Problem
Most "RL trading" projects are trained offline on a frozen dataset and then shown as a backtest chart — which hides the two hard problems: learning online from non-stationary streaming data, and knowing what the network is actually doing while it trades. For my AI/ML bootcamp capstone (April 2026) I wanted the opposite: an agent that trains continuously on live market data, and a dashboard honest enough to show every layer of it — including the P&L, whatever it turned out to be.
What I built
A live online-learning RL system for BTC perpetual futures. A Soft Actor-Critic agent trains continuously as data streams in — no train/deploy split, the weights that trade are the weights being updated.
16 live feature blocks → a 116-dim state vector, ingested in real time: 5Hz L2 order-book microstructure, funding and open interest, cross-exchange premium, liquidations, Deribit options flow, stablecoin supply, and social sentiment, among others — each block with its own freshness tracking on the dashboard.
Four neural encoders turn that state into latents:
- an order-book CNN over the top 15 price levels × 60 ticks of L2 data
- a 2-layer GRU temporal encoder over the recent feature history
- a 116-dim full-state MLP over the raw feature vector
- a cross-block attention encoder treating the 16 blocks as tokens
A continuous SAC head sits on top, with two additions that came out of watching it trade: an anti-churn temperature integrator (entropy temperature steered to keep the agent from thrashing in and out of positions) and a hold-triangle reward that pays for holding a position through favorable moves instead of scalping every tick.
The dashboard is an X-ray of the whole network updating live: encoder latent heatmaps, Q-values, actor/critic losses, every fill on a trade tape, and the NAV curve — all streamed over WebSocket from the training loop while it runs against the Bitunix perpetuals API.
Outcome
The demo is the live system itself — not a backtest, not a recording. It runs 24/7 on a Hetzner box: twelve market feeds, the inference tick, and the SAC trainer, all under a 1Hz latency certification (p50 ~400 ms, p99 under 1.2 s), streaming into the X-ray dashboard over WebSocket. Whatever the NAV reads when you open it is what the agent is actually doing right now.
The point of the project is that machinery: genuinely online RL on live, messy, multi-source market data, with full visibility into every layer — encoder activations, Q-values, actor/critic losses, every fill on the tape, the NAV curve. No cherry-picked backtest, no survivorship, no hiding the P&L — a real system on real data, shown as it trades, small net loss and all. (An earlier recorded April 2026 run remains archived as a replay; the live board is the honest, running artifact.)