Calibrated Conviction

A crypto trading Skill that knows how sure it should be — and bets accordingly.

Real out-of-sample result: Brier 0.238 · ECE 0.039 · ~¼ the drawdown of a naive full-size baseline across 19,168 hourly points. Calibration is the proof; survival is the strategy.

🔗 Live page · BNB Hack submission · Skill spec

Submission to BNB Hack: AI Trading Agent Edition (Track 2 — Strategy Skills), built on the CoinMarketCap AI Agent Hub data layer.

Every other strategy is a signal generator — RSI + MACD + Fear & Greed tells you what to trade. Calibrated Conviction answers the questions that actually keep traders alive:

How sure should I be? Is that confidence earned? So how much should I bet?

It emits a calibrated probability of an up-move, proves it's honest (Brier score + reliability curve), sizes by fractional Kelly, and abstains when the edge is thin or the regime looks chaotic.

Real-data evidence (the key proof)

Run on real, free, public data (Binance klines + funding, alternative.me Fear & Greed), pooling 8 liquid pairs at hourly resolution — 19,168 strictly out-of-sample points — via python realdata.py:

• Calibrated on real markets: out-of-sample Brier 0.238 (< 0.25 coin-flip) and ECE 0.039. A "55% call" really does come up ~55% of the time — see the reliability curve, where the heavily-populated bins sit on the diagonal.
• Calibrated sizing survives: on each symbol's own coherent timeline, calibrated sizing + abstention takes a median max drawdown ~¼ that of the naive baseline, at comparable-or-higher Sharpe.

Honest read: real short-horizon edge is small (Brier just under 0.25). The win is not a magic return — it's honest probabilities + disciplined sizing that takes a fraction of the drawdown.

Reproducible synthetic stress test

Where the thesis is designed to be visible: a synthetic market with calm and chaotic, crash-prone regimes, vs a naive full-size baseline, across 40 independent draws:

	Calibrated Conviction	Naive full-size
Median Sharpe	11.3	6.8
Median max drawdown	5%	20%
Worst-case drawdown	11%	27%

The naive agent wins the headline-return contest by running its drawdown ~5× deeper — toward a typical 30% disqualification cap, one bad week from elimination. Synthetic figures illustrate behaviour under engineered stress, not return claims.

Run it

pip install -r requirements.txt
python test_strategy.py        # 6 passing tests
python backtest.py             # out-of-sample backtest + calibration metrics
python backtest.py --robust    # 40-market robustness + 30% DQ-gate analysis
python realdata.py             # REAL pooled multi-asset backtest (needs internet)
python realdata.py --demo      # pipeline self-test, no network

Files

• features.py — multi-source features (price, funding, OI, social, F&G) + the live regime-uncertainty tell
• model.py — calibrated logistic probability + Brier / ECE / reliability curve
• strategy.py — fractional-Kelly sizing + edge & regime abstention gates
• backtest.py — OOS backtest + 40-market robustness vs naive baseline
• realdata.py — real multi-asset walk-forward backtest; writes reliability_curve.csv + .png
• test_strategy.py — calibration, sizing, abstention, risk-adjusted-win tests
• skill_spec.md — the strategy authored as a CMC Skill spec

Honesty notes

• Real-data calibration pools all 8 symbols (per-sample, order-independent). The P&L / drawdown comparison is computed on each symbol's own coherent timeline and aggregated (median) — series are never concatenated into a fake equity curve. Cost is a realistic per-notional taker fee (~6 bps).
• The synthetic market is seeded and gives only a modest, noisy, regime-dependent edge — the claim is about sizing a weak signal correctly, not a magic alpha.
• This is a strategy/research Skill — it does not custody funds or execute trades, so it carries no trading or regulatory risk by itself.

MIT licensed. Author: Ruiyang Zhang (CFA, ASA) · ruiyang.co · GitHub @ryonzhang