How we measure every signal.

1. Aggressive Limit Fill Model

Starting with ses.168 (May 2026), the entry price used for P&L calculation is no longer the midpoint of the entry zone. It is the actual fill price of a limit order placed at the most conservative edge of the zone, with a bonus when the activation candle opens through a gap.

In plain words: for LONG, if the market opens below entryMax(gap-down), our limit fills at the real open — a benefit for the user. If the open is above entryMax, the fill happens at the worst edge (entryMax) — the most conservative scenario. Symmetric for SHORT with entryMin.

The fill must happen after the signal exists

A signal is created part-way through a candle. That candle's high and low therefore describe a window that includes price action from before the signal existed. Testing the entry zone against the whole candle would let a signal be “filled” at a price that had already come and gone. We resolve that one straddling candle at 1-minute granularity and accept a fill only if the entry zone is reached by a minute bar that opens at or after the signal's creation timestamp. If minute data is unavailable for the instrument, we refuse to fill on that candle and wait. A late fill is recoverable; a fabricated one is not.

The same rule is enforced at write time as an invariant on every closed signal:created ≤ published ≤ closed.

One denominator

Every result — take-profits and stop-losses alike — is(exit − fill) ÷ fill. Not the mid-point for gains and the fill for losses; the same denominator for both. Until ses.309 that was not true: take-profit results were divided by the entry-zone midpoint while stop results were divided by the real fill, which understated every gain and left every loss exact. The historical rows were recomputed onto the single basis, which moved the published gains up — an honest correction that happens to flatter us, and is disclosed here for that reason.

No cherry-picking: every signal that was filled is included, including ones that hit stop-loss immediately after fill. Signals whose entry zone was never reached committed no capital, have no realized P&L, and are shown as “Never entered” — excluded from the win rate, still listed publicly. A filled position that ran out of clock is a different thing entirely: it is closed at market, shown as “Closed at time limit”, and it counts in the win rate by the sign of its realized P&L, win or lose. These two used to share one grey “Expired” badge, which quietly removed the second class — the losing one — from the statistics.

One historical signal (XRP/USDT 15M, 13 Jul 2026) has been withdrawn from the track record. It was recorded as a win, but the audit showed its entry zone was only ever touched before the signal was published — it could not have been traded. The row is retained in our database, marked as quarantined, and excluded from every statistic on this site. It is named here rather than deleted quietly.

2. How an outcome is decided — and what we do not do

This section used to claim that we compute a Deflated Sharpe Ratio over a Combinatorial Purged Cross-Validation framework. That was not true. The code for both exists in our repository, but neither has ever run on the numbers you see: the cross-validation routine has no callers at all, and the deflated-Sharpe endpoint requires a minimum sample it has never had, so it has only ever returned “insufficient data”. The claim has been removed rather than softened, and what follows is what the track record actually rests on.

Triple-barrier resolution, on one venue

Once a signal is filled it has exactly three ways out: the take-profit level, the stop-loss level, or the time limit for its timeframe. Whichever is reached first ends the trade. Every barrier check is run against candles from a single pinned venue (Binance for crypto pairs) — not whichever exchange happened to answer first. Different exchanges disagree on the wicks, and a win that depends on which server replied fastest is not a win.

Barrier touches require the candle to close beyond the level, not merely to wick through it. That is the pessimistic reading: a level tagged for one tick and rejected does not count as reached — in either direction. When a single candle contains both the take-profit and the stop-loss, we cannot know from OHLC data which came first; the resolution rule is documented in the code and applied identically to every signal.

Small numbers, stated as small numbers

A win rate is an estimate, and on a few dozen trades it is a noisy estimate. We publish it with a 95% Wilson confidence interval and the number of trades it was computed from, always together. On our current sample that interval is wide enough to include outcomes that would be indistinguishable from chance — which is the honest state of the evidence, and the reason we show the interval rather than the headline alone. Below a minimum sample we publish no percentage at all: a “100% win rate” on two trades is a slogan, not a statistic.

What is built but not wired

We maintain machinery that is not currently used to produce any published number, and we would rather name it than let its presence in the codebase imply otherwise:

• Combinatorial purged cross-validation — implemented, zero callers. It validates nothing today.
• Deflated Sharpe Ratio — implemented, but its sample threshold has never been met, so it has never produced a value.
• Conformal prediction intervals — implemented and scored, but measured against held-out data the coverage was 19/36 (52.8%) against a 90% nominal. We do not publish it as a guarantee, because on this data it is not one.
• Probability calibration (isotonic) — trained, and deliberately not promoted: it does not beat a constant out-of-sample on our current history.

If and when any of these starts constraining a live decision, this page changes first. A mechanism that exists is not a mechanism that runs.

3. CRPS Forecast Scoring

For macro / geopolitical forecasts (output: continuous distribution with p10/p50/p90 quantiles), we evaluate quality through the Continuous Ranked Probability Score — a proper scoring rule that penalizes both bias and over/under-confidence.

Quantile validation (post-ses.175)

Quantiles must respect the natural ordering p10 ≤ p50 ≤ p90. Starting with ses.175, an auto-repair guard detects crossing (rare, but possible when personas diverge) and re-orders the quantiles before scoring, with logging for audit.

Per-category rolling aggregation

CRPS is not a global score — we aggregate it per category (rates, conflict, election, energy, etc.) on rolling 30/90-day windows. The internal leaderboard shows mean / median / p90 / sample count per category; values below 0.15 are considered excellent, 0.15–0.30 acceptable, above 0.30 problematic.

Implemented in lib/forecast/calibration/crps.ts + crpsRolling.ts (ses.172 N-05).

4. Persona Divergence Labels

The Multi-LLM Council uses several independent voices (Anthropic, OpenAI, Google, xAI; active set varies with budget allocation). For each scoring cycle, we measure how aligned the verdicts are via standard deviation and entropy over the scores.

• CONSENSUS — low stdev + low entropy. The voices converge to the same verdict. High confidence.
• MIXED — moderate stdev. Voices aligned on direction, but with different magnitudes.
• CHAOS — high stdev + high entropy. Voices are fundamentally disagreeing. Treated with reduced confidence and different routing at synthesis.

System-level impact

• Synthesis routing: CONSENSUS routes to the cheapest model tier, CHAOS routes to the strongest one — the harder the disagreement, the more capacity we spend arbitrating it.
• Cache TTL (POST-12): CONSENSUS results can be cached more aggressively; CHAOS forces re-evaluation on the next cycle.
• Confidence dampening: CHAOS applies an informational penalty to the final score, signalling that the models do not “know” with certainty.

Implemented in lib/multiLLM/personaDivergence.ts (ses.172 N-06). The label is persisted on every signal in the SignalSnapshot.divergenceAdj field and is visible on the AI Council page.