How the reports earn trust

A professional reader of any analytical publication asks four questions: How are the signals defined? Can I inspect the methodology? Can I reproduce the numbers? How well do they calibrate over time? This page answers all four — and every answer below is live-derivable from the platform, not marketing copy about it.

1How are the signals defined?

Every signal is a named boolean predicate over a flat metrics dict — a key, an operator, and a threshold. Nothing is discretionary at firing time. The full catalog is machine-introspectable: derived_signals(schema=True) returns exactly what is rendered below (53 signals, 57 accepted input aliases, 16 contradictions).

Contradictions — the headline layer

Single signals describe; contradictions — named combinations that a desk reads as one thesis-level statement, whether the legs pull against each other (refiner_margin_trap, freight_locked_arb) or reinforce (global_stocks_tight) — are what a desk actually argues about, so they are the report headlines. Each has a required signal set, optional exclusions, a score (average of component scores), and a fixed narrative. Fourteen of the sixteen require two or more signals; two (diesel_clean_long, flow_freight_divergence) are a single required signal with an exclusion. (us_comfortable_eu_tight was rebuilt 2026-08-11: it previously paired one requirement with an exclusion, which together with global_stocks_tight exhaustively partitioned a boolean — a guaranteed headline whenever the EU leg fired. Its US leg is now an assertion, crude_oversupplied.) A sample:

How the contradiction layer is scored — and why it mostly isn't. Most contradictions combine conditions that are individually uncommon (2σ z-thresholds, or 20th/80th/90th-percentile cuts), so the joint base rate keeps the sample in single digits however long the history runs. The two single-signal-plus-exclusion forms are scoreable to the same extent as their required signal, and inherit its sample. So the signal layer is scored against its own firings; the contradiction layer is largely argued. Each one is named, its component signals are published, and its reading is fixed in advance — so it can be disagreed with precisely, which is the honest alternative to a statistic that will never exist.

Full catalog — all 53 predicates, the 57-alias input map, and all 16 contradiction definitions — at /signals (generated from the running server) or via derived_signals(schema=True) on any connected client.

2Can I inspect the methodology?

Yes — the rules the reports operate under are written down, versioned, and enforced by the tooling rather than by good intentions.

Backtest rules (signal_backtest)

Every claimed edge is an event study over the signal's own historical firings, under always-on honesty guards:

• Point-in-time evaluation — predicates are re-evaluated with expanding windows using only data available at each historical date. No look-ahead.
• Rolling de-clustering — a firing within the gap of the previous firing extends the episode; a sustained run of any length is one episode. (Episode-start de-clustering overstates n by 5–8× on this data.)
• The overlap rule — episodes closer together than the forward horizon share forward-window days and pseudo-replicate. Independent cluster counts at ≥-horizon separation are reported beside n, and sub-cluster n is never cited alone. This rule exists because an adversarial review caught a "n=23, edge +7%" result that was really 2 independent episodes — the finding amended the spec itself.
• Base rates beside conditionals — the unconditional same-horizon move is always shown; the headline number is edge = conditional − base, never a scary conditional the market does anyway.
• Small-sample and regime flags — n < 10 is flagged low confidence; >50% of episodes in one calendar year is flagged single-regime.
• Incomplete windows excluded — an ongoing episode never scores its own unfinished future.
• Inconvenient results are published — in the July 2026 pack, the backtest withdrew one of the report's own claims, refuted another, and found a third unscoreable. All three appear in the report table.
• A known gap, disclosed: live and replayed z-scores are not yet the same statistic — the live engine normalizes some z-based signals over the full sample, while the backtest replays them point-in-time over expanding windows. Where the two definitions diverge the backtest refuses to emit numbers rather than score a signal the engine doesn't fire (cushing_tight currently returns exactly that warning). Until the engine computes point-in-time statistics natively, backtest evidence certifies the point-in-time variant of a rule, not the full-sample one. This affects 14 of the 53 signals — the 13 z-based predicates plus vix_oil_supply_shock, whose rolling correlation behaves the same way. The percentile- and level-threshold signals are unaffected.

Revision policy

• Every artifact carries a revision trail in its footer — what changed, why, in which revision. Silent edits don't happen; the July update pack ships with five documented revisions (r2–r6), including the ones that made it look worse.
• Trigger amendments are always explicit — if an invalidation threshold moves, every occurrence discloses the old value and the reason, and scoreboards name the rule set they were scored on ("0 of 4 fired" always says which four).
• Statistical windows are stated where they are known — a z-score is meaningless without its window, so where the window is published it appears next to the number (crack z over 2020–, the same window the signal fires on). Not every signal carries one yet: cushing_z, bwet_z, vix_corr and wti_price_pctl currently ship without a stated window, and a per-signal window field is being added to the schema so the claim becomes mechanically true rather than a matter of diligence. Where a window is stated, it is authoritative.

3Can I reproduce the numbers?

Every quantitative claim in a report traces to a named tool call against versioned data.

• Tool appendix — each report ends with the full list of tools called, call counts, and what each produced, plus a per-series data appendix with vintages (the as-of date of every input).
• Measured vs judgment tags — every claim is tagged. Measured values reproduce from the tools; judgment is labeled as judgment and never wears false precision.
• Press is fenced — qualitative news context lives in its own cited section, outside the audited quantitative core, and informs judgment only.
• The snapshot is canonical — report(preset=...) emits the flat metrics dict the narrative was derived from; save_report_run stores it with the HTML, and compare_runs diffs any two runs server-side. The engine's "what changed" is checkable against the report's "what changed".
• Reproduce it yourself — the same tools that build the reports are the product: run report(watchlist=..., preset="monthly_risk"), derived_signals(metrics=...), and signal_backtest(signal=...) against the live server and compare.
• An external check, not a self-assessment — reproducibility claims are cheap when the platform grades its own homework. So we rebuilt the final edition of EIA's This Week in Petroleum (29 Oct 2025) from the published series, point-in-time, and compared every headline number against what EIA printed: 8 of 8 within their own rounding.

4How well do they calibrate?

The honest answer today: being measured — first resolution August 2026. The machinery is running and the ledger is open, but one unresolved entry is not a track record. This section is a promissory note with a date on it; it becomes evidence a year from now, and the reader is entitled to hold it to that.

The Report performance panel

Every edition ends by scoring the previous one, under one non-negotiable condition: scoring is pre-registered, never retrospective. Trigger-based calls score mechanically against thresholds frozen at publication; prose claims are editorially scored and flagged as such; and every new call ships with its scoring rule attached, so the next edition cannot choose how to grade this one. In the panel's first outing, the mechanical calls held — and none of the three editorially-scored prose claims survived the backtest as stated. That result was printed.

See it in action

The demo reports below were produced under exactly these rules — revision trails, appendices, and performance panels included.