Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

feat(backtest): audit every sniper backtest for look-ahead, pin its dataset with a hash, and order data sources by canary health

nirholasbfd8ca754df5
diff --git a/STRUCTURE.md b/STRUCTURE.md
index 63ec26e9a..c86d7ac9b 100644
--- a/STRUCTURE.md
+++ b/STRUCTURE.md
@@ -336,6 +336,7 @@ packages, a generated standalone GitHub mirror). See
| Autonomous Trading hub (front door) | [pages/trading.html](pages/trading.html) `→ /trading` + [src/trading-hub.js](src/trading-hub.js), pure formatters [src/trading-hub-format.js](src/trading-hub-format.js), surface + doc directory [src/trading-hub-data.js](src/trading-hub-data.js), styles [src/trading-hub.css](src/trading-hub.css) | Live | One entry point for a trading system that had grown to ~20 pages with no front door. Live fleet vitals from `GET /api/sniper/status` (mode, armed strategies, open positions, feed health, funding, uptime), the agent scoreboard from `GET /api/sniper/leaderboard` (realized P&L only, closed trades only, with cumulative sparkline), the five-gate entry-to-exit explainer, and directories of every trading surface and doc. Feed health is reported separately from process health because a beating worker with a silent feed reads as green while seeing nothing. Directory is data, so [tests/trading-hub.test.js](tests/trading-hub.test.js) asserts every link resolves against `data/pages.json` and the docs tree. Docs: [docs/trading-hub.md](docs/trading-hub.md), tutorial [docs/tutorials/read-the-trading-fleet.md](docs/tutorials/read-the-trading-fleet.md) |
| Strategy Objects (settings, research gates, ask-before-trade) | library and builder [pages/strategies.html](pages/strategies.html) `→ /strategies` (`?editor=new`), form + JSON editor [src/shared/strategy-forms.js](src/shared/strategy-forms.js), live preview, text compiler box and gated backtest [src/shared/strategy-settings.js](src/shared/strategy-settings.js), agent panel with mode switch, approvals and decision log [src/shared/agent-strategy-panel.js](src/shared/agent-strategy-panel.js); schema and gate evaluator [api/_lib/strategy-schema.js](api/_lib/strategy-schema.js), gate report [api/_lib/strategy-research.js](api/_lib/strategy-research.js), approvals and decision log [api/_lib/strategy-approvals.js](api/_lib/strategy-approvals.js), preview [api/_lib/strategy-preview.js](api/_lib/strategy-preview.js), runtime [api/_lib/agent-strategy-runtime.js](api/_lib/agent-strategy-runtime.js), APIs [api/strategies.js](api/strategies.js) `→ /api/strategies` and [api/agents/agent-strategy-objects.js](api/agents/agent-strategy-objects.js) `→ /api/agents/:id/strategies`, migration [api/_lib/migrations/20261010140000_strategy_research_gates.sql](api/_lib/migrations/20261010140000_strategy_research_gates.sql) | Built, ships with the next deploy | Version 2 Strategy Objects: source filter, market cap band, SOL size, a price-impact cap that only tightens the agent breaker, nine research gates (holders, largest holder, liquidity, firewall score, mint and freeze authority, creator launches, graduations, dev sold) that fail closed on unknown data, and `ask` (default) or `auto` mode. Every blocked coin is logged with the check and reason; ask mode files a payload-hash approval that executes once after re-checking every guard. The builder edits one config through sliders, toggles and raw JSON, shows a live count of recent launches the settings would have bought, compiles plain English into the fields, and backtests with the gates applied. Doc: [docs/strategy-objects.md](docs/strategy-objects.md). Tests: [tests/strategy-research-gates.test.js](tests/strategy-research-gates.test.js), [tests/strategy-schema.test.js](tests/strategy-schema.test.js) |
| Sniper Experiments (rules vs LLM A/B fleet) | [pages/sniper-experiments.html](pages/sniper-experiments.html) + [src/sniper-experiments.js](src/sniper-experiments.js) `→ /sniper/experiments`, aggregate [api/sniper/experiments.js](api/sniper/experiments.js), LLM judge [workers/agent-sniper/llm-judge.js](workers/agent-sniper/llm-judge.js), fleet seeding [scripts/seed-sniper-experiments.mjs](scripts/seed-sniper-experiments.mjs) | Live | The sniper fleet runs labeled rule-set variants side by side (strict shields, oracle-gated, and `decision_mode=llm` arms judged per-launch by Grok / Claude / OpenRouter auto with no shields), all behind the same safety rails. The page is the live scoreboard of which entry conditions actually make money; real on-chain fills only. |
+| Backtest integrity and research/live wall | audit, manifests, reproducibility [api/_lib/backtest-integrity.js](api/_lib/backtest-integrity.js), read-only handle [api/_lib/research-context.js](api/_lib/research-context.js), canaries [api/_lib/data-canaries.js](api/_lib/data-canaries.js) + [api/_lib/data-canary-health.js](api/_lib/data-canary-health.js) + cron [api/cron/data-canary.js](api/cron/data-canary.js), import wall [scripts/lib/research-wall.mjs](scripts/lib/research-wall.mjs), view [src/shared/backtest-integrity-view.js](src/shared/backtest-integrity-view.js) | Built, ships with the next deploy; scheduler sync owner-gated | Look-ahead audit of the sniper entry gate on every backtest, dataset manifest with content hash (older results marked unreproducible), 30 minute price and candle canaries that reorder fallback chains, and a test that fails if research code can reach a live execution module. Docs: [docs/backtest-integrity.md](docs/backtest-integrity.md). Tests: [tests/backtest-integrity.test.js](tests/backtest-integrity.test.js) |
| Arm your agent (Oracle conviction on autopilot) | [pages/arm.html](pages/arm.html) + [src/arm.js](src/arm.js) `→ /oracle/arm` (also `/arm`), config + graded ledger API [api/oracle/watch.js](api/oracle/watch.js) `→ /api/oracle/watch`, alert test [api/oracle/test-alert.js](api/oracle/test-alert.js), live stream [api/oracle/feed.js](api/oracle/feed.js), proof of edge [api/oracle/backtest.js](api/oracle/backtest.js) | Live | The rules screen that turns an Oracle score into an agent that acts. Four steps: the conviction floor (Prime / Strong+ / Lean+, each showing its real 30-day win rate, average peak and 3x count for that exact bar), position sizing with hard daily and open-position caps, an optional personal Telegram alert with a send-test button, and simulate-or-live arming. Beside the form is a live preview of every coin clearing the rules as they stand, updated every 20 seconds, with the size the agent would commit to each: tighten the bar and the list shrinks in front of you. The agent spends only from its own custodial Solana wallet, whose balance and remaining trade runway sit in the control bar, and every action it takes is graded against the outcome in the ledger below. The conviction preview and edge readout are public endpoints, so a signed-out visitor sees the product working before being asked for an account. Arming in live mode is gated on the shared risk acknowledgement ([src/shared/risk-ack.js](src/shared/risk-ack.js)). |
| Oracle Lab (the conviction model, published) | [pages/oracle-lab.html](pages/oracle-lab.html) + [src/oracle-lab.js](src/oracle-lab.js) `→ /oracle-lab`, model API [api/oracle/model.js](api/oracle/model.js) `→ /api/oracle/model`, SDK [packages/oracle-model/](packages/oracle-model) `→ @three-ws/oracle-model` | Live | The glass box over the Oracle. Publishes the scoring model itself, not a summary of it: every bucket weight with the number of labeled launches behind it and what happened to them, the held-out reliability curve for each tier claim, the version history of every refit INCLUDING the candidates the promotion gate refused and why, and a scorer that runs the real weights in the visitor's browser. The model is a lookup table (~32KB), so `@three-ws/oracle-model` downloads it once and reproduces any score offline with no key, no rate limit and no telemetry; `verify()` re-measures our published AUC/Brier/reliability against a caller's own labeled outcomes, and `diff()` shows every weight that moved between two versions. `?view=registry` and `?view=diff&from=&to=` serve the same data server-side. Docs: [docs/oracle-model.md](docs/oracle-model.md) `→ /docs/oracle-model`. Tests: [packages/oracle-model/test/](packages/oracle-model/test) (23), [tests/oracle/fit.test.js](tests/oracle/fit.test.js) (43) |
| Oracle conviction model (three heads, learned on a clock) | Fitter [api/_lib/oracle/fit.js](api/_lib/oracle/fit.js), scorer [api/_lib/oracle/conviction.js](api/_lib/oracle/conviction.js), DB-backed weights [api/_lib/oracle/model-store.js](api/_lib/oracle/model-store.js), learning loop [api/cron/oracle-refit.js](api/cron/oracle-refit.js) (every 6h), CLI [scripts/oracle-fit.mjs](scripts/oracle-fit.mjs) | Live, v3 | v2 ranked "spikes 3x or graduates", so a coin that ran 3x and went to zero counted as a hit: Prime-tier coins hit that target 58.2% of the time and rugged 53.0% of the time. v3 fits three heads over one design matrix (`win` = ran and the holder kept it, `rug` = holder down more than half, `moon` = ran at all) and anchors the public score on `win`, publishing `giveBackRisk = 1 - P(win)/P(moon)` beside it: given it runs, how often a coin like this hands the run straight back. Weights moved out of the container image into `oracle_model_versions`, so the refit cron can promote a better model between deploys; it was frozen at its 2026-08-09 fit for nineteen days because nothing but a human could refit it. Every candidate faces a promotion gate (complete fit, absolute AUC floor, every tier band still earning its claim, no feature-set collapse, a material gain over the incumbent, no regression on the other heads) and is stored either way with the reason. Thin buckets are shrunk toward zero by `n/(n+200)` so 34 samples cannot reverse a verdict. Docs: [docs/oracle-model.md](docs/oracle-model.md) |
diff --git a/api/_lib/backtest-integrity.js b/api/_lib/backtest-integrity.js
new file mode 100644
index 000000000..f69f863a1
--- /dev/null
+++ b/api/_lib/backtest-integrity.js
@@ -0,0 +1,186 @@
+// Backtest integrity: the three checks that decide whether a backtest number can
+// be trusted.
+//
+// 1. Look-ahead audit. Replays a decision function with the data truncated at
+// each decision time and fails if any decision changes when the future is
+// removed. A decision that depends on data after its own timestamp is
+// look-ahead bias, and it makes a backtest look better than live can be.
+// 2. Dataset manifest. Every backtest records which dataset it ran on (id,
+// range, source, row count, content hash), so a second run can prove it saw
+// the same bytes. A result without a manifest is marked unreproducible.
+// 3. Prior causality. Per-subject running counters (a creator's prior launches)
+// must never fall or stall as time advances; if they do, they were computed
+// with knowledge of the future.
+//
+// Pure functions only. No network, no database, no wallet.
+
+import { createHash } from 'node:crypto';
+
+export const MANIFEST_VERSION = 1;
+const MAX_REPORTED = 50;
+const DEFAULT_MAX_CHECKS = 400;
+
+function canonical(v) {
+ if (v === undefined) return null;
+ if (v instanceof Date) return v.toISOString();
+ if (typeof v === 'bigint') return v.toString();
+ if (Array.isArray(v)) return v.map(canonical);
+ if (v && typeof v === 'object') {
+ const out = {};
+ for (const k of Object.keys(v).sort()) out[k] = canonical(v[k]);
+ return out;
+ }
+ return v;
+}
+
+const same = (a, b) => JSON.stringify(canonical(a)) === JSON.stringify(canonical(b));
+
+/**
+ * Indexes to audit: every index when there are few, otherwise an even stride that
+ * always includes the first and last decision.
+ */
+function auditIndexes(total, maxChecks) {
+ if (total <= maxChecks) return Array.from({ length: total }, (_, i) => i);
+ const out = new Set([0, total - 1]);
+ const stride = (total - 1) / (maxChecks - 1);
+ for (let k = 0; k < maxChecks; k++) out.add(Math.round(k * stride));
+ return [...out].sort((a, b) => a - b);
+}
+
+/**
+ * Look-ahead audit.
+ *
+ * `decide(visible, index)` returns the decision for `visible[index]` (any
+ * JSON-comparable value). It is called twice per audited index: once with the full
+ * series and once with the series truncated right after that decision, so
+ * `visible.length === index + 1`. If the two decisions differ, the decision used
+ * data from the future.
+ *
+ * @param {object} args
+ * @param {any[]} args.series Time-ordered bars or events.
+ * @param {(visible:any[], index:number)=>any} args.decide
+ * @param {number} [args.maxChecks=400]
+ * @returns {{ ok: boolean, method: string, total: number, checked: number, offending: number[], offending_truncated: boolean, errors: number }}
+ */
+export function auditLookahead({ series, decide, maxChecks = DEFAULT_MAX_CHECKS }) {
+ if (!Array.isArray(series)) throw new TypeError('auditLookahead: series must be an array');
+ if (typeof decide !== 'function') throw new TypeError('auditLookahead: decide must be a function');
+ const total = series.length;
+ const indexes = auditIndexes(total, Math.max(2, Math.floor(maxChecks)));
+ const offending = [];
+ let errors = 0;
+ for (const i of indexes) {
+ let full;
+ let truncated;
+ try {
+ full = decide(series, i);
+ truncated = decide(series.slice(0, i + 1), i);
+ } catch {
+ errors++;
+ offending.push(i);
+ continue;
+ }
+ if (!same(full, truncated)) offending.push(i);
+ }
+ return {
+ ok: offending.length === 0,
+ method: 'truncate-future',
+ total,
+ checked: indexes.length,
+ offending: offending.slice(0, MAX_REPORTED),
+ offending_truncated: offending.length > MAX_REPORTED,
+ errors,
+ };
+}
+
+/**
+ * Causality check for per-subject running counters. `rows` must be time-ordered.
+ * For each subject, the counter at a later row must be at least one higher than at
+ * the previous row (each earlier row is itself one more prior occurrence).
+ *
+ * @returns {{ ok: boolean, checked: number, offending: number[] }}
+ */
+export function checkPriorsCausal({ rows, subjectOf, priorOf }) {
+ const last = new Map();
+ const offending = [];
+ let checked = 0;
+ rows.forEach((row, i) => {
+ const subject = subjectOf(row);
+ const prior = priorOf(row);
+ if (subject == null || prior == null) return;
+ checked++;
+ const before = last.get(subject);
+ if (before != null && prior < before + 1) offending.push(i);
+ last.set(subject, prior);
+ });
+ return { ok: offending.length === 0, checked, offending: offending.slice(0, MAX_REPORTED) };
+}
+
+/**
+ * Describe the exact dataset a backtest ran on.
+ *
+ * @param {object} args
+ * @param {string} args.id Stable dataset id (e.g. table plus network plus window).
+ * @param {string} args.source Where the rows came from.
+ * @param {any[]} args.rows The rows, in the order the replay consumed them.
+ * @param {(row:any)=>any} args.project Picks the fields that determine the result.
+ * @param {(row:any)=>any} args.timeOf Row timestamp (Date, ISO string or epoch ms).
+ */
+export function buildDatasetManifest({ id, source, rows, project, timeOf }) {
+ const hash = createHash('sha256');
+ let from = null;
+ let to = null;
+ for (const row of rows) {
+ hash.update(JSON.stringify(canonical(project(row))));
+ hash.update('\n');
+ const t = timeOf(row);
+ const ms = t instanceof Date ? t.getTime() : typeof t === 'number' ? t : Date.parse(t);
+ if (Number.isFinite(ms)) {
+ if (from == null || ms < from) from = ms;
+ if (to == null || ms > to) to = ms;
+ }
+ }
+ return {
+ version: MANIFEST_VERSION,
+ id,
+ source,
+ range: {
+ from: from == null ? null : new Date(from).toISOString(),
+ to: to == null ? null : new Date(to).toISOString(),
+ },
+ row_count: rows.length,
+ content_hash: hash.digest('hex'),
+ };
+}
+
+/** True when a manifest carries every field needed to reproduce the run. */
+export function isCompleteManifest(m) {
+ return !!(
+ m &&
+ typeof m === 'object' &&
+ typeof m.id === 'string' && m.id &&
+ typeof m.source === 'string' && m.source &&
+ m.range && typeof m.range === 'object' && 'from' in m.range && 'to' in m.range &&
+ Number.isInteger(m.row_count) && m.row_count >= 0 &&
+ typeof m.content_hash === 'string' && /^[0-9a-f]{64}$/.test(m.content_hash)
+ );
+}
+
+/**
+ * Stamp `reproducible` on a result. A result with no complete manifest (every
+ * snapshot saved before manifests existed) is explicitly unreproducible.
+ */
+export function withReproducibility(result) {
+ if (!result || typeof result !== 'object') return result;
+ const manifest = result.integrity?.manifest;
+ if (isCompleteManifest(manifest)) return { ...result, reproducible: true };
+ return {
+ ...result,
+ reproducible: false,
+ integrity: {
+ ...(result.integrity || {}),
+ manifest: null,
+ unreproducible_reason: 'No dataset manifest was recorded for this run, so it cannot be re-run against the same data.',
+ },
+ };
+}
diff --git a/api/_lib/data-canaries.js b/api/_lib/data-canaries.js
new file mode 100644
index 000000000..2f9094bc9
--- /dev/null
+++ b/api/_lib/data-canaries.js
@@ -0,0 +1,122 @@
+// Market-data canaries. Each probe asks a source for something whose true value is
+// known and checks the answer, so a source that is up but wrong (stuck price,
+// wrong pair, shifted candles) fails the same way a dead one does.
+//
+// price every SOL/USD provider in sol-price.js must answer a positive number
+// within PRICE_BAND of the median of all answering providers.
+// candle a closed historical daily SOL/USD candle (2025-01-01 UTC). It cannot
+// change, so the probe checks the timestamp, OHLC consistency and that
+// the close sits within CANDLE_TOLERANCE of the reference close.
+//
+// Verdicts are stored by data-canary-health.js and consumed by `orderByCanary`.
+
+import { PROVIDERS as SOL_PRICE_PROVIDERS } from './sol-price.js';
+import { recordCanary } from './data-canary-health.js';
+
+const PROBE_TIMEOUT_MS = 6000;
+const PRICE_BAND = 0.05;
+const CANDLE_TOLERANCE = 0.02;
+const UA = 'three.ws-data-canary/1.0';
+
+/** Closed daily SOL/USD candle, 2025-01-01 UTC. Same bar on both venues. */
+export const KNOWN_CANDLE = Object.freeze({ time: 1735689600, close: 193.84, low: 187.6, high: 194.99 });
+
+const num = (v) => {
+ const n = Number(v);
+ return Number.isFinite(n) ? n : null;
+};
+
+/**
+ * Judge a normalized candle {time, open, high, low, close} against the reference.
+ * @returns {{ ok: boolean, detail: string|null }}
+ */
+export function judgeCandle(c, ref = KNOWN_CANDLE, tolerance = CANDLE_TOLERANCE) {
+ if (!c) return { ok: false, detail: 'no_candle' };
+ if (c.time !== ref.time) return { ok: false, detail: `time_mismatch:${c.time}` };
+ const { open, high, low, close } = c;
+ if (![open, high, low, close].every((x) => x != null && x > 0)) return { ok: false, detail: 'non_positive' };
+ if (high < Math.max(open, close) || low > Math.min(open, close) || high < low) return { ok: false, detail: 'ohlc_inconsistent' };
+ if (Math.abs(close - ref.close) / ref.close > tolerance) return { ok: false, detail: `close_off:${close}` };
+ return { ok: true, detail: null };
+}
+
+/**
+ * Judge a set of price answers by agreement with their median.
+ * @param {Record<string, number|null>} prices
+ * @returns {Record<string, { ok: boolean, detail: string|null }>}
+ */
+export function judgePrices(prices, band = PRICE_BAND) {
+ const live = Object.values(prices).filter((p) => p != null && p > 0).sort((a, b) => a - b);
+ const median = live.length ? live[Math.floor((live.length - 1) / 2)] : null;
+ const out = {};
+ for (const [id, p] of Object.entries(prices)) {
+ if (p == null || !(p > 0)) out[id] = { ok: false, detail: 'no_price' };
+ else if (live.length < 2) out[id] = { ok: true, detail: null };
+ else out[id] = Math.abs(p - median) / median <= band ? { ok: true, detail: null } : { ok: false, detail: `off_median:${p}` };
+ }
+ return out;
+}
+
+async function timed(fn) {
+ const t0 = Date.now();
+ try {
+ return { value: await fn(), ms: Date.now() - t0, error: null };
+ } catch (err) {
+ return { value: null, ms: Date.now() - t0, error: err?.name === 'AbortError' ? 'timeout' : err?.message || 'error' };
+ }
+}
+
+async function fetchJson(url, headers = {}) {
+ const res = await fetch(url, { headers: { accept: 'application/json', 'user-agent': UA, ...headers }, signal: AbortSignal.timeout(PROBE_TIMEOUT_MS) });
+ if (!res.ok) throw new Error(`http_${res.status}`);
+ return res;
+}
+
+export const CANDLE_PROBES = [
+ {
+ id: 'kraken-ohlc',
+ run: async () => {
+ const res = await fetchJson(`https://api.kraken.com/0/public/OHLC?pair=SOLUSD&interval=1440&since=${KNOWN_CANDLE.time - 1}`);
+ const row = (await res.json())?.result?.SOLUSD?.[0];
+ return row ? { time: num(row[0]), open: num(row[1]), high: num(row[2]), low: num(row[3]), close: num(row[4]) } : null;
+ },
+ },
+ {
+ id: 'coinbase-candles',
+ run: async () => {
+ const iso = new Date(KNOWN_CANDLE.time * 1000).toISOString();
+ const res = await fetchJson(`https://api.exchange.coinbase.com/products/SOL-USD/candles?granularity=86400&start=${iso}&end=${iso}`);
+ const row = (await res.json())?.[0];
+ // Coinbase order: [time, low, high, open, close, volume]
+ return row ? { time: num(row[0]), low: num(row[1]), high: num(row[2]), open: num(row[3]), close: num(row[4]) } : null;
+ },
+ },
+];
+
+/** Run every probe, record every verdict, and return them. */
+export async function runCanaries(now = Date.now()) {
+ const verdicts = [];
+
+ const priceRuns = await Promise.all(
+ SOL_PRICE_PROVIDERS.map(async (p) => {
+ const r = await timed(async () => {
+ const res = await fetch(p.url, { headers: { accept: 'application/json', 'user-agent': UA, ...(p.init?.headers || {}) }, signal: AbortSignal.timeout(PROBE_TIMEOUT_MS) });
+ if (!res.ok) throw new Error(`http_${res.status}`);
+ return p.parse ? p.parse(res) : res.json();
+ });
+ return { id: p.name, ...r };
+ }),
+ );
+ const priceJudged = judgePrices(Object.fromEntries(priceRuns.map((r) => [r.id, num(r.value)])));
+ for (const r of priceRuns) {
+ const j = r.error ? { ok: false, detail: r.error } : priceJudged[r.id];
+ verdicts.push(await recordCanary({ id: r.id, kind: 'price', ok: j.ok, latencyMs: r.ms, detail: j.detail, now }));
+ }
+
+ for (const probe of CANDLE_PROBES) {
+ const r = await timed(probe.run);
+ const j = r.error ? { ok: false, detail: r.error } : judgeCandle(r.value);
+ verdicts.push(await recordCanary({ id: probe.id, kind: 'candle', ok: j.ok, latencyMs: r.ms, detail: j.detail, now }));
+ }
+ return verdicts;
+}
diff --git a/api/_lib/data-canary-health.js b/api/_lib/data-canary-health.js
new file mode 100644
index 000000000..ae66df7ed
--- /dev/null
+++ b/api/_lib/data-canary-health.js
@@ -0,0 +1,89 @@
+// Canary health store and fallback ordering for market-data sources.
+//
+// The canary cron (api/cron/data-canary.js) probes each source against a value
+// that is known to be true and records the verdict here. The data layer then asks
+// `orderByCanary` to sort a provider list so sources that just passed their probe
+// are tried first, sources never probed keep their configured position, and
+// sources that are failing sink to the back (they stay in the list: a canary can
+// be wrong, and a failing source is still better than no source).
+//
+// Storage is the shared cache adapter (Redis when configured, memory otherwise).
+// This module has no dependency on any provider list, so provider modules can
+// import it without a cycle.
+
+import { cacheGet, cacheSet } from './cache.js';
+
+export const CANARY_KEY = (id) => `data-canary:v1:${id}`;
+/** A verdict older than this is treated as unknown, not as healthy. */
+export const CANARY_TTL_S = 6 * 3600;
+export const CANARY_STALE_MS = 3 * 3600 * 1000;
+
+/**
+ * @typedef {{ id: string, kind: string, ok: boolean, checked_at: number, latency_ms: number, consecutive_failures: number, detail: string|null }} CanaryVerdict
+ */
+
+/** Persist one verdict, carrying the failure streak forward from the previous one. */
+export async function recordCanary({ id, kind, ok, latencyMs, detail = null, now = Date.now() }) {
+ let prev = null;
+ try {
+ prev = await cacheGet(CANARY_KEY(id));
+ } catch {
+ prev = null;
+ }
+ /** @type {CanaryVerdict} */
+ const verdict = {
+ id,
+ kind,
+ ok: !!ok,
+ checked_at: now,
+ latency_ms: Math.max(0, Math.round(latencyMs || 0)),
+ consecutive_failures: ok ? 0 : (prev?.consecutive_failures || 0) + 1,
+ detail: ok ? null : String(detail || 'failed').slice(0, 200),
+ };
+ await cacheSet(CANARY_KEY(id), verdict, CANARY_TTL_S);
+ return verdict;
+}
+
+export async function readCanaries(ids) {
+ const out = {};
+ await Promise.all(
+ ids.map(async (id) => {
+ try {
+ out[id] = (await cacheGet(CANARY_KEY(id))) || null;
+ } catch {
+ out[id] = null;
+ }
+ }),
+ );
+ return out;
+}
+
+/**
+ * Rank: 0 = passed recently, 1 = unknown or stale (configured order preserved),
+ * 2 = failing. Pure; exported for tests.
+ */
+export function canaryRank(verdict, now = Date.now()) {
+ if (!verdict || typeof verdict.checked_at !== 'number') return 1;
+ if (now - verdict.checked_at > CANARY_STALE_MS) return 1;
+ return verdict.ok ? 0 : 2;
+}
+
+/**
+ * Stable sort of `providers` (objects with a `name`) by canary health. Providers
+ * with no canary keep their relative order. Never drops a provider.
+ */
+export function orderProviders(providers, verdicts, now = Date.now()) {
+ return providers
+ .map((p, i) => ({ p, i, r: canaryRank(verdicts[p.name], now) }))
+ .sort((a, b) => a.r - b.r || a.i - b.i)
+ .map((x) => x.p);
+}
+
+export async function orderByCanary(providers, now = Date.now()) {
+ try {
+ const verdicts = await readCanaries(providers.map((p) => p.name));
+ return orderProviders(providers, verdicts, now);
+ } catch {
+ return providers;
+ }
+}
diff --git a/api/_lib/market-fallbacks.js b/api/_lib/market-fallbacks.js
index 14350275a..5e979c697 100644
--- a/api/_lib/market-fallbacks.js
+++ b/api/_lib/market-fallbacks.js
@@ -25,6 +25,7 @@ import { PAPRIKA_BASE, paprikaGet } from './coinpaprika.js';
import { CEX_BASE_BY_ID, cexPriceProviders, cexCandleProviders } from './cex-public.js';
import { downsample } from '../../src/shared/coin-format.js';
import { cacheGet, cacheSet } from './cache.js';
+import { orderByCanary } from './data-canary-health.js';
// null/undefined and the empty string must stay null, not become 0: Number(null)
// is 0, which is how a coin CoinGecko reports with market_cap_rank: null landed
@@ -483,7 +484,7 @@ export async function fetchExchangeChart(id, days) {
const base = CEX_BASE_BY_ID[id];
if (base) providers.push(...cexCandleProviders(base, days));
try {
- const { value } = await fetchFirst(providers, { timeoutMs: 8000, label: `chart:${id}` });
+ const { value } = await fetchFirst(await orderByCanary(providers), { timeoutMs: 8000, label: `chart:${id}` });
return value;
} catch {
return null;
diff --git a/api/_lib/research-context.js b/api/_lib/research-context.js
new file mode 100644
index 000000000..eb8397d5c
--- /dev/null
+++ b/api/_lib/research-context.js
@@ -0,0 +1,46 @@
+// ResearchContext: the only handle a backtest or research module gets.
+//
+// It is a frozen, read-only view of a recorded dataset plus its manifest. It has
+// no signing, no wallet, no order, no network and no database function, so
+// research code that receives one cannot move funds even by mistake.
+// tests/research-live-wall.test.js enforces the other half of the wall: no
+// module in RESEARCH_MODULES may import (directly or transitively) a live
+// execution module.
+
+import { buildDatasetManifest } from './backtest-integrity.js';
+
+/** Files that must stay on the research side of the wall. */
+export const RESEARCH_MODULES = Object.freeze([
+ 'api/_lib/research-context.js',
+ 'api/_lib/backtest-integrity.js',
+ 'api/_lib/strategy-backtest.js',
+ 'api/_lib/exit-replay.js',
+ 'api/_lib/strategy-preview.js',
+]);
+
+/**
+ * @param {object} args
+ * @param {string} args.datasetId
+ * @param {string} args.source
+ * @param {any[]} args.rows
+ * @param {(row:any)=>any} args.project
+ * @param {(row:any)=>any} args.timeOf
+ */
+export function createResearchContext({ datasetId, source, rows, project, timeOf }) {
+ const frozenRows = Object.freeze([...rows]);
+ const manifest = Object.freeze(buildDatasetManifest({ id: datasetId, source, rows: frozenRows, project, timeOf }));
+ return Object.freeze({
+ kind: 'research',
+ manifest,
+ rows: frozenRows,
+ /** Rows visible at or before `cutoff` (Date, ISO string or epoch ms). */
+ asOf(cutoff) {
+ const c = cutoff instanceof Date ? cutoff.getTime() : typeof cutoff === 'number' ? cutoff : Date.parse(cutoff);
+ return frozenRows.filter((r) => {
+ const t = timeOf(r);
+ const ms = t instanceof Date ? t.getTime() : typeof t === 'number' ? t : Date.parse(t);
+ return ms <= c;
+ });
+ },
+ });
+}
diff --git a/api/_lib/sol-price.js b/api/_lib/sol-price.js
index 3464fa42c..e6b112cad 100644
--- a/api/_lib/sol-price.js
+++ b/api/_lib/sol-price.js
@@ -15,6 +15,7 @@
// independently, so it takes a near-total outage to exhaust the chain.
import { fetchFirst } from '../../src/shared/failover-fetch.js';
+import { orderByCanary } from './data-canary-health.js';
import { redstoneProvider } from './redstone.js';
import { SWITCHBOARD_SOL_USD_FEED, switchboardProvider } from './switchboard.js';
@@ -128,7 +129,7 @@ let _at = 0;
export async function solPriceUsd(now = Date.now()) {
if (now - _at < TTL_MS && _price > 0) return _price;
try {
- const { value } = await fetchFirst(PROVIDERS, { timeoutMs: 3000, label: 'sol-price' });
+ const { value } = await fetchFirst(await orderByCanary(PROVIDERS), { timeoutMs: 3000, label: 'sol-price' });
_price = value;
_at = now;
} catch {
diff --git a/api/_lib/strategy-backtest.js b/api/_lib/strategy-backtest.js
index a4e8271cb..c95fd8207 100644
--- a/api/_lib/strategy-backtest.js
+++ b/api/_lib/strategy-backtest.js
@@ -23,6 +23,8 @@ import { decideExit } from '../../workers/agent-sniper/exit-logic.js';
import { getLearnedWeights } from '../../workers/agent-sniper/intel/store.js';
import { normalizeStrategyConfig, researchGatesActive } from './strategy-schema.js';
import { historicalGateReport, judgeHistoricalGates } from './strategy-preview.js';
+import { auditLookahead, checkPriorsCausal, withReproducibility } from './backtest-integrity.js';
+import { createResearchContext } from './research-context.js';
const LAMPORTS_PER_SOL = 1_000_000_000;
const MAX_CANDIDATES = 6000;
@@ -108,6 +110,61 @@ export function toIntelRecord(row) {
};
}
+function entryMarketCap(row) {
+ const athMult = n(row.ath_multiple);
+ const athMcUsd = n(row.ath_market_cap_usd);
+ return athMult != null && athMult > 0 && athMcUsd != null ? athMcUsd / athMult : null;
+}
+
+// The entry gate for one launch row: the same call the live worker makes.
+function entryVerdict(row, strategy, weights, useIntel) {
+ return useIntel
+ ? scoreIntel(toIntelRecord(row), strategy, weights)
+ : scoreMint(toMintEvent(row, entryMarketCap(row)), strategy);
+}
+
+// Fields of a row that can change a backtest result. Their hash is the dataset
+// manifest's content hash.
+const MANIFEST_FIELDS = [
+ 'mint', 'creator', 'first_seen_at', 'outcome', 'graduated', 'ath_multiple', 'ath_market_cap_usd',
+ 'last_market_cap_usd', 'quality_score', 'bundle_score', 'organic_score', 'category',
+ 'dev_buy_lamports', 'dev_sold', 'twitter', 'telegram', 'website', 'unique_buyers', 'sell_volume_lamports', 'buy_volume_lamports', 'creator_launches_prior', 'creator_graduated_prior',
+];
+const projectRow = (row) => [...MANIFEST_FIELDS.map((k) => row[k] ?? null), row.signals ?? null, row.risk_flags ?? null];
+
+/**
+ * Look-ahead audit of the entry gate over a captured universe. Each launch's
+ * verdict is recomputed with every later launch removed; it must not change.
+ * Creator priors are also checked for causality (they must rise with each
+ * earlier launch by the same creator, never fall).
+ */
+export function auditBacktestLookahead(rows, strategy, weights = null) {
+ const useIntel = strategy.trigger === 'intel_confirmed';
+ const gate = auditLookahead({
+ series: rows,
+ decide: (visible, i) => {
+ const v = entryVerdict(visible[i], strategy, weights, useIntel);
+ return { pass: !!v.pass, score: v.score ?? null };
+ },
+ });
+ const priors = checkPriorsCausal({
+ rows,
+ subjectOf: (r) => r.creator,
+ priorOf: (r) => n(r.creator_launches_prior),
+ });
+ return {
+ ok: gate.ok && priors.ok,
+ method: gate.method,
+ total: gate.total,
+ checked: gate.checked,
+ offending: gate.offending,
+ offending_truncated: gate.offending_truncated,
+ priors_causal: priors.ok,
+ priors_checked: priors.checked,
+ priors_offending: priors.offending,
+ };
+}
+
/**
* Model the realized return of one entered launch, honestly, from the two real
* price points we recorded: the peak multiple (ath_multiple) and the terminal
@@ -193,6 +250,14 @@ export async function runBacktest(strategy, { windowDays = 30, network = 'mainne
`;
const universeSize = rows.length;
+ const ctx = createResearchContext({
+ datasetId: `pump_coin_intel+pump_coin_outcomes:${network}:${days}d`,
+ source: 'postgres:pump_coin_intel+pump_coin_outcomes',
+ rows,
+ project: projectRow,
+ timeOf: (r) => r.first_seen_at,
+ });
+ const integrity = { manifest: ctx.manifest, lookahead: auditBacktestLookahead(ctx.rows, strategy, weights) };
const useIntel = strategy.trigger === 'intel_confirmed';
const perTradeSol = Number(BigInt(strategy.per_trade_lamports || '0')) / LAMPORTS_PER_SOL;
const stakeSol = perTradeSol > 0 ? perTradeSol : 0.1; // notional stake for drawdown/exposure when unset
@@ -209,13 +274,10 @@ export async function runBacktest(strategy, { windowDays = 30, network = 'mainne
for (const row of rows) {
const athMult = n(row.ath_multiple);
- const athMcUsd = n(row.ath_market_cap_usd);
- const entryMcUsd = athMult != null && athMult > 0 && athMcUsd != null ? athMcUsd / athMult : null;
+ const entryMcUsd = entryMarketCap(row);
// Entry gate — the SAME function the live worker runs.
- const verdict = useIntel
- ? scoreIntel(toIntelRecord(row), strategy, weights)
- : scoreMint(toMintEvent(row, entryMcUsd), strategy);
+ const verdict = entryVerdict(row, strategy, weights, useIntel);
if (!verdict.pass) { skip.gate++; continue; }
if (researchConfig) {
const gate = judgeHistoricalGates(researchConfig, historicalGateReport(row));
@@ -283,6 +345,8 @@ export async function runBacktest(strategy, { windowDays = 30, network = 'mainne
skipped: skip,
research_blocked_by: researchBlockedBy,
caveats,
+ integrity,
+ reproducible: true,
};
}
@@ -346,6 +410,8 @@ export async function runBacktest(strategy, { windowDays = 30, network = 'mainne
sample_hits: sampleHits,
sample_misses: sampleMisses,
caveats,
+ integrity,
+ reproducible: true,
ran_at: null, // stamped by the caller (Date.now is unavailable inside workflows)
};
}
@@ -372,7 +438,7 @@ export async function getCachedBacktest(hash, ttlMinutes = 30) {
AND ran_at > now() - (${Math.max(1, ttlMinutes)} || ' minutes')::interval
ORDER BY ran_at DESC LIMIT 1
`;
- return row ? { ...row.metrics, sample_size: row.sample_size, ran_at: row.ran_at, cached: true } : null;
+ return row ? { ...withReproducibility(row.metrics), sample_size: row.sample_size, ran_at: row.ran_at, cached: true } : null;
} catch {
return null;
}
diff --git a/api/cron/data-canary.js b/api/cron/data-canary.js
new file mode 100644
index 000000000..924d99d54
--- /dev/null
+++ b/api/cron/data-canary.js
@@ -0,0 +1,16 @@
+// GET /api/cron/data-canary: probe every market-data source against a known
+// price and a known historical candle, and record the verdicts that order the
+// data layer's fallback chains. Library: api/_lib/data-canaries.js.
+
+import { json, method, wrapCron } from '../_lib/http.js';
+import { requireCron } from '../_lib/cron-auth.js';
+import { runCanaries } from '../_lib/data-canaries.js';
+
+export default wrapCron(async (req, res) => {
+ if (!method(req, res, ['GET', 'POST'])) return;
+ if (!requireCron(req, res)) return;
+
+ const verdicts = await runCanaries();
+ const failing = verdicts.filter((v) => !v.ok).map((v) => ({ id: v.id, detail: v.detail, consecutive_failures: v.consecutive_failures }));
+ return json(res, 200, { ok: true, probed: verdicts.length, healthy: verdicts.length - failing.length, failing }, { 'cache-control': 'no-store' });
+});
diff --git a/data/changelog.json b/data/changelog.json
index e80a339a9..6df6bfc44 100644
--- a/data/changelog.json
+++ b/data/changelog.json
@@ -1,6 +1,13 @@
{
"$comment": "Curated changelog entries for three.ws, the editorial layer of the public changelog. New PAGE launches are tracked automatically via the `added` field in data/pages.json; this file is for everything else holders care about: improvements, fixes, SDK releases, trading/payments capabilities, security work. scripts/build-page-index.mjs merges both sources into CHANGELOG.md, public/changelog.json, and public/changelog.xml. Newest first. Allowed tags: feature, improvement, fix, sdk, infra, docs, security, desktop (desktop entries become the desktop app's release notes). `link` is optional and must be a live three.ws path.",
"entries": [
+ {
+ "date": "2026-10-23",
+ "title": "Backtests now prove they are free of look-ahead and can be repeated",
+ "summary": "Every strategy backtest replays its entry gate with the future removed and reports any decision that changed, pins the exact dataset it ran on with a content hash, and marks older unpinned results as unreproducible. A new canary job checks each price and candle source against a known value every 30 minutes and tries the healthy ones first.",
+ "tags": ["feature"],
+ "link": "/strategies"
+ },
{
"date": "2026-10-23",
"title": "One earnings ledger for everything your agents earn, with reinvest rules",
diff --git a/data/pages.json b/data/pages.json
index 864621d4c..99523855d 100644
--- a/data/pages.json
+++ b/data/pages.json
@@ -465,6 +465,14 @@
"changefreq": "monthly",
"added": "2026-10-23"
},
+ {
+ "path": "/docs/backtest-integrity",
+ "title": "Docs · Backtest integrity",
+ "description": "How every strategy backtest is audited for look-ahead bias, pinned to a hashed dataset manifest, backed by price and candle canaries, and walled off from live trading code.",
+ "priority": 0.6,
+ "changefreq": "monthly",
+ "added": "2026-10-23"
+ },
{
"path": "/commerce",
"title": "Agent Commerce",
diff --git a/docs/backtest-integrity.md b/docs/backtest-integrity.md
new file mode 100644
index 000000000..d324a1d8b
--- /dev/null
+++ b/docs/backtest-integrity.md
@@ -0,0 +1,34 @@
+# Backtest integrity
+
+A backtest number is only worth showing if it can be trusted and repeated. Three checks run on every sniper backtest (`POST /api/sniper/backtest`) and a fourth keeps research code away from live trading.
+
+## Look-ahead audit
+
+`auditLookahead` in [api/_lib/backtest-integrity.js](../api/_lib/backtest-integrity.js) replays a decision function twice per audited point: once with the full series and once with everything after that point removed. If the decision changes, it used future data. The result lists the offending indexes.
+
+For the sniper backtest, the decision is the entry gate (`scoreMint` or `scoreIntel`, the same call the live worker makes) for each captured launch, so `auditBacktestLookahead` in [api/_lib/strategy-backtest.js](../api/_lib/strategy-backtest.js) audits the real gate. Creator history counters (launches and graduations before this launch) are also checked for causality: for one creator they must rise with every earlier launch, never stall or fall. Series longer than 400 decisions are sampled on an even stride that always includes the first and last decision (`checked` vs `total` in the report).
+
+The result sits under `integrity.lookahead` in the response and is shown on the Strategy tab of the wallet hub and in the strategy settings backtest panel.
+
+## Dataset manifests
+
+Each backtest records `integrity.manifest`: dataset id (table, network, window), source, range, row count and a SHA-256 over the fields that determine the result. Two runs with the same hash saw the same data. A snapshot without a complete manifest (every one saved before this shipped) comes back with `reproducible: false` and an `unreproducible_reason`; the UI says to run it again.
+
+## Canaries
+
+`/api/cron/data-canary` runs every 30 minutes ([api/cron/data-canary.js](../api/cron/data-canary.js), schedule in `vercel.json`). It probes:
+
+- every SOL/USD price provider in [api/_lib/sol-price.js](../api/_lib/sol-price.js): positive and within 5% of the median of all answering providers;
+- a closed historical daily SOL/USD candle (2025-01-01 UTC) on Kraken and Coinbase: right timestamp, consistent OHLC, close within 2% of the reference.
+
+Verdicts are stored in the shared cache ([api/_lib/data-canary-health.js](../api/_lib/data-canary-health.js)). `orderByCanary` sorts a provider list: recently passing sources first, never-probed or stale ones in configured order, failing ones last. Nothing is ever dropped from a chain. It is applied to the SOL price chain and the coin chart candle chain.
+
+Register the schedule with the usual `npm run create:gcp-scheduler` flow at the next deploy; the entry is in `vercel.json` only.
+
+## Research/live wall
+
+Research modules receive a `ResearchContext` ([api/_lib/research-context.js](../api/_lib/research-context.js)): a frozen, read-only view of a recorded dataset and its manifest. It has no signing, wallet, order, network or database function.
+
+`RESEARCH_MODULES` lists the files on the research side. [scripts/lib/research-wall.mjs](../scripts/lib/research-wall.mjs) follows their static and literal dynamic imports transitively and reports any path into a live execution module (execution engine, wallets, custody, signers, the sniper executor, order and swap modules) or a signing package, with the import chain. `tests/backtest-integrity.test.js` fails the suite on any violation. Add a new backtest or research file to `RESEARCH_MODULES`.
+
+Tests: [tests/backtest-integrity.test.js](../tests/backtest-integrity.test.js).
diff --git a/scripts/lib/research-wall.mjs b/scripts/lib/research-wall.mjs
new file mode 100644
index 000000000..6153133f7
--- /dev/null
+++ b/scripts/lib/research-wall.mjs
@@ -0,0 +1,84 @@
+// Static import-graph check for the research/live wall.
+//
+// Starting from each research module, follow every relative static import,
+// re-export and dynamic import('...') with a literal path. Report any reachable
+// file whose repo-relative path matches a live-execution pattern, with the import
+// chain that reaches it. Bare package specifiers that can sign or send are checked
+// by name.
+
+import { readFileSync, existsSync, statSync } from 'node:fs';
+import { dirname, resolve, relative, join } from 'node:path';
+
+/** Repo-relative path patterns of modules that can sign, hold keys, send or order. */
+export const LIVE_PATH_PATTERNS = [
+ /(^|\/)execution-engine\.js$/,
+ /(^|\/)(landing-race|tx-landing|jito|send-transaction)[^/]*\.js$/,
+ /(^|\/)(agent-)?wallet[^/]*\.js$/,
+ /(^|\/)custody[^/]*\.js$/,
+ /(^|\/)(signer|signing|sign-[^/]*|keypair[^/]*)\.js$/,
+ /(^|\/)workers\/agent-sniper\/(executor|buyer|seller|position-loop|index|run|trade|swap)[^/]*\.js$/,
+ /(^|\/)api\/_lib\/pump\.js$/,
+ /(^|\/)api\/_lib\/(order|orders|swap|jupiter-swap|trade)[^/]*\.js$/,
+ /(^|\/)api\/_lib\/x402-pay[^/]*\.js$/,
+];
+
+/** Bare specifiers that give a module the ability to sign or send. */
+export const LIVE_PACKAGE_PATTERNS = [/^@solana\/web3\.js$/, /^@solana\/kit$/, /^ethers$/, /^viem(\/|$)/, /^tweetnacl$/, /^@coral-xyz\/anchor$/];
+
+const IMPORT_RE = /(?:^|[\s;])(?:import|export)\s+(?:[^'"()]*?\s+from\s+)?['"]([^'"]+)['"]|import\(\s*['"]([^'"]+)['"]\s*\)/g;
+
+function stripComments(src) {
+ return src.replace(/\/\*[\s\S]*?\*\//g, '').replace(/(^|[^:'"`])\/\/[^\n]*/g, '$1');
+}
+
+function resolveRel(fromFile, spec) {
+ const base = resolve(dirname(fromFile), spec);
+ for (const cand of [base, `${base}.js`, `${base}.mjs`, join(base, 'index.js')]) {
+ if (existsSync(cand) && statSync(cand).isFile()) return cand;
+ }
+ return null;
+}
+
+/**
+ * @param {string[]} entries Repo-relative research module paths.
+ * @param {{ root: string, patterns?: RegExp[], packages?: RegExp[] }} opts
+ * @returns {{ violations: { entry: string, target: string, chain: string[] }[], visited: number }}
+ */
+export function findWallViolations(entries, { root, patterns = LIVE_PATH_PATTERNS, packages = LIVE_PACKAGE_PATTERNS }) {
+ const violations = [];
+ const seenAll = new Set();
+ for (const entry of entries) {
+ const start = resolve(root, entry);
+ const seen = new Set([start]);
+ const queue = [[start, [entry]]];
+ while (queue.length) {
+ const [file, chain] = queue.shift();
+ seenAll.add(file);
+ const rel = relative(root, file);
+ if (file !== start && patterns.some((p) => p.test(rel))) {
+ violations.push({ entry, target: rel, chain });
+ continue;
+ }
+ let src;
+ try {
+ src = stripComments(readFileSync(file, 'utf8'));
+ } catch {
+ continue;
+ }
+ for (const m of src.matchAll(IMPORT_RE)) {
+ const spec = m[1] || m[2];
+ if (!spec) continue;
+ if (spec.startsWith('.')) {
+ const next = resolveRel(file, spec);
+ if (next && !seen.has(next)) {
+ seen.add(next);
+ queue.push([next, [...chain, relative(root, next)]]);
+ }
+ } else if (!spec.startsWith('node:') && packages.some((p) => p.test(spec))) {
+ violations.push({ entry, target: spec, chain: [...chain, spec] });
+ }
+ }
+ }
+ }
+ return { violations, visited: seenAll.size };
+}
diff --git a/src/agent-wallet-hub/tabs/snipe.js b/src/agent-wallet-hub/tabs/snipe.js
index 9711acd25..c5f73d601 100644
--- a/src/agent-wallet-hub/tabs/snipe.js
+++ b/src/agent-wallet-hub/tabs/snipe.js
@@ -20,6 +20,7 @@
import { registerWalletTab } from '../registry.js';
import { consumeCsrfToken } from '../../api.js';
import { ensureRiskAck } from '../../shared/risk-ack.js';
+import { renderIntegrity } from '../../shared/backtest-integrity-view.js';
import { formatSol, shortAddress, explorerAddressUrl } from '../util.js';
const LAMPORTS_PER_SOL = 1_000_000_000;
@@ -385,6 +386,7 @@ registerWalletTab({
${b.sample_misses?.length ? `<h4 style="margin-top:10px">Worst simulated entries</h4>${b.sample_misses.map(trade).join('')}` : ''}
</div>` : ''}
+ ${renderIntegrity(b)}
<div class="sb-caveats">${b.stake_assumed ? `Modeled at a ${b.stake_sol} SOL notional stake (set a per-trade size to model your real size). ` : `Modeled at ${b.stake_sol} SOL per trade. `}
<ul>${(b.caveats?.items || []).map((c) => `<li>${esc(c)}</li>`).join('')}</ul>
</div>
diff --git a/src/shared/backtest-integrity-view.js b/src/shared/backtest-integrity-view.js
new file mode 100644
index 000000000..1caaadbe3
--- /dev/null
+++ b/src/shared/backtest-integrity-view.js
@@ -0,0 +1,40 @@
+// Renders the backtest integrity report (look-ahead audit + dataset manifest)
+// returned by POST /api/sniper/backtest. Shared by the wallet hub Strategy tab
+// and the strategy settings panel so both show the same verdict.
+
+const esc = (s) =>
+ String(s ?? '').replace(/[&<>"']/g, (c) => ({ '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;' })[c]);
+
+const BOX = 'margin-top:12px;padding:10px 12px;border:1px solid var(--border,rgba(255,255,255,.14));border-radius:8px;font-size:.78rem;line-height:1.55;';
+const TAG = (color, text) => `<b style="color:${color}">${text}</b>`;
+
+/**
+ * @param {object} r A backtest response.
+ * @returns {string} HTML, or '' when the response carries no integrity data and
+ * is not an old snapshot (nothing truthful to say).
+ */
+export function renderIntegrity(r) {
+ if (!r || typeof r !== 'object') return '';
+ const i = r.integrity || {};
+ const la = i.lookahead;
+ const m = i.manifest;
+ const rows = [];
+
+ if (la) {
+ if (la.ok) {
+ rows.push(`${TAG('var(--ok,#4ade80)', 'Look-ahead audit passed.')} ${la.checked} of ${la.total} entry decisions were replayed with every later launch removed and none changed.`);
+ } else {
+ const idx = (la.offending || []).slice(0, 8).join(', ');
+ const priors = la.priors_causal === false ? ' Creator history counters moved backwards in time.' : '';
+ rows.push(`${TAG('var(--err,#f87171)', 'Look-ahead audit failed.')} ${(la.offending || []).length ? `Decision${la.offending.length === 1 ? '' : 's'} at index ${esc(idx)}${la.offending_truncated ? ' and more' : ''} changed when future data was removed.` : ''}${priors} Do not trust these numbers.`);
+ }
+ }
+
+ if (m && r.reproducible !== false) {
+ rows.push(`${TAG('var(--ok,#4ade80)', 'Reproducible.')} Dataset <code>${esc(m.id)}</code>, ${Number(m.row_count).toLocaleString()} rows, ${esc((m.range?.from || '').slice(0, 10))} to ${esc((m.range?.to || '').slice(0, 10))}, hash <code title="${esc(m.content_hash)}">${esc(String(m.content_hash).slice(0, 12))}</code>.`);
+ } else if (r.reproducible === false) {
+ rows.push(`${TAG('var(--warn,#fbbf24)', 'Unreproducible.')} ${esc(i.unreproducible_reason || 'No dataset manifest was recorded for this run.')} Run the backtest again to get a pinned result.`);
+ }
+
+ return rows.length ? `<div class="bt-integrity" role="note" style="${BOX}">${rows.map((x) => `<div>${x}</div>`).join('')}</div>` : '';
+}
diff --git a/src/shared/strategy-settings.js b/src/shared/strategy-settings.js
index 29e2b8d96..1e3240765 100644
--- a/src/shared/strategy-settings.js
+++ b/src/shared/strategy-settings.js
@@ -14,6 +14,7 @@
import { apiFetch } from '../api.js';
import { esc, shortAddr, timeAgo, VIOLET } from './strategy-forms.js';
+import { renderIntegrity } from './backtest-integrity-view.js';
// Plain-language names for every reason a launch can be filtered out. Entry
// keys come from matchesEntry; research keys mirror RESEARCH_CHECK_LABELS in
@@ -419,6 +420,7 @@ function renderBacktest(r) {
</div>
<p class="sx-note">${fmtN(r.sample_size)} of ${fmtN(r.universe_size)} recorded launches over ${r.window_days} days passed your rules${r.cached ? ' (cached result for these exact settings)' : ''}.</p>
${blockedList(r)}
+ ${renderIntegrity(r)}
${(r.caveats || []).length ? `<details class="sx-det"><summary>Caveats</summary><ul>${r.caveats.map((c) => `<li>${esc(c)}</li>`).join('')}</ul></details>` : ''}`;
}
diff --git a/tests/backtest-integrity.test.js b/tests/backtest-integrity.test.js
new file mode 100644
index 000000000..9f1986a16
--- /dev/null
+++ b/tests/backtest-integrity.test.js
@@ -0,0 +1,216 @@
+// Backtest integrity: look-ahead audit, dataset manifests, prior causality,
+// canary ordering, the research/live import wall, and the audit wired into the
+// real entry gate.
+
+import { describe, it, expect } from 'vitest';
+import { mkdtempSync, mkdirSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import {
+ auditLookahead,
+ checkPriorsCausal,
+ buildDatasetManifest,
+ isCompleteManifest,
+ withReproducibility,
+} from '../api/_lib/backtest-integrity.js';
+import { createResearchContext, RESEARCH_MODULES } from '../api/_lib/research-context.js';
+import { auditBacktestLookahead } from '../api/_lib/strategy-backtest.js';
+import { orderProviders, canaryRank, CANARY_STALE_MS } from '../api/_lib/data-canary-health.js';
+import { judgeCandle, judgePrices, KNOWN_CANDLE } from '../api/_lib/data-canaries.js';
+import { findWallViolations } from '../scripts/lib/research-wall.mjs';
+
+const bars = Array.from({ length: 60 }, (_, i) => ({ t: i, close: 100 + Math.sin(i / 5) * 10 + i * 0.1 }));
+
+// Clean: decides from the bars up to and including its own index.
+const cleanDecide = (visible, i) => {
+ const window = visible.slice(Math.max(0, i - 4), i + 1);
+ const sma = window.reduce((a, b) => a + b.close, 0) / window.length;
+ return visible[i].close > sma ? 'buy' : 'hold';
+};
+// Look-ahead: peeks at the next bar's close.
+const peekDecide = (visible, i) => {
+ const next = visible[i + 1];
+ return next && next.close > visible[i].close ? 'buy' : 'hold';
+};
+
+describe('auditLookahead', () => {
+ it('passes a clean strategy', () => {
+ const r = auditLookahead({ series: bars, decide: cleanDecide });
+ expect(r.ok).toBe(true);
+ expect(r.offending).toEqual([]);
+ expect(r.checked).toBe(60);
+ });
+
+ it('fails a strategy that reads the next bar and reports the offending indexes', () => {
+ const r = auditLookahead({ series: bars, decide: peekDecide });
+ expect(r.ok).toBe(false);
+ expect(r.offending.length).toBeGreaterThan(0);
+ // The last decision has no next bar in either view, so it cannot differ.
+ expect(r.offending).not.toContain(59);
+ for (const i of r.offending) expect(i).toBeLessThan(59);
+ });
+
+ it('samples evenly but always includes the first and last decision on long series', () => {
+ const long = Array.from({ length: 5000 }, (_, i) => ({ t: i, close: i }));
+ const r = auditLookahead({ series: long, decide: (v, i) => v[i].close, maxChecks: 100 });
+ expect(r.ok).toBe(true);
+ expect(r.checked).toBeLessThanOrEqual(102);
+ expect(r.total).toBe(5000);
+ });
+
+ it('treats a decision that throws on truncated data as a failure', () => {
+ const r = auditLookahead({ series: bars, decide: (v, i) => v[i + 1].close });
+ expect(r.ok).toBe(false);
+ expect(r.errors).toBeGreaterThan(0);
+ });
+});
+
+describe('checkPriorsCausal', () => {
+ const mk = (priors) => priors.map((p, i) => ({ creator: 'c', prior: p, i }));
+ it('accepts counters that rise with each earlier launch', () => {
+ expect(checkPriorsCausal({ rows: mk([0, 1, 2, 5]), subjectOf: (r) => r.creator, priorOf: (r) => r.prior }).ok).toBe(true);
+ });
+ it('flags a counter that stalls or falls', () => {
+ const r = checkPriorsCausal({ rows: mk([0, 1, 1, 0]), subjectOf: (x) => x.creator, priorOf: (x) => x.prior });
+ expect(r.ok).toBe(false);
+ expect(r.offending).toEqual([2, 3]);
+ });
+});
+
+describe('dataset manifest', () => {
+ const rows = [
+ { id: 'a', at: '2026-01-02T00:00:00Z', v: 1 },
+ { id: 'b', at: '2026-01-05T00:00:00Z', v: 2 },
+ ];
+ const args = { id: 'ds', source: 'unit', project: (r) => [r.id, r.v], timeOf: (r) => r.at };
+
+ it('records id, source, range, row count and a stable content hash', () => {
+ const m = buildDatasetManifest({ ...args, rows });
+ expect(isCompleteManifest(m)).toBe(true);
+ expect(m.row_count).toBe(2);
+ expect(m.range).toEqual({ from: '2026-01-02T00:00:00.000Z', to: '2026-01-05T00:00:00.000Z' });
+ expect(buildDatasetManifest({ ...args, rows }).content_hash).toBe(m.content_hash);
+ });
+
+ it('changes the hash when any row changes', () => {
+ const a = buildDatasetManifest({ ...args, rows });
+ const b = buildDatasetManifest({ ...args, rows: [rows[0], { ...rows[1], v: 3 }] });
+ expect(b.content_hash).not.toBe(a.content_hash);
+ });
+
+ it('marks a result without a manifest unreproducible and one with a manifest reproducible', () => {
+ const old = withReproducibility({ ok: true, sample_size: 3 });
+ expect(old.reproducible).toBe(false);
+ expect(old.integrity.unreproducible_reason).toMatch(/manifest/);
+ const m = buildDatasetManifest({ ...args, rows });
+ expect(withReproducibility({ ok: true, integrity: { manifest: m } }).reproducible).toBe(true);
+ expect(withReproducibility({ ok: true, integrity: { manifest: { id: 'x' } } }).reproducible).toBe(false);
+ });
+
+ it('exposes a frozen ResearchContext with no live capability', () => {
+ const ctx = createResearchContext({ datasetId: 'ds', source: 'unit', rows, project: args.project, timeOf: args.timeOf });
+ expect(Object.isFrozen(ctx)).toBe(true);
+ expect(ctx.asOf('2026-01-03T00:00:00Z')).toHaveLength(1);
+ const names = Object.keys(ctx).join(' ');
+ expect(names).not.toMatch(/sign|wallet|order|send|swap|key/i);
+ });
+});
+
+describe('entry gate audit on the real scorer', () => {
+ const row = (i, creator, prior) => ({
+ mint: `THREEsynthetic${i}`,
+ creator,
+ first_seen_at: new Date(Date.UTC(2026, 0, 1 + i)),
+ outcome: 'flat',
+ ath_multiple: 2,
+ ath_market_cap_usd: 40000,
+ last_market_cap_usd: 20000,
+ creator_launches_prior: prior,
+ creator_graduated_prior: 0,
+ signals: {},
+ risk_flags: [],
+ });
+ const strategy = { trigger: 'new_mint', slippage_bps: 500, require_sol_quote: true, min_market_cap_usd: 5000 };
+
+ it('passes when each launch is judged on its own recorded fields', () => {
+ const rows = [row(0, 'cA', 0), row(1, 'cB', 0), row(2, 'cA', 1)];
+ const r = auditBacktestLookahead(rows, strategy);
+ expect(r.ok).toBe(true);
+ expect(r.priors_causal).toBe(true);
+ });
+
+ it('fails when creator history counters go backwards', () => {
+ const rows = [row(0, 'cA', 2), row(1, 'cA', 0)];
+ const r = auditBacktestLookahead(rows, strategy);
+ expect(r.ok).toBe(false);
+ expect(r.priors_causal).toBe(false);
+ expect(r.priors_offending).toEqual([1]);
+ });
+});
+
+describe('canaries', () => {
+ const now = 1_800_000_000_000;
+ const ok = { ok: true, checked_at: now - 1000 };
+ const bad = { ok: false, checked_at: now - 1000 };
+
+ it('orders healthy sources first, unknown next in configured order, failing last', () => {
+ const providers = [{ name: 'a' }, { name: 'b' }, { name: 'c' }, { name: 'd' }];
+ const out = orderProviders(providers, { a: bad, b: null, c: ok, d: null }, now).map((p) => p.name);
+ expect(out).toEqual(['c', 'b', 'd', 'a']);
+ });
+
+ it('never drops a provider and treats a stale verdict as unknown', () => {
+ const providers = [{ name: 'a' }, { name: 'b' }];
+ const stale = { ok: false, checked_at: now - CANARY_STALE_MS - 1 };
+ expect(canaryRank(stale, now)).toBe(1);
+ expect(orderProviders(providers, { a: stale, b: bad }, now)).toHaveLength(2);
+ });
+
+ it('judges the known candle', () => {
+ const good = { time: KNOWN_CANDLE.time, open: 188.82, high: 194.99, low: 187.6, close: 193.84 };
+ expect(judgeCandle(good).ok).toBe(true);
+ expect(judgeCandle({ ...good, time: good.time + 86400 }).ok).toBe(false);
+ expect(judgeCandle({ ...good, close: 150 }).ok).toBe(false);
+ expect(judgeCandle({ ...good, high: 100 }).ok).toBe(false);
+ expect(judgeCandle(null).ok).toBe(false);
+ });
+
+ it('flags a price that disagrees with the median and one that is missing', () => {
+ const j = judgePrices({ a: 190, b: 191, c: 192, d: 400, e: null });
+ expect(j.a.ok && j.b.ok && j.c.ok).toBe(true);
+ expect(j.d.ok).toBe(false);
+ expect(j.e.ok).toBe(false);
+ });
+});
+
+describe('research/live wall', () => {
+ const root = process.cwd();
+
+ it('no research module reaches a live execution module', () => {
+ const { violations, visited } = findWallViolations(RESEARCH_MODULES, { root });
+ expect(visited).toBeGreaterThan(RESEARCH_MODULES.length);
+ expect(violations.map((v) => `${v.entry} -> ${v.chain.join(' > ')}`)).toEqual([]);
+ });
+
+ it('fails when a research file imports an execution module (transitively)', () => {
+ const dir = mkdtempSync(join(tmpdir(), 'wall-'));
+ mkdirSync(join(dir, 'api/_lib'), { recursive: true });
+ writeFileSync(join(dir, 'api/_lib/execution-engine.js'), 'export const submit = () => {};\n');
+ writeFileSync(join(dir, 'api/_lib/helper.js'), "import { submit } from './execution-engine.js';\nexport const h = submit;\n");
+ writeFileSync(join(dir, 'api/_lib/research-x.js'), "import { h } from './helper.js';\nexport const x = h;\n");
+ const { violations } = findWallViolations(['api/_lib/research-x.js'], { root: dir });
+ expect(violations).toHaveLength(1);
+ expect(violations[0].target).toBe('api/_lib/execution-engine.js');
+ expect(violations[0].chain).toEqual(['api/_lib/research-x.js', 'api/_lib/helper.js', 'api/_lib/execution-engine.js']);
+ });
+
+ it('fails on a dynamic import and on a signing package', () => {
+ const dir = mkdtempSync(join(tmpdir(), 'wall-'));
+ mkdirSync(join(dir, 'api/_lib'), { recursive: true });
+ writeFileSync(join(dir, 'api/_lib/wallet-sign.js'), 'export const s = 1;\n');
+ writeFileSync(join(dir, 'api/_lib/r1.js'), "export const f = () => import('./wallet-sign.js');\n");
+ writeFileSync(join(dir, 'api/_lib/r2.js'), "import { Keypair } from '@solana/web3.js';\nexport const k = Keypair;\n");
+ expect(findWallViolations(['api/_lib/r1.js'], { root: dir }).violations).toHaveLength(1);
+ expect(findWallViolations(['api/_lib/r2.js'], { root: dir }).violations).toHaveLength(1);
+ });
+});