Skip to content

Claude Code. Any Model. The most powerful AI coding agent now speaks every language.

docs: record the v3 remediation and d0c1538 reviews

Jack Rudenko8bb4097fca45
diff --git a/ai-docs/reports/REVIEW_models_index_v3_d0c1538-20260914.md b/ai-docs/reports/REVIEW_models_index_v3_d0c1538-20260914.md
new file mode 100644
index 00000000..535149cb
--- /dev/null
+++ b/ai-docs/reports/REVIEW_models_index_v3_d0c1538-20260914.md
@@ -0,0 +1,342 @@
+# Follow-up review — models-index v3, commit `d0c1538`
+
+**Reviewer:** claudish client team
+**Target:** `d0c1538` ("fix(catalog): close v3 publication blockers"), PR #151, parent `ba4fee1` (a merge of upstream main into the v3 branch; the code change under review is `d7acfd7..d0c1538`)
+**Prior reviews:** `REVIEW_models_index_v3_impl-20260913.md` (six findings), `REVIEW_models_index_v3_remediation-20260913.md` (FAIL — CRITICAL-1 + HIGH-1(new))
+**Spec:** `~/.claude/plans/gleaming-discovering-marble.md`
+**Date:** 2026-09-14
+**Mode:** read-only. Nothing in that repo was modified, staged, executed, or tested. Their worktree is still mid-merge with conflict markers on disk, so **every line below was read out of the commit object itself**, never the filesystem. Anyone re-checking this must do the same; the on-disk tree will mislead you.
+
+## Verdict
+
+**CONDITIONAL** — 0 CRITICAL, 1 HIGH, 3 MEDIUM, 2 LOW.
+
+**Both publication blockers from the last round are genuinely closed.** CRITICAL-1's
+unconditional `throw` is gone and the replacement is coherent rather than a papering-over:
+`candidateModelIds` is now provably the exact final published id set, and every consumer that
+previously disagreed about it reads that one set. HIGH-1(new) is fixed in the correct direction
+and asserted end-to-end. **This is no longer a pre-cutover blocker.** By the letter of the
+severity thresholds (0 CRITICAL, fewer than 3 HIGH) this is a PASS; I am reporting CONDITIONAL
+rather than PASS because the one HIGH is a silent partial-catalog outage in precisely the area
+the last two rounds were about, and because it is invisible to their own operator alerting as
+well as to us.
+
+Nothing here touches the wire contract. Our 426 guard remains correct and pinned to their
+committed fixture.
+
+## Item-by-item against what you asked
+
+| # | Question | Result |
+|---|---|---|
+| — | Can the CRITICAL-1 filter drop a legitimate model? | **YES — HIGH-1 below.** Resolution is real but gated on a ~43-entry hand-maintained slug table, and the drop is unrecorded |
+| 1 | exact-ID official-callability for all Mistral routes | **VERIFIED SAFE** — narrower than it reads; no model is dropped and no aggregator route is touched |
+| 2 | HIGH-1(new) supplemental reconciliation | **FIXED** for the reported case, asserted; introduces MEDIUM-1 (expiry now strips a live inclusion, untested) |
+| 3 | `merger.ts` identity prep survived the 519-line merge | **VERIFIED functionally** — `index.ts:458-489` still prepares once and feeds both consumers |
+| 4 | seven §4.2 rows still publish; reasons still split 2/5 | **VERIFIED** (a)(b) — and that seam test's fixtures *are* production-faithful |
+| 5 | `canonicalizeModelId` promoting `mistral-medium-latest` | **STILL OPEN — MEDIUM-2**, and now unreachable-by-construction for three pointers |
+| 6 | diff projection sorting roster entries / inclusions | **STILL OPEN — MEDIUM-3** |
+| 7 | LOW error-vocabulary deferral recorded durably | **STILL OPEN — LOW-1**; a repo-wide search of `docs/` and `CHANGELOG.md` finds nothing |
+
+---
+
+## Findings, most severe first
+
+### HIGH-1 — The quarantine now withholds a Mistral model silently, and the "owner-source-backed" resolution that is supposed to save it is gated on a hand-maintained 43-row slug table. The withholding is invisible to clients *and* to their own provider-drop alert. (a) for every step; (b)/(c) for which pointers occur live
+
+**Locations:** `catalog-v3-builder.ts:109-111` (`publishableMergedModels`), `:402-437`
+(`collectServingEvidence` skip), `:440-444`, `:641-665` (`buildModelChangesV3`);
+`merger.ts:402-444` (`applyMistralPointerTargets`), `:446-476`
+(`sourceBackedMistralPointerTargets`); `collectors/scraper/mistral-model-pages.ts:164-170`,
+`:268-302`; `collectors/scraper/mistral-pricing.ts:158-176`, `:426-437`;
+`collectors/api/mistral.ts:185`; `index.ts:529-536`.
+
+**The drop is unconditional and unrecorded.** `publishableMergedModels` filters the row out and
+`collectServingEvidence` skips it. Neither records a violation. The orphan-route guard at
+`:150-155` cannot see it either, because the evidence was skipped rather than orphaned.
+`buildModelChangesV3` only diffs active-vs-current, so a pointer that never published produces
+no change document. Nothing on the wire says a model was withheld.
+
+**The resolution path is narrower than the contract doc now claims.**
+`docs/subscription-routing-contract.md` says pointers resolve "when an owner source binds them
+to an immutable identity". In production that requires a **non-`mistral-api`** Mistral-owner row
+whose `externalId` **is** the pointer and whose `canonicalId` is a non-pointer
+(`merger.ts:446-476`, which discards any row where `target === pointerId`). Two facts collapse
+that set:
+
+1. **`mistral-model-pages-scrape` can never satisfy it.** `parseMistralModelPage` returns
+ `externalId: modelId, canonicalId: modelId` (`mistral-model-pages.ts:167-168`) — always
+ equal. Its `extractModelId` resolves the *slug*, so the OCR-4 page yields `mistral-ocr-4-0`
+ for both fields and the page's copy-to-clipboard `mistral-ocr-latest` is discarded. Their own
+ `mistral-model-pages.test.ts:210-228` asserts exactly that.
+2. **So the only backing source is `mistral-pricing-scrape`**, whose `canonicalId` comes from
+ `canonicalModelIdFromMistralCardLink` or `canonicalIdFromPricingRecord`
+ (`mistral-pricing.ts:426-437`) → `MISTRAL_PINNED_API_ID_MAP` (7 rows) or
+ `canonicalModelIdFromMistralDisplayName` → `findMappedMistralSlugForTitle` →
+ **`MISTRAL_MODEL_PAGE_SLUG_ID_MAP`, roughly 43 hand-written slug→dated rows**, plus one
+ dashed-decimal regex. On a miss it returns `apiName`, i.e. the pointer — no backing. And
+ `mistral-pricing.ts:158` `continue`s on any card whose pricing metrics do not parse, so such
+ a card emits no row at all.
+
+So the generic-sounding rule has the same reach as a curated table; it was relocated from
+`mistral-identity-policy.ts` into a scraper, not generalised.
+
+**Concrete failure scenario A — a new model is invisible.** Mistral ships a model and
+`/v1/models` exposes it as `X-latest` hours or days before the pricing page and slug map carry
+it. `reviewedMistralCanonicalIdForPointerV3` misses, `sourceBackedTargets` misses, the fallback
+at `merger.ts:437-441` keeps `canonicalId` as the pointer, `mergePreparedResults` publishes a
+doc whose `modelId` is the pointer, and `:109-111` drops it. If no other collector produced a
+dated doc, **the model is absent from the catalog** behind a well-formed 200 with an advancing
+`generationId`. For us: `search_models` never returns it, `mistralai@X` has no catalog row, and
+any routing chain that would have included mistralai for that model loses the provider — with
+no error and not even an `unknown` evidence row to hint at it.
+
+**Concrete failure scenario B — a published model freezes forever.** The pricing card's layout
+changes so the pointer-to-dated binding disappears while the API keeps serving the pointer.
+Because `collectors/api/mistral.ts:185` declares
+`completeness: { models: "partial", servingRoutes: "partial" }`, `mistral-api` is never in
+`completeModelCollectorIds`, so `retainedActiveModelIds` (`:115-130`) always retains the stale
+dated doc and `reconcileAggregatorRoutes([], active.aggregators, …)` keeps its stale mistralai
+route. The model therefore never disappears — it stops updating. Context window, pricing and
+deprecation status freeze at the last run that resolved the pointer. The only signal is
+`dataFreshnessWarning: true` on the model doc.
+
+**Their operator alert is blind to it too.** `index.ts:531` calls `buildProviderCounts(merged)`
+on the **pre-filter** merged list. A withheld Mistral pointer still counts toward `mistralai`,
+so `detectProviderDrops` sees no drop and `alertProviderDrop` never fires. The one mechanism in
+the repo built to catch "a provider lost models" runs upstream of the filter that loses them.
+
+**The new test proves the rule, not the path — this is the sixth instance of the pattern.**
+`catalog-v3-builder.test.ts` "maps an OCR API pointer through owner evidence and keeps it
+non-callable" builds its backing row as
+`mistralRawModel("mistral-ocr-4-0", "mistral-ocr-latest", "mistral-model-pages-scrape")` —
+`canonicalId` different from `externalId`, from the one collector that can only ever emit them
+equal. The assertion is right and the production path it stands for does not exist. Contrast the
+seven-row seam test, whose `mistral-model-pages-scrape` fixture uses
+`canonicalId === externalId === dated`; that one *is* faithful.
+
+**Suggestion.** Two small changes, in this order. (1) Record the withholding: collect the
+filtered ids at `:109-111` and emit them as a candidate-level diagnostic — ideally something a
+client can read, at minimum something `alertCatalogResults` reports and something
+`buildProviderCounts` is computed *after*. A silent partial is the failure mode this whole
+contract exists to remove, and this is now the only unlogged decision in the builder.
+(2) Make `parseMistralModelPage` emit the page's copy-to-clipboard API id as `externalId` when
+it differs from the slug-derived `canonicalId`. That is precisely the owner-observed
+pointer-to-dated pair `sourceBackedMistralPointerTargets` is written to consume, it comes from
+Mistral's own page rather than a table, and it would make the OCR test's fixture real. Then add
+the test that is still missing: a `mistral-api` pointer with **no** backing at all, asserting
+the intended outcome *and* the diagnostic.
+
+### MEDIUM-1 — New this round: roster *expiry* now strips supplemental plan membership even when the model is still published, with no test for that case and no wire signal that distinguishes it from an empty plan (a)
+
+**Location:** `subscription-plan-reconciliation.ts:872-882` — the new first branch inside
+`retained`.
+
+At `d7acfd7` an inclusion was stripped only when
+`mapped && !modelIds.has(modelId) && key ∈ supplementalEntryKeys`. `d0c1538` adds, ahead of it:
+
+```ts
+if (supplementalProvenanceEntryKeys.has(key) && !applicableSupplementalEntryKeys.has(key)) {
+ return false;
+}
+```
+
+That is unconditional on model presence. I tried hard to find a case where it strips a live
+inclusion wrongly and could not: `mergeSupplementalRosters:1110-1124` unions **every** unexpired
+active entry into the applicable roster, so `key ∈ provenance` and `key ∉ applicable` together
+imply either the entry expired or the whole roster failed validation *and* carry-forward
+(`reconcileRosterEvidenceV3:760-795`). Both are the intended removal semantics. **The rule is
+right.** What is new is its reach.
+
+**Concrete failure scenario.** The OpenAI Codex public-registry collector fails for more than
+24h (`LIVE_ROSTER_MAX_AGE_HOURS`). `validateRosterForRequirement(active, …)` now fails on roster
+expiry, so nothing is carried, `applicable` is empty for `openai-codex`, and the new branch
+strips **every** `provider_model` inclusion — while `retainedActiveModelIds` keeps every Codex
+model published and callable. `projectSubscriptionPlanMembershipV3` then derives no
+`subscriptionPlanIds`. **Published output: every Codex model present and callable, and the
+`openai-codex` plan with zero members.** At `d7acfd7` those inclusions survived, stale but
+present. Because the Codex roster is `requiredForActivation: false`, no violation is raised and
+the generation activates cleanly. On our side `[email protected]` stops being recognised as
+plan-included and is priced per-token against a flat-rate Codex subscription — the same money
+direction as the HIGH-1(new) you just fixed, relocated from retention to expiry.
+
+**Why we cannot detect it.** `rosterCoverageAt:955-962` returns the constant
+`{status: "unknown", reason: "supplemental_only"}` for any plan with no authoritative
+requirement, which `openai-codex` is by design. So coverage reads identically before and after,
+and an emptied `inclusions` list is indistinguishable on the wire from a genuinely empty plan.
+
+**Untested.** `subscription-plan-reconciliation.test.ts` is unchanged by this commit. The builder
+test that covers expiry, "uses an expired active Codex roster as provenance when carrying its
+plan", has the carried model **absent** from `mergedModels`, so it passed at `d7acfd7` through
+the old model-missing branch and does not distinguish the new one. The case that matters —
+provenance key present, roster expired, **model present** — has no assertion anywhere.
+
+**Suggestion.** Either keep the positive and let it decay through coverage — which is what §5's
+"expiry removes authority" and `canAuthoritativelyExcludeModelV3` already express: for a
+supplemental roster, absence must never harden into not-served — or, if removing on expiry is
+the deliberate choice, give the plan a distinguishable state so a client can tell "no members"
+from "evidence expired", and add the missing test with the model present.
+
+### MEDIUM-2 — Prior MEDIUM-1 unresolved, and this round made it unreachable by construction for three pointers (a)
+
+**Locations:** `schema-runtime.ts:427-448`; `merger.ts:425-441`, `:459-461`.
+
+`canonicalizeMistralModelId` still promotes `mistral-medium-latest`, `mistral-medium-3.5`,
+`-3-5` and `-3.5-128b` to `mistral-medium-2604`; `voxtral-mini-latest` to `voxtral-mini-2602`;
+`voxtral-mini-tts-latest` to `voxtral-mini-tts-2603`. This round changed the lookup key to
+`pointerId = canonicalizeModelId(raw.externalId)` (`merger.ts:425`), so for those three
+`pointerId` is **already the dated id**. Consequences, all read in source:
+
+- `reviewedMistralCanonicalIdForPointerV3(pointerId)` can never match — its keys are pointer
+ spellings.
+- `sourceBackedMistralPointerTargets` computes its key the same way, so the pricing row for
+ `mistral-medium-latest` yields `pointerId === target === mistral-medium-2604` and is discarded
+ by `if (target === pointerId) continue` (`:459-461`).
+- All three therefore **always** take the fallback `return { ...raw, canonicalId: pointerId }`
+ (`:437-441`), which — unlike both resolved branches — does **not** call
+ `stripRemappedMutableAliasMetadata`. The pointer row alone publishes a dated canonical model
+ carrying the pointer's own `contextWindow`, `pricing`, `capabilities`, `description` and
+ `status`, even when no owner source observed the dated id that run. §2.2 holds in letter (the
+ published id is dated) and not in substance. The fallback's own comment — "Keep it visibly
+ mutable so the v3 candidate gate can reject it" — is still false for exactly these three.
+- The route is safe: `subscription-alias-validation.ts:151-155` yields `mutable_pointer_only`.
+ But that reason contradicts the reviewed table, which assigns `mistral-medium-2604` the reason
+ `identity_conflict` with pointer `magistral-medium-latest`
+ (`mistral-identity-policy.ts:22-29`). Two reviewed statements about one model still disagree,
+ in two files.
+
+**Suggestion** unchanged: use the literal normalized `externalId` in the fallback, and decide
+`mistral-medium-latest`'s target in one place — either add it to the table with a reason, or
+delete the stale canonicalizer rule. Note this is now *safe* to do: with CRITICAL-1's throw
+replaced by a filter, making the fallback honest no longer risks an outage — it risks a silent
+drop, which is HIGH-1's ask.
+
+### MEDIUM-3 — Prior MEDIUM-2 unresolved: the deliberate projection still does not sort roster entries or plan inclusions, so order-only upstream churn still refuses activation (a for the code, (c) for the trigger)
+
+`catalog-v3-deliberate-diff.ts:133-152`: the five top-level arrays are sorted, but
+`deliberateRoster` maps `entries` without sorting and `deliberatePlan` only strips
+`lastUpdated`. Persistence does not compensate: `createRosterWithSnapshotCount:553-568` maps
+`input.entries` in arrival order and never sorts, and only `mergeSupplementalRosters:1130-1132`
+sorts — so every roster taken through `rosters.push(cloneRoster(incoming))` keeps upstream
+order, which covers the authoritative live rosters. `dedupeInclusions([...retained,
+...rosterInclusions])` is unsorted.
+
+**Concrete:** the OpenCode Go roster page re-ranks its table; entry content is byte-identical;
+`surfaces.rosters` differs; `buildCatalogCandidateV3` throws
+`CatalogDeliberateDiffApprovalRequiredV3`; `publishCatalogV3` stages a pending candidate and
+rethrows; `collectModelCatalog` errors. The catalog freezes until an operator supplies a digest —
+the original HIGH-1 failure mode, narrowed from "every run" to "any run where an upstream
+reorders a list". Still ~4 lines to close, inside the projection only: sort `entries` by
+`externalModelId` and `inclusions` by a stable key.
+
+### LOW-1 — The error-vocabulary deferral is still recorded nowhere durable
+
+A case-insensitive search for `405`, `method_not_allowed`, `defer` and `amendment` across
+`docs/` and `CHANGELOG.md` at `d0c1538` returns nothing; the same search finds `invalid_cursor`
+twice in `docs/api-reference.md`, so the pathspec is sound. The 405-carrying-`catalog_unavailable`
+deferral survives only in a PR comment, which leaves the repo when #151 merges. Two lines beside
+the error table closes it. No client impact.
+
+### LOW-2 — `prepareCollectorResultsForCatalogV3` still re-slices positionally, and HIGH-1's fix is the change that would break it
+
+`merger.ts:96-112` flattens all rows, transforms, then walks a running `offset` slicing
+`result.models.length` back per collector. Still correct at `d0c1538` —
+`applyMistralPointerTargets` and `recanonicalizeAliasBackedRawModels` are both `.map()`, and this
+round wisely put the quarantine drop in the *builder* on `mergedModels` rather than here.
+Flagging it again only because HIGH-1's suggested "drop the row and record it" is exactly the
+edit that would shift the slices and silently re-attribute rows to neighbouring collectors, which
+decides `sourceProviderIdForCollector` and hence the published route. One line closes it:
+`if (preparedRaw.length !== allRaw.length) throw`.
+
+---
+
+## What checks out, with the reasoning
+
+**CRITICAL-1 is properly fixed, not papered over (a).** `publishableMergedModels` (`:109-111`)
+replaces the throw, and — the part worth crediting — the `mergedModelIds`/published-set
+discrepancy that caused HIGH-1(new) is gone at the root: `candidateModelIds` (`:131-134`) is
+`incomingModelIds` unioned with `retainedActiveModelIds`, and the final `models` array is exactly
+`publishableMergedModels.map(…)` plus the members of `retainedActiveModelIds` (`:170`, `:202`).
+Both `applyRosterEvidenceToPlansV3` (`:139`) and the orphan-route guard (`:154`) now read that one
+set. I checked the refactored retention predicate for equivalence rather than trusting it:
+`previousSourceIds.length === 0 || previousSourceIds.some(id => !complete.has(id))` is exactly
+`!(previousSourceIds.length > 0 && retainedSourceIds.length === 0)`, the condition deleted from
+the loop body — no behaviour change. Excluding quarantined actives from retention (`:120`)
+correctly prevents a pointer published by an older generation from being carried forward forever.
+
+**HIGH-1(new) is fixed in the right direction and asserted (a)(b).**
+`catalog-v3-builder.test.ts` "preserves an unexpired Codex inclusion when its model is retained"
+drives two real generations and asserts the inclusion **and** `subscriptionPlanIds` survive when
+the model is absent from the fresh merge but re-added by retention. The strip still fires where it
+should: "uses an expired active Codex roster as provenance when carrying its plan" asserts
+removal, and "keeps a dangling authoritative inclusion fatal" asserts the throw. MEDIUM-1 is the
+one new edge the refactor opened.
+
+**The broadened "exact-ID official-callability for all Mistral routes" is safe, and much narrower
+than it reads (a).** The entire Mistral block in `projectServingEvidence` is gated on
+`evidence.sourceProviderId === "mistralai"` (`subscription-alias-validation.ts:137`), and
+`mistral-api` is the only collector that maps to that source provider. Removing
+`isDatedMistralModelIdV3` from the last two branches therefore touches **no aggregator route on
+any Mistral model** — OpenRouter/Together/Fireworks rows still project `mapped`. What it does
+change is that a *non-dated* Mistral canonical (`mistral-ocr-4-0`, `shieldstral-1.0`,
+`mistral-embed`) now also needs `externalModelId === canonicalModelId && officialCallableEvidence
+=== true` for a callable direct-API route. Since `officialCallableEvidence` is set at
+`catalog-v3-builder.ts:429-431` under precisely that condition, the two agree and an exact API id
+stays mapped. A failure degrades to `unknown`; it never drops a model. This is the one place where
+broadening a reviewed decision into a category-wide rule did **not** repeat CRITICAL-1's mistake,
+because the failure mode of the broadened rule is a weaker route rather than a rejected model.
+
+**The `merger.ts` identity preparation survived the merge functionally (a).** `index.ts:458-460`
+still calls `prepareCollectorResultsForCatalogV3(collectorResults)` once, and `:473` and `:489`
+feed the same `catalogCollectorResults` to `mergePreparedResults` and to the builder's
+`collectorResults`. The pre-merge/post-merge identity split cannot reappear. `isMistralModel` was
+broadened with `devstral|pixtral|voxtral|leanstral` (`catalog-v3-builder.ts:450-451`); all four
+are Mistral families and the predicate is only ever used together with
+`isMutableMistralPointerV3`, so I found no non-Mistral model it can capture.
+
+**The seven §4.2 rows still publish, and their seam test is production-faithful (a)(b).**
+`catalog-v3-builder.test.ts` "projects every reviewed Mistral API pointer as unknown evidence on
+dated models" still runs `prepareCollectorResultsForCatalogV3` then `mergePreparedResults` then
+`buildCatalogCandidateV3`, and loops `Object.entries(MISTRAL_POINTER_QUARANTINE_V3)`. Its
+`mistral-api` fixture uses `canonicalId === externalId === pointer` and its
+`mistral-model-pages-scrape` fixture uses `canonicalId === externalId === dated` — both are the
+shapes those collectors actually emit, so unlike the OCR test this one does carry a document
+through the seam. `mistral-medium-2604` and `mistral-small-2603` remain `identity_conflict`, the
+other five `mutable_pointer_only` (`mistral-identity-policy.ts:1-29`). That the seven reviewed
+lookups still resolve after the key became `canonicalizeModelId(externalId)` is (b) — it rests on
+their 107/107 claim, since that test is the only thing checking it.
+
+## What I did not reach
+
+- **Their test suite was not run**, per the read-only constraint, and the emulator smoke was not
+ executed. All test conclusions come from reading the test source. Their 107/107 and
+ 1,183/1,183 claims are unverified by me.
+- **The live contents of `https://api.mistral.ai/v1/models`.** HIGH-1's frequency — how often a
+ pointer arrives with no pricing-card backing — depends on it. Their own fixtures show the
+ binding working for `mistral-medium-latest`, `mistral-small-latest`, `codestral-latest`,
+ `mistral-ocr-latest` and `devstral-small-latest`, so the common cases are covered today; the
+ exposure is new releases and card-layout drift. A dump of that roster diffed against
+ `MISTRAL_MODEL_PAGE_SLUG_ID_MAP` plus `MISTRAL_PINNED_API_ID_MAP` settles the severity.
+- `catalog-generation.ts` write/CAS/readback/retention internals — unchanged by this commit, and
+ still the un-audited half of Priority 3 item 10.
+- Which manual mapping rules exist in production Firestore.
+- `web/`, the roughly twenty test files untouched by `d0c1538`, and the non-Mistral,
+ non-Codex collectors.
+- Not re-verified because you closed it: the 426 envelope and
+ `docs/fixtures/catalog-v3-upgrade-required.json`.
+
+## What claudish should do next
+
+1. **Ask for HIGH-1's diagnostic before cutover, not the resolution fix.** We can live with
+ Mistral pointers being withheld; we cannot live with not knowing. A candidate-level list of
+ withheld ids, plus computing `buildProviderCounts` on the post-filter set, are both small and
+ convert this from silent to observable.
+2. **Raise MEDIUM-1 as a money item.** After 24h of Codex-registry failure the `openai-codex`
+ plan publishes with zero members and we cannot tell that from an empty plan. If they decline
+ to change the policy we need a distinguishable signal, or we will mis-price `cx@` for
+ flat-rate users.
+3. **No change to our 426 parser.**
+4. **A client-side canary is still worth having**, but its shape has changed: a frozen
+ `generationId` no longer indicates the Mistral failure, because the generation now advances
+ happily while withholding models. Watch instead for Mistral models carrying
+ `dataFreshnessWarning: true` across several generations, and for plans whose `inclusions` drop
+ to zero.
diff --git a/ai-docs/reports/REVIEW_models_index_v3_remediation-20260913.md b/ai-docs/reports/REVIEW_models_index_v3_remediation-20260913.md
new file mode 100644
index 00000000..7e7c66db
--- /dev/null
+++ b/ai-docs/reports/REVIEW_models_index_v3_remediation-20260913.md
@@ -0,0 +1,500 @@
+# Follow-up review — models-index v3 remediation, commit `d7acfd7`
+
+**Reviewer:** claudish client team
+**Target:** `d7acfd7` ("fix(catalog): remediate v3 review findings"), PR #151, baseline `a0658df`
+**Prior review:** `ai-docs/reports/REVIEW_models_index_v3_impl-20260913.md`
+**Spec:** `~/.claude/plans/gleaming-discovering-marble.md` (the remediation plan; its sections 1-6 map 1:1 to the six findings)
+**Date:** 2026-09-13
+**Mode:** read-only. Nothing in that repo was modified, staged, executed, or tested.
+
+## Method note — the working tree is NOT the commit
+
+That worktree is mid-merge with `origin/main`: its status shows `UU`/`DU`/`AA` conflict
+states, and `functions/src/collectors/api/mistral.ts` on disk literally contains a
+`<<<<<<< HEAD` marker. `merger.ts`, `index.ts`, and `schema-runtime.ts` on disk differ from
+`d7acfd7`. **Everything below was read out of the commit object itself**, except files I
+first proved byte-identical to the commit (`catalog-v3-builder.ts`,
+`catalog-v3-deliberate-diff.ts`, `mistral-identity-policy.ts`,
+`subscription-plan-reconciliation.ts`, `subscription-alias-validation.ts`, the cutover
+workflow, and every test file except `merger-aliases.test.ts`). Anyone re-checking this
+report must do the same; the on-disk tree will mislead you.
+
+## Verdict
+
+**FAIL** — 1 CRITICAL, 1 HIGH, 2 MEDIUM, 2 LOW.
+
+Five of the six original findings are genuinely fixed, several of them well: HIGH-1,
+MEDIUM-1, MEDIUM-2 (for the reproduced case), and MEDIUM-3 are clean, and the Mistral
+quarantine's seven reviewed rows are now reachable and asserted end-to-end through the
+real merge path — including the pre-remapped case, which now has a test.
+
+But the remediation introduces **two new defects that are worse than what they replaced**,
+and both are the same pattern the review was told to hunt: a correct rule whose execution
+path meets inputs the reviewed table does not describe.
+
+- **CRITICAL-1** — Mistral's live `/v1/models` contains `-latest` pointers beyond the
+ reviewed seven. Each one now becomes a canonical model id, which `buildCatalogCandidateV3`
+ rejects by throwing — before staging, before activation. That is a total,
+ every-run catalog-generation outage, not a frozen catalog.
+- **HIGH-1(new)** — the dangling-supplemental cleanup is keyed on the pre-retention merged
+ model set, so it silently strips a **legitimate** Codex plan inclusion from a model the
+ same generation still publishes. That is the "costs a flat-rate user money" outcome the
+ cleanup was supposed to avoid, converted from a loud throw into a silent under-report.
+
+Nothing here touches the wire contract. Our 426 guard remains correct and is now pinned to
+a committed fixture on their side too.
+
+## Item-by-item
+
+| # | Original finding | Their claim | Result |
+|---|---|---|---|
+| 1 | HIGH-1 deliberate-diff gate | freshness-only projection | **VERIFIED** |
+| 2 | HIGH-2 Mistral quarantine | shared identity prep + fail-closed guards | **PARTIAL** — seven rows fixed; introduces CRITICAL-1 and MEDIUM-1 |
+| 3 | MEDIUM-1 unscoped alias index | retired v2 graph removed | **VERIFIED** (gone, not merely unused) |
+| 4 | MEDIUM-2 dangling inclusion | supplemental cleanup w/ prior provenance | **PARTIAL** — reproduced case fixed; introduces HIGH-1(new) |
+| 5 | MEDIUM-3 Accept verification | no-Accept 426 asserted in `verify-public` | **VERIFIED** |
+| 6 | LOW-1 error vocabulary | deferred to a contract amendment | **PARTIAL** — correctly un-applied, but not recorded in the repo |
+
+### 1. HIGH-1, the deliberate-diff gate — VERIFIED (a)(b)
+
+`catalog-v3-deliberate-diff.ts:125-159` projects away exactly the four freshness surfaces
+the plan names and nothing else: `deliberatePlan` drops `lastUpdated`; `deliberateRoster`
+drops roster `observedAt`/`expiresAt`/`contentHash` **and** per-entry
+`observedAt`/`expiresAt`; `deliberateRedirect` drops `observedAt`.
+
+I checked the inverse risk you flagged, because dropping `contentHash` is the dangerous
+part — a hash is a proxy for content, and removing it could remove the only witness to a
+content change. It does not: the projection retains every field of
+`AuthoritativeRosterV3Schema` that carries meaning (`schema-runtime.ts:1031-1050`) —
+`entries[]` with `sourceText`/`externalModelId`/`modelId`, plus `snapshotEntryCount`,
+`planIds`, `authority`, `scope`, `sourceKind`, `sourceUrl`, `refreshOwner`. A model added
+to or removed from a roster changes `entries` **and** `snapshotEntryCount`, so it still
+trips the gate. `contentHash` was removable precisely because it is derived from data that
+stays in the projection.
+
+**Content-change-coinciding-with-timestamp-change is safe by construction, not by luck.**
+`addChangedSurface` compares canonical JSON of the *projected* value, and the projection is
+per-field: the timestamps are gone from both sides before comparison, so a simultaneous
+content edit is the only thing left to differ. Their test at `:84-101` proves the
+content-only half (`entries[0].sourceText`, and a redirect `toModelId` remap), and `:206-287`
+table-drives 13 semantic mutations including `rosters[0].sourceUrl` and plan `features`,
+each asserting `changed === true` **and** a digest different from the unchanged digest.
+
+I also confirmed no *other* volatile field survives in the compared surfaces:
+`PlanBaseV3SchemaShape` (`:789-800`) has exactly one timestamp (`lastUpdated`, stripped);
+`ModelInclusionV3Schema` (`:736-773`) and `RosterRequirementV3Schema` (`:782-787`) have none;
+`CatalogModelV3Schema` (`:1147-1174`) has no `lastUpdated`, no popularity, and no
+observation timestamp at all. So the `plans` and `rosters` surfaces are now genuinely
+stable across a no-op collection.
+
+Path reached, with the inputs that occur in practice: `catalog-v3-publication.test.ts:20-46`
+publishes G0 then G1 from the same collection input at a later `generatedAt` and asserts
+`g1.publication.activated === true` with `control.revision === 2`. That is the exact
+scenario I predicted would freeze forever, now asserted green end-to-end through
+`publishCatalogV3`. The former "must re-submit with `approvedDiffDigest`" test was rewritten
+to use a real semantic mutation (`mergedModels[0].displayName`), as the plan required.
+
+Their decision to skip staging-path retention is sound given this fix: the 8-pending-per-day
+accumulation mode was a consequence of the freeze, not an independent defect.
+
+### 2. HIGH-2, the Mistral quarantine — PARTIAL
+
+**What is genuinely fixed (a)(b).** The keying defect is gone.
+`prepareCollectorResultsForCatalogV3` (`merger.ts` at `d7acfd7`, the block replacing the old
+`mergeResults` body) runs `recanonicalizeAliasBackedRawModels` **once** over a deep clone of
+every collector row and re-slices the result back per collector; `mergePreparedResults` and
+`collectServingEvidence` then consume the *same* rows. `index.ts:427-486` wires the real
+pipeline to it — `mergePreparedResults(catalogCollectorResults)` for models and
+`collectorResults: catalogCollectorResults` for the builder — so the pre-merge/post-merge
+split that made the quarantine unreachable no longer exists. The re-slice is order- and
+length-safe today: both stages inside `recanonicalizeAliasBackedRawModels` are `.map()`.
+
+**All seven rows are now reachable AND published, not merely decided (b).**
+`catalog-v3-builder.test.ts:97-135` is a real collector-to-builder integration test: it feeds
+seven `mistral-api` rows whose `canonicalId === externalId ===` the pointer, plus
+`mistral-model-pages-scrape` rows for the seven dated targets, runs
+`prepareCollectorResultsForCatalogV3` then `mergePreparedResults` then
+`buildCatalogCandidateV3`, then loops `Object.entries(MISTRAL_POINTER_QUARANTINE_V3)`
+asserting for each that no model is published under the pointer id **and** that the dated
+model's `aggregators` contains `{sourceProviderId:"mistralai",
+sourceCollectorId:"mistral-api", routeStatus:"unknown", reason: <policy.reason>,
+observedExternalModelId: <pointer>}`. That covers `mistral-medium-2604` and
+`mistral-small-2603` as `identity_conflict` and the other five as `mutable_pointer_only`,
+exactly as section 4.2 requires. The `unknown` variant still carries no `route` and no
+`externalModelId` — enforced structurally by the `.strict()` `unknown` branch of
+`AggregatorRouteV3Schema` (`schema-runtime.ts:1002-1013`), unchanged.
+
+**The pre-remapped path is now covered by a test, not only reasoned about (b).**
+`merger-aliases.test.ts` (new test "does not let gateway alias evidence stabilize a Mistral
+API pointer") is precisely your case: a `mistral-api` row with `externalId:
+"mistral-large-latest"` whose `canonicalId` is **already** `mistral-large-2512`, plus an
+OpenRouter row carrying the alias. It asserts the prepared row's `canonicalId` is reset to
+`"mistral-large-latest"` — i.e. ingress's remap is undone because no *owner* source
+independently observed the dated id (`isOwnerTopLevelSource`, `merger.ts:951-958`; OpenRouter
+is not an owner for `mistralai`). Combined with `catalog-v3-builder.test.ts:178-199`
+("blocks unresolved mutable Mistral canonical identities"), the chain is established. This
+closes the gap their session told us about.
+
+**A dated canonical still cannot carry a pointer as its mapped `externalModelId` (a).**
+`projectServingEvidence` (`subscription-alias-validation.ts:149-165`) keeps all four layered
+guards, and the catch-all still demands `externalModelId === canonicalModelId &&
+officialCallableEvidence === true` for every dated Mistral id. `officialCallableEvidence` is
+still set in exactly one place and only for `mistral-api` rows where
+`externalId === canonicalId` (`catalog-v3-builder.ts:396-400`) — note this now reads the
+*prepared* canonicalId, so a remapped pointer row can never claim it. The dated-token regex
+gap I reported is also closed: `isDatedMistralModelIdV3` now anchors with `(?=-|$)`
+(`mistral-identity-policy.ts:52-54`) and is tested against
+`mistral-ocr-2505-completion` (`subscription-alias-validation.test.ts:114-131`).
+
+**The orphan guard requested by plan section 2 item 6 exists**
+(`catalog-v3-builder.ts:124-135`): any `routesByModelId` key absent from `mergedModelIds` is
+a violation, tested at `catalog-v3-builder.test.ts:201-217`. Evidence is no longer silently
+discarded.
+
+**What is not fixed, and what the fix broke: see CRITICAL-1 and MEDIUM-1 below.**
+
+### 3. MEDIUM-1, the unscoped alias index — VERIFIED, it is gone (a)
+
+Not merely unreferenced — deleted. The commit's recursive tree listing has no
+`subscription-plan-membership.ts`, `writer.ts`, `recommender.ts`,
+`recommended-route-contract.ts`, `algolia-catalog.ts`, `search-bag-builder.ts`,
+`plugin-defaults.ts`, `seed-plugin-defaults.ts`, `recommended-subscription-coverage.ts`,
+`recommended-subscription-projection.test.ts`, or
+`scripts/check-recommended-subscription-coverage.ts`, plus all their tests
+(-9,291 lines in the commit; `writer.ts` and `writer.test.ts` are in the tail of the stat).
+The only files left matching those names are the v3 ones, `catalog-v3-recommender.ts` and
+`plugin-defaults-handler.ts`.
+
+`attachSubscriptionPlanMembership` and `canonicalizeSubscriptionPlanId` return zero hits
+repo-wide. The retired Zod graph (`RecommendedModelsDocSchema`, `RecommendedModelEntrySchema`,
+`RecommendedSubscriptionSchema`, `validateRecommendedDoc`) is stripped from
+`schema-runtime.ts`; `schema.ts` loses 110 lines. `providerUid` and `includedModels` now
+survive **only** as forbidden-field names in the runtime scan list
+(`schema-runtime.ts:1334-1335`), as negative test assertions, and as an unrelated local
+variable in the MiniMax collector (`collectors/scraper/minimax.ts:487`). `algoliasearch` is
+out of `functions/package.json`; the dead CI steps and the `catalog-invariants.yml` workflow
+that ran the retired check are removed. `ACCESS_METHODS` no longer exists anywhere — the
+plan's "extract then migrate" became "delete the only consumer", which is a better outcome.
+
+There is no path by which a future caller reintroduces the unscoped join without writing it
+from scratch.
+
+### 4. MEDIUM-2, the generation-blocking dangling inclusion — PARTIAL (a)(b)
+
+The no-applicable-roster case I reproduced **is** fixed, and fixed at the right spot. In
+`applyRosterEvidenceToPlansV3` the `retained` filter is now computed **before** the
+`applicable.length === 0` early return, and that return re-emits the plan with the pruned
+inclusions (`subscription-plan-reconciliation.ts`, the
+`if (applicable.length === 0) return retained.length === plan.inclusions.length ? plan : {...plan, inclusions: retained}`
+block). The stale positive is matched by a `provider` + `externalModelId` key against
+`supplementalEntryKeys`, which unions current applicable rosters with
+`supplementalProvenanceRosters` — passed by the builder as `input.activeV3?.rosters`
+(`catalog-v3-builder.ts:96-101`), so an **expired** prior roster still serves as provenance
+for removing its own positives while being barred from adding any.
+
+Tested for the case I reproduced, not only the easy one:
+`subscription-plan-reconciliation.test.ts` adds "uses an expired supplemental roster only to
+remove its carried positives" (empty `rosters`, provenance-only, asserts the unrelated
+inclusion survives and the dangling one goes) and "drops dangling supplemental positives
+without granting membership"; `catalog-v3-builder.test.ts:344-414` and `:416-490` drive both
+through two real generations. Authority is preserved: the entry-level skip is guarded on
+`roster.authority === "supplemental"`, and "keeps a dangling authoritative inclusion fatal"
+asserts the throw still fires.
+
+**The cleanup can nonetheless remove a legitimate inclusion — see HIGH-1(new).**
+
+### 5. MEDIUM-3, Accept verification — VERIFIED, and it covers the case that matters (a)
+
+`.github/workflows/catalog-v3-cutover.yml` `verify-public` now issues the no-Accept request
+through `node:https.request` with only a `cache-control` header — deliberately avoiding
+`fetch`, which injects `accept: */*` and would have tested the wrong thing. `*/*` is itself a
+426 in their implementation, so the bug would have hidden. `assertUpgradeRequired` asserts
+HTTP 426, top-level `contractVersion === 3`, `error.code ===
+"catalog_client_upgrade_required"`, `error.minimumContractVersion === 3`, and
+`Object.hasOwn(body,"data") === false`, and throws otherwise (failing the step and the
+cutover). The `accept: application/json` case is asserted the same way. **This is the exact
+request claudish sends, now verified against production rather than attested.**
+
+They also committed `docs/fixtures/catalog-v3-upgrade-required.json` and replaced the inline
+literal in `catalog-v3-http.test.ts:94` with a `toEqual` against that file, so their contract
+test and our parser test can pin the same artifact. We asked for this; we got it.
+
+### 6. LOW-1, the deferral — PARTIAL (a)
+
+**Nothing half-applies it, confirmed.** `CatalogErrorCodeV3` (`schema.ts:390-394`) is
+unchanged at four codes; no method-specific code was invented; the 405 still carries
+`catalog_unavailable` and still only on the admin endpoint
+(`catalog-v3-manual-handler.ts:73`); `invalid_cursor` is unchanged and still documented.
+No cursor or method failure was reclassified as an availability failure.
+
+**But the deferral is not recorded anywhere durable in the repo.** Searches for `405`,
+`method_not_allowed`, `deferr`, and `amendment` across `docs/` and the v3 HTTP/schema
+sources at `d7acfd7` return nothing; `docs/api-reference.md` gained only the fixture link,
+and `CLAUDE.md`'s four changed lines are about the test command. The plan itself (section 6)
+only ever asked for it to be recorded "in the PR update" — i.e. a PR comment, which
+disappears from the repo's own history the moment PR #151 merges. Ask for two lines in
+`docs/api-reference.md` next to the error table.
+
+---
+
+## Findings, most severe first
+
+Regressions introduced by the remediation come first, per your ranking rule.
+
+### CRITICAL-1 — Every Mistral `-latest` id outside the reviewed seven now becomes a canonical model and hard-blocks the whole catalog generation, on every run (a, with the live-roster premise at (b))
+
+**Locations:** `merger.ts` at `d7acfd7` — `applyReviewedMistralPointerTargets` and the
+`unresolvedMistralPointer` short-circuit inside `recanonicalizeAliasBackedRawModels`;
+`catalog-v3-builder.ts:105-116`; `collectors/api/mistral.ts` at `d7acfd7`, `collect()`;
+`schema-runtime.ts:420-456` `canonicalizeMistralModelId`.
+
+**The mechanism, step by step — every step read in the committed source.**
+
+1. `MistralCollector.collect()` emits one `RawModel` for **every** id in
+ `https://api.mistral.ai/v1/models`, `-latest` pointers included, with
+ `canonicalId: selectCanonicalMistralId(m)` — which is the API id verbatim
+ (`selectCanonicalMistralId` is a one-line `normalizeMistralId(model.id) ?? model.id`).
+ The only rows skipped are embed/moderation ids that also carry a non-chat `type`.
+2. `applyReviewedMistralPointerTargets` intercepts every `mistral-api` row whose
+ `externalId` is a pointer and looks it up in the **seven-entry** inverse map
+ `MISTRAL_CANONICAL_ID_BY_POINTER_V3` (`mistral-identity-policy.ts:35-50`). A hit whose
+ dated target is independently observed gets the reviewed dated `canonicalId`. **A miss
+ falls through to `canonicalId: canonicalizeModelId(raw.externalId)`, which for an
+ unreviewed pointer is the pointer itself**, with the comment "Keep it visibly mutable so
+ the v3 candidate gate can reject it".
+3. `unresolvedMistralPointer` then disables **both** alias-remap paths for that row
+ (`externalMappedCanonical` and `mayUseCurrentCanonicalAlias`), so the owner-page and
+ pricing evidence that used to fold the pointer onto its dated model is now refused.
+4. `mergePreparedResults` groups by `canonicalId` and therefore publishes a `ModelDoc` whose
+ `modelId` **is** the pointer.
+5. `buildCatalogCandidateV3:105-116` computes
+ `mergedModels.filter(m => isMistralModel(m) && isMutableMistralPointerV3(m.modelId))` and
+ throws `mutable Mistral pointers cannot be canonical models: ...` unconditionally.
+6. `publishCatalogV3` (`catalog-v3-publication.ts:103-128`) catches **only**
+ `CatalogDeliberateDiffApprovalRequiredV3`; a plain `Error` rethrows, so no candidate is
+ built, nothing is staged, nothing is activated, and `runCatalogV3Collection` errors out.
+
+**Their own tests document steps 3-4 as the intended new behaviour.** The test formerly
+called "breaks alias cycles using the strongest canonical evidence" was renamed to
+"does not collapse circular mutable-only Mistral evidence" and its assertion inverted from
+`expect(docs).toHaveLength(1); expect(docs[0].modelId).toBe("mistral-ocr-2505")` to
+`expect(docs.map(d => d.modelId).sort()).toEqual(["mistral-ocr-2505", "mistral-ocr-latest"])`
+— a `mistral-api` row for `mistral-ocr-latest`, an id that is **not** in the reviewed seven,
+now yields a canonical model called `mistral-ocr-latest`. That test calls `mergeResults` and
+stops there. No test carries that same input into `buildCatalogCandidateV3`, which is the
+only place the two halves meet — and where it throws.
+
+**The live-roster premise.** This is fatal only if Mistral's API actually serves pointers
+outside the seven. Three independent pieces of evidence from their own repo say it does:
+`collectors/scraper/mistral-model-pages.test.ts:213` is a captured model-page fixture whose
+copy-to-clipboard API id is `mistral-ocr-latest`;
+`collectors/scraper/mistral-pricing.test.ts:79` is a captured pricing payload whose
+`api_endpoint` values include `mistral-small-latest` and `codestral-latest`; and the repo
+references `devstral-small-latest`, `devstral-latest`, `mistral-tiny-latest`,
+`mistral-vibe-cli-latest`, and `voxtral-mini-latest` besides. None of those is a key of the
+seven-entry map. I verified by reading `canonicalizeMistralModelId` that only three pointers
+get rewritten to a dated id by ingress (`mistral-medium-latest`, `voxtral-mini-latest`,
+`voxtral-mini-tts-latest`); everything else stays mutable. I also verified the seven reviewed
+dated targets are fixed points of `canonicalizeModelId`, so the reviewed path itself is
+self-consistent — the problem is only the *set*, not the *rule*.
+
+**Concrete failure scenario.** 02:00 UTC. `MistralCollector` returns its usual rows
+including `codestral-latest` and `mistral-ocr-latest`. Neither is in the reviewed map, so
+both keep mutable canonical ids; the owner-page rows that used to fold them are now ignored.
+`buildCatalogCandidateV3` throws `mutable Mistral pointers cannot be canonical models:
+codestral-latest, mistral-ocr-latest`. `collectModelCatalog` errors. **No generation is
+produced — not even a pending one.** Identical at 05:00, 08:00, and so on. The active
+generation stays served, so claudish keeps getting well-formed 200s with a `generationId`
+that never advances; within 24 hours every live-roster plan (`opencode-go`, `ollama-cloud`,
+both Alibaba token plans, `routing-run`, `streamlake-kwaikat-coding-plan`) decays to
+`rosterCoverage: {status:"unknown", reason:"expired"}` while new models never appear. This is
+strictly worse than the HIGH-1 freeze it replaced, which at least sealed a pending candidate
+a human could approve.
+
+**Contradiction inside the commit.** `subscription-alias-validation.test.ts:114-131` (added
+by this commit) asserts the projection for serving evidence pairing canonical
+`mistral-ocr-2505-completion` with observed `mistral-ocr-latest` — a pairing the new merger
+path can no longer produce, because that pointer is never remapped onto a dated target. One
+half of the commit tests a pairing the other half makes unreachable while also making it
+fatal.
+
+**Suggestion.** Either (a) let owner-observed dated evidence resolve a pointer generically —
+the pointer-to-dated pair asserted by `mistral-model-pages-scrape` or
+`mistral-pricing-scrape` for the *same owner* is the same class of evidence the reviewed
+table encodes by hand — and keep the hard throw only for a pointer with no owner-observed
+target at all; or (b) keep the table-only policy but make the unresolved case **drop the
+pointer row** (with a recorded violation) instead of aborting the generation, so one vendor's
+unreviewed alias cannot take down the entire catalog. Whichever you choose, add the missing
+integration test: feed `buildCatalogCandidateV3` a `mistral-api` roster containing a pointer
+that is *not* in the table and assert the intended outcome. That test is the one that would
+have caught this.
+
+### HIGH-1 (new) — The dangling-supplemental cleanup uses the pre-retention model set, so it silently strips a legitimate plan inclusion from a model the same generation publishes (a)
+
+**Locations:** `catalog-v3-builder.ts:93` (`mergedModelIds`), `:96-101` (passed to
+`applyRosterEvidenceToPlansV3`), `:187-208` (active-model retention), `:212`
+(`projectSubscriptionPlanMembershipV3`); the entry-level and inclusion-level `modelIds.has(...)`
+tests in `applyRosterEvidenceToPlansV3`.
+
+**Problem.** Two different "does this model exist?" answers are used in one generation.
+`mergedModelIds` is built at `:93` from `input.mergedModels` — this run's fresh merge only —
+and is the set the new cleanup consults. But the **published** model set is assembled later
+at `:187-208`, where the builder deliberately re-adds every active-generation model that is
+missing from the fresh merge and whose sources did not certify completeness, stamping it
+`dataFreshnessWarning: true`. So a model can be absent from `mergedModelIds` and present in
+`candidate.models`. For that model the cleanup fires: its supplemental inclusion is dropped
+as "dangling" even though the very same candidate publishes it. `:212`
+`projectSubscriptionPlanMembershipV3` then derives `subscriptionPlanIds` from the pruned
+inclusions, so the membership disappears too — and because the inclusion is gone, the guard
+that would have shouted (`plan ... inclusion references missing model ...`) never fires. The
+loud failure I reported has become a silent one.
+
+**Concrete failure scenario, with the wrong output.** `gpt-5.5-codex` is listed by the
+OpenAI Codex public registry and, being Codex-only, is reported by no aggregator — the
+`openai-api`/codex collectors are its sole source. One run, that collector returns a
+transient 500. `input.mergedModels` therefore lacks `gpt-5.5-codex`, so `mergedModelIds`
+lacks it. The Codex supplemental roster is fine (or carried forward), and its entry names
+`gpt-5.5-codex`, so `supplementalEntryKeys` contains that provider/external pair and the
+inclusion is pruned. Retention at `:187-208` then re-adds the model with
+`dataFreshnessWarning: true`. The generation activates cleanly. **Published output:
+`gpt-5.5-codex` present, callable, and with no `openai-codex` in `subscriptionPlanIds`; the
+`openai-codex` plan lists one fewer member.** On our side a `[email protected]` request is no
+longer recognised as plan-included, so the model is priced per-token and accrues spend for a
+user on a flat-rate Codex subscription — the exact money-costing under-report you asked me to
+watch for. It self-heals on the next successful collection, which makes it harder to notice,
+not less expensive.
+
+**Why this is the same pattern.** The rule ("drop a supplemental positive whose target no
+longer exists") is right. Its input — "the set of models that exist" — is read one stage too
+early, from the set that is known to be incomplete precisely in the scenario the rule fires in.
+
+**Suggestion.** Compute the cleanup against the final published id set, or at minimum
+`mergedModelIds` unioned with the active model ids that retention will re-add. The simplest
+correct shape is to move the inclusion pruning after `:208` and pass
+`new Set(models.map(m => m.modelId))`. Then add the missing test: an active generation whose
+model is absent from the next run's `mergedModels` but retained by `:187-208`, asserting the
+Codex inclusion and `subscriptionPlanIds` both **survive**.
+
+### MEDIUM-1 — For the three pointers ingress already rewrites, the "keep it visibly mutable" fallback does the opposite, and a pointer alone can still manufacture a dated canonical model (a)
+
+**Locations:** `applyReviewedMistralPointerTargets`'s fallback branch (`merger.ts` at
+`d7acfd7`); `schema-runtime.ts:420-441` `canonicalizeMistralModelId`;
+`mistral-identity-policy.ts:22-29`.
+
+The fallback's own comment says "Ingress canonicalizers ... may not turn a moving API pointer
+into catalog identity. Keep it visibly mutable so the v3 candidate gate can reject it" — and
+then implements that by calling `canonicalizeModelId(raw.externalId)`, i.e. **the ingress
+canonicalizer it just declared untrustworthy**. For the three pointers that have a hardcoded
+rule there, the result is a dated id, not a mutable one:
+`mistral-medium-latest` to `mistral-medium-2604`, `voxtral-mini-latest` to
+`voxtral-mini-2602`, `voxtral-mini-tts-latest` to `voxtral-mini-tts-2603`.
+
+Consequences for `mistral-medium-latest`, which is **not** a key of the reviewed map:
+
+- If no owner source observes the dated id this run, the pointer row alone creates the
+ canonical model `mistral-medium-2604`, carrying the pointer's context window, capabilities
+ and description. Section 2.2's "mutable pointers cannot manufacture canonical identity" is
+ satisfied in letter (the published id is dated) and broken in substance. The route is safe —
+ dated canonical plus `-latest` external hits the third guard, giving `unknown` — so no
+ callable moving target reaches us; the defect is in identity and metadata provenance.
+- The reason is also wrong per the reviewed table: this pairing publishes
+ `mutable_pointer_only`, while section 4.2 assigns `mistral-medium-2604` the reason
+ `identity_conflict` (its reviewed pointer is `magistral-medium-latest`). The commit thus
+ contains two contradicting reviewed statements about the same model: the canonicalizer says
+ `mistral-medium-latest` *is* `mistral-medium-2604`, and the quarantine table says that
+ model's pointer is a Magistral id.
+
+**Suggestion.** In the fallback, use the literal normalized `externalId` (lowercase and trim
+only) rather than `canonicalizeModelId`, so the comment becomes true; then decide
+`mistral-medium-latest`'s reviewed target explicitly — either add it to the table with a
+reason, or delete the stale canonicalizer rule. Whichever way, both statements should live in
+one place. Note the interaction with CRITICAL-1: making the fallback honest here *adds* one
+more id to the set that blocks generation, so fix CRITICAL-1 first.
+
+### MEDIUM-2 — The deliberate gate still fires on order-only churn, which reintroduces the HIGH-1 freeze whenever an upstream reorders a list (a for the code, (b)/(c) for the trigger)
+
+`semanticCandidate` sorts the five **top-level** arrays (`models` by `modelId`, `plans` by
+`id`, `rosters` by `rosterId`, `redirects` by a composite key, `changes` by `modelId`), but
+`deliberateRoster` (`catalog-v3-deliberate-diff.ts:143-150`) maps `entries` without sorting
+them, and plan `inclusions` are left in `dedupeInclusions([...retained, ...rosterInclusions])`
+order. So two collections with identical content but a different upstream ordering produce
+different canonical JSON for `rosters` and `plans`, `changed` becomes `true`, and activation
+is refused pending an operator digest — the same silent freeze as HIGH-1, narrowed from
+"every run" to "any run where an upstream list order changes".
+
+Whether that happens is a property of the upstreams I cannot read from here: a JSON array
+from the Codex registry is probably stable, whereas a roster derived from an object/map
+iteration or a scraped table with re-ranked rows is not. Supplemental merging appends carried
+entries after incoming ones (visible in `catalog-v3-builder.test.ts:327-331`, which expects
+`[gpt-5.4-mini, gpt-5.5]` — incoming then carried), so the ordering is a function of upstream
+order plus which entries happened to expire. Low cost to close: sort `entries` by
+`externalModelId` and `inclusions` by a stable key inside the *projection only*, exactly as
+the top-level arrays already are. The persisted documents need not change.
+
+### LOW-1 — `prepareCollectorResultsForCatalogV3` re-slices rows positionally, so any future filter silently mis-attributes models to the wrong collector (a)
+
+The function flattens all collector rows, transforms them, then walks a running `offset`
+slicing `result.models.length` rows back into each result. That is correct today because both
+stages are `.map()`. If either ever filters or expands — which is exactly what a future
+"drop unresolved pointer rows" change would do, and what CRITICAL-1's fix may require — the
+slices shift and rows are re-attributed to neighbouring collectors. The consequences are
+silent and severe, because `collectorId` determines `sourceProviderIdForCollector` and hence
+the route a model is published with; a shifted row becomes a wrong but well-formed route.
+Key by an explicit index or a per-row collector tag instead of trusting positional
+alignment, or assert `preparedRaw.length === allRaw.length` at minimum.
+
+### LOW-2 — The LOW-1 deferral is not recorded in the repo
+
+Covered under item 6 above: correct non-application, no durable record. Two lines beside the
+error table in `docs/api-reference.md` would close it.
+
+---
+
+## What I verified, and what I did not reach
+
+**Verified by reading the committed code (a):** `catalog-v3-deliberate-diff.ts` in full and
+every schema it projects; `merger.ts`'s new entry points, `applyReviewedMistralPointerTargets`,
+`recanonicalizeAliasBackedRawModels`, `isOwnerTopLevelSource`, `COLLECTOR_TO_VENDOR`;
+`catalog-v3-builder.ts` `:53-265` and `:360-420`; `applyRosterEvidenceToPlansV3` in full;
+`subscription-alias-validation.ts`'s diff and `projectServingEvidence`;
+`mistral-identity-policy.ts` in full; `canonicalizeModelId` and every Mistral and PDecimal
+canonicalizer it calls; `collectors/api/mistral.ts` `collect()`;
+`manual-overrides/engine.ts` `applyManualModelMappings` and `doc-engine.ts`
+`applyManualModelDocMappings` (neither drops nor renames a doc, and synthetic rows pass
+`validateRawModel`, so they cannot produce a non-canonical serving key);
+`enrichVendorRecords` (maps only); `publishCatalogV3`'s error handling; the full
+`catalog-v3-cutover.yml` addition; `orchestrator.ts`'s four Mistral collector registrations;
+and the deletion set via the commit's tree listing.
+
+**Inferred from their tests (b):** the G1 auto-activation outcome; the seven published
+quarantine rows; the pre-remapped reset; the supplemental-cleanup cases; and the live
+composition of Mistral's `/v1/models` roster, which I inferred from their captured
+model-page and pricing fixtures rather than from the API.
+
+**Could not determine (c):** the actual current contents of
+`https://api.mistral.ai/v1/models` — CRITICAL-1's severity rests on it containing at least
+one `-latest` id outside the reviewed seven, which their own fixtures indicate but I could
+not confirm live (I hold no Mistral key and would not call a paid API from a review). **If
+their team dumps that roster and diffs it against the seven-key map, that single command
+settles CRITICAL-1 either way.** Also not reached: which manual mapping rules exist in
+production Firestore; `catalog-generation.ts`'s write/CAS/retention internals (unchanged by
+this commit); the emulator smoke run and their test suite, both deliberately not executed per
+the read-only constraint; and the roughly twenty test files untouched by `d7acfd7`.
+
+**Not re-verified because you closed it:** the 426 wire envelope and
+`docs/fixtures/catalog-v3-upgrade-required.json`. I did confirm the fixture is now the single
+source for their own contract test (`catalog-v3-http.test.ts:14-18, 94`).
+
+## What claudish should do next
+
+1. **Escalate CRITICAL-1 before anything else.** It is a pre-cutover blocker: deployed as-is,
+ catalog generation stops on the first scheduled run and we silently serve an ageing
+ generation. Ask for the live `/v1/models` id dump as the deciding evidence.
+2. **Escalate HIGH-1(new) as a money bug.** It is quiet, self-healing, and drops a flat-rate
+ user's plan membership — the failure mode we least want to debug from the client side.
+3. **No change to our 426 parser**, and we can now pin our test to their committed fixture
+ path instead of a hand-written literal.
+4. **Do not build on the assumption that the catalog advances after cutover** until
+ CRITICAL-1 is resolved. A frozen `generationId` with well-formed 200s is indistinguishable
+ from health on our side; if we want a client-side canary, `generationId` staleness plus
+ `rosterCoverage: {status:"unknown", reason:"expired"}` across multiple plans is the signal.