Skip to content

πŸͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

chore(pricing): document gemini-3.5/3.7-flash permanent-alias lifecycle gap (#18900)

langfuse-bot3c9f45d2b72b
diff --git a/.agents/skills/add-model-price/references/model-audit-memory.md b/.agents/skills/add-model-price/references/model-audit-memory.md
index 63b1e710c8..99f084cefb 100644
--- a/.agents/skills/add-model-price/references/model-audit-memory.md
+++ b/.agents/skills/add-model-price/references/model-audit-memory.md
@@ -12,11 +12,52 @@ audit date.
## Latest useful snapshot
-**Audit date:** 2026-10-14
+**Audit date:** 2026-10-21
All prices listed as `$X / MTok` (per million tokens). Per-token JSON values: divide by 1,000,000.
-The 2026-10-14 run re-fetched the full Anthropic pricing page (model table, cache-hits
+The 2026-10-21 run re-fetched the full Anthropic pricing page (model table, cache-hits
+footnote, Fast mode table, Batch table) and the Anthropic model-deprecations page (full
+"Model status" table), the OpenAI aggregate Standard pricing table (every
+`gpt-6`/`gpt-5.6`/`gpt-5.5`/`gpt-5.4`/`gpt-5.3-codex`/`gpt-5.2`/`gpt-5.1`/`gpt-5`/o-series/
+`gpt-4.1`/`gpt-4o` row, plus every GPT-6/GPT-5.6-family Large Context row) and deprecations
+page, the OpenAI model catalog, both Gemini pricing pages (3.x and 2.5 families), the Gemini
+models catalog, the Gemini deprecations page, the AWS Bedrock pricing page, and the TypeSafe
+Jev models page. Every price was cross-checked field-by-field against the live pricing JSON
+with `jq` (not just visually against a WebFetch summary) and matched exactly; no pricing-file
+edits were needed. This run resolved one standing gap and reconfirmed one recurring fetch
+hazard, neither requiring a code change:
+1. **gemini-3.5-flash and gemini-3.7-flash's "model not found" pricing-page gap (open since
+ October 13, 2026) is now explained, not just worked around**: the Gemini deprecations page
+ carries an explicit row for each with Shutdown date `N/A` and a replacement note stating
+ the endpoint "is not shutting down" but auto-routes requests to `gemini-3.6-flash` /
+ `gemini-3.8-flash` respectively, and that it is "not available to select in Google AI
+ Studio." Both IDs are also fully absent from the models catalog page (not even labeled
+ "Shut down"). This is a new lifecycle category the existing three removal criteria in
+ `automated-audit.md` do not cover (not shut down, not a preview/dated snapshot, still
+ callable β€” just silently proxied) β€” left unchanged in both selectable arrays and reported
+ as unresolved; see the detailed entry in `provider-sources-and-price-keys.md` for why this
+ needs a human decision, especially since `gemini-3.5-flash` is the first/default entry of
+ `vertexAIModels`.
+2. **A second confirmed instance of the OpenAI deprecations-page date-confusion failure mode**
+ (first seen October 9, 2026): a WebFetch of `developers.openai.com/api/docs/deprecations`
+ mischaracterized the October 23, 2026 "Legacy GPT model snapshots" shutdown (still 2 days
+ in the future as of this run) as "has now expired (2 days ago)." Not actioned; this batch
+ remains correctly un-removed from `openAIModels` pending October 23, 2026 actually passing.
+No other drift was found: `claude-haiku-4-5-20251001`'s "Not sooner than October 15, 2026"
+retirement floor has passed without a firm date being announced (still `Active`,
+`Deprecated: N/A`) β€” not actionable, a floor rather than a trigger. The January 14, 2027
+"GPT-4 series" batch is unchanged and still 85 days out. No new flagship text/chat/reasoning
+model was found on any provider. AWS Bedrock's pricing page again yielded no extractable
+per-model numeric rows for any Claude model (standing limitation). TypeSafe's Jev page
+confirms `jev-1.13.0`/`jev-1.14.0` unchanged. This run independently re-verified (via `jq`
+against the live JSON) every Anthropic model in the main pricing table, every OpenAI
+`gpt-6`/`gpt-5.6`/`gpt-5.5`/`gpt-5.4`/`gpt-5.3-codex`/`gpt-5.2`/`gpt-5.1`/`gpt-5`/o-series/
+`gpt-4.1`/`gpt-4o` entry (Standard and Large Context tiers), every priced Gemini 2.5/3.x
+model, and the Jev entry; the legacy OpenAI catalog tail (`gpt-4-turbo`, `gpt-4-0613`,
+`gpt-3.5-turbo*`, `davinci-002`, `babbage-002`) was not independently re-fetched this run,
+consistent with prior audits' "flagship model" scoping.
+The 2026-10-14 run before it re-fetched the full Anthropic pricing page (model table, cache-hits
footnote, Fast mode, Batch, and tool-use tables) and the Anthropic model-deprecations page,
the OpenAI aggregate Standard pricing table and the OpenAI deprecations page, both Gemini
3.x pricing fetches and the Gemini models catalog, the AWS Bedrock pricing page, and the
diff --git a/.agents/skills/add-model-price/references/provider-sources-and-price-keys.md b/.agents/skills/add-model-price/references/provider-sources-and-price-keys.md
index aeb6f10993..1b901d8d81 100644
--- a/.agents/skills/add-model-price/references/provider-sources-and-price-keys.md
+++ b/.agents/skills/add-model-price/references/provider-sources-and-price-keys.md
@@ -1218,21 +1218,40 @@ json_literal = "<price_per_mtok>e-6"
above. Re-attempt a full tabular fetch in a future audit if a Bedrock-only price (such as
the Public Extended Access SKU) needs independent reconfirmation.
- **Gemini 3.x pricing-page single-model fetch gap for gemini-3.5-flash and
- gemini-3.7-flash (found October 13 2026)** β€” Targeted WebFetch requests against
- `https://ai.google.dev/gemini-api/docs/pricing` asking only about `gemini-3.5-flash` (the
- non-Lite sibling) or `gemini-3.7-flash` each returned "model not found on this page," even
- though broader fetches of the same page in the same run returned data for
- `gemini-3.8-flash`, `gemini-3.6-flash`, and `gemini-3.5-flash-lite`, and the Gemini models
- catalog (`ai.google.dev/gemini-api/docs/models`) lists both as active (not labeled "Shut
- down") in the same run. This is a different failure mode from the known free/paid
- column-collapse issue: the row appears to be silently dropped rather than
- misattributed. Do not treat a "model not found" result for these two IDs as evidence of
- removal β€” cross-check the models catalog's shutdown labels first, and prefer a fetch that
- asks for every row on the page at once (risking column-collapse, correctable with a
- Free/Paid-explicit follow-up) over a single-model-targeted fetch on this specific page
- (risking the row being dropped entirely). `gemini-3.5-flash` and `gemini-3.7-flash` prices
- were left unchanged this run, carried forward from their last confirmed values
- ($1.50/$9.00/$0.15 and $0.75/$3.75/$0.075 respectively).
+ gemini-3.7-flash (found October 13 2026; root cause confirmed October 21 2026)** β€”
+ Targeted WebFetch requests against `https://ai.google.dev/gemini-api/docs/pricing` asking
+ only about `gemini-3.5-flash` (the non-Lite sibling) or `gemini-3.7-flash` each returned
+ "model not found on this page," even though broader fetches of the same page in the same
+ run returned data for `gemini-3.8-flash`, `gemini-3.6-flash`, and `gemini-3.5-flash-lite`,
+ and the Gemini models catalog (`ai.google.dev/gemini-api/docs/models`) lists both as active
+ (not labeled "Shut down") in the same run. The October 21 2026 audit resolved this: it is
+ not a WebFetch artifact. `https://ai.google.dev/gemini-api/docs/deprecations`'s "Gemini 3
+ models" table carries an explicit row for each, under columns "Model", "Release date",
+ "Shutdown date", "Recommended replacement": `gemini-3.5-flash` β†’ Shutdown date `N/A`,
+ replacement `gemini-3.6-flash` ("not available to select in Google AI Studio; the endpoint
+ is not shutting down and automatically routes API requests to `gemini-3.6-flash`");
+ `gemini-3.7-flash` β†’ Shutdown date `N/A`, replacement `gemini-3.8-flash` (same wording,
+ routes to `gemini-3.8-flash`). Both IDs are also completely absent from the models catalog
+ page now (not even labeled "Shut down" β€” simply not listed), which is why a single-model
+ pricing-page fetch finds no row: Google no longer prices them as independent products, it
+ silently proxies the request to the stated successor at the successor's price. This is a
+ new lifecycle category not covered by the three removal criteria in
+ `automated-audit.md`'s "Selectable Model Availability": the model is explicitly **not**
+ shut down and has no shutdown date, so it is not "shut down or past its retirement date"; it
+ is a GA model ID (not a preview/experimental/dated snapshot); and it remains callable
+ through the text-generation endpoint (requests still succeed, just silently billed and
+ served as the successor model). Do not remove either ID from a selectable array on this
+ evidence alone β€” report it as unresolved for a human to decide whether a fourth removal
+ criterion ("confirmed not selectable in the provider's own console, with requests
+ auto-routing to a successor already present in the array") should be added, since this
+ precedent will likely recur for other Gemini point releases. Note `gemini-3.5-flash` is
+ currently the **first (default) entry of `vertexAIModels`**; removing it would promote
+ `gemini-2.5-flash` to default, which itself carries the September 2026 access-restriction
+ notice (see below) β€” exactly the "next entry is itself deprecated" scenario the hard
+ constraints say to report rather than resolve automatically. `gemini-3.5-flash` and
+ `gemini-3.7-flash` prices were left unchanged this run, carried forward from their last
+ confirmed values ($1.50/$9.00/$0.15 and $0.75/$3.75/$0.075 respectively) since neither has
+ an independent current price to confirm against.
- **Gemini 4 Argon (added October 14 2026)** β€” `gemini-4-argon` is Google's new
flagship reasoning model, confirmed "New Stable" via
@@ -1298,6 +1317,47 @@ json_literal = "<price_per_mtok>e-6"
already-tracked October 23, 2026 "Legacy GPT model snapshots" batch (see
the October 9 2026 entry above) β€” do not conflate the two shutdown dates or
model lists when a future audit processes either one.
+- **October 21 2026 audit: no price drift; a second confirmed instance of the
+ deprecations-page date-confusion failure mode; gemini-3.5-flash/gemini-3.7-flash
+ resolved as a new "permanent alias, not shut down" lifecycle category (see the
+ updated entry above)** β€” Re-fetched the full Anthropic pricing page (model table,
+ cache-hits footnote, Fast mode table, Batch table) and the Anthropic
+ model-deprecations page (full "Model status" table), the OpenAI aggregate Standard
+ pricing table (every `gpt-6`/`gpt-5.6`/`gpt-5.5`/`gpt-5.4`/`gpt-5.3-codex`/`gpt-5.2`/
+ `gpt-5.1`/`gpt-5`/o-series/`gpt-4.1`/`gpt-4o` row, plus every GPT-6/GPT-5.6-family
+ Large Context row), `https://developers.openai.com/api/docs/deprecations`, the OpenAI
+ model catalog, both Gemini pricing pages (3.x and 2.5 families), the Gemini models
+ catalog, the Gemini deprecations page, the AWS Bedrock pricing page, and the TypeSafe
+ Jev models page. Every price checked β€” cross-verified field-by-field against the live
+ JSON with `jq`, not just visually compared β€” matched exactly; no pricing-file edits
+ were needed. Two findings:
+ 1. **The October 23, 2026 "Legacy GPT model snapshots" shutdown date has still not
+ passed (it is 2 days in the future as of this run's date, 2026-10-21), but a
+ generically-worded WebFetch of the deprecations page again mischaracterized it as
+ "has now expired (2 days ago)"** β€” the same date-direction confusion documented in
+ the October 9 2026 entry above, now confirmed recurring. The summarizer appears to
+ anchor on the gap between today and the date without reliably determining which
+ direction it runs. Do not trust a WebFetch's own "expired" / "upcoming" framing for
+ a deprecation date close to the audit's run date; always compare the literal date
+ string against the run date yourself (`date -u +%Y-%m-%dT00:00:00.000Z`) before
+ treating a shutdown as actionable. This batch remains correctly un-removed from
+ `openAIModels`; re-check on or after October 23, 2026 per the October 9 2026 entry's
+ model list.
+ 2. **gemini-3.5-flash and gemini-3.7-flash's "model not found" pricing-page fetch gap
+ (open since October 13, 2026) is resolved** β€” see the updated entry above for the
+ full finding: both are permanently aliased to a successor via the Gemini
+ deprecations page, with no shutdown date, left unchanged in both selectable arrays
+ pending a human decision on the removal criteria.
+ No other drift was found: `claude-haiku-4-5-20251001`'s tentative retirement floor
+ ("Not sooner than October 15, 2026") has passed without a firm date being announced
+ (still `Active`, `Deprecated: N/A` on the Anthropic model-deprecations page) β€” not
+ actionable, consistent with the "not sooner than" wording meaning a floor, not a
+ trigger. The January 14, 2027 "GPT-4 series" batch (`gpt-4o`/`gpt-4o-2024-08-06`/
+ `gpt-4.1`/`gpt-4.1-2025-04-14`/`gpt-4.1-mini`/`gpt-4.1-mini-2025-04-14`) is unchanged
+ and still 85 days out. No new flagship text/chat/reasoning model was found on any
+ provider. AWS Bedrock's pricing page again could not be made to yield extractable
+ per-model numeric rows for any Claude model (standing limitation); TypeSafe's Jev page
+ confirms `jev-1.13.0`/`jev-1.14.0` unchanged at $42/Btok input, output free.
## Provider Usage Keys