Skip to content

armgit / avibe

Public Imported from GitHub
0

Make your computer a home for your AI partner. Avibe is the local-first Agent OS: Vibey, its built-in AI partner, works out of the box, and Claude Code, Codex & OpenCode run unchanged beside it — reach them from your browser, phone, or any chat app.

126 KB
Raw Blame
"""Bounded, credential-free persistence for Model Hub token usage.

Only the code that sees a complete upstream model response can count tokens, and
one turn can make several upstream calls. Two callers therefore report calls here,
over populations that ``InvokeHandle.stream is not None`` keeps disjoint: the
resolver reports every call it consumed itself, including a failover hop that
billed us before the turn moved on, and the turn gateway reports every call whose
body it forwarded. This module owns what happens to those counts afterwards: one
bounded daily aggregate per source, model, and context tier, and the read shape the
settings page consumes.

Two properties are deliberate. `requests` is self-measured by our own code and is
always available; token counts are vendor-reported and may be absent, which is
why `token_reports` is tracked separately instead of treating a missing report as
zero usage. And nothing here ever feeds admission, routing, or cooldown — a
hostile upstream must not be able to change resolution behavior by lying about
usage.

What "bounded" bounds is the file: a fixed number of daily rows over a fixed
retention window, each carrying counters the settings page can still read back
exactly. It was never a bound on the counts themselves — how much a user spends
is not ours to cap — so every aggregate here is an exact sum, and
`USAGE_COUNTER_CEILING` constrains only what crosses into the file and back out
of it. A row carrying a counter past it is dropped at whichever door it reaches,
loudly, rather than rewritten into a number nobody spent.

A search-service request reaches no model, so it gets its own row in the same file
and the same daily and hourly lifecycle: one per Source and catalog `service_id`,
counting sent requests rather than model calls (`web-search-services.md` §4.3). It
is told apart by the `service_id` a model row never carries, and a row naming both
a model and a service is unreadable. Model counters never include it.
"""

from __future__ import annotations

import asyncio
import json
import logging
import math
import queue
import threading
from concurrent.futures import Executor
from concurrent.futures import Future as CFuture
from dataclasses import dataclass, replace
from datetime import date, datetime, time, timedelta, timezone
from pathlib import Path
from time import monotonic
from typing import Callable, Final, Mapping, Optional, Sequence

from .fast_mode import FAST_SPEED
from .identifiers import persisted_ledger_key, usage_ledger_key
from .pricing import Cost, ModelPrice, PriceTable, ServiceCost, context_tier_size, row_cost, service_row_cost
from .state_file import write_state_document
from .stream_wire import ProtocolUsageReport

logger = logging.getLogger(__name__)

# Roughly two months of daily rows: long enough for a monthly view plus the
# previous cycle, short enough that the file stays small on a busy machine.
USAGE_RETENTION_DAYS: Final = 62
USAGE_MAX_ROWS: Final = 400
USAGE_DEFAULT_WINDOW_DAYS: Final = 30
USAGE_HOURLY_RETENTION_HOURS: Final = 24
USAGE_WINDOW_KEYS: Final = ("24h", "7d", "30d", "60d")
# The largest integer the settings page holds exactly. Nothing on this side of the
# wire needs it: `json.loads` reads a counter of any size into an exact `int`, and
# so do the rpc and client hops. The doubles appear one boundary later, in the
# browser parsing the published summary, and this is the only number in the module
# that comes from there — which is worth saying plainly, because calling it a
# property of the file is what made it look reusable as a bound on a sum.
#
# It is applied at the file's doors because that is where a value can still be
# refused. A published aggregate is a sum, and refusing a sum is the defect this
# module was fixed for; admitting a row only up to this instead bounds every
# published count at `USAGE_MAX_ROWS` times it, a range the consumer holds as a
# finite double and the contract can therefore declare and satisfy — with no
# ceiling anywhere on how much a user may spend.
#
# Deliberately not the per-report ceiling, and nothing between the two doors
# clamps to it. Nothing this module writes can approach it either: a window spans
# at most `USAGE_RETENTION_DAYS` days of calls each bounded by
# `stream_wire.USAGE_REPORT_TOKEN_CEILING`, so what it actually guards is a corrupt
# or hand-edited file. There it drops the row rather than saturating it: a
# saturated counter is a number nobody spent, and saturating a subset with its
# superset forces a cached-input share to exactly 100% — the artifact this module
# exists to not produce.
USAGE_COUNTER_CEILING: Final = 2**53 - 1
# The largest count `summary` can publish, and the maximum
# `usage-summary.schema.json` declares. Derived rather than imposed: a published
# count is a sum over one window, `window` reports at most `USAGE_MAX_ROWS` rows,
# and no row reaches it carrying a counter past `USAGE_COUNTER_CEILING` — not from
# the file, where `_counter` clears each one, and not from a merge, which `_read`
# re-checks because a sum of cleared addends is not itself cleared.
#
# Both capacities binding the reported set is what makes the product a bound on
# what this module can publish, rather than a description of the files it happens
# to write. Deriving it from the write path alone was true of every file this
# ledger produces and false of every other one — a reader handed a larger file
# published a total above its own contract.
#
# Saying it out loud is what lets the contract carry a maximum that is both true
# and satisfiable; the two things tried before were a maximum the producer could
# violate, and none at all, which left a consumer nothing to size against and said
# nothing about where exactness ends. Never a clamp: it states the range, and
# nothing compares a sum against it. Every value in the range is a finite double,
# only the part above `USAGE_COUNTER_CEILING` rounds, and reaching that at all
# takes a corrupt file — a window spans at most `USAGE_RETENTION_DAYS` days of
# real calls.
USAGE_PUBLISHED_COUNT_BOUND: Final = USAGE_MAX_ROWS * USAGE_COUNTER_CEILING
# Anything older than every instant this ledger can hold, so a row whose recency
# cannot be read — it recorded no instant, or recorded one that has not happened —
# sorts as the least recently metered.
_OLDEST_INSTANT: Final = datetime.min.replace(tzinfo=timezone.utc)
# A persisted owner day has no timezone of its own. Every IANA timezone currently
# in use fits inside UTC-12..UTC+14, so this is the smallest conservative envelope
# that can contain one whole local day without consulting today's host timezone.
_OWNER_DAY_MAX_AHEAD: Final = timedelta(hours=14)
_OWNER_DAY_MAX_BEHIND: Final = timedelta(hours=12)

_COUNTER_KEYS: Final = (
    "requests",
    "token_reports",
    "input_tokens",
    "cached_input_tokens",
    "output_tokens",
    "cache_write_input_tokens",
    "cache_write_1h_input_tokens",
    "cache_write_30m_input_tokens",
    "cache_write_12h_input_tokens",
    "cache_write_uncaptured_reports",
    "cache_write_ttl_uncaptured_reports",
)
# Counters added after files were already written. A released row lacks them, and
# what it lacks is knowledge, not usage: its cache writes are inside its input but
# were never split out, so every token report it carries counts as uncaptured.
# Additive only — no stored row is rewritten to acquire them.
_NEW_CACHE_WRITE_KEYS: Final = (
    "cache_write_30m_input_tokens",
    "cache_write_12h_input_tokens",
    "cache_write_ttl_uncaptured_reports",
)
# The cross-field guarantees the read contract makes, as (subset, superset).
# Each one is repaired on read, so a corrupt or hand-edited file degrades into a
# smaller true statement instead of publishing an impossible one: a coverage
# figure above 100% is more misleading than a conservative one. Order matters:
# a superset is repaired before the subsets that are read against it.
_COUNTER_SUBSETS: Final = (
    ("cached_input_tokens", "input_tokens"),
    ("token_reports", "requests"),
    ("cache_write_uncaptured_reports", "token_reports"),
    ("cache_write_ttl_uncaptured_reports", "token_reports"),
    ("cache_write_1h_input_tokens", "cache_write_input_tokens"),
    ("cache_write_30m_input_tokens", "cache_write_input_tokens"),
    ("cache_write_12h_input_tokens", "cache_write_input_tokens"),
)


def _released_cache_write_counters(value: dict) -> dict[str, int]:
    """What a counter set written before cache writes were metered stands for."""

    reports = value.get("token_reports")
    uncaptured = reports if isinstance(reports, int) and not isinstance(reports, bool) else 0
    return {
        "cache_write_input_tokens": 0,
        "cache_write_1h_input_tokens": 0,
        "cache_write_30m_input_tokens": 0,
        "cache_write_12h_input_tokens": 0,
        "cache_write_uncaptured_reports": max(0, uncaptured),
        "cache_write_ttl_uncaptured_reports": 0,
    }


def _repair_cache_writes(counters: dict) -> None:
    """Keep cache writes inside the input that cache reads did not already claim."""

    counters["cache_write_input_tokens"] = min(
        counters["cache_write_input_tokens"],
        max(0, counters["input_tokens"] - counters["cached_input_tokens"]),
    )
    counters["cache_write_1h_input_tokens"] = min(
        counters["cache_write_1h_input_tokens"], counters["cache_write_input_tokens"]
    )
    remaining = max(0, counters["cache_write_input_tokens"] - counters["cache_write_1h_input_tokens"])
    counters["cache_write_30m_input_tokens"] = min(counters["cache_write_30m_input_tokens"], remaining)
    counters["cache_write_12h_input_tokens"] = min(
        counters["cache_write_12h_input_tokens"],
        max(0, remaining - counters["cache_write_30m_input_tokens"]),
    )


# A service row's counts (`web-search-services.md` §4.3): requests sent, those the
# catalog's billing rule charges, those whose billing is unknown, and those the
# vendor priced itself. Every billable request is valued at the request's list
# price, so no separate list-priced quantity exists. `reported_usd`, the vendor's
# own figures, rides beside them as an amount rather than a count.
_SERVICE_COUNTER_KEYS: Final = (
    "requests",
    "billable_requests",
    "uncertain_requests",
    "reported_requests",
)
# What one request's reported cost can be at most: a fixed bound above every catalog
# version's own per-request ceiling, so a stored figure past it is corruption.
_SERVICE_REPORTED_USD_CEILING: Final = 1.0


def _is_service(value: Mapping) -> bool:
    """Whether a normalized row, hour slice or counter set counts search requests.

    Every service counter set carries `billable_requests` and no model one does, so
    the counters themselves say which arithmetic applies wherever they travel.
    """

    return "billable_requests" in value


def _counter_keys(value: Mapping) -> tuple[str, ...]:
    return _SERVICE_COUNTER_KEYS if _is_service(value) else _COUNTER_KEYS


def _amount(value: object) -> Optional[float]:
    """Read one persisted dollar amount, or None when it is not finite and non-negative."""

    if isinstance(value, bool) or not isinstance(value, (int, float)):
        return None
    try:
        amount = float(value)
    except OverflowError:
        return None
    if not math.isfinite(amount) or amount < 0:
        return None
    return amount


def _service_relations_hold(counters: Mapping) -> bool:
    """The relations a service counter set keeps on every door (§4.3).

    The reported amount is bounded by its count rather than compared for equality
    anywhere: it is a float sum, so two orders of the same additions may differ in
    their last place, while each addend's bound survives any order.
    """

    return (
        counters["billable_requests"] + counters["uncertain_requests"] <= counters["requests"]
        and counters["reported_requests"] <= counters["billable_requests"]
        and counters["reported_usd"] <= counters["reported_requests"] * _SERVICE_REPORTED_USD_CEILING
    )


def _normalize_service_totals(value: object) -> Optional[dict]:
    """Read strict service counters for a retained slice or pruned history, or None.

    Like a model slice, nested evidence is refused rather than repaired: the daily
    row is the authoritative aggregate.
    """

    if not isinstance(value, dict):
        return None
    totals: dict = {}
    for key in _SERVICE_COUNTER_KEYS:
        counter = value.get(key)
        if not isinstance(counter, int) or isinstance(counter, bool) or counter < 0:
            return None
        if counter > USAGE_COUNTER_CEILING:
            return None
        totals[key] = counter
    amount = _amount(value.get("reported_usd"))
    if amount is None:
        return None
    totals["reported_usd"] = amount
    return totals if _service_relations_hold(totals) else None


def _repair_service_counters(row: Mapping) -> Optional[tuple[dict, bool]]:
    """Read one daily service row's counts, repaired to their relations (§4.3).

    None when a count is past the ceiling, which makes the row unreadable exactly as
    it does a model row. A field the row lacks reads as zero. A row that breaks a
    relation keeps `requests`, has its counts reduced to their bounds and drops the
    reported quantities that a reduced count no longer covers, or that break their
    own bound (orphaned dollars included); the second value says so, and the caller
    marks the row a lower bound.
    """

    counters: dict = {}
    broken = False
    for key in _SERVICE_COUNTER_KEYS:
        raw = row.get(key, 0)
        counter = _counter(raw)
        if counter is None:
            return None
        broken = broken or type(raw) is not int or raw < 0
        counters[key] = counter
    amount = _amount(row.get("reported_usd", 0.0))
    billable_reduced = counters["billable_requests"] > counters["requests"]
    if billable_reduced:
        counters["billable_requests"] = counters["requests"]
        broken = True
    unbilled = counters["requests"] - counters["billable_requests"]
    if counters["uncertain_requests"] > unbilled:
        counters["uncertain_requests"] = unbilled
        broken = True
    if (
        billable_reduced
        or amount is None
        or counters["reported_requests"] > counters["billable_requests"]
        or amount > counters["reported_requests"] * _SERVICE_REPORTED_USD_CEILING
    ):
        counters["reported_requests"] = 0
        amount = 0.0
        broken = True
    counters["reported_usd"] = amount
    return counters, broken


def _utc_now() -> datetime:
    return datetime.now(timezone.utc)


def _carried(moment: datetime) -> Optional[datetime]:
    """Carry one moment through every conversion this module performs, or refuse it.

    `datetime` conversion is not total. A value near either end of the representable
    range, offset far enough, leaves that range on the way to another zone and raises
    `OverflowError` — which is an `ArithmeticError`, so it passes straight through a
    handler written for bad data and takes the flush task with it, stopping metering
    for the rest of the process while the row that caused it stays on disk.

    Bounding the accepted years would leave a free parameter for the next value to
    probe. The conversion itself is the bound instead: what this module can carry is
    what its own conversions return, measured rather than declared. Both are
    performed here — UTC for the spelling it publishes, local for the day it buckets
    — so nothing that escapes this door can raise at either of them later.
    """

    try:
        carried = moment.astimezone(timezone.utc)
        carried.astimezone()
    except (OverflowError, OSError, ValueError):
        return None
    return carried


def _aware(moment: datetime) -> datetime:
    """Read one caller-supplied moment, taking a naive one as local time.

    The same rule `_instant` applies to persisted text, so a naive value means the
    same thing however it reached this module.

    Total, unlike the persisted-read door: a caller is reporting a call an upstream
    already billed, and a moment this module cannot carry is a reason to date the row
    by the only instant it can measure, never a reason to lose the row.
    """

    return _carried(moment) or _utc_now()


def local_usage_day(moment: datetime) -> date:
    """Bucket one moment into a local-calendar day.

    Avibe is local-first and the settings page already presents local days, so a
    day boundary here is the user's midnight, not UTC's.
    """

    return _aware(moment).astimezone().date()


def _counter(value: object) -> Optional[int]:
    """Read one persisted counter, or None when its whole row is unreadable.

    Two unusable shapes, two answers, because they cost different things. A value
    that is not a count — not an integer, a bool, negative — carries no magnitude,
    and zero is the one substitute that claims none either: it is the identity of
    every sum this module performs, so the rest of the row still reports what it
    does know.

    A value above `USAGE_COUNTER_CEILING` is the opposite problem. It carries a
    magnitude no published document could carry to its reader, so every substitute
    invents one in its place, and an
    invented counter that large dominates whatever aggregate it enters; saturating
    it would additionally hand a subset and its superset the same number and force
    a cached-input share to exactly 100%. So that row is not read at all. It is
    dropped and counted with every other unusable row, and the next write is what
    takes it out of the file.

    No reading of a real count is refused here: the ceiling is a property of the
    consumer, and `_write` is what keeps this module from ever persisting a counter
    past it.
    """

    if not isinstance(value, int) or isinstance(value, bool):
        return 0
    if value < 0:
        return 0
    if value > USAGE_COUNTER_CEILING:
        return None
    return value


def _text(value: object) -> Optional[str]:
    """Read one persisted text field, dropping anything that is not text.

    No length bound here: the only text fields this reads are a calendar day and
    an instant, and the parsers below already reject anything that is not one.
    The key fields go through the two key functions instead, and which one depends
    on the direction: a live call is keyed by `usage_ledger_key`, which folds a long
    identity rather than refusing it because the hub already served that call, while
    a row read back from the file goes through `persisted_ledger_key`, which
    recognizes the key a previous write derived instead of deriving it again.
    """

    if not isinstance(value, str):
        return None
    return value.strip() or None


def _calendar_day(value: str) -> Optional[date]:
    """Read one ISO calendar day, or None when the text is not a bare date."""

    try:
        return date.fromisoformat(value)
    except ValueError:
        return None


def _instant(value: object) -> Optional[datetime]:
    """Parse one persisted instant, or None when it is not an ISO date-time.

    A bare date is a day, not an instant, and the date-time parser would read one
    as midnight — inventing a time of day this ledger never wrote. The same parser
    that recognizes the ``day`` key decides that here, so the module holds one
    notion of what a day is.

    A naive value is read as local time, the same calendar the day buckets use.

    Parsing is not the whole of reading it: a value `fromisoformat` accepts may still
    be one no conversion can carry, so it goes through `_carried` before it escapes
    and degrades to absent when it cannot make the trip.
    """

    if not isinstance(value, str) or _calendar_day(value) is not None:
        return None
    try:
        parsed = datetime.fromisoformat(value)
    except ValueError:
        return None
    return _carried(parsed)


def _timestamp(value: object) -> Optional[str]:
    """Publish one persisted instant in the one spelling this module writes.

    The read surface promises an RFC 3339 date-time, and `datetime.fromisoformat`
    accepts far more than RFC 3339 describes — naive, space-separated, and, after
    an offset, even seconds. Publishing the file's text, or re-publishing the
    parsed value in the file's own offset, both leave the output shape decided by
    the input: fix one spelling and the next one is still reachable.

    So the file supplies only the instant, and this module supplies the spelling.
    Normalizing to UTC means the offset is `+00:00` by construction, whatever the
    file said, and there is no remaining spelling for a hand-edited value to
    reach. An unparseable value degrades the field to absent.
    """

    parsed = _instant(_text(value))
    if parsed is None:
        return None
    return parsed.isoformat()


def _local(moment: datetime) -> datetime:
    """Return one carried instant in the server's local timezone."""

    return _aware(moment).astimezone()


def _local_midnight(day: date) -> datetime:
    """Construct a server-local midnight for a calendar date."""

    # Start naive so the OS applies the offset and DST rule for this date,
    # rather than borrowing today's offset for a historical calendar day.
    return datetime.combine(day, time.min).astimezone()


def _utc_owner_day_envelope(day: date) -> Optional[tuple[datetime, datetime]]:
    """Return the conservative UTC interval a timezone-less owner day can occupy."""

    try:
        start = (
            datetime.combine(day, time.min).replace(tzinfo=timezone.utc)
            - _OWNER_DAY_MAX_AHEAD
        )
        end = (
            datetime.combine(day + timedelta(days=1), time.min).replace(
                tzinfo=timezone.utc,
            )
            + _OWNER_DAY_MAX_BEHIND
        )
    except (OverflowError, OSError, ValueError):
        return None
    return start, end


def _local_owner_day_window(day: date) -> Optional[tuple[datetime, datetime]]:
    """Return the current-zone interval for an owner day with a usable instant."""

    try:
        return (
            _local_midnight(day).astimezone(timezone.utc),
            _local_midnight(day + timedelta(days=1)).astimezone(timezone.utc),
        )
    except (OverflowError, OSError, ValueError):
        return None


def _intervals_overlap(
    first_start: datetime,
    first_end: datetime,
    second_start: datetime,
    second_end: datetime,
) -> bool:
    """Whether two half-open instant intervals overlap."""

    return first_start < second_end and first_end > second_start


def _has_unresolved_hourly_mass(row: dict) -> bool:
    """Whether daily counters still contain usage no hourly field can locate."""

    hours = row.get("hours")
    expired = row.get("hourly_expired_totals")
    if not isinstance(hours, list):
        return True
    service = _is_service(row)
    if expired is None:
        expired = _empty_totals(service=service)
    elif not isinstance(expired, dict):
        return True
    known = _empty_totals(service=service)
    for item in hours:
        if not isinstance(item, dict):
            return True
        _accumulate(known, item)
    _accumulate(known, expired)
    return any(known[key] != row[key] for key in _counter_keys(row))


def _hourly_owner_day_window(
    row: dict,
) -> Optional[tuple[datetime, datetime]]:
    """Return the window that bounds any hourly uncertainty in a row.

    A timestamp-less daily-only row has no trustworthy hour, so its owner date
    uses the timezone-independent envelope. A merged row with an hour list can
    also contain older residual mass that the later exact timestamp cannot
    locate; that residual uses the same envelope. A timestamp on a daily-only
    legacy row keeps the existing local-day compatibility semantics.
    Fully reconciled rows keep the local-day window for future or otherwise
    incomplete slice validation.
    """

    day = row.get("day")
    if not isinstance(day, str):
        return None
    calendar_day = _calendar_day(day)
    if calendar_day is None:
        return None
    unresolved = _has_unresolved_hourly_mass(row)
    if unresolved and (
        isinstance(row.get("hours"), list)
        or _instant(row.get("last_metered_at")) is None
    ):
        envelope = _utc_owner_day_envelope(calendar_day)
        if envelope is not None:
            return envelope
    return _local_owner_day_window(calendar_day)


def _hourly_uncertainty_was_dropped(
    rows: Sequence[dict],
    held: Sequence[dict],
    start: datetime,
    end: datetime,
) -> bool:
    """Whether capacity removed unresolved mass that overlaps the hourly view."""

    held_keys = {_row_key(row) for row in held}
    for row in rows:
        if _row_key(row) in held_keys or not _has_unresolved_hourly_mass(row):
            continue
        owner_day_window = _hourly_owner_day_window(row)
        if owner_day_window is not None and _intervals_overlap(
            start,
            end,
            *owner_day_window,
        ):
            return True
    return False


def _overlaps_local_day(start: datetime, end: datetime, day: date) -> bool:
    """Whether an instant interval intersects a local calendar day.

    A UTC hour can straddle local midnight in a fractional-offset zone. Its
    contributions then belong to two daily owners, not just the start's date.
    Unrepresentable persisted dates are invalid evidence, never a read failure.
    """

    window = _local_owner_day_window(day)
    return (
        window is not None
        and start < end
        and _intervals_overlap(start, end, *window)
    )


def _hour_start(moment: datetime) -> datetime:
    """Return the UTC boundary of the actual hour containing ``moment``."""

    return _aware(moment).astimezone(timezone.utc).replace(
        minute=0,
        second=0,
        microsecond=0,
    )


def _hour_key(moment: datetime) -> str:
    """Spell one hourly bucket with its canonical UTC boundary."""

    return _hour_start(moment).isoformat(timespec="seconds")


def _hour_key_parts(value: object, *, day: Optional[date] = None) -> Optional[tuple[str, datetime]]:
    """Parse and normalize a persisted hourly key to its UTC boundary."""

    if not isinstance(value, str) or not value.strip() or _calendar_day(value.strip()) is not None:
        return None
    try:
        parsed = datetime.fromisoformat(value.strip())
    except ValueError:
        return None
    try:
        if parsed.tzinfo is None or parsed.utcoffset() is None:
            return None
    except (OverflowError, OSError, ValueError):
        return None
    carried = _carried(parsed)
    if carried is None:
        return None
    if carried.minute or carried.second or carried.microsecond:
        return None
    if day is not None:
        try:
            end = carried + timedelta(hours=1)
        except OverflowError:
            return None
        if not _overlaps_local_day(carried, end, day):
            return None
    return carried.isoformat(timespec="seconds"), carried


def _hour_starts(now: datetime) -> tuple[datetime, ...]:
    """Return the 24 actual consecutive UTC-hour starts ending at ``now``."""

    current = _hour_start(now)
    return tuple(
        current - timedelta(hours=offset)
        for offset in range(USAGE_HOURLY_RETENTION_HOURS - 1, -1, -1)
    )


def _merge_hour_slices(target: dict, incoming: dict) -> None:
    """Merge optional hourly slices without changing the daily aggregate."""

    target_hours = target.get("hours")
    incoming_hours = incoming.get("hours")
    target_complete = target.get("hourly_history_complete") is True
    incoming_complete = incoming.get("hourly_history_complete") is True

    if target_hours is None:
        target["hours"] = None if incoming_hours is None else [dict(item) for item in incoming_hours]
    elif incoming_hours is not None:
        by_key = {item["key"]: item for item in target_hours}
        for item in incoming_hours:
            existing = by_key.get(item["key"])
            if existing is None:
                by_key[item["key"]] = dict(item)
                continue
            _accumulate(existing, item)
            existing["last_metered_at"] = _newer_timestamp(
                existing.get("last_metered_at"),
                item.get("last_metered_at"),
            )
        target["hours"] = sorted(by_key.values(), key=lambda item: item["key"])

    target_expired = target.get("hourly_expired_totals")
    incoming_expired = incoming.get("hourly_expired_totals")
    # A zero aggregate has no expiry interval. Every nonzero contributor must
    # supply its boundary; merging cannot repair an older undated aggregate.
    expiry_bounds = [
        _instant(row.get("hourly_expired_before"))
        for row in (target, incoming)
        if any((row.get("hourly_expired_totals") or {}).values())
    ]
    target["hourly_expired_before"] = (
        max(expiry_bounds).isoformat()
        if expiry_bounds and all(bound is not None for bound in expiry_bounds)
        else None
    )
    if target_expired is None or incoming_expired is None:
        target["hourly_expired_totals"] = None
    else:
        expired = dict(target_expired)
        _accumulate(expired, incoming_expired)
        target["hourly_expired_totals"] = expired
    target["hourly_history_complete"] = target_complete and incoming_complete


def _normalize_hour_slice(item: object, *, service: bool = False) -> Optional[dict]:
    """Normalize one nested hourly slice, or drop it as corrupt."""

    if not isinstance(item, dict):
        return None
    # The UTC key is durable evidence. Re-checking it against the row's local
    # calendar day would reinterpret old data after the host timezone changes and
    # discard a slice that still belongs in the 24-hour projection.
    parts = _hour_key_parts(item.get("key"))
    if parts is None:
        return None
    key, _start = parts
    counters = _normalize_hourly_totals(item, service=service)
    if counters is None:
        return None
    return {
        "key": key,
        **counters,
        "last_metered_at": _timestamp(item.get("last_metered_at")),
    }


def _normalize_hourly_totals(value: object, *, service: bool = False) -> Optional[dict]:
    """Read strict counters for either a retained slice or pruned history."""

    if service:
        return _normalize_service_totals(value)
    if not isinstance(value, dict):
        return None
    if any(key not in value for key in _NEW_CACHE_WRITE_KEYS):
        value = {**_released_cache_write_counters(value), **value}
    totals: dict[str, int] = {}
    for key in _COUNTER_KEYS:
        counter = value.get(key)
        if not isinstance(counter, int) or isinstance(counter, bool) or counter < 0:
            return None
        if counter > USAGE_COUNTER_CEILING:
            return None
        totals[key] = counter
    for subset, superset in _COUNTER_SUBSETS:
        if totals[subset] > totals[superset]:
            return None
    if totals["cache_write_input_tokens"] + totals["cached_input_tokens"] > totals["input_tokens"]:
        return None
    if totals["token_reports"] == 0 and any(
        totals[key] for key in ("input_tokens", "cached_input_tokens", "output_tokens")
    ):
        # Token magnitudes require a report, which in turn requires a request.
        # Reject impossible hourly evidence without altering released daily totals.
        return None
    return totals


def _normalize_row(row: object) -> Optional[dict]:
    """Project one persisted row onto the current shape, or drop it.

    Only ever handed keys, never live identities: a caller metering a call derives
    its key first, so this function asks the one question a row can be asked.
    """

    if not isinstance(row, dict):
        return None
    if "service_id" in row:
        return _normalize_service_row(row)
    day = _text(row.get("day"))
    source_id = persisted_ledger_key(row.get("source_id"))
    model_id = persisted_ledger_key(row.get("model_id"))
    if day is None or source_id is None or model_id is None:
        return None
    # Same rule as the instant above: the file supplies the day, this module
    # supplies its spelling. `date.fromisoformat` also reads `20260818` and
    # `2026-W34-2`, and the window bounds are compared as `YYYY-MM-DD` text — so
    # a row kept in another valid spelling would be silently outside every
    # window it belongs to, without even counting as dropped.
    calendar_day = _calendar_day(day)
    if calendar_day is None:
        return None
    # Part of the row's identity, so a value that is not a tier size leaves no
    # row to fold it into. Absent on every row a released version wrote.
    context_tier = row.get("context_tier", 0)
    if not isinstance(context_tier, int) or isinstance(context_tier, bool) or context_tier < 0:
        return None
    # The same for the processing tier: a call its response said ran at the
    # vendor's fast tier is billed at fast rates (docs/plans/fast-mode.md).
    # Absent on every standard-tier row and on every row a released version wrote,
    # which reads a fast row as a standard one and keeps every count.
    speed = row.get("speed")
    if speed is not None and speed != FAST_SPEED:
        return None
    legacy_cache_write_shape = any(key not in row for key in _NEW_CACHE_WRITE_KEYS)
    if legacy_cache_write_shape:
        row = {**_released_cache_write_counters(row), **row}
    counters: dict[str, int] = {}
    for key in _COUNTER_KEYS:
        counter = _counter(row.get(key))
        if counter is None:
            return None
        counters[key] = counter
    normalized = {
        "day": calendar_day.isoformat(),
        "source_id": source_id,
        "model_id": model_id,
        # Written only past the base tier, so a base row keeps the released shape.
        **({"context_tier": context_tier} if context_tier else {}),
        **({"speed": speed} if speed else {}),
        **counters,
    }
    if legacy_cache_write_shape:
        normalized["_legacy_cache_write_shape"] = True
    if row.get("history_degraded") is True:
        # Internal witness that an earlier read lost evidence. It never crosses
        # the public usage-summary boundary.
        normalized["history_degraded"] = True
    for subset, superset in _COUNTER_SUBSETS:
        normalized[subset] = min(normalized[subset], normalized[superset])
    _repair_cache_writes(normalized)
    normalized["last_metered_at"] = _timestamp(row.get("last_metered_at"))
    return _normalize_hours(normalized, row)


def _normalize_service_row(row: dict) -> Optional[dict]:
    """Project one persisted search-service row, repaired to its relations, or drop it.

    A row naming both a model and a service is neither; the same day, Source and
    key rules as a model row apply otherwise.
    """

    if "model_id" in row:
        return None
    day = _text(row.get("day"))
    source_id = persisted_ledger_key(row.get("source_id"))
    service_id = persisted_ledger_key(row.get("service_id"))
    if day is None or source_id is None or service_id is None or row.get("tool", "web_search") not in ("web_search", "web_fetch"):
        return None
    calendar_day = _calendar_day(day)
    if calendar_day is None:
        return None
    repaired = _repair_service_counters(row)
    if repaired is None:
        return None
    counters, broken = repaired
    normalized = {
        "day": calendar_day.isoformat(),
        "source_id": source_id,
        "service_id": service_id,
        **({"tool": "web_fetch"} if row.get("tool") == "web_fetch" else {}),
        **counters,
        # A broken relation leaves a smaller true statement, so what it values is
        # a floor; so is a request whose billing is unknown.
        "lower_bound": row.get("lower_bound") is True or broken or counters["uncertain_requests"] > 0,
    }
    if row.get("history_degraded") is True:
        normalized["history_degraded"] = True
    normalized["last_metered_at"] = _timestamp(row.get("last_metered_at"))
    # Repaired counts no longer match the slices that were written beside them,
    # so where in the day the remainder fell is unknown.
    return _normalize_hours(normalized, row, invalid=broken)


def _normalize_hours(normalized: dict, row: dict, *, invalid: bool = False) -> dict:
    """Attach a normalized row's optional hourly history, checked against its daily counts."""

    service = _is_service(normalized)
    counters = {key: normalized[key] for key in _counter_keys(normalized)}
    raw_hours = row.get("hours")
    if isinstance(raw_hours, list):
        # An empty list with the producer's complete flag can be ordinary
        # retention expiry: the daily owner outlives its 24-hour slices.
        by_key: dict[str, dict] = {}
        invalid_hour = invalid
        duplicate_hour = False
        over_capacity = False
        for item in raw_hours:
            normalized_hour = _normalize_hour_slice(item, service=service)
            if normalized_hour is None:
                invalid_hour = True
                continue
            key = normalized_hour["key"]
            if key in by_key:
                # A duplicate key is ambiguous persisted evidence. Discard the
                # nested history immediately so an unbounded duplicate list
                # cannot consume memory or be counted repeatedly.
                duplicate_hour = True
                invalid_hour = True
                break
            by_key[key] = normalized_hour
            if len(by_key) > USAGE_HOURLY_RETENTION_HOURS:
                # A corrupt file can contain arbitrarily many unique slices.
                # Once the explicit capacity is exceeded, no temporal subset is
                # authoritative enough to repair, so stop retaining them.
                over_capacity = True
                invalid_hour = True
                break
        if duplicate_hour or over_capacity:
            hours = []
        else:
            hours = list(by_key.values())
        nested_totals = _empty_totals(service=service)
        for item in hours:
            _accumulate(nested_totals, item)
        if any(nested_totals[key] > counters[key] for key in counters):
            # The daily row is the authoritative released aggregate. A nested
            # corruption that exceeds it cannot be repaired by clamping one
            # arbitrary hour, so discard all hourly slices and report the
            # temporal history as unavailable.
            hours = []
            invalid_hour = True
        normalized["hours"] = sorted(hours, key=lambda item: item["key"])
        normalized["hourly_history_complete"] = (
            row.get("hourly_history_complete") is True and not invalid_hour
        )
        normalized["hourly_expired_totals"] = _normalize_hourly_totals(
            row.get("hourly_expired_totals"), service=service
        )
        expiry = _hour_key_parts(row.get("hourly_expired_before"))
        normalized["hourly_expired_before"] = expiry[0] if expiry else None
        if (
            row.get("hourly_expired_totals") is not None
            and normalized["hourly_expired_totals"] is None
        ):
            normalized["hourly_history_complete"] = False
    else:
        # Released files have no hourly field. Their daily totals remain valid,
        # but no hour may be invented from the daily row's last timestamp.
        normalized["hours"] = None
        normalized["hourly_history_complete"] = False
        normalized["hourly_expired_totals"] = None
        normalized["hourly_expired_before"] = None
    return normalized


@dataclass(frozen=True)
class SourceIdentity:
    """One Source as config holds it, for joining onto the rows metered under it.

    Nested rather than two flat maps, because a metered model's identity *is* the
    pair: the same common model ID lives on several Sources, and a flat model map
    cannot say which one it came from — it answered for a model removed from Source A
    as long as Source B still listed it, labelling a retained row as though the
    identity were still there. Arity is the whole of that defect, so it is carried in
    the type rather than in a rule each caller has to remember.

    `label` is the only genuinely caller-owned text here. A model's label is its own
    identity, which the ledger can derive, and which matters only because a folded row
    publishes a key rather than the identifier the user typed.
    """

    source_id: str
    label: Optional[str] = None
    model_ids: Sequence[str] = ()


@dataclass(frozen=True)
class ServiceCatalogEntry:
    """One search service as the current catalog lists it, for labelling and valuing its rows.

    Caller-owned for the same reason a Source's label is: the catalog belongs to the
    release, not to the file, so a row a later catalog no longer lists stays
    readable, with no label and no price, instead of being priced from a stored copy.
    """

    service_id: str
    label: str
    usd_per_1k: float
    # The date the catalog's price was verified (`price.as_of`).
    price_date: str


def _keyed_catalog(catalog: Optional[Sequence[ServiceCatalogEntry]]) -> dict[str, ServiceCatalogEntry]:
    """Key the catalog the way the service rows it values were keyed."""

    keyed: dict[str, ServiceCatalogEntry] = {}
    for entry in catalog or ():
        key = usage_ledger_key(entry.service_id)
        if key is not None:
            keyed[key] = entry
    return keyed


def _service_price(row: Mapping, keyed_catalog: Mapping[str, ServiceCatalogEntry]) -> Optional[float]:
    entry = keyed_catalog.get(row["service_id"])
    return entry.usd_per_1k if entry is not None else None


def _service_fields(target: dict, cost: Optional[ServiceCost], *, degraded: bool) -> dict:
    """A service aggregate's counts, plus its cost fields exactly when the report is priced.

    The ledger keeps the reported amount regardless; publishing it is the cost
    presence rule's, so an unpriced report carries counts only.
    """

    published = {key: value for key, value in target.items() if key != "reported_usd"}
    if cost is None:
        return published
    fields = cost.fields()
    # A degraded read lost rows of unknown size, so no figure from it is exact.
    fields["api_cost_lower_bound"] = fields["api_cost_lower_bound"] or degraded
    return {**published, **fields}


@dataclass(frozen=True)
class _LedgerRead:
    rows: list[dict]
    degraded: bool


def _keyed_identities(
    identities: Optional[Sequence[SourceIdentity]],
) -> tuple[dict[str, Optional[str]], dict[tuple[str, str], str]]:
    """Key what config holds the way the rows it will join were keyed.

    The read half of the rule `record_many` applies on the write half: an identity a
    caller holds is not the key its rows carry, and a long one differs from it
    entirely. Deriving here rather than in the caller is what keeps a join from
    silently missing exactly the identities the fold exists for — and keying both
    levels here is what keeps a model's join inside its own Source.

    Two identities that key the same are the same identity spelled differently —
    `usage_ledger_key` is injective over anything else — so the later one wins, as it
    would in the mapping a caller built.
    """

    sources: dict[str, Optional[str]] = {}
    models: dict[tuple[str, str], str] = {}
    for identity in identities or ():
        source_key = usage_ledger_key(identity.source_id)
        if source_key is None:
            continue
        sources[source_key] = identity.label
        for model_id in identity.model_ids:
            model_key = usage_ledger_key(model_id)
            if model_key is not None:
                models[(source_key, model_key)] = model_id
    return sources, models


def _row_price(
    row: Mapping, prices: PriceTable, keyed_model_labels: Mapping[tuple[str, str], str]
) -> Optional[ModelPrice]:
    """Price a row by the identity it was metered under; a folded key is not a model name."""

    return prices.price(keyed_model_labels.get((row["source_id"], row["model_id"]), row["model_id"]))


def _bucket_rows(
    rows: Sequence[dict],
    prices: Optional[PriceTable],
    keyed_model_labels: Mapping[tuple[str, str], str],
    *,
    keyed_catalog: Optional[Mapping[str, ServiceCatalogEntry]] = None,
    degraded: bool = False,
) -> list[dict]:
    """Publish one bucket's rows: one per Source and model, summed across context tiers.

    Each row is valued at its own tier before the sum, because a tier's rates
    apply only to the calls recorded in it. Search-service rows follow, one per
    Source and service, in their own `kind: "service"` shape: a service is never
    dressed as a model.
    """

    published: dict[tuple[str, str], dict] = {}
    costs: dict[tuple[str, str], Cost] = {}
    services: dict[tuple[str, str, str], dict] = {}
    service_costs: dict[tuple[str, str, str], ServiceCost] = {}
    for row in rows:
        if _is_service(row):
            key = (row["source_id"], row["service_id"], row.get("tool", "web_search"))
            target = services.setdefault(
                key,
                {
                    "kind": "service",
                    "source_id": row["source_id"],
                    "service_id": row["service_id"],
                    **({"tool": row["tool"]} if "tool" in row else {}),
                    **_empty_totals(service=True),
                },
            )
            _accumulate(target, row)
            if prices is not None:
                service_costs.setdefault(key, ServiceCost()).add(
                    service_row_cost(row, _service_price(row, keyed_catalog or {}))
                )
            continue
        key = (row["source_id"], row["model_id"])
        target = published.setdefault(
            key, {"source_id": row["source_id"], "model_id": row["model_id"], **_empty_totals()}
        )
        _accumulate(target, row)
        _count_fast(target, row)
        if prices is not None:
            costs.setdefault(key, Cost()).add(row_cost(row, _row_price(row, prices, keyed_model_labels)))
    service_rows = [
        _service_fields(services[key], service_costs.get(key), degraded=degraded)
        for key in sorted(services)
    ]
    if prices is None:
        return [published[key] for key in sorted(published)] + service_rows
    priced = []
    for key in sorted(published):
        cost = costs[key]
        # A degraded read lost rows of unknown size, so no figure from it is exact.
        cost.api_cost_lower_bound = cost.api_cost_lower_bound or degraded
        priced.append({**published[key], **cost.fields()})
    return priced + service_rows


def _recency(
    row: dict, ceiling: datetime, *, utc_owner_days: bool = False,
) -> tuple[str, datetime]:
    """Order rows oldest-metered first, so the bound evicts what costs least.

    Ordering by key instead would evict by spelling: an early-sorting model would
    be recreated and evicted again on every write while later-sorting stale rows
    survived, so its usage could never accumulate. Instants are compared as points
    in time — text order is not time order once two rows carry different offsets.

    An instant later than `ceiling` is not a recency at all. Nothing was metered
    after the reading the caller just took, so such a row is not the set's most
    recently used one — it is one whose recency cannot be read, which is the case
    `_OLDEST_INSTANT` already answers for a row that recorded no instant. It gets
    the same answer, and not because a corrupt row deserves to lose: a row this
    ordering cannot place is the only row it can evict without discarding usage it
    can account for.

    Bounding it to `ceiling` instead is not enough, which is worth stating because
    it is the obvious remedy. That makes the row as recent as the reading, so it
    still outranks every row metered before it and still evicts real usage; it
    narrows the lie without changing who pays for it. Placing the bound here at all
    — rather than leaving it to whoever assembled the rows — is what stops this from
    recurring: `_retained` keeps a future instant out of the file, and each time
    that was the only place it happened, the report path ordered by the raw value.

    Daily reports retain the existing owner-day ordering. Write retention and
    hourly reads also admit old-zone owners through UTC evidence, so their recency
    calendar label must follow that evidence in the current zone. An owner with
    neither a usable instant nor a usable hourly key is admitted by its
    plausibility envelope but ranks below every known instant, so uncertainty
    cannot evict a newer call. This changes eviction only, never persisted daily
    ownership.
    """

    metered = _instant(row["last_metered_at"])
    if utc_owner_days:
        evidence = [metered] if metered is not None and metered <= ceiling else []
        for item in row.get("hours") or ():
            parts = _hour_key_parts(item.get("key"))
            if parts is None or parts[1] > ceiling:
                continue
            evidence.append(parts[1])
            latest = _instant(item.get("last_metered_at"))
            if latest is not None and latest <= ceiling:
                evidence.append(latest)
        if evidence:
            latest = max(evidence)
            return (local_usage_day(latest).isoformat(), latest)
        if metered is None or metered > ceiling:
            return ("", _OLDEST_INSTANT)
    if metered is None or metered > ceiling:
        return (row["day"], _OLDEST_INSTANT)
    return (row["day"], metered)


def _persistable_row(row: dict) -> dict:
    """Keep released cache-write rows byte-compatible until a new write touches them."""

    if not row.get("_legacy_cache_write_shape"):
        return row

    def strip(value: object) -> object:
        if isinstance(value, list):
            return [strip(item) for item in value]
        if not isinstance(value, dict):
            return value
        return {
            key: strip(item)
            for key, item in value.items()
            if key not in _NEW_CACHE_WRITE_KEYS and key != "_legacy_cache_write_shape"
        }

    return strip(row)
def _cache_write_increment(usage: Optional[ProtocolUsageReport]) -> dict[str, int]:
    return {
        "cache_write_input_tokens": usage.cache_write_input_tokens if usage else 0,
        "cache_write_1h_input_tokens": usage.cache_write_1h_input_tokens if usage else 0,
        "cache_write_30m_input_tokens": usage.cache_write_30m_input_tokens if usage else 0,
        "cache_write_12h_input_tokens": usage.cache_write_12h_input_tokens if usage else 0,
        "cache_write_uncaptured_reports": 0,
        "cache_write_ttl_uncaptured_reports": usage.cache_write_ttl_uncaptured_reports if usage else 0,
    }


def _row_key(row: dict) -> tuple[str, str, str, int, bool, str, str]:
    """A row is one day of one model on one Source, in one context tier and processing tier.

    Or of one hosted service/tool on one Source. The boolean keeps model and
    service identities apart; the tool keeps shared Exa search/fetch attribution
    separate without changing its service/cooldown identity. The last member is a
    model row's speed (`"fast"`, else empty), so calls billed at fast rates never
    share a row with standard ones.
    """

    if _is_service(row):
        return (row["day"], row["source_id"], row["service_id"], 0, True, row.get("tool", "web_search"), "")
    return (row["day"], row["source_id"], row["model_id"], row.get("context_tier", 0), False, "", row.get("speed", ""))


def _empty_totals(*, service: bool = False) -> dict:
    if service:
        return {**{key: 0 for key in _SERVICE_COUNTER_KEYS}, "reported_usd": 0.0}
    return {key: 0 for key in _COUNTER_KEYS}


def _count_fast(target: dict, row: dict) -> None:
    """Publish how many of an aggregate's requests ran at the vendor's fast tier.

    Present only once a fast row joins, so an aggregate without one keeps the
    released shape. Read-side only: no persisted row carries it.
    """

    if row.get("speed") == FAST_SPEED:
        target["fast_requests"] = target.get("fast_requests", 0) + row["requests"]


def _accumulate(target: dict, row: dict) -> None:
    """Add one row's counters into an aggregate, exactly.

    Nothing is clamped here, and that absence is the point. Every addend already
    passed the door that bounds it — a live report at
    `stream_wire.USAGE_REPORT_TOKEN_CEILING`, a persisted one at
    `USAGE_COUNTER_CEILING` — so a ceiling on the sum would no longer protect the
    aggregate from a hostile upstream. It would cap how much usage the user is
    allowed to have had, and it read as exactly that: a truncated total is
    indistinguishable from a real one, and clamping a subset and its superset
    independently drove every cached-input share to exactly 100% once either
    saturated, which presents a broken number as perfect caching.

    Two callers persist what they accumulate — the duplicate-key merge in `_read`
    and the fold in `record_many` — and neither makes a bound belong here. A sum
    that outgrows the file is a fact about the file, so `_write` answers it, for
    every writer at once and without a published aggregate ever being quietly
    reduced to keep a row writable.
    """

    for key in _counter_keys(target):
        target[key] += row[key]
    if _is_service(target):
        # A float sum, so it is bounded by its count rather than compared exactly;
        # see `_service_relations_hold`.
        target["reported_usd"] += row["reported_usd"]
        if "lower_bound" in target:
            target["lower_bound"] = target["lower_bound"] or row.get("lower_bound") is True


def _newer_timestamp(current: Optional[str], candidate: Optional[str]) -> Optional[str]:
    """Keep the later of two instants, comparing points in time.

    Text order is not time order once two rows carry different UTC offsets, which
    a state file merged across machines or an older release can hold.
    """

    if candidate is None:
        return current
    if current is None:
        return candidate
    current_instant = _instant(current)
    candidate_instant = _instant(candidate)
    if current_instant is None:
        return candidate
    if candidate_instant is None:
        return current
    return candidate if candidate_instant > current_instant else current


def _retain_hour_slices(row: dict, measured: datetime) -> dict:
    """Keep only the recent, usable hour slices needed by the 24-hour report."""

    hours = row.get("hours")
    if hours is None:
        return row

    current_start = _hour_start(measured).astimezone(timezone.utc)
    oldest_start = current_start - timedelta(hours=USAGE_HOURLY_RETENTION_HOURS - 1)
    retained: dict[str, dict] = {}
    expired = row.get("hourly_expired_totals")
    expired_known = isinstance(expired, dict)
    if expired_known:
        expired = dict(expired)
    expired_before = _instant(row.get("hourly_expired_before"))
    boundary_known = expired_before is not None or not any((expired or {}).values())
    incomplete = row.get("hourly_history_complete") is not True
    if any((expired or {}).values()) and (
        expired_before is None or oldest_start < expired_before
    ):
        # Counts alone prove reconciliation, not that pruned hours remain
        # outside this report. A clock rollback can bring them back in range.
        incomplete = True
    for item in hours:
        parts = _hour_key_parts(item.get("key"))
        if parts is None:
            incomplete = True
            continue
        key, start = parts
        instant = start.astimezone(timezone.utc)
        metered = _instant(item.get("last_metered_at"))
        if instant > measured or (metered is not None and metered > measured):
            # Future slices cannot be evidence of usage and must not occupy the
            # bounded recent history. A current-hour aggregate may also contain
            # future-stamped calls; its counters cannot be split retrospectively.
            incomplete = True
            continue
        if instant < oldest_start:
            # Keep the expired portion of the daily aggregate separate so a
            # partial oldest local day can distinguish ordinary retention from a
            # missing in-horizon slice.
            if expired_known:
                _accumulate(expired, item)
                if boundary_known:
                    end = instant + timedelta(hours=1)
                    expired_before = max(expired_before, end) if expired_before else end
            continue
        retained[key] = {**item, "key": key}
    latest_metered = _instant(row.get("last_metered_at"))
    if (
        not incomplete
        and row.get("requests", 0) > 0
        and latest_metered is not None
        and oldest_start <= latest_metered <= measured
        and _hour_key(latest_metered) not in retained
    ):
        logger.warning(
            "Model Hub usage ledger row %s has complete hourly history without "
            "its latest metered hour; publishing it as incomplete",
            _row_key(row),
        )
        incomplete = True
    row_day_overlaps = False
    if not incomplete:
        owner_day_window = _hourly_owner_day_window(row)
        if owner_day_window is not None:
            row_day_overlaps = _intervals_overlap(
                oldest_start,
                measured,
                *owner_day_window,
            )
    if not incomplete and (row_day_overlaps or retained or expired_known):
        # Compare durable UTC slices even when a host timezone change makes the
        # persisted local owner day fall outside the current display grid.
        nested_totals = _empty_totals(service=_is_service(row))
        for item in retained.values():
            _accumulate(nested_totals, item)
        if expired_known:
            _accumulate(nested_totals, expired)
        # Counts reconcile exactly; a reported amount is a float sum whose order
        # differs between the day and its hours, so it is bounded, never compared.
        if not expired_known or any(
            nested_totals[key] != row[key] for key in _counter_keys(row)
        ):
            logger.warning(
                "Model Hub usage ledger row %s cannot reconcile its complete "
                "in-horizon day with retained and expired hourly counters; "
                "publishing it as incomplete",
                _row_key(row),
            )
            incomplete = True
    if len(retained) > USAGE_HOURLY_RETENTION_HOURS:
        retained = dict(
            sorted(
                retained.items(),
                key=lambda item: _hour_key_parts(item[0])[1],
            )[-USAGE_HOURLY_RETENTION_HOURS:]
        )
        incomplete = True
    return {
        **row,
        "hours": sorted(retained.values(), key=lambda item: item["key"]),
        "hourly_expired_totals": expired if expired_known else None,
        "hourly_expired_before": expired_before.isoformat() if expired_before else None,
        "hourly_history_complete": not incomplete,
    }


def _has_recent_hourly_evidence(row: dict, start: datetime, end: datetime) -> bool:
    """Use the same durable UTC evidence for write retention and hourly reads."""

    latest = _instant(row.get("last_metered_at"))
    if latest is not None and start <= latest <= end:
        return True
    for item in row.get("hours") or ():
        parts = _hour_key_parts(item.get("key"))
        if parts is not None and start <= parts[1] <= end:
            # Selection preserves evidence of uncertainty too. The shared
            # retention validator rejects future-stamped counters afterward.
            return True
    return False


def _model_increment(
    call: "UsageCall", source_key: str, *, owner_day: str, metered_at: datetime
) -> Optional[dict]:
    """One metered model call as an unnormalized row, or None when its model has no key."""

    model_key = usage_ledger_key(call.model_id)
    if model_key is None:
        return None
    usage = call.usage
    return {
        "day": owner_day,
        "source_id": source_key,
        "model_id": model_key,
        "context_tier": call.context_tier,
        **({"speed": call.speed} if call.speed else {}),
        "requests": call.requests,
        "token_reports": call.requests if usage is not None else 0,
        "input_tokens": usage.input_tokens if usage else 0,
        "cached_input_tokens": usage.cached_input_tokens if usage else 0,
        "output_tokens": usage.output_tokens if usage else 0,
        **_cache_write_increment(usage),
        "last_metered_at": metered_at.isoformat(),
        "hours": [
            {
                "key": _hour_key(metered_at),
                "requests": call.requests,
                "token_reports": call.requests if usage is not None else 0,
                "input_tokens": usage.input_tokens if usage else 0,
                "cached_input_tokens": usage.cached_input_tokens if usage else 0,
                "output_tokens": usage.output_tokens if usage else 0,
                **_cache_write_increment(usage),
                "last_metered_at": metered_at.isoformat(),
            }
        ],
        "hourly_expired_totals": _empty_totals(),
        "hourly_history_complete": True,
    }


def _service_increment(
    call: "ServiceUsageCall", source_key: str, *, owner_day: str, metered_at: datetime
) -> Optional[dict]:
    """Sent search requests as an unnormalized row, or None when the service has no key.

    It meets the same repair a persisted row does, so a call whose counts break a
    relation is recorded as the floor that repair leaves, never as written.
    """

    service_key = usage_ledger_key(call.service_id)
    if service_key is None:
        return None
    counts = {
        "requests": call.requests,
        "billable_requests": call.billable_requests,
        "uncertain_requests": call.uncertain_requests,
        "reported_requests": call.reported_requests,
        "reported_usd": call.reported_usd,
    }
    return {
        "day": owner_day,
        "source_id": source_key,
        "service_id": service_key,
        **({"tool": call.tool} if call.tool == "web_fetch" else {}),
        **counts,
        "last_metered_at": metered_at.isoformat(),
        "hours": [
            {"key": _hour_key(metered_at), **counts, "last_metered_at": metered_at.isoformat()}
        ],
        "hourly_expired_totals": _empty_totals(service=True),
        "hourly_history_complete": True,
    }


class BoundedUsageLedger:
    """Persist metered upstream-call token counts as a bounded daily aggregate."""

    def __init__(
        self,
        path: Path,
        *,
        max_rows: int = USAGE_MAX_ROWS,
        retention_days: int = USAGE_RETENTION_DAYS,
        now: Callable[[], datetime] = _utc_now,
    ):
        self.path = path
        self.max_rows = max_rows
        self.retention_days = retention_days
        # Not a second clock: hub callers pass their own so a fixed service clock
        # still decides every day this ledger writes. What matters is that this one
        # is *read* where the write happens rather than handed in from a call.
        self._now = now
        self._lock = threading.RLock()

    def _within_capacity(
        self, rows: list[dict], *, measured: datetime, utc_owner_days: bool = False,
    ) -> list[dict]:
        """Bound an already-reportable set using its admission policy.

        Daily readers preserve owner-day ordering. Writes and hourly readers use
        UTC evidence for recency too, so a future old-zone owner cannot displace a
        newer call merely because its original calendar label sorts later.
        """

        if len(rows) <= self.max_rows:
            return rows
        return sorted(
            rows,
            key=lambda row: _recency(row, measured, utc_owner_days=utc_owner_days),
        )[-self.max_rows :]

    def _read(self) -> _LedgerRead:
        if not self.path.exists():
            return _LedgerRead([], False)
        # Degrading to empty keeps a broken optional-feature file from failing
        # startup, but the next write replaces that file — so this is the last
        # moment its history is recoverable, and saying nothing would erase it
        # silently.
        # Every way decoding can fail, by category rather than by the shapes a
        # particular file happens to hold: invalid UTF-8 and an integer past the
        # digit limit both raise plain `ValueError`, deep nesting raises
        # `RecursionError`, and either one escaping here would take down the read
        # route and then stop metering entirely — the loud-failure outcome this
        # degradation exists to prevent.
        try:
            payload = json.loads(self.path.read_text(encoding="utf-8"))
        except (OSError, ValueError, RecursionError) as exc:
            logger.warning("Model Hub usage ledger %s is unreadable: %s", self.path, exc)
            return _LedgerRead([], True)
        if not isinstance(payload, list):
            logger.warning("Model Hub usage ledger %s is not a list of rows", self.path)
            return _LedgerRead([], True)
        rows: dict[tuple[str, str, str, int], dict] = {}
        dropped = 0
        for item in payload:
            row = _normalize_row(item)
            if row is None:
                dropped += 1
                continue
            existing = rows.get(_row_key(row))
            if existing is None:
                rows[_row_key(row)] = row
                continue
            if existing.get("_legacy_cache_write_shape") and not row.get("_legacy_cache_write_shape"):
                existing.pop("_legacy_cache_write_shape", None)
            _accumulate(existing, row)
            existing["last_metered_at"] = _newer_timestamp(
                existing["last_metered_at"],
                row["last_metered_at"],
            )
            if row.get("history_degraded") is True:
                existing["history_degraded"] = True
            _merge_hour_slices(existing, row)
        if dropped:
            logger.warning(
                "Model Hub usage ledger %s dropped %d unusable row(s)", self.path, dropped
            )
        # A merge builds a counter no row in the file carried. `_counter` cleared
        # each addend, and their sum can still land past the ceiling — only a
        # file holding duplicate keys reaches this, which is one this ledger never
        # wrote. Same answer as every other door, for the same reason: a row whose
        # magnitude no published document could carry is dropped, not saturated.
        held = []
        degraded = dropped > 0 or any(
            row.get("history_degraded") is True for row in rows.values()
        )
        for row in rows.values():
            if all(row[key] <= USAGE_COUNTER_CEILING for key in _counter_keys(row)):
                held.append(row)
                continue
            degraded = True
            logger.warning(
                "Model Hub usage ledger %s dropped row %s: merged counters outgrew "
                "what the file can carry",
                self.path,
                _row_key(row),
            )
        return _LedgerRead(sorted(held, key=_row_key), degraded)

    def _write(self, rows: list[dict], *, measured: datetime) -> None:
        """Persist the rows the file can hold, at both of the capacities it has.

        `max_rows` is one, and `_within_capacity` is where it is applied for this
        door and for `window` together — holding it here alone is what let a larger
        file be read back and published whole. The other capacity is what a stored
        counter may be without putting the published document out of the
        range its reader holds exactly, and it needs the same door for a reason
        the exact merges upstream make unavoidable: folding an increment onto a row
        near the ceiling, or merging two duplicate-keyed rows a corrupt file holds,
        can produce a counter past it. Persisting that would put the loss somewhere
        worse than here — the next read would find a row it cannot use, and the
        usage would go missing with nothing having said so.

        So the row stops where it stops fitting, once, with its identity in the
        log. That is also what heals the file: a bucket whose stored counters were
        never real disappears instead of saturating, and the next call recorded
        against it starts a row that means what it says.
        """

        holdable: list[dict] = []
        for row in rows:
            if all(row[key] <= USAGE_COUNTER_CEILING for key in _counter_keys(row)):
                holdable.append(_retain_hour_slices(row, measured))
                continue
            logger.warning(
                "Model Hub usage ledger %s dropped row %s: counters outgrew what the "
                "file can carry",
                self.path,
                _row_key(row),
            )
        retained = self._within_capacity(holdable, measured=measured, utc_owner_days=True)
        if _hourly_uncertainty_was_dropped(
            holdable,
            retained,
            _hour_starts(measured)[0],
            measured,
        ):
            # Capacity may evict an uncertain owner in favor of a known newer
            # call. Preserve the existing degradation witness on a survivor so
            # reopening cannot publish the retained row as complete history.
            retained = [{**row, "history_degraded": True} for row in retained]
        write_state_document(self.path, [_persistable_row(row) for row in sorted(retained, key=_row_key)])

    def record(
        self,
        *,
        source_id: str,
        model_id: str,
        usage: Optional[ProtocolUsageReport],
        at: datetime,
        speed: Optional[str] = None,
    ) -> None:
        """Fold one metered upstream call into its day's row.

        The new row goes through the same normalization as a persisted one, so
        what this module writes and what it reads can never disagree about a
        bound, a subset, or an identity.

        `at` is a value the caller captured when its call ended; the horizon and
        the ceiling below are a reading taken where the write happens. The two are
        not interchangeable, because metering runs off the event loop and reaches
        this lock in whatever order the executor ran it: a call stamped just before
        local midnight can persist after one stamped just after it. Handing the
        captured value to a ledger-wide bound would then date the newer row into
        the future and drop it. So `at` decides this call's own bucket and stamp
        and nothing else — up to the moment it is persisted, since nothing is
        metered later than the write that records it.
        """

        self.record_many((UsageCall(source_id=source_id, model_id=model_id, usage=usage, at=at, speed=speed),))

    def record_many(self, calls: Sequence["UsageCall | ServiceUsageCall"]) -> None:
        """Fold a batch of metered calls into their days' rows in one transaction.

        The batch is the reason a backlog cannot grow: one read, one retention
        pass and one fsync serve however many calls arrived while the previous
        batch was being written, so the queue drains in bursts of whatever size
        it reached rather than one durable write at a time. Folding is what makes
        that safe — every call still lands in its own day's row with its own
        stamp, so a batch of ten is arithmetically the ten separate writes it
        replaces, never a summary of them. A call that already speaks for several
        (`UsageCall.requests`) folds by the same arithmetic onto the same row those
        calls would each have reached.

        One `persisted_at` reading covers the batch, for the same reason `record`
        takes its own: it is the moment these calls reached the disk, and none of
        them was metered later than that.
        """

        if not calls:
            return
        with self._lock:
            persisted_at = _aware(self._now())
            read = self._read()
            rows = {_row_key(row): row for row in read.rows}
            if read.degraded:
                # A successful write must not turn damaged history into a
                # complete report. Carry the uncertainty forward in the
                # surviving/new rows until the ledger is explicitly repaired.
                for row in rows.values():
                    row["history_degraded"] = True
            folded = False
            for call in calls:
                metered_at = min(_aware(call.at), persisted_at)
                # Ordinary delayed writes retain the date captured before queueing,
                # even if the host zone changes. Future calls still clamp to the
                # persistence clock, including its local date.
                owner_day = (
                    local_usage_day(persisted_at).isoformat()
                    if _aware(call.at) > persisted_at
                    else call.owner_day
                )
                # A live call carries identities, not keys, and this is the one place
                # they become one: derive here and every row downstream — new,
                # accumulated, or read back next restart — is already a key, so no
                # later surface can re-derive one that was folded and orphan its row.
                source_key = usage_ledger_key(call.source_id)
                increment = None
                if source_key is not None:
                    increment = _normalize_row(
                        _service_increment(call, source_key, owner_day=owner_day, metered_at=metered_at)
                        if isinstance(call, ServiceUsageCall)
                        else _model_increment(call, source_key, owner_day=owner_day, metered_at=metered_at)
                    )
                if increment is None:
                    # Reachable only for a value no config can hold — not text, or
                    # empty — because a long identity is folded to a bounded key
                    # rather than refused. Loud rather than silent so a caller that
                    # invents an identifier shows up as lost metering instead of as
                    # a quietly incomplete tab.
                    logger.warning(
                        "Model Hub usage metering skipped a call with an unusable identifier",
                        extra={"source_id_usable": source_key is not None},
                    )
                    continue
                if read.degraded:
                    increment["history_degraded"] = True
                existing = rows.get(_row_key(increment))
                if existing is None:
                    rows[_row_key(increment)] = increment
                else:
                    if existing.get("_legacy_cache_write_shape"):
                        existing.pop("_legacy_cache_write_shape", None)
                    _accumulate(existing, increment)
                    existing["last_metered_at"] = _newer_timestamp(
                        existing["last_metered_at"],
                        increment["last_metered_at"],
                    )
                    _merge_hour_slices(existing, increment)
                folded = True
            if not folded:
                return
            retained = self._retained(list(rows.values()), persisted_at)
            self._write(retained, measured=persisted_at)

    def _retained(self, rows: list[dict], measured: datetime) -> list[dict]:
        """Keep daily owners or real evidence needed by the hourly reader.

        A host timezone change can move an old local owner outside the current
        daily window without aging its UTC evidence. Retain that owner unchanged;
        only daily reports use its calendar label. Future timestamps alone never
        rescue an otherwise unreportable row or consume the bounded capacity.
        `measured` is read at the write, never handed in from a call — see `record`.
        """

        today = local_usage_day(measured)
        oldest = (today - timedelta(days=self.retention_days - 1)).isoformat()
        newest = today.isoformat()
        ceiling = _aware(measured).isoformat()
        hourly_start = _hour_starts(measured)[0]
        placed = []
        for row in rows:
            has_recent_hourly_evidence = _has_recent_hourly_evidence(
                row, hourly_start, measured,
            )
            owner_day_window = _hourly_owner_day_window(row)
            has_plausible_hourly_evidence = (
                owner_day_window is not None
                and _intervals_overlap(
                    hourly_start,
                    measured,
                    *owner_day_window,
                )
            )
            if (
                not oldest <= row["day"] <= newest
                and not has_recent_hourly_evidence
                and not has_plausible_hourly_evidence
            ):
                continue
            metered = _instant(row["last_metered_at"])
            if metered is not None and metered > measured:
                row = {**row, "last_metered_at": ceiling}
            placed.append(_retain_hour_slices(row, measured))
        return placed

    def _window_rows(self, *, days: int, now: datetime) -> tuple[list[dict], bool]:
        """Return reportable rows and whether reading the ledger degraded."""

        bounded_days = max(1, min(int(days), self.retention_days))
        today = local_usage_day(now)
        first_day = (today - timedelta(days=bounded_days - 1)).isoformat()
        last_day = today.isoformat()
        with self._lock:
            read = self._read()
        rows = read.rows
        placed = [row for row in rows if first_day <= row["day"] <= last_day]
        # The reportable set is formed here, so this is where the file's row
        # capacity bounds what `summary` can publish — after the date filter, never
        # before it. Every row left is one a reader can see, which is the condition
        # `_within_capacity` needs to evict the right one.
        #
        # `_write` never emits more rows than the file holds, so reaching this at
        # all means reading a file this ledger did not write. That is worth saying
        # once, unlike eviction on the write path, which is retention working as
        # designed and happens constantly.
        held = self._within_capacity(placed, measured=_aware(now))
        if len(held) < len(placed):
            logger.warning(
                "Model Hub usage ledger %s held %d reportable row(s) over its "
                "capacity of %d; dropped the least recently metered",
                self.path,
                len(placed) - len(held),
                self.max_rows,
            )
        return sorted(held, key=_row_key), read.degraded

    def window(self, *, days: int, now: datetime) -> list[dict]:
        """Return the rows inside the trailing local-day window, oldest first."""

        rows, _degraded = self._window_rows(days=days, now=now)
        return rows

    def daily_costs(
        self,
        *,
        days: int,
        now: datetime,
        identities: Sequence[SourceIdentity],
        prices: PriceTable,
    ) -> dict[str, dict[str, Cost]]:
        """Price the window's rows per Source and local day, keyed by config Source ID.

        The quota page compares spans the ledger does not know (a billing cycle, a
        week), so it gets days to sum rather than one total. Only the given
        identities are answered for; a row keyed to anything else is not theirs.
        """

        keyed_sources = {
            key: identity.source_id
            for identity in identities
            if (key := usage_ledger_key(identity.source_id)) is not None
        }
        _labels, keyed_model_labels = _keyed_identities(identities)
        rows, degraded = self._window_rows(days=days, now=now)
        costs: dict[str, dict[str, Cost]] = {}
        for row in rows:
            source_id = keyed_sources.get(row["source_id"])
            if source_id is None or _is_service(row):
                # A search request is not a model call a plan's quota covers.
                continue
            cost = row_cost(row, _row_price(row, prices, keyed_model_labels))
            costs.setdefault(source_id, {}).setdefault(row["day"], Cost()).add(cost)
        if degraded:
            # Rows of unknown size were lost, so every Source's value is a floor:
            # today lies in every span the quota page sums.
            today = local_usage_day(now).isoformat()
            for source_id in keyed_sources.values():
                costs.setdefault(source_id, {}).setdefault(today, Cost()).add(Cost(api_cost_lower_bound=True))
        return costs

    def _hourly_rows(
        self,
        *,
        starts: Sequence[datetime],
        now: datetime,
    ) -> tuple[list[dict], bool]:
        """Select hourly owners by durable UTC evidence before local projection."""

        report_instant = _aware(now)
        horizon_start = starts[0].astimezone(timezone.utc)
        with self._lock:
            read = self._read()

        candidates: list[dict] = []
        for row in read.rows:
            if _has_recent_hourly_evidence(row, horizon_start, report_instant):
                candidates.append(row)
                continue
            latest = _instant(row.get("last_metered_at"))
            if (
                latest is not None
                and latest < horizon_start
                and not (
                    isinstance(row.get("hours"), list)
                    and _has_unresolved_hourly_mass(row)
                )
            ):
                continue
            owner_day_window = _hourly_owner_day_window(row)
            if owner_day_window is not None and _intervals_overlap(
                horizon_start,
                report_instant,
                *owner_day_window,
            ):
                candidates.append(row)
                continue

        held = self._within_capacity(candidates, measured=report_instant, utc_owner_days=True)
        if len(held) < len(candidates):
            logger.warning(
                "Model Hub usage ledger %s held %d hourly report row(s) over "
                "its capacity of %d; dropped the least recently metered",
                self.path,
                len(candidates) - len(held),
                self.max_rows,
            )
        capacity_degraded = _hourly_uncertainty_was_dropped(
            candidates,
            held,
            horizon_start,
            report_instant,
        )
        return sorted(held, key=_row_key), read.degraded or capacity_degraded

    def summary(
        self,
        *,
        days: int = USAGE_DEFAULT_WINDOW_DAYS,
        now: datetime,
        identities: Optional[Sequence[SourceIdentity]] = None,
        prices: Optional[PriceTable] = None,
        service_catalog: Optional[Sequence[ServiceCatalogEntry]] = None,
    ) -> dict:
        """Aggregate the window into the read shape the settings page consumes.

        Labels are joined here rather than persisted, because a display name is
        user-supplied text this ledger has no business storing and a join keeps a
        rename visible immediately instead of freezing old copies. What arrives is
        what config holds — identities, nested by Source — and this method keys it,
        because a row is keyed and the component holding config is not the one that
        knows that: the caller that keyed its own map looked a label up by
        `row["source_id"]`, so a source whose ID is past the admission bound reported
        no label while still existing, and its renames never appeared.

        The nesting is the other half of the same rule. A model's identity is its
        Source and its ID together, so a flat model map answers for the wrong Source
        whenever a common model ID is listed on more than one — which is every
        interesting case. `SourceIdentity` makes that arity part of the argument's
        type, so there is no keying convention left for a caller to hold.

        A model's label is its own identity, which only matters for a folded row.
        There the key is a head plus a digest, so a tab drawing `model_id` would
        show a string nobody typed; the label carries the identity back.

        `service_catalog` is the current release's search services, which label and
        value service rows the way the price table values model tokens.
        """

        bounded_days = max(1, min(int(days), self.retention_days))
        today = local_usage_day(now)
        rows, degraded = self._window_rows(days=bounded_days, now=now)
        return self._summary_from_rows(
            rows,
            window_days=bounded_days,
            from_day=(today - timedelta(days=bounded_days - 1)).isoformat(),
            to_day=today.isoformat(),
            identities=identities,
            prices=prices,
            service_catalog=service_catalog,
            degraded=degraded,
        )

    def _summary_from_rows(
        self,
        rows: Sequence[dict],
        *,
        window_days: int,
        from_day: str,
        to_day: str,
        identities: Optional[Sequence[SourceIdentity]],
        prices: Optional[PriceTable] = None,
        service_catalog: Optional[Sequence[ServiceCatalogEntry]] = None,
        degraded: bool = False,
    ) -> dict:
        """Aggregate one exact set of daily or hourly rows.

        With a price table, every aggregate also carries its API-price valuation,
        summed from the same rows so a total always equals the sum of its parts.
        A `degraded` read lost rows of unknown size, so every valuation is a floor.

        Search-service rows never enter a model count or `sources[]`: they are
        `services[]`, and only their valuation joins the totals and days, beside
        the vendors' own reported figures (`web-search-services.md` §4.3).
        """

        keyed_source_labels, keyed_model_labels = _keyed_identities(identities)
        keyed_catalog = _keyed_catalog(service_catalog)
        costs: dict[int, Cost] = {}
        service_costs: dict[int, ServiceCost] = {}

        def priced(target: dict, cost: Optional[Cost]) -> None:
            if cost is not None:
                costs.setdefault(id(target), Cost()).add(cost)

        def service_priced(target: dict, cost: Optional[ServiceCost]) -> None:
            if cost is not None:
                service_costs.setdefault(id(target), ServiceCost()).add(cost)

        def with_cost(target: dict, *, omit: str = "", with_services: bool = False) -> dict:
            published = {key: value for key, value in target.items() if key != omit}
            if prices is None:
                return published
            cost = costs.get(id(target), Cost())
            cost.api_cost_lower_bound = cost.api_cost_lower_bound or degraded
            fields = cost.fields()
            if with_services:
                # The services' list valuation joins the one API-price figure;
                # their reported costs stay beside it, never added to it.
                service = service_costs.get(id(target), ServiceCost())
                fields["api_cost_usd"] = round(cost.api_cost_usd + service.api_cost_usd, 6)
                fields["api_cost_lower_bound"] = fields["api_cost_lower_bound"] or service.lower_bound
                fields["excluded_service_requests"] = service.excluded_requests
                fields["service_reported_usd"] = round(service.reported_usd, 6)
            return {**published, **fields}

        totals = _empty_totals()
        sources: dict[str, dict] = {}
        services: dict[tuple[str, str, str], dict] = {}
        by_day: dict[str, dict] = {}
        for row in rows:
            if _is_service(row):
                entry = keyed_catalog.get(row["service_id"])
                service_cost = (
                    None
                    if prices is None
                    else service_row_cost(row, entry.usd_per_1k if entry is not None else None)
                )
                service_priced(totals, service_cost)
                day = by_day.setdefault(row["day"], {"day": row["day"], **_empty_totals()})
                service_priced(day, service_cost)
                service = services.setdefault(
                    (row["source_id"], row["service_id"], row.get("tool", "web_search")),
                    {
                        "source_id": row["source_id"],
                        "service_id": row["service_id"],
                        **({"tool": row["tool"]} if "tool" in row else {}),
                        "label": keyed_source_labels.get(row["source_id"]),
                        "service_label": entry.label if entry is not None else None,
                        **_empty_totals(service=True),
                        "last_metered_at": None,
                    },
                )
                _accumulate(service, row)
                service_priced(service, service_cost)
                service["last_metered_at"] = _newer_timestamp(
                    service["last_metered_at"],
                    row["last_metered_at"],
                )
                continue
            _accumulate(totals, row)
            _count_fast(totals, row)
            cost = (
                None
                if prices is None
                else row_cost(row, _row_price(row, prices, keyed_model_labels))
            )
            priced(totals, cost)

            source = sources.setdefault(
                row["source_id"],
                {
                    "source_id": row["source_id"],
                    "label": keyed_source_labels.get(row["source_id"]),
                    **_empty_totals(),
                    "last_metered_at": None,
                    "models": {},
                },
            )
            _accumulate(source, row)
            _count_fast(source, row)
            priced(source, cost)
            source["last_metered_at"] = _newer_timestamp(
                source["last_metered_at"],
                row["last_metered_at"],
            )
            model = source["models"].setdefault(
                row["model_id"],
                {
                    "model_id": row["model_id"],
                    "label": keyed_model_labels.get((row["source_id"], row["model_id"])),
                    **_empty_totals(),
                },
            )
            _accumulate(model, row)
            _count_fast(model, row)
            priced(model, cost)
            if prices is not None:
                model["priced"] = _row_price(row, prices, keyed_model_labels) is not None

            day = by_day.setdefault(row["day"], {"day": row["day"], **_empty_totals()})
            _accumulate(day, row)
            _count_fast(day, row)
            priced(day, cost)

        published_services = []
        for key in sorted(services, key=lambda key: (-services[key]["requests"], key)):
            service = services[key]
            fields = _service_fields(
                service,
                service_costs.get(id(service), ServiceCost()) if prices is not None else None,
                degraded=degraded,
            )
            if prices is not None:
                fields["priced"] = service["service_id"] in keyed_catalog
            published_services.append(fields)
        summary = {
            "window_days": window_days,
            "from_day": from_day,
            "to_day": to_day,
            "totals": with_cost(totals, with_services=True),
            "sources": [
                {
                    **with_cost(source, omit="models"),
                    "models": [
                        with_cost(model)
                        for model in sorted(
                            source["models"].values(),
                            key=lambda model: (-model["requests"], model["model_id"]),
                        )
                    ],
                }
                for source in sorted(
                    sources.values(),
                    key=lambda source: (-source["requests"], source["source_id"]),
                )
            ],
            "services": published_services,
            "days": [with_cost(by_day[day], with_services=True) for day in sorted(by_day)],
        }
        if prices is not None:
            summary["pricing"] = prices.fields()
            price_dates = [
                keyed_catalog[service["service_id"]].price_date
                for service in published_services
                if service["service_id"] in keyed_catalog
            ]
            if price_dates:
                # The catalog prices that valued `services[]`, by when the oldest of
                # them was verified; with no service valued there is nothing to date.
                summary["pricing"]["service_price_date"] = min(price_dates)
        return summary

    def report(
        self,
        *,
        window: str,
        now: datetime,
        identities: Optional[Sequence[SourceIdentity]] = None,
        prices: Optional[PriceTable] = None,
        service_catalog: Optional[Sequence[ServiceCatalogEntry]] = None,
    ) -> dict:
        """Build one modern dense report without reallocating daily history."""

        if window not in USAGE_WINDOW_KEYS:
            raise ValueError(f"unsupported usage window: {window!r}")
        if window == "24h":
            return self._hourly_report(
                now=now, identities=identities, prices=prices, service_catalog=service_catalog
            )
        days = int(window[:-1])
        bounded_days = max(1, min(days, self.retention_days))
        report_local = _local(now)
        today = report_local.date()
        from_day = today - timedelta(days=bounded_days - 1)
        rows, read_degraded = self._window_rows(days=bounded_days, now=now)
        summary = self._summary_from_rows(
            rows,
            window_days=bounded_days,
            from_day=from_day.isoformat(),
            to_day=today.isoformat(),
            identities=identities,
            prices=prices,
            service_catalog=service_catalog,
            degraded=read_degraded,
        )
        _source_labels, keyed_model_labels = _keyed_identities(identities)
        keyed_catalog = _keyed_catalog(service_catalog)
        by_day = {
            row["day"]: []
            for row in rows
            if from_day.isoformat() <= row["day"] <= today.isoformat()
        }
        for row in rows:
            if row["day"] in by_day:
                by_day[row["day"]].append(row)
        buckets = []
        for index in range(bounded_days):
            bucket_day = from_day + timedelta(days=index)
            start = _local_midnight(bucket_day)
            end = (
                report_local
                if bucket_day == today
                else _local_midnight(bucket_day + timedelta(days=1))
            )
            buckets.append(
                {
                    "key": bucket_day.isoformat(),
                    "start_at": start.isoformat(),
                    "end_at": end.isoformat(),
                    "history_complete": not read_degraded,
                    "rows": _bucket_rows(
                        by_day.get(bucket_day.isoformat(), []),
                        prices,
                        keyed_model_labels,
                        keyed_catalog=keyed_catalog,
                        degraded=read_degraded,
                    ),
                }
            )
        return {
            **summary,
            "window_key": window,
            "granularity": "day",
            "from_at": buckets[0]["start_at"],
            "to_at": report_local.isoformat(),
            "buckets": buckets,
        }

    def _hourly_report(
        self,
        *,
        now: datetime,
        identities: Optional[Sequence[SourceIdentity]],
        prices: Optional[PriceTable] = None,
        service_catalog: Optional[Sequence[ServiceCatalogEntry]] = None,
    ) -> dict:
        """Project only measured nested slices onto 24 actual consecutive hours."""

        _source_labels, keyed_model_labels = _keyed_identities(identities)
        keyed_catalog = _keyed_catalog(service_catalog)

        report_instant = _aware(now)
        starts = _hour_starts(now)
        interval_by_instant = {
            start.astimezone(timezone.utc): (start, index)
            for index, start in enumerate(starts)
        }
        first_start = starts[0]
        report_local = _local(now)
        last_day = report_local.date()
        # DST plus a fractional offset can make these 24 actual intervals touch
        # three local dates. Select rows from their durable UTC-hour evidence
        # before applying the current host timezone to their local presentation.
        rows, read_degraded = self._hourly_rows(starts=starts, now=report_instant)
        measured_by_bucket: list[dict[tuple[str, str, int], dict]] = [
            {} for _ in starts
        ]
        incomplete: set[int] = set(range(len(starts))) if read_degraded else set()

        for row in rows:
            # Read-only projection must apply the same temporal evidence policy
            # as persistence, including future slices in an otherwise valid day.
            owner_day_window = _hourly_owner_day_window(row)
            row = _retain_hour_slices(row, report_instant)
            row_day = _calendar_day(row["day"])
            if row.get("hourly_history_complete") is not True:
                overlapping = False
                latest = _instant(row.get("last_metered_at"))
                owner_mismatch = (
                    latest is not None
                    and first_start <= latest <= report_instant
                    and local_usage_day(latest) != row_day
                )
                if owner_day_window is not None:
                    for index, start in enumerate(starts):
                        end = (
                            starts[index + 1]
                            if index + 1 < len(starts)
                            else report_instant
                        )
                        if _intervals_overlap(start, end, *owner_day_window):
                            incomplete.add(index)
                            overlapping = True
                if not overlapping or owner_mismatch:
                    # Even partial owner-day overlap cannot localize missing
                    # calls when durable UTC evidence disagrees with that day.
                    # No old-zone offset exists for reconstructing their hours.
                    incomplete.update(range(len(starts)))
            # A damaged request counter cannot erase the incomplete witness.
            if row["requests"] <= 0:
                continue
            for item in row.get("hours") or ():
                parts = _hour_key_parts(item.get("key"))
                if parts is None:
                    continue
                _key, start = parts
                bucket = interval_by_instant.get(start)
                if bucket is None:
                    continue
                _start_utc, index = bucket
                key = _row_key(row)[1:]
                projected = measured_by_bucket[index].get(key)
                if projected is None:
                    start_local = start.astimezone()
                    service = _is_service(row)
                    projected = {
                        "day": start_local.date().isoformat(),
                        "source_id": row["source_id"],
                        **({"service_id": row["service_id"], **({"tool": row["tool"]} if "tool" in row else {})}
                           if service else {"model_id": row["model_id"]}),
                        **({"context_tier": row["context_tier"]} if "context_tier" in row else {}),
                        **({"speed": row["speed"]} if "speed" in row else {}),
                        **_empty_totals(service=service),
                        "last_metered_at": None,
                    }
                    measured_by_bucket[index][key] = projected
                _accumulate(projected, item)
                if _is_service(row) and row.get("lower_bound") and row.get("uncertain_requests", 0) == 0:
                    projected["lower_bound"] = True
                projected["last_metered_at"] = _newer_timestamp(
                    projected["last_metered_at"],
                    item.get("last_metered_at"),
                )

        bucket_rows = []
        summary_rows = []
        for index, start in enumerate(starts):
            end = starts[index + 1] if index + 1 < len(starts) else report_instant
            start_local = start.astimezone()
            end_local = end.astimezone()
            projected_rows = _bucket_rows(
                list(measured_by_bucket[index].values()),
                prices,
                keyed_model_labels,
                keyed_catalog=keyed_catalog,
                # An incomplete hour holds usage it cannot place, so its cost is a floor.
                degraded=index in incomplete,
            )
            bucket_rows.append(
                {
                    "key": start_local.isoformat(timespec="seconds"),
                    "start_at": start_local.isoformat(),
                    "end_at": end_local.isoformat(),
                    "history_complete": index not in incomplete,
                    "rows": projected_rows,
                }
            )
            summary_rows.extend(measured_by_bucket[index].values())

        days = sorted({
            *(start.astimezone().date().isoformat() for start in starts),
            last_day.isoformat(),
        })
        summary = self._summary_from_rows(
            summary_rows,
            window_days=len(days),
            from_day=days[0],
            to_day=days[-1],
            identities=identities,
            prices=prices,
            service_catalog=service_catalog,
            degraded=bool(incomplete),
        )
        return {
            **summary,
            "window_key": "24h",
            "granularity": "hour",
            "from_at": first_start.astimezone().isoformat(),
            "to_at": report_local.isoformat(),
            "buckets": bucket_rows,
        }


class _AbandonableWriter(Executor):
    """One serialized worker for ledger writes that the process can walk away from.

    `ThreadPoolExecutor` is the obvious way to spell "one worker thread", and it is
    the one mechanism this module may not use: its workers are non-daemon and it
    registers an `atexit` hook that joins them. A worker wedged in `fsync` on an
    unresponsive disk therefore holds interpreter shutdown open forever — *after*
    every bounded wait in this module has already returned and told its caller the
    write was unfinished. Bounding the coroutine is not the same as being able to
    abandon the work, and this is the resource where only the second one counts:
    metering is optional, so a stop or restart must never wait on it.

    A daemon thread says exactly that, and CPython does not join one at shutdown.
    The `Executor` interface is kept so `run_in_executor` still bridges the thread
    to the loop; a worker is started on the first write and lives as long as the
    process, which is the same single idle thread the pool cost.
    """

    def __init__(self) -> None:
        self._queue: "queue.SimpleQueue[tuple[CFuture, Callable, tuple, dict]]" = queue.SimpleQueue()
        self._worker = threading.Thread(
            target=self._serve,
            name="model-hub-usage",
            daemon=True,
        )
        self._worker.start()

    def submit(self, fn: Callable, /, *args: object, **kwargs: object) -> CFuture:
        future: CFuture = CFuture()
        self._queue.put((future, fn, args, kwargs))
        return future

    def _serve(self) -> None:
        while True:
            future, fn, args, kwargs = self._queue.get()
            if not future.set_running_or_notify_cancel():
                continue
            try:
                result = fn(*args, **kwargs)
            except BaseException as exc:  # noqa: BLE001 - relayed to the waiting future
                future.set_exception(exc)
            else:
                future.set_result(result)


_LEDGER_EXECUTOR_LOCK: Final = threading.Lock()
_LEDGER_EXECUTOR: Optional[_AbandonableWriter] = None
# How long a metering caller waits for its own row to reach the disk. Sized for a
# local read-modify-write and an fsync plus scheduling slack, which is orders of
# magnitude under it, so nothing but a disk that has stopped answering gets here.
_DURABILITY_WAIT_SECONDS: Final = 2.0
# How long a price table the writer read decides context tiers before a flush reads
# it again. A read parses the whole models.dev copy, megabytes, so it happens on the
# writer's thread rather than once per call; the override file is hand-edited, and
# minutes are soon enough for an edit to reach new calls.
_PRICE_TABLE_TTL_SECONDS: Final = 300.0


def _ledger_executor() -> _AbandonableWriter:
    """The one thread ledger writes run on, created when something is first metered.

    Deliberately not the loop's default executor. `record_many` holds the ledger's
    lock across an fsync, so submitting one job per completed call there would
    occupy that many shared workers while all but one waited on the lock — and
    whatever else in the process reaches for a thread would wait behind metering
    for no gain, since the lock admits one writer regardless. Owning the
    serialization makes it explicit instead of emergent, and it costs a single
    idle thread once anything has been metered at all.

    Owning it is also what makes it abandonable; see `_AbandonableWriter` for why
    the stdlib pool is the one mechanism that cannot be used here.
    """

    global _LEDGER_EXECUTOR
    with _LEDGER_EXECUTOR_LOCK:
        if _LEDGER_EXECUTOR is None:
            _LEDGER_EXECUTOR = _AbandonableWriter()
        return _LEDGER_EXECUTOR


@dataclass(frozen=True)
class UsageCall:
    """One metered upstream call, in the shape the ledger folds it in.

    Or several of them. `requests` is how many calls this value speaks for, so a
    queue can fold the ones that will land on a single row before they reach the
    disk without a second shape existing for a folded call. Every call folded
    together agrees on whether tokens were reported — that is part of what makes
    them one row — so `token_reports` stays derivable from `usage` instead of
    becoming a second additive field able to disagree with `requests`.
    """

    source_id: str
    model_id: str
    usage: Optional[ProtocolUsageReport]
    at: datetime
    requests: int = 1
    owner_day: str = ""
    # The context tier the call was billed in (`pricing.context_tier_size`), 0 for
    # the base rates. Decided per call, since a folded row can no longer tell.
    context_tier: int = 0
    # `"fast"` when the call's response said it ran at the vendor's fast tier, so it
    # is billed at fast rates; None for the standard tier.
    speed: Optional[str] = None

    def __post_init__(self) -> None:
        # An init field intentionally survives dataclasses.replace during a fold.
        # Recomputing it from `at` later would use a possibly different host zone.
        if not self.owner_day:
            object.__setattr__(self, "owner_day", local_usage_day(self.at).isoformat())

    @property
    def fold_key(self) -> tuple[str, str, str, str, int, str, bool]:
        """What makes two calls one daily row and one hourly slice.

        The ledger's own row key plus whether tokens were reported, which is the
        part that keeps a fold arithmetically identical to the calls it replaces.
        The row key includes the context tier and the speed, so calls billed at
        different rates never fold: one call's tier would otherwise price the
        other's tokens.
        The hour is part of the queue key so a flush cannot merge calls from
        different hours before the ledger sees their temporal identity. The local
        date must also remain: a UTC hour can cross local midnight, and folding
        those calls would move the earlier day's counts into the later day.
        """

        return (
            self.owner_day,
            _hour_key(self.at),
            self.source_id,
            self.model_id,
            self.context_tier,
            self.speed or "",
            self.usage is not None,
        )

    def folded_with(self, other: "UsageCall") -> "UsageCall":
        """Fold a call that shares this one's row into a single value.

        Arithmetically the calls it replaces: counts add, token counts add, and
        the stamp is the later of the two — which is what the row would have kept
        anyway, since `record_many` keeps the newer stamp. Nothing is bounded here;
        every counter is bounded where rows are normalized, so a fold that
        saturates the ceiling saturates it identically.
        """

        reports = [report for report in (self.usage, other.usage) if report is not None]
        usage = None
        if reports:
            usage = ProtocolUsageReport.of(
                input_tokens=sum(report.input_tokens for report in reports),
                cached_input_tokens=sum(report.cached_input_tokens for report in reports),
                output_tokens=sum(report.output_tokens for report in reports),
                cache_write_input_tokens=sum(report.cache_write_input_tokens for report in reports),
                cache_write_1h_input_tokens=sum(
                    report.cache_write_1h_input_tokens for report in reports
                ),
                cache_write_30m_input_tokens=sum(
                    report.cache_write_30m_input_tokens for report in reports
                ),
                cache_write_12h_input_tokens=sum(
                    report.cache_write_12h_input_tokens for report in reports
                ),
                cache_write_ttl_uncaptured_reports=sum(
                    report.cache_write_ttl_uncaptured_reports for report in reports
                ),
            )
        return replace(
            self,
            usage=usage,
            at=max(_aware(self.at), _aware(other.at)),
            requests=self.requests + other.requests,
        )


@dataclass(frozen=True)
class ServiceUsageCall:
    """Sent search-service requests, in the shape the ledger folds them in.

    The service counterpart of `UsageCall`: a request reaches no model, so it is
    counted by what the catalog's billing rule makes of it (`web-search-services.md`
    §4.3) rather than by tokens. `requests` is how many sent requests this value
    speaks for; the other counts are the subsets the relations bound. A request
    that failed before it was sent is never one of them.
    """

    source_id: str
    service_id: str
    at: datetime
    requests: int = 1
    billable_requests: int = 0
    uncertain_requests: int = 0
    reported_requests: int = 0
    reported_usd: float = 0.0
    owner_day: str = ""
    tool: str = "web_search"

    def __post_init__(self) -> None:
        if self.tool not in ("web_search", "web_fetch"):
            raise ValueError("unsupported hosted service tool")
        if not self.owner_day:
            object.__setattr__(self, "owner_day", local_usage_day(self.at).isoformat())

    @property
    def fold_key(self) -> tuple[str, str, str, str, str, str]:
        """What makes two calls one daily row and one hourly slice; never a model's key."""

        return ("service", self.owner_day, _hour_key(self.at), self.source_id, self.service_id, self.tool)

    def folded_with(self, other: "ServiceUsageCall") -> "ServiceUsageCall":
        """Fold a call that shares this one's row: every count adds, the later stamp wins."""

        return replace(
            self,
            at=max(_aware(self.at), _aware(other.at)),
            requests=self.requests + other.requests,
            billable_requests=self.billable_requests + other.billable_requests,
            uncertain_requests=self.uncertain_requests + other.uncertain_requests,
            reported_requests=self.reported_requests + other.reported_requests,
            reported_usd=self.reported_usd + other.reported_usd,
        )


def _flag_count(flag: object) -> Optional[int]:
    """One request's count for a billing flag; None for an answer that is not a flag.

    Anything but a bool, an int included, is refused rather than coerced: `int()`
    would read 2 as two requests and fail on text. A refused count keeps the call
    out of the fold, and the ledger repairs it into a floor that keeps the request.
    """

    if flag is True:
        return 1
    if flag is False:
        return 0
    return None


def _service_call_is_foldable(call: ServiceUsageCall) -> bool:
    """Only fold service calls whose own counters already satisfy the relations.

    Invalid calls must reach ``_normalize_row`` separately: folding an invalid
    reported amount with a valid call could make the combined row look valid and
    erase the required lower-bound witness.
    """

    counters = {
        "requests": call.requests,
        "billable_requests": call.billable_requests,
        "uncertain_requests": call.uncertain_requests,
        "reported_requests": call.reported_requests,
    }
    if any(
        type(value) is not int or value < 0 or value > USAGE_COUNTER_CEILING
        for value in counters.values()
    ):
        return False
    amount = _amount(call.reported_usd)
    return (
        amount is not None
        and counters["billable_requests"] + counters["uncertain_requests"] <= counters["requests"]
        and counters["reported_requests"] <= counters["billable_requests"]
        and amount <= counters["reported_requests"] * _SERVICE_REPORTED_USD_CEILING
    )


def _queue_key(call: UsageCall | ServiceUsageCall) -> tuple:
    """Keep invalid service calls separate until the ledger repairs each one."""

    key = call.fold_key
    if isinstance(call, ServiceUsageCall) and not _service_call_is_foldable(call):
        return (*key, id(call))
    return key


@dataclass
class _QueuedRow:
    """One queued row and the future every call folded into it is waiting on."""

    call: UsageCall | ServiceUsageCall
    done: "asyncio.Future[None]"


class UsageWriter:
    """Async ownership of every ledger write, so nothing can take one along.

    Metering has two populations — a call either hands its body onward or it does
    not — but the property that a queued write outlives whatever queued it belongs
    to neither of them. It lives here once. A write is this object's from the
    moment it exists, so a caller cancelled mid-flight loses its own ordering and
    never the row, and a population added later inherits that by construction
    rather than by remembering to.

    Queue, not fan-out, and folded rather than capped. Calls accumulate while the
    flush ahead of them is on disk and the next flush takes all of them in one
    transaction; the ones heading for a single row fold into a single queued row as
    they arrive. The backlog is therefore bounded by the identities config holds
    rather than by how hard the hub is driven or by how long one write takes — a
    flush stuck on an unresponsive disk cannot grow it past that bound, which is
    the whole of what "bounded" can mean here. The alternatives both cost
    something this module exists to prevent: a capacity means choosing rows to
    drop, which loses billed usage, and blocking the caller stalls a served turn
    on a disk that is already failing.

    A call's context tier is part of its row, so the writer decides it before the
    call folds, from the price table it holds: read when the writer is built, at
    controller startup, and refreshed on its own thread.
    """

    def __init__(
        self,
        ledger: BoundedUsageLedger,
        *,
        durability_wait: float = _DURABILITY_WAIT_SECONDS,
        price_table: Optional[Callable[[], PriceTable]] = None,
    ):
        self.ledger = ledger
        self._durability_wait = durability_wait
        self._pending: dict[tuple, _QueuedRow] = {}
        # Where context tiers come from: the valuation's table factory, and the
        # table last read from it with the monotonic instant it was read.
        self._price_table = price_table
        self._prices: Optional[tuple[PriceTable, float]] = None
        if price_table is not None:
            # Read here, once: the writer is built with Model Hub's service at
            # controller startup, before anything is metered or served, so every
            # call finds a table and is tiered when it is recorded.
            self._read_prices()
        # The batch on its way to disk, kept visible so a drain that times out
        # can count it: in the executor is not the same as persisted.
        self._writing: tuple[_QueuedRow, ...] = ()
        self._flush: Optional[asyncio.Task[None]] = None
        # Whether the last flush lost its batch, so an outage is reported when it
        # starts and ends rather than once per failed write.
        self._dropping = False

    @property
    def held_prices(self) -> Optional[PriceTable]:
        """The price table this writer last read, without any I/O; None before its first read."""

        held = self._prices
        return held[0] if held is not None else None

    def record(
        self,
        *,
        source_id: str,
        model_id: str,
        usage: Optional[ProtocolUsageReport],
        at: datetime,
        speed: Optional[str] = None,
    ) -> "asyncio.Future[None]":
        """Own one call and queue it, handing back what will finish it.

        Synchronous on purpose: nothing may suspend between a caller deciding to
        meter a call and this object owning the result, or a cancellation could
        land in a window where the call is neither metered nor still meterable.

        `at` is the caller's own reading from when its call ended, carried rather
        than re-read, because the flush that persists this can start well after
        it and a queued write is still a report about the moment it finished.

        A caller that wants the row on disk before it returns hands the result to
        `wait_recorded`; one that does not can simply drop it. Callers whose calls
        fold together share one future, which is the same promise each would have
        had alone: it completes when their row reaches the disk.

        The call's context tier is decided here from the price table the writer
        holds, a lookup in memory. A writer built without a table, or holding one
        without tiers (no models.dev copy yet), records the base tier. `speed` is
        `"fast"` only when the call's response said it ran at the fast tier.
        """

        speed = FAST_SPEED if speed == FAST_SPEED else None
        return self._queue(
            UsageCall(
                source_id=source_id,
                model_id=model_id,
                usage=usage,
                at=at,
                context_tier=self._context_tier(model_id, usage, speed),
                speed=speed,
            )
        )

    def record_service(
        self,
        *,
        source_id: str,
        service_id: str,
        billable: bool,
        uncertain: bool = False,
        reported_usd: Optional[float] = None,
        at: datetime,
        tool: str = "web_search",
    ) -> "asyncio.Future[None]":
        """Own one sent search-service request and queue it, as `record` does a model call.

        Synchronous for the same reason, so a stopped turn still meters a request
        it sent (`web-search-services.md` §4.3). `billable` is the catalog's billing
        rule's answer; `uncertain` is a request whose billing is unknown, which is
        never also billable. `reported_usd` is the vendor's own figure for a billable
        request, already bounded by the caller; the ledger's relations are the
        backstop, and a figure that breaks them is recorded as a floor, not as sent.
        """

        return self._queue(
            ServiceUsageCall(
                source_id=source_id,
                service_id=service_id,
                at=at,
                billable_requests=_flag_count(billable),
                uncertain_requests=_flag_count(uncertain),
                reported_requests=int(reported_usd is not None),
                reported_usd=reported_usd if reported_usd is not None else 0.0,
                tool=tool,
            )
        )

    def _queue(self, call: UsageCall | ServiceUsageCall) -> "asyncio.Future[None]":
        done = asyncio.get_running_loop().create_future
        key = _queue_key(call)
        if (queued := self._pending.get(key)) is None:
            queued = _QueuedRow(call=call, done=done())
            self._pending[key] = queued
        else:
            queued.call = queued.call.folded_with(call)
        if self._flush is None or self._flush.done():
            self._flush = asyncio.create_task(self._flush_pending())
        return queued.done

    async def wait_recorded(self, write: "asyncio.Future[None]") -> bool:
        """Wait out one queued write, bounded, and answer whether it landed.

        Two properties every metering caller wants and none of them owns. The
        shield is why a caller cancelled here loses its ordering and not the row.
        The bound is why a ledger that has stopped answering cannot hold a served
        turn open behind it: the wait exists so a caller reading the usage tab
        right after its own call sees that call, which is an ordering convenience,
        not a durability requirement the turn is allowed to fail for.

        Both used to be spelled at each metering call site, which made the bound
        the one property a new call site could omit by writing the obvious thing —
        and it did: awaiting a write with no deadline puts a stuck disk on the
        turn's critical path. Here there is nothing to omit.

        A timed-out write is not lost or cancelled. It stays this writer's, keeps
        its place in the queue, and keeps counting in `unpersisted` until the disk
        answers; only the caller's wait for it ends.
        """

        try:
            await asyncio.wait_for(asyncio.shield(write), self._durability_wait)
        except asyncio.TimeoutError:
            return False
        return True

    @property
    def unpersisted(self) -> int:
        """How many metered calls this writer still owes the ledger.

        Queued and in-flight both count: handed to the writing thread is not the
        same as on disk. Calls, not queued rows — folding is how this object stays
        bounded, and answering in rows would make a backlog look like it shrank
        because the hub got busier.
        """

        return sum(
            queued.call.requests
            for queued in (*self._pending.values(), *self._writing)
        )

    async def drain(self, *, timeout: float) -> int:
        """Wait out the calls still queued; answers how many did not reach disk.

        Bounded, because a ledger that cannot be reached must not hold a shutdown
        open — the same trade every owned drain in the hub makes.
        """

        flush = self._flush
        if flush is not None and not flush.done():
            await asyncio.wait((flush,), timeout=timeout)
        return self.unpersisted

    async def _flush_pending(self) -> None:
        """Write queued calls a batch at a time until nothing is left waiting.

        The loop ends only with the queue empty and without having awaited since
        it saw that, which is what lets `record` treat a finished flush as proof
        that it must start the next one.
        """

        loop = asyncio.get_running_loop()
        while self._pending:
            batch = tuple(self._pending.values())
            self._pending = {}
            self._writing = batch
            try:
                calls = tuple(queued.call for queued in batch)
                if self._prices_due():
                    # A refresh for the calls after this batch; these are already tiered.
                    await loop.run_in_executor(_ledger_executor(), self._read_prices)
                await loop.run_in_executor(_ledger_executor(), self.ledger.record_many, calls)
            except (OSError, ValueError) as exc:
                self._report_dropped(batch, exc)
            else:
                self._report_recovered()
            finally:
                self._writing = ()
            for queued in batch:
                if not queued.done.done():
                    queued.done.set_result(None)

    def _context_tier(self, model_id: str, usage: Optional[ProtocolUsageReport], speed: Optional[str] = None) -> int:
        """The tier the held price table puts one call in; the base tier without usage or a table.

        A fast call is placed by the fast rates' own context sizes, the rates its
        row is valued at; where the table lists no fast rates, by the standard ones.
        """

        held = self._prices
        if usage is None or held is None:
            return 0
        price = held[0].price(model_id)
        if speed == FAST_SPEED and price is not None and price.fast is not None:
            price = price.fast
        return context_tier_size(price, usage.input_tokens)

    def _prices_due(self) -> bool:
        """Whether the next flush re-reads the price table: none held, too old, or still being fetched."""

        if self._price_table is None:
            return False
        held = self._prices
        return held is None or held[0].pending or monotonic() - held[1] >= _PRICE_TABLE_TTL_SECONDS

    def _read_prices(self) -> None:
        """Read the price table; one that cannot be read leaves the held one (or none).

        Never on the controller loop: at construction, before anything is served,
        and afterwards on the writer's thread. A valuation must never cost a
        metered call.
        """

        if self._price_table is None:
            return
        try:
            self._prices = (self._price_table(), monotonic())
        except Exception:  # noqa: BLE001 - valuation is optional; metering is not
            logger.warning("Model Hub usage metering could not read the price table", exc_info=True)

    def _report_dropped(self, batch: Sequence[_QueuedRow], exc: BaseException) -> None:
        """Report that metering stopped, once per outage rather than per flush.

        A ledger that cannot be written must not hold up the turns it meters, so a
        failed batch is lost by design — but a state directory that has gone
        read-only then looks exactly like a hub with nothing to record, which is
        the one reading that makes this module's absence invisible. Saying it at
        `debug` said it to nobody.

        The transition carries the information, so the transition is what is
        logged: bounded by how often the ledger changes state rather than by how
        long an outage lasts, which is the bound a counter would be approximating.
        """

        calls = sum(queued.call.requests for queued in batch)
        if self._dropping:
            logger.debug(
                "Model Hub usage metering still cannot write %s, dropped %d call(s): %s",
                self.ledger.path,
                calls,
                exc,
            )
            return
        self._dropping = True
        logger.warning(
            "Model Hub usage metering cannot write %s and is dropping metered calls "
            "(%d lost in this batch); the usage tab will under-report until it recovers: %s",
            self.ledger.path,
            calls,
            exc,
        )

    def _report_recovered(self) -> None:
        """Close an outage the same way it was opened, so the gap has both edges."""

        if not self._dropping:
            return
        self._dropping = False
        logger.warning("Model Hub usage metering recovered; %s is writable again", self.ledger.path)