Garry's Opinionated OpenClaw/Hermes Agent Brain
auto_chronicle date quality (capy/chronicle-date-quality, v0.60.49.0)
The gbrain-evals off-versus-on lift eval (garrytan/gbrain-evals#62, gbrain
739e5cc) failed auto_chronicle's quality gate: 0.96 false + premature events
per judged labeled page against a gate of 0.20. Two patterns caused it:
- CL-1, premature events. Plans, follow-ups and scheduled meetings in a past page were written as events on their future dates.
- CL-2, invented days. A vague past date became a specific day ("back in 2024" → 2024-01-01, "last month" → the first of the month).
Fix
- Prompt. The judge extracts only what happened by the end of the page's
day, dates a commitment on the meeting day rather than its due date, and
writes a vague past date at its real precision (
2024,2026-03). future_dated(deterministic). A proposal dated after the page's own day is dropped before publication. The page's own day is the latest of the eligibility date (authored effective date, else frontmatterdate/start), a calendar invite'sendand a conversation's last parsed message, as a day inchronicle.tz; a date-only value keeps its own day in every time zone. Nothing after today passes either, which also bounds undated pages.date_imprecise(deterministic). A proposal whosewhenhas no day (YYYY,YYYY-MM) is dropped. The schema has no date precision:timeline_entries.dateisDATE NOT NULL, event slugs andevent_dateare days, andchronicle_day/since/last_seenread days. Storing a partial date would need a precision column and changes to every reader, so the extractor drops it rather than invent a day. No migration.- Reporting. Drops are counted in the phase result's
events_dropped, the row outcome and the direct extractor result. A page whose every proposal was dropped records the drop reason on its extracted ledger row (and counts asno_eventsin the phase details). Both codes areCHRONICLE_REASONSentries andchronicle_skippedreasons.
CHRONICLE_EXTRACTOR_VERSION is unchanged: bumping it would re-extract every
page once at one paid call each. Existing premature events are retired when
their page is extracted again.
Measured run
gbrain-evals branch capy/eval-repin-739e5cc (chronicle-lift.ts run --arms on-a --no-qa), corpus amara-life-v1 (48 judged pages; 38 labeled events on
28 meeting and calendar pages; 96 controls), judge
anthropic:claude-sonnet-4-6, node_modules/gbrain pointed at each checkout.
Scored with chronicle-lift.ts score --review and the published
review.json; unmatched events the published review did not cover were
classified by hand with the same rubric. Counted strictly, as the published
review does, a quarter result dated on the meeting that discussed it is false.
master 5bd9e849 |
branch, run 1 | branch, run 2 | |
|---|---|---|---|
| Recall | 35/38 (92.1%) | 38/38 (100%) | 38/38 (100%) |
| Meetings / invites | 15/18, 20/20 | 18/18, 20/20 | 18/18, 20/20 |
| Premature | 26 | 0 | 0 |
| False | 5 | 2 | 3 |
| False + premature per judged labeled page | 1.11 | 0.07 | 0.11 |
| Slack events dated after their page | 32 of 299 | 0 of 227 | 0 of 236 |
Dropped date_imprecise / future_dated |
— | 9 / 0 | 6 / 0 |
| Events written | 386 | 303 | 315 |
| Control pages judged | 0 | 0 | 0 |
| Metered cost | $0.5719 | $0.5044 | $0.5112 |
The false events on master: two quarter results on the meeting day, one "back in 2024" on 2024-01-01, and the two the published review already classified. On the branch every false event is a quarter result on the meeting day ("CarbonLoop Q1 revenue reported at $2.1M", "Meridian Labs Q1 numbers came in strong"). Read as statements made at the meeting, each run has one.
In both branch runs the prompt alone kept every proposal on or before its
page's day, so future_dated dropped nothing; the deterministic rule is the
guarantee for models and pages where the prompt does not hold, and the unit
and E2E tests exercise it. date_imprecise dropped 9 and 6 proposals.
Total provider spend: $1.59.
The labeled 31-page fixture from the auto_chronicle fix wave
(capy-fix-wave-chronicle.md) is not committed, so it was not rerun.
gbrain eval chronicle (deterministic): 6/6.