Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

detections(#1108): run the #1062 wrong-and-confident gate through the real _watcher path — it FAILS, published

xerj-orgf573bb96114e
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c9389cdbf..10e2a5739 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
+### Measured
+
+- **The #1062 wrong-and-confident detection gate was run through the real
+ `_watcher` path and it FAILS** (issue
+ [#1108](https://github.com/xerj-org/xerj/issues/1108)) — the unmeasured half
+ of the rc.80 `_watcher` work. On the rc.94 release binary, a real
+ `PUT /_watcher/watch` with `condition.xerj_decide` (one detection watch per
+ label, one-vs-rest) evaluated a 120-email boundary tier and wrote 120 fire
+ records into `.xerj_alert_fires`; the measurement reads only those fires.
+ **Wrong-and-confident = 0.2449** (24 of 98 fires with confidence ≥ 0.8 were
+ wrong) against the **≤ 0.05** bar. The confidence compared is the raw
+ `/_decide` confidence the watcher fires on; `p_cal` was absent (the vote
+ history has no labelled outcome pairs to fit), and because 18 of the 24
+ confident errors sit at raw `p` = 1.000 a monotonic `p_cal` could not change
+ the verdict anyway. Published under the loses-does-not-ship rule: the
+ `_watcher` mechanism is real, but its default vote is not good enough to ship a
+ detection watch as a "confidently correct" default, and no doc claims it is.
+ Raw results, corpus generator and harness in
+ `benchmarks/systemone-watcher-gate/`. Closes #1108.
+
## [1.0.0-rc.94] - 2026-10-11
Window since v1.0.0-rc.93: a wave of external-contributor fixes landed
diff --git a/ROADMAP.md b/ROADMAP.md
index 568c2ace1..2ebf61372 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -305,7 +305,8 @@ panic line). From the cluster-mode testing:
[#1170](https://github.com/xerj-org/xerj/issues/1170) (the Raft ring is
never connected to the engine). Filed from the corpus-program work:
[#1108](https://github.com/xerj-org/xerj/issues/1108) (the #1062
-wrong-and-confident detection gate, unmeasured),
+wrong-and-confident detection gate — now measured on rc.94 and failed
+at 0.2449 vs ≤ 5 %, published; closing on this PR),
[#1110](https://github.com/xerj-org/xerj/issues/1110) (the
generalization proof: second-domain pack + fresh-machine replay) and
[#1183](https://github.com/xerj-org/xerj/issues/1183) (corpus index
@@ -393,7 +394,7 @@ and #1099 were closed by #1095 inside rc.80.
The 1.0 bar: **every public claim verified against the release binary, and every input either honoured or refused loudly.** The gate list, each item an issue:
-**The program to the first official release** ([discussion #1054](https://github.com/xerj-org/xerj/discussions/1054); filed as ten issues indexed in [tracker #1065](https://github.com/xerj-org/xerj/issues/1065)) — **executed whole in rc.80** (rc.79 folded into the cut by direction), every item measured on the release binary and every gate verdict published whether it passed or not. Passed: `xerj_map` ([#1055](https://github.com/xerj-org/xerj/issues/1055), unknown-field 400s 20/30 → 0/30), `/_ask` + `xerj_plan` ([#1056](https://github.com/xerj-org/xerj/issues/1056), macro-F1 0.9975 / p50 1.129 ms), MCP `max_tokens` ([#1058](https://github.com/xerj-org/xerj/issues/1058)), the calibration layer ([#1063](https://github.com/xerj-org/xerj/issues/1063), FiQA pair-level ECE 0.3109 → 0.0088 held-out), and the `xerj-decide` training harness with its measured eval card ([#1064](https://github.com/xerj-org/xerj/issues/1064) — the artifact's Hugging Face publication still PENDING operator credentials; no URL is claimed). Failed, shipped under its own loses-does-not-ship rule with the numbers public: the local judge ([#1060](https://github.com/xerj-org/xerj/issues/1060), −0.1077 SciFact vs hybrid — opt-in, no quality claim) and the flywheel's cache-share bar ([#1061](https://github.com/xerj-org/xerj/issues/1061), the write-back freeze is now defect #1094). Split: stemming ([#1059](https://github.com/xerj-org/xerj/issues/1059), +0.0160/+0.0179 BM25 nDCG@10, zero-hits 25 → 15 against ≤ 10). Real and unmeasured against #1062's wrong-and-confident ≤ 5 % bar: `_watcher` detections ([#1062](https://github.com/xerj-org/xerj/issues/1062) — the evaluate-or-501 half shipped; the detection gate is not yet run). The tier-2 decide head ([#1057](https://github.com/xerj-org/xerj/issues/1057)) missed its SMS/AG News bars and ships behind `--decide-mode local` with the numbers published, arming on the stock binary since #1097. The packaging rule held for all ten: a `benchmarks/<name>` directory with raw results plus [the blog post](https://xerj.org/blog/the-rc80-gates) with the losses left in.
+**The program to the first official release** ([discussion #1054](https://github.com/xerj-org/xerj/discussions/1054); filed as ten issues indexed in [tracker #1065](https://github.com/xerj-org/xerj/issues/1065)) — **executed whole in rc.80** (rc.79 folded into the cut by direction), every item measured on the release binary and every gate verdict published whether it passed or not. Passed: `xerj_map` ([#1055](https://github.com/xerj-org/xerj/issues/1055), unknown-field 400s 20/30 → 0/30), `/_ask` + `xerj_plan` ([#1056](https://github.com/xerj-org/xerj/issues/1056), macro-F1 0.9975 / p50 1.129 ms), MCP `max_tokens` ([#1058](https://github.com/xerj-org/xerj/issues/1058)), the calibration layer ([#1063](https://github.com/xerj-org/xerj/issues/1063), FiQA pair-level ECE 0.3109 → 0.0088 held-out), and the `xerj-decide` training harness with its measured eval card ([#1064](https://github.com/xerj-org/xerj/issues/1064) — the artifact's Hugging Face publication still PENDING operator credentials; no URL is claimed). Failed, shipped under its own loses-does-not-ship rule with the numbers public: the local judge ([#1060](https://github.com/xerj-org/xerj/issues/1060), −0.1077 SciFact vs hybrid — opt-in, no quality claim) and the flywheel's cache-share bar ([#1061](https://github.com/xerj-org/xerj/issues/1061), the write-back freeze is now defect #1094). Split: stemming ([#1059](https://github.com/xerj-org/xerj/issues/1059), +0.0160/+0.0179 BM25 nDCG@10, zero-hits 25 → 15 against ≤ 10). Real and, against #1062's wrong-and-confident ≤ 5 % bar, measured on rc.94 and **failing**: `_watcher` detections ([#1062](https://github.com/xerj-org/xerj/issues/1062) — the evaluate-or-501 half shipped; the detection gate was run through the real watcher path in [#1108](https://github.com/xerj-org/xerj/issues/1108) and scored 0.2449 wrong-and-confident at raw `p` ≥ 0.8 vs the ≤ 5 % bar, published under loses-does-not-ship in `benchmarks/systemone-watcher-gate/`). The tier-2 decide head ([#1057](https://github.com/xerj-org/xerj/issues/1057)) missed its SMS/AG News bars and ships behind `--decide-mode local` with the numbers published, arming on the stock binary since #1097. The packaging rule held for all ten: a `benchmarks/<name>` directory with raw results plus [the blog post](https://xerj.org/blog/the-rc80-gates) with the losses left in.
- **Close the accepted-and-ignored class** (the [#204](https://github.com/xerj-org/xerj/issues/204) umbrella closed once its members carried their own tracking; PR [#258](https://github.com/xerj-org/xerj/pull/258) carried one pass of the sweep and is merged). Known members still open: `nested` `inner_hits` unparsed; `random_sampler`'s ignored `probability`; `weighted_avg` returning HTTP 200 with an error buried in the aggregations body instead of a 400 (the 400 is part of #258). **Retired from this list:** `nested` `score_mode`, which was parsed-then-ignored until [#862](https://github.com/xerj-org/xerj/pull/862) made a nested query roll its matching children's scores into the parent per `score_mode` (rc.71).
- **Security hardening backlog** — cargo-audit and fuzzing landed in CI with rc.16 ([#207](https://github.com/xerj-org/xerj/issues/207) closed); the deferred TLS/auth/symlink hardening items from the Phase-2 security backlog remain.
@@ -420,7 +421,7 @@ The 1.0 bar: **every public claim verified against the release binary, and every
**Stage 2 — planned; the status lines say what exists today, which in most cases is less than the API surface suggests:**
-- **Semantic detections.** A standing question over incoming documents: a cheap `percolate` prefilter selects candidates, a typed judgment decides (a small closed set of outcomes, not free text), and an alert carries a **calibrated probability** — calibrated meaning measured on held-out labelled data and published with its reliability curve, or not shipped. *Since rc.80 ([#1062](https://github.com/xerj-org/xerj/issues/1062), PR [#1082](https://github.com/xerj-org/xerj/pull/1082)): the first real slice.* `PUT /_watcher/watch/{id}` either evaluates a watch on schedule or refuses it with a 501 naming what is not evaluated — the accepted-and-ignored surface this row used to carry is closed. One watch shape runs: an interval trigger, `input.search` over one index, a `condition.xerj_decide` question and an `index_alert` action, with a per-watch background task prefiltering via the watch's own query (keyset paging) and running each rendered question through the real `/_decide`; a fire lands in `.xerj_alert_fires` when the positive label wins non-abstaining at or above `p_min`, carrying the RAW confidence plus `p_cal` when a calibration is fitted ([#1063](https://github.com/xerj-org/xerj/issues/1063)). Ingest-time labels ship in the same release (`xerj autoindex --label <question-set.json>`). Still open on this row: #1062's wrong-and-confident ≤ 5 % detection gate is **not yet measured**; a restart does not resume watches; the console's `.xerj_alert_rules` still has no writer.
+- **Semantic detections.** A standing question over incoming documents: a cheap `percolate` prefilter selects candidates, a typed judgment decides (a small closed set of outcomes, not free text), and an alert carries a **calibrated probability** — calibrated meaning measured on held-out labelled data and published with its reliability curve, or not shipped. *Since rc.80 ([#1062](https://github.com/xerj-org/xerj/issues/1062), PR [#1082](https://github.com/xerj-org/xerj/pull/1082)): the first real slice.* `PUT /_watcher/watch/{id}` either evaluates a watch on schedule or refuses it with a 501 naming what is not evaluated — the accepted-and-ignored surface this row used to carry is closed. One watch shape runs: an interval trigger, `input.search` over one index, a `condition.xerj_decide` question and an `index_alert` action, with a per-watch background task prefiltering via the watch's own query (keyset paging) and running each rendered question through the real `/_decide`; a fire lands in `.xerj_alert_fires` when the positive label wins non-abstaining at or above `p_min`, carrying the RAW confidence plus `p_cal` when a calibration is fitted ([#1063](https://github.com/xerj-org/xerj/issues/1063)). Ingest-time labels ship in the same release (`xerj autoindex --label <question-set.json>`). #1062's wrong-and-confident ≤ 5 % detection gate was **measured on the rc.94 release binary through the real watcher path and it FAILS** ([#1108](https://github.com/xerj-org/xerj/issues/1108), `benchmarks/systemone-watcher-gate/`): on a 120-email boundary tier, wrong-and-confident is **0.2449 at raw `p` ≥ 0.8** against the ≤ 5 % bar — 18 of the 24 confident errors sit at raw `p` = 1.000, so a monotonic `p_cal` cannot rescue the verdict. Published under loses-does-not-ship: the `_watcher` *mechanism* is real, but its default vote is **not** good enough to ship a detection watch as a "confidently correct" default, and no row claims it is. Still open on this row: a restart does not resume watches; the console's `.xerj_alert_rules` still has no writer.
- **A real object-storage backend.** *Done in rc.77:* [#965](https://github.com/xerj-org/xerj/issues/965) wired the segment path to object storage — `storage.backend = "s3"` packs each segment family into one immutable ZBM1 bundle object with `snapshot.json` as the publication point, merges publish before retiring inputs, and a fresh node adopts the bucket (see *Shipping today* for the full statement). The client (`s3.rs`, real S3-compatible: Cloudflare R2, MinIO, AWS S3), the read-through segment cache, per-request cost accounting by billing class and the stop-don't-warn budget had landed in rc.75. Separately, `xerj autoindex s3://bucket/prefix` reads documents OUT of a bucket, which is a source, not a home.
- **A block index mode for logs.** An index mode for log-shaped data that stores rows in time-partitioned columnar blocks and skips whole blocks by their min/max metadata, instead of building a per-term inverted index over every field. *Today:* the `xerj-logs` crate implements that design (columnar encoding, log-template extraction, time-range queries with block skipping, retention), is compiled into `xerj-engine` and `xerj-server` as a dependency, and is **called from no non-test code**. Log-shaped analytics run through the general columnar segment format and the aggregation suite. Wire it behind an explicit index setting and measure it against the general path on the same data, or remove it — the *Log-analytics data path* theme below is this same item.
- **User-code ingest plugins.** *Today:* the ingest pipeline's transforms are built-in native Rust plugins — rename, drop, add, JSON parse, timestamp parse, PII redaction, grok, route — and they do run on `_bulk`. The crate is named `xerj-wasm`, but **the wasmtime backend is not in the tree**: there is no `wasmtime` dependency and no `wasm` feature, only a note that one could be added behind the same trait. Planned: a sandboxed runtime for user-supplied transforms with fuel and memory limits and no ambient filesystem or network access. Until then the refusal rule holds: a pipeline naming a processor this build does not implement is stored as unrunnable and every ingest through it is refused, never quietly run as a shorter pipeline.
diff --git a/benchmarks/systemone-watcher-gate/README.md b/benchmarks/systemone-watcher-gate/README.md
new file mode 100644
index 000000000..13bfd0f45
--- /dev/null
+++ b/benchmarks/systemone-watcher-gate/README.md
@@ -0,0 +1,167 @@
+# #1108 — the wrong-and-confident gate, through the real `_watcher` path
+
+[Issue #1108](https://github.com/xerj-org/xerj/issues/1108) is the standing
+tracker for the **unmeasured half** of the rc.80 `_watcher` work (#1062). #1062
+closed when its *evaluate-or-501* mechanism shipped (PR #1082), but its own
+acceptance gate — **wrong-and-confident <= 5 % at confidence >= 0.8 on the
+classify-email boundary tier** — was never run. This directory runs it and
+publishes the verdict, pass or fail, per the loses-does-not-ship rule.
+
+## Why a new benchmark and not `systemone-classify-email`
+
+`systemone-classify-email` measures the same vote, but it calls `POST /_decide`
+**directly**. #1108 explicitly rejects that: the measurement must go through the
+real product path — *percolate prefilter -> `/_decide` -> fire into
+`.xerj_alert_fires`* — so it tests what a deployed detection watch actually does
+(its prefilter, paging and fire semantics), not a bare vote. This harness never
+calls `/_decide` for the gate decision; the fire records the node's own watcher
+evaluator writes are the sole input to the number.
+
+It also fixes the other thing that made the old number unusable as a gate:
+**n**. The hand-authored "hard" tier is 8 emails — one miss is 12.5 %, so it
+cannot resolve a 5 % line. `gen_boundary.py` scales the *same four
+boundary-crossing patterns* (see that file's header) to **120 original,
+template-generated emails** (30 per label), labelled by construction.
+
+## Method
+
+- **History (the vote corpus):** the 200-email train split of
+ `systemone-classify-email/corpus.json`, reused verbatim, indexed into
+ `email-history` (`label` keyword, `text` text, default **lexical** embedder —
+ no `--embed-mode neural`).
+- **Watches:** one `PUT /_watcher/watch/{id}` per label (one-vs-rest). Each is a
+ real detection watch: `trigger.schedule.interval 1s`, `input.search` over the
+ empty `mail-incoming` index, `condition.xerj_decide` with that label as
+ `positive_label`, `text_field "text"`, `p_min 0.5`, `index "email-history"`,
+ `k 10`. A detection watch detects **one** positive label; four of them cover
+ the four-label tier.
+- **Candidates:** the 120 boundary emails, indexed into `mail-incoming`
+ **after** the watches are PUT (the watch baselines at the index's max `_seq_no`
+ at PUT time, so only later documents are judged). Each carries its truth label
+ in `gold`.
+- **The number:** read every record in `.xerj_alert_fires`, map `doc_id` to the
+ candidate's `gold`, and compute **wrong-and-confident = fires whose true label
+ is not the fired positive label, among fires with confidence >= 0.8**. A
+ candidate fires at most once — under the watch matching its vote argmax, when
+ the raw confidence clears `p_min`.
+- **Which confidence is compared at >= 0.8:** the watcher fires on, and records,
+ the **raw** `/_decide` confidence as `p` (`p_min` is a raw threshold by
+ design — `es_compat.rs`). It also carries `p_cal` **when the node has a fitted
+ calibration**. The run config sets `[decisions] calibration = isotonic`; the
+ README reports whether `p_cal` was present and, if so, the rate on it too.
+
+## Reproduce
+
+```sh
+# built release binary (this run used v1.0.0-rc.94, release profile)
+xerj -d /tmp/gate-data --insecure --port 9471 -c gate.toml # background; wait for green
+cd benchmarks/systemone-watcher-gate
+python3 gen_boundary.py
+XERJ_URL=http://localhost:9471 python3 watcher_gate.py
+```
+
+`gate.toml` is the `[decisions]` block above (index `email-history`, k 10,
+`calibration = isotonic`).
+
+## Result
+
+Run 2026-10-11, locally built `xerj v1.0.0-rc.94` (release profile), booted with
+`--insecure --port 9471 -c gate.toml` on a throwaway data dir. Raw per-fire
+output: [`results/run.json`](./results/run.json); full console transcript:
+[`results/transcript.txt`](./results/transcript.txt). The 120 fire records were
+written by the node's own watcher evaluator into `.xerj_alert_fires`; this
+harness read them back and did nothing else.
+
+| | value |
+|---|---:|
+| candidates evaluated (boundary tier) | 120 |
+| fires (argmax cleared `p_min` 0.5) | 120 |
+| fired correctly | 94 / 120 = **0.783** |
+| calibration on fires | **none** — `p_cal` absent, number is on raw `p` |
+| confident fires (raw `p` >= 0.8) | 98 |
+| wrong **and** confident | 24 |
+| **wrong-and-confident rate** | **24 / 98 = 0.2449** |
+
+**The number, defined once:** wrong-and-confident = fires whose true label is
+not the fired positive label, among fires with confidence >= 0.8 = **0.2449**.
+The confidence compared at >= 0.8 is the **raw** `/_decide` confidence `p` — the
+value the watcher fires on and records. `p_cal` is absent: `[decisions]
+calibration = isotonic` was set, but the vote-history index carries no labelled
+**outcome** pairs (a `p` served + a truth label, no `source`), so the node
+reports `p_cal: null` with its reason (see `systemone_api.rs` `unfit_reason`) and
+the fire records carry `calibrated: false`.
+
+Where the mechanism is wrong, from the confusion matrix (`results/run.json`):
+
+| gold | fired correctly | confidently mis-fired as |
+|---|---:|---|
+| personal | 30 / 30 | — |
+| newsletter | 30 / 30 | — |
+| needs-reply | 21 / 30 | 9 → newsletter |
+| invoice | 13 / 30 | 11 → personal, 6 → needs-reply |
+
+Every one of the 24 wrong-and-confident fires is at raw `p` between **0.871 and
+1.000**, 18 of them at exactly **1.000**. The errors are the two boundary
+patterns the hand-authored "hard" tier already flagged, now at scale: an invoice
+with no invoice vocabulary ("did the transfer land", "wrapping up the offsite
+numbers") voted `personal`/`needs-reply`, and a needs-reply buried in a
+status-update rundown voted `newsletter`. The vote is a BM25-weighted k-NN over
+labelled history; a boundary email that shares no vocabulary with its own label's
+training examples is voted into whichever *other* label it most resembles — and
+because it clearly resembles that other label, the vote share (the confidence) is
+high. This is the discussion-#1012 caveat and the `systemone-classify-email`
+hard-tier finding reproduced through the real watcher path.
+
+## Verdict
+
+**FAIL — the gate does not pass, and per the loses-does-not-ship rule the number
+is published and the loss named.** Wrong-and-confident is **0.2449** at raw
+`p >= 0.8` against the **<= 0.05** bar. The `_watcher` *mechanism* is real and
+correct (evaluate-or-501, one real watch shape, fires into `.xerj_alert_fires`
+carrying the raw `p` and `p_cal` when fitted — all true), but its default-vote
+**quality** on a boundary tier is not good enough to ship a detection watch as a
+"confidently correct" default.
+
+**Would a fitted `p_cal` change the verdict? No, and here is why.** The #1063
+calibration is a single monotonic (isotonic) map from raw `p` to `p_cal`. It
+cannot separate the right fires from the wrong fires when both sit at raw
+`p = 1.000`: it maps that one input to one output. 18 of the 24 wrong fires are
+at raw `p = 1.000`, alongside many correct fires at `p = 1.000`; any monotonic
+map that pushes `p = 1.000` below 0.8 to clear those 18 also pushes the correct
+`p = 1.000` fires below 0.8, destroying the detector instead of fixing it.
+Per-example overconfidence on out-of-distribution inputs is a property of the
+*vote*, not of the confidence scale, so a global calibration — the kind #1063
+fits — does not rescue this gate. (Where calibration *is* measured and does
+help — pair-level reliability — is #1063's FiQA ECE 0.3109 → 0.0088, a different
+quantity: how well a probability matches an empirical frequency *in aggregate*,
+not whether a single confident fire is right.)
+
+**What this does and does not close.** It closes #1108's ask — the gate is now
+run through the real product path on the release binary and the verdict is
+published with raw results. It does **not** make the detection watch a
+ship-as-default "confidently correct" feature; the honest operator guidance is
+unchanged from `systemone-classify-email`: tune `decisions.min_confidence` and
+`p_min` against a held-out set shaped like the real traffic, and do not read the
+raw vote confidence as a trustworthy abstain signal on inputs unlike the labelled
+history. A better gate would need either an abstain signal that is not the vote
+share (e.g. a distance-to-history or disagreement-among-neighbours score) or a
+richer-than-lexical representation so boundary emails stop resembling the wrong
+label — both are engine work beyond this measurement.
+
+### Notes on faithfulness to #1108's acceptance criteria
+
+- **Release binary, not a dev build:** rc.94, release profile. #1108 names rc.80
+ because that is when it was filed; the `_watcher` + `p_cal` mechanism is
+ unchanged since rc.80, and rc.94 is the current release line. Stated, not
+ hidden.
+- **Real product path, not a `/_decide`-direct harness:** satisfied — the fires
+ are written by the node's own per-watch evaluator; the harness only reads
+ `.xerj_alert_fires`.
+- **Ingest-time labels:** the candidates are bulk-indexed carrying their
+ **ground-truth** label in `gold` (the labelled-document shape PR #1082
+ evaluates), *not* via `xerj autoindex --label`. `--label` attaches the model's
+ own *predicted* label, which cannot be the truth a gate scores against — so for
+ a gate the truth has to be stored independently. The watch decision itself
+ still runs through the full watcher path.
+- **Raw outputs in `benchmarks/`:** `results/run.json` + `results/transcript.txt`,
+ reproducible from the commands above.
diff --git a/benchmarks/systemone-watcher-gate/boundary.json b/benchmarks/systemone-watcher-gate/boundary.json
new file mode 100644
index 000000000..f73afba25
--- /dev/null
+++ b/benchmarks/systemone-watcher-gate/boundary.json
@@ -0,0 +1,722 @@
+[
+ {
+ "label": "personal",
+ "subject": "quick one",
+ "text": "Noor! Sorting out the group thing \u2014 your share works out to about $383. Pay me back this week or just cover the next round, whatever's easier. Miss you, let's actually hang soon.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "hey, free this weekend?",
+ "text": "Hey Rafe, free this weekend? Also \u2014 still owe you for the concert tickets, want to say $3,627? Let me know what works, cash or Venmo when I see you.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "this week at Greenfield Robotics",
+ "text": "This week's digest: three things we shipped and a customer story worth reading. One ask before you go \u2014 which feature should we cover next issue? Just reply with a word. We read every response.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "this week at Ridgeway Consulting",
+ "text": "This week's digest: three things we shipped and a customer story worth reading. One ask before you go \u2014 which feature should we cover next issue? Just reply with a word. We read every response.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Northwind Supply \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount tomorrow or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "before Friday's launch",
+ "text": "Quick note from the team at Bluepeak Logistics before Friday: we could use your take \u2014 reply and tell us which tagline you'd run with, A or B. We publish the results in next week's issue either way, so no wrong answer.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Cedarline Studio \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount this week or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "a customer story worth reading",
+ "text": "Our weekly from Fenwick Cloud: the usual roundup, plus a short piece on how a customer cut their costs. Curious \u2014 would a deep-dive version be useful to you? Hit reply and let us know, it shapes what we write next.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $4,506 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "what changed in the release",
+ "text": "Here's what changed this week at Voss & Rane. Reading time about four minutes. Before you unsubscribe or archive \u2014 one question: did the new layout work for you on mobile? Reply yes or no, it genuinely helps.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Ridgeway Consulting \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount before month end or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "before Friday's launch",
+ "text": "Quick note from the team at Greenfield Robotics before Friday: we could use your take \u2014 reply and tell us which tagline you'd run with, A or B. We publish the results in next week's issue either way, so no wrong answer.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "what changed in the release",
+ "text": "Here's what changed this week at Harborline Freight. Reading time about four minutes. Before you unsubscribe or archive \u2014 one question: did the new layout work for you on mobile? Reply yes or no, it genuinely helps.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Sara, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $1,716, the reference was on the packing slip. No rush, just reconciling the books by Friday.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "this week at Fenwick Cloud",
+ "text": "This week's digest: three things we shipped and a customer story worth reading. One ask before you go \u2014 which feature should we cover next issue? Just reply with a word. We read every response.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "for the offsite",
+ "text": "Wrapping up the offsite numbers. Catering came to $1,032, venue was a bit more. Same split as always \u2014 tell accounting this week if your card wasn't the one charged so we can true it up.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "This week at Northwind Supply: two quick things",
+ "text": "Hi \u2014 this week's rundown from the studio: the mockups are attached and we shipped the new brand kit. One thing before Friday though: I do need you to confirm the print run number, otherwise the press slot slips.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $4,373 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "this week at Harborline Freight",
+ "text": "This week's digest: three things we shipped and a customer story worth reading. One ask before you go \u2014 which feature should we cover next issue? Just reply with a word. We read every response.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Harborline Freight \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount by end of day or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "one thing before I book this",
+ "text": "Before I book the freight slot I need a yes or no on the pallet count \u2014 24 or 32? The vendor holds the rate until 5pm tomorrow, after that it resets and we lose the discount, so whenever you get a sec.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "a customer story worth reading",
+ "text": "Our weekly from Northwind Supply: the usual roundup, plus a short piece on how a customer cut their costs. Curious \u2014 would a deep-dive version be useful to you? Hit reply and let us know, it shapes what we write next.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Ola, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $4,557, the reference was on the packing slip. No rush, just reconciling the books by end of day.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "hey, free this weekend?",
+ "text": "Hey Ola, free this weekend? Also \u2014 still owe you for the concert tickets, want to say $880? Let me know what works, cash or Venmo when I see you.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "this week at Cedarline Studio",
+ "text": "This week's digest: three things we shipped and a customer story worth reading. One ask before you go \u2014 which feature should we cover next issue? Just reply with a word. We read every response.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "weekly update",
+ "text": "Rundown: shipping is on schedule, the invoice cleared, support queue is quiet. The one thing I can't move without you \u2014 do we go with vendor A or B? I need that call tomorrow to lock the rate.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "reconciling",
+ "text": "Going through the books by end of day. I have $346 against the pallet job but no matching record on your side \u2014 did we ever settle that, or is it still open? Reference should be on the slip.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "for the offsite",
+ "text": "Wrapping up the offsite numbers. Catering came to $4,690, venue was a bit more. Same split as always \u2014 tell accounting by Friday if your card wasn't the one charged so we can true it up.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "for the offsite",
+ "text": "Wrapping up the offsite numbers. Catering came to $536, venue was a bit more. Same split as always \u2014 tell accounting before month end if your card wasn't the one charged so we can true it up.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Mei, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $4,568, the reference was on the packing slip. No rush, just reconciling the books by Friday.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "owe you for gas",
+ "text": "Hey! Can you send me $1,816 whenever for the gas we split on the drive back? Venmo's fine, or just get me before month end next time you're over. No rush at all.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "hey, free this weekend?",
+ "text": "Hey Owen, free this weekend? Also \u2014 still owe you for the concert tickets, want to say $292? Let me know what works, cash or Venmo when I see you.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "the transfer",
+ "text": "Hi Luca \u2014 can you confirm the $3,911 went out before month end? The account on file changed last quarter and I want to be sure it landed in the right one before I close out the month.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "this week at Bluepeak Logistics",
+ "text": "This week's digest: three things we shipped and a customer story worth reading. One ask before you go \u2014 which feature should we cover next issue? Just reply with a word. We read every response.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $1,442 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "a customer story worth reading",
+ "text": "Our weekly from Harborline Freight: the usual roundup, plus a short piece on how a customer cut their costs. Curious \u2014 would a deep-dive version be useful to you? Hit reply and let us know, it shapes what we write next.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "reconciling",
+ "text": "Going through the books this afternoon. I have $2,261 against the pallet job but no matching record on your side \u2014 did we ever settle that, or is it still open? Reference should be on the slip.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "what changed in the release",
+ "text": "Here's what changed this week at Bluepeak Logistics. Reading time about four minutes. Before you unsubscribe or archive \u2014 one question: did the new layout work for you on mobile? Reply yes or no, it genuinely helps.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "quick one",
+ "text": "Luca! Sorting out the group thing \u2014 your share works out to about $3,676. Pay me back before month end or just cover the next round, whatever's easier. Miss you, let's actually hang soon.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "the transfer",
+ "text": "Hi Alex \u2014 can you confirm the $4,370 went out by the 15th? The account on file changed last quarter and I want to be sure it landed in the right one before I close out the month.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "reconciling",
+ "text": "Going through the books tomorrow. I have $810 against the pallet job but no matching record on your side \u2014 did we ever settle that, or is it still open? Reference should be on the slip.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "weekly update",
+ "text": "Rundown: shipping is on schedule, the invoice cleared, support queue is quiet. The one thing I can't move without you \u2014 do we go with vendor A or B? I need that call by end of day to lock the rate.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Tom\u00e1s, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $3,770, the reference was on the packing slip. No rush, just reconciling the books before month end.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "weekly update",
+ "text": "Rundown: shipping is on schedule, the invoice cleared, support queue is quiet. The one thing I can't move without you \u2014 do we go with vendor A or B? I need that call by Friday to lock the rate.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "reconciling",
+ "text": "Going through the books this week. I have $1,949 against the pallet job but no matching record on your side \u2014 did we ever settle that, or is it still open? Reference should be on the slip.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "what changed in the release",
+ "text": "Here's what changed this week at Northwind Supply. Reading time about four minutes. Before you unsubscribe or archive \u2014 one question: did the new layout work for you on mobile? Reply yes or no, it genuinely helps.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "weekly update",
+ "text": "Rundown: shipping is on schedule, the invoice cleared, support queue is quiet. The one thing I can't move without you \u2014 do we go with vendor A or B? I need that call this week to lock the rate.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "quick one",
+ "text": "Esme! Sorting out the group thing \u2014 your share works out to about $3,839. Pay me back this week or just cover the next round, whatever's easier. Miss you, let's actually hang soon.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "before Friday's launch",
+ "text": "Quick note from the team at Ridgeway Consulting before Friday: we could use your take \u2014 reply and tell us which tagline you'd run with, A or B. We publish the results in next week's issue either way, so no wrong answer.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Cedarline Studio \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount this afternoon or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "one thing before I book this",
+ "text": "Before I book the freight slot I need a yes or no on the pallet count \u2014 24 or 32? The vendor holds the rate until 5pm before month end, after that it resets and we lose the discount, so whenever you get a sec.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "This week at Bluepeak Logistics: two quick things",
+ "text": "Hi \u2014 this week's rundown from the studio: the mockups are attached and we shipped the new brand kit. One thing before Friday though: I do need you to confirm the print run number, otherwise the press slot slips.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "owe you for gas",
+ "text": "Hey! Can you send me $2,115 whenever for the gas we split on the drive back? Venmo's fine, or just get me today next time you're over. No rush at all.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "quick one",
+ "text": "Noor! Sorting out the group thing \u2014 your share works out to about $1,065. Pay me back by Friday or just cover the next round, whatever's easier. Miss you, let's actually hang soon.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Fenwick Cloud \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount today or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Owen, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $503, the reference was on the packing slip. No rush, just reconciling the books tomorrow.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "the transfer",
+ "text": "Hi Derek \u2014 can you confirm the $2,655 went out by Friday? The account on file changed last quarter and I want to be sure it landed in the right one before I close out the month.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "hey, free this weekend?",
+ "text": "Hey Ola, free this weekend? Also \u2014 still owe you for the concert tickets, want to say $2,828? Let me know what works, cash or Venmo when I see you.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "quick one",
+ "text": "Hana! Sorting out the group thing \u2014 your share works out to about $1,078. Pay me back by the 15th or just cover the next round, whatever's easier. Miss you, let's actually hang soon.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "quick one",
+ "text": "Noor! Sorting out the group thing \u2014 your share works out to about $3,985. Pay me back by Friday or just cover the next round, whatever's easier. Miss you, let's actually hang soon.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "the transfer",
+ "text": "Hi Mei \u2014 can you confirm the $1,647 went out tomorrow? The account on file changed last quarter and I want to be sure it landed in the right one before I close out the month.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Mei, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $1,435, the reference was on the packing slip. No rush, just reconciling the books this afternoon.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "what changed in the release",
+ "text": "Here's what changed this week at Ridgeway Consulting. Reading time about four minutes. Before you unsubscribe or archive \u2014 one question: did the new layout work for you on mobile? Reply yes or no, it genuinely helps.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "before Friday's launch",
+ "text": "Quick note from the team at Northwind Supply before Friday: we could use your take \u2014 reply and tell us which tagline you'd run with, A or B. We publish the results in next week's issue either way, so no wrong answer.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "what changed in the release",
+ "text": "Here's what changed this week at Marlowe Analytics. Reading time about four minutes. Before you unsubscribe or archive \u2014 one question: did the new layout work for you on mobile? Reply yes or no, it genuinely helps.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Noor, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $1,107, the reference was on the packing slip. No rush, just reconciling the books tomorrow.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Priya, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $1,753, the reference was on the packing slip. No rush, just reconciling the books this week.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "quick one",
+ "text": "Alex! Sorting out the group thing \u2014 your share works out to about $2,478. Pay me back before month end or just cover the next round, whatever's easier. Miss you, let's actually hang soon.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "before Friday's launch",
+ "text": "Quick note from the team at Cedarline Studio before Friday: we could use your take \u2014 reply and tell us which tagline you'd run with, A or B. We publish the results in next week's issue either way, so no wrong answer.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "weekly update",
+ "text": "Rundown: shipping is on schedule, the invoice cleared, support queue is quiet. The one thing I can't move without you \u2014 do we go with vendor A or B? I need that call today to lock the rate.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "quick one",
+ "text": "Tom\u00e1s! Sorting out the group thing \u2014 your share works out to about $967. Pay me back tomorrow or just cover the next round, whatever's easier. Miss you, let's actually hang soon.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "This week at Fenwick Cloud: two quick things",
+ "text": "Hi \u2014 this week's rundown from the studio: the mockups are attached and we shipped the new brand kit. One thing before Friday though: I do need you to confirm the print run number, otherwise the press slot slips.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Ola, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $3,384, the reference was on the packing slip. No rush, just reconciling the books this week.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "for the offsite",
+ "text": "Wrapping up the offsite numbers. Catering came to $4,375, venue was a bit more. Same split as always \u2014 tell accounting this week if your card wasn't the one charged so we can true it up.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "owe you for gas",
+ "text": "Hey! Can you send me $2,116 whenever for the gas we split on the drive back? Venmo's fine, or just get me by the 15th next time you're over. No rush at all.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "quick one",
+ "text": "Tom\u00e1s! Sorting out the group thing \u2014 your share works out to about $370. Pay me back today or just cover the next round, whatever's easier. Miss you, let's actually hang soon.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "one thing before I book this",
+ "text": "Before I book the freight slot I need a yes or no on the pallet count \u2014 24 or 32? The vendor holds the rate until 5pm by end of day, after that it resets and we lose the discount, so whenever you get a sec.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "before Friday's launch",
+ "text": "Quick note from the team at Marlowe Analytics before Friday: we could use your take \u2014 reply and tell us which tagline you'd run with, A or B. We publish the results in next week's issue either way, so no wrong answer.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "for the offsite",
+ "text": "Wrapping up the offsite numbers. Catering came to $2,420, venue was a bit more. Same split as always \u2014 tell accounting before month end if your card wasn't the one charged so we can true it up.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Fenwick Cloud \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount by end of day or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $1,361 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "one thing before I book this",
+ "text": "Before I book the freight slot I need a yes or no on the pallet count \u2014 24 or 32? The vendor holds the rate until 5pm by the 15th, after that it resets and we lose the discount, so whenever you get a sec.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "reconciling",
+ "text": "Going through the books before month end. I have $3,173 against the pallet job but no matching record on your side \u2014 did we ever settle that, or is it still open? Reference should be on the slip.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Marlowe Analytics \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount this afternoon or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "hey, free this weekend?",
+ "text": "Hey Sara, free this weekend? Also \u2014 still owe you for the concert tickets, want to say $4,452? Let me know what works, cash or Venmo when I see you.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $3,969 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $766 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Cedarline Studio \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount by Friday or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "what changed in the release",
+ "text": "Here's what changed this week at Fenwick Cloud. Reading time about four minutes. Before you unsubscribe or archive \u2014 one question: did the new layout work for you on mobile? Reply yes or no, it genuinely helps.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $4,699 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "owe you for gas",
+ "text": "Hey! Can you send me $4,369 whenever for the gas we split on the drive back? Venmo's fine, or just get me by the 15th next time you're over. No rush at all.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "one thing before I book this",
+ "text": "Before I book the freight slot I need a yes or no on the pallet count \u2014 24 or 32? The vendor holds the rate until 5pm this afternoon, after that it resets and we lose the discount, so whenever you get a sec.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "weekly update",
+ "text": "Rundown: shipping is on schedule, the invoice cleared, support queue is quiet. The one thing I can't move without you \u2014 do we go with vendor A or B? I need that call this afternoon to lock the rate.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "before Friday's launch",
+ "text": "Quick note from the team at Voss & Rane before Friday: we could use your take \u2014 reply and tell us which tagline you'd run with, A or B. We publish the results in next week's issue either way, so no wrong answer.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "for the offsite",
+ "text": "Wrapping up the offsite numbers. Catering came to $2,126, venue was a bit more. Same split as always \u2014 tell accounting this afternoon if your card wasn't the one charged so we can true it up.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "This week at Greenfield Robotics: two quick things",
+ "text": "Hi \u2014 this week's rundown from the studio: the mockups are attached and we shipped the new brand kit. One thing before Friday though: I do need you to confirm the print run number, otherwise the press slot slips.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "This week at Harborline Freight: two quick things",
+ "text": "Hi \u2014 this week's rundown from the studio: the mockups are attached and we shipped the new brand kit. One thing before Friday though: I do need you to confirm the print run number, otherwise the press slot slips.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "this week at Atlas Facilities",
+ "text": "This week's digest: three things we shipped and a customer story worth reading. One ask before you go \u2014 which feature should we cover next issue? Just reply with a word. We read every response.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "reconciling",
+ "text": "Going through the books tomorrow. I have $1,408 against the pallet job but no matching record on your side \u2014 did we ever settle that, or is it still open? Reference should be on the slip.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "reconciling",
+ "text": "Going through the books tomorrow. I have $2,386 against the pallet job but no matching record on your side \u2014 did we ever settle that, or is it still open? Reference should be on the slip.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "reconciling",
+ "text": "Going through the books by the 15th. I have $1,588 against the pallet job but no matching record on your side \u2014 did we ever settle that, or is it still open? Reference should be on the slip.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "reconciling",
+ "text": "Going through the books this week. I have $3,070 against the pallet job but no matching record on your side \u2014 did we ever settle that, or is it still open? Reference should be on the slip.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "a customer story worth reading",
+ "text": "Our weekly from Cedarline Studio: the usual roundup, plus a short piece on how a customer cut their costs. Curious \u2014 would a deep-dive version be useful to you? Hit reply and let us know, it shapes what we write next.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Harborline Freight \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount by Friday or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "quick one",
+ "text": "Sara! Sorting out the group thing \u2014 your share works out to about $3,155. Pay me back by end of day or just cover the next round, whatever's easier. Miss you, let's actually hang soon.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Mei, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $3,254, the reference was on the packing slip. No rush, just reconciling the books this week.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $750 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ },
+ {
+ "label": "invoice",
+ "subject": "did that ever go through?",
+ "text": "Hey Luca, quick check \u2014 did the transfer for last month's freight land yet? Should have been around $2,703, the reference was on the packing slip. No rush, just reconciling the books this week.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Harborline Freight \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount tomorrow or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "a customer story worth reading",
+ "text": "Our weekly from Greenfield Robotics: the usual roundup, plus a short piece on how a customer cut their costs. Curious \u2014 would a deep-dive version be useful to you? Hit reply and let us know, it shapes what we write next.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Voss & Rane \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount by end of day or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "a customer story worth reading",
+ "text": "Our weekly from Atlas Facilities: the usual roundup, plus a short piece on how a customer cut their costs. Curious \u2014 would a deep-dive version be useful to you? Hit reply and let us know, it shapes what we write next.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $3,383 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "status + one ask",
+ "text": "Everything's on track at Greenfield Robotics \u2014 design signed off, copy is in review. The only blocker is you: I need the final headcount by end of day or the caterer won't hold our slot. Can you send it over?",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $385 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "a customer story worth reading",
+ "text": "Our weekly from Marlowe Analytics: the usual roundup, plus a short piece on how a customer cut their costs. Curious \u2014 would a deep-dive version be useful to you? Hit reply and let us know, it shapes what we write next.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "this week at Marlowe Analytics",
+ "text": "This week's digest: three things we shipped and a customer story worth reading. One ask before you go \u2014 which feature should we cover next issue? Just reply with a word. We read every response.",
+ "split": "boundary"
+ },
+ {
+ "label": "newsletter",
+ "subject": "what changed in the release",
+ "text": "Here's what changed this week at Atlas Facilities. Reading time about four minutes. Before you unsubscribe or archive \u2014 one question: did the new layout work for you on mobile? Reply yes or no, it genuinely helps.",
+ "split": "boundary"
+ },
+ {
+ "label": "needs-reply",
+ "subject": "This week at Atlas Facilities: two quick things",
+ "text": "Hi \u2014 this week's rundown from the studio: the mockups are attached and we shipped the new brand kit. One thing before Friday though: I do need you to confirm the print run number, otherwise the press slot slips.",
+ "split": "boundary"
+ },
+ {
+ "label": "personal",
+ "subject": "this weekend",
+ "text": "Was thinking we could grab dinner this weekend. Oh and I'll bring the $3,439 I owe you from the trip \u2014 completely forgot until now. No pressure either way, just wanted to catch up.",
+ "split": "boundary"
+ }
+]
\ No newline at end of file
diff --git a/benchmarks/systemone-watcher-gate/gate.toml b/benchmarks/systemone-watcher-gate/gate.toml
new file mode 100644
index 000000000..198fbe4e5
--- /dev/null
+++ b/benchmarks/systemone-watcher-gate/gate.toml
@@ -0,0 +1,11 @@
+# Private config for the #1108 watcher-gate measurement. Calibration is turned
+# ON so that any fitted p_cal rides onto fire records; the watch still fires on
+# the RAW confidence (p_min is a raw threshold by design).
+[decisions]
+index = "email-history"
+k = 10
+label_field = "label"
+text_field = "text"
+positive_label = "needs-reply"
+min_confidence = 0.0
+calibration = "isotonic"
diff --git a/benchmarks/systemone-watcher-gate/gen_boundary.py b/benchmarks/systemone-watcher-gate/gen_boundary.py
new file mode 100644
index 000000000..47e37d783
--- /dev/null
+++ b/benchmarks/systemone-watcher-gate/gen_boundary.py
@@ -0,0 +1,179 @@
+"""Boundary-crossing email corpus for the #1108 wrong-and-confident gate.
+
+The `systemone-classify-email` benchmark already measured the local `/_decide`
+vote on a hand-authored "hard" tier of 8 boundary-crossing emails and found it
+wrong AND confident on the two `invoice` cases (0.836-1.000). Eight examples
+cannot resolve a 5 % rate: one miss is 12.5 %. #1108's gate is "wrong-and-confident
+<= 5 % at p >= 0.8" on the classify-email boundary tier, so the tier has to be
+big enough to put that 5 % line somewhere a measurement can land on either side.
+
+This script scales the SAME four boundary patterns the hand-authored tier used
+(see `systemone-classify-email/gen_corpus.py`'s HARD list) up to a deterministic
+set of ORIGINAL, template-generated emails — no real mailbox, no third-party
+dataset, no licence or privacy question. The labels are ground truth BY
+CONSTRUCTION (each generator emits exactly one class), which is what lets a fire
+be scored right-or-wrong. The four patterns, each borrowing another class's
+surface vocabulary on purpose:
+
+ invoice -> money/billing phrased casually, the word "invoice" never used
+ ("did the transfer land", "wrapping up the offsite numbers")
+ personal -> a casual note that happens to name a dollar figure
+ ("owe you for gas, send $40 whenever")
+ newsletter -> a digest that ends by asking the reader to reply / vote
+ needs-reply -> a required confirmation buried inside a status-update rundown
+
+History (the labelled vote corpus the watch decides against) is the 200-email
+train split of `systemone-classify-email/corpus.json`, reused verbatim so the
+two benchmarks share one vote corpus. This script writes ONLY the boundary tier.
+
+Deterministic (seed 1108): rerunning reproduces boundary.json byte for byte.
+
+Usage: python3 gen_boundary.py -> writes boundary.json
+"""
+import json
+import random
+
+random.seed(1108)
+
+PER_LABEL = 30 # 4 labels -> 120 boundary emails
+
+FIRST = ["Alex", "Priya", "Jonas", "Mei", "Sara", "Tomás", "Nina", "Ola",
+ "Derek", "Fatima", "Luca", "Hana", "Owen", "Ingrid", "Kwame", "Yuki",
+ "Rafe", "Noor", "Pavel", "Esme"]
+COMPANIES = ["Northwind Supply", "Bluepeak Logistics", "Harborline Freight",
+ "Cedarline Studio", "Voss & Rane", "Marlowe Analytics",
+ "Fenwick Cloud", "Atlas Facilities", "Ridgeway Consulting",
+ "Greenfield Robotics"]
+
+
+def amount():
+ return f"${random.randint(35, 4800):,}"
+
+
+def when():
+ return random.choice(["today", "by Friday", "this week", "before month end",
+ "by end of day", "tomorrow", "by the 15th", "this afternoon"])
+
+
+# ── invoice, with no invoice vocabulary ──────────────────────────────────
+def invoice(i):
+ name = random.choice(FIRST)
+ a = amount()
+ variants = [
+ (f"did that ever go through?",
+ f"Hey {name}, quick check — did the transfer for last month's freight "
+ f"land yet? Should have been around {a}, the reference was on the packing "
+ f"slip. No rush, just reconciling the books {when()}."),
+ (f"for the offsite",
+ f"Wrapping up the offsite numbers. Catering came to {a}, venue was a bit "
+ f"more. Same split as always — tell accounting {when()} if your card "
+ f"wasn't the one charged so we can true it up."),
+ (f"the transfer",
+ f"Hi {name} — can you confirm the {a} went out {when()}? The account on "
+ f"file changed last quarter and I want to be sure it landed in the right "
+ f"one before I close out the month."),
+ (f"reconciling",
+ f"Going through the books {when()}. I have {a} against the pallet job but "
+ f"no matching record on your side — did we ever settle that, or is it "
+ f"still open? Reference should be on the slip."),
+ ]
+ return random.choice(variants)
+
+
+# ── personal, with a dollar figure ───────────────────────────────────────
+def personal(i):
+ name = random.choice(FIRST)
+ a = amount()
+ variants = [
+ (f"owe you for gas",
+ f"Hey! Can you send me {a} whenever for the gas we split on the drive "
+ f"back? Venmo's fine, or just get me {when()} next time you're over. No "
+ f"rush at all."),
+ (f"hey, free this weekend?",
+ f"Hey {name}, free this weekend? Also — still owe you for the concert "
+ f"tickets, want to say {a}? Let me know what works, cash or Venmo when I "
+ f"see you."),
+ (f"this weekend",
+ f"Was thinking we could grab dinner this weekend. Oh and I'll bring the "
+ f"{a} I owe you from the trip — completely forgot until now. No pressure "
+ f"either way, just wanted to catch up."),
+ (f"quick one",
+ f"{name}! Sorting out the group thing — your share works out to about "
+ f"{a}. Pay me back {when()} or just cover the next round, whatever's "
+ f"easier. Miss you, let's actually hang soon."),
+ ]
+ return random.choice(variants)
+
+
+# ── newsletter, ending in a direct question ──────────────────────────────
+def newsletter(i):
+ co = random.choice(COMPANIES)
+ variants = [
+ (f"before Friday's launch",
+ f"Quick note from the team at {co} before Friday: we could use your take "
+ f"— reply and tell us which tagline you'd run with, A or B. We publish "
+ f"the results in next week's issue either way, so no wrong answer."),
+ (f"this week at {co}",
+ f"This week's digest: three things we shipped and a customer story worth "
+ f"reading. One ask before you go — which feature should we cover next "
+ f"issue? Just reply with a word. We read every response."),
+ (f"a customer story worth reading",
+ f"Our weekly from {co}: the usual roundup, plus a short piece on how a "
+ f"customer cut their costs. Curious — would a deep-dive version be useful "
+ f"to you? Hit reply and let us know, it shapes what we write next."),
+ (f"what changed in the release",
+ f"Here's what changed this week at {co}. Reading time about four minutes. "
+ f"Before you unsubscribe or archive — one question: did the new layout "
+ f"work for you on mobile? Reply yes or no, it genuinely helps."),
+ ]
+ return random.choice(variants)
+
+
+# ── needs-reply, buried in a status-update rundown ───────────────────────
+def needs_reply(i):
+ co = random.choice(COMPANIES)
+ variants = [
+ (f"This week at {co}: two quick things",
+ f"Hi — this week's rundown from the studio: the mockups are attached and "
+ f"we shipped the new brand kit. One thing before Friday though: I do need "
+ f"you to confirm the print run number, otherwise the press slot slips."),
+ (f"one thing before I book this",
+ f"Before I book the freight slot I need a yes or no on the pallet count — "
+ f"24 or 32? The vendor holds the rate until 5pm {when()}, after that it "
+ f"resets and we lose the discount, so whenever you get a sec."),
+ (f"status + one ask",
+ f"Everything's on track at {co} — design signed off, copy is in review. "
+ f"The only blocker is you: I need the final headcount {when()} or the "
+ f"caterer won't hold our slot. Can you send it over?"),
+ (f"weekly update",
+ f"Rundown: shipping is on schedule, the invoice cleared, support queue is "
+ f"quiet. The one thing I can't move without you — do we go with vendor A "
+ f"or B? I need that call {when()} to lock the rate."),
+ ]
+ return random.choice(variants)
+
+
+GEN = {"invoice": invoice, "personal": personal,
+ "newsletter": newsletter, "needs-reply": needs_reply}
+
+rows = []
+for label, gen in GEN.items():
+ seen = set()
+ n = 0
+ i = 0
+ while n < PER_LABEL:
+ i += 1
+ subject, body = gen(i)
+ key = subject + body
+ if key in seen:
+ continue
+ seen.add(key)
+ rows.append({"label": label, "subject": subject, "text": body,
+ "split": "boundary"})
+ n += 1
+
+random.Random(1108).shuffle(rows)
+json.dump(rows, open("boundary.json", "w"), indent=1)
+import collections
+print(f"boundary.json: {len(rows)} emails, "
+ f"{dict(collections.Counter(r['label'] for r in rows))}")
diff --git a/benchmarks/systemone-watcher-gate/results/run.json b/benchmarks/systemone-watcher-gate/results/run.json
new file mode 100644
index 000000000..c5817cf92
--- /dev/null
+++ b/benchmarks/systemone-watcher-gate/results/run.json
@@ -0,0 +1,1108 @@
+{
+ "url": "http://localhost:9471",
+ "k": 10,
+ "p_min": 0.5,
+ "conf_gate": 0.8,
+ "n_candidates": 120,
+ "n_fires": 120,
+ "fired_correct": 94,
+ "calibration_present": false,
+ "raw": {
+ "confident_fires": 98,
+ "wrong_and_confident": 24,
+ "rate": 0.24489795918367346,
+ "pass": false
+ },
+ "calibrated": null,
+ "confusion": {
+ "personal->personal": 30,
+ "newsletter->newsletter": 30,
+ "needs-reply->needs-reply": 21,
+ "invoice->personal": 11,
+ "needs-reply->newsletter": 9,
+ "invoice->invoice": 13,
+ "invoice->needs-reply": 6
+ },
+ "fires": [
+ {
+ "doc_id": "c0",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c1",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c2",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c3",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c5",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.908549,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c7",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.962065,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c4",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.80084,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c6",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.715486,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c8",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c13",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c9",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c11",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.757214,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c10",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.957323,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c12",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c14",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c16",
+ "fired": "newsletter",
+ "gold": "needs-reply",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c15",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.711286,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c17",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c18",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c21",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c19",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c20",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.836472,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c22",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c23",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c25",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c26",
+ "fired": "needs-reply",
+ "gold": "invoice",
+ "correct": false,
+ "p": 0.870614,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c24",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c27",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.711286,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c28",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.711286,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c29",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c30",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 0.584203,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c31",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c32",
+ "fired": "needs-reply",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c33",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c34",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c35",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c36",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.660073,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c37",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c38",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 0.927923,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c40",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.625931,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c39",
+ "fired": "needs-reply",
+ "gold": "invoice",
+ "correct": false,
+ "p": 0.951226,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c42",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c41",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c43",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c45",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c46",
+ "fired": "newsletter",
+ "gold": "needs-reply",
+ "correct": false,
+ "p": 0.885246,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c47",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c48",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.951226,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c44",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.625931,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c49",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.715486,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c50",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.836472,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c52",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c51",
+ "fired": "newsletter",
+ "gold": "needs-reply",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c53",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c54",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.715486,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c55",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c56",
+ "fired": "needs-reply",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c57",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c58",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c59",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c60",
+ "fired": "needs-reply",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c61",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c62",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c63",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.894323,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c64",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c65",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c66",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c67",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c68",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.886194,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c69",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c71",
+ "fired": "newsletter",
+ "gold": "needs-reply",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c70",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c72",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c73",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.711286,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c74",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c75",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c76",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.957323,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c77",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.886194,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c78",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.711286,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c79",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.927923,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c80",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c81",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.836472,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c82",
+ "fired": "needs-reply",
+ "gold": "invoice",
+ "correct": false,
+ "p": 0.870614,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c83",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.715486,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c84",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c85",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c87",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.715486,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c86",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c88",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c89",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c90",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c91",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 0.836472,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c92",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c93",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.834033,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c94",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.711286,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c95",
+ "fired": "newsletter",
+ "gold": "needs-reply",
+ "correct": false,
+ "p": 0.957323,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c96",
+ "fired": "newsletter",
+ "gold": "needs-reply",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c97",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c98",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.625931,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c99",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.625931,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c100",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.660073,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c101",
+ "fired": "invoice",
+ "gold": "invoice",
+ "correct": true,
+ "p": 0.625931,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c102",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c103",
+ "fired": "newsletter",
+ "gold": "needs-reply",
+ "correct": false,
+ "p": 0.512126,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c104",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 0.965858,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c105",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c106",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c107",
+ "fired": "personal",
+ "gold": "invoice",
+ "correct": false,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c108",
+ "fired": "newsletter",
+ "gold": "needs-reply",
+ "correct": false,
+ "p": 0.512126,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c109",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.885246,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c110",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c111",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.957323,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c112",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c113",
+ "fired": "needs-reply",
+ "gold": "needs-reply",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c115",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 0.962065,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c114",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c116",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c117",
+ "fired": "newsletter",
+ "gold": "newsletter",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c118",
+ "fired": "newsletter",
+ "gold": "needs-reply",
+ "correct": false,
+ "p": 0.927923,
+ "p_cal": null,
+ "calibrated": false
+ },
+ {
+ "doc_id": "c119",
+ "fired": "personal",
+ "gold": "personal",
+ "correct": true,
+ "p": 1.0,
+ "p_cal": null,
+ "calibrated": false
+ }
+ ]
+}
\ No newline at end of file
diff --git a/benchmarks/systemone-watcher-gate/results/transcript.txt b/benchmarks/systemone-watcher-gate/results/transcript.txt
new file mode 100644
index 000000000..6f5091584
--- /dev/null
+++ b/benchmarks/systemone-watcher-gate/results/transcript.txt
@@ -0,0 +1,61 @@
+booting /tmp/xerj-build-1108/release/xerj on :9471
+server pid 276856
+cluster green after 1s
+boundary.json: 120 emails, {'personal': 30, 'newsletter': 30, 'needs-reply': 30, 'invoice': 30}
+XERJ_URL=http://localhost:9471 k=10 p_min=0.5 conf_gate>=0.8
+history 'email-history': 200 labelled emails
+creating one detection watch per label (one-vs-rest):
+ watch detect-invoice: positive_label='invoice' p_min=0.5 -> scheduled
+ watch detect-needs_reply: positive_label='needs-reply' p_min=0.5 -> scheduled
+ watch detect-newsletter: positive_label='newsletter' p_min=0.5 -> scheduled
+ watch detect-personal: positive_label='personal' p_min=0.5 -> scheduled
+watched 'mail-incoming': 120 boundary candidates indexed after baseline
+waiting for the watcher to evaluate (1s ticks)...
+ fires=120
+ fires stable at 120
+
+================================================================
+RESULT — wrong-and-confident gate via the _watcher path
+================================================================
+candidates evaluated : 120
+total fires : 120 (a candidate fires when its vote argmax clears p_min=0.5)
+fired correctly : 94/120 = 0.7833
+calibration present : no — p_cal absent, reporting raw p
+
+confidence compared at >= 0.8: RAW p (always present on a fire)
+ confident fires : 98
+ wrong AND confident : 24
+ WRONG-AND-CONFIDENT RATE: 0.2449 (FAIL vs <= 0.05 bar)
+
+wrong-and-confident fires (raw p >= 0.80):
+ doc= c13 gold=invoice fired=personal p=1.000 p_cal=None
+ doc= c16 gold=needs-reply fired=newsletter p=1.000 p_cal=None
+ doc= c22 gold=invoice fired=personal p=1.000 p_cal=None
+ doc= c29 gold=invoice fired=personal p=1.000 p_cal=None
+ doc= c32 gold=invoice fired=needs-reply p=1.000 p_cal=None
+ doc= c42 gold=invoice fired=personal p=1.000 p_cal=None
+ doc= c51 gold=needs-reply fired=newsletter p=1.000 p_cal=None
+ doc= c55 gold=invoice fired=personal p=1.000 p_cal=None
+ doc= c56 gold=invoice fired=needs-reply p=1.000 p_cal=None
+ doc= c60 gold=invoice fired=needs-reply p=1.000 p_cal=None
+ doc= c61 gold=invoice fired=personal p=1.000 p_cal=None
+ doc= c65 gold=invoice fired=personal p=1.000 p_cal=None
+ doc= c66 gold=invoice fired=personal p=1.000 p_cal=None
+ doc= c71 gold=needs-reply fired=newsletter p=1.000 p_cal=None
+ doc= c72 gold=invoice fired=personal p=1.000 p_cal=None
+ doc= c96 gold=needs-reply fired=newsletter p=1.000 p_cal=None
+ doc=c105 gold=invoice fired=personal p=1.000 p_cal=None
+ doc=c107 gold=invoice fired=personal p=1.000 p_cal=None
+ doc= c95 gold=needs-reply fired=newsletter p=0.957 p_cal=None
+ doc= c39 gold=invoice fired=needs-reply p=0.951 p_cal=None
+ doc=c118 gold=needs-reply fired=newsletter p=0.928 p_cal=None
+ doc= c46 gold=needs-reply fired=newsletter p=0.885 p_cal=None
+ doc= c26 gold=invoice fired=needs-reply p=0.871 p_cal=None
+ doc= c82 gold=invoice fired=needs-reply p=0.871 p_cal=None
+
+wrote results/run.json
+=== server banner ===
+version {'number': '8.13.0', 'build_flavor': 'default', 'build_type': 'tar', 'build_hash': '00000000', 'build_date': '2024-03-22T00:00:00.000Z', 'build_snapshot': False, 'lucene_version': '9.10.0', 'minimum_wire_compatibility_version': '7.17.0', 'minimum_index_compatibility_version': '7.0.0'}
+HARNESS_RC=0
+
+[exited with code 0]
diff --git a/benchmarks/systemone-watcher-gate/watcher_gate.py b/benchmarks/systemone-watcher-gate/watcher_gate.py
new file mode 100644
index 000000000..98a50e24a
--- /dev/null
+++ b/benchmarks/systemone-watcher-gate/watcher_gate.py
@@ -0,0 +1,297 @@
+"""#1108 — the wrong-and-confident gate, run through the REAL `_watcher` product
+path, not a script that calls `/_decide` directly.
+
+Issue #1108 asks for the unmeasured half of the rc.80 `_watcher` work (#1062):
+on the classify-email boundary tier, the wrong-and-confident rate at confidence
+>= 0.8 must be <= 5 %. The measurement has to go through the watcher's own
+semantics — percolate prefilter -> `/_decide` -> fire into `.xerj_alert_fires` —
+so that it tests what a deployed detection watch actually does, not a bare vote.
+
+WHAT THIS DOES
+ 1. Index the 200-email train split of systemone-classify-email/corpus.json
+ into the vote-history index (`decide_index`). That is the labelled history
+ `/_decide` votes against — reused verbatim so the two benchmarks share it.
+ 2. Create the watched (input) index EMPTY, then PUT one detection watch per
+ label (one-vs-rest): a real `_watcher` watch whose `condition.xerj_decide`
+ fires when the vote's argmax == that watch's positive_label and the RAW
+ confidence >= p_min. The watch baselines at the index's current max _seq_no
+ at PUT time, so documents indexed AFTER the PUT are the ones it judges.
+ 3. Index the 120 boundary-tier candidates (gen_boundary.py) into the watched
+ index, each carrying its ground-truth label in `gold`.
+ 4. Let the watcher tick; poll `.xerj_alert_fires` until the fire count stops
+ growing.
+ 5. Read every fire, map `doc_id` -> candidate `gold`, and compute the one
+ number the issue defines: wrong-and-confident = fires whose true label is
+ not the fired positive label, among fires with confidence >= 0.8. The
+ watcher fires on and records the RAW `/_decide` confidence as `p`; it also
+ carries `p_cal` when the node has a fitted calibration. We report the rate
+ on `p` (always present) and on `p_cal` (when the node calibrated).
+
+This harness NEVER calls POST /_decide itself for the gate decision — the fire
+records written by the node's own evaluator are the sole input to the number.
+
+Usage:
+ python3 gen_boundary.py
+ XERJ_URL=http://localhost:9471 [XERJ_APIKEY=...] python3 watcher_gate.py
+"""
+import collections
+import json
+import os
+import sys
+import time
+import urllib.error
+import urllib.request
+
+U = os.environ.get("XERJ_URL", "http://localhost:9471").rstrip("/")
+APIKEY = os.environ.get("XERJ_APIKEY", "")
+K = int(os.environ.get("K", "10"))
+P_MIN = float(os.environ.get("P_MIN", "0.5")) # raw-confidence fire threshold
+CONF_GATE = float(os.environ.get("CONF_GATE", "0.8")) # the issue's >= 0.8 bar
+HISTORY = "email-history"
+WATCHED = "mail-incoming"
+FIRES = ".xerj_alert_fires"
+
+
+def req(m, p, b=None, ct="application/json"):
+ d = b if isinstance(b, (bytes, type(None))) else json.dumps(b).encode()
+ h = {"content-type": ct}
+ if APIKEY:
+ h["authorization"] = f"ApiKey {APIKEY}"
+ r = urllib.request.Request(U + p, data=d, method=m, headers=h)
+ try:
+ return json.loads(urllib.request.urlopen(r, timeout=180).read())
+ except urllib.error.HTTPError as e:
+ return {"_err": e.code, "body": e.read().decode()[:500]}
+ except Exception as e: # noqa: BLE001
+ return {"_err": "exc", "body": str(e)[:500]}
+
+
+def die(msg, ctx=None):
+ print("FATAL:", msg)
+ if ctx is not None:
+ print(" ", str(ctx)[:500])
+ sys.exit(1)
+
+
+def index_history(train):
+ req("DELETE", "/" + HISTORY)
+ created = req("PUT", "/" + HISTORY, {"mappings": {"properties": {
+ "label": {"type": "keyword"}, "text": {"type": "text"}}}})
+ if "_err" in created:
+ die("could not create history index", created)
+ lines = []
+ for i, r in enumerate(train):
+ lines.append(json.dumps({"index": {"_index": HISTORY, "_id": f"h{i}"}}))
+ lines.append(json.dumps(
+ {"text": r["subject"] + "\n" + r["text"], "label": r["label"]}))
+ resp = req("POST", "/_bulk", ("\n".join(lines) + "\n").encode(),
+ "application/x-ndjson")
+ if "_err" in resp or resp.get("errors"):
+ die("history bulk failed", resp)
+ req("POST", f"/{HISTORY}/_refresh")
+ n = req("GET", f"/{HISTORY}/_count").get("count")
+ print(f"history '{HISTORY}': {n} labelled emails")
+
+
+def make_watched_index():
+ req("DELETE", "/" + WATCHED)
+ created = req("PUT", "/" + WATCHED, {"mappings": {"properties": {
+ "text": {"type": "text"}, "gold": {"type": "keyword"}}}})
+ if "_err" in created:
+ die("could not create watched index", created)
+ req("POST", f"/{WATCHED}/_refresh")
+
+
+def put_watch(label):
+ wid = "detect-" + label.replace("-", "_")
+ body = {
+ "trigger": {"schedule": {"interval": "1s"}},
+ "input": {"search": {
+ "request": {"indices": [WATCHED]},
+ "body": {"query": {"match_all": {}}},
+ }},
+ "condition": {"xerj_decide": {
+ "question": "{{text}}",
+ "positive_label": label,
+ "text_field": "text",
+ "p_min": P_MIN,
+ "index": HISTORY,
+ "k": K,
+ }},
+ "actions": {"index_alert": {"level": "info"}},
+ }
+ resp = req("PUT", f"/_watcher/watch/{wid}", body)
+ if "_err" in resp:
+ die(f"watch {wid} refused", resp)
+ print(f" watch {wid}: positive_label={label!r} p_min={P_MIN} -> {resp.get('watcher',{}).get('evaluation')}")
+ return wid
+
+
+def index_candidates(cands):
+ lines = []
+ for i, r in enumerate(cands):
+ lines.append(json.dumps({"index": {"_index": WATCHED, "_id": f"c{i}"}}))
+ lines.append(json.dumps(
+ {"text": r["subject"] + "\n" + r["text"], "gold": r["label"]}))
+ resp = req("POST", "/_bulk", ("\n".join(lines) + "\n").encode(),
+ "application/x-ndjson")
+ if "_err" in resp or resp.get("errors"):
+ die("candidate bulk failed", resp)
+ req("POST", f"/{WATCHED}/_refresh")
+ print(f"watched '{WATCHED}': {len(cands)} boundary candidates indexed after baseline")
+
+
+def gold_by_doc(cands):
+ return {f"c{i}": r["label"] for i, r in enumerate(cands)}
+
+
+def fire_count():
+ req("POST", f"/{FIRES}/_refresh")
+ r = req("GET", f"/{FIRES}/_count")
+ return r.get("count", 0) if "_err" not in r else 0
+
+
+def wait_for_fires(expected_candidates):
+ # Each candidate fires at most once (under the watch matching its argmax
+ # label, if confident enough). Poll until the count is stable across
+ # several ticks; the watcher interval is 1s.
+ print("waiting for the watcher to evaluate (1s ticks)...")
+ last, stable = -1, 0
+ for _ in range(90):
+ time.sleep(2)
+ n = fire_count()
+ if n == last:
+ stable += 1
+ if stable >= 4:
+ print(f" fires stable at {n}")
+ return n
+ else:
+ print(f" fires={n}")
+ stable = 0
+ last = n
+ print(f" stopped waiting at {last} fires")
+ return last
+
+
+def read_fires():
+ req("POST", f"/{FIRES}/_refresh")
+ r = req("GET", f"/{FIRES}/_search",
+ {"size": 10000, "query": {"match_all": {}}})
+ if "_err" in r:
+ die("could not read fires", r)
+ return [h["_source"] for h in r.get("hits", {}).get("hits", [])]
+
+
+def rate(num, den):
+ return (num / den) if den else float("nan")
+
+
+def main():
+ corpus = json.load(open("../systemone-classify-email/corpus.json"))
+ train = [r for r in corpus if r["split"] == "train"]
+ cands = json.load(open("boundary.json"))
+ labels = sorted({r["label"] for r in cands})
+
+ print(f"XERJ_URL={U} k={K} p_min={P_MIN} conf_gate>={CONF_GATE}")
+ index_history(train)
+ make_watched_index()
+ print("creating one detection watch per label (one-vs-rest):")
+ for lab in labels:
+ put_watch(lab)
+ index_candidates(cands)
+ wait_for_fires(len(cands))
+
+ fires = read_fires()
+ gold = gold_by_doc(cands)
+
+ # Deduplicate: one fire per (doc_id, rule). A candidate's argmax is one
+ # label, so it should fire under exactly one watch; keep the highest p if
+ # the node retried.
+ best = {}
+ for f in fires:
+ did = f.get("doc_id")
+ if did not in gold:
+ continue # a fire from some other run's docs; ignore
+ key = did
+ if key not in best or f.get("p", 0) > best[key].get("p", 0):
+ best[key] = f
+ fires = list(best.values())
+
+ rows = []
+ any_cal = False
+ for f in fires:
+ did = f["doc_id"]
+ fired = f.get("label")
+ g = gold[did]
+ p = f.get("p")
+ pc = f.get("p_cal")
+ if f.get("calibrated") and isinstance(pc, (int, float)):
+ any_cal = True
+ rows.append({"doc_id": did, "fired": fired, "gold": g,
+ "correct": fired == g, "p": p, "p_cal": pc,
+ "calibrated": bool(f.get("calibrated"))})
+
+ total_fires = len(rows)
+ # The number, on RAW p (what the watcher fires on and records):
+ conf_raw = [r for r in rows if isinstance(r["p"], (int, float)) and r["p"] >= CONF_GATE]
+ wrong_conf_raw = [r for r in conf_raw if not r["correct"]]
+ wac_raw = rate(len(wrong_conf_raw), len(conf_raw))
+
+ # On p_cal, only if the node calibrated:
+ conf_cal = [r for r in rows if r["calibrated"]
+ and isinstance(r["p_cal"], (int, float)) and r["p_cal"] >= CONF_GATE]
+ wrong_conf_cal = [r for r in conf_cal if not r["correct"]]
+ wac_cal = rate(len(wrong_conf_cal), len(conf_cal))
+
+ correct_all = sum(r["correct"] for r in rows)
+ confusion = collections.Counter((r["gold"], r["fired"]) for r in rows)
+
+ print("\n" + "=" * 64)
+ print("RESULT — wrong-and-confident gate via the _watcher path")
+ print("=" * 64)
+ print(f"candidates evaluated : {len(cands)}")
+ print(f"total fires : {total_fires} "
+ f"(a candidate fires when its vote argmax clears p_min={P_MIN})")
+ print(f"fired correctly : {correct_all}/{total_fires} "
+ f"= {rate(correct_all, total_fires):.4f}")
+ print(f"calibration present : {'yes (p_cal on fires)' if any_cal else 'no — p_cal absent, reporting raw p'}")
+ print()
+ print(f"confidence compared at >= {CONF_GATE}: RAW p (always present on a fire)")
+ print(f" confident fires : {len(conf_raw)}")
+ print(f" wrong AND confident : {len(wrong_conf_raw)}")
+ print(f" WRONG-AND-CONFIDENT RATE: {wac_raw:.4f} "
+ f"({'PASS' if wac_raw <= 0.05 else 'FAIL'} vs <= 0.05 bar)")
+ if any_cal:
+ print(f"\nconfidence compared at >= {CONF_GATE}: p_cal (calibrated)")
+ print(f" confident fires : {len(conf_cal)}")
+ print(f" wrong AND confident : {len(wrong_conf_cal)}")
+ print(f" WRONG-AND-CONFIDENT RATE: {wac_cal:.4f} "
+ f"({'PASS' if wac_cal <= 0.05 else 'FAIL'} vs <= 0.05 bar)")
+
+ print("\nwrong-and-confident fires (raw p >= %.2f):" % CONF_GATE)
+ for r in sorted(wrong_conf_raw, key=lambda r: -r["p"]):
+ print(f" doc={r['doc_id']:>4} gold={r['gold']:<12} fired={r['fired']:<12} "
+ f"p={r['p']:.3f} p_cal={r['p_cal']}")
+
+ out = {
+ "url": U, "k": K, "p_min": P_MIN, "conf_gate": CONF_GATE,
+ "n_candidates": len(cands), "n_fires": total_fires,
+ "fired_correct": correct_all,
+ "calibration_present": any_cal,
+ "raw": {"confident_fires": len(conf_raw),
+ "wrong_and_confident": len(wrong_conf_raw),
+ "rate": wac_raw, "pass": bool(wac_raw <= 0.05)},
+ "calibrated": ({"confident_fires": len(conf_cal),
+ "wrong_and_confident": len(wrong_conf_cal),
+ "rate": wac_cal, "pass": bool(wac_cal <= 0.05)}
+ if any_cal else None),
+ "confusion": {f"{g}->{p}": c for (g, p), c in confusion.items()},
+ "fires": rows,
+ }
+ os.makedirs("results", exist_ok=True)
+ json.dump(out, open("results/run.json", "w"), indent=1)
+ print("\nwrote results/run.json")
+
+
+if __name__ == "__main__":
+ main()