AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
bench(systemone): measure local /_decide for discussion-1012 email labelling
Vincenzo Lombardodf0ce65e0620
diff --git a/benchmarks/systemone-classify-email/README.md b/benchmarks/systemone-classify-email/README.mdnew file mode 100644index 000000000..3403dbd53--- /dev/null+++ b/benchmarks/systemone-classify-email/README.md@@ -0,0 +1,129 @@+# Labelling documents at `autoindex` time — a local prototype for discussion #1012++[discussions/1012](https://github.com/xerj-org/xerj/discussions/1012) asks whether+`xerj autoindex` should be able to attach a best-label + score to each document while+indexing it, starting with email. Before writing any `xerj-autoindex` code, this+measures the mechanism the discussion's Q2 flagged as the local alternative to a+hosted Jev call: `POST /_decide` (`engine/crates/xerj-api/src/systemone_api.rs`), a+weighted k-nearest-neighbour vote over a labelled-history index that is **already+shipped** and sends nothing off the node.++## What this is not++- **Not a hosted-Jev measurement.** The discussion's hosted path needs the `jev-reranker`+ client extended to send classification questions instead of yes/no (its own text: "the+ current client only sends yes/no questions, it would need extending"). That extension+ does not exist, so no hosted number is claimed here — only the local `/_decide` path,+ which needed no new client code.+- **Not a real-mailbox accuracy number.** `gen_corpus.py` generates its own labelled+ emails from templates (see that file's docstring) — original text, no licence+ question, no privacy question, but also more separable than a real inbox: a real+ "personal" email sometimes has a dollar figure in it, a real newsletter sometimes asks+ a direct question. Read the number below as "does the mechanism work at all", not as a+ production accuracy estimate.++## Method++- 4 labels, matching the discussion's own example: `invoice`, `newsletter`, `personal`,+ `needs-reply`.+- 50 train + 15 held-out test emails per label (200 train / 60 test), generated by+ `gen_corpus.py`, seed 7 (deterministic — rerunning it reproduces `corpus.json` byte+ for byte). Templated text, so the 4 labels have near-exclusive vocabulary+ ("Invoice", "unsubscribe", "Hey `<name>`", "confirm by") — see the second tier below+ for why that number alone would be misleading.+- **A second, hand-authored "hard" tier** (8 emails, 2 per label): deliberately+ boundary-crossing — a personal note that mentions a dollar figure, an invoice with no+ word "invoice" anywhere in it, a newsletter that ends by asking a direct question, a+ needs-reply email phrased like a status update. Added after the first run scored a+ suspicious 100% — see [Result](#result).+- `run.py` bulk-indexes the 200 train emails into a throwaway index+ (`email-classify-proto`, `label: keyword`, `text: text`, default lexical embedder —+ no `--embed-mode neural`, no vector setup), then calls `POST /_decide` once per held-out+ email with `k=10` — the same k the published Banking77 / SMS-spam+ `decisions-as-retrieval` numbers used — and compares the returned `label` to the truth.+- `decisions.min_confidence` was left at its default (`0.0` — never abstain), so every+ test item gets a label; a real deployment would likely set this above `0.0` and leave+ low-confidence documents unlabelled instead of guessing.++## Result++Run 2026-09-22, locally built `xerj v1.0.0-rc.76`, `--insecure --port 9420`, throwaway+`--data-dir`, default lexical embedder. Raw numbers: [`results/run.json`](./results/run.json).++| Tier | n | accuracy | mean confidence | ms/item |+|---|---:|---:|---:|---:|+| held-out test (same generator as train) | 60 | **1.000** | 1.000 | 2.6 |+| hard (hand-authored, boundary-crossing) | 8 | **0.625** | 0.902 | 1.7 |++The first row is not a claim about anything — it says the templates are separable by+whatever vocabulary a template happens to use once per label, which is exactly the+failure mode this second tier exists to catch. On the hard tier, both `invoice` misses+(`P=0.000 R=0.000` on that split, `n=2`) are cases with no invoice-shaped vocabulary at+all — a casual "did the transfer land yet?" — and both were misclassified with+confidence 0.836–1.000, i.e. **wrong and confident**, not wrong and hedging. That matches+the discussion's own calibration caveat about the hosted path (ECE 0.31 on FiQA) more+than it contradicts it: this mechanism is also not calibrated to say "I don't know" on+inputs that look nothing like its labelled history — it votes on lexical/BM25 similarity+to what it has seen, and a boundary-crossing email that shares no vocabulary with its+label's training examples gets voted into whichever OTHER label it does resemble,+confidently. `decisions.min_confidence` would need to be tuned well above 0 to catch+this in practice, and even then the confidence number itself was not a reliable flag here+(0.836–1.000 covers both hits and misses in this small sample).++3 misses, all on the 8-email hard tier — full detail in `results/run.json`:++| subject | gold | predicted | confidence |+|---|---|---|---:|+| "This week at Cedarline: two quick things" | needs-reply | newsletter | 1.000 |+| "for the offsite" | invoice | needs-reply | 0.836 |+| "did that ever go through?" | invoice | personal | 1.000 |++## Reproduce++```sh+# from a built release binary (engine/target/release/xerj)+xerj -d /tmp/classify-proto-data --insecure --port 9420 &+cd benchmarks/systemone-classify-email+python3 gen_corpus.py+XERJ_URL=http://localhost:9420 python3 run.py+```++## Answering the discussion's four questions, from what this measured++1. **Emails, or something else?** Not answered by this prototype — it only shows the+ *mechanism* generalizes across a label set (nothing email-specific in `/_decide`); it+ says nothing about which document kind benefits most.+2. **Hosted vs. local-only.** The local `/_decide` path needs zero new client code and+ zero egress. It also has a cost the hosted path does not, visible in the hard-tier+ result above: it needs labelled examples *before* it can classify anything, it votes+ on similarity to what it has already seen, and it is not more calibrated than the+ hosted judge's own documented weakness — a boundary-crossing email got voted+ confidently into the wrong label here too. Whether that trade-off (no egress, no+ labelling cost saved) beats a hosted judge's own miscalibration is a product+ question this benchmark cannot settle; it can only say neither path arrives+ pre-calibrated.+3. **One best label, or a score per label?** `/_decide` already returns both — the+ winning `label` plus `confidence` (the winning label's vote share), and the raw+ per-neighbour votes are visible in its response if a caller wants to reconstruct a+ full distribution. No new work needed either way — but per the hard-tier result,+ don't read `confidence` as a trustworthy abstain signal without tuning+ `decisions.min_confidence` against a held-out set shaped like the real traffic first;+ this measurement is too small (n=8 on the hard tier) to say where that threshold+ should sit, only that 0 is not it.+4. **Where does the label live?** `label`/`text` on the record matches+ `xerj-autoindex`'s own convention (`DOC_BODY_FIELDS = ["body", "text"]` in+ `engine/crates/xerj-autoindex/src/infer/mod.rs`) closely enough that a real+ integration would likely add a plain field (e.g. `_label` / `_label_confidence`) on+ the record rather than a separate index — a separate index is what `/_decide` already+ *reads from* (the labelled history), not what it should write to.++## What would still need building++This prototype calls `/_decide` from a throwaway Python script against an+already-labelled index. Wiring it into `xerj autoindex` needs, at minimum: a CLI flag+(off by default, per the discussion's own "what I'd want to keep true"), a step in the+indexing pipeline that calls `/_decide` per document and attaches `label`/+`label_confidence` on failure-or-abstain-tolerant terms (a `/_decide` error or abstain+must not stop the run — same policy `rerank` already uses), and a decision on whether+the labelled-history index is built ahead of time by the operator or bootstrapped from a+`--label-examples` file. None of that is written yet.diff --git a/benchmarks/systemone-classify-email/corpus.json b/benchmarks/systemone-classify-email/corpus.jsonnew file mode 100644index 000000000..d5223a6e7--- /dev/null+++ b/benchmarks/systemone-classify-email/corpus.json@@ -0,0 +1,1610 @@+[+ {+ "label": "needs-reply",+ "subject": "Need your input: tell me which option you prefer",+ "text": "Hi \u2014 following up from Voss & Rane. Could you tell me which option you prefer by end of day Jun 3? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: pick a time for the walkthrough",+ "text": "Hi \u2014 following up from Voss & Rane. Could you pick a time for the walkthrough by end of day Sep 6? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm your seat for the workshop",+ "text": "Hi \u2014 following up from Marlowe Analytics. Could you confirm your seat for the workshop by end of day Mar 2? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10052 from Fenwick Cloud \u2014 $30,433.51 due Dec 3",+ "text": "Please find attached invoice INV-10052 for the annual support contract. Amount due: $30,433.51, payable by Dec 3. Remit to the account on file and reference INV-10052 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: pick a time for the walkthrough",+ "text": "Hi \u2014 following up from Marlowe Analytics. Could you pick a time for the walkthrough by end of day Mar 12? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Bluepeak Logistics: what changed in the release",+ "text": "Hi there \u2014 here is this week's digest from Bluepeak Logistics. This issue: what changed in the release. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: tell me which option you prefer",+ "text": "Hi \u2014 following up from Bluepeak Logistics. Could you tell me which option you prefer by end of day Feb 14? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10013 from Ridgeway Consulting \u2014 $32,527.43 due Dec 15",+ "text": "Please find attached invoice INV-10013 for the API usage overage. Amount due: $32,527.43, payable by Dec 15. Remit to the account on file and reference INV-10013 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10057 from Cedarline Studio \u2014 $1,914.32 due Apr 10",+ "text": "Please find attached invoice INV-10057 for the office furniture delivery. Amount due: $1,914.32, payable by Apr 10. Remit to the account on file and reference INV-10057 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: pick a time for the walkthrough",+ "text": "Hi \u2014 following up from Atlas Facilities. Could you pick a time for the walkthrough by end of day Aug 6? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: pick a time for the walkthrough",+ "text": "Hi \u2014 following up from Atlas Facilities. Could you pick a time for the walkthrough by end of day Mar 1? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "re: Alex's thing",+ "text": "Hey Ingrid, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Alex said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Marlowe Analytics: a recap of the community call",+ "text": "Hi there \u2014 here is this week's digest from Marlowe Analytics. This issue: a recap of the community call. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Alex, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Fatima said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Fatima, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Kwame said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "This week at Cedarline: two quick things",+ "text": "Hi \u2014 this week's rundown from the studio: the mockups are attached and we shipped the new brand kit. One thing before Friday though: I do need you to confirm the print run number, otherwise the press slot slips.",+ "split": "hard"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10061 from Atlas Facilities \u2014 $12,080.77 due Jan 25",+ "text": "Please find attached invoice INV-10061 for the annual support contract. Amount due: $12,080.77, payable by Jan 25. Remit to the account on file and reference INV-10061 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "for the offsite",+ "text": "Hi team, wrapping up the offsite numbers. Catering came to $1,180, venue was $2,600. Same split as always \u2014 let accounting know if your card wasn't the one charged so we can true it up before month end.",+ "split": "hard"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10042 from Ridgeway Consulting \u2014 $24,112.21 due Jun 25",+ "text": "Please find attached invoice INV-10042 for the office furniture delivery. Amount due: $24,112.21, payable by Jun 25. Remit to the account on file and reference INV-10042 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: decide between the two vendors",+ "text": "Hi \u2014 following up from Bluepeak Logistics. Could you decide between the two vendors by end of day Jan 4? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Voss & Rane: a recap of the community call",+ "text": "Hi there \u2014 here is this week's digest from Voss & Rane. This issue: a recap of the community call. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Sara, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Priya said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10058 from Greenfield Robotics \u2014 $21,444.33 due Sep 14",+ "text": "Please find attached invoice INV-10058 for the annual support contract. Amount due: $21,444.33, payable by Sep 14. Remit to the account on file and reference INV-10058 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Bluepeak Logistics: our hiring plans",+ "text": "Hi there \u2014 here is this week's digest from Bluepeak Logistics. This issue: our hiring plans. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10015 from Marlowe Analytics \u2014 $10,040.62 due Jul 2",+ "text": "Please find attached invoice INV-10015 for consulting services (Q3). Amount due: $10,040.62, payable by Jul 2. Remit to the account on file and reference INV-10015 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Ridgeway Consulting: our product roadmap",+ "text": "Hi there \u2014 here is this week's digest from Ridgeway Consulting. This issue: our product roadmap. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Ridgeway Consulting: a recap of the community call",+ "text": "Hi there \u2014 here is this week's digest from Ridgeway Consulting. This issue: a recap of the community call. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: sign off on the draft before Friday",+ "text": "Hi \u2014 following up from Northwind Supply. Could you sign off on the draft before Friday by end of day Apr 24? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Fenwick Cloud: upcoming maintenance windows",+ "text": "Hi there \u2014 here is this week's digest from Fenwick Cloud. This issue: upcoming maintenance windows. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: let me know if the numbers look right",+ "text": "Hi \u2014 following up from Bluepeak Logistics. Could you let me know if the numbers look right by end of day Oct 12? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Ridgeway Consulting: what shipped this week",+ "text": "Hi there \u2014 here is this week's digest from Ridgeway Consulting. This issue: what shipped this week. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "re: Owen's thing",+ "text": "Hey Ingrid, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Owen said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10039 from Atlas Facilities \u2014 $45,434.20 due Sep 1",+ "text": "Please find attached invoice INV-10039 for the office furniture delivery. Amount due: $45,434.20, payable by Sep 1. Remit to the account on file and reference INV-10039 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: approve the attached budget",+ "text": "Hi \u2014 following up from Greenfield Robotics. Could you approve the attached budget by end of day Sep 7? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Ridgeway Consulting: three things we shipped",+ "text": "Hi there \u2014 here is this week's digest from Ridgeway Consulting. This issue: three things we shipped. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: pick a time for the walkthrough",+ "text": "Hi \u2014 following up from Marlowe Analytics. Could you pick a time for the walkthrough by end of day Oct 10? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: tell me which option you prefer",+ "text": "Hi \u2014 following up from Voss & Rane. Could you tell me which option you prefer by end of day Aug 8? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm the delivery window",+ "text": "Hi \u2014 following up from Cedarline Studio. Could you confirm the delivery window by end of day Jun 14? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Harborline Freight: a customer story worth reading",+ "text": "Hi there \u2014 here is this week's digest from Harborline Freight. This issue: a customer story worth reading. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Greenfield Robotics: our hiring plans",+ "text": "Hi there \u2014 here is this week's digest from Greenfield Robotics. This issue: our hiring plans. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Voss & Rane: a customer story worth reading",+ "text": "Hi there \u2014 here is this week's digest from Voss & Rane. This issue: a customer story worth reading. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10044 from Greenfield Robotics \u2014 $12,869.30 due Jul 24",+ "text": "Please find attached invoice INV-10044 for the office furniture delivery. Amount due: $12,869.30, payable by Jul 24. Remit to the account on file and reference INV-10044 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Voss & Rane: what changed in the release",+ "text": "Hi there \u2014 here is this week's digest from Voss & Rane. This issue: what changed in the release. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Northwind Supply: a customer story worth reading",+ "text": "Hi there \u2014 here is this week's digest from Northwind Supply. This issue: a customer story worth reading. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm your seat for the workshop",+ "text": "Hi \u2014 following up from Greenfield Robotics. Could you confirm your seat for the workshop by end of day Apr 2? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10045 from Cedarline Studio \u2014 $34,003.63 due Jun 24",+ "text": "Please find attached invoice INV-10045 for the March hardware order. Amount due: $34,003.63, payable by Jun 24. Remit to the account on file and reference INV-10045 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10043 from Ridgeway Consulting \u2014 $35,572.99 due Sep 11",+ "text": "Please find attached invoice INV-10043 for the office furniture delivery. Amount due: $35,572.99, payable by Sep 11. Remit to the account on file and reference INV-10043 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: pick a time for the walkthrough",+ "text": "Hi \u2014 following up from Northwind Supply. Could you pick a time for the walkthrough by end of day Nov 26? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "quick one",+ "text": "Hey Jonas, it's been way too long. Was thinking we could join for the hike next weekend \u2014 no pressure either way, just wanted to catch up. Nina said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10053 from Harborline Freight \u2014 $8,405.03 due Mar 19",+ "text": "Please find attached invoice INV-10053 for the freight for pallet #4. Amount due: $8,405.03, payable by Mar 19. Remit to the account on file and reference INV-10053 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10056 from Bluepeak Logistics \u2014 $34,590.95 due Mar 14",+ "text": "Please find attached invoice INV-10056 for the office furniture delivery. Amount due: $34,590.95, payable by Mar 14. Remit to the account on file and reference INV-10056 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "hey Priya",+ "text": "Hey Priya, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Derek said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10065 from Northwind Supply \u2014 $6,485.64 due Aug 18",+ "text": "Please find attached invoice INV-10065 for the March hardware order. Amount due: $6,485.64, payable by Aug 18. Remit to the account on file and reference INV-10065 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10037 from Atlas Facilities \u2014 $30,619.61 due Aug 10",+ "text": "Please find attached invoice INV-10037 for consulting services (Q3). Amount due: $30,619.61, payable by Aug 10. Remit to the account on file and reference INV-10037 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm your seat for the workshop",+ "text": "Hi \u2014 following up from Northwind Supply. Could you confirm your seat for the workshop by end of day Nov 7? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Northwind Supply: what shipped this week",+ "text": "Hi there \u2014 here is this week's digest from Northwind Supply. This issue: what shipped this week. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Atlas Facilities: our product roadmap",+ "text": "Hi there \u2014 here is this week's digest from Atlas Facilities. This issue: our product roadmap. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Marlowe Analytics: what changed in the release",+ "text": "Hi there \u2014 here is this week's digest from Marlowe Analytics. This issue: what changed in the release. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: let me know if the numbers look right",+ "text": "Hi \u2014 following up from Cedarline Studio. Could you let me know if the numbers look right by end of day Jan 13? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "re: Kwame's thing",+ "text": "Hey Mei, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Kwame said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Ola, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Alex said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10029 from Greenfield Robotics \u2014 $37,195.40 due Mar 23",+ "text": "Please find attached invoice INV-10029 for the March hardware order. Amount due: $37,195.40, payable by Mar 23. Remit to the account on file and reference INV-10029 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: approve the attached budget",+ "text": "Hi \u2014 following up from Bluepeak Logistics. Could you approve the attached budget by end of day Jul 16? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Yuki, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Alex said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Atlas Facilities: our hiring plans",+ "text": "Hi there \u2014 here is this week's digest from Atlas Facilities. This issue: our hiring plans. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: decide between the two vendors",+ "text": "Hi \u2014 following up from Atlas Facilities. Could you decide between the two vendors by end of day Mar 8? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Northwind Supply: a recap of the community call",+ "text": "Hi there \u2014 here is this week's digest from Northwind Supply. This issue: a recap of the community call. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "hey Kwame",+ "text": "Hey Kwame, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Tom\u00e1s said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10036 from Marlowe Analytics \u2014 $39,550.46 due Aug 4",+ "text": "Please find attached invoice INV-10036 for consulting services (Q3). Amount due: $39,550.46, payable by Aug 4. Remit to the account on file and reference INV-10036 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: tell me which option you prefer",+ "text": "Hi \u2014 following up from Greenfield Robotics. Could you tell me which option you prefer by end of day Feb 12? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Priya, it's been way too long. Was thinking we could swing by before the game \u2014 no pressure either way, just wanted to catch up. Sara said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: sign off on the draft before Friday",+ "text": "Hi \u2014 following up from Northwind Supply. Could you sign off on the draft before Friday by end of day Dec 25? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: sign off on the draft before Friday",+ "text": "Hi \u2014 following up from Fenwick Cloud. Could you sign off on the draft before Friday by end of day Jul 10? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10009 from Marlowe Analytics \u2014 $6,465.70 due Dec 3",+ "text": "Please find attached invoice INV-10009 for the March hardware order. Amount due: $6,465.70, payable by Dec 3. Remit to the account on file and reference INV-10009 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "hey Ingrid",+ "text": "Hey Ingrid, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Mei said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Ola, it's been way too long. Was thinking we could join for the hike next weekend \u2014 no pressure either way, just wanted to catch up. Yuki said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Ingrid, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Hana said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10063 from Ridgeway Consulting \u2014 $4,127.41 due Nov 17",+ "text": "Please find attached invoice INV-10063 for the freight for pallet #4. Amount due: $4,127.41, payable by Nov 17. Remit to the account on file and reference INV-10063 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Fatima, it's been way too long. Was thinking we could call you this weekend \u2014 no pressure either way, just wanted to catch up. Nina said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Atlas Facilities: what shipped this week",+ "text": "Hi there \u2014 here is this week's digest from Atlas Facilities. This issue: what shipped this week. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "hey, free this weekend?",+ "text": "Hey Owen, free this weekend? Also \u2014 still owe you for the concert tickets, want to say $65? Let me know what works, either Venmo or cash when I see you.",+ "split": "hard"+ },+ {+ "label": "personal",+ "subject": "hey Sara",+ "text": "Hey Sara, it's been way too long. Was thinking we could grab dinner Thursday \u2014 no pressure either way, just wanted to catch up. Ingrid said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: tell me which option you prefer",+ "text": "Hi \u2014 following up from Bluepeak Logistics. Could you tell me which option you prefer by end of day Jan 26? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10033 from Marlowe Analytics \u2014 $39,449.06 due Feb 1",+ "text": "Please find attached invoice INV-10033 for the annual support contract. Amount due: $39,449.06, payable by Feb 1. Remit to the account on file and reference INV-10033 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm the delivery window",+ "text": "Hi \u2014 following up from Marlowe Analytics. Could you confirm the delivery window by end of day May 24? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "owe you for gas",+ "text": "Hey! Can you send me $40 whenever for the gas we split on the drive back? Venmo's fine, or just get me back next time you're over. No rush at all.",+ "split": "hard"+ },+ {+ "label": "newsletter",+ "subject": "This week at Voss & Rane: our hiring plans",+ "text": "Hi there \u2014 here is this week's digest from Voss & Rane. This issue: our hiring plans. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "hey Ola",+ "text": "Hey Ola, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Derek said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10007 from Voss & Rane \u2014 $27,548.18 due Sep 4",+ "text": "Please find attached invoice INV-10007 for the API usage overage. Amount due: $27,548.18, payable by Sep 4. Remit to the account on file and reference INV-10007 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: approve the attached budget",+ "text": "Hi \u2014 following up from Cedarline Studio. Could you approve the attached budget by end of day Aug 23? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: approve the attached budget",+ "text": "Hi \u2014 following up from Fenwick Cloud. Could you approve the attached budget by end of day Oct 20? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: decide between the two vendors",+ "text": "Hi \u2014 following up from Atlas Facilities. Could you decide between the two vendors by end of day Feb 14? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Alex, it's been way too long. Was thinking we could grab dinner Thursday \u2014 no pressure either way, just wanted to catch up. Tom\u00e1s said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Cedarline Studio: a recap of the community call",+ "text": "Hi there \u2014 here is this week's digest from Cedarline Studio. This issue: a recap of the community call. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Marlowe Analytics: what shipped this week",+ "text": "Hi there \u2014 here is this week's digest from Marlowe Analytics. This issue: what shipped this week. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "quick one",+ "text": "Hey Luca, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Owen said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Ridgeway Consulting: what changed in the release",+ "text": "Hi there \u2014 here is this week's digest from Ridgeway Consulting. This issue: what changed in the release. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Kwame, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Sara said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "before Friday's launch",+ "text": "Quick note from the team before Friday: we could use your take \u2014 reply and tell us which tagline you'd run with, A or B. We publish the results in next week's issue either way, so no wrong answer, just curious.",+ "split": "hard"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10021 from Atlas Facilities \u2014 $23,375.21 due Oct 4",+ "text": "Please find attached invoice INV-10021 for the freight for pallet #4. Amount due: $23,375.21, payable by Oct 4. Remit to the account on file and reference INV-10021 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10025 from Ridgeway Consulting \u2014 $18,326.90 due Jul 12",+ "text": "Please find attached invoice INV-10025 for the printer maintenance plan. Amount due: $18,326.90, payable by Jul 12. Remit to the account on file and reference INV-10025 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Bluepeak Logistics: our product roadmap",+ "text": "Hi there \u2014 here is this week's digest from Bluepeak Logistics. This issue: our product roadmap. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Northwind Supply: our hiring plans",+ "text": "Hi there \u2014 here is this week's digest from Northwind Supply. This issue: our hiring plans. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Harborline Freight: upcoming maintenance windows",+ "text": "Hi there \u2014 here is this week's digest from Harborline Freight. This issue: upcoming maintenance windows. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Ingrid, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Luca said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10040 from Ridgeway Consulting \u2014 $23,787.18 due Dec 18",+ "text": "Please find attached invoice INV-10040 for the March hardware order. Amount due: $23,787.18, payable by Dec 18. Remit to the account on file and reference INV-10040 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm the delivery window",+ "text": "Hi \u2014 following up from Bluepeak Logistics. Could you confirm the delivery window by end of day Aug 18? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm your seat for the workshop",+ "text": "Hi \u2014 following up from Bluepeak Logistics. Could you confirm your seat for the workshop by end of day Jun 9? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10024 from Fenwick Cloud \u2014 $36,088.35 due Mar 27",+ "text": "Please find attached invoice INV-10024 for the printer maintenance plan. Amount due: $36,088.35, payable by Mar 27. Remit to the account on file and reference INV-10024 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10050 from Bluepeak Logistics \u2014 $43,372.15 due Jul 26",+ "text": "Please find attached invoice INV-10050 for the office furniture delivery. Amount due: $43,372.15, payable by Jul 26. Remit to the account on file and reference INV-10050 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Mei, it's been way too long. Was thinking we could swing by before the game \u2014 no pressure either way, just wanted to catch up. Jonas said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm the delivery window",+ "text": "Hi \u2014 following up from Atlas Facilities. Could you confirm the delivery window by end of day Nov 14? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Voss & Rane: upcoming maintenance windows",+ "text": "Hi there \u2014 here is this week's digest from Voss & Rane. This issue: upcoming maintenance windows. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: decide between the two vendors",+ "text": "Hi \u2014 following up from Northwind Supply. Could you decide between the two vendors by end of day Apr 27? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "re: Ingrid's thing",+ "text": "Hey Kwame, it's been way too long. Was thinking we could swing by before the game \u2014 no pressure either way, just wanted to catch up. Ingrid said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm the delivery window",+ "text": "Hi \u2014 following up from Voss & Rane. Could you confirm the delivery window by end of day Feb 21? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: decide between the two vendors",+ "text": "Hi \u2014 following up from Northwind Supply. Could you decide between the two vendors by end of day Feb 26? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Priya, it's been way too long. Was thinking we could swing by before the game \u2014 no pressure either way, just wanted to catch up. Fatima said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "re: Yuki's thing",+ "text": "Hey Priya, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Yuki said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "re: Owen's thing",+ "text": "Hey Jonas, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Owen said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Bluepeak Logistics: a customer story worth reading",+ "text": "Hi there \u2014 here is this week's digest from Bluepeak Logistics. This issue: a customer story worth reading. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: approve the attached budget",+ "text": "Hi \u2014 following up from Cedarline Studio. Could you approve the attached budget by end of day May 10? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10032 from Cedarline Studio \u2014 $4,493.26 due Aug 6",+ "text": "Please find attached invoice INV-10032 for consulting services (Q3). Amount due: $4,493.26, payable by Aug 6. Remit to the account on file and reference INV-10032 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Cedarline Studio: our product roadmap",+ "text": "Hi there \u2014 here is this week's digest from Cedarline Studio. This issue: our product roadmap. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Fatima, it's been way too long. Was thinking we could meet at the usual place at six \u2014 no pressure either way, just wanted to catch up. Owen said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10027 from Cedarline Studio \u2014 $870.62 due Oct 6",+ "text": "Please find attached invoice INV-10027 for the API usage overage. Amount due: $870.62, payable by Oct 6. Remit to the account on file and reference INV-10027 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10012 from Cedarline Studio \u2014 $11,861.89 due Apr 3",+ "text": "Please find attached invoice INV-10012 for the API usage overage. Amount due: $11,861.89, payable by Apr 3. Remit to the account on file and reference INV-10012 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10020 from Voss & Rane \u2014 $47,044.49 due Nov 12",+ "text": "Please find attached invoice INV-10020 for the March hardware order. Amount due: $47,044.49, payable by Nov 12. Remit to the account on file and reference INV-10020 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "quick one",+ "text": "Hey Ingrid, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Jonas said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "quick one",+ "text": "Hey Derek, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Derek said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm your seat for the workshop",+ "text": "Hi \u2014 following up from Ridgeway Consulting. Could you confirm your seat for the workshop by end of day Mar 14? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Ridgeway Consulting: our hiring plans",+ "text": "Hi there \u2014 here is this week's digest from Ridgeway Consulting. This issue: our hiring plans. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: pick a time for the walkthrough",+ "text": "Hi \u2014 following up from Fenwick Cloud. Could you pick a time for the walkthrough by end of day Jul 4? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Marlowe Analytics: a customer story worth reading",+ "text": "Hi there \u2014 here is this week's digest from Marlowe Analytics. This issue: a customer story worth reading. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Harborline Freight: what shipped this week",+ "text": "Hi there \u2014 here is this week's digest from Harborline Freight. This issue: what shipped this week. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Nina, it's been way too long. Was thinking we could grab dinner Thursday \u2014 no pressure either way, just wanted to catch up. Jonas said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10038 from Harborline Freight \u2014 $6,776.95 due Jun 24",+ "text": "Please find attached invoice INV-10038 for the API usage overage. Amount due: $6,776.95, payable by Jun 24. Remit to the account on file and reference INV-10038 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: let me know if the numbers look right",+ "text": "Hi \u2014 following up from Fenwick Cloud. Could you let me know if the numbers look right by end of day Apr 1? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: approve the attached budget",+ "text": "Hi \u2014 following up from Bluepeak Logistics. Could you approve the attached budget by end of day May 20? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Northwind Supply: our product roadmap",+ "text": "Hi there \u2014 here is this week's digest from Northwind Supply. This issue: our product roadmap. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10048 from Bluepeak Logistics \u2014 $14,946.60 due Apr 11",+ "text": "Please find attached invoice INV-10048 for the office furniture delivery. Amount due: $14,946.60, payable by Apr 11. Remit to the account on file and reference INV-10048 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Yuki, it's been way too long. Was thinking we could call you this weekend \u2014 no pressure either way, just wanted to catch up. Tom\u00e1s said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10006 from Northwind Supply \u2014 $14,568.05 due Sep 28",+ "text": "Please find attached invoice INV-10006 for the annual support contract. Amount due: $14,568.05, payable by Sep 28. Remit to the account on file and reference INV-10006 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Voss & Rane: our product roadmap",+ "text": "Hi there \u2014 here is this week's digest from Voss & Rane. This issue: our product roadmap. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "re: Alex's thing",+ "text": "Hey Ola, it's been way too long. Was thinking we could grab dinner Thursday \u2014 no pressure either way, just wanted to catch up. Alex said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "re: Nina's thing",+ "text": "Hey Priya, it's been way too long. Was thinking we could grab dinner Thursday \u2014 no pressure either way, just wanted to catch up. Nina said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Ingrid, it's been way too long. Was thinking we could swing by before the game \u2014 no pressure either way, just wanted to catch up. Priya said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: sign off on the draft before Friday",+ "text": "Hi \u2014 following up from Voss & Rane. Could you sign off on the draft before Friday by end of day Jun 9? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Priya, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Luca said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10030 from Atlas Facilities \u2014 $44,682.71 due Jul 13",+ "text": "Please find attached invoice INV-10030 for the printer maintenance plan. Amount due: $44,682.71, payable by Jul 13. Remit to the account on file and reference INV-10030 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "hey Nina",+ "text": "Hey Nina, it's been way too long. Was thinking we could join for the hike next weekend \u2014 no pressure either way, just wanted to catch up. Tom\u00e1s said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Ingrid, it's been way too long. Was thinking we could call you this weekend \u2014 no pressure either way, just wanted to catch up. Hana said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10034 from Ridgeway Consulting \u2014 $6,729.46 due Oct 1",+ "text": "Please find attached invoice INV-10034 for consulting services (Q3). Amount due: $6,729.46, payable by Oct 1. Remit to the account on file and reference INV-10034 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "quick one",+ "text": "Hey Luca, it's been way too long. Was thinking we could meet at the usual place at six \u2014 no pressure either way, just wanted to catch up. Jonas said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "re: Nina's thing",+ "text": "Hey Nina, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Nina said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Marlowe Analytics: upcoming maintenance windows",+ "text": "Hi there \u2014 here is this week's digest from Marlowe Analytics. This issue: upcoming maintenance windows. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: approve the attached budget",+ "text": "Hi \u2014 following up from Cedarline Studio. Could you approve the attached budget by end of day Jan 7? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Fatima, it's been way too long. Was thinking we could call you this weekend \u2014 no pressure either way, just wanted to catch up. Sara said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10046 from Northwind Supply \u2014 $18,391.60 due May 7",+ "text": "Please find attached invoice INV-10046 for the catering for the offsite. Amount due: $18,391.60, payable by May 7. Remit to the account on file and reference INV-10046 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Bluepeak Logistics: what shipped this week",+ "text": "Hi there \u2014 here is this week's digest from Bluepeak Logistics. This issue: what shipped this week. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Fenwick Cloud: what shipped this week",+ "text": "Hi there \u2014 here is this week's digest from Fenwick Cloud. This issue: what shipped this week. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: pick a time for the walkthrough",+ "text": "Hi \u2014 following up from Ridgeway Consulting. Could you pick a time for the walkthrough by end of day Aug 20? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10003 from Northwind Supply \u2014 $5,712.55 due Jul 3",+ "text": "Please find attached invoice INV-10003 for the office furniture delivery. Amount due: $5,712.55, payable by Jul 3. Remit to the account on file and reference INV-10003 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10026 from Cedarline Studio \u2014 $9,970.10 due Mar 5",+ "text": "Please find attached invoice INV-10026 for the office furniture delivery. Amount due: $9,970.10, payable by Mar 5. Remit to the account on file and reference INV-10026 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10064 from Bluepeak Logistics \u2014 $36,799.07 due Apr 7",+ "text": "Please find attached invoice INV-10064 for the API usage overage. Amount due: $36,799.07, payable by Apr 7. Remit to the account on file and reference INV-10064 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10035 from Cedarline Studio \u2014 $40,323.48 due Mar 21",+ "text": "Please find attached invoice INV-10035 for the API usage overage. Amount due: $40,323.48, payable by Mar 21. Remit to the account on file and reference INV-10035 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10031 from Fenwick Cloud \u2014 $6,865.61 due Nov 13",+ "text": "Please find attached invoice INV-10031 for the March hardware order. Amount due: $6,865.61, payable by Nov 13. Remit to the account on file and reference INV-10031 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: let me know if the numbers look right",+ "text": "Hi \u2014 following up from Atlas Facilities. Could you let me know if the numbers look right by end of day May 14? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "Bluepeak Logistics: a customer story worth reading",+ "text": "This issue: a customer story worth reading, plus what shipped this week. We publish every Friday. Reading time about four minutes. Unsubscribe any time with the link below, no hard feelings.",+ "split": "hard"+ },+ {+ "label": "newsletter",+ "subject": "This week at Marlowe Analytics: our hiring plans",+ "text": "Hi there \u2014 here is this week's digest from Marlowe Analytics. This issue: our hiring plans. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Harborline Freight: what changed in the release",+ "text": "Hi there \u2014 here is this week's digest from Harborline Freight. This issue: what changed in the release. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10023 from Fenwick Cloud \u2014 $25,701.63 due Feb 6",+ "text": "Please find attached invoice INV-10023 for the freight for pallet #4. Amount due: $25,701.63, payable by Feb 6. Remit to the account on file and reference INV-10023 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Cedarline Studio: upcoming maintenance windows",+ "text": "Hi there \u2014 here is this week's digest from Cedarline Studio. This issue: upcoming maintenance windows. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Ridgeway Consulting: a customer story worth reading",+ "text": "Hi there \u2014 here is this week's digest from Ridgeway Consulting. This issue: a customer story worth reading. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Fatima, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Nina said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: let me know if the numbers look right",+ "text": "Hi \u2014 following up from Harborline Freight. Could you let me know if the numbers look right by end of day Aug 20? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "hey Hana",+ "text": "Hey Hana, it's been way too long. Was thinking we could call you this weekend \u2014 no pressure either way, just wanted to catch up. Sara said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: let me know if the numbers look right",+ "text": "Hi \u2014 following up from Voss & Rane. Could you let me know if the numbers look right by end of day Feb 2? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10019 from Voss & Rane \u2014 $42,490.73 due Nov 27",+ "text": "Please find attached invoice INV-10019 for the freight for pallet #4. Amount due: $42,490.73, payable by Nov 27. Remit to the account on file and reference INV-10019 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Northwind Supply: what changed in the release",+ "text": "Hi there \u2014 here is this week's digest from Northwind Supply. This issue: what changed in the release. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm your seat for the workshop",+ "text": "Hi \u2014 following up from Greenfield Robotics. Could you confirm your seat for the workshop by end of day Jul 14? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: sign off on the draft before Friday",+ "text": "Hi \u2014 following up from Marlowe Analytics. Could you sign off on the draft before Friday by end of day May 1? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Fenwick Cloud: what changed in the release",+ "text": "Hi there \u2014 here is this week's digest from Fenwick Cloud. This issue: what changed in the release. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Greenfield Robotics: three things we shipped",+ "text": "Hi there \u2014 here is this week's digest from Greenfield Robotics. This issue: three things we shipped. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10014 from Greenfield Robotics \u2014 $4,877.15 due Sep 14",+ "text": "Please find attached invoice INV-10014 for the annual support contract. Amount due: $4,877.15, payable by Sep 14. Remit to the account on file and reference INV-10014 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10054 from Harborline Freight \u2014 $40,160.76 due Aug 22",+ "text": "Please find attached invoice INV-10054 for the catering for the offsite. Amount due: $40,160.76, payable by Aug 22. Remit to the account on file and reference INV-10054 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10055 from Harborline Freight \u2014 $36,036.70 due Mar 1",+ "text": "Please find attached invoice INV-10055 for the March hardware order. Amount due: $36,036.70, payable by Mar 1. Remit to the account on file and reference INV-10055 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10001 from Marlowe Analytics \u2014 $9,966.50 due Nov 2",+ "text": "Please find attached invoice INV-10001 for consulting services (Q3). Amount due: $9,966.50, payable by Nov 2. Remit to the account on file and reference INV-10001 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "re: Yuki's thing",+ "text": "Hey Ola, it's been way too long. Was thinking we could call you this weekend \u2014 no pressure either way, just wanted to catch up. Yuki said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10005 from Cedarline Studio \u2014 $41,408.80 due Oct 2",+ "text": "Please find attached invoice INV-10005 for the printer maintenance plan. Amount due: $41,408.80, payable by Oct 2. Remit to the account on file and reference INV-10005 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: tell me which option you prefer",+ "text": "Hi \u2014 following up from Atlas Facilities. Could you tell me which option you prefer by end of day Jun 18? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Voss & Rane: what shipped this week",+ "text": "Hi there \u2014 here is this week's digest from Voss & Rane. This issue: what shipped this week. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Marlowe Analytics: our product roadmap",+ "text": "Hi there \u2014 here is this week's digest from Marlowe Analytics. This issue: our product roadmap. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: approve the attached budget",+ "text": "Hi \u2014 following up from Fenwick Cloud. Could you approve the attached budget by end of day Sep 25? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Northwind Supply: upcoming maintenance windows",+ "text": "Hi there \u2014 here is this week's digest from Northwind Supply. This issue: upcoming maintenance windows. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10028 from Voss & Rane \u2014 $348.18 due Jul 18",+ "text": "Please find attached invoice INV-10028 for the catering for the offsite. Amount due: $348.18, payable by Jul 18. Remit to the account on file and reference INV-10028 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: let me know if the numbers look right",+ "text": "Hi \u2014 following up from Cedarline Studio. Could you let me know if the numbers look right by end of day Jun 25? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Ingrid, it's been way too long. Was thinking we could call you this weekend \u2014 no pressure either way, just wanted to catch up. Luca said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10018 from Voss & Rane \u2014 $31,150.89 due Nov 3",+ "text": "Please find attached invoice INV-10018 for the March hardware order. Amount due: $31,150.89, payable by Nov 3. Remit to the account on file and reference INV-10018 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: sign off on the draft before Friday",+ "text": "Hi \u2014 following up from Cedarline Studio. Could you sign off on the draft before Friday by end of day Mar 27? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Fenwick Cloud: a recap of the community call",+ "text": "Hi there \u2014 here is this week's digest from Fenwick Cloud. This issue: a recap of the community call. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Jonas, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Sara said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10059 from Northwind Supply \u2014 $23,265.58 due Nov 19",+ "text": "Please find attached invoice INV-10059 for the printer maintenance plan. Amount due: $23,265.58, payable by Nov 19. Remit to the account on file and reference INV-10059 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Jonas, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Derek said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Harborline Freight: a recap of the community call",+ "text": "Hi there \u2014 here is this week's digest from Harborline Freight. This issue: a recap of the community call. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Bluepeak Logistics: a recap of the community call",+ "text": "Hi there \u2014 here is this week's digest from Bluepeak Logistics. This issue: a recap of the community call. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Owen, it's been way too long. Was thinking we could grab dinner Thursday \u2014 no pressure either way, just wanted to catch up. Nina said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: decide between the two vendors",+ "text": "Hi \u2014 following up from Marlowe Analytics. Could you decide between the two vendors by end of day Jun 26? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Jonas, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Hana said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Bluepeak Logistics: three things we shipped",+ "text": "Hi there \u2014 here is this week's digest from Bluepeak Logistics. This issue: three things we shipped. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Atlas Facilities: three things we shipped",+ "text": "Hi there \u2014 here is this week's digest from Atlas Facilities. This issue: three things we shipped. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: approve the attached budget",+ "text": "Hi \u2014 following up from Harborline Freight. Could you approve the attached budget by end of day Nov 6? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Marlowe Analytics: three things we shipped",+ "text": "Hi there \u2014 here is this week's digest from Marlowe Analytics. This issue: three things we shipped. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10022 from Northwind Supply \u2014 $14,380.98 due May 5",+ "text": "Please find attached invoice INV-10022 for the office furniture delivery. Amount due: $14,380.98, payable by May 5. Remit to the account on file and reference INV-10022 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "quick one",+ "text": "Hey Priya, it's been way too long. Was thinking we could swing by before the game \u2014 no pressure either way, just wanted to catch up. Alex said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10002 from Ridgeway Consulting \u2014 $6,248.46 due Oct 2",+ "text": "Please find attached invoice INV-10002 for the office furniture delivery. Amount due: $6,248.46, payable by Oct 2. Remit to the account on file and reference INV-10002 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Atlas Facilities: a recap of the community call",+ "text": "Hi there \u2014 here is this week's digest from Atlas Facilities. This issue: a recap of the community call. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Cedarline Studio: our hiring plans",+ "text": "Hi there \u2014 here is this week's digest from Cedarline Studio. This issue: our hiring plans. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "hey Kwame",+ "text": "Hey Kwame, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Ingrid said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10060 from Ridgeway Consulting \u2014 $8,649.68 due Mar 17",+ "text": "Please find attached invoice INV-10060 for the March hardware order. Amount due: $8,649.68, payable by Mar 17. Remit to the account on file and reference INV-10060 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Sara, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Derek said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10051 from Atlas Facilities \u2014 $11,779.55 due Nov 11",+ "text": "Please find attached invoice INV-10051 for consulting services (Q3). Amount due: $11,779.55, payable by Nov 11. Remit to the account on file and reference INV-10051 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: pick a time for the walkthrough",+ "text": "Hi \u2014 following up from Fenwick Cloud. Could you pick a time for the walkthrough by end of day Apr 14? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "hey Fatima",+ "text": "Hey Fatima, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Yuki said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Cedarline Studio: a customer story worth reading",+ "text": "Hi there \u2014 here is this week's digest from Cedarline Studio. This issue: a customer story worth reading. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Harborline Freight: our hiring plans",+ "text": "Hi there \u2014 here is this week's digest from Harborline Freight. This issue: our hiring plans. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Atlas Facilities: what changed in the release",+ "text": "Hi there \u2014 here is this week's digest from Atlas Facilities. This issue: what changed in the release. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: tell me which option you prefer",+ "text": "Hi \u2014 following up from Atlas Facilities. Could you tell me which option you prefer by end of day Jun 12? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10008 from Ridgeway Consulting \u2014 $44,775.23 due Feb 19",+ "text": "Please find attached invoice INV-10008 for the office furniture delivery. Amount due: $44,775.23, payable by Feb 19. Remit to the account on file and reference INV-10008 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Yuki, it's been way too long. Was thinking we could join for the hike next weekend \u2014 no pressure either way, just wanted to catch up. Ola said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10062 from Harborline Freight \u2014 $9,357.60 due Oct 24",+ "text": "Please find attached invoice INV-10062 for consulting services (Q3). Amount due: $9,357.60, payable by Oct 24. Remit to the account on file and reference INV-10062 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: tell me which option you prefer",+ "text": "Hi \u2014 following up from Harborline Freight. Could you tell me which option you prefer by end of day Apr 5? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: decide between the two vendors",+ "text": "Hi \u2014 following up from Northwind Supply. Could you decide between the two vendors by end of day Jun 2? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10010 from Greenfield Robotics \u2014 $13,577.63 due Nov 18",+ "text": "Please find attached invoice INV-10010 for the printer maintenance plan. Amount due: $13,577.63, payable by Nov 18. Remit to the account on file and reference INV-10010 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10016 from Ridgeway Consulting \u2014 $37,633.40 due Jun 23",+ "text": "Please find attached invoice INV-10016 for the catering for the offsite. Amount due: $37,633.40, payable by Jun 23. Remit to the account on file and reference INV-10016 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10017 from Greenfield Robotics \u2014 $32,630.74 due Aug 3",+ "text": "Please find attached invoice INV-10017 for consulting services (Q3). Amount due: $32,630.74, payable by Aug 3. Remit to the account on file and reference INV-10017 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "quick one",+ "text": "Hey Kwame, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Owen said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: tell me which option you prefer",+ "text": "Hi \u2014 following up from Harborline Freight. Could you tell me which option you prefer by end of day Jun 28? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10004 from Bluepeak Logistics \u2014 $36,193.54 due Jan 27",+ "text": "Please find attached invoice INV-10004 for consulting services (Q3). Amount due: $36,193.54, payable by Jan 27. Remit to the account on file and reference INV-10004 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Nina, it's been way too long. Was thinking we could bring the kids over Sunday \u2014 no pressure either way, just wanted to catch up. Owen said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10041 from Ridgeway Consulting \u2014 $19,615.82 due Feb 23",+ "text": "Please find attached invoice INV-10041 for the API usage overage. Amount due: $19,615.82, payable by Feb 23. Remit to the account on file and reference INV-10041 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "hey Kwame",+ "text": "Hey Kwame, it's been way too long. Was thinking we could call you this weekend \u2014 no pressure either way, just wanted to catch up. Kwame said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "hey Hana",+ "text": "Hey Hana, it's been way too long. Was thinking we could call you this weekend \u2014 no pressure either way, just wanted to catch up. Derek said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Tom\u00e1s, it's been way too long. Was thinking we could join for the hike next weekend \u2014 no pressure either way, just wanted to catch up. Owen said you've been swamped. Say hi to everyone for me!",+ "split": "test"+ },+ {+ "label": "personal",+ "subject": "re: Sara's thing",+ "text": "Hey Ola, it's been way too long. Was thinking we could swing by before the game \u2014 no pressure either way, just wanted to catch up. Sara said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Cedarline Studio: what shipped this week",+ "text": "Hi there \u2014 here is this week's digest from Cedarline Studio. This issue: what shipped this week. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "test"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: let me know if the numbers look right",+ "text": "Hi \u2014 following up from Voss & Rane. Could you let me know if the numbers look right by end of day Jan 10? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "test"+ },+ {+ "label": "newsletter",+ "subject": "This week at Greenfield Robotics: a customer story worth reading",+ "text": "Hi there \u2014 here is this week's digest from Greenfield Robotics. This issue: a customer story worth reading. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "quick one",+ "text": "Hey Tom\u00e1s, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Tom\u00e1s said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10011 from Marlowe Analytics \u2014 $30,593.74 due Aug 12",+ "text": "Please find attached invoice INV-10011 for the API usage overage. Amount due: $30,593.74, payable by Aug 12. Remit to the account on file and reference INV-10011 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Derek, it's been way too long. Was thinking we could drop off the keys tomorrow \u2014 no pressure either way, just wanted to catch up. Ola said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "long time!",+ "text": "Hey Priya, it's been way too long. Was thinking we could call you this weekend \u2014 no pressure either way, just wanted to catch up. Alex said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: sign off on the draft before Friday",+ "text": "Hi \u2014 following up from Fenwick Cloud. Could you sign off on the draft before Friday by end of day Apr 17? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: approve the attached budget",+ "text": "Hi \u2014 following up from Greenfield Robotics. Could you approve the attached budget by end of day Jan 27? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "this weekend?",+ "text": "Hey Mei, it's been way too long. Was thinking we could grab dinner Thursday \u2014 no pressure either way, just wanted to catch up. Luca said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10047 from Atlas Facilities \u2014 $47,470.44 due Jun 3",+ "text": "Please find attached invoice INV-10047 for the office furniture delivery. Amount due: $47,470.44, payable by Jun 3. Remit to the account on file and reference INV-10047 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Fenwick Cloud: our product roadmap",+ "text": "Hi there \u2014 here is this week's digest from Fenwick Cloud. This issue: our product roadmap. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "personal",+ "subject": "quick one",+ "text": "Hey Nina, it's been way too long. Was thinking we could catch the early train Saturday \u2014 no pressure either way, just wanted to catch up. Ola said you've been swamped. Say hi to everyone for me!",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: tell me which option you prefer",+ "text": "Hi \u2014 following up from Ridgeway Consulting. Could you tell me which option you prefer by end of day Nov 26? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "Invoice INV-10049 from Atlas Facilities \u2014 $40,978.78 due Jan 16",+ "text": "Please find attached invoice INV-10049 for the catering for the offsite. Amount due: $40,978.78, payable by Jan 16. Remit to the account on file and reference INV-10049 on the transfer. A late fee of 1.5% applies after the due date. Questions about this invoice can go to [email protected].",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Voss & Rane: three things we shipped",+ "text": "Hi there \u2014 here is this week's digest from Voss & Rane. This issue: three things we shipped. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: let me know if the numbers look right",+ "text": "Hi \u2014 following up from Bluepeak Logistics. Could you let me know if the numbers look right by end of day Nov 18? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Northwind Supply: three things we shipped",+ "text": "Hi there \u2014 here is this week's digest from Northwind Supply. This issue: three things we shipped. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "newsletter",+ "subject": "This week at Fenwick Cloud: three things we shipped",+ "text": "Hi there \u2014 here is this week's digest from Fenwick Cloud. This issue: three things we shipped. We publish every Friday and read every reply, though this address is not monitored for support. If this isn't useful anymore, unsubscribe any time with the link at the bottom, no hard feelings.",+ "split": "train"+ },+ {+ "label": "invoice",+ "subject": "did that ever go through?",+ "text": "Hey Priya, quick check \u2014 did the transfer for last month's freight land yet? Should have been around forty-eight hundred, reference was on the packing slip. No rush, just reconciling the books this week.",+ "split": "hard"+ },+ {+ "label": "needs-reply",+ "subject": "one thing before I book this",+ "text": "Before I book the freight slot I need a yes or no on the pallet count \u2014 24 or 32? Vendor holds the rate until 5pm today, after that it resets and we lose the discount, so whenever you get a sec.",+ "split": "hard"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: sign off on the draft before Friday",+ "text": "Hi \u2014 following up from Cedarline Studio. Could you sign off on the draft before Friday by end of day Jan 20? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ },+ {+ "label": "needs-reply",+ "subject": "Need your input: confirm your seat for the workshop",+ "text": "Hi \u2014 following up from Atlas Facilities. Could you confirm your seat for the workshop by end of day Jan 10? I don't want to move ahead without hearing from you first. Let me know either way, even if the answer is 'not yet'.",+ "split": "train"+ }+]\ No newline at end of filediff --git a/benchmarks/systemone-classify-email/gen_corpus.py b/benchmarks/systemone-classify-email/gen_corpus.pynew file mode 100644index 000000000..1d4ac04e0--- /dev/null+++ b/benchmarks/systemone-classify-email/gen_corpus.py@@ -0,0 +1,193 @@+"""Synthetic labelled-email corpus for the discussion-1012 classification prototype.++This corpus is ORIGINAL TEXT generated by this script, not sourced from any+real mailbox or third-party dataset — written for this measurement so there+is no licence question and no privacy question. It is a sanity-check corpus,+not a claim about accuracy on real inboxes: template-generated text is more+separable than a real mailbox (real "personal" mail sometimes contains a+dollar figure, real newsletters sometimes ask a direct question). Treat the+numbers in README.md as "does the mechanism work at all", not as a production+accuracy estimate.++Four labels, chosen to match the ones discussion #1012 used as its example:+invoice, newsletter, personal, needs-reply. Deterministic (seed 7): rerunning+this script reproduces byte-identical corpus.json.++Usage: python3 gen_corpus.py -> writes corpus.json (train+test, labelled)+"""+import json+import random++random.seed(7)++FIRST_NAMES = ["Alex", "Priya", "Jonas", "Mei", "Sara", "Tomás", "Nina", "Ola",+ "Derek", "Fatima", "Luca", "Hana", "Owen", "Ingrid", "Kwame", "Yuki"]+COMPANIES = ["Northwind Supply", "Bluepeak Logistics", "Harborline Freight",+ "Cedarline Studio", "Voss & Rane", "Marlowe Analytics",+ "Fenwick Cloud", "Atlas Facilities", "Ridgeway Consulting",+ "Greenfield Robotics"]+PRODUCTS = ["the March hardware order", "consulting services (Q3)",+ "the annual support contract", "the office furniture delivery",+ "the API usage overage", "the catering for the offsite",+ "the printer maintenance plan", "the freight for pallet #4"]+TOPICS = ["what shipped this week", "our product roadmap",+ "three things we shipped", "a customer story worth reading",+ "what changed in the release", "upcoming maintenance windows",+ "our hiring plans", "a recap of the community call"]+PLANS = ["grab dinner Thursday", "catch the early train Saturday",+ "swing by before the game", "call you this weekend",+ "bring the kids over Sunday", "meet at the usual place at six",+ "drop off the keys tomorrow", "join for the hike next weekend"]+QUESTIONS = ["confirm the delivery window", "approve the attached budget",+ "pick a time for the walkthrough",+ "tell me which option you prefer",+ "sign off on the draft before Friday",+ "confirm your seat for the workshop",+ "let me know if the numbers look right",+ "decide between the two vendors"]+++def rand_amount():+ return f"${random.randint(80, 48000):,}.{random.randint(0, 99):02d}"+++def rand_date():+ month = random.choice(["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul",+ "Aug", "Sep", "Oct", "Nov", "Dec"])+ return f"{month} {random.randint(1, 28)}"+++def invoice(i):+ company = random.choice(COMPANIES)+ amount = rand_amount()+ due = rand_date()+ inv_no = f"INV-{10000 + i}"+ subject = f"Invoice {inv_no} from {company} — {amount} due {due}"+ body = (+ f"Please find attached invoice {inv_no} for {random.choice(PRODUCTS)}. "+ f"Amount due: {amount}, payable by {due}. Remit to the account on file "+ f"and reference {inv_no} on the transfer. A late fee of 1.5% applies "+ f"after the due date. Questions about this invoice can go to "+ f"billing@{company.lower().replace(' ', '').replace('&', '')}.example."+ )+ return subject, body+++def newsletter(i):+ company = random.choice(COMPANIES)+ topic = random.choice(TOPICS)+ subject = f"This week at {company}: {topic}"+ body = (+ f"Hi there — here is this week's digest from {company}. This issue: "+ f"{topic}. We publish every Friday and read every reply, though this "+ f"address is not monitored for support. If this isn't useful anymore, "+ f"unsubscribe any time with the link at the bottom, no hard feelings."+ )+ return subject, body+++def personal(i):+ name = random.choice(FIRST_NAMES)+ friend = random.choice(FIRST_NAMES)+ plan = random.choice(PLANS)+ subject = random.choice([f"hey {name}", "quick one", "this weekend?",+ "long time!", f"re: {friend}'s thing"])+ body = (+ f"Hey {name}, it's been way too long. Was thinking we could "+ f"{plan} — no pressure either way, just wanted to catch up. "+ f"{friend} said you've been swamped. Say hi to everyone for me!"+ )+ return subject, body+++def needs_reply(i):+ company = random.choice(COMPANIES)+ q = random.choice(QUESTIONS)+ subject = f"Need your input: {q}"+ body = (+ f"Hi — following up from {company}. Could you {q} by end of day "+ f"{rand_date()}? I don't want to move ahead without hearing from "+ f"you first. Let me know either way, even if the answer is 'not yet'."+ )+ return subject, body+++GENERATORS = {+ "invoice": invoice,+ "newsletter": newsletter,+ "personal": personal,+ "needs-reply": needs_reply,+}++PER_LABEL_TRAIN = 50+PER_LABEL_TEST = 15++rows = []+for label, gen in GENERATORS.items():+ seen = set()+ n = 0+ i = 0+ while n < PER_LABEL_TRAIN + PER_LABEL_TEST:+ i += 1+ subject, body = gen(i)+ key = subject + body+ if key in seen:+ continue+ seen.add(key)+ rows.append({"label": label, "subject": subject, "text": body,+ "split": "train" if n < PER_LABEL_TRAIN else "test"})+ n += 1++# Hand-authored, deliberately boundary-crossing. The templated generators+# above give each label near-exclusive vocabulary ("Invoice", "unsubscribe",+# "Hey <name>", "confirm by") which the vote can key on trivially — these+# instead borrow another label's words and topic on purpose (money in a+# personal note, a direct question in a newsletter, no "invoice" anywhere in+# an invoice) so a perfect score here would mean something.+HARD = [+ ("invoice", "did that ever go through?",+ "Hey Priya, quick check — did the transfer for last month's freight land "+ "yet? Should have been around forty-eight hundred, reference was on the "+ "packing slip. No rush, just reconciling the books this week."),+ ("personal", "owe you for gas",+ "Hey! Can you send me $40 whenever for the gas we split on the drive back? "+ "Venmo's fine, or just get me back next time you're over. No rush at all."),+ ("newsletter", "before Friday's launch",+ "Quick note from the team before Friday: we could use your take — reply "+ "and tell us which tagline you'd run with, A or B. We publish the results "+ "in next week's issue either way, so no wrong answer, just curious."),+ ("needs-reply", "This week at Cedarline: two quick things",+ "Hi — this week's rundown from the studio: the mockups are attached and "+ "we shipped the new brand kit. One thing before Friday though: I do need "+ "you to confirm the print run number, otherwise the press slot slips."),+ ("invoice", "for the offsite",+ "Hi team, wrapping up the offsite numbers. Catering came to $1,180, venue "+ "was $2,600. Same split as always — let accounting know if your card "+ "wasn't the one charged so we can true it up before month end."),+ ("personal", "hey, free this weekend?",+ "Hey Owen, free this weekend? Also — still owe you for the concert "+ "tickets, want to say $65? Let me know what works, either Venmo or cash "+ "when I see you."),+ ("newsletter", "Bluepeak Logistics: a customer story worth reading",+ "This issue: a customer story worth reading, plus what shipped this "+ "week. We publish every Friday. Reading time about four minutes. "+ "Unsubscribe any time with the link below, no hard feelings."),+ ("needs-reply", "one thing before I book this",+ "Before I book the freight slot I need a yes or no on the pallet count — "+ "24 or 32? Vendor holds the rate until 5pm today, after that it resets "+ "and we lose the discount, so whenever you get a sec."),+]+seen_hard = set()+for label, subject, body in HARD:+ key = subject + body+ assert key not in seen_hard, "duplicate hard example"+ seen_hard.add(key)+ rows.append({"label": label, "subject": subject, "text": body, "split": "hard"})++random.Random(11).shuffle(rows)+json.dump(rows, open("corpus.json", "w"), indent=1)+train = sum(r["split"] == "train" for r in rows)+test = sum(r["split"] == "test" for r in rows)+hard = sum(r["split"] == "hard" for r in rows)+print(f"corpus.json written: {len(rows)} emails ({train} train / {test} test / "+ f"{hard} hard), {len(GENERATORS)} labels")diff --git a/benchmarks/systemone-classify-email/results/run.json b/benchmarks/systemone-classify-email/results/run.jsonnew file mode 100644index 000000000..0f600b34d--- /dev/null+++ b/benchmarks/systemone-classify-email/results/run.json@@ -0,0 +1,528 @@+{+ "easy": {+ "accuracy": 1.0,+ "abstained": 0,+ "mean_confidence": 1.0,+ "ms_per_item": 2.6117324829101562,+ "k": 10,+ "n": 60,+ "confusion": {+ "invoice": {+ "invoice": 15+ },+ "needs-reply": {+ "needs-reply": 15+ },+ "newsletter": {+ "newsletter": 15+ },+ "personal": {+ "personal": 15+ }+ },+ "per_item": [+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: pick a time for the walkthrough"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10052 from Fenwick Cloud \u2014 $30,433.51 due Dec 3"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: pick a time for the walkthrough"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10057 from Cedarline Studio \u2014 $1,914.32 due Apr 10"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: pick a time for the walkthrough"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10061 from Atlas Facilities \u2014 $12,080.77 due Jan 25"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10058 from Greenfield Robotics \u2014 $21,444.33 due Sep 14"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: let me know if the numbers look right"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Ridgeway Consulting: what shipped this week"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Greenfield Robotics: our hiring plans"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Voss & Rane: a customer story worth reading"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: pick a time for the walkthrough"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "quick one"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10053 from Harborline Freight \u2014 $8,405.03 due Mar 19"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10056 from Bluepeak Logistics \u2014 $34,590.95 due Mar 14"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10065 from Northwind Supply \u2014 $6,485.64 due Aug 18"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: confirm your seat for the workshop"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Atlas Facilities: our product roadmap"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Marlowe Analytics: what changed in the release"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: approve the attached budget"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Northwind Supply: a recap of the community call"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "hey Kwame"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: sign off on the draft before Friday"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10063 from Ridgeway Consulting \u2014 $4,127.41 due Nov 17"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "long time!"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "hey Sara"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Voss & Rane: our hiring plans"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "hey Ola"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: approve the attached budget"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Northwind Supply: our hiring plans"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "this weekend?"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "this weekend?"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: pick a time for the walkthrough"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: let me know if the numbers look right"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "long time!"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "re: Nina's thing"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "long time!"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "this weekend?"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "hey Nina"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Bluepeak Logistics: what shipped this week"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Fenwick Cloud: what shipped this week"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10064 from Bluepeak Logistics \u2014 $36,799.07 due Apr 7"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: confirm your seat for the workshop"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Greenfield Robotics: three things we shipped"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10054 from Harborline Freight \u2014 $40,160.76 due Aug 22"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10055 from Harborline Freight \u2014 $36,036.70 due Mar 1"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: sign off on the draft before Friday"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10059 from Northwind Supply \u2014 $23,265.58 due Nov 19"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "this weekend?"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "this weekend?"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Marlowe Analytics: three things we shipped"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Atlas Facilities: a recap of the community call"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10060 from Ridgeway Consulting \u2014 $8,649.68 due Mar 17"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10051 from Atlas Facilities \u2014 $11,779.55 due Nov 11"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Atlas Facilities: what changed in the release"+ },+ {+ "gold": "invoice",+ "pred": "invoice",+ "confidence": 1.0,+ "ok": true,+ "subject": "Invoice INV-10062 from Harborline Freight \u2014 $9,357.60 due Oct 24"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: decide between the two vendors"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "this weekend?"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "This week at Cedarline Studio: what shipped this week"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 1.0,+ "ok": true,+ "subject": "Need your input: let me know if the numbers look right"+ }+ ]+ },+ "hard": {+ "accuracy": 0.625,+ "abstained": 0,+ "mean_confidence": 0.902096625,+ "ms_per_item": 1.665741205215454,+ "k": 10,+ "n": 8,+ "confusion": {+ "invoice": {+ "needs-reply": 1,+ "personal": 1+ },+ "needs-reply": {+ "newsletter": 1,+ "needs-reply": 1+ },+ "newsletter": {+ "newsletter": 2+ },+ "personal": {+ "personal": 2+ }+ },+ "per_item": [+ {+ "gold": "needs-reply",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": false,+ "subject": "This week at Cedarline: two quick things"+ },+ {+ "gold": "invoice",+ "pred": "needs-reply",+ "confidence": 0.836472,+ "ok": false,+ "subject": "for the offsite"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "hey, free this weekend?"+ },+ {+ "gold": "personal",+ "pred": "personal",+ "confidence": 1.0,+ "ok": true,+ "subject": "owe you for gas"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 0.543829,+ "ok": true,+ "subject": "before Friday's launch"+ },+ {+ "gold": "newsletter",+ "pred": "newsletter",+ "confidence": 1.0,+ "ok": true,+ "subject": "Bluepeak Logistics: a customer story worth reading"+ },+ {+ "gold": "invoice",+ "pred": "personal",+ "confidence": 1.0,+ "ok": false,+ "subject": "did that ever go through?"+ },+ {+ "gold": "needs-reply",+ "pred": "needs-reply",+ "confidence": 0.836472,+ "ok": true,+ "subject": "one thing before I book this"+ }+ ]+ }+}\ No newline at end of filediff --git a/benchmarks/systemone-classify-email/run.py b/benchmarks/systemone-classify-email/run.pynew file mode 100644index 000000000..9f811608b--- /dev/null+++ b/benchmarks/systemone-classify-email/run.py@@ -0,0 +1,141 @@+"""Prototype for discussion #1012: label documents at autoindex time from a+local labelled-history index, via the already-shipped POST /_decide endpoint+(engine/crates/xerj-api/src/systemone_api.rs) — no outbound call, no key,+nothing added to the wire client. This measures whether that existing local+mechanism is good enough at the classification the discussion asked about+("invoice / newsletter / personal / needs-reply"), before any autoindex code+is written.++Corpus: gen_corpus.py's synthetic, originally-authored emails (see that+file's docstring for why synthetic, not a real mailbox). k = 10, matching the+k used in the published Banking77 / SMS-spam decisions-as-retrieval numbers.++Usage:+ python3 gen_corpus.py # writes corpus.json (once)+ XERJ_URL=http://localhost:9420 python3 run.py+"""+import collections+import json+import os+import sys+import time+import urllib.request++U = os.environ.get("XERJ_URL", "http://localhost:9420")+K = int(os.environ.get("K", "10"))+INDEX = "email-classify-proto"+++def req(m, p, b=None, ct="application/json"):+ d = b if isinstance(b, (bytes, type(None))) else json.dumps(b).encode()+ r = urllib.request.Request(U + p, data=d, method=m, headers={"content-type": ct})+ try:+ return json.loads(urllib.request.urlopen(r, timeout=120).read())+ except urllib.error.HTTPError as e:+ return {"_err": e.code, "body": e.read().decode()[:400]}+++def load_train(rows):+ train = [r for r in rows if r["split"] == "train"]+ req("DELETE", "/" + INDEX)+ created = req("PUT", "/" + INDEX, {"mappings": {"properties": {+ "label": {"type": "keyword"}, "text": {"type": "text"}}}})+ if "_err" in created:+ print("index create failed:", created)+ sys.exit(1)+ lines = []+ for i, r in enumerate(train):+ lines.append(json.dumps({"index": {"_index": INDEX, "_id": str(i)}}))+ lines.append(json.dumps({"text": r["subject"] + "\n" + r["text"], "label": r["label"]}))+ resp = req("POST", "/_bulk", ("\n".join(lines) + "\n").encode(), "application/x-ndjson")+ if resp.get("errors") or "_err" in resp:+ print("bulk problem:", str(resp)[:400])+ sys.exit(1)+ req("POST", f"/{INDEX}/_refresh")+ count = req("GET", f"/{INDEX}/_count").get("count")+ print(f"{INDEX}: indexed {len(train)} train emails, count={count}")+++def decide(text):+ return req("POST", "/_decide", {"index": INDEX, "question": text, "k": K})+++def confusion_table(labels, cm):+ w = max(len(l) for l in labels) + 1+ header = " " * w + "".join(f"{l[:10]:>11s}" for l in labels) + " <- predicted"+ print(header)+ for gold in labels:+ row = f"{gold:<{w}}" + "".join(f"{cm[gold][pred]:11d}" for pred in labels)+ print(row)+++def evaluate(name, rows, labels):+ cm = {g: collections.Counter() for g in labels}+ correct = 0+ abstained = 0+ confidences = []+ t0 = time.time()+ per_item = []+ for r in rows:+ text = r["subject"] + "\n" + r["text"]+ resp = decide(text)+ if "_err" in resp:+ print("decide failed:", resp)+ sys.exit(1)+ pred = resp.get("label")+ conf = resp.get("confidence", 0.0)+ if resp.get("abstain"):+ abstained += 1+ cm[r["label"]][pred if pred else "(abstain)"] += 1+ confidences.append(conf)+ ok = pred == r["label"]+ correct += ok+ per_item.append({"gold": r["label"], "pred": pred, "confidence": conf,+ "ok": ok, "subject": r["subject"]})+ took_ms = (time.time() - t0) * 1000 / len(rows)++ print(f"\n== {name}: {len(rows)} emails, k={K}, index={INDEX}\n")+ print(f"accuracy: {correct}/{len(rows)} = {correct/len(rows):.4f}")+ print(f"abstained (below decisions.min_confidence, default 0.0 = never): {abstained}")+ print(f"mean confidence: {sum(confidences)/len(confidences):.4f}")+ print(f"latency: {took_ms:.1f} ms/item (local /_decide call, no egress)\n")+ confusion_table(labels, cm)++ for label in labels:+ tp = cm[label][label]+ fn = sum(v for k, v in cm[label].items() if k != label)+ fp = sum(cm[g][label] for g in labels if g != label)+ p = tp / max(tp + fp, 1)+ r_ = tp / max(tp + fn, 1)+ f1 = 2 * p * r_ / max(p + r_, 1e-9)+ print(f" {label:<12s} P={p:.3f} R={r_:.3f} F1={f1:.3f}")++ misses = sorted((i for i in per_item if not i["ok"]), key=lambda i: i["confidence"])+ print(f"\nmisses ({len(misses)}):")+ for m in misses:+ print(f" \"{m['subject'][:40]}\" gold={m['gold']:<12s} "+ f"pred={str(m['pred']):<12s} conf={m['confidence']:.3f}")++ return {"accuracy": correct / len(rows), "abstained": abstained,+ "mean_confidence": sum(confidences) / len(confidences),+ "ms_per_item": took_ms, "k": K, "n": len(rows),+ "confusion": {g: dict(cm[g]) for g in labels}, "per_item": per_item}+++def main():+ rows = json.load(open("corpus.json"))+ load_train(rows)+ labels = sorted({r["label"] for r in rows})++ easy = evaluate("held-out test (templated, same generator as train)",+ [r for r in rows if r["split"] == "test"], labels)+ hard = evaluate("hard set (hand-authored, boundary-crossing vocabulary)",+ [r for r in rows if r["split"] == "hard"], labels)++ json.dump({"easy": easy, "hard": hard},+ open("results/run.json", "w"), indent=1)+ print("\nwrote results/run.json")+++if __name__ == "__main__":+ main()