Skip to content

AI-powered virtual executive team — a single coherent executive persona backed by 8 specialist agents (FastAPI + Next.js).

fix(watchlist): link proposals to their evidence; ground profile entries by name; match short names

Rufusc4f5cc84cc10
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 44f59091..a69cb208 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -28,6 +28,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
department watch interests and the decisions that name a known entity, so
a new watched entity or a relevant decision triggers the next scan.
`/watchlist` shows "for: <department>" on routed rows.
+- **Watch proposals are linked to their evidence, and profile entries ground
+ by name.** `propose_watch` now requires `finding_index`, and when the model
+ omits it the policy links the finding that cites the source itself instead
+ of rejecting the proposal for lack of evidence. Profile competitor / vendor
+ / ticker entries and department watched entities are parsed to their names
+ ("Tesla (TSLA) — Model Y…" grounds as Tesla plus the ticker TSLA; "GM /
+ Chevrolet (…)" as both), so short names such as BYD, GM or Kia match and
+ the description text can no longer stand in for the entity.
- **Research watchlist policy — grounded watches go straight in, uncertain
ones become suggestions.** The research council's watchlist pass no longer
adds watches itself; its only tool is `propose_watch`, and deterministic
diff --git a/packages/core/openexecutive/architecture/architecture-facts.yaml b/packages/core/openexecutive/architecture/architecture-facts.yaml
index 1de6bb62..103a1624 100644
--- a/packages/core/openexecutive/architecture/architecture-facts.yaml
+++ b/packages/core/openexecutive/architecture/architecture-facts.yaml
@@ -1381,7 +1381,14 @@ external_monitoring:
finding_index). `monitoring.research.watch_policy` decides deterministically:
- grounding vocabulary = company name + competitive_landscape.primary_competitors
+ profile.vendors + profile.tickers + active initiative titles +
- strategic_priorities.current_year + each department's `watched_entities`
+ strategic_priorities.current_year + each department's `watched_entities`.
+ Named-entity lists are PARSED (watch_policy.entity_names): only the
+ name(s) before " — " / " - " / ":" ground, "A / B" yields both names, and
+ a parenthesised ticker ("Tesla (TSLA)") becomes a ticker term of its own;
+ the free-text description is dropped (it hid short names such as BYD
+ inside a sentence and would have made "global" BYD's own-source word).
+ Short names (BYD, GM, Kia, AWS) match as whole words because a
+ vocabulary term of any length is a distinctive token.
(kind department_entity, strong) + each department's charter scope
phrases and goal key results (kind department_scope, suggestion-only)
(+ existing watch labels, which only corroborate). The entity MUST
@@ -1419,8 +1426,16 @@ external_monitoring:
grounding_kind) at >= 5 samples (>= 0.7 / <= 0.3 approval rate).
- finding points (high / verified / consensus) count only when the cited
finding concerns THIS source: a URL target shares its site with a cited
- URL; a ticker/query target appears in the finding text or is the named
- entity. A strong finding cannot be borrowed for an unrelated page.
+ URL; a ticker/query target appears in the finding text (as a phrase —
+ "1211.HK" is two words once normalized) or is the named entity. A strong
+ finding cannot be borrowed for an unrelated page.
+ - `finding_index` is REQUIRED in the propose_watch schema and the pass's
+ RULES say so; when the model still omits it (three dev runs on Haiku
+ and Sonnet omitted it on every proposal), apply_proposals auto-links the
+ first finding that concerns the source by the same test
+ (watch_policy.auto_link_finding) and records "linked to finding #N by
+ its cited source" as the first reason. A proposal nothing cites is
+ still rejected as having no evidence.
- direct add at score >= 4 with the mandatory match AND the entity's OWN
source AND a linked finding that concerns this source (company data
plus a decision mention can reach the bar alone; nothing goes live on
diff --git a/packages/core/openexecutive/architecture/prebuilt/external_monitoring.json b/packages/core/openexecutive/architecture/prebuilt/external_monitoring.json
index 730828ca..29f5e2ba 100644
--- a/packages/core/openexecutive/architecture/prebuilt/external_monitoring.json
+++ b/packages/core/openexecutive/architecture/prebuilt/external_monitoring.json
@@ -1,7 +1,7 @@
{
"section_id": "external_monitoring",
"title": "External Monitoring",
- "markdown": "`openexecutive.monitoring` watches the outside world on a heartbeat and turns detected changes into briefing proposals via the same alert pipeline that handles inbound email and Slack.\n\nThe `external_monitor_scan` heartbeat (`__internal__` channel, default every 5 min) walks `list_watchlist(enabled_only=True)`, skipping rows fresher than their per-source cadence floor. For each due row it fires a read-only source adapter resolved from `_REGISTRY` by `signal_type`:\n\n- `vendor_status` — Statuspage Atom/RSS incident feeds (AWS, Stripe, GitHub, …). Reads each incident's current status out of the entry body, parses `<updated>` into `published_at`, and keys its dedup on `(entry id, <updated>)`; a **seeding** source (see below).\n- `rss` — generic Atom/RSS via feedparser (changelogs, layoffs.fyi, filings). Parses each entry's `<pubDate>`/`<updated>` into `published_at`; a **seeding** source (see below).\n- `stock` — Yahoo Finance chart endpoint; fires on `abs_change_pct_gte` moves.\n- `page_watch` — change-detection on a feed-less page. Fetches via the keyless httpx fetcher by default, or via xcrawl's scrape API when the row carries `config.fetch = \"xcrawl\"` — so JS-rendered / bot-blocked pages that publish no RSS (most modern news / marketing sites) are still monitorable.\n- `edgar` and the BILLED `query` web-search adapter.\n\nEach `Signal` is trigger-filtered, severity-clamped to the row's `[floor, ceiling]`, and INSERTed into `external_signals` where `UNIQUE(dedup_key)` drops replays. Survivors pass a suppression cascade (`baseline` → `stale` → `dry_run` → `below_floor` → capture-time relevance enrichment → promote), then `promote_signal_to_alert` builds an `AlertEvent` handed to `alerts.pipeline.schedule_evaluation` and surfaced as a `/today` proposal.\n\nEvery signal carries two timestamps that are never conflated: `captured_at` (when the monitor first saw it) and `published_at` (when the upstream says it happened — RSS `<pubDate>`, EDGAR filing date; null for `stock`, `page_watch`, `query`). Two freshness gates run first in the cascade, before anything that costs an LLM call. **Baseline:** the first poll that returns entries for a row whose adapter declares `seed_on_first_poll` (`rss`, `edgar`) records every entry already in the feed — trigger misses included, so the first entry that ever matches a keyword is news and a later, wider trigger can't resurface old ones — as `suppressed_baseline`, promotes nothing, and stamps `watchlist.baselined_at`, so a new watch reports what changes from now on instead of replaying the feed's back-catalogue (the same idea as `page_watch`'s first-observation baseline). It is keyed on `baselined_at` rather than `last_polled_at`, so a failed first fetch can't forfeit it; the claim and the recording are one transaction (a compare-and-swap on the stamp plus the rows, all-or-nothing), so of two overlapping scans exactly one records the back-catalogue and a failure part-way leaves nothing behind to replay; the other scan promotes only entries dated after that stamp (undated entries count as baseline, since the feed controls the date; entries dated more than a few minutes past its own clock are deferred rather than recorded); and baseline rows don't count against `max_signals_per_scan`, so a large feed can't be half-baselined. `vendor_status` used to sit outside both gates for a structural reason: its dedup key was `sha256(slug, entry id)`, and an incident's id never changes across its updates, so any suppressed insert muted that incident for good. Keying on `(entry id, <updated>)` instead (issue #90) makes a suppression cost one *state* of an incident rather than the incident, so both gates now apply to vendor feeds too — and an incident moving Investigating → Resolved mints a new key and surfaces, while re-polling an unchanged feed still dedups. Because a history feed's archive is mostly incidents resolved months ago, the adapter also carves live news out of its own baseline: it reads each incident's current status from the entry body (Statuspage renders the newest update first as `<strong>Resolved</strong>` / `Investigating` / `Monitoring` / … — the body is parsed by a real HTML parser, off the event loop, and the reading is deliberately asymmetric: any sign that the incident is over closes it, wherever it appears, while only a marked-up label can say it is still open. Working out which update is newest turned out to be something tag soup cannot be trusted to express, and every attempt reported a long-resolved incident as live) and exempts the ones still OPEN via the optional `promote_on_baseline` hook, so a watch added during an outage reports that outage on its first poll while the resolved archive is recorded as `suppressed_baseline`. An unrecognised status counts as closed — a vendor changing feed format must not be able to promote its whole archive at HIGH, and nothing is lost, since the next `<updated>` bump surfaces normally. An exempt entry must still pass the row's trigger filter (one that misses stays in the baseline, so widening a trigger later cannot resurface it) and is still judged by the stale gate below: the exemption buys an entry past the baseline, never past its own timestamp, because \"this is live\" is a reading of vendor-controlled markup and an entry the feed dates outside the age window is not news whatever that reading says. **Stale:** a signal whose `published_at` is older than `EXTERNAL_MONITOR_MAX_SIGNAL_AGE_DAYS` (default 7; `0` disables) at capture time is recorded as `suppressed_stale` and never promoted — a January article a feed resurfaces in September is not September news. An entry dated more than `EXTERNAL_MONITOR_MAX_FUTURE_SKEW_HOURS` ahead (default 24; `0` disables; `config_json.max_future_skew_hours` overrides it per row for calendar or maintenance feeds that legitimately date entries ahead) is deferred instead: skipped for the tick and not recorded (recording would burn its dedup key and let a feed mute an announcement for good by post-dating it), with one aggregated `external_signal_deferred` audit row per poll — carrying each held-back entry's summary and link, and emitted even if the poll later raises — then judged normally once the date passes. Inside the window, the alert body handed to triage carries `Published:` and `Discovered:` lines so the item is judged on when it happened. Adapters prefer a plausible date key over one more than a day in the future (and keep the future value when every key is implausible, so the pipeline defers it rather than waving an undated entry through), and the alert body collapses whitespace in the summary so no source can forge its own labeled body line.\n\nThe `query` adapter is a special case: it emits every web hit flat-LOW (it can't judge materiality per-hit), so for query signals enrichment runs BEFORE the floor gate and `_grade_query_severity` regrades from the relevance read (≥0.8 HIGH, ≥0.5 MEDIUM, else LOW). Without this, any query watch item with a `severity_floor` above LOW would suppress every hit — which is what left the Monitoring section empty even while standing queries were returning material findings.\n\nWhich lane a proposal lands in is decided by `briefing.ranking.categorize`: an unrouted, low/medium signal from a monitoring source (`stock`, `vendor_status`, `rss`, `query`, `edgar`, `page_watch`, or any `external:*` topic tag) is `monitoring` (the collapsible Monitoring section); a routed or high/urgent one is `action` (the \"Needs you\" queue). The triage prompt's External Signals rule covers all six source kinds so it trusts the source and emits the `external:*` tags that drive this split.\n\nThe watchlist is editable from chat (`add_watchlist_entry`, `tune_watchlist_entry`) and HTTP (`/watchlist`); the `executive_research` workflow proposes what to watch through a deterministic policy (next section). Because that workflow is LLM-driven and historically proposed feed URLs it never verified (most were dead — 404 / 403 / not-actually-a-feed), every `rss` target is validated at insert time (`monitoring.target_validation`): a URL that fetches and parses as a feed is kept; a public page that isn't a feed is converted to a scrape-backed `page_watch` when xcrawl can read it, otherwise rejected (HTTP 422 / tool error); SSRF-blocked or unparseable URLs are rejected outright and never handed to xcrawl. Non-`rss` types insert unchanged with no network I/O.\n\n### One watch, one card — and dismissals teach the watch\n\n`promote_signal_to_alert` stamps the `AlertEvent` with `dedup_hint=\"watch:<slug>\"`, which `alerts.pipeline` uses as the *alert* dedup key. A repeat from the same watch coalesces into the open card (`occurrence_count`, `last_seen_at`, severity max) rather than minting another, and re-pings only when severity rose into high/urgent — so a ticker that stays over its threshold or an incident that keeps updating no longer produces a new \"Needs you\" item every day. The signal-level `dedup_key` (stock's UTC-date bucket, vendor_status's `<updated>`) is untouched, so every observation still lands in `external_signals`. Dismissing a watch-sourced alert calls `store.record_dismissal(slug)` (`dismiss_count += 1`, `trust_score = max(0.2, trust × 0.8)`); approving calls `record_confirmation` (+0.05). `trust_score` discounts the briefing ranking and is shown to the Executive's alert review as evidence, alongside the newer signals for that watch (a \"Resolved\" incident state, a reversed move) that let it close the card with evidence.\n\n### What the research pass may put on the watchlist\n\nEvery watchlist row carries an `origin`: `manual` (the principal, via UI/API), `executive` (the chat tool acting on a request), `research` (the research pass added it on its own) or `research_proposed` (a pending suggestion). The `executive_research` workflow's watchlist pass no longer calls `add_watchlist_entry`; its only tool is `propose_watch` (slug, signal_type, target, `grounding_entity`, `rationale`, `certainty`), and `monitoring.research.watch_policy` decides what happens to each proposal from company data, not from the model's confidence in itself:\n\n- **Direct add** — the row lands live, `origin=research`, with quiet defaults the model cannot loosen (daily cadence — hourly for `vendor_status`, weekly for `page_watch`/`query` — `severity_floor=medium`, a keyword trigger of material-event words + the current-year priority terms for `rss` / `query` only — never for `page_watch`, whose summary is just \"page changed\" and would never match, and never the entity's own name, which appears in every feed entry's label and would fire on everything — and `abs_change_pct_gte=5` for stocks). Requires the `grounding_entity` to name an external entity in company data — the company itself, a `competitive_landscape.primary_competitors` entry, one of the profile's `vendors` / `tickers`, or an entity a department lists under its **watched entities** (an initiative title, a current-year priority, a department's charter-scope phrase or a goal's key result also grounds a proposal, but only ever to a suggestion: they are bags of common words a planted finding can echo) — *and* a score ≥ 4 on a fixed ledger: +2 entity in company data (mandatory), +1 entity already watched under another source, +1 target is the entity's own source (for stock/edgar the grounding entity must be the ticker itself; a URL whose site label — `acme` in `status.acme.com` / `acme.co.uk`, never a subdomain or TLD — is the entity's leading name or all its words run together, never a generic trailing word; or a primary-source host such as sec.gov), +1 finding confidence high, +1 verification confirmed, +1 cross-specialist consensus (the three finding points count only when the cited finding concerns this source: a URL target must share its site with one of the finding's cited URLs, a ticker must appear in its text — a strong finding cannot be borrowed for an unrelated page), +1 the entity (a company-data term, never a bare watch label) is named in one of the ten most recent episodic decisions from the last 90 days — the whole term, or every distinctive word of it, as whole words — ±1 from the policy's own approval history for that `(signal_type, grounding kind)` once it has ≥5 samples. A direct add is further reserved for the entity's **own** source (its ticker, its site, its status page): a well-corroborated third-party page about a competitor is the principal's call. It also always needs a linked finding that cites the source — company data plus a decision mention can reach the score bar alone, but nothing goes live on hearsay. The model's `certainty` can only downgrade a direct add to a suggestion; a `query` watch is always a suggestion. At most `WATCHLIST_RESEARCH_MAX_DIRECT_ADDS` (2) per run. When one term appears in several sources the strongest kind wins (company > competitor > vendor > ticker > department entity > initiative > priority > department scope > watch label); a term that came from a department carries the department along even when a stronger profile kind wins, so a profile vendor that Finance also lists stays a vendor and still routes to Finance.\n- **Suggestion** — anything that validates, has a linked finding and scores at least 1 but misses the bar is inserted in `dry_run` (polls and records signals, never alerts) with `origin=research_proposed` and the rationale in `notes`, at most `WATCHLIST_RESEARCH_MAX_PROPOSALS` (2) per run. `/watchlist` shows them in a \"Suggested by the Executive\" section with Approve (`POST /watchlist/{slug}/approve` → `active`, `origin=research`) and Decline (`POST /watchlist/{slug}/decline` with a reason: `not_relevant` blacklists the entity, `wrong_source` blacklists only this target, `too_noisy` keeps the source live with `severity_floor=high` — the one way a suggestion goes live without an approve — and counts as a decline for calibration). A suggestion nobody reviews within `WATCHLIST_PROPOSAL_TTL_DAYS` (14) is removed. The morning brief mentions the pending count in one line; a `watchlist_suggestions` alert nudges only once ≥3 suggestions have waited ≥7 days — one card for the whole pile (an open card is refreshed in place; a dismissed one is not re-minted until the next ISO week), so acting on a suggestion never spawns another nudge.\n- **Routed to a department** — a watch grounded in a department's term (or whose entity a recent decision recorded for that department names) is inserted with `route_to_department` and `route_to_person_id` set to the department and its head at insert time. `_signal_to_alert_event` now maps both onto the `AlertEvent` for every watch (`routed_to_person_id`, `department=<slug>` — a tag hint, deliberately not the `channel` field triage reads as a team room), so the alerts it raises queue on the head's `/today` (as action items, since routed alerts are never hidden in Monitoring) and carry the `department:<slug>` tag instead of landing in the principal's unrouted queue. Suggestions in a department's area are put in front of its head, not the principal: after each run, one \"Watch suggestions for <Department>\" card per department is filed through `propose_via_alert` (routed to the head, or the principal when there is no head; an open card is refreshed in place and an acknowledged one is not re-minted until the next ISO week), and the principal's pile-up nudge counts only suggestions that reached no head — the principal's own and those of a department without a head — so none can become invisible. `/watchlist` labels routed rows \"for: <department>\".\n- **Reject** — an invalid target, a second watch on the same site + signal type (disabled rows included), a declined target or entity, nothing vouching for it (no linked finding, score 0), the enabled-watch ceiling `WATCHLIST_MAX_ENABLED` (40; suggestions poll too, so the ceiling stops them as well), or a spent budget. Every decision is audited (`watchlist_research_added` / `_suggested` / `_rejected`) with the score and reasons; the stamp also lives in the row's `config_json._policy` so approve/decline/auto-disable can attribute the outcome.\n\n**Declines are remembered and enforced.** `watchlist_declines` is keyed by the normalized target (upper-cased ticker; URL with `https` forced, `www.` dropped, host and path lower-cased, query, fragment and trailing slash removed; free-text queries case- and whitespace-folded). A `not_relevant` decline also blacklists the *entity*: any later proposal whose `grounding_entity` names it is refused whatever the target. An explicit decline — the Decline button, \"Stop watching…\" on a research-added card, `remove_watchlist_entry` or `DELETE /watchlist/{slug}` on a research-origin row — is permanent; an unreviewed expiry may be re-proposed once after 90 days. The `propose_watch` handler refuses a declined target or entity before anything is fetched, so a new slug cannot route around a decline (URL keys also fold percent-encoding, dot segments and a trailing host dot); the list is also rendered to the model as DECLINED. The routing pass that runs before the watchlist pass has no watchlist write tools at all (`add_watchlist_entry`, `tune_watchlist_entry`, `remove_watchlist_entry` are withheld from synthesis), and `tune_watchlist_entry` accepts only `notes` on a pending suggestion — never `mode` / `enabled` (only the principal's approve or decline moves one out of review) nor cadence / floor / trigger (it goes live with the quiet defaults policy gave it). Every URL-typed target (`rss`, `vendor_status`, `page_watch`) is checked against the SSRF guard at insert time, and a suggestion's `source_url` link is kept only when it is a public http(s) URL.\n\n**The policy learns from outcomes.** Approve, decline, expiry and auto-disable write `watchlist_policy_outcomes` rows keyed by `(signal_type, grounding kind, specialist)`; the pass renders the tallies (\"stock watches grounded in a competitor: 4 approved / 1 declined\"; \"proposals from cso: …\") and the score ledger reads them. The research turn also shows each existing watch's `origin`, `fired_count`, `dismiss_count` and `trust_score`.\n\n**Research watches retire themselves.** The scheduler's 15-minute sweep (`watch_policy.sweep`, alongside the alert expiry sweep) disables a `research`-origin watch — audited as `watchlist_auto_disabled`, re-enable from `/watchlist` — when it fired ≥5 alerts of which ≥2 were dismissed and trust ≤ 0.5, or after 3 consecutive poll failures (silence is deliberately not a rule: an unchanged page or a healthy vendor is a working watch); the retired target is also recorded as a 90-day decline so the next run cannot re-add it under a new slug with fresh counters (a disabled row already counts as \"same source watched\"). Approve, decline and expiry are compare-and-act on \"still pending\", so a concurrent approve always wins over a decline or the sweep. Direct adds and auto-disables appear in the brief's \"handled overnight\" block as `watching` / `stopped_watching`. The research skip-if-unchanged fingerprint hashes only enabled, `active` rows, so approving or declining a suggestion never triggers a fresh 7-specialist run.\n\n### Invariants\n\n- Adapters are read-only — they never write upstream and never auto-execute.\n- Every `Signal` carries a deterministic `dedup_key` and a mandatory `provenance_url`.\n- Fetches are SSRF-guarded and bounded by `external_monitor_max_fetch_bytes`; xcrawl scrape (off by default, gated by `XCRAWL_ENABLED` + `XCRAWL_API_KEY`) is the one path that delegates the fetch to an external API.\n- Suppressed signals are still persisted so audits show the full picture.\n- A repeat from the same watch never becomes a second open card: the alert-level `watch:<slug>` key coalesces it, and only a severity escalation re-dispatches.\n- The research model never writes the watchlist: it proposes, deterministic policy adds (only when grounded in company data) or suggests, and a target the principal declined is refused at the handler.",
+ "markdown": "`openexecutive.monitoring` watches the outside world on a heartbeat and turns detected changes into briefing proposals via the same alert pipeline that handles inbound email and Slack.\n\nThe `external_monitor_scan` heartbeat (`__internal__` channel, default every 5 min) walks `list_watchlist(enabled_only=True)`, skipping rows fresher than their per-source cadence floor. For each due row it fires a read-only source adapter resolved from `_REGISTRY` by `signal_type`:\n\n- `vendor_status` — Statuspage Atom/RSS incident feeds (AWS, Stripe, GitHub, …). Reads each incident's current status out of the entry body, parses `<updated>` into `published_at`, and keys its dedup on `(entry id, <updated>)`; a **seeding** source (see below).\n- `rss` — generic Atom/RSS via feedparser (changelogs, layoffs.fyi, filings). Parses each entry's `<pubDate>`/`<updated>` into `published_at`; a **seeding** source (see below).\n- `stock` — Yahoo Finance chart endpoint; fires on `abs_change_pct_gte` moves.\n- `page_watch` — change-detection on a feed-less page. Fetches via the keyless httpx fetcher by default, or via xcrawl's scrape API when the row carries `config.fetch = \"xcrawl\"` — so JS-rendered / bot-blocked pages that publish no RSS (most modern news / marketing sites) are still monitorable.\n- `edgar` and the BILLED `query` web-search adapter.\n\nEach `Signal` is trigger-filtered, severity-clamped to the row's `[floor, ceiling]`, and INSERTed into `external_signals` where `UNIQUE(dedup_key)` drops replays. Survivors pass a suppression cascade (`baseline` → `stale` → `dry_run` → `below_floor` → capture-time relevance enrichment → promote), then `promote_signal_to_alert` builds an `AlertEvent` handed to `alerts.pipeline.schedule_evaluation` and surfaced as a `/today` proposal.\n\nEvery signal carries two timestamps that are never conflated: `captured_at` (when the monitor first saw it) and `published_at` (when the upstream says it happened — RSS `<pubDate>`, EDGAR filing date; null for `stock`, `page_watch`, `query`). Two freshness gates run first in the cascade, before anything that costs an LLM call. **Baseline:** the first poll that returns entries for a row whose adapter declares `seed_on_first_poll` (`rss`, `edgar`) records every entry already in the feed — trigger misses included, so the first entry that ever matches a keyword is news and a later, wider trigger can't resurface old ones — as `suppressed_baseline`, promotes nothing, and stamps `watchlist.baselined_at`, so a new watch reports what changes from now on instead of replaying the feed's back-catalogue (the same idea as `page_watch`'s first-observation baseline). It is keyed on `baselined_at` rather than `last_polled_at`, so a failed first fetch can't forfeit it; the claim and the recording are one transaction (a compare-and-swap on the stamp plus the rows, all-or-nothing), so of two overlapping scans exactly one records the back-catalogue and a failure part-way leaves nothing behind to replay; the other scan promotes only entries dated after that stamp (undated entries count as baseline, since the feed controls the date; entries dated more than a few minutes past its own clock are deferred rather than recorded); and baseline rows don't count against `max_signals_per_scan`, so a large feed can't be half-baselined. `vendor_status` used to sit outside both gates for a structural reason: its dedup key was `sha256(slug, entry id)`, and an incident's id never changes across its updates, so any suppressed insert muted that incident for good. Keying on `(entry id, <updated>)` instead (issue #90) makes a suppression cost one *state* of an incident rather than the incident, so both gates now apply to vendor feeds too — and an incident moving Investigating → Resolved mints a new key and surfaces, while re-polling an unchanged feed still dedups. Because a history feed's archive is mostly incidents resolved months ago, the adapter also carves live news out of its own baseline: it reads each incident's current status from the entry body (Statuspage renders the newest update first as `<strong>Resolved</strong>` / `Investigating` / `Monitoring` / … — the body is parsed by a real HTML parser, off the event loop, and the reading is deliberately asymmetric: any sign that the incident is over closes it, wherever it appears, while only a marked-up label can say it is still open. Working out which update is newest turned out to be something tag soup cannot be trusted to express, and every attempt reported a long-resolved incident as live) and exempts the ones still OPEN via the optional `promote_on_baseline` hook, so a watch added during an outage reports that outage on its first poll while the resolved archive is recorded as `suppressed_baseline`. An unrecognised status counts as closed — a vendor changing feed format must not be able to promote its whole archive at HIGH, and nothing is lost, since the next `<updated>` bump surfaces normally. An exempt entry must still pass the row's trigger filter (one that misses stays in the baseline, so widening a trigger later cannot resurface it) and is still judged by the stale gate below: the exemption buys an entry past the baseline, never past its own timestamp, because \"this is live\" is a reading of vendor-controlled markup and an entry the feed dates outside the age window is not news whatever that reading says. **Stale:** a signal whose `published_at` is older than `EXTERNAL_MONITOR_MAX_SIGNAL_AGE_DAYS` (default 7; `0` disables) at capture time is recorded as `suppressed_stale` and never promoted — a January article a feed resurfaces in September is not September news. An entry dated more than `EXTERNAL_MONITOR_MAX_FUTURE_SKEW_HOURS` ahead (default 24; `0` disables; `config_json.max_future_skew_hours` overrides it per row for calendar or maintenance feeds that legitimately date entries ahead) is deferred instead: skipped for the tick and not recorded (recording would burn its dedup key and let a feed mute an announcement for good by post-dating it), with one aggregated `external_signal_deferred` audit row per poll — carrying each held-back entry's summary and link, and emitted even if the poll later raises — then judged normally once the date passes. Inside the window, the alert body handed to triage carries `Published:` and `Discovered:` lines so the item is judged on when it happened. Adapters prefer a plausible date key over one more than a day in the future (and keep the future value when every key is implausible, so the pipeline defers it rather than waving an undated entry through), and the alert body collapses whitespace in the summary so no source can forge its own labeled body line.\n\nThe `query` adapter is a special case: it emits every web hit flat-LOW (it can't judge materiality per-hit), so for query signals enrichment runs BEFORE the floor gate and `_grade_query_severity` regrades from the relevance read (≥0.8 HIGH, ≥0.5 MEDIUM, else LOW). Without this, any query watch item with a `severity_floor` above LOW would suppress every hit — which is what left the Monitoring section empty even while standing queries were returning material findings.\n\nWhich lane a proposal lands in is decided by `briefing.ranking.categorize`: an unrouted, low/medium signal from a monitoring source (`stock`, `vendor_status`, `rss`, `query`, `edgar`, `page_watch`, or any `external:*` topic tag) is `monitoring` (the collapsible Monitoring section); a routed or high/urgent one is `action` (the \"Needs you\" queue). The triage prompt's External Signals rule covers all six source kinds so it trusts the source and emits the `external:*` tags that drive this split.\n\nThe watchlist is editable from chat (`add_watchlist_entry`, `tune_watchlist_entry`) and HTTP (`/watchlist`); the `executive_research` workflow proposes what to watch through a deterministic policy (next section). Because that workflow is LLM-driven and historically proposed feed URLs it never verified (most were dead — 404 / 403 / not-actually-a-feed), every `rss` target is validated at insert time (`monitoring.target_validation`): a URL that fetches and parses as a feed is kept; a public page that isn't a feed is converted to a scrape-backed `page_watch` when xcrawl can read it, otherwise rejected (HTTP 422 / tool error); SSRF-blocked or unparseable URLs are rejected outright and never handed to xcrawl. Non-`rss` types insert unchanged with no network I/O.\n\n### One watch, one card — and dismissals teach the watch\n\n`promote_signal_to_alert` stamps the `AlertEvent` with `dedup_hint=\"watch:<slug>\"`, which `alerts.pipeline` uses as the *alert* dedup key. A repeat from the same watch coalesces into the open card (`occurrence_count`, `last_seen_at`, severity max) rather than minting another, and re-pings only when severity rose into high/urgent — so a ticker that stays over its threshold or an incident that keeps updating no longer produces a new \"Needs you\" item every day. The signal-level `dedup_key` (stock's UTC-date bucket, vendor_status's `<updated>`) is untouched, so every observation still lands in `external_signals`. Dismissing a watch-sourced alert calls `store.record_dismissal(slug)` (`dismiss_count += 1`, `trust_score = max(0.2, trust × 0.8)`); approving calls `record_confirmation` (+0.05). `trust_score` discounts the briefing ranking and is shown to the Executive's alert review as evidence, alongside the newer signals for that watch (a \"Resolved\" incident state, a reversed move) that let it close the card with evidence.\n\n### What the research pass may put on the watchlist\n\nEvery watchlist row carries an `origin`: `manual` (the principal, via UI/API), `executive` (the chat tool acting on a request), `research` (the research pass added it on its own) or `research_proposed` (a pending suggestion). The `executive_research` workflow's watchlist pass no longer calls `add_watchlist_entry`; its only tool is `propose_watch` (slug, signal_type, target, `grounding_entity`, `rationale`, `certainty`), and `monitoring.research.watch_policy` decides what happens to each proposal from company data, not from the model's confidence in itself:\n\n- **Direct add** — the row lands live, `origin=research`, with quiet defaults the model cannot loosen (daily cadence — hourly for `vendor_status`, weekly for `page_watch`/`query` — `severity_floor=medium`, a keyword trigger of material-event words + the current-year priority terms for `rss` / `query` only — never for `page_watch`, whose summary is just \"page changed\" and would never match, and never the entity's own name, which appears in every feed entry's label and would fire on everything — and `abs_change_pct_gte=5` for stocks). Requires the `grounding_entity` to name an external entity in company data — the company itself, a `competitive_landscape.primary_competitors` entry, one of the profile's `vendors` / `tickers`, or an entity a department lists under its **watched entities** (each list entry is parsed to its name: \"Tesla (TSLA) — Model Y is the benchmark\" grounds as `Tesla` plus the ticker `TSLA`, \"GM / Chevrolet (…)\" as both names; the description after the dash never becomes a term, and short names such as BYD or GM match as whole words) (an initiative title, a current-year priority, a department's charter-scope phrase or a goal's key result also grounds a proposal, but only ever to a suggestion: they are bags of common words a planted finding can echo) — *and* a score ≥ 4 on a fixed ledger: +2 entity in company data (mandatory), +1 entity already watched under another source, +1 target is the entity's own source (for stock/edgar the grounding entity must be the ticker itself; a URL whose site label — `acme` in `status.acme.com` / `acme.co.uk`, never a subdomain or TLD — is the entity's leading name or all its words run together, never a generic trailing word; or a primary-source host such as sec.gov), +1 finding confidence high, +1 verification confirmed, +1 cross-specialist consensus (the three finding points count only when the cited finding concerns this source: a URL target must share its site with one of the finding's cited URLs, a ticker must appear in its text — a strong finding cannot be borrowed for an unrelated page), +1 the entity (a company-data term, never a bare watch label) is named in one of the ten most recent episodic decisions from the last 90 days — the whole term, or every distinctive word of it, as whole words — ±1 from the policy's own approval history for that `(signal_type, grounding kind)` once it has ≥5 samples. A direct add is further reserved for the entity's **own** source (its ticker, its site, its status page): a well-corroborated third-party page about a competitor is the principal's call. It also always needs a linked finding that cites the source — company data plus a decision mention can reach the score bar alone, but nothing goes live on hearsay. The model's `certainty` can only downgrade a direct add to a suggestion; a `query` watch is always a suggestion. At most `WATCHLIST_RESEARCH_MAX_DIRECT_ADDS` (2) per run. When one term appears in several sources the strongest kind wins (company > competitor > vendor > ticker > department entity > initiative > priority > department scope > watch label); a term that came from a department carries the department along even when a stronger profile kind wins, so a profile vendor that Finance also lists stays a vendor and still routes to Finance.\n- **Linked finding.** `finding_index` is required on `propose_watch` and the pass is told so; when the model omits it anyway, the policy links the first finding that concerns the source (a cited URL on the target's site, or the ticker / entity named in its text) and says so in the first reason. A proposal that no finding cites is rejected as having no evidence.\n- **Suggestion** — anything that validates, has a linked finding and scores at least 1 but misses the bar is inserted in `dry_run` (polls and records signals, never alerts) with `origin=research_proposed` and the rationale in `notes`, at most `WATCHLIST_RESEARCH_MAX_PROPOSALS` (2) per run. `/watchlist` shows them in a \"Suggested by the Executive\" section with Approve (`POST /watchlist/{slug}/approve` → `active`, `origin=research`) and Decline (`POST /watchlist/{slug}/decline` with a reason: `not_relevant` blacklists the entity, `wrong_source` blacklists only this target, `too_noisy` keeps the source live with `severity_floor=high` — the one way a suggestion goes live without an approve — and counts as a decline for calibration). A suggestion nobody reviews within `WATCHLIST_PROPOSAL_TTL_DAYS` (14) is removed. The morning brief mentions the pending count in one line; a `watchlist_suggestions` alert nudges only once ≥3 suggestions have waited ≥7 days — one card for the whole pile (an open card is refreshed in place; a dismissed one is not re-minted until the next ISO week), so acting on a suggestion never spawns another nudge.\n- **Routed to a department** — a watch grounded in a department's term (or whose entity a recent decision recorded for that department names) is inserted with `route_to_department` and `route_to_person_id` set to the department and its head at insert time. `_signal_to_alert_event` now maps both onto the `AlertEvent` for every watch (`routed_to_person_id`, `department=<slug>` — a tag hint, deliberately not the `channel` field triage reads as a team room), so the alerts it raises queue on the head's `/today` (as action items, since routed alerts are never hidden in Monitoring) and carry the `department:<slug>` tag instead of landing in the principal's unrouted queue. Suggestions in a department's area are put in front of its head, not the principal: after each run, one \"Watch suggestions for <Department>\" card per department is filed through `propose_via_alert` (routed to the head, or the principal when there is no head; an open card is refreshed in place and an acknowledged one is not re-minted until the next ISO week), and the principal's pile-up nudge counts only suggestions that reached no head — the principal's own and those of a department without a head — so none can become invisible. `/watchlist` labels routed rows \"for: <department>\".\n- **Reject** — an invalid target, a second watch on the same site + signal type (disabled rows included), a declined target or entity, nothing vouching for it (no linked finding, score 0), the enabled-watch ceiling `WATCHLIST_MAX_ENABLED` (40; suggestions poll too, so the ceiling stops them as well), or a spent budget. Every decision is audited (`watchlist_research_added` / `_suggested` / `_rejected`) with the score and reasons; the stamp also lives in the row's `config_json._policy` so approve/decline/auto-disable can attribute the outcome.\n\n**Declines are remembered and enforced.** `watchlist_declines` is keyed by the normalized target (upper-cased ticker; URL with `https` forced, `www.` dropped, host and path lower-cased, query, fragment and trailing slash removed; free-text queries case- and whitespace-folded). A `not_relevant` decline also blacklists the *entity*: any later proposal whose `grounding_entity` names it is refused whatever the target. An explicit decline — the Decline button, \"Stop watching…\" on a research-added card, `remove_watchlist_entry` or `DELETE /watchlist/{slug}` on a research-origin row — is permanent; an unreviewed expiry may be re-proposed once after 90 days. The `propose_watch` handler refuses a declined target or entity before anything is fetched, so a new slug cannot route around a decline (URL keys also fold percent-encoding, dot segments and a trailing host dot); the list is also rendered to the model as DECLINED. The routing pass that runs before the watchlist pass has no watchlist write tools at all (`add_watchlist_entry`, `tune_watchlist_entry`, `remove_watchlist_entry` are withheld from synthesis), and `tune_watchlist_entry` accepts only `notes` on a pending suggestion — never `mode` / `enabled` (only the principal's approve or decline moves one out of review) nor cadence / floor / trigger (it goes live with the quiet defaults policy gave it). Every URL-typed target (`rss`, `vendor_status`, `page_watch`) is checked against the SSRF guard at insert time, and a suggestion's `source_url` link is kept only when it is a public http(s) URL.\n\n**The policy learns from outcomes.** Approve, decline, expiry and auto-disable write `watchlist_policy_outcomes` rows keyed by `(signal_type, grounding kind, specialist)`; the pass renders the tallies (\"stock watches grounded in a competitor: 4 approved / 1 declined\"; \"proposals from cso: …\") and the score ledger reads them. The research turn also shows each existing watch's `origin`, `fired_count`, `dismiss_count` and `trust_score`.\n\n**Research watches retire themselves.** The scheduler's 15-minute sweep (`watch_policy.sweep`, alongside the alert expiry sweep) disables a `research`-origin watch — audited as `watchlist_auto_disabled`, re-enable from `/watchlist` — when it fired ≥5 alerts of which ≥2 were dismissed and trust ≤ 0.5, or after 3 consecutive poll failures (silence is deliberately not a rule: an unchanged page or a healthy vendor is a working watch); the retired target is also recorded as a 90-day decline so the next run cannot re-add it under a new slug with fresh counters (a disabled row already counts as \"same source watched\"). Approve, decline and expiry are compare-and-act on \"still pending\", so a concurrent approve always wins over a decline or the sweep. Direct adds and auto-disables appear in the brief's \"handled overnight\" block as `watching` / `stopped_watching`. The research skip-if-unchanged fingerprint hashes only enabled, `active` rows, so approving or declining a suggestion never triggers a fresh 7-specialist run.\n\n### Invariants\n\n- Adapters are read-only — they never write upstream and never auto-execute.\n- Every `Signal` carries a deterministic `dedup_key` and a mandatory `provenance_url`.\n- Fetches are SSRF-guarded and bounded by `external_monitor_max_fetch_bytes`; xcrawl scrape (off by default, gated by `XCRAWL_ENABLED` + `XCRAWL_API_KEY`) is the one path that delegates the fetch to an external API.\n- Suppressed signals are still persisted so audits show the full picture.\n- A repeat from the same watch never becomes a second open card: the alert-level `watch:<slug>` key coalesces it, and only a severity escalation re-dispatches.\n- The research model never writes the watchlist: it proposes, deterministic policy adds (only when grounded in company data) or suggests, and a target the principal declined is refused at the handler.",
"mermaid": "flowchart TD\n ADD[\"add (chat tool / POST /watchlist / research)\"]\n VAL[\"target_validation: rss feed? → keep / convert→page_watch(xcrawl) / reject\"]\n WL[(\"watchlist\")]\n HB[\"external_monitor_scan heartbeat (~5 min)\"]\n AD[\"read-only adapters: vendor_status / rss / stock / query / edgar / page_watch (httpx | xcrawl)\"]\n SIG[(\"external_signals UNIQUE(dedup_key)\")]\n ENRICH[\"capture-time enrichment<br/>(relevance score + why_it_matters)\"]\n PROMO[\"promote_signal_to_alert\"]\n PIPE[\"alert pipeline (triage)\"]\n TODAY[\"/today proposals\"]\n ADD --> VAL\n VAL -->|\"insert valid rows\"| WL\n HB -->|\"walk enabled rows\"| WL\n WL -->|\"poll due rows\"| AD\n AD -->|\"emit Signal\"| SIG\n SIG -->|\"baseline / stale / dry_run / floor gates\"| ENRICH\n ENRICH -->|\"≥ relevance floor\"| PROMO\n PROMO --> PIPE\n PIPE --> TODAY\n class HB hot\n class WL storage\n class VAL compute\n class AD external\n class SIG storage\n class ENRICH compute\n class PROMO compute\n class PIPE compute\n class TODAY entry\n class ADD compute",
"generated_at": "2026-09-12T12:03:38.574978+00:00"
}
diff --git a/packages/core/openexecutive/monitoring/research/scheduler.py b/packages/core/openexecutive/monitoring/research/scheduler.py
index e776e59e..4d055b9c 100644
--- a/packages/core/openexecutive/monitoring/research/scheduler.py
+++ b/packages/core/openexecutive/monitoring/research/scheduler.py
@@ -171,7 +171,7 @@ def compute_research_state_hash(db_path: Path | None = None) -> str:
return [
f"{getattr(decision, 'id', '')}::{str(getattr(decision, 'summary', '') or '')[:120]}"
for decision in watch_policy.recent_decisions(db_path=db_path)
- if any(watch_policy.named_in_decision(term, decision) for term in vocabulary)
+ if any(watch_policy.named_in_decision(term, decision, vocabulary) for term in vocabulary)
]
_append_part(parts, "DECISIONS", _matching_decisions)
diff --git a/packages/core/openexecutive/monitoring/research/watch_policy.py b/packages/core/openexecutive/monitoring/research/watch_policy.py
index 976fb31a..98984eb4 100644
--- a/packages/core/openexecutive/monitoring/research/watch_policy.py
+++ b/packages/core/openexecutive/monitoring/research/watch_policy.py
@@ -292,6 +292,38 @@ def _norm(term: str) -> str:
return " ".join(_TOKEN_RE.findall((term or "").lower()))
+# A profile list entry is often "Name (TICKER) — free-text description" or
+# "Name A / Name B (products…)". Only the NAME grounds; the description is a
+# bag of common words that would both hide the name (short names such as BYD
+# never match inside a sentence) and poison own-source checks ("global" in
+# "BYD — global cost leader" would make global.com BYD's own site).
+_ENTRY_SPLIT_RE = re.compile(r"\s+[—–-]\s+|[:;]")
+_PAREN_RE = re.compile(r"\(([^)]*)\)")
+_TICKER_RE = re.compile(r"^(?:[A-Z]{1,5}(?:\.[A-Z]{1,3})?|\d{3,6}\.[A-Z]{2,3})$")
+
+
+def entity_names(raw: str) -> tuple[list[str], list[str]]:
+ """``(names, tickers)`` parsed from one profile / department entry.
+
+ "Tesla (TSLA) — Model Y is the benchmark" → (["Tesla"], ["TSLA"]);
+ "GM / Chevrolet (Equinox EV, Blazer EV) — direct competitor" →
+ (["GM", "Chevrolet"], []); "Stripe" → (["Stripe"], [])."""
+ text = str(raw or "").strip()
+ if not text:
+ return [], []
+ head = _ENTRY_SPLIT_RE.split(text, maxsplit=1)[0]
+ tickers: list[str] = []
+ for group in _PAREN_RE.findall(head):
+ for part in re.split(r"[,/]", group):
+ candidate = part.strip()
+ if _TICKER_RE.match(candidate) and candidate not in tickers:
+ tickers.append(candidate)
+ bare = _PAREN_RE.sub(" ", head)
+ names = [n.strip(" ,") for n in re.split(r"\s*/\s*", bare)]
+ names = [n for n in names if len(_norm(n)) >= 2]
+ return names, tickers
+
+
def grounding_vocabulary(
profile: Any,
initiatives: list[Any],
@@ -338,23 +370,32 @@ def grounding_vocabulary(
add(item.slug.replace("-", " "), KIND_WATCH)
for i in initiatives:
add(str(getattr(i, "title", "") or ""), KIND_INITIATIVE)
+ def add_entity(raw: str, kind: str, department: str = "") -> None:
+ # Named-entity lists: only the parsed name(s) ground; a parenthesised
+ # ticker is a ticker term of its own.
+ names, tickers = entity_names(raw)
+ for name in names:
+ add(name, kind, department)
+ for ticker in tickers:
+ add(ticker, KIND_TICKER, department)
+
if profile is not None:
for p in list(getattr(getattr(profile, "strategic_priorities", None), "current_year", []) or []):
add(str(p), KIND_PRIORITY)
for t in list(getattr(profile, "tickers", []) or []):
- add(str(t), KIND_TICKER)
+ add_entity(str(t), KIND_TICKER)
for v in list(getattr(profile, "vendors", []) or []):
- add(str(v), KIND_VENDOR)
+ add_entity(str(v), KIND_VENDOR)
for c in list(getattr(getattr(profile, "competitive_landscape", None), "primary_competitors", []) or []):
- add(str(c), KIND_COMPETITOR)
- add(str(getattr(profile, "name", "") or ""), KIND_COMPANY)
+ add_entity(str(c), KIND_COMPETITOR)
+ add_entity(str(getattr(profile, "name", "") or ""), KIND_COMPANY)
for state in departments or []:
config = getattr(state, "config", None)
slug = str(getattr(config, "slug", "") or "")
if not slug:
continue
for entity in list(getattr(config, "watched_entities", []) or []):
- add(str(entity), KIND_DEPARTMENT_ENTITY, slug)
+ add_entity(str(entity), KIND_DEPARTMENT_ENTITY, slug)
charter = getattr(config, "charter", None)
for phrase in list(getattr(charter, "scope", []) or []):
add(str(phrase), KIND_DEPARTMENT_SCOPE, slug)
@@ -426,15 +467,20 @@ def priority_terms(profile: Any) -> list[str]:
return out[:6]
-def _distinctive_tokens(text: str, vocabulary: dict[str, Any] | None = None) -> set[str]:
- """Tokens that may carry a partial match: not stopwords, and either long
- enough or a whole vocabulary term in their own right (short names such
- as IBM, AWS, SAP)."""
+def _name_tokens(text: str, vocabulary: dict[str, Any] | None = None) -> list[str]:
+ """Ordered tokens that may carry a partial match: not stopwords, and
+ either long enough or a whole vocabulary term in their own right (short
+ names such as IBM, AWS, BYD, GM)."""
vocab = vocabulary or {}
- return {
+ return [
t for t in text.split()
if t not in _STOPWORDS and (len(t) >= _MIN_PARTIAL_TOKEN or t in vocab)
- }
+ ]
+
+
+def _distinctive_tokens(text: str, vocabulary: dict[str, Any] | None = None) -> set[str]:
+ """Set form of :func:`_name_tokens`."""
+ return set(_name_tokens(text, vocabulary))
def _entry(value: Any) -> VocabEntry:
@@ -490,7 +536,9 @@ def match_vocab(entity: str, vocabulary: dict[str, Any]) -> tuple[str, VocabEntr
return best
-def named_in_decision(entity_term: str, decision: Any) -> bool:
+def named_in_decision(
+ entity_term: str, decision: Any, vocabulary: dict[str, Any] | None = None,
+) -> bool:
"""Does a recent episodic decision name this entity? The whole term as
a phrase, or EVERY distinctive token of the term as a whole word — so
"Acme Corp" is named by "Acme pricing review" (corp is a stopword) but
@@ -500,9 +548,7 @@ def named_in_decision(entity_term: str, decision: Any) -> bool:
return False
if f" {entity_term} " in f" {text} ":
return True
- entity_tokens = {
- t for t in entity_term.split() if len(t) >= _MIN_PARTIAL_TOKEN and t not in _STOPWORDS
- }
+ entity_tokens = _distinctive_tokens(entity_term, vocabulary)
return bool(entity_tokens) and entity_tokens <= set(text.split())
@@ -511,7 +557,7 @@ def _decision_bonus(entity_term: str, ctx: PolicyContext) -> tuple[int, str, str
when one names the entity; the department it was recorded for (when
the policy knows that department) is offered as a routing hint."""
for recent in ctx.recent_decisions:
- if not named_in_decision(entity_term, recent):
+ if not named_in_decision(entity_term, recent, ctx.vocabulary):
continue
decided_for = str(getattr(recent, "department", "") or "")
return 1, "named in a recent decision", decided_for if decided_for in ctx.departments else ""
@@ -531,7 +577,10 @@ def entity_declined(entity: str, declines: list[Any]) -> bool:
dn = _norm(str(getattr(d, "entity", "") or ""))
if len(dn) < 2:
continue
- if dn == n:
+ # Exact, or a one-word declined name ("BYD") as a whole word of the
+ # entity ("BYD Auto") and vice versa — short names have no
+ # distinctive tokens, so the token rule below cannot see them.
+ if dn == n or (" " not in dn and dn in n.split()) or (" " not in n and n in dn.split()):
return True
d_tokens = {t for t in dn.split() if t not in _STOPWORDS and len(t) >= _MIN_PARTIAL_TOKEN}
if tokens and d_tokens and (d_tokens <= tokens or tokens <= d_tokens):
@@ -565,9 +614,7 @@ def _is_own_source(proposal: WatchProposal, entity_term: str, vocabulary: dict[s
# "Acme Payments" is not payments.io. Entities are free text
# (departments type them), so a generic trailing word must never hand
# an unrelated domain "own source" credit.
- entity_tokens = [
- t for t in entity_term.split() if len(t) >= _MIN_PARTIAL_TOKEN and t not in _STOPWORDS
- ]
+ entity_tokens = _name_tokens(entity_term, vocabulary)
if not entity_tokens:
return False
return _site_label(host) in {entity_tokens[0], "".join(entity_tokens)}
@@ -591,19 +638,25 @@ def _site_label(host: str) -> str:
return labels[-2]
-def _finding_supports(proposal: WatchProposal, finding: ResearchFinding, entity_term: str) -> bool:
+def _finding_supports(
+ proposal: WatchProposal,
+ finding: ResearchFinding,
+ entity_term: str,
+ vocabulary: dict[str, Any] | None = None,
+) -> bool:
"""Does the cited finding actually concern this target? A URL target
must share its site with one of the finding's cited URLs; a ticker or
query target must appear in the finding's text or be about the entity
the finding names."""
text = _norm(f"{finding.title} {finding.summary}")
text_tokens = set(text.split())
- entity_tokens = {
- t for t in entity_term.split() if len(t) >= _MIN_PARTIAL_TOKEN and t not in _STOPWORDS
- }
- names_entity = bool(entity_term) and (entity_term in text or bool(entity_tokens & text_tokens))
+ entity_tokens = _distinctive_tokens(entity_term, vocabulary)
+ names_entity = bool(entity_term) and (
+ f" {entity_term} " in f" {text} " or bool(entity_tokens & text_tokens)
+ )
if proposal.signal_type in (SOURCE_KIND_STOCK, SOURCE_KIND_EDGAR):
- return _norm(proposal.target) in text_tokens or names_entity
+ # Phrase containment: a dotted ticker normalizes to two words.
+ return f" {_norm(proposal.target)} " in f" {text} " or names_entity
if proposal.signal_type == SOURCE_KIND_QUERY:
return names_entity
# A URL target must share its site with a cited URL: a finding that
@@ -708,7 +761,7 @@ def classify(
if own_source:
score += 1
reasons.append("target is the entity's own source")
- supported = finding is not None and _finding_supports(proposal, finding, entity_term)
+ supported = finding is not None and _finding_supports(proposal, finding, entity_term, ctx.vocabulary)
if finding is not None and not supported:
# The cited finding says nothing about THIS source, so it lends it
# no credit — the model cannot borrow a strong finding's points for
@@ -842,6 +895,22 @@ def _policy_stamp(decision: Decision, proposal: WatchProposal, finding: Research
}
+def auto_link_finding(
+ proposal: WatchProposal, findings: list[ResearchFinding], ctx: PolicyContext,
+) -> int | None:
+ """Index of the first finding that concerns this proposal's source (a
+ cited URL on the target's site; for a ticker or query, the ticker or
+ the grounding entity named in its text), for a proposal the model filed
+ without a ``finding_index``. The link is the same test the score uses,
+ so it can never lend a proposal a finding the score would not honour."""
+ match = match_vocab(proposal.grounding_entity, ctx.vocabulary)
+ entity_term = match[0] if match else _norm(proposal.grounding_entity)
+ for index, finding in enumerate(findings):
+ if _finding_supports(proposal, finding, entity_term, ctx.vocabulary):
+ return index
+ return None
+
+
def policy_stamp_of(item: WatchlistItem) -> dict[str, Any]:
stamp = item.config_json.get("_policy") if isinstance(item.config_json, dict) else None
return dict(stamp) if isinstance(stamp, dict) else {}
@@ -878,7 +947,19 @@ def apply_proposals(
continue
seen_targets.add(proposal.normalized_target)
+ linked_note = ""
+ if finding is None:
+ # The model routinely omits finding_index; the finding that
+ # cites this source is still the evidence, so link it.
+ auto_index = auto_link_finding(proposal, findings, ctx)
+ if auto_index is not None:
+ finding = findings[auto_index]
+ proposal.finding_index = auto_index
+ linked_note = f"linked to finding #{auto_index + 1} by its cited source"
+
decision = classify(proposal, finding, ctx)
+ if linked_note:
+ decision.reasons.insert(0, linked_note)
if decision.tier == TIER_DIRECT and direct_left <= 0:
decision.tier = TIER_SUGGEST
decision.reasons.append("direct-add budget spent this run")
@@ -1252,9 +1333,11 @@ __all__ = [
"VocabEntry",
"WatchProposal",
"apply_proposals",
+ "auto_link_finding",
"classify",
"department_refs",
"entity_declined",
+ "entity_names",
"grounding_vocabulary",
"load_departments",
"match_entity",
diff --git a/packages/core/openexecutive/orchestrator/watchlist_tools.py b/packages/core/openexecutive/orchestrator/watchlist_tools.py
index a5e6712f..d3911b10 100644
--- a/packages/core/openexecutive/orchestrator/watchlist_tools.py
+++ b/packages/core/openexecutive/orchestrator/watchlist_tools.py
@@ -339,7 +339,11 @@ PROPOSE_WATCH_TOOL: dict[str, Any] = {
},
"finding_index": {
"type": "integer",
- "description": "The 1-based '#N' of the finding this proposal comes from.",
+ "description": (
+ "REQUIRED. The 1-based '#N' of the finding this proposal comes "
+ "from — the one whose urls or text name this source. A proposal "
+ "with no finding behind it is rejected as having no evidence."
+ ),
},
"certainty": {
"type": "string",
@@ -354,7 +358,9 @@ PROPOSE_WATCH_TOOL: dict[str, Any] = {
"trigger": {"type": "object", "description": "Optional adapter trigger; policy adds quiet defaults."},
"route_to_specialist": {"type": "string"},
},
- "required": ["slug", "signal_type", "target", "grounding_entity", "rationale", "certainty"],
+ "required": [
+ "slug", "signal_type", "target", "grounding_entity", "rationale", "finding_index", "certainty",
+ ],
},
}
@@ -565,6 +571,8 @@ async def handle_propose_watch(tool_input: dict[str, Any], collector: list[Any])
finding_index: int | None
try:
# 1-based "#N" from the findings turn; 0 or negative means "none".
+ # Tolerated here even though the schema requires it: the policy
+ # links the finding that cites the source when it is missing.
finding_index = int(raw_index) - 1 if raw_index is not None and int(raw_index) >= 1 else None
except (TypeError, ValueError):
finding_index = None
diff --git a/packages/core/openexecutive/workflows/executive_research.py b/packages/core/openexecutive/workflows/executive_research.py
index 6fb79b82..7efa412f 100644
--- a/packages/core/openexecutive/workflows/executive_research.py
+++ b/packages/core/openexecutive/workflows/executive_research.py
@@ -258,6 +258,9 @@ def _build_watchlist_system(max_direct: int, max_suggest: int) -> str:
" - `target` MUST be concrete and real: a ticker symbol, or a URL "
"taken from a finding's `urls` list — do NOT invent feed URLs. If you "
"cannot give a real target, do not propose.\n"
+ " - `finding_index` is REQUIRED: the #N of the finding whose urls or "
+ "text this source comes from. A proposal with no finding behind it is "
+ "rejected as having no evidence, however well grounded.\n"
" - `certainty` = 'confident' ONLY when the entity is in the company "
"context AND the target is the entity's own source (its ticker, its "
"site, its status page). Otherwise 'unsure'.\n"
diff --git a/packages/core/tests/unit/test_executive_research_watchlist.py b/packages/core/tests/unit/test_executive_research_watchlist.py
index d353f44a..bdfefdb9 100644
--- a/packages/core/tests/unit/test_executive_research_watchlist.py
+++ b/packages/core/tests/unit/test_executive_research_watchlist.py
@@ -284,6 +284,30 @@ def test_research_context_renders_recent_decisions() -> None:
assert "RECENT DECISIONS" not in without
+def test_propose_watch_requires_a_finding_and_the_prompt_says_so() -> None:
+ from openexecutive.orchestrator.watchlist_tools import PROPOSE_WATCH_TOOL
+
+ assert "finding_index" in PROPOSE_WATCH_TOOL["input_schema"]["required"]
+ system = er._build_watchlist_system(2, 2)
+ assert "`finding_index` is REQUIRED" in system
+
+
+async def test_watchlist_pass_links_the_finding_when_the_model_omits_it(
+ db: Path, monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ call = _propose("rss-tesla-ir", "rss", "https://ir.tesla.com/press.xml", entity="Tesla")
+ del call[1]["finding_index"]
+ _stub_provider(monkeypatch, [_resp([call]), _resp([])])
+ out = await er._watchlist_analysis_loop(
+ [_finding(title="Tesla cut Model Y prices", url="https://ir.tesla.com/press-release/model-y")],
+ [], profile=_profile(), initiatives=[],
+ )
+ assert [c["outcome"] for c in out] == ["added"]
+ row = monitoring_store.get_watchlist_item_by_slug("rss-tesla-ir", db_path=db)
+ assert row is not None and row.mode == "active"
+
+
@pytest.mark.asyncio
async def test_watchlist_pass_routes_department_grounded_watch(
db: Path, monkeypatch: pytest.MonkeyPatch,
diff --git a/packages/core/tests/unit/test_watch_policy.py b/packages/core/tests/unit/test_watch_policy.py
index 6877e0b6..d931d59b 100644
--- a/packages/core/tests/unit/test_watch_policy.py
+++ b/packages/core/tests/unit/test_watch_policy.py
@@ -913,3 +913,101 @@ def test_nudge_counts_only_suggestions_no_head_received(db: Path) -> None:
)
_backdate(db, f"rss-n{i}", 8)
assert wp.sweep(datetime.now(UTC), db_path=db)["nudged"] == 1
+
+
+# --------------------------------------------------------------------- #
+# Profile entry parsing, short names, finding auto-link
+# --------------------------------------------------------------------- #
+
+
+def test_entity_names_parse_profile_entries() -> None:
+ assert wp.entity_names("Tesla (TSLA) — Model Y is the volume benchmark") == (["Tesla"], ["TSLA"])
+ assert wp.entity_names("GM / Chevrolet (Equinox EV, Blazer EV) — direct competitor") == (["GM", "Chevrolet"], [])
+ assert wp.entity_names("BYD — global cost leader; tariff-gated out of the US") == (["BYD"], [])
+ assert wp.entity_names("Hyundai / Kia (Ioniq 5, EV6, EV9) - strong value") == (["Hyundai", "Kia"], [])
+ assert wp.entity_names("Rivian (RIVN, 1211.HK)") == (["Rivian"], ["RIVN", "1211.HK"])
+ assert wp.entity_names("Stripe: payments") == (["Stripe"], [])
+ assert wp.entity_names(" ") == ([], [])
+
+
+def _descriptive_profile() -> CompanyProfile:
+ return CompanyProfile.model_validate({
+ "name": "Halcyon Motors",
+ "competitive_landscape": {"primary_competitors": [
+ "Tesla (TSLA) — Model Y is the volume benchmark and price-cut pace-setter",
+ "BYD — global cost leader; tariff-gated out of the US for now",
+ "GM / Chevrolet (Equinox EV, Blazer EV) — direct mainstream-price competitor",
+ "Hyundai / Kia (Ioniq 5, EV6, EV9) — strong value + charging speed",
+ ]},
+ "vendors": ["CATL — sole LFP cell source"],
+ "tickers": ["1211.HK"],
+ })
+
+
+def test_descriptive_profile_entries_ground_by_name_only() -> None:
+ vocab = wp.grounding_vocabulary(_descriptive_profile(), [], [])
+ assert vocab["tesla"].kind == wp.KIND_COMPETITOR and vocab["tsla"].kind == wp.KIND_TICKER
+ assert vocab["byd"].kind == wp.KIND_COMPETITOR and vocab["gm"].kind == wp.KIND_COMPETITOR
+ assert vocab["chevrolet"].kind == wp.KIND_COMPETITOR and vocab["kia"].kind == wp.KIND_COMPETITOR
+ assert vocab["catl"].kind == wp.KIND_VENDOR and vocab["1211 hk"].kind == wp.KIND_TICKER
+ # The description never becomes a term, so "Model Y" or "global" ground nothing.
+ assert wp.match_entity("Model Y", vocab) is None and wp.match_entity("global cost leader", vocab) is None
+ # Short names match as whole words, in longer forms too.
+ assert wp.match_entity("BYD", vocab) == ("byd", wp.KIND_COMPETITOR)
+ assert wp.match_entity("BYD Auto", vocab) == ("byd", wp.KIND_COMPETITOR)
+ assert wp.match_entity("GM", vocab) == ("gm", wp.KIND_COMPETITOR)
+ assert wp.match_entity("General Motors", vocab) is None # a different name is not a match
+
+
+def test_short_names_get_own_source_and_finding_support() -> None:
+ profile = _descriptive_profile()
+ vocab = wp.grounding_vocabulary(profile, [], [])
+ ctx = wp.PolicyContext(vocabulary=vocab, priority_terms=[], existing=[])
+ f = _finding(title="BYD enters Mexico", summary="BYD launched the Seal in Mexico on 2026-09-05.",
+ relevant_urls=["https://www.byd.com/news/mexico"], source_specialist="cso")
+ p = _proposal(slug="rss-byd", target="https://www.byd.com/news/feed", grounding_entity="BYD")
+ d = wp.classify(p, f, ctx)
+ assert d.tier == wp.TIER_DIRECT and "target is the entity's own source" in d.reasons
+ # A dotted ticker: own source when grounded as the ticker, and named in
+ # text as a phrase.
+ f2 = _finding(title="BYD (1211.HK) slides", summary="1211.HK fell 6% on 2026-09-05.", relevant_urls=[])
+ stock = _proposal(slug="stock-byd", signal_type="stock", target="1211.HK", grounding_entity="1211.HK")
+ d2 = wp.classify(stock, f2, ctx)
+ assert d2.tier == wp.TIER_DIRECT and d2.grounding_kind == wp.KIND_TICKER
+ # Grounded as the competitor name instead, the ticker is not its own
+ # source, so it is a suggestion.
+ assert wp.classify(_proposal(slug="stock-byd2", signal_type="stock", target="1211.HK",
+ grounding_entity="BYD"), f2, ctx).tier == wp.TIER_SUGGEST
+ decided = SimpleNamespace(summary="Benchmark C1 pricing against BYD monthly", department="", timestamp="")
+ assert wp.named_in_decision("byd", decided, vocab)
+ # A short name inside a longer term needs the vocabulary to count.
+ two_word = SimpleNamespace(summary="Benchmark against BYD Auto monthly", department="", timestamp="")
+ assert wp.named_in_decision("byd auto", two_word, vocab)
+ # Every distinctive word must appear: "BYD Auto" is not named by "BYD pricing".
+ assert not wp.named_in_decision("byd auto", SimpleNamespace(summary="Watch BYD pricing"), vocab)
+ assert wp.entity_declined("BYD Auto", [SimpleNamespace(reason="not_relevant", entity="BYD")])
+ assert not wp.entity_declined("Rivian", [SimpleNamespace(reason="not_relevant", entity="BYD")])
+
+
+def test_proposal_without_finding_index_is_linked_to_the_citing_finding(db: Path) -> None:
+ findings = [
+ _finding(title="Unrelated", summary="Globex raised prices.", relevant_urls=["https://globex.com/blog"]),
+ _finding(title="Acme cut prices", summary="Acme Corp cut list prices 20%.",
+ relevant_urls=["https://www.acme.com/blog/pricing"]),
+ _finding(title="ACME slides", summary="ACME fell 8% on 2026-09-01.", relevant_urls=[]),
+ ]
+ ctx = _ctx()
+ unlinked = _proposal(finding_index=None)
+ assert wp.auto_link_finding(unlinked, findings, ctx) == 1
+ stock = _proposal(slug="stock-acme", signal_type="stock", target="ACME", grounding_entity="ACME",
+ finding_index=None)
+ assert wp.auto_link_finding(stock, findings, ctx) == 1 # names the entity first
+ nowhere = _proposal(slug="rss-else", target="https://elsewhere.com/feed", grounding_entity="Acme Corp",
+ finding_index=None)
+ assert wp.auto_link_finding(nowhere, findings, ctx) is None
+ out = wp.apply_proposals([unlinked, nowhere], findings, ctx, db_path=db)
+ assert [o["outcome"] for o in out] == ["added", "rejected"]
+ row = ms.get_watchlist_item_by_slug("rss-acme-blog", db_path=db)
+ assert row is not None
+ assert wp.policy_stamp_of(row)["reasons"][0] == "linked to finding #2 by its cited source"
+ assert wp.policy_stamp_of(row)["specialist"] == "cso"