Skip to content

AI-powered virtual executive team — a single coherent executive persona backed by 8 specialist agents (FastAPI + Next.js).

fix(monitoring): stop old RSS/EDGAR items surfacing as recent alerts (#80)

johnrufusonee84654615951
diff --git a/packages/core/openexecutive/api/routes/watchlist.py b/packages/core/openexecutive/api/routes/watchlist.py
index 96ba6d3d..33c949ae 100644
--- a/packages/core/openexecutive/api/routes/watchlist.py
+++ b/packages/core/openexecutive/api/routes/watchlist.py
@@ -104,6 +104,9 @@ class SignalRow(BaseModel):
source_kind: str
source_external_id: str
captured_at: str
+ # Upstream publish time (RSS <pubDate>, EDGAR filing date) — None when the
+ # source has none or the row predates the column (issue #80).
+ published_at: str | None = None
normalized_summary: str
provenance_url: str
severity_hint: str
diff --git a/packages/core/openexecutive/architecture/architecture-facts.yaml b/packages/core/openexecutive/architecture/architecture-facts.yaml
index 0a2e1009..3bbb3a18 100644
--- a/packages/core/openexecutive/architecture/architecture-facts.yaml
+++ b/packages/core/openexecutive/architecture/architecture-facts.yaml
@@ -1233,12 +1233,32 @@ external_monitoring:
is intentional — the briefing layer can answer "what did you ignore
overnight" and the trust loop has a denominator for hit-rate metrics.
Columns: id, watchlist_id, source_kind, source_external_id,
- captured_at, normalized_summary, raw_payload_json, provenance_url,
- severity_hint, dedup_key, processed_at, processed_outcome
- ('alerted' | 'suppressed_dry_run' | 'suppressed_below_floor' |
+ captured_at, published_at, normalized_summary, raw_payload_json,
+ provenance_url, severity_hint, dedup_key, processed_at,
+ processed_outcome
+ ('alerted' | 'suppressed_baseline' | 'suppressed_stale' |
+ 'suppressed_dry_run' | 'suppressed_below_floor' |
'suppressed_dup' | 'suppressed_triage' | 'suppressed_low_relevance' |
'failed'),
promoted_alert_id, enrichment_json.
+ captured_at is when the monitor first saw the event; published_at
+ (nullable, added via the same idempotent additive ALTER pattern) is
+ when the upstream says it happened — RSS <pubDate>/<updated>, EDGAR
+ filing date; NULL for stock / page_watch / query. The two are never
+ conflated (issue #80). Two freshness gates run FIRST in the
+ suppression cascade, before dry_run / floor / enrichment, so they
+ never spend an LLM call: (1) `suppressed_baseline` — on the FIRST
+ poll of a row (last_polled_at IS NULL) whose adapter declares
+ `seed_on_first_poll` (rss, edgar), every entry in the feed is
+ recorded as seen and nothing is promoted, so a new watch reports
+ what changes from now on rather than replaying the feed's
+ back-catalogue (mirrors page_watch's first-observation baseline);
+ (2) `suppressed_stale` — a signal whose published_at is older than
+ EXTERNAL_MONITOR_MAX_SIGNAL_AGE_DAYS (default 7; 0 disables) at
+ capture time is recorded but never promoted. The alert body handed
+ to triage carries `Published:` (when known) and `Discovered:`
+ lines so anything inside the window is still judged on when it
+ happened, not when it was found.
enrichment_json holds the capture-time relevance read
(relevance_score 0..1, why_it_matters, matched_initiative) written by
the enrichment stage; '{}' when enrichment is off or missed. Added via
@@ -1270,10 +1290,10 @@ external_monitoring:
add a top-level fact in this YAML when they ship.
shipped:
- "vendor_status — polls Statuspage Atom history feeds (AWS, Stripe, Twilio, GitHub, Cloudflare, …) and emits one Signal per incident entry. Severity hint HIGH by default. Optional `trigger_json.keywords` narrows to entries whose title contains a keyword."
- - "rss — generic Atom/RSS feed adapter via feedparser; handles competitor changelogs, layoffs.fyi, USPTO patent feeds, regulatory filings, blog RSS. Severity hint LOW by default (most RSS is noisy). Optional `trigger_json.keywords` filters by title + summary."
+ - "rss — generic Atom/RSS feed adapter via feedparser; handles competitor changelogs, layoffs.fyi, USPTO patent feeds, regulatory filings, blog RSS. Severity hint LOW by default (most RSS is noisy). Optional `trigger_json.keywords` filters by title + summary. Each entry's <pubDate>/<updated> is parsed into Signal.published_at; declares `seed_on_first_poll` so a new watch baselines the feed's existing entries instead of firing them, and entries older than EXTERNAL_MONITOR_MAX_SIGNAL_AGE_DAYS are recorded as `suppressed_stale` (issue #80)."
- "stock — Yahoo Finance Chart endpoint (the same endpoint yfinance scrapes, used directly via httpx to avoid pandas/numpy). Watchlist `target` is the ticker; signals fire when |intraday vs previous close| ≥ `trigger.abs_change_pct_gte` (default 5%). Severity hint scales with the magnitude of the move (≥10% URGENT, ≥5% HIGH, ≥2% MEDIUM, else LOW). v2 candidate: swap behind the same Source interface for a paid Alpha Vantage / Finnhub feed if Yahoo brittleness bites."
- "query — standing web-search query. Watchlist `target` is a natural-language query (e.g. 'Acme Corp layoffs OR restructuring OR funding'); each poll runs it through `BaseAgent.analyze_with_tools` with the web_search tool (model pinned to `get_research_model()`, honouring an OpenRouter Council override — the provider layer translates native web_search into OpenRouter's `plugins:[{id:web}]`, see providers/translator). The model reports distinct results via the `emit_query_results` tool; each result with a valid clickable URL becomes a Signal (the adapter emits a flat severity hint LOW because it can't judge materiality per-hit — the pipeline regrades this from the capture-time relevance read BEFORE the floor gate via `_grade_query_severity`, so a material hit clears a medium/high floor instead of being suppressed; dedup key `query:` + first 32 hex chars of sha256(slug + NUL + canonical_url); URL-less results dropped to uphold the provenance invariant). UNLIKE the keyless feed adapters this source is BILLED (one LLM call + web searches per poll), so it polls slowly (default 360 min), is bounded by `web_search_max_uses` + per-row `config.max_results` (default 5, ceiling 10), and has its own kill switch `EXTERNAL_MONITOR_QUERY_ENABLED`. Optional `trigger.keywords` filters like rss; `config.allowed_domains`/`blocked_domains` are best-effort prompt hints only (NOT enforced, and not honoured by the OpenRouter web plugin). This is the 'watch arbitrary conditions' primitive that no feed publishes."
- - "edgar — SEC EDGAR filings. Watchlist `target` is a ticker (e.g. 'AAPL') or CIK; polls SEC's browse-edgar Atom feed (`/cgi-bin/browse-edgar?action=getcompany&CIK=…&output=atom`, whose CIK param resolves a ticker server-side, so no ticker→CIK table is shipped), parsed with feedparser. Emits one Signal per new filing whose form is in the filter (`trigger.forms`, default {8-K, 10-K, 10-Q}; amendments like 8-K/A match their base form). Dedup on the filing's accession number (globally unique — the cleanest natural key of any source). Provenance is the filing-index URL. Severity hint: 8-K HIGH (material-event report), 10-K/10-Q MEDIUM, else LOW. Keyless, but SEC's fair-access policy requires a descriptive User-Agent with a contact — sent via the EDGAR_USER_AGENT setting (operators should override the default with a real contact). Reuses fetch_bounded (now with a per-call user_agent override), validate_target_url, and the standard dedup/Signal pattern."
+ - "edgar — SEC EDGAR filings. Watchlist `target` is a ticker (e.g. 'AAPL') or CIK; polls SEC's browse-edgar Atom feed (`/cgi-bin/browse-edgar?action=getcompany&CIK=…&output=atom`, whose CIK param resolves a ticker server-side, so no ticker→CIK table is shipped), parsed with feedparser. Emits one Signal per new filing whose form is in the filter (`trigger.forms`, default {8-K, 10-K, 10-Q}; amendments like 8-K/A match their base form). Dedup on the filing's accession number (globally unique — the cleanest natural key of any source). Provenance is the filing-index URL. Severity hint: 8-K HIGH (material-event report), 10-K/10-Q MEDIUM, else LOW. Keyless, but SEC's fair-access policy requires a descriptive User-Agent with a contact — sent via the EDGAR_USER_AGENT setting (operators should override the default with a real contact). Reuses fetch_bounded (now with a per-call user_agent override), validate_target_url, and the standard dedup/Signal pattern. The entry's <updated> lands on Signal.published_at and the adapter declares `seed_on_first_poll`, so a new watch baselines past filings instead of replaying them and filings older than EXTERNAL_MONITOR_MAX_SIGNAL_AGE_DAYS are `suppressed_stale`."
- "page_watch — generic web-page change detection for feed-less pages (pricing, careers, leadership, ToS). Watchlist `target` is the page URL. STATEFUL (the first stateful adapter): each poll fetches the page (bounded, SSRF-guarded), reduces it to normalized visible text (drops script/style/comments/tags, unescapes entities, collapses whitespace — so markup/layout churn doesn't register), hashes the FULL normalized text (so a change anywhere registers — no length blind spot), and compares to the last-seen hash in `page_watch_state`. First observation stores a baseline and emits NOTHING; an unchanged page emits nothing; a changed page emits ONE Signal carrying a difflib-based change summary (~N% differs + the added text; the diff/snapshot are length-capped but detection is not) and updates the baseline. Severity hint LOW (page diffs are noisy — enrichment + severity_floor filter). Dedup key folds slug + UTC date + content hash (mirrors stock — a state seen again on a later day re-alerts; same-day repeats dedup). Optional `trigger.keywords` only surfaces a change when the change itself (summary + added text) matches. Reads/writes its state via the db_path threaded through Source.poll. Optional `config.fetch = \"xcrawl\"` routes the fetch through xcrawl's scrape API (clean markdown) instead of the keyless httpx fetcher — so JS-rendered / bot-blocked feed-less pages (most modern news/marketing sites) are monitorable; off by default, a no-op when xcrawl is disabled, and the scraped markdown is size-capped to mirror the httpx byte cap."
insert_validation: |
monitoring.target_validation.validate_and_normalize_target — runs at
diff --git a/packages/core/openexecutive/architecture/prebuilt/external_monitoring.json b/packages/core/openexecutive/architecture/prebuilt/external_monitoring.json
index 8345fa7e..94843871 100644
--- a/packages/core/openexecutive/architecture/prebuilt/external_monitoring.json
+++ b/packages/core/openexecutive/architecture/prebuilt/external_monitoring.json
@@ -1,7 +1,7 @@
{
"section_id": "external_monitoring",
"title": "External Monitoring",
- "markdown": "`openexecutive.monitoring` watches the outside world on a heartbeat and turns detected changes into briefing proposals via the same alert pipeline that handles inbound email and Slack.\n\nThe `external_monitor_scan` heartbeat (`__internal__` channel, default every 5 min) walks `list_watchlist(enabled_only=True)`, skipping rows fresher than their per-source cadence floor. For each due row it fires a read-only source adapter resolved from `_REGISTRY` by `signal_type`:\n\n- `vendor_status` — Statuspage Atom incident feeds (AWS, Stripe, GitHub, …).\n- `rss` — generic Atom/RSS via feedparser (changelogs, layoffs.fyi, filings).\n- `stock` — Yahoo Finance chart endpoint; fires on `abs_change_pct_gte` moves.\n- `page_watch` — change-detection on a feed-less page. Fetches via the keyless httpx fetcher by default, or via xcrawl's scrape API when the row carries `config.fetch = \"xcrawl\"` — so JS-rendered / bot-blocked pages that publish no RSS (most modern news / marketing sites) are still monitorable.\n- `edgar` and the BILLED `query` web-search adapter.\n\nEach `Signal` is trigger-filtered, severity-clamped to the row's `[floor, ceiling]`, and INSERTed into `external_signals` where `UNIQUE(dedup_key)` drops replays. Survivors pass a suppression cascade (`dry_run` → `below_floor` → capture-time relevance enrichment → promote), then `promote_signal_to_alert` builds an `AlertEvent` handed to `alerts.pipeline.schedule_evaluation` and surfaced as a `/today` proposal.\n\nThe `query` adapter is a special case: it emits every web hit flat-LOW (it can't judge materiality per-hit), so for query signals enrichment runs BEFORE the floor gate and `_grade_query_severity` regrades from the relevance read (≥0.8 HIGH, ≥0.5 MEDIUM, else LOW). Without this, any query watch item with a `severity_floor` above LOW would suppress every hit — which is what left the Monitoring section empty even while standing queries were returning material findings.\n\nWhich lane a proposal lands in is decided by `briefing.ranking.categorize`: an unrouted, low/medium signal from a monitoring source (`stock`, `vendor_status`, `rss`, `query`, `edgar`, `page_watch`, or any `external:*` topic tag) is `monitoring` (the collapsible Monitoring section); a routed or high/urgent one is `action` (the \"Needs you\" queue). The triage prompt's External Signals rule covers all six source kinds so it trusts the source and emits the `external:*` tags that drive this split.\n\nThe watchlist is editable from chat (`add_watchlist_entry`, `tune_watchlist_entry`) and HTTP (`/watchlist`); the `executive_research` workflow proposes what to watch. Because that workflow is LLM-driven and historically proposed feed URLs it never verified (most were dead — 404 / 403 / not-actually-a-feed), every `rss` target is validated at insert time (`monitoring.target_validation`): a URL that fetches and parses as a feed is kept; a public page that isn't a feed is converted to a scrape-backed `page_watch` when xcrawl can read it, otherwise rejected (HTTP 422 / tool error); SSRF-blocked or unparseable URLs are rejected outright and never handed to xcrawl. Non-`rss` types insert unchanged with no network I/O.\n\n### Invariants\n\n- Adapters are read-only — they never write upstream and never auto-execute.\n- Every `Signal` carries a deterministic `dedup_key` and a mandatory `provenance_url`.\n- Fetches are SSRF-guarded and bounded by `external_monitor_max_fetch_bytes`; xcrawl scrape (off by default, gated by `XCRAWL_ENABLED` + `XCRAWL_API_KEY`) is the one path that delegates the fetch to an external API.\n- Suppressed signals are still persisted so audits show the full picture.",
+ "markdown": "`openexecutive.monitoring` watches the outside world on a heartbeat and turns detected changes into briefing proposals via the same alert pipeline that handles inbound email and Slack.\n\nThe `external_monitor_scan` heartbeat (`__internal__` channel, default every 5 min) walks `list_watchlist(enabled_only=True)`, skipping rows fresher than their per-source cadence floor. For each due row it fires a read-only source adapter resolved from `_REGISTRY` by `signal_type`:\n\n- `vendor_status` — Statuspage Atom incident feeds (AWS, Stripe, GitHub, …).\n- `rss` — generic Atom/RSS via feedparser (changelogs, layoffs.fyi, filings). Parses each entry's `<pubDate>`/`<updated>` into `published_at`; a **seeding** source (see below).\n- `stock` — Yahoo Finance chart endpoint; fires on `abs_change_pct_gte` moves.\n- `page_watch` — change-detection on a feed-less page. Fetches via the keyless httpx fetcher by default, or via xcrawl's scrape API when the row carries `config.fetch = \"xcrawl\"` — so JS-rendered / bot-blocked pages that publish no RSS (most modern news / marketing sites) are still monitorable.\n- `edgar` and the BILLED `query` web-search adapter.\n\nEach `Signal` is trigger-filtered, severity-clamped to the row's `[floor, ceiling]`, and INSERTed into `external_signals` where `UNIQUE(dedup_key)` drops replays. Survivors pass a suppression cascade (`baseline` → `stale` → `dry_run` → `below_floor` → capture-time relevance enrichment → promote), then `promote_signal_to_alert` builds an `AlertEvent` handed to `alerts.pipeline.schedule_evaluation` and surfaced as a `/today` proposal.\n\nEvery signal carries two timestamps that are never conflated: `captured_at` (when the monitor first saw it) and `published_at` (when the upstream says it happened — RSS `<pubDate>`, EDGAR filing date; null for `stock`, `page_watch`, `query`). Two freshness gates run first in the cascade, before anything that costs an LLM call. **Baseline:** the first poll of a row whose adapter declares `seed_on_first_poll` (`rss`, `edgar`) records every entry already in the feed as `suppressed_baseline` and promotes nothing, so a new watch reports what changes from now on instead of replaying the feed's back-catalogue (the same idea as `page_watch`'s first-observation baseline). **Stale:** a signal whose `published_at` is older than `EXTERNAL_MONITOR_MAX_SIGNAL_AGE_DAYS` (default 7; `0` disables) at capture time is recorded as `suppressed_stale` and never promoted — a January article a feed resurfaces in September is not September news. Inside the window, the alert body handed to triage carries `Published:` and `Discovered:` lines so the item is judged on when it happened.\n\nThe `query` adapter is a special case: it emits every web hit flat-LOW (it can't judge materiality per-hit), so for query signals enrichment runs BEFORE the floor gate and `_grade_query_severity` regrades from the relevance read (≥0.8 HIGH, ≥0.5 MEDIUM, else LOW). Without this, any query watch item with a `severity_floor` above LOW would suppress every hit — which is what left the Monitoring section empty even while standing queries were returning material findings.\n\nWhich lane a proposal lands in is decided by `briefing.ranking.categorize`: an unrouted, low/medium signal from a monitoring source (`stock`, `vendor_status`, `rss`, `query`, `edgar`, `page_watch`, or any `external:*` topic tag) is `monitoring` (the collapsible Monitoring section); a routed or high/urgent one is `action` (the \"Needs you\" queue). The triage prompt's External Signals rule covers all six source kinds so it trusts the source and emits the `external:*` tags that drive this split.\n\nThe watchlist is editable from chat (`add_watchlist_entry`, `tune_watchlist_entry`) and HTTP (`/watchlist`); the `executive_research` workflow proposes what to watch. Because that workflow is LLM-driven and historically proposed feed URLs it never verified (most were dead — 404 / 403 / not-actually-a-feed), every `rss` target is validated at insert time (`monitoring.target_validation`): a URL that fetches and parses as a feed is kept; a public page that isn't a feed is converted to a scrape-backed `page_watch` when xcrawl can read it, otherwise rejected (HTTP 422 / tool error); SSRF-blocked or unparseable URLs are rejected outright and never handed to xcrawl. Non-`rss` types insert unchanged with no network I/O.\n\n### Invariants\n\n- Adapters are read-only — they never write upstream and never auto-execute.\n- Every `Signal` carries a deterministic `dedup_key` and a mandatory `provenance_url`.\n- Fetches are SSRF-guarded and bounded by `external_monitor_max_fetch_bytes`; xcrawl scrape (off by default, gated by `XCRAWL_ENABLED` + `XCRAWL_API_KEY`) is the one path that delegates the fetch to an external API.\n- Suppressed signals are still persisted so audits show the full picture.",
"mermaid": "flowchart TD\n ADD[\"add (chat tool / POST /watchlist / research)\"]\n VAL[\"target_validation: rss feed? → keep / convert→page_watch(xcrawl) / reject\"]\n WL[(\"watchlist\")]\n HB[\"external_monitor_scan heartbeat (~5 min)\"]\n AD[\"read-only adapters: vendor_status / rss / stock / query / edgar / page_watch (httpx | xcrawl)\"]\n SIG[(\"external_signals UNIQUE(dedup_key)\")]\n ENRICH[\"capture-time enrichment<br/>(relevance score + why_it_matters)\"]\n PROMO[\"promote_signal_to_alert\"]\n PIPE[\"alert pipeline (triage)\"]\n TODAY[\"/today proposals\"]\n ADD --> VAL\n VAL -->|\"insert valid rows\"| WL\n HB -->|\"walk enabled rows\"| WL\n WL -->|\"poll due rows\"| AD\n AD -->|\"emit Signal\"| SIG\n SIG -->|\"dry_run / floor gates\"| ENRICH\n ENRICH -->|\"≥ relevance floor\"| PROMO\n PROMO --> PIPE\n PIPE --> TODAY\n class HB hot\n class WL storage\n class VAL compute\n class AD external\n class SIG storage\n class ENRICH compute\n class PROMO compute\n class PIPE compute\n class TODAY entry\n class ADD compute",
- "generated_at": "2026-06-04T00:00:00+00:00"
+ "generated_at": "2026-09-07T12:18:25+00:00"
}
diff --git a/packages/core/openexecutive/config.py b/packages/core/openexecutive/config.py
index ef8e693e..184a267a 100644
--- a/packages/core/openexecutive/config.py
+++ b/packages/core/openexecutive/config.py
@@ -624,6 +624,16 @@ class Settings(BaseSettings):
external_monitor_max_signals_per_scan: int = Field(
50, alias="EXTERNAL_MONITOR_MAX_SIGNALS_PER_SCAN"
)
+ # Freshness gate for sources that carry an upstream publish timestamp
+ # (rss <pubDate>, edgar filing date). A signal whose ``published_at`` is
+ # older than this many days at capture time is recorded but never
+ # promoted (outcome ``suppressed_stale``) — a feed that resurfaces a
+ # January article in September must not become September news (issue
+ # #80). 0 disables the gate. Sources without an upstream timestamp
+ # (stock, page_watch, query) are unaffected.
+ external_monitor_max_signal_age_days: int = Field(
+ 7, alias="EXTERNAL_MONITOR_MAX_SIGNAL_AGE_DAYS"
+ )
# Adapter-fetch ceiling (bytes). Caps the body we read from any single
# external feed — defence against runaway sources (e.g. malformed RSS
# that streams forever) and a soft guard against XML-bomb shapes.
diff --git a/packages/core/openexecutive/monitoring/models.py b/packages/core/openexecutive/monitoring/models.py
index 7cca754d..c53210a3 100644
--- a/packages/core/openexecutive/monitoring/models.py
+++ b/packages/core/openexecutive/monitoring/models.py
@@ -121,7 +121,14 @@ class Signal(BaseModel):
watchlist_id: int
source_kind: str
source_external_id: str
- captured_at: str # ISO 8601 UTC
+ captured_at: str # ISO 8601 UTC — when WE first saw the event
+ # When the upstream says the event happened (ISO 8601 UTC), when the
+ # source publishes one — RSS <pubDate>, Atom <updated>, EDGAR filing
+ # date. None for sources with no upstream timestamp (stock, page_watch,
+ # query). The pipeline's age gate reads this; ``captured_at`` is never
+ # a proxy for it (issue #80: a January article discovered in September
+ # is not September news).
+ published_at: str | None = None
normalized_summary: str # one-line human description
raw_payload: dict[str, Any] = Field(default_factory=dict)
provenance_url: str
@@ -142,10 +149,20 @@ OUTCOME_SUPPRESSED_TRIAGE = "suppressed_triage"
# configured floor (external_monitor_enrichment_min_relevance). Recorded, not
# promoted — the gate is opt-in (default threshold 0.0 = off) until calibrated.
OUTCOME_SUPPRESSED_LOW_RELEVANCE = "suppressed_low_relevance"
+# Recorded on the FIRST poll of a feed-listing source (rss, edgar): every
+# entry already in the feed is persisted as "seen" so dedup covers it, but
+# nothing is promoted — the watch reports what changes from now on, not
+# the feed's back-catalogue. Mirrors page_watch's first-observation baseline.
+OUTCOME_SUPPRESSED_BASELINE = "suppressed_baseline"
+# Upstream ``published_at`` is older than EXTERNAL_MONITOR_MAX_SIGNAL_AGE_DAYS
+# at capture time. Persisted for the audit trail + dedup; never promoted.
+OUTCOME_SUPPRESSED_STALE = "suppressed_stale"
OUTCOME_FAILED = "failed"
_VALID_OUTCOMES = frozenset({
OUTCOME_ALERTED,
+ OUTCOME_SUPPRESSED_BASELINE,
+ OUTCOME_SUPPRESSED_STALE,
OUTCOME_SUPPRESSED_DRY_RUN,
OUTCOME_SUPPRESSED_BELOW_FLOOR,
OUTCOME_SUPPRESSED_DUP,
diff --git a/packages/core/openexecutive/monitoring/pipeline.py b/packages/core/openexecutive/monitoring/pipeline.py
index 81eb42f6..4a006b70 100644
--- a/packages/core/openexecutive/monitoring/pipeline.py
+++ b/packages/core/openexecutive/monitoring/pipeline.py
@@ -42,9 +42,11 @@ from openexecutive.monitoring.models import (
MODE_DRY_RUN,
OUTCOME_ALERTED,
OUTCOME_FAILED,
+ OUTCOME_SUPPRESSED_BASELINE,
OUTCOME_SUPPRESSED_BELOW_FLOOR,
OUTCOME_SUPPRESSED_DRY_RUN,
OUTCOME_SUPPRESSED_LOW_RELEVANCE,
+ OUTCOME_SUPPRESSED_STALE,
SOURCE_KIND_QUERY,
Signal,
WatchlistItem,
@@ -98,6 +100,35 @@ def _due_for_poll(item: WatchlistItem, now: datetime) -> bool:
return now - last >= floor
+# --------------------------------------------------------------------- #
+# Freshness gate
+# --------------------------------------------------------------------- #
+
+
+def _is_stale(signal: Signal, now: datetime, max_age_days: int) -> bool:
+ """True when the upstream publish time is older than the age gate.
+
+ Only signals that carry ``published_at`` can be judged; sources without
+ an upstream timestamp (stock, page_watch, query) always pass. An
+ unparseable timestamp also passes — the adapter writes ISO 8601, so a
+ bad value is a bug to surface downstream, not a reason to drop the
+ event silently. ``max_age_days <= 0`` disables the gate.
+ """
+ if max_age_days <= 0 or not signal.published_at:
+ return False
+ try:
+ published = datetime.fromisoformat(signal.published_at)
+ except ValueError:
+ logger.warning(
+ "monitoring.pipeline: unparseable published_at %r on %s — "
+ "treating as fresh", signal.published_at, signal.dedup_key,
+ )
+ return False
+ if published.tzinfo is None:
+ published = published.replace(tzinfo=UTC)
+ return now - published > timedelta(days=max_age_days)
+
+
# --------------------------------------------------------------------- #
# Signal → alert promotion
# --------------------------------------------------------------------- #
@@ -187,6 +218,12 @@ def _signal_to_alert_event(
# makes them robust to the model's tendency to skim long bodies.
parts.append(f"Watchlist: {item.slug}")
parts.append(f"Severity hint: {signal.severity_hint.value}")
+ # When it happened vs when we noticed it — distinct on purpose (issue
+ # #80). The age gate already drops clearly stale items before this
+ # point; these lines let triage weigh anything inside the window.
+ if signal.published_at:
+ parts.append(f"Published: {signal.published_at}")
+ parts.append(f"Discovered: {signal.captured_at}")
target = signal.raw_payload.get("target_url")
if target:
parts.append(f"Source: {target}")
@@ -314,6 +351,15 @@ async def _poll_one_watchlist_item(
)
return 0
+ # First poll of this row → feed-listing sources are baselined (see
+ # Source.seed_on_first_poll). Read BEFORE the poll: ``mark_polled`` in
+ # the ``finally`` below flips the DB row, and ``item`` is the pre-poll
+ # snapshot. A failed first fetch still marks the row polled, so the
+ # next successful poll is NOT a baseline — the age gate below is the
+ # backstop for that case.
+ is_baseline_poll = item.last_polled_at is None and src.seed_on_first_poll
+ max_age_days = get_settings().external_monitor_max_signal_age_days
+
try:
emitted = await src.poll(item, db_path=db_path)
except Exception:
@@ -382,6 +428,28 @@ async def _poll_one_watchlist_item(
# paired external_signal_suppressed audit event so a reader can
# tell from the audit log alone (without joining external_signals)
# what was noticed but not surfaced.
+ #
+ # Freshness gates come first: a baseline entry or a stale one is
+ # "already happened" news regardless of mode / floor, and neither
+ # should spend an enrichment call.
+ if is_baseline_poll:
+ store.mark_signal_processed(
+ signal_id, OUTCOME_SUPPRESSED_BASELINE, db_path=db_path
+ )
+ _audit_suppressed(
+ signal_id, item, signal, OUTCOME_SUPPRESSED_BASELINE,
+ )
+ continue
+
+ if _is_stale(signal, now, max_age_days):
+ store.mark_signal_processed(
+ signal_id, OUTCOME_SUPPRESSED_STALE, db_path=db_path
+ )
+ _audit_suppressed(
+ signal_id, item, signal, OUTCOME_SUPPRESSED_STALE,
+ )
+ continue
+
if item.mode == MODE_DRY_RUN:
store.mark_signal_processed(
signal_id, OUTCOME_SUPPRESSED_DRY_RUN, db_path=db_path
diff --git a/packages/core/openexecutive/monitoring/sources/base.py b/packages/core/openexecutive/monitoring/sources/base.py
index 009ab572..923adcd6 100644
--- a/packages/core/openexecutive/monitoring/sources/base.py
+++ b/packages/core/openexecutive/monitoring/sources/base.py
@@ -22,8 +22,11 @@ Invariants every adapter MUST uphold:
"""
from __future__ import annotations
+import calendar
+from collections.abc import Mapping
+from datetime import UTC, datetime
from pathlib import Path
-from typing import Protocol
+from typing import Any, Protocol
from openexecutive.monitoring.models import Signal, WatchlistItem
@@ -34,6 +37,13 @@ class Source(Protocol):
kind: str # matches WatchlistItem.signal_type
default_poll_interval_minutes: int
+ # True for feed-listing sources (rss, edgar) whose poll returns the
+ # feed's whole back-catalogue, not just what changed: on the FIRST poll
+ # of a row the pipeline records every entry as seen (outcome
+ # ``suppressed_baseline``) and promotes nothing, so the watch reports
+ # what changes from now on. False for point-in-time sources (stock,
+ # query) and for page_watch, which keeps its own baseline.
+ seed_on_first_poll: bool
async def poll(
self, item: WatchlistItem, *, db_path: Path | None = None
@@ -64,3 +74,26 @@ class Source(Protocol):
# Re-export Signal so adapter code can ``from .base import Signal``.
__all__ = ["Signal", "Source"]
+
+
+def feed_entry_published_at(entry: Mapping[str, Any]) -> str | None:
+ """ISO 8601 UTC publish timestamp of a feedparser entry, or None.
+
+ feedparser normalises ``<pubDate>`` / ``<published>`` / ``<updated>``
+ into ``*_parsed`` ``time.struct_time`` values already converted to
+ UTC; we prefer ``published`` (when the item first appeared) over
+ ``updated`` (last edit). Entries with no parseable date return None —
+ the pipeline's age gate then can't judge them, and only the first-poll
+ baseline protects against replaying them as new.
+ """
+ for key in ("published_parsed", "updated_parsed"):
+ st = entry.get(key)
+ if not st:
+ continue
+ try:
+ # struct_time is already UTC (feedparser normalises), so timegm —
+ # never mktime, which would apply the host's local offset.
+ return datetime.fromtimestamp(calendar.timegm(st), tz=UTC).isoformat()
+ except (TypeError, ValueError, OverflowError, OSError):
+ continue
+ return None
diff --git a/packages/core/openexecutive/monitoring/sources/edgar.py b/packages/core/openexecutive/monitoring/sources/edgar.py
index c4bfa70e..9e87f0fd 100644
--- a/packages/core/openexecutive/monitoring/sources/edgar.py
+++ b/packages/core/openexecutive/monitoring/sources/edgar.py
@@ -51,6 +51,7 @@ from openexecutive.monitoring.sources._http import (
fetch_bounded,
validate_target_url,
)
+from openexecutive.monitoring.sources.base import feed_entry_published_at
logger = logging.getLogger(__name__)
@@ -87,6 +88,9 @@ _IDENT_RE = re.compile(rf"^[A-Za-z0-9.\-]{{1,{_MAX_IDENT_LEN}}}$")
class EdgarSource:
kind: str = SOURCE_KIND_EDGAR
default_poll_interval_minutes: int = _DEFAULT_POLL_MINUTES
+ # The Atom feed lists the last N filings on every poll; the first poll
+ # is a baseline so a new watch doesn't replay past filings as news.
+ seed_on_first_poll: bool = True
async def poll(
self, item: WatchlistItem, *, db_path: Path | None = None
@@ -228,6 +232,7 @@ def _entry_to_signal(
source_kind=SOURCE_KIND_EDGAR,
source_external_id=accession[:500],
captured_at=datetime.now(UTC).isoformat(),
+ published_at=feed_entry_published_at(entry),
normalized_summary=summary[:500],
raw_payload={
"label": label,
diff --git a/packages/core/openexecutive/monitoring/sources/page_watch.py b/packages/core/openexecutive/monitoring/sources/page_watch.py
index c96a12cb..28177ef6 100644
--- a/packages/core/openexecutive/monitoring/sources/page_watch.py
+++ b/packages/core/openexecutive/monitoring/sources/page_watch.py
@@ -90,6 +90,8 @@ _WS_RE = re.compile(r"\s+")
class PageWatchSource:
kind: str = SOURCE_KIND_PAGE_WATCH
default_poll_interval_minutes: int = _DEFAULT_POLL_MINUTES
+ # Keeps its own first-observation baseline in page_watch_state.
+ seed_on_first_poll: bool = False
async def poll(
self, item: WatchlistItem, *, db_path: Path | None = None
diff --git a/packages/core/openexecutive/monitoring/sources/query.py b/packages/core/openexecutive/monitoring/sources/query.py
index a557c003..0c2b27df 100644
--- a/packages/core/openexecutive/monitoring/sources/query.py
+++ b/packages/core/openexecutive/monitoring/sources/query.py
@@ -152,6 +152,8 @@ class _QueryResearchAgent(BaseAgent):
class QuerySource:
kind: str = SOURCE_KIND_QUERY
default_poll_interval_minutes: int = _DEFAULT_POLL_MINUTES
+ # A standing query's first results are the research the user asked for.
+ seed_on_first_poll: bool = False
async def poll(
self, item: WatchlistItem, *, db_path: Path | None = None
diff --git a/packages/core/openexecutive/monitoring/sources/rss.py b/packages/core/openexecutive/monitoring/sources/rss.py
index 05b0d404..91e808e4 100644
--- a/packages/core/openexecutive/monitoring/sources/rss.py
+++ b/packages/core/openexecutive/monitoring/sources/rss.py
@@ -15,6 +15,14 @@ Watchlist row shape:
— when present, only entries whose title OR summary contain at least
one keyword surface. Otherwise every new entry surfaces.
+Freshness: each entry's ``<pubDate>`` / ``<updated>`` is parsed into
+``Signal.published_at`` so the pipeline can tell *when it happened* apart
+from *when we first saw it* (``captured_at``). Two guards follow from
+that (issue #80 — a January article was surfacing as September news):
+the first poll of a row records the feed's existing entries as a
+baseline without promoting them, and entries older than
+``EXTERNAL_MONITOR_MAX_SIGNAL_AGE_DAYS`` are recorded but never promoted.
+
Severity hint defaults to ``LOW`` — RSS is by far the noisiest source,
so a low default lets the watchlist's ``severity_floor`` filter
aggressively. Operators expecting HIGH-severity feeds (e.g. SEC EDGAR
@@ -48,6 +56,7 @@ from openexecutive.monitoring.sources._http import (
strip_url_query,
validate_target_url,
)
+from openexecutive.monitoring.sources.base import feed_entry_published_at
logger = logging.getLogger(__name__)
@@ -61,6 +70,10 @@ class RssSource:
# RSS publishers update less often than status pages — 30 min is the
# sweet spot between freshness and politeness toward upstream.
default_poll_interval_minutes: int = 30
+ # A feed returns its whole back-catalogue on every poll, so the first
+ # poll is a baseline (see Source.seed_on_first_poll) — otherwise every
+ # historical entry would fire as "news" the moment a watch is added.
+ seed_on_first_poll: bool = True
async def poll(
self, item: WatchlistItem, *, db_path: Path | None = None
@@ -124,6 +137,7 @@ class RssSource:
source_kind=self.kind,
source_external_id=entry_id[:500],
captured_at=datetime.now(UTC).isoformat(),
+ published_at=feed_entry_published_at(entry),
normalized_summary=summary[:500],
raw_payload={
"feed_label": feed_label,
diff --git a/packages/core/openexecutive/monitoring/sources/stock.py b/packages/core/openexecutive/monitoring/sources/stock.py
index b20a66fe..b79c1617 100644
--- a/packages/core/openexecutive/monitoring/sources/stock.py
+++ b/packages/core/openexecutive/monitoring/sources/stock.py
@@ -72,6 +72,8 @@ class StockSource:
# Stock prices update fast but we don't want to be a high-frequency
# trader's data feed — 15 min is a reasonable middle ground.
default_poll_interval_minutes: int = 15
+ # Point-in-time threshold check — a crossing on the first poll is real.
+ seed_on_first_poll: bool = False
async def poll(
self, item: WatchlistItem, *, db_path: Path | None = None
diff --git a/packages/core/openexecutive/monitoring/sources/vendor_status.py b/packages/core/openexecutive/monitoring/sources/vendor_status.py
index 1f07e4de..dea1be64 100644
--- a/packages/core/openexecutive/monitoring/sources/vendor_status.py
+++ b/packages/core/openexecutive/monitoring/sources/vendor_status.py
@@ -69,6 +69,8 @@ _MAX_ENTRIES_PER_FEED = 100
class VendorStatusSource:
kind: str = SOURCE_KIND_VENDOR_STATUS
default_poll_interval_minutes: int = 5
+ # Open incidents on a vendor we just started watching are actionable now.
+ seed_on_first_poll: bool = False
async def poll(
self, item: WatchlistItem, *, db_path: Path | None = None
diff --git a/packages/core/openexecutive/monitoring/store.py b/packages/core/openexecutive/monitoring/store.py
index 39af7305..e58f282b 100644
--- a/packages/core/openexecutive/monitoring/store.py
+++ b/packages/core/openexecutive/monitoring/store.py
@@ -95,6 +95,7 @@ def initialize_db(db_path: Path | None = None) -> None:
source_kind TEXT NOT NULL,
source_external_id TEXT NOT NULL,
captured_at TEXT NOT NULL,
+ published_at TEXT,
normalized_summary TEXT NOT NULL,
raw_payload_json TEXT NOT NULL DEFAULT '{}',
provenance_url TEXT NOT NULL,
@@ -130,6 +131,9 @@ def initialize_db(db_path: Path | None = None) -> None:
conn, "external_signals", "enrichment_json",
"TEXT NOT NULL DEFAULT '{}'",
)
+ # Upstream publish timestamp (issue #80). Nullable: pre-existing rows
+ # and sources without an upstream timestamp carry NULL.
+ _ensure_column(conn, "external_signals", "published_at", "TEXT")
def _ensure_column(
@@ -393,14 +397,15 @@ def insert_signal(signal: Signal, db_path: Path | None = None) -> int | None:
cursor = conn.execute(
"INSERT OR IGNORE INTO external_signals "
"(watchlist_id, source_kind, source_external_id, captured_at, "
- "normalized_summary, raw_payload_json, provenance_url, "
- "severity_hint, dedup_key) "
- "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
+ "published_at, normalized_summary, raw_payload_json, "
+ "provenance_url, severity_hint, dedup_key) "
+ "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(
signal.watchlist_id,
signal.source_kind,
signal.source_external_id,
signal.captured_at,
+ signal.published_at,
signal.normalized_summary,
json.dumps(signal.raw_payload),
signal.provenance_url,
diff --git a/packages/core/openexecutive/prompts/triage_prompt.py b/packages/core/openexecutive/prompts/triage_prompt.py
index 301cd48e..d1541564 100644
--- a/packages/core/openexecutive/prompts/triage_prompt.py
+++ b/packages/core/openexecutive/prompts/triage_prompt.py
@@ -58,9 +58,11 @@ Events with `source` ∈ `{"vendor_status", "rss", "stock", "query", "edgar", "p
```
Watchlist: <slug> # the user-named entry that captured this signal
Severity hint: <low|medium|high|urgent> # the adapter's pre-graded severity
+Published: <ISO timestamp> # when the upstream says it happened (only when the source publishes one)
+Discovered: <ISO timestamp> # when the monitor first saw it
```
-Use those two lines plus `source` as your primary inputs. Follow these rules in addition to the general rubric:
+Use the first two lines plus `source` as your primary inputs. `Published:` and `Discovered:` are distinct on purpose: an item is only as recent as its `Published:` line, never its `Discovered:` line. Follow these rules in addition to the general rubric:
- The watchlist already decided what's worth surfacing, so trust the source. Default to `alert=true` unless the event clearly duplicates a recent one (apply the dedup rule) or is muted.
- Use the `Severity hint:` line as your starting severity. Upgrade only when something else justifies it; downgrade only when the body makes clear the event resolved or is irrelevant.
@@ -72,6 +74,7 @@ Use those two lines plus `source` as your primary inputs. Follow these rules in
- ALWAYS add two topic tags: `external:<source>` (e.g. `external:vendor_status` or `external:query`, from the `source` field) AND `external:<watchlist_slug>` (e.g. `external:stock-aapl`, copied verbatim from the `Watchlist:` line in the body). These tags drive the "external" chip on the briefing card; missing the source tag leaves the principal without provenance.
- When the source is `stock`, include a short rationale in `body` linking the move to a plausible cause (earnings, macro, peer move) when one is obvious from `<recent_alerts>`. When no rationale is obvious, say "no obvious driver from open signals" — don't invent one.
- `suggested_action` for external signals is the single concrete action you (the Executive) will take on the user's behalf if they approve — first-person imperative, naming the source/channel/tool: "Check Stripe's status page and confirm checkout impact, then report back" / "Re-read Acme's changelog v2 and refresh our battlecard if pricing changed". Never longer than one sentence.
+- When `Published:` is present and clearly predates `Discovered:` (days, not hours), describe the item as dated in `body` and do not treat it as breaking news — keep or lower the severity hint rather than upgrading it. Old news that is still material to an active initiative can stay `alert=true`, but say when it happened.
- Quote, don't paraphrase, when summarising filings or status-page incidents — the body is the principal's last chance to see what the source actually said before clicking through.
The blast-radius rules above (privacy invariant on board/comp/legal) still apply: an external signal that names a competitor's funding round is NOT board-scoped and stays per-Person.
diff --git a/packages/core/tests/unit/test_monitoring_edgar.py b/packages/core/tests/unit/test_monitoring_edgar.py
index c635bb3c..be833cea 100644
--- a/packages/core/tests/unit/test_monitoring_edgar.py
+++ b/packages/core/tests/unit/test_monitoring_edgar.py
@@ -108,6 +108,10 @@ async def test_edgar_emits_signal_per_matching_filing(
assert by_form["8-K"].provenance_url.endswith("-index.htm")
assert by_form["8-K"].normalized_summary.startswith("[Apple] 8-K filed 2026-05-28")
assert all(s.dedup_key.startswith("edgar:") for s in signals)
+ # <updated>2026-05-28T16:30:00-04:00</updated> → published_at in UTC, so
+ # the pipeline's age gate judges the filing date, not the poll time.
+ assert by_form["8-K"].published_at == "2026-05-28T20:30:00+00:00"
+ assert by_form["8-K"].captured_at != by_form["8-K"].published_at
# The ticker target reaches the EDGAR URL; the configured UA is sent.
from openexecutive.config import get_settings
@@ -261,5 +265,10 @@ def test_severity_for_form() -> None:
assert _severity_for_form("4") == AlertSeverity.LOW
+def test_edgar_is_a_seeding_source() -> None:
+ """The Atom feed lists past filings on every poll — first poll must baseline."""
+ assert EdgarSource.seed_on_first_poll is True
+
+
def test_edgar_registered() -> None:
assert "edgar" in list_registered_kinds()
diff --git a/packages/core/tests/unit/test_monitoring_pipeline.py b/packages/core/tests/unit/test_monitoring_pipeline.py
index 1927d2e8..18f7568d 100644
--- a/packages/core/tests/unit/test_monitoring_pipeline.py
+++ b/packages/core/tests/unit/test_monitoring_pipeline.py
@@ -10,7 +10,7 @@ pipeline orchestration code instead.
from __future__ import annotations
import asyncio
-from datetime import UTC, datetime
+from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
@@ -25,8 +25,10 @@ from openexecutive.monitoring.models import (
MODE_ACTIVE,
MODE_DRY_RUN,
OUTCOME_ALERTED,
+ OUTCOME_SUPPRESSED_BASELINE,
OUTCOME_SUPPRESSED_BELOW_FLOOR,
OUTCOME_SUPPRESSED_DRY_RUN,
+ OUTCOME_SUPPRESSED_STALE,
Signal,
WatchlistItem,
)
@@ -57,10 +59,12 @@ class _FakeSource:
kind: str = "vendor_status" # reuse a registered kind for routing
default_poll_interval_minutes: int = 5
+ seed_on_first_poll: bool = False
- def __init__(self, signals: list[Signal]) -> None:
+ def __init__(self, signals: list[Signal], *, seed: bool = False) -> None:
self._signals = signals
self.calls = 0
+ self.seed_on_first_poll = seed
async def poll(
self, item: WatchlistItem, *, db_path: Path | None = None
@@ -84,8 +88,8 @@ def install_fake_source(monkeypatch: pytest.MonkeyPatch):
original = SOURCE_REGISTRY.get("vendor_status")
installed: list[_FakeSource] = []
- def _set(signals: list[Signal]) -> _FakeSource:
- fake = _FakeSource(signals)
+ def _set(signals: list[Signal], *, seed: bool = False) -> _FakeSource:
+ fake = _FakeSource(signals, seed=seed)
SOURCE_REGISTRY["vendor_status"] = fake # type: ignore[assignment]
installed.append(fake)
return fake
@@ -103,12 +107,14 @@ def _make_signal(
dedup_key: str = "vendor_status:abc",
severity: AlertSeverity = AlertSeverity.HIGH,
summary: str = "Test incident",
+ published_at: str | None = None,
) -> Signal:
return Signal(
watchlist_id=0, # set by _FakeSource at poll time
source_kind="vendor_status",
source_external_id="upstream-1",
captured_at=datetime.now(UTC).isoformat(),
+ published_at=published_at,
normalized_summary=summary,
raw_payload={"source": "fake"},
provenance_url="https://example.com/incident/1",
@@ -320,6 +326,221 @@ def test_dedup_key_collision_is_silent(
assert len(ms.list_recent_signals(db_path=db)) == 1 # still 1
+# --------------------------------------------------------------------- #
+# Freshness gates (issue #80) — published_at vs captured_at
+# --------------------------------------------------------------------- #
+
+
+def _install_promotion_recorder(monkeypatch: pytest.MonkeyPatch) -> list[Any]:
+ promoted: list[Any] = []
+ monkeypatch.setattr(
+ "openexecutive.monitoring.pipeline.schedule_evaluation",
+ lambda event: promoted.append(event),
+ )
+ return promoted
+
+
+def _iso_days_ago(days: float) -> str:
+ return (datetime.now(UTC) - timedelta(days=days)).isoformat()
+
+
+def test_stale_published_signal_is_recorded_but_not_promoted(
+ db: Path,
+ install_fake_source,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """A January article fetched in September is not September news: a
+ signal whose upstream published_at is past the age gate lands in
+ external_signals (audit + dedup) with outcome suppressed_stale and
+ never reaches triage."""
+ promoted = _install_promotion_recorder(monkeypatch)
+ ms.insert_watchlist_item(
+ slug="acme-blog", signal_type="vendor_status",
+ target="https://example.com/feed", db_path=db,
+ )
+ install_fake_source([
+ _make_signal(dedup_key="vendor_status:old", published_at=_iso_days_ago(60)),
+ ])
+
+ written = asyncio.run(mp.run_external_monitor_scan(db_path=db))
+ assert written == 1
+ assert promoted == []
+ rows = ms.list_recent_signals(db_path=db)
+ assert len(rows) == 1
+ assert rows[0]["processed_outcome"] == OUTCOME_SUPPRESSED_STALE
+ assert rows[0]["published_at"] is not None
+ # Stale suppression is not a "fire" — trust stats must not move.
+ item = ms.list_watchlist(db_path=db)[0]
+ assert item.fired_count == 0
+
+
+def test_fresh_published_signal_promotes(
+ db: Path,
+ install_fake_source,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Inside the age window the gate is transparent — same path as before."""
+ promoted = _install_promotion_recorder(monkeypatch)
+ ms.insert_watchlist_item(
+ slug="acme-blog", signal_type="vendor_status",
+ target="https://example.com/feed", db_path=db,
+ )
+ install_fake_source([
+ _make_signal(dedup_key="vendor_status:new", published_at=_iso_days_ago(1)),
+ ])
+
+ asyncio.run(mp.run_external_monitor_scan(db_path=db))
+ assert len(promoted) == 1
+ rows = ms.list_recent_signals(db_path=db)
+ assert rows[0]["processed_outcome"] == OUTCOME_ALERTED
+
+
+def test_age_gate_disabled_when_zero(
+ db: Path,
+ install_fake_source,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """EXTERNAL_MONITOR_MAX_SIGNAL_AGE_DAYS=0 turns the gate off entirely."""
+ promoted = _install_promotion_recorder(monkeypatch)
+ from openexecutive.config import get_settings
+ base = get_settings()
+ monkeypatch.setattr(
+ "openexecutive.monitoring.pipeline.get_settings",
+ lambda: base.model_copy(update={"external_monitor_max_signal_age_days": 0}),
+ )
+ ms.insert_watchlist_item(
+ slug="acme-blog", signal_type="vendor_status",
+ target="https://example.com/feed", db_path=db,
+ )
+ install_fake_source([
+ _make_signal(dedup_key="vendor_status:old", published_at=_iso_days_ago(400)),
+ ])
+
+ asyncio.run(mp.run_external_monitor_scan(db_path=db))
+ assert len(promoted) == 1
+
+
+def test_undated_signal_passes_age_gate() -> None:
+ """Sources with no upstream timestamp (stock, page_watch, query) are
+ never judged stale; a malformed timestamp is treated as fresh too —
+ dropping an event on a parse bug would hide the bug."""
+ now = datetime.now(UTC)
+ assert mp._is_stale(_make_signal(), now, 7) is False
+ assert mp._is_stale(_make_signal(published_at="not-a-date"), now, 7) is False
+ assert mp._is_stale(_make_signal(published_at=_iso_days_ago(8)), now, 7) is True
+ assert mp._is_stale(_make_signal(published_at=_iso_days_ago(6)), now, 7) is False
+ # Naive timestamps are read as UTC rather than blowing up on comparison.
+ naive = (now - timedelta(days=30)).replace(tzinfo=None).isoformat()
+ assert mp._is_stale(_make_signal(published_at=naive), now, 7) is True
+
+
+def test_first_poll_baselines_seeding_source(
+ db: Path,
+ install_fake_source,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """A feed-listing source (rss / edgar) returns its whole back-catalogue
+ on every poll. The first poll of a new watch must record those entries
+ as seen without promoting any of them; only entries that appear on a
+ later poll fire."""
+ promoted = _install_promotion_recorder(monkeypatch)
+ ms.insert_watchlist_item(
+ slug="acme-changelog", signal_type="vendor_status",
+ target="https://example.com/feed", db_path=db,
+ )
+ fake = install_fake_source(
+ [
+ _make_signal(dedup_key="vendor_status:e1", published_at=_iso_days_ago(1)),
+ _make_signal(dedup_key="vendor_status:e2"), # undated
+ ],
+ seed=True,
+ )
+
+ # Tick 1 — baseline: both recorded, nothing promoted, nothing "fired".
+ written = asyncio.run(mp.run_external_monitor_scan(db_path=db))
+ assert written == 2
+ assert promoted == []
+ outcomes = {r["dedup_key"]: r["processed_outcome"] for r in ms.list_recent_signals(db_path=db)}
+ assert outcomes == {
+ "vendor_status:e1": OUTCOME_SUPPRESSED_BASELINE,
+ "vendor_status:e2": OUTCOME_SUPPRESSED_BASELINE,
+ }
+ item = ms.list_watchlist(db_path=db)[0]
+ assert item.fired_count == 0
+ assert item.last_polled_at is not None
+
+ # Tick 2 — a genuinely new entry appears alongside the old ones.
+ fake._signals.append(
+ _make_signal(dedup_key="vendor_status:e3", published_at=_iso_days_ago(0.1)),
+ )
+ ms.mark_polled(item.id or 0, datetime(2000, 1, 1, tzinfo=UTC), db_path=db)
+ asyncio.run(mp.run_external_monitor_scan(db_path=db))
+ assert [e.external_id for e in promoted] == ["vendor_status:e3"]
+ outcomes = {r["dedup_key"]: r["processed_outcome"] for r in ms.list_recent_signals(db_path=db)}
+ assert outcomes["vendor_status:e3"] == OUTCOME_ALERTED
+ assert outcomes["vendor_status:e1"] == OUTCOME_SUPPRESSED_BASELINE # untouched
+
+
+def test_non_seeding_source_promotes_on_first_poll(
+ db: Path,
+ install_fake_source,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Point-in-time sources (stock threshold, open vendor incident) are
+ actionable on the very first poll — no baseline for them."""
+ promoted = _install_promotion_recorder(monkeypatch)
+ ms.insert_watchlist_item(
+ slug="vendor-stripe", signal_type="vendor_status",
+ target="https://example.com/feed", db_path=db,
+ )
+ install_fake_source([_make_signal(dedup_key="vendor_status:open")], seed=False)
+ asyncio.run(mp.run_external_monitor_scan(db_path=db))
+ assert len(promoted) == 1
+
+
+def test_published_at_column_added_to_existing_db(tmp_path: Path) -> None:
+ """DBs created before published_at existed get the column via the
+ idempotent additive ALTER, and the value round-trips on insert/read."""
+ import sqlite3
+
+ db_path = tmp_path / "legacy.db"
+ with sqlite3.connect(db_path) as conn:
+ conn.executescript("""
+ CREATE TABLE external_signals (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ watchlist_id INTEGER NOT NULL,
+ source_kind TEXT NOT NULL,
+ source_external_id TEXT NOT NULL,
+ captured_at TEXT NOT NULL,
+ normalized_summary TEXT NOT NULL,
+ raw_payload_json TEXT NOT NULL DEFAULT '{}',
+ provenance_url TEXT NOT NULL,
+ severity_hint TEXT NOT NULL DEFAULT 'low',
+ dedup_key TEXT NOT NULL UNIQUE,
+ processed_at TEXT,
+ processed_outcome TEXT,
+ promoted_alert_id INTEGER,
+ enrichment_json TEXT NOT NULL DEFAULT '{}'
+ );
+ """)
+ ms.initialize_db(db_path)
+ ms.initialize_db(db_path) # idempotent — second run must not fail
+ with sqlite3.connect(db_path) as conn:
+ cols = {row[1] for row in conn.execute("PRAGMA table_info(external_signals)")}
+ assert "published_at" in cols
+
+ sig = _make_signal(published_at="2026-01-15T08:00:00+00:00")
+ sig = sig.model_copy(update={"watchlist_id": 1})
+ assert ms.insert_signal(sig, db_path=db_path) is not None
+ rows = ms.list_recent_signals(db_path=db_path)
+ assert rows[0]["published_at"] == "2026-01-15T08:00:00+00:00"
+ # Rows written before the column existed read back as None, not "".
+ undated = _make_signal(dedup_key="vendor_status:legacy").model_copy(update={"watchlist_id": 1})
+ ms.insert_signal(undated, db_path=db_path)
+ by_key = {r["dedup_key"]: r for r in ms.list_recent_signals(db_path=db_path)}
+ assert by_key["vendor_status:legacy"]["published_at"] is None
+
+
# --------------------------------------------------------------------- #
# Heartbeat lifecycle
# --------------------------------------------------------------------- #
@@ -480,6 +701,14 @@ def test_signal_to_alert_event_carries_slug_and_severity_for_triage() -> None:
assert "Provenance: https://finance.yahoo.com/quote/AAPL" in event.body
assert event.source == "stock"
assert event.external_id == "stock:abc123"
+ # No upstream timestamp → no Published line, but Discovered is always there.
+ assert "Published:" not in event.body
+ assert "Discovered: 2026-05-28T12:00:00+00:00" in event.body
+
+ dated = signal.model_copy(update={"published_at": "2026-01-15T08:00:00+00:00"})
+ body = _signal_to_alert_event(dated, item).body
+ assert "Published: 2026-01-15T08:00:00+00:00" in body
+ assert "Discovered: 2026-05-28T12:00:00+00:00" in body
def test_strip_url_query_drops_tokens() -> None:
diff --git a/packages/core/tests/unit/test_monitoring_query_enrichment.py b/packages/core/tests/unit/test_monitoring_query_enrichment.py
index 2566b40a..ae71c8d2 100644
--- a/packages/core/tests/unit/test_monitoring_query_enrichment.py
+++ b/packages/core/tests/unit/test_monitoring_query_enrichment.py
@@ -372,6 +372,7 @@ def db(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
class _FakeSource:
kind: str = "vendor_status"
default_poll_interval_minutes: int = 5
+ seed_on_first_poll: bool = False
def __init__(self, signals: list[Signal]) -> None:
self._signals = signals
@@ -536,6 +537,7 @@ async def test_query_kill_switch_skips_query_rows(
class _SpyQuery:
kind = "query"
default_poll_interval_minutes = 360
+ seed_on_first_poll: bool = False
async def poll(
self, item: WatchlistItem, *, db_path: Path | None = None
@@ -576,6 +578,7 @@ class _FakeQuerySource:
kind: str = "query"
default_poll_interval_minutes: int = 360
+ seed_on_first_poll: bool = False
def __init__(self, signals: list[Signal]) -> None:
self._signals = signals
diff --git a/packages/core/tests/unit/test_monitoring_sources_rss_stock.py b/packages/core/tests/unit/test_monitoring_sources_rss_stock.py
index 1ac0872b..5b803f05 100644
--- a/packages/core/tests/unit/test_monitoring_sources_rss_stock.py
+++ b/packages/core/tests/unit/test_monitoring_sources_rss_stock.py
@@ -96,6 +96,45 @@ async def test_rss_emits_signals_for_each_entry(
assert signals[0].severity_hint == AlertSeverity.LOW
assert signals[0].dedup_key.startswith("rss:")
assert signals[0].dedup_key != signals[1].dedup_key
+ # <pubDate> lands on published_at as ISO 8601 UTC — distinct from
+ # captured_at (issue #80: when it happened vs when we first saw it).
+ assert signals[0].published_at == "2026-05-28T12:00:00+00:00"
+ assert signals[1].published_at == "2026-05-27T09:00:00+00:00"
+ assert signals[0].captured_at != signals[0].published_at
+
+
+async def test_rss_entry_without_date_has_no_published_at(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """A feed item with no <pubDate>/<updated> can't be aged — published_at
+ stays None so the pipeline's age gate passes it through (the first-poll
+ baseline is the only replay guard for such feeds)."""
+ undated = _SAMPLE_RSS.replace(
+ b"<pubDate>Wed, 28 May 2026 12:00:00 GMT</pubDate>", b""
+ )
+
+ async def fake_fetch(url: str, max_bytes: int) -> bytes:
+ return undated
+
+ monkeypatch.setattr(
+ "openexecutive.monitoring.sources.rss.fetch_bounded", fake_fetch
+ )
+ monkeypatch.setattr(
+ "openexecutive.monitoring.sources.rss.validate_target_url",
+ lambda u: (True, ""),
+ )
+
+ signals = await RssSource().poll(_make_item())
+ assert len(signals) == 2
+ assert signals[0].published_at is None
+ assert signals[1].published_at == "2026-05-27T09:00:00+00:00"
+
+
+def test_rss_is_a_seeding_source() -> None:
+ """A feed returns its back-catalogue on every poll, so the first poll of a
+ row must be a baseline (see Source.seed_on_first_poll)."""
+ assert RssSource.seed_on_first_poll is True
def test_rss_matches_trigger_keywords() -> None:
diff --git a/packages/ui/src/app/watchlist/[slug]/page.tsx b/packages/ui/src/app/watchlist/[slug]/page.tsx
index 9313d59f..49ca324c 100644
--- a/packages/ui/src/app/watchlist/[slug]/page.tsx
+++ b/packages/ui/src/app/watchlist/[slug]/page.tsx
@@ -49,7 +49,10 @@ function SignalRow({ signal }: { signal: WatchlistSignal }) {
{signal.normalized_summary}
</div>
<div className="text-[11px] text-fg-muted mt-0.5">
- {formatRelTime(signal.captured_at)} ago · severity {signal.severity_hint}
+ {signal.published_at
+ ? `published ${formatRelTime(signal.published_at)} ago · seen ${formatRelTime(signal.captured_at)} ago`
+ : `${formatRelTime(signal.captured_at)} ago`}
+ {" "}· severity {signal.severity_hint}
</div>
</div>
<span
diff --git a/packages/ui/src/lib/api.ts b/packages/ui/src/lib/api.ts
index 6a8c7e6b..63aed833 100644
--- a/packages/ui/src/lib/api.ts
+++ b/packages/ui/src/lib/api.ts
@@ -2693,6 +2693,8 @@ export interface WatchlistSignal {
source_kind: string;
source_external_id: string;
captured_at: string;
+ /** Upstream publish time (RSS pubDate, EDGAR filing date); null when the source has none. */
+ published_at: string | null;
normalized_summary: string;
provenance_url: string;
severity_hint: string;