Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

chore(signals): move the anomaly scout notebook recipe to markdown notebooks (#119294)

posthog[bot]069e3b3acc87
diff --git a/products/signals/skills/signals-scout-anomaly-detection/SKILL.md b/products/signals/skills/signals-scout-anomaly-detection/SKILL.md
index f653a1d6c66c..e877df89f045 100644
--- a/products/signals/skills/signals-scout-anomaly-detection/SKILL.md
+++ b/products/signals/skills/signals-scout-anomaly-detection/SKILL.md
@@ -87,7 +87,7 @@ Memory is continuous, not a final step. Maintain the watchlist and baselines as
For each candidate anomaly, classify against prior runs, the inbox, and the scratchpad (net-new / material-update / already-covered / addressed-or-noise — full classifier in [`references/watchlist-and-memory.md`](references/watchlist-and-memory.md)). You file findings on the **report channel**: a scored, attributed anomaly you'd stand behind is a finished, 1:1 inbox report, not a weak signal for the pipeline to cluster — so you author it directly. Then:
-- **Author** a fresh report via `scout-emit-report` when the move is net-new and clears the bar. **Before you author, write the anomaly up in a notebook** (`notebooks-create`) — the report `summary` is the inbox surface, but the notebook is the durable artifact a human opens to see the charts, the baseline math, and the attribution behind the call. Build it first, then link its URL from the report `summary` and cite it as an `evidence` entry. Attach the scored series to the report itself via `charts` too — reuse the widened-window node you embedded in the notebook — so the break and its baseline are visible in the inbox without opening the notebook. The report contract _and_ the notebook structure — the title/summary prose contract, evidence, actionability, suggested reviewers, the notebook layout + embedded-chart recipe, worked example — are in [`references/report-contract.md`](references/report-contract.md). For this scout a report-worthy anomaly is: robust z ≥ ~3.5 on the latest complete bucket, the move not explained by seasonality or a known data-pipeline gap, with the insight `short_id`, the bucket value, the baseline, the z-score, and the time window in the evidence. **Search the inbox first** (`inbox-reports-list`, plus your `report:` scratchpad pointer) — the channel is not idempotent, so never author a duplicate.
+- **Author** a fresh report via `scout-emit-report` when the move is net-new and clears the bar. **Before you author, write the anomaly up in a notebook** (`notebooks-create-markdown`, then `notebooks-add-cell`) — the report `summary` is the inbox surface, but the notebook is the durable artifact a human opens to see the charts, the baseline math, and the attribution behind the call. Build it first, then link its URL from the report `summary` and cite it as an `evidence` entry. Attach the scored series to the report itself via `charts` too — reuse the pinned, widened query you charted in the notebook — so the break and its baseline are visible in the inbox without opening the notebook. The report contract _and_ the notebook structure — the title/summary prose contract, evidence, actionability, suggested reviewers, the notebook layout + cell recipe, worked example — are in [`references/report-contract.md`](references/report-contract.md). For this scout a report-worthy anomaly is: robust z ≥ ~3.5 on the latest complete bucket, the move not explained by seasonality or a known data-pipeline gap, with the insight `short_id`, the bucket value, the baseline, the z-score, and the time window in the evidence. **Search the inbox first** (`inbox-reports-list`, plus your `report:` scratchpad pointer) — the channel is not idempotent, so never author a duplicate.
- **Edit** the existing report via `scout-edit-report` when one already covers this insight's anomaly (found via the inbox search or a `report:anomaly_detection:insight:<short_id>` pointer) and you have a material update — it's still firing, escalated, or correlates with a fresh deploy. Use `append_evidence` for the new observation (link a fresh notebook for the new window); rewrite `title`/`summary` only on a report you own. Don't author a second report for the same ongoing move.
- **Remember** if it's suggestive but below the bar, or to refresh a baseline / record what you ruled out.
- **Skip** if a `noise:` / `addressed:` / `report:` entry already covers it without new evidence.
@@ -127,7 +127,7 @@ Local: `Bash` + `python3` — the distribution-shift lens: run a pure-stdlib two
Write (user-facing):
- `scout-emit-report` / `scout-edit-report` (gated on `signal_scout_report:write`) — the report channel: author a full inbox report for an anomaly, or update the existing one on a recurrence. Field-level contract in [`references/report-contract.md`](references/report-contract.md).
-- `notebooks-create` (gated on `notebook:write`) — the durable write-up that backs an authored report. Build it _before_ authoring and reference its URL from the report `summary` and an `evidence` entry. Layout + embedded-chart recipe (embed the anomalous insight with a `SavedInsightNode`; chart a SQL-fallback series with a `DataVisualizationNode`) is in [`references/report-contract.md`](references/report-contract.md).
+- `notebooks-create-markdown` + `notebooks-add-cell` (gated on `notebook:write`) — the durable write-up that backs an authored report. Build it _before_ authoring and reference its URL from the report `summary` and an `evidence` entry. Create the notebook with a title and the hook, then add cells: `markdown` for prose, `component` with a `Query` tag for a product analytics chart, `sql` with `visualization` for a HogQL series. The layout and cell recipe is in [`references/report-contract.md`](references/report-contract.md). `notebooks-create` is retired. Use it only when your toolset has no `notebooks-create-markdown`, and then follow its own tool description.
- `notebooks-destroy` — clean up the write-up if the report did not surface (preflight gate-skip, or the safety judge suppressed it) so a non-surfacing run leaves no orphan artifact. See [`references/report-contract.md`](references/report-contract.md).
- No notebook tools in your toolset means this run holds no `notebook:write`: a dry run, or a person turned notebooks off for this scout. Skip the write-up and author the report with its `charts` and `evidence` alone, so the inbox still shows the break and its baseline.
diff --git a/products/signals/skills/signals-scout-anomaly-detection/references/report-contract.md b/products/signals/skills/signals-scout-anomaly-detection/references/report-contract.md
index 053a39a6340a..dadca62dcbdc 100644
--- a/products/signals/skills/signals-scout-anomaly-detection/references/report-contract.md
+++ b/products/signals/skills/signals-scout-anomaly-detection/references/report-contract.md
@@ -82,7 +82,7 @@ characters behind a "Show more" toggle. Front-load, structure, no walls:
Cite the insight and dashboard inline as markdown links so a human pivots straight to the
source, and close by linking the notebook: "Full write-up with charts: [notebook](url)."
The harness prompt (_Linking what you reference_) carries the link rule: reuse the url the
-returning tool attached (`notebooks-create` returns the notebook's), else `generate-app-url`.
+returning tool attached (`notebooks-create-markdown` returns the notebook's), else `generate-app-url`.
Keep the title and the summary's first line plain text, because the inbox renders both as
text: a link there shows up as literal markdown.
@@ -90,7 +90,7 @@ text: a link there shows up as literal markdown.
The notebook stays the durable write-up, but attach the scored series to the report itself via
`charts` too, so the inbox reader sees the break without leaving the report. Reuse the same
-widened-window node you embed in the notebook — but a `SavedInsightNode` only when the insight's
+pinned, widened query you chart in the notebook — but a `SavedInsightNode` only when the insight's
saved range is **absolute** and contains both the baseline and the anomaly (a relative range
like `-63d` shows them today and then slides until the anomaly scrolls off the durable report),
else a widened ad-hoc `InsightVizNode` or `DataVisualizationNode` with pinned dates (and its
@@ -180,90 +180,96 @@ on a recurrence).
### Create it
-Call `notebooks-create` with a `title` and `content` (ProseMirror rich-text JSON). The response
-carries the new notebook's `short_id` and a clickable URL in `_posthogUrl` (the tool enriches the
-result with `/notebooks/{short_id}`) — surface that verbatim, don't hand-build it. If you ever
-need to build the link yourself, it is `generate-app-url` with `url=/notebooks/{shortId}`.
+Call `notebooks-create-markdown` with a `title` and an optional `markdown` body. The `markdown`
+body holds prose only (the quantified hook); add every chart as a cell afterwards. The response
+carries the notebook's short id in `notebook_id` and a clickable URL in `_posthogUrl`. Surface
+that URL verbatim, don't hand-build it. If you ever need to build the link yourself, it is
+`generate-app-url` with `url=/notebooks/{shortId}`.
+
+Then add the cells in reading order with `notebooks-add-cell`, passing the `notebook_id`. Each
+call appends at the end of the document, so no `after_node_id` is needed.
- **Title** — name the metric, the direction, and the date, e.g.
- `Anomaly: daily signups dropped ~60% (2026-06-06)`.
+ `Anomaly: daily signups dropped ~60% (2026-06-06)`. The tool writes it as the leading
+ `# heading`, so do not repeat it in `markdown`.
- **One notebook per report** — never append a new anomaly to a prior run's notebook. A
recurrence on a later day is a new notebook, linked from the `edit_report` note.
+`notebooks-create` (ProseMirror `content` with `ph-query` nodes) is retired. Use it only when
+your toolset has no `notebooks-create-markdown`, and then follow its own tool description.
+
### What goes in it
Lead with the same hook the inbox sees, then the evidence the ~300-char preview can't hold:
1. **Summary** — the quantified hook (bucket value vs baseline, robust z, priority), one or two
- sentences. Same claim as the report summary, so the notebook stands alone.
-2. **The chart** — embed the anomalous series so the spike/drop is visible, with the window wide
- enough (e.g. `-63d`) that the baseline _and_ the break are both on screen. A `SavedInsightNode`
- renders the insight's own saved date range and carries no date override — so use it only when
- that saved range already shows the baseline; if the insight is saved to a short window (often
- `-7d`), embed an inline widened `DataVisualizationNode` (or `InsightVizNode`) for the scored
- window instead.
+ sentences, in the `markdown` body of `notebooks-create-markdown`. Same claim as the report
+ summary, so the notebook stands alone.
+2. **The chart** — chart the anomalous series so the spike/drop is visible, with **pinned
+ absolute dates** wide enough (e.g. 63 days back from the scored bucket) that the baseline
+ _and_ the break are both on screen. A relative range like `-63d` slides, and the anomaly
+ scrolls off the durable write-up. A `saved_insight` cell renders the insight's own saved date
+ range with no override, so use it only when that saved range is absolute and shows the
+ baseline. Otherwise add a widened `Query` component or SQL cell for the scored window.
3. **Baseline & method** — the seasonality-matched baseline (median + MAD per bucket), the
z-score, which detector(s) `alert-simulate` fired, and that the partial bucket was excluded.
4. **Attribution** — which breakdown segment(s) drove the move, and whether it's broad
(regression) or one segment (often expected).
5. **Hypothesis & next step** — suspected cause and what to check, matching the report summary.
-### Embedded-chart recipe
-
-Charts are `{type: "ph-query", attrs: {nodeId: "<unique>", query: <query>}}` nodes inside
-`content`. `query` is one of:
-
-- **Embed the anomalous saved insight** —
- `{kind: "SavedInsightNode", shortId: "<short_id>"}`. Renders the insight's _saved_ date range
- (no override); fine when that range shows the baseline, otherwise prefer a widened node below.
-- **Chart a SQL-fallback series** —
- `{kind: "DataVisualizationNode", source: {kind: "HogQLQuery", query: "SELECT ..."}, display: "ActionsLineGraph"}`.
- Do **not** wrap a `HogQLQuery` in an `InsightVizNode`.
-- **Build an ad-hoc product-analytics chart** —
- `{kind: "InsightVizNode", source: {kind: "TrendsQuery", ...}}`.
-
-Prefer embedding the saved insight you scored — it stays in sync with the source and is the thing
-the human will open next. Give each `ph-query` node a distinct `nodeId`.
-
-`content` is a ProseMirror doc (use this skeleton). Text is
-`paragraph` / `heading` (with `attrs.level`) / `bulletList` → `listItem` → `paragraph`; charts are
-`ph-query` nodes. A minimal working shape:
-
-```json
-{
- "type": "doc",
- "content": [
- {
- "type": "heading",
- "attrs": { "level": 1 },
- "content": [{ "type": "text", "text": "Anomaly: <metric> <direction> (<date>)" }]
- },
- { "type": "paragraph", "content": [{ "type": "text", "text": "<the quantified hook>" }] },
- {
- "type": "ph-query",
- "attrs": { "nodeId": "scored-insight", "query": { "kind": "SavedInsightNode", "shortId": "<short_id>" } }
- },
- { "type": "heading", "attrs": { "level": 2 }, "content": [{ "type": "text", "text": "Baseline & method" }] },
- {
- "type": "bulletList",
- "content": [
- {
- "type": "listItem",
- "content": [
- {
- "type": "paragraph",
- "content": [{ "type": "text", "text": "<baseline median + MAD, the z, partial bucket excluded>" }]
- }
- ]
- }
- ]
- }
- ]
-}
+### Cell recipe
+
+Give every non-markdown cell a short `title` that says what it shows. Markdown cells take no
+`title`; start them with a `##` heading instead.
+
+| What | `notebooks-add-cell` arguments |
+| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Prose section | `cell_type: "markdown"`, `markdown: "## Baseline & method\n\n- ..."` |
+| Product analytics chart (pinned) | `cell_type: "component"`, `tag_name: "Query"`, `props: {"query": {"kind": "InsightVizNode", "source": <TrendsQuery with an absolute dateRange>}}` |
+| HogQL series (the SQL-fallback score) | `cell_type: "sql"`, `code: "SELECT ..."` (pinned dates in the `WHERE`), `visualization: {"display": "ActionsLineGraph", "chartSettings": {"xAxis": {"column": "day"}, "yAxis": [{"column": "signups"}]}}` |
+| Saved insight (absolute range only) | `cell_type: "saved_insight"`, `insight_short_id: "<short_id>"` |
+
+- **Query component.** `props.query` takes an `InsightVizNode` wrapping a `TrendsQuery`,
+ `FunnelsQuery`, `RetentionQuery`, `PathsQuery`, `StickinessQuery`, or `LifecycleQuery`. Copy the
+ scored insight's query and replace its `dateRange` with absolute `date_from` / `date_to`. Do
+ not put a `HogQLQuery` here; it is refused.
+- **SQL cell.** It runs as soon as you add it and returns the columns and a row preview. Read
+ the result and fix the query in place with `notebooks-update-cell` if it is wrong. Name the
+ axes in `chartSettings` with columns the query returns, and read `visualization_warnings` in
+ the response: a warning means the chart does not draw what you expect.
+
+A minimal write-up, as four calls:
+
+```yaml
+# 1. notebooks-create-markdown
+title: 'Anomaly: daily signups dropped ~60% (2026-06-06)'
+markdown: 'Daily signups fell to 412 on 2026-06-06 against a same-weekday median of 1,048 (robust z = 4.8, P1).'
+# → notebook_id: aB12cD34, _posthogUrl: <notebook url>
+
+# 2. notebooks-add-cell
+notebook_id: aB12cD34
+cell_type: component
+tag_name: Query
+title: Daily signups, 2026-04-04 to 2026-06-06
+props:
+ query:
+ kind: InsightVizNode
+ source:
+ kind: TrendsQuery
+ interval: day
+ dateRange: { date_from: '2026-04-04', date_to: '2026-06-06T23:59:59' }
+ series: [{ kind: EventsNode, event: signed_up, math: total, custom_name: Signups }]
+
+# 3. notebooks-add-cell
+notebook_id: aB12cD34
+cell_type: markdown
+markdown: "## Baseline & method\n\n- 8-same-weekday median 1,048, MAD 95, robust z = 4.8.\n- Partial bucket for 2026-06-07 excluded."
+
+# 4. notebooks-add-cell (attribution and hypothesis, same shape as 3)
```
-For a SQL-fallback chart, swap the `ph-query` query for
-`{ "kind": "DataVisualizationNode", "source": { "kind": "HogQLQuery", "query": "SELECT ..." }, "display": "ActionsLineGraph" }`.
+For a SQL-fallback series, replace call 2 with a `sql` cell that selects the bucket and the value
+over the same pinned window, and set `visualization` as in the table.
### Clean up if the report doesn't surface
@@ -273,7 +279,7 @@ preflight gates and safety judge. If the report **did not surface** — `emit_re
report created), or the safety judge suppressed it (`emitted=false`) — **delete the just-created
notebook with `notebooks-destroy`** so a non-surfacing run leaves no orphan user-facing artifact.
Only a notebook behind a surfaced report should survive the run. Skipping notebook _creation_ is
-acceptable only if `notebooks-create` fails — then author anyway and note the missing artifact in
+acceptable only if `notebooks-create-markdown` fails — then author anyway and note the missing artifact in
the summary.
## Worked example
@@ -300,7 +306,7 @@ evidence:
±1.5 z. Latest complete day only; today's partial bucket excluded.
- source_id: aB12cD34
description: >
- [Write-up](https://us.posthog.com/project/41233/notebooks/aB12cD34) with the -63d chart,
+ [Write-up](https://us.posthog.com/project/41233/notebooks/aB12cD34) with the pinned 63-day chart,
the per-weekday baseline, and the segment attribution.
summary: |
Daily signups dropped to 412 yesterday (2026-06-06) against an ~1,048 same-weekday baseline (robust z = 4.8, MAD 95).