Skip to content

armgit / AiSOC

Public Imported from GitHub
0

Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.

chore: remove stale internal progress trackers (#232)

Beenu Aroradeae03f68b42
diff --git a/AI_STACK_PLAN_PROGRESS.md b/AI_STACK_PLAN_PROGRESS.md
deleted file mode 100644
index 1f3b94074..000000000
--- a/AI_STACK_PLAN_PROGRESS.md
+++ /dev/null
@@ -1,681 +0,0 @@
-# AI Stack & Data Integration Plan — Progress Tracker
-
-Tracking progress against the `ai-stack-data-integration-plan_e90071ca.plan.md`
-spec (also attached as `uploads/ai-stack-data-integration-plan_e90071ca.plan-L1-L332-0.md`).
-
-**Plan file is read-only — never edit it. Update this tracker instead.**
-
----
-
-## How to resume after a session restart
-
-1. Re-open `/Users/beenu/Desktop/AiSOC` in your editor.
-2. Read this file top-to-bottom.
-3. Re-create the todo list from the snapshot in the
- "Live todo snapshot" section below — keep the same `id`s.
-4. Resume from the section marked **"Resume here"** at the bottom.
-5. Workspace rule: continue without stopping until every todo is done.
-
----
-
-## Workstream status
-
-| WS | Title | Status | Anchor |
-|----|------------------------------------|-------------|--------|
-| 1 | Repo + plan alignment | DONE | pre-session |
-| 2 | Click-and-connect OAuth | DONE | pre-session |
-| 3 | Catalog expansion (P0 batch) | DONE | 14 connectors landed + manifests + docs |
-| 4 | Capability taxonomy | DONE | enum + per-instance scoping + `/agents/tools` |
-| 5 | Self-healing | DONE | OAuth refresh + backfill + freshness SLO + UI badges |
-| 6 | Universal capture | DONE | `/v1/inbox/*` in `services/ingest` + tokens panel |
-| 7 | Tenant lake API | DONE | endpoints + rewriter + rate limiter + 69/69 tests |
-| 8 | Bidirectional ITSM | DONE | push_case + `/v1/inbox/itsm` + fan-out service + 113/113 tests |
-| 9 | Docs + Sonali | DONE | `api-coverage.md` + `itsm-as-source-of-truth.md` + `sonali-consultation-questions.md` + sidebars + SYSTEM_DESIGN §12 |
-
----
-
-## GitHub hygiene workstream (in flight)
-
-User asked for:
-1. GitHub fully updated (docs/architecture in sync with code).
-2. Code-scanning alerts at https://github.com/beenuar/AiSOC/security/code-scanning fixed.
-3. Contributor graph showing only `beenu` (no automation accounts, no `AiSOC Bot`).
-4. **Author identity for all commits: `Beenu Arora <[email protected]>`** — no
- co-author trailers, no automation addresses, no `users.noreply.github.com`.
-
-### Status
-
-| ID | Task | Status |
-|----------------------------|------------------------------------------------------------------------------|-------------|
-| gh-1 | Audit current git state, branches, identities | DONE |
-| gh-4 | Secret scan with gitleaks + triage false positives | DONE (`.gitleaksignore` committed) |
-| gh-2 | Pull live code-scanning alerts and triage | DONE (47 alerts triaged) |
-| gh-6 | Fix code-scanning issues (critical/high first) | DONE for SSRF + log-injection in `cases.py` + `fusion.py`; remaining notes (unused imports, bind-all, weak hash) triaged as benign |
-| gh-5 | Sync docs/architecture so GitHub matches current state | DONE (`docs/architecture/SYSTEM_DESIGN.md` §12 added; topology + service table updated) |
-| gh-commit-pre-rewrite | Commit fixes + tracker with `Beenu Arora <[email protected]>`, no co-authors | DONE |
-| gh-3a..gh-3e | History-rewrite plan (filter-repo + force-push) | DONE (3 passes; verified clean; CI workflows patched) |
-| gh-prs | Close 19 dependabot PRs; comment on #37 for K4R7IK to rebase | DONE |
-| gh-7 | Verify code-scanning alerts cleared after push | DONE (0 open high/medium/error alerts; 3 false positives dismissed with rationales) |
-| gh-7a | Add `safe_log_value` helper + apply to 19 `py/log-injection` sites | DONE |
-| gh-7b | Fix `py/url-redirection` in `oauth.py` (whitelist + URL reconstruction) | DONE |
-| gh-7c | Fix/dismiss 3 `py/incomplete-url-substring-sanitization` alerts | DONE |
-| gh-7d | Dismiss CodeQL false positives with rationales | DONE |
-| gh-7e | Clean up note-level alerts (unused imports/vars, empty except) | DONE |
-| tryaisoc-cj | Customer journey review of tryaisoc.com — find and fix bugs | DONE |
-
-### Co-author trailer issue (resolved at the local layer)
-
-A shell harness was injecting an unwanted `Co-authored-by:` trailer into
-every commit message at shell-invocation time, even though local git config
-is clean (`user.name=Beenu Arora`, `[email protected]`, no
-`commit.template`, no active hooks, no mailmap).
-
-Workaround applied: after each commit, run
-
-```bash
-FILTER_BRANCH_SQUELCH_WARNING=1 git filter-branch -f \
- --msg-filter "sed '/^Co-authored-by: /d' \
- | awk 'NF{p=1} p' \
- | awk 'BEGIN{n=0} {lines[n++]=\$0} END{e=n-1; while(e>=0 && lines[e]==\"\") e--; for(i=0;i<=e;i++) print lines[i]}'" \
- HEAD~N..HEAD
-```
-
-…where `N` is the number of recent commits with the trailer. The two AwK
-passes strip leading and trailing blank lines so the rewritten commit message
-is byte-clean.
-
-Verification command:
-
-```bash
-git log --format='%h %an <%ae>%n%(trailers:only=true)' -n 5
-```
-
----
-
-## WS7 — Tenant lake API (shipped)
-
-### What landed
-
-- `services/api/pyproject.toml` — `sqlglot>=23.0.0,<27.0.0` + `clickhouse-driver`.
-- `services/api/app/db/clickhouse.py` — async ClickHouse client wrapper:
- `get_clickhouse_client`, `close_clickhouse`, `execute_lake_query`,
- `fetch_lake_schema`. Exceptions: `LakeQueryNotConfiguredError`,
- `LakeQueryTimeoutError`, `LakeQueryError`. Returns `LakeQueryResult` dataclass.
-- `services/api/app/services/lake_sql.py` — sqlglot rewriter:
- - `rewrite_for_tenant(sql, tenant_id, *, max_limit, allowlist)` →
- `RewriteResult(sql, referenced_tables, applied_limit)`.
- - SELECT-only allowlist; rejects DML/DDL/`KILL`/table-valued functions.
- - Recursive `tenant_id` predicate injection via `optimizer.scope`.
- - Empty-projection `SELECT` rejection (treats bare `SELECT` as syntax error).
- - **Subtle bug fixed**: `_direct_tables` previously read `select.args.get("from_")`;
- sqlglot stores the FROM clause under the dict key `"from"` (the Python attr
- is `from_`, but `args[]` uses the YAML-style key). All FROMs were silently
- skipped before the fix → no predicate injection, empty `referenced_tables`.
- - **Forbidden roots**: `(exp.Insert, exp.Update, exp.Delete, exp.Drop, exp.Alter,
- exp.Create, exp.TruncateTable, exp.Command, exp.Kill)`.
-- `services/api/app/services/lake_rate_limit.py` — per-tenant in-memory token
- bucket. `LakeRateLimiter.acquire(tenant_id, cost=1.0)` → `RateLimitDecision`
- with `to_headers()` for `X-RateLimit-*` + `Retry-After`. Uses
- `time.monotonic` for refill clock; `_TokenBucket.last_refill` is a
- `field(default_factory=time.monotonic)` — see "Test gotcha" below.
-- `services/api/app/api/v1/endpoints/lake.py` — `POST /api/v1/lake/sql` and
- `GET /api/v1/lake/schema`. Uses `AuthUser`, `DBSession`,
- `require_permission("lake:query"|"lake:read_schema")`. Helpers:
- `_acquire_or_429`, `_scrub_sql_for_log`, `_audit_query_attempt`.
- Pydantic models: `LakeQueryRequest`, `LakeQueryResponse`, `LakeColumnInfo`,
- `LakeTableInfo`, `LakeSchemaResponse`.
-- `services/api/migrations/034_lake_permissions.sql` — registers
- `lake:query` + `lake:read_schema` in `role_permissions`.
-- `services/mcp/src/tools/lake.ts` — `aisoc_lake_query` + `aisoc_lake_schema`
- MCP tools (typed surface, prompt-injection guards, calls API endpoints).
-- `services/mcp/tests/lake.test.ts` — comprehensive tests for both tools
- (Zod schemas, multi-statement guard, forbidden tables, response shape).
-
-### Tests — done
-
-- `services/api/tests/test_lake_sql.py` — **44/44 passing**.
-- `services/api/tests/test_lake_rate_limit.py` — **25/25 passing**.
-
-### Test gotcha — `LakeRateLimiter` + `time.monotonic`
-
-`_TokenBucket.last_refill` is set via `field(default_factory=time.monotonic)`,
-which captures the **real** clock at bucket-creation time. If a test patches
-`app.services.lake_rate_limit.time.monotonic` *after* the bucket has been
-created, `_refill` computes `elapsed = patched_now - real_creation_now`,
-which is hugely negative and gets clamped to 0 → no tokens refill.
-
-Workaround already used in `test_lake_rate_limit.py`:
-
-```python
-async def _seed_bucket(limiter, tenant, *, at_time):
- """Force-create the tenant's bucket and align last_refill with the patched clock."""
- await limiter.acquire(tenant, cost=0.0)
- bucket = limiter._buckets[tenant]
- bucket.last_refill = at_time
-```
-
-Apply the same pattern when writing endpoint tests that depend on rate-limit
-timing.
-
----
-
-## Security fixes (gh-6)
-
-### `services/api/app/api/v1/endpoints/fusion.py`
-
-- Added `_SAFE_PATH_RE = re.compile(r"^/[A-Za-z0-9_\-./%]*$")` and
- `_validate_proxy_path(path)` that rejects `..`, protocol-relative `//host/...`,
- and any character outside the allowlist.
-- `_proxy_get` validates the path before any `httpx.AsyncClient.get` and logs
- the *validated* path (kills the `py/log-injection` sink).
-- `entity_risk_detail` now URL-encodes `entity_type` + `entity_value` via
- `urllib.parse.quote(safe="")` before composing the path.
-
-### `services/api/app/api/v1/endpoints/cases.py`
-
-- Same path validator pattern (`_SAFE_PROXY_PATH_RE`, `_validate_agents_path`),
- same `//` rejection.
-- `case_investigation_run` URL-encodes `run_id` before proxying.
-- `case_investigation_pdf` URL-encodes `run_id` and scrubs both `case_id` and
- `run_id` for the `Content-Disposition` filename via
- `_safe_filename_segment` (kills CR/LF/quote injection in headers).
-
-### `.gitleaksignore`
-
-Six triaged false positives, each with a comment:
-
-- `detections/fixtures/positive/jwt-none-alg.json:jwt:2` — positive test fixture.
-- `render.yaml:generic-api-key:70` and `:124` — env var **names**
- (`AISOC_DISABLE_NEO4J=true`).
-- `scripts/detection_specs_part3_application.py:generic-api-key:61` —
- detection rule literal `count_5min_per_ip_gt: 30`.
-- `scripts/detection_specs_part2.py:jwt:1111` — positive sample for jwt-none-alg.
-- `services/api/tests/test_security_defaults.py:generic-api-key:34` —
- pre-generated Fernet key for tests; production injects the real one via env.
-
----
-
-## WS7 — Verification
-
-```bash
-cd services/api && source .venv/bin/activate
-python -m pytest tests/test_lake_sql.py tests/test_lake_rate_limit.py -q
-# 69 passed
-```
-
-`ws7-verify` (full pytest + MCP build) is still pending and tracked under the
-post-rewrite block.
-
----
-
-## WS8 — Bidirectional ITSM (shipped)
-
-### What landed
-
-- `services/connectors/app/connectors/base.py` — extended `BaseConnector` with
- abstract `push_case` and `push_status_change` methods, plus
- `Capability.PUSH_CASE` / `Capability.PUSH_STATUS` enum members.
-- `services/connectors/app/connectors/jira_connector.py` —
- - `push_case`: creates Jira issue with ADF-formatted description, severity →
- priority mapping, summary truncation at 255 chars, `aisoc-case-{id}` label
- for round-trip identification, returns `{external_id, external_url, vendor,
- external_status}`.
- - `push_status_change`: discovers transitions via
- `GET /rest/api/3/issue/{key}/transitions`, picks the matching transition by
- target status name, POSTs the transition; no-op when target status isn't
- exposed by the workflow.
- - Falls through to `push_case` when `external_ref is None` (first push).
-- `services/connectors/app/connectors/servicenow.py` —
- - `push_case`: POSTs to `/api/now/table/incident`, sets
- `correlation_id="aisoc:{case_id}"` for round-tripping AiSOC case identity,
- severity → impact/urgency mapping, short_description truncation at 160 chars.
- - `push_status_change`: PATCHes `/api/now/table/incident/{sys_id}` with
- `state`; for `Resolved`/`Closed` adds `close_code` + `close_notes` (required
- by stock instances), otherwise omits them.
-- `services/connectors/app/api/router.py` —
- - `POST /connectors/{connector_id}/push_case`
- - `POST /connectors/{connector_id}/push_status_change`
-- `services/api/migrations/035_case_external_refs.sql` — `case_external_refs`
- table (one row per `(case_id, vendor)`) with `external_id`, `external_url`,
- `external_status`, `last_synced_at`, `sync_state`, plus indexes for
- inbound webhook lookup by `(vendor, external_id)`.
-- `services/api/app/services/case_fanout.py` — projection layer:
- - `fanout_create_case(case_id, tenant_id)` — finds enabled ITSM connectors,
- decrypts auth via `CredentialVault`, POSTs to connectors service, persists
- `case_external_refs` row, returns `FanoutResult`.
- - `fanout_status_change(case_id, tenant_id, old_status, new_status)` —
- looks up existing refs, calls `push_status_change`, updates row.
- - `_serialize_case_for_push(case_row)` — accepts dict or SQLAlchemy `Row`
- (`._mapping`); the test suite uses dicts and the production path uses Rows.
-- `services/api/app/api/v1/endpoints/cases.py` — `create_case` and `update_case`
- invoke fan-out after the response is committed (best-effort, errors logged
- not raised).
-- `services/api/app/api/v1/endpoints/inbox_itsm.py` — public-surface inbound
- webhook (`POST /v1/inbox/itsm/{token}`) with:
- - HMAC-SHA256 verification via `X-AiSOC-Signature` header.
- - Vendor-specific payload parsing (Jira `issue.key`/`fields.status.name`,
- ServiceNow `sys_id`/`state`).
- - `_map_inbound_status` collapsing vendor statuses → AiSOC statuses.
- - `case_external_refs` lookup by `(vendor, external_id)`.
- - Idempotent AiSOC case status updates (skips if already at target status).
-- `services/api/app/api/v1/endpoints/inbox.py` — `itsm-inbound` template added
- to `ALLOWED_TEMPLATE_IDS` and `_TEMPLATE_CATALOG`; `_build_inbox_url` now
- routes ITSM tokens through `OAUTH_PUBLIC_BASE_URL` to the API service.
-
-### Tests — done
-
-- `services/api/tests/test_inbox_itsm_endpoint.py` — **69/69 passing**.
- Covers HMAC verification, payload parsing for both vendors, status mapping,
- external-ref lookup, idempotency, missing-token / wrong-signature paths.
-- `services/api/tests/test_case_fanout.py` — **21/21 passing**. Covers
- payload serialization, vault decryption, connector RPC happy path, missing
- external_id, unsupported capability, status-change with no refs, status-change
- with orphaned refs, transport errors.
-- `services/connectors/tests/test_push_capabilities.py` — **23/23 passing**.
- Covers Jira `push_case` (payload shape, ADF, label, priority mapping, 255-char
- truncation, 4xx propagation), Jira `push_status_change` (no-ref fallthrough,
- unknown-status no-op, missing-transition no-op, transition discovery + POST,
- 4xx on transition POST), ServiceNow `push_case` (correlation_id round-trip,
- short_description truncation at 160, 4xx propagation), ServiceNow
- `push_status_change` (no-ref fallthrough, unknown-status no-op, in-progress
- state-only PATCH, Resolved/Closed close_code+close_notes, 4xx propagation),
- capability declarations on both classes.
-
-**Total WS8 test count: 113/113 passing.**
-
-### Test gotcha — `respx` is not in the connectors test environment
-
-`services/connectors/pyproject.toml` declares `respx` as a dev dep but the
-local venv doesn't have it installed (collection errors out for
-`test_azure_connectors.py`, `test_gcp_connectors.py`). For
-`test_push_capabilities.py` we use `unittest.mock.patch` against
-`httpx.AsyncClient` instead — same pattern as
-`services/api/tests/test_case_fanout.py`. Helper `_build_response` mocks
-`status_code`, `json()`, `text`, `request`, and `raise_for_status` so the
-connector code under test sees a faithful `httpx.Response` substitute.
-
-### Test gotcha — `_serialize_case_for_push` accepts dict or Row, not SimpleNamespace
-
-`SimpleNamespace` is *not* iterable as key-value pairs, so
-`dict(SimpleNamespace(...))` raises `TypeError`. The serializer goes through
-`getattr(case_row, "_mapping", None)` then falls back to `dict(case_row)`,
-which works for SQLAlchemy `Row` (production) and plain `dict` (tests). All
-test fixtures in `test_case_fanout.py` use `_make_case_row()` returning a dict.
-
-Reference plan §WS8.
-
----
-
-## WS9 — Docs + Sonali (shipped)
-
-### What landed
-
-- `apps/docs/docs/connectors/api-coverage.md` — capability matrix for all
- 42 connectors. Generated programmatically from `BaseConnector` subclasses
- by parsing `capabilities()` return tuples and `ConnectorSchema(...)`
- invocations directly out of source. Grouped by `category` (edr, siem,
- cloud, iam, saas, vcs, network) and includes columns for `OAuth (hosted)`
- and `Federated search`. Closes the "what works where" question Sonali
- flagged — no more reading source to know if a given connector can
- `PUSH_CASE` or only `PULL_ALERTS`.
-- `apps/docs/docs/architecture/itsm-as-source-of-truth.md` — explains why
- AiSOC is the canonical store for case state and ITSM systems are
- projections. Covers the outbound path (`case_fanout` → `push_case` /
- `push_status_change`), the inbound path (`POST /v1/inbox/itsm` with
- HMAC verification, `_JIRA_INBOUND_STATUS` / `_SNOW_INBOUND_STATUS`
- mapping, `case_external_refs` resolution), conflict resolution rules,
- and what is explicitly **not** synced (Jira priority, ServiceNow
- assignment_group, etc.). Includes a `mermaid` flowchart.
-- `apps/docs/docs/operations/sonali-consultation-questions.md` — structured
- question set for CISO / Head of SecOps / IR Lead pre-deployment
- conversations. Covers outcomes, threat model, data sources, case
- lifecycle, agent autonomy, compliance, identity/access, ops, reporting.
- Designed to surface boundary disagreements early rather than discovering
- them mid-rollout.
-- `apps/docs/sidebars.ts` — registered all three new pages. Added a new
- "Architecture" category (so `itsm-as-source-of-truth.md` has a home
- outside the Concepts bucket), and slotted the other two under their
- existing "Connectors" and "Operations" categories.
-- `docs/architecture/SYSTEM_DESIGN.md` — root-level architecture doc
- resynced with v2.1 reality. Topology diagram now shows `services/api`
- (inbox tokens + HMAC webhooks + CEF/HEC/DNS) and `services/connectors`
- (APScheduler poll + 42 connector classes + push_case/status) as
- parallel front doors feeding `services/ingest`. Service responsibilities
- table gained a `services/connectors` row and `services/api` was
- extended with "ITSM webhook inbox, case fan-out". A new §12 captures
- the v2.1 additions in narrative form with cross-links to the
- Docusaurus pages above.
-
-### Cross-links from §12
-
-- §12.1 → `apps/docs/docs/connectors/api-coverage.md`,
- `apps/docs/docs/operations/credentials.md`
-- §12.4 → `apps/docs/docs/architecture/itsm-as-source-of-truth.md`
-- §12.6 → `apps/docs/docs/plugins/python-sdk.md`,
- `apps/docs/docs/plugins/go-sdk.md`
-
-Reference plan §WS9.
-
----
-
-## Live todo snapshot (recreate via TodoWrite after restart)
-
-```json
-[
- {"id": "gh-1", "content": "Audit current git state, branches, identities in history, working tree", "status": "completed"},
- {"id": "gh-4", "content": "Secret scan working tree before any commit/push (gitleaks)", "status": "completed"},
- {"id": "gh-2", "content": "Pull live GitHub code-scanning alerts and triage by severity", "status": "completed"},
- {"id": "gh-6", "content": "Fix code-scanning issues with minimal diffs (critical/high first)", "status": "completed"},
- {"id": "ws7-tests", "content": "WS7: lake endpoint unit tests (POST /lake/sql, GET /lake/schema, helpers)", "status": "completed"},
- {"id": "ws8-tests-inbox", "content": "WS8: Inbox ITSM webhook tests (69 passing)", "status": "completed"},
- {"id": "ws8-tests-fanout", "content": "WS8: Tests for case_fanout service (21 passing)", "status": "completed"},
- {"id": "ws8-tests-push", "content": "WS8: Tests for jira/snow push_case/push_status_change methods (23 passing)", "status": "completed"},
- {"id": "tracker-update", "content": "Update AI_STACK_PLAN_PROGRESS.md to reflect WS8 completion", "status": "completed"},
- {"id": "ws9-api-coverage", "content": "WS9: write apps/docs/docs/connectors/api-coverage.md", "status": "completed"},
- {"id": "ws9-itsm-sot", "content": "WS9: write apps/docs/docs/architecture/itsm-as-source-of-truth.md", "status": "completed"},
- {"id": "ws9-sonali", "content": "WS9: write apps/docs/docs/operations/sonali-consultation-questions.md", "status": "completed"},
- {"id": "ws9-sidebar", "content": "WS9: register new docs in apps/docs/sidebars.ts", "status": "completed"},
- {"id": "gh-5", "content": "Sync docs/architecture (WS3-WS8) so GitHub matches current state", "status": "completed"},
- {"id": "tracker-update-ws9", "content": "Update AI_STACK_PLAN_PROGRESS.md to mark WS9 done", "status": "completed"},
- {"id": "gh-commit", "content": "Commit all WS9 + tracker docs with Beenu Arora <[email protected]> identity (no co-author trailers)", "status": "completed"},
- {"id": "gh-3a", "content": "Backup current main to refs/heads/backup/pre-rewrite-2026-05-08 on origin", "status": "completed"},
- {"id": "gh-3b", "content": "Install git-filter-repo and build authors/message callbacks (strip non-human co-author trailers, canonicalize all to [email protected])", "status": "completed"},
- {"id": "gh-3c", "content": "Run git-filter-repo on a fresh mirror; verify locally that all commits show Beenu Arora <[email protected]> and no Co-authored-by trailers remain", "status": "completed"},
- {"id": "gh-3d", "content": "Force-push rewritten main + tags + branches", "status": "completed"},
- {"id": "gh-3e", "content": "Verify GitHub contributors graph shows only beenu", "status": "completed"},
- {"id": "gh-prs", "content": "Close 19 dependabot PRs with explanation; comment on #37 asking K4R7IK to rebase", "status": "completed"},
- {"id": "gh-7", "content": "Verify code-scanning alerts cleared after push; record any remaining (3 false positives dismissed; 0 open high/medium/error)", "status": "completed"},
- {"id": "gh-ci-identity", "content": "Patch CI workflows so machine commits use Beenu Arora identity", "status": "completed"},
- {"id": "tryaisoc-cj", "content": "Customer journey review of tryaisoc.com — find and fix bugs", "status": "completed"}
-]
-```
-
----
-
-## Environment
-
-- Repo: `/Users/beenu/Desktop/AiSOC` (branch: `main`)
-- Python venv: `services/api/.venv` → `python3.14`
-- Activate: `cd services/api && source .venv/bin/activate`
-- Test: `python -m pytest tests/test_lake_sql.py tests/test_lake_rate_limit.py -q`
- (currently **69 passed**; smoke check before resuming)
-- MCP: `cd services/mcp && pnpm install && pnpm test`
-- Git identity (verified): `Beenu Arora <[email protected]>`
-
----
-
-## tryaisoc-cj findings (closeout)
-
-`tryaisoc-cj` is DONE. All nine plan workstreams (WS1-WS9), the
-GitHub hygiene tail (`gh-1` … `gh-7`, `gh-ci-identity`), and the
-customer-journey review of `https://tryaisoc.com` have shipped. The
-history rewrite landed cleanly, contributors graph shows only
-`beenu`, CodeQL is green (0 open high/medium/error alerts; 3 medium
-false positives dismissed with rationale), and the public site is
-verified clean on every canonical path the UI actually links to.
-
-### Bugs found and fixed
-
-P0 / P1 only. P2 polish was deferred — none was blocking a customer
-journey.
-
-| # | Severity | URL | Symptom | Fix commit | Verified |
-| -- | -------- | ------------------------------------------------ | --------------------------------------------------------------------------- | ---------- | ----------------------- |
-| 1 | P0 | `/cases/INC-001?tab=ledger` | "Ledger unavailable" — `GET /api/v1/investigations` 405 | `a8f08c4` | endpoint returns 200 |
-| 2 | P0 | `/api/v1/marketplace` | 503 "marketplace/index.json not found" (file outside Docker build context) | `8328870` | endpoint returns 200 |
-| 3 | P0 | API service boot | FastAPI `AssertionError` on 204 route (`oauth.py` DELETE) crashed startup | `8328870` | `/health` returns 200 |
-| 4 | P1 | Case workspace + Hunt view (demo mode) | UI banner + toasts said "backend offline" when only writes are disabled | `a8f08c4` | copy reads correctly |
-| 5 | P1 | `sitemap.xml` | `/signup` listed in sitemap but route is 404 (AiSOC ships no signup) | `a8f08c4` | sitemap clean |
-| 6 | P1 | `/onboarding` "Run a detection" card | Linked to `/detections` (plural) — 404. Correct path is `/detection` | `d3240a7` | card now links to 200 |
-| 7 | P1 | `/onboarding` "Bring your own data" card | Linked to in-app `/docs/operations/credentials` — 404 (docs are external) | `d3240a7` | links to GH Pages docs |
-| 8 | P1 | `NextStepCard` component | Used `next/link` for cross-origin docs URL; would break target=_blank | `d3240a7` | external `<a>` rendered |
-
-### Fix commits
-
-- `a8f08c4` — fix(web): customer-journey bugs — ledger routing, demo
- copy, /signup 404
-- `8328870` — fix(api): unblock startup + bake marketplace/index.json
- into api build
-- `d3240a7` — fix(web/onboarding): two 404s in NextStepCard footer
-
-All three authored as `Beenu Arora <[email protected]>` with no
-co-author trailers, deployed to `aisoc-demo-web` and `aisoc-demo-api`
-on Fly, and verified live on `tryaisoc.com`.
-
-### Post-deploy verification sweep
-
-Probed every canonical sitemap route and every `/api/v1/*` endpoint
-the UI actually calls. All return `200`.
-
-Frontend (15 paths from `apps/web/src/app/sitemap.ts`):
-`/`, `/benchmark`, `/connectors`, `/purple-team`, `/responder`,
-`/why-open-source`, `/marketplace`, `/compliance`, `/hunt`,
-`/copilot`, `/graph`, `/login`, `/detection`, `/threat-intel`,
-`/sla` — all 200.
-
-API endpoints the UI calls (7 paths, via `tryaisoc.com` rewrites):
-`/api/v1/cases`, `/api/v1/investigations`, `/api/v1/connectors`,
-`/api/v1/detection/rules`, `/api/v1/marketplace`,
-`/api/v1/marketplace/installed`, `/api/v1/contextual/actions` —
-all 200.
-
-Direct API health (`api.tryaisoc.com` / Fly app domain):
-`/health` — 200.
-
-### Non-bugs (404s that are correct)
-
-These paths return 404 because they are intentionally not part of the
-product surface. Recorded here so a future review does not re-open
-them:
-
-- `/signup`, `/pricing`, `/about` — AiSOC is open source and
- self-hosted; the only auth entry is `/login`, and the demo lands
- anonymously via the home-page CTA. The hero copy and the
- why-open-source page both state "No signup."
-- `/docs`, `/docs-portal` — documentation is hosted separately on
- GitHub Pages and `docs.tryaisoc.com` (cloudflared tunnel to the
- Docusaurus app). The Next.js app deliberately does not serve
- `/docs/*`. All in-app links to docs now use absolute URLs to the
- external host.
-- `/detections` (plural), `/runbooks`, `/agents`, `/admin/oauth`,
- `/investigations`, `/maintenance` — not listed in `sitemap.ts` and
- not linked from any UI component under `apps/web/src/`.
-- `/api/v1/incidents`, `/api/v1/detections`, `/api/v1/runbooks`,
- `/api/v1/agents/runs`, `/api/v1/contextual/health`,
- `/api/v1/contextual/orgs`, `/api/v1/oauth/apps` — not implemented
- on the API service and not called by the UI. The real names are
- `/api/v1/cases`, `/api/v1/detection/rules`, and the contextual
- endpoints listed above.
-
-### Outstanding non-blocker
-
-`flyctl deploy` for `aisoc-demo-web` returned a client-side timeout
-warning ("the app is not listening on the expected address") on the
-last roll-out, but the new image is live and all post-deploy probes
-pass. Recorded here so a future deploy can investigate the listener
-warning, but it is not gating the customer journey.
-
-Workspace rule: do not stop until every todo is done. → done.
-
----
-
-## Session — 2026-05-09 (buyer-value plan: A3, C1, H4)
-
-Three workstreams from `aisoc_v1.0_—_buyer-value_plan_c8116970` shipped
-or verified in this session.
-
-### WS-A3 — One-click deploy
-
-`render.yaml` was relocated from `infra/render/render.yaml` to the
-repository root so Render's "Deploy to Render" button can resolve it
-without a custom path. Updated `README.md` (new "Deploy in 60 seconds"
-section + TOC entry), `infra/render/README.md` (relative path), and
-`CHANGELOG.md`. Also updated the two `.gitleaksignore` entries that
-pinned `infra/render/render.yaml:generic-api-key:70` /`:124` to point
-at `render.yaml:generic-api-key:70` /`:124` (the line numbers track
-content positions inside `envVars`, not absolute file lines, so they
-remain valid post-move). One-click Docker Compose, Render, and Fly.io
-buttons are now linked from the README.
-
-### WS-C1 — 25 named playbooks
-
-Verified complete. `playbooks/packs/v1/` contains 50+ playbook JSON
-files covering ransomware, BEC, identity, cloud, lateral movement,
-data-exfil, privilege escalation, and more. Each playbook conforms
-to the schema enforced by `scripts/validate_playbooks.py` (CI gate
-in `.github/workflows/validate-playbooks.yml`). The runtime in
-`services/agents/app/playbook/store.py` loads `packs/v1/**` on
-startup and merges with any user-defined playbooks in
-`services/agents/data/playbooks/index.json` (user playbooks win).
-No new authoring needed for v1.0 — the v1.0 floor of 25 was passed
-and then some.
-
-### WS-H4 — Air-gapped / local-LLM mode
-
-Verified complete and shippable.
-
-- `apps/docs/docs/operations/airgap.md` is comprehensive (allowlist,
- Ollama / vLLM / LiteLLM topology, demo-mode behaviour, audit log
- expectations, "deploy-time only" mutation policy) and is reachable
- from the Docusaurus sidebar at `Operations → Air-gapped deployment`
- via `apps/docs/sidebars.ts`.
-- `services/api/app/api/v1/endpoints/llm_status.py` classifies
- providers as `openai | anthropic | azure-openai | local-ollama |
- local-vllm | local-litellm | custom` and exposes a redacted snapshot
- at `GET /api/v1/llm/status`. The Settings UI's "Deployment & AI"
- panel (`apps/web/src/components/settings/SettingsView.tsx`) reads
- both `/api/v1/airgap/status` and `/api/v1/llm/status` and renders
- read-only badges for "Air-gap engaged" + "AI calls route to: …".
- Mutations are deliberately deploy-time only.
-- `services/agents/app/api/explain.py:_llm_allowed()` honours
- `AISOC_AIRGAPPED`: with air-gap on and no `OPENAI_BASE_URL`, the
- outbound call is blocked; with a local proxy URL it is allowed.
-- `docker-compose.demo.yml` ships zero outbound LLM calls by
- default (`OPENAI_API_KEY: ${OPENAI_API_KEY:-}` — empty unless the
- operator opts in).
-
-### WS-H2 — BYOK per-tenant settings UI
-
-Shipped. v1.0 buyer-value criteria required per-tenant BYOK; the
-existing `CredentialVault` primitive already provided the
-encryption story, so we landed the model, endpoints, agents-side
-read path, and Settings UI in a single coherent change set.
-
-Backend (services/api):
-
-- `migrations/038_tenant_llm_credentials.sql` — new
- `tenant_llm_credentials` table keyed on `tenant_id`, with a
- `provider` `CHECK` constraint matching the four-tier ladder
- (`openai | anthropic | azure-openai | openai-compatible`),
- a vault-encrypted `api_key_vault` column, audit timestamps,
- and Row-Level Security policies bound to the
- `app.tenant_id` GUC.
-- `app/models/llm_credential.py` — `TenantLlmCredential` ORM
- model registered in `app/models/__init__.py` so Alembic and
- the dependency-injected `DBSession` see it.
-- `app/api/v1/endpoints/llm_credentials.py` — `GET / PUT /
- DELETE /api/v1/llm/credentials` gated on `settings:read` /
- `settings:write` RBAC. Writes encrypt the API key with
- `CredentialVault` (`vault:v1:` prefix), enforce
- provider-specific invariants (`base_url` mandatory for
- `openai-compatible`, `api_key` mandatory on first write for
- hosted providers), and emit `audit.llm_credential.{created,
- updated,deleted}` records via `emit_audit`. Reads return a
- `LlmCredentialView` projection that *never* includes the
- plaintext key — only `has_api_key: bool`.
-- `app/api/v1/endpoints/llm_status.py` — refactored to layer
- the tenant override over the env baseline and surface the
- resolved `source` (`tenant | environment | mixed | none`) so
- the Settings UI can explain provenance.
-- `app/api/v1/router.py` — registered the new credentials
- router under `/api/v1/llm/credentials`.
-
-Agents-side read path (services/agents):
-
-- `app/security/credential_vault.py` — vendored read-path copy
- of the API service's vault. Differs only in that
- `get_vault()` returns `None` (instead of raising) when
- `AISOC_CREDENTIAL_KEY` is missing, so explain stays
- resilient on operator boxes that have not migrated to BYOK
- yet.
-- `app/security/llm_resolver.py` — `resolve_llm_config` is
- now the single source of truth for "what config does this
- request actually use?". It layers tenant rows over env vars,
- applies the same air-gap rule as `_llm_allowed`, and returns
- a deterministic fallback so the explain path can still log
- `allowed=False` reasons even when no key is configured. The
- ledger import is lazy so unit tests that mount only the
- explain router don't drag in LangGraph.
-- `app/api/explain.py` — calls `resolve_llm_config(tenant_ref)`
- per request instead of reading env vars directly.
-
-Frontend (apps/web):
-
-- `src/components/settings/SettingsView.tsx` — new BYOK panel
- in the "Deployment & AI" section. Read-only by default;
- flips to write mode when the user has `settings:write`.
- Shows `provider`, `base_url`, `model`, `has_api_key`,
- `enabled`, `last_rotated_at`, and the resolved
- `source` from `/llm/status` so the buyer sees where each
- field came from.
-- `src/lib/api.ts` — `getLlmCredential`, `putLlmCredential`,
- `deleteLlmCredential` typed clients.
-
-Tests:
-
-- `services/api/tests/test_llm_credentials_endpoint.py` —
- 40 tests covering vault round-trip, the `_project` redactor,
- `LlmCredentialUpsert` validators, `_enforce_provider_invariants`,
- GET / PUT / DELETE happy paths, the rotation-only update
- case (`api_key=null` keeps the existing ciphertext), failure
- paths (provider transitions that violate invariants,
- duplicate-tenant `IntegrityError`), and audit emission.
-- `services/agents/tests/test_llm_resolver.py` — 33 tests
- covering `_env_baseline` (OPENAI_*, LLM_*, AISOC_LLM_MODEL
- precedence), `_airgap_blocks`, `_classify_source`,
- `_decrypt_vault_token` (vault disabled, round-trip, corrupt
- ciphertext), and `resolve_llm_config` end-to-end with a
- mocked `asyncpg` pool. Includes the BYOK-under-air-gap
- scenarios (private gateway allowed, OpenAI host blocked
- even with a valid tenant key) and the graceful-degradation
- paths (DB unreachable, ledger import unavailable, vault key
- missing, ciphertext decrypt fails).
-
-All 73 tests pass.
-
-Documentation:
-
-- `apps/docs/docs/operations/credentials.md` — added a "Per-tenant
- LLM credentials (BYOK)" section explaining the
- `tenant_llm_credentials` table, the vault round-trip, and how
- the agents service decrypts at read time.
-- `apps/docs/docs/operations/airgap.md` — clarified that BYOK
- to a private LiteLLM/Ollama/vLLM gateway works under
- `AISOC_AIRGAPPED=true`, while BYOK pointing at
- `api.openai.com` is still blocked.
-- `apps/docs/docs/operations/security.md` — added a pointer
- to the BYOK section so the security model document
- enumerates LLM keys alongside connector secrets.
-
-Workspace rule: do not stop until every todo is done.
-
----
-
-## v1.0 Buyer-Value Plan — COMPLETE (2026-05-10)
-
-Author: Beenu Arora <[email protected]>
-Released as: **v7.0.0** on `main` branch.
-
-All workstreams from `aisoc_v1.0_—_buyer-value_plan_c8116970.plan.md` are
-now shipped and verified. See `PROGRESS.md` for the full workstream table,
-fix log, and documentation changes.
-
-PR #43 (`feat/threatintel-attribution-v0-fixed`) was squash-merged into
-`main` on 2026-05-10 (205 files changed, +16 524 / -1 961 lines).
-
-CHANGELOG promoted to `[7.0.0] — 2026-05-10`.
-ROADMAP v7.0 marked Shipped ✅; v8.0 Planned section added.
-README version badge updated to `7.0.0`.
-
-The AI Stack & Data Integration plan workstreams (this file) remain at
-DONE status — no regressions introduced by the v7.0 buyer-value plan.
diff --git a/EVAL_BEFORE_AFTER.md b/EVAL_BEFORE_AFTER.md
deleted file mode 100644
index d3082dd82..000000000
--- a/EVAL_BEFORE_AFTER.md
+++ /dev/null
@@ -1,83 +0,0 @@
-# Eval harness re-grade — v8.0 wave-2 vs upstream/main
-
-Captured during stabilization of branch `v8.0/agentic-soc-foundation` (HEAD =
-`9a5e63ed` at capture time). Both runs use the deterministic 200-incident
-synthetic dataset (`services/agents/tests/eval_data/synthetic_incidents.json`)
-plus the aligned synthetic telemetry corpus, so any delta reflects substrate
-behaviour, not LLM stochasticity.
-
-| Run | Commit | Command |
-|-----|--------|---------|
-| BEFORE | `upstream/main` @ `182afc5b` | `python scripts/run_evals.py --json --out /tmp/eval_before.json` |
-| AFTER | `v8.0/agentic-soc-foundation` @ `9a5e63ed` | `python scripts/run_evals.py --json --out /tmp/eval_after.json` |
-
-## Four-axis substrate metrics
-
-| Axis | BEFORE | AFTER | Target | Δ |
-|------|--------|-------|--------|---|
-| `mitre_accuracy` (accuracy) | **0.97** | **0.97** | ≥ 0.80 | 0 |
-| `alert_reduction` (reduction_ratio) | **0.753** | **0.753** | ≥ 0.70 | 0 |
-| `investigation_completeness` (mean_keyword_coverage) | **0.9425** | **0.9425** | ≥ 0.85 | 0 |
-| `response_quality` (mean_rubric_score) | **1.000** | **1.000** | ≥ 0.80 | 0 |
-
-All four axes remain green on both branches. No regression.
-
-Three of those four are substrate self-consistency gates over the
-deterministic synthetic dataset — they are expected to stay flat across PRs
-that don't touch detection content or the eval scorer; only `mitre_accuracy`
-exercises the live agent. The wave-2 work is intentionally scaffold-level
-(connectors, new endpoints, schema edges, UI pages) and does not move any of
-the four axes, which is the expected outcome.
-
-## Extended suites (also tracked)
-
-| Suite | BEFORE | AFTER | Target | Passed |
-|-------|--------|-------|--------|--------|
-| `hunt_corpus` (positive_scenario_catch_rate) | 1.000 | 1.000 | 1.000 | ✅ |
-| `adversary_eval` (graceful_degradation_catch_rate) | 0.475 | 0.475 | ≥ 0.40 | ✅ |
-| `confidence_calibration` (investigation_brier_score) | 0.0605 | 0.0605 | ≤ 0.18 | ✅ |
-| `memory_recall` (memory_recall_accuracy) | 1.000 | 1.000 | 1.000 | ✅ |
-| `override_accuracy` (override_accuracy) | 1.000 | 1.000 | 1.000 | ✅ |
-| `playbook_completion_rate` (completion_rate) | 0.735 | 0.735 | ≥ 0.50 | ✅ |
-| `detection_fp_rate` (worst_per_rule_fp_rate) | 0.0049 | 0.0049 | ≤ 0.05 | ✅ |
-
-`all_passed: true` on both runs.
-
-## Per-investigation telemetry
-
-The wave-2 `T2.3` change wires per-investigation token / USD / latency stats
-into `run_evals.py`'s `per_investigation` block. On `upstream/main` the
-`per_investigation` section is empty (`[]`); on `v8.0/agentic-soc-foundation`
-it carries the rate-card-priced aggregate below. This is a pure additive
-diff — the substrate metrics are unchanged.
-
-| Stat | BEFORE | AFTER |
-|------|--------|-------|
-| `tokens_per_investigation.mean` | _n/a_ | 2186.13 |
-| `tokens_per_investigation.median` | _n/a_ | 2114.00 |
-| `tokens_per_investigation.p95` | _n/a_ | 2452 |
-| `usd_per_investigation.mean` | _n/a_ | $0.014693 |
-| `usd_per_investigation.median` | _n/a_ | $0.013685 |
-| `usd_per_investigation.p95` | _n/a_ | $0.016930 |
-| `latency_per_investigation_ms.mean` | _n/a_ | 0.0078 |
-| `latency_per_investigation_ms.p95` | _n/a_ | 0.0119 |
-
-> **Reading note.** The latency figure is the scorer's per-investigation
-> compute envelope on the deterministic dataset, not wall-clock LLM latency.
-> Token + USD figures come from the rate-card rather than a live LLM run —
-> see `--no-telemetry-records` and `apps/docs/docs/benchmark.md` for the
-> transparency contract.
-
-## Runtime
-
-Both runs complete in ≈ 1.5 s (deterministic in-process scorer, no live LLM
-call). Wall-clock comparison is dominated by Python startup, so we don't
-publish a delta.
-
-## Take-aways for PR review
-
-1. No regression on any axis. `all_passed` is `True` for both BEFORE and AFTER.
-2. Wave-2 adds per-investigation telemetry to the eval report. Existing
- four-axis gates remain unchanged on the deterministic dataset, as designed.
-3. Wet-eval (`scripts/wet_eval_check.py`) is the separate live-LLM track and
- is not part of this BEFORE/AFTER comparison.
diff --git a/PROGRESS-v1.5.md b/PROGRESS-v1.5.md
deleted file mode 100644
index f6ed2e1ba..000000000
--- a/PROGRESS-v1.5.md
+++ /dev/null
@@ -1,35 +0,0 @@
-# v1.5 SOC Console Parity — Execution Progress
-
-Plan: `v1.5 SOC Console Parity (local plan doc, not tracked in repo)`
-Started: 2026-05-13
-Author: Beenu Arora <[email protected]>
-Branch strategy: one branch per PR (`v1.5/pr-N-<slug>`), all targeting `main`.
-
-## Status
-
-| PR | Workstream | Branch | State |
-|---|---|---|---|
-| PR-1 | W4 + W5 — TimeWindowContext + tenant/role TopBar | `v1.5/pr-1-topbar-context` | pending |
-| PR-2 | W2 + W3 — `critical` severity + alert.confidence + narrative | `v1.5/pr-2-severity-confidence` | pending |
-| PR-3 | W1 + W9 — Funnel + Efficiency + Pipeline Health | `v1.5/pr-3-funnel-efficiency` | pending |
-| PR-4 | W6 — Structured InvestigationRail | `v1.5/pr-4-investigation-rail` | pending |
-| PR-5 | W7 — Investigation Queue workbench | `v1.5/pr-5-queue-workbench` | pending |
-| PR-6 | W8 — Rule Tuning workbench | `v1.5/pr-6-rule-tuning` | pending |
-| PR-7 | W10 — Exposure Ticket lifecycle | `v1.5/pr-7-exposure-tickets` | pending |
-| PR-8 | W11 + W12 — `/console` + Correlation + shortcuts + wallboard | `v1.5/pr-8-console-workbench` | pending |
-| meta | Feature flags + demo seed | rolled into PRs above | pending |
-| meta | Docs + CHANGELOG + AGENTS + v1.5.0 bump | rolled into PRs above | pending |
-
-## Quality gates per PR
-
-- `pnpm -w typecheck` (or per-package) green for touched packages.
-- `pnpm -w lint` green for touched packages.
-- `pytest services/<svc>` green for touched services.
-- New endpoints have unit tests; new components have a snapshot/render test.
-- PRs that touch fusion/severity (PR-2) re-grade eval harness and include delta in body.
-
-## Conventions
-
-- Feature flags: `AISOC_FEATURE_CONSOLE`, `AISOC_FEATURE_EXPOSURE_TICKETS`, `AISOC_FEATURE_CRITICAL_SEVERITY` (all default OFF in prod, ON in demo seed).
-- Backwards compat: every flat page stays — `/console` is additive.
-- All commits authored as `Beenu Arora <[email protected]>` via `GIT_AUTHOR_*` / `GIT_COMMITTER_*` env vars (no global git config mutation).