Understand any codebase instantly. System intelligence for codebases, built for humans and AI.
TypeScript aiai-memoryclaude-codeclicode-analysiscode-mappingcode-memorycodebase-understandingcodexdeveloper-toolsllmopenclawpersistent-memoryprogram-analysissoftware-architecturesystem-intelligencesystem-mapping
ci(e2e): run the correctness harness on every PR against a pinned backend
KageBinarye3078be6da83
diff --git a/.github/dependabot.yml b/.github/dependabot.ymlindex 65000dae..ad9753c9 100644--- a/.github/dependabot.yml+++ b/.github/dependabot.yml@@ -8,6 +8,14 @@ updates:actions:patterns: ["*"]+ # The backend image the real-backend harness runs against, pinned by digest+ # in ix-cli/test/e2e/compose.e2e.yml. A bump PR runs the scenarios against the+ # new release before any CLI PR sees it.+ - package-ecosystem: docker-compose+ directory: /ix-cli/test/e2e+ schedule:+ interval: weekly+- package-ecosystem: npmdirectory: /ix-clischedule:diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex af95550d..2729e71a 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -295,9 +295,36 @@ jobs:echo "::endgroup::"node -e "const d=require('./doctor-unmapped.json');const c=(d.checks||[]).find(c=>c.name==='Workspace for this directory');if(!c){console.error('no workspace check in doctor output');process.exit(1)}if(c.ok||d.healthy){console.error('doctor reported healthy in a directory with no workspace');process.exit(1)}console.log('unmapped cwd reported:',c.detail)"+ # The correctness harness (ix-cli/test/e2e): edit sequences against a+ # backend pinned by digest in compose.e2e.yml, not the :latest the smoke+ # step uses, so a backend release cannot turn an unrelated CLI PR red.+ # Its own ports (8290/8729), next to the smoke stack. Scenarios that fail+ # until a fix lands are marked failsUntil([...]) and pass while they fail.+ - name: Start harness backend (pinned)+ working-directory: ix-cli+ run: npm run e2e:up++ - name: Correctness scenarios (incremental equals fresh)+ working-directory: ix-cli+ env:+ IX_E2E: "1"+ IX_E2E_DIFF_DIR: ${{ runner.temp }}/e2e-diffs+ # dist/ is already built above; test:e2e would rebuild it.+ run: npx vitest run --config vitest.e2e.config.ts++ - name: Upload graph diffs on failure+ if: failure()+ uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1+ with:+ name: e2e-graph-diffs+ path: ${{ runner.temp }}/e2e-diffs+ if-no-files-found: ignore+- name: Backend logs on failureif: failure()- run: docker compose -f docker-compose.standalone.yml logs --tail 200+ run: |+ docker compose -f docker-compose.standalone.yml logs --tail 200+ (cd ix-cli && node test/e2e/stack.mjs logs)# ── API-reference parity gate ────────────────────────────────────────────# docs/api/openapi.yaml claims to be generated from the client source; thisdiff --git a/ix-cli/package.json b/ix-cli/package.jsonindex 40a57654..734bd051 100644--- a/ix-cli/package.json+++ b/ix-cli/package.json@@ -21,8 +21,8 @@"test:unit": "vitest run","test:coverage": "vitest run --coverage","test:e2e": "npm run build && vitest run --config vitest.e2e.config.ts",- "e2e:up": "docker compose -f test/e2e/compose.e2e.yml up -d --wait --wait-timeout 240",- "e2e:down": "docker compose -f test/e2e/compose.e2e.yml down -v",+ "e2e:up": "node test/e2e/stack.mjs up",+ "e2e:down": "node test/e2e/stack.mjs down","smoke": "node test/parser.smoke.mjs","typecheck": "tsc --noEmit","lint": "eslint src",diff --git a/ix-cli/test/e2e/README.md b/ix-cli/test/e2e/README.mdindex 23820d2a..02ab2231 100644--- a/ix-cli/test/e2e/README.md+++ b/ix-cli/test/e2e/README.md@@ -36,7 +36,7 @@ endpoint that is not loopback or that uses a port of a stack with real data:| `IX_E2E_PROJECT` | `ix-e2e` | compose project name || `IX_E2E_BACKEND_PORT` | `8290` | compose and harness: host port of the memory-layer || `IX_E2E_ARANGO_PORT` | `8729` | compose and harness: host port of ArangoDB |-| `IX_E2E_BACKEND_IMAGE` | `ghcr.io/ix-infrastructure/ix-memory-layer:1.0.31@sha256:7f972fb0…` | compose: the backend under test |+| `IX_E2E_BACKEND_IMAGE` | the pin in `compose.e2e.yml` (1.0.31 by digest) | `stack.mjs`: run this image instead || `IX_E2E_ENDPOINT` | `http://127.0.0.1:$IX_E2E_BACKEND_PORT` | harness: backend URL, overrides the port || `IX_E2E_ARANGO_URL` | `http://127.0.0.1:$IX_E2E_ARANGO_PORT` | harness and `graph-sig.mjs`: ArangoDB URL || `IX_E2E_DIFF_DIR` | unset | harness: write every graph difference to `<dir>/<scenario>.txt` |@@ -44,6 +44,11 @@ endpoint that is not loopback or that uses a port of a stack with real data:Set the same `IX_E2E_*` values for `e2e:up`, `test:e2e` and `e2e:down`.+`e2e:up` and `e2e:down` run `stack.mjs`, which calls `docker compose` on+`compose.e2e.yml` and adds `compose.image-override.yml` when+`IX_E2E_BACKEND_IMAGE` is set. The pin itself is a literal line so Dependabot+can propose bumps (`.github/dependabot.yml`, ecosystem `docker-compose`).+### Several stacks at onceEach stack needs its own project name and ports. One stack is capped at about@@ -108,6 +113,14 @@ crashed `ix map` or an unreachable database, fails the run either way.To see what a marked scenario still gets wrong, set `IX_E2E_DIFF_DIR`.+## In CI++Job `e2e` in `.github/workflows/ci.yml` runs the scenarios on every PR after+the smoke test, against the pinned image on the default ports, and is part of+`CI Passed`. On failure it uploads the graph diffs (`e2e-graph-diffs`) and both+stacks' logs. A backend fix reaches this job only once its image is released+and the pin is moved to it, normally by the Dependabot PR.+## Fixtures`fixtures/polyglot/` has 23 files: Python with cross-file calls (`app/`),diff --git a/ix-cli/test/e2e/compose.e2e.yml b/ix-cli/test/e2e/compose.e2e.ymlindex 15268d38..08e6a7dd 100644--- a/ix-cli/test/e2e/compose.e2e.yml+++ b/ix-cli/test/e2e/compose.e2e.yml@@ -3,11 +3,12 @@# Throwaway backend for the real-backend harness (test/e2e). Every scenario# resets this database, so it gets its own compose project and ports and must# never be pointed at a stack anything else uses. See test/e2e/README.md.+# Start and stop it with `npm run e2e:up` / `npm run e2e:down` (stack.mjs).## IX_E2E_PROJECT compose project name (default ix-e2e)# IX_E2E_BACKEND_PORT host port of memory-layer (default 8290)# IX_E2E_ARANGO_PORT host port of ArangoDB (default 8729)-# IX_E2E_BACKEND_IMAGE memory-layer image (default: v1.0.31, pinned by digest)+# IX_E2E_BACKEND_IMAGE memory-layer image (default: the pin below; see stack.mjs)## Memory is capped: ArangoDB's caches otherwise size themselves from host RAM# (a 9 GB block cache on a 30 GB host). The whole stack stays under ~2.7 GB.@@ -47,7 +48,9 @@ services:retries: 15memory-layer:- image: ${IX_E2E_BACKEND_IMAGE:-ghcr.io/ix-infrastructure/ix-memory-layer:1.0.31@sha256:7f972fb084d9667ef1b7cd7e7c40b24aad76fb4cb38d1cc07a0319444c0ab979}+ # Literal, so Dependabot can propose bumps. Set IX_E2E_BACKEND_IMAGE to run+ # another image (stack.mjs then adds compose.image-override.yml).+ image: ghcr.io/ix-infrastructure/ix-memory-layer:1.0.31@sha256:7f972fb084d9667ef1b7cd7e7c40b24aad76fb4cb38d1cc07a0319444c0ab979networks: [backend]ports:- "127.0.0.1:${IX_E2E_BACKEND_PORT:-8290}:8090"diff --git a/ix-cli/test/e2e/compose.image-override.yml b/ix-cli/test/e2e/compose.image-override.ymlnew file mode 100644index 00000000..68fbf5d2--- /dev/null+++ b/ix-cli/test/e2e/compose.image-override.yml@@ -0,0 +1,9 @@+# Copyright 2026 Ix Infrastructure Inc.+#+# Added by stack.mjs only when IX_E2E_BACKEND_IMAGE is set: runs that image+# instead of the pinned one in compose.e2e.yml (for example a backend built+# from an Ix-memory branch).++services:+ memory-layer:+ image: ${IX_E2E_BACKEND_IMAGE:?set IX_E2E_BACKEND_IMAGE}diff --git a/ix-cli/test/e2e/stack.mjs b/ix-cli/test/e2e/stack.mjsnew file mode 100644index 00000000..03122d69--- /dev/null+++ b/ix-cli/test/e2e/stack.mjs@@ -0,0 +1,30 @@+// Copyright 2026 Ix Infrastructure Inc.++// Starts or stops the harness stack: `node test/e2e/stack.mjs up|down|logs`.+//+// compose.e2e.yml pins the backend image as a literal line so Dependabot can+// propose bumps. To test another image, set IX_E2E_BACKEND_IMAGE: this script+// then adds compose.image-override.yml, which swaps it in.++/* global process */++import { spawnSync } from "node:child_process";+import { dirname, join } from "node:path";+import { fileURLToPath } from "node:url";++const here = dirname(fileURLToPath(import.meta.url));+const files = ["-f", join(here, "compose.e2e.yml")];+if (process.env.IX_E2E_BACKEND_IMAGE) files.push("-f", join(here, "compose.image-override.yml"));++const actions = {+ up: ["up", "-d", "--wait", "--wait-timeout", "240"],+ down: ["down", "-v"],+ logs: ["logs", "--tail", "200"],+};+const action = actions[process.argv[2]];+if (!action) {+ process.stderr.write("usage: node test/e2e/stack.mjs up|down|logs\n");+ process.exit(2);+}+const res = spawnSync("docker", ["compose", ...files, ...action], { stdio: "inherit" });+process.exit(res.status ?? 1);