Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
Seed the test memo from trusted CI runs and document it
[._.]/ Adam Eivyb46cccfcab65
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex ce3e12fbb3..2fde6b13fc 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -354,9 +354,25 @@ jobs:if: needs.impact.outputs.slashdo == 'true'run: git submodule update --init lib/slashdo+ # The test memo (docs/GITHUB_ACTIONS.md "Test memo"): the runner skips a+ # selected test file whose every input — loaded modules, fixtures,+ # package versions, Node/platform, gating env — is byte-identical to one+ # a trusted green run verified. Pull requests only read. The scheduled,+ # dispatched and called full runs save, so the cache holds nothing a PR+ # branch wrote. Each job and shard keeps its own lineage under the key+ # prefix; a restore takes the newest.+ - name: Restore the test memo+ uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0+ with:+ path: ${{ runner.temp }}/portos-test-memo+ key: test-memo-${{ runner.os }}-${{ runner.arch }}-server-${{ matrix.shard }}of${{ strategy.job-total }}-${{ github.run_id }}+ restore-keys: |+ test-memo-${{ runner.os }}-${{ runner.arch }}-server-${{ matrix.shard }}of${{ strategy.job-total }}-+- name: Run server testsif: needs.impact.outputs.server_mode != 'skip'env:+ PORTOS_TEST_MEMO_DIR: ${{ runner.temp }}/portos-test-memoPORTOS_CI_BROWSER_OWNER: unitCI_TEST_MODE: ${{ needs.impact.outputs.server_mode }}CI_TEST_FILES: ${{ needs.impact.outputs.server_files }}@@ -371,6 +387,15 @@ jobs:PGPASSWORD: portosrun: node scripts/run-ci-tests.js server+ # Only trusted full runs on the default/release branches seed the memo;+ # a red shard still saves the files that passed (entries are per file).+ - name: Save the test memo+ if: github.event_name != 'pull_request' && !cancelled()+ uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0+ with:+ path: ${{ runner.temp }}/portos-test-memo+ key: test-memo-${{ runner.os }}-${{ runner.arch }}-server-${{ matrix.shard }}of${{ strategy.job-total }}-${{ github.run_id }}+# Smoke-boot does not need Postgres: NODE_ENV=test selects the file# backend. It does need the native rebuild (server boot loads node-pty),# which is why it lives on this job rather than a third installer.@@ -456,15 +481,33 @@ jobs:if: needs.impact.outputs.client_mode != 'skip'run: bash scripts/ci-warm-test-chrome.sh+ # Same contract as the server job's memo steps.+ - name: Restore the test memo+ if: needs.impact.outputs.client_mode != 'skip'+ uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0+ with:+ path: ${{ runner.temp }}/portos-test-memo+ key: test-memo-${{ runner.os }}-${{ runner.arch }}-client-${{ matrix.shard }}of${{ strategy.job-total }}-${{ github.run_id }}+ restore-keys: |+ test-memo-${{ runner.os }}-${{ runner.arch }}-client-${{ matrix.shard }}of${{ strategy.job-total }}-+- name: Run client testsif: needs.impact.outputs.client_mode != 'skip'env:+ PORTOS_TEST_MEMO_DIR: ${{ runner.temp }}/portos-test-memoCI_TEST_MODE: ${{ needs.impact.outputs.client_mode }}CI_TEST_FILES: ${{ needs.impact.outputs.client_files }}CI_TEST_SOURCES: ${{ needs.impact.outputs.client_sources }}CI_SHARD: ${{ matrix.shard }}/${{ strategy.job-total }}run: node scripts/run-ci-tests.js client+ - name: Save the test memo+ if: needs.impact.outputs.client_mode != 'skip' && github.event_name != 'pull_request' && !cancelled()+ uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0+ with:+ path: ${{ runner.temp }}/portos-test-memo+ key: test-memo-${{ runner.os }}-${{ runner.arch }}-client-${{ matrix.shard }}of${{ strategy.job-total }}-${{ github.run_id }}+- name: Build clientif: needs.impact.outputs.build == 'true' && matrix.shard == 1run: npm run build --prefix client@@ -846,6 +889,16 @@ jobs:if: steps.client-modules.outputs.cache-hit != 'true'run: npm ci --prefix client+ # Same contract as the server job's memo steps. The browser profile is+ # part of every key, so a smoke verdict never stands in for a full one.+ - name: Restore the test memo+ uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0+ with:+ path: ${{ runner.temp }}/portos-test-memo+ key: test-memo-${{ runner.os }}-${{ runner.arch }}-browser-${{ github.run_id }}+ restore-keys: |+ test-memo-${{ runner.os }}-${{ runner.arch }}-browser-+# Fail-closed: PORTOS_REQUIRE_BROWSER_SUITES makes the suites' canRun guard# throw (see server/lib/browserSuiteGate.js) when Chrome, ffmpeg or the# client bundler is missing, so a skip can never pass this step.@@ -854,6 +907,7 @@ jobs:# this 15m step limit is only a fallback before the 20m job boundary.timeout-minutes: 15env:+ PORTOS_TEST_MEMO_DIR: ${{ runner.temp }}/portos-test-memoPORTOS_BROWSER_TEST_PROFILE: ${{ github.event_name == 'pull_request' && github.base_ref != 'release' && inputs.full != true && 'smoke' || 'full' }}PORTOS_CI_DIAGNOSTICS_DIR: ${{ runner.temp }}/portos-ci-diagnosticsCI_TEST_MODE: files@@ -862,6 +916,13 @@ jobs:PORTOS_TEST_QUIET: 1run: node scripts/watch-ci-tests.js server+ - name: Save the test memo+ if: github.event_name != 'pull_request' && !cancelled()+ uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0+ with:+ path: ${{ runner.temp }}/portos-test-memo+ key: test-memo-${{ runner.os }}-${{ runner.arch }}-browser-${{ github.run_id }}+- name: Preserve browser test failure diagnosticsif: failure()continue-on-error: true@@ -987,12 +1048,23 @@ jobs:- name: Check server entry-point syntaxrun: node --check server/index.js+ # Same contract as the server job's memo steps; the platform is part of+ # every key, so Windows reads only Windows verdicts.+ - name: Restore the test memo+ uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0+ with:+ path: ${{ runner.temp }}/portos-test-memo+ key: test-memo-${{ runner.os }}-${{ runner.arch }}-windows-server-${{ matrix.shard }}of${{ strategy.job-total }}-${{ github.run_id }}+ restore-keys: |+ test-memo-${{ runner.os }}-${{ runner.arch }}-windows-server-${{ matrix.shard }}of${{ strategy.job-total }}-+- name: Run server tests on Windows# A step timeout FAILS the step (log kept, `if: failure()` reporting# runs); the job-level timeout would cancel the job and lose the log# (#10415). Observed healthy runs take 2-10 minutes.timeout-minutes: 15env:+ PORTOS_TEST_MEMO_DIR: ${{ runner.temp }}/portos-test-memo# Outside the launcher's disposable temp root; recoverable after exit.PORTOS_CI_DIAGNOSTICS_DIR: ${{ runner.temp }}/portos-ci-diagnosticsCI_TEST_MODE: ${{ needs.impact.outputs.windows_mode }}@@ -1006,6 +1078,13 @@ jobs:PORTOS_CI_MAX_WORKERS: 2run: node scripts/run-ci-tests.js server+ - name: Save the test memo+ if: github.event_name != 'pull_request' && !cancelled()+ uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0+ with:+ path: ${{ runner.temp }}/portos-test-memo+ key: test-memo-${{ runner.os }}-${{ runner.arch }}-windows-server-${{ matrix.shard }}of${{ strategy.job-total }}-${{ github.run_id }}+- name: Preserve Windows test failure diagnosticsif: failure()continue-on-error: truediff --git a/AGENTS.md b/AGENTS.mdindex 49e96fda15..8ceb143815 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -56,6 +56,8 @@ npm test --prefix client # Vitest (happy-dom) — component/unit testsnpm run test:db # DB-backed suites → portos_test ONLY (see Security Model)npm run pregate # BEFORE EVERY PUSH — CI's plan run locally; receipt in .cache/pregate/report.json+# pregate and CI skip test files whose every input already passed (the test+# memo, docs/GITHUB_ACTIONS.md "Test memo"); PORTOS_TEST_MEMO=off runs them all.```**Run `npm run pregate` before every push and again after rebasing onto a moved base.** It uses CI's planner and runners, including tree-wide guards for import boundaries, catalog merges, and generated manifests. The local plan includes both committed and uncommitted changes, including untracked files.diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.mdindex 0d94a808b4..c86c91831a 100644--- a/docs/CONTRIBUTING.md+++ b/docs/CONTRIBUTING.md@@ -253,8 +253,18 @@ For server watch mode, run this alternative from the repository root:npm run test:watch --prefix server```+`npm run pregate` (and CI, through the same runner) also consults the test+memo: a selected test file whose every input — the modules it loads, the+fixtures and package versions it reaches, the Node major and platform, and+the gating environment variables — is byte-identical to a run that already+passed is skipped, and new green results are recorded under+`~/.portos/test-memo/`. Tree-wide scanners and anything that reads repository+state are never memoized. `PORTOS_TEST_MEMO=off` disables it, and+`PORTOS_TEST_MEMO_VERBOSE=1` lists the skipped files; see+[GITHUB_ACTIONS.md](./GITHUB_ACTIONS.md#test-memo).+Pull requests into `main` run the tests for affected feature directories, with-conservative fallbacks to Vitest related-test mode or the complete suite. The+conservative fallbacks to the runner's related-test mode or the complete suite. Thepull request into `release`, nightly CI, and manual CI runs always run thecomplete server, client, DB, lint, build, and smoke checks. Pushes to `main`run nothing — the pull request gate already covered that tree. Releasediff --git a/docs/GITHUB_ACTIONS.md b/docs/GITHUB_ACTIONS.mdindex b1b3114473..79e4697583 100644--- a/docs/GITHUB_ACTIONS.md+++ b/docs/GITHUB_ACTIONS.md@@ -65,8 +65,8 @@ A release therefore pays for one full run (on its PR), not three.PRs into `main` use `scripts/ci-test-plan.js` to classify the changed filesbefore installing dependencies. Directory-scoped features add their server and-client feature tests, and every behavioral source (nested or flat) uses-Vitest's import-graph-aware `related` mode, fed the changed behavioral source+client feature tests, and every behavioral source (nested or flat) uses the+runner's import-graph-aware `related` mode, fed the changed behavioral sourcepaths plus the planner's explicit test files. The planner deliberately chooses full CI for sharedcomposition roots, test configuration, dependency manifests, workflow changes,unknown artifacts, or wide diffs.@@ -452,8 +452,9 @@ pull request is the merge ref plus both of its parents — and thefirst parent *is* the base-branch commit the pull request is diffed against.`scripts/ci-base-sha.js` reads it (`HEAD^1`) and exports `CI_BASE_SHA` for therest of the job, so the planner's `git diff <base>...HEAD` resolves without-deeper history. The planner passes the resulting source paths directly to-`vitest related`; Vitest no longer performs its own Git diff.+deeper history. The planner passes the resulting source paths to the runner,+which resolves them through its own module graph; Vitest no longer performs+its own Git diff.Reading the base off the checkout rather than `github.event.pull_request.base.sha`is also more correct: GitHub rebuilds the merge ref when the base branch moves,@@ -771,14 +772,89 @@ failing closed to the full suite for a script nothing names. A `.py` outside`scripts/` is still unclassified.Targeted `files` plans run the planner's exact test files once. `related` plans-run `vitest related` once with changed behavioral source paths and the cheap-structural/repository contract files as inputs. Vitest treats a test-file input-as directly selected, so contracts and changed tests share the import-graph run-without being repeated in a second process.+resolve the changed behavioral source paths through the runner's own module+graph (`scripts/lib/testMemo.js`: static, dynamic and `vi.mock` edges, the+same edges `vitest related` follows): a test is selected when a changed+source is in its closure or when one of its edges cannot be resolved, plus+the cheap structural/repository contract files, in one Vitest invocation.+The exact list reaches Vitest through a selection file+(`PORTOS_CI_TEST_SELECTION`, read by both workspace configs) rather than+command-line selectors, because a related set can exceed the Windows+command-line limit. Computing the set in the runner is what lets the test+memo below subtract the files a trusted green run already verified.There is no buffered discovery pass: the old `vitest list --changed` path couldspend minutes printing every test name, overflow Node's buffer, discard theresult, and rerun the same graph.+### Test memo++`scripts/run-ci-tests.js` skips a selected test file whose every input is+byte-identical to one a trusted green run already verified, locally under+`npm run pregate` and in CI through the same code path. Each selected file+gets two content-addressed keys (`scripts/lib/testMemo.js`):++- the **floor** key hashes the working-tree bytes of the file's static ++ dynamic module closure (`import`/`export … from`, `require()`, `import()`,+ `vi.mock`/`vi.doMock`/`vi.importActual`/`vi.importMock`/`vi.unmock`, all+ with literal specifiers);+- the **recorded** key hashes the modules the file actually loaded in the+ green run that produced its entry (Vitest's+ `experimental.recordDependencies` depfile). A module the test mocks away or+ never reaches cannot change its outcome, so a change to the lazily+ imported server cluster invalidates only the suites that loaded it; a new+ edge still invalidates because the importer's own bytes changed.++Both keys also cover: every non-JS file a member names through a literal path+anchored at `import.meta.url`, `__dirname`, `process.cwd()` or an identifier+bound to one of those (a literal directory contributes its whole tree, up to+400 files; snapshot files and the planner's `FIXTURE_OWNERS` directories are+added for the test itself); the installed version of every package the+members reach (so a lockfile bump invalidates exactly the suites that reach+the bumped packages); the workspace Vitest config closure, setup files,+`scripts/vitestCiPool.js`, `package.json` and lockfile; the Node major,+`process.platform`, `process.arch`; and `TEST_MEMO_ENV` (`CI`, `NODE_ENV`,+`PORTOS_BROWSER_TEST_PROFILE`, `PORTOS_CI_BROWSER_OWNER`,+`PORTOS_REQUIRE_BROWSER_SUITES`, `PORTOS_REQUIRE_LINUX_CONTAINMENT`,+`PORTOS_TEST_QUIET`, `CI_EXPECT_SLASHDO_SUBMODULE`, …).++**Never memoized (fails closed).** A file in the planner's `ALWAYS_RUN_TESTS`+or in `TEST_MEMO_NEVER` (the tree-wide convention and catalog scanners); a+file whose closure contains the `// test-memo: never` marker; a test file or+`scripts/` module that spawns `git` (repository state is not in the key —+product modules that run git on their own targets are under the test's+control); a member that reads a path the analyzer cannot resolve+(`import.meta.glob`, anything under `.git/`, a literal directory with more+than 400 files); a bare specifier that resolves to no installed package; and,+for the floor key only, a non-literal `import()`/`require()` or an+unparseable member. A recorded set that reaches outside the repository is+not memoizable either. Every unknown construct lands in one of those buckets+rather than silently narrowing the key.++**Store.** `~/.portos/test-memo/` (override with `PORTOS_TEST_MEMO_DIR`):+`entries/<key[0..2]>/<key>.json` holds `{ test, workspace, verifiedAt,+source: 'local'|'ci', commit }`, `records/<workspace>/<sha256(test)>.json`+the loaded-module list. Writes are temp-file-plus-rename so concurrent+worktree pregates never tear a file; any read error is a miss. Entries are+written only for files Vitest's JSON report shows passed with no failed and+no skipped case (a browser profile's name pattern is the one accepted skip+reason). The runner prints one line —+`🧠 Test memo: skipped N of M selected files (closure unchanged); running K.`+— and lists the skipped files under `PORTOS_TEST_MEMO_VERBOSE=1`. When every+selected file is memoized nothing is launched and the job exits 0.++**CI seeding.** The server, client, browser and Windows jobs restore+`${{ runner.temp }}/portos-test-memo` with `actions/cache/restore` under a+per-job, per-shard key prefix (`test-memo-<os>-<arch>-<job>-<shard>of<n>-`).+Pull requests only read. The scheduled nightly, `workflow_dispatch` and+`workflow_call` runs on the default/release branches save a new entry keyed+by run id, so the cache holds nothing a pull-request branch wrote and every+PR restores the newest trusted lineage. The DB job does not go through+`run-ci-tests.js` and is not memoized.++**Disable.** `PORTOS_TEST_MEMO=off` turns reads and writes off together and+restores the pre-memo behavior. A test spawned under Vitest (`VITEST` set)+never touches the developer's store unless it sets `PORTOS_TEST_MEMO_DIR`.+No third-party change-filter action is used. The planner passes test paths as aJSON argument array to `spawnSync`, never through shell interpolation.