Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

fix: teach goal-fidelity rubric that a local-catalog entry's added key field counts as delivered

[._.]/ Adam Eivy953579c9e79a
diff --git a/server/services/codeReview.js b/server/services/codeReview.js
index 7800a8655b..3dff39b0ee 100644
--- a/server/services/codeReview.js
+++ b/server/services/codeReview.js
@@ -729,6 +729,8 @@ For an objective to add buttons or links that send an item to other pages (for e
For an objective to add a newly launched AI model to the provider catalog, presets, pricing, or model-comparison seed, judge the sourced facts the diff supplies: the model id in the provider/preset lists, its published prices, and a migration for existing installs when the diff changes shipped provider data. A model-comparison row whose quality, latency, or throughput fields are null (or whose benchmark scores are absent) is NOT missing work when the diff or commit states the vendor has not published those figures; recording an unpublished benchmark would be fabrication. Count a null benchmark as missing only when the objective explicitly asks for benchmarks and the diff offers no statement that none are published. This exempts only unpublished benchmark metrics for a new model; a missing model id, price, preset entry, or required migration remains missing.
+For an objective to add a newly released model to the local LLM catalog (a \`LOCAL_LLM_CATALOG\` array in \`localLlmCatalog.js\`), the entry's identifier is its \`key\` field (for example \`+ key: 'example-model',\`), alongside \`name\`, \`ollama\`/\`lmstudio\` ids, \`capabilities\` and \`context\`. Before reporting a catalog field missing, search the added (\`+\`) lines of that entry: a field present there is delivered, and the first added line of a new object literal is often the \`key\`. Entries in companion tables such as hardware-requirement or vision-heuristic lists (keyed by the same model key or name pattern) are supporting registrations, not unrequested work. This exempts only fields the added lines actually contain: an entry with no \`key\`, a key absent from the companion registration the objective requires, or missing runtime ids remains missing.
+
For an objective about clearing a federated peer's displayed schema-mismatch warning, a diff that makes the unchanged-checksum shortcut conditional on an already saved peer/category schema gap is a production retry, not a check-only change. The saved gap sends that category through the existing snapshot apply path again, where the receiver checks the newly fetched envelope's \`portosMeta.schemaVersions\`; a successful apply clears the saved warning for that peer/category. \`portosMeta.portosVersion\` is only the friendly label captured for display, while \`schemaVersions\` controls compatibility, so an older displayed label does not make this retry ineffective. Count the bypass as requested behavior only when the diff scopes it to an existing peer/category gap and preserves the ordinary unchanged-checksum return otherwise. An unconditional retry, a display-label-only edit, or a generic checksum change does not receive this interpretation and must be judged against the objective as usual.
For an objective to align a default process manifest with production startup and remove a retired process from the UI's expected-process list, judge manifest membership at the registry boundary, even without a literal UI assertion or a changed UI file. Apply this only to paired production removal plus fresh/legacy registry evidence:
diff --git a/server/services/codeReview.test.js b/server/services/codeReview.test.js
index 57bae62e97..1209a663f9 100644
--- a/server/services/codeReview.test.js
+++ b/server/services/codeReview.test.js
@@ -1223,6 +1223,38 @@ describe('codeReview helpers', () => {
expect(withoutPricing).toMatchObject({ ok: true, verdict: 'fix-first', missing: ['published pricing'] })
})
+ it('ships a local-catalog model entry whose added key field is present, but not one with no key', async () => {
+ const objective = 'Example Model 2 has just launched; add it to the local LLM catalog.'
+ const diff = [
+ 'diff --git a/server/lib/localLlmCatalog.js b/server/lib/localLlmCatalog.js',
+ '+ {',
+ "+ key: 'example-model-2',",
+ "+ name: 'Example Model 2',",
+ "+ ollama: 'example-model-2',",
+ '+ },',
+ ].join('\n')
+ const noKeyDiff = diff.replace(/^\+ key:.*\n/m, '')
+
+ global.fetch = vi.fn(async (_url, init) => {
+ const request = JSON.parse(init.body)
+ const rubric = request.messages[0].content
+ const recognizes = rubric.includes('the entry\'s identifier is its \`key\` field')
+ && rubric.includes('an entry with no \`key\`')
+ const hasKey = request.messages[1].content.includes('+ key:')
+ return mockJsonResponse({
+ choices: [{ message: { content: JSON.stringify(recognizes && hasKey
+ ? { verdict: 'ship', missing: [], unrequested: [], evidence: 'entry carries its added key' }
+ : { verdict: 'fix-first', missing: ['key field'], unrequested: [], evidence: 'no key' }) } }],
+ })
+ })
+
+ const withKey = await runLocalGoalFidelityReview({ backend: 'ollama', model: 'example-model', objective, diff })
+ const withoutKey = await runLocalGoalFidelityReview({ backend: 'ollama', model: 'example-model', objective, diff: noKeyDiff })
+
+ expect(withKey).toMatchObject({ ok: true, verdict: 'ship', missing: [] })
+ expect(withoutKey).toMatchObject({ ok: true, verdict: 'fix-first', missing: ['key field'] })
+ })
+
it('ships a same-checksum retry scoped to a saved federated schema gap, not an unconditional retry', async () => {
const objective = 'On our federated instances page, I see a schema version mismatch.'
const scopedRetryDiff = [