Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

feat: score the audited slice per run and roll the category score server-side

[._.]/ Adam Eivy4b2a281fa1b7
diff --git a/client/src/components/apps/AppQuality.jsx b/client/src/components/apps/AppQuality.jsx
index 4cf55ae294..60532c003d 100644
--- a/client/src/components/apps/AppQuality.jsx
+++ b/client/src/components/apps/AppQuality.jsx
@@ -110,9 +110,9 @@ export default function AppQuality({ app, detail = false }) {
<h3 className="flex items-center gap-1.5 font-semibold text-white">
{label}
<InfoTooltip label="How the quality score works" placement="below" align="start" width={320}>
- <p>Equal-weight mean of broad, medium/high-confidence assessments from the last 30 days. Missing, partial, low-confidence and stale assessments are excluded, not counted as perfect. Assessments describe the code before fixes.</p>
- <p className="mt-1.5">Scores are the auditing agent’s evidence-based judgment, not an issue count: 90–100 no material defect · 70–89 localized debt · 40–69 significant problems · 10–39 severe defects · 0–9 pervasive failure. A run with no findings can score below 100.</p>
- {federation && <p className="mt-1.5">Unified score: the newest assessment per category across this install and sync peers with the same repository. Offline peers do not contribute.</p>}
+ <p>Equal-weight mean of the category scores. Each category score is the confidence-weighted mean of its last five qualifying runs from the last 30 days (broad signal coverage, medium/high confidence, scored); one run’s slice pick no longer swings the figure. Missing, partial, low-confidence and stale runs are excluded, not counted as perfect. Assessments describe the code before fixes.</p>
+ <p className="mt-1.5">Each run rates only the slice it deep-reviewed, as the auditing agent’s evidence-based judgment, not an issue count: 90–100 no material defect · 70–89 localized debt · 40–69 significant problems · 10–39 severe defects · 0–9 pervasive failure. A run with no findings can score below 100.</p>
+ {federation && <p className="mt-1.5">Unified score: the recent runs per category across this install and sync peers with the same repository, one per day. Offline peers do not contribute.</p>}
{score == null && !hasAssessments && <p className="mt-1.5">Maintenance tasks only supply a score when they return a valid quality report. Earlier runs are not scored retroactively.</p>}
</InfoTooltip>
</h3>
@@ -192,7 +192,9 @@ export default function AppQuality({ app, detail = false }) {
}
function CategoryRow({ category, below, runLink }) {
- const details = category.summary || category.totalFiles > 0 || category.id === 'better-dependency-freedom';
+ const details = category.summary || category.slice || category.totalFiles > 0 || category.id === 'better-dependency-freedom';
+ // Pre-#12076 servers send no window; a row with one shows how many runs fed its score.
+ const windowRuns = category.window?.runs ?? 0;
const menuItems = [
{ id: 'settings', label: 'Runner settings', to: `/cos/schedule?task=${encodeURIComponent(category.id)}` },
...(!category.sourcePeerId && category.agentId ? [{ id: 'run', label: 'View audit run', to: `/cos/agents/${category.agentId}` }] : []),
@@ -203,6 +205,7 @@ function CategoryRow({ category, below, runLink }) {
{category.stale ? 'Stale · ' : ''}{category.coverage}{category.confidence && ` · ${category.confidence}`}
{category.assessedAt && ` · ${formatDateShort(category.assessedAt)}`}
{category.sourcePeerId || category.sourcePeerName ? ` · ${category.sourcePeerName || 'federated peer'}` : ''}
+ {windowRuns > 0 && ` · ${pluralize(windowRuns, 'run')}`}
</>;
return (
<tr className={`grid grid-cols-[1fr_auto] items-center gap-x-3 gap-y-1.5 py-2.5 px-2 sm:px-0 sm:py-0 border-t border-port-border rounded-lg sm:rounded-none sm:table-row sm:align-middle${below ? ' bg-port-warning/10' : ''}${inapplicable ? ' text-gray-500' : ''}`}>
@@ -211,6 +214,8 @@ function CategoryRow({ category, below, runLink }) {
{category.label}
{details && <InfoTooltip label={`${category.label} assessment details`} placement="below" align="start" width={288} panelClassName="font-normal">
{category.summary && <p className="break-words">{category.summary}</p>}
+ {category.slice && <p className="mt-1 break-words">Latest slice: {category.slice}{category.runScore != null && ` · rated ${category.runScore}/100`}</p>}
+ {windowRuns > 0 && <p className="mt-1">Score: mean of the last {pluralize(windowRuns, 'qualifying run')}{category.window.worstSeverity != null && ` · worst severity in window: ${category.window.worstSeverity}/10`}</p>}
{category.totalFiles > 0 && <p className="mt-1">{category.scannedFiles}/{category.totalFiles} files scanned · Worst severity: {category.worstSeverity}/10</p>}
{category.id === 'better-dependency-freedom' && <p className="mt-1">Assesses whether packages earn their place; dependency count carries no automatic penalty.</p>}
</InfoTooltip>}
diff --git a/client/src/components/apps/AppQuality.test.jsx b/client/src/components/apps/AppQuality.test.jsx
index 0e70ea27db..bb1db83573 100644
--- a/client/src/components/apps/AppQuality.test.jsx
+++ b/client/src/components/apps/AppQuality.test.jsx
@@ -85,6 +85,25 @@ it('keeps secondary category actions in a row menu beside the run shortcut', asy
expect(screen.getByRole('menuitem', { name: 'View audit run' })).toHaveAttribute('href', '/cos/agents/run-1');
});
+// #12076: the row's score is a rolling window; the tooltip names the latest
+// slice and how many runs fed the figure. A pre-window server sends neither.
+it('shows the rolling window and the latest slice behind the category score', async () => {
+ const app = { id: 'example', quality: { score: 48, ratedCategories: 1, categories: [
+ { id: 'security', label: 'Security', score: 48, runScore: 60, slice: 'server/services/auth', coverage: 'partial', confidence: 'high',
+ summary: 'Credential handling is sound; one stale allowlist.', scannedFiles: 12, totalFiles: 12, worstSeverity: 4, window: { runs: 3, worstSeverity: 7 } },
+ { id: 'ux', label: 'UX', score: 80, coverage: 'broad', confidence: 'high' },
+ ] } };
+ render(<MemoryRouter><AppQuality app={app} detail /></MemoryRouter>);
+ await screen.findByText(/No scored assessments/);
+ const security = screen.getByRole('row', { name: /Security/ });
+ expect(security).toHaveTextContent('48/100');
+ expect(security).toHaveTextContent('partial · high · 3 runs');
+ fireEvent.click(screen.getByRole('button', { name: 'Security assessment details' }));
+ expect(screen.getByRole('tooltip')).toHaveTextContent('Latest slice: server/services/auth · rated 60/100');
+ expect(screen.getByRole('tooltip')).toHaveTextContent('Score: mean of the last 3 qualifying runs · worst severity in window: 7/10');
+ expect(screen.getByRole('row', { name: /UX/ })).not.toHaveTextContent(/run/);
+});
+
it('sorts the category breakdown by worst score or oldest last run', async () => {
const app = { id: 'example', quality: { categories: [
{ id: 'ux', label: 'UX', score: 80, coverage: 'broad', assessedAt: '2026-09-12T00:00:00Z' },
diff --git a/client/src/components/apps/AppQualityHistory.jsx b/client/src/components/apps/AppQualityHistory.jsx
index 1b93ee4079..2f4114f060 100644
--- a/client/src/components/apps/AppQualityHistory.jsx
+++ b/client/src/components/apps/AppQualityHistory.jsx
@@ -39,7 +39,7 @@ export default function AppQualityHistory({ appId, categories = [] }) {
<h4 className="flex items-center gap-1.5 text-sm font-medium">
Quality over time
<InfoTooltip label="About quality history" placement="below" align="start" width={288}>
- Daily snapshots (UTC), higher is healthier. Scores carry forward for up to 30 days; gaps mean no fresh evidence. Coverage changes can move the overall mean. Category views include provisional assessments.
+ Daily snapshots (UTC), higher is healthier. Each category value is the rolling mean of its recent qualifying runs as of that day; runs carry forward for up to 30 days, and gaps mean no fresh evidence. Coverage changes can move the overall mean. Category views include provisional assessments.
{state.data?.federation && ` Includes ${state.data.federation.available ?? 0} available full-sync peers; peer availability can change historical coverage.`}
</InfoTooltip>
</h4>
diff --git a/docs/features/app-quality.md b/docs/features/app-quality.md
index bcdd03ac53..ccff8efefd 100644
--- a/docs/features/app-quality.md
+++ b/docs/features/app-quality.md
@@ -35,14 +35,27 @@ material finding verified). An issue being filed, or one fix being made, is not
an automatic score improvement. Scores express an auditor's judgment supported
by the scan and inspection, not a certification or an exhaustive proof.
-The overall score is the rounded, equal-weight average of the latest eligible
-category assessments: **broad** discovery coverage, medium/high confidence, and
-at most 30 days old. Broad means the signal scan covered the entire declared
-eligible inventory; deep investigation still stays bounded. Partial, stale,
-low-confidence, unavailable and inapplicable assessments remain visible but do
-not contribute. Missing categories do not count as 100. The contributing category
-count is shown beside the score so a single good category cannot imply complete
-coverage. Its denominator is the number of categories that **apply** to the
+Each run rates only the **slice** it deep-reviewed (report `version: 2`:
+`slice` names the top-level paths or area, `sliceScore` is its 0–100 rating).
+A bounded run cannot honestly rate the whole category, and when it was asked to
+(`version: 1`, a category-wide `score`) the dashboard moved with whichever
+slice the agent picked — a 20-point swing between consecutive runs was normal.
+Stored v1 reports stay readable; their `score` is read as that run's score.
+
+A category's score is derived server-side: the confidence-weighted mean (high
+1, medium 0.5) of its newest five eligible runs — **broad** discovery coverage,
+medium/high confidence, scored, and at most 30 days old — one run per UTC day.
+The breakdown shows how many runs fed the figure and the worst severity any of
+them found; the row's coverage, confidence, date, slice and summary describe the
+newest run. A newer partial run does not erase a fresh scored one. With no
+eligible run the row shows the newest run's own score beside its stale, partial
+or low-confidence label and does not contribute. The overall score is the
+rounded, equal-weight average of the contributing category scores. Broad means
+the signal scan covered the entire declared eligible inventory; deep
+investigation still stays bounded. Partial, stale, low-confidence, unavailable
+and inapplicable assessments remain visible but do not contribute. Missing
+categories do not count as 100. The contributing category count is shown beside
+the score so a single good category cannot imply complete coverage. Its denominator is the number of categories that **apply** to the
repository (`applicableCategories`): a category is inapplicable when the
repository scan says so (see "Which checks" below — the app detail read runs it)
or its own fresh assessment reported `not-applicable`, unless a fresh rated
@@ -58,9 +71,12 @@ markers and malformed JSON. Missing/invalid output never invents a score or
replaces a previous valid measurement; the prior measurement keeps its original
date and expires normally. Assessment collection does not relax commit/PR gates.
The run's recorded start time orders measurements so delayed recovery cannot
-replace newer evidence. PostgreSQL retains each run measurement and projects the latest per app/category,
-locally and within the ordinary database backup; no old runs are guessed into
-scores and no extra audit batch is enabled automatically.
+replace newer evidence. PostgreSQL retains each run measurement and projects the
+recent runs per app/category into the rolling score, locally and within the
+ordinary database backup; no old runs are guessed into scores and no extra
+audit batch is enabled automatically. Peers and the `.quality.json` snapshot
+receive the numeric v1 wire shape with a v2 run's `sliceScore` in `score`, so
+an older install keeps reading it; the slice text never leaves the machine.
## Coverage review
@@ -121,8 +137,9 @@ ranges and an overall/category selector. Both selections are shareable URL
parameters. The expandable history table provides dates, values and coverage
without relying on color or hover. Refresh history reloads measurements.
-Daily snapshots use the latest assessment available on that date, carrying it
-forward for at most 30 days. Missing or expired evidence creates a gap. The
+Daily snapshots apply the same rolling derivation to the runs known on that
+date, carrying each run forward for at most 30 days. Missing or expired evidence
+creates a gap. The
current day ends at the request time. Category charts include provisional
partial/low-confidence scores; the overall chart applies the same eligibility
rules as today's score. Coverage is shown with each value because adding or
diff --git a/server/lib/README.md b/server/lib/README.md
index e8f71285c4..17d6694c16 100644
--- a/server/lib/README.md
+++ b/server/lib/README.md
@@ -810,7 +810,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub
| `voiceEngines.js` | Shared TTS engine IDs, display metadata, supported-engine set, persisted configuration keys, and retired TTS config migration. |
| `voiceModelAssets.js` | Completion contract for downloaded voice model assets (Whisper `.bin`, Piper `.onnx` + `.onnx.json`). `inspectVoiceAsset(kind, path)` is the read-only readiness check (`verified` / `unverified` / `incomplete` / `missing` — stat plus one small JSON read, no hashing or network); `verifyVoiceAssetHashes` and `writeVoiceAssetReceipt` are the script-side halves that re-hash and record the `.portos-complete.json` receipt after promotion. `isManagedVoiceAsset` separates setup-script-owned files from user-supplied ones. Shared by `voice/bootstrap.js`, `voice/health.js` and `scripts/voice-asset.js`. |
-| `auditQuality.js` | Category discovery strategies, assessment prompt and strict schema, sentinel parsing, and freshness-aware app quality aggregation with deterministic latest-per-category selection across peers. |
+| `auditQuality.js` | Category discovery strategies, slice-scored assessment prompt and v1/v2 report schema, sentinel parsing, and freshness-aware app quality aggregation: per-category rolling window of recent run scores over daily winners across peers. |
| `mindToolRecipes.js` | Closed recipe schemas, current semantic-contract validation, and safe JSON field binding resolution. |
| `deckTemplates.js` | Decks (playing cards / tarot): kind alphabet, per-kind card roster (`deckCardRoster`), exact playing-card rank/suit identity cues (`deckCardIdentityPrompt`), shared layout and face-orientation prompts (`deckCardOrientationPrompt`), render prompt composition (`composeCardRenderPrompt`), per-card status and `deckCompletion`, plus `deckCardSize` / `deckCardAspectStyle` (persisted trim → thumbnail box). Pure — re-exported by `client/src/lib/decks.js`. |
diff --git a/server/lib/auditQuality.js b/server/lib/auditQuality.js
index e3a24319dd..20bda4e956 100644
--- a/server/lib/auditQuality.js
+++ b/server/lib/auditQuality.js
@@ -51,11 +51,11 @@ This contract overrides narrower slice-selection or ranking advice in the missio
2. Category search: ${AUDIT_DISCOVERY[category]}
3. Rank at least the top five candidates (or all if fewer), with paths, measured signals, impact, reach, confidence and why higher candidates win. Validate the top candidates through callers, tests and relevant history before filing or fixing. Severity and user impact outrank ease, small scope and recency. Churn and fan-in break ties; inactivity alone never dismisses a severe offender. If the raw worst is exempt, already filed or not actionable, report why and continue down the ranking. Existing unresolved issues still count against quality. Never pick a nit while a verified major problem remains actionable.
4. Keep investigation bounded after this scan. Report scanned roots, tools/commands, candidate ranking, exclusions, reviewed paths, unreviewed inventory and stopping reason. If tools or budget prevent broad discovery, report partial coverage, never claim a global worst or a clean repository.
-5. Assess the PRE-FIX codebase for this category on 0–100 (higher is healthier): 90–100 no material defect found after broad evidence; 70–89 localized moderate debt; 40–69 significant recurring or widespread problems; 10–39 severe/core-workflow defects; 0–9 pervasive critical failure. Score the category, not the agent's performance or issue count. Explain the score using severity, prevalence and concrete evidence. Do not award points merely for filing or fixing this run. Use score:null for unavailable or inapplicable assessment; zero is a real score.
+5. Assess the PRE-FIX state of the SLICE you deep-reviewed on 0–100 (higher is healthier): 90–100 no material defect found in the slice after real inspection; 70–89 localized moderate debt; 40–69 significant recurring or widespread problems; 10–39 severe/core-workflow defects; 0–9 pervasive critical failure. Name the slice (the top-level paths or area actually read, not the whole inventory) and rate only what you read; the dashboard derives the category trend server-side from recent slice scores, so never extrapolate a repository-wide number from a signal scan. Rate the slice, not the agent's performance or issue count. Explain the sliceScore using severity, prevalence and concrete evidence from the slice. Do not award points merely for filing or fixing this run. Use sliceScore:null for unavailable or inapplicable assessment; zero is a real score.
6. Score the worst finding's severity separately on 1–10: 1–3 minor localized impact, 4–6 material recurring cost, 7–8 major workflow/reliability impact, 9–10 critical data/security/availability failure. Use 0 only when no material finding was verified. List this severity for every finding in the narrative.
7. Include exactly one single-line QUALITY_AUDIT_JSON: {...} in your completion sentinel summary AND final response, with no secrets or personal data. Replace the example values with observed evidence. The category is fixed to this task; do not rate other categories. Keep ordinary completion/PR instructions and summaries too.
-QUALITY_AUDIT_JSON: {"version":1,"category":"${category}","score":null,"worstSeverity":0,"coverage":"unavailable","confidence":"low","summary":"Explain the assessment and strongest evidence","scannedFiles":0,"totalFiles":0}
-Allowed coverage: broad, partial, unavailable, not-applicable. Allowed confidence: low, medium, high. scannedFiles counts files actually included in the category signal scan; totalFiles is the eligible inventory. Broad requires the whole inventory to be scanned (deep review remains bounded). Partial assessments are excluded from the overall score. For partial coverage, use score:null when the inspected evidence cannot support a category score; explain why rather than inventing one. A numeric partial score is provisional. If unavailable or not-applicable, score must be null and explain why. Even a zero-finding audit returns this assessment.`;
+QUALITY_AUDIT_JSON: {"version":2,"category":"${category}","slice":"top-level paths or area deep-reviewed","sliceScore":null,"worstSeverity":0,"coverage":"unavailable","confidence":"low","summary":"Explain the slice assessment and strongest evidence","scannedFiles":0,"totalFiles":0}
+Allowed coverage: broad, partial, unavailable, not-applicable. Allowed confidence: low, medium, high. coverage describes the signal scan in step 1, not the deep review: scannedFiles counts files actually included in the category signal scan; totalFiles is the eligible inventory. Broad requires the whole inventory to be scanned (deep review remains bounded). Partial assessments are excluded from the overall score. For partial coverage, use sliceScore:null when the inspected evidence cannot support a slice score; explain why rather than inventing one. A numeric partial score is provisional. If unavailable or not-applicable, sliceScore must be null and explain why. Even a zero-finding audit returns this assessment.`;
}
export const appQualityQuerySchema = z.object({ includeQuality: z.enum(['true', 'false']).optional() });
@@ -68,21 +68,53 @@ export const appListQuerySchema = appQualityQuerySchema.extend({
view: z.enum(['nav', 'probe']).optional(),
});
-export const auditQualityReportSchema = z.object({
- version: z.literal(1),
- category: z.string().transform(normalizeAuditTaskType)
- .refine(value => Object.hasOwn(AUDIT_DEFINITIONS, value)),
- score: z.number().int().min(0).max(100).nullable(),
+const scoreSchema = z.number().int().min(0).max(100).nullable();
+const categorySchema = z.string().transform(normalizeAuditTaskType)
+ .refine(value => Object.hasOwn(AUDIT_DEFINITIONS, value));
+// Key order is the stored JSON order; keep the score fields where v1 wrote them.
+const reportFields = {
worstSeverity: z.number().int().min(0).max(10),
coverage: z.enum(['broad', 'partial', 'unavailable', 'not-applicable']),
confidence: z.enum(['low', 'medium', 'high']),
summary: z.string().trim().min(1).max(2000),
scannedFiles: z.number().int().nonnegative(),
totalFiles: z.number().int().nonnegative(),
-}).strict().refine(r => r.scannedFiles <= r.totalFiles)
- .refine(r => ['unavailable', 'not-applicable'].includes(r.coverage) ? r.score === null : r.scannedFiles > 0 && (r.coverage === 'partial' || r.score !== null))
+};
+
+/**
+ * Two report shapes are stored side by side (#12076). v1 carried a category-wide
+ * `score` the agent extrapolated from a bounded slice, which moved with the slice
+ * it happened to pick. v2 names the `slice` and rates only it (`sliceScore`); the
+ * category figure is derived server-side from a rolling window of run scores
+ * (`summarizeAppQuality`). Both shapes stay parseable forever: stored v1 rows and
+ * older peers keep rendering, and the run score is read from either.
+ */
+export const auditQualityReportSchema = z.discriminatedUnion('version', [
+ z.object({ version: z.literal(1), category: categorySchema, score: scoreSchema, ...reportFields }).strict(),
+ z.object({ version: z.literal(2), category: categorySchema, slice: z.string().trim().min(1).max(500), sliceScore: scoreSchema, ...reportFields }).strict(),
+]).refine(r => r.scannedFiles <= r.totalFiles)
+ .refine(r => ['unavailable', 'not-applicable'].includes(r.coverage) ? qualityRunScore(r) === null : r.scannedFiles > 0 && (r.coverage === 'partial' || qualityRunScore(r) !== null))
.refine(r => r.coverage !== 'broad' || (r.totalFiles > 0 && r.scannedFiles === r.totalFiles));
+/** The one run's 0–100 assessment from either report shape; `null` when unscored. */
+function qualityRunScore(report) {
+ if (!report || typeof report !== 'object') return null;
+ return (report.version === 2 ? report.sliceScore : report.score) ?? null;
+}
+
+/**
+ * The numeric wire/file shape (`PORTOS_SCHEMA_VERSIONS.appQuality` v1, the
+ * `.quality.json` row) for any stored report: a v2 run score travels in the v1
+ * `score` field, so an older peer reads it as the per-run score it already
+ * expects, and the `slice` text — which names repository paths — never leaves
+ * the machine. The result is still validated by its consumer's schema.
+ */
+export function numericQualityReport(report) {
+ if (!report || typeof report !== 'object' || report.version !== 2) return report;
+ const { category, worstSeverity, coverage, confidence, scannedFiles, totalFiles } = report;
+ return { version: 1, category, score: qualityRunScore(report), worstSeverity, coverage, confidence, scannedFiles, totalFiles };
+}
+
export function parseAuditQualityReport(summary, category) {
if (typeof summary !== 'string') return null;
const lines = summary.split(/\r?\n/).filter(line => line.startsWith(REPORT_PREFIX));
@@ -91,7 +123,7 @@ export function parseAuditQualityReport(summary, category) {
// An agent that scanned only part of the inventory but labelled it "broad"
// (the common slip) still measured something real. Record it as the partial
// assessment it honestly is rather than dropping the whole run's measurement.
- const claim = raw && typeof raw === 'object' && raw.coverage === 'broad' && raw.score !== null
+ const claim = raw && typeof raw === 'object' && raw.coverage === 'broad' && qualityRunScore(raw) !== null
&& Number.isFinite(raw.scannedFiles) && Number.isFinite(raw.totalFiles)
&& raw.scannedFiles > 0 && raw.scannedFiles < raw.totalFiles
? { ...raw, coverage: 'partial' } : raw;
@@ -134,8 +166,43 @@ export function isFreshAssessment(assessedAt, now = Date.now()) {
return Number.isFinite(age) && age >= 0 && age <= AUDIT_FRESHNESS_MS;
}
+/** How many recent run scores a category's dashboard figure averages (#12076). */
+export const QUALITY_WINDOW_RUNS = 5;
+// A low-confidence run never contributes; the window weights the rest so a
+// hedged assessment moves the figure half as far as a confident one.
+const CONFIDENCE_WEIGHT = Object.freeze({ high: 1, medium: 0.5 });
+
+/** Whether one run's parsed report qualifies for the category window: fresh, broad signal scan, not hedged, scored. */
+function qualifiesForWindow(run, now) {
+ return run.valid && isFreshAssessment(run.record.assessedAt, now) && run.report.coverage === 'broad'
+ && run.report.confidence !== 'low' && qualityRunScore(run.report) !== null;
+}
+
+/** Confidence-weighted mean of the window's run scores, rounded; `null` for an empty window. */
+function windowScore(window) {
+ if (!window.length) return null;
+ const weighted = window.reduce((acc, run) => {
+ const weight = CONFIDENCE_WEIGHT[run.report.confidence];
+ return { sum: acc.sum + weight * qualityRunScore(run.report), weight: acc.weight + weight };
+ }, { sum: 0, weight: 0 });
+ return Math.round(weighted.sum / weighted.weight);
+}
+
/**
- * Roll the newest measurement per category into the app's quality summary.
+ * Roll each category's recent measurements into the app's quality summary.
+ *
+ * A category's `score` is the confidence-weighted mean of its newest
+ * `QUALITY_WINDOW_RUNS` qualifying runs (fresh, broad coverage, medium/high
+ * confidence, scored); `window` reports how many runs fed it and the worst
+ * severity any of them found. One run's slice score is slice noise — a 20-point
+ * swing between consecutive runs carried no information — so the newest run
+ * alone no longer decides the figure, and a newer partial run does not erase a
+ * fresh scored one. With no qualifying run the row falls back to the newest
+ * run's own score, shown beside its stale/partial/low-confidence evidence as
+ * before, and does not contribute. Everything else about the row (coverage,
+ * confidence, slice, `runScore`, summary, provenance, staleness) describes the
+ * newest run.
+ * The overall score stays the equal-weight mean of the contributing categories.
*
* `inapplicable` maps category → reason for categories this repository cannot
* have findings for (from `resolveQualityChecks`; the caller supplies it only
@@ -147,26 +214,37 @@ export function isFreshAssessment(assessedAt, now = Date.now()) {
* `totalCategories` keeps its catalog-size meaning for existing readers.
*/
export function summarizeAppQuality(records = [], now = Date.now(), { inapplicable = {} } = {}) {
- const normalizedRecords = records.map(record => ({
- ...record,
- category: normalizeAuditTaskType(record.category),
- }));
+ const runsByCategory = new Map();
+ for (const record of records) {
+ const category = normalizeAuditTaskType(record.category);
+ const report = auditQualityReportSchema.safeParse(record.report);
+ const valid = report.success && report.data.category === category;
+ if (!runsByCategory.has(category)) runsByCategory.set(category, []);
+ runsByCategory.get(category).push({ record, valid, report: valid ? report.data : null });
+ }
const categories = Object.entries(AUDIT_DEFINITIONS).map(([id, definition]) => {
- const record = normalizedRecords.find(row => row.category === id);
- const report = auditQualityReportSchema.safeParse(record?.report);
- const valid = report.success && report.data.category === id;
+ const runs = (runsByCategory.get(id) || []).sort((a, b) => compareQualityRecords(b.record, a.record));
+ const latest = runs[0];
+ const record = latest?.record;
+ const valid = Boolean(latest?.valid);
const assessedAt = record?.assessedAt;
const stale = valid && !isFreshAssessment(assessedAt, now);
- const contributes = valid && !stale && report.data.coverage === 'broad' && report.data.confidence !== 'low' && report.data.score !== null;
+ const window = runs.filter(run => qualifiesForWindow(run, now)).slice(0, QUALITY_WINDOW_RUNS);
+ const contributes = window.length > 0;
// Only this install's own ruling, as the dispatch gate reads it: a peer's
// or a release snapshot's describes a different checkout of the repository.
const local = !record?.sourcePeerId && !record?.sourcePeerName;
- const reportedNotApplicable = valid && !stale && local && report.data.coverage === 'not-applicable';
+ const reportedNotApplicable = valid && !stale && local && latest.report.coverage === 'not-applicable';
const detectedReason = Object.hasOwn(inapplicable, id) ? inapplicable[id] : null;
const applicable = contributes || !(reportedNotApplicable || detectedReason);
const inapplicableReason = applicable ? null
: detectedReason || 'the latest audit reported this category as not applicable';
- return { id, label: definition.label, ...(valid ? report.data : { score: null, coverage: 'unavailable' }), assessedAt: assessedAt || null, agentId: record?.agentId || null, sourcePeerId: record?.sourcePeerId || null, sourcePeerName: record?.sourcePeerName || null, stale, contributes, applicable, inapplicableReason };
+ const { version: _version, category: _category, score: _score, sliceScore: _sliceScore, slice, ...latestFields } = latest?.report || {};
+ const runScore = valid ? qualityRunScore(latest.report) : null;
+ return { id, label: definition.label, ...(valid ? latestFields : { coverage: 'unavailable' }),
+ score: contributes ? windowScore(window) : runScore, runScore, slice: valid ? slice ?? null : null,
+ window: { runs: window.length, worstSeverity: window.length ? Math.max(...window.map(run => run.report.worstSeverity)) : null },
+ assessedAt: assessedAt || null, agentId: record?.agentId || null, sourcePeerId: record?.sourcePeerId || null, sourcePeerName: record?.sourcePeerName || null, stale, contributes, applicable, inapplicableReason };
});
const rated = categories.filter(c => c.contributes);
return {
@@ -180,14 +258,17 @@ export function summarizeAppQuality(records = [], now = Date.now(), { inapplicab
export const appQualityHistoryQuerySchema = z.object({ days: z.enum(['30', '90', '365']).default('90').transform(Number) });
-/** Daily UTC snapshots of evidence available then; never backfill past scores. */
+/**
+ * Daily UTC snapshots of evidence available then; never backfill past scores.
+ * Each point applies the same rolling derivation as the live summary to the
+ * runs known by that day, so the chart and the breakdown agree.
+ */
export function buildAppQualityHistory(records, days, now = Date.now()) {
const dayMs = 86400000;
const today = Math.floor(now / dayMs) * dayMs;
- const sorted = records.filter(r => Number.isFinite(Date.parse(r.assessedAt)))
- .map(record => ({ ...record, category: normalizeAuditTaskType(record.category) }))
- .sort(compareQualityRecords);
+ const sorted = dailyQualityRecords(records.filter(r => Number.isFinite(Date.parse(r.assessedAt))));
const latest = new Map();
+ let known = [];
let cursor = 0;
const points = [];
for (let day = today - (days - 1) * dayMs; day <= today; day += dayMs) {
@@ -195,8 +276,12 @@ export function buildAppQualityHistory(records, days, now = Date.now()) {
while (cursor < sorted.length && Date.parse(sorted[cursor].assessedAt) <= asOf) {
const record = sorted[cursor++];
latest.set(record.category, record);
+ known.push(record);
}
- const summary = summarizeAppQuality([...latest.values()], asOf);
+ // A run older than the freshness window can no longer feed any category's
+ // rolling figure; only the newest per category still matters, as its evidence row.
+ known = known.filter(record => Date.parse(record.assessedAt) >= asOf - AUDIT_FRESHNESS_MS || latest.get(record.category) === record);
+ const summary = summarizeAppQuality(known, asOf);
points.push({ date: new Date(day).toISOString().slice(0, 10), score: summary.score,
ratedCategories: summary.ratedCategories,
categories: Object.fromEntries(summary.categories.map(c => [c.id, {
@@ -206,14 +291,23 @@ export function buildAppQualityHistory(records, days, now = Date.now()) {
return { days, totalCategories: Object.keys(AUDIT_DEFINITIONS).length, points };
}
-/** One newest measurement per category, with a stable run tie-break across all installs. */
-export function latestQualityRecords(records) {
- const latest = new Map();
+/**
+ * One measurement per category per UTC day — the newest run that day, with a
+ * stable run tie-break across all installs — in ascending order. The same run
+ * can reach a reader twice: as the local row and again through the release
+ * snapshot it was published into, which keeps only the day. Collapsing to the
+ * day is what stops the rolling window counting that run twice, and matches the
+ * daily winners the history query and the wire export already select.
+ */
+export function dailyQualityRecords(records) {
+ const byDay = new Map();
for (const record of [...records].sort(compareQualityRecords)) {
const normalized = { ...record, category: normalizeAuditTaskType(record.category) };
- latest.set(normalized.category, normalized);
+ const time = Date.parse(normalized.assessedAt);
+ const day = Number.isFinite(time) ? new Date(time).toISOString().slice(0, 10) : String(normalized.assessedAt);
+ byDay.set(`${normalized.category}|${day}`, normalized);
}
- return [...latest.values()];
+ return [...byDay.values()].sort(compareQualityRecords);
}
export function compareQualityRecords(a, b) {
diff --git a/server/services/appQuality.js b/server/services/appQuality.js
index 871a1553de..022519f9cf 100644
--- a/server/services/appQuality.js
+++ b/server/services/appQuality.js
@@ -4,7 +4,7 @@ import { ensureSchema, query } from '../lib/db.js';
import { doneSentinelPath, parseSentinelPayload } from '../lib/agentSentinel.js';
import { tryReadFile } from '../lib/jsonIo.js';
import { normalizeAuditTaskType } from '../lib/auditCatalog.js';
-import { parseAuditQualityReport, summarizeAppQuality, buildAppQualityHistory, latestQualityRecords } from '../lib/auditQuality.js';
+import { AUDIT_FRESHNESS_MS, parseAuditQualityReport, summarizeAppQuality, buildAppQualityHistory, dailyQualityRecords } from '../lib/auditQuality.js';
import { collectAppQuality, qualityRecord, readQualityRecords, readReleaseQuality } from './appQualityFederation.js';
export async function recordAuditQuality({ task, taskType, agentId, workspacePath, success, assessedAt, deepDiscoveryComplete = false }, deps = {}) {
@@ -41,10 +41,17 @@ export async function recordAuditQuality({ task, taskType, agentId, workspacePat
*/
export async function enrichAppsWithQuality(apps, deps = {}) {
if (!apps.length) return apps;
- // One app-scoped query per list, not one per tile or transcript scan.
+ // One app-scoped query per list, not one per tile or transcript scan. Every
+ // run inside the freshness window feeds a category's rolling figure; the
+ // newest run per category is kept past it as the row's evidence (#12076).
+ const now = deps.now ?? Date.now();
const result = await (deps.query || query)(
- `SELECT DISTINCT ON (app_id, category) app_id, category, agent_id, assessed_at, report FROM app_quality_measurements WHERE app_id = ANY($1::text[]) ORDER BY app_id, category, assessed_at DESC, agent_id DESC`,
- [apps.map(app => app.id)]
+ `SELECT app_id, category, agent_id, assessed_at, report FROM (
+ SELECT app_id, category, agent_id, assessed_at, report,
+ row_number() OVER (PARTITION BY app_id, category ORDER BY assessed_at DESC, agent_id DESC) AS run_rank
+ FROM app_quality_measurements WHERE app_id = ANY($1::text[])) runs
+ WHERE run_rank = 1 OR assessed_at >= $2`,
+ [apps.map(app => app.id), new Date(now - AUDIT_FRESHNESS_MS)]
).catch(err => {
console.error(`❌ App quality unavailable: ${err.message}`);
return null;
@@ -60,10 +67,10 @@ export async function enrichAppsWithQuality(apps, deps = {}) {
deps.resolveApplicability ? deps.resolveApplicability(app).catch(() => ({})) : {},
]);
return { ...app,
- quality: result ? { ...summarizeAppQuality(latestQualityRecords([
+ quality: result ? { ...summarizeAppQuality(dailyQualityRecords([
...result.rows.filter(row => row.app_id === app.id).map(qualityRecord),
...shared.records, ...releases[index],
- ]), deps.now ?? Date.now(), { inapplicable }), federation: shared.federation }
+ ]), now, { inapplicable }), federation: shared.federation }
: { ...summarizeAppQuality(), unavailable: true },
};
}));
diff --git a/server/services/appQuality.test.js b/server/services/appQuality.test.js
index 25045b8b51..4a40c62197 100644
--- a/server/services/appQuality.test.js
+++ b/server/services/appQuality.test.js
@@ -1,10 +1,13 @@
import { describe, it, expect, vi } from 'vitest';
import { recordAuditQuality, enrichAppsWithQuality } from './appQuality.js';
import { AUDIT_DEFINITIONS } from '../lib/auditCatalog.js';
-import { AUDIT_DISCOVERY, auditQualityInstructions, parseAuditQualityReport, parseRecommendedAudits, summarizeAppQuality, AUDIT_FRESHNESS_MS, buildAppQualityHistory } from '../lib/auditQuality.js';
+import { AUDIT_DISCOVERY, auditQualityInstructions, parseAuditQualityReport, parseRecommendedAudits, summarizeAppQuality, AUDIT_FRESHNESS_MS, buildAppQualityHistory, dailyQualityRecords, numericQualityReport, QUALITY_WINDOW_RUNS } from '../lib/auditQuality.js';
const report = (overrides = {}) => ({ version: 1, category: 'better-complexity', score: 35, worstSeverity: 8,
coverage: 'broad', confidence: 'high', summary: 'The checkout has a major branching hotspot in the job dispatcher.', scannedFiles: 100, totalFiles: 100, ...overrides });
+// The v2 shape (#12076): the agent names the slice it read and rates only that.
+const sliceReport = (overrides = {}) => ({ version: 2, category: 'better-complexity', slice: 'server/services', sliceScore: 35, worstSeverity: 8,
+ coverage: 'broad', confidence: 'high', summary: 'The job dispatcher has a major branching hotspot.', scannedFiles: 100, totalFiles: 100, ...overrides });
const sentinel = value => `Completed audit.\nQUALITY_AUDIT_JSON: ${JSON.stringify(value)}\n`;
const task = { metadata: { app: 'portos-default' } };
const assessedAt = '2026-09-10T00:00:00Z';
@@ -62,6 +65,69 @@ describe('scheduled audit measurement workflow', () => {
expect(parseAuditQualityReport(sentinel(report({ score: 0 })), 'better-complexity')?.score).toBe(0);
});
+ // #12076: a run rates the slice it deep-reviewed; the category-wide claim is gone
+ // from the prompt, and every stored v1 record stays readable beside v2 ones.
+ it('asks for a named slice score, parses the v2 report, and still reads stored v1 reports', () => {
+ const instructions = auditQualityInstructions('better-complexity');
+ expect(instructions).toContain('"version":2');
+ expect(instructions).toContain('"slice":"top-level paths or area deep-reviewed"');
+ expect(instructions).toContain('"sliceScore":null');
+ expect(instructions).not.toMatch(/score the category|assess the pre-fix codebase for this category/i);
+ const v2 = parseAuditQualityReport(sentinel(sliceReport()), 'better-complexity');
+ expect(v2).toEqual(sliceReport());
+ expect(numericQualityReport(v2)).toEqual({ version: 1, category: 'better-complexity', score: 35, worstSeverity: 8, coverage: 'broad', confidence: 'high', scannedFiles: 100, totalFiles: 100 });
+ expect(numericQualityReport(report())).toEqual(report());
+ expect(numericQualityReport(sliceReport({ sliceScore: 0 })).score).toBe(0);
+ for (const broken of [{ ...sliceReport(), slice: ' ' }, { ...sliceReport(), score: 35 }, sliceReport({ coverage: 'unavailable' }), sliceReport({ sliceScore: 101 })]) {
+ expect(parseAuditQualityReport(sentinel(broken), 'better-complexity')).toBeNull();
+ }
+ expect(parseAuditQualityReport(sentinel(sliceReport({ scannedFiles: 59, totalFiles: 89 })), 'better-complexity')).toMatchObject({ coverage: 'partial', sliceScore: 35 });
+ });
+
+ // #12076: the dashboard figure is a server-derived rolling window, so one
+ // run's slice pick no longer swings the category by 20 points.
+ it('derives the category score from a confidence-weighted window of recent run scores and keeps the newest run as evidence', () => {
+ const now = Date.parse('2026-09-10T12:00:00Z');
+ const daysAgo = n => new Date(now - n * 86400000).toISOString();
+ const run = (n, sliceScore, overrides = {}) => ({ category: 'security', agentId: `run-${n}`, assessedAt: daysAgo(n),
+ report: sliceReport({ category: 'security', slice: `slice-${n}`, sliceScore, worstSeverity: 3, ...overrides }) });
+ const records = [
+ run(0, 60, { coverage: 'partial', worstSeverity: 9 }), // newest: provisional, so it is the evidence row but not in the window
+ run(1, 80),
+ run(2, 40, { confidence: 'medium' }), // half weight
+ run(3, 90, { confidence: 'low' }), // never contributes
+ { category: 'security', agentId: 'run-4', assessedAt: daysAgo(4), report: report({ category: 'security', score: 50, worstSeverity: 3 }) }, // a stored v1 run counts too
+ run(5, 10, { worstSeverity: 7 }),
+ run(6, 20),
+ run(7, 100), // sixth qualifying run: outside the window
+ run(40, 0), // older than the freshness window
+ ];
+ expect(QUALITY_WINDOW_RUNS).toBe(5);
+ const quality = summarizeAppQuality(records, now);
+ // (80·1 + 40·0.5 + 50·1 + 10·1 + 20·1) / 4.5
+ expect(quality).toMatchObject({ score: 40, ratedCategories: 1 });
+ expect(quality.categories.find(c => c.id === 'security')).toMatchObject({
+ score: 40, runScore: 60, slice: 'slice-0', coverage: 'partial', worstSeverity: 9, agentId: 'run-0', stale: false,
+ window: { runs: 5, worstSeverity: 7 },
+ });
+ // No qualifying run: the row shows the newest run's own score beside its evidence label, and does not contribute.
+ const unqualified = summarizeAppQuality([run(3, 90, { confidence: 'low' }), run(40, 0)], now);
+ expect(unqualified).toMatchObject({ score: null, ratedCategories: 0 });
+ expect(unqualified.categories.find(c => c.id === 'security')).toMatchObject({ score: 90, runScore: 90, window: { runs: 0, worstSeverity: null } });
+ });
+
+ // The same run reaches the window as the local row and again through the
+ // day-only release snapshot it was published into; it must count once.
+ it('collapses each category to its newest run per UTC day before rolling', () => {
+ const now = Date.parse('2026-09-10T12:00:00Z');
+ const local = { category: 'security', agentId: 'run-a', assessedAt: '2026-09-10T10:00:00Z', report: sliceReport({ category: 'security', sliceScore: 80 }) };
+ const release = { category: 'security', measurementId: 'f'.repeat(64), assessedAt: '2026-09-10T00:00:00.000Z', sourcePeerName: 'Release snapshot', report: report({ category: 'security', score: 80 }) };
+ const older = { category: 'react-lifecycle', agentId: 'run-b', assessedAt: '2026-09-08T10:00:00Z', report: sliceReport({ category: 'react-lifecycle', sliceScore: 20 }) };
+ const daily = dailyQualityRecords([release, local, older, { ...local, agentId: 'run-c', assessedAt: '2026-09-08T09:00:00Z', report: sliceReport({ category: 'security', sliceScore: 20 }) }]);
+ expect(daily.map(r => [r.category, r.agentId])).toEqual([['security', 'run-c'], ['ui-lifecycle', 'run-b'], ['security', 'run-a']]);
+ expect(summarizeAppQuality(daily, now).categories.find(c => c.id === 'security')).toMatchObject({ score: 50, window: { runs: 2 } });
+ });
+
// Unique regression: a "broad" claim over a partial scan (5907/8923 files) used to drop the whole run's measurement.
it('records a broad claim over a partial scan as partial coverage instead of discarding it', () => {
const parsed = parseAuditQualityReport(sentinel(report({ scannedFiles: 59, totalFiles: 89 })), 'better-complexity');
@@ -151,10 +217,11 @@ describe('scheduled audit measurement workflow', () => {
});
});
-it('retains historical scores without hindsight, expires old evidence and exposes changed coverage', () => {
+it('retains historical scores without hindsight, expires old evidence, and rolls each day the same way as the live summary', () => {
const now = Date.parse('2026-09-10T12:00:00Z');
const rows = [
{ category: 'better-complexity', assessedAt: '2026-08-01T12:00:00Z', report: report({ score: 20 }) },
+ { category: 'better-complexity', assessedAt: '2026-09-08T12:00:00Z', report: sliceReport({ sliceScore: 30 }) },
{ category: 'better-complexity', assessedAt: '2026-09-09T12:00:00Z', report: report({ score: 70 }) },
{ category: 'security', assessedAt: '2026-09-10T10:00:00Z', report: report({ category: 'security', score: 90 }) },
];
@@ -162,8 +229,11 @@ it('retains historical scores without hindsight, expires old evidence and expose
expect(points.find(p => p.date === '2026-07-31').score).toBeNull();
expect(points.find(p => p.date === '2026-08-01')).toMatchObject({ score: 20, ratedCategories: 1 });
expect(points.find(p => p.date === '2026-09-01').score).toBeNull();
- expect(points.find(p => p.date === '2026-09-09')).toMatchObject({ score: 70, ratedCategories: 1 });
- expect(points.at(-1)).toMatchObject({ score: 80, ratedCategories: 2 });
+ expect(points.find(p => p.date === '2026-09-08')).toMatchObject({ score: 30, ratedCategories: 1 });
+ // 09-09 averages the two fresh runs (the 08-01 run has expired); the next day adds security.
+ expect(points.find(p => p.date === '2026-09-09')).toMatchObject({ score: 50, ratedCategories: 1 });
+ expect(points.at(-1)).toMatchObject({ score: 70, ratedCategories: 2 });
+ expect(points.at(-1).categories['better-complexity']).toMatchObject({ score: 50, coverage: 'broad', confidence: 'high' });
});
@@ -180,7 +250,7 @@ it('records honest unscored partial Deep evidence without inventing a score or c
expect(summary.categories.find(category => category.id === 'observability')).toMatchObject({ score: null, coverage: 'partial' });
expect(parseAuditQualityReport(sentinel({ ...partial, coverage: 'broad', scannedFiles: 10636 }), 'observability')).toBeNull();
expect(parseAuditQualityReport(sentinel({ ...partial, scannedFiles: 0 }), 'observability')).toBeNull();
- expect(auditQualityInstructions('observability')).toContain('For partial coverage, use score:null');
+ expect(auditQualityInstructions('observability')).toContain('For partial coverage, use sliceScore:null');
});
// The code-quality lane's triage output (#12085): a ranked list under a
diff --git a/server/services/appQualityFederation.js b/server/services/appQualityFederation.js
index 2770043de2..d46462d6a5 100644
--- a/server/services/appQualityFederation.js
+++ b/server/services/appQualityFederation.js
@@ -5,7 +5,7 @@ import { z } from 'zod';
import { query } from '../lib/db.js';
import { PORTOS_APP_ID } from '../lib/appIdentity.js';
import { AUDIT_TASK_TYPE_LIST, LEGACY_AUDIT_TASK_TYPE_ALIASES, normalizeAuditTaskType } from '../lib/auditCatalog.js';
-import { auditQualityReportSchema, AUDIT_FRESHNESS_MS } from '../lib/auditQuality.js';
+import { auditQualityReportSchema, AUDIT_FRESHNESS_MS, numericQualityReport } from '../lib/auditQuality.js';
import { PORTOS_SCHEMA_VERSIONS } from '../lib/schemaVersions.js';
import { getOriginInfo } from '../lib/gitRemote.js';
import { peerBaseUrl } from '../lib/peerUrl.js';
@@ -16,8 +16,11 @@ import { releaseTieId } from './appQualitySnapshotFormat.js';
const MAX_PAYLOAD_BYTES = 4 * 1024 * 1024;
const sharedSummary = 'Numeric assessment shared by a federated instance; evidence remains on the source machine.';
const hash = value => createHash('sha256').update(value).digest('hex');
+// A stored v2 (slice-scored) report travels as the v1 numeric shape: the wire
+// contract is unchanged, so a peer that predates v2 reads the run score it
+// already expects, and the slice text never leaves this machine (#12076).
const numericReportSchema = z.preprocess(value => value && typeof value === 'object'
- ? { ...value, summary: sharedSummary } : value, auditQualityReportSchema)
+ ? { ...numericQualityReport(value), summary: sharedSummary } : value, auditQualityReportSchema)
.transform(({ summary: _summary, ...report }) => report);
const measurementSchema = z.object({
measurementId: z.string().regex(/^[a-f0-9]{64}$/),
diff --git a/server/services/appQualityFederation.test.js b/server/services/appQualityFederation.test.js
index b02f8796b9..18746fa040 100644
--- a/server/services/appQualityFederation.test.js
+++ b/server/services/appQualityFederation.test.js
@@ -38,15 +38,16 @@ it('combines newest categories in app view and UTC history without changing othe
const remote = await exportPortosQuality(peer.instanceId, 30, deps([row('security', 80), row('performance', 60)]));
const local = { ...deps([row('security', 20, '2026-09-09T10:00:00Z')]), peerFetch: vi.fn(async () => response(remote)) };
const apps = await enrichAppsWithQuality([{ id: 'portos-default' }, { id: 'other' }], local);
- expect(apps[0].quality).toMatchObject({ score: 70, ratedCategories: 2, federation: { peers: 1, available: 1, unavailable: 0 } });
+ // security rolls the peer's run (80) with the local one (20); performance is the peer's alone.
+ expect(apps[0].quality).toMatchObject({ score: 55, ratedCategories: 2, federation: { peers: 1, available: 1, unavailable: 0 } });
expect(apps[0].quality.categories.find(c => c.id === 'security')).toMatchObject({ sourcePeerId: 'peer-a', agentId: null });
expect(apps[1].quality.score).toBeNull();
expect(local.peerFetch.mock.calls[0]).toEqual([expect.stringMatching(/^http:\/\/192\.0\.2\.1:5555\/api\/apps\/quality-federation\?days=30&categories=security%2C/),
expect.objectContaining({ signal: expect.any(AbortSignal), redirect: 'error', maxBytes: 4194304 }), peer]);
const history = await getAppQualityHistory({ id: 'portos-default' }, 30, local);
expect(history.points.find(p => p.date === '2026-09-09').score).toBe(20);
- expect(history.points.at(-1)).toMatchObject({ score: 70, ratedCategories: 2 });
- expect((await enrichAppsWithQuality([{ id: 'portos-default' }], local))[0].quality.score).toBe(70);
+ expect(history.points.at(-1)).toMatchObject({ score: 55, ratedCategories: 2 });
+ expect((await enrichAppsWithQuality([{ id: 'portos-default' }], local))[0].quality.score).toBe(55);
expect((await exportPortosQuality(peer.instanceId, 30, local)).measurements.map(m => m.report.score)).toEqual([20]);
});
@@ -100,14 +101,17 @@ it('keeps local scores on old, offline, malformed, oversize or mismatched peers
expect((await collectPortosQuality(30, { ...local, peerFetch: async () => response(future) })).records).toEqual([]);
});
-it('lets newer partial evidence supersede older broad scores and breaks ties consistently', async () => {
+it('keeps a fresh scored run in the window under newer partial evidence and breaks ties consistently', async () => {
const partial = row('security', 80);
partial.report.coverage = 'partial';
partial.report.scannedFiles = 1;
const payload = await exportPortosQuality(peer.instanceId, 30, deps([partial]));
const partialDeps = { ...deps([row('security', 20, '2026-09-09T10:00:00Z')]), peerFetch: async () => response(payload) };
- expect((await enrichAppsWithQuality([{ id: 'portos-default' }], partialDeps))[0].quality.score).toBeNull();
- expect((await getAppQualityHistory({ id: 'portos-default' }, 30, partialDeps)).points.at(-1).score).toBeNull();
+ // #12076: the partial run is the evidence row but the rolling figure still rests on the qualifying run.
+ const [partialApp] = await enrichAppsWithQuality([{ id: 'portos-default' }], partialDeps);
+ expect(partialApp.quality.score).toBe(20);
+ expect(partialApp.quality.categories.find(c => c.id === 'security')).toMatchObject({ score: 20, runScore: 80, coverage: 'partial', sourcePeerId: 'peer-a', window: { runs: 1 } });
+ expect((await getAppQualityHistory({ id: 'portos-default' }, 30, partialDeps)).points.at(-1).score).toBe(20);
const a = row('security', 25, undefined, { agent_id: 'run-a' });
const b = row('security', 75, undefined, { agent_id: 'run-b' });
const scores = [];
@@ -137,6 +141,22 @@ it('matches managed repositories with different local ids and excludes other ori
expect(await exportPortosQuality(peer.instanceId, 30, source, '0'.repeat(64))).toBeNull();
});
+// #12076: the wire contract stays v1 numeric. A v2 (slice-scored) run exports
+// its run score where an older peer already reads one; the slice text, which
+// names repository paths, never leaves the machine.
+it('exports a slice-scored v2 record in the v1 numeric wire shape and reads it back into the window', async () => {
+ const v2 = row('security', null);
+ v2.report = { version: 2, category: 'security', slice: 'server/services/auth and /Users/person/private', sliceScore: 64, worstSeverity: 5,
+ coverage: 'broad', confidence: 'high', summary: 'Private narrative', scannedFiles: 12, totalFiles: 12 };
+ const payload = await exportPortosQuality(peer.instanceId, 30, deps([v2]));
+ expect(payload).toMatchObject({ schemaVersion: 1, measurements: [{ report: { version: 1, category: 'security', score: 64, worstSeverity: 5, coverage: 'broad', confidence: 'high', scannedFiles: 12, totalFiles: 12 } }] });
+ expect(JSON.stringify(payload)).not.toMatch(/slice|Users|narrative|summary/i);
+ expect(qualityFileFromWireSnapshot(payload)).toContain(',64,');
+ const local = { ...deps([v2]), peerFetch: async () => response({ ...payload, measurements: [{ ...payload.measurements[0], measurementId: 'a'.repeat(64), assessedAt: '2026-09-09T10:00:00Z' }] }) };
+ const [app] = await enrichAppsWithQuality([{ id: 'portos-default' }], local);
+ expect(app.quality.categories.find(c => c.id === 'security')).toMatchObject({ score: 64, runScore: 64, slice: v2.report.slice, window: { runs: 2 } });
+});
+
// PortOS's own release evidence goes through the same `.quality.json` path as
// every managed app: same filename, same reader, same guards.
it('loads release evidence without peers and never re-exports it', async () => {
diff --git a/server/services/promptSections/completion.js b/server/services/promptSections/completion.js
index 4212dbd399..df0b647e55 100644
--- a/server/services/promptSections/completion.js
+++ b/server/services/promptSections/completion.js
@@ -500,7 +500,7 @@ export function buildAuditOutputCompletionSection(task, sentinelPath) {
Before writing the completion sentinel, prepare the validated QUALITY_AUDIT_JSON report described in the audit instructions for category "${category}".
Write your human summary AND exactly one single-line QUALITY_AUDIT_JSON: {...} report together to \`${sentinelPath}\`, then stop. This applies to every provider, including agents that normally finish by exiting.
The report is required even when no issues are filed or no code changes are needed. A successful issue or PR does not supply a quality assessment. A final response alone is not enough: PortOS reads the sentinel file to save the dashboard score.
-Preserve the report when following the summary template above; the template's length guidance does not replace this required output. If assessment is unavailable, use score:null and explain the missing evidence rather than inventing a score.`;
+Preserve the report when following the summary template above; the template's length guidance does not replace this required output. If assessment is unavailable, use sliceScore:null and explain the missing evidence rather than inventing a score.`;
}
/**