Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
/**
* Layered Intelligence — proposal APPROVAL FUNNEL metrics (#3120).
*
* The outcome store already answers "how did my resolved proposals fare?"
* (`summarizeOutcomeStats`) and "did my approved proposals get delivered?"
* (`computeProposalOutcomeMetrics`). Neither can answer the question this module exists
* for: **where in the human-approval funnel are my proposals stuck, and is the user's
* triage behavior changing?** Those are the loop's own bottleneck signals —
* a backlog of undecided proposals is a reason to STOP filing, and neither a lifetime
* merge rate nor a delivery rate makes it visible.
*
* Five metrics, all derived from the proposal LIFECYCLE (`filedAt` → `outcome` +
* `outcomeAt`) that `data/cos/li-outcomes/` already persists — explicitly NOT from
* tracker labels, and with no second on-disk aggregate to drift from its source:
* 1. Approval rate over a sliding window — measured over the decisions the user made
* INSIDE the window, so it tracks current triage behavior rather than restating the
* lifetime rate.
* 2. Pending-review count — proposals awaiting a human decision right now.
* 3. Median time from filing to a human decision (and, separately, to an APPROVAL).
* 4. Approver-delay buckets — how many undecided proposals are past 1 / 3 / 7 days.
* 5. Proposal-PHASE delivery: what share of everything LI filed has received any
* verdict at all. Deliberately distinct from `cosMetrics`, which measures the
* AGENT-TASK lifecycle ("did the run finish?") and says nothing about whether a
* filed proposal was ever looked at.
*
* Pure + side-effect-free, and NO LLM call: every number is a read of records the loop
* already keeps (the "no cold-bootstrap LLM calls" policy). `now` is an injected clock
* seam so the ages are testable and so one selfEval run reasons against a single "now".
*
* DECISION TIMESTAMPS MUST BE AUTHORITATIVE. Only a tracker-reported close time dates
* the human's decision. The `plan` tracker reports none, so `reconcileOutcomes`
* synthesizes one from the reconcile clock and marks it `outcomeAtSource: 'observed'`;
* this module treats such a record as UNDATED rather than as a decision made now, so a
* PLAN.md item checked off months ago but first reconciled today cannot inflate the
* window cohort or report a latency the user never took. A `plan` record that predates
* the provenance field is inferred the same way from its `tracker` — see
* `hasAuthoritativeDecisionDate` for why that cannot be left to a backfill.
*
* SENTINEL DISCIPLINE throughout — a rate whose denominator is zero is `null`, never 0:
* "no decision has been made in this window" and "every decision was a rejection" are
* opposite facts, and a 0% approval rate in the prompt is a direct instruction to the
* reasoner to hold back. A record whose timestamp cannot be parsed is counted in an
* explicit `undated` slot rather than being placed in a window or an age bucket it
* cannot be shown to belong to.
*/
import { DAY, formatDuration } from '../../lib/fileUtils.js';
import { LI_APPROVAL_FUNNEL_WINDOW_MS, LI_APPROVAL_DELAY_BUCKET_DAYS } from './constants.js';
import { summarizeDurations } from './awareness.js';
// The bucket key for an age threshold — `1` → `'1d'`. One place so the computed object's
// keys and the rendered line can't disagree.
export const delayBucketKey = (days) => `${days}d`;
/** A parsed epoch-ms timestamp, or null when the value is absent/unparseable. */
const parsedMs = (value) => {
const ms = Date.parse(value);
return Number.isFinite(ms) ? ms : null;
};
/**
* Clamp a timestamp at `now`. Nothing can have elapsed INTO the future, but a record's
* timestamps are not guaranteed to be in the past: cross-peer clock skew, a repaired
* record, or a tracker reporting in a different timezone offset can all leave a
* `filedAt`/`outcomeAt` ahead of the local clock. Unclamped that yields a NEGATIVE
* pending age (which `formatDuration` renders as a nonsensical "-1m") and an inflated
* filing-to-decision latency that drags the median. Clamping reads a future-dated record
* as "this just happened", which is the closest true statement available — and it is
* applied only to ELAPSED-TIME math, never to the window membership test: a decision is
* still a decision whenever it is dated.
*/
const elapsedSince = (ms, now) => Math.max(0, now - ms);
// Trackers that report NO close timestamp, so a resolved record's `outcomeAt` can only
// ever have been synthesized from the reconcile clock. `plan` (PLAN.md checkboxes) is the
// only one: `extractPlanSlugs` documents that a checked item carries no `closedAt`.
const TIMESTAMPLESS_TRACKERS = ['plan'];
/**
* Can this record's `outcomeAt` be read as WHEN THE HUMAN DECIDED?
*
* Only a tracker-reported close time can. Two ways a record fails that:
* - `outcomeAtSource === 'observed'` — reconcile explicitly stamped it as synthesized.
* - a `plan`-tracker record with NO provenance stamp — written before the field
* existed. These cannot be trusted the way a legacy forge/jira record can: forge and
* jira always report `closedAt`, so their legacy `outcomeAt` IS a real close time,
* but a `plan` record's never was. And they will not self-heal, because
* `reconcileOutcomes` no-ops on an already-settled record and so never reaches the
* provenance write — hence deriving it here at READ time rather than backfilling.
* Anything else (an explicit `'tracker'` stamp, or a legacy timestamped-tracker record)
* is authoritative.
*/
const hasAuthoritativeDecisionDate = (record) => {
if (record?.outcomeAtSource === 'observed') return false;
if (record?.outcomeAtSource === 'tracker') return true;
return !TIMESTAMPLESS_TRACKERS.includes(record?.tracker);
};
/**
* Compute the approval-funnel metrics for one app's outcome records.
*
* @param {Array} [outcomes] - the app's li-outcomes records (from listOutcomes). A
* NON-array is treated as an empty set here; callers that must distinguish a FAILED
* store read from a genuinely empty history check that BEFORE calling (the report
* formatter below does).
* @param {object} [opts]
* @param {number} [opts.now] - clock seam for ages and the window edge.
* @param {number} [opts.windowMs] - sliding-window span (default the shared constant).
* @returns {{ windowMs: number, windowDays: number,
* window: { decided: number, approved: number, rejected: number, abandoned: number,
* undated: number, approvalRate: number|null, timeToDecision: object,
* timeToApproval: object },
* pending: { count: number, oldestAgeMs: number|null, undated: number,
* byAge: Object<string, number> },
* proposalPhase: { totalFiled: number, totalDecided: number, totalPending: number,
* decisionRate: number|null } }}
*/
export function computeApprovalFunnel(outcomes = [], { now = Date.now(), windowMs = LI_APPROVAL_FUNNEL_WINDOW_MS } = {}) {
const records = (Array.isArray(outcomes) ? outcomes : []).filter(r => r && typeof r === 'object');
const windowStart = now - windowMs;
// --- Decision cohort: verdicts the user issued inside the window --------------
// Keyed on `outcomeAt` (WHEN the user decided), not `filedAt`: the metric is about the
// user's current triage behavior, so a 5-month-old proposal decided yesterday belongs
// in this window and a proposal filed yesterday and still open does not.
const decided = records.filter(r => r.outcome);
const decidedInWindow = [];
let undatedDecisions = 0;
for (const r of decided) {
// A SYNTHESIZED decision timestamp is the reconcile clock, not the human's decision:
// the `plan` tracker reports no close time, so a PLAN.md item checked off months ago
// but first reconciled today carries today's date. Treated exactly like an
// unparseable one — it lands in `undated` rather than being read as a decision made
// now, which would both inflate the window cohort and report a filing-to-decision
// latency the user never took.
const decidedMs = hasAuthoritativeDecisionDate(r) ? parsedMs(r.outcomeAt) : null;
// An undatable decision cannot be placed in or out of the window. Counting it in
// would inflate a "recent" rate with an unknown-age verdict; counting it out
// silently would undercount the user's activity. It gets its own slot.
if (decidedMs === null) { undatedDecisions += 1; continue; }
// Lower bound only, deliberately. A FUTURE-dated decision (clock skew on the peer
// that reconciled it, a repaired record) is still a decision the user made, and it
// is by definition the most recent one — so it belongs in the window. Excluding it
// would be the more dangerous choice, not the safer one: it would silently drop a
// real verdict from BOTH sides of the rate while `proposalPhase.totalDecided` still
// counted it, and a dropped REJECTION inflates the approval rate — exactly the
// direction that must never be flattered. The elapsed-time math clamps at `now`
// separately (see `latency`), so the skew cannot distort a duration.
if (decidedMs >= windowStart) decidedInWindow.push({ record: r, decidedMs });
}
const countOutcome = (name) => decidedInWindow.filter(({ record }) => record.outcome === name).length;
const approvedInWindow = decidedInWindow.filter(({ record }) => record.outcome === 'merged');
// Filing-to-decision latency. Only measurable when BOTH timestamps parse and the
// decision does not precede the filing (a clock-skewed or hand-edited record);
// summarizeDurations drops the NaN/negative values for us. Both ends are clamped at
// `now` so a future-dated record cannot contribute a latency longer than the proposal
// has existed and drag the median.
const latency = (entries) => summarizeDurations(
entries.map(({ record, decidedMs }) => {
const filedMs = parsedMs(record.filedAt);
return filedMs === null ? NaN : Math.min(decidedMs, now) - Math.min(filedMs, now);
})
);
// --- Right-now snapshot: what is awaiting a human decision -------------------
// Deliberately NOT windowed: a proposal that has sat undecided for 40 days is the
// single most important thing in this block, and a window would hide it.
const pending = records.filter(r => !r.outcome);
const byAge = Object.create(null);
for (const days of LI_APPROVAL_DELAY_BUCKET_DAYS) byAge[delayBucketKey(days)] = 0;
let oldestAgeMs = null;
let undatedPending = 0;
for (const r of pending) {
const filedMs = parsedMs(r.filedAt);
if (filedMs === null) { undatedPending += 1; continue; }
// Clamped at zero: a future-dated filing has waited no time at all, not a negative
// amount. Without this the oldest-wait line renders as "-1m" and a skewed record
// could sort itself to the top of the backlog.
const ageMs = elapsedSince(filedMs, now);
if (oldestAgeMs === null || ageMs > oldestAgeMs) oldestAgeMs = ageMs;
// CUMULATIVE buckets: a 9-day-old proposal counts in 1d, 3d AND 7d, so the three
// numbers read as a decay curve. `>=` so an age exactly at the threshold counts —
// "older than 1 day" at precisely 24h is past the mark, not short of it.
for (const days of LI_APPROVAL_DELAY_BUCKET_DAYS) {
if (ageMs >= days * DAY) byAge[delayBucketKey(days)] += 1;
}
}
return {
windowMs,
windowDays: Math.round(windowMs / DAY),
window: {
decided: decidedInWindow.length,
approved: approvedInWindow.length,
rejected: countOutcome('rejected'),
abandoned: countOutcome('abandoned'),
// Decisions the store holds but cannot date — reported so a window that looks
// empty is distinguishable from a store full of undatable verdicts.
undated: undatedDecisions,
// Null, never 0, when the user decided nothing in the window.
approvalRate: decidedInWindow.length > 0
? (approvedInWindow.length / decidedInWindow.length) * 100
: null,
timeToDecision: latency(decidedInWindow),
timeToApproval: latency(approvedInWindow)
},
pending: {
count: pending.length,
oldestAgeMs,
undated: undatedPending,
byAge
},
proposalPhase: {
totalFiled: records.length,
totalDecided: decided.length,
totalPending: pending.length,
// The proposal-phase analogue of a delivery rate: of everything LI filed, what
// share ever got a human verdict? Null (not 0) when nothing has been filed.
decisionRate: records.length > 0 ? (decided.length / records.length) * 100 : null
}
};
}
/**
* Render the funnel as the lines folded into the `liSelfEval` block (#3120), so the
* reasoner sees its own approval bottleneck on EVERY run rather than only in a dashboard.
*
* Takes the SAME three-valued `outcomes` argument `computeSelfEvalSummary` receives:
* - not an array → the outcomes source is off / the store was unreadable. Reported as
* UNAVAILABLE. A failed read must never render as a clean zero-proposal funnel.
* - `[]` → read fine, genuinely nothing filed. Reported as such.
* Returns an array of lines (never a bare string) so the caller splices them into its
* own list without re-splitting.
*/
export function formatApprovalFunnelLines({ outcomes = null, now = Date.now(), windowMs = LI_APPROVAL_FUNNEL_WINDOW_MS } = {}) {
if (!Array.isArray(outcomes)) {
return ['- LI approval funnel: UNAVAILABLE — no outcome history was gathered this run, so how your proposals move through human review is unknown.'];
}
const funnel = computeApprovalFunnel(outcomes, { now, windowMs });
if (funnel.proposalPhase.totalFiled === 0) {
return [`- LI approval funnel: no proposals filed yet for this app — nothing has entered human review, so there is no approval bottleneck to read (this is an empty history, NOT a failed read).`];
}
const { window, pending, proposalPhase, windowDays } = funnel;
const lines = [];
lines.push(
`- LI approval rate (last ${windowDays}d): `
+ (window.approvalRate === null
// Deliberately free of any literal percentage: an idle approver means the rate is
// UNKNOWN, and even a rate quoted inside a disclaimer is a number the reasoner can
// latch onto as its own record.
? `no decisions in the window — ${proposalPhase.totalPending} proposal(s) awaiting review; rate unknown. An idle approver is not a poor approval rate.`
: `${Math.round(window.approvalRate)}% — ${window.approved} of ${window.decided} decisions were approvals`
+ `${window.rejected ? `, ${window.rejected} rejected` : ''}${window.abandoned ? `, ${window.abandoned} abandoned` : ''}.`)
+ (window.undated ? ` (${window.undated} recorded decision(s) carry no usable timestamp and are excluded from the window.)` : '')
);
lines.push(
'- LI time-to-decision: '
+ (window.timeToDecision.medianMs === null
? 'no dated decision in the window — median unknown.'
: `median ${formatDuration(window.timeToDecision.medianMs)} from filing to a human decision (p90 ${formatDuration(window.timeToDecision.p90Ms)}, ${window.timeToDecision.count} decided).`)
+ (window.timeToApproval.medianMs === null
? ' No approval in the window, so time-to-APPROVAL is unknown.'
: ` Median time-to-APPROVAL ${formatDuration(window.timeToApproval.medianMs)} (${window.timeToApproval.count} approved).`)
);
// "Pending human review" indicator — emitted ONLY when something is actually pending
// (the issue's explicit requirement). A "0 awaiting review" line every run would be
// noise, and worse, would read as reassurance in exactly the case where the real
// signal is elsewhere.
if (pending.count > 0) {
const stalled = LI_APPROVAL_DELAY_BUCKET_DAYS
.map(days => ({ days, count: pending.byAge[delayBucketKey(days)] }))
.filter(({ count }) => count > 0)
.map(({ days, count }) => `${count} past ${days}d`)
.join(', ');
lines.push(
`- PENDING HUMAN REVIEW: ${pending.count} filed proposal(s) are awaiting your approver's decision`
+ `${stalled ? ` (${stalled})` : ''}`
+ `${pending.oldestAgeMs === null ? '' : `; oldest has waited ${formatDuration(pending.oldestAgeMs)}`}.`
+ `${pending.undated ? ` ${pending.undated} of them carry no usable filing date and are excluded from the age buckets.` : ''}`
// The actionable read: an approval backlog means the constraint is human review
// capacity, not idea supply. Filing more into a full queue lowers the odds any of
// it is read — so the correct response is a higher bar, not more volume.
+ `${pending.byAge[delayBucketKey(LI_APPROVAL_DELAY_BUCKET_DAYS.at(-1))] > 0
? ` Proposals are sitting past ${LI_APPROVAL_DELAY_BUCKET_DAYS.at(-1)} days without a verdict — your bottleneck is human review, not idea supply. Adding to this queue makes it LESS likely any of it is read: file only something clearly more valuable than what is already waiting, or return proposal: null.`
: ''}`
);
}
lines.push(
`- LI proposal-phase throughput (distinct from the cosMetrics agent-task lifecycle): `
+ `${proposalPhase.totalDecided} of ${proposalPhase.totalFiled} filed proposals have received any verdict`
// The percentage is only rendered once a verdict exists. At zero decisions the
// fraction already says everything ("0 of 2"), and a bare "0%" in the self-eval
// block reads to the reasoner as a failure verdict on a pipeline that may simply be
// young — the same reason `computeSelfEvalSummary` keeps an unresolved merge rate
// out of percentage form. The computed `decisionRate` is unchanged for the
// queryable route payload, where a numeric 0 over a real denominator is correct.
+ `${proposalPhase.totalDecided > 0 ? ` (${Math.round(proposalPhase.decisionRate)}%)` : ''}`
+ `${proposalPhase.totalPending ? `; ${proposalPhase.totalPending} still unjudged` : ''}.`
);
return lines;
}