Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

10.2 KB
Raw Blame

Historical exhaustive Deep audits

This describes retained certification records from before extended Deep. They remain incomplete history. New launches and normal resume no longer execute this workflow. The descriptions below document the stored evidence format, not current launch instructions.

Deep audits

Deep is an audit depth, independent of File issues or Audit and fix delivery. Quick remains the default and retains its broad scan followed by a bounded investigation.

Run and resume

In an app's Quality tab, choose the categories, delivery policy, provider/model and Audit depth → Deep, then Run now. A launch performs one server-assigned batch of one pass, not the whole audit. Each subsystem/scenario is partitioned deterministically by sorted source paths into units of at most 24 files. Every source path belongs to exactly one partition of each scenario. Batches contain at most 12 partitions and 96 distinct owned paths. A very large individual file or unavailable required runtime evidence can still block a partition. Untouched obligations precede repeated blockers, with pass rotation and attempt counts preventing starvation. A partition advances through its own prerequisite passes; a blocked neighbor does not prevent useful trace, adversarial or challenge work elsewhere. The agent receives a compact assignment with pinned sources, prior evidence and relevant candidates; the full ledger remains available for targeted lookup. Batch limits never reduce the full coverage denominator. Publication-only retries at an unchanged revision use bounded rotating re-attestation batches while retaining the complete global coverage requirements. They are explicitly marked delivery retries, not new full-scope reviews. Failed publication remains incomplete; a changed validation revision invalidates prior post-fix receipts. A partial checkpoint stops the maintenance run. Resume Deep audit starts another invocation using the same coverage ledger. It can continue a partition into its next pass while other partitions still need earlier passes. Failed Deep tasks are parked rather than automatically retried. Completion keeps the task held until worktree cleanup finishes, then atomically parks it with any surviving branch/worktree pointer. Explicit resume waits for that handoff and reuses the persisted audit instructions. Recovery of an interrupted invocation also parks it for explicit resume instead of treating a commit or retry budget as permission to continue. Boot and reading status do not launch work.

An app's Automation tab also exposes depth separately from delivery. An explicitly configured schedule can start a Deep audit; a partial task then parks and blocks further automatic dispatch for that app/category. Use Run Now to resume explicitly. Its app/category/delivery identity keeps the same ledger across those invocations. Custom agent jobs accept taskMetadata.auditDepth: "deep" and an optional deepAuditId for an explicitly selected checkpoint. A custom job without an audit category uses the code-quality coverage category. Different apps, categories and delivery modes cannot share a checkpoint.

Manual runs show discovery, reviewed units, satisfied/required pass requirements, blockers, pending candidates and delivery separately. Incomplete Deep runs remain in history even beyond the ordinary finished-run cap. The full local ledger and derived progress are available through authenticated GET /api/cos/schedule/deep-audits/:id; manual IDs are the maintenance step IDs, and scheduled/custom IDs are returned in the agent's result.deepAudit.id. Normal maintenance start/stop/resume endpoints remain the launch controls; start accepts auditDepth: "quick" | "deep".

Coverage and completion

The server pins a clean Git revision, tracked paths and blob IDs, inventory hash, detected capabilities, explicit exclusions, category prompt version, Deep contract version and effective task prompt hash. It inventories root files, code, tests, docs and configuration rather than filtering to popular source suffixes. Fresh isolated workspaces initialize submodules recursively at their recorded commits before inventory. Retained submodule changes are refused rather than reset. Fresh resume workspaces use the server-owned ledger revision even when the default branch has advanced; an unavailable pinned commit blocks launch instead of falling back to a different source. An unexpanded submodule blocks completion. Ignored runtime assets are outside the source inventory; required runtime evidence that cannot be obtained must remain blocked.

The coverage groups are category × source directory/subsystem × normal, failure, and concurrency/recovery scenarios. Each group owns deterministic source partitions. Group completion is the conjunction of actual partition receipts, never a synthetic review receipt. This is a minimum, not a claim that those three scenarios exhaust every workflow. Agents can add concrete workflows or entry-point scenarios through additionalUnits; additions increase the denominator and require the same passes. They cannot delete inventoried units.

Every required unit needs evidence from:

  1. Static inspection, accounting for every file in that unit.
  2. End-to-end tracing, recording entry, exit, inbound callers, outbound dependencies, boundary observations and unresolved boundary questions. Follow relevant callers/callees outside the owned partition and cite their pinned identities as context; those references do not credit their separate coverage.
  3. Adversarial investigation, recording scenario, expected behavior, observed outcome and failures across the identified boundaries.
  4. An independent challenge invocation, bound to the preceding evidence and attributed to a different server-issued agent identity, explicitly challenging cross-boundary assumptions.

Statuses are derived from receipts: pending, scanned, traced, validated, reviewed, blocked, or inapplicable with a reason. Inapplicability needs source references and pass-specific explanations, including independent confirmation. A scan count, quality score, successful process, first fix or five findings cannot complete discovery. Required units and passes must be accounted for and candidates triaged. A verified defect is reviewed evidence, not automatically a coverage blocker: a dangling tracked symlink can be inspected and its missing target verified without claiming to review nonexistent contents. Genuinely inaccessible required evidence remains blocked. The findings register retains confirmed, rejected, duplicate, deferred and resolved candidates separately from small remediation PRs.

Fix delivery also requires a post-fix pass against a clean, exact tested revision. All current post-fix receipts must bind to that revision; evidence from different revisions cannot combine into completion. A resolved finding names that tested revision and its fix/test evidence; changing the validation revision requires re-attesting prior resolutions. Discovery, remediation and delivery have separate outcomes: deferred fixes keep remediation incomplete. Actual execution and verification results settle delivery after evidence ingestion. Existing PR policy still governs delivery; selecting Deep does not authorize merging or deploying.

Persistence and limitations

deep_audit_ledgers is machine-local PostgreSQL state. A row lock serializes checkpoint merges, and server-assigned attempt IDs plus report hashes make completion replay idempotent. The DB stores evidence, candidates, assignments and invalidated history. It is covered by PostgreSQL backup and does not federate. Additive, idempotent schema initialization provisions the table on upgrades and fresh installs; there is no seed or legacy data transform.

Agents atomically replace their checkpoint files after units, under data/cos/deep-audit-checkpoints/, outside source worktrees. Finalization imports them on success, failure or interruption before worktree cleanup. The JSON file preserves progress during an invocation; the UI's persisted counters advance when finalization imports it. Missing or malformed reports remain partial. The files contain private audit evidence and must not be committed. They are retained with CoS data for recovery; database receipts remain authoritative after import.

Contract v3 upgrades v1/v2 ledgers transactionally on explicit resume. The complete prior generation remains historical, and new partition/boundary obligations receive no inherited receipt credit. When the contract is the only scope change, enrolled groups and findings are retained. Legacy findings remain explicitly group-scoped, visible from each assigned partition in that group without conferring coverage credit. New findings must bind to assigned partitions. Migration records both old and new required-pass counts and is idempotent; stale in-flight reports cannot update the replacement generation.

Source, inventory, capabilities, exclusions or other prompt changes conservatively invalidate current evidence and retain it as historical. This first version invalidates the full generation rather than trying to prove which dependencies were unaffected. That includes resuming against a changed HEAD after an interrupted fix; the earlier discovery remains readable but does not certify the new source. Preserve and use the retained worktree/branch when recovering incomplete remediation. Git cannot move a linked worktree containing submodules: for a verified inactive, clean owned workspace, Deep resume detaches the old tree at its exact HEAD and attaches the preserved branch in a new workspace. The old files remain intact. Dirty, locked or ambiguous submodule workspaces stay blocked for explicit recovery; they are never force-deinitialized or reset.

Evidence is agent-reported, source-verified. The server checks assignment, scope, source blob identities, required evidence fields, ordering, separate invocation identity and exact revision bindings. It cannot prove that a natural-language trace is truthful or that a reported command actually ran. Independent challenge is a distinct invocation, not a guarantee of a different provider/model. Existing test and PR review requirements remain necessary. Deep mode may take many explicitly resumed invocations; it has no automatic repeat-until-perfect loop and launches no additional categories on its own.