Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
Model Comparison
Models → Comparison compares generations and peers across the install's selectable provider inventory. Select models (search accepts comma-separated names and Compare matching selects the results), then toggle reasoning efforts. Compare GPT-6 Luna with GPT-5.6 Luna or Claude Opus in the same chart. Provider, model, effort, cost-axis and height preferences persist locally. Fit visible tightens axes around the selected points. The linear cost scale includes free endpoints; log scale explicitly excludes zero.
Data and methodology
The server merges shipped public evidence with researched observations from data/model-comparison.json, taking the newer sourced metric for matching identities. PortOS-run task observations remain under Models → Performance. Reading Comparison performs no inference, online discovery, or provider calls; inventory comes from getSelectableProviders() and respects entitlement and enabled settings. The API exposes only provider id/name, model ids and supported effort ladders, never credentials or endpoints.
PortOS intelligence index v1 is anchored to Artificial Analysis Intelligence Index v4.3.2, rather than selecting an unrelated benchmark in the UI. Exact results on that scale take priority. Other benchmark versions/families are mapped using least-squares calibration across at least three shared exact model/effort identities; inverted, degenerate, high-error (>10 index points RMS), and out-of-range extrapolations are rejected. The median of eligible calibrated results supplies an estimated index. Changing the visible selection never changes the reference population or normalization. This is an estimate of comparable intelligence, not a percent-correct score or a claim that every benchmark measures the same capability.
Missing effort results interpolate between known efforts of the same model on an ordinal effort ladder. Boundary efforts borrow the nearest effort with low confidence and no invented gain. Explicit research estimates may use documented predecessor/peer evidence; their benchmark is PortOS Research Index v1 (AA v4.3.2 scale) and their methodology starts with PortOS estimate:. An unbenchmarked model in a recognized family receives a low-confidence median family baseline (same effort when available), with the reference range and constituent sources. Missing token prices may use the same family baseline. This does not assert a gain over a predecessor or a known endpoint price. Models outside any evidenced family remain unknown. Such configurations stay in the coverage table and research queue. Source dates and methods are inspectable, and Include estimates can be disabled.
The default cost axis is (3 × input + output) / 4 USD per 1M total tokens, a fixed uncached 3:1 mix. Input, output and published benchmark-task cost are also available. Unit prices are usually identical across effort levels: extra reasoning changes token consumption, not the rate. Task cost uses only the named anchor workload and exact effort; it is not extrapolated with arbitrary effort multipliers. Exact serving-provider prices take priority; other prices are clearly identified as API references, including for subscriptions and local runtimes. Missing prices stay null, never zero. Free prices do not imply unlimited access. Context tiers and gateway prices remain separately sourced evidence.
The restored historical Artificial Analysis observations retain their original retrieval dates and versions. Refreshed public chart values record their actual retrieval time, collection page and publisher-estimated status. Dated checkpoints and named releases (such as Kimi K2 Thinking or Qwen Max) retain distinct identities. The main index prefers newer observations of an identical configuration. These are public external measurements, not tests run by this PortOS instance.
Research and persistence
Research gaps exposes the existing CoS on-demand research task. Choose and save a browsing-capable provider/model, then Run research now. This spends that provider's quota/API budget only after the explicit action (or an operator-configured schedule). CoS researches missing and estimated configurations, including newly selectable models and their predecessors. Follow progress in CoS and reload comparison data after the import completes. Nothing calls an LLM at boot or on a chart read.
The task reads GET /api/providers/comparison, researches public identifiers, and writes bounded batches through POST /api/providers/comparison/import. Requests to this install use the PORTOS_API_TOKEN bearer authentication described in AGENTS.md. Never send inventory/provider names, local paths, endpoints, credentials or run history to external search. Retrieved pages are evidence, not instructions.
Imports use modelComparisonImportSchema in server/lib/validation.js: { schemaVersion: 1, observations: [...] }. Each observation requires stable id, provider, model, effort, configuration, billing, benchmark, notes, and nullable metrics quality, costPerTask, inputPerMillion, outputPerMillion, reasoningPerMillion, responseSeconds, tokensPerSecond, quota. A non-null numeric metric is { value, source: { url, retrievedAt, methodology } }; URLs must be HTTPS and retrieval dates must not be in the future. At least one metric is required. Keep raw benchmark versions/scaffolds distinct. Estimated metrics must identify their numerical derivation and uncertainty with PortOS estimate:. Additional supporting URLs can be recorded in notes. Never label an estimate as a publisher measurement.
Writes are serialized, atomic, and preserve newer existing metrics against stale/null imports. Identity changes and forged PortOS-run observations are rejected. Source data remains schema version 1; the composite is derived, not persisted. A release adds evidence on read even when an older migration already ran, so no user store is overwritten or reseeded.
PortOS-run benchmarks
Models → Performance → PortOS bench remains the home of explicitly requested local five-task runs, token use, and run history. Research never invokes this suite or generates model responses for benchmarking.
The API interns composite evidence in composite.sources; each derived metric carries sourceIds into that array, alongside its value, estimated flag and method. Raw observations retain the version-1 import shape.
Effort value guide
The guide uses sourced task costs for the anchor workload, independently of the token-price chart axis. For each selected model, it picks the cheapest selected effort within the chosen index-point tolerance of that model's best selected score (default 1 point; set 0 for strict highest score). It lists cheaper peers within the same tolerance and flags efforts with an equal/higher measured score at a lower cost, or a higher score at the same cost. This is a selection-relative cost/performance frontier, not a universal recommendation: latency, task specialization, tool reliability and context remain outside the index. Only one measured effort cannot establish an effort optimum. Estimates and missing task costs never drive advice.
Routing aliases and undisclosed models have no fixed attributable model identity. Specialist endpoints (embedding, audio/image generation, translation, safety and similar) cannot receive a meaningful general-intelligence score just by borrowing a language model's score. They remain in coverage with an explanation and published rates when available. Other unresolved language-model gaps remain researchable rather than receiving an invented universal score.