Test Coverage Matrix
Canonical mapping of every product feature to its test source(s). Drives gap-fill PRs (#967, #968, #969, #970, #971) under epic #773.
Status legend
| Symbol |
Meaning |
| ✅ |
Covered — at least one test asserts the behaviour |
| 🟡 |
Partial — touched by a broader spec, no dedicated assertion |
| ❌ |
Missing — no test today |
| 🚫 |
Not driver-automatable — manual smoke (release-cut checklist, see #971) |
Layer abbreviations
| Code |
Layer |
RU |
Rust unit (sibling *_tests.rs under crates/openhuman-core/src/, or under vendor/ where marked vendored) |
RI |
Rust integration (tests/*.rs) |
VU |
Vitest unit (app/src/**/*.test.ts(x)) |
WD |
WDIO E2E (app/test/e2e/specs/*.spec.ts) — Linux tauri-driver + macOS Appium Mac2 |
PW |
Playwright browser E2E (app/test/playwright/specs/*.spec.ts) |
MS |
Manual smoke (release-cut checklist) |
Update contract — when a PR adds, removes, or changes a feature leaf, the matrix row must be updated in the same PR. Tracking guard: see #965.
0. Application Lifecycle
0.1 Application Download
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 0.1.1 |
Direct Download Access |
MS |
release-manual-smoke (see #971) |
🚫 |
DMG hosting + version landing page |
| 0.1.2 |
Version Compatibility Check |
MS |
release-manual-smoke |
🚫 |
Driver cannot assert OS-version gates |
| 0.1.3 |
Corrupted Installer Handling |
MS |
release-manual-smoke |
🚫 |
Mutated DMG validation; manual repro |
| 0.1.4 |
Download Reward Confirmation |
VU |
app/src/utils/__tests__/desktopDeepLinkListener.test.ts, app/src/services/api/waitlistApi.test.ts |
✅ |
openhuman://waitlist open confirms the tokenmaxxxing download |
0.2 Installation & Launch
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 0.2.1 |
DMG Installation Flow |
MS |
release-manual-smoke |
🚫 |
OS-level Finder drag |
| 0.2.2 |
Gatekeeper Validation |
MS |
release-manual-smoke |
🚫 |
OS-level signature check |
| 0.2.3 |
Code Signing Verification |
MS |
release-manual-smoke |
🚫 |
codesign --verify capture in checklist |
| 0.2.4 |
First Launch Permissions Prompt |
MS |
release-manual-smoke |
🚫 |
TCC prompts non-driver-automatable |
0.3 Updates & Reinstallation
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 0.3.1 |
Auto Update Check |
RU+RI+MS |
crates/openhuman-core/src/platform/update/ (Rust unit), tests/json_rpc_e2e.rs, release smoke |
🟡 |
Core check/update policy covered; desktop prompt + release upgrade still manual |
| 0.3.2 |
Forced Update Handling |
MS |
release-manual-smoke |
🚫 |
End-to-end gating verified at release |
| 0.3.3 |
Reinstall with Existing State |
MS |
release-manual-smoke |
🚫 |
Workspace persistence on reinstall |
| 0.3.4 |
Clean Uninstall |
MS |
release-manual-smoke |
🚫 |
OS removal paths |
1. Authentication & Identity
1.1 Multi-Provider Authentication
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 1.1.1 |
Google Login |
WD |
app/test/e2e/specs/login-flow.spec.ts |
✅ |
Deep-link branch covered |
| 1.1.2 |
GitHub Login |
WD |
login-flow.spec.ts |
✅ |
Deep-link branch covered |
| 1.1.3 |
Twitter (X) Login |
WD |
login-flow.spec.ts |
🟡 |
Generic OAuth path; assert provider tag in #968 |
| 1.1.4 |
Discord Login |
WD |
login-flow.spec.ts |
🟡 |
Same — discord branch unasserted |
1.2 Account Management
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 1.2.1 |
Account Creation & Mapping |
WD+RI |
login-flow.spec.ts, tests/json_rpc_e2e.rs |
✅ |
|
| 1.2.2 |
Multi-Provider Linking |
WD |
missing — tracked #968 |
❌ |
Need spec linking 4 providers to one account |
| 1.2.3 |
Duplicate Account Handling |
WD |
missing — tracked #968 |
❌ |
Collision UX path |
1.3 Session Management
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 1.3.1 |
Token Issuance |
WD+RU+RI |
login-flow.spec.ts, crates/openhuman-tinyhumans/src/session/{client,manager}_tests.rs, json_rpc_e2e.rs |
✅ |
|
| 1.3.2 |
Session Persistence |
WD+RU |
logout-relogin-onboarding.spec.ts, crates/openhuman-core/src/security/credentials/ops_credential_tests.rs |
✅ |
|
| 1.3.3 |
Refresh Token Rotation |
VU |
missing — tracked #968 |
❌ |
Slice-level refresh logic |
| 1.3.4 |
Product Identity Header (x-sdk-name) |
RU |
crates/openhuman-tinyhumans/src/backend/headers_tests.rs, crates/openhuman-tinyhumans/src/transport/transport_tests.rs, crates/openhuman-tinyhumans/src/session/client_tests.rs, crates/openhuman-core/src/integrations/client_tests.rs |
✅ |
#5406 — backend requests carry a sanitized product identity. RU covers the sanitizer, default, and override on the SDK and raw HTTP client paths. |
| 1.3.5 |
Credential handoff to core (auth.set_credential) |
RU+RI |
crates/openhuman-core/src/security/credentials/ops_credential_tests.rs, crates/openhuman-tinyhumans/src/session/link_tests.rs, tests/in_process/config_auth_app_state_connectivity_e2e.rs |
✅ |
The core takes a JWT / API key / local token as given (no /auth/me): expired JWTs and sessions without a user id are refused, same-token/same-user is a cheap refresh, a different user is signed out first, the snapshot reports the handed-over user and never calls the backend. |
1.4 Logout & Revocation
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 1.4.1 |
Session Logout |
RU+WD |
logout-relogin-onboarding.spec.ts, crates/openhuman-core/src/channels/runtime/session_tests.rs, crates/openhuman-core/src/security/credentials/ops_provider_oauth_tests.rs |
✅ |
Logout invalidates channel lifetimes, including pending startup; local inference remains independent of backend login. |
| 1.4.2 |
Global Logout |
WD |
missing — tracked #968 |
❌ |
Multi-session invalidation |
| 1.4.3 |
Token Invalidation |
WD |
missing — tracked #968 |
❌ |
Server-side revocation propagation |
2. Permissions & System Access
2.1 macOS Permissions
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 2.1.1 |
Accessibility Permission |
MS |
release-manual-smoke |
🚫 |
TCC OS-level prompt |
| 2.1.2 |
Input Monitoring Permission |
MS |
release-manual-smoke |
🚫 |
TCC OS-level prompt |
| 2.1.3 |
Microphone Permission |
MS |
release-manual-smoke |
🚫 |
TCC OS-level prompt |
2.2 Permission Lifecycle
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 2.2.1 |
Permission Grant Flow |
RU |
vendor/tinycomputer/crates/tinycomputer-accessibility/ |
🟡 |
Core branch covered; UX manual |
| 2.2.2 |
Permission Denial Handling |
RU |
vendor/tinycomputer/crates/tinycomputer-accessibility/ |
🟡 |
Same |
| 2.2.3 |
Permission Re-Sync / Refresh |
WD |
missing — tracked #968 |
❌ |
App-restart re-sync |
| 2.2.4 |
Partial Permission State Handling |
WD |
missing — tracked #968 |
❌ |
macOS-only spec |
3. Local AI Runtime (Ollama + LM Studio)
Local inference runs against a runtime the user installs, starts, and pulls models into; OpenHuman only calls the endpoint (crates/openhuman-core/src/inference/provider/factory/local_runtime.rs, host bindings in crates/openhuman-core/src/inference/host_runtime/). Rows for model download, runtime install, daemon lifecycle, RAM tiers, and tier presets are kept for catalog continuity and marked Retired.
3.1 Model Management
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 3.1.1 |
Model Detection |
RU+RI |
tests/json_rpc_e2e.rs (json_rpc_local_ai_lm_studio_config_diagnostics_and_prompt), vendor/tinyagents/vendor/tinyinference/crates/tinyinference-local/src/models.rs |
🟡 |
Detection here means listing the models the user's runtime already has (the provider model picker). The old local-model-runtime.spec.ts citation was false: the spec was skipped and now asserts only the legacy route redirect. ⚠️ tinyinference-local is not built by any OpenHuman lane — vendor is in the root [workspace] exclude and no CI step names the package, so its unit tests are cited history, not executed coverage. |
| 3.1.2 |
Model Download & Installation |
— |
retired |
🚫 |
Retired. OpenHuman no longer downloads models, installs runtimes, manages the Ollama daemon, or picks models by RAM tier: the user runs Ollama / LM Studio / MLX / an OpenAI-compatible server, pulls models themselves, and adds the endpoint as a provider. The download RPCs and LocalAIDownloadSnackbar were removed; users run ollama pull <model> (or load the model in LM Studio). |
| 3.1.3 |
Model Version Handling |
RU |
crates/openhuman-core/src/config/schema/types/model_ids.rs |
✅ |
Path corrected (was crates/openhuman-core/src/inference/model_ids.rs). |
| 3.1.4 |
LM Studio Model Discovery |
RU+RI |
vendor/tinyagents/vendor/tinyinference/crates/tinyinference-local/src/service/ollama_admin_tests.rs (LM Studio cases), tests/json_rpc_e2e.rs |
✅ |
Uses LM Studio's OpenAI-compatible /v1/models surface. Path corrected (was crates/openhuman-core/src/inference/local/service/ollama_admin_tests.rs, which moved upstream with the runtime). ⚠️ tinyinference-local is not built by any OpenHuman lane — vendor is in the root [workspace] exclude and no CI step names the package, so its unit tests are cited history, not executed coverage. |
| 3.1.5 |
Model Context-Window Requirement Gate |
RU |
vendor/tinyagents/vendor/tinyinference/crates/tinyinference-local/src/model_requirements.rs, vendor/tinyagents/vendor/tinyinference/crates/tinyinference-local/src/ollama.rs |
🟡 |
Rejects Ollama models whose native context window is below the memory-layer minimum (local_ai.model_context_check). The model is one the user pulled; OpenHuman no longer suggests or pulls a replacement. ⚠️ tinyinference-local is not built by any OpenHuman lane — vendor is in the root [workspace] exclude and no CI step names the package, so its unit tests are cited history, not executed coverage. |
3.2 Runtime Execution
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 3.2.1 |
Local Inference Execution |
RU+RI |
crates/openhuman-core/src/inference/provider/factory_crate_native_tests.rs, crates/openhuman-core/src/inference/host_runtime/ops_tests.rs, crates/openhuman-core/src/flows/ops_agent_binding_tests.rs, crates/openhuman-core/src/flows/host_gates/agent_refs_tests.rs, tests/json_rpc_e2e.rs |
✅ |
Caller-owned model construction and local agent-flow readiness work without an OpenHuman session; cloud session refusal is retained. host_runtime/ops_tests.rs pins the disabled-runtime refusals. |
| 3.2.2 |
Resource Handling (CPU/GPU/Memory) |
— |
retired |
🚫 |
Retired. OpenHuman no longer downloads models, installs runtimes, manages the Ollama daemon, or picks models by RAM tier: the user runs Ollama / LM Studio / MLX / an OpenAI-compatible server, pulls models themselves, and adds the endpoint as a provider. Hardware sizing is the user's runtime's concern; device.rs-driven tier selection and local_ai_presets are no longer product behavior. |
| 3.2.3 |
Runtime Failure Handling |
VU |
app/src/components/settings/panels/__tests__/AIPanel.test.tsx (add-time probe and rollback) |
🟡 |
Daemon lifecycle rows (owned spawn, shutdown, adoption, stale-marker recovery) are retired with Ollama daemon management. What remains is an unreachable user-run runtime: adding the provider probes its model list and rolls the save back on failure (useProviderConnect.ts). The cited tests cover the probe and rollback for key providers; the local-runtime branch has no dedicated assertion yet. |
| 3.2.4 |
LM Studio Chat Completions |
RU+RI |
vendor/tinyagents/vendor/tinyinference/crates/tinyinference-local/src/service/public_infer_tests.rs, tests/json_rpc_e2e.rs |
✅ |
Covers prompt/chat success and non-success status errors. Path corrected (was crates/openhuman-core/src/inference/local/service/public_infer_tests.rs). ⚠️ tinyinference-local is not built by any OpenHuman lane — vendor is in the root [workspace] exclude and no CI step names the package, so its unit tests are cited history, not executed coverage. |
3.3 Runtime Configuration
3.3.1 RAM Allocation Control
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 3.3.1.1 |
RAM Limit Selection |
— |
retired |
🚫 |
Retired. OpenHuman no longer downloads models, installs runtimes, manages the Ollama daemon, or picks models by RAM tier: the user runs Ollama / LM Studio / MLX / an OpenAI-compatible server, pulls models themselves, and adds the endpoint as a provider. There was never a RAM-limit control; the RAM→tier mapping it stood for is gone. |
| 3.3.1.2 |
RAM Availability Detection |
— |
retired |
🚫 |
Retired. OpenHuman no longer downloads models, installs runtimes, manages the Ollama daemon, or picks models by RAM tier: the user runs Ollama / LM Studio / MLX / an OpenAI-compatible server, pulls models themselves, and adds the endpoint as a provider. OpenHuman no longer reads host RAM to gate or size local inference. |
| 3.3.1.3 |
Over-Allocation Prevention |
— |
retired |
🚫 |
Retired. OpenHuman no longer downloads models, installs runtimes, manages the Ollama daemon, or picks models by RAM tier: the user runs Ollama / LM Studio / MLX / an OpenAI-compatible server, pulls models themselves, and adds the endpoint as a provider. The MIN_RAM_GB_FOR_LOCAL_AI floor no longer decides whether local inference is offered. |
| 3.3.1.4 |
Under-Allocation Handling |
— |
retired |
🚫 |
Retired. OpenHuman no longer downloads models, installs runtimes, manages the Ollama daemon, or picks models by RAM tier: the user runs Ollama / LM Studio / MLX / an OpenAI-compatible server, pulls models themselves, and adds the endpoint as a provider. Mirror of 3.3.1.3. |
3.3.2 Dynamic Resource Adjustment
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 3.3.2.1 |
Runtime Scaling Based on Load |
— |
retired |
🚫 |
Never implemented; with the runtime now user-run, OpenHuman has nothing to scale. |
| 3.3.2.2 |
Model Switching Based on Memory |
— |
retired |
🚫 |
Retired. OpenHuman no longer downloads models, installs runtimes, manages the Ollama daemon, or picks models by RAM tier: the user runs Ollama / LM Studio / MLX / an OpenAI-compatible server, pulls models themselves, and adds the endpoint as a provider. RAM → recommend_tier → apply_preset_to_config is gone; the user picks the model. |
3.3.3 Configuration Persistence
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 3.3.3.1 |
Save RAM Settings |
— |
retired |
🚫 |
Retired. OpenHuman no longer downloads models, installs runtimes, manages the Ollama daemon, or picks models by RAM tier: the user runs Ollama / LM Studio / MLX / an OpenAI-compatible server, pulls models themselves, and adds the endpoint as a provider. There is no RAM setting or tier preset to persist. |
| 3.3.3.2 |
Apply on Restart |
— |
retired |
🚫 |
Retired. OpenHuman no longer downloads models, installs runtimes, manages the Ollama daemon, or picks models by RAM tier: the user runs Ollama / LM Studio / MLX / an OpenAI-compatible server, pulls models themselves, and adds the endpoint as a provider. The row tracked RAM/tier settings surviving a restart. Endpoint and provider persistence is covered by 3.3.3.4. |
| 3.3.3.3 |
Reset to Default |
— |
retired |
🚫 |
Retired. OpenHuman no longer downloads models, installs runtimes, manages the Ollama daemon, or picks models by RAM tier: the user runs Ollama / LM Studio / MLX / an OpenAI-compatible server, pulls models themselves, and adds the endpoint as a provider. The reachable default is LocalAiConfig::default() (local AI off); there is no preset tier to reset to. |
| 3.3.3.4 |
Provider Selection Persistence |
RU+RI+VU |
crates/openhuman-core/src/config/ops_tests.rs, tests/json_rpc_e2e.rs, app/src/utils/tauriCommands/config.test.ts |
✅ |
Covers lm_studio normalization and config round-trip |
4. Chat Interface (Core Interaction)
4.1 Chat Sessions
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 4.1.1 |
Session Creation |
WD |
conversations-web-channel-flow.spec.ts |
✅ |
|
| 4.1.2 |
Session Persistence |
WD |
conversations-web-channel-flow.spec.ts |
✅ |
|
| 4.1.3 |
Multi-Session Handling |
RU+WD |
vendor/tinyagents/crates/tinyagents-session/src/threads/ (vendored tinyagents), agent-review.spec.ts, conversations-web-channel-flow.spec.ts |
🟡 |
100 concurrent transcript writers use independent per-thread locks without loss/corruption; no dedicated frontend multi-thread switch test |
4.2 Messaging
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 4.2.1 |
User Message Handling |
WD+RI |
conversations-web-channel-flow.spec.ts, tests/json_rpc_e2e.rs |
✅ |
|
| 4.2.2 |
AI Response Generation |
WD |
agent-review.spec.ts |
✅ |
Mock LLM |
| 4.2.3 |
Streaming Responses |
RI |
tests/json_rpc_e2e.rs, tests/agent_harness_e2e.rs |
✅ |
tests/agent_harness_e2e.rs adds provider-level SSE tool-arg accumulation (chunked args reassembled + parsed) and engine-level delta forwarding (#3471) |
| 4.2.4 |
Parallel inference (cross-thread + within-thread forked turns) |
RU+VU |
crates/openhuman-core/src/web_chat/web_tests.rs, app/src/store/__tests__/chatRuntimeSlice.test.ts, app/src/providers/__tests__/ChatRuntimeProvider.test.tsx |
🟡 |
Concurrent same-/cross-thread dispatch, cooperative CancellationToken teardown, and parallel-lane stream routing covered; dedicated WD E2E is a follow-up |
| 4.2.6 |
Background-activity panel (chat-header Background tasks button) |
VU+WD |
app/src/features/conversations/hooks/useBackgroundActivity.test.ts, app/src/features/conversations/components/__tests__/BackgroundActivityRows.test.tsx, app/test/e2e/specs/chat-background-activity-panel.spec.ts |
✅ |
View-only panel surfacing this chat's async sub-agents + global cron jobs and memory syncing; freshness-only "Syncing now" labeling; E2E opens the panel and asserts its sections render and close |
| 4.2.7 |
Plan-mode review (Approve / Reject / Send-feedback before execute) |
RU+RI+VU |
crates/openhuman-core/src/plan_review/gate.rs, crates/openhuman-core/src/plan_review/tool.rs, crates/openhuman-core/src/plan_review/schemas.rs, tests/json_rpc_e2e.rs, app/src/features/conversations/components/PlanReviewCard.test.tsx, app/src/pages/__tests__/Conversations.render.test.tsx |
✅ |
Interactive turns call request_plan_review, which parks the LIVE turn on the in-memory PlanReviewGate (oneshot) until the user decides; plan_review_request socket event drives PlanReviewCard, which resolves via openhuman.plan_review_decide (approve resumes-and-executes / reject resumes-and-stops / revise resumes-with-feedback). RU covers gate park/resolve/timeout + tool auto-approve + parking; RI covers the decide RPC; VU covers the card + provider wiring. WD E2E (agent-driven park flow) tracked as follow-up |
| 4.2.8 | Composer attachments (image / video / document; drag-drop + paste) | VU | app/src/lib/attachments.test.ts, app/src/components/chat/__tests__/ChatComposer.test.tsx, app/src/pages/__tests__/Conversations.attachments.test.tsx, app/src/features/conversations/components/AssistantUiChat.slots.test.tsx | 🟡 | Attach affordance gated on the resolved vision tier (images/video need vision; documents flow on any model); video is sampled into still frames client-side and forwarded through the existing [IMAGE:] vision path; drag-drop + clipboard-paste reuse the picker ingest. Both composers are covered: the legacy mic-cloud one (ChatComposer.test.tsx) and the shipping assistant-ui surface, where drag-drop and paste route through the host onComposerFiles sink and the composer slots are pinned against the remount that blanked previews and swallowed picks (#6246). VU covers MIME/kind/limits/marker building + drag-drop + paste + slot identity; real video decode and the frames→vision round-trip are manual-smoke only (jsdom has no video codec). WD E2E is a follow-up |
| 4.2.9 | Share cards ("Look what my agent did" post to X / LinkedIn) | VU | app/src/features/share/shareContent.test.ts, app/src/features/share/shareTargets.test.ts, app/src/features/share/shareCaption.test.ts, app/src/features/share/shareCard.test.ts, app/src/features/share/ShareMessageButton.test.tsx, app/src/features/share/ShareCardModal.test.tsx | ✅ | Hover Share button under completed agent messages opens a Canvas 2D branded card; LLM-drafted headline/caption via inference_agent_chat_simple with deterministic offline fallback; X/LinkedIn intent URLs + copy/save PNG; secrets/paths/emails redacted before render. MVP wires only the completed-task trigger; workflow-run + Human-tab triggers are follow-ups. WD E2E deferred |
| 4.2.10 | Durable agent reply (core stores it before announcing chat_done) | RU+VU | crates/openhuman-core/src/web_chat/reply_persistence_tests.rs, crates/openhuman-core/src/web_chat/presentation_tests.rs, app/src/providers/__tests__/ChatRuntimeProvider.test.tsx | ✅ | deliver_response persists an unsegmented reply under agent:<request_id> before publishing the terminal event, so a dropped socket, a failed threads_message_append or a reloaded webview costs a repaint rather than the answer; the client's append collapses onto that row by id. RU covers the write, its idempotency, the empty-reply and missing-thread paths, and that delivery still announces when the store refuses. VU covers the mirrored id, the segmented exception, the refetch that recovers a failed append, and the reconnect that rejoins orphaned thread rooms. WD E2E (kill the append mid-turn) is a follow-up |
| 4.2.11 | Links open in the default browser; the main webview never loads a remote page | RU+VU+MS | crates/openhuman-app/src/external_navigation_tests.rs, app/src/utils/openUrl.test.ts, app/src/utils/externalLinkGuard.test.ts, app/src/components/Announcement/AnnouncementModal.test.tsx, release-manual-smoke | 🟡 | RU covers the shell's navigation policy: remote http(s) navigation of main is cancelled and handed to the OS opener; tauri://localhost, tauri.localhost, about:/data:/blob: and the tauri dev server stay in the webview; other webviews are untouched. VU covers openUrl never falling back to window.open in the desktop shell (including the IPC bridge bootstrap gap) and the click guard. Real macOS click verified manually; automated desktop E2E is a follow-up (the WDIO desktop runner still assumes CEF) |
| 4.2.12 | Live stream renders like the reopened thread (parity, append-only, atomic settle) | VU+WD | app/src/providers/__tests__/liveHistoryParity.test.tsx, app/src/store/__tests__/chatRuntimeSlice.liveParity.test.ts, app/src/features/conversations/derived/mapDisplayItems.test.ts, app/src/services/chatDeltaCoalescer.test.ts, app/src/features/conversations/components/AssistantUiToolCall.disclosure.test.tsx, app/src/components/assistant-ui/elements/reasoning-panel.test.tsx, app/src/components/assistant-ui/thread.followBottom.test.tsx, chat-live-history-parity.spec.ts | ✅ | One scripted turn (app/src/test/fixtures/streamedTurn.ts) is told as socket events and as the core transcript. VU drives it through the real socket handlers: the settled reply equals the reload projection, the tail is append-only while it streams, and chat_done swaps tail for reply in one store update at one index. Also covered: in-place sub-agent promotion on both sides, args-delta row adoption, a superseded settle, frame-coalesced deltas, disclosure state across remounts, and following that survives a collapse. WD samples the real DOM while streaming, checks the viewport still follows, and compares with the reopened thread |
| 4.2.13 | Final-reply remix with region slang and memes (TinyMemes) | — | retired | 🚫 | Retired. TinyMemes was removed from OpenHuman (it was off by default behind OPENHUMAN_TINYMEMES); the reply remix now lives in chottu-core as chottu-personality and is tested there. Old [memes] / components.memes config keys are ignored on load (crates/openhuman-core/src/config/schema/load/parse_tests.rs) |
| 4.2.14 | Tool UI widgets in chat (MCP Apps ui:// and Apps SDK templates) | RU+VU | crates/openhuman-core/src/mcp/ui/resolve_tests.rs, crates/openhuman-core/src/mcp/ui/ops_tests.rs, crates/openhuman-core/src/mcp/ui/decorate_tests.rs, crates/openhuman-core/src/mcp/ui/tools_tests.rs, crates/openhuman-app/src/widget_scheme_tests.rs, app/src/features/conversations/tools/mcpUi/mcpAppBridge.test.ts, app/src/features/conversations/tools/mcpUi/McpAppFrame.test.tsx, app/src/components/assistant-ui/thread.mcpWidgets.test.tsx | 🟡 | RU covers template resolution, resource reads, widget tool-call policy, result decoration and the ohwidget:// proxy scheme. VU covers the bridge, the sandboxed frame, link and QR fallback, and widget placement in the assistant turn. The live widget round-trip against a real MCP server is manual smoke only |
4.3 Tool Invocation
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 4.3.1 |
Tool Trigger via Chat |
WD |
skill-execution-flow.spec.ts, skill-multi-round.spec.ts |
✅ |
|
| 4.3.2 |
Permission-Based Execution |
RU+WD |
crates/openhuman-core/src/tools/, skill-execution-flow.spec.ts |
✅ |
|
| 4.3.3 |
Tool Failure Handling |
WD |
skill-execution-flow.spec.ts |
✅ |
|
| 4.3.4 |
Subagent Mascot Visualization |
VU |
app/src/features/human/SubMascotLayer.test.tsx, app/src/features/human/chatMascot/ChatMascotOverlay.test.tsx |
✅ |
Renders spawned/completed/failed subagent timeline rows as colored companion mascots with activity bubbles |
| 4.3.5 |
Image Tool Contracts |
RU |
crates/openhuman-core/src/media/image/ |
✅ |
High-level image_generation / view_image schema, gating, serialization, prompt guidance, and contract e2e coverage for #2984 |
| 4.3.6 |
Media generation (OpenRouter via backend proxy) |
RU+E2E |
crates/openhuman-core/src/media/generation/tools_tests.rs, tests/media_generation_e2e.rs::media_tools_deliver_images_and_videos_through_the_backend_proxy, scripts/mock-api/routes/__tests__/media.test.mjs |
✅ |
media_generate_image / media_generate_video run TinyInference generators through /agent-integrations/openrouter (envelope, credential, x-sdk-name); a video job reporting completed with no outputs is polled through, not failed; one billed submit per call; local references confined to action/workspace dirs. |
| 4.3.7 |
Mascot Avatar Animation |
VU |
app/src/features/human/Mascot/RiveMascot.test.tsx, app/src/features/human/Mascot/riveMaps.test.ts |
✅ |
Rive MascotSM state machine: face→pose mapping, Oculus→visme_codes viseme normalization, and idle random pose rotation for the tiny_mascot.riv upgrade |
| 4.3.8 |
Tool rules (allow / deny / hide / approval patterns) |
RU |
crates/openhuman-core/src/tools/rules/ops_tests.rs, crates/openhuman-core/src/agent/session_host/runtime_session/tool_rules_tests.rs, crates/openhuman-core/src/agent/tinyagents/turn_plan_tests.rs, crates/openhuman-core/src/agent/tinyagents/host/definition_registry_tests.rs, crates/openhuman-embed/src/agent/definition_tests.rs, vendor/tinyagents/crates/tinyagents-harness/src/agent_loop/tool_rules_tests.rs |
✅ |
[tool_rules] config, definition disallowed_tools/tool_rules, registry entries and AgentDefinitionSpec::tool_rules compose into rule layers; sub-agents inherit operator layers only; the turn harness carries them as RunPolicy::tool_rules, which the TinyAgents loop applies to the catalogue, tool_search and every call (loop behaviour covered upstream) |
4.4 Agent Harness Behaviors
End-to-end coverage of the agent harness via the web-chat RPC surface against an in-test scripted upstream (tests/agent_harness_e2e.rs) plus browser-level Redux/DOM assertions (app/test/e2e/specs/agent-harness-behaviors.spec.ts). Added for #3471.
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 4.4.1 |
Subagent Delegation |
RI+WD |
tests/agent_harness_e2e.rs, app/test/e2e/specs/agent-harness-behaviors.spec.ts |
✅ |
Orchestrator→agent_memory (retrieve_memory) delegation asserts ≥3 upstream calls, distinct subagent persona, no "Unknown tool:" |
| 4.4.2 |
Subagent Clarification (input-required) |
RI+WD |
tests/agent_harness_e2e.rs, app/test/e2e/specs/agent-harness-behaviors.spec.ts, chat-harness-subagent-continue.spec.ts |
✅ |
Subagent question surfaced + reply completes turn; full continue_subagent resume covered by the continue spec |
| 4.4.3 |
Approval Gate — Approve |
RI+WD |
tests/agent_harness_e2e.rs, app/test/e2e/specs/agent-harness-behaviors.spec.ts |
✅ |
approval_request event → approve → tool executes (timeline success, file written) |
| 4.4.4 |
Approval Gate — Deny |
RI+WD |
tests/agent_harness_e2e.rs, app/test/e2e/specs/agent-harness-behaviors.spec.ts |
✅ |
Deny → tool not executed (timeline error, file unchanged), agent explains gracefully |
| 4.4.5 |
Approval Gate — TTL Timeout |
RI |
tests/agent_harness_e2e.rs |
✅ |
OPENHUMAN_APPROVAL_TTL_SECS (debug-only) → no decision → auto-deny |
| 4.4.6 |
Subagent + Approval Gate |
RI |
tests/agent_harness_e2e.rs |
✅ |
Approval fires inside subagent context; decision propagates; artifact content asserted |
| 4.4.7 |
Multi-turn State Persistence |
RI+RU |
tests/agent_harness_e2e.rs, crates/openhuman-core/src/agent/session_host/runtime_adapter_tests.rs::a_cold_resumed_thread_can_send_again, crates/openhuman-core/src/agent/session_host/transcript_compat_tests.rs |
✅ |
Turn-2 request carries turn-1 history; a cold host resumes the committed thread and accepts another turn (#6608). Historical tool results replay after executor retirement; a missing executor recovers as an unknown-tool response without restoring execution authority (#7410) |
| 4.4.8 |
Parallel Subagent Fan-out |
RI |
tests/agent_harness_e2e.rs |
✅ |
spawn_parallel_agents 2-task fan-out; both child results reach synthesis call |
| 4.4.9 |
Max Iterations / Empty Response |
RI |
tests/agent_harness_e2e.rs |
✅ |
Tool-call limit surface + empty-completion graceful terminal (both skip Sentry) |
| 4.4.10 |
Provider Error Retry |
RI |
tests/agent_harness_e2e.rs |
✅ |
First upstream 500 retried by ReliableProvider; second succeeds |
| 4.4.11 |
Inference Phase Transitions |
WD |
app/test/e2e/specs/agent-harness-behaviors.spec.ts |
✅ |
Redux inferenceStatusByThread observes subagent phase then clears to idle |
| 4.4.12 |
Tool Timeline Completeness |
WD |
app/test/e2e/specs/agent-harness-behaviors.spec.ts |
✅ |
Timeline entries carry id/name/status/round; subagent row reaches success; rounds non-decreasing |
| 4.4.13 |
Grounded Close (no final text / breaker halt) |
RU |
crates/openhuman-core/src/agent/session_host/turn_final_reply_grounding_tests.rs, crates/openhuman-core/src/agent/session_host/turn_checkpoint_tests.rs |
✅ |
Tool-records wrap-up, check rejects intent narration / contradicted claims, fallback quotes failure messages; breaker stop note never shown verbatim (#6278, #6279) |
| 4.4.14 |
Session todo list (the todo tool) |
RI+VU+WD |
tests/agent_harness_e2e.rs, crates/openhuman-core/src/agent/tools/todo_tests.rs, vendor/tinyagents/crates/tinyagents-graph/src/todos/{mod_tests.rs,sequence_tests.rs,middleware_tests.rs}, app/src/features/conversations/aui/TodoListPart.test.tsx, app/src/components/assistant-ui/elements/todo-list.test.tsx, app/src/features/conversations/utils/harnessState.test.ts, app/src/features/conversations/components/TodoChecklist.test.tsx, app/src/features/conversations/hooks/useThreadHarnessState.test.ts, app/test/e2e/specs/chat-todos-goals.spec.ts |
✅ |
Whole-list write per call (Claude/Codex shape), steps run one at a time in order (complete/cancel ticks, out-of-order ticks refused, a final answer with open steps held once, an ignored reminder gates other tools), cancelled steps shown as skipped, per-session scoping; five items ticked off across turns, live on the socket, persisted in the turn state, and rendered as the chat pane's checklist |
| 4.4.15 |
Thread goal (goal_set / goal_get / goal_complete) |
RI+RU+VU+WD |
tests/agent_harness_e2e.rs, crates/openhuman-core/src/agent/goals/{tools_tests.rs,runtime_tests.rs,continuation_tests.rs}, vendor/tinyagents/crates/tinyagents-graph/src/goals/test.rs, app/src/features/conversations/utils/harnessState.test.ts, app/src/features/conversations/components/GoalBanner.test.tsx, app/src/features/conversations/hooks/useThreadHarnessState.test.ts, app/test/e2e/specs/chat-todos-goals.spec.ts |
✅ |
Objective + token budget set, read back across turns, and completed; structured {goal, text} payload drives the chat pane's goal banner. Budget accounting and the budget-limit stop hook are unit-covered (the scripted e2e upstream reports no usage) |
| 4.4.16 |
Builder-supplied session definition (library host) |
RU |
crates/openhuman-core/src/agent/session_host/builder/builder_tests_session_definition_tests.rs |
✅ |
SessionHostBuilder::agent_definition stamps a caller's own AgentDefinition on the session, so a hosted root invocation resolves an id the process registry never held; it outranks a registry entry sharing its id, it carries the turn's tool allow-list, the session is stamped with the definition's own id, a contradicting agent_definition_name fails the build, and without it the same id stays unknown |
| 4.4.17 |
Chat failure presentation |
RU+VU |
crates/openhuman-core/src/web_chat/web_tests_rate_limit_classification_tests.rs, app/src/providers/__tests__/assistantUiMessages.test.ts, app/src/features/conversations/aui/ChatErrorNotice.test.tsx, app/src/components/assistant-ui/thread.reloadCapability.test.tsx |
✅ |
Failed turns use assistant-ui's error card; old custom link markup is removed from saved errors (#6609); uncommitted failures and guardrail notices hide invalid regenerate controls while completed replies retain Refresh (#6613) |
| 4.4.18 |
Agent trace export |
RU |
crates/openhuman-core/src/core/observability/trace_export_adoption_tests.rs, crates/openhuman-core/src/agent/subagent_host/ops_tests_trace_export_tests.rs, crates/openhuman-core/src/agent/tinyagents/routes_tests.rs, crates/openhuman-core/src/agent/session_host/runtime_session_tests.rs |
🟡 |
Native span snapshots cover call scoping, child lineage, same-time failure precedence and orphan bounds; host tests cover durable child export and routed generation enrichment. Live proxy canary follows backend deployment |
5. Built-in Intelligence Skills
5.1 Voice Intelligence
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 5.1.1 |
Voice Input Capture |
WD |
voice-mode.spec.ts |
✅ |
|
| 5.1.2 |
Speech-to-Text Processing |
WD |
voice-mode.spec.ts |
✅ |
|
| 5.1.3 |
Voice Command Execution |
WD |
voice-mode.spec.ts |
✅ |
|
| 5.1.4 |
Mascot Voice Selection |
VU |
app/src/store/__tests__/mascotSlice.test.ts, app/src/components/settings/panels/__tests__/VoicePanel.test.tsx, app/src/features/human/useHumanMascot.test.ts |
✅ |
Slice validation + persist REHYDRATE, Settings picker UI (#1762), synthesizeSpeech voiceId override propagation |
| 5.1.5 |
Live Voice Agent Session |
RU+VU+PW |
crates/openhuman-core/src/voice/live/*_tests.rs, crates/openhuman-core/src/agent/tinyagents/live_harness_tests.rs, app/src/features/human/voice/live/useLiveVoiceSession.test.tsx, app/test/playwright/specs/live-voice.spec.ts |
✅ |
/ws/live-voice framing, transcript persistence, tool calls through approval/policy middleware; provider protocols tested in tinyliveagents |
| 5.1.6 |
Live Voice Provider Settings |
RI+VU |
tests/json_rpc_e2e.rs (voice_live_settings_round_trip_over_json_rpc), app/src/components/settings/panels/LiveVoicePanel.test.tsx |
✅ |
Connections → Voice agents: providers, BYOK keys, default provider, test button |
5.2 Persona
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 5.2.1 |
Persona Name & Description |
VU |
app/src/store/personaSlice.test.ts, app/src/components/settings/panels/PersonaPanel.test.tsx |
✅ |
Slice validation + persist REHYDRATE scrub; Settings identity fields persist on save (#2345) |
| 5.2.2 |
SOUL.md Edit & Reset |
RU+VU |
crates/openhuman-core/src/config/workspace/rpc.rs, app/src/components/settings/panels/PersonaPanel.test.tsx |
✅ |
Core read/write/reset with allowlist + size cap; panel loads, saves, resets over RPC (#2345) |
| 5.2.3 |
Persona Settings Surface |
VU |
app/src/components/settings/panels/PersonaPanel.test.tsx |
✅ |
Bundles identity + SOUL.md + link to Mascot avatar/voice (#2345) |
6. System Tools & Agent Capabilities
6.1 File System
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 6.1.1 |
File Read Access |
RU+WD |
vendor/tinyagents/crates/tinyagents-harness-tools/src/fs/filesystem/file_read.rs, app/test/e2e/specs/tool-filesystem-flow.spec.ts |
✅ |
Was 🟡 — WDIO drives memory_read_file + asserts via Node fs. Byte offset paging (continues a paged artifact read, #6284): file-read tests in tinyagents-harness-tools |
| 6.1.2 |
File Write Access |
RU+WD |
vendor/tinyagents/crates/tinyagents-harness-tools/src/fs/filesystem/file_write.rs, app/test/e2e/specs/tool-filesystem-flow.spec.ts |
✅ |
Was 🟡 — WDIO drives memory_write_file + asserts bytes match on disk |
| 6.1.3 |
Path Restriction Enforcement |
RU+WD |
crates/openhuman-core/src/tools/impl/filesystem/gate_tests.rs, app/test/e2e/specs/tool-filesystem-flow.spec.ts |
✅ |
Was 🟡 — WDIO asserts traversal + absolute-path denial envelope |
| 6.1.4 |
Agent deliverables in the visible Files folder |
RU+VU |
crates/openhuman-core/src/artifacts/files_tests.rs, crates/openhuman-core/src/artifacts/migrate_tests.rs, crates/openhuman-core/src/artifacts/ops_tests.rs, crates/openhuman-app/src/artifact_commands_tests.rs, app/src/services/__tests__/artifactDownloadService.test.ts, crates/openhuman-core/src/config/ops_files_dir_tests.rs, app/src/components/settings/panels/__tests__/FilesFolderSection.test.tsx, app/src/features/conversations/aui/MessageArtifactCards.test.tsx, app/src/features/conversations/aui/ArtifactPreviewPanel.refresh.test.tsx |
✅ |
Files land in ~/OpenHuman/projects/Files with human names (create_new, (2) on collision, shared across accounts without clobbering; per-account listing); escape guard on the recorded file; "file missing" for a file removed outside OpenHuman; crash-safe idempotent migration of legacy files; Download resolves an artifact id through the core; the folder is chosen in Settings → Agent OS access and validated (absolute, not protected, not inside the data folder); Show in folder on the settings row (#5505); in chat, each ready file is a card in the reply that made it and opens in the preview panel, whose Download saves it through the OS save dialog (the chat Files panel was removed in #7888) |
6.2 Shell & Git
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 6.2.1 |
Shell Command Execution |
RU+WD |
crates/openhuman-core/src/tools/impl/system/shell.rs, app/test/e2e/specs/tool-shell-git-flow.spec.ts |
✅ |
Was 🟡 — WDIO asserts agent runtime + the shell-bearing agent registry contract (the orchestrator since tools_agent was removed); full LLM path tracked #68 |
| 6.2.2 |
Command Restriction Handling |
RU+WD |
crates/openhuman-core/src/security/policy/policy_tests.rs (+ policy_injection_tests.rs), app/test/e2e/specs/tool-shell-git-flow.spec.ts |
✅ |
Was 🟡 — WDIO locks denial envelope shape { ok:false, error } consumed by the React UI |
| 6.2.3 |
Git Read Operations |
RU+WD |
vendor/tinyagents/crates/tinyagents-harness-tools/src/fs/filesystem/git_operations.rs, app/test/e2e/specs/tool-shell-git-flow.spec.ts |
✅ |
Was 🟡 — WDIO seeds a fixture repo in OPENHUMAN_WORKSPACE and asserts read ops succeed |
| 6.2.4 |
Git Write Operations |
RU+WD |
vendor/tinyagents/crates/tinyagents-harness-tools/src/fs/filesystem/git_operations.rs, app/test/e2e/specs/tool-shell-git-flow.spec.ts |
✅ |
Was 🟡 — WDIO commits into the same fixture and asserts log advances |
6.3 Sub-agent Orchestration
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 6.3.1 |
Steer a running sub-agent |
RU |
crates/openhuman-core/src/agent/orchestration/running_subagents.rs, crates/openhuman-core/src/agent/orchestration/tools/steer_subagent.rs |
✅ |
steer_subagent injects a steer/collect message into a running async sub-agent's run-queue; registry enforces parent ownership + terminal guard. |
| 6.3.2 |
Wait for a sub-agent result |
RU |
crates/openhuman-core/src/agent/orchestration/running_subagents.rs, crates/openhuman-core/src/agent/orchestration/tools/wait_subagent.rs |
✅ |
wait_subagent blocks on the completion watch with a timeout; prunes terminal entries, leaves entries intact on timeout. |
| 6.3.3 |
Steer lands in child history |
RU |
crates/openhuman-core/src/agent/subagent_host/ops_tests_slug_filter_typed_mode_tests.rs::run_queue_steer_lands_in_subagent_history |
🟡 |
Migration coverage: the direct host adapter retains the child queue/history assertion while Phase 6 completes its lifecycle and persistence cutover. |
| 6.3.9 |
Vision sub-agent reads attached images |
RU |
crates/openhuman-core/src/agent_registry/agents/loader_tests_builtin_registration_tests.rs::vision_agent_loads_on_vision_hint, crates/openhuman-core/src/inference/model_context_tests.rs::oh_tier_vision_map_is_exhaustively_pinned, crates/openhuman-core/src/agent/tinyagents/routes_tests.rs::turn_required_capabilities_gates_only_vision, crates/openhuman-core/src/agent/multimodal_attachment_handling_tests.rs::extract_image_placeholders_pulls_att_tokens_in_order |
✅ |
Orchestrator keeps the image as a placeholder, delegates to vision_agent on the vision route, which rehydrates the on-disk attachment and reads it. Engine gate prefers per-tier current_model_vision; turn placeholders forwarded into the sub-agent prompt. |
| 6.3.11 | Async-by-default archetype delegation (durable ref) | RU | crates/openhuman-core/src/agent/orchestration/tools/tools_e2e_tests.rs::archetype_delegation_defaults_to_async_with_durable_session_e2e, crates/openhuman-core/src/agent/orchestration/tools/archetype_delegation.rs::parameters_schema_advertises_async_default_blocking_opt_in | ✅ | delegate_* / build_workflow with a parent turn + chat thread return [async_subagent_ref] (task*id + subagent_session_id) immediately, persist a durable reusable session, and queue the result for background delivery as a new turn; blocking: true opts back into inline dispatch, and no-thread contexts fall back automatically. |
| 6.3.12 | Cross-turn sub-agent roster (durable store merge) | RU | crates/openhuman-core/src/agent/orchestration/running_subagents.rs::snapshot_and_block_scope_to_parent_and_reflect_live_status | ✅ | [active_subagents] merges the in-memory registry with the per-workspace subagent_sessions store, so a cold-booted parent still sees earlier workers (id, status, task title) and resumes instead of re-delegating; Closed sessions excluded, other parents' sessions never leak. |
| 6.3.13 | continue_subagent durable resume (no pause checkpoint) | RU | crates/openhuman-core/src/agent/orchestration/tools/tools_e2e_tests.rs::{continue_subagent_resumes_idle_durable_session_e2e,continue_subagent_without_checkpoint_or_durable_session_names_the_roster} | ✅ | With no ask_user_clarification checkpoint, continue_subagent resolves the durable session by subagent_session_id / task id, resumes it via the reusable async path seeded with persisted history, and keeps the same durable id; missing sessions error with a pointer at the roster. |
| 6.3.14 | Workflow proposal durability (async builder → chat card) | RU+VU | crates/openhuman-core/src/agent/orchestration/tools/spawn_async_subagent.rs::{extract_workflow_proposal_finds_last_proposal_tool_result,attach_workflow_proposal_persists_thread_message_and_extends_summary,attach_workflow_proposal_without_proposal_returns_summary_unchanged}, app/src/lib/workflows/workflowProposal.test.ts | ✅ | A workflow_proposal payload in a finished async child's history is persisted as a parent-thread message (metadata scope workflow_proposal) and embedded in the delivery notice; the frontend rehydrates the newest unconsumed proposal into the card on thread load, and Save/Dismiss mark the source message consumed. |
| 6.3.15 | Background reply persisted once (core-owned id, idempotent append) | RU+VU | crates/openhuman-core/src/agent/orchestration/background_delivery.rs::run_system_turn_on_thread, crates/openhuman-core/src/memory/conversations/store/store_tests.rs::append_message_is_idempotent_by_message_id, crates/openhuman-core/src/web_chat/presentation_tests.rs::single_bubble_delivery_emits_one_unsegmented_chat_done, app/src/providers/__tests__/ChatRuntimeProvider.test.tsx (system chat_done/chat_error reuse the core id), app/src/services/api/threadApi.test.ts (legacy assistant sender folded onto agent), app/src/store/__tests__/threadSlice.test.ts (same-id cache upsert) | ✅ | #5933 — background delivery persists a reply FIRST as agent:<run_id> (sender: agent, extraMetadata.requestId), then announces one unsegmented chat_done; the frontend reuses that id so the id-idempotent store keeps one row. |
| 6.3.16 | Sub-agent spawn/delegate tool refusal (#4452 invariant) | RU | crates/openhuman-core/src/agent/subagent_host/ops/graph_policy_tests.rs::{a_sub_agent_cannot_reach_a_spawn_tool_and_the_healthy_run_is_quiet,an_allowlist_that_readmits_a_spawn_tool_is_refused_loudly}, crates/openhuman-core/src/agent/subagent_host/tool_prep_tests.rs::{dynamic_tools_keep_ordinary_actions_and_lose_spawn_tools,dynamic_tools_lose_unprefixed_delegate_name_overrides,a_dynamic_tool_list_without_spawn_tools_is_untouched} | 🟡 | Migration coverage: the host still fail-closes spawn_subagent/delegate*\*/spawn_worker_threadwhile Phase 6 moves lifecycle ownership to TinyAgents. Completion requires the direct-driver and durable-resume checks in the extraction plan. |
| 6.3.17 | Host-authored turns run on the thread's cached session (no competing root transcript) | RU |crates/openhuman-core/src/web_chat/session_checkout_tests.rs::{checkout_cold_boots_from_the_thread_transcript_and_checkin_keeps_it_warm,checkin_if_vacant_yields_to_a_turn_that_re_cached_meanwhile,a_system_turn_adopts_the_cached_agent_and_its_fingerprint,a_fork_never_takes_or_returns_the_cached_agent}| ✅ | Background delivery and goal continuation go throughweb_chat::run_system_turn_on_thread→checkout_session_agent, so they see the conversation and append to the thread's transcript. A throwaway host bound to the thread used to write a competing root transcript that the next cold-boot resume preferred (newest created), dropping every earlier turn after a restart. checkin_session_agent_if_vacantnever clobbers a user turn that re-cached meanwhile; forks stay isolated. |
| 6.3.18 | Mid-conversation availability notes are status, not instructions | RU |crates/openhuman-core/src/agent/session_host/runtime_adapter_tests.rs::availability_notes_are_status_not_instructions| ✅ |[integration update]/[MCP update]/[skills update]prepended to the next user message no longer say "act on them immediately" (which sent the orchestrator to the integrations agent mid-conversation); they defer to the user's message and only forbid the "reconnect/restart" reply. |
| 6.3.19 |use_skilldispatches packed delegates with the live parent | RU |crates/openhuman-core/src/agent/tinyagents/use_skill_dispatch_tests.rs::use_skill_dispatch_reaches_live_parent_for_packed_archetype_delegate| ✅ |use_skill {skill, tool:"create_image"}routes through the typed delegation dispatch with the real parentRunContext instead of failing with "delegation requires a live harness run context"; disclosure and not-found paths unchanged. |
6.4 Managed Cloud File Storage
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 6.4.1 |
storage_upload_file (multipart, quota/TTL args, path safety) |
RU |
crates/openhuman-core/src/integrations/file_storage/tools_tests.rs |
✅ |
wiremock upload against the backend envelope; rejects workspace-escaping/symlinked paths, bad visibility/ttl; surfaces backend errors (e.g. insufficient balance). |
| 6.4.2 |
storage_download_file (redirect follow + persist to workspace) |
RU |
crates/openhuman-core/src/integrations/file_storage/tools_tests.rs |
✅ |
Follows the 302 to the presigned URL, persists bytes under the action dir, honors explicit filename. |
| 6.4.3 |
storage_list_files / storage_get_link |
RU |
crates/openhuman-core/src/integrations/file_storage/tools_tests.rs |
✅ |
List + usage rendering; presigned link generation with expiry arg validation. |
| 6.4.4 |
storage_set_visibility / storage_delete_file |
RU |
crates/openhuman-core/src/integrations/file_storage/tools_tests.rs |
✅ |
Public/private toggle surfaces the stable public URL; delete confirms backend deleted flag; both are Write-level tools with external effect. |
6.5 Native Desktop Control
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 6.5.1 |
Local connection, permissions, and read-only probe |
RU+VU+MS |
crates/openhuman-core/src/desktop/control/ops_tests.rs, app/src/components/desktop/DesktopConnectionPage.test.tsx |
✅ |
Enablement stays local and off by default; the native module checks Accessibility before observation. OS grants need release smoke. |
| 6.5.2 |
Deferred tool discovery and bounded desktop goals |
RU+MS |
crates/openhuman-core/src/tools/ops_tests_domain_family_tests.rs, crates/openhuman-core/src/desktop/control/tools_tests.rs |
✅ |
Desktop schemas stay behind tool_search; the live core smoke in scripts/debug/desktop-live.mjs needs an unlocked desktop. |
| 6.5.3 |
Desktop approval default and continuation behavior |
RU+VU |
crates/openhuman-core/src/desktop/control/tools_tests.rs, app/src/components/desktop/DesktopConnectionPage.test.tsx |
✅ |
Approval waits are off by default for registered desktop tools; opt-in confirmation uses a one-use, revalidated handle. |
7. Web & Network Capabilities
7.1 Browser
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 7.1.1 |
Open URL |
RU+WD |
crates/openhuman-core/src/tools/impl/browser/browser_open_tests.rs, app/test/e2e/specs/tool-browser-flow.spec.ts |
✅ |
browser_open is a deferred one-shot TinyComputer entry point. |
| 7.1.2 |
Browser Automation |
RU+WD |
crates/openhuman-core/src/tools/impl/browser/browser_computer_tests.rs, app/test/e2e/specs/tool-browser-flow.spec.ts |
✅ |
browser is deferred until tool_search; tests cover task limits, TinyComputer task pauses, session reuse, policy rebinds, and host confirmation. |
| 7.1.3 |
Browser Connections |
VU+RU |
app/src/components/settings/panels/BrowserConnectionsPanel.test.tsx, crates/openhuman-core/src/modules/browser_tests.rs |
✅ |
Readiness, launch settings, and the shared website policy are configured in Connections. |
| 7.1.4 |
Browser site memory |
RU+VU |
crates/openhuman-core/src/modules/site_memory_host_tests.rs, vendor/tinycomputer (site_memory_tests.rs, sites_tests.rs), app/src/components/settings/panels/BrowserConnectionsPanel.test.tsx |
✅ |
A finished task's plan (for the same goal and facts) and the elements it found are kept per site in the workspace and handed to the next task there. Fact values, page text, rescued runs' plans and reused plans that fail are not kept; entries expire after 30 days unused. The learning switch and "forget learned sites" live in Browser settings. |
| 7.1.5 |
Unattended browser actions |
RU |
crates/openhuman-core/src/tools/impl/browser/browser_unattended_tests.rs, crates/openhuman-core/src/config/schema/tools/browser_tests.rs, crates/openhuman-core/src/config/schema/load_env_overlay_context_tests.rs, crates/openhuman-core/src/security/approval/gate_forced_tests.rs |
🟡 |
[browser] unattended_actions lets Cron/Background/approval-free Workflow turns take listed gated actions (and task_step) without the forced gate; WebChat, channels, CLI, Unknown and require_approval workflows stay gated. RU drives BrowserTool::execute under a CoreContext-scoped turn origin for every origin; allowed actions log kind + digest only, unknown names are counted, not echoed. No E2E: the action needs the native TinyComputer module and Chromium, which the test lanes do not carry and must not download. |
7.2 Network
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 7.2.1 |
HTTP / API Requests |
RU+WD |
service-connectivity-flow.spec.ts |
✅ |
|
| 7.2.2 |
Web Search Execution |
WD |
harness-search-tool-flow.spec.ts::S3.2 |
✅ |
Citation corrected 2026-09-24: the row cited skill-execution-flow.spec.ts, whose three tests are login-shell / core.ping / installed-tools and touch no search path. harness-search-tool-flow.spec.ts::S3.2 drives web_search_tool end to end and asserts the final reply cites results; it runs in the webhooks suite |
| 7.2.3 |
TinyFish Integration Tools |
RU |
vendor/tinysearch provider tests, crates/openhuman-core/src/modules/search/config_tests.rs, crates/openhuman-core/src/search/providers_tests.rs |
✅ |
TinyFish is a managed TinySearch provider serving the search and contents roles; host tests cover its managed-only route and usability; request mapping is tested in the module |
| 7.2.4 |
Exa search (managed + BYOK) |
RU+RI+VU |
vendor/tinysearch provider tests, crates/openhuman-core/src/modules/search/config_tests.rs, tests/raw_coverage/worker_b_raw_coverage_e2e.rs::tools_web_search_success_path_uses_backend_session_and_shapes_results, crates/openhuman-core/src/config/ops/search_tests.rs, app/src/components/settings/panels/SearchPanel.test.tsx |
✅ |
Managed Exa posts {objective, searchQueries} (no mode) to /agent-integrations/exa/search through the loaded TinySearch module against a local backend; direct Exa keeps its key private in the module configuration; the panel switches route and saves/clears the key |
| 7.2.5 |
Tavily BYOK search + extract (direct API) |
RU+RI+VU |
vendor/tinysearch provider tests, crates/openhuman-core/src/config/ops_loader_and_search_tests.rs (apply_search_settings_stores_and_clears_tavily_key), tests/in_process/config_auth_app_state_connectivity_e2e.rs, app/src/components/settings/panels/SearchPanel.test.tsx |
✅ |
Tavily serves the search and contents roles with the user's key; the key round-trips through the settings RPC and secret encryption and is never returned; clearing it drops Tavily from effective_roles |
| 7.2.6 |
Multi-provider roles, fallback and grounded answers |
RU+RI+VU+WD |
crates/openhuman-core/src/search/{providers,render,tools,bus}_tests.rs, crates/openhuman-core/src/config/schema/tools/search_search_config_tests_tests.rs, crates/openhuman-core/src/integrations/recorded_tools_tests.rs, crates/openhuman-core/src/mcp/server/tools_tests_params_and_specs_tests.rs, tests/raw_coverage/worker_b_raw_coverage_e2e.rs::tools_web_answer_uses_managed_gemini_grounding_and_returns_citations, app/src/components/settings/panels/SearchPanel.test.tsx, app/src/features/conversations/tools/parseWebSearchResult.test.ts, app/test/e2e/specs/chip-tabs-keyboard.spec.ts |
✅ |
Several providers at once: one agent tool per role (web_search_tool, web_answer_tool, web_contents_tool) served by the first usable provider with fallback (module tests); managed providers need a session and refresh on CredentialChanged; managed Gemini grounding returns an answer with citations end to end; single-engine settings migrate on load and Parallel stays as an own-key-only provider; resumed threads keep recorded role tools; chat renders answers, citations and fallback |
| 7.2.8 |
Backend socket connect deadline + reconnect diagnostics |
RU |
crates/openhuman-core/src/platform/socket/ws_loop_deadline_tests.rs (connect_with_redirects_times_out_when_upgrade_never_answers, connect_timeout_is_not_tighter_than_a_handshake_read, ws_loop_reports_reconnecting_between_attempts, ws_loop_reconnects_after_ping_deadline_on_a_silent_server), crates/openhuman-core/src/core/observability_error_classification_network_tests.rs (classifies_socket_connect_deadline_as_network_unreachable) (#6256) |
✅ |
One connect attempt (DNS + TCP + TLS + upgrade) is bounded at 10 s per redirect hop and fails as the operation timed out transport shape the Sentry classifier demotes; the loop reports reconnecting between attempts and disconnected only when stopped; the ping-timeout warning carries connection age / EIO pings received / frames sent, and a handshake after an outage logs Reconnected after … |
| 7.2.7 |
Connectivity chip — core↔hosted backend link |
VU |
app/src/services/__tests__/coreHealthMonitor.test.ts, app/src/store/__tests__/connectivitySlice.test.ts, app/src/store/__tests__/connectivitySelectors.test.ts, app/src/components/__tests__/ConnectionIndicator.test.tsx (#6256) |
✅ |
connectivity_diag.socket_state + socket_loop_active are mirrored into connectivity.hosted by the health poll; with the loop alive, connecting / reconnecting / disconnected (server closed the namespace) render the amber hosted-degraded chip, ranked below every other channel; socket_loop_active=false or uninitialized collapse to unknown (no chip: signed out, local session, boot) unless socket_loop_stopped_on_failure says the loop exited on an unusable session, which renders hosted-stopped with "sign in again" copy; error (server error event on a live transport) is stored but is not a chip state; a degraded link selects the 5 s poll so recovery shows within seconds |
| 7.2.9 |
Per-agent pinned search keys (embedded hosts) |
RU+RI |
crates/openhuman-core/src/config/schema/tools/search_host_tests.rs, crates/openhuman-core/src/modules/search/config_tests.rs, crates/openhuman-core/src/modules/search/proxy_tests.rs, tests/search_pinned_keys_e2e.rs |
✅ |
Two agents in one process search through the loaded TinySearch module with their own in-memory keys against a loopback provider; each call carries its own key, in turn and concurrently, and no key reaches the environment, a file or a log line |
8. Memory System (Memory v2: Recall, Fetch, Store)
Spec: docs/specs/memory-v2.md (host) and vendor/tinymemory/docs/specs/memory-v2.md (engine contract). Core unit tests sit beside each module in crates/openhuman-core/src/memory/ (*_tests.rs).
8.1 Memory Operations
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 8.1.1 |
Learn / store a learning |
RU+RI+WD |
crates/openhuman-core/src/memory/ops_tests.rs, crates/openhuman-core/src/memory/tools_tests.rs, crates/openhuman-core/src/memory/tool_writes_tests.rs, crates/openhuman-core/src/memory/state_store_tests.rs, tests/memory_v2_e2e.rs (learn_list_fetch_recall_and_forget_round_trip), app/test/playwright/specs/memory-v2.spec.ts |
✅ |
Agent learn returns a durable local queued acknowledgement without awaiting the engine; explicit openhuman.memory_learn retains its RPC contract. Held-engine, scrub-before-persist, state-KV persistence, workspace isolation, capacity, persistence failure, restart replay and invocation-ID deduplication regressions cover the outbox |
| 8.1.2 |
Recall (answer with citations) |
RU+RI+WD |
crates/openhuman-core/src/memory/ops_tests.rs, crates/openhuman-core/src/memory/tool_budget_tests.rs, tests/memory_v2_e2e.rs, app/test/playwright/specs/memory-v2.spec.ts |
✅ |
Ask chip; explicit agent-tool recall is bounded to 15 seconds per call, 30 seconds aggregate and 8 attempts while the bounded run record remains tracked; a timeout disables optional memory for the tracked run (idle records may be evicted under churn) |
| 8.1.3 |
Fetch and list |
RU+RI |
crates/openhuman-core/src/memory/ops_tests.rs, crates/openhuman-core/src/memory/tool_budget_tests.rs, tests/memory_v2_e2e.rs |
✅ |
Hybrid only on both launch engines; other modes answer UNSUPPORTED. Explicit agent-tool fetch shares the bounded recall budget |
| 8.1.4 |
Forget |
RU+RI+WD |
crates/openhuman-core/src/memory/ops_tests.rs, crates/openhuman-core/src/memory/tool_writes_tests.rs, tests/memory_v2_e2e.rs, app/test/playwright/specs/memory-v2.spec.ts |
✅ |
By id; agent-tool forget queues trusted-reach removal, preserves learn-before-forget order and retains failed writes for retry; a queued result does not claim removal |
| 8.1.5 |
Agent memory tool |
RU |
crates/openhuman-core/src/memory/tools_tests.rs, crates/openhuman-core/src/memory/tools_read_tests.rs, crates/openhuman-core/src/memory/tool_budget_tests.rs, crates/openhuman-core/src/memory/tool_writes_tests.rs |
✅ |
Actions recall, fetch, learn, forget; recall returns the engine's evidence (RecallMode::Evidence, no engine-written answer) capped at about 2,500 tokens, cut only at line ends; fetch returns five matches at a time (no limit argument), as dated lines with their ids, plus the engine's next_cursor for the next five of the same search; and recall/fetch leave out the calling thread's turns still verbatim in the prompt (exclude_thread; turns compacted out of the prompt stay readable, and with no known window the whole thread is left out); write acknowledgements distinguish queueing from engine acceptance, budget exhaustion cannot fail chat, and explicit memory disable rejects new writes and pauses pending retries |
| 8.1.6 |
Erase all and queued-write privacy fence |
RU |
crates/openhuman-core/src/memory/ops_tests.rs, crates/openhuman-core/src/memory/tool_writes_tests.rs, crates/openhuman-core/src/memory/lifecycle/prefetch_tests.rs |
✅ |
Explicit erase holds an owner mutation barrier through engine erasure, waits only for current mutations, clears earlier pending tool writes across that owner's team/layout partitions, retires queued automatic turn logs, waits for active automatic work, and preserves later enqueues and other owners. Restart and successive-fence regressions prevent old queued learnings from reappearing. Source/import/belief jobs and legacy post-turn logging retain their existing lifecycle |
8.2 Engines, Ingestion and Context
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 8.2.1 |
Engine list, select, status; memory off |
RU+RI+WD |
crates/openhuman-core/src/memory/engine_tests.rs, crates/openhuman-core/src/memory/status_tests.rs, crates/openhuman-core/src/memory/error_tests.rs, tests/memory_v2_e2e.rs (memory_is_off_when_signed_out, signing_in_turns_memory_on, engines_list_get_and_set, engine_failures_surface_as_structured_errors), app/test/playwright/specs/memory-v2.spec.ts |
✅ |
TinyHumans and CortexDB |
| 8.2.2 |
Per-account isolation |
RU+RI |
crates/openhuman-core/src/memory/tool_writes_tests.rs, crates/openhuman-core/src/memory/lifecycle/prefetch_tests.rs, tests/memory_v2_e2e.rs (memory_is_isolated_per_account) |
✅ |
Shared-workspace outboxes and mutation barriers isolate owners; cached packs isolate identity, config, thread, workspace and core context. Credential changes reject late in-flight results |
| 8.2.3 |
Turn logging and per-turn memory pack |
RU+RI |
crates/openhuman-core/src/memory/lifecycle/hooks_tests.rs, crates/openhuman-core/src/memory/lifecycle/prefetch_tests.rs, crates/openhuman-core/src/agent/tinyagents/memory_summarizer_tests.rs, crates/openhuman-core/src/memory/lifecycle/mod_tests.rs, crates/openhuman-core/src/memory/bus_tests.rs, crates/openhuman-core/src/memory/backfill_tests.rs, tests/memory_v2_e2e.rs (a_chat_turn_carries_its_pack_and_is_logged) |
✅ |
Past chats from before logging are stored by the consented backfill, in bulk with accepted writes (search catches up after the run). Chat and transcript compaction consume only completed, scoped packs; refresh, turn logging, date extraction and compaction recall run in background. Held-engine and slow-refresh regressions prove these hooks do not delay inference or compaction. Packs expire after 60 seconds, are bounded, and stay ephemeral (<memory-context>); tool results are one line, no arguments |
| 8.2.4 |
Background jobs (belief builds) |
RU+RI |
crates/openhuman-core/src/memory/lifecycle/jobs_tests.rs, crates/openhuman-core/src/memory/tool_writes_tests.rs, tests/memory_v2_e2e.rs (brain_pack_preview_and_jobs_round_trip) |
✅ |
memory_background cron, every 5 minutes; tool-write outboxes drain immediately and retry on credential/cron events after restart. Failures retain the FIFO head; background mutation deadlines allow up to 120 seconds without holding a chat turn |
| 8.2.5 |
Document sources: add, sync, list, remove |
RU+RI+WD |
crates/openhuman-core/src/memory/sources/sync_tests.rs, crates/openhuman-core/src/memory/sources/mod_tests.rs, tests/memory_v2_e2e.rs (sources_add_sync_list_and_remove), app/test/playwright/specs/memory-v2.spec.ts |
✅ |
folder, file, link, github, rss |
| 8.2.6 |
Folder-source path picker |
RU+VU |
crates/openhuman-app/src/directory_picker.rs, app/src/utils/tauriCommands/directoryPicker.test.ts |
✅ |
|
| 8.2.7 |
Pack preview (Ask tab) and brain views |
RU+RI+WD |
crates/openhuman-core/src/memory/lifecycle/views_tests.rs, crates/openhuman-core/src/memory/brain_tests.rs, tests/memory_v2_e2e.rs, app/test/playwright/specs/memory-v2.spec.ts |
✅ |
No context.md; pack never persisted in the transcript |
| 8.2.8 |
One-time import of previous (v1) memory |
RU+RI+WD |
crates/openhuman-core/src/memory/import_tests.rs, tests/memory_v2_e2e.rs, app/test/playwright/specs/memory-v2.spec.ts (importing previous memory needs explicit consent, a stopped import resumes only after consent), app/src/components/memory/MemoryImportBanner.test.tsx |
✅ |
Refused without consent; stops (not skips) on exhausted credits (exhausted_credits_stop_the_import_instead_of_skipping_everything) and resumes on its own only while memory work is free, per billing::free_period_active (an_import_out_of_credits_resumes_only_once_automatic_runs_are_allowed); batches are stored accepted and the last waited for once before Done (batches_are_stored_accepted_and_the_last_waited_for_once), a wait that never sees the listing still finishes Done (a_last_batch_never_listed_still_finishes_the_import); an unavailable engine is retried in-run (a_transient_failure_is_retried_within_the_run) and then resumed by the background job (an_engine_that_stays_unavailable_is_resumed_by_the_background_job); refused items are kept and retried (a_refused_item_is_kept_and_a_retry_stores_it); state written atomically and a resume keeps its total (the_import_state_is_written_whole_and_leaves_no_staging_file, a_resumed_import_keeps_its_total_instead_of_rescanning); the background job resumes an import the app quit during (the_background_job_resumes_an_interrupted_import), never while paused, including a pause mid-run (nothing_resumes_while_background_work_is_paused, a_pause_that_lands_after_the_check_stops_the_run_at_the_next_batch); a resume that cannot start is stopped, not retried (an_automatic_resume_that_cannot_start_is_stopped_not_retried); Resume button after an error, behind consent (a stopped import resumes only after consent) |
| 8.2.9 |
MCP memory tools |
RU |
crates/openhuman-core/src/mcp/server/tools/ |
✅ |
memory.recall, memory.fetch, memory.list, memory.learn, memory.forget |
| 8.2.10 |
Memory UI chips and legacy redirects |
VU+WD |
app/src/components/memory/*.test.tsx, app/test/playwright/specs/memory-v2.spec.ts |
✅ |
Engine, Ask, Explorer, Learnings, Conversations, Brain, Background, Settings; /brain and /settings/memory-engine redirect |
| 8.2.11 |
Move into the per-user layout (v3) |
RU+VU+WD |
crates/openhuman-core/src/memory/layout_migration/*_tests.rs, app/src/components/memory/MemoryImportBanner.test.tsx, app/test/playwright/specs/memory-v2.spec.ts (migrate now moves the legacy memory and the banner goes away, a tree other accounts may share is taken only after consent) |
✅ |
Items are stored again and verified, never copied as events; no switch before a full copy, then a catch-up copy; a source is removed only after its twin is verified (an_item_without_its_twin_is_never_removed), by kind-scope erasure or by id where the engine cannot erase, and only by id after an import whose last batch was not confirmed listed (after_an_unconfirmed_import_moved_items_are_forgotten_by_id_never_erased), which is then copied again until a pass moves nothing new, so an item listed late is still moved (after_an_unconfirmed_import_an_item_listed_late_is_still_moved); resumable after a pause or crash; runs on its own only while moving is free; a shared self-hosted tree moves only with consent and only one account may claim it (a_tree_another_account_took_is_left_alone, the_first_account_takes_the_tree_and_keeps_it); items are placed in the v3 layout before the setting switches (places_into_the_v3_layout_before_the_switch); signed out there is nothing to move; never starts while an import of old local memory is unfinished, counting a reserved import before its first state write (nothing_starts_while_an_import_is_unfinished, an_import_counts_as_in_progress_from_its_reservation), and starts on its own once the import is done, free period or not (a_finished_import_starts_organizing_into_the_per_user_tree); one banner shows import then organize (does not organize while the import is still running) |
8.3 TinyMemory Engine Contract
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 8.3.1 |
Conformance suite |
RU |
vendor/tinymemory/crates/tinymemory-api/src/conformance/ |
✅ |
Reference engine and both CortexDB wires through an HTTP double |
| 8.3.2 |
Sources, documents, safety, context, import crates |
RU |
vendor/tinymemory/crates/tinymemory-tools/src/, vendor/tinymemory/crates/tinymemory-integrations/src/ |
✅ |
SSRF guard, scrubbing, budgeted context, legacy reader |
8.4 Chat Thread Store (not memory)
The thread JSONL store moved to tinyagents_session::threads (vendor/tinyagents); the host wrapper is crates/openhuman-core/src/threads/store/. See 4.1.
9. Automation Engine
9.1 Task Scheduling
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 9.1.1 |
Task Creation |
PW |
app/test/playwright/specs/cron-jobs-flow.spec.ts |
✅ |
Verifies an actual cron row and persisted state. |
| 9.1.2 |
Task Update |
PW |
app/test/playwright/specs/cron-jobs-flow.spec.ts |
✅ |
Pause and resume persist across refresh. |
| 9.1.3 |
Task Deletion |
PW |
app/test/playwright/specs/cron-jobs-flow.spec.ts |
✅ |
Remove action clears the row and RPC state. |
9.2 Cron Jobs
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 9.2.1 |
Cron Expression Validation |
RU |
crates/openhuman-core/src/cron/ |
✅ |
Incl. the 5-minute agent-job floor (validate_agent_schedule, #6158) |
| 9.2.2 |
Recurring Execution |
WD+RI |
cron-jobs-flow.spec.ts |
✅ |
|
| 9.2.3 |
Agent Job Transcript Isolation |
RU |
crates/openhuman-core/src/cron/scheduler_transcript_isolation_tests.rs, agent/session_host/runtime_adapter_tests.rs |
✅ |
Cron orchestration suppresses transcript autoload; the session host consumes that override as ResumeMode::Never before dispatch. |
9.3 Remote Execution
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 9.3.1 |
Remote Agent Scheduling |
RI |
tests/in_process/domain_modules_e2e.rs::cron_update_applies_a_partial_patch_without_clobbering_unset_fields (CRUD); manual execution + history (cron_run/cron_runs) in tests/raw_coverage/automation_scheduling_e2e.rs::cron_run_records_history_and_remove_is_not_idempotent (live again: restored from the #6382 quarantine) |
🟡 |
Evidence corrected 2026-09-24; updated when the file was restored. This row cited tests/json_rpc_e2e.rs, which contains zero occurrences of cron (verified with a positive control: 425 openhuman. matches in the same file). So "Coverage thin" was assessed against a file with no scheduling coverage at all. What is live is CRUD plus a manual cron_run with its history; nothing live asserts a scheduled agent job actually RUNS on its schedule (the scheduler tick), which is why this stays 🟡. |
| 9.3.2 |
Execution Trigger |
WD |
cron-jobs-flow.spec.ts |
✅ |
|
| 9.3.3 |
Retry Handling |
RU |
crates/openhuman-core/src/cron/ |
🟡 |
Backoff branches partial |
9.4 Workflow Run History
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 9.4.1 |
Aggregate run history (all workflows) |
RU+VU |
crates/openhuman-core/src/flows/ops_tests.rs::flows_list_all_runs_aggregates_across_flows_newest_first, app/src/pages/WorkflowRunsPage.test.tsx |
✅ |
New openhuman.flows_list_all_runs RPC (store::list_all_flow_runs) backs the Workflows page "Workflow Runs" sub-view. RU asserts cross-flow, newest-first aggregation; VU asserts the list/empty/error render. |
9.5 Flow Pre-authorization (save+enable approval card)
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 9.5.1 |
Approval manifest computation |
RU |
crates/openhuman-core/src/flows/ops_approval_manifest_and_catalog_tests.rs::{approval_manifest_lists_gated_nodes_and_skips_curated_reads,approval_manifest_marks_blocked_classes_under_readonly_tier,approval_manifest_dedupes_repeated_tools_and_flags_dynamic_slugs,approval_manifest_discloses_agent_ref_nodes_only}, tests/tinyflows_published_prompt_e2e.rs::published_module_public_import_uses_authoring_bus_in_a_cold_process |
✅ |
compute_approval_manifest mirrors runtime gating: curated Composio reads excluded, tier-Blocked classes informational, trust-key dedupe, dynamic = slugs and agent_ref nodes disclosed. The published-module E2E also verifies Catalog.ApprovalManifest migration and host classification over TinyBus. flows.approval_manifest RPC joins against existing flow_tool_trust. |
| 9.5.2 |
Flow trust store (list / delete / preauth audit) |
RU |
crates/openhuman-core/src/security/approval/store.rs::tests::{list_flow_trust_returns_sorted_grants_scoped_to_flow,delete_flow_trust_named_removes_only_named_grants,delete_flow_trust_all_purges_only_that_flow,record_flow_preauthorization_lands_in_audit_not_pending} |
✅ |
Batch pre-grants via approval.preauthorize_flow are idempotent, flow-scoped, and each new grant lands as a born-decided approve_always_for_flow audit row (Approval history, never list_pending). Flow deletion purges the flow's grants. |
| 9.5.3 |
Consolidated Approve-all/Deny card + enable flow |
VU |
app/src/hooks/useFlowPreauthorization.test.ts, app/src/components/flows/FlowPreauthorizationCard.test.tsx |
✅ |
Hook: direct enable when nothing missing, grant-then-enable ordering on Approve all, Deny leaves the flow saved-but-disabled, fail-open on manifest RPC errors. Card: exactly two actions, per-kind rows (approvable/blocked/dynamic/agent), busy + error states. |
10. Unified Messaging Hub
10.1 Integration Setup
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 10.1.1 |
Telegram Connection |
WD |
telegram-flow.spec.ts |
✅ |
|
| 10.1.2 |
WhatsApp Connection |
WD |
app/test/e2e/specs/whatsapp-flow.spec.ts |
🚫 |
Corrected 2026-09-24 — this row was ✅ on evidence that does not support it. WhatsApp has no entry in all_channel_definitions(), is not a Composio toolkit, and its provider sits behind the non-default whatsapp-web feature, which is absent from scripts/ci/product-features.txt — so there is no WhatsApp connect flow in the shipped build to test. The cited whatsapp-flow.spec.ts correctly asserts that the retired /accounts route redirects; it never connects anything. Manual-only until the feature ships |
| 10.1.3 |
Gmail Connection |
WD |
gmail-flow.spec.ts |
✅ |
|
| 10.1.4 |
Slack Connection |
WD |
app/test/e2e/specs/connector-composio-contract.spec.ts (Slack row) |
✅ |
Evidence corrected 2026-09-24: Slack is a Composio toolkit, not a native channel — it has no entry in all_channel_definitions(), so channels_connect cannot reach it. Real coverage is the 10-case connector contract (connector-contract.ts), which runs Slack in the gated connectors suite. The previously cited slack-flow.spec.ts is a 26-line surface smoke whose run() call is commented out at e2e-run-all-flows.sh:343, so no lane executes it Was ❌ |
| 10.1.5 |
Yuanbao Connection |
RU+WD |
vendor/tinychannels/src/providers/yuanbao/ (vendored tinychannels), crates/openhuman-core/src/channels/controllers/ops_yuanbao_email_tests.rs::connect_yuanbao_*, crates/openhuman-core/src/channels/controllers/ops_connect_status_tests.rs::connect_yuanbao_rejects_invalid_credentials, crates/openhuman-core/src/channels/runtime/startup_yuanbao_secret_tests_tests.rs, app/test/e2e/specs/credential-channels-flow.spec.ts |
✅ |
New API-key channel for Tencent Yuanbao. RU covers sign-token preflight (valid/invalid creds, env-override cluster routing), credentials store hydration (incl. stale app_key guard), and WS reconnect/shutdown. WDIO connect-flow added 2026-09-24 (credential-channels-flow.spec.ts, suite providers): definition presence in channels_list, the api_key field set channels_describe serves, connect/status/disconnect round trip, and rejection of credentials missing a required field. Previously: no WDIO spec — connect-flow UI is rendered via the generic ChannelSetupModal already exercised by other channel flow specs. |
| 10.1.6 |
Email (IMAP/SMTP) Connection |
RU+VU+WD |
crates/openhuman-core/src/channels/controllers/ops/connect_email_config_tests_tests.rs, crates/openhuman-core/src/channels/controllers/ops_yuanbao_email_tests.rs::{persist_email_config_*,disconnect_email_*,connect_email_rejects_invalid_port_*,test_channel_email_rejects_invalid_port_*}, app/src/components/channels/CredentialChannelConfig.test.tsx, app/src/components/channels/ChannelConfigPanel.test.tsx, app/test/e2e/specs/credential-channels-flow.spec.ts |
✅ |
#4280 — native IMAP/SMTP for non-Gmail/Outlook mailboxes surfacing the existing EmailChannel. RU covers credentials→EmailConfig mapping/defaults, port/sender parsing, definition/validation, config persist + disconnect, and pre-network invalid-port rejection. VU covers the connect form rendering/submit + panel routing. WDIO connect-flow added 2026-09-24 (credential-channels-flow.spec.ts). Live IMAP verify remains a follow-up and is deliberately out of scope for E2E — it would be a real third-party call. |
10.2 Authentication & Authorization
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 10.2.1 |
OAuth / API Token Handling |
WD |
skill-oauth.spec.ts |
✅ |
|
| 10.2.2 |
Scope Selection (Read/Write/Initiate) |
WD |
gmail-flow.spec.ts, skill-oauth.spec.ts, composio-triggers-flow.spec.ts |
🟡 |
Multi-scope matrix not exhaustive; Gmail trigger OAuth read scope covered |
| 10.2.3 |
Token Storage & Encryption |
RU |
crates/openhuman-core/src/security/encryption/, crates/openhuman-core/src/security/credentials/ |
✅ |
|
10.3 Message Sync & Ingestion
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 10.3.1 |
Incoming Message Sync |
RU+WD |
crates/openhuman-core/src/channels/tests/, gmail-flow.spec.ts |
✅ |
|
| 10.3.2 |
Message Deduplication |
RU |
crates/openhuman-core/src/channels/tests/ |
✅ |
|
| 10.3.4 |
Real-Time vs Delayed Sync |
RU |
crates/openhuman-core/src/channels/tests/runtime_dispatch.rs |
✅ |
|
| 10.3.5 |
Inbound Event Queues (Composio triggers, agent webhooks) |
RU+RI |
crates/openhuman-core/src/integrations/composio/bus/trigger_subscriber_tests.rs, crates/openhuman-core/src/skills/webhooks/bus_tests.rs, crates/openhuman-core/src/inbound/, tests/components_e2e.rs, tests/inbound_module_e2e.rs |
✅ |
TinyEventQueues owns retry, deduplication, and delivery behavior. inbound_module_e2e exercises host admission against the staged digest-verified module and asserts that an accepted trigger is persisted before acknowledgment. inbound.* RPC answers COMPONENT_DISABLED:eventqueues when off. Host callbacks require the verified local-file source; linked factories are not eligible for them. |
| 10.3.6 |
Inbound external sources (inbound.offer, pinned agent turns, inbound.results / inbound.result_ack) |
RU |
crates/openhuman-core/src/inbound/ (*_tests.rs beside pinned.rs, results.rs, events.rs, settings.rs, ops.rs, schemas.rs), crates/openhuman-core/src/config/schema/inbound_sources_tests.rs, crates/openhuman-core/src/triage/origin_tests.rs |
✅ |
Against the real tinyeventqueues module linked into the test binary: three offers inside the debounce window become one turn on the configured agent, one stored result and one InboundTurnCompleted; max_batch splits a larger batch; a prompt-injection event is left out of the turn and listed as blocked; an all-blocked batch makes no model call; a completed turn is not re-run; an unconfigured or reserved source is refused; the turn runs under the parked-approval external origin |
10.4 Messaging Operations
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 10.4.1 |
Send Message |
WD+RI |
gmail-flow.spec.ts, telegram-flow.spec.ts |
✅ |
|
| 10.4.2 |
Reply to Thread |
WD |
gmail-flow.spec.ts |
✅ |
|
| 10.4.3 |
Initiate Conversation |
WD |
gmail-flow.spec.ts |
🟡 |
Telegram/WhatsApp/Slack not exercised |
| 10.4.4 |
Attachment Handling |
WD |
gmail-flow.spec.ts |
🟡 |
Attachment branch shallow |
| 10.4.5 |
Outbound Message Edit |
RU |
crates/openhuman-tinyhumans/src/client/backend_channel_tests.rs, crates/openhuman-core/src/channels/bus_tests.rs |
🟡 |
#5230 — drives the progressive draft + ephemeral "💭 Thinking:" bubble. RU covers 404 classification (route absence vs deleted message), the message-id retention invariant each recovery depends on, and the per-provider capability latch. The backend implements no PATCH /channels/:channel/messages/:messageId, so edits degrade (post-once + replace) instead of updating in place; live-edit coverage needs that route first — separate tinyhumansai/backend PR. |
10.5 Cross-Channel Behavior
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 10.5.1 |
Channel Isolation |
RU |
crates/openhuman-core/src/channels/tests/identity.rs |
✅ |
|
| 10.5.2 |
Unified Inbox Handling |
WD+RI |
channels-smoke.spec.ts, tests/in_process/worker_c_modules_e2e.rs |
🟡 |
UI assertion shallow; RI covers config-only channel status after connect/disconnect |
| 10.5.3 |
Context Preservation |
RU |
crates/openhuman-core/src/channels/tests/context.rs |
✅ |
|
10.6 Permission Enforcement
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 10.6.1 |
Read Access Enforcement |
RU+WD |
auth-access-control.spec.ts |
✅ |
|
| 10.6.2 |
Write Access Enforcement |
RU+WD |
auth-access-control.spec.ts |
✅ |
|
| 10.6.3 |
Initiate Action Enforcement |
RU |
crates/openhuman-core/src/channels/ |
🟡 |
E2E thin |
10.7 Disconnect & Re-Setup
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 10.7.1 |
Integration Disconnect |
WD+RI |
gmail-flow.spec.ts, tests/in_process/worker_c_modules_e2e.rs |
✅ |
RI covers channels_disconnect clearing config-only iMessage state |
| 10.7.2 |
Token Revocation |
RU |
crates/openhuman-core/src/security/credentials/ |
✅ |
|
| 10.7.3 |
Re-Authorization Flow |
WD |
connector-contract.ts (all contract toolkits) |
✅ |
Re-auth post-revoke asserted 2026-09-24 in connector-contract.ts ('pressing Reconnect after expiry re-authorizes and restores the connection'): the expired modal's Reconnect button is clicked, a POST to /composio/authorize must follow, and the connection must read ACTIVE afterwards. Inherited by all 11 contract toolkits plus GitHub/Jira/Gmail/Discord |
| 10.7.4 |
Permission Re-Sync |
WD |
missing — tracked #968 |
❌ |
|
11. Intelligence & Insights
11.1 Analysis Engine
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 11.1.1 |
Multi-Source Analysis |
RI |
tests/memory_v2_e2e.rs (sources_add_sync_list_and_remove) |
✅ |
Sources sync into the selected memory engine; the memory graph was removed in v2 |
| 11.1.2 |
Actionable Item Extraction |
VU |
app/src/components/intelligence/__tests__/utils.test.ts |
✅ |
Was ❌ |
| 11.1.3 |
Analyze Trigger |
WD |
app/test/e2e/specs/insights-dashboard.spec.ts mounts the route; explicit analyze-handler invocation TBD |
🟡 |
Route mounts and search/filter UI assert — full analyze trigger flow tracked as follow-up |
| 11.1.4 |
MCP server (stdio + HTTP) |
RU |
crates/openhuman-core/src/mcp/server/ |
✅ |
Stdio framing plus Streamable HTTP/SSE session lifecycle; McpHttpClient round-trip tests |
| 11.1.5 |
Global tool registry |
RI |
crates/openhuman-core/src/tools/registry/, tests/json_rpc_e2e.rs, tests/in_process/domain_modules_e2e.rs, tests/in_process/worker_b_domain_e2e.rs |
✅ |
Read-only MCP/controller discovery with routes, schemas, version, allowed agents, and health |
| 11.1.6 |
SearXNG MCP search |
RU |
crates/openhuman-core/src/search/tools/searxng.rs, crates/openhuman-core/src/mcp/server/tools/, crates/openhuman-core/src/tools/schemas.rs |
✅ |
Self-hosted search config, normalized results, MCP argument validation, and mocked HTTP execution |
| 11.1.7 |
Bundled prompt resources |
RU |
crates/openhuman-core/src/mcp/server/resources.rs, crates/openhuman-core/src/mcp/server/protocol.rs |
✅ |
resources/list catalog + resources/read happy path, -32002 unknown URI, -32602 missing param, catalog-mirrors-BUILTINS parity test |
| 11.1.8 |
Resource templates list |
RU |
crates/openhuman-core/src/mcp/server/resources.rs, crates/openhuman-core/src/mcp/server/protocol.rs |
✅ |
resources/templates/list returns {resourceTemplates: []} (static catalog), tolerates unknown/cursor params |
| 11.1.10 |
MCP declare (mcp.json)→connect→tool_call |
RI |
tests/json_rpc_e2e.rs (mcp_clients_declare_connect_tool_call_happy_path, mcp_clients_config_round_trip_merges_credentials_and_removes_absent_servers), tests/mcp_registry_e2e.rs, tests/agent_harness_e2e.rs (agent_calls_a_tool_on_an_mcp_server_installed_from_the_registry), tests/skills_mcp_registry_live.rs (manual, #[ignore]), crates/openhuman-core/src/mcp/registry/config_doc_tests.rs |
✅ |
HTTP-RPC happy path config_set→connect→tool_call→update_env against test-mcp-stub; the mcp.json round trip (write-only credentials merged on re-save, a read carries names only, an entry dropped from the document is uninstalled, a refusal names the entry and field); registry search→declare→connect over RPC, then a scripted agent turn calls the server's tool and its answer reaches the model; live official-registry smoke (manual) |
| 11.1.11 |
MCP env reconfigure + registry creds |
RI/VU |
tests/json_rpc_e2e.rs (mcp_clients_registry_settings_roundtrip), crates/openhuman-core/src/mcp/registry/ops.rs (mcp_clients_registry_settings_get), vendor/tinymcp/crates/tinymcp/src/registry/sources/official/ (vendored tinymcp), app/src/components/channels/mcp/InstalledServerDetail.test.tsx (#3039) |
✅ |
update_env persist+reconnect; registry_settings get/set with secrets write-only (config-first, env-fallback); reconfigure form validation |
| 11.1.12 |
MCP UI surface (Servers / mcp.json / Registry tabs) |
VU/RU |
app/src/components/channels/mcp/McpServersPage.test.tsx, app/src/components/channels/mcp/McpJsonEditor.test.tsx, app/src/components/channels/mcp/mcpJson.test.ts, app/src/components/channels/mcp/McpRegistryBrowser.test.tsx, app/src/components/channels/mcp/registryAdd.test.ts, app/src/services/api/mcpClientsApi.test.ts, crates/openhuman-core/src/mcp/registry/schemas_tests.rs, vendor/tinymcp/crates/tinymcp/src/registry/{curation,sources}/ (vendored tinymcp) |
✅ |
Connections → MCP Servers owns its page header with three tabs in it, after the OpenCompany MCP page: the server rows (icon, name → detail, transport and status, the command or endpoint, icon controls for connect/disconnect, enable/disable, remove), the mcp.json editor (local shape check, core refusal shown verbatim, Save/Revert, re-reads the rows after a save) and the Registry (one-click add for hosted servers that need no setup, declared into mcp.json then connected or signed in; any other row opens the server's own page); curated catalog with official-vendor badge + official-first order; namespace-stripped relevance search + server-side Stdio/Hosted transport filter |
| 11.1.17 |
MCP server tools as mcp_<server>_<tool> agent tools + persistent tool cache |
RU/RI |
vendor/tinymcp/crates/tinymcp/src/tools/ (vendored tinymcp, tools feature), crates/openhuman-core/src/mcp/registry/action_tool_tests.rs, crates/openhuman-core/src/tools/impl/network/mcp_server_tools_tests.rs, crates/openhuman-core/src/agent/session_host/runtime_session_tests.rs::connected_mcp_actions_enter_search_and_leave_on_disconnect, crates/openhuman-embed/src/agent/build_tests.rs, tests/agent_harness_e2e.rs (orchestrator_calls_a_connected_mcp_tool_directly) |
✅ |
Each remote tool is its own agent tool, deferred behind tool_search by default or direct via expose/direct_tools; installed and configured servers' tool lists persist in tinymcp's mcp_tool_cache (fingerprinted by definition, dropped on disable/uninstall/credential change) so tools are offered before reconnect while calls still require a live connection; recorded pre-readable hashed names are restored as aliases on resume; configured-server output is scrubbed of credentials |
| 11.1.13 |
MCP HTTP-remote auth (token / Bearer / OAuth) + redirect resolution |
RU/VU |
vendor/tinymcp/crates/tinymcp/src/registry/connections/ (build_http_auth*, resolve_final_url), vendor/tinymcp/crates/tinymcp/src/registry/oauth/ (PKCE/token/bundle/callback port) (vendored tinymcp), app/src/components/channels/mcp/ConnectAuthModal.test.tsx (#3495) |
✅ |
Bearer/raw scheme + custom headers; redirect-final-URL resolved before auth; OAuth dynamic client registration + PKCE + refresh; tokens MERGED into stored env; credentials stored encrypted locally, never sent to backend |
| 11.1.15 |
Agent uses connected MCP servers in chat |
RU |
crates/openhuman-core/src/agent_registry/agents/loader.rs (orchestrator_does_not_delegate_mcp_calls, orchestrator_reaches_mcp_directly_and_skills_through_hand_offs, planner_has_readonly_mcp_discovery_not_execute), crates/openhuman-core/src/agent_registry/agents/orchestrator/prompt.rs (connected_mcp_block_*), crates/openhuman-core/src/agent/session_host/turn_tests.rs (mcp_announcement_fires_once_for_new_server), tests/agent_harness_e2e.rs (orchestrator_calls_a_connected_mcp_tool_directly: tool_search → deferred MCP action via tool_call), crates/openhuman-core/src/mcp/registry/action_tool_tests.rs, crates/openhuman-core/src/agent/session_host/runtime_session_tests.rs (connected_mcp_actions_enter_search_and_leave_on_disconnect), crates/openhuman-core/src/mcp/registry/tools_tests.rs, vendor/tinymcp/crates/tinymcp/src/registry/connections/ (vendored tinymcp) (#3495) |
✅ |
connected MCP actions enter the JEV-backed tool_search catalogue as deferred tools and call directly; generic mcp_registry_list_tools / mcp_registry_tool_call remain a fallback; orchestrator ## Connected MCP Servers prompt block + mid-session connect announcement on the user turn; planner read-only MCP discovery (no tool_call) |
| 11.1.16 |
MCP supervisor observability (Event Log + notifications) |
RU |
crates/openhuman-core/src/mcp/registry/supervisor_events_tests.rs, crates/openhuman-core/src/core/events_tests.rs (mcp_supervisor_events_*, mcp_lifecycle_events_*), crates/openhuman-core/src/desktop/notifications/bus_tests_2_tests.rs (mcp_*, *_workspace*, an_outage_is_filed_under_its_own_workspace_not_the_bridge_s, *_announced), app/src/components/settings/panels/__tests__/EventLogPanel.test.tsx (detail line + filter), vendor/tinymcp/crates/tinymcp/src/registry/supervisor/test.rs ("The report" section) (#5931) |
✅ |
tinymcp::Supervisor::tick returns a TickReport; the host maps it to McpServer{ProbeTimedOut,TransportDropped,Reconnected,ReconnectFailed,Parked} domain events (Event Log, mcp_client domain, server name in the agent column, failure reason via DomainEvent::log_detail) and the notification bridge notifies only on the first failed reconnect of an episode, on recovery after failures, and on a parked server — filing each under the workspace the event names and announcing it only when that workspace is live, since one process supervises every workspace it has opened and the bridge's own binding may be the stale half after a switch |
11.2 Insights Dashboard
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 11.2.1 |
Memory View |
WD |
app/test/playwright/specs/memory-v2.spec.ts |
✅ |
Connections > Memory chips |
| 11.2.2 |
Memory Retrieval |
RU+RI |
crates/openhuman-core/src/memory/ and tests/in_process/ |
✅ |
Engine-selectable retrieval paths |
| 11.2.3 |
Memory Storage |
RU+RI |
crates/openhuman-core/src/memory/ and tests/in_process/ |
✅ |
Engine-selectable storage paths |
------ | --------------------------------------------------------------------- | ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
12. Rewards & Progression
Removed. The Rewards page, its API client, slice and Playwright specs were deleted; the section id is kept so later numbering is stable.
12.1 Role Unlocking (removed)
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 12.1.1 |
Activity-Based Unlock |
— |
— |
❌ |
Removed with the Rewards domain; catalog ID retained. |
| 12.1.2 |
Integration-Based Unlock |
— |
— |
❌ |
Removed with the Rewards domain; catalog ID retained. |
| 12.1.3 |
Plan-Based Unlock |
— |
— |
❌ |
Removed with the Rewards domain; catalog ID retained. |
12.2 Progress Tracking (removed)
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 12.2.1 |
Message Count Tracking |
— |
— |
❌ |
Removed with the Rewards domain; catalog ID retained. |
| 12.2.2 |
Usage Metrics |
— |
— |
❌ |
Removed with the Rewards domain; catalog ID retained. |
| 12.2.3 |
State Persistence |
— |
— |
❌ |
Removed with the Rewards domain; catalog ID retained. |
13. Settings & Developer Tools
13.1 Account & Security
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 13.1.1 |
Profile Management |
VU |
app/src/components/settings/panels/AccountPanel.test.tsx |
🟡 |
Citation corrected: the previous PrivacyPanel.test.tsx covers the analytics toggle and capability rendering and asserts nothing about profiles. AccountPanel.test.tsx is the real evidence (avatar initial + full name, username fallback, no-user omission). Stays 🟡 for a reason the old note hid: AccountPanel.tsx is 49 lines and read-only — identity summary plus logout/clear, no editing. There is no profile management to cover; a case asserting one would test a feature that does not exist. |
| 13.1.2 |
Linked Accounts |
WD |
auth-access-control.spec.ts |
🟡 |
UI surface unasserted |
| 13.1.4 |
Wallet Balances Panel |
VU |
app/src/components/settings/panels/__tests__/WalletBalancesPanel.test.tsx, app/src/services/walletApi.test.ts |
✅ |
Loading/error/empty/loaded states; Retry + Refresh re-invocation; chain badges; truncated address; providerStatus chip |
| 13.1.5 |
Approval History |
VU |
app/src/components/settings/panels/__tests__/ApprovalHistoryPanel.test.tsx, app/src/services/api/approvalApi.test.ts |
✅ |
Was ❌ — read-only audit surface over approval_list_recent_decisions; covers loaded/empty/error/refresh states, per-decision badge, and the bare-array vs {result,logs} envelope normalization |
| 13.1.6 |
Action Timeout |
VU, RU, RI |
app/src/components/settings/panels/__tests__/AgentAccessPanel.test.tsx, crates/openhuman-core/src/tools/timeout/mod.rs, crates/openhuman-core/src/config/ops_tests.rs, tests/json_rpc_e2e.rs |
✅ |
New (#3100) — UI control over config_get/update_agent_settings for [agent].agent_timeout_secs; covers load/persist-on-blur/range-rejection/no-op/env-override-disable, the runtime-mutable tool_timeout resolver + env precedence, ops apply/reject, and the RPC roundtrip |
| 13.1.7 |
Core Gateways (run the core elsewhere) |
RU, VU |
crates/openhuman-app/src/gateway/{types,store,ops,provision,registry} (sibling *_tests.rs), app/src/services/__tests__/gatewayService.test.ts, app/src/components/settings/panels/core/__tests__/GatewaySection.test.tsx, app/src/lib/bootCheck/index.test.ts |
✅ |
New — routing the frontend to a core in a Docker container, on a machine over SSH, or both, via tinybox. Covers the reach/confinement pairing, the wire shape, store round-trips and the reserved desktop id, the box spec (published port, network policy, fixed(guest, host) order) and core command, port-conflict retry classification, boot re-activation and its failure path, and an #[ignore]d end-to-end provisioning test that spawns a real detached process and polls /health |
13.2 Automation & Channels
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 13.2.1 |
Channel Configuration |
WD |
app/test/e2e/specs/settings-channels-permissions.spec.ts |
✅ |
|
| 13.2.2 |
Permission Settings |
WD |
app/test/e2e/specs/settings-channels-permissions.spec.ts |
✅ |
|
13.3 AI & Skills
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 13.3.1 |
Model Configuration |
PW |
app/test/playwright/specs/settings-ai-skills.spec.ts (the routing page shows the core-pinned default model and keeps it across a reload) |
✅ |
Was 🟡: both layers asserted only that the LLM tab mounts, so nothing configured anything. The PW case now pins the contract — what the core holds in default_model is what the routing page shows, before and after a reload, with no write-back on render. Driven over inference_update_model_settings rather than through the picker dialog on purpose: the picker is being rebuilt under #6395, so a click-through spec would be rewritten with it and meanwhile cover nothing. The picker → core direction stays at VU (AIPanel.test.tsx). |
| 13.3.2 |
Skill Toggle |
PW |
app/test/playwright/specs/skill-lifecycle.spec.ts |
✅ |
|
| 13.3.3 |
Azure deployment name (off-catalog model id) |
VU |
app/src/components/settings/panels/__tests__/azureDeployment.test.ts, app/src/components/settings/panels/__tests__/ModelEntryField.test.tsx, app/src/components/settings/panels/__tests__/AIPanel.test.tsx |
✅ |
Azure routes by deployment name, not the base model id its /models catalog lists (#5213). Covers endpoint-host detection (incl. dot-boundary lookalikes and the excluded Foundry serverless hosts), /openai/v1 base-URL detection and its inline nudge, the shared entry-mode field used by both pickers (free text, catalog, loading, probe-error branches), no auto-seeding of a catalog id for Azure, the deployment name surviving verbatim into persisted routing, and a provider staying creatable when the live /models probe fails |
| 13.3.4 |
ChatGPT sign-in in AI settings |
VU |
app/src/components/settings/oauth/__tests__/OpenAiOAuthConnect.test.tsx, app/src/components/settings/panels/__tests__/AIPanel.test.tsx |
✅ |
Mocked core RPCs cover sign-in, disconnect, provider/routing persistence, failure paths, and delayed initial status. Live desktop OAuth remains a release smoke check. |
| 13.3.5 |
OpenRouter sign-in (PKCE) in AI settings |
RU+VU |
crates/openhuman-app/src/loopback_oauth_tests.rs, app/src/utils/__tests__/openrouterOAuth.test.ts, app/src/utils/__tests__/loopbackOauthListener.test.ts, app/src/components/settings/panels/__tests__/AIPanel.test.tsx |
✅ |
A pending sign-in stays dismissable (Cancel/Esc/click-outside abort the flow and free the loopback port); Deny, error= and missing-code callbacks get a "Sign-in was not completed." page and a dialog error instead of "You're signed in."; cancel clears the listener timeout so a retry is not stopped; the key exchange honours the abort signal (#6263). Live Authorize remains a release smoke check. |
| 13.3.6 |
Skill registry search→install→agent runs the skill |
RI |
tests/agent_harness_e2e.rs (agent_installs_a_registry_skill_then_runs_it), tests/in_process/skill_registry_e2e.rs, tests/skills_mcp_registry_live.rs (manual, #[ignore]) |
✅ |
Agent finds a loopback-catalog skill via use_skill(skills/setup_skills), installs it behind the approval gate, the SKILL.md lands on disk, then the orchestrator runs it through its own run_workflow (describe_workflow returns its body to the model). The live smoke reports, per catalog source, how many listed skills have a download URL and installs one sample each. |
| 13.3.7 |
Skill registry install sources + unique entry ids |
RU+VU |
crates/openhuman-core/src/skills/catalog/ops_tests.rs, crates/openhuman-core/src/skills/catalog/ops_install_tests.rs, crates/openhuman-core/src/skills/ops_create_and_url_tests.rs, app/src/components/skills/__tests__/SkillsExplorerTab.test.tsx |
🟡 |
Entry ids and SKILL.md resolution come from the tinyskills registry: entries without a SKILL.md are installable: false, show "Not installable" with a source link, and installing one fails with SKILL_REGISTRY_NO_DIRECT_DOWNLOAD naming the page; a missing id suggests the closest ones; a throttled document host reports SKILL_REGISTRY_RATE_LIMITED with its retry delay. Live upstream hosts remain a manual check. |
| 13.3.12 |
Skill registry paging, detail, freshness and outages |
RU+RI+VU+PW |
crates/openhuman-core/src/skills/catalog/ops_tests.rs, crates/openhuman-core/src/skills/catalog/transport_tests.rs, crates/openhuman-core/src/skills/catalog/registry_tests.rs, crates/openhuman-core/src/skills/catalog/tools_tests.rs, tests/in_process/skill_registry_e2e.rs, app/src/components/skills/__tests__/SkillsExplorerRegistryState.test.tsx, app/test/playwright/specs/skills-registry-catalog.spec.ts |
✅ |
Browse/search return one server page with total/total_pages; detail returns the overview and resolved download URL; the transport connects only to pinned addresses and returns redirects unfollowed; a 503 registry keeps serving the held catalog with last_error, which the app shows as an offline banner whose Retry forces a refresh; rate-limit and unreachable errors get their own copy. |
| 13.3.13 |
Skill install supply-chain scan gate |
RU+RI+VU+PW |
crates/openhuman-core/src/skills/ops_install/scan_gate_tests.rs, crates/openhuman-core/src/skills/catalog/ops_install_tests.rs, crates/openhuman-core/src/skills/catalog/tools_tests.rs, crates/openhuman-core/src/skills/tools_tests.rs, tests/in_process/skill_registry_e2e.rs, app/src/components/skills/__tests__/ScanBlockedDialog.test.tsx, app/src/components/skills/__tests__/SkillsExplorerTab.test.tsx, app/src/components/skills/__tests__/InstallSkillDialog.test.tsx, app/test/playwright/specs/skills-registry-catalog.spec.ts |
✅ |
A blocking scan or failed fetch is fetched and scanned once more; a document that still blocks is not installed and returns status: "scan_blocked" with findings for catalog and pasted-URL installs; acknowledged_digest set to the refusal's digest installs that document only, and a changed document is refused afresh with its new digest (or installs if it now scans clean); the agent tools neither expose nor honour the flag; the app prompts with Block install focused and installs only on Install anyway; the registry cools down 45 s after a failed refresh and the retry does not refetch. |
| 13.3.8 | Managed default model (one OpenRouter model for every managed workload) | RU+RI | crates/openhuman-core/src/inference/provider/factory_route_resolution_tests.rs (resolve_model_for_hint_maps_every_managed_hint_to_the_default_model, resolve_model_for_hint_uses_the_pinned_managed_default, resolve_model_for_hint_treats_retired_tier_slugs_as_role_aliases), crates/openhuman-core/src/inference/provider/factory_crate_native_tests.rs, crates/openhuman-core/src/triage/routing_tests.rs, crates/openhuman-core/src/agent/cost_tests.rs, crates/openhuman-core/src/platform/cost/route_tests.rs, crates/openhuman-core/src/config/migrations/retire_managed_tier_slugs_tests.rs, crates/openhuman-core/src/config/migrations/mod_tests.rs (run_pending_v12_to_v13_*), tests/json_rpc_e2e.rs (routing cases, flows arc) | ✅ | The chat-v1/agentic-v1/… tier endpoints are retired: every managed workload role resolves to config.default_model when it names a catalog model, else openrouter/deepseek/deepseek-v4-flash; hint:* and retired slugs stay role aliases; migration 12→13 rewrites persisted slugs; cost classification and the flash price row |
| 13.3.9 | Routing page: Default model row + per-workload tables always shown; providers header action | VU | app/src/components/settings/panels/__tests__/AIPanel.test.tsx (shows the per-workload routing tables directly, with no mode selector, pins a managed default model from the routing page), app/src/services/api/__tests__/aiSettingsApi.test.ts (sends default_model only when the pinned default model changed), app/src/components/settings/panels/ai/ProviderModelPickerDialog.test.tsx | ✅ | No Managed / Own / Advanced selector; the Default model row opens the managed catalog picker (list, alphabetical, search filters models) and persists default_model; "Add provider" lives in the page header |
| 13.3.10 | Composer model pick is the global default | RU+RI+VU+PW | app/test/playwright/specs/chat-model-managed-catalog.spec.ts (Managed OpenRouter catalog in the model picker), crates/openhuman-core/src/web_chat/session_routing_tests.rs, tests/json_rpc_e2e.rs, app/src/features/conversations/Conversations.processSourceCommand.test.tsx | 🟡 | The managed catalog suite is active and covers catalog display and selection. Its individual a composer pick is the global default and survives a fresh page case remains test.skip under #6395, so persistence across a fresh page is not asserted by Playwright. Backend provider/model routing is covered by web_chat::session::session_routing_tests and tests/json_rpc_e2e.rs; composer send defaults, picker overrides, delayed clears, rejected clears, and draft restoration are covered by Conversations.processSourceCommand.test.tsx. |
| 13.3.11 | Skills page (Installed → Registry) with run/edit/remove controls | VU | app/src/components/skills/__tests__/SkillsExplorerTab.test.tsx | ✅ | Page-owned header + tabs in the MCP page's shape: installed rows with Run / Edit / Remove icon controls, Install from URL and New skill; registry rows with search, source filter and Install |
13.4 Developer Options
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 13.4.1 |
Webhook Inspection |
WD |
app/test/e2e/specs/settings-dev-options.spec.ts |
✅ |
|
| 13.4.2 |
Runtime Logs |
WD |
app/test/e2e/specs/settings-dev-options.spec.ts |
✅ |
|
| 13.4.3 |
Memory Debug |
WD |
app/test/e2e/specs/settings-dev-options.spec.ts |
✅ |
|
| 13.4.4 |
Event Log workspace scope (envelope handle + core-notification routing) |
RU+VU |
crates/openhuman-core/src/config/workspace_handle_tests.rs, crates/openhuman-core/src/config/schema/load/active_workspace_tests.rs, crates/openhuman-core/src/core/events_tests.rs (every_workspace_bound_variant_is_reachable_through_one_accessor, events_without_a_workspace_report_none, the_workspace_switch_announcement_is_not_itself_workspace_bound, an_empty_artifact_workspace_reads_as_unbound_not_as_a_workspace), app/src/components/settings/panels/__tests__/EventLogPanel.workspaceScope.test.tsx, app/src/lib/nativeNotifications/__tests__/workspaceRouting.test.ts, app/src/lib/nativeNotifications/__tests__/socketListeners.test.ts |
✅ |
One process serves every workspace it has opened, so the /events/domain stream mixed them. Each row now carries an opaque workspace handle (SHA-256 of workspace_dir, never the path — the panel exports NDJSON) plus the handle that was active when it was emitted; the panel defaults to the active workspace and switches to all. DomainEvent::workspace_dir is the single accessor across the channel, artifact and MCP-supervisor families, replacing the bridge's MCP-only private list. CoreNotificationEvent.workspace lets the client re-check routing on render instead of trusting the publish-time gate, which resolves and sends in two steps (#5966) |
13.5 Data Management
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 13.5.1 |
Clear App Data |
WD |
app/test/e2e/specs/settings-data-management.spec.ts |
✅ |
Destructive — confirm-then-reset |
| 13.5.2 |
Cache Reset |
WD |
app/test/e2e/specs/settings-data-management.spec.ts |
✅ |
|
| 13.5.3 |
Full State Reset |
WD |
app/test/e2e/specs/settings-data-management.spec.ts |
✅ |
Restart-and-verify fresh-install state |
| 13.5.4 |
Migration from another assistant (OpenClaw) |
VU+RU |
app/src/components/settings/panels/__tests__/MigrationPanel.test.tsx, crates/openhuman-core/src/config/migration_helpers/ops.rs (existing) |
✅ |
Was ❌ — UI now wraps the existing openhuman.migrate_openclaw RPC with preview-then-apply + confirm. Hermes tracked as follow-up under #1440 |
13.6 Keyboard Shortcuts & Command Surface
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 13.6.1 |
Command Palette (⌘K / ⌘P) |
VU+WD |
app/src/lib/commands/__tests__/globalActions.test.tsx, app/src/components/commands/__tests__/CommandProvider.test.tsx, app/test/e2e/specs/command-palette.spec.ts |
✅ |
Opens via ⌘K, runs an action, lists seed nav actions, Esc closes |
| 13.6.2 |
Keyboard Shortcuts help directory (? / ⌘/) |
VU+WD |
app/src/components/shortcuts/__tests__/shortcutsView.test.tsx, app/src/components/commands/__tests__/CommandProvider.test.tsx, app/test/e2e/specs/command-palette.spec.ts |
✅ |
Registry-driven grouped list; opens via ? and ⌘/ (mutually exclusive with palette); also reachable from the sidebar shortcut icon + Settings → Keyboard Shortcuts |
| 13.6.3 |
Global shortcut map (nav / chat / view) |
VU |
app/src/lib/commands/__tests__/globalActions.test.tsx |
✅ |
Control-based nav (ctrl/mod per-OS), New Chat / Toggle Sidebar handlers, and alias keys |
13.7 Language
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 13.7.1 |
Japanese UI locale |
VU+PW+MS |
app/src/components/LanguageSelect.test.tsx, app/src/components/settings/panels/__tests__/ComposioPanel.test.tsx, app/src/store/localeSlice.test.ts, app/src/lib/i18n/__tests__/I18nContext.test.tsx, app/src/lib/i18n/__tests__/{coverage,mascotDismissPath}.test.ts, app/test/e2e/specs/japanese-locale.browser.ts (discovered through app/test/playwright/specs/japanese-locale.spec.ts), docs/RELEASE-MANUAL-SMOKE.md |
✅ |
Browser E2E asserts settings selection, reload persistence, fresh ja-JP detection, a saved English override, Memory v2 counts and model attribution at 1280x720. Native-script picker, Japanese browser-language detection, switching back to English, unknown-key fallback, complete dictionary, exact interpolation placeholders, and matching menu labels in mascot dismissal instructions |
14. Tabbed Terminal UI (openhuman-tui)
14.1 Standalone TUI crate
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 14.1.1 |
Transcript reducer (history, deltas, tools, agents, artifacts, errors) |
RU |
crates/openhuman-tui/src/state.rs |
✅ |
Client+thread filtering, projected-history restore, reasoning/tool/sub-agent/artifact rows, structured retry timing, and chat_done finalization |
| 14.1.2 |
Runner flags + thread resolution + RPC name pins |
RU+RI |
crates/openhuman-tui/src/runner.rs, crates/openhuman-tui/src/app.rs, crates/openhuman-tui/tests/cli_e2e.rs |
✅ |
--thread/--new/--last/--resume/--no-alt-screen + positional prompt; compiled-binary help/alias coverage; canonical RPC registry pins |
| 14.1.3 |
Agent cockpit navigation, composer, pickers, and overlays |
RU |
crates/openhuman-tui/src/ui_state.rs, crates/openhuman-tui/src/composer.rs, crates/openhuman-tui/src/cockpit.rs, crates/openhuman-tui/src/render.rs |
✅ |
Chat-first four-tab layout, multiline Unicode editing, history, slash/file completion, approval/plan-review and domain overlays via ratatui TestBackend |
| 14.1.4 |
Historical-command migration diagnostic |
RU |
crates/openhuman-core/src/core/cli_tests.rs |
✅ |
openhuman tui / chat point to openhuman-tui, not unknown namespace |
| 14.1.6 |
Bounded live core-log buffer |
RU |
crates/openhuman-core/src/core/logging.rs |
✅ |
Preserves ordering; bounds both line count and individual line length |
| 14.1.7 |
Interactive terminal session (raw mode, streaming) |
MS |
manual PTY smoke; docs/RELEASE-MANUAL-SMOKE.md |
🚫 |
Requires a real PTY to verify steering/queue input, overlays, and terminal restoration |
16. Embedded Library Harness (openhuman_embed::Harness)
16.1 One-call agent turns for a host process
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 16.1.1 |
End-to-end turn with no transport and no background services |
RI |
crates/openhuman-embed/tests/harness_embed.rs |
✅ |
Real login-free HostKind::Library turn against a wiremock provider; asserts the request landed on the caller's endpoint, no RPC listener was bound, continuity across two turns, then 100 distinct sessions overlap on the same live core without fake auth; also checks session_db placement and workspace removal on drop |
| 16.1.2 |
Turn wire contract (TurnRequest ↔ inference.agent_chat) |
RU |
crates/openhuman-embed/src/turn_tests.rs |
✅ |
Field names checked both ways against the registered controller schema — AgentChatParams carries no rename_all, so an upstream rename is otherwise a silent runtime failure; plus absent-option omission |
| 16.1.3 |
Embedder-supplied config reaches handlers, not just boot |
RU |
crates/openhuman-core/src/core/runtime/context.rs (*_embedder_config) |
✅ |
CoreBuilder::config is published on CoreContext and preferred by load_config_with_timeout; without it every handler re-resolves the process-global workspace |
| 16.1.4 |
Workspace lifetime and credential isolation |
RU |
crates/openhuman-embed/src/harness/workspace_tests.rs |
✅ |
Ephemeral removed on drop, Dir persists, action_dir never inside the workspace, and config_path beside it — its parent is where auth profiles and the keyring resolve |
| 16.1.5 |
Access tier sets the autonomy level and the turn origin |
RU |
crates/openhuman-embed/src/harness/access_tests.rs |
✅ |
The fail-closed trap: tier alone leaves every acting tool refusing while the transcript still reads plausibly. Also pins tool-install and auto_approve_all staying off at full() |
| 16.1.6 |
Per-call provider route is never persisted |
RU |
crates/openhuman-embed/src/harness/provider_tests.rs, crates/openhuman-embed/src/harness/builder_tests.rs |
✅ |
Compiles to EphemeralRoute; asserts no cloud_providers entry and no api_url write, so borrowing an endpoint cannot repoint the operator's install |
| 16.1.7 |
Session store (Core::auth) |
RU |
crates/openhuman-embed/src/auth_tests.rs |
✅ |
Local-session token shape + user payload, backend/local discrimination, AuthState and token-payload decoding incl. the signed-out shapes |
| 16.1.8 |
MCP servers declared programmatically |
RU |
crates/openhuman-embed/src/harness/mcp_tests.rs |
✅ |
stdio/HTTP transports mutually exclusive, enabled by default, env accumulation, allow/deny filters, auth carried through |
| 16.1.9 |
Skill bundles installed into the workspace |
RU |
crates/openhuman-embed/src/harness/skills_tests.rs |
✅ |
Copied not linked (discovery rejects symlinked bundles and manifests by design); symlinks skipped both as bundles and inside them; Inherit refuses the copy |
| 16.1.10 |
One runtime per process |
RU+RI |
crates/openhuman-embed/src/runtime/builder_tests.rs, crates/openhuman-embed/tests/harness_embed.rs |
✅ |
A second Runtime (or Harness) returns AlreadyRunning rather than sharing process-global keyring/event-bus/subscribers; a failed build releases the slot so a retry is possible |
| 16.1.11 |
Many agents on one runtime |
RU+RI |
crates/openhuman-embed/src/agent/spec_tests.rs, crates/openhuman-embed/tests/runtime_agents.rs, crates/openhuman-embed/tests/agent_lifecycle.rs, crates/openhuman-embed/tests/approval_callbacks.rs, crates/openhuman-embed/tests/flow_approvals.rs, crates/openhuman-core/src/security/approval/registration_scope_tests.rs, crates/openhuman-core/src/security/approval/flow_surface_tests.rs, crates/openhuman-embed/src/agent/lifecycle_tests.rs |
✅ |
Three agents with different providers, access tiers, skills, MCP servers and action dirs; turns land on their own provider; per-agent MCP visibility; thread-scoped resume; duplicate/invalid ids and widening are refused; removal settles pending approvals with agent_removed before cancelling turns; closing immediately refuses new admission even while an accepted registration or facade decision holds its barrier; instance-owned approval registration closes before the denial snapshot and rejects late registrations without persisting rows; polling and callback approval decisions serialize with the removal claim; the shared gate also refuses direct, agent-scoped and RPC decisions during the denial-snapshot gap; flow workspace resolution precedes registration, and registration publishes generic, flow and desktop approval surfaces together under the instance barrier |
| 16.1.12 |
API-key credential for library mode |
RU+RI |
crates/openhuman-core/src/security/credentials/api_key_tests.rs, crates/openhuman-embed/tests/runtime_agents.rs |
✅ |
Key stored as an api-key auth profile; wins over an expired session; managed inference sends it as a bearer with no x-api-key; auth state reports credential = "api-key" |
| 16.1.14 |
Host-owned tool policy extension |
RI |
crates/openhuman-embed/tests/host_only_tools.rs |
✅ |
Host executors declare read-only requirements through the public ToolPolicy facade; HostOnly advertises the exact host catalog and refuses built-in shell/write attempts. Application-specific query validation and data handling belong to the host. |
| 16.1.15 |
Strict structured output and bounded repair |
RI |
crates/openhuman-embed/tests/structured_validation.rs, crates/openhuman-embed/tests/structured_turns.rs, crates/openhuman-embed/tests/unit/turn_usage.rs |
✅ |
Wrong types and numeric combinators refused, invalid/external schemas make no call, repair bounded with usage, terminal length rejected even for complete JSON; successful charged usage reaches outcome/meter, invalid buyer charges stay unknown, root overlays retain child spend and session context. |
| 16.1.16 |
Explicit routing ladders and required exploration |
RI |
crates/openhuman-embed/tests/completion_routing.rs, crates/openhuman-embed/tests/tool_required_routing.rs, crates/openhuman-embed/tests/structured_turns.rs, crates/openhuman-embed/tests/unit/completion_cost.rs |
✅ |
Ordered fallbacks, per-rung option/cap overrides, bounded case-insensitive length/max_tokens truncation retries, unpinned terminal provider selection, image/options preservation, accumulated buyer-first reported costs with invalid-charge guards, actual answering model and refusal of schema-valid output before successful tool execution |
| 16.1.19 |
Pinned standalone Embed source consumer |
Script+RI |
scripts/__tests__/embed-consumer-bootstrap.test.mjs |
✅ |
Exact SHA verification, inherited workspace edition, generated active patches, stable locked offline builds with optional Embed enabled/disabled, default dependency tree without HTTP, no untracked secrets or Git metadata, refused malformed/escaping patches and existing destinations |
| 16.1.17 |
Enforced shared turn/run budgets |
RU+RI |
crates/openhuman-embed/tests/budget_fanout.rs, vendor/tinyagents/vendor/tinyinference/crates/tinyinference-llm/src/model/budget_tests.rs |
✅ |
Atomic parent/child reservations, physical provider admission, conservative unknown spend, cancellation and output cap |
| 16.1.18 |
Ordered bounded fanout |
RI |
crates/openhuman-embed/tests/budget_fanout.rs |
✅ |
Input-order results, per-branch and child error isolation, branch ceilings, shared-budget concurrent refusal and empty fanout |
| 16.1.13 |
Awaited per-turn cancellation |
RU+RI |
crates/openhuman-embed/tests/turn_cancellation.rs, crates/openhuman-embed/tests/process_cancellation.rs, crates/openhuman-embed/src/turn_control_tests.rs |
✅ |
Before send, during inference and during a builtin shell command; agent reuse, concurrent cancellation handles, failure/drop acknowledgement, bounded and unbounded descendant termination plus direct-child reaping on Linux; cancellation acknowledgement waits for descendants even when they close output pipes; deadline errors stay typed when the native cancellation relay interrupts dispatch; later cancellation/deadlines do not replace completed outcomes |
| 16.1.21 |
Agent and turn hook isolation |
RI |
crates/openhuman-embed/tests/scoped_hooks.rs |
✅ |
Additive same-named callbacks, overlapping worker turns, one-turn callback lifetime, resumed-session snapshots, core-scheduled turn callbacks and reuse of a removed agent id without registry leakage |
| 16.1.22 |
Agent-turn route attribution headers |
RU+RI |
crates/openhuman-embed/tests/route_headers.rs, crates/openhuman-embed/src/turn_tests.rs |
✅ |
Per-route headers reach only the named endpoint and turn, remain absent on other agents and backend requests, and are redacted in Debug; wire fields pinned against the registered controller schema |
| 16.1.23 |
Tool-hook turn identity and cwd |
RU+RI |
crates/openhuman-embed/tests/tool_hook_context.rs, crates/openhuman-core/src/hooks/bridge_tests.rs |
✅ |
Before and after hooks report the worker/session and working root used by builtin shell; a later turn returns to its agent default; configured hooks use resolved cwd even when tool arguments omit it |
| 16.1.24 |
Inline agent/turn permission callback |
RI |
crates/openhuman-embed/tests/inline_permissions.rs |
✅ |
UI decisions precede execution; denials narrow authority, concurrent agents remain independent, cancellation closes a pending receiver and a closed UI refuses execution; repeated agent callbacks preserve earlier denials |
| 16.1.25 |
Per-turn host-tool replacement |
RI |
crates/openhuman-embed/tests/turn_tools.rs |
✅ |
Replacement and empty-belt revocation apply to resumed HostOnly and mixed definitions, without restoring historical declarations or affecting other agents and later turns |
| 16.1.26 |
Scoped usage observation and budget stop |
RI |
crates/openhuman-embed/tests/usage_hooks.rs |
✅ |
Cumulative charged cost and tokens, known-zero versus unpriced calls, per-agent policy and per-turn observer isolation; no summary-repair model calls after a stop |
| 16.1.27 |
Scoped builtin command environments |
RI |
crates/openhuman-embed/tests/tool_environment.rs, crates/openhuman-embed/tests/tool_hook_context.rs |
✅ |
Overlapping subprocess maps exclude host inheritance on Unix; a real shell receives its turn's variables while excluding host HOME and preserving command-owned runtime/security additions |
| 16.1.20 |
Host turn observer privacy and runtime propagation |
RI |
crates/openhuman-embed/tests/turn_observers.rs, crates/openhuman-embed/tests/observed_turns.rs |
✅ |
Sanitized terminal failures, opt-in payloads, root runtime scope propagation, real model/tool callbacks and answering-model usage |
| 16.1.28 |
Host-installed key stores and default in-memory library secrets |
RU+RI+MS |
crates/openhuman-core/src/security/secret_storage/encrypted_file_backend/key_source_tests.rs, crates/openhuman-core/src/security/secret_storage/store_tests_2_tests.rs, crates/openhuman-core/src/security/secret_storage/memory_backend_tests.rs, crates/openhuman-rpc/src/keyring/*_tests.rs, crates/openhuman-embed/tests/headless_keys.rs, tests/keyring_headless_e2e.rs |
🟡 |
In-memory CRUD, isolation and threaded access, no-file embed defaults, installed-provider precedence, denied-source preservation, private workspace-key creation/reuse, and explicit encrypted-store restart persistence are automated. Existing native desktop profile decryption remains a cross-platform release smoke check. |
| 16.1.29 |
Host-supplied turn attachments (Turn::attach) |
RU+RI |
crates/openhuman-core/src/attachments/host_tests.rs, crates/openhuman-embed/tests/turn_attachments.rs, crates/openhuman-embed/tests/orchestrator_attachments.rs |
✅ |
Bytes are staged under the turn's thread and lead the message in order as image/document blocks with no payload in the reference; generic MIME types are sniffed; image/file count, size and empty files are refused before any model call; a caller-built orchestrator refuses with attachments_unsupported; Debug prints no payload |
| 16.1.30 |
Steering a running turn |
RU+RI |
crates/openhuman-embed/tests/turn_steering.rs, crates/openhuman-embed/src/turn_steering_tests.rs |
✅ |
A steer sent during a plain chat turn's only model call follows the draft answer and earns one more call in the same turn; a steer queued before send reaches the first call; steers the turn did not apply come back on the outcome or, after a failure or cancellation, from the handle; a steer after the turn ended is refused with its text |
16.2 Dynamic runtime and agent configuration
| ID |
Feature |
Layer |
Test path(s) |
Status |
Notes |
| 16.2.1 |
Defaults, templates and typed configuration |
RU+RI |
crates/openhuman-embed/src/runtime/defaults_tests.rs, crates/openhuman-embed/src/runtime/info_tests.rs, crates/openhuman-embed/src/agent/definition_tests.rs, crates/openhuman-embed/tests/runtime_configuration.rs |
✅ |
Config, runtime, template, agent and turn overrides preserve unspecified fields; invalid sampling, unknown templates and capability widening are rejected. |
| 16.2.2 |
Versioned capabilities and runtime weight |
RU |
crates/openhuman-embed/src/runtime/info_tests.rs, crates/openhuman-rpc/src/host_tests.rs, crates/openhuman-embed/examples/capability_report.rs |
✅ |
Reports compiled gates, selected domains/services/modules and effective limits; credentials, endpoints and prompts stay out of JSON. Unavailable modules identify their required feature. |
| 16.2.3 |
Configured storage and driver feature gates |
RU+RI |
crates/openhuman-embed/src/runtime/info_tests.rs, crates/openhuman-embed/tests/runtime_configuration.rs, crates/openhuman-embed/examples/storage.rs |
✅ |
Configured memory/SQLite storage is installed before boot and supplies session stores; MongoDB requires its Cargo feature and operator configuration. Automated recipes use local storage without contacting MongoDB. |
| 16.2.4 |
Native providers, role pins and ordered fallback |
RU+RI |
crates/openhuman-core/src/inference/turn_models_tests.rs, crates/openhuman-embed/tests/runtime_observation.rs, crates/openhuman-embed/examples/custom_provider.rs, vendor/tinyagents/vendor/tinyinference/crates/tinyinference-llm/src/model/fallback_tests.rs |
✅ |
Agent-local native providers receive effective sampling parameters; crates/openhuman-core/src/agent_runs/agent_chat/session_turn_tests.rs pins explicit turn temperature over the stored definition. Explicit role pins override config routing. Generic fallback preserves deferred polling and never switches after visible content. |
| 16.2.5 |
Live hooks and isolated session stores |
RU+RI |
crates/openhuman-core/src/agent/host_overrides_tests.rs, crates/openhuman-embed/src/runtime/seams_tests.rs, crates/openhuman-embed/tests/runtime_observation.rs, crates/openhuman-embed/examples/runtime_events.rs |
✅ |
Runtime hooks compose with local hooks, can be replaced or removed between turns, and restore prior registrations on drop; agent-local session stores do not leak transcripts. |
| 16.2.6 |
Runtime events and approval callbacks |
RU+RI |
crates/openhuman-embed/src/events_tests.rs, crates/openhuman-embed/src/turn_tests.rs, crates/openhuman-embed/tests/runtime_observation.rs, crates/openhuman-embed/tests/approval_lifecycle.rs, crates/openhuman-embed/examples/approvals.rs |
✅ |
Ordered, bounded, content-free events identify agents and individual calls, retain approval origins, and report lag. Callback approve/deny and subscription-drop cancellation exercise real shell markers. Removed-agent handles are invalidated and IDs stay reserved until teardown completes, including cancelled removals. |
| 16.2.7 |
Streaming and cancellation under backpressure |
RU+RI |
crates/openhuman-embed/tests/stream_cancellation.rs, crates/openhuman-embed/examples/streaming.rs |
✅ |
Progress precedes the final result; dropping the stream or cancelling its supplied token releases native model work even with an unread bounded receiver. |
| 16.2.8 |
Executable recipes and generated documentation |
scripts+RI |
scripts/__tests__/run-embed-examples.test.mjs, scripts/__tests__/generate-embed-docs.test.mjs, scripts/run-embed-examples.mjs |
✅ |
Every recipe runs with loopback fixtures and an explicit output contract. Snippets, cookbook metadata, builder signatures and capability output regenerate from source; drift, hidden rustdoc lines and broken links fail checks. |
Summary
| Status |
Count |
| ✅ Covered |
89 |
| 🟡 Partial |
27 |
| ❌ Missing |
26 |
| 🚫 Manual smoke |
11 |
| Total leaves |
153 explicit + nested = 213 product features |
PR-A delta: 13 leaves moved from ❌ → ✅ via 5 WDIO specs + 2 Vitest + 1 Rust integration test.
Section 16.1 adds 11 leaves (10 ✅, 1 🟡) for the embedded library harness. Section 16.2 adds eight covered leaves for dynamic runtime configuration, lifecycle and generated documentation.
Remaining gaps tracked under sub-issues #965 (process), #966 (docs), #967 (tools), #968 (auth/perm), #969 (settings), #970 (rewards), #971 (manual smoke).