Personal AI, On Personal Devices
Python
Feature/afm3 support (#904)
MRH113dbd4a1dfd75e
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 4171889d..e2a6fb0a 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,50 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).### Added+**Apple Foundation Models (AFM 3)** — a new in-process `afm` engine drives+Apple's `apple-fm-sdk` directly, with no HTTP hop and no second process whose+CPU draw would land inside the same energy measurement window. Install with+`DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer uv sync --extra afm`+(the SDK compiles Swift bindings, so a full Xcode is required), then+`jarvis ask --engine afm --model afm-3-core "..."`. Requires an Apple Silicon+Mac on macOS 26+ with Apple Intelligence enabled.++Token counts are real, via `SystemLanguageModel.token_count` (added in SDK+0.2.1, hence the floor). Because `token_count` rejects a value and+instructions together, both ends are measured against the session transcript:+`prompt_tokens = <transcript before> + <prompt alone>` and+`completion_tokens = <transcript after> - prompt_tokens`.++Two caveats worth reading before comparing AFM numbers to other backends:+`stream_response` yields cumulative snapshots batching roughly 8-10 tokens, so+`ttft` is time-to-first-*chunk* (~450 ms on an M1 Pro) and derived+inter-token latencies are inter-chunk; and the SDK exposes no variant+selector, so `afm-3` / `afm-3-core` / `afm-3-core-advanced` are run labels+only — the framework's dynamic profile chooses what actually executes.+Bench results record `metadata["engine_info"]` (SDK version, context size,+host chip) so a run can be attributed after the fact.++**Eval results record which rail did the work.** Result rows and summaries+previously carried only `energy_joules` and `power_watts`, so nothing in an+eval artifact distinguished a Neural Engine run from an idle GPU, or a+hardware measurement from a modelled estimate. Rows now include the per-rail+breakdown (`cpu`/`gpu`/`dram`/`ane`/`soc`), `energy_basis`, and+`energy_method`; summaries add `energy_joules_by_rail` alongside both. On a+ToolCall-15 run against AFM the ANE rail is the largest single component —+91.7 J of 223.7 J, against 3.0 J on the GPU — which is now visible in the+file rather than only through live instrumentation.++**ANE and whole-SoC energy** — `EnergySample.ane_energy_joules` existed but+died at the monitor boundary. Apple Neural Engine and a whole-SoC total now+flow through `TelemetryRecord`, the SQLite schema (with an `ADD COLUMN`+migration), the aggregator, `TelemetrySession`, eval `TurnTrace`/`QueryTrace`,+and bench metadata, alongside an `energy_basis` marker (`"soc"` on Apple+Silicon, `"gpu"` elsewhere) so downstream repeats the monitor's choice instead+of guessing. This matters because AFM runs predominantly on the ANE: measured+on an M1 Pro, one generation drew 0.98 J on the ANE against 0.014 J on the+GPU rail — 70x — where an MLX matmul on the same machine drew 8.99 J on the+GPU and nothing on the ANE.+**Vision input for `jarvis ask`** — attach images to a query with`-i`/`--image` (repeatable) or capture the current screen with`-S`/`--screen`, for vision-capable models such as `gemma3:4b`. Images flow@@ -21,6 +65,49 @@ stack with `mss`/`Pillow` fallbacks on other platforms. Adds the`JARVIS_NUM_CTX` environment variable to tune the Ollama context window(default `16384`).+### Fixed++**Apple Silicon energy was never measured, only modelled.**+`telemetry/energy_apple.py` imported `AppleSiliconMonitor` from+`zeus.device.soc.apple`; no such class has ever existed there. The import+raised on every machine, so the CPU-time fallback -- `wall_clock x TDP x 0.60`+split by four hardcoded ratios -- was the only path that ever ran. It read no+counters and no utilization, so a ten-second sleep and a ten-second generation+produced identical joules, and every Apple Silicon figure in the telemetry DB,+eval traces and `jarvis bench` came from it. Renaming the import would not+have helped: `zeus.device.soc` landed on zeus master after the last zeus+release, so no published `zeus-ml` contains it, and `zeus-ml` has no `apple`+extra either. `energy-apple` now installs `zeus-apple-silicon`, the IOReport+extension zeus itself wraps, which needs no root. Measured on an M1 Pro:+8.24 W under load against 3.72 W idle, where the previous code could not tell+the two windows apart.++`AppleEnergyMonitor.available()` now reports `False` when nothing is+measurable, instead of `True` on any Apple Silicon host, so the factory falls+through rather than letting a model masquerade as a measurement. The estimate+remains reachable through the new `telemetry.allow_energy_estimates` setting+and still reports `energy_method = "tdp_estimate"`.++`snapshot()` is implemented for the first time, so `TelemetrySession`'s+background sampler -- the path agentic evals use -- records real readings+rather than the ABC's empty default. Relatedly, `TelemetrySample` hardcoded+`cpu_power_w = 0.0`, which zeroed CPU power in every eval trace on every+platform.++**`apple_fm` shim reported zero token counts.** It hardcoded zeros with a note+that the SDK did not expose counts; SDK 0.2.1 added `token_count`. Zeroed+`completion_tokens` zeroes throughput, `energy_per_output_token_joules` and+`tokens_per_joule` for everything served through the shim. Streaming responses+now also carry `usage` on the final chunk. The install hint no longer tells+users to clone from GitHub -- `apple-fm-sdk` is on PyPI. System messages become+the session's `instructions` rather than being prefixed onto the prompt as+`[System] ...`, which discarded a distinction the SDK draws and inflated the+prompt-token count.++**`scripts/setup-energy-monitor.sh` did not exist**, though+`jarvis bench --setup-energy` built a path to it and offered to run it -- so+the offer silently did nothing.+### Security**WebSocket API keys no longer appear in request URLs.** Browser clients nowdiff --git a/configs/openjarvis/config.toml b/configs/openjarvis/config.tomlindex 943d31c4..fe605853 100644--- a/configs/openjarvis/config.toml+++ b/configs/openjarvis/config.toml@@ -65,7 +65,15 @@ host = "http://localhost:18181"host = "http://localhost:8080"[engine.apple_fm]-host = "http://localhost:8079"+host = "http://localhost:8079" # OpenAI-compatible shim++# In-process Apple Foundation Models (macOS 26+, Apple Intelligence enabled).+# Install with: uv pip install -e '.[afm]'+[engine.afm]+# instructions = "" # applied to every session+# use_case = "general" # or "content_tagging"+# guardrails = "default" # or "permissive_content_transformations"+sampling = "greedy" # or "random" (the SDK default)# ═══════════════════════════════════════════════════════════════# PILLAR 5: Learning — Improvement Methodologiesdiff --git a/docs/architecture/engine.md b/docs/architecture/engine.mdindex 7fad6232..ba4be3c1 100644--- a/docs/architecture/engine.md+++ b/docs/architecture/engine.md@@ -118,7 +118,8 @@ All providers produce the same output format consumed by agents:| **Nexa** | `nexa` | OpenAI-compatible | 18181 | No (CPU/GPU) | On-device inference with GGUF models || **Lemonade** | `lemonade` | OpenAI-compatible | 13305 | AMD GPU/NPU | AMD consumer GPUs (RDNA), Ryzen AI NPUs || **Uzu** | `uzu` | OpenAI-compatible | 8000 | Varies | Uzu inference runtime |-| **Apple FM** | `apple_fm` | OpenAI-compatible | 8079 | Apple Silicon | Apple Foundation Model on-device inference |+| **Apple FM (shim)** | `apple_fm` | OpenAI-compatible | 8079 | Apple Silicon | Apple Foundation Models over HTTP, for external OpenAI clients |+| **Apple FM (in-process)** | `afm` | Python SDK (in-process) | — | Apple Silicon | Apple Foundation Models with no HTTP hop — the preferred path || **LiteLLM** | `litellm` | OpenAI-compatible | — | No | Unified proxy to 100+ LLM providers || **Cloud** | `cloud` | Provider SDKs | — | No | OpenAI, Anthropic, Google API access |@@ -218,17 +219,42 @@ The Uzu backend connects to the Uzu inference runtime. Unlike other OpenAI-compa- **Health check:** `GET /models`- **Best for:** Uzu-optimized inference workloads-### Apple FM+### Apple Foundation Models (AFM 3)-The Apple FM backend connects to Apple's Foundation Model SDK via a FastAPI shim (`apple_fm_shim.py`). It wraps `python-apple-fm-sdk` as an OpenAI-compatible API. Requires macOS 15+ with Apple Silicon.+Two backends serve the same on-device model. Both require an Apple Silicon Mac on **macOS 26+** with Apple Intelligence enabled.-!!! note "Token counts"- The Apple FM SDK does not expose token counts. The shim returns 0 for all token counts. Benchmark throughput and energy-per-token metrics will reflect this limitation.+**`afm` — in-process (preferred).** Drives `apple_fm_sdk` directly, with no HTTP hop and no second process. Prefer it for anything energy-measured: a separate uvicorn process burns CPU inside the same measurement window.-- **Default host:** `http://localhost:8079`-- **Health check:** `GET /v1/models`-- **Install:** `pip install python-apple-fm-sdk`-- **Best for:** Running Apple Foundation Models natively on Apple Silicon hardware+**`apple_fm` — OpenAI-compatible shim.** A FastAPI server (`apple_fm_shim.py`) on port 8079, for pointing external OpenAI clients at AFM.++**Install.** The SDK compiles Swift bindings at install time, so a full Xcode is required — Command Line Tools alone fail:++```bash+DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \+ uv sync --extra afm+```++`>=0.2.1` is the floor: that release added `SystemLanguageModel.token_count` and `.context_size`, which both backends use for token accounting.++```bash+jarvis ask --engine afm --model afm-3-core "..." # in-process+jarvis host afm-3 --backend apple_fm # launch the shim+```++- **Model labels:** `afm-3`, `afm-3-core`, `afm-3-core-advanced`+- **Context length:** 4096 tokens+- **Configuration:** `[engine.afm]` — `instructions`, `use_case`, `guardrails`, `sampling`++!!! warning "The model label does not select the model"+ The SDK exposes no variant selector. The Foundation Models framework's dynamic profile picks AFM 3 Core (dense ~3B) or AFM 3 Core Advanced (20B sparse MoE, 1–4B active per request) based on the host device. The label only tags the run. `AppleFMEngine.describe()` records the SDK version, context size and host chip so a run can be attributed after the fact.++!!! warning "Latency figures are not comparable to other backends"+ `stream_response` yields cumulative text snapshots batching roughly 8–10 tokens each, not individual tokens. `ttft` is therefore time-to-first-*chunk* — around 450 ms on an M1 Pro, versus a few tens of ms of real first-token latency — and derived inter-token latencies are inter-*chunk* latencies. Token counts, throughput and per-token energy are unaffected.++!!! note "Energy"+ AFM runs predominantly on the Neural Engine, so its energy is invisible on a GPU-only rail. Install `openjarvis[energy-apple]` to measure CPU/GPU/DRAM/ANE separately; records are then written with `energy_basis = "soc"` and an `ane_energy_joules` column. Without it, no Apple energy is recorded at all.++There is no Private Cloud Compute path in the Python SDK — inference is always on-device, which is what makes the energy measurement meaningful.### LiteLLMdiff --git a/pyproject.toml b/pyproject.tomlindex b7705194..561bfaef 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -98,10 +98,27 @@ desktop = ["soundfile>=0.12",]openhands = ["openhands-sdk>=1.0; python_version >= '3.12'"]+# Apple Foundation Models (on-device AFM 3). Needs an Apple Silicon Mac on+# macOS 26+ with Apple Intelligence enabled. The SDK compiles Swift bindings+# at install time, so a full Xcode is required -- Command Line Tools alone+# fail:+# DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \+# uv pip install -e '.[afm]'+# 0.2.1 is the floor: it added SystemLanguageModel.context_size and+# token_count, which the engine uses for token accounting.+afm = ["apple-fm-sdk>=0.2.1; sys_platform == 'darwin'"]gpu-metrics = ["pynvml>=12.0"]energy-amd = ["amdsmi>=6.1"]-energy-apple = ["zeus-ml[apple]"]-energy-all = ["pynvml>=12.0", "amdsmi>=6.1", "zeus-ml[apple]"]+# `zeus-ml[apple]` was the previous spelling and never worked: zeus-ml has no+# `apple` extra, and its `zeus.device.soc` package (which wraps this library)+# landed on master after the last zeus release, so no published zeus-ml+# contains it. Depend on the underlying IOReport extension directly.+energy-apple = ["zeus-apple-silicon>=1.1.0; sys_platform == 'darwin'"]+energy-all = [+ "pynvml>=12.0",+ "amdsmi>=6.1",+ "zeus-apple-silicon>=1.1.0; sys_platform == 'darwin'",+]orchestrator-training = ["torch>=2.0", "transformers>=4.40"]learning-dspy = ["dspy>=2.6"]learning-gepa = ["gepa>=0.1"]diff --git a/scripts/setup-energy-monitor.sh b/scripts/setup-energy-monitor.shnew file mode 100755index 00000000..4439ad03--- /dev/null+++ b/scripts/setup-energy-monitor.sh@@ -0,0 +1,71 @@+#!/usr/bin/env bash+# Install the energy-measurement backend for this machine.+#+# Referenced by `jarvis bench --setup-energy` (see cli/bench_cmd.py), which+# built a path to this script and offered to run it while the file did not+# exist -- the offer silently did nothing.+#+# Picks the right extra by platform:+# Apple Silicon -> energy-apple (zeus-apple-silicon, IOReport, no root)+# Linux + NVIDIA -> gpu-metrics (pynvml)+# Linux + AMD -> energy-amd (amdsmi)+# Linux, neither -> nothing to install; RAPL is read from sysfs+set -euo pipefail++cd "$(dirname "$0")/.."++if ! command -v uv >/dev/null 2>&1; then+ echo "error: uv is required. See https://docs.astral.sh/uv/" >&2+ exit 1+fi++os="$(uname -s)"+arch="$(uname -m)"++case "$os" in+Darwin)+ if [ "$arch" != "arm64" ]; then+ echo "Intel Macs expose no energy counters OpenJarvis can read." >&2+ exit 1+ fi+ extra="energy-apple"+ ;;+Linux)+ if command -v nvidia-smi >/dev/null 2>&1; then+ extra="gpu-metrics"+ elif command -v rocm-smi >/dev/null 2>&1; then+ extra="energy-amd"+ elif [ -d /sys/class/powercap/intel-rapl ]; then+ echo "RAPL is readable from sysfs; no extra package needed."+ echo "If readings come back empty, check permissions on"+ echo " /sys/class/powercap/intel-rapl/*/energy_uj"+ exit 0+ else+ echo "No supported energy backend detected on this host." >&2+ exit 1+ fi+ ;;+*)+ echo "Energy measurement is not supported on $os." >&2+ exit 1+ ;;+esac++echo "Installing openjarvis[$extra] ..."+# --inexact: add this extra without pruning extras the user already has.+# A plain `uv sync --extra` would uninstall e.g. [dev] and [server].+uv sync --inexact --extra "$extra"++echo+echo "Verifying ..."+uv run python - <<'PY'+from openjarvis.telemetry.energy_monitor import create_energy_monitor++monitor = create_energy_monitor()+if monitor is None:+ raise SystemExit(+ "Energy monitor still unavailable. Run `jarvis doctor` for details."+ )+print(f"OK: {monitor.vendor().value} via {monitor.energy_method()}")+monitor.close()+PYdiff --git a/src/openjarvis/bench/_stubs.py b/src/openjarvis/bench/_stubs.pyindex 0da2ed96..e944b459 100644--- a/src/openjarvis/bench/_stubs.py+++ b/src/openjarvis/bench/_stubs.py@@ -29,6 +29,46 @@ class BenchmarkResult:energy_method: str = ""+def engine_info(engine: Any) -> Optional[Dict[str, Any]]:+ """Run metadata from an engine that can describe itself, or ``None``.++ Duck-typed the way IPW's runner writes ``client_info`` into+ ``summary.json``: ``describe`` is not on the ``InferenceEngine`` ABC, so+ engines opt in by defining it. The AFM engine uses this to record its SDK+ version, context size and host chip, none of which is recoverable from the+ benchmark results alone.++ The return value is validated rather than trusted. It lands in a JSONL+ file, so anything that is not a plain dict of JSON-safe scalars is+ dropped or stringified -- an engine returning something exotic must not be+ able to fail the whole benchmark write.+ """+ describe = getattr(engine, "describe", None)+ if not callable(describe):+ return None+ try:+ info = describe()+ except Exception:+ return None+ if not isinstance(info, dict):+ return None++ safe: Dict[str, Any] = {}+ for key, value in info.items():+ if not isinstance(key, str):+ continue+ if value is None or isinstance(value, (bool, int, float, str)):+ safe[key] = value+ else:+ try:+ json.dumps(value)+ except (TypeError, ValueError):+ safe[key] = repr(value)+ else:+ safe[key] = value+ return safe or None++class BaseBenchmark(ABC):"""Base class for all benchmark implementations.diff --git a/src/openjarvis/bench/energy.py b/src/openjarvis/bench/energy.pyindex abc35e1f..3ede389c 100644--- a/src/openjarvis/bench/energy.py+++ b/src/openjarvis/bench/energy.py@@ -7,7 +7,7 @@ import timefrom typing import Any, List, Optionalfrom openjarvis.bench._stats import compute_stats-from openjarvis.bench._stubs import BaseBenchmark, BenchmarkResult+from openjarvis.bench._stubs import BaseBenchmark, BenchmarkResult, engine_infofrom openjarvis.core.registry import BenchmarkRegistryfrom openjarvis.core.types import Message, Rolefrom openjarvis.engine._stubs import InferenceEngine@@ -59,6 +59,15 @@ class EnergyBenchmark(BaseBenchmark):per_tokens: List[float] = []errors = 0energy_method = ""+ # Per-rail totals, so an ANE-resident backend's energy is attributable+ # rather than just folded into one number.+ rail_totals: dict[str, float] = {+ "cpu": 0.0,+ "gpu": 0.0,+ "dram": 0.0,+ "ane": 0.0,+ }+ energy_basis = ""if energy_monitor is not None:from openjarvis.telemetry.steady_state import SteadyStateDetector@@ -80,6 +89,12 @@ class EnergyBenchmark(BaseBenchmark):tps = tokens / elapsed if elapsed > 0 else 0.0ept = energy_j / tokens if tokens > 0 else 0.0+ rail_totals["cpu"] += sample.cpu_energy_joules+ rail_totals["gpu"] += sample.gpu_energy_joules+ rail_totals["dram"] += sample.dram_energy_joules+ rail_totals["ane"] += sample.ane_energy_joules+ energy_basis = sample.basis or energy_basis+per_energy_j.append(energy_j)per_power_w.append(power_w)per_tps.append(tps)@@ -127,10 +142,17 @@ class EnergyBenchmark(BaseBenchmark):metrics["total_time_seconds"] = total_timemetadata: dict[str, Any] = {}- if engine.engine_id == "apple_fm":- metadata["token_estimation"] = (- "~4 chars/token (Apple FM SDK does not expose counts)"- )+ info = engine_info(engine)+ if info:+ metadata["engine_info"] = info+ if energy_basis:+ # Which rail set total_energy_joules and energy_per_token came+ # from. Figures are only comparable across runs sharing a basis.+ metadata["energy_basis"] = energy_basis+ if any(v > 0 for v in rail_totals.values()):+ metadata["energy_joules_by_rail"] = {+ rail: round(v, 6) for rail, v in rail_totals.items() if v > 0+ }return BenchmarkResult(benchmark_name=self.name,diff --git a/src/openjarvis/bench/throughput.py b/src/openjarvis/bench/throughput.pyindex 6dc42db8..bc92a20b 100644--- a/src/openjarvis/bench/throughput.py+++ b/src/openjarvis/bench/throughput.py@@ -7,7 +7,7 @@ import timefrom typing import Any, Listfrom openjarvis.bench._stats import compute_stats-from openjarvis.bench._stubs import BaseBenchmark, BenchmarkResult+from openjarvis.bench._stubs import BaseBenchmark, BenchmarkResult, engine_infofrom openjarvis.core.registry import BenchmarkRegistryfrom openjarvis.core.types import Message, Rolefrom openjarvis.engine._stubs import InferenceEngine@@ -76,10 +76,9 @@ class ThroughputBenchmark(BaseBenchmark):metrics["total_time_seconds"] = total_timemetadata: dict[str, Any] = {}- if engine.engine_id == "apple_fm":- metadata["token_estimation"] = (- "~4 chars/token (Apple FM SDK does not expose counts)"- )+ info = engine_info(engine)+ if info:+ metadata["engine_info"] = inforeturn BenchmarkResult(benchmark_name=self.name,diff --git a/src/openjarvis/cli/ask.py b/src/openjarvis/cli/ask.pyindex 14052cd6..7c06e71b 100644--- a/src/openjarvis/cli/ask.py+++ b/src/openjarvis/cli/ask.py@@ -34,6 +34,24 @@ from openjarvis.telemetry.store import TelemetryStorelogger = logging.getLogger(__name__)+# Engines that run inference on this machine. Used by the image privacy guard+# below: a screenshot is sensitive and OpenJarvis is local-first, so sending+# one to a non-local engine warrants a warning first. Module-level so it can be+# asserted against directly -- an engine missing from here produces a false+# "your image is leaving this machine" warning.+LOCAL_ENGINES = {+ "afm",+ "apple_fm",+ "exo",+ "gemma_cpp",+ "llamacpp",+ "nexa",+ "ollama",+ "sglang",+ "uzu",+ "vllm",+}+def _run_research(*,@@ -833,6 +851,7 @@ def ask(energy_monitor = create_energy_monitor(prefer_vendor=config.telemetry.energy_vendor or None,+ allow_estimates=config.telemetry.allow_energy_estimates,)except Exception as exc:logger.debug("Failed to create energy monitor: %s", exc)@@ -943,21 +962,7 @@ def ask(return# Direct-to-engine mode (no agent)- # Privacy guard: a screenshot/image is sensitive, and OpenJarvis is- # local-first. If the active engine isn't local, warn before the image- # leaves the machine rather than silently uploading it to a third party.- _LOCAL_ENGINES = {- "ollama",- "llamacpp",- "vllm",- "sglang",- "exo",- "nexa",- "uzu",- "apple_fm",- "gemma_cpp",- }- if image_b64 and engine_name not in _LOCAL_ENGINES:+ if image_b64 and engine_name not in LOCAL_ENGINES:console.print(f"[yellow]Privacy warning:[/yellow] sending {len(image_b64)} "f"image(s) to a non-local engine ('{engine_name}'). The image will "diff --git a/src/openjarvis/cli/bench_cmd.py b/src/openjarvis/cli/bench_cmd.pyindex 24fd8ccb..76292310 100644--- a/src/openjarvis/cli/bench_cmd.py+++ b/src/openjarvis/cli/bench_cmd.py@@ -260,6 +260,7 @@ def run(energy_monitor = create_energy_monitor(prefer_vendor=config.telemetry.energy_vendor or None,+ allow_estimates=config.telemetry.allow_energy_estimates,)except Exception as exc:logger.debug("Energy monitor init skipped: %s", exc)@@ -298,6 +299,7 @@ def run(energy_monitor = create_energy_monitor(prefer_vendor=config.telemetry.energy_vendor or None,+ allow_estimates=config.telemetry.allow_energy_estimates,)if energy_monitor is not None:console.print("[green]Energy monitor installed.[/green]")diff --git a/src/openjarvis/cli/doctor_cmd.py b/src/openjarvis/cli/doctor_cmd.pyindex 2c40ee61..c3fc1c90 100644--- a/src/openjarvis/cli/doctor_cmd.py+++ b/src/openjarvis/cli/doctor_cmd.py@@ -206,7 +206,12 @@ def _check_optional_deps() -> List[CheckResult]:("pynvml", "openjarvis[gpu-metrics]", "NVIDIA energy monitoring"),("amdsmi", "openjarvis[energy-amd]", "AMD energy monitoring"),("colbert", "openjarvis[memory-colbert]", "ColBERT memory backend"),- ("zeus", "openjarvis[energy-apple]", "Apple Silicon energy monitoring"),+ (+ "zeus_apple_silicon",+ "openjarvis[energy-apple]",+ "Apple Silicon energy monitoring",+ ),+ ("apple_fm_sdk", "openjarvis[afm]", "Apple Foundation Models (AFM 3)"),]for pkg, install_hint, description in optional_packages:try:diff --git a/src/openjarvis/cli/host_cmd.py b/src/openjarvis/cli/host_cmd.pyindex 8013dfb4..e765938b 100644--- a/src/openjarvis/cli/host_cmd.py+++ b/src/openjarvis/cli/host_cmd.py@@ -72,6 +72,15 @@ _BACKENDS = {"default_port": 52415,"binary": "exo",},+ "apple_fm": {+ "display": "Apple Foundation Models (shim)",+ "package": "apple-fm-sdk",+ "import_check": "apple_fm_sdk",+ "pip_spec": "apple-fm-sdk>=0.2.1",+ "uv_extra": "afm",+ "platform": "darwin",+ "default_port": 8079,+ },"uzu": {"display": "Uzu","package": "uzu",@@ -280,6 +289,22 @@ def _build_serve_command(backend: str, model: str, port: int) -> list[str]:if backend == "uzu":return ["uzu", "serve", "--port", str(port)]+ if backend == "apple_fm":+ # There is one on-device model, so `model` is a label and is not+ # passed through -- the shim serves whatever the Foundation Models+ # dynamic profile selects. For in-process use prefer the `afm`+ # engine, which needs no server at all.+ return [+ sys.executable,+ "-m",+ "uvicorn",+ "openjarvis.engine.apple_fm_shim:app",+ "--host",+ "127.0.0.1",+ "--port",+ str(port),+ ]+raise ValueError(f"Unknown backend: {backend}")diff --git a/src/openjarvis/core/config.py b/src/openjarvis/core/config.pyindex 701c0e0f..6159a25a 100644--- a/src/openjarvis/core/config.py+++ b/src/openjarvis/core/config.py@@ -447,6 +447,26 @@ class GemmaCppEngineConfig:num_threads: int = 0+@dataclass(slots=True)+class AfmEngineConfig:+ """Per-engine config for in-process Apple Foundation Models.++ No ``host``: this engine runs the SDK in-process rather than talking to a+ server. (The separate ``apple_fm`` engine does have a host — it speaks to+ the FastAPI shim.)+ """++ # Standing instructions applied to every session, on top of any system+ # message in the request.+ instructions: str = ""+ # "general" or "content_tagging".+ use_case: str = "general"+ # "default" or "permissive_content_transformations".+ guardrails: str = "default"+ # "greedy" (reproducible, the default here) or "random" (the SDK default).+ sampling: str = "greedy"++@dataclass(slots=True)class LemonadeEngineConfig:"""Per-engine config for Lemonade."""@@ -469,6 +489,7 @@ class EngineConfig:nexa: NexaEngineConfig = field(default_factory=NexaEngineConfig)uzu: UzuEngineConfig = field(default_factory=UzuEngineConfig)apple_fm: AppleFmEngineConfig = field(default_factory=AppleFmEngineConfig)+ afm: AfmEngineConfig = field(default_factory=AfmEngineConfig)gemma_cpp: GemmaCppEngineConfig = field(default_factory=GemmaCppEngineConfig)lemonade: LemonadeEngineConfig = field(default_factory=LemonadeEngineConfig)@@ -1165,6 +1186,12 @@ class TelemetryConfig:gpu_metrics: bool = Falsegpu_poll_interval_ms: int = 50energy_vendor: str = "" # auto-detect or force "nvidia"/"amd"/"apple"/"cpu_rapl"+ # Permit a modelled energy estimate when no hardware counters are+ # readable. Off by default: an estimate written to the telemetry DB is+ # indistinguishable from a measurement at query time except via+ # `energy_method`, and on Apple Silicon the estimate is a function of+ # wall-clock alone, so it cannot tell an idle window from a busy one.+ allow_energy_estimates: bool = Falsewarmup_samples: int = 0steady_state_window: int = 5steady_state_threshold: float = 0.05diff --git a/src/openjarvis/core/types.py b/src/openjarvis/core/types.pyindex add45385..9b93437f 100644--- a/src/openjarvis/core/types.py+++ b/src/openjarvis/core/types.py@@ -185,6 +185,18 @@ class TelemetryRecord:cpu_energy_joules: float = 0.0gpu_energy_joules: float = 0.0dram_energy_joules: float = 0.0+ # Apple Neural Engine. Populated on Apple Silicon only, and the reason the+ # rail exists here at all: Apple Foundation Models runs predominantly on+ # the ANE, so a record carrying only CPU/GPU/DRAM would make it look+ # nearly free.+ ane_energy_joules: float = 0.0+ # Every rail this record's monitor measured, summed. On Apple Silicon each+ # rail above is a partial view of one package, so the derived per-token+ # and per-watt figures come from this rather than from GPU alone.+ soc_energy_joules: float = 0.0+ # Which rail set the derived figures were computed from: "soc" on Apple+ # Silicon, "gpu" elsewhere. Empty on records written before this existed.+ energy_basis: str = ""tokens_per_joule: float = 0.0# Version tag for the token-counting methodology used when this record# was produced. `None` (= legacy) means the record predates per-recorddiff --git a/src/openjarvis/engine/__init__.py b/src/openjarvis/engine/__init__.pyindex ef88acab..726318ec 100644--- a/src/openjarvis/engine/__init__.py+++ b/src/openjarvis/engine/__init__.py@@ -24,10 +24,14 @@ logger = logging.getLogger(__name__)def _register_optional_engines() -> None:"""Register available optional engines and retain import diagnostics."""- for optional in ("cloud", "litellm", "gemma_cpp"):+ # OSError is caught alongside ImportError for apple_fm: its Swift bindings+ # load at import time and raise OSError, not ImportError, on a host whose+ # macOS or Xcode is too old. Letting that escape would take down every+ # other engine too.+ for optional in ("cloud", "litellm", "gemma_cpp", "apple_fm"):try:importlib.import_module(f".{optional}", __name__)- except ImportError as exc:+ except (ImportError, OSError) as exc:logger.debug("Optional engine %r unavailable: %s", optional, exc)diff --git a/src/openjarvis/engine/_apple_fm_support.py b/src/openjarvis/engine/_apple_fm_support.pynew file mode 100644index 00000000..3a32b92c--- /dev/null+++ b/src/openjarvis/engine/_apple_fm_support.py@@ -0,0 +1,151 @@+"""Pure helpers for the Apple Foundation Models engine and shim.++Deliberately free of any ``apple_fm_sdk`` import so this logic stays testable+on CI runners that cannot install the SDK — it builds Swift bindings and needs+a full Xcode on an Apple Silicon Mac, and OpenJarvis has no macOS runner.++Ported from IPW's ``ipw/clients/_afm_support.py``.+"""++from __future__ import annotations++from typing import Any, Dict, List, Mapping, Sequence, Tuple++# Model labels accepted by the engine. The SDK exposes no variant selector:+# the Foundation Models framework's dynamic profile picks AFM 3 Core (dense+# ~3B) or AFM 3 Core Advanced (20B sparse MoE, 1-4B active per request) from+# the host device's capabilities. These labels therefore only tag a run -- they+# do not steer which model executes.+MODEL_LABELS: Tuple[str, ...] = ("afm-3", "afm-3-core", "afm-3-core-advanced")++# Sampling modes exposed by fm.SamplingMode as constructors. ``top`` and+# ``probability_threshold`` are fields of the random mode, not modes.+SAMPLING_MODES: Tuple[str, ...] = ("greedy", "random")++_RANDOM_SAMPLING_FIELDS: Tuple[str, ...] = ("top", "probability_threshold", "seed")+++def validate_model_label(model: str) -> str:+ """Normalize and check a model label.++ Raises ``ValueError`` naming the accepted labels: a typo would otherwise+ silently mislabel a run's telemetry and FLOPs lookup.+ """+ label = (model or "").strip().lower()+ if label not in MODEL_LABELS:+ raise ValueError(+ f"Unknown AFM model label {model!r}. Expected one of: "+ f"{', '.join(MODEL_LABELS)}. Note that these are run labels only -- "+ "the on-device variant is chosen by the framework's dynamic "+ "profile, not by this name."+ )+ return label+++def coerce_scalar(value: Any) -> Any:+ """Coerce a config string into bool/int/float when it looks like one."""+ if not isinstance(value, str):+ return value+ text = value.strip()+ lowered = text.lower()+ if lowered in ("true", "false"):+ return lowered == "true"+ try:+ if "." in text or "e" in lowered:+ return float(text)+ return int(text)+ except ValueError:+ return text+++class SnapshotAccumulator:+ """Turn ``session.stream_response``'s cumulative snapshots into deltas.++ The SDK yields the *entire* text generated so far on each iteration, not+ the newly added piece. Appending each chunk -- the pattern every other+ streaming backend uses -- would duplicate text quadratically.++ ``add`` returns the newly arrived text, or ``""`` when a snapshot repeats.+ Guided generation can revise a snapshot rather than extend it, so a+ snapshot that does not continue the previous one is treated as a+ replacement and reported whole.+ """++ __slots__ = ("text",)++ def __init__(self) -> None:+ self.text: str = ""++ def add(self, snapshot: Any) -> str:+ if snapshot is None:+ return ""+ if not isinstance(snapshot, str):+ # Defensive: tolerate a chunk object exposing text via an attribute.+ snapshot = getattr(snapshot, "content", None) or str(snapshot)+ if snapshot == self.text:+ return ""+ if snapshot.startswith(self.text):+ delta = snapshot[len(self.text) :]+ else:+ delta = snapshot+ self.text = snapshot+ return delta+++def snapshot_deltas(snapshots: Sequence[Any]) -> List[str]:+ """Eager form of :class:`SnapshotAccumulator`, for tests and offline use."""+ accumulator = SnapshotAccumulator()+ return [delta for delta in (accumulator.add(s) for s in snapshots) if delta]+++def build_options_kwargs(params: Mapping[str, Any]) -> Dict[str, Any]:+ """Map OpenJarvis params onto ``fm.GenerationOptions`` fields.++ Only non-``None`` values are returned so the SDK's own defaults apply for+ anything the caller did not set. ``max_tokens`` is OpenJarvis's+ cross-backend name for what the SDK calls ``maximum_response_tokens``.+ """+ options: Dict[str, Any] = {}++ max_tokens = params.get("max_tokens")+ if max_tokens is not None:+ options["maximum_response_tokens"] = int(coerce_scalar(max_tokens))++ temperature = params.get("temperature")+ if temperature is not None:+ options["temperature"] = float(coerce_scalar(temperature))++ return options+++def parse_sampling_spec(params: Mapping[str, Any]) -> Tuple[str, Dict[str, Any]]:+ """Resolve the sampling mode name and its keyword arguments.++ Defaults to ``greedy`` so repeated runs are reproducible; the SDK itself+ defaults to random sampling.+ """+ mode = str(params.get("sampling") or "greedy").strip().lower()+ if mode not in SAMPLING_MODES:+ raise ValueError(+ f"Unknown AFM sampling mode {mode!r}. Expected one of: "+ f"{', '.join(SAMPLING_MODES)}."+ )++ kwargs: Dict[str, Any] = {}+ if mode == "random":+ for field in _RANDOM_SAMPLING_FIELDS:+ if params.get(field) is not None:+ kwargs[field] = coerce_scalar(params[field])+ return mode, kwargs+++__all__ = [+ "MODEL_LABELS",+ "SAMPLING_MODES",+ "SnapshotAccumulator",+ "build_options_kwargs",+ "coerce_scalar",+ "parse_sampling_spec",+ "snapshot_deltas",+ "validate_model_label",+]diff --git a/src/openjarvis/engine/_async_loop.py b/src/openjarvis/engine/_async_loop.pynew file mode 100644index 00000000..6b743fe3--- /dev/null+++ b/src/openjarvis/engine/_async_loop.py@@ -0,0 +1,60 @@+"""Run an asyncio event loop on a background thread.++``InferenceEngine.generate`` is synchronous, but some backends expose+async-only APIs — Apple's Foundation Models SDK is one. This bridges the two+without an ``asyncio.run`` per call, which would tear down and rebuild a loop+for every request and break SDKs holding loop-bound state.++Ported from IPW's ``ipw/clients/_async_loop.py``.+"""++from __future__ import annotations++import asyncio+import threading+from typing import Any+++class AsyncLoopRunner:+ """Own an event loop on a daemon thread and run coroutines against it."""++ def __init__(self, name: str = "openjarvis-async") -> None:+ self._loop = asyncio.new_event_loop()+ self._thread = threading.Thread(target=self._run_loop, name=name, daemon=True)+ self._thread.start()++ def run(self, coro: Any) -> Any:+ """Submit *coro* to the background loop and block for its result."""+ future = asyncio.run_coroutine_threadsafe(coro, self._loop)+ return future.result()++ def shutdown(self) -> None:+ if self._loop.is_closed():+ return++ async def _drain() -> None:+ current = asyncio.current_task()+ tasks = [+ task+ for task in asyncio.all_tasks()+ if task is not current and not task.done()+ ]+ for task in tasks:+ task.cancel()+ if tasks:+ await asyncio.gather(*tasks, return_exceptions=True)++ try:+ asyncio.run_coroutine_threadsafe(_drain(), self._loop).result(timeout=5.0)+ except Exception: # pragma: no cover - shutdown is best-effort+ pass+ self._loop.call_soon_threadsafe(self._loop.stop)+ self._thread.join(timeout=2.0)+ self._loop.close()++ def _run_loop(self) -> None:+ asyncio.set_event_loop(self._loop)+ self._loop.run_forever()+++__all__ = ["AsyncLoopRunner"]diff --git a/src/openjarvis/engine/_discovery.py b/src/openjarvis/engine/_discovery.pyindex d3a9acc9..047f7f1e 100644--- a/src/openjarvis/engine/_discovery.py+++ b/src/openjarvis/engine/_discovery.py@@ -28,6 +28,8 @@ _HOST_MAP: Dict[str, str | None] = {"cloud": None,"litellm": None,"gemma_cpp": None,+ # In-process: drives the Apple FM SDK directly, so there is no host.+ "afm": None,}@@ -51,6 +53,16 @@ def _make_engine(key: str, config: JarvisConfig) -> InferenceEngine:num_threads=cfg.num_threads,)+ # afm: in-process engine, configured by behaviour rather than a host+ if key == "afm":+ cfg = config.engine.afm+ return cls(+ instructions=cfg.instructions,+ use_case=cfg.use_case,+ guardrails=cfg.guardrails,+ sampling=cfg.sampling,+ )+host_attr = _HOST_MAP.get(key)if host_attr is not None:host = getattr(config.engine, host_attr, None)diff --git a/src/openjarvis/engine/apple_fm.py b/src/openjarvis/engine/apple_fm.pynew file mode 100644index 00000000..7bc67a9e--- /dev/null+++ b/src/openjarvis/engine/apple_fm.py@@ -0,0 +1,580 @@+"""In-process Apple Foundation Models (AFM 3) engine.++Drives Apple's ``apple_fm_sdk`` against the on-device model behind Apple+Intelligence, with no HTTP hop and no second process — so the energy measured+around a request is the energy the request actually cost. (The+``apple_fm_shim`` + ``apple_fm`` engine pair remains available for pointing+external OpenAI-compatible clients at the same model.)++Requires an Apple Silicon Mac on macOS 26+ with Apple Intelligence enabled,+and a full Xcode — not just Command Line Tools — because the SDK compiles+Swift bindings at install time::++ DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \\+ uv pip install -e '.[afm]'++ jarvis ask --engine afm --model afm-3-core "..."++Measurement caveats specific to this backend — read before comparing numbers+against MLX or Ollama:++* ``session.stream_response`` yields **cumulative text snapshots**, each+ batching roughly 8-10 tokens. ``ttft`` is therefore time-to-first-*chunk*+ (an upper bound on true TTFT — around 450ms on an M1 Pro versus a few tens+ of ms of real first-token latency), and derived inter-token latencies are+ inter-*chunk* latencies. Neither is comparable to a backend that streams one+ token at a time. Token counts, throughput and per-token energy are+ unaffected.+* The SDK exposes no way to select AFM 3 Core (dense ~3B) versus AFM 3 Core+ Advanced (20B sparse MoE, 1-4B active per request); the framework's dynamic+ profile picks one from the host device's capabilities. ``model`` is a run+ label only — :meth:`AppleFMEngine.describe` records the SDK version, context+ size and host chip so a run can be attributed after the fact.+* There is no Private Cloud Compute path in the Python SDK, which suits+ OpenJarvis: off-device inference would make the on-device energy+ measurement meaningless.+"""++from __future__ import annotations++import asyncio+import logging+import platform+import subprocess+import time+from collections.abc import AsyncIterator, Sequence+from importlib.metadata import PackageNotFoundError+from importlib.metadata import version as _dist_version+from typing import Any, Dict, List, Optional++# Top-level import on purpose: it gates registration. ``engine/__init__.py``+# imports this module inside a try/except, so on a host without the SDK the+# engine simply does not register.+import apple_fm_sdk as fm++from openjarvis.core.registry import EngineRegistry+from openjarvis.core.types import Message, Role+from openjarvis.engine._apple_fm_support import (+ MODEL_LABELS,+ SnapshotAccumulator,+ build_options_kwargs,+ parse_sampling_spec,+ validate_model_label,+)+from openjarvis.engine._async_loop import AsyncLoopRunner+from openjarvis.engine._base import (+ EngineConnectionError,+ EngineContextLengthError,+ InferenceEngine,+)+from openjarvis.engine._stubs import StreamChunk++logger = logging.getLogger(__name__)++DEFAULT_MODEL_LABEL = "afm-3"+++def _require(path: str) -> Any:+ """Fetch a dotted attribute from the SDK, or fail as an ImportError.++ An SDK that imports but lacks the surface this engine needs (anything+ below 0.2.1, which introduced ``token_count`` and ``context_size``) is an+ unusable install, not a broken engine. Raising ImportError lets+ ``engine/__init__.py``'s guarded import skip registration instead of+ letting an AttributeError take down every other engine.+ """+ obj: Any = fm+ for part in path.split("."):+ try:+ obj = getattr(obj, part)+ except AttributeError as exc:+ raise ImportError(+ f"apple_fm_sdk is missing {path!r}. The AFM engine needs "+ "apple-fm-sdk>=0.2.1; upgrade with "+ "`uv pip install -U 'apple-fm-sdk>=0.2.1'`."+ ) from exc+ return obj+++_USE_CASES = {+ "general": _require("SystemLanguageModelUseCase.GENERAL"),+ "content_tagging": _require("SystemLanguageModelUseCase.CONTENT_TAGGING"),+}+_GUARDRAILS = {+ "default": _require("SystemLanguageModelGuardrails.DEFAULT"),+ "permissive_content_transformations": _require(+ "SystemLanguageModelGuardrails.PERMISSIVE_CONTENT_TRANSFORMATIONS"+ ),+}+# token_count / context_size are the 0.2.1 additions the engine's whole token+# accounting rests on -- check them at import so a stale SDK degrades to "not+# registered" rather than to silently zeroed usage.+_require("SystemLanguageModel.token_count")+_require("SystemLanguageModel.context_size")+++def _resolve_error(name: str) -> Optional[type]:+ """Look an SDK exception class up by name.++ Resolved by name rather than imported so a future SDK release renaming one+ of these degrades to a narrower catch instead of breaking import outright.+ """+ exc = getattr(fm, name, None)+ if isinstance(exc, type) and issubclass(exc, BaseException):+ return exc+ return None+++def _error_tuple(*names: str) -> tuple[type[BaseException], ...]:+ found = tuple(e for e in (_resolve_error(n) for n in names) if e is not None)+ if not found: # pragma: no cover - guards a wholesale SDK rename+ logger.warning(+ "No known apple_fm_sdk error classes found among %s; AFM failures "+ "will surface as generic errors.",+ ", ".join(names),+ )+ return found+++# Overflow is routine at a 4096-token context, and callers already have a+# dedicated "conversation too long" path for it.+_CONTEXT_ERRORS = _error_tuple("ExceededContextWindowSizeError")+# Refusals and guardrail trips are per-query outcomes, not engine faults: a+# long eval run must not die because one prompt was declined.+_REFUSAL_ERRORS = _error_tuple(+ "GuardrailViolationError",+ "RefusalError",+ "UnsupportedLanguageOrLocaleError",+)+++def _host_chip() -> str:+ """Identify the Apple Silicon chip, e.g. "Apple M1 Pro".++ ``platform.processor()`` returns a bare "arm" on every Apple Silicon Mac,+ which is useless for the one job this has: the executing AFM variant is+ not observable, so the host chip is how a run gets attributed later.+ """+ try:+ result = subprocess.run(+ ["sysctl", "-n", "machdep.cpu.brand_string"],+ capture_output=True,+ text=True,+ timeout=5.0,+ check=True,+ )+ except (OSError, subprocess.SubprocessError): # pragma: no cover - host dependent+ return platform.processor() or platform.machine()+ return result.stdout.strip() or platform.processor() or platform.machine()++[email protected]("afm")+class AppleFMEngine(InferenceEngine):+ """Apple Foundation Models, driven in-process via ``apple_fm_sdk``."""++ engine_id = "afm"++ def __init__(+ self,+ instructions: str = "",+ use_case: str = "general",+ guardrails: str = "default",+ sampling: str = "greedy",+ ) -> None:+ self._instructions = str(instructions or "").strip()+ self._use_case_name = str(use_case or "general").strip().lower()+ self._guardrails_name = str(guardrails or "default").strip().lower()+ self._sampling = str(sampling or "greedy").strip().lower()++ if self._use_case_name not in _USE_CASES:+ raise ValueError(+ f"Unknown AFM use_case {use_case!r}. Expected one of: "+ f"{', '.join(_USE_CASES)}."+ )+ if self._guardrails_name not in _GUARDRAILS:+ raise ValueError(+ f"Unknown AFM guardrails {guardrails!r}. Expected one of: "+ f"{', '.join(_GUARDRAILS)}."+ )+ # Validate eagerly so a typo fails at construction rather than midway+ # through a run.+ parse_sampling_spec({"sampling": self._sampling})++ self._model: Any = None+ self._context_size: Optional[int] = None+ self._prepared = False+ self._loop: Optional[AsyncLoopRunner] = None+ self._skipped: Dict[str, int] = {}++ # ---- lifecycle -------------------------------------------------------++ def _ensure_model(self) -> Any:+ if self._model is None:+ self._model = fm.SystemLanguageModel(+ use_case=_USE_CASES[self._use_case_name],+ guardrails=_GUARDRAILS[self._guardrails_name],+ )+ return self._model++ def _ensure_loop(self) -> AsyncLoopRunner:+ if self._loop is None:+ self._loop = AsyncLoopRunner(name="openjarvis-afm")+ return self._loop++ def health(self) -> bool:+ """Report on-device model availability.++ Returns a bool for the ABC, but logs the SDK's reason first: a bare+ False would hide whether the cause is Apple Intelligence being switched+ off, an ineligible device, or assets still downloading.+ """+ try:+ available, reason = self._ensure_model().is_available()+ except Exception as exc: # pragma: no cover - depends on host state+ logger.warning("Could not initialize Apple Foundation Models: %s", exc)+ return False+ if not available:+ logger.warning(+ "Apple Foundation Models unavailable: %s. Check that Apple "+ "Intelligence is enabled in System Settings, that this device "+ "is eligible, and that model assets have finished downloading.",+ getattr(reason, "name", reason),+ )+ return bool(available)++ def list_models(self) -> List[str]:+ return list(MODEL_LABELS)++ def can_serve(self, model: str) -> bool:+ """Only the AFM labels — this engine cannot serve arbitrary model ids.++ Unlike the other local engines, "is the model installed" is not a+ separate concern here: there is exactly one on-device model, so+ accepting any id would let engine selection route e.g. a Llama request+ to AFM and silently answer with a different model.+ """+ return (model or "").strip().lower() in MODEL_LABELS++ def prepare(self, model: str) -> None:+ """Validate the label, then warm up outside the first energy window.++ The first request after boot pays a large one-off cost for model asset+ load and XPC setup (~10s versus ~450ms warm on an M1 Pro). Burning it+ here keeps it out of the telemetry window of the first real request.+ """+ validate_model_label(model)+ language_model = self._ensure_model()++ available, reason = language_model.is_available()+ if not available:+ raise EngineConnectionError(+ "Apple Foundation Models is unavailable "+ f"({getattr(reason, 'name', reason)}). Enable Apple "+ "Intelligence in System Settings and wait for model assets to "+ "download."+ )++ self._context_size = int(language_model.context_size)+ self._ensure_loop().run(self._warmup())+ self._prepared = True++ def close(self) -> None:+ if self._skipped:+ logger.warning(+ "AFM declined %d requests: %s",+ sum(self._skipped.values()),+ ", ".join(f"{k}={v}" for k, v in sorted(self._skipped.items())),+ )+ if self._loop is not None:+ self._loop.shutdown()+ self._loop = None+ self._model = None+ self._prepared = False++ def describe(self) -> Dict[str, Any]:+ """Run metadata. The executing variant is not observable, so the host+ chip and SDK version are the only way to attribute a run later."""+ try:+ sdk_version: Optional[str] = _dist_version("apple-fm-sdk")+ except PackageNotFoundError: # pragma: no cover - installed by definition+ sdk_version = None+ return {+ "engine_id": self.engine_id,+ "afm_sdk_version": sdk_version,+ "afm_context_size": self._context_size,+ "afm_use_case": self._use_case_name,+ "afm_guardrails": self._guardrails_name,+ "afm_instructions": self._instructions or None,+ "afm_sampling": self._sampling,+ "host_chip": _host_chip(),+ "host_os_version": platform.mac_ver()[0] or None,+ "variant_selection": (+ "device-chosen by the Foundation Models dynamic profile; "+ "the model name is a label only"+ ),+ "declined_requests": dict(self._skipped) or None,+ }++ # ---- prompt / options ------------------------------------------------++ def _split_messages(self, messages: Sequence[Message]) -> tuple[str, str]:+ """Split *messages* into (instructions, prompt).++ System messages become the session's ``instructions``, which is where+ the SDK wants them — folding them into the prompt text (as the shim+ historically did) both loses that distinction and inflates the+ prompt-token count.+ """+ system_parts: List[str] = []+ if self._instructions:+ system_parts.append(self._instructions)+ prompt_parts: List[str] = []+ for m in messages:+ if m.role == Role.SYSTEM:+ system_parts.append(m.content)+ elif m.role in (Role.USER, Role.ASSISTANT):+ prompt_parts.append(m.content)+ return "\n".join(p for p in system_parts if p), "\n".join(prompt_parts)++ def _build_options(self, temperature: float, max_tokens: int) -> Any:+ kwargs = build_options_kwargs(+ {"temperature": temperature, "max_tokens": max_tokens}+ )+ mode, mode_kwargs = parse_sampling_spec({"sampling": self._sampling})+ factory = getattr(fm.SamplingMode, mode, None)+ if callable(factory):+ kwargs["sampling"] = factory(**mode_kwargs)+ return fm.GenerationOptions(**kwargs)++ def _new_session(self, instructions: str) -> Any:+ """A fresh session per request.++ Sessions accumulate a transcript, so reusing one would grow the prompt+ until it overflows the 4096-token context after a handful of requests,+ and would make per-request energy depend on preceding requests.+ """+ return fm.LanguageModelSession(+ instructions=instructions or None,+ model=self._ensure_model(),+ )++ # ---- token accounting ------------------------------------------------++ async def _count(self, *args: Any, **kwargs: Any) -> Optional[int]:+ """``token_count`` that degrades to ``None`` rather than failing."""+ try:+ return int(await self._ensure_model().token_count(*args, **kwargs))+ except Exception:+ logger.debug("AFM token_count failed", exc_info=True)+ return None++ @staticmethod+ def _usage(+ prompt_tokens: Optional[int],+ completion_tokens: Optional[int],+ ) -> Dict[str, int]:+ prompt = prompt_tokens or 0+ completion = completion_tokens or 0+ return {+ "prompt_tokens": prompt,+ "prompt_tokens_evaluated": prompt,+ "completion_tokens": completion,+ "total_tokens": prompt + completion,+ }++ # ---- error mapping ---------------------------------------------------++ @staticmethod+ def _sdk_detail(exc: BaseException) -> str:+ """The SDK's own message, minus its noise.++ The SDK formats messages as "<reason>: <detail>" and passes ``None``+ as the detail when it has none, so the raw text ends in a dangling+ ": None". Left alone that reads as a bug in the error path rather+ than a plain context overflow, and it lands in every affected eval+ row.+ """+ text = str(exc).strip()+ while text.endswith(": None"):+ text = text[: -len(": None")].strip()+ return text++ def _translate(self, exc: BaseException) -> BaseException:+ if _CONTEXT_ERRORS and isinstance(exc, _CONTEXT_ERRORS):+ size = self._context_size or "the model's"+ detail = self._sdk_detail(exc)+ message = (+ f"Prompt exceeds the {size}-token context window for Apple "+ "Foundation Models"+ )+ # Only append the SDK's text when it says something the sentence+ # above does not already convey.+ if detail and "context window size exceeded" not in detail.lower():+ message = f"{message}: {detail}"+ return EngineContextLengthError(message)+ return exc++ def _note_declined(self, exc: BaseException) -> None:+ reason = type(exc).__name__+ self._skipped[reason] = self._skipped.get(reason, 0) + 1+ logger.warning("AFM declined a request (%s): %s", reason, exc)++ # ---- inference -------------------------------------------------------++ def generate(+ self,+ messages: Sequence[Message],+ *,+ model: str = DEFAULT_MODEL_LABEL,+ temperature: float = 0.7,+ max_tokens: int = 1024,+ **kwargs: Any,+ ) -> Dict[str, Any]:+ if not self._prepared:+ self.prepare(model)+ return self._ensure_loop().run(+ self._respond(messages, model, temperature, max_tokens)+ )++ async def _respond(+ self,+ messages: Sequence[Message],+ model: str,+ temperature: float,+ max_tokens: int,+ ) -> Dict[str, Any]:+ instructions, prompt = self._split_messages(messages)+ options = self._build_options(temperature, max_tokens)+ session = self._new_session(instructions)++ # Token accounting: `token_count` rejects a value and instructions+ # together ("Provide either a value or instructions"), and counting+ # instructions standalone does not match how the transcript encodes+ # them. So both ends are measured against the transcript instead:+ # prompt_tokens = <transcript before generating> + <prompt alone>+ # completion_tokens = <transcript after generating> - prompt_tokens+ pre_tokens = await self._count(session.transcript)+ prompt_only = await self._count(prompt)++ accumulator = SnapshotAccumulator()+ start = time.time()+ ttft = 0.0++ try:+ async for snapshot in session.stream_response(prompt, options=options):+ if not accumulator.add(snapshot):+ continue+ if ttft == 0.0:+ ttft = time.time() - start+ except _REFUSAL_ERRORS as exc:+ self._note_declined(exc)+ return {+ "content": "",+ "usage": self._usage(prompt_only, 0),+ "model": model,+ "finish_reason": "content_filter",+ "ttft": 0.0,+ }+ except Exception as exc:+ raise self._translate(exc) from exc++ post_tokens = await self._count(session.transcript)+ prompt_tokens = (+ pre_tokens + prompt_only+ if pre_tokens is not None and prompt_only is not None+ else prompt_only+ )+ completion_tokens = (+ max(post_tokens - prompt_tokens, 0)+ if post_tokens is not None and prompt_tokens is not None+ else None+ )++ return {+ "content": accumulator.text,+ "usage": self._usage(prompt_tokens, completion_tokens),+ "model": model,+ "finish_reason": "stop",+ "ttft": ttft,+ }++ async def stream(+ self,+ messages: Sequence[Message],+ *,+ model: str = DEFAULT_MODEL_LABEL,+ temperature: float = 0.7,+ max_tokens: int = 1024,+ **kwargs: Any,+ ) -> AsyncIterator[str]:+ async for chunk in self.stream_full(+ messages,+ model=model,+ temperature=temperature,+ max_tokens=max_tokens,+ **kwargs,+ ):+ if chunk.content:+ yield chunk.content++ async def stream_full(+ self,+ messages: Sequence[Message],+ *,+ model: str = DEFAULT_MODEL_LABEL,+ temperature: float = 0.7,+ max_tokens: int = 1024,+ **kwargs: Any,+ ) -> AsyncIterator[StreamChunk]:+ if not self._prepared:+ # prepare() blocks for the ~10s cold start; off-thread so it does+ # not stall the caller's event loop.+ await asyncio.to_thread(self.prepare, model)++ instructions, prompt = self._split_messages(messages)+ options = self._build_options(temperature, max_tokens)+ session = self._new_session(instructions)++ pre_tokens = await self._count(session.transcript)+ prompt_only = await self._count(prompt)+ accumulator = SnapshotAccumulator()++ try:+ async for snapshot in session.stream_response(prompt, options=options):+ delta = accumulator.add(snapshot)+ if delta:+ yield StreamChunk(content=delta)+ except _REFUSAL_ERRORS as exc:+ self._note_declined(exc)+ yield StreamChunk(+ finish_reason="content_filter",+ usage=self._usage(prompt_only, 0),+ )+ return+ except Exception as exc:+ raise self._translate(exc) from exc++ post_tokens = await self._count(session.transcript)+ prompt_tokens = (+ pre_tokens + prompt_only+ if pre_tokens is not None and prompt_only is not None+ else prompt_only+ )+ completion_tokens = (+ max(post_tokens - prompt_tokens, 0)+ if post_tokens is not None and prompt_tokens is not None+ else None+ )+ yield StreamChunk(+ finish_reason="stop",+ usage=self._usage(prompt_tokens, completion_tokens),+ )++ async def _warmup(self) -> None:+ session = self._new_session(self._instructions)+ options = fm.GenerationOptions(maximum_response_tokens=4)+ async for _ in session.stream_response("warmup", options=options):+ pass+++__all__ = ["AppleFMEngine"]diff --git a/src/openjarvis/engine/apple_fm_shim.py b/src/openjarvis/engine/apple_fm_shim.pyindex 9f2acee8..1b1bad43 100644--- a/src/openjarvis/engine/apple_fm_shim.py+++ b/src/openjarvis/engine/apple_fm_shim.py@@ -1,14 +1,15 @@"""Apple Foundation Models shim.-Thin FastAPI server exposing Apple Intelligence's on-device foundation-model as an OpenAI-compatible API. Only runs on macOS 26+ with Apple-Intelligence enabled. Wraps Apple's `apple-fm-sdk`'s-``LanguageModelSession`` as ``/v1/chat/completions`` and ``/v1/models``-endpoints.+Thin FastAPI server exposing Apple Intelligence's on-device foundation model+as an OpenAI-compatible API, for pointing external OpenAI clients at AFM.+Only runs on macOS 26+ with Apple Intelligence enabled. Wraps+``apple-fm-sdk``'s ``LanguageModelSession`` as ``/v1/chat/completions`` and+``/v1/models``.-**Token counts:** The Apple FM SDK does not expose token counts. The-shim returns 0 for all token counts. Throughput and energy benchmarks-will reflect this limitation.+OpenJarvis's own preferred path is the in-process ``afm`` engine+(``openjarvis.engine.apple_fm``), which avoids an HTTP hop and a second+process whose CPU energy would otherwise land inside the same measurement+window.Usage:uvicorn openjarvis.engine.apple_fm_shim:app \@@ -31,24 +32,30 @@ try:import apple_fm_sdk # type: ignore[import-untyped]except ImportError:print(- "apple_fm_shim: apple-fm-sdk is not available. The SDK is not on\n"- "PyPI yet; clone https://github.com/apple/python-apple-fm-sdk and\n"- "install from source:\n"- " git clone https://github.com/apple/python-apple-fm-sdk\n"- " uv pip install -e ./python-apple-fm-sdk\n"- "Requires macOS 26+, Xcode 26+, and Apple Intelligence enabled.",+ "apple_fm_shim: apple-fm-sdk is not available. Install it with:\n"+ " DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \\\n"+ " uv pip install 'apple-fm-sdk>=0.2.1'\n"+ "The SDK compiles Swift bindings at install time, so a full Xcode is\n"+ "required -- Command Line Tools alone fail. Requires macOS 26+ and\n"+ "Apple Intelligence enabled.",file=sys.stderr,)sys.exit(1)import json+import loggingimport timeimport uuid+from typing import Any, Optionalfrom fastapi import FastAPIfrom fastapi.responses import JSONResponse, StreamingResponsefrom pydantic import BaseModel+from openjarvis.engine._apple_fm_support import SnapshotAccumulator++logger = logging.getLogger(__name__)+app = FastAPI(title="Apple FM Shim")MODEL_ID = "apple-fm"@@ -67,21 +74,28 @@ class ChatRequest(BaseModel):stream: bool = False-def _build_prompt(messages: list[ChatMessage]) -> str:- parts: list[str] = []- for m in messages:- if m.role == "system":- parts.append(f"[System] {m.content}")- elif m.role in ("user", "assistant"):- parts.append(m.content)- return "\n".join(parts)+def _system_language_model() -> Any:+ return apple_fm_sdk.SystemLanguageModel()+++def _split_messages(messages: list[ChatMessage]) -> tuple[str, str]:+ """Split into (instructions, prompt).++ System messages become the session's ``instructions``, which is where the+ SDK wants them. Earlier versions prefixed them onto the prompt as+ ``[System] ...``, which both discards the distinction and inflates the+ prompt-token count.+ """+ system_parts = [m.content for m in messages if m.role == "system"]+ prompt_parts = [m.content for m in messages if m.role in ("user", "assistant")]+ return "\n".join(system_parts), "\n".join(prompt_parts)def _generation_options(req: ChatRequest) -> apple_fm_sdk.GenerationOptions:"""Build the per-request GenerationOptions from a ChatRequest.- Apple FM doesn't take ``max_tokens`` / ``temperature`` as positional- args to ``respond`` / ``stream_response`` — they live on a+ Apple FM doesn't take ``max_tokens`` / ``temperature`` as positional args+ to ``respond`` / ``stream_response`` -- they live on a``GenerationOptions`` object passed via the ``options`` kwarg."""return apple_fm_sdk.GenerationOptions(@@ -90,12 +104,40 @@ def _generation_options(req: ChatRequest) -> apple_fm_sdk.GenerationOptions:)+async def _count(model: Any, value: Any) -> Optional[int]:+ """``token_count`` that degrades to ``None`` rather than failing a request."""+ try:+ return int(await model.token_count(value))+ except Exception:+ logger.debug("apple_fm_shim: token_count failed", exc_info=True)+ return None+++def _usage(prompt_tokens: Optional[int], completion_tokens: Optional[int]) -> dict:+ prompt = prompt_tokens or 0+ completion = completion_tokens or 0+ return {+ "prompt_tokens": prompt,+ "completion_tokens": completion,+ "total_tokens": prompt + completion,+ }+++def _completion_tokens(+ post: Optional[int],+ prompt_tokens: Optional[int],+) -> Optional[int]:+ if post is None or prompt_tokens is None:+ return None+ return max(post - prompt_tokens, 0)++@app.get("/health")def health() -> JSONResponse:# SystemLanguageModel.is_available() is an *instance* method that# returns (bool, reason | None). Unpack so we can both gate the# response code and surface the reason for unavailability.- available, reason = apple_fm_sdk.SystemLanguageModel().is_available()+ available, reason = _system_language_model().is_available()if available:return JSONResponse({"status": "ok"}, status_code=200)return JSONResponse(@@ -106,43 +148,57 @@ def health() -> JSONResponse:@app.get("/v1/models")def list_models() -> JSONResponse:- return JSONResponse(- {- "object": "list",- "data": [- {"id": MODEL_ID, "object": "model", "owned_by": "apple"},- ],- }- )+ entry: dict[str, Any] = {+ "id": MODEL_ID,+ "object": "model",+ "owned_by": "apple",+ }+ try:+ entry["context_length"] = int(_system_language_model().context_size)+ except Exception:+ logger.debug("apple_fm_shim: context_size unavailable", exc_info=True)+ return JSONResponse({"object": "list", "data": [entry]})@app.post("/v1/chat/completions", response_model=None)async def chat_completions(req: ChatRequest,) -> JSONResponse | StreamingResponse:- prompt = _build_prompt(req.messages)- session = apple_fm_sdk.LanguageModelSession()+ instructions, prompt = _split_messages(req.messages)+ model = _system_language_model()+ session = apple_fm_sdk.LanguageModelSession(+ instructions=instructions or None,+ model=model,+ )options = _generation_options(req)+ # Token accounting. `token_count` rejects a value and instructions+ # together, and counting instructions standalone does not match how the+ # transcript encodes them, so both ends are measured against the+ # transcript instead:+ # prompt_tokens = <transcript before> + <prompt alone>+ # completion_tokens = <transcript after> - prompt_tokens+ pre_tokens = await _count(model, session.transcript)+ prompt_only = await _count(model, prompt)+ prompt_tokens = (+ pre_tokens + prompt_only+ if pre_tokens is not None and prompt_only is not None+ else prompt_only+ )+if req.stream:async def generate():cid = f"chatcmpl-{uuid.uuid4().hex[:12]}"# Apple FM yields cumulative snapshots, OpenAI clients expect- # incremental deltas — diff against the last snapshot to convert- # (see #378).- sent = ""+ # incremental deltas (see #378). The accumulator also handles+ # guided generation revising a snapshot rather than extending it.+ accumulator = SnapshotAccumulator()async for snapshot in session.stream_response(prompt,options=options,):- if not snapshot.startswith(sent):- # Snapshot diverged (model revised earlier text);- # fall back to resending the full snapshot.- delta = snapshot- else:- delta = snapshot[len(sent) :]- sent = snapshot+ delta = accumulator.add(snapshot)if not delta:continuechunk = {@@ -159,6 +215,8 @@ async def chat_completions(],}yield f"data: {json.dumps(chunk)}\n\n"++ post_tokens = await _count(model, session.transcript)final = {"id": cid,"object": "chat.completion.chunk",@@ -171,6 +229,12 @@ async def chat_completions("finish_reason": "stop",}],+ # Streaming clients that track cost/throughput read usage off+ # the final chunk; omitting it left them with nothing.+ "usage": _usage(+ prompt_tokens,+ _completion_tokens(post_tokens, prompt_tokens),+ ),}yield f"data: {json.dumps(final)}\n\n"yield "data: [DONE]\n\n"@@ -181,6 +245,7 @@ async def chat_completions()text = await session.respond(prompt, options=options)+ post_tokens = await _count(model, session.transcript)cid = f"chatcmpl-{uuid.uuid4().hex[:12]}"return JSONResponse({@@ -195,10 +260,9 @@ async def chat_completions("finish_reason": "stop",}],- "usage": {- "prompt_tokens": 0,- "completion_tokens": 0,- "total_tokens": 0,- },+ "usage": _usage(+ prompt_tokens,+ _completion_tokens(post_tokens, prompt_tokens),+ ),})diff --git a/src/openjarvis/evals/backends/jarvis_agent.py b/src/openjarvis/evals/backends/jarvis_agent.pyindex ef5625da..ac60d922 100644--- a/src/openjarvis/evals/backends/jarvis_agent.py+++ b/src/openjarvis/evals/backends/jarvis_agent.py@@ -193,6 +193,13 @@ class JarvisAgentBackend(InferenceBackend):"energy_joules": energy_joules,"peak_power_w": peak_power_w,"power_watts": telemetry_data.get("power_watts", 0.0),+ "energy_method": telemetry_data.get("energy_method", ""),+ "energy_basis": telemetry_data.get("energy_basis", ""),+ "cpu_energy_joules": telemetry_data.get("cpu_energy_joules", 0.0),+ "gpu_energy_joules": telemetry_data.get("gpu_energy_joules", 0.0),+ "dram_energy_joules": telemetry_data.get("dram_energy_joules", 0.0),+ "ane_energy_joules": telemetry_data.get("ane_energy_joules", 0.0),+ "soc_energy_joules": telemetry_data.get("soc_energy_joules", 0.0),"gpu_utilization_pct": telemetry_data.get("gpu_utilization_pct", 0.0),"throughput_tok_per_sec": telemetry_data.get("throughput_tok_per_sec", 0.0),"trace_data": trace_data,diff --git a/src/openjarvis/evals/backends/jarvis_direct.py b/src/openjarvis/evals/backends/jarvis_direct.pyindex b90ba5cd..e3a539ca 100644--- a/src/openjarvis/evals/backends/jarvis_direct.py+++ b/src/openjarvis/evals/backends/jarvis_direct.py@@ -124,6 +124,13 @@ class JarvisDirectBackend(InferenceBackend):"energy_joules": energy_joules,"peak_power_w": peak_power_w,"power_watts": telemetry_data.get("power_watts", 0.0),+ "energy_method": telemetry_data.get("energy_method", ""),+ "energy_basis": telemetry_data.get("energy_basis", ""),+ "cpu_energy_joules": telemetry_data.get("cpu_energy_joules", 0.0),+ "gpu_energy_joules": telemetry_data.get("gpu_energy_joules", 0.0),+ "dram_energy_joules": telemetry_data.get("dram_energy_joules", 0.0),+ "ane_energy_joules": telemetry_data.get("ane_energy_joules", 0.0),+ "soc_energy_joules": telemetry_data.get("soc_energy_joules", 0.0),"gpu_utilization_pct": telemetry_data.get("gpu_utilization_pct", 0.0),"throughput_tok_per_sec": telemetry_data.get("throughput_tok_per_sec", 0.0),"tool_calls": 0,diff --git a/src/openjarvis/evals/cli.py b/src/openjarvis/evals/cli.pyindex d218770d..6beacc0b 100644--- a/src/openjarvis/evals/cli.py+++ b/src/openjarvis/evals/cli.py@@ -935,10 +935,14 @@ def _run_agentic(# Build TelemetrySession (optional — only if energy monitoring available)telemetry_session = Nonetry:+ from openjarvis.core.config import load_configfrom openjarvis.telemetry.energy_monitor import create_energy_monitorfrom openjarvis.telemetry.session import TelemetrySession- monitor = create_energy_monitor()+ # The eval config carries only a telemetry on/off flag, so the+ # estimate opt-in has to come from the user's JarvisConfig.+ allow_estimates = load_config().telemetry.allow_energy_estimates+ monitor = create_energy_monitor(allow_estimates=allow_estimates)if monitor is not None:telemetry_session = TelemetrySession(monitor=monitor,diff --git a/src/openjarvis/evals/core/agentic_runner.py b/src/openjarvis/evals/core/agentic_runner.pyindex 6c262bbd..0388992d 100644--- a/src/openjarvis/evals/core/agentic_runner.py+++ b/src/openjarvis/evals/core/agentic_runner.py@@ -45,6 +45,23 @@ def _compute_energy_delta(return None+def _resolve_energy_basis(readings: list[Any]) -> Optional[str]:+ """Which rail set these readings should be interpreted against.++ A reading that carries a non-zero ANE or SoC rail came from an Apple+ Silicon monitor, where the GPU rail alone is a partial view of one+ package. Returns ``None`` when there is nothing to judge from, so a trace+ recorded before per-rail sampling stays unlabelled rather than being+ mislabelled "gpu".+ """+ if not readings:+ return None+ for r in readings:+ if getattr(r, "soc_energy_j", 0.0) or getattr(r, "ane_energy_j", 0.0):+ return "soc"+ return "gpu"++def _compute_power_avg(readings: list[Any],power_field: str = "gpu_power_w",@@ -602,8 +619,13 @@ class AgenticRunner:# Query-level energy from telemetry windowquery_gpu_energy = _compute_energy_delta(readings, "gpu_energy_j")query_cpu_energy = _compute_energy_delta(readings, "cpu_energy_j")+ query_ane_energy = _compute_energy_delta(readings, "ane_energy_j")+ query_soc_energy = _compute_energy_delta(readings, "soc_energy_j")query_gpu_power_avg = _compute_power_avg(readings, "gpu_power_w")query_cpu_power_avg = _compute_power_avg(readings, "cpu_power_w")+ query_ane_power_avg = _compute_power_avg(readings, "ane_power_w")+ query_soc_power_avg = _compute_power_avg(readings, "soc_power_w")+ query_energy_basis = _resolve_energy_basis(readings)# Extract MBU from telemetry readingsmbu_values = [@@ -623,6 +645,11 @@ class AgenticRunner:completed=True,query_gpu_energy_joules=query_gpu_energy,query_cpu_energy_joules=query_cpu_energy,+ query_ane_energy_joules=query_ane_energy,+ query_soc_energy_joules=query_soc_energy,+ query_ane_power_avg_watts=query_ane_power_avg,+ query_soc_power_avg_watts=query_soc_power_avg,+ energy_basis=query_energy_basis,query_gpu_power_avg_watts=query_gpu_power_avg,query_cpu_power_avg_watts=query_cpu_power_avg,is_resolved=record.metadata.get("is_resolved"),@@ -653,13 +680,21 @@ class AgenticRunner:window = [r for r in readings if start_ns <= r.timestamp_ns <= end_ns]gpu_energy = _compute_energy_delta(window, "gpu_energy_j")cpu_energy = _compute_energy_delta(window, "cpu_energy_j")+ ane_energy = _compute_energy_delta(window, "ane_energy_j")+ soc_energy = _compute_energy_delta(window, "soc_energy_j")avg_gpu_power = _compute_power_avg(window, "gpu_power_w")avg_cpu_power = _compute_power_avg(window, "cpu_power_w")+ avg_ane_power = _compute_power_avg(window, "ane_power_w")+ avg_soc_power = _compute_power_avg(window, "soc_power_w")return {"gpu_energy_joules": gpu_energy,"cpu_energy_joules": cpu_energy,+ "ane_energy_joules": ane_energy,+ "soc_energy_joules": soc_energy,"avg_gpu_power_watts": avg_gpu_power,"avg_cpu_power_watts": avg_cpu_power,+ "avg_ane_power_watts": avg_ane_power,+ "avg_soc_power_watts": avg_soc_power,}def _build_turn_traces(@@ -800,17 +835,20 @@ class AgenticRunner:if has_turn_energy:return trace- total_gpu_energy = _compute_energy_delta(readings, "gpu_energy_j")- total_cpu_energy = _compute_energy_delta(readings, "cpu_energy_j")+ totals = {+ "gpu_energy_joules": _compute_energy_delta(readings, "gpu_energy_j"),+ "cpu_energy_joules": _compute_energy_delta(readings, "cpu_energy_j"),+ "ane_energy_joules": _compute_energy_delta(readings, "ane_energy_j"),+ "soc_energy_joules": _compute_energy_delta(readings, "soc_energy_j"),+ }total_wall = sum(t.wall_clock_s for t in trace.turns)if total_wall > 0:for turn in trace.turns:fraction = turn.wall_clock_s / total_wall- if total_gpu_energy is not None:- turn.gpu_energy_joules = total_gpu_energy * fraction- if total_cpu_energy is not None:- turn.cpu_energy_joules = total_cpu_energy * fraction+ for attr, total in totals.items():+ if total is not None:+ setattr(turn, attr, total * fraction)return tracediff --git a/src/openjarvis/evals/core/runner.py b/src/openjarvis/evals/core/runner.pyindex f40475e6..6daf113b 100644--- a/src/openjarvis/evals/core/runner.py+++ b/src/openjarvis/evals/core/runner.py@@ -322,6 +322,13 @@ class EvalRunner:ttft=full.get("ttft", 0.0) or 0.0,energy_joules=energy_j,power_watts=power_w,+ cpu_energy_joules=full.get("cpu_energy_joules", 0.0) or 0.0,+ gpu_energy_joules=full.get("gpu_energy_joules", 0.0) or 0.0,+ dram_energy_joules=full.get("dram_energy_joules", 0.0) or 0.0,+ ane_energy_joules=full.get("ane_energy_joules", 0.0) or 0.0,+ soc_energy_joules=full.get("soc_energy_joules", 0.0) or 0.0,+ energy_basis=full.get("energy_basis", "") or "",+ energy_method=full.get("energy_method", "") or "",gpu_utilization_pct=full.get("gpu_utilization_pct", 0.0) or 0.0,throughput_tok_per_sec=full.get("throughput_tok_per_sec", 0.0) or 0.0,trace_data=full.get("trace_data"),@@ -483,6 +490,13 @@ class EvalRunner:ttft=full.get("ttft", 0.0) or 0.0,energy_joules=energy_j,power_watts=power_w,+ cpu_energy_joules=full.get("cpu_energy_joules", 0.0) or 0.0,+ gpu_energy_joules=full.get("gpu_energy_joules", 0.0) or 0.0,+ dram_energy_joules=full.get("dram_energy_joules", 0.0) or 0.0,+ ane_energy_joules=full.get("ane_energy_joules", 0.0) or 0.0,+ soc_energy_joules=full.get("soc_energy_joules", 0.0) or 0.0,+ energy_basis=full.get("energy_basis", "") or "",+ energy_method=full.get("energy_method", "") or "",gpu_utilization_pct=full.get("gpu_utilization_pct", 0.0) or 0.0,throughput_tok_per_sec=throughput,mfu_pct=mfu,@@ -861,6 +875,13 @@ class EvalRunner:"ttft": result.ttft,"energy_joules": result.energy_joules,"power_watts": result.power_watts,+ "cpu_energy_joules": result.cpu_energy_joules,+ "gpu_energy_joules": result.gpu_energy_joules,+ "dram_energy_joules": result.dram_energy_joules,+ "ane_energy_joules": result.ane_energy_joules,+ "soc_energy_joules": result.soc_energy_joules,+ "energy_basis": result.energy_basis,+ "energy_method": result.energy_method,"gpu_utilization_pct": result.gpu_utilization_pct,"throughput_tok_per_sec": result.throughput_tok_per_sec,"mfu_pct": result.mfu_pct,@@ -1004,6 +1025,28 @@ class EvalRunner:]total_energy = sum(r.energy_joules for r in results)+ # Per-rail totals, so a summary says which hardware did the work+ # rather than only how much it cost in aggregate.+ rail_totals = {+ rail: round(+ sum(getattr(r, f"{rail}_energy_joules", 0.0) for r in results), 6+ )+ for rail in ("cpu", "gpu", "dram", "ane", "soc")+ }+ rail_totals = {k: v for k, v in rail_totals.items() if v > 0}++ # How the energy was obtained, and from which rail set. Both are+ # per-record, but a run uses one monitor throughout; report the value+ # when it is unambiguous and flag genuine mixtures rather than+ # silently picking one.+ def _consensus(attr: str) -> str:+ seen = {getattr(r, attr, "") for r in results if getattr(r, attr, "")}+ if not seen:+ return ""+ return seen.pop() if len(seen) == 1 else "mixed"++ energy_method_value = _consensus("energy_method")+ energy_basis_value = _consensus("energy_basis")total_estimated_flops = sum(r.estimated_flops for r in results)total_input_tokens = sum(r.prompt_tokens for r in results)total_output_tokens = sum(r.completion_tokens for r in results)@@ -1015,6 +1058,9 @@ class EvalRunner:"total_energy_joules": round(total_energy, 6),"avg_power_watts": round(avg_power, 4),"total_estimated_flops": total_estimated_flops,+ "energy_method": energy_method_value,+ "energy_basis": energy_basis_value,+ "energy_joules_by_rail": rail_totals or None,"ipj": (round(accuracy / total_energy, 6) if total_energy > 0 else None),"ipw": (round(accuracy / avg_power, 6) if avg_power > 0 else None),}@@ -1057,6 +1103,7 @@ class EvalRunner:input_token_stats=_metric_stats([float(v) for v in input_tok_vals]),output_token_stats=_metric_stats([float(v) for v in output_tok_vals]),total_energy_joules=round(total_energy, 6),+ energy_method=energy_method_value,total_estimated_flops=total_estimated_flops,flops_stats=_metric_stats(flops_vals),warmup_samples_excluded=cfg.warmup_samples,diff --git a/src/openjarvis/evals/core/trace.py b/src/openjarvis/evals/core/trace.pyindex 8e2e1a1b..d465ba75 100644--- a/src/openjarvis/evals/core/trace.py+++ b/src/openjarvis/evals/core/trace.py@@ -23,8 +23,17 @@ class TurnTrace:# Energy and cost fieldsgpu_energy_joules: Optional[float] = Nonecpu_energy_joules: Optional[float] = None+ # Apple Neural Engine, populated on Apple Silicon only. Without it an+ # ANE-resident model (Apple Foundation Models runs mostly on the ANE)+ # records almost no energy for the work it actually did.+ ane_energy_joules: Optional[float] = None+ # Every rail measured, summed -- the figure per-token energy should be+ # derived from on a unified-memory SoC.+ soc_energy_joules: Optional[float] = Nonegpu_power_avg_watts: Optional[float] = Nonecpu_power_avg_watts: Optional[float] = None+ ane_power_avg_watts: Optional[float] = None+ soc_power_avg_watts: Optional[float] = Nonecost_usd: Optional[float] = None# Per-action energy breakdown (lm_inference vs tool_call granularity)action_energy_breakdown: Optional[List[Dict[str, Any]]] = None@@ -43,8 +52,12 @@ class TurnTrace:"error": self.error,"gpu_energy_joules": self.gpu_energy_joules,"cpu_energy_joules": self.cpu_energy_joules,+ "ane_energy_joules": self.ane_energy_joules,+ "soc_energy_joules": self.soc_energy_joules,"gpu_power_avg_watts": self.gpu_power_avg_watts,"cpu_power_avg_watts": self.cpu_power_avg_watts,+ "ane_power_avg_watts": self.ane_power_avg_watts,+ "soc_power_avg_watts": self.soc_power_avg_watts,"cost_usd": self.cost_usd,"action_energy_breakdown": self.action_energy_breakdown,"tool_calls": [dict(tc) for tc in self.tool_calls],@@ -63,6 +76,10 @@ class TurnTrace:error=d.get("error"),gpu_energy_joules=d.get("gpu_energy_joules"),cpu_energy_joules=d.get("cpu_energy_joules"),+ ane_energy_joules=d.get("ane_energy_joules"),+ soc_energy_joules=d.get("soc_energy_joules"),+ ane_power_avg_watts=d.get("ane_power_avg_watts"),+ soc_power_avg_watts=d.get("soc_power_avg_watts"),gpu_power_avg_watts=d.get("gpu_power_avg_watts"),cpu_power_avg_watts=d.get("cpu_power_avg_watts"),cost_usd=d.get("cost_usd"),@@ -86,8 +103,16 @@ class QueryTrace:# Query-level energy fields (populated even when turns are empty)query_gpu_energy_joules: Optional[float] = Nonequery_cpu_energy_joules: Optional[float] = None+ query_ane_energy_joules: Optional[float] = None+ query_soc_energy_joules: Optional[float] = Nonequery_gpu_power_avg_watts: Optional[float] = Nonequery_cpu_power_avg_watts: Optional[float] = None+ query_ane_power_avg_watts: Optional[float] = None+ query_soc_power_avg_watts: Optional[float] = None+ # Rail set the energy figures above should be read from: "soc" on Apple+ # Silicon, "gpu" elsewhere. Energy and power must be taken from the same+ # basis -- SoC joules over GPU watts is a meaningless ratio.+ energy_basis: Optional[str] = Noneis_resolved: Optional[bool] = Nonequery_mbu_avg_pct: Optional[float] = Nonequery_mbu_max_pct: Optional[float] = None@@ -137,6 +162,36 @@ class QueryTrace:return sum(values)return self.query_cpu_energy_joules+ @property+ def total_ane_energy_joules(self) -> Optional[float]:+ values = [+ t.ane_energy_joules for t in self.turns if t.ane_energy_joules is not None+ ]+ if values:+ return sum(values)+ return self.query_ane_energy_joules++ @property+ def total_soc_energy_joules(self) -> Optional[float]:+ values = [+ t.soc_energy_joules for t in self.turns if t.soc_energy_joules is not None+ ]+ if values:+ return sum(values)+ return self.query_soc_energy_joules++ @property+ def total_basis_energy_joules(self) -> Optional[float]:+ """Energy on whichever rail set this trace was measured against.++ Use this rather than ``total_gpu_energy_joules`` for per-token or+ per-watt figures: on Apple Silicon the GPU rail alone misses the CPU+ and ANE work that actually ran.+ """+ if self.energy_basis == "soc":+ return self.total_soc_energy_joules+ return self.total_gpu_energy_joules+@propertydef total_cost_usd(self) -> Optional[float]:values = [t.cost_usd for t in self.turns if t.cost_usd is not None]@@ -198,8 +253,13 @@ class QueryTrace:"timed_out": self.timed_out,"query_gpu_energy_joules": self.query_gpu_energy_joules,"query_cpu_energy_joules": self.query_cpu_energy_joules,+ "query_ane_energy_joules": self.query_ane_energy_joules,+ "query_soc_energy_joules": self.query_soc_energy_joules,"query_gpu_power_avg_watts": self.query_gpu_power_avg_watts,"query_cpu_power_avg_watts": self.query_cpu_power_avg_watts,+ "query_ane_power_avg_watts": self.query_ane_power_avg_watts,+ "query_soc_power_avg_watts": self.query_soc_power_avg_watts,+ "energy_basis": self.energy_basis,"is_resolved": self.is_resolved,"query_mbu_avg_pct": self.query_mbu_avg_pct,"query_mbu_max_pct": self.query_mbu_max_pct,@@ -220,8 +280,13 @@ class QueryTrace:timed_out=d.get("timed_out", False),query_gpu_energy_joules=d.get("query_gpu_energy_joules"),query_cpu_energy_joules=d.get("query_cpu_energy_joules"),+ query_ane_energy_joules=d.get("query_ane_energy_joules"),+ query_soc_energy_joules=d.get("query_soc_energy_joules"),query_gpu_power_avg_watts=d.get("query_gpu_power_avg_watts"),query_cpu_power_avg_watts=d.get("query_cpu_power_avg_watts"),+ query_ane_power_avg_watts=d.get("query_ane_power_avg_watts"),+ query_soc_power_avg_watts=d.get("query_soc_power_avg_watts"),+ energy_basis=d.get("energy_basis"),is_resolved=d.get("is_resolved"),query_mbu_avg_pct=d.get("query_mbu_avg_pct"),query_mbu_max_pct=d.get("query_mbu_max_pct"),diff --git a/src/openjarvis/evals/core/types.py b/src/openjarvis/evals/core/types.pyindex 78605634..1beaf337 100644--- a/src/openjarvis/evals/core/types.py+++ b/src/openjarvis/evals/core/types.py@@ -36,6 +36,19 @@ class EvalResult:ttft: float = 0.0energy_joules: float = 0.0power_watts: float = 0.0+ # Per-rail breakdown. Without these a reader cannot tell which hardware+ # actually did the work -- and on Apple Silicon that is the whole point:+ # an ANE-resident model draws almost nothing on the GPU rail.+ cpu_energy_joules: float = 0.0+ gpu_energy_joules: float = 0.0+ dram_energy_joules: float = 0.0+ ane_energy_joules: float = 0.0+ soc_energy_joules: float = 0.0+ # Which rail set energy_joules was taken from ("soc" / "gpu"), and how it+ # was obtained. Without energy_method a modelled estimate is+ # indistinguishable from a hardware measurement at read time.+ energy_basis: str = ""+ energy_method: str = ""gpu_utilization_pct: float = 0.0throughput_tok_per_sec: float = 0.0mfu_pct: float = 0.0diff --git a/src/openjarvis/intelligence/model_catalog.py b/src/openjarvis/intelligence/model_catalog.pyindex 52b3b49d..ea010ea6 100644--- a/src/openjarvis/intelligence/model_catalog.py+++ b/src/openjarvis/intelligence/model_catalog.py@@ -1015,6 +1015,55 @@ BUILTIN_MODELS: List[ModelSpec] = ["url": "https://ai.google.dev/gemini-api/docs/models",},),+ # -----------------------------------------------------------------------+ # Apple Foundation Models (on-device, AFM 3)+ #+ # These ids are run *labels*, not selectors: the Foundation Models+ # framework's dynamic profile picks AFM 3 Core (dense ~3B) or AFM 3 Core+ # Advanced (20B sparse MoE, 1-4B active) from the host device, and the SDK+ # exposes no way to choose. Requesting one label does not guarantee that+ # variant executed -- `AppleFMEngine.describe()` records the host chip and+ # SDK version so a run can be attributed after the fact.+ # -----------------------------------------------------------------------+ ModelSpec(+ model_id="afm-3",+ name="Apple Foundation Model 3",+ parameter_count_b=3.0,+ context_length=4096,+ supported_engines=("afm", "apple_fm"),+ provider="apple",+ metadata={+ "architecture": "dense",+ "variant_selection": "device-chosen; model id is a label only",+ },+ ),+ ModelSpec(+ model_id="afm-3-core",+ name="Apple Foundation Model 3 Core",+ parameter_count_b=3.0,+ context_length=4096,+ supported_engines=("afm", "apple_fm"),+ provider="apple",+ metadata={+ "architecture": "dense",+ "variant_selection": "device-chosen; model id is a label only",+ },+ ),+ ModelSpec(+ model_id="afm-3-core-advanced",+ name="Apple Foundation Model 3 Core Advanced",+ # 20B total, but Instruction-Following Pruning activates only 1-4B per+ # request; the 4B upper bound is what FLOPs accounting should use.+ parameter_count_b=20.0,+ active_parameter_count_b=4.0,+ context_length=4096,+ supported_engines=("afm", "apple_fm"),+ provider="apple",+ metadata={+ "architecture": "sparse-moe",+ "variant_selection": "device-chosen; model id is a label only",+ },+ ),]diff --git a/src/openjarvis/sdk.py b/src/openjarvis/sdk.pyindex 43976c41..5f17216e 100644--- a/src/openjarvis/sdk.py+++ b/src/openjarvis/sdk.py@@ -230,6 +230,7 @@ class Jarvis:energy_monitor = create_energy_monitor(prefer_vendor=self._config.telemetry.energy_vendor or None,+ allow_estimates=self._config.telemetry.allow_energy_estimates,)except Exception as exc:logger.debug("Failed to create energy monitor: %s", exc)diff --git a/src/openjarvis/security/data_boundary_audit.py b/src/openjarvis/security/data_boundary_audit.pyindex 025966c6..d104d4f9 100644--- a/src/openjarvis/security/data_boundary_audit.py+++ b/src/openjarvis/security/data_boundary_audit.py@@ -33,6 +33,10 @@ CLOUD_PROVIDER_KEYS = {}LOCAL_ENGINE_KEYS = {+ # Apple Foundation Models, both the in-process engine and the shim. The+ # Python SDK has no Private Cloud Compute path, so inference is+ # unconditionally on-device.+ "afm","apple_fm","exo","gemma_cpp",diff --git a/src/openjarvis/system/builder.py b/src/openjarvis/system/builder.pyindex 7fc9804a..0696ea5a 100644--- a/src/openjarvis/system/builder.py+++ b/src/openjarvis/system/builder.py@@ -165,6 +165,7 @@ class SystemBuilder:energy_monitor = create_energy_monitor(poll_interval_ms=config.telemetry.gpu_poll_interval_ms,prefer_vendor=config.telemetry.energy_vendor or None,+ allow_estimates=config.telemetry.allow_energy_estimates,)except ImportError:passdiff --git a/src/openjarvis/telemetry/aggregator.py b/src/openjarvis/telemetry/aggregator.pyindex b6bb95a9..eae7dc2b 100644--- a/src/openjarvis/telemetry/aggregator.py+++ b/src/openjarvis/telemetry/aggregator.py@@ -33,6 +33,11 @@ class ModelStats:avg_throughput_per_watt: float = 0.0total_prefill_energy_joules: float = 0.0total_decode_energy_joules: float = 0.0+ # Per-rail totals. `total_ane_energy_joules` is what makes an ANE-resident+ # backend (Apple Foundation Models) legible here at all -- on the GPU rail+ # alone it looks like it did no work.+ total_ane_energy_joules: float = 0.0+ total_soc_energy_joules: float = 0.0avg_mean_itl_ms: float = 0.0avg_median_itl_ms: float = 0.0avg_p95_itl_ms: float = 0.0@@ -57,6 +62,11 @@ class EngineStats:avg_throughput_per_watt: float = 0.0total_prefill_energy_joules: float = 0.0total_decode_energy_joules: float = 0.0+ # Per-rail totals. `total_ane_energy_joules` is what makes an ANE-resident+ # backend (Apple Foundation Models) legible here at all -- on the GPU rail+ # alone it looks like it did no work.+ total_ane_energy_joules: float = 0.0+ total_soc_energy_joules: float = 0.0avg_mean_itl_ms: float = 0.0avg_median_itl_ms: float = 0.0avg_p95_itl_ms: float = 0.0@@ -77,6 +87,11 @@ class AggregatedStats:avg_throughput_per_watt: float = 0.0total_prefill_energy_joules: float = 0.0total_decode_energy_joules: float = 0.0+ # Per-rail totals. `total_ane_energy_joules` is what makes an ANE-resident+ # backend (Apple Foundation Models) legible here at all -- on the GPU rail+ # alone it looks like it did no work.+ total_ane_energy_joules: float = 0.0+ total_soc_energy_joules: float = 0.0avg_mean_itl_ms: float = 0.0avg_median_itl_ms: float = 0.0avg_p95_itl_ms: float = 0.0@@ -155,6 +170,7 @@ class TelemetryAggregator:has_derived = self._safe_col("energy_per_output_token_joules")has_phase = self._safe_col("prefill_energy_joules")has_itl = self._safe_col("mean_itl_ms")+ has_rails = self._safe_col("ane_energy_joules")if has_pte:extra_cols += ", SUM(prompt_tokens_evaluated) AS prompt_tokens_evaluated"@@ -177,6 +193,11 @@ class TelemetryAggregator:", AVG(median_itl_ms) AS avg_median_itl_ms"", AVG(p95_itl_ms) AS avg_p95_itl_ms")+ if has_rails:+ extra_cols += (+ ", SUM(ane_energy_joules) AS total_ane_energy_joules"+ ", SUM(soc_energy_joules) AS total_soc_energy_joules"+ )sql = ("SELECT model_id,"@@ -228,6 +249,9 @@ class TelemetryAggregator:ms.avg_mean_itl_ms = r["avg_mean_itl_ms"] or 0.0ms.avg_median_itl_ms = r["avg_median_itl_ms"] or 0.0ms.avg_p95_itl_ms = r["avg_p95_itl_ms"] or 0.0+ if has_rails:+ ms.total_ane_energy_joules = r["total_ane_energy_joules"] or 0.0+ ms.total_soc_energy_joules = r["total_soc_energy_joules"] or 0.0result.append(ms)return result@@ -247,6 +271,7 @@ class TelemetryAggregator:has_derived = self._safe_col("energy_per_output_token_joules")has_phase = self._safe_col("prefill_energy_joules")has_itl = self._safe_col("mean_itl_ms")+ has_rails = self._safe_col("ane_energy_joules")if has_tpj:extra_cols += ", AVG(tokens_per_joule) AS avg_tokens_per_joule"@@ -267,6 +292,11 @@ class TelemetryAggregator:", AVG(median_itl_ms) AS avg_median_itl_ms"", AVG(p95_itl_ms) AS avg_p95_itl_ms")+ if has_rails:+ extra_cols += (+ ", SUM(ane_energy_joules) AS total_ane_energy_joules"+ ", SUM(soc_energy_joules) AS total_soc_energy_joules"+ )sql = ("SELECT engine,"@@ -312,6 +342,9 @@ class TelemetryAggregator:es.avg_mean_itl_ms = r["avg_mean_itl_ms"] or 0.0es.avg_median_itl_ms = r["avg_median_itl_ms"] or 0.0es.avg_p95_itl_ms = r["avg_p95_itl_ms"] or 0.0+ if has_rails:+ es.total_ane_energy_joules = r["total_ane_energy_joules"] or 0.0+ es.total_soc_energy_joules = r["total_soc_energy_joules"] or 0.0result.append(es)return result@@ -364,6 +397,8 @@ class TelemetryAggregator:total_decode_energy_joules=sum(m.total_decode_energy_joules for m in model_stats),+ total_ane_energy_joules=sum(m.total_ane_energy_joules for m in model_stats),+ total_soc_energy_joules=sum(m.total_soc_energy_joules for m in model_stats),avg_mean_itl_ms=_weighted_avg("avg_mean_itl_ms"),avg_median_itl_ms=_weighted_avg("avg_median_itl_ms"),avg_p95_itl_ms=_weighted_avg("avg_p95_itl_ms"),diff --git a/src/openjarvis/telemetry/energy_apple.py b/src/openjarvis/telemetry/energy_apple.pyindex 4cffef1d..ab5318eb 100644--- a/src/openjarvis/telemetry/energy_apple.py+++ b/src/openjarvis/telemetry/energy_apple.py@@ -1,15 +1,35 @@-"""Apple Silicon energy monitor — via zeus-ml[apple] or CPU-time estimation."""+"""Apple Silicon energy monitor — per-rail IOReport counters via zeus-apple-silicon.++Reads the SoC's own energy counters (CPU, GPU, DRAM, Apple Neural Engine)+through ``zeus_apple_silicon``, a nanobind extension over IOReport. No root+required, unlike ``powermetrics``.++**Why not via ``zeus-ml``:** zeus wraps this same library as+``zeus.device.soc.apple.AppleSilicon``, but that module landed on zeus master+in April 2025 and no zeus release has shipped since February 2025 — every+published ``zeus-ml`` (through 0.11.0.post1) has no ``zeus.device.soc``+package at all, and no ``apple`` extra. Depending on the underlying library+directly is the only spelling that actually resolves.++The ANE rail is what makes this worth doing: Apple Foundation Models runs+predominantly on the Neural Engine, so a GPU-only reading makes it look+almost free.+"""from __future__ import annotations+import atexitimport loggingimport platformimport subprocessimport time+import weakreffrom contextlib import contextmanager-from typing import Generator+from dataclasses import dataclass+from typing import Any, Generator, Optionalfrom openjarvis.telemetry.energy_monitor import (+ BASIS_SOC,EnergyMonitor,EnergySample,EnergyVendor,@@ -18,14 +38,33 @@ from openjarvis.telemetry.energy_monitor import (logger = logging.getLogger(__name__)try:- from zeus.device.soc.apple import AppleSiliconMonitor+ from zeus_apple_silicon import AppleEnergyMonitor as _AppleRailReader- _ZEUS_APPLE_AVAILABLE = True+ _NATIVE_AVAILABLE = Trueexcept ImportError:- _ZEUS_APPLE_AVAILABLE = False+ _AppleRailReader = None # type: ignore[assignment]+ _NATIVE_AVAILABLE = False+++# Monitors holding a live native handle. Long-lived callers (`jarvis serve`)+# create one for the process lifetime and never close it, which leaves the+# nanobind object alive when the extension's types are torn down -- the+# extension then prints "leaked instance" warnings on exit. Releasing the+# handles at interpreter shutdown keeps that teardown clean.+_LIVE_MONITORS: "weakref.WeakSet[AppleEnergyMonitor]" = weakref.WeakSet()+++def _release_live_monitors() -> None: # pragma: no cover - shutdown hook+ for monitor in list(_LIVE_MONITORS):+ try:+ monitor.close()+ except Exception:+ pass-# Typical package TDP (watts) by chip family. Used for the CPU-time fallback.+# Typical package TDP (watts) by chip family. Used only for the opt-in+# modelled estimate, never for measured readings._CHIP_TDP: dict[str, float] = {"M1": 15.0,"M1 Pro": 30.0,@@ -42,11 +81,19 @@ _CHIP_TDP: dict[str, float] = {"M4": 15.0,"M4 Pro": 30.0,"M4 Max": 60.0,+ "M5": 15.0,+ "M5 Pro": 30.0,+ "M5 Max": 60.0,+ "M5 Ultra": 90.0,}def _detect_chip() -> tuple[str, float]:- """Return (chip_name, tdp_watts) for the current Apple Silicon SoC."""+ """Return (chip_name, tdp_watts) for the current Apple Silicon SoC.++ ``platform.processor()`` reports a bare "arm" on every Apple Silicon Mac,+ so ``sysctl`` is the only way to name the actual part.+ """try:r = subprocess.run(["sysctl", "-n", "machdep.cpu.brand_string"],@@ -61,124 +108,302 @@ def _detect_chip() -> tuple[str, float]:for chip, tdp in sorted(_CHIP_TDP.items(), key=lambda kv: -len(kv[0])):if chip in brand:- return chip, tdp+ return brand or chip, tdpreturn brand or "Apple Silicon", 20.0+@dataclass+class _Rails:+ """One reading of the SoC's energy rails, in joules.++ A rail is ``None`` when this chip does not expose that counter — which is+ not the same as zero, and must not be summed as if it were.+ """++ cpu: Optional[float] = None+ gpu: Optional[float] = None+ dram: Optional[float] = None+ ane: Optional[float] = None++ @property+ def total(self) -> float:+ """Sum of the rails that are actually present."""+ rails = (self.cpu, self.gpu, self.dram, self.ane)+ return sum(v for v in rails if v is not None)++ def __sub__(self, other: "_Rails") -> "_Rails":+ def diff(a: Optional[float], b: Optional[float]) -> Optional[float]:+ if a is None or b is None:+ return None+ # Counters are monotonic; a negative delta means a wrapped or+ # reset counter, which is better reported as "unknown" than as a+ # negative energy.+ return a - b if a >= b else None++ return _Rails(+ cpu=diff(self.cpu, other.cpu),+ gpu=diff(self.gpu, other.gpu),+ dram=diff(self.dram, other.dram),+ ane=diff(self.ane, other.ane),+ )+++def _mj(value: Any) -> Optional[float]:+ """Convert one millijoule counter to joules, or ``None`` if absent."""+ if value is None:+ return None+ try:+ return float(value) / 1000.0+ except (TypeError, ValueError):+ return None+++def _rails_from_metrics(metrics: Any) -> _Rails:+ """Map a ``zeus_apple_silicon.AppleEnergyMetrics`` onto :class:`_Rails`.++ ``cpu_total_mj`` already subsumes the per-core, per-cluster and+ core-manager breakdowns the object also exposes (verified on an M1 Pro:+ the cluster fields sum to ``cpu_total_mj``), so adding those would double+ count. ``gpu_sram_mj`` is a separate rail from ``gpu_mj`` and does need+ adding.+ """+ gpu = _mj(getattr(metrics, "gpu_mj", None))+ gpu_sram = _mj(getattr(metrics, "gpu_sram_mj", None))+ if gpu is not None and gpu_sram is not None:+ gpu = gpu + gpu_sram+ elif gpu is None:+ gpu = gpu_sram++ return _Rails(+ cpu=_mj(getattr(metrics, "cpu_total_mj", None)),+ gpu=gpu,+ dram=_mj(getattr(metrics, "dram_mj", None)),+ ane=_mj(getattr(metrics, "ane_mj", None)),+ )++class AppleEnergyMonitor(EnergyMonitor):"""Apple Silicon energy monitor.- Prefers ``zeus-ml[apple]`` ``AppleSiliconMonitor`` when available.- Falls back to a CPU-time-based estimation using the chip's known TDP,- which gives order-of-magnitude correct energy readings without root.+ Measures CPU, GPU, DRAM and ANE rails via ``zeus-apple-silicon``. When+ that backend is absent, :meth:`available` reports ``False`` so the factory+ falls through rather than inventing numbers; a modelled TDP estimate is+ reachable only by explicitly constructing with ``allow_estimates=True``."""- def __init__(self, poll_interval_ms: int = 50) -> None:+ def __init__(+ self,+ poll_interval_ms: int = 50,+ allow_estimates: bool = False,+ ) -> None:self._poll_interval_ms = poll_interval_ms- self._monitor = None- self._zeus_ok = False+ self._allow_estimates = allow_estimates+ self._reader: Any = Noneself._chip_name, self._tdp_watts = _detect_chip()+ # Previous cumulative reading, for deriving power in snapshot().+ self._prev: Optional[tuple[float, _Rails]] = None- if _ZEUS_APPLE_AVAILABLE and platform.system() == "Darwin":+ if _NATIVE_AVAILABLE and platform.system() == "Darwin":try:- self._monitor = AppleSiliconMonitor()- self._zeus_ok = True+ self._reader = _AppleRailReader()+ _LIVE_MONITORS.add(self)except Exception as exc:logger.debug("Failed to initialize Apple Silicon energy monitor: %s",exc,)+ # ---- capability probes ---------------------------------------------+@staticmethoddef available() -> bool:+ """True only when rails can actually be *measured*.++ Deliberately stricter than "is this a Mac": returning True on any+ Apple Silicon host is what previously let a modelled estimate+ masquerade as a measurement for every reader downstream.+ """+ if not AppleEnergyMonitor.estimate_available():+ return False+ if not _NATIVE_AVAILABLE:+ return False+ try:+ _AppleRailReader()+ except Exception as exc:+ logger.debug("Apple Silicon energy backend unusable: %s", exc)+ return False+ return True++ @staticmethod+ def estimate_available() -> bool:+ """True on any Apple Silicon host, measurable or not."""return platform.system() == "Darwin" and platform.machine() == "arm64"def vendor(self) -> EnergyVendor:return EnergyVendor.APPLEdef energy_method(self) -> str:- return "zeus" if self._zeus_ok else "cpu_time_estimate"+ return "ioreport" if self._reader is not None else "tdp_estimate"- @contextmanager- def sample(self) -> Generator[EnergySample, None, None]:- result = EnergySample(+ # ---- measurement ----------------------------------------------------++ def _new_sample(self) -> EnergySample:+ return EnergySample(vendor=EnergyVendor.APPLE.value,device_name=self._chip_name,device_count=1,energy_method=self.energy_method(),+ basis=BASIS_SOC,)- if self._zeus_ok and self._monitor is not None:- yield from self._sample_zeus(result)+ @contextmanager+ def sample(self) -> Generator[EnergySample, None, None]:+ result = self._new_sample()++ if self._reader is not None:+ yield from self._sample_ioreport(result)+ elif self._allow_estimates:+ yield from self._sample_estimate(result)else:- yield from self._sample_cputime(result)+ # No backend and no opt-in: hand back an empty sample rather than+ # a fabricated one. Callers see zeros and energy_method tells them+ # why.+ yield result- def _sample_zeus(+ def _sample_ioreport(self,result: EnergySample,) -> Generator[EnergySample, None, None]:- assert self._monitor is not None- window_name = f"openjarvis_{time.monotonic_ns()}"+ assert self._reader is not None+ window = f"openjarvis_{time.monotonic_ns()}"t_start = time.monotonic()- self._monitor.begin_window(window_name)-- yield result-- measurement = self._monitor.end_window(window_name)- wall = time.monotonic() - t_start-- cpu_j = getattr(measurement, "cpu_energy", 0.0)- gpu_j = getattr(measurement, "gpu_energy", 0.0)- dram_j = getattr(measurement, "dram_energy", 0.0)- ane_j = getattr(measurement, "ane_energy", 0.0)+ self._reader.begin_window(window)++ try:+ yield result+ finally:+ # No `return` in this block: returning from a finally discards an+ # exception the body raised, so a failed end_window would swallow+ # the caller's error.+ try:+ metrics = self._reader.end_window(window)+ except Exception as exc:+ logger.debug("Apple energy window failed to close: %s", exc)+ else:+ self._apply(+ result,+ _rails_from_metrics(metrics),+ time.monotonic() - t_start,+ )- result.cpu_energy_joules = float(cpu_j)- result.gpu_energy_joules = float(gpu_j)- result.dram_energy_joules = float(dram_j)- result.ane_energy_joules = float(ane_j)- result.energy_joules = cpu_j + gpu_j + dram_j + ane_j+ @staticmethod+ def _apply(result: EnergySample, rails: _Rails, wall: float) -> None:+ """Write a rail reading and its derived totals onto *result*."""+ result.cpu_energy_joules = rails.cpu or 0.0+ result.gpu_energy_joules = rails.gpu or 0.0+ result.dram_energy_joules = rails.dram or 0.0+ result.ane_energy_joules = rails.ane or 0.0++ total = rails.total+ result.soc_energy_joules = total+ # On Apple Silicon the SoC sum *is* the energy figure — every rail+ # here is on the same package.+ result.energy_joules = totalresult.duration_seconds = wallif wall > 0:- result.mean_power_watts = result.energy_joules / wall-- def _sample_cputime(+ result.mean_power_watts = total / wall+ result.soc_power_watts = total / wall+ result.cpu_power_watts = (rails.cpu or 0.0) / wall+ result.gpu_power_watts = (rails.gpu or 0.0) / wall+ result.ane_power_watts = (rails.ane or 0.0) / wall++ def snapshot(self) -> EnergySample:+ """Instantaneous reading, for background polling by ``TelemetrySession``.++ Energy fields carry the *cumulative* since-boot counters, which is what+ the eval runners difference across a window+ (``_compute_energy_delta`` takes last-minus-first). Power is derived+ from the delta against the previous snapshot, so the first call in a+ session reports zero watts.+ """+ result = self._new_sample()+ if self._reader is None:+ return result++ try:+ metrics = self._reader.get_cumulative_energy()+ except Exception as exc:+ logger.debug("Apple cumulative energy read failed: %s", exc)+ return result++ now = time.monotonic()+ rails = _rails_from_metrics(metrics)++ result.cpu_energy_joules = rails.cpu or 0.0+ result.gpu_energy_joules = rails.gpu or 0.0+ result.dram_energy_joules = rails.dram or 0.0+ result.ane_energy_joules = rails.ane or 0.0+ result.soc_energy_joules = rails.total+ result.energy_joules = rails.total+ result.num_snapshots = 1++ prev = self._prev+ self._prev = (now, rails)+ if prev is not None:+ dt = now - prev[0]+ if dt > 0:+ delta = rails - prev[1]+ result.duration_seconds = dt+ result.cpu_power_watts = (delta.cpu or 0.0) / dt+ result.gpu_power_watts = (delta.gpu or 0.0) / dt+ result.ane_power_watts = (delta.ane or 0.0) / dt+ result.soc_power_watts = delta.total / dt+ result.mean_power_watts = result.soc_power_watts++ return result++ # ---- opt-in modelled estimate --------------------------------------++ def _sample_estimate(self,result: EnergySample,) -> Generator[EnergySample, None, None]:- """Estimate energy from user+system CPU time and chip TDP.-- Energy ≈ (cpu_seconds / wall_seconds) × TDP × wall_seconds- = cpu_seconds × TDP-- This is an approximation — real power varies with clock speed,- workload mix, and thermal state — but it gives useful non-zero- readings without requiring root or external libraries.+ """Model energy as wall-clock × TDP, split by fixed rail ratios.++ This is not a measurement. It reads no counters and no utilization: a+ ten-second sleep and a ten-second generation produce identical joules,+ and the rail split is four constants. It exists only so a host without+ ``zeus-apple-silicon`` can produce order-of-magnitude figures when the+ caller has explicitly asked for estimates, and it always reports+ ``energy_method="tdp_estimate"`` so those figures stay separable from+ real ones."""_ACTIVE_RATIO = 0.60t0 = time.monotonic()- yield result-- wall = time.monotonic() - t0- if wall <= 0:- result.duration_seconds = 0.0- return-- power_w = self._tdp_watts * _ACTIVE_RATIO- energy_j = power_w * wall-- result.gpu_energy_joules = energy_j * 0.55- result.cpu_energy_joules = energy_j * 0.25- result.dram_energy_joules = energy_j * 0.15- result.ane_energy_joules = energy_j * 0.05- result.energy_joules = energy_j- result.duration_seconds = wall- result.mean_power_watts = power_w+ try:+ yield result+ finally:+ # No `return` in this block -- see _sample_ioreport.+ wall = time.monotonic() - t0+ if wall <= 0:+ result.duration_seconds = 0.0+ else:+ power_w = self._tdp_watts * _ACTIVE_RATIO+ energy_j = power_w * wall+ rails = _Rails(+ cpu=energy_j * 0.25,+ gpu=energy_j * 0.55,+ dram=energy_j * 0.15,+ ane=energy_j * 0.05,+ )+ self._apply(result, rails, wall)def close(self) -> None:- self._monitor = None- self._zeus_ok = False+ self._reader = None+ self._prev = None+ _LIVE_MONITORS.discard(self)__all__ = ["AppleEnergyMonitor"]diff --git a/src/openjarvis/telemetry/energy_monitor.py b/src/openjarvis/telemetry/energy_monitor.pyindex 9b869b69..cc25eda5 100644--- a/src/openjarvis/telemetry/energy_monitor.py+++ b/src/openjarvis/telemetry/energy_monitor.py@@ -12,6 +12,14 @@ from typing import Generator, Optionallogger = logging.getLogger(__name__)+# Rail sets that derived per-token / per-watt figures can be computed from.+# ``BASIS_SOC`` means CPU + GPU + ANE (+ DRAM where measured); ``BASIS_GPU``+# means the GPU rail alone. Energy and power must always be read from the+# *same* basis -- SoC joules divided by GPU watts is a meaningless number.+BASIS_SOC = "soc"+BASIS_GPU = "gpu"++class EnergyVendor(str, Enum):"""Supported energy measurement vendors."""@@ -48,7 +56,11 @@ class EnergySample:vendor: str = ""device_name: str = ""device_count: int = 0- energy_method: str = "" # "hw_counter", "polling", "rapl", "zeus"+ # How the numbers above were obtained. "hw_counter" / "polling" (NVIDIA,+ # AMD), "rapl" (Linux sysfs), "ioreport" (Apple Silicon), or+ # "tdp_estimate" -- the last being modelled rather than measured, and+ # opt-in.+ energy_method: str = ""# Per-component breakdown (joules)cpu_energy_joules: float = 0.0@@ -56,6 +68,26 @@ class EnergySample:dram_energy_joules: float = 0.0ane_energy_joules: float = 0.0+ # Per-component instantaneous power (watts). Populated by monitors that+ # can attribute power per rail; ``snapshot()`` callers difference the+ # energy fields above and average these.+ cpu_power_watts: float = 0.0+ gpu_power_watts: float = 0.0+ ane_power_watts: float = 0.0++ # Whole-SoC energy: every rail this monitor measured, summed. On Apple+ # Silicon each rail above is a partial view -- a model that runs on the+ # Neural Engine draws almost nothing on the GPU rail -- so per-token+ # energy must be derived from the sum, not from GPU alone.+ soc_energy_joules: float = 0.0+ soc_power_watts: float = 0.0++ # Which rail set the derived per-token/per-watt numbers should come from:+ # "soc" on Apple Silicon, "gpu" on discrete-GPU hosts. Recorded so+ # downstream aggregation repeats this monitor's choice instead of+ # guessing. Empty on samples produced before this field existed.+ basis: str = ""+class EnergyMonitor(ABC):"""Abstract base class for energy measurement backends.@@ -76,7 +108,13 @@ class EnergyMonitor(ABC):@abstractmethoddef energy_method(self) -> str:- """Return the measurement method: 'hw_counter', 'polling', 'rapl', or 'zeus'."""+ """Return the measurement method.++ One of ``hw_counter``, ``polling``, ``rapl``, ``ioreport``, or+ ``tdp_estimate``. Everything but the last is a real measurement;+ ``tdp_estimate`` is modelled and only reachable when the caller+ explicitly opts in.+ """@abstractmethod@contextmanager@@ -103,12 +141,21 @@ class EnergyMonitor(ABC):def create_energy_monitor(poll_interval_ms: int = 50,prefer_vendor: Optional[str] = None,+ allow_estimates: bool = False,) -> Optional[EnergyMonitor]:"""Factory — auto-detect and return the best available EnergyMonitor.Detection order: NVIDIA > AMD > Apple > CPU RAPL.If *prefer_vendor* is set, try that vendor first.+ Every monitor returned by the normal path reports *measured* energy. When+ *allow_estimates* is set and nothing measurable is present, a modelled+ estimator may be returned instead — currently only on Apple Silicon, and+ only when the IOReport backend is missing. Estimators always report a+ distinct ``energy_method`` so callers can tell modelled joules from+ measured ones. Off by default: a fabricated number that lands in the+ telemetry DB is indistinguishable from a real one at query time.+Returns ``None`` if no energy monitoring is available."""# Build ordered candidate list — imports are defensive because vendor@@ -165,10 +212,29 @@ def create_energy_monitor(logger.debug("Energy monitor candidate failed: %s", exc)continue+ if allow_estimates:+ try:+ from openjarvis.telemetry.energy_apple import AppleEnergyMonitor++ if AppleEnergyMonitor.estimate_available():+ logger.warning(+ "No measurable energy backend found; falling back to a "+ "modelled Apple Silicon estimate. Install "+ "`openjarvis[energy-apple]` for real measurements."+ )+ return AppleEnergyMonitor(+ poll_interval_ms=poll_interval_ms,+ allow_estimates=True,+ )+ except Exception as exc:+ logger.debug("Energy estimate fallback unavailable: %s", exc)+return None__all__ = [+ "BASIS_GPU",+ "BASIS_SOC","EnergyMonitor","EnergySample","EnergyVendor",diff --git a/src/openjarvis/telemetry/flops.py b/src/openjarvis/telemetry/flops.pyindex 67e7b06b..c9195700 100644--- a/src/openjarvis/telemetry/flops.py+++ b/src/openjarvis/telemetry/flops.py@@ -28,6 +28,17 @@ MODEL_PARAMS_B: dict[str, float] = {"llama-3.1-8b": 8.0,"mistral-7b": 7.0,"mixtral-8x7b": 47.0,+ # Apple Foundation Models (on-device, AFM 3). "Core" is a dense ~3B model.+ # "Core Advanced" is a 20B sparse MoE that activates only 1-4B parameters+ # per request via Instruction-Following Pruning, so the 4B upper bound is+ # used here -- FLOPs for that label estimate active compute rather than+ # measuring it. Apple has not published exact parameter counts.+ # Core Advanced is listed first because `_get_params_b` falls back to a+ # prefix scan in insertion order, and "afm-3" is a prefix of both longer+ # labels.+ "afm-3-core-advanced": 4.0,+ "afm-3-core": 3.0,+ "afm-3": 3.0,}diff --git a/src/openjarvis/telemetry/instrumented_engine.py b/src/openjarvis/telemetry/instrumented_engine.pyindex e0007768..129bdb3a 100644--- a/src/openjarvis/telemetry/instrumented_engine.py+++ b/src/openjarvis/telemetry/instrumented_engine.py@@ -10,6 +10,7 @@ from typing import Any, Dict, List, Optional, Sequencefrom openjarvis.core.events import EventBus, EventTypefrom openjarvis.core.types import TOKEN_COUNTING_VERSION, Message, TelemetryRecordfrom openjarvis.engine._stubs import InferenceEngine, StreamChunk+from openjarvis.telemetry.energy_monitor import BASIS_GPUfrom openjarvis.telemetry.gpu_monitor import GpuSample# ---------------------------------------------------------------------------@@ -146,6 +147,9 @@ class InstrumentedEngine(InferenceEngine):cpu_energy_joules = 0.0gpu_energy_joules = 0.0dram_energy_joules = 0.0+ ane_energy_joules = 0.0+ soc_energy_joules = 0.0+ energy_basis = ""if energy_sample is not None:# New multi-vendor EnergyMonitor path@@ -159,6 +163,9 @@ class InstrumentedEngine(InferenceEngine):cpu_energy_joules = energy_sample.cpu_energy_joulesgpu_energy_joules = energy_sample.gpu_energy_joulesdram_energy_joules = energy_sample.dram_energy_joules+ ane_energy_joules = energy_sample.ane_energy_joules+ soc_energy_joules = energy_sample.soc_energy_joules+ energy_basis = energy_sample.basiselif gpu_sample is not None:# Legacy GpuMonitor pathenergy_joules = gpu_sample.energy_joules@@ -168,6 +175,10 @@ class InstrumentedEngine(InferenceEngine):gpu_temperature_c = gpu_sample.mean_temperature_cenergy_method = "polling"energy_vendor = "nvidia"+ # The legacy monitor reads one GPU rail and nothing else.+ gpu_energy_joules = gpu_sample.energy_joules+ soc_energy_joules = gpu_sample.energy_joules+ energy_basis = BASIS_GPUif ttft > 0:prefill_latency = ttft@@ -233,6 +244,9 @@ class InstrumentedEngine(InferenceEngine):cpu_energy_joules=cpu_energy_joules,gpu_energy_joules=gpu_energy_joules,dram_energy_joules=dram_energy_joules,+ ane_energy_joules=ane_energy_joules,+ soc_energy_joules=soc_energy_joules,+ energy_basis=energy_basis,tokens_per_joule=tokens_per_joule,# Stamp every new record with the current methodology version# so the leaderboard aggregator can drop legacy rows (those@@ -294,6 +308,9 @@ class InstrumentedEngine(InferenceEngine):"cpu_energy_joules": cpu_energy_joules,"gpu_energy_joules": gpu_energy_joules,"dram_energy_joules": dram_energy_joules,+ "ane_energy_joules": ane_energy_joules,+ "soc_energy_joules": soc_energy_joules,+ "energy_basis": energy_basis,}return result@@ -381,6 +398,9 @@ class InstrumentedEngine(InferenceEngine):cpu_energy_joules = 0.0gpu_energy_joules = 0.0dram_energy_joules = 0.0+ ane_energy_joules = 0.0+ soc_energy_joules = 0.0+ energy_basis = ""if energy_sample is not None:energy_joules = energy_sample.energy_joules@@ -393,6 +413,9 @@ class InstrumentedEngine(InferenceEngine):cpu_energy_joules = energy_sample.cpu_energy_joulesgpu_energy_joules = energy_sample.gpu_energy_joulesdram_energy_joules = energy_sample.dram_energy_joules+ ane_energy_joules = energy_sample.ane_energy_joules+ soc_energy_joules = energy_sample.soc_energy_joules+ energy_basis = energy_sample.basiselif gpu_sample is not None:energy_joules = gpu_sample.energy_joulespower_watts = gpu_sample.mean_power_watts@@ -401,6 +424,10 @@ class InstrumentedEngine(InferenceEngine):gpu_temperature_c = gpu_sample.mean_temperature_cenergy_method = "polling"energy_vendor = "nvidia"+ # The legacy monitor reads one GPU rail and nothing else.+ gpu_energy_joules = gpu_sample.energy_joules+ soc_energy_joules = gpu_sample.energy_joules+ energy_basis = BASIS_GPUprefill_latency = ttft if ttft > 0 else 0.0@@ -459,6 +486,9 @@ class InstrumentedEngine(InferenceEngine):cpu_energy_joules=cpu_energy_joules,gpu_energy_joules=gpu_energy_joules,dram_energy_joules=dram_energy_joules,+ ane_energy_joules=ane_energy_joules,+ soc_energy_joules=soc_energy_joules,+ energy_basis=energy_basis,tokens_per_joule=tokens_per_joule,# Stamp the methodology version on streaming records too.token_counting_version=TOKEN_COUNTING_VERSION,diff --git a/src/openjarvis/telemetry/session.py b/src/openjarvis/telemetry/session.pyindex 3946912b..234bc4e5 100644--- a/src/openjarvis/telemetry/session.py+++ b/src/openjarvis/telemetry/session.py@@ -25,8 +25,12 @@ class TelemetrySample:timestamp_ns: intgpu_power_w: float = 0.0cpu_power_w: float = 0.0+ ane_power_w: float = 0.0+ soc_power_w: float = 0.0gpu_energy_j: float = 0.0cpu_energy_j: float = 0.0+ ane_energy_j: float = 0.0+ soc_energy_j: float = 0.0gpu_util_pct: float = 0.0gpu_temp_c: float = 0.0gpu_mem_gb: float = 0.0@@ -99,12 +103,20 @@ class TelemetrySession:try:if self._monitor is not None:es = self._monitor.snapshot()+ # `cpu_power_w` was previously hardcoded to 0.0, so every+ # eval trace reported zero CPU power regardless of vendor.+ # Monitors that attribute power per rail now fill it in;+ # those that don't still report 0.0.sample = TelemetrySample(timestamp_ns=time.time_ns(),- gpu_power_w=es.mean_power_watts,- cpu_power_w=0.0,+ gpu_power_w=es.gpu_power_watts or es.mean_power_watts,+ cpu_power_w=es.cpu_power_watts,+ ane_power_w=es.ane_power_watts,+ soc_power_w=es.soc_power_watts or es.mean_power_watts,gpu_energy_j=es.gpu_energy_joules,cpu_energy_j=es.cpu_energy_joules,+ ane_energy_j=es.ane_energy_joules,+ soc_energy_j=es.soc_energy_joules or es.energy_joules,gpu_util_pct=es.mean_utilization_pct,gpu_temp_c=es.mean_temperature_c,gpu_mem_gb=es.mean_memory_used_gb,diff --git a/src/openjarvis/telemetry/store.py b/src/openjarvis/telemetry/store.pyindex d2fdfc97..b93ea873 100644--- a/src/openjarvis/telemetry/store.py+++ b/src/openjarvis/telemetry/store.py@@ -44,6 +44,12 @@ CREATE TABLE IF NOT EXISTS telemetry (cpu_energy_joules REAL NOT NULL DEFAULT 0.0,gpu_energy_joules REAL NOT NULL DEFAULT 0.0,dram_energy_joules REAL NOT NULL DEFAULT 0.0,+ ane_energy_joules REAL NOT NULL DEFAULT 0.0,+ soc_energy_joules REAL NOT NULL DEFAULT 0.0,+ -- Rail set the derived per-token/per-watt columns were computed from:+ -- 'soc' on Apple Silicon, 'gpu' elsewhere. '' on rows written before the+ -- column existed, which are GPU-basis by construction.+ energy_basis TEXT NOT NULL DEFAULT '',tokens_per_joule REAL NOT NULL DEFAULT 0.0,energy_per_output_token_joules REAL NOT NULL DEFAULT 0.0,throughput_per_watt REAL NOT NULL DEFAULT 0.0,@@ -92,6 +98,7 @@ INSERT INTO telemetry (throughput_tok_per_sec, prefill_latency_seconds, decode_latency_seconds,energy_method, energy_vendor, batch_id, is_warmup,cpu_energy_joules, gpu_energy_joules, dram_energy_joules,+ ane_energy_joules, soc_energy_joules, energy_basis,tokens_per_joule,energy_per_output_token_joules, throughput_per_watt,prefill_energy_joules, decode_energy_joules,@@ -104,7 +111,8 @@ INSERT INTO telemetry (?, ?, ?, ?, ?, ?, ?, ?, ?, ?,?, ?, ?, ?, ?, ?, ?, ?, ?, ?,?, ?, ?, ?, ?, ?, ?, ?, ?, ?,- ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?+ ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,+ ?, ?, ?, ?)"""@@ -129,6 +137,9 @@ _MIGRATE_COLUMNS = [("cpu_energy_joules", "REAL NOT NULL DEFAULT 0.0"),("gpu_energy_joules", "REAL NOT NULL DEFAULT 0.0"),("dram_energy_joules", "REAL NOT NULL DEFAULT 0.0"),+ ("ane_energy_joules", "REAL NOT NULL DEFAULT 0.0"),+ ("soc_energy_joules", "REAL NOT NULL DEFAULT 0.0"),+ ("energy_basis", "TEXT NOT NULL DEFAULT ''"),("tokens_per_joule", "REAL NOT NULL DEFAULT 0.0"),("energy_per_output_token_joules", "REAL NOT NULL DEFAULT 0.0"),("throughput_per_watt", "REAL NOT NULL DEFAULT 0.0"),@@ -262,6 +273,9 @@ class TelemetryStore:rec.cpu_energy_joules,rec.gpu_energy_joules,rec.dram_energy_joules,+ rec.ane_energy_joules,+ rec.soc_energy_joules,+ rec.energy_basis,rec.tokens_per_joule,rec.energy_per_output_token_joules,rec.throughput_per_watt,diff --git a/tests/bench/test_engine_info.py b/tests/bench/test_engine_info.pynew file mode 100644index 00000000..705489b8--- /dev/null+++ b/tests/bench/test_engine_info.py@@ -0,0 +1,88 @@+"""Tests for ``engine_info`` — run metadata from self-describing engines.++Benchmark results are written to JSONL, so whatever ``describe()`` returns+has to survive ``json.dumps``. It does not come from a trusted source: it is+duck-typed off whatever object the caller passed as the engine.+"""++from __future__ import annotations++import json+from unittest.mock import MagicMock++from openjarvis.bench._stubs import engine_info+++class _Described:+ def __init__(self, payload):+ self._payload = payload++ def describe(self):+ return self._payload+++def test_returns_none_when_engine_cannot_describe_itself():+ engine = object()+ assert engine_info(engine) is None+++def test_returns_the_described_payload():+ info = engine_info(+ _Described(+ {+ "engine_id": "afm",+ "afm_context_size": 4096,+ "afm_instructions": None,+ }+ )+ )+ assert info == {+ "engine_id": "afm",+ "afm_context_size": 4096,+ "afm_instructions": None,+ }+++def test_mock_engine_does_not_poison_the_jsonl_write():+ """A MagicMock answers ``describe()`` with another MagicMock.++ Passing that straight into benchmark metadata made the whole run fail with+ "Object of type MagicMock is not JSON serializable" at write time -- which+ is exactly the shape of an engine returning something unexpected.+ """+ engine = MagicMock()+ assert engine_info(engine) is None+++def test_non_dict_payloads_are_rejected():+ assert engine_info(_Described("just a string")) is None+ assert engine_info(_Described(None)) is None+++def test_describe_that_raises_is_not_fatal():+ class _Broken:+ def describe(self):+ raise RuntimeError("host query failed")++ assert engine_info(_Broken()) is None+++def test_unserializable_values_are_stringified_not_dropped():+ info = engine_info(_Described({"engine_id": "afm", "handle": object()}))+ assert info["engine_id"] == "afm"+ assert isinstance(info["handle"], str)+ # The whole point: the result must round-trip through JSON.+ json.dumps(info)+++def test_result_is_always_json_serializable():+ payload = {+ "engine_id": "afm",+ "nested": {"a": [1, 2, 3]},+ "bad": MagicMock(),+ 7: "non-string key is dropped",+ }+ info = engine_info(_Described(payload))+ json.dumps(info)+ assert 7 not in info+ assert info["nested"] == {"a": [1, 2, 3]}diff --git a/tests/cli/test_doctor_labels.py b/tests/cli/test_doctor_labels.pyindex 0b570c80..d510ee1e 100644--- a/tests/cli/test_doctor_labels.py+++ b/tests/cli/test_doctor_labels.py@@ -41,7 +41,7 @@ class TestDoctorOptionalLabels:def test_labels_show_install_hint_on_missing(self) -> None:"""When a package is missing, show install hint in status."""- blocker = _selective_import_blocker("zeus")+ blocker = _selective_import_blocker("zeus_apple_silicon")with mock.patch("builtins.__import__", side_effect=blocker):runner = CliRunner()result = runner.invoke(cli, ["doctor", "--json"])diff --git a/tests/engine/test_apple_fm_engine.py b/tests/engine/test_apple_fm_engine.pynew file mode 100644index 00000000..ecd78889--- /dev/null+++ b/tests/engine/test_apple_fm_engine.py@@ -0,0 +1,610 @@+"""Tests for the in-process Apple Foundation Models engine.++The real ``apple-fm-sdk`` builds Swift bindings and only runs on macOS 26++with Apple Intelligence, and there is no macOS CI runner, so these tests+inject a stub SDK into ``sys.modules`` before importing the engine. The stub+mirrors the shapes the engine actually depends on:++* ``stream_response`` yields **cumulative** snapshots, not deltas+* ``token_count`` is async and takes either a value or a transcript+* ``is_available`` is an *instance* method returning ``(bool, reason)``++The pure helpers in ``_apple_fm_support`` need no stub at all and are tested+directly.+"""++from __future__ import annotations++import asyncio+import importlib+import sys+import types+from typing import Any++import pytest++from openjarvis.core.types import Message, Role++# ---------------------------------------------------------------------------+# Pure helpers — no SDK needed+# ---------------------------------------------------------------------------+++class TestSnapshotAccumulator:+ def test_cumulative_snapshots_become_deltas(self):+ from openjarvis.engine._apple_fm_support import snapshot_deltas++ # The SDK re-sends everything generated so far on each iteration.+ snapshots = ["The", "The capital", "The capital is Paris."]+ assert snapshot_deltas(snapshots) == [+ "The",+ " capital",+ " is Paris.",+ ]++ def test_naive_concatenation_would_have_duplicated(self):+ from openjarvis.engine._apple_fm_support import SnapshotAccumulator++ acc = SnapshotAccumulator()+ for snap in ["a", "ab", "abc"]:+ acc.add(snap)+ # Appending each chunk (what every delta-based backend does) would+ # give "aababc"; the accumulator tracks the snapshot instead.+ assert acc.text == "abc"++ def test_repeated_snapshot_yields_nothing(self):+ from openjarvis.engine._apple_fm_support import snapshot_deltas++ assert snapshot_deltas(["ab", "ab", "abc"]) == ["ab", "c"]++ def test_revised_snapshot_is_reported_whole(self):+ """Guided generation can revise rather than extend."""+ from openjarvis.engine._apple_fm_support import snapshot_deltas++ assert snapshot_deltas(["hello wor", "goodbye"]) == ["hello wor", "goodbye"]+++class TestModelLabels:+ def test_known_labels_normalize(self):+ from openjarvis.engine._apple_fm_support import validate_model_label++ assert validate_model_label(" AFM-3-Core ") == "afm-3-core"++ def test_unknown_label_names_the_alternatives(self):+ from openjarvis.engine._apple_fm_support import validate_model_label++ with pytest.raises(ValueError, match="afm-3-core-advanced"):+ validate_model_label("afm-4")+++class TestOptionsAndSampling:+ def test_max_tokens_maps_to_sdk_name(self):+ from openjarvis.engine._apple_fm_support import build_options_kwargs++ kwargs = build_options_kwargs({"max_tokens": "128", "temperature": "0.5"})+ assert kwargs == {"maximum_response_tokens": 128, "temperature": 0.5}++ def test_unset_values_defer_to_the_sdk(self):+ from openjarvis.engine._apple_fm_support import build_options_kwargs++ assert build_options_kwargs({}) == {}++ def test_sampling_defaults_to_greedy_for_reproducibility(self):+ from openjarvis.engine._apple_fm_support import parse_sampling_spec++ assert parse_sampling_spec({}) == ("greedy", {})++ def test_random_sampling_carries_its_fields(self):+ from openjarvis.engine._apple_fm_support import parse_sampling_spec++ mode, kwargs = parse_sampling_spec(+ {"sampling": "random", "seed": "7", "top": 3}+ )+ assert mode == "random"+ assert kwargs == {"seed": 7, "top": 3}++ def test_unknown_sampling_mode_rejected(self):+ from openjarvis.engine._apple_fm_support import parse_sampling_spec++ with pytest.raises(ValueError, match="greedy"):+ parse_sampling_spec({"sampling": "beam"})+++# ---------------------------------------------------------------------------+# Stub SDK+# ---------------------------------------------------------------------------+++class _Recorder(dict):+ pass+++def _install_stub_sdk(+ *,+ available: tuple[bool, Any] = (True, None),+ snapshots: list[str] | None = None,+ token_counts: list[int] | None = None,+ raise_on_stream: str | None = None,+) -> _Recorder:+ """Inject a fake ``apple_fm_sdk`` and return a call recorder."""+ rec = _Recorder(+ options=[],+ sessions=[],+ stream_calls=[],+ token_count_calls=[],+ models=[],+ )+ sdk = types.ModuleType("apple_fm_sdk")++ class SystemLanguageModelUseCase:+ GENERAL = "general"+ CONTENT_TAGGING = "content_tagging"++ class SystemLanguageModelGuardrails:+ DEFAULT = "default"+ PERMISSIVE_CONTENT_TRANSFORMATIONS = "permissive"++ class SamplingMode:+ @staticmethod+ def greedy(**kw):+ return ("greedy", kw)++ @staticmethod+ def random(**kw):+ return ("random", kw)++ class GenerationOptions:+ def __init__(+ self, temperature=None, maximum_response_tokens=None, sampling=None+ ):+ self.temperature = temperature+ self.maximum_response_tokens = maximum_response_tokens+ self.sampling = sampling+ rec["options"].append(self)++ class ExceededContextWindowSizeError(Exception):+ pass++ class GuardrailViolationError(Exception):+ pass++ class RefusalError(Exception):+ pass++ class UnsupportedLanguageOrLocaleError(Exception):+ pass++ class SystemLanguageModel:+ context_size = 4096++ def __init__(self, use_case=None, guardrails=None):+ self.use_case = use_case+ self.guardrails = guardrails+ rec["models"].append(self)++ def is_available(self):+ return available++ async def token_count(self, value=None, *, instructions=None):+ rec["token_count_calls"].append(+ {+ "value": value,+ "instructions": instructions,+ }+ )+ counts = token_counts or []+ idx = len(rec["token_count_calls"]) - 1+ return counts[idx] if idx < len(counts) else 0++ class _Transcript:+ pass++ class LanguageModelSession:+ def __init__(self, instructions=None, model=None, tools=None):+ self.instructions = instructions+ self.model = model+ self.transcript = _Transcript()+ rec["sessions"].append(self)++ async def stream_response(self, prompt, options=None):+ rec["stream_calls"].append({"prompt": prompt, "options": options})+ if raise_on_stream is not None:+ # Named rather than passed as an instance so callers don't+ # need a stub SDK in hand before installing one.+ raise getattr(sdk, raise_on_stream)("stub failure")+ for snap in snapshots or []:+ yield snap++ sdk.SystemLanguageModelUseCase = SystemLanguageModelUseCase+ sdk.SystemLanguageModelGuardrails = SystemLanguageModelGuardrails+ sdk.SamplingMode = SamplingMode+ sdk.GenerationOptions = GenerationOptions+ sdk.SystemLanguageModel = SystemLanguageModel+ sdk.LanguageModelSession = LanguageModelSession+ sdk.ExceededContextWindowSizeError = ExceededContextWindowSizeError+ sdk.GuardrailViolationError = GuardrailViolationError+ sdk.RefusalError = RefusalError+ sdk.UnsupportedLanguageOrLocaleError = UnsupportedLanguageOrLocaleError++ sys.modules["apple_fm_sdk"] = sdk+ return rec+++def _load_engine_module():+ """Import a fresh engine module against whatever stub is installed.++ ``tests/conftest.py`` wipes ``EngineRegistry`` between tests, so the+ module must be re-imported (not just fetched from the registry) for the+ ``@EngineRegistry.register`` decorator to run again.+ """+ sys.modules.pop("openjarvis.engine.apple_fm", None)+ return importlib.import_module("openjarvis.engine.apple_fm")+++def engine_mod():+ created = []++ def _make(**stub_kwargs):+ rec = _install_stub_sdk(**stub_kwargs)+ mod = _load_engine_module()+ created.append(mod)+ return mod, rec++ yield _make+ sys.modules.pop("openjarvis.engine.apple_fm", None)+ sys.modules.pop("apple_fm_sdk", None)+++def _engine(mod, **kwargs):+ eng = mod.AppleFMEngine(**kwargs)+ eng._prepared = True+ eng._context_size = 4096+ return eng+++# ---------------------------------------------------------------------------+# Engine behaviour+# ---------------------------------------------------------------------------+++class TestRegistration:+ def test_registers_under_afm(self, engine_mod):+ from openjarvis.core.registry import EngineRegistry++ mod, _ = engine_mod()+ assert EngineRegistry.get("afm") is mod.AppleFMEngine+ assert mod.AppleFMEngine.engine_id == "afm"++ def test_rejects_unknown_use_case(self, engine_mod):+ mod, _ = engine_mod()+ with pytest.raises(ValueError, match="content_tagging"):+ mod.AppleFMEngine(use_case="nonsense")++ def test_rejects_unknown_sampling_at_construction(self, engine_mod):+ """A typo must fail now, not midway through a long run."""+ mod, _ = engine_mod()+ with pytest.raises(ValueError, match="greedy"):+ mod.AppleFMEngine(sampling="beam")+++class TestServeability:+ def test_only_serves_afm_labels(self, engine_mod):+ """Unlike other local engines, this one must not accept any model id.++ There is exactly one on-device model, so accepting arbitrary ids would+ let engine selection route a Llama request here and answer with a+ different model entirely.+ """+ mod, _ = engine_mod()+ eng = _engine(mod)+ assert eng.can_serve("afm-3-core") is True+ assert eng.can_serve("llama3.2:3b") is False+++class TestHealth:+ def test_reports_unavailable_reason(self, engine_mod, caplog):+ mod, _ = engine_mod(available=(False, types.SimpleNamespace(name="ASSETS")))+ eng = mod.AppleFMEngine()+ with caplog.at_level("WARNING"):+ assert eng.health() is False+ assert "ASSETS" in caplog.text+++class TestGenerate:+ def test_returns_content_and_real_token_counts(self, engine_mod):+ # token_count call order: transcript-before, prompt, transcript-after+ mod, rec = engine_mod(+ snapshots=["Paris", "Paris is", "Paris is the capital."],+ token_counts=[12, 8, 27],+ )+ eng = _engine(mod)++ result = eng.generate(+ [Message(role=Role.USER, content="Capital of France?")],+ model="afm-3-core",+ max_tokens=64,+ )+ eng.close()++ assert result["content"] == "Paris is the capital."+ # prompt = transcript-before (12) + prompt alone (8)+ assert result["usage"]["prompt_tokens"] == 20+ # completion = transcript-after (27) - prompt (20)+ assert result["usage"]["completion_tokens"] == 7+ assert result["usage"]["total_tokens"] == 27+ assert result["finish_reason"] == "stop"++ def test_system_message_becomes_session_instructions(self, engine_mod):+ """Not folded into the prompt text.++ The shim historically prefixed system content onto the prompt, which+ both loses the distinction the SDK draws and inflates prompt tokens.+ """+ mod, rec = engine_mod(snapshots=["ok"], token_counts=[1, 1, 2])+ eng = _engine(mod)+ eng.generate(+ [+ Message(role=Role.SYSTEM, content="Be terse."),+ Message(role=Role.USER, content="Hi"),+ ],+ model="afm-3",+ )+ eng.close()++ assert rec["sessions"][-1].instructions == "Be terse."+ assert rec["stream_calls"][-1]["prompt"] == "Hi"++ def test_configured_instructions_precede_request_system_message(self, engine_mod):+ mod, rec = engine_mod(snapshots=["ok"], token_counts=[1, 1, 2])+ eng = _engine(mod, instructions="Always answer in English.")+ eng.generate(+ [+ Message(role=Role.SYSTEM, content="Be terse."),+ Message(role=Role.USER, content="Hi"),+ ],+ model="afm-3",+ )+ eng.close()+ assert (+ rec["sessions"][-1].instructions == "Always answer in English.\nBe terse."+ )++ def test_generation_options_carry_sampling_and_limits(self, engine_mod):+ mod, rec = engine_mod(snapshots=["x"], token_counts=[0, 0, 1])+ eng = _engine(mod, sampling="random")+ eng.generate(+ [Message(role=Role.USER, content="Hi")],+ model="afm-3",+ temperature=0.25,+ max_tokens=99,+ )+ eng.close()++ opts = rec["stream_calls"][-1]["options"]+ assert opts.temperature == 0.25+ assert opts.maximum_response_tokens == 99+ assert opts.sampling == ("random", {})++ def test_each_request_gets_a_fresh_session(self, engine_mod):+ """Reused sessions accumulate a transcript that overflows the 4096-token+ context after a handful of requests, and would make per-request energy+ depend on the requests before it."""+ mod, rec = engine_mod(snapshots=["x"], token_counts=[0, 0, 1] * 3)+ eng = _engine(mod)+ msgs = [Message(role=Role.USER, content="Hi")]+ for _ in range(3):+ eng.generate(msgs, model="afm-3")+ eng.close()++ assert len(rec["sessions"]) == 3+ assert len({id(s) for s in rec["sessions"]}) == 3++ def test_token_count_failure_degrades_to_zero_not_an_error(self, engine_mod):+ mod, _ = engine_mod(snapshots=["hi"])+ eng = _engine(mod)++ async def _boom(*a, **k):+ raise RuntimeError("counter unavailable")++ eng._ensure_model().token_count = _boom++ result = eng.generate([Message(role=Role.USER, content="Hi")], model="afm-3")+ eng.close()+ assert result["content"] == "hi"+ assert result["usage"]["completion_tokens"] == 0+++class TestErrorMapping:+ def test_context_overflow_becomes_a_context_length_error(self, engine_mod):+ from openjarvis.engine._base import (+ EngineContextLengthError,+ looks_like_context_length_error,+ )++ mod, _ = engine_mod(raise_on_stream="ExceededContextWindowSizeError")+ eng = _engine(mod)+ with pytest.raises(EngineContextLengthError) as exc:+ eng.generate([Message(role=Role.USER, content="x" * 100)], model="afm-3")+ eng.close()+ # The shared heuristic must recognise the phrasing too, so callers+ # branching on text rather than type still get "conversation too long".+ assert looks_like_context_length_error(str(exc.value))+ # The SDK formats its messages as "<reason>: <detail>" and passes+ # None as the detail, so the raw text ends in ": None". That must not+ # reach the user -- it reads as a broken error path rather than a+ # plain overflow, and it lands in every affected eval row.+ assert not str(exc.value).rstrip().endswith("None")+ assert "4096" in str(exc.value)++ def test_context_error_does_not_repeat_the_sdk_reason(self, engine_mod):+ from openjarvis.engine._base import EngineContextLengthError++ mod, _ = engine_mod(raise_on_stream="ExceededContextWindowSizeError")+ eng = _engine(mod)+ with pytest.raises(EngineContextLengthError) as exc:+ eng.generate([Message(role=Role.USER, content="x")], model="afm-3")+ eng.close()+ # "context window" should appear once, from our own sentence.+ assert str(exc.value).lower().count("context window") == 1++ def test_refusal_returns_empty_instead_of_killing_the_run(self, engine_mod):+ """A declined prompt is a per-query outcome, not an engine fault.++ A long eval run must not die because one prompt tripped a guardrail.+ """+ mod, _ = engine_mod(+ raise_on_stream="GuardrailViolationError",+ token_counts=[3, 5],+ )+ eng = _engine(mod)+ result = eng.generate([Message(role=Role.USER, content="...")], model="afm-3")++ assert result["content"] == ""+ assert result["finish_reason"] == "content_filter"+ assert eng._skipped == {"GuardrailViolationError": 1}+ eng.close()+++class TestStreaming:+ def test_stream_yields_deltas_that_reassemble(self, engine_mod):+ mod, _ = engine_mod(+ snapshots=["The", "The cap", "The capital is Paris."],+ token_counts=[4, 6, 20],+ )+ eng = _engine(mod)++ async def _collect():+ return [+ c+ async for c in eng.stream(+ [Message(role=Role.USER, content="Hi")], model="afm-3"+ )+ ]++ chunks = asyncio.run(_collect())+ eng.close()+ assert "".join(chunks) == "The capital is Paris."++ def test_stream_full_reports_usage_at_end(self, engine_mod):+ mod, _ = engine_mod(snapshots=["ab", "abc"], token_counts=[4, 6, 15])+ eng = _engine(mod)++ async def _collect():+ return [+ c+ async for c in eng.stream_full(+ [Message(role=Role.USER, content="Hi")], model="afm-3"+ )+ ]++ chunks = asyncio.run(_collect())+ eng.close()+ final = chunks[-1]+ assert final.finish_reason == "stop"+ assert final.usage["prompt_tokens"] == 10+ assert final.usage["completion_tokens"] == 5+++class TestPrepare:+ def test_warms_up_and_records_context_size(self, engine_mod):+ """Warmup runs inside prepare() so the ~10s cold start lands outside+ the first request's energy window."""+ mod, rec = engine_mod(snapshots=["w"])+ eng = mod.AppleFMEngine()+ eng.prepare("afm-3-core")+ eng.close()++ assert eng._context_size == 4096+ assert rec["stream_calls"][0]["prompt"] == "warmup"+ assert rec["stream_calls"][0]["options"].maximum_response_tokens == 4++ def test_unavailable_model_raises_connection_error(self, engine_mod):+ from openjarvis.engine._base import EngineConnectionError++ mod, _ = engine_mod(+ available=(False, types.SimpleNamespace(name="APPLE_INTELLIGENCE_OFF"))+ )+ eng = mod.AppleFMEngine()+ with pytest.raises(EngineConnectionError, match="APPLE_INTELLIGENCE_OFF"):+ eng.prepare("afm-3-core")++ def test_bad_label_rejected_before_touching_the_model(self, engine_mod):+ mod, _ = engine_mod()+ eng = mod.AppleFMEngine()+ with pytest.raises(ValueError, match="afm-3-core-advanced"):+ eng.prepare("gpt-4")+++class TestDescribe:+ def test_records_what_the_variant_cannot_tell_us(self, engine_mod):+ mod, _ = engine_mod()+ eng = _engine(mod, use_case="content_tagging")+ info = eng.describe()+ eng.close()++ assert info["engine_id"] == "afm"+ assert info["afm_use_case"] == "content_tagging"+ assert info["afm_context_size"] == 4096+ # The executing variant is not observable, so the host chip is how a+ # run gets attributed after the fact.+ assert info["host_chip"]+ assert "dynamic profile" in info["variant_selection"]+++# ---------------------------------------------------------------------------+# On-hardware smoke test+# ---------------------------------------------------------------------------+++class TestOnRealHardware:+ def test_generates_with_real_token_counts(self):+ sys.modules.pop("apple_fm_sdk", None)+ sys.modules.pop("openjarvis.engine.apple_fm", None)+ pytest.importorskip("apple_fm_sdk")+ mod = importlib.import_module("openjarvis.engine.apple_fm")++ eng = mod.AppleFMEngine(instructions="Answer in one word.")+ if not eng.health():+ pytest.skip("Apple Intelligence unavailable on this host")++ result = eng.generate(+ [Message(role=Role.USER, content="Name one planet.")],+ model="afm-3-core",+ max_tokens=16,+ )+ eng.close()++ assert result["content"].strip()+ # The whole point of the 0.2.1 SDK floor: the shim reported 0 here,+ # which zeroed throughput and every per-token energy figure.+ assert result["usage"]["completion_tokens"] > 0+ assert result["usage"]["prompt_tokens"] > 0+++class TestAfmIsTreatedAsLocal:+ """`afm` must be recognised as an on-device engine everywhere it matters.++ The Python SDK has no Private Cloud Compute path, so AFM inference never+ leaves the machine. An engine key missing from these sets produces a false+ "your data is leaving this machine" signal.+ """++ def test_data_boundary_audit_classifies_afm_as_local(self):+ from openjarvis.security.data_boundary_audit import (+ LOCAL_ENGINE_KEYS,+ _target_is_cloud,+ )++ assert "afm" in LOCAL_ENGINE_KEYS+ assert _target_is_cloud("afm", "afm-3-core") is False++ def test_image_privacy_guard_does_not_warn_for_afm(self):+ """`jarvis ask -i photo.png --engine afm` must not claim the image+ leaves the machine."""+ from openjarvis.cli.ask import LOCAL_ENGINES++ assert "afm" in LOCAL_ENGINES+ assert "openai" not in LOCAL_ENGINESdiff --git a/tests/engine/test_apple_fm_shim.py b/tests/engine/test_apple_fm_shim.pyindex 8d8447cc..e126a7c3 100644--- a/tests/engine/test_apple_fm_shim.py+++ b/tests/engine/test_apple_fm_shim.py@@ -25,6 +25,8 @@ def _install_stub_sdk(available: tuple[bool, Any] = (True, None),stream_tokens: list[str] | None = None,respond_text: str = "Hello from Apple FM.",+ token_counts: list[int] | None = None,+ ctx_size: int = 4096,) -> dict[str, Any]:"""Inject a fake ``apple_fm_sdk`` module and return a recorder dict.@@ -32,8 +34,19 @@ def _install_stub_sdk(args passed to ``stream_response`` / ``respond`` so tests can assertthe shim wires ``options=`` through (PR #377) rather than the old``max_tokens=`` positional kwarg.++ The stub mirrors the SDK 0.2.1 surface: ``LanguageModelSession`` takes+ ``instructions`` / ``model``, and ``SystemLanguageModel`` exposes the+ async ``token_count`` plus ``context_size`` that the shim now uses for+ real usage numbers."""- rec: dict[str, Any] = {"options": [], "stream_calls": [], "respond_calls": []}+ rec: dict[str, Any] = {+ "options": [],+ "stream_calls": [],+ "respond_calls": [],+ "sessions": [],+ "token_count_calls": [],+ }sdk = types.ModuleType("apple_fm_sdk")class GenerationOptions:@@ -42,11 +55,49 @@ def _install_stub_sdk(self.maximum_response_tokens = maximum_response_tokensrec["options"].append(self)+ class SystemLanguageModelUseCase:+ GENERAL = "general"+ CONTENT_TAGGING = "content_tagging"++ class SystemLanguageModelGuardrails:+ DEFAULT = "default"+ PERMISSIVE_CONTENT_TRANSFORMATIONS = "permissive"++ class SamplingMode:+ @staticmethod+ def greedy(**kw):+ return ("greedy", kw)++ @staticmethod+ def random(**kw):+ return ("random", kw)+class SystemLanguageModel:+ def __init__(self, use_case: Any = None, guardrails: Any = None):+ self.use_case = use_case+ self.guardrails = guardrails++ context_size = ctx_size+def is_available(self): # instance method returning (bool, reason)return available+ async def token_count(self, value: Any = None, *, instructions: Any = None):+ rec["token_count_calls"].append(value)+ counts = token_counts or []+ idx = len(rec["token_count_calls"]) - 1+ return counts[idx] if idx < len(counts) else 0++ class _Transcript:+ pass+class LanguageModelSession:+ def __init__(self, instructions: Any = None, model: Any = None, tools=None):+ self.instructions = instructions+ self.model = model+ self.transcript = _Transcript()+ rec["sessions"].append(self)+async def stream_response(self, prompt: str, *, options: Any = None):rec["stream_calls"].append({"prompt": prompt, "options": options})for tok in stream_tokens or []:@@ -59,6 +110,11 @@ def _install_stub_sdk(sdk.GenerationOptions = GenerationOptions # type: ignore[attr-defined]sdk.SystemLanguageModel = SystemLanguageModel # type: ignore[attr-defined]sdk.LanguageModelSession = LanguageModelSession # type: ignore[attr-defined]+ sdk.SystemLanguageModelUseCase = SystemLanguageModelUseCase # type: ignore[attr-defined]+ sdk.SystemLanguageModelGuardrails = ( # type: ignore[attr-defined]+ SystemLanguageModelGuardrails+ )+ sdk.SamplingMode = SamplingMode # type: ignore[attr-defined]sys.modules["apple_fm_sdk"] = sdkreturn rec@@ -69,7 +125,11 @@ def shim(monkeypatch):"""Import a fresh copy of the shim against the stub SDK on any platform."""# The module bails with sys.exit unless it sees Darwin + the SDK import.monkeypatch.setattr(platform, "system", lambda: "Darwin")- rec = _install_stub_sdk(stream_tokens=["Sure! ", "Sure! The ", "Sure! The answer."])+ # token_count order: transcript-before, prompt, transcript-after.+ rec = _install_stub_sdk(+ stream_tokens=["Sure! ", "Sure! The ", "Sure! The answer."],+ token_counts=[6, 4, 17],+ )sys.modules.pop("openjarvis.engine.apple_fm_shim", None)mod = importlib.import_module("openjarvis.engine.apple_fm_shim")mod = importlib.reload(mod)@@ -171,3 +231,97 @@ class TestAppleFmShimSdkMigration:sys.modules.pop("openjarvis.engine.apple_fm_shim", None)sys.modules.pop("apple_fm_sdk", None)_ = rec+++class TestAppleFmShimTokenCounts:+ """SDK 0.2.1 added ``token_count``; the shim used to hardcode zeros.++ Zeroed ``completion_tokens`` zeroes throughput, per-token energy and+ tokens-per-joule for every request served through this shim, so these+ numbers are the point of the upgrade rather than a nicety.+ """++ def test_non_streaming_reports_real_usage(self, shim):+ mod, rec = shim+ client = TestClient(mod.app)+ resp = client.post(+ "/v1/chat/completions",+ json={"messages": [{"role": "user", "content": "hi"}], "stream": False},+ )+ usage = resp.json()["usage"]+ # token_count order: transcript-before (6), prompt alone (4),+ # transcript-after (17). prompt = 6 + 4; completion = 17 - 10.+ assert usage["prompt_tokens"] == 10+ assert usage["completion_tokens"] == 7+ assert usage["total_tokens"] == 17++ def test_streaming_reports_usage_on_the_final_chunk(self, shim):+ import json as _json++ mod, _ = shim+ client = TestClient(mod.app)+ with client.stream(+ "POST",+ "/v1/chat/completions",+ json={"messages": [{"role": "user", "content": "hi"}], "stream": True},+ ) as resp:+ body = "".join(resp.iter_text())++ payloads = [+ _json.loads(line[len("data:") :].strip())+ for line in body.splitlines()+ if line.startswith("data:") and "[DONE]" not in line+ ]+ final = payloads[-1]+ assert final["choices"][0]["finish_reason"] == "stop"+ assert final["usage"]["completion_tokens"] == 7++ def test_token_count_failure_does_not_fail_the_request(self, shim):+ mod, _ = shim++ async def _boom(*a, **k):+ raise RuntimeError("counter unavailable")++ import apple_fm_sdk as stub++ stub.SystemLanguageModel.token_count = _boom+ client = TestClient(mod.app)+ resp = client.post(+ "/v1/chat/completions",+ json={"messages": [{"role": "user", "content": "hi"}], "stream": False},+ )+ assert resp.status_code == 200+ assert resp.json()["usage"]["completion_tokens"] == 0+++class TestAppleFmShimInstructions:+ def test_system_message_becomes_session_instructions(self, shim):+ """Not prefixed onto the prompt as "[System] ...".++ The SDK draws a distinction between instructions and prompt; folding+ them together discards it and inflates the prompt-token count.+ """+ mod, rec = shim+ client = TestClient(mod.app)+ client.post(+ "/v1/chat/completions",+ json={+ "messages": [+ {"role": "system", "content": "Be terse."},+ {"role": "user", "content": "hi"},+ ],+ "stream": False,+ },+ )+ assert rec["sessions"][-1].instructions == "Be terse."+ assert rec["respond_calls"][-1]["prompt"] == "hi"+++class TestAppleFmShimModels:+ def test_models_endpoint_reports_context_length(self, shim):+ mod, _ = shim+ client = TestClient(mod.app)+ data = client.get("/v1/models").json()+ entry = data["data"][0]+ assert entry["id"] == "apple-fm"+ assert entry["context_length"] == 4096diff --git a/tests/evals/test_energy_rails_in_results.py b/tests/evals/test_energy_rails_in_results.pynew file mode 100644index 00000000..76d268d2--- /dev/null+++ b/tests/evals/test_energy_rails_in_results.py@@ -0,0 +1,84 @@+"""Eval output must record which hardware did the work, and how it was measured.++Before this, an eval result row carried only ``energy_joules`` and+``power_watts``. On Apple Silicon that hides the entire point of the+measurement: an ANE-resident model draws almost nothing on the GPU rail, so a+reader could not tell a Neural Engine run from an idle machine, nor a hardware+measurement from a modelled estimate. Both distinctions now have to survive+the trip from the monitor to the JSONL file.+"""++from __future__ import annotations++from openjarvis.evals.core.types import EvalResult+++def _apple_result() -> EvalResult:+ """A result shaped like an AFM run: ANE-dominant, GPU near zero."""+ return EvalResult(+ record_id="r1",+ model_answer="ok",+ energy_joules=43.104,+ power_watts=4.69,+ cpu_energy_joules=27.851,+ gpu_energy_joules=0.670,+ dram_energy_joules=8.902,+ ane_energy_joules=5.681,+ soc_energy_joules=43.104,+ energy_basis="soc",+ energy_method="ioreport",+ )+++class TestEvalResultCarriesRails:+ def test_rail_fields_exist_with_safe_defaults(self):+ r = EvalResult(record_id="r", model_answer="")+ for field in (+ "cpu_energy_joules",+ "gpu_energy_joules",+ "dram_energy_joules",+ "ane_energy_joules",+ "soc_energy_joules",+ ):+ assert getattr(r, field) == 0.0+ assert r.energy_basis == ""+ assert r.energy_method == ""++ def test_ane_is_distinguishable_from_gpu(self):+ """The claim the whole Apple path rests on.++ A GPU-only view of this run would report 0.67 J and look idle; the+ work actually happened on the Neural Engine.+ """+ r = _apple_result()+ assert r.ane_energy_joules > r.gpu_energy_joules * 8+ assert r.soc_energy_joules > r.gpu_energy_joules++ def test_method_separates_measurement_from_estimate(self):+ assert _apple_result().energy_method == "ioreport"+ estimated = EvalResult(+ record_id="r", model_answer="", energy_method="tdp_estimate"+ )+ assert estimated.energy_method != "ioreport"+++class TestSerializedRow:+ def test_written_row_includes_rails_and_method(self):+ """Guards the JSONL schema itself.++ The runner builds result rows by hand, so a field can exist on+ EvalResult and still never reach the file — which is exactly how this+ gap arose.+ """+ import inspect++ from openjarvis.evals.core import runner as runner_mod++ source = inspect.getsource(runner_mod)+ for key in (+ '"ane_energy_joules": result.ane_energy_joules',+ '"soc_energy_joules": result.soc_energy_joules',+ '"energy_basis": result.energy_basis',+ '"energy_method": result.energy_method',+ ):+ assert key in source, f"result row no longer writes {key}"diff --git a/tests/telemetry/test_energy_apple.py b/tests/telemetry/test_energy_apple.pyindex 3c6e5860..a3f00a3d 100644--- a/tests/telemetry/test_energy_apple.py+++ b/tests/telemetry/test_energy_apple.py@@ -1,176 +1,392 @@-"""Tests for AppleEnergyMonitor -- mock zeus (no real Apple Silicon required)."""+"""Tests for AppleEnergyMonitor.++The fake backend below mirrors the *real* ``zeus_apple_silicon`` API:+``AppleEnergyMonitor`` with ``begin_window`` / ``end_window`` /+``get_cumulative_energy``, returning an object whose rail counters are+millijoules and are individually absent on chips that do not expose them.++This matters. The previous version of this file invented a+``zeus.device.soc.apple.AppleSiliconMonitor`` class with joule-valued+``cpu_energy`` / ``ane_energy`` attributes. No such class or field has ever+existed in zeus, so the suite validated its own mock while the production+import failed on every machine and the monitor silently fell back to a+modelled estimate. Keep this stub faithful to the real library.+"""from __future__ import annotationsimport time-import types-from unittest.mock import MagicMock, patch+from unittest.mock import patchimport pytest-from tests.telemetry.energy_test_helpers import (- assert_sample_result_basics,-)+from tests.telemetry.energy_test_helpers import assert_sample_result_basics# ----------------------------------------------------------------------------# Helpers: build a fake zeus module+# Fake zeus_apple_silicon backend# ----------------------------------------------------------------------------def _make_fake_zeus():- """Return a fake zeus.device.soc.apple module with AppleSiliconMonitor."""- # Build the nested module hierarchy- zeus = types.ModuleType("zeus")- zeus_device = types.ModuleType("zeus.device")- zeus_device_soc = types.ModuleType("zeus.device.soc")- zeus_device_soc_apple = types.ModuleType("zeus.device.soc.apple")-- mock_monitor_cls = MagicMock()- zeus_device_soc_apple.AppleSiliconMonitor = mock_monitor_cls-- zeus.device = zeus_device- zeus_device.soc = zeus_device_soc- zeus_device_soc.apple = zeus_device_soc_apple-- return zeus, zeus_device, zeus_device_soc, zeus_device_soc_apple, mock_monitor_cls+class FakeMetrics:+ """Stands in for ``zeus_apple_silicon.zeus_ext.AppleEnergyMetrics``.++ All counters are millijoules. ``None`` means the chip does not expose that+ rail, which is distinct from a zero reading.+ """++ def __init__(+ self,+ cpu_total_mj=None,+ gpu_mj=None,+ gpu_sram_mj=None,+ dram_mj=None,+ ane_mj=None,+ ):+ self.cpu_total_mj = cpu_total_mj+ self.gpu_mj = gpu_mj+ self.gpu_sram_mj = gpu_sram_mj+ self.dram_mj = dram_mj+ self.ane_mj = ane_mj+ # The real object also exposes per-core / per-cluster breakdowns that+ # cpu_total_mj already subsumes. Present here so a regression that+ # starts summing them shows up as double counting.+ self.efficiency_cores_mj = [100, 100]+ self.performance_cores_mj = [200, 200]+ self.efficiency_cluster_mj = [200]+ self.performance_cluster_mj = [400]+ self.efficiency_core_manager_mj = 50+ self.performance_core_manager_mj = 50+++class FakeReader:+ """Stands in for ``zeus_apple_silicon.AppleEnergyMonitor``."""++ def __init__(self, window_metrics=None, cumulative=None):+ self._window_metrics = window_metrics or FakeMetrics()+ self._cumulative = cumulative or []+ self._cumulative_idx = 0+ self.begun: list[str] = []+ self.ended: list[str] = []++ def begin_window(self, key, restart=False):+ self.begun.append(key)++ def end_window(self, key):+ if key not in self.begun:+ raise RuntimeError("No measurement with provided key had been started.")+ self.ended.append(key)+ return self._window_metrics++ def get_cumulative_energy(self):+ if not self._cumulative:+ return FakeMetrics()+ idx = min(self._cumulative_idx, len(self._cumulative) - 1)+ self._cumulative_idx += 1+ return self._cumulative[idx]+++def _monitor_with(reader, *, allow_estimates=False, chip="Apple M1 Pro", tdp=30.0):+ """Build a monitor bypassing __init__'s hardware probe."""+ from openjarvis.telemetry.energy_apple import AppleEnergyMonitor++ m = AppleEnergyMonitor.__new__(AppleEnergyMonitor)+ m._poll_interval_ms = 50+ m._allow_estimates = allow_estimates+ m._reader = reader+ m._chip_name = chip+ m._tdp_watts = tdp+ m._prev = None+ return m# ----------------------------------------------------------------------------# Tests: available()+# available()# ---------------------------------------------------------------------------class TestAvailable:- def test_available_false_on_non_darwin(self):- with patch("platform.system", return_value="Linux"):- from openjarvis.telemetry.energy_apple import AppleEnergyMonitor+ def test_false_on_non_darwin(self):+ from openjarvis.telemetry.energy_apple import AppleEnergyMonitor+ with patch("platform.system", return_value="Linux"):assert AppleEnergyMonitor.available() is False+ assert AppleEnergyMonitor.estimate_available() is False++ def test_false_without_native_backend(self):+ """No IOReport backend means nothing measurable — not "available anyway".- def test_available_true_without_zeus(self):- """Monitor is available on Apple Silicon even without Zeus."""+ The old behaviour returned True on any Apple Silicon host, which is+ how a modelled estimate came to masquerade as a measurement.+ """import openjarvis.telemetry.energy_apple as mod- orig = mod._ZEUS_APPLE_AVAILABLE- mod._ZEUS_APPLE_AVAILABLE = False+ orig = mod._NATIVE_AVAILABLE+ mod._NATIVE_AVAILABLE = Falsetry:with (patch("platform.system", return_value="Darwin"),patch("platform.machine", return_value="arm64"),):- assert mod.AppleEnergyMonitor.available() is True- monitor = mod.AppleEnergyMonitor.__new__(mod.AppleEnergyMonitor)- monitor._zeus_ok = False- assert monitor.energy_method() == "cpu_time_estimate"+ assert mod.AppleEnergyMonitor.available() is False+ # ...but the host is still Apple Silicon, so an explicitly+ # requested estimate remains possible.+ assert mod.AppleEnergyMonitor.estimate_available() is Truefinally:- mod._ZEUS_APPLE_AVAILABLE = orig+ mod._NATIVE_AVAILABLE = orig++ def test_false_when_backend_raises(self):+ """A backend that imports but cannot construct is not available."""+ import openjarvis.telemetry.energy_apple as mod++ def _boom():+ raise RuntimeError("IOReport unavailable")++ with (+ patch("platform.system", return_value="Darwin"),+ patch("platform.machine", return_value="arm64"),+ patch.object(mod, "_NATIVE_AVAILABLE", True),+ patch.object(mod, "_AppleRailReader", _boom),+ ):+ assert mod.AppleEnergyMonitor.available() is False# ----------------------------------------------------------------------------# Tests: energy_method()+# energy_method()# ---------------------------------------------------------------------------class TestEnergyMethod:- def test_returns_zeus(self):- from openjarvis.telemetry.energy_apple import AppleEnergyMonitor+ def test_ioreport_when_reader_present(self):+ assert _monitor_with(FakeReader()).energy_method() == "ioreport"- monitor = AppleEnergyMonitor.__new__(AppleEnergyMonitor)- monitor._zeus_ok = True- assert monitor.energy_method() == "zeus"+ def test_tdp_estimate_without_reader(self):+ assert _monitor_with(None).energy_method() == "tdp_estimate"# ----------------------------------------------------------------------------# Tests: sample() component breakdown+# sample() — rail extraction# ----------------------------------------------------------------------------class TestSampleComponentBreakdown:- def test_component_energy_extraction(self):- """Mock begin_window/end_window and verify cpu/gpu/dram/ane extraction."""- mock_measurement = MagicMock()- mock_measurement.cpu_energy = 1.5- mock_measurement.gpu_energy = 3.0- mock_measurement.dram_energy = 0.5- mock_measurement.ane_energy = 2.0-- mock_zeus_monitor = MagicMock()- mock_zeus_monitor.begin_window = MagicMock()- mock_zeus_monitor.end_window = MagicMock(return_value=mock_measurement)+class TestSampleRails:+ def test_millijoules_are_converted_to_joules(self):+ reader = FakeReader(+ FakeMetrics(+ cpu_total_mj=10_000,+ gpu_mj=2_000,+ gpu_sram_mj=500,+ dram_mj=1_000,+ ane_mj=4_000,+ )+ )+ monitor = _monitor_with(reader)- from openjarvis.telemetry.energy_apple import AppleEnergyMonitor+ with monitor.sample() as result:+ time.sleep(0.01)- monitor = AppleEnergyMonitor.__new__(AppleEnergyMonitor)- monitor._poll_interval_ms = 50- monitor._monitor = mock_zeus_monitor- monitor._zeus_ok = True- monitor._chip_name = "M1"+ assert reader.begun and reader.ended+ assert result.cpu_energy_joules == pytest.approx(10.0)+ # GPU is gpu_mj + gpu_sram_mj — two separate rails.+ assert result.gpu_energy_joules == pytest.approx(2.5)+ assert result.dram_energy_joules == pytest.approx(1.0)+ assert result.ane_energy_joules == pytest.approx(4.0)+ assert_sample_result_basics(result, vendor="apple", energy_method="ioreport")++ def test_soc_total_and_basis(self):+ reader = FakeReader(+ FakeMetrics(cpu_total_mj=1_000, gpu_mj=2_000, dram_mj=300, ane_mj=700)+ )+ monitor = _monitor_with(reader)with monitor.sample() as result:- time.sleep(0.01)+ pass- mock_zeus_monitor.begin_window.assert_called_once()- mock_zeus_monitor.end_window.assert_called_once()+ expected = 1.0 + 2.0 + 0.3 + 0.7+ assert result.soc_energy_joules == pytest.approx(expected)+ assert result.energy_joules == pytest.approx(expected)+ assert result.basis == "soc"- assert result.cpu_energy_joules == pytest.approx(1.5)- assert result.gpu_energy_joules == pytest.approx(3.0)- assert result.dram_energy_joules == pytest.approx(0.5)- assert result.ane_energy_joules == pytest.approx(2.0)- assert_sample_result_basics(result, vendor="apple", energy_method="zeus")+ def test_cpu_total_is_not_double_counted(self):+ """cpu_total_mj already subsumes the cluster/core breakdowns.- def test_total_energy_is_sum_of_components(self):- """total = cpu + gpu + dram + ane."""- mock_measurement = MagicMock()- mock_measurement.cpu_energy = 1.0- mock_measurement.gpu_energy = 2.0- mock_measurement.dram_energy = 0.3- mock_measurement.ane_energy = 0.7+ Verified on an M1 Pro: efficiency_cluster + performance_cluster sums+ to cpu_total_mj. Summing both would roughly double CPU energy.+ """+ reader = FakeReader(FakeMetrics(cpu_total_mj=600))+ monitor = _monitor_with(reader)- mock_zeus_monitor = MagicMock()- mock_zeus_monitor.begin_window = MagicMock()- mock_zeus_monitor.end_window = MagicMock(return_value=mock_measurement)+ with monitor.sample() as result:+ pass- from openjarvis.telemetry.energy_apple import AppleEnergyMonitor+ assert result.cpu_energy_joules == pytest.approx(0.6)+ assert result.soc_energy_joules == pytest.approx(0.6)- monitor = AppleEnergyMonitor.__new__(AppleEnergyMonitor)- monitor._poll_interval_ms = 50- monitor._monitor = mock_zeus_monitor- monitor._zeus_ok = True- monitor._chip_name = "M1"+ def test_absent_rail_is_not_counted_as_zero(self):+ """A chip without an ANE counter reports no ANE, and the total omits it."""+ reader = FakeReader(FakeMetrics(cpu_total_mj=2_000, gpu_mj=1_000, ane_mj=None))+ monitor = _monitor_with(reader)with monitor.sample() as result:pass- expected_total = 1.0 + 2.0 + 0.3 + 0.7- assert result.energy_joules == pytest.approx(expected_total)+ assert result.ane_energy_joules == 0.0+ assert result.soc_energy_joules == pytest.approx(3.0)++ def test_power_is_derived_per_rail(self):+ reader = FakeReader(FakeMetrics(cpu_total_mj=1_000, gpu_mj=1_000, ane_mj=2_000))+ monitor = _monitor_with(reader)++ with monitor.sample() as result:+ time.sleep(0.05)++ assert result.duration_seconds > 0+ assert result.mean_power_watts == pytest.approx(+ result.soc_energy_joules / result.duration_seconds+ )+ assert result.ane_power_watts == pytest.approx(+ result.ane_energy_joules / result.duration_seconds+ )++ def test_body_error_survives_a_failing_end_window(self):+ """The caller's exception must not be swallowed by cleanup.++ `return` inside a `finally` discards an in-flight exception, so a+ failed end_window would have hidden whatever the body raised.+ """+ reader = FakeReader(FakeMetrics(cpu_total_mj=1_000))++ def _explode(key):+ raise RuntimeError("IOReport went away")++ reader.end_window = _explode+ monitor = _monitor_with(reader)++ with pytest.raises(ValueError, match="original"):+ with monitor.sample():+ raise ValueError("original failure")++ def test_window_closes_even_when_body_raises(self):+ reader = FakeReader(FakeMetrics(cpu_total_mj=1_000))+ monitor = _monitor_with(reader)++ with pytest.raises(ValueError):+ with monitor.sample():+ raise ValueError("boom")++ assert reader.ended == reader.begun# ----------------------------------------------------------------------------# Tests: sample() with uninitialized monitor+# sample() — no backend# ----------------------------------------------------------------------------class TestSampleUninitialized:- def test_uninitialized_monitor_empty_result(self):- """When monitor is not initialized, sample yields empty result."""- from openjarvis.telemetry.energy_apple import AppleEnergyMonitor+class TestSampleWithoutBackend:+ def test_reports_zero_rather_than_fabricating(self):+ """Without a backend and without opt-in, no numbers are invented."""+ monitor = _monitor_with(None)++ with monitor.sample() as result:+ time.sleep(0.01)++ assert result.energy_joules == 0.0+ assert result.soc_energy_joules == 0.0+ assert result.ane_energy_joules == 0.0+ assert result.vendor == "apple"+ assert result.energy_method == "tdp_estimate"- monitor = AppleEnergyMonitor.__new__(AppleEnergyMonitor)- monitor._poll_interval_ms = 50- monitor._monitor = None- monitor._zeus_ok = False- monitor._chip_name = "Apple Silicon"- monitor._tdp_watts = 20.0+ def test_estimate_is_produced_only_when_opted_in(self):+ monitor = _monitor_with(None, allow_estimates=True, tdp=30.0)with monitor.sample() as result:- pass+ time.sleep(0.05)++ assert result.energy_joules > 0.0+ assert result.mean_power_watts == pytest.approx(30.0 * 0.60, rel=1e-6)+ assert result.energy_method == "tdp_estimate"+++# ---------------------------------------------------------------------------+# snapshot()+# ---------------------------------------------------------------------------+++class TestSnapshot:+ def test_reports_cumulative_energy_and_derived_power(self):+ """Energy fields are cumulative counters; power comes from the delta.- # CPU-time fallback produces small but non-zero estimates- assert result.energy_joules >= 0.0- assert result.cpu_energy_joules >= 0.0- assert result.gpu_energy_joules >= 0.0- assert result.dram_energy_joules >= 0.0- assert result.ane_energy_joules >= 0.0- assert_sample_result_basics(- result, vendor="apple", energy_method="cpu_time_estimate"+ The eval runners difference these across a window+ (``_compute_energy_delta`` is last-minus-first), so a snapshot must+ carry the running counter, not a per-interval delta.+ """+ reader = FakeReader(+ cumulative=[+ FakeMetrics(cpu_total_mj=1_000, gpu_mj=500, ane_mj=100),+ FakeMetrics(cpu_total_mj=3_000, gpu_mj=700, ane_mj=900),+ ])+ monitor = _monitor_with(reader)++ first = monitor.snapshot()+ assert first.cpu_energy_joules == pytest.approx(1.0)+ # No previous reading to difference against yet.+ assert first.soc_power_watts == 0.0++ time.sleep(0.05)+ second = monitor.snapshot()+ assert second.cpu_energy_joules == pytest.approx(3.0)+ assert second.soc_power_watts > 0.0+ assert second.ane_power_watts > 0.0+ assert second.basis == "soc"++ def test_counter_reset_reports_no_power_rather_than_negative(self):+ reader = FakeReader(+ cumulative=[+ FakeMetrics(cpu_total_mj=5_000),+ FakeMetrics(cpu_total_mj=10),+ ]+ )+ monitor = _monitor_with(reader)++ monitor.snapshot()+ second = monitor.snapshot()++ assert second.cpu_power_watts == 0.0++ def test_empty_without_backend(self):+ result = _monitor_with(None).snapshot()++ assert result.energy_joules == 0.0+ assert result.vendor == "apple"+++# ---------------------------------------------------------------------------+# On-hardware smoke test+# ---------------------------------------------------------------------------+++class TestOnRealHardware:+ def test_measures_more_energy_under_load_than_idle(self):+ from openjarvis.telemetry.energy_apple import AppleEnergyMonitor++ if not AppleEnergyMonitor.available():+ pytest.skip("zeus-apple-silicon not installed")++ monitor = AppleEnergyMonitor()++ with monitor.sample() as idle:+ time.sleep(0.5)++ with monitor.sample() as busy:+ t = time.time()+ x = 0+ while time.time() - t < 0.5:+ x = (x * 31 + 7) % 1000003++ assert busy.soc_energy_joules > 0+ # The whole point of measuring rather than modelling: load must move+ # the number. The old wall-clock × TDP estimate could not tell these+ # two windows apart.+ assert busy.mean_power_watts > idle.mean_power_watts+ monitor.close()diff --git a/tests/telemetry/test_energy_wiring.py b/tests/telemetry/test_energy_wiring.pyindex 06ec1cb6..27ad5a37 100644--- a/tests/telemetry/test_energy_wiring.py+++ b/tests/telemetry/test_energy_wiring.py@@ -19,6 +19,7 @@ from openjarvis.core.config import JarvisConfigfrom openjarvis.core.events import EventBus, EventTypefrom openjarvis.core.types import Message, Role, TelemetryRecordfrom openjarvis.telemetry.aggregator import AggregatedStats, TelemetryAggregator+from openjarvis.telemetry.energy_monitor import BASIS_GPU, EnergySamplefrom openjarvis.telemetry.instrumented_engine import InstrumentedEnginefrom openjarvis.telemetry.store import TelemetryStore@@ -51,27 +52,40 @@ def _mock_engine(content="Test response"):def _mock_energy_monitor():- """Return a mock energy monitor with realistic sample data."""+ """Return a mock energy monitor with realistic sample data.++ The yielded sample is a real ``EnergySample``, not a ``MagicMock``. A mock+ returns a ``MagicMock`` for any attribute the test forgot to set, which+ sqlite then refuses to bind — so a newly added telemetry field would make+ every record silently fail to persist and surface here as an unrelated+ "0 records" assertion. A real dataclass picks up new fields with their+ real defaults.+ """monitor = MagicMock()monitor.close = MagicMock()- sample = MagicMock()- sample.energy_joules = 42.5- sample.mean_power_watts = 250.0- sample.peak_power_watts = 350.0- sample.mean_utilization_pct = 78.0- sample.peak_utilization_pct = 95.0- sample.mean_memory_used_gb = 16.0- sample.peak_memory_used_gb = 20.0- sample.mean_temperature_c = 65.0- sample.peak_temperature_c = 72.0- sample.duration_seconds = 0.5- sample.num_snapshots = 10- sample.energy_method = "hw_counter"- sample.vendor = "nvidia"- sample.cpu_energy_joules = 0.0- sample.gpu_energy_joules = 42.5- sample.dram_energy_joules = 0.0+ sample = EnergySample(+ energy_joules=42.5,+ mean_power_watts=250.0,+ peak_power_watts=350.0,+ mean_utilization_pct=78.0,+ peak_utilization_pct=95.0,+ mean_memory_used_gb=16.0,+ peak_memory_used_gb=20.0,+ mean_temperature_c=65.0,+ peak_temperature_c=72.0,+ duration_seconds=0.5,+ num_snapshots=10,+ energy_method="hw_counter",+ vendor="nvidia",+ cpu_energy_joules=0.0,+ gpu_energy_joules=42.5,+ dram_energy_joules=0.0,+ gpu_power_watts=250.0,+ soc_energy_joules=42.5,+ soc_power_watts=250.0,+ basis=BASIS_GPU,+ )@contextmanagerdef _sample():diff --git a/uv.lock b/uv.lockindex 5443be5e..3846aa7e 100644--- a/uv.lock+++ b/uv.lock@@ -44,7 +44,7 @@ name = "aiofile"version = "3.9.0"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "caio" },+ { name = "caio", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/67/e2/d7cb819de8df6b5c1968a2756c3cb4122d4fa2b8fc768b53b7c9e5edb646/aiofile-3.9.0.tar.gz", hash = "sha256:e5ad718bb148b265b6df1b3752c4d1d83024b93da9bd599df74b9d9ffcf7919b", size = 17943, upload-time = "2024-10-08T10:39:35.846Z" }wheels = [@@ -285,6 +285,17 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/b5/23/1b7dc5f0807f83098183a57db6ee85b2c93b646d74a6e03781c9208aaeb0/apache_tvm_ffi-0.1.9-cp312-abi3-win_amd64.whl", hash = "sha256:d1dcf4c041d5ec05e3da1d545800c33cdbb95c113baa7705085ff79fa262752b", size = 1973200, upload-time = "2026-02-27T19:27:32.367Z" },]+[[package]]+name = "apple-fm-sdk"+version = "0.2.1"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "build", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },+ { name = "setuptools", version = "80.10.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'" },+ { name = "setuptools", version = "82.0.0", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (python_full_version < '3.12' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },+]+sdist = { url = "https://files.pythonhosted.org/packages/91/02/5cfad13c6ab9acd22e81bc083789db02f3551069c10a1551ec60dd04a273/apple_fm_sdk-0.2.1.tar.gz", hash = "sha256:e1801d68ae0517f8524c31723b1dc1a662bf9d06b7a9cc765c9dddb77e647980", size = 107506, upload-time = "2026-06-29T23:30:06.383Z" }+[[package]]name = "astor"version = "0.8.1"@@ -369,7 +380,7 @@ name = "authlib"version = "1.6.8"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "cryptography" },+ { name = "cryptography", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/6b/6c/c88eac87468c607f88bc24df1f3b31445ee6fc9ba123b09e666adf687cd9/authlib-1.6.8.tar.gz", hash = "sha256:41ae180a17cf672bc784e4a518e5c82687f1fe1e98b0cafaeda80c8e4ab2d1cb", size = 165074, upload-time = "2026-02-14T04:02:17.941Z" }wheels = [@@ -617,6 +628,22 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/bd/dc/cadbf64b335a2ee0f31a84d05f34551c2199caa6f639a90c9157b564d0d6/blurhash-1.1.5-py2.py3-none-any.whl", hash = "sha256:96a8686e8b9fced1676550b814e59256214e2d4033202b16c91271ed4d317fec", size = 6632, upload-time = "2025-08-17T10:36:11.404Z" },]+[[package]]+name = "build"+version = "1.5.0"+source = { registry = "https://pypi.org/simple" }+dependencies = [+ { name = "colorama", marker = "(python_full_version < '3.11' and os_name == 'nt' and sys_platform == 'emscripten') or (python_full_version < '3.11' and os_name == 'nt' and sys_platform == 'win32') or (os_name == 'nt' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },+ { name = "importlib-metadata", marker = "python_full_version < '3.10.2' and sys_platform != 'linux'" },+ { name = "packaging", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },+ { name = "pyproject-hooks", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },+ { name = "tomli", marker = "python_full_version < '3.11' and sys_platform != 'linux'" },+]+sdist = { url = "https://files.pythonhosted.org/packages/78/e0/df5e171f685f82f37b12e1f208064e24244911079d7b767447d1af7e0d70/build-1.5.0.tar.gz", hash = "sha256:302c22c3ba2a0fd5f3911918651341ebb3896176cbdec15bd421f80b1afc7647", size = 89796, upload-time = "2026-04-30T03:18:25.17Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/0d/fe/6bea5c9162869c5beba5d9c8abbed835ec85bf1ec1fba05a3822325c45f3/build-1.5.0-py3-none-any.whl", hash = "sha256:13f3eecb844759ab66efec90ca17639bbf14dc06cb2fdf37a9010322d9c50a6f", size = 26018, upload-time = "2026-04-30T03:18:23.644Z" },+]+[[package]]name = "cachetools"version = "7.0.1"@@ -1157,7 +1184,7 @@ resolution-markers = ["python_full_version < '3.11' and sys_platform == 'linux'",]dependencies = [- { name = "cuda-pathfinder" },+ { name = "cuda-pathfinder", marker = "sys_platform == 'linux'" },]wheels = [{ url = "https://files.pythonhosted.org/packages/37/31/bfcc870f69c6a017c4ad5c42316207fc7551940db6f3639aa4466ec5faf3/cuda_bindings-12.9.4-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a022c96b8bd847e8dc0675523431149a4c3e872f440e3002213dbb9e08f0331a", size = 11800959, upload-time = "2025-10-21T14:51:26.458Z" },@@ -1189,7 +1216,7 @@ resolution-markers = ["python_full_version < '3.11' and sys_platform != 'linux'",]dependencies = [- { name = "cuda-pathfinder" },+ { name = "cuda-pathfinder", marker = "sys_platform != 'linux'" },]wheels = [{ url = "https://files.pythonhosted.org/packages/ec/ea/81999d01375645f34596c76eb046b4b36d58cc6fe2bddb2410f8a7b7a827/cuda_bindings-13.2.0-cp310-cp310-win_amd64.whl", hash = "sha256:845025438a1b9e20718b9fb42add3e0eb72e85458bcab3eeb80bfd8f0a9dab33", size = 5600047, upload-time = "2026-03-11T00:12:34.848Z" },@@ -1217,7 +1244,7 @@ resolution-markers = ["python_full_version < '3.11' and sys_platform == 'linux'",]dependencies = [- { name = "cuda-bindings", version = "12.9.4", source = { registry = "https://pypi.org/simple" } },+ { name = "cuda-bindings", version = "12.9.4", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform == 'linux'" },]wheels = [{ url = "https://files.pythonhosted.org/packages/af/f3/6b032a554019cfb3447e671798c1bd3e79b5f1af20d10253f56cea269ef2/cuda_python-12.9.4-py3-none-any.whl", hash = "sha256:d2cacea882a69863f1e7d27ee71d75f0684f4c76910aff839067e4f89c902279", size = 7594, upload-time = "2025-10-21T14:55:12.846Z" },@@ -1240,8 +1267,8 @@ resolution-markers = ["python_full_version < '3.11' and sys_platform != 'linux'",]dependencies = [- { name = "cuda-bindings", version = "13.2.0", source = { registry = "https://pypi.org/simple" } },- { name = "cuda-pathfinder" },+ { name = "cuda-bindings", version = "13.2.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'linux'" },+ { name = "cuda-pathfinder", marker = "sys_platform != 'linux'" },]wheels = [{ url = "https://files.pythonhosted.org/packages/4a/da/b4dbe129f941afe1c24a09ba53521b78875626763d96414798a74763282f/cuda_python-13.2.0-py3-none-any.whl", hash = "sha256:2f092b0ec13a860115fa595411889ee939ad203450ea4f91e9461b174ea7b084", size = 8145, upload-time = "2026-03-11T13:55:19.143Z" },@@ -1252,10 +1279,10 @@ name = "cyclopts"version = "4.6.0"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "attrs" },- { name = "docstring-parser" },- { name = "rich" },- { name = "rich-rst" },+ { name = "attrs", marker = "python_full_version >= '3.12'" },+ { name = "docstring-parser", marker = "python_full_version >= '3.12'" },+ { name = "rich", marker = "python_full_version >= '3.12'" },+ { name = "rich-rst", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/49/5c/88a4068c660a096bbe87efc5b7c190080c9e86919c36ec5f092cb08d852f/cyclopts-4.6.0.tar.gz", hash = "sha256:483c4704b953ea6da742e8de15972f405d2e748d19a848a4d61595e8e5360ee5", size = 162724, upload-time = "2026-02-23T15:44:49.286Z" }wheels = [@@ -1344,7 +1371,7 @@ name = "deprecation"version = "2.1.0"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "packaging" },+ { name = "packaging", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/5a/d3/8ae2869247df154b64c1884d7346d412fed0c49df84db635aab2d1c40e62/deprecation-2.1.0.tar.gz", hash = "sha256:72b3bde64e5d778694b0cf68178aed03d15e15477116add3fb773e581f9518ff", size = 173788, upload-time = "2020-04-20T14:23:38.738Z" }wheels = [@@ -1515,7 +1542,7 @@ name = "exceptiongroup"version = "1.3.1"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "typing-extensions", marker = "python_full_version < '3.13'" },+ { name = "typing-extensions", marker = "python_full_version < '3.11' or python_full_version == '3.12.*'" },]sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" }wheels = [@@ -1735,26 +1762,26 @@ name = "fastmcp"version = "3.0.2"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "authlib" },- { name = "cyclopts" },- { name = "exceptiongroup" },- { name = "httpx" },- { name = "jsonref" },- { name = "jsonschema-path" },- { name = "mcp" },- { name = "openapi-pydantic" },- { name = "opentelemetry-api" },- { name = "packaging" },- { name = "platformdirs" },- { name = "py-key-value-aio", extra = ["filetree", "keyring", "memory"] },- { name = "pydantic", extra = ["email"] },- { name = "pyperclip" },- { name = "python-dotenv" },- { name = "pyyaml" },- { name = "rich" },- { name = "uvicorn" },- { name = "watchfiles" },- { name = "websockets" },+ { name = "authlib", marker = "python_full_version >= '3.12'" },+ { name = "cyclopts", marker = "python_full_version >= '3.12'" },+ { name = "exceptiongroup", marker = "python_full_version >= '3.12'" },+ { name = "httpx", marker = "python_full_version >= '3.12'" },+ { name = "jsonref", marker = "python_full_version >= '3.12'" },+ { name = "jsonschema-path", marker = "python_full_version >= '3.12'" },+ { name = "mcp", marker = "python_full_version >= '3.12'" },+ { name = "openapi-pydantic", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-api", marker = "python_full_version >= '3.12'" },+ { name = "packaging", marker = "python_full_version >= '3.12'" },+ { name = "platformdirs", marker = "python_full_version >= '3.12'" },+ { name = "py-key-value-aio", extra = ["filetree", "keyring", "memory"], marker = "python_full_version >= '3.12'" },+ { name = "pydantic", extra = ["email"], marker = "python_full_version >= '3.12'" },+ { name = "pyperclip", marker = "python_full_version >= '3.12'" },+ { name = "python-dotenv", marker = "python_full_version >= '3.12'" },+ { name = "pyyaml", marker = "python_full_version >= '3.12'" },+ { name = "rich", marker = "python_full_version >= '3.12'" },+ { name = "uvicorn", marker = "python_full_version >= '3.12'" },+ { name = "watchfiles", marker = "python_full_version >= '3.12'" },+ { name = "websockets", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/11/6b/1a7ec89727797fb07ec0928e9070fa2f45e7b35718e1fe01633a34c35e45/fastmcp-3.0.2.tar.gz", hash = "sha256:6bd73b4a3bab773ee6932df5249dcbcd78ed18365ed0aeeb97bb42702a7198d7", size = 17239351, upload-time = "2026-02-22T16:32:28.843Z" }wheels = [@@ -2555,7 +2582,7 @@ name = "jaraco-classes"version = "3.4.0"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "more-itertools" },+ { name = "more-itertools", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/06/c0/ed4a27bc5571b99e3cff68f8a9fa5b56ff7df1c2251cc715a652ddd26402/jaraco.classes-3.4.0.tar.gz", hash = "sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd", size = 11780, upload-time = "2024-03-31T07:27:36.643Z" }wheels = [@@ -2576,7 +2603,7 @@ name = "jaraco-functools"version = "4.4.0"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "more-itertools" },+ { name = "more-itertools", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/0f/27/056e0638a86749374d6f57d0b0db39f29509cce9313cf91bdc0ac4d91084/jaraco_functools-4.4.0.tar.gz", hash = "sha256:da21933b0417b89515562656547a77b4931f98176eb173644c0d35032a33d6bb", size = 19943, upload-time = "2025-12-21T09:29:43.6Z" }wheels = [@@ -2732,9 +2759,9 @@ name = "jsonschema-path"version = "0.4.2"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "pathable" },- { name = "pyyaml" },- { name = "referencing" },+ { name = "pathable", marker = "python_full_version >= '3.12'" },+ { name = "pyyaml", marker = "python_full_version >= '3.12'" },+ { name = "referencing", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/b3/da/1ebeb1c0ff579c330e200e8b06e6200653e3d0758136d8bd86762d63e7de/jsonschema_path-0.4.2.tar.gz", hash = "sha256:5f5ff183150030ea24bb51cf1ddac9bf5dbf030272e2792a7ffe8262f7eea2a5", size = 13417, upload-time = "2026-02-23T16:21:36.602Z" }wheels = [@@ -2758,12 +2785,12 @@ name = "keyring"version = "25.7.0"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "jaraco-classes" },- { name = "jaraco-context" },- { name = "jaraco-functools" },- { name = "jeepney", marker = "sys_platform == 'linux'" },- { name = "pywin32-ctypes", marker = "sys_platform == 'win32'" },- { name = "secretstorage", marker = "sys_platform == 'linux'" },+ { name = "jaraco-classes", marker = "python_full_version >= '3.12'" },+ { name = "jaraco-context", marker = "python_full_version >= '3.12'" },+ { name = "jaraco-functools", marker = "python_full_version >= '3.12'" },+ { name = "jeepney", marker = "python_full_version >= '3.12' and sys_platform == 'linux'" },+ { name = "pywin32-ctypes", marker = "python_full_version >= '3.12' and sys_platform == 'win32'" },+ { name = "secretstorage", marker = "python_full_version >= '3.12' and sys_platform == 'linux'" },]sdist = { url = "https://files.pythonhosted.org/packages/43/4b/674af6ef2f97d56f0ab5153bf0bfa28ccb6c3ed4d1babf4305449668807b/keyring-25.7.0.tar.gz", hash = "sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b", size = 63516, upload-time = "2025-11-16T16:26:09.482Z" }wheels = [@@ -2901,22 +2928,22 @@ name = "lmnr"version = "0.7.42"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "grpcio" },- { name = "httpx" },- { name = "opentelemetry-api" },- { name = "opentelemetry-exporter-otlp-proto-grpc" },- { name = "opentelemetry-exporter-otlp-proto-http" },- { name = "opentelemetry-instrumentation" },- { name = "opentelemetry-instrumentation-threading" },- { name = "opentelemetry-sdk" },- { name = "opentelemetry-semantic-conventions" },- { name = "opentelemetry-semantic-conventions-ai" },- { name = "orjson" },- { name = "packaging" },- { name = "pydantic" },- { name = "python-dotenv" },- { name = "tenacity" },- { name = "tqdm" },+ { name = "grpcio", marker = "python_full_version >= '3.12'" },+ { name = "httpx", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-api", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-exporter-otlp-proto-grpc", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-exporter-otlp-proto-http", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-instrumentation", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-instrumentation-threading", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-sdk", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-semantic-conventions", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-semantic-conventions-ai", marker = "python_full_version >= '3.12'" },+ { name = "orjson", marker = "python_full_version >= '3.12'" },+ { name = "packaging", marker = "python_full_version >= '3.12'" },+ { name = "pydantic", marker = "python_full_version >= '3.12'" },+ { name = "python-dotenv", marker = "python_full_version >= '3.12'" },+ { name = "tenacity", marker = "python_full_version >= '3.12'" },+ { name = "tqdm", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/0a/95/5b9dac7022e81494871aa228ccb0578668781cbc4241b59bd89ebe37536e/lmnr-0.7.42.tar.gz", hash = "sha256:2bea334ca201470120f1c5dd18eccf6aee503c8d962addda2761dfa681f74845", size = 240415, upload-time = "2026-02-23T14:23:24.39Z" }wheels = [@@ -3402,7 +3429,7 @@ name = "mlx"version = "0.31.2"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "mlx-metal" },+ { name = "mlx-metal", marker = "sys_platform == 'darwin'" },]wheels = [{ url = "https://files.pythonhosted.org/packages/29/7c/c16d52494a1ba6d90443f31fa26bc810bf878d532dfa9a7a13f49ef9542d/mlx-0.31.2-cp310-cp310-macosx_14_0_arm64.whl", hash = "sha256:b29cf940f34205f09bb552ac60465ae833c4ae640b52777c6d725ddbad8461ca", size = 586942, upload-time = "2026-04-22T03:14:21.97Z" },@@ -3424,13 +3451,13 @@ name = "mlx-lm"version = "0.31.3"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "jinja2" },- { name = "mlx" },- { name = "numpy" },- { name = "protobuf" },- { name = "pyyaml" },- { name = "sentencepiece" },- { name = "transformers" },+ { name = "jinja2", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },+ { name = "mlx", marker = "sys_platform == 'darwin'" },+ { name = "numpy", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },+ { name = "protobuf", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },+ { name = "pyyaml", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },+ { name = "sentencepiece", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },+ { name = "transformers", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },]sdist = { url = "https://files.pythonhosted.org/packages/84/94/9a38d6b0c6fcca995b9136c94eb7da1e9c5165652edf228b96b29960fa7a/mlx_lm-0.31.3.tar.gz", hash = "sha256:61eb0e3ba09444f77f874aff295401d7ccd20b39495cbbce0c782a15474ce733", size = 304318, upload-time = "2026-04-22T07:37:27.922Z" }wheels = [@@ -3851,7 +3878,7 @@ name = "nvidia-cudnn-cu12"version = "9.10.2.21"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "nvidia-cublas-cu12" },+ { name = "nvidia-cublas-cu12", marker = "sys_platform == 'linux'" },]wheels = [{ url = "https://files.pythonhosted.org/packages/ba/51/e123d997aa098c61d029f76663dedbfb9bc8dcf8c60cbd6adbe42f76d049/nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:949452be657fa16687d0930933f032835951ef0892b37d2d53824d1a84dc97a8", size = 706758467, upload-time = "2025-06-06T21:54:08.597Z" },@@ -3881,7 +3908,7 @@ name = "nvidia-cufft-cu12"version = "11.3.3.83"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "nvidia-nvjitlink-cu12" },+ { name = "nvidia-nvjitlink-cu12", marker = "sys_platform == 'linux'" },]wheels = [{ url = "https://files.pythonhosted.org/packages/1f/13/ee4e00f30e676b66ae65b4f08cb5bcbb8392c03f54f2d5413ea99a5d1c80/nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4d2dd21ec0b88cf61b62e6b43564355e5222e4a3fb394cac0db101f2dd0d4f74", size = 193118695, upload-time = "2025-03-07T01:45:27.821Z" },@@ -3908,9 +3935,9 @@ name = "nvidia-cusolver-cu12"version = "11.7.3.90"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "nvidia-cublas-cu12" },- { name = "nvidia-cusparse-cu12" },- { name = "nvidia-nvjitlink-cu12" },+ { name = "nvidia-cublas-cu12", marker = "sys_platform == 'linux'" },+ { name = "nvidia-cusparse-cu12", marker = "sys_platform == 'linux'" },+ { name = "nvidia-nvjitlink-cu12", marker = "sys_platform == 'linux'" },]wheels = [{ url = "https://files.pythonhosted.org/packages/85/48/9a13d2975803e8cf2777d5ed57b87a0b6ca2cc795f9a4f59796a910bfb80/nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:4376c11ad263152bd50ea295c05370360776f8c3427b30991df774f9fb26c450", size = 267506905, upload-time = "2025-03-07T01:47:16.273Z" },@@ -3921,7 +3948,7 @@ name = "nvidia-cusparse-cu12"version = "12.5.8.93"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "nvidia-nvjitlink-cu12" },+ { name = "nvidia-nvjitlink-cu12", marker = "sys_platform == 'linux'" },]wheels = [{ url = "https://files.pythonhosted.org/packages/c2/f5/e1854cb2f2bcd4280c44736c93550cc300ff4b8c95ebe370d0aa7d2b473d/nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1ec05d76bbbd8b61b06a80e1eaf8cf4959c3d4ce8e711b65ebd0443bb0ebb13b", size = 288216466, upload-time = "2025-03-07T01:48:13.779Z" },@@ -4095,7 +4122,7 @@ name = "openapi-pydantic"version = "0.5.1"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "pydantic" },+ { name = "pydantic", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/02/2e/58d83848dd1a79cb92ed8e63f6ba901ca282c5f09d04af9423ec26c56fd7/openapi_pydantic-0.5.1.tar.gz", hash = "sha256:ff6835af6bde7a459fb93eb93bb92b8749b754fc6e51b2f1590a19dc3005ee0d", size = 60892, upload-time = "2025-01-08T19:29:27.083Z" }wheels = [@@ -4125,17 +4152,17 @@ name = "openhands-sdk"version = "1.11.5"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "deprecation" },- { name = "fastmcp" },- { name = "filelock" },- { name = "httpx" },- { name = "litellm" },- { name = "lmnr" },- { name = "pydantic" },- { name = "python-frontmatter" },- { name = "python-json-logger" },- { name = "tenacity" },- { name = "websockets" },+ { name = "deprecation", marker = "python_full_version >= '3.12'" },+ { name = "fastmcp", marker = "python_full_version >= '3.12'" },+ { name = "filelock", marker = "python_full_version >= '3.12'" },+ { name = "httpx", marker = "python_full_version >= '3.12'" },+ { name = "litellm", marker = "python_full_version >= '3.12'" },+ { name = "lmnr", marker = "python_full_version >= '3.12'" },+ { name = "pydantic", marker = "python_full_version >= '3.12'" },+ { name = "python-frontmatter", marker = "python_full_version >= '3.12'" },+ { name = "python-json-logger", marker = "python_full_version >= '3.12'" },+ { name = "tenacity", marker = "python_full_version >= '3.12'" },+ { name = "websockets", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/3a/fc/cfb73768099be94c9f92b2e160f29d1f6d3a4dd84fea5e33a2b0984449cc/openhands_sdk-1.11.5.tar.gz", hash = "sha256:dd6225876b7b8dbb6c608559f2718c3d0bf44d0bb741e990b185c6cdc5150c5a", size = 295069, upload-time = "2026-02-20T22:16:47.102Z" }wheels = [@@ -4161,6 +4188,9 @@ dependencies = [][package.optional-dependencies]+afm = [+ { name = "apple-fm-sdk", marker = "sys_platform == 'darwin'" },+]browser = [{ name = "playwright" },]@@ -4249,13 +4279,13 @@ docs = [energy-all = [{ name = "amdsmi" },{ name = "pynvml" },- { name = "zeus-ml" },+ { name = "zeus-apple-silicon", marker = "sys_platform == 'darwin'" },]energy-amd = [{ name = "amdsmi" },]energy-apple = [- { name = "zeus-ml" },+ { name = "zeus-apple-silicon", marker = "sys_platform == 'darwin'" },]eval-sheets = [{ name = "google-auth" },@@ -4372,6 +4402,7 @@ requires-dist = [{ name = "amdsmi", marker = "extra == 'energy-all'", specifier = ">=6.1" },{ name = "amdsmi", marker = "extra == 'energy-amd'", specifier = ">=6.1" },{ name = "anthropic", marker = "extra == 'inference-cloud'", specifier = ">=0.30" },+ { name = "apple-fm-sdk", marker = "sys_platform == 'darwin' and extra == 'afm'", specifier = ">=0.2.1" },{ name = "click", specifier = ">=8" },{ name = "colbert-ai", marker = "extra == 'memory-colbert'", specifier = ">=0.2" },{ name = "croniter", marker = "extra == 'scheduler'", specifier = ">=2.0" },@@ -4467,11 +4498,11 @@ requires-dist = [{ name = "wandb", marker = "extra == 'eval-wandb'", specifier = ">=0.17" },{ name = "wasmtime", marker = "extra == 'sandbox-wasm'", specifier = ">=25" },{ name = "websockets", specifier = ">=15.0.1" },- { name = "zeus-ml", extras = ["apple"], marker = "extra == 'energy-all'" },- { name = "zeus-ml", extras = ["apple"], marker = "extra == 'energy-apple'" },+ { name = "zeus-apple-silicon", marker = "sys_platform == 'darwin' and extra == 'energy-all'", specifier = ">=1.1.0" },+ { name = "zeus-apple-silicon", marker = "sys_platform == 'darwin' and extra == 'energy-apple'", specifier = ">=1.1.0" },{ name = "zulip", marker = "extra == 'channel-zulip'", specifier = ">=0.9" },]-provides-extras = ["browser", "channel-discord", "channel-gmail", "channel-line", "channel-mastodon", "channel-messenger", "channel-nostr", "channel-reddit", "channel-rocketchat", "channel-slack", "channel-telegram", "channel-twilio", "channel-twitch", "channel-twitter", "channel-viber", "channel-xmpp", "channel-zulip", "dashboard", "desktop", "dev", "docs", "energy-all", "energy-amd", "energy-apple", "eval-sheets", "eval-wandb", "framework-comparison", "gpu-metrics", "inference-cloud", "inference-gemma", "inference-google", "inference-litellm", "inference-mlx", "inference-vllm", "learning-dspy", "learning-gepa", "media", "memory-bm25", "memory-colbert", "memory-faiss", "memory-pdf", "mining-pearl-cpu", "mining-pearl-vllm", "openhands", "orchestrator-training", "pdf", "sandbox-docker", "sandbox-wasm", "scheduler", "security-signing", "server", "speech", "speech-deepgram", "tools-search"]+provides-extras = ["afm", "browser", "channel-discord", "channel-gmail", "channel-line", "channel-mastodon", "channel-messenger", "channel-nostr", "channel-reddit", "channel-rocketchat", "channel-slack", "channel-telegram", "channel-twilio", "channel-twitch", "channel-twitter", "channel-viber", "channel-xmpp", "channel-zulip", "dashboard", "desktop", "dev", "docs", "energy-all", "energy-amd", "energy-apple", "eval-sheets", "eval-wandb", "framework-comparison", "gpu-metrics", "inference-cloud", "inference-gemma", "inference-google", "inference-litellm", "inference-mlx", "inference-vllm", "learning-dspy", "learning-gepa", "media", "memory-bm25", "memory-colbert", "memory-faiss", "memory-pdf", "mining-pearl-cpu", "mining-pearl-vllm", "openhands", "orchestrator-training", "pdf", "sandbox-docker", "sandbox-wasm", "scheduler", "security-signing", "server", "speech", "speech-deepgram", "tools-search"][package.metadata.requires-dev]desktop-native = [{ name = "openjarvis-rust", directory = "rust/crates/openjarvis-python" }]@@ -4561,10 +4592,10 @@ name = "opentelemetry-instrumentation"version = "0.60b1"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "opentelemetry-api" },- { name = "opentelemetry-semantic-conventions" },- { name = "packaging" },- { name = "wrapt" },+ { name = "opentelemetry-api", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-semantic-conventions", marker = "python_full_version >= '3.12'" },+ { name = "packaging", marker = "python_full_version >= '3.12'" },+ { name = "wrapt", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/41/0f/7e6b713ac117c1f5e4e3300748af699b9902a2e5e34c9cf443dde25a01fa/opentelemetry_instrumentation-0.60b1.tar.gz", hash = "sha256:57ddc7974c6eb35865af0426d1a17132b88b2ed8586897fee187fd5b8944bd6a", size = 31706, upload-time = "2025-12-11T13:36:42.515Z" }wheels = [@@ -4576,9 +4607,9 @@ name = "opentelemetry-instrumentation-threading"version = "0.60b1"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "opentelemetry-api" },- { name = "opentelemetry-instrumentation" },- { name = "wrapt" },+ { name = "opentelemetry-api", marker = "python_full_version >= '3.12'" },+ { name = "opentelemetry-instrumentation", marker = "python_full_version >= '3.12'" },+ { name = "wrapt", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/9b/0a/e36123ec4c0910a3936b92982545a53e9bca5b26a28df06883751a783f84/opentelemetry_instrumentation_threading-0.60b1.tar.gz", hash = "sha256:20b18a68abe5801fa9474336b7c27487d4af3e00b66f6a8734e4fdd75c8b0b43", size = 8768, upload-time = "2025-12-11T13:37:16.29Z" }wheels = [@@ -4784,10 +4815,10 @@ resolution-markers = ["python_full_version < '3.11' and sys_platform != 'linux'",]dependencies = [- { name = "numpy" },- { name = "python-dateutil" },- { name = "pytz" },- { name = "tzdata" },+ { name = "numpy", marker = "python_full_version < '3.11'" },+ { name = "python-dateutil", marker = "python_full_version < '3.11'" },+ { name = "pytz", marker = "python_full_version < '3.11'" },+ { name = "tzdata", marker = "python_full_version < '3.11'" },]sdist = { url = "https://files.pythonhosted.org/packages/33/01/d40b85317f86cf08d853a4f495195c73815fdf205eef3993821720274518/pandas-2.3.3.tar.gz", hash = "sha256:e05e1af93b977f7eafa636d043f9f94c7ee3ac81af99c13508215942e64c993b", size = 4495223, upload-time = "2025-09-29T23:34:51.853Z" }wheels = [@@ -4846,9 +4877,9 @@ resolution-markers = ["python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'",]dependencies = [- { name = "numpy" },- { name = "python-dateutil" },- { name = "tzdata", marker = "sys_platform == 'emscripten' or sys_platform == 'win32'" },+ { name = "numpy", marker = "python_full_version >= '3.11'" },+ { name = "python-dateutil", marker = "python_full_version >= '3.11'" },+ { name = "tzdata", marker = "(python_full_version >= '3.11' and sys_platform == 'emscripten') or (python_full_version >= '3.11' and sys_platform == 'win32')" },]sdist = { url = "https://files.pythonhosted.org/packages/de/da/b1dc0481ab8d55d0f46e343cfe67d4551a0e14fcee52bd38ca1bd73258d8/pandas-3.0.0.tar.gz", hash = "sha256:0facf7e87d38f721f0af46fe70d97373a37701b1c09f7ed7aeeb292ade5c050f", size = 4633005, upload-time = "2026-01-21T15:52:04.726Z" }wheels = [@@ -5305,8 +5336,8 @@ name = "py-key-value-aio"version = "0.4.4"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "beartype" },- { name = "typing-extensions" },+ { name = "beartype", marker = "python_full_version >= '3.12'" },+ { name = "typing-extensions", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/04/3c/0397c072a38d4bc580994b42e0c90c5f44f679303489e4376289534735e5/py_key_value_aio-0.4.4.tar.gz", hash = "sha256:e3012e6243ed7cc09bb05457bd4d03b1ba5c2b1ca8700096b3927db79ffbbe55", size = 92300, upload-time = "2026-02-16T21:21:43.245Z" }wheels = [@@ -5315,14 +5346,14 @@ wheels = [[package.optional-dependencies]filetree = [- { name = "aiofile" },- { name = "anyio" },+ { name = "aiofile", marker = "python_full_version >= '3.12'" },+ { name = "anyio", marker = "python_full_version >= '3.12'" },]keyring = [- { name = "keyring" },+ { name = "keyring", marker = "python_full_version >= '3.12'" },]memory = [- { name = "cachetools" },+ { name = "cachetools", marker = "python_full_version >= '3.12'" },][[package]]@@ -5884,6 +5915,15 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/df/80/fc9d01d5ed37ba4c42ca2b55b4339ae6e200b456be3a1aaddf4a9fa99b8c/pyperclip-1.11.0-py3-none-any.whl", hash = "sha256:299403e9ff44581cb9ba2ffeed69c7aa96a008622ad0c46cb575ca75b5b84273", size = 11063, upload-time = "2025-09-26T14:40:36.069Z" },]+[[package]]+name = "pyproject-hooks"+version = "1.2.0"+source = { registry = "https://pypi.org/simple" }+sdist = { url = "https://files.pythonhosted.org/packages/e7/82/28175b2414effca1cdac8dc99f76d660e7a4fb0ceefa4b4ab8f5f6742925/pyproject_hooks-1.2.0.tar.gz", hash = "sha256:1e859bd5c40fae9448642dd871adf459e5e2084186e8d2c2a79a824c970da1f8", size = 19228, upload-time = "2024-09-29T09:24:13.293Z" }+wheels = [+ { url = "https://files.pythonhosted.org/packages/bd/24/12818598c362d7f300f18e74db45963dbcb85150324092410c8b49405e42/pyproject_hooks-1.2.0-py3-none-any.whl", hash = "sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913", size = 10216, upload-time = "2024-09-29T09:24:11.978Z" },+]+[[package]]name = "pytest"version = "9.0.2"@@ -5982,7 +6022,7 @@ name = "python-frontmatter"version = "1.1.0"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "pyyaml" },+ { name = "pyyaml", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/96/de/910fa208120314a12f9a88ea63e03707261692af782c99283f1a2c8a5e6f/python-frontmatter-1.1.0.tar.gz", hash = "sha256:7118d2bd56af9149625745c58c9b51fb67e8d1294a0c76796dafdc72c36e5f6d", size = 16256, upload-time = "2024-01-16T18:50:04.052Z" }wheels = [@@ -6395,8 +6435,8 @@ name = "rich-rst"version = "1.3.2"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "docutils" },- { name = "rich" },+ { name = "docutils", marker = "python_full_version >= '3.12'" },+ { name = "rich", marker = "python_full_version >= '3.12'" },]sdist = { url = "https://files.pythonhosted.org/packages/bc/6d/a506aaa4a9eaa945ed8ab2b7347859f53593864289853c5d6d62b77246e0/rich_rst-1.3.2.tar.gz", hash = "sha256:a1196fdddf1e364b02ec68a05e8ff8f6914fee10fbca2e6b6735f166bb0da8d4", size = 14936, upload-time = "2025-10-14T16:49:45.332Z" }wheels = [@@ -6686,10 +6726,10 @@ resolution-markers = ["python_full_version < '3.11' and sys_platform != 'linux'",]dependencies = [- { name = "joblib" },- { name = "numpy" },- { name = "scipy", version = "1.15.3", source = { registry = "https://pypi.org/simple" } },- { name = "threadpoolctl" },+ { name = "joblib", marker = "python_full_version < '3.11'" },+ { name = "numpy", marker = "python_full_version < '3.11'" },+ { name = "scipy", version = "1.15.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },+ { name = "threadpoolctl", marker = "python_full_version < '3.11'" },]sdist = { url = "https://files.pythonhosted.org/packages/98/c2/a7855e41c9d285dfe86dc50b250978105dce513d6e459ea66a6aeb0e1e0c/scikit_learn-1.7.2.tar.gz", hash = "sha256:20e9e49ecd130598f1ca38a1d85090e1a600147b9c02fa6f15d69cb53d968fda", size = 7193136, upload-time = "2025-09-09T08:21:29.075Z" }wheels = [@@ -6739,10 +6779,10 @@ resolution-markers = ["python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'",]dependencies = [- { name = "joblib" },- { name = "numpy" },- { name = "scipy", version = "1.17.0", source = { registry = "https://pypi.org/simple" } },- { name = "threadpoolctl" },+ { name = "joblib", marker = "python_full_version >= '3.11'" },+ { name = "numpy", marker = "python_full_version >= '3.11'" },+ { name = "scipy", version = "1.17.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },+ { name = "threadpoolctl", marker = "python_full_version >= '3.11'" },]sdist = { url = "https://files.pythonhosted.org/packages/0e/d4/40988bf3b8e34feec1d0e6a051446b1f66225f8529b9309becaeef62b6c4/scikit_learn-1.8.0.tar.gz", hash = "sha256:9bccbb3b40e3de10351f8f5068e105d0f4083b1a65fa07b6634fbc401a6287fd", size = 7335585, upload-time = "2025-12-10T07:08:53.618Z" }wheels = [@@ -6781,7 +6821,7 @@ resolution-markers = ["python_full_version < '3.11' and sys_platform != 'linux'",]dependencies = [- { name = "numpy" },+ { name = "numpy", marker = "python_full_version < '3.11'" },]sdist = { url = "https://files.pythonhosted.org/packages/0f/37/6964b830433e654ec7485e45a00fc9a27cf868d622838f6b6d9c5ec0d532/scipy-1.15.3.tar.gz", hash = "sha256:eae3cf522bc7df64b42cad3925c876e1b0b6c35c1337c93e12c0f366f55b0eaf", size = 59419214, upload-time = "2025-05-08T16:13:05.955Z" }wheels = [@@ -6851,7 +6891,7 @@ resolution-markers = ["python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'",]dependencies = [- { name = "numpy" },+ { name = "numpy", marker = "python_full_version >= '3.11'" },]sdist = { url = "https://files.pythonhosted.org/packages/56/3e/9cca699f3486ce6bc12ff46dc2031f1ec8eb9ccc9a320fdaf925f1417426/scipy-1.17.0.tar.gz", hash = "sha256:2591060c8e648d8b96439e111ac41fd8342fdeff1876be2e19dea3fe8930454e", size = 30396830, upload-time = "2026-01-10T21:34:23.009Z" }wheels = [@@ -6902,8 +6942,8 @@ name = "secretstorage"version = "3.5.0"source = { registry = "https://pypi.org/simple" }dependencies = [- { name = "cryptography" },- { name = "jeepney" },+ { name = "cryptography", marker = "python_full_version >= '3.12' and sys_platform == 'linux'" },+ { name = "jeepney", marker = "python_full_version >= '3.12' and sys_platform == 'linux'" },]sdist = { url = "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz", hash = "sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", size = 19884, upload-time = "2025-11-23T19:02:53.191Z" }wheels = [@@ -7128,9 +7168,9 @@ resolution-markers = ["python_full_version < '3.11' and sys_platform != 'linux'",]dependencies = [- { name = "aiodns" },- { name = "pyasn1" },- { name = "pyasn1-modules" },+ { name = "aiodns", marker = "python_full_version < '3.11'" },+ { name = "pyasn1", marker = "python_full_version < '3.11'" },+ { name = "pyasn1-modules", marker = "python_full_version < '3.11'" },]sdist = { url = "https://files.pythonhosted.org/packages/d9/03/925a18ce8aced370a5135b3e0ef870ec71a30da005f747891c83df98ca53/slixmpp-1.12.0.tar.gz", hash = "sha256:7469f6dcaf5742fd62e0e66ee447c5338a74520c1982a70784e7b790def2fdd5", size = 715300, upload-time = "2025-10-19T17:50:18.975Z" }wheels = [@@ -7171,9 +7211,9 @@ resolution-markers = ["python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'",]dependencies = [- { name = "aiodns" },- { name = "pyasn1" },- { name = "pyasn1-modules" },+ { name = "aiodns", marker = "python_full_version >= '3.11'" },+ { name = "pyasn1", marker = "python_full_version >= '3.11'" },+ { name = "pyasn1-modules", marker = "python_full_version >= '3.11'" },]sdist = { url = "https://files.pythonhosted.org/packages/8b/1c/6ce021fb5524b41330d583956d1ff8e508c95d6c1bb0ed34790e754cc8d2/slixmpp-1.13.2.tar.gz", hash = "sha256:1b6f7c4c273ae5d34c4e54f6d66984a27467de225312f19f942f971f38b78da2", size = 757698, upload-time = "2026-02-08T19:05:21.165Z" }wheels = [@@ -7765,9 +7805,9 @@ resolution-markers = ["python_full_version < '3.11' and sys_platform != 'linux'",]dependencies = [- { name = "aiohttp" },- { name = "iso8601" },- { name = "typing-extensions" },+ { name = "aiohttp", marker = "python_full_version < '3.11'" },+ { name = "iso8601", marker = "python_full_version < '3.11'" },+ { name = "typing-extensions", marker = "python_full_version < '3.11'" },]sdist = { url = "https://files.pythonhosted.org/packages/de/61/3f177d7e48af2816bd7bc639a7d5c4f2fa0ff3ae46faf36754b3c3676de9/twitchio-2.10.0.tar.gz", hash = "sha256:3ae5e17b3764eff24ad7d12decb473c9d34f18510b5d705e0bbe6fcf0b06fd75", size = 114393, upload-time = "2024-06-25T14:47:22.463Z" }wheels = [@@ -7793,25 +7833,13 @@ resolution-markers = ["python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'",]dependencies = [- { name = "aiohttp" },+ { name = "aiohttp", marker = "python_full_version >= '3.11'" },]sdist = { url = "https://files.pythonhosted.org/packages/16/55/d0b8f1c6552f23994d925301fa7d5743109568cc8f079fee9866c5a52499/twitchio-3.2.1.tar.gz", hash = "sha256:a1be858a4028625173978db91224326b910a0aaa4c7db879a8867003642a0115", size = 248627, upload-time = "2026-01-29T14:14:44.803Z" }wheels = [{ url = "https://files.pythonhosted.org/packages/7e/a1/2f7e31066eb2c6e78cecd95cf1dd66773cf8f90738339320e12df5e2da2d/twitchio-3.2.1-py3-none-any.whl", hash = "sha256:576d131fdfb0378ee0bfd7f35f8381636f9a911c3961589add97055a578da555", size = 320563, upload-time = "2026-01-29T14:14:43.252Z" },]-[[package]]-name = "typeguard"-version = "4.5.1"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "typing-extensions" },-]-sdist = { url = "https://files.pythonhosted.org/packages/2b/e8/66e25efcc18542d58706ce4e50415710593721aae26e794ab1dec34fb66f/typeguard-4.5.1.tar.gz", hash = "sha256:f6f8ecbbc819c9bc749983cc67c02391e16a9b43b8b27f15dc70ed7c4a007274", size = 80121, upload-time = "2026-02-19T16:09:03.392Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/91/88/b55b3117287a8540b76dbdd87733808d4d01c8067a3b339408c250bb3600/typeguard-4.5.1-py3-none-any.whl", hash = "sha256:44d2bf329d49a244110a090b55f5f91aa82d9a9834ebfd30bcc73651e4a8cc40", size = 36745, upload-time = "2026-02-19T16:09:01.6Z" },-]-[[package]]name = "typer"version = "0.24.0"@@ -7848,20 +7876,6 @@ wheels = [{ url = "https://files.pythonhosted.org/packages/dc/9b/47798a6c91d8bdb567fe2698fe81e0c6b7cb7ef4d13da4114b41d239f65d/typing_inspection-0.4.2-py3-none-any.whl", hash = "sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7", size = 14611, upload-time = "2025-10-01T02:14:40.154Z" },]-[[package]]-name = "tyro"-version = "1.0.8"-source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "docstring-parser" },- { name = "typeguard" },- { name = "typing-extensions" },-]-sdist = { url = "https://files.pythonhosted.org/packages/9e/89/f4d76ba5ac4ca627c3a0413fdc8d90f202929c45d9c25deec29c9a86a0ae/tyro-1.0.8.tar.gz", hash = "sha256:c22bf238ce029b8cc262759129fccfd968300c3761f00ce186570af3891e14bb", size = 460626, upload-time = "2026-02-25T00:18:45.388Z" }-wheels = [- { url = "https://files.pythonhosted.org/packages/61/7c/9c5a4ab9e24f1428fd5ab2728b7f68878f54d549a11fa431ead76d3d127b/tyro-1.0.8-py3-none-any.whl", hash = "sha256:abb6a4054f20616d9ca5181ba0c6ae0e35cf8f35ae0c78d3d4dbd1f678e0643d", size = 181935, upload-time = "2026-02-25T00:18:43.689Z" },-]-[[package]]name = "tzdata"version = "2025.3"@@ -8670,26 +8684,16 @@ wheels = [][[package]]-name = "zeus-ml"-version = "0.11.0.post1"+name = "zeus-apple-silicon"+version = "1.1.0"source = { registry = "https://pypi.org/simple" }-dependencies = [- { name = "amdsmi" },- { name = "httpx" },- { name = "numpy" },- { name = "nvidia-ml-py" },- { name = "pandas", version = "2.3.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },- { name = "pandas", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },- { name = "pydantic" },- { name = "python-dateutil" },- { name = "rich" },- { name = "scikit-learn", version = "1.7.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },- { name = "scikit-learn", version = "1.8.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },- { name = "tyro" },-]-sdist = { url = "https://files.pythonhosted.org/packages/c7/26/e7e70121272fffc4c9ce3ec232e457d33d02b490eefe1b7a2b022bf296aa/zeus_ml-0.11.0.post1.tar.gz", hash = "sha256:0bc061b6c34edcfc2e86b3de81000157e30227fe11e7d0c683f6f5cf7bc3be22", size = 178924, upload-time = "2025-02-03T01:17:07.93Z" }+sdist = { url = "https://files.pythonhosted.org/packages/64/a5/7da627c178365b90d2e4c83c78ed3ccb085251d4247584d8ca9929e86ffb/zeus_apple_silicon-1.1.0.tar.gz", hash = "sha256:ca99b33e9e375498b2f5b9671eee50258ed7483c10d62eac7595cbb882ce3a4e", size = 46173, upload-time = "2026-03-29T05:30:46.405Z" }wheels = [- { url = "https://files.pythonhosted.org/packages/37/f4/d787c837308a26d7db69154761ae8b7e7f0a0703952cd57dc40fd13471c8/zeus_ml-0.11.0.post1-py3-none-any.whl", hash = "sha256:d0554ff3a4b8a27d818f4ea8688daa500be5f10322fb5824365aef24c1682e43", size = 227090, upload-time = "2025-02-03T01:17:04.018Z" },+ { url = "https://files.pythonhosted.org/packages/b9/2c/2d2cd9d53167b4ab951217d54fa13f3ac183808958193de7d83455903d10/zeus_apple_silicon-1.1.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:ed75237100da14552c50618ad8a18e16eb8d7fb1bdf31c43f5134bf2ab47830d", size = 62858, upload-time = "2026-03-29T05:30:37.676Z" },+ { url = "https://files.pythonhosted.org/packages/4a/cd/69179ff6d638103440457cbe1c8c0b30d0a22a6bb6f2c043a42995fb99e4/zeus_apple_silicon-1.1.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:fca4ff0211b1440841fa8454ea6effedc55c83a17be770ad92bb39da961a2214", size = 62752, upload-time = "2026-03-29T05:30:38.966Z" },+ { url = "https://files.pythonhosted.org/packages/4b/50/dae39d71cdfed918eea04596ca582f3a8428bb8e44d76e88020d02f04b75/zeus_apple_silicon-1.1.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:c862421e3477344260fad80276cb6b8f6ed7189b5472ef3fca9a54cd5a780aa7", size = 61615, upload-time = "2026-03-29T05:30:40.228Z" },+ { url = "https://files.pythonhosted.org/packages/c8/0c/e162be99be598d9be1ee48732a95475a42473b311c4809fc01dcc13a68c7/zeus_apple_silicon-1.1.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:410ecf3f594fc94202ad778d274131a24a403553c9b56c7907344edc9107ec03", size = 61526, upload-time = "2026-03-29T05:30:41.568Z" },+ { url = "https://files.pythonhosted.org/packages/a9/a9/ed1a74db7dbad082bb4ea8fc91660f96d9adcc52d4dea1afecc74015a96c/zeus_apple_silicon-1.1.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:e111ac377a08f2e06cb1303af8625927a8a9dcd5cf6d2796a655eddf18debe4b", size = 59941, upload-time = "2026-03-29T05:30:44.092Z" },][[package]]