Skip to content

Personal AI, On Personal Devices

Python

Feature/afm3 support (#904)

MRH113dbd4a1dfd75e
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4171889d..e2a6fb0a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,50 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
### Added
+**Apple Foundation Models (AFM 3)** — a new in-process `afm` engine drives
+Apple's `apple-fm-sdk` directly, with no HTTP hop and no second process whose
+CPU draw would land inside the same energy measurement window. Install with
+`DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer uv sync --extra afm`
+(the SDK compiles Swift bindings, so a full Xcode is required), then
+`jarvis ask --engine afm --model afm-3-core "..."`. Requires an Apple Silicon
+Mac on macOS 26+ with Apple Intelligence enabled.
+
+Token counts are real, via `SystemLanguageModel.token_count` (added in SDK
+0.2.1, hence the floor). Because `token_count` rejects a value and
+instructions together, both ends are measured against the session transcript:
+`prompt_tokens = <transcript before> + <prompt alone>` and
+`completion_tokens = <transcript after> - prompt_tokens`.
+
+Two caveats worth reading before comparing AFM numbers to other backends:
+`stream_response` yields cumulative snapshots batching roughly 8-10 tokens, so
+`ttft` is time-to-first-*chunk* (~450 ms on an M1 Pro) and derived
+inter-token latencies are inter-chunk; and the SDK exposes no variant
+selector, so `afm-3` / `afm-3-core` / `afm-3-core-advanced` are run labels
+only — the framework's dynamic profile chooses what actually executes.
+Bench results record `metadata["engine_info"]` (SDK version, context size,
+host chip) so a run can be attributed after the fact.
+
+**Eval results record which rail did the work.** Result rows and summaries
+previously carried only `energy_joules` and `power_watts`, so nothing in an
+eval artifact distinguished a Neural Engine run from an idle GPU, or a
+hardware measurement from a modelled estimate. Rows now include the per-rail
+breakdown (`cpu`/`gpu`/`dram`/`ane`/`soc`), `energy_basis`, and
+`energy_method`; summaries add `energy_joules_by_rail` alongside both. On a
+ToolCall-15 run against AFM the ANE rail is the largest single component —
+91.7 J of 223.7 J, against 3.0 J on the GPU — which is now visible in the
+file rather than only through live instrumentation.
+
+**ANE and whole-SoC energy** — `EnergySample.ane_energy_joules` existed but
+died at the monitor boundary. Apple Neural Engine and a whole-SoC total now
+flow through `TelemetryRecord`, the SQLite schema (with an `ADD COLUMN`
+migration), the aggregator, `TelemetrySession`, eval `TurnTrace`/`QueryTrace`,
+and bench metadata, alongside an `energy_basis` marker (`"soc"` on Apple
+Silicon, `"gpu"` elsewhere) so downstream repeats the monitor's choice instead
+of guessing. This matters because AFM runs predominantly on the ANE: measured
+on an M1 Pro, one generation drew 0.98 J on the ANE against 0.014 J on the
+GPU rail — 70x — where an MLX matmul on the same machine drew 8.99 J on the
+GPU and nothing on the ANE.
+
**Vision input for `jarvis ask`** — attach images to a query with
`-i`/`--image` (repeatable) or capture the current screen with
`-S`/`--screen`, for vision-capable models such as `gemma3:4b`. Images flow
@@ -21,6 +65,49 @@ stack with `mss`/`Pillow` fallbacks on other platforms. Adds the
`JARVIS_NUM_CTX` environment variable to tune the Ollama context window
(default `16384`).
+### Fixed
+
+**Apple Silicon energy was never measured, only modelled.**
+`telemetry/energy_apple.py` imported `AppleSiliconMonitor` from
+`zeus.device.soc.apple`; no such class has ever existed there. The import
+raised on every machine, so the CPU-time fallback -- `wall_clock x TDP x 0.60`
+split by four hardcoded ratios -- was the only path that ever ran. It read no
+counters and no utilization, so a ten-second sleep and a ten-second generation
+produced identical joules, and every Apple Silicon figure in the telemetry DB,
+eval traces and `jarvis bench` came from it. Renaming the import would not
+have helped: `zeus.device.soc` landed on zeus master after the last zeus
+release, so no published `zeus-ml` contains it, and `zeus-ml` has no `apple`
+extra either. `energy-apple` now installs `zeus-apple-silicon`, the IOReport
+extension zeus itself wraps, which needs no root. Measured on an M1 Pro:
+8.24 W under load against 3.72 W idle, where the previous code could not tell
+the two windows apart.
+
+`AppleEnergyMonitor.available()` now reports `False` when nothing is
+measurable, instead of `True` on any Apple Silicon host, so the factory falls
+through rather than letting a model masquerade as a measurement. The estimate
+remains reachable through the new `telemetry.allow_energy_estimates` setting
+and still reports `energy_method = "tdp_estimate"`.
+
+`snapshot()` is implemented for the first time, so `TelemetrySession`'s
+background sampler -- the path agentic evals use -- records real readings
+rather than the ABC's empty default. Relatedly, `TelemetrySample` hardcoded
+`cpu_power_w = 0.0`, which zeroed CPU power in every eval trace on every
+platform.
+
+**`apple_fm` shim reported zero token counts.** It hardcoded zeros with a note
+that the SDK did not expose counts; SDK 0.2.1 added `token_count`. Zeroed
+`completion_tokens` zeroes throughput, `energy_per_output_token_joules` and
+`tokens_per_joule` for everything served through the shim. Streaming responses
+now also carry `usage` on the final chunk. The install hint no longer tells
+users to clone from GitHub -- `apple-fm-sdk` is on PyPI. System messages become
+the session's `instructions` rather than being prefixed onto the prompt as
+`[System] ...`, which discarded a distinction the SDK draws and inflated the
+prompt-token count.
+
+**`scripts/setup-energy-monitor.sh` did not exist**, though
+`jarvis bench --setup-energy` built a path to it and offered to run it -- so
+the offer silently did nothing.
+
### Security
**WebSocket API keys no longer appear in request URLs.** Browser clients now
diff --git a/configs/openjarvis/config.toml b/configs/openjarvis/config.toml
index 943d31c4..fe605853 100644
--- a/configs/openjarvis/config.toml
+++ b/configs/openjarvis/config.toml
@@ -65,7 +65,15 @@ host = "http://localhost:18181"
host = "http://localhost:8080"
[engine.apple_fm]
-host = "http://localhost:8079"
+host = "http://localhost:8079" # OpenAI-compatible shim
+
+# In-process Apple Foundation Models (macOS 26+, Apple Intelligence enabled).
+# Install with: uv pip install -e '.[afm]'
+[engine.afm]
+# instructions = "" # applied to every session
+# use_case = "general" # or "content_tagging"
+# guardrails = "default" # or "permissive_content_transformations"
+sampling = "greedy" # or "random" (the SDK default)
# ═══════════════════════════════════════════════════════════════
# PILLAR 5: Learning — Improvement Methodologies
diff --git a/docs/architecture/engine.md b/docs/architecture/engine.md
index 7fad6232..ba4be3c1 100644
--- a/docs/architecture/engine.md
+++ b/docs/architecture/engine.md
@@ -118,7 +118,8 @@ All providers produce the same output format consumed by agents:
| **Nexa** | `nexa` | OpenAI-compatible | 18181 | No (CPU/GPU) | On-device inference with GGUF models |
| **Lemonade** | `lemonade` | OpenAI-compatible | 13305 | AMD GPU/NPU | AMD consumer GPUs (RDNA), Ryzen AI NPUs |
| **Uzu** | `uzu` | OpenAI-compatible | 8000 | Varies | Uzu inference runtime |
-| **Apple FM** | `apple_fm` | OpenAI-compatible | 8079 | Apple Silicon | Apple Foundation Model on-device inference |
+| **Apple FM (shim)** | `apple_fm` | OpenAI-compatible | 8079 | Apple Silicon | Apple Foundation Models over HTTP, for external OpenAI clients |
+| **Apple FM (in-process)** | `afm` | Python SDK (in-process) | — | Apple Silicon | Apple Foundation Models with no HTTP hop — the preferred path |
| **LiteLLM** | `litellm` | OpenAI-compatible | — | No | Unified proxy to 100+ LLM providers |
| **Cloud** | `cloud` | Provider SDKs | — | No | OpenAI, Anthropic, Google API access |
@@ -218,17 +219,42 @@ The Uzu backend connects to the Uzu inference runtime. Unlike other OpenAI-compa
- **Health check:** `GET /models`
- **Best for:** Uzu-optimized inference workloads
-### Apple FM
+### Apple Foundation Models (AFM 3)
-The Apple FM backend connects to Apple's Foundation Model SDK via a FastAPI shim (`apple_fm_shim.py`). It wraps `python-apple-fm-sdk` as an OpenAI-compatible API. Requires macOS 15+ with Apple Silicon.
+Two backends serve the same on-device model. Both require an Apple Silicon Mac on **macOS 26+** with Apple Intelligence enabled.
-!!! note "Token counts"
- The Apple FM SDK does not expose token counts. The shim returns 0 for all token counts. Benchmark throughput and energy-per-token metrics will reflect this limitation.
+**`afm` — in-process (preferred).** Drives `apple_fm_sdk` directly, with no HTTP hop and no second process. Prefer it for anything energy-measured: a separate uvicorn process burns CPU inside the same measurement window.
-- **Default host:** `http://localhost:8079`
-- **Health check:** `GET /v1/models`
-- **Install:** `pip install python-apple-fm-sdk`
-- **Best for:** Running Apple Foundation Models natively on Apple Silicon hardware
+**`apple_fm` — OpenAI-compatible shim.** A FastAPI server (`apple_fm_shim.py`) on port 8079, for pointing external OpenAI clients at AFM.
+
+**Install.** The SDK compiles Swift bindings at install time, so a full Xcode is required — Command Line Tools alone fail:
+
+```bash
+DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
+ uv sync --extra afm
+```
+
+`>=0.2.1` is the floor: that release added `SystemLanguageModel.token_count` and `.context_size`, which both backends use for token accounting.
+
+```bash
+jarvis ask --engine afm --model afm-3-core "..." # in-process
+jarvis host afm-3 --backend apple_fm # launch the shim
+```
+
+- **Model labels:** `afm-3`, `afm-3-core`, `afm-3-core-advanced`
+- **Context length:** 4096 tokens
+- **Configuration:** `[engine.afm]` — `instructions`, `use_case`, `guardrails`, `sampling`
+
+!!! warning "The model label does not select the model"
+ The SDK exposes no variant selector. The Foundation Models framework's dynamic profile picks AFM 3 Core (dense ~3B) or AFM 3 Core Advanced (20B sparse MoE, 1–4B active per request) based on the host device. The label only tags the run. `AppleFMEngine.describe()` records the SDK version, context size and host chip so a run can be attributed after the fact.
+
+!!! warning "Latency figures are not comparable to other backends"
+ `stream_response` yields cumulative text snapshots batching roughly 8–10 tokens each, not individual tokens. `ttft` is therefore time-to-first-*chunk* — around 450 ms on an M1 Pro, versus a few tens of ms of real first-token latency — and derived inter-token latencies are inter-*chunk* latencies. Token counts, throughput and per-token energy are unaffected.
+
+!!! note "Energy"
+ AFM runs predominantly on the Neural Engine, so its energy is invisible on a GPU-only rail. Install `openjarvis[energy-apple]` to measure CPU/GPU/DRAM/ANE separately; records are then written with `energy_basis = "soc"` and an `ane_energy_joules` column. Without it, no Apple energy is recorded at all.
+
+There is no Private Cloud Compute path in the Python SDK — inference is always on-device, which is what makes the energy measurement meaningful.
### LiteLLM
diff --git a/pyproject.toml b/pyproject.toml
index b7705194..561bfaef 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -98,10 +98,27 @@ desktop = [
"soundfile>=0.12",
]
openhands = ["openhands-sdk>=1.0; python_version >= '3.12'"]
+# Apple Foundation Models (on-device AFM 3). Needs an Apple Silicon Mac on
+# macOS 26+ with Apple Intelligence enabled. The SDK compiles Swift bindings
+# at install time, so a full Xcode is required -- Command Line Tools alone
+# fail:
+# DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
+# uv pip install -e '.[afm]'
+# 0.2.1 is the floor: it added SystemLanguageModel.context_size and
+# token_count, which the engine uses for token accounting.
+afm = ["apple-fm-sdk>=0.2.1; sys_platform == 'darwin'"]
gpu-metrics = ["pynvml>=12.0"]
energy-amd = ["amdsmi>=6.1"]
-energy-apple = ["zeus-ml[apple]"]
-energy-all = ["pynvml>=12.0", "amdsmi>=6.1", "zeus-ml[apple]"]
+# `zeus-ml[apple]` was the previous spelling and never worked: zeus-ml has no
+# `apple` extra, and its `zeus.device.soc` package (which wraps this library)
+# landed on master after the last zeus release, so no published zeus-ml
+# contains it. Depend on the underlying IOReport extension directly.
+energy-apple = ["zeus-apple-silicon>=1.1.0; sys_platform == 'darwin'"]
+energy-all = [
+ "pynvml>=12.0",
+ "amdsmi>=6.1",
+ "zeus-apple-silicon>=1.1.0; sys_platform == 'darwin'",
+]
orchestrator-training = ["torch>=2.0", "transformers>=4.40"]
learning-dspy = ["dspy>=2.6"]
learning-gepa = ["gepa>=0.1"]
diff --git a/scripts/setup-energy-monitor.sh b/scripts/setup-energy-monitor.sh
new file mode 100755
index 00000000..4439ad03
--- /dev/null
+++ b/scripts/setup-energy-monitor.sh
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+# Install the energy-measurement backend for this machine.
+#
+# Referenced by `jarvis bench --setup-energy` (see cli/bench_cmd.py), which
+# built a path to this script and offered to run it while the file did not
+# exist -- the offer silently did nothing.
+#
+# Picks the right extra by platform:
+# Apple Silicon -> energy-apple (zeus-apple-silicon, IOReport, no root)
+# Linux + NVIDIA -> gpu-metrics (pynvml)
+# Linux + AMD -> energy-amd (amdsmi)
+# Linux, neither -> nothing to install; RAPL is read from sysfs
+set -euo pipefail
+
+cd "$(dirname "$0")/.."
+
+if ! command -v uv >/dev/null 2>&1; then
+ echo "error: uv is required. See https://docs.astral.sh/uv/" >&2
+ exit 1
+fi
+
+os="$(uname -s)"
+arch="$(uname -m)"
+
+case "$os" in
+Darwin)
+ if [ "$arch" != "arm64" ]; then
+ echo "Intel Macs expose no energy counters OpenJarvis can read." >&2
+ exit 1
+ fi
+ extra="energy-apple"
+ ;;
+Linux)
+ if command -v nvidia-smi >/dev/null 2>&1; then
+ extra="gpu-metrics"
+ elif command -v rocm-smi >/dev/null 2>&1; then
+ extra="energy-amd"
+ elif [ -d /sys/class/powercap/intel-rapl ]; then
+ echo "RAPL is readable from sysfs; no extra package needed."
+ echo "If readings come back empty, check permissions on"
+ echo " /sys/class/powercap/intel-rapl/*/energy_uj"
+ exit 0
+ else
+ echo "No supported energy backend detected on this host." >&2
+ exit 1
+ fi
+ ;;
+*)
+ echo "Energy measurement is not supported on $os." >&2
+ exit 1
+ ;;
+esac
+
+echo "Installing openjarvis[$extra] ..."
+# --inexact: add this extra without pruning extras the user already has.
+# A plain `uv sync --extra` would uninstall e.g. [dev] and [server].
+uv sync --inexact --extra "$extra"
+
+echo
+echo "Verifying ..."
+uv run python - <<'PY'
+from openjarvis.telemetry.energy_monitor import create_energy_monitor
+
+monitor = create_energy_monitor()
+if monitor is None:
+ raise SystemExit(
+ "Energy monitor still unavailable. Run `jarvis doctor` for details."
+ )
+print(f"OK: {monitor.vendor().value} via {monitor.energy_method()}")
+monitor.close()
+PY
diff --git a/src/openjarvis/bench/_stubs.py b/src/openjarvis/bench/_stubs.py
index 0da2ed96..e944b459 100644
--- a/src/openjarvis/bench/_stubs.py
+++ b/src/openjarvis/bench/_stubs.py
@@ -29,6 +29,46 @@ class BenchmarkResult:
energy_method: str = ""
+def engine_info(engine: Any) -> Optional[Dict[str, Any]]:
+ """Run metadata from an engine that can describe itself, or ``None``.
+
+ Duck-typed the way IPW's runner writes ``client_info`` into
+ ``summary.json``: ``describe`` is not on the ``InferenceEngine`` ABC, so
+ engines opt in by defining it. The AFM engine uses this to record its SDK
+ version, context size and host chip, none of which is recoverable from the
+ benchmark results alone.
+
+ The return value is validated rather than trusted. It lands in a JSONL
+ file, so anything that is not a plain dict of JSON-safe scalars is
+ dropped or stringified -- an engine returning something exotic must not be
+ able to fail the whole benchmark write.
+ """
+ describe = getattr(engine, "describe", None)
+ if not callable(describe):
+ return None
+ try:
+ info = describe()
+ except Exception:
+ return None
+ if not isinstance(info, dict):
+ return None
+
+ safe: Dict[str, Any] = {}
+ for key, value in info.items():
+ if not isinstance(key, str):
+ continue
+ if value is None or isinstance(value, (bool, int, float, str)):
+ safe[key] = value
+ else:
+ try:
+ json.dumps(value)
+ except (TypeError, ValueError):
+ safe[key] = repr(value)
+ else:
+ safe[key] = value
+ return safe or None
+
+
class BaseBenchmark(ABC):
"""Base class for all benchmark implementations.
diff --git a/src/openjarvis/bench/energy.py b/src/openjarvis/bench/energy.py
index abc35e1f..3ede389c 100644
--- a/src/openjarvis/bench/energy.py
+++ b/src/openjarvis/bench/energy.py
@@ -7,7 +7,7 @@ import time
from typing import Any, List, Optional
from openjarvis.bench._stats import compute_stats
-from openjarvis.bench._stubs import BaseBenchmark, BenchmarkResult
+from openjarvis.bench._stubs import BaseBenchmark, BenchmarkResult, engine_info
from openjarvis.core.registry import BenchmarkRegistry
from openjarvis.core.types import Message, Role
from openjarvis.engine._stubs import InferenceEngine
@@ -59,6 +59,15 @@ class EnergyBenchmark(BaseBenchmark):
per_tokens: List[float] = []
errors = 0
energy_method = ""
+ # Per-rail totals, so an ANE-resident backend's energy is attributable
+ # rather than just folded into one number.
+ rail_totals: dict[str, float] = {
+ "cpu": 0.0,
+ "gpu": 0.0,
+ "dram": 0.0,
+ "ane": 0.0,
+ }
+ energy_basis = ""
if energy_monitor is not None:
from openjarvis.telemetry.steady_state import SteadyStateDetector
@@ -80,6 +89,12 @@ class EnergyBenchmark(BaseBenchmark):
tps = tokens / elapsed if elapsed > 0 else 0.0
ept = energy_j / tokens if tokens > 0 else 0.0
+ rail_totals["cpu"] += sample.cpu_energy_joules
+ rail_totals["gpu"] += sample.gpu_energy_joules
+ rail_totals["dram"] += sample.dram_energy_joules
+ rail_totals["ane"] += sample.ane_energy_joules
+ energy_basis = sample.basis or energy_basis
+
per_energy_j.append(energy_j)
per_power_w.append(power_w)
per_tps.append(tps)
@@ -127,10 +142,17 @@ class EnergyBenchmark(BaseBenchmark):
metrics["total_time_seconds"] = total_time
metadata: dict[str, Any] = {}
- if engine.engine_id == "apple_fm":
- metadata["token_estimation"] = (
- "~4 chars/token (Apple FM SDK does not expose counts)"
- )
+ info = engine_info(engine)
+ if info:
+ metadata["engine_info"] = info
+ if energy_basis:
+ # Which rail set total_energy_joules and energy_per_token came
+ # from. Figures are only comparable across runs sharing a basis.
+ metadata["energy_basis"] = energy_basis
+ if any(v > 0 for v in rail_totals.values()):
+ metadata["energy_joules_by_rail"] = {
+ rail: round(v, 6) for rail, v in rail_totals.items() if v > 0
+ }
return BenchmarkResult(
benchmark_name=self.name,
diff --git a/src/openjarvis/bench/throughput.py b/src/openjarvis/bench/throughput.py
index 6dc42db8..bc92a20b 100644
--- a/src/openjarvis/bench/throughput.py
+++ b/src/openjarvis/bench/throughput.py
@@ -7,7 +7,7 @@ import time
from typing import Any, List
from openjarvis.bench._stats import compute_stats
-from openjarvis.bench._stubs import BaseBenchmark, BenchmarkResult
+from openjarvis.bench._stubs import BaseBenchmark, BenchmarkResult, engine_info
from openjarvis.core.registry import BenchmarkRegistry
from openjarvis.core.types import Message, Role
from openjarvis.engine._stubs import InferenceEngine
@@ -76,10 +76,9 @@ class ThroughputBenchmark(BaseBenchmark):
metrics["total_time_seconds"] = total_time
metadata: dict[str, Any] = {}
- if engine.engine_id == "apple_fm":
- metadata["token_estimation"] = (
- "~4 chars/token (Apple FM SDK does not expose counts)"
- )
+ info = engine_info(engine)
+ if info:
+ metadata["engine_info"] = info
return BenchmarkResult(
benchmark_name=self.name,
diff --git a/src/openjarvis/cli/ask.py b/src/openjarvis/cli/ask.py
index 14052cd6..7c06e71b 100644
--- a/src/openjarvis/cli/ask.py
+++ b/src/openjarvis/cli/ask.py
@@ -34,6 +34,24 @@ from openjarvis.telemetry.store import TelemetryStore
logger = logging.getLogger(__name__)
+# Engines that run inference on this machine. Used by the image privacy guard
+# below: a screenshot is sensitive and OpenJarvis is local-first, so sending
+# one to a non-local engine warrants a warning first. Module-level so it can be
+# asserted against directly -- an engine missing from here produces a false
+# "your image is leaving this machine" warning.
+LOCAL_ENGINES = {
+ "afm",
+ "apple_fm",
+ "exo",
+ "gemma_cpp",
+ "llamacpp",
+ "nexa",
+ "ollama",
+ "sglang",
+ "uzu",
+ "vllm",
+}
+
def _run_research(
*,
@@ -833,6 +851,7 @@ def ask(
energy_monitor = create_energy_monitor(
prefer_vendor=config.telemetry.energy_vendor or None,
+ allow_estimates=config.telemetry.allow_energy_estimates,
)
except Exception as exc:
logger.debug("Failed to create energy monitor: %s", exc)
@@ -943,21 +962,7 @@ def ask(
return
# Direct-to-engine mode (no agent)
- # Privacy guard: a screenshot/image is sensitive, and OpenJarvis is
- # local-first. If the active engine isn't local, warn before the image
- # leaves the machine rather than silently uploading it to a third party.
- _LOCAL_ENGINES = {
- "ollama",
- "llamacpp",
- "vllm",
- "sglang",
- "exo",
- "nexa",
- "uzu",
- "apple_fm",
- "gemma_cpp",
- }
- if image_b64 and engine_name not in _LOCAL_ENGINES:
+ if image_b64 and engine_name not in LOCAL_ENGINES:
console.print(
f"[yellow]Privacy warning:[/yellow] sending {len(image_b64)} "
f"image(s) to a non-local engine ('{engine_name}'). The image will "
diff --git a/src/openjarvis/cli/bench_cmd.py b/src/openjarvis/cli/bench_cmd.py
index 24fd8ccb..76292310 100644
--- a/src/openjarvis/cli/bench_cmd.py
+++ b/src/openjarvis/cli/bench_cmd.py
@@ -260,6 +260,7 @@ def run(
energy_monitor = create_energy_monitor(
prefer_vendor=config.telemetry.energy_vendor or None,
+ allow_estimates=config.telemetry.allow_energy_estimates,
)
except Exception as exc:
logger.debug("Energy monitor init skipped: %s", exc)
@@ -298,6 +299,7 @@ def run(
energy_monitor = create_energy_monitor(
prefer_vendor=config.telemetry.energy_vendor or None,
+ allow_estimates=config.telemetry.allow_energy_estimates,
)
if energy_monitor is not None:
console.print("[green]Energy monitor installed.[/green]")
diff --git a/src/openjarvis/cli/doctor_cmd.py b/src/openjarvis/cli/doctor_cmd.py
index 2c40ee61..c3fc1c90 100644
--- a/src/openjarvis/cli/doctor_cmd.py
+++ b/src/openjarvis/cli/doctor_cmd.py
@@ -206,7 +206,12 @@ def _check_optional_deps() -> List[CheckResult]:
("pynvml", "openjarvis[gpu-metrics]", "NVIDIA energy monitoring"),
("amdsmi", "openjarvis[energy-amd]", "AMD energy monitoring"),
("colbert", "openjarvis[memory-colbert]", "ColBERT memory backend"),
- ("zeus", "openjarvis[energy-apple]", "Apple Silicon energy monitoring"),
+ (
+ "zeus_apple_silicon",
+ "openjarvis[energy-apple]",
+ "Apple Silicon energy monitoring",
+ ),
+ ("apple_fm_sdk", "openjarvis[afm]", "Apple Foundation Models (AFM 3)"),
]
for pkg, install_hint, description in optional_packages:
try:
diff --git a/src/openjarvis/cli/host_cmd.py b/src/openjarvis/cli/host_cmd.py
index 8013dfb4..e765938b 100644
--- a/src/openjarvis/cli/host_cmd.py
+++ b/src/openjarvis/cli/host_cmd.py
@@ -72,6 +72,15 @@ _BACKENDS = {
"default_port": 52415,
"binary": "exo",
},
+ "apple_fm": {
+ "display": "Apple Foundation Models (shim)",
+ "package": "apple-fm-sdk",
+ "import_check": "apple_fm_sdk",
+ "pip_spec": "apple-fm-sdk>=0.2.1",
+ "uv_extra": "afm",
+ "platform": "darwin",
+ "default_port": 8079,
+ },
"uzu": {
"display": "Uzu",
"package": "uzu",
@@ -280,6 +289,22 @@ def _build_serve_command(backend: str, model: str, port: int) -> list[str]:
if backend == "uzu":
return ["uzu", "serve", "--port", str(port)]
+ if backend == "apple_fm":
+ # There is one on-device model, so `model` is a label and is not
+ # passed through -- the shim serves whatever the Foundation Models
+ # dynamic profile selects. For in-process use prefer the `afm`
+ # engine, which needs no server at all.
+ return [
+ sys.executable,
+ "-m",
+ "uvicorn",
+ "openjarvis.engine.apple_fm_shim:app",
+ "--host",
+ "127.0.0.1",
+ "--port",
+ str(port),
+ ]
+
raise ValueError(f"Unknown backend: {backend}")
diff --git a/src/openjarvis/core/config.py b/src/openjarvis/core/config.py
index 701c0e0f..6159a25a 100644
--- a/src/openjarvis/core/config.py
+++ b/src/openjarvis/core/config.py
@@ -447,6 +447,26 @@ class GemmaCppEngineConfig:
num_threads: int = 0
+@dataclass(slots=True)
+class AfmEngineConfig:
+ """Per-engine config for in-process Apple Foundation Models.
+
+ No ``host``: this engine runs the SDK in-process rather than talking to a
+ server. (The separate ``apple_fm`` engine does have a host — it speaks to
+ the FastAPI shim.)
+ """
+
+ # Standing instructions applied to every session, on top of any system
+ # message in the request.
+ instructions: str = ""
+ # "general" or "content_tagging".
+ use_case: str = "general"
+ # "default" or "permissive_content_transformations".
+ guardrails: str = "default"
+ # "greedy" (reproducible, the default here) or "random" (the SDK default).
+ sampling: str = "greedy"
+
+
@dataclass(slots=True)
class LemonadeEngineConfig:
"""Per-engine config for Lemonade."""
@@ -469,6 +489,7 @@ class EngineConfig:
nexa: NexaEngineConfig = field(default_factory=NexaEngineConfig)
uzu: UzuEngineConfig = field(default_factory=UzuEngineConfig)
apple_fm: AppleFmEngineConfig = field(default_factory=AppleFmEngineConfig)
+ afm: AfmEngineConfig = field(default_factory=AfmEngineConfig)
gemma_cpp: GemmaCppEngineConfig = field(default_factory=GemmaCppEngineConfig)
lemonade: LemonadeEngineConfig = field(default_factory=LemonadeEngineConfig)
@@ -1165,6 +1186,12 @@ class TelemetryConfig:
gpu_metrics: bool = False
gpu_poll_interval_ms: int = 50
energy_vendor: str = "" # auto-detect or force "nvidia"/"amd"/"apple"/"cpu_rapl"
+ # Permit a modelled energy estimate when no hardware counters are
+ # readable. Off by default: an estimate written to the telemetry DB is
+ # indistinguishable from a measurement at query time except via
+ # `energy_method`, and on Apple Silicon the estimate is a function of
+ # wall-clock alone, so it cannot tell an idle window from a busy one.
+ allow_energy_estimates: bool = False
warmup_samples: int = 0
steady_state_window: int = 5
steady_state_threshold: float = 0.05
diff --git a/src/openjarvis/core/types.py b/src/openjarvis/core/types.py
index add45385..9b93437f 100644
--- a/src/openjarvis/core/types.py
+++ b/src/openjarvis/core/types.py
@@ -185,6 +185,18 @@ class TelemetryRecord:
cpu_energy_joules: float = 0.0
gpu_energy_joules: float = 0.0
dram_energy_joules: float = 0.0
+ # Apple Neural Engine. Populated on Apple Silicon only, and the reason the
+ # rail exists here at all: Apple Foundation Models runs predominantly on
+ # the ANE, so a record carrying only CPU/GPU/DRAM would make it look
+ # nearly free.
+ ane_energy_joules: float = 0.0
+ # Every rail this record's monitor measured, summed. On Apple Silicon each
+ # rail above is a partial view of one package, so the derived per-token
+ # and per-watt figures come from this rather than from GPU alone.
+ soc_energy_joules: float = 0.0
+ # Which rail set the derived figures were computed from: "soc" on Apple
+ # Silicon, "gpu" elsewhere. Empty on records written before this existed.
+ energy_basis: str = ""
tokens_per_joule: float = 0.0
# Version tag for the token-counting methodology used when this record
# was produced. `None` (= legacy) means the record predates per-record
diff --git a/src/openjarvis/engine/__init__.py b/src/openjarvis/engine/__init__.py
index ef88acab..726318ec 100644
--- a/src/openjarvis/engine/__init__.py
+++ b/src/openjarvis/engine/__init__.py
@@ -24,10 +24,14 @@ logger = logging.getLogger(__name__)
def _register_optional_engines() -> None:
"""Register available optional engines and retain import diagnostics."""
- for optional in ("cloud", "litellm", "gemma_cpp"):
+ # OSError is caught alongside ImportError for apple_fm: its Swift bindings
+ # load at import time and raise OSError, not ImportError, on a host whose
+ # macOS or Xcode is too old. Letting that escape would take down every
+ # other engine too.
+ for optional in ("cloud", "litellm", "gemma_cpp", "apple_fm"):
try:
importlib.import_module(f".{optional}", __name__)
- except ImportError as exc:
+ except (ImportError, OSError) as exc:
logger.debug("Optional engine %r unavailable: %s", optional, exc)
diff --git a/src/openjarvis/engine/_apple_fm_support.py b/src/openjarvis/engine/_apple_fm_support.py
new file mode 100644
index 00000000..3a32b92c
--- /dev/null
+++ b/src/openjarvis/engine/_apple_fm_support.py
@@ -0,0 +1,151 @@
+"""Pure helpers for the Apple Foundation Models engine and shim.
+
+Deliberately free of any ``apple_fm_sdk`` import so this logic stays testable
+on CI runners that cannot install the SDK — it builds Swift bindings and needs
+a full Xcode on an Apple Silicon Mac, and OpenJarvis has no macOS runner.
+
+Ported from IPW's ``ipw/clients/_afm_support.py``.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, List, Mapping, Sequence, Tuple
+
+# Model labels accepted by the engine. The SDK exposes no variant selector:
+# the Foundation Models framework's dynamic profile picks AFM 3 Core (dense
+# ~3B) or AFM 3 Core Advanced (20B sparse MoE, 1-4B active per request) from
+# the host device's capabilities. These labels therefore only tag a run -- they
+# do not steer which model executes.
+MODEL_LABELS: Tuple[str, ...] = ("afm-3", "afm-3-core", "afm-3-core-advanced")
+
+# Sampling modes exposed by fm.SamplingMode as constructors. ``top`` and
+# ``probability_threshold`` are fields of the random mode, not modes.
+SAMPLING_MODES: Tuple[str, ...] = ("greedy", "random")
+
+_RANDOM_SAMPLING_FIELDS: Tuple[str, ...] = ("top", "probability_threshold", "seed")
+
+
+def validate_model_label(model: str) -> str:
+ """Normalize and check a model label.
+
+ Raises ``ValueError`` naming the accepted labels: a typo would otherwise
+ silently mislabel a run's telemetry and FLOPs lookup.
+ """
+ label = (model or "").strip().lower()
+ if label not in MODEL_LABELS:
+ raise ValueError(
+ f"Unknown AFM model label {model!r}. Expected one of: "
+ f"{', '.join(MODEL_LABELS)}. Note that these are run labels only -- "
+ "the on-device variant is chosen by the framework's dynamic "
+ "profile, not by this name."
+ )
+ return label
+
+
+def coerce_scalar(value: Any) -> Any:
+ """Coerce a config string into bool/int/float when it looks like one."""
+ if not isinstance(value, str):
+ return value
+ text = value.strip()
+ lowered = text.lower()
+ if lowered in ("true", "false"):
+ return lowered == "true"
+ try:
+ if "." in text or "e" in lowered:
+ return float(text)
+ return int(text)
+ except ValueError:
+ return text
+
+
+class SnapshotAccumulator:
+ """Turn ``session.stream_response``'s cumulative snapshots into deltas.
+
+ The SDK yields the *entire* text generated so far on each iteration, not
+ the newly added piece. Appending each chunk -- the pattern every other
+ streaming backend uses -- would duplicate text quadratically.
+
+ ``add`` returns the newly arrived text, or ``""`` when a snapshot repeats.
+ Guided generation can revise a snapshot rather than extend it, so a
+ snapshot that does not continue the previous one is treated as a
+ replacement and reported whole.
+ """
+
+ __slots__ = ("text",)
+
+ def __init__(self) -> None:
+ self.text: str = ""
+
+ def add(self, snapshot: Any) -> str:
+ if snapshot is None:
+ return ""
+ if not isinstance(snapshot, str):
+ # Defensive: tolerate a chunk object exposing text via an attribute.
+ snapshot = getattr(snapshot, "content", None) or str(snapshot)
+ if snapshot == self.text:
+ return ""
+ if snapshot.startswith(self.text):
+ delta = snapshot[len(self.text) :]
+ else:
+ delta = snapshot
+ self.text = snapshot
+ return delta
+
+
+def snapshot_deltas(snapshots: Sequence[Any]) -> List[str]:
+ """Eager form of :class:`SnapshotAccumulator`, for tests and offline use."""
+ accumulator = SnapshotAccumulator()
+ return [delta for delta in (accumulator.add(s) for s in snapshots) if delta]
+
+
+def build_options_kwargs(params: Mapping[str, Any]) -> Dict[str, Any]:
+ """Map OpenJarvis params onto ``fm.GenerationOptions`` fields.
+
+ Only non-``None`` values are returned so the SDK's own defaults apply for
+ anything the caller did not set. ``max_tokens`` is OpenJarvis's
+ cross-backend name for what the SDK calls ``maximum_response_tokens``.
+ """
+ options: Dict[str, Any] = {}
+
+ max_tokens = params.get("max_tokens")
+ if max_tokens is not None:
+ options["maximum_response_tokens"] = int(coerce_scalar(max_tokens))
+
+ temperature = params.get("temperature")
+ if temperature is not None:
+ options["temperature"] = float(coerce_scalar(temperature))
+
+ return options
+
+
+def parse_sampling_spec(params: Mapping[str, Any]) -> Tuple[str, Dict[str, Any]]:
+ """Resolve the sampling mode name and its keyword arguments.
+
+ Defaults to ``greedy`` so repeated runs are reproducible; the SDK itself
+ defaults to random sampling.
+ """
+ mode = str(params.get("sampling") or "greedy").strip().lower()
+ if mode not in SAMPLING_MODES:
+ raise ValueError(
+ f"Unknown AFM sampling mode {mode!r}. Expected one of: "
+ f"{', '.join(SAMPLING_MODES)}."
+ )
+
+ kwargs: Dict[str, Any] = {}
+ if mode == "random":
+ for field in _RANDOM_SAMPLING_FIELDS:
+ if params.get(field) is not None:
+ kwargs[field] = coerce_scalar(params[field])
+ return mode, kwargs
+
+
+__all__ = [
+ "MODEL_LABELS",
+ "SAMPLING_MODES",
+ "SnapshotAccumulator",
+ "build_options_kwargs",
+ "coerce_scalar",
+ "parse_sampling_spec",
+ "snapshot_deltas",
+ "validate_model_label",
+]
diff --git a/src/openjarvis/engine/_async_loop.py b/src/openjarvis/engine/_async_loop.py
new file mode 100644
index 00000000..6b743fe3
--- /dev/null
+++ b/src/openjarvis/engine/_async_loop.py
@@ -0,0 +1,60 @@
+"""Run an asyncio event loop on a background thread.
+
+``InferenceEngine.generate`` is synchronous, but some backends expose
+async-only APIs — Apple's Foundation Models SDK is one. This bridges the two
+without an ``asyncio.run`` per call, which would tear down and rebuild a loop
+for every request and break SDKs holding loop-bound state.
+
+Ported from IPW's ``ipw/clients/_async_loop.py``.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import threading
+from typing import Any
+
+
+class AsyncLoopRunner:
+ """Own an event loop on a daemon thread and run coroutines against it."""
+
+ def __init__(self, name: str = "openjarvis-async") -> None:
+ self._loop = asyncio.new_event_loop()
+ self._thread = threading.Thread(target=self._run_loop, name=name, daemon=True)
+ self._thread.start()
+
+ def run(self, coro: Any) -> Any:
+ """Submit *coro* to the background loop and block for its result."""
+ future = asyncio.run_coroutine_threadsafe(coro, self._loop)
+ return future.result()
+
+ def shutdown(self) -> None:
+ if self._loop.is_closed():
+ return
+
+ async def _drain() -> None:
+ current = asyncio.current_task()
+ tasks = [
+ task
+ for task in asyncio.all_tasks()
+ if task is not current and not task.done()
+ ]
+ for task in tasks:
+ task.cancel()
+ if tasks:
+ await asyncio.gather(*tasks, return_exceptions=True)
+
+ try:
+ asyncio.run_coroutine_threadsafe(_drain(), self._loop).result(timeout=5.0)
+ except Exception: # pragma: no cover - shutdown is best-effort
+ pass
+ self._loop.call_soon_threadsafe(self._loop.stop)
+ self._thread.join(timeout=2.0)
+ self._loop.close()
+
+ def _run_loop(self) -> None:
+ asyncio.set_event_loop(self._loop)
+ self._loop.run_forever()
+
+
+__all__ = ["AsyncLoopRunner"]
diff --git a/src/openjarvis/engine/_discovery.py b/src/openjarvis/engine/_discovery.py
index d3a9acc9..047f7f1e 100644
--- a/src/openjarvis/engine/_discovery.py
+++ b/src/openjarvis/engine/_discovery.py
@@ -28,6 +28,8 @@ _HOST_MAP: Dict[str, str | None] = {
"cloud": None,
"litellm": None,
"gemma_cpp": None,
+ # In-process: drives the Apple FM SDK directly, so there is no host.
+ "afm": None,
}
@@ -51,6 +53,16 @@ def _make_engine(key: str, config: JarvisConfig) -> InferenceEngine:
num_threads=cfg.num_threads,
)
+ # afm: in-process engine, configured by behaviour rather than a host
+ if key == "afm":
+ cfg = config.engine.afm
+ return cls(
+ instructions=cfg.instructions,
+ use_case=cfg.use_case,
+ guardrails=cfg.guardrails,
+ sampling=cfg.sampling,
+ )
+
host_attr = _HOST_MAP.get(key)
if host_attr is not None:
host = getattr(config.engine, host_attr, None)
diff --git a/src/openjarvis/engine/apple_fm.py b/src/openjarvis/engine/apple_fm.py
new file mode 100644
index 00000000..7bc67a9e
--- /dev/null
+++ b/src/openjarvis/engine/apple_fm.py
@@ -0,0 +1,580 @@
+"""In-process Apple Foundation Models (AFM 3) engine.
+
+Drives Apple's ``apple_fm_sdk`` against the on-device model behind Apple
+Intelligence, with no HTTP hop and no second process — so the energy measured
+around a request is the energy the request actually cost. (The
+``apple_fm_shim`` + ``apple_fm`` engine pair remains available for pointing
+external OpenAI-compatible clients at the same model.)
+
+Requires an Apple Silicon Mac on macOS 26+ with Apple Intelligence enabled,
+and a full Xcode — not just Command Line Tools — because the SDK compiles
+Swift bindings at install time::
+
+ DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \\
+ uv pip install -e '.[afm]'
+
+ jarvis ask --engine afm --model afm-3-core "..."
+
+Measurement caveats specific to this backend — read before comparing numbers
+against MLX or Ollama:
+
+* ``session.stream_response`` yields **cumulative text snapshots**, each
+ batching roughly 8-10 tokens. ``ttft`` is therefore time-to-first-*chunk*
+ (an upper bound on true TTFT — around 450ms on an M1 Pro versus a few tens
+ of ms of real first-token latency), and derived inter-token latencies are
+ inter-*chunk* latencies. Neither is comparable to a backend that streams one
+ token at a time. Token counts, throughput and per-token energy are
+ unaffected.
+* The SDK exposes no way to select AFM 3 Core (dense ~3B) versus AFM 3 Core
+ Advanced (20B sparse MoE, 1-4B active per request); the framework's dynamic
+ profile picks one from the host device's capabilities. ``model`` is a run
+ label only — :meth:`AppleFMEngine.describe` records the SDK version, context
+ size and host chip so a run can be attributed after the fact.
+* There is no Private Cloud Compute path in the Python SDK, which suits
+ OpenJarvis: off-device inference would make the on-device energy
+ measurement meaningless.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+import platform
+import subprocess
+import time
+from collections.abc import AsyncIterator, Sequence
+from importlib.metadata import PackageNotFoundError
+from importlib.metadata import version as _dist_version
+from typing import Any, Dict, List, Optional
+
+# Top-level import on purpose: it gates registration. ``engine/__init__.py``
+# imports this module inside a try/except, so on a host without the SDK the
+# engine simply does not register.
+import apple_fm_sdk as fm
+
+from openjarvis.core.registry import EngineRegistry
+from openjarvis.core.types import Message, Role
+from openjarvis.engine._apple_fm_support import (
+ MODEL_LABELS,
+ SnapshotAccumulator,
+ build_options_kwargs,
+ parse_sampling_spec,
+ validate_model_label,
+)
+from openjarvis.engine._async_loop import AsyncLoopRunner
+from openjarvis.engine._base import (
+ EngineConnectionError,
+ EngineContextLengthError,
+ InferenceEngine,
+)
+from openjarvis.engine._stubs import StreamChunk
+
+logger = logging.getLogger(__name__)
+
+DEFAULT_MODEL_LABEL = "afm-3"
+
+
+def _require(path: str) -> Any:
+ """Fetch a dotted attribute from the SDK, or fail as an ImportError.
+
+ An SDK that imports but lacks the surface this engine needs (anything
+ below 0.2.1, which introduced ``token_count`` and ``context_size``) is an
+ unusable install, not a broken engine. Raising ImportError lets
+ ``engine/__init__.py``'s guarded import skip registration instead of
+ letting an AttributeError take down every other engine.
+ """
+ obj: Any = fm
+ for part in path.split("."):
+ try:
+ obj = getattr(obj, part)
+ except AttributeError as exc:
+ raise ImportError(
+ f"apple_fm_sdk is missing {path!r}. The AFM engine needs "
+ "apple-fm-sdk>=0.2.1; upgrade with "
+ "`uv pip install -U 'apple-fm-sdk>=0.2.1'`."
+ ) from exc
+ return obj
+
+
+_USE_CASES = {
+ "general": _require("SystemLanguageModelUseCase.GENERAL"),
+ "content_tagging": _require("SystemLanguageModelUseCase.CONTENT_TAGGING"),
+}
+_GUARDRAILS = {
+ "default": _require("SystemLanguageModelGuardrails.DEFAULT"),
+ "permissive_content_transformations": _require(
+ "SystemLanguageModelGuardrails.PERMISSIVE_CONTENT_TRANSFORMATIONS"
+ ),
+}
+# token_count / context_size are the 0.2.1 additions the engine's whole token
+# accounting rests on -- check them at import so a stale SDK degrades to "not
+# registered" rather than to silently zeroed usage.
+_require("SystemLanguageModel.token_count")
+_require("SystemLanguageModel.context_size")
+
+
+def _resolve_error(name: str) -> Optional[type]:
+ """Look an SDK exception class up by name.
+
+ Resolved by name rather than imported so a future SDK release renaming one
+ of these degrades to a narrower catch instead of breaking import outright.
+ """
+ exc = getattr(fm, name, None)
+ if isinstance(exc, type) and issubclass(exc, BaseException):
+ return exc
+ return None
+
+
+def _error_tuple(*names: str) -> tuple[type[BaseException], ...]:
+ found = tuple(e for e in (_resolve_error(n) for n in names) if e is not None)
+ if not found: # pragma: no cover - guards a wholesale SDK rename
+ logger.warning(
+ "No known apple_fm_sdk error classes found among %s; AFM failures "
+ "will surface as generic errors.",
+ ", ".join(names),
+ )
+ return found
+
+
+# Overflow is routine at a 4096-token context, and callers already have a
+# dedicated "conversation too long" path for it.
+_CONTEXT_ERRORS = _error_tuple("ExceededContextWindowSizeError")
+# Refusals and guardrail trips are per-query outcomes, not engine faults: a
+# long eval run must not die because one prompt was declined.
+_REFUSAL_ERRORS = _error_tuple(
+ "GuardrailViolationError",
+ "RefusalError",
+ "UnsupportedLanguageOrLocaleError",
+)
+
+
+def _host_chip() -> str:
+ """Identify the Apple Silicon chip, e.g. "Apple M1 Pro".
+
+ ``platform.processor()`` returns a bare "arm" on every Apple Silicon Mac,
+ which is useless for the one job this has: the executing AFM variant is
+ not observable, so the host chip is how a run gets attributed later.
+ """
+ try:
+ result = subprocess.run(
+ ["sysctl", "-n", "machdep.cpu.brand_string"],
+ capture_output=True,
+ text=True,
+ timeout=5.0,
+ check=True,
+ )
+ except (OSError, subprocess.SubprocessError): # pragma: no cover - host dependent
+ return platform.processor() or platform.machine()
+ return result.stdout.strip() or platform.processor() or platform.machine()
+
+
+class AppleFMEngine(InferenceEngine):
+ """Apple Foundation Models, driven in-process via ``apple_fm_sdk``."""
+
+ engine_id = "afm"
+
+ def __init__(
+ self,
+ instructions: str = "",
+ use_case: str = "general",
+ guardrails: str = "default",
+ sampling: str = "greedy",
+ ) -> None:
+ self._instructions = str(instructions or "").strip()
+ self._use_case_name = str(use_case or "general").strip().lower()
+ self._guardrails_name = str(guardrails or "default").strip().lower()
+ self._sampling = str(sampling or "greedy").strip().lower()
+
+ if self._use_case_name not in _USE_CASES:
+ raise ValueError(
+ f"Unknown AFM use_case {use_case!r}. Expected one of: "
+ f"{', '.join(_USE_CASES)}."
+ )
+ if self._guardrails_name not in _GUARDRAILS:
+ raise ValueError(
+ f"Unknown AFM guardrails {guardrails!r}. Expected one of: "
+ f"{', '.join(_GUARDRAILS)}."
+ )
+ # Validate eagerly so a typo fails at construction rather than midway
+ # through a run.
+ parse_sampling_spec({"sampling": self._sampling})
+
+ self._model: Any = None
+ self._context_size: Optional[int] = None
+ self._prepared = False
+ self._loop: Optional[AsyncLoopRunner] = None
+ self._skipped: Dict[str, int] = {}
+
+ # ---- lifecycle -------------------------------------------------------
+
+ def _ensure_model(self) -> Any:
+ if self._model is None:
+ self._model = fm.SystemLanguageModel(
+ use_case=_USE_CASES[self._use_case_name],
+ guardrails=_GUARDRAILS[self._guardrails_name],
+ )
+ return self._model
+
+ def _ensure_loop(self) -> AsyncLoopRunner:
+ if self._loop is None:
+ self._loop = AsyncLoopRunner(name="openjarvis-afm")
+ return self._loop
+
+ def health(self) -> bool:
+ """Report on-device model availability.
+
+ Returns a bool for the ABC, but logs the SDK's reason first: a bare
+ False would hide whether the cause is Apple Intelligence being switched
+ off, an ineligible device, or assets still downloading.
+ """
+ try:
+ available, reason = self._ensure_model().is_available()
+ except Exception as exc: # pragma: no cover - depends on host state
+ logger.warning("Could not initialize Apple Foundation Models: %s", exc)
+ return False
+ if not available:
+ logger.warning(
+ "Apple Foundation Models unavailable: %s. Check that Apple "
+ "Intelligence is enabled in System Settings, that this device "
+ "is eligible, and that model assets have finished downloading.",
+ getattr(reason, "name", reason),
+ )
+ return bool(available)
+
+ def list_models(self) -> List[str]:
+ return list(MODEL_LABELS)
+
+ def can_serve(self, model: str) -> bool:
+ """Only the AFM labels — this engine cannot serve arbitrary model ids.
+
+ Unlike the other local engines, "is the model installed" is not a
+ separate concern here: there is exactly one on-device model, so
+ accepting any id would let engine selection route e.g. a Llama request
+ to AFM and silently answer with a different model.
+ """
+ return (model or "").strip().lower() in MODEL_LABELS
+
+ def prepare(self, model: str) -> None:
+ """Validate the label, then warm up outside the first energy window.
+
+ The first request after boot pays a large one-off cost for model asset
+ load and XPC setup (~10s versus ~450ms warm on an M1 Pro). Burning it
+ here keeps it out of the telemetry window of the first real request.
+ """
+ validate_model_label(model)
+ language_model = self._ensure_model()
+
+ available, reason = language_model.is_available()
+ if not available:
+ raise EngineConnectionError(
+ "Apple Foundation Models is unavailable "
+ f"({getattr(reason, 'name', reason)}). Enable Apple "
+ "Intelligence in System Settings and wait for model assets to "
+ "download."
+ )
+
+ self._context_size = int(language_model.context_size)
+ self._ensure_loop().run(self._warmup())
+ self._prepared = True
+
+ def close(self) -> None:
+ if self._skipped:
+ logger.warning(
+ "AFM declined %d requests: %s",
+ sum(self._skipped.values()),
+ ", ".join(f"{k}={v}" for k, v in sorted(self._skipped.items())),
+ )
+ if self._loop is not None:
+ self._loop.shutdown()
+ self._loop = None
+ self._model = None
+ self._prepared = False
+
+ def describe(self) -> Dict[str, Any]:
+ """Run metadata. The executing variant is not observable, so the host
+ chip and SDK version are the only way to attribute a run later."""
+ try:
+ sdk_version: Optional[str] = _dist_version("apple-fm-sdk")
+ except PackageNotFoundError: # pragma: no cover - installed by definition
+ sdk_version = None
+ return {
+ "engine_id": self.engine_id,
+ "afm_sdk_version": sdk_version,
+ "afm_context_size": self._context_size,
+ "afm_use_case": self._use_case_name,
+ "afm_guardrails": self._guardrails_name,
+ "afm_instructions": self._instructions or None,
+ "afm_sampling": self._sampling,
+ "host_chip": _host_chip(),
+ "host_os_version": platform.mac_ver()[0] or None,
+ "variant_selection": (
+ "device-chosen by the Foundation Models dynamic profile; "
+ "the model name is a label only"
+ ),
+ "declined_requests": dict(self._skipped) or None,
+ }
+
+ # ---- prompt / options ------------------------------------------------
+
+ def _split_messages(self, messages: Sequence[Message]) -> tuple[str, str]:
+ """Split *messages* into (instructions, prompt).
+
+ System messages become the session's ``instructions``, which is where
+ the SDK wants them — folding them into the prompt text (as the shim
+ historically did) both loses that distinction and inflates the
+ prompt-token count.
+ """
+ system_parts: List[str] = []
+ if self._instructions:
+ system_parts.append(self._instructions)
+ prompt_parts: List[str] = []
+ for m in messages:
+ if m.role == Role.SYSTEM:
+ system_parts.append(m.content)
+ elif m.role in (Role.USER, Role.ASSISTANT):
+ prompt_parts.append(m.content)
+ return "\n".join(p for p in system_parts if p), "\n".join(prompt_parts)
+
+ def _build_options(self, temperature: float, max_tokens: int) -> Any:
+ kwargs = build_options_kwargs(
+ {"temperature": temperature, "max_tokens": max_tokens}
+ )
+ mode, mode_kwargs = parse_sampling_spec({"sampling": self._sampling})
+ factory = getattr(fm.SamplingMode, mode, None)
+ if callable(factory):
+ kwargs["sampling"] = factory(**mode_kwargs)
+ return fm.GenerationOptions(**kwargs)
+
+ def _new_session(self, instructions: str) -> Any:
+ """A fresh session per request.
+
+ Sessions accumulate a transcript, so reusing one would grow the prompt
+ until it overflows the 4096-token context after a handful of requests,
+ and would make per-request energy depend on preceding requests.
+ """
+ return fm.LanguageModelSession(
+ instructions=instructions or None,
+ model=self._ensure_model(),
+ )
+
+ # ---- token accounting ------------------------------------------------
+
+ async def _count(self, *args: Any, **kwargs: Any) -> Optional[int]:
+ """``token_count`` that degrades to ``None`` rather than failing."""
+ try:
+ return int(await self._ensure_model().token_count(*args, **kwargs))
+ except Exception:
+ logger.debug("AFM token_count failed", exc_info=True)
+ return None
+
+ @staticmethod
+ def _usage(
+ prompt_tokens: Optional[int],
+ completion_tokens: Optional[int],
+ ) -> Dict[str, int]:
+ prompt = prompt_tokens or 0
+ completion = completion_tokens or 0
+ return {
+ "prompt_tokens": prompt,
+ "prompt_tokens_evaluated": prompt,
+ "completion_tokens": completion,
+ "total_tokens": prompt + completion,
+ }
+
+ # ---- error mapping ---------------------------------------------------
+
+ @staticmethod
+ def _sdk_detail(exc: BaseException) -> str:
+ """The SDK's own message, minus its noise.
+
+ The SDK formats messages as "<reason>: <detail>" and passes ``None``
+ as the detail when it has none, so the raw text ends in a dangling
+ ": None". Left alone that reads as a bug in the error path rather
+ than a plain context overflow, and it lands in every affected eval
+ row.
+ """
+ text = str(exc).strip()
+ while text.endswith(": None"):
+ text = text[: -len(": None")].strip()
+ return text
+
+ def _translate(self, exc: BaseException) -> BaseException:
+ if _CONTEXT_ERRORS and isinstance(exc, _CONTEXT_ERRORS):
+ size = self._context_size or "the model's"
+ detail = self._sdk_detail(exc)
+ message = (
+ f"Prompt exceeds the {size}-token context window for Apple "
+ "Foundation Models"
+ )
+ # Only append the SDK's text when it says something the sentence
+ # above does not already convey.
+ if detail and "context window size exceeded" not in detail.lower():
+ message = f"{message}: {detail}"
+ return EngineContextLengthError(message)
+ return exc
+
+ def _note_declined(self, exc: BaseException) -> None:
+ reason = type(exc).__name__
+ self._skipped[reason] = self._skipped.get(reason, 0) + 1
+ logger.warning("AFM declined a request (%s): %s", reason, exc)
+
+ # ---- inference -------------------------------------------------------
+
+ def generate(
+ self,
+ messages: Sequence[Message],
+ *,
+ model: str = DEFAULT_MODEL_LABEL,
+ temperature: float = 0.7,
+ max_tokens: int = 1024,
+ **kwargs: Any,
+ ) -> Dict[str, Any]:
+ if not self._prepared:
+ self.prepare(model)
+ return self._ensure_loop().run(
+ self._respond(messages, model, temperature, max_tokens)
+ )
+
+ async def _respond(
+ self,
+ messages: Sequence[Message],
+ model: str,
+ temperature: float,
+ max_tokens: int,
+ ) -> Dict[str, Any]:
+ instructions, prompt = self._split_messages(messages)
+ options = self._build_options(temperature, max_tokens)
+ session = self._new_session(instructions)
+
+ # Token accounting: `token_count` rejects a value and instructions
+ # together ("Provide either a value or instructions"), and counting
+ # instructions standalone does not match how the transcript encodes
+ # them. So both ends are measured against the transcript instead:
+ # prompt_tokens = <transcript before generating> + <prompt alone>
+ # completion_tokens = <transcript after generating> - prompt_tokens
+ pre_tokens = await self._count(session.transcript)
+ prompt_only = await self._count(prompt)
+
+ accumulator = SnapshotAccumulator()
+ start = time.time()
+ ttft = 0.0
+
+ try:
+ async for snapshot in session.stream_response(prompt, options=options):
+ if not accumulator.add(snapshot):
+ continue
+ if ttft == 0.0:
+ ttft = time.time() - start
+ except _REFUSAL_ERRORS as exc:
+ self._note_declined(exc)
+ return {
+ "content": "",
+ "usage": self._usage(prompt_only, 0),
+ "model": model,
+ "finish_reason": "content_filter",
+ "ttft": 0.0,
+ }
+ except Exception as exc:
+ raise self._translate(exc) from exc
+
+ post_tokens = await self._count(session.transcript)
+ prompt_tokens = (
+ pre_tokens + prompt_only
+ if pre_tokens is not None and prompt_only is not None
+ else prompt_only
+ )
+ completion_tokens = (
+ max(post_tokens - prompt_tokens, 0)
+ if post_tokens is not None and prompt_tokens is not None
+ else None
+ )
+
+ return {
+ "content": accumulator.text,
+ "usage": self._usage(prompt_tokens, completion_tokens),
+ "model": model,
+ "finish_reason": "stop",
+ "ttft": ttft,
+ }
+
+ async def stream(
+ self,
+ messages: Sequence[Message],
+ *,
+ model: str = DEFAULT_MODEL_LABEL,
+ temperature: float = 0.7,
+ max_tokens: int = 1024,
+ **kwargs: Any,
+ ) -> AsyncIterator[str]:
+ async for chunk in self.stream_full(
+ messages,
+ model=model,
+ temperature=temperature,
+ max_tokens=max_tokens,
+ **kwargs,
+ ):
+ if chunk.content:
+ yield chunk.content
+
+ async def stream_full(
+ self,
+ messages: Sequence[Message],
+ *,
+ model: str = DEFAULT_MODEL_LABEL,
+ temperature: float = 0.7,
+ max_tokens: int = 1024,
+ **kwargs: Any,
+ ) -> AsyncIterator[StreamChunk]:
+ if not self._prepared:
+ # prepare() blocks for the ~10s cold start; off-thread so it does
+ # not stall the caller's event loop.
+ await asyncio.to_thread(self.prepare, model)
+
+ instructions, prompt = self._split_messages(messages)
+ options = self._build_options(temperature, max_tokens)
+ session = self._new_session(instructions)
+
+ pre_tokens = await self._count(session.transcript)
+ prompt_only = await self._count(prompt)
+ accumulator = SnapshotAccumulator()
+
+ try:
+ async for snapshot in session.stream_response(prompt, options=options):
+ delta = accumulator.add(snapshot)
+ if delta:
+ yield StreamChunk(content=delta)
+ except _REFUSAL_ERRORS as exc:
+ self._note_declined(exc)
+ yield StreamChunk(
+ finish_reason="content_filter",
+ usage=self._usage(prompt_only, 0),
+ )
+ return
+ except Exception as exc:
+ raise self._translate(exc) from exc
+
+ post_tokens = await self._count(session.transcript)
+ prompt_tokens = (
+ pre_tokens + prompt_only
+ if pre_tokens is not None and prompt_only is not None
+ else prompt_only
+ )
+ completion_tokens = (
+ max(post_tokens - prompt_tokens, 0)
+ if post_tokens is not None and prompt_tokens is not None
+ else None
+ )
+ yield StreamChunk(
+ finish_reason="stop",
+ usage=self._usage(prompt_tokens, completion_tokens),
+ )
+
+ async def _warmup(self) -> None:
+ session = self._new_session(self._instructions)
+ options = fm.GenerationOptions(maximum_response_tokens=4)
+ async for _ in session.stream_response("warmup", options=options):
+ pass
+
+
+__all__ = ["AppleFMEngine"]
diff --git a/src/openjarvis/engine/apple_fm_shim.py b/src/openjarvis/engine/apple_fm_shim.py
index 9f2acee8..1b1bad43 100644
--- a/src/openjarvis/engine/apple_fm_shim.py
+++ b/src/openjarvis/engine/apple_fm_shim.py
@@ -1,14 +1,15 @@
"""Apple Foundation Models shim.
-Thin FastAPI server exposing Apple Intelligence's on-device foundation
-model as an OpenAI-compatible API. Only runs on macOS 26+ with Apple
-Intelligence enabled. Wraps Apple's `apple-fm-sdk`'s
-``LanguageModelSession`` as ``/v1/chat/completions`` and ``/v1/models``
-endpoints.
+Thin FastAPI server exposing Apple Intelligence's on-device foundation model
+as an OpenAI-compatible API, for pointing external OpenAI clients at AFM.
+Only runs on macOS 26+ with Apple Intelligence enabled. Wraps
+``apple-fm-sdk``'s ``LanguageModelSession`` as ``/v1/chat/completions`` and
+``/v1/models``.
-**Token counts:** The Apple FM SDK does not expose token counts. The
-shim returns 0 for all token counts. Throughput and energy benchmarks
-will reflect this limitation.
+OpenJarvis's own preferred path is the in-process ``afm`` engine
+(``openjarvis.engine.apple_fm``), which avoids an HTTP hop and a second
+process whose CPU energy would otherwise land inside the same measurement
+window.
Usage:
uvicorn openjarvis.engine.apple_fm_shim:app \
@@ -31,24 +32,30 @@ try:
import apple_fm_sdk # type: ignore[import-untyped]
except ImportError:
print(
- "apple_fm_shim: apple-fm-sdk is not available. The SDK is not on\n"
- "PyPI yet; clone https://github.com/apple/python-apple-fm-sdk and\n"
- "install from source:\n"
- " git clone https://github.com/apple/python-apple-fm-sdk\n"
- " uv pip install -e ./python-apple-fm-sdk\n"
- "Requires macOS 26+, Xcode 26+, and Apple Intelligence enabled.",
+ "apple_fm_shim: apple-fm-sdk is not available. Install it with:\n"
+ " DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \\\n"
+ " uv pip install 'apple-fm-sdk>=0.2.1'\n"
+ "The SDK compiles Swift bindings at install time, so a full Xcode is\n"
+ "required -- Command Line Tools alone fail. Requires macOS 26+ and\n"
+ "Apple Intelligence enabled.",
file=sys.stderr,
)
sys.exit(1)
import json
+import logging
import time
import uuid
+from typing import Any, Optional
from fastapi import FastAPI
from fastapi.responses import JSONResponse, StreamingResponse
from pydantic import BaseModel
+from openjarvis.engine._apple_fm_support import SnapshotAccumulator
+
+logger = logging.getLogger(__name__)
+
app = FastAPI(title="Apple FM Shim")
MODEL_ID = "apple-fm"
@@ -67,21 +74,28 @@ class ChatRequest(BaseModel):
stream: bool = False
-def _build_prompt(messages: list[ChatMessage]) -> str:
- parts: list[str] = []
- for m in messages:
- if m.role == "system":
- parts.append(f"[System] {m.content}")
- elif m.role in ("user", "assistant"):
- parts.append(m.content)
- return "\n".join(parts)
+def _system_language_model() -> Any:
+ return apple_fm_sdk.SystemLanguageModel()
+
+
+def _split_messages(messages: list[ChatMessage]) -> tuple[str, str]:
+ """Split into (instructions, prompt).
+
+ System messages become the session's ``instructions``, which is where the
+ SDK wants them. Earlier versions prefixed them onto the prompt as
+ ``[System] ...``, which both discards the distinction and inflates the
+ prompt-token count.
+ """
+ system_parts = [m.content for m in messages if m.role == "system"]
+ prompt_parts = [m.content for m in messages if m.role in ("user", "assistant")]
+ return "\n".join(system_parts), "\n".join(prompt_parts)
def _generation_options(req: ChatRequest) -> apple_fm_sdk.GenerationOptions:
"""Build the per-request GenerationOptions from a ChatRequest.
- Apple FM doesn't take ``max_tokens`` / ``temperature`` as positional
- args to ``respond`` / ``stream_response`` — they live on a
+ Apple FM doesn't take ``max_tokens`` / ``temperature`` as positional args
+ to ``respond`` / ``stream_response`` -- they live on a
``GenerationOptions`` object passed via the ``options`` kwarg.
"""
return apple_fm_sdk.GenerationOptions(
@@ -90,12 +104,40 @@ def _generation_options(req: ChatRequest) -> apple_fm_sdk.GenerationOptions:
)
+async def _count(model: Any, value: Any) -> Optional[int]:
+ """``token_count`` that degrades to ``None`` rather than failing a request."""
+ try:
+ return int(await model.token_count(value))
+ except Exception:
+ logger.debug("apple_fm_shim: token_count failed", exc_info=True)
+ return None
+
+
+def _usage(prompt_tokens: Optional[int], completion_tokens: Optional[int]) -> dict:
+ prompt = prompt_tokens or 0
+ completion = completion_tokens or 0
+ return {
+ "prompt_tokens": prompt,
+ "completion_tokens": completion,
+ "total_tokens": prompt + completion,
+ }
+
+
+def _completion_tokens(
+ post: Optional[int],
+ prompt_tokens: Optional[int],
+) -> Optional[int]:
+ if post is None or prompt_tokens is None:
+ return None
+ return max(post - prompt_tokens, 0)
+
+
@app.get("/health")
def health() -> JSONResponse:
# SystemLanguageModel.is_available() is an *instance* method that
# returns (bool, reason | None). Unpack so we can both gate the
# response code and surface the reason for unavailability.
- available, reason = apple_fm_sdk.SystemLanguageModel().is_available()
+ available, reason = _system_language_model().is_available()
if available:
return JSONResponse({"status": "ok"}, status_code=200)
return JSONResponse(
@@ -106,43 +148,57 @@ def health() -> JSONResponse:
@app.get("/v1/models")
def list_models() -> JSONResponse:
- return JSONResponse(
- {
- "object": "list",
- "data": [
- {"id": MODEL_ID, "object": "model", "owned_by": "apple"},
- ],
- }
- )
+ entry: dict[str, Any] = {
+ "id": MODEL_ID,
+ "object": "model",
+ "owned_by": "apple",
+ }
+ try:
+ entry["context_length"] = int(_system_language_model().context_size)
+ except Exception:
+ logger.debug("apple_fm_shim: context_size unavailable", exc_info=True)
+ return JSONResponse({"object": "list", "data": [entry]})
@app.post("/v1/chat/completions", response_model=None)
async def chat_completions(
req: ChatRequest,
) -> JSONResponse | StreamingResponse:
- prompt = _build_prompt(req.messages)
- session = apple_fm_sdk.LanguageModelSession()
+ instructions, prompt = _split_messages(req.messages)
+ model = _system_language_model()
+ session = apple_fm_sdk.LanguageModelSession(
+ instructions=instructions or None,
+ model=model,
+ )
options = _generation_options(req)
+ # Token accounting. `token_count` rejects a value and instructions
+ # together, and counting instructions standalone does not match how the
+ # transcript encodes them, so both ends are measured against the
+ # transcript instead:
+ # prompt_tokens = <transcript before> + <prompt alone>
+ # completion_tokens = <transcript after> - prompt_tokens
+ pre_tokens = await _count(model, session.transcript)
+ prompt_only = await _count(model, prompt)
+ prompt_tokens = (
+ pre_tokens + prompt_only
+ if pre_tokens is not None and prompt_only is not None
+ else prompt_only
+ )
+
if req.stream:
async def generate():
cid = f"chatcmpl-{uuid.uuid4().hex[:12]}"
# Apple FM yields cumulative snapshots, OpenAI clients expect
- # incremental deltas — diff against the last snapshot to convert
- # (see #378).
- sent = ""
+ # incremental deltas (see #378). The accumulator also handles
+ # guided generation revising a snapshot rather than extending it.
+ accumulator = SnapshotAccumulator()
async for snapshot in session.stream_response(
prompt,
options=options,
):
- if not snapshot.startswith(sent):
- # Snapshot diverged (model revised earlier text);
- # fall back to resending the full snapshot.
- delta = snapshot
- else:
- delta = snapshot[len(sent) :]
- sent = snapshot
+ delta = accumulator.add(snapshot)
if not delta:
continue
chunk = {
@@ -159,6 +215,8 @@ async def chat_completions(
],
}
yield f"data: {json.dumps(chunk)}\n\n"
+
+ post_tokens = await _count(model, session.transcript)
final = {
"id": cid,
"object": "chat.completion.chunk",
@@ -171,6 +229,12 @@ async def chat_completions(
"finish_reason": "stop",
}
],
+ # Streaming clients that track cost/throughput read usage off
+ # the final chunk; omitting it left them with nothing.
+ "usage": _usage(
+ prompt_tokens,
+ _completion_tokens(post_tokens, prompt_tokens),
+ ),
}
yield f"data: {json.dumps(final)}\n\n"
yield "data: [DONE]\n\n"
@@ -181,6 +245,7 @@ async def chat_completions(
)
text = await session.respond(prompt, options=options)
+ post_tokens = await _count(model, session.transcript)
cid = f"chatcmpl-{uuid.uuid4().hex[:12]}"
return JSONResponse(
{
@@ -195,10 +260,9 @@ async def chat_completions(
"finish_reason": "stop",
}
],
- "usage": {
- "prompt_tokens": 0,
- "completion_tokens": 0,
- "total_tokens": 0,
- },
+ "usage": _usage(
+ prompt_tokens,
+ _completion_tokens(post_tokens, prompt_tokens),
+ ),
}
)
diff --git a/src/openjarvis/evals/backends/jarvis_agent.py b/src/openjarvis/evals/backends/jarvis_agent.py
index ef5625da..ac60d922 100644
--- a/src/openjarvis/evals/backends/jarvis_agent.py
+++ b/src/openjarvis/evals/backends/jarvis_agent.py
@@ -193,6 +193,13 @@ class JarvisAgentBackend(InferenceBackend):
"energy_joules": energy_joules,
"peak_power_w": peak_power_w,
"power_watts": telemetry_data.get("power_watts", 0.0),
+ "energy_method": telemetry_data.get("energy_method", ""),
+ "energy_basis": telemetry_data.get("energy_basis", ""),
+ "cpu_energy_joules": telemetry_data.get("cpu_energy_joules", 0.0),
+ "gpu_energy_joules": telemetry_data.get("gpu_energy_joules", 0.0),
+ "dram_energy_joules": telemetry_data.get("dram_energy_joules", 0.0),
+ "ane_energy_joules": telemetry_data.get("ane_energy_joules", 0.0),
+ "soc_energy_joules": telemetry_data.get("soc_energy_joules", 0.0),
"gpu_utilization_pct": telemetry_data.get("gpu_utilization_pct", 0.0),
"throughput_tok_per_sec": telemetry_data.get("throughput_tok_per_sec", 0.0),
"trace_data": trace_data,
diff --git a/src/openjarvis/evals/backends/jarvis_direct.py b/src/openjarvis/evals/backends/jarvis_direct.py
index b90ba5cd..e3a539ca 100644
--- a/src/openjarvis/evals/backends/jarvis_direct.py
+++ b/src/openjarvis/evals/backends/jarvis_direct.py
@@ -124,6 +124,13 @@ class JarvisDirectBackend(InferenceBackend):
"energy_joules": energy_joules,
"peak_power_w": peak_power_w,
"power_watts": telemetry_data.get("power_watts", 0.0),
+ "energy_method": telemetry_data.get("energy_method", ""),
+ "energy_basis": telemetry_data.get("energy_basis", ""),
+ "cpu_energy_joules": telemetry_data.get("cpu_energy_joules", 0.0),
+ "gpu_energy_joules": telemetry_data.get("gpu_energy_joules", 0.0),
+ "dram_energy_joules": telemetry_data.get("dram_energy_joules", 0.0),
+ "ane_energy_joules": telemetry_data.get("ane_energy_joules", 0.0),
+ "soc_energy_joules": telemetry_data.get("soc_energy_joules", 0.0),
"gpu_utilization_pct": telemetry_data.get("gpu_utilization_pct", 0.0),
"throughput_tok_per_sec": telemetry_data.get("throughput_tok_per_sec", 0.0),
"tool_calls": 0,
diff --git a/src/openjarvis/evals/cli.py b/src/openjarvis/evals/cli.py
index d218770d..6beacc0b 100644
--- a/src/openjarvis/evals/cli.py
+++ b/src/openjarvis/evals/cli.py
@@ -935,10 +935,14 @@ def _run_agentic(
# Build TelemetrySession (optional — only if energy monitoring available)
telemetry_session = None
try:
+ from openjarvis.core.config import load_config
from openjarvis.telemetry.energy_monitor import create_energy_monitor
from openjarvis.telemetry.session import TelemetrySession
- monitor = create_energy_monitor()
+ # The eval config carries only a telemetry on/off flag, so the
+ # estimate opt-in has to come from the user's JarvisConfig.
+ allow_estimates = load_config().telemetry.allow_energy_estimates
+ monitor = create_energy_monitor(allow_estimates=allow_estimates)
if monitor is not None:
telemetry_session = TelemetrySession(
monitor=monitor,
diff --git a/src/openjarvis/evals/core/agentic_runner.py b/src/openjarvis/evals/core/agentic_runner.py
index 6c262bbd..0388992d 100644
--- a/src/openjarvis/evals/core/agentic_runner.py
+++ b/src/openjarvis/evals/core/agentic_runner.py
@@ -45,6 +45,23 @@ def _compute_energy_delta(
return None
+def _resolve_energy_basis(readings: list[Any]) -> Optional[str]:
+ """Which rail set these readings should be interpreted against.
+
+ A reading that carries a non-zero ANE or SoC rail came from an Apple
+ Silicon monitor, where the GPU rail alone is a partial view of one
+ package. Returns ``None`` when there is nothing to judge from, so a trace
+ recorded before per-rail sampling stays unlabelled rather than being
+ mislabelled "gpu".
+ """
+ if not readings:
+ return None
+ for r in readings:
+ if getattr(r, "soc_energy_j", 0.0) or getattr(r, "ane_energy_j", 0.0):
+ return "soc"
+ return "gpu"
+
+
def _compute_power_avg(
readings: list[Any],
power_field: str = "gpu_power_w",
@@ -602,8 +619,13 @@ class AgenticRunner:
# Query-level energy from telemetry window
query_gpu_energy = _compute_energy_delta(readings, "gpu_energy_j")
query_cpu_energy = _compute_energy_delta(readings, "cpu_energy_j")
+ query_ane_energy = _compute_energy_delta(readings, "ane_energy_j")
+ query_soc_energy = _compute_energy_delta(readings, "soc_energy_j")
query_gpu_power_avg = _compute_power_avg(readings, "gpu_power_w")
query_cpu_power_avg = _compute_power_avg(readings, "cpu_power_w")
+ query_ane_power_avg = _compute_power_avg(readings, "ane_power_w")
+ query_soc_power_avg = _compute_power_avg(readings, "soc_power_w")
+ query_energy_basis = _resolve_energy_basis(readings)
# Extract MBU from telemetry readings
mbu_values = [
@@ -623,6 +645,11 @@ class AgenticRunner:
completed=True,
query_gpu_energy_joules=query_gpu_energy,
query_cpu_energy_joules=query_cpu_energy,
+ query_ane_energy_joules=query_ane_energy,
+ query_soc_energy_joules=query_soc_energy,
+ query_ane_power_avg_watts=query_ane_power_avg,
+ query_soc_power_avg_watts=query_soc_power_avg,
+ energy_basis=query_energy_basis,
query_gpu_power_avg_watts=query_gpu_power_avg,
query_cpu_power_avg_watts=query_cpu_power_avg,
is_resolved=record.metadata.get("is_resolved"),
@@ -653,13 +680,21 @@ class AgenticRunner:
window = [r for r in readings if start_ns <= r.timestamp_ns <= end_ns]
gpu_energy = _compute_energy_delta(window, "gpu_energy_j")
cpu_energy = _compute_energy_delta(window, "cpu_energy_j")
+ ane_energy = _compute_energy_delta(window, "ane_energy_j")
+ soc_energy = _compute_energy_delta(window, "soc_energy_j")
avg_gpu_power = _compute_power_avg(window, "gpu_power_w")
avg_cpu_power = _compute_power_avg(window, "cpu_power_w")
+ avg_ane_power = _compute_power_avg(window, "ane_power_w")
+ avg_soc_power = _compute_power_avg(window, "soc_power_w")
return {
"gpu_energy_joules": gpu_energy,
"cpu_energy_joules": cpu_energy,
+ "ane_energy_joules": ane_energy,
+ "soc_energy_joules": soc_energy,
"avg_gpu_power_watts": avg_gpu_power,
"avg_cpu_power_watts": avg_cpu_power,
+ "avg_ane_power_watts": avg_ane_power,
+ "avg_soc_power_watts": avg_soc_power,
}
def _build_turn_traces(
@@ -800,17 +835,20 @@ class AgenticRunner:
if has_turn_energy:
return trace
- total_gpu_energy = _compute_energy_delta(readings, "gpu_energy_j")
- total_cpu_energy = _compute_energy_delta(readings, "cpu_energy_j")
+ totals = {
+ "gpu_energy_joules": _compute_energy_delta(readings, "gpu_energy_j"),
+ "cpu_energy_joules": _compute_energy_delta(readings, "cpu_energy_j"),
+ "ane_energy_joules": _compute_energy_delta(readings, "ane_energy_j"),
+ "soc_energy_joules": _compute_energy_delta(readings, "soc_energy_j"),
+ }
total_wall = sum(t.wall_clock_s for t in trace.turns)
if total_wall > 0:
for turn in trace.turns:
fraction = turn.wall_clock_s / total_wall
- if total_gpu_energy is not None:
- turn.gpu_energy_joules = total_gpu_energy * fraction
- if total_cpu_energy is not None:
- turn.cpu_energy_joules = total_cpu_energy * fraction
+ for attr, total in totals.items():
+ if total is not None:
+ setattr(turn, attr, total * fraction)
return trace
diff --git a/src/openjarvis/evals/core/runner.py b/src/openjarvis/evals/core/runner.py
index f40475e6..6daf113b 100644
--- a/src/openjarvis/evals/core/runner.py
+++ b/src/openjarvis/evals/core/runner.py
@@ -322,6 +322,13 @@ class EvalRunner:
ttft=full.get("ttft", 0.0) or 0.0,
energy_joules=energy_j,
power_watts=power_w,
+ cpu_energy_joules=full.get("cpu_energy_joules", 0.0) or 0.0,
+ gpu_energy_joules=full.get("gpu_energy_joules", 0.0) or 0.0,
+ dram_energy_joules=full.get("dram_energy_joules", 0.0) or 0.0,
+ ane_energy_joules=full.get("ane_energy_joules", 0.0) or 0.0,
+ soc_energy_joules=full.get("soc_energy_joules", 0.0) or 0.0,
+ energy_basis=full.get("energy_basis", "") or "",
+ energy_method=full.get("energy_method", "") or "",
gpu_utilization_pct=full.get("gpu_utilization_pct", 0.0) or 0.0,
throughput_tok_per_sec=full.get("throughput_tok_per_sec", 0.0) or 0.0,
trace_data=full.get("trace_data"),
@@ -483,6 +490,13 @@ class EvalRunner:
ttft=full.get("ttft", 0.0) or 0.0,
energy_joules=energy_j,
power_watts=power_w,
+ cpu_energy_joules=full.get("cpu_energy_joules", 0.0) or 0.0,
+ gpu_energy_joules=full.get("gpu_energy_joules", 0.0) or 0.0,
+ dram_energy_joules=full.get("dram_energy_joules", 0.0) or 0.0,
+ ane_energy_joules=full.get("ane_energy_joules", 0.0) or 0.0,
+ soc_energy_joules=full.get("soc_energy_joules", 0.0) or 0.0,
+ energy_basis=full.get("energy_basis", "") or "",
+ energy_method=full.get("energy_method", "") or "",
gpu_utilization_pct=full.get("gpu_utilization_pct", 0.0) or 0.0,
throughput_tok_per_sec=throughput,
mfu_pct=mfu,
@@ -861,6 +875,13 @@ class EvalRunner:
"ttft": result.ttft,
"energy_joules": result.energy_joules,
"power_watts": result.power_watts,
+ "cpu_energy_joules": result.cpu_energy_joules,
+ "gpu_energy_joules": result.gpu_energy_joules,
+ "dram_energy_joules": result.dram_energy_joules,
+ "ane_energy_joules": result.ane_energy_joules,
+ "soc_energy_joules": result.soc_energy_joules,
+ "energy_basis": result.energy_basis,
+ "energy_method": result.energy_method,
"gpu_utilization_pct": result.gpu_utilization_pct,
"throughput_tok_per_sec": result.throughput_tok_per_sec,
"mfu_pct": result.mfu_pct,
@@ -1004,6 +1025,28 @@ class EvalRunner:
]
total_energy = sum(r.energy_joules for r in results)
+ # Per-rail totals, so a summary says which hardware did the work
+ # rather than only how much it cost in aggregate.
+ rail_totals = {
+ rail: round(
+ sum(getattr(r, f"{rail}_energy_joules", 0.0) for r in results), 6
+ )
+ for rail in ("cpu", "gpu", "dram", "ane", "soc")
+ }
+ rail_totals = {k: v for k, v in rail_totals.items() if v > 0}
+
+ # How the energy was obtained, and from which rail set. Both are
+ # per-record, but a run uses one monitor throughout; report the value
+ # when it is unambiguous and flag genuine mixtures rather than
+ # silently picking one.
+ def _consensus(attr: str) -> str:
+ seen = {getattr(r, attr, "") for r in results if getattr(r, attr, "")}
+ if not seen:
+ return ""
+ return seen.pop() if len(seen) == 1 else "mixed"
+
+ energy_method_value = _consensus("energy_method")
+ energy_basis_value = _consensus("energy_basis")
total_estimated_flops = sum(r.estimated_flops for r in results)
total_input_tokens = sum(r.prompt_tokens for r in results)
total_output_tokens = sum(r.completion_tokens for r in results)
@@ -1015,6 +1058,9 @@ class EvalRunner:
"total_energy_joules": round(total_energy, 6),
"avg_power_watts": round(avg_power, 4),
"total_estimated_flops": total_estimated_flops,
+ "energy_method": energy_method_value,
+ "energy_basis": energy_basis_value,
+ "energy_joules_by_rail": rail_totals or None,
"ipj": (round(accuracy / total_energy, 6) if total_energy > 0 else None),
"ipw": (round(accuracy / avg_power, 6) if avg_power > 0 else None),
}
@@ -1057,6 +1103,7 @@ class EvalRunner:
input_token_stats=_metric_stats([float(v) for v in input_tok_vals]),
output_token_stats=_metric_stats([float(v) for v in output_tok_vals]),
total_energy_joules=round(total_energy, 6),
+ energy_method=energy_method_value,
total_estimated_flops=total_estimated_flops,
flops_stats=_metric_stats(flops_vals),
warmup_samples_excluded=cfg.warmup_samples,
diff --git a/src/openjarvis/evals/core/trace.py b/src/openjarvis/evals/core/trace.py
index 8e2e1a1b..d465ba75 100644
--- a/src/openjarvis/evals/core/trace.py
+++ b/src/openjarvis/evals/core/trace.py
@@ -23,8 +23,17 @@ class TurnTrace:
# Energy and cost fields
gpu_energy_joules: Optional[float] = None
cpu_energy_joules: Optional[float] = None
+ # Apple Neural Engine, populated on Apple Silicon only. Without it an
+ # ANE-resident model (Apple Foundation Models runs mostly on the ANE)
+ # records almost no energy for the work it actually did.
+ ane_energy_joules: Optional[float] = None
+ # Every rail measured, summed -- the figure per-token energy should be
+ # derived from on a unified-memory SoC.
+ soc_energy_joules: Optional[float] = None
gpu_power_avg_watts: Optional[float] = None
cpu_power_avg_watts: Optional[float] = None
+ ane_power_avg_watts: Optional[float] = None
+ soc_power_avg_watts: Optional[float] = None
cost_usd: Optional[float] = None
# Per-action energy breakdown (lm_inference vs tool_call granularity)
action_energy_breakdown: Optional[List[Dict[str, Any]]] = None
@@ -43,8 +52,12 @@ class TurnTrace:
"error": self.error,
"gpu_energy_joules": self.gpu_energy_joules,
"cpu_energy_joules": self.cpu_energy_joules,
+ "ane_energy_joules": self.ane_energy_joules,
+ "soc_energy_joules": self.soc_energy_joules,
"gpu_power_avg_watts": self.gpu_power_avg_watts,
"cpu_power_avg_watts": self.cpu_power_avg_watts,
+ "ane_power_avg_watts": self.ane_power_avg_watts,
+ "soc_power_avg_watts": self.soc_power_avg_watts,
"cost_usd": self.cost_usd,
"action_energy_breakdown": self.action_energy_breakdown,
"tool_calls": [dict(tc) for tc in self.tool_calls],
@@ -63,6 +76,10 @@ class TurnTrace:
error=d.get("error"),
gpu_energy_joules=d.get("gpu_energy_joules"),
cpu_energy_joules=d.get("cpu_energy_joules"),
+ ane_energy_joules=d.get("ane_energy_joules"),
+ soc_energy_joules=d.get("soc_energy_joules"),
+ ane_power_avg_watts=d.get("ane_power_avg_watts"),
+ soc_power_avg_watts=d.get("soc_power_avg_watts"),
gpu_power_avg_watts=d.get("gpu_power_avg_watts"),
cpu_power_avg_watts=d.get("cpu_power_avg_watts"),
cost_usd=d.get("cost_usd"),
@@ -86,8 +103,16 @@ class QueryTrace:
# Query-level energy fields (populated even when turns are empty)
query_gpu_energy_joules: Optional[float] = None
query_cpu_energy_joules: Optional[float] = None
+ query_ane_energy_joules: Optional[float] = None
+ query_soc_energy_joules: Optional[float] = None
query_gpu_power_avg_watts: Optional[float] = None
query_cpu_power_avg_watts: Optional[float] = None
+ query_ane_power_avg_watts: Optional[float] = None
+ query_soc_power_avg_watts: Optional[float] = None
+ # Rail set the energy figures above should be read from: "soc" on Apple
+ # Silicon, "gpu" elsewhere. Energy and power must be taken from the same
+ # basis -- SoC joules over GPU watts is a meaningless ratio.
+ energy_basis: Optional[str] = None
is_resolved: Optional[bool] = None
query_mbu_avg_pct: Optional[float] = None
query_mbu_max_pct: Optional[float] = None
@@ -137,6 +162,36 @@ class QueryTrace:
return sum(values)
return self.query_cpu_energy_joules
+ @property
+ def total_ane_energy_joules(self) -> Optional[float]:
+ values = [
+ t.ane_energy_joules for t in self.turns if t.ane_energy_joules is not None
+ ]
+ if values:
+ return sum(values)
+ return self.query_ane_energy_joules
+
+ @property
+ def total_soc_energy_joules(self) -> Optional[float]:
+ values = [
+ t.soc_energy_joules for t in self.turns if t.soc_energy_joules is not None
+ ]
+ if values:
+ return sum(values)
+ return self.query_soc_energy_joules
+
+ @property
+ def total_basis_energy_joules(self) -> Optional[float]:
+ """Energy on whichever rail set this trace was measured against.
+
+ Use this rather than ``total_gpu_energy_joules`` for per-token or
+ per-watt figures: on Apple Silicon the GPU rail alone misses the CPU
+ and ANE work that actually ran.
+ """
+ if self.energy_basis == "soc":
+ return self.total_soc_energy_joules
+ return self.total_gpu_energy_joules
+
@property
def total_cost_usd(self) -> Optional[float]:
values = [t.cost_usd for t in self.turns if t.cost_usd is not None]
@@ -198,8 +253,13 @@ class QueryTrace:
"timed_out": self.timed_out,
"query_gpu_energy_joules": self.query_gpu_energy_joules,
"query_cpu_energy_joules": self.query_cpu_energy_joules,
+ "query_ane_energy_joules": self.query_ane_energy_joules,
+ "query_soc_energy_joules": self.query_soc_energy_joules,
"query_gpu_power_avg_watts": self.query_gpu_power_avg_watts,
"query_cpu_power_avg_watts": self.query_cpu_power_avg_watts,
+ "query_ane_power_avg_watts": self.query_ane_power_avg_watts,
+ "query_soc_power_avg_watts": self.query_soc_power_avg_watts,
+ "energy_basis": self.energy_basis,
"is_resolved": self.is_resolved,
"query_mbu_avg_pct": self.query_mbu_avg_pct,
"query_mbu_max_pct": self.query_mbu_max_pct,
@@ -220,8 +280,13 @@ class QueryTrace:
timed_out=d.get("timed_out", False),
query_gpu_energy_joules=d.get("query_gpu_energy_joules"),
query_cpu_energy_joules=d.get("query_cpu_energy_joules"),
+ query_ane_energy_joules=d.get("query_ane_energy_joules"),
+ query_soc_energy_joules=d.get("query_soc_energy_joules"),
query_gpu_power_avg_watts=d.get("query_gpu_power_avg_watts"),
query_cpu_power_avg_watts=d.get("query_cpu_power_avg_watts"),
+ query_ane_power_avg_watts=d.get("query_ane_power_avg_watts"),
+ query_soc_power_avg_watts=d.get("query_soc_power_avg_watts"),
+ energy_basis=d.get("energy_basis"),
is_resolved=d.get("is_resolved"),
query_mbu_avg_pct=d.get("query_mbu_avg_pct"),
query_mbu_max_pct=d.get("query_mbu_max_pct"),
diff --git a/src/openjarvis/evals/core/types.py b/src/openjarvis/evals/core/types.py
index 78605634..1beaf337 100644
--- a/src/openjarvis/evals/core/types.py
+++ b/src/openjarvis/evals/core/types.py
@@ -36,6 +36,19 @@ class EvalResult:
ttft: float = 0.0
energy_joules: float = 0.0
power_watts: float = 0.0
+ # Per-rail breakdown. Without these a reader cannot tell which hardware
+ # actually did the work -- and on Apple Silicon that is the whole point:
+ # an ANE-resident model draws almost nothing on the GPU rail.
+ cpu_energy_joules: float = 0.0
+ gpu_energy_joules: float = 0.0
+ dram_energy_joules: float = 0.0
+ ane_energy_joules: float = 0.0
+ soc_energy_joules: float = 0.0
+ # Which rail set energy_joules was taken from ("soc" / "gpu"), and how it
+ # was obtained. Without energy_method a modelled estimate is
+ # indistinguishable from a hardware measurement at read time.
+ energy_basis: str = ""
+ energy_method: str = ""
gpu_utilization_pct: float = 0.0
throughput_tok_per_sec: float = 0.0
mfu_pct: float = 0.0
diff --git a/src/openjarvis/intelligence/model_catalog.py b/src/openjarvis/intelligence/model_catalog.py
index 52b3b49d..ea010ea6 100644
--- a/src/openjarvis/intelligence/model_catalog.py
+++ b/src/openjarvis/intelligence/model_catalog.py
@@ -1015,6 +1015,55 @@ BUILTIN_MODELS: List[ModelSpec] = [
"url": "https://ai.google.dev/gemini-api/docs/models",
},
),
+ # -----------------------------------------------------------------------
+ # Apple Foundation Models (on-device, AFM 3)
+ #
+ # These ids are run *labels*, not selectors: the Foundation Models
+ # framework's dynamic profile picks AFM 3 Core (dense ~3B) or AFM 3 Core
+ # Advanced (20B sparse MoE, 1-4B active) from the host device, and the SDK
+ # exposes no way to choose. Requesting one label does not guarantee that
+ # variant executed -- `AppleFMEngine.describe()` records the host chip and
+ # SDK version so a run can be attributed after the fact.
+ # -----------------------------------------------------------------------
+ ModelSpec(
+ model_id="afm-3",
+ name="Apple Foundation Model 3",
+ parameter_count_b=3.0,
+ context_length=4096,
+ supported_engines=("afm", "apple_fm"),
+ provider="apple",
+ metadata={
+ "architecture": "dense",
+ "variant_selection": "device-chosen; model id is a label only",
+ },
+ ),
+ ModelSpec(
+ model_id="afm-3-core",
+ name="Apple Foundation Model 3 Core",
+ parameter_count_b=3.0,
+ context_length=4096,
+ supported_engines=("afm", "apple_fm"),
+ provider="apple",
+ metadata={
+ "architecture": "dense",
+ "variant_selection": "device-chosen; model id is a label only",
+ },
+ ),
+ ModelSpec(
+ model_id="afm-3-core-advanced",
+ name="Apple Foundation Model 3 Core Advanced",
+ # 20B total, but Instruction-Following Pruning activates only 1-4B per
+ # request; the 4B upper bound is what FLOPs accounting should use.
+ parameter_count_b=20.0,
+ active_parameter_count_b=4.0,
+ context_length=4096,
+ supported_engines=("afm", "apple_fm"),
+ provider="apple",
+ metadata={
+ "architecture": "sparse-moe",
+ "variant_selection": "device-chosen; model id is a label only",
+ },
+ ),
]
diff --git a/src/openjarvis/sdk.py b/src/openjarvis/sdk.py
index 43976c41..5f17216e 100644
--- a/src/openjarvis/sdk.py
+++ b/src/openjarvis/sdk.py
@@ -230,6 +230,7 @@ class Jarvis:
energy_monitor = create_energy_monitor(
prefer_vendor=self._config.telemetry.energy_vendor or None,
+ allow_estimates=self._config.telemetry.allow_energy_estimates,
)
except Exception as exc:
logger.debug("Failed to create energy monitor: %s", exc)
diff --git a/src/openjarvis/security/data_boundary_audit.py b/src/openjarvis/security/data_boundary_audit.py
index 025966c6..d104d4f9 100644
--- a/src/openjarvis/security/data_boundary_audit.py
+++ b/src/openjarvis/security/data_boundary_audit.py
@@ -33,6 +33,10 @@ CLOUD_PROVIDER_KEYS = {
}
LOCAL_ENGINE_KEYS = {
+ # Apple Foundation Models, both the in-process engine and the shim. The
+ # Python SDK has no Private Cloud Compute path, so inference is
+ # unconditionally on-device.
+ "afm",
"apple_fm",
"exo",
"gemma_cpp",
diff --git a/src/openjarvis/system/builder.py b/src/openjarvis/system/builder.py
index 7fc9804a..0696ea5a 100644
--- a/src/openjarvis/system/builder.py
+++ b/src/openjarvis/system/builder.py
@@ -165,6 +165,7 @@ class SystemBuilder:
energy_monitor = create_energy_monitor(
poll_interval_ms=config.telemetry.gpu_poll_interval_ms,
prefer_vendor=config.telemetry.energy_vendor or None,
+ allow_estimates=config.telemetry.allow_energy_estimates,
)
except ImportError:
pass
diff --git a/src/openjarvis/telemetry/aggregator.py b/src/openjarvis/telemetry/aggregator.py
index b6bb95a9..eae7dc2b 100644
--- a/src/openjarvis/telemetry/aggregator.py
+++ b/src/openjarvis/telemetry/aggregator.py
@@ -33,6 +33,11 @@ class ModelStats:
avg_throughput_per_watt: float = 0.0
total_prefill_energy_joules: float = 0.0
total_decode_energy_joules: float = 0.0
+ # Per-rail totals. `total_ane_energy_joules` is what makes an ANE-resident
+ # backend (Apple Foundation Models) legible here at all -- on the GPU rail
+ # alone it looks like it did no work.
+ total_ane_energy_joules: float = 0.0
+ total_soc_energy_joules: float = 0.0
avg_mean_itl_ms: float = 0.0
avg_median_itl_ms: float = 0.0
avg_p95_itl_ms: float = 0.0
@@ -57,6 +62,11 @@ class EngineStats:
avg_throughput_per_watt: float = 0.0
total_prefill_energy_joules: float = 0.0
total_decode_energy_joules: float = 0.0
+ # Per-rail totals. `total_ane_energy_joules` is what makes an ANE-resident
+ # backend (Apple Foundation Models) legible here at all -- on the GPU rail
+ # alone it looks like it did no work.
+ total_ane_energy_joules: float = 0.0
+ total_soc_energy_joules: float = 0.0
avg_mean_itl_ms: float = 0.0
avg_median_itl_ms: float = 0.0
avg_p95_itl_ms: float = 0.0
@@ -77,6 +87,11 @@ class AggregatedStats:
avg_throughput_per_watt: float = 0.0
total_prefill_energy_joules: float = 0.0
total_decode_energy_joules: float = 0.0
+ # Per-rail totals. `total_ane_energy_joules` is what makes an ANE-resident
+ # backend (Apple Foundation Models) legible here at all -- on the GPU rail
+ # alone it looks like it did no work.
+ total_ane_energy_joules: float = 0.0
+ total_soc_energy_joules: float = 0.0
avg_mean_itl_ms: float = 0.0
avg_median_itl_ms: float = 0.0
avg_p95_itl_ms: float = 0.0
@@ -155,6 +170,7 @@ class TelemetryAggregator:
has_derived = self._safe_col("energy_per_output_token_joules")
has_phase = self._safe_col("prefill_energy_joules")
has_itl = self._safe_col("mean_itl_ms")
+ has_rails = self._safe_col("ane_energy_joules")
if has_pte:
extra_cols += ", SUM(prompt_tokens_evaluated) AS prompt_tokens_evaluated"
@@ -177,6 +193,11 @@ class TelemetryAggregator:
", AVG(median_itl_ms) AS avg_median_itl_ms"
", AVG(p95_itl_ms) AS avg_p95_itl_ms"
)
+ if has_rails:
+ extra_cols += (
+ ", SUM(ane_energy_joules) AS total_ane_energy_joules"
+ ", SUM(soc_energy_joules) AS total_soc_energy_joules"
+ )
sql = (
"SELECT model_id,"
@@ -228,6 +249,9 @@ class TelemetryAggregator:
ms.avg_mean_itl_ms = r["avg_mean_itl_ms"] or 0.0
ms.avg_median_itl_ms = r["avg_median_itl_ms"] or 0.0
ms.avg_p95_itl_ms = r["avg_p95_itl_ms"] or 0.0
+ if has_rails:
+ ms.total_ane_energy_joules = r["total_ane_energy_joules"] or 0.0
+ ms.total_soc_energy_joules = r["total_soc_energy_joules"] or 0.0
result.append(ms)
return result
@@ -247,6 +271,7 @@ class TelemetryAggregator:
has_derived = self._safe_col("energy_per_output_token_joules")
has_phase = self._safe_col("prefill_energy_joules")
has_itl = self._safe_col("mean_itl_ms")
+ has_rails = self._safe_col("ane_energy_joules")
if has_tpj:
extra_cols += ", AVG(tokens_per_joule) AS avg_tokens_per_joule"
@@ -267,6 +292,11 @@ class TelemetryAggregator:
", AVG(median_itl_ms) AS avg_median_itl_ms"
", AVG(p95_itl_ms) AS avg_p95_itl_ms"
)
+ if has_rails:
+ extra_cols += (
+ ", SUM(ane_energy_joules) AS total_ane_energy_joules"
+ ", SUM(soc_energy_joules) AS total_soc_energy_joules"
+ )
sql = (
"SELECT engine,"
@@ -312,6 +342,9 @@ class TelemetryAggregator:
es.avg_mean_itl_ms = r["avg_mean_itl_ms"] or 0.0
es.avg_median_itl_ms = r["avg_median_itl_ms"] or 0.0
es.avg_p95_itl_ms = r["avg_p95_itl_ms"] or 0.0
+ if has_rails:
+ es.total_ane_energy_joules = r["total_ane_energy_joules"] or 0.0
+ es.total_soc_energy_joules = r["total_soc_energy_joules"] or 0.0
result.append(es)
return result
@@ -364,6 +397,8 @@ class TelemetryAggregator:
total_decode_energy_joules=sum(
m.total_decode_energy_joules for m in model_stats
),
+ total_ane_energy_joules=sum(m.total_ane_energy_joules for m in model_stats),
+ total_soc_energy_joules=sum(m.total_soc_energy_joules for m in model_stats),
avg_mean_itl_ms=_weighted_avg("avg_mean_itl_ms"),
avg_median_itl_ms=_weighted_avg("avg_median_itl_ms"),
avg_p95_itl_ms=_weighted_avg("avg_p95_itl_ms"),
diff --git a/src/openjarvis/telemetry/energy_apple.py b/src/openjarvis/telemetry/energy_apple.py
index 4cffef1d..ab5318eb 100644
--- a/src/openjarvis/telemetry/energy_apple.py
+++ b/src/openjarvis/telemetry/energy_apple.py
@@ -1,15 +1,35 @@
-"""Apple Silicon energy monitor — via zeus-ml[apple] or CPU-time estimation."""
+"""Apple Silicon energy monitor — per-rail IOReport counters via zeus-apple-silicon.
+
+Reads the SoC's own energy counters (CPU, GPU, DRAM, Apple Neural Engine)
+through ``zeus_apple_silicon``, a nanobind extension over IOReport. No root
+required, unlike ``powermetrics``.
+
+**Why not via ``zeus-ml``:** zeus wraps this same library as
+``zeus.device.soc.apple.AppleSilicon``, but that module landed on zeus master
+in April 2025 and no zeus release has shipped since February 2025 — every
+published ``zeus-ml`` (through 0.11.0.post1) has no ``zeus.device.soc``
+package at all, and no ``apple`` extra. Depending on the underlying library
+directly is the only spelling that actually resolves.
+
+The ANE rail is what makes this worth doing: Apple Foundation Models runs
+predominantly on the Neural Engine, so a GPU-only reading makes it look
+almost free.
+"""
from __future__ import annotations
+import atexit
import logging
import platform
import subprocess
import time
+import weakref
from contextlib import contextmanager
-from typing import Generator
+from dataclasses import dataclass
+from typing import Any, Generator, Optional
from openjarvis.telemetry.energy_monitor import (
+ BASIS_SOC,
EnergyMonitor,
EnergySample,
EnergyVendor,
@@ -18,14 +38,33 @@ from openjarvis.telemetry.energy_monitor import (
logger = logging.getLogger(__name__)
try:
- from zeus.device.soc.apple import AppleSiliconMonitor
+ from zeus_apple_silicon import AppleEnergyMonitor as _AppleRailReader
- _ZEUS_APPLE_AVAILABLE = True
+ _NATIVE_AVAILABLE = True
except ImportError:
- _ZEUS_APPLE_AVAILABLE = False
+ _AppleRailReader = None # type: ignore[assignment]
+ _NATIVE_AVAILABLE = False
+
+
+# Monitors holding a live native handle. Long-lived callers (`jarvis serve`)
+# create one for the process lifetime and never close it, which leaves the
+# nanobind object alive when the extension's types are torn down -- the
+# extension then prints "leaked instance" warnings on exit. Releasing the
+# handles at interpreter shutdown keeps that teardown clean.
+_LIVE_MONITORS: "weakref.WeakSet[AppleEnergyMonitor]" = weakref.WeakSet()
+
+
+def _release_live_monitors() -> None: # pragma: no cover - shutdown hook
+ for monitor in list(_LIVE_MONITORS):
+ try:
+ monitor.close()
+ except Exception:
+ pass
-# Typical package TDP (watts) by chip family. Used for the CPU-time fallback.
+# Typical package TDP (watts) by chip family. Used only for the opt-in
+# modelled estimate, never for measured readings.
_CHIP_TDP: dict[str, float] = {
"M1": 15.0,
"M1 Pro": 30.0,
@@ -42,11 +81,19 @@ _CHIP_TDP: dict[str, float] = {
"M4": 15.0,
"M4 Pro": 30.0,
"M4 Max": 60.0,
+ "M5": 15.0,
+ "M5 Pro": 30.0,
+ "M5 Max": 60.0,
+ "M5 Ultra": 90.0,
}
def _detect_chip() -> tuple[str, float]:
- """Return (chip_name, tdp_watts) for the current Apple Silicon SoC."""
+ """Return (chip_name, tdp_watts) for the current Apple Silicon SoC.
+
+ ``platform.processor()`` reports a bare "arm" on every Apple Silicon Mac,
+ so ``sysctl`` is the only way to name the actual part.
+ """
try:
r = subprocess.run(
["sysctl", "-n", "machdep.cpu.brand_string"],
@@ -61,124 +108,302 @@ def _detect_chip() -> tuple[str, float]:
for chip, tdp in sorted(_CHIP_TDP.items(), key=lambda kv: -len(kv[0])):
if chip in brand:
- return chip, tdp
+ return brand or chip, tdp
return brand or "Apple Silicon", 20.0
+@dataclass
+class _Rails:
+ """One reading of the SoC's energy rails, in joules.
+
+ A rail is ``None`` when this chip does not expose that counter — which is
+ not the same as zero, and must not be summed as if it were.
+ """
+
+ cpu: Optional[float] = None
+ gpu: Optional[float] = None
+ dram: Optional[float] = None
+ ane: Optional[float] = None
+
+ @property
+ def total(self) -> float:
+ """Sum of the rails that are actually present."""
+ rails = (self.cpu, self.gpu, self.dram, self.ane)
+ return sum(v for v in rails if v is not None)
+
+ def __sub__(self, other: "_Rails") -> "_Rails":
+ def diff(a: Optional[float], b: Optional[float]) -> Optional[float]:
+ if a is None or b is None:
+ return None
+ # Counters are monotonic; a negative delta means a wrapped or
+ # reset counter, which is better reported as "unknown" than as a
+ # negative energy.
+ return a - b if a >= b else None
+
+ return _Rails(
+ cpu=diff(self.cpu, other.cpu),
+ gpu=diff(self.gpu, other.gpu),
+ dram=diff(self.dram, other.dram),
+ ane=diff(self.ane, other.ane),
+ )
+
+
+def _mj(value: Any) -> Optional[float]:
+ """Convert one millijoule counter to joules, or ``None`` if absent."""
+ if value is None:
+ return None
+ try:
+ return float(value) / 1000.0
+ except (TypeError, ValueError):
+ return None
+
+
+def _rails_from_metrics(metrics: Any) -> _Rails:
+ """Map a ``zeus_apple_silicon.AppleEnergyMetrics`` onto :class:`_Rails`.
+
+ ``cpu_total_mj`` already subsumes the per-core, per-cluster and
+ core-manager breakdowns the object also exposes (verified on an M1 Pro:
+ the cluster fields sum to ``cpu_total_mj``), so adding those would double
+ count. ``gpu_sram_mj`` is a separate rail from ``gpu_mj`` and does need
+ adding.
+ """
+ gpu = _mj(getattr(metrics, "gpu_mj", None))
+ gpu_sram = _mj(getattr(metrics, "gpu_sram_mj", None))
+ if gpu is not None and gpu_sram is not None:
+ gpu = gpu + gpu_sram
+ elif gpu is None:
+ gpu = gpu_sram
+
+ return _Rails(
+ cpu=_mj(getattr(metrics, "cpu_total_mj", None)),
+ gpu=gpu,
+ dram=_mj(getattr(metrics, "dram_mj", None)),
+ ane=_mj(getattr(metrics, "ane_mj", None)),
+ )
+
+
class AppleEnergyMonitor(EnergyMonitor):
"""Apple Silicon energy monitor.
- Prefers ``zeus-ml[apple]`` ``AppleSiliconMonitor`` when available.
- Falls back to a CPU-time-based estimation using the chip's known TDP,
- which gives order-of-magnitude correct energy readings without root.
+ Measures CPU, GPU, DRAM and ANE rails via ``zeus-apple-silicon``. When
+ that backend is absent, :meth:`available` reports ``False`` so the factory
+ falls through rather than inventing numbers; a modelled TDP estimate is
+ reachable only by explicitly constructing with ``allow_estimates=True``.
"""
- def __init__(self, poll_interval_ms: int = 50) -> None:
+ def __init__(
+ self,
+ poll_interval_ms: int = 50,
+ allow_estimates: bool = False,
+ ) -> None:
self._poll_interval_ms = poll_interval_ms
- self._monitor = None
- self._zeus_ok = False
+ self._allow_estimates = allow_estimates
+ self._reader: Any = None
self._chip_name, self._tdp_watts = _detect_chip()
+ # Previous cumulative reading, for deriving power in snapshot().
+ self._prev: Optional[tuple[float, _Rails]] = None
- if _ZEUS_APPLE_AVAILABLE and platform.system() == "Darwin":
+ if _NATIVE_AVAILABLE and platform.system() == "Darwin":
try:
- self._monitor = AppleSiliconMonitor()
- self._zeus_ok = True
+ self._reader = _AppleRailReader()
+ _LIVE_MONITORS.add(self)
except Exception as exc:
logger.debug(
"Failed to initialize Apple Silicon energy monitor: %s",
exc,
)
+ # ---- capability probes ---------------------------------------------
+
@staticmethod
def available() -> bool:
+ """True only when rails can actually be *measured*.
+
+ Deliberately stricter than "is this a Mac": returning True on any
+ Apple Silicon host is what previously let a modelled estimate
+ masquerade as a measurement for every reader downstream.
+ """
+ if not AppleEnergyMonitor.estimate_available():
+ return False
+ if not _NATIVE_AVAILABLE:
+ return False
+ try:
+ _AppleRailReader()
+ except Exception as exc:
+ logger.debug("Apple Silicon energy backend unusable: %s", exc)
+ return False
+ return True
+
+ @staticmethod
+ def estimate_available() -> bool:
+ """True on any Apple Silicon host, measurable or not."""
return platform.system() == "Darwin" and platform.machine() == "arm64"
def vendor(self) -> EnergyVendor:
return EnergyVendor.APPLE
def energy_method(self) -> str:
- return "zeus" if self._zeus_ok else "cpu_time_estimate"
+ return "ioreport" if self._reader is not None else "tdp_estimate"
- @contextmanager
- def sample(self) -> Generator[EnergySample, None, None]:
- result = EnergySample(
+ # ---- measurement ----------------------------------------------------
+
+ def _new_sample(self) -> EnergySample:
+ return EnergySample(
vendor=EnergyVendor.APPLE.value,
device_name=self._chip_name,
device_count=1,
energy_method=self.energy_method(),
+ basis=BASIS_SOC,
)
- if self._zeus_ok and self._monitor is not None:
- yield from self._sample_zeus(result)
+ @contextmanager
+ def sample(self) -> Generator[EnergySample, None, None]:
+ result = self._new_sample()
+
+ if self._reader is not None:
+ yield from self._sample_ioreport(result)
+ elif self._allow_estimates:
+ yield from self._sample_estimate(result)
else:
- yield from self._sample_cputime(result)
+ # No backend and no opt-in: hand back an empty sample rather than
+ # a fabricated one. Callers see zeros and energy_method tells them
+ # why.
+ yield result
- def _sample_zeus(
+ def _sample_ioreport(
self,
result: EnergySample,
) -> Generator[EnergySample, None, None]:
- assert self._monitor is not None
- window_name = f"openjarvis_{time.monotonic_ns()}"
+ assert self._reader is not None
+ window = f"openjarvis_{time.monotonic_ns()}"
t_start = time.monotonic()
- self._monitor.begin_window(window_name)
-
- yield result
-
- measurement = self._monitor.end_window(window_name)
- wall = time.monotonic() - t_start
-
- cpu_j = getattr(measurement, "cpu_energy", 0.0)
- gpu_j = getattr(measurement, "gpu_energy", 0.0)
- dram_j = getattr(measurement, "dram_energy", 0.0)
- ane_j = getattr(measurement, "ane_energy", 0.0)
+ self._reader.begin_window(window)
+
+ try:
+ yield result
+ finally:
+ # No `return` in this block: returning from a finally discards an
+ # exception the body raised, so a failed end_window would swallow
+ # the caller's error.
+ try:
+ metrics = self._reader.end_window(window)
+ except Exception as exc:
+ logger.debug("Apple energy window failed to close: %s", exc)
+ else:
+ self._apply(
+ result,
+ _rails_from_metrics(metrics),
+ time.monotonic() - t_start,
+ )
- result.cpu_energy_joules = float(cpu_j)
- result.gpu_energy_joules = float(gpu_j)
- result.dram_energy_joules = float(dram_j)
- result.ane_energy_joules = float(ane_j)
- result.energy_joules = cpu_j + gpu_j + dram_j + ane_j
+ @staticmethod
+ def _apply(result: EnergySample, rails: _Rails, wall: float) -> None:
+ """Write a rail reading and its derived totals onto *result*."""
+ result.cpu_energy_joules = rails.cpu or 0.0
+ result.gpu_energy_joules = rails.gpu or 0.0
+ result.dram_energy_joules = rails.dram or 0.0
+ result.ane_energy_joules = rails.ane or 0.0
+
+ total = rails.total
+ result.soc_energy_joules = total
+ # On Apple Silicon the SoC sum *is* the energy figure — every rail
+ # here is on the same package.
+ result.energy_joules = total
result.duration_seconds = wall
if wall > 0:
- result.mean_power_watts = result.energy_joules / wall
-
- def _sample_cputime(
+ result.mean_power_watts = total / wall
+ result.soc_power_watts = total / wall
+ result.cpu_power_watts = (rails.cpu or 0.0) / wall
+ result.gpu_power_watts = (rails.gpu or 0.0) / wall
+ result.ane_power_watts = (rails.ane or 0.0) / wall
+
+ def snapshot(self) -> EnergySample:
+ """Instantaneous reading, for background polling by ``TelemetrySession``.
+
+ Energy fields carry the *cumulative* since-boot counters, which is what
+ the eval runners difference across a window
+ (``_compute_energy_delta`` takes last-minus-first). Power is derived
+ from the delta against the previous snapshot, so the first call in a
+ session reports zero watts.
+ """
+ result = self._new_sample()
+ if self._reader is None:
+ return result
+
+ try:
+ metrics = self._reader.get_cumulative_energy()
+ except Exception as exc:
+ logger.debug("Apple cumulative energy read failed: %s", exc)
+ return result
+
+ now = time.monotonic()
+ rails = _rails_from_metrics(metrics)
+
+ result.cpu_energy_joules = rails.cpu or 0.0
+ result.gpu_energy_joules = rails.gpu or 0.0
+ result.dram_energy_joules = rails.dram or 0.0
+ result.ane_energy_joules = rails.ane or 0.0
+ result.soc_energy_joules = rails.total
+ result.energy_joules = rails.total
+ result.num_snapshots = 1
+
+ prev = self._prev
+ self._prev = (now, rails)
+ if prev is not None:
+ dt = now - prev[0]
+ if dt > 0:
+ delta = rails - prev[1]
+ result.duration_seconds = dt
+ result.cpu_power_watts = (delta.cpu or 0.0) / dt
+ result.gpu_power_watts = (delta.gpu or 0.0) / dt
+ result.ane_power_watts = (delta.ane or 0.0) / dt
+ result.soc_power_watts = delta.total / dt
+ result.mean_power_watts = result.soc_power_watts
+
+ return result
+
+ # ---- opt-in modelled estimate --------------------------------------
+
+ def _sample_estimate(
self,
result: EnergySample,
) -> Generator[EnergySample, None, None]:
- """Estimate energy from user+system CPU time and chip TDP.
-
- Energy ≈ (cpu_seconds / wall_seconds) × TDP × wall_seconds
- = cpu_seconds × TDP
-
- This is an approximation — real power varies with clock speed,
- workload mix, and thermal state — but it gives useful non-zero
- readings without requiring root or external libraries.
+ """Model energy as wall-clock × TDP, split by fixed rail ratios.
+
+ This is not a measurement. It reads no counters and no utilization: a
+ ten-second sleep and a ten-second generation produce identical joules,
+ and the rail split is four constants. It exists only so a host without
+ ``zeus-apple-silicon`` can produce order-of-magnitude figures when the
+ caller has explicitly asked for estimates, and it always reports
+ ``energy_method="tdp_estimate"`` so those figures stay separable from
+ real ones.
"""
_ACTIVE_RATIO = 0.60
t0 = time.monotonic()
- yield result
-
- wall = time.monotonic() - t0
- if wall <= 0:
- result.duration_seconds = 0.0
- return
-
- power_w = self._tdp_watts * _ACTIVE_RATIO
- energy_j = power_w * wall
-
- result.gpu_energy_joules = energy_j * 0.55
- result.cpu_energy_joules = energy_j * 0.25
- result.dram_energy_joules = energy_j * 0.15
- result.ane_energy_joules = energy_j * 0.05
- result.energy_joules = energy_j
- result.duration_seconds = wall
- result.mean_power_watts = power_w
+ try:
+ yield result
+ finally:
+ # No `return` in this block -- see _sample_ioreport.
+ wall = time.monotonic() - t0
+ if wall <= 0:
+ result.duration_seconds = 0.0
+ else:
+ power_w = self._tdp_watts * _ACTIVE_RATIO
+ energy_j = power_w * wall
+ rails = _Rails(
+ cpu=energy_j * 0.25,
+ gpu=energy_j * 0.55,
+ dram=energy_j * 0.15,
+ ane=energy_j * 0.05,
+ )
+ self._apply(result, rails, wall)
def close(self) -> None:
- self._monitor = None
- self._zeus_ok = False
+ self._reader = None
+ self._prev = None
+ _LIVE_MONITORS.discard(self)
__all__ = ["AppleEnergyMonitor"]
diff --git a/src/openjarvis/telemetry/energy_monitor.py b/src/openjarvis/telemetry/energy_monitor.py
index 9b869b69..cc25eda5 100644
--- a/src/openjarvis/telemetry/energy_monitor.py
+++ b/src/openjarvis/telemetry/energy_monitor.py
@@ -12,6 +12,14 @@ from typing import Generator, Optional
logger = logging.getLogger(__name__)
+# Rail sets that derived per-token / per-watt figures can be computed from.
+# ``BASIS_SOC`` means CPU + GPU + ANE (+ DRAM where measured); ``BASIS_GPU``
+# means the GPU rail alone. Energy and power must always be read from the
+# *same* basis -- SoC joules divided by GPU watts is a meaningless number.
+BASIS_SOC = "soc"
+BASIS_GPU = "gpu"
+
+
class EnergyVendor(str, Enum):
"""Supported energy measurement vendors."""
@@ -48,7 +56,11 @@ class EnergySample:
vendor: str = ""
device_name: str = ""
device_count: int = 0
- energy_method: str = "" # "hw_counter", "polling", "rapl", "zeus"
+ # How the numbers above were obtained. "hw_counter" / "polling" (NVIDIA,
+ # AMD), "rapl" (Linux sysfs), "ioreport" (Apple Silicon), or
+ # "tdp_estimate" -- the last being modelled rather than measured, and
+ # opt-in.
+ energy_method: str = ""
# Per-component breakdown (joules)
cpu_energy_joules: float = 0.0
@@ -56,6 +68,26 @@ class EnergySample:
dram_energy_joules: float = 0.0
ane_energy_joules: float = 0.0
+ # Per-component instantaneous power (watts). Populated by monitors that
+ # can attribute power per rail; ``snapshot()`` callers difference the
+ # energy fields above and average these.
+ cpu_power_watts: float = 0.0
+ gpu_power_watts: float = 0.0
+ ane_power_watts: float = 0.0
+
+ # Whole-SoC energy: every rail this monitor measured, summed. On Apple
+ # Silicon each rail above is a partial view -- a model that runs on the
+ # Neural Engine draws almost nothing on the GPU rail -- so per-token
+ # energy must be derived from the sum, not from GPU alone.
+ soc_energy_joules: float = 0.0
+ soc_power_watts: float = 0.0
+
+ # Which rail set the derived per-token/per-watt numbers should come from:
+ # "soc" on Apple Silicon, "gpu" on discrete-GPU hosts. Recorded so
+ # downstream aggregation repeats this monitor's choice instead of
+ # guessing. Empty on samples produced before this field existed.
+ basis: str = ""
+
class EnergyMonitor(ABC):
"""Abstract base class for energy measurement backends.
@@ -76,7 +108,13 @@ class EnergyMonitor(ABC):
@abstractmethod
def energy_method(self) -> str:
- """Return the measurement method: 'hw_counter', 'polling', 'rapl', or 'zeus'."""
+ """Return the measurement method.
+
+ One of ``hw_counter``, ``polling``, ``rapl``, ``ioreport``, or
+ ``tdp_estimate``. Everything but the last is a real measurement;
+ ``tdp_estimate`` is modelled and only reachable when the caller
+ explicitly opts in.
+ """
@abstractmethod
@contextmanager
@@ -103,12 +141,21 @@ class EnergyMonitor(ABC):
def create_energy_monitor(
poll_interval_ms: int = 50,
prefer_vendor: Optional[str] = None,
+ allow_estimates: bool = False,
) -> Optional[EnergyMonitor]:
"""Factory — auto-detect and return the best available EnergyMonitor.
Detection order: NVIDIA > AMD > Apple > CPU RAPL.
If *prefer_vendor* is set, try that vendor first.
+ Every monitor returned by the normal path reports *measured* energy. When
+ *allow_estimates* is set and nothing measurable is present, a modelled
+ estimator may be returned instead — currently only on Apple Silicon, and
+ only when the IOReport backend is missing. Estimators always report a
+ distinct ``energy_method`` so callers can tell modelled joules from
+ measured ones. Off by default: a fabricated number that lands in the
+ telemetry DB is indistinguishable from a real one at query time.
+
Returns ``None`` if no energy monitoring is available.
"""
# Build ordered candidate list — imports are defensive because vendor
@@ -165,10 +212,29 @@ def create_energy_monitor(
logger.debug("Energy monitor candidate failed: %s", exc)
continue
+ if allow_estimates:
+ try:
+ from openjarvis.telemetry.energy_apple import AppleEnergyMonitor
+
+ if AppleEnergyMonitor.estimate_available():
+ logger.warning(
+ "No measurable energy backend found; falling back to a "
+ "modelled Apple Silicon estimate. Install "
+ "`openjarvis[energy-apple]` for real measurements."
+ )
+ return AppleEnergyMonitor(
+ poll_interval_ms=poll_interval_ms,
+ allow_estimates=True,
+ )
+ except Exception as exc:
+ logger.debug("Energy estimate fallback unavailable: %s", exc)
+
return None
__all__ = [
+ "BASIS_GPU",
+ "BASIS_SOC",
"EnergyMonitor",
"EnergySample",
"EnergyVendor",
diff --git a/src/openjarvis/telemetry/flops.py b/src/openjarvis/telemetry/flops.py
index 67e7b06b..c9195700 100644
--- a/src/openjarvis/telemetry/flops.py
+++ b/src/openjarvis/telemetry/flops.py
@@ -28,6 +28,17 @@ MODEL_PARAMS_B: dict[str, float] = {
"llama-3.1-8b": 8.0,
"mistral-7b": 7.0,
"mixtral-8x7b": 47.0,
+ # Apple Foundation Models (on-device, AFM 3). "Core" is a dense ~3B model.
+ # "Core Advanced" is a 20B sparse MoE that activates only 1-4B parameters
+ # per request via Instruction-Following Pruning, so the 4B upper bound is
+ # used here -- FLOPs for that label estimate active compute rather than
+ # measuring it. Apple has not published exact parameter counts.
+ # Core Advanced is listed first because `_get_params_b` falls back to a
+ # prefix scan in insertion order, and "afm-3" is a prefix of both longer
+ # labels.
+ "afm-3-core-advanced": 4.0,
+ "afm-3-core": 3.0,
+ "afm-3": 3.0,
}
diff --git a/src/openjarvis/telemetry/instrumented_engine.py b/src/openjarvis/telemetry/instrumented_engine.py
index e0007768..129bdb3a 100644
--- a/src/openjarvis/telemetry/instrumented_engine.py
+++ b/src/openjarvis/telemetry/instrumented_engine.py
@@ -10,6 +10,7 @@ from typing import Any, Dict, List, Optional, Sequence
from openjarvis.core.events import EventBus, EventType
from openjarvis.core.types import TOKEN_COUNTING_VERSION, Message, TelemetryRecord
from openjarvis.engine._stubs import InferenceEngine, StreamChunk
+from openjarvis.telemetry.energy_monitor import BASIS_GPU
from openjarvis.telemetry.gpu_monitor import GpuSample
# ---------------------------------------------------------------------------
@@ -146,6 +147,9 @@ class InstrumentedEngine(InferenceEngine):
cpu_energy_joules = 0.0
gpu_energy_joules = 0.0
dram_energy_joules = 0.0
+ ane_energy_joules = 0.0
+ soc_energy_joules = 0.0
+ energy_basis = ""
if energy_sample is not None:
# New multi-vendor EnergyMonitor path
@@ -159,6 +163,9 @@ class InstrumentedEngine(InferenceEngine):
cpu_energy_joules = energy_sample.cpu_energy_joules
gpu_energy_joules = energy_sample.gpu_energy_joules
dram_energy_joules = energy_sample.dram_energy_joules
+ ane_energy_joules = energy_sample.ane_energy_joules
+ soc_energy_joules = energy_sample.soc_energy_joules
+ energy_basis = energy_sample.basis
elif gpu_sample is not None:
# Legacy GpuMonitor path
energy_joules = gpu_sample.energy_joules
@@ -168,6 +175,10 @@ class InstrumentedEngine(InferenceEngine):
gpu_temperature_c = gpu_sample.mean_temperature_c
energy_method = "polling"
energy_vendor = "nvidia"
+ # The legacy monitor reads one GPU rail and nothing else.
+ gpu_energy_joules = gpu_sample.energy_joules
+ soc_energy_joules = gpu_sample.energy_joules
+ energy_basis = BASIS_GPU
if ttft > 0:
prefill_latency = ttft
@@ -233,6 +244,9 @@ class InstrumentedEngine(InferenceEngine):
cpu_energy_joules=cpu_energy_joules,
gpu_energy_joules=gpu_energy_joules,
dram_energy_joules=dram_energy_joules,
+ ane_energy_joules=ane_energy_joules,
+ soc_energy_joules=soc_energy_joules,
+ energy_basis=energy_basis,
tokens_per_joule=tokens_per_joule,
# Stamp every new record with the current methodology version
# so the leaderboard aggregator can drop legacy rows (those
@@ -294,6 +308,9 @@ class InstrumentedEngine(InferenceEngine):
"cpu_energy_joules": cpu_energy_joules,
"gpu_energy_joules": gpu_energy_joules,
"dram_energy_joules": dram_energy_joules,
+ "ane_energy_joules": ane_energy_joules,
+ "soc_energy_joules": soc_energy_joules,
+ "energy_basis": energy_basis,
}
return result
@@ -381,6 +398,9 @@ class InstrumentedEngine(InferenceEngine):
cpu_energy_joules = 0.0
gpu_energy_joules = 0.0
dram_energy_joules = 0.0
+ ane_energy_joules = 0.0
+ soc_energy_joules = 0.0
+ energy_basis = ""
if energy_sample is not None:
energy_joules = energy_sample.energy_joules
@@ -393,6 +413,9 @@ class InstrumentedEngine(InferenceEngine):
cpu_energy_joules = energy_sample.cpu_energy_joules
gpu_energy_joules = energy_sample.gpu_energy_joules
dram_energy_joules = energy_sample.dram_energy_joules
+ ane_energy_joules = energy_sample.ane_energy_joules
+ soc_energy_joules = energy_sample.soc_energy_joules
+ energy_basis = energy_sample.basis
elif gpu_sample is not None:
energy_joules = gpu_sample.energy_joules
power_watts = gpu_sample.mean_power_watts
@@ -401,6 +424,10 @@ class InstrumentedEngine(InferenceEngine):
gpu_temperature_c = gpu_sample.mean_temperature_c
energy_method = "polling"
energy_vendor = "nvidia"
+ # The legacy monitor reads one GPU rail and nothing else.
+ gpu_energy_joules = gpu_sample.energy_joules
+ soc_energy_joules = gpu_sample.energy_joules
+ energy_basis = BASIS_GPU
prefill_latency = ttft if ttft > 0 else 0.0
@@ -459,6 +486,9 @@ class InstrumentedEngine(InferenceEngine):
cpu_energy_joules=cpu_energy_joules,
gpu_energy_joules=gpu_energy_joules,
dram_energy_joules=dram_energy_joules,
+ ane_energy_joules=ane_energy_joules,
+ soc_energy_joules=soc_energy_joules,
+ energy_basis=energy_basis,
tokens_per_joule=tokens_per_joule,
# Stamp the methodology version on streaming records too.
token_counting_version=TOKEN_COUNTING_VERSION,
diff --git a/src/openjarvis/telemetry/session.py b/src/openjarvis/telemetry/session.py
index 3946912b..234bc4e5 100644
--- a/src/openjarvis/telemetry/session.py
+++ b/src/openjarvis/telemetry/session.py
@@ -25,8 +25,12 @@ class TelemetrySample:
timestamp_ns: int
gpu_power_w: float = 0.0
cpu_power_w: float = 0.0
+ ane_power_w: float = 0.0
+ soc_power_w: float = 0.0
gpu_energy_j: float = 0.0
cpu_energy_j: float = 0.0
+ ane_energy_j: float = 0.0
+ soc_energy_j: float = 0.0
gpu_util_pct: float = 0.0
gpu_temp_c: float = 0.0
gpu_mem_gb: float = 0.0
@@ -99,12 +103,20 @@ class TelemetrySession:
try:
if self._monitor is not None:
es = self._monitor.snapshot()
+ # `cpu_power_w` was previously hardcoded to 0.0, so every
+ # eval trace reported zero CPU power regardless of vendor.
+ # Monitors that attribute power per rail now fill it in;
+ # those that don't still report 0.0.
sample = TelemetrySample(
timestamp_ns=time.time_ns(),
- gpu_power_w=es.mean_power_watts,
- cpu_power_w=0.0,
+ gpu_power_w=es.gpu_power_watts or es.mean_power_watts,
+ cpu_power_w=es.cpu_power_watts,
+ ane_power_w=es.ane_power_watts,
+ soc_power_w=es.soc_power_watts or es.mean_power_watts,
gpu_energy_j=es.gpu_energy_joules,
cpu_energy_j=es.cpu_energy_joules,
+ ane_energy_j=es.ane_energy_joules,
+ soc_energy_j=es.soc_energy_joules or es.energy_joules,
gpu_util_pct=es.mean_utilization_pct,
gpu_temp_c=es.mean_temperature_c,
gpu_mem_gb=es.mean_memory_used_gb,
diff --git a/src/openjarvis/telemetry/store.py b/src/openjarvis/telemetry/store.py
index d2fdfc97..b93ea873 100644
--- a/src/openjarvis/telemetry/store.py
+++ b/src/openjarvis/telemetry/store.py
@@ -44,6 +44,12 @@ CREATE TABLE IF NOT EXISTS telemetry (
cpu_energy_joules REAL NOT NULL DEFAULT 0.0,
gpu_energy_joules REAL NOT NULL DEFAULT 0.0,
dram_energy_joules REAL NOT NULL DEFAULT 0.0,
+ ane_energy_joules REAL NOT NULL DEFAULT 0.0,
+ soc_energy_joules REAL NOT NULL DEFAULT 0.0,
+ -- Rail set the derived per-token/per-watt columns were computed from:
+ -- 'soc' on Apple Silicon, 'gpu' elsewhere. '' on rows written before the
+ -- column existed, which are GPU-basis by construction.
+ energy_basis TEXT NOT NULL DEFAULT '',
tokens_per_joule REAL NOT NULL DEFAULT 0.0,
energy_per_output_token_joules REAL NOT NULL DEFAULT 0.0,
throughput_per_watt REAL NOT NULL DEFAULT 0.0,
@@ -92,6 +98,7 @@ INSERT INTO telemetry (
throughput_tok_per_sec, prefill_latency_seconds, decode_latency_seconds,
energy_method, energy_vendor, batch_id, is_warmup,
cpu_energy_joules, gpu_energy_joules, dram_energy_joules,
+ ane_energy_joules, soc_energy_joules, energy_basis,
tokens_per_joule,
energy_per_output_token_joules, throughput_per_watt,
prefill_energy_joules, decode_energy_joules,
@@ -104,7 +111,8 @@ INSERT INTO telemetry (
?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
- ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?
+ ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
+ ?, ?, ?, ?
)
"""
@@ -129,6 +137,9 @@ _MIGRATE_COLUMNS = [
("cpu_energy_joules", "REAL NOT NULL DEFAULT 0.0"),
("gpu_energy_joules", "REAL NOT NULL DEFAULT 0.0"),
("dram_energy_joules", "REAL NOT NULL DEFAULT 0.0"),
+ ("ane_energy_joules", "REAL NOT NULL DEFAULT 0.0"),
+ ("soc_energy_joules", "REAL NOT NULL DEFAULT 0.0"),
+ ("energy_basis", "TEXT NOT NULL DEFAULT ''"),
("tokens_per_joule", "REAL NOT NULL DEFAULT 0.0"),
("energy_per_output_token_joules", "REAL NOT NULL DEFAULT 0.0"),
("throughput_per_watt", "REAL NOT NULL DEFAULT 0.0"),
@@ -262,6 +273,9 @@ class TelemetryStore:
rec.cpu_energy_joules,
rec.gpu_energy_joules,
rec.dram_energy_joules,
+ rec.ane_energy_joules,
+ rec.soc_energy_joules,
+ rec.energy_basis,
rec.tokens_per_joule,
rec.energy_per_output_token_joules,
rec.throughput_per_watt,
diff --git a/tests/bench/test_engine_info.py b/tests/bench/test_engine_info.py
new file mode 100644
index 00000000..705489b8
--- /dev/null
+++ b/tests/bench/test_engine_info.py
@@ -0,0 +1,88 @@
+"""Tests for ``engine_info`` — run metadata from self-describing engines.
+
+Benchmark results are written to JSONL, so whatever ``describe()`` returns
+has to survive ``json.dumps``. It does not come from a trusted source: it is
+duck-typed off whatever object the caller passed as the engine.
+"""
+
+from __future__ import annotations
+
+import json
+from unittest.mock import MagicMock
+
+from openjarvis.bench._stubs import engine_info
+
+
+class _Described:
+ def __init__(self, payload):
+ self._payload = payload
+
+ def describe(self):
+ return self._payload
+
+
+def test_returns_none_when_engine_cannot_describe_itself():
+ engine = object()
+ assert engine_info(engine) is None
+
+
+def test_returns_the_described_payload():
+ info = engine_info(
+ _Described(
+ {
+ "engine_id": "afm",
+ "afm_context_size": 4096,
+ "afm_instructions": None,
+ }
+ )
+ )
+ assert info == {
+ "engine_id": "afm",
+ "afm_context_size": 4096,
+ "afm_instructions": None,
+ }
+
+
+def test_mock_engine_does_not_poison_the_jsonl_write():
+ """A MagicMock answers ``describe()`` with another MagicMock.
+
+ Passing that straight into benchmark metadata made the whole run fail with
+ "Object of type MagicMock is not JSON serializable" at write time -- which
+ is exactly the shape of an engine returning something unexpected.
+ """
+ engine = MagicMock()
+ assert engine_info(engine) is None
+
+
+def test_non_dict_payloads_are_rejected():
+ assert engine_info(_Described("just a string")) is None
+ assert engine_info(_Described(None)) is None
+
+
+def test_describe_that_raises_is_not_fatal():
+ class _Broken:
+ def describe(self):
+ raise RuntimeError("host query failed")
+
+ assert engine_info(_Broken()) is None
+
+
+def test_unserializable_values_are_stringified_not_dropped():
+ info = engine_info(_Described({"engine_id": "afm", "handle": object()}))
+ assert info["engine_id"] == "afm"
+ assert isinstance(info["handle"], str)
+ # The whole point: the result must round-trip through JSON.
+ json.dumps(info)
+
+
+def test_result_is_always_json_serializable():
+ payload = {
+ "engine_id": "afm",
+ "nested": {"a": [1, 2, 3]},
+ "bad": MagicMock(),
+ 7: "non-string key is dropped",
+ }
+ info = engine_info(_Described(payload))
+ json.dumps(info)
+ assert 7 not in info
+ assert info["nested"] == {"a": [1, 2, 3]}
diff --git a/tests/cli/test_doctor_labels.py b/tests/cli/test_doctor_labels.py
index 0b570c80..d510ee1e 100644
--- a/tests/cli/test_doctor_labels.py
+++ b/tests/cli/test_doctor_labels.py
@@ -41,7 +41,7 @@ class TestDoctorOptionalLabels:
def test_labels_show_install_hint_on_missing(self) -> None:
"""When a package is missing, show install hint in status."""
- blocker = _selective_import_blocker("zeus")
+ blocker = _selective_import_blocker("zeus_apple_silicon")
with mock.patch("builtins.__import__", side_effect=blocker):
runner = CliRunner()
result = runner.invoke(cli, ["doctor", "--json"])
diff --git a/tests/engine/test_apple_fm_engine.py b/tests/engine/test_apple_fm_engine.py
new file mode 100644
index 00000000..ecd78889
--- /dev/null
+++ b/tests/engine/test_apple_fm_engine.py
@@ -0,0 +1,610 @@
+"""Tests for the in-process Apple Foundation Models engine.
+
+The real ``apple-fm-sdk`` builds Swift bindings and only runs on macOS 26+
+with Apple Intelligence, and there is no macOS CI runner, so these tests
+inject a stub SDK into ``sys.modules`` before importing the engine. The stub
+mirrors the shapes the engine actually depends on:
+
+* ``stream_response`` yields **cumulative** snapshots, not deltas
+* ``token_count`` is async and takes either a value or a transcript
+* ``is_available`` is an *instance* method returning ``(bool, reason)``
+
+The pure helpers in ``_apple_fm_support`` need no stub at all and are tested
+directly.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import importlib
+import sys
+import types
+from typing import Any
+
+import pytest
+
+from openjarvis.core.types import Message, Role
+
+# ---------------------------------------------------------------------------
+# Pure helpers — no SDK needed
+# ---------------------------------------------------------------------------
+
+
+class TestSnapshotAccumulator:
+ def test_cumulative_snapshots_become_deltas(self):
+ from openjarvis.engine._apple_fm_support import snapshot_deltas
+
+ # The SDK re-sends everything generated so far on each iteration.
+ snapshots = ["The", "The capital", "The capital is Paris."]
+ assert snapshot_deltas(snapshots) == [
+ "The",
+ " capital",
+ " is Paris.",
+ ]
+
+ def test_naive_concatenation_would_have_duplicated(self):
+ from openjarvis.engine._apple_fm_support import SnapshotAccumulator
+
+ acc = SnapshotAccumulator()
+ for snap in ["a", "ab", "abc"]:
+ acc.add(snap)
+ # Appending each chunk (what every delta-based backend does) would
+ # give "aababc"; the accumulator tracks the snapshot instead.
+ assert acc.text == "abc"
+
+ def test_repeated_snapshot_yields_nothing(self):
+ from openjarvis.engine._apple_fm_support import snapshot_deltas
+
+ assert snapshot_deltas(["ab", "ab", "abc"]) == ["ab", "c"]
+
+ def test_revised_snapshot_is_reported_whole(self):
+ """Guided generation can revise rather than extend."""
+ from openjarvis.engine._apple_fm_support import snapshot_deltas
+
+ assert snapshot_deltas(["hello wor", "goodbye"]) == ["hello wor", "goodbye"]
+
+
+class TestModelLabels:
+ def test_known_labels_normalize(self):
+ from openjarvis.engine._apple_fm_support import validate_model_label
+
+ assert validate_model_label(" AFM-3-Core ") == "afm-3-core"
+
+ def test_unknown_label_names_the_alternatives(self):
+ from openjarvis.engine._apple_fm_support import validate_model_label
+
+ with pytest.raises(ValueError, match="afm-3-core-advanced"):
+ validate_model_label("afm-4")
+
+
+class TestOptionsAndSampling:
+ def test_max_tokens_maps_to_sdk_name(self):
+ from openjarvis.engine._apple_fm_support import build_options_kwargs
+
+ kwargs = build_options_kwargs({"max_tokens": "128", "temperature": "0.5"})
+ assert kwargs == {"maximum_response_tokens": 128, "temperature": 0.5}
+
+ def test_unset_values_defer_to_the_sdk(self):
+ from openjarvis.engine._apple_fm_support import build_options_kwargs
+
+ assert build_options_kwargs({}) == {}
+
+ def test_sampling_defaults_to_greedy_for_reproducibility(self):
+ from openjarvis.engine._apple_fm_support import parse_sampling_spec
+
+ assert parse_sampling_spec({}) == ("greedy", {})
+
+ def test_random_sampling_carries_its_fields(self):
+ from openjarvis.engine._apple_fm_support import parse_sampling_spec
+
+ mode, kwargs = parse_sampling_spec(
+ {"sampling": "random", "seed": "7", "top": 3}
+ )
+ assert mode == "random"
+ assert kwargs == {"seed": 7, "top": 3}
+
+ def test_unknown_sampling_mode_rejected(self):
+ from openjarvis.engine._apple_fm_support import parse_sampling_spec
+
+ with pytest.raises(ValueError, match="greedy"):
+ parse_sampling_spec({"sampling": "beam"})
+
+
+# ---------------------------------------------------------------------------
+# Stub SDK
+# ---------------------------------------------------------------------------
+
+
+class _Recorder(dict):
+ pass
+
+
+def _install_stub_sdk(
+ *,
+ available: tuple[bool, Any] = (True, None),
+ snapshots: list[str] | None = None,
+ token_counts: list[int] | None = None,
+ raise_on_stream: str | None = None,
+) -> _Recorder:
+ """Inject a fake ``apple_fm_sdk`` and return a call recorder."""
+ rec = _Recorder(
+ options=[],
+ sessions=[],
+ stream_calls=[],
+ token_count_calls=[],
+ models=[],
+ )
+ sdk = types.ModuleType("apple_fm_sdk")
+
+ class SystemLanguageModelUseCase:
+ GENERAL = "general"
+ CONTENT_TAGGING = "content_tagging"
+
+ class SystemLanguageModelGuardrails:
+ DEFAULT = "default"
+ PERMISSIVE_CONTENT_TRANSFORMATIONS = "permissive"
+
+ class SamplingMode:
+ @staticmethod
+ def greedy(**kw):
+ return ("greedy", kw)
+
+ @staticmethod
+ def random(**kw):
+ return ("random", kw)
+
+ class GenerationOptions:
+ def __init__(
+ self, temperature=None, maximum_response_tokens=None, sampling=None
+ ):
+ self.temperature = temperature
+ self.maximum_response_tokens = maximum_response_tokens
+ self.sampling = sampling
+ rec["options"].append(self)
+
+ class ExceededContextWindowSizeError(Exception):
+ pass
+
+ class GuardrailViolationError(Exception):
+ pass
+
+ class RefusalError(Exception):
+ pass
+
+ class UnsupportedLanguageOrLocaleError(Exception):
+ pass
+
+ class SystemLanguageModel:
+ context_size = 4096
+
+ def __init__(self, use_case=None, guardrails=None):
+ self.use_case = use_case
+ self.guardrails = guardrails
+ rec["models"].append(self)
+
+ def is_available(self):
+ return available
+
+ async def token_count(self, value=None, *, instructions=None):
+ rec["token_count_calls"].append(
+ {
+ "value": value,
+ "instructions": instructions,
+ }
+ )
+ counts = token_counts or []
+ idx = len(rec["token_count_calls"]) - 1
+ return counts[idx] if idx < len(counts) else 0
+
+ class _Transcript:
+ pass
+
+ class LanguageModelSession:
+ def __init__(self, instructions=None, model=None, tools=None):
+ self.instructions = instructions
+ self.model = model
+ self.transcript = _Transcript()
+ rec["sessions"].append(self)
+
+ async def stream_response(self, prompt, options=None):
+ rec["stream_calls"].append({"prompt": prompt, "options": options})
+ if raise_on_stream is not None:
+ # Named rather than passed as an instance so callers don't
+ # need a stub SDK in hand before installing one.
+ raise getattr(sdk, raise_on_stream)("stub failure")
+ for snap in snapshots or []:
+ yield snap
+
+ sdk.SystemLanguageModelUseCase = SystemLanguageModelUseCase
+ sdk.SystemLanguageModelGuardrails = SystemLanguageModelGuardrails
+ sdk.SamplingMode = SamplingMode
+ sdk.GenerationOptions = GenerationOptions
+ sdk.SystemLanguageModel = SystemLanguageModel
+ sdk.LanguageModelSession = LanguageModelSession
+ sdk.ExceededContextWindowSizeError = ExceededContextWindowSizeError
+ sdk.GuardrailViolationError = GuardrailViolationError
+ sdk.RefusalError = RefusalError
+ sdk.UnsupportedLanguageOrLocaleError = UnsupportedLanguageOrLocaleError
+
+ sys.modules["apple_fm_sdk"] = sdk
+ return rec
+
+
+def _load_engine_module():
+ """Import a fresh engine module against whatever stub is installed.
+
+ ``tests/conftest.py`` wipes ``EngineRegistry`` between tests, so the
+ module must be re-imported (not just fetched from the registry) for the
+ ``@EngineRegistry.register`` decorator to run again.
+ """
+ sys.modules.pop("openjarvis.engine.apple_fm", None)
+ return importlib.import_module("openjarvis.engine.apple_fm")
+
+
+def engine_mod():
+ created = []
+
+ def _make(**stub_kwargs):
+ rec = _install_stub_sdk(**stub_kwargs)
+ mod = _load_engine_module()
+ created.append(mod)
+ return mod, rec
+
+ yield _make
+ sys.modules.pop("openjarvis.engine.apple_fm", None)
+ sys.modules.pop("apple_fm_sdk", None)
+
+
+def _engine(mod, **kwargs):
+ eng = mod.AppleFMEngine(**kwargs)
+ eng._prepared = True
+ eng._context_size = 4096
+ return eng
+
+
+# ---------------------------------------------------------------------------
+# Engine behaviour
+# ---------------------------------------------------------------------------
+
+
+class TestRegistration:
+ def test_registers_under_afm(self, engine_mod):
+ from openjarvis.core.registry import EngineRegistry
+
+ mod, _ = engine_mod()
+ assert EngineRegistry.get("afm") is mod.AppleFMEngine
+ assert mod.AppleFMEngine.engine_id == "afm"
+
+ def test_rejects_unknown_use_case(self, engine_mod):
+ mod, _ = engine_mod()
+ with pytest.raises(ValueError, match="content_tagging"):
+ mod.AppleFMEngine(use_case="nonsense")
+
+ def test_rejects_unknown_sampling_at_construction(self, engine_mod):
+ """A typo must fail now, not midway through a long run."""
+ mod, _ = engine_mod()
+ with pytest.raises(ValueError, match="greedy"):
+ mod.AppleFMEngine(sampling="beam")
+
+
+class TestServeability:
+ def test_only_serves_afm_labels(self, engine_mod):
+ """Unlike other local engines, this one must not accept any model id.
+
+ There is exactly one on-device model, so accepting arbitrary ids would
+ let engine selection route a Llama request here and answer with a
+ different model entirely.
+ """
+ mod, _ = engine_mod()
+ eng = _engine(mod)
+ assert eng.can_serve("afm-3-core") is True
+ assert eng.can_serve("llama3.2:3b") is False
+
+
+class TestHealth:
+ def test_reports_unavailable_reason(self, engine_mod, caplog):
+ mod, _ = engine_mod(available=(False, types.SimpleNamespace(name="ASSETS")))
+ eng = mod.AppleFMEngine()
+ with caplog.at_level("WARNING"):
+ assert eng.health() is False
+ assert "ASSETS" in caplog.text
+
+
+class TestGenerate:
+ def test_returns_content_and_real_token_counts(self, engine_mod):
+ # token_count call order: transcript-before, prompt, transcript-after
+ mod, rec = engine_mod(
+ snapshots=["Paris", "Paris is", "Paris is the capital."],
+ token_counts=[12, 8, 27],
+ )
+ eng = _engine(mod)
+
+ result = eng.generate(
+ [Message(role=Role.USER, content="Capital of France?")],
+ model="afm-3-core",
+ max_tokens=64,
+ )
+ eng.close()
+
+ assert result["content"] == "Paris is the capital."
+ # prompt = transcript-before (12) + prompt alone (8)
+ assert result["usage"]["prompt_tokens"] == 20
+ # completion = transcript-after (27) - prompt (20)
+ assert result["usage"]["completion_tokens"] == 7
+ assert result["usage"]["total_tokens"] == 27
+ assert result["finish_reason"] == "stop"
+
+ def test_system_message_becomes_session_instructions(self, engine_mod):
+ """Not folded into the prompt text.
+
+ The shim historically prefixed system content onto the prompt, which
+ both loses the distinction the SDK draws and inflates prompt tokens.
+ """
+ mod, rec = engine_mod(snapshots=["ok"], token_counts=[1, 1, 2])
+ eng = _engine(mod)
+ eng.generate(
+ [
+ Message(role=Role.SYSTEM, content="Be terse."),
+ Message(role=Role.USER, content="Hi"),
+ ],
+ model="afm-3",
+ )
+ eng.close()
+
+ assert rec["sessions"][-1].instructions == "Be terse."
+ assert rec["stream_calls"][-1]["prompt"] == "Hi"
+
+ def test_configured_instructions_precede_request_system_message(self, engine_mod):
+ mod, rec = engine_mod(snapshots=["ok"], token_counts=[1, 1, 2])
+ eng = _engine(mod, instructions="Always answer in English.")
+ eng.generate(
+ [
+ Message(role=Role.SYSTEM, content="Be terse."),
+ Message(role=Role.USER, content="Hi"),
+ ],
+ model="afm-3",
+ )
+ eng.close()
+ assert (
+ rec["sessions"][-1].instructions == "Always answer in English.\nBe terse."
+ )
+
+ def test_generation_options_carry_sampling_and_limits(self, engine_mod):
+ mod, rec = engine_mod(snapshots=["x"], token_counts=[0, 0, 1])
+ eng = _engine(mod, sampling="random")
+ eng.generate(
+ [Message(role=Role.USER, content="Hi")],
+ model="afm-3",
+ temperature=0.25,
+ max_tokens=99,
+ )
+ eng.close()
+
+ opts = rec["stream_calls"][-1]["options"]
+ assert opts.temperature == 0.25
+ assert opts.maximum_response_tokens == 99
+ assert opts.sampling == ("random", {})
+
+ def test_each_request_gets_a_fresh_session(self, engine_mod):
+ """Reused sessions accumulate a transcript that overflows the 4096-token
+ context after a handful of requests, and would make per-request energy
+ depend on the requests before it."""
+ mod, rec = engine_mod(snapshots=["x"], token_counts=[0, 0, 1] * 3)
+ eng = _engine(mod)
+ msgs = [Message(role=Role.USER, content="Hi")]
+ for _ in range(3):
+ eng.generate(msgs, model="afm-3")
+ eng.close()
+
+ assert len(rec["sessions"]) == 3
+ assert len({id(s) for s in rec["sessions"]}) == 3
+
+ def test_token_count_failure_degrades_to_zero_not_an_error(self, engine_mod):
+ mod, _ = engine_mod(snapshots=["hi"])
+ eng = _engine(mod)
+
+ async def _boom(*a, **k):
+ raise RuntimeError("counter unavailable")
+
+ eng._ensure_model().token_count = _boom
+
+ result = eng.generate([Message(role=Role.USER, content="Hi")], model="afm-3")
+ eng.close()
+ assert result["content"] == "hi"
+ assert result["usage"]["completion_tokens"] == 0
+
+
+class TestErrorMapping:
+ def test_context_overflow_becomes_a_context_length_error(self, engine_mod):
+ from openjarvis.engine._base import (
+ EngineContextLengthError,
+ looks_like_context_length_error,
+ )
+
+ mod, _ = engine_mod(raise_on_stream="ExceededContextWindowSizeError")
+ eng = _engine(mod)
+ with pytest.raises(EngineContextLengthError) as exc:
+ eng.generate([Message(role=Role.USER, content="x" * 100)], model="afm-3")
+ eng.close()
+ # The shared heuristic must recognise the phrasing too, so callers
+ # branching on text rather than type still get "conversation too long".
+ assert looks_like_context_length_error(str(exc.value))
+ # The SDK formats its messages as "<reason>: <detail>" and passes
+ # None as the detail, so the raw text ends in ": None". That must not
+ # reach the user -- it reads as a broken error path rather than a
+ # plain overflow, and it lands in every affected eval row.
+ assert not str(exc.value).rstrip().endswith("None")
+ assert "4096" in str(exc.value)
+
+ def test_context_error_does_not_repeat_the_sdk_reason(self, engine_mod):
+ from openjarvis.engine._base import EngineContextLengthError
+
+ mod, _ = engine_mod(raise_on_stream="ExceededContextWindowSizeError")
+ eng = _engine(mod)
+ with pytest.raises(EngineContextLengthError) as exc:
+ eng.generate([Message(role=Role.USER, content="x")], model="afm-3")
+ eng.close()
+ # "context window" should appear once, from our own sentence.
+ assert str(exc.value).lower().count("context window") == 1
+
+ def test_refusal_returns_empty_instead_of_killing_the_run(self, engine_mod):
+ """A declined prompt is a per-query outcome, not an engine fault.
+
+ A long eval run must not die because one prompt tripped a guardrail.
+ """
+ mod, _ = engine_mod(
+ raise_on_stream="GuardrailViolationError",
+ token_counts=[3, 5],
+ )
+ eng = _engine(mod)
+ result = eng.generate([Message(role=Role.USER, content="...")], model="afm-3")
+
+ assert result["content"] == ""
+ assert result["finish_reason"] == "content_filter"
+ assert eng._skipped == {"GuardrailViolationError": 1}
+ eng.close()
+
+
+class TestStreaming:
+ def test_stream_yields_deltas_that_reassemble(self, engine_mod):
+ mod, _ = engine_mod(
+ snapshots=["The", "The cap", "The capital is Paris."],
+ token_counts=[4, 6, 20],
+ )
+ eng = _engine(mod)
+
+ async def _collect():
+ return [
+ c
+ async for c in eng.stream(
+ [Message(role=Role.USER, content="Hi")], model="afm-3"
+ )
+ ]
+
+ chunks = asyncio.run(_collect())
+ eng.close()
+ assert "".join(chunks) == "The capital is Paris."
+
+ def test_stream_full_reports_usage_at_end(self, engine_mod):
+ mod, _ = engine_mod(snapshots=["ab", "abc"], token_counts=[4, 6, 15])
+ eng = _engine(mod)
+
+ async def _collect():
+ return [
+ c
+ async for c in eng.stream_full(
+ [Message(role=Role.USER, content="Hi")], model="afm-3"
+ )
+ ]
+
+ chunks = asyncio.run(_collect())
+ eng.close()
+ final = chunks[-1]
+ assert final.finish_reason == "stop"
+ assert final.usage["prompt_tokens"] == 10
+ assert final.usage["completion_tokens"] == 5
+
+
+class TestPrepare:
+ def test_warms_up_and_records_context_size(self, engine_mod):
+ """Warmup runs inside prepare() so the ~10s cold start lands outside
+ the first request's energy window."""
+ mod, rec = engine_mod(snapshots=["w"])
+ eng = mod.AppleFMEngine()
+ eng.prepare("afm-3-core")
+ eng.close()
+
+ assert eng._context_size == 4096
+ assert rec["stream_calls"][0]["prompt"] == "warmup"
+ assert rec["stream_calls"][0]["options"].maximum_response_tokens == 4
+
+ def test_unavailable_model_raises_connection_error(self, engine_mod):
+ from openjarvis.engine._base import EngineConnectionError
+
+ mod, _ = engine_mod(
+ available=(False, types.SimpleNamespace(name="APPLE_INTELLIGENCE_OFF"))
+ )
+ eng = mod.AppleFMEngine()
+ with pytest.raises(EngineConnectionError, match="APPLE_INTELLIGENCE_OFF"):
+ eng.prepare("afm-3-core")
+
+ def test_bad_label_rejected_before_touching_the_model(self, engine_mod):
+ mod, _ = engine_mod()
+ eng = mod.AppleFMEngine()
+ with pytest.raises(ValueError, match="afm-3-core-advanced"):
+ eng.prepare("gpt-4")
+
+
+class TestDescribe:
+ def test_records_what_the_variant_cannot_tell_us(self, engine_mod):
+ mod, _ = engine_mod()
+ eng = _engine(mod, use_case="content_tagging")
+ info = eng.describe()
+ eng.close()
+
+ assert info["engine_id"] == "afm"
+ assert info["afm_use_case"] == "content_tagging"
+ assert info["afm_context_size"] == 4096
+ # The executing variant is not observable, so the host chip is how a
+ # run gets attributed after the fact.
+ assert info["host_chip"]
+ assert "dynamic profile" in info["variant_selection"]
+
+
+# ---------------------------------------------------------------------------
+# On-hardware smoke test
+# ---------------------------------------------------------------------------
+
+
+class TestOnRealHardware:
+ def test_generates_with_real_token_counts(self):
+ sys.modules.pop("apple_fm_sdk", None)
+ sys.modules.pop("openjarvis.engine.apple_fm", None)
+ pytest.importorskip("apple_fm_sdk")
+ mod = importlib.import_module("openjarvis.engine.apple_fm")
+
+ eng = mod.AppleFMEngine(instructions="Answer in one word.")
+ if not eng.health():
+ pytest.skip("Apple Intelligence unavailable on this host")
+
+ result = eng.generate(
+ [Message(role=Role.USER, content="Name one planet.")],
+ model="afm-3-core",
+ max_tokens=16,
+ )
+ eng.close()
+
+ assert result["content"].strip()
+ # The whole point of the 0.2.1 SDK floor: the shim reported 0 here,
+ # which zeroed throughput and every per-token energy figure.
+ assert result["usage"]["completion_tokens"] > 0
+ assert result["usage"]["prompt_tokens"] > 0
+
+
+class TestAfmIsTreatedAsLocal:
+ """`afm` must be recognised as an on-device engine everywhere it matters.
+
+ The Python SDK has no Private Cloud Compute path, so AFM inference never
+ leaves the machine. An engine key missing from these sets produces a false
+ "your data is leaving this machine" signal.
+ """
+
+ def test_data_boundary_audit_classifies_afm_as_local(self):
+ from openjarvis.security.data_boundary_audit import (
+ LOCAL_ENGINE_KEYS,
+ _target_is_cloud,
+ )
+
+ assert "afm" in LOCAL_ENGINE_KEYS
+ assert _target_is_cloud("afm", "afm-3-core") is False
+
+ def test_image_privacy_guard_does_not_warn_for_afm(self):
+ """`jarvis ask -i photo.png --engine afm` must not claim the image
+ leaves the machine."""
+ from openjarvis.cli.ask import LOCAL_ENGINES
+
+ assert "afm" in LOCAL_ENGINES
+ assert "openai" not in LOCAL_ENGINES
diff --git a/tests/engine/test_apple_fm_shim.py b/tests/engine/test_apple_fm_shim.py
index 8d8447cc..e126a7c3 100644
--- a/tests/engine/test_apple_fm_shim.py
+++ b/tests/engine/test_apple_fm_shim.py
@@ -25,6 +25,8 @@ def _install_stub_sdk(
available: tuple[bool, Any] = (True, None),
stream_tokens: list[str] | None = None,
respond_text: str = "Hello from Apple FM.",
+ token_counts: list[int] | None = None,
+ ctx_size: int = 4096,
) -> dict[str, Any]:
"""Inject a fake ``apple_fm_sdk`` module and return a recorder dict.
@@ -32,8 +34,19 @@ def _install_stub_sdk(
args passed to ``stream_response`` / ``respond`` so tests can assert
the shim wires ``options=`` through (PR #377) rather than the old
``max_tokens=`` positional kwarg.
+
+ The stub mirrors the SDK 0.2.1 surface: ``LanguageModelSession`` takes
+ ``instructions`` / ``model``, and ``SystemLanguageModel`` exposes the
+ async ``token_count`` plus ``context_size`` that the shim now uses for
+ real usage numbers.
"""
- rec: dict[str, Any] = {"options": [], "stream_calls": [], "respond_calls": []}
+ rec: dict[str, Any] = {
+ "options": [],
+ "stream_calls": [],
+ "respond_calls": [],
+ "sessions": [],
+ "token_count_calls": [],
+ }
sdk = types.ModuleType("apple_fm_sdk")
class GenerationOptions:
@@ -42,11 +55,49 @@ def _install_stub_sdk(
self.maximum_response_tokens = maximum_response_tokens
rec["options"].append(self)
+ class SystemLanguageModelUseCase:
+ GENERAL = "general"
+ CONTENT_TAGGING = "content_tagging"
+
+ class SystemLanguageModelGuardrails:
+ DEFAULT = "default"
+ PERMISSIVE_CONTENT_TRANSFORMATIONS = "permissive"
+
+ class SamplingMode:
+ @staticmethod
+ def greedy(**kw):
+ return ("greedy", kw)
+
+ @staticmethod
+ def random(**kw):
+ return ("random", kw)
+
class SystemLanguageModel:
+ def __init__(self, use_case: Any = None, guardrails: Any = None):
+ self.use_case = use_case
+ self.guardrails = guardrails
+
+ context_size = ctx_size
+
def is_available(self): # instance method returning (bool, reason)
return available
+ async def token_count(self, value: Any = None, *, instructions: Any = None):
+ rec["token_count_calls"].append(value)
+ counts = token_counts or []
+ idx = len(rec["token_count_calls"]) - 1
+ return counts[idx] if idx < len(counts) else 0
+
+ class _Transcript:
+ pass
+
class LanguageModelSession:
+ def __init__(self, instructions: Any = None, model: Any = None, tools=None):
+ self.instructions = instructions
+ self.model = model
+ self.transcript = _Transcript()
+ rec["sessions"].append(self)
+
async def stream_response(self, prompt: str, *, options: Any = None):
rec["stream_calls"].append({"prompt": prompt, "options": options})
for tok in stream_tokens or []:
@@ -59,6 +110,11 @@ def _install_stub_sdk(
sdk.GenerationOptions = GenerationOptions # type: ignore[attr-defined]
sdk.SystemLanguageModel = SystemLanguageModel # type: ignore[attr-defined]
sdk.LanguageModelSession = LanguageModelSession # type: ignore[attr-defined]
+ sdk.SystemLanguageModelUseCase = SystemLanguageModelUseCase # type: ignore[attr-defined]
+ sdk.SystemLanguageModelGuardrails = ( # type: ignore[attr-defined]
+ SystemLanguageModelGuardrails
+ )
+ sdk.SamplingMode = SamplingMode # type: ignore[attr-defined]
sys.modules["apple_fm_sdk"] = sdk
return rec
@@ -69,7 +125,11 @@ def shim(monkeypatch):
"""Import a fresh copy of the shim against the stub SDK on any platform."""
# The module bails with sys.exit unless it sees Darwin + the SDK import.
monkeypatch.setattr(platform, "system", lambda: "Darwin")
- rec = _install_stub_sdk(stream_tokens=["Sure! ", "Sure! The ", "Sure! The answer."])
+ # token_count order: transcript-before, prompt, transcript-after.
+ rec = _install_stub_sdk(
+ stream_tokens=["Sure! ", "Sure! The ", "Sure! The answer."],
+ token_counts=[6, 4, 17],
+ )
sys.modules.pop("openjarvis.engine.apple_fm_shim", None)
mod = importlib.import_module("openjarvis.engine.apple_fm_shim")
mod = importlib.reload(mod)
@@ -171,3 +231,97 @@ class TestAppleFmShimSdkMigration:
sys.modules.pop("openjarvis.engine.apple_fm_shim", None)
sys.modules.pop("apple_fm_sdk", None)
_ = rec
+
+
+class TestAppleFmShimTokenCounts:
+ """SDK 0.2.1 added ``token_count``; the shim used to hardcode zeros.
+
+ Zeroed ``completion_tokens`` zeroes throughput, per-token energy and
+ tokens-per-joule for every request served through this shim, so these
+ numbers are the point of the upgrade rather than a nicety.
+ """
+
+ def test_non_streaming_reports_real_usage(self, shim):
+ mod, rec = shim
+ client = TestClient(mod.app)
+ resp = client.post(
+ "/v1/chat/completions",
+ json={"messages": [{"role": "user", "content": "hi"}], "stream": False},
+ )
+ usage = resp.json()["usage"]
+ # token_count order: transcript-before (6), prompt alone (4),
+ # transcript-after (17). prompt = 6 + 4; completion = 17 - 10.
+ assert usage["prompt_tokens"] == 10
+ assert usage["completion_tokens"] == 7
+ assert usage["total_tokens"] == 17
+
+ def test_streaming_reports_usage_on_the_final_chunk(self, shim):
+ import json as _json
+
+ mod, _ = shim
+ client = TestClient(mod.app)
+ with client.stream(
+ "POST",
+ "/v1/chat/completions",
+ json={"messages": [{"role": "user", "content": "hi"}], "stream": True},
+ ) as resp:
+ body = "".join(resp.iter_text())
+
+ payloads = [
+ _json.loads(line[len("data:") :].strip())
+ for line in body.splitlines()
+ if line.startswith("data:") and "[DONE]" not in line
+ ]
+ final = payloads[-1]
+ assert final["choices"][0]["finish_reason"] == "stop"
+ assert final["usage"]["completion_tokens"] == 7
+
+ def test_token_count_failure_does_not_fail_the_request(self, shim):
+ mod, _ = shim
+
+ async def _boom(*a, **k):
+ raise RuntimeError("counter unavailable")
+
+ import apple_fm_sdk as stub
+
+ stub.SystemLanguageModel.token_count = _boom
+ client = TestClient(mod.app)
+ resp = client.post(
+ "/v1/chat/completions",
+ json={"messages": [{"role": "user", "content": "hi"}], "stream": False},
+ )
+ assert resp.status_code == 200
+ assert resp.json()["usage"]["completion_tokens"] == 0
+
+
+class TestAppleFmShimInstructions:
+ def test_system_message_becomes_session_instructions(self, shim):
+ """Not prefixed onto the prompt as "[System] ...".
+
+ The SDK draws a distinction between instructions and prompt; folding
+ them together discards it and inflates the prompt-token count.
+ """
+ mod, rec = shim
+ client = TestClient(mod.app)
+ client.post(
+ "/v1/chat/completions",
+ json={
+ "messages": [
+ {"role": "system", "content": "Be terse."},
+ {"role": "user", "content": "hi"},
+ ],
+ "stream": False,
+ },
+ )
+ assert rec["sessions"][-1].instructions == "Be terse."
+ assert rec["respond_calls"][-1]["prompt"] == "hi"
+
+
+class TestAppleFmShimModels:
+ def test_models_endpoint_reports_context_length(self, shim):
+ mod, _ = shim
+ client = TestClient(mod.app)
+ data = client.get("/v1/models").json()
+ entry = data["data"][0]
+ assert entry["id"] == "apple-fm"
+ assert entry["context_length"] == 4096
diff --git a/tests/evals/test_energy_rails_in_results.py b/tests/evals/test_energy_rails_in_results.py
new file mode 100644
index 00000000..76d268d2
--- /dev/null
+++ b/tests/evals/test_energy_rails_in_results.py
@@ -0,0 +1,84 @@
+"""Eval output must record which hardware did the work, and how it was measured.
+
+Before this, an eval result row carried only ``energy_joules`` and
+``power_watts``. On Apple Silicon that hides the entire point of the
+measurement: an ANE-resident model draws almost nothing on the GPU rail, so a
+reader could not tell a Neural Engine run from an idle machine, nor a hardware
+measurement from a modelled estimate. Both distinctions now have to survive
+the trip from the monitor to the JSONL file.
+"""
+
+from __future__ import annotations
+
+from openjarvis.evals.core.types import EvalResult
+
+
+def _apple_result() -> EvalResult:
+ """A result shaped like an AFM run: ANE-dominant, GPU near zero."""
+ return EvalResult(
+ record_id="r1",
+ model_answer="ok",
+ energy_joules=43.104,
+ power_watts=4.69,
+ cpu_energy_joules=27.851,
+ gpu_energy_joules=0.670,
+ dram_energy_joules=8.902,
+ ane_energy_joules=5.681,
+ soc_energy_joules=43.104,
+ energy_basis="soc",
+ energy_method="ioreport",
+ )
+
+
+class TestEvalResultCarriesRails:
+ def test_rail_fields_exist_with_safe_defaults(self):
+ r = EvalResult(record_id="r", model_answer="")
+ for field in (
+ "cpu_energy_joules",
+ "gpu_energy_joules",
+ "dram_energy_joules",
+ "ane_energy_joules",
+ "soc_energy_joules",
+ ):
+ assert getattr(r, field) == 0.0
+ assert r.energy_basis == ""
+ assert r.energy_method == ""
+
+ def test_ane_is_distinguishable_from_gpu(self):
+ """The claim the whole Apple path rests on.
+
+ A GPU-only view of this run would report 0.67 J and look idle; the
+ work actually happened on the Neural Engine.
+ """
+ r = _apple_result()
+ assert r.ane_energy_joules > r.gpu_energy_joules * 8
+ assert r.soc_energy_joules > r.gpu_energy_joules
+
+ def test_method_separates_measurement_from_estimate(self):
+ assert _apple_result().energy_method == "ioreport"
+ estimated = EvalResult(
+ record_id="r", model_answer="", energy_method="tdp_estimate"
+ )
+ assert estimated.energy_method != "ioreport"
+
+
+class TestSerializedRow:
+ def test_written_row_includes_rails_and_method(self):
+ """Guards the JSONL schema itself.
+
+ The runner builds result rows by hand, so a field can exist on
+ EvalResult and still never reach the file — which is exactly how this
+ gap arose.
+ """
+ import inspect
+
+ from openjarvis.evals.core import runner as runner_mod
+
+ source = inspect.getsource(runner_mod)
+ for key in (
+ '"ane_energy_joules": result.ane_energy_joules',
+ '"soc_energy_joules": result.soc_energy_joules',
+ '"energy_basis": result.energy_basis',
+ '"energy_method": result.energy_method',
+ ):
+ assert key in source, f"result row no longer writes {key}"
diff --git a/tests/telemetry/test_energy_apple.py b/tests/telemetry/test_energy_apple.py
index 3c6e5860..a3f00a3d 100644
--- a/tests/telemetry/test_energy_apple.py
+++ b/tests/telemetry/test_energy_apple.py
@@ -1,176 +1,392 @@
-"""Tests for AppleEnergyMonitor -- mock zeus (no real Apple Silicon required)."""
+"""Tests for AppleEnergyMonitor.
+
+The fake backend below mirrors the *real* ``zeus_apple_silicon`` API:
+``AppleEnergyMonitor`` with ``begin_window`` / ``end_window`` /
+``get_cumulative_energy``, returning an object whose rail counters are
+millijoules and are individually absent on chips that do not expose them.
+
+This matters. The previous version of this file invented a
+``zeus.device.soc.apple.AppleSiliconMonitor`` class with joule-valued
+``cpu_energy`` / ``ane_energy`` attributes. No such class or field has ever
+existed in zeus, so the suite validated its own mock while the production
+import failed on every machine and the monitor silently fell back to a
+modelled estimate. Keep this stub faithful to the real library.
+"""
from __future__ import annotations
import time
-import types
-from unittest.mock import MagicMock, patch
+from unittest.mock import patch
import pytest
-from tests.telemetry.energy_test_helpers import (
- assert_sample_result_basics,
-)
+from tests.telemetry.energy_test_helpers import assert_sample_result_basics
# ---------------------------------------------------------------------------
-# Helpers: build a fake zeus module
+# Fake zeus_apple_silicon backend
# ---------------------------------------------------------------------------
-def _make_fake_zeus():
- """Return a fake zeus.device.soc.apple module with AppleSiliconMonitor."""
- # Build the nested module hierarchy
- zeus = types.ModuleType("zeus")
- zeus_device = types.ModuleType("zeus.device")
- zeus_device_soc = types.ModuleType("zeus.device.soc")
- zeus_device_soc_apple = types.ModuleType("zeus.device.soc.apple")
-
- mock_monitor_cls = MagicMock()
- zeus_device_soc_apple.AppleSiliconMonitor = mock_monitor_cls
-
- zeus.device = zeus_device
- zeus_device.soc = zeus_device_soc
- zeus_device_soc.apple = zeus_device_soc_apple
-
- return zeus, zeus_device, zeus_device_soc, zeus_device_soc_apple, mock_monitor_cls
+class FakeMetrics:
+ """Stands in for ``zeus_apple_silicon.zeus_ext.AppleEnergyMetrics``.
+
+ All counters are millijoules. ``None`` means the chip does not expose that
+ rail, which is distinct from a zero reading.
+ """
+
+ def __init__(
+ self,
+ cpu_total_mj=None,
+ gpu_mj=None,
+ gpu_sram_mj=None,
+ dram_mj=None,
+ ane_mj=None,
+ ):
+ self.cpu_total_mj = cpu_total_mj
+ self.gpu_mj = gpu_mj
+ self.gpu_sram_mj = gpu_sram_mj
+ self.dram_mj = dram_mj
+ self.ane_mj = ane_mj
+ # The real object also exposes per-core / per-cluster breakdowns that
+ # cpu_total_mj already subsumes. Present here so a regression that
+ # starts summing them shows up as double counting.
+ self.efficiency_cores_mj = [100, 100]
+ self.performance_cores_mj = [200, 200]
+ self.efficiency_cluster_mj = [200]
+ self.performance_cluster_mj = [400]
+ self.efficiency_core_manager_mj = 50
+ self.performance_core_manager_mj = 50
+
+
+class FakeReader:
+ """Stands in for ``zeus_apple_silicon.AppleEnergyMonitor``."""
+
+ def __init__(self, window_metrics=None, cumulative=None):
+ self._window_metrics = window_metrics or FakeMetrics()
+ self._cumulative = cumulative or []
+ self._cumulative_idx = 0
+ self.begun: list[str] = []
+ self.ended: list[str] = []
+
+ def begin_window(self, key, restart=False):
+ self.begun.append(key)
+
+ def end_window(self, key):
+ if key not in self.begun:
+ raise RuntimeError("No measurement with provided key had been started.")
+ self.ended.append(key)
+ return self._window_metrics
+
+ def get_cumulative_energy(self):
+ if not self._cumulative:
+ return FakeMetrics()
+ idx = min(self._cumulative_idx, len(self._cumulative) - 1)
+ self._cumulative_idx += 1
+ return self._cumulative[idx]
+
+
+def _monitor_with(reader, *, allow_estimates=False, chip="Apple M1 Pro", tdp=30.0):
+ """Build a monitor bypassing __init__'s hardware probe."""
+ from openjarvis.telemetry.energy_apple import AppleEnergyMonitor
+
+ m = AppleEnergyMonitor.__new__(AppleEnergyMonitor)
+ m._poll_interval_ms = 50
+ m._allow_estimates = allow_estimates
+ m._reader = reader
+ m._chip_name = chip
+ m._tdp_watts = tdp
+ m._prev = None
+ return m
# ---------------------------------------------------------------------------
-# Tests: available()
+# available()
# ---------------------------------------------------------------------------
class TestAvailable:
- def test_available_false_on_non_darwin(self):
- with patch("platform.system", return_value="Linux"):
- from openjarvis.telemetry.energy_apple import AppleEnergyMonitor
+ def test_false_on_non_darwin(self):
+ from openjarvis.telemetry.energy_apple import AppleEnergyMonitor
+ with patch("platform.system", return_value="Linux"):
assert AppleEnergyMonitor.available() is False
+ assert AppleEnergyMonitor.estimate_available() is False
+
+ def test_false_without_native_backend(self):
+ """No IOReport backend means nothing measurable — not "available anyway".
- def test_available_true_without_zeus(self):
- """Monitor is available on Apple Silicon even without Zeus."""
+ The old behaviour returned True on any Apple Silicon host, which is
+ how a modelled estimate came to masquerade as a measurement.
+ """
import openjarvis.telemetry.energy_apple as mod
- orig = mod._ZEUS_APPLE_AVAILABLE
- mod._ZEUS_APPLE_AVAILABLE = False
+ orig = mod._NATIVE_AVAILABLE
+ mod._NATIVE_AVAILABLE = False
try:
with (
patch("platform.system", return_value="Darwin"),
patch("platform.machine", return_value="arm64"),
):
- assert mod.AppleEnergyMonitor.available() is True
- monitor = mod.AppleEnergyMonitor.__new__(mod.AppleEnergyMonitor)
- monitor._zeus_ok = False
- assert monitor.energy_method() == "cpu_time_estimate"
+ assert mod.AppleEnergyMonitor.available() is False
+ # ...but the host is still Apple Silicon, so an explicitly
+ # requested estimate remains possible.
+ assert mod.AppleEnergyMonitor.estimate_available() is True
finally:
- mod._ZEUS_APPLE_AVAILABLE = orig
+ mod._NATIVE_AVAILABLE = orig
+
+ def test_false_when_backend_raises(self):
+ """A backend that imports but cannot construct is not available."""
+ import openjarvis.telemetry.energy_apple as mod
+
+ def _boom():
+ raise RuntimeError("IOReport unavailable")
+
+ with (
+ patch("platform.system", return_value="Darwin"),
+ patch("platform.machine", return_value="arm64"),
+ patch.object(mod, "_NATIVE_AVAILABLE", True),
+ patch.object(mod, "_AppleRailReader", _boom),
+ ):
+ assert mod.AppleEnergyMonitor.available() is False
# ---------------------------------------------------------------------------
-# Tests: energy_method()
+# energy_method()
# ---------------------------------------------------------------------------
class TestEnergyMethod:
- def test_returns_zeus(self):
- from openjarvis.telemetry.energy_apple import AppleEnergyMonitor
+ def test_ioreport_when_reader_present(self):
+ assert _monitor_with(FakeReader()).energy_method() == "ioreport"
- monitor = AppleEnergyMonitor.__new__(AppleEnergyMonitor)
- monitor._zeus_ok = True
- assert monitor.energy_method() == "zeus"
+ def test_tdp_estimate_without_reader(self):
+ assert _monitor_with(None).energy_method() == "tdp_estimate"
# ---------------------------------------------------------------------------
-# Tests: sample() component breakdown
+# sample() — rail extraction
# ---------------------------------------------------------------------------
-class TestSampleComponentBreakdown:
- def test_component_energy_extraction(self):
- """Mock begin_window/end_window and verify cpu/gpu/dram/ane extraction."""
- mock_measurement = MagicMock()
- mock_measurement.cpu_energy = 1.5
- mock_measurement.gpu_energy = 3.0
- mock_measurement.dram_energy = 0.5
- mock_measurement.ane_energy = 2.0
-
- mock_zeus_monitor = MagicMock()
- mock_zeus_monitor.begin_window = MagicMock()
- mock_zeus_monitor.end_window = MagicMock(return_value=mock_measurement)
+class TestSampleRails:
+ def test_millijoules_are_converted_to_joules(self):
+ reader = FakeReader(
+ FakeMetrics(
+ cpu_total_mj=10_000,
+ gpu_mj=2_000,
+ gpu_sram_mj=500,
+ dram_mj=1_000,
+ ane_mj=4_000,
+ )
+ )
+ monitor = _monitor_with(reader)
- from openjarvis.telemetry.energy_apple import AppleEnergyMonitor
+ with monitor.sample() as result:
+ time.sleep(0.01)
- monitor = AppleEnergyMonitor.__new__(AppleEnergyMonitor)
- monitor._poll_interval_ms = 50
- monitor._monitor = mock_zeus_monitor
- monitor._zeus_ok = True
- monitor._chip_name = "M1"
+ assert reader.begun and reader.ended
+ assert result.cpu_energy_joules == pytest.approx(10.0)
+ # GPU is gpu_mj + gpu_sram_mj — two separate rails.
+ assert result.gpu_energy_joules == pytest.approx(2.5)
+ assert result.dram_energy_joules == pytest.approx(1.0)
+ assert result.ane_energy_joules == pytest.approx(4.0)
+ assert_sample_result_basics(result, vendor="apple", energy_method="ioreport")
+
+ def test_soc_total_and_basis(self):
+ reader = FakeReader(
+ FakeMetrics(cpu_total_mj=1_000, gpu_mj=2_000, dram_mj=300, ane_mj=700)
+ )
+ monitor = _monitor_with(reader)
with monitor.sample() as result:
- time.sleep(0.01)
+ pass
- mock_zeus_monitor.begin_window.assert_called_once()
- mock_zeus_monitor.end_window.assert_called_once()
+ expected = 1.0 + 2.0 + 0.3 + 0.7
+ assert result.soc_energy_joules == pytest.approx(expected)
+ assert result.energy_joules == pytest.approx(expected)
+ assert result.basis == "soc"
- assert result.cpu_energy_joules == pytest.approx(1.5)
- assert result.gpu_energy_joules == pytest.approx(3.0)
- assert result.dram_energy_joules == pytest.approx(0.5)
- assert result.ane_energy_joules == pytest.approx(2.0)
- assert_sample_result_basics(result, vendor="apple", energy_method="zeus")
+ def test_cpu_total_is_not_double_counted(self):
+ """cpu_total_mj already subsumes the cluster/core breakdowns.
- def test_total_energy_is_sum_of_components(self):
- """total = cpu + gpu + dram + ane."""
- mock_measurement = MagicMock()
- mock_measurement.cpu_energy = 1.0
- mock_measurement.gpu_energy = 2.0
- mock_measurement.dram_energy = 0.3
- mock_measurement.ane_energy = 0.7
+ Verified on an M1 Pro: efficiency_cluster + performance_cluster sums
+ to cpu_total_mj. Summing both would roughly double CPU energy.
+ """
+ reader = FakeReader(FakeMetrics(cpu_total_mj=600))
+ monitor = _monitor_with(reader)
- mock_zeus_monitor = MagicMock()
- mock_zeus_monitor.begin_window = MagicMock()
- mock_zeus_monitor.end_window = MagicMock(return_value=mock_measurement)
+ with monitor.sample() as result:
+ pass
- from openjarvis.telemetry.energy_apple import AppleEnergyMonitor
+ assert result.cpu_energy_joules == pytest.approx(0.6)
+ assert result.soc_energy_joules == pytest.approx(0.6)
- monitor = AppleEnergyMonitor.__new__(AppleEnergyMonitor)
- monitor._poll_interval_ms = 50
- monitor._monitor = mock_zeus_monitor
- monitor._zeus_ok = True
- monitor._chip_name = "M1"
+ def test_absent_rail_is_not_counted_as_zero(self):
+ """A chip without an ANE counter reports no ANE, and the total omits it."""
+ reader = FakeReader(FakeMetrics(cpu_total_mj=2_000, gpu_mj=1_000, ane_mj=None))
+ monitor = _monitor_with(reader)
with monitor.sample() as result:
pass
- expected_total = 1.0 + 2.0 + 0.3 + 0.7
- assert result.energy_joules == pytest.approx(expected_total)
+ assert result.ane_energy_joules == 0.0
+ assert result.soc_energy_joules == pytest.approx(3.0)
+
+ def test_power_is_derived_per_rail(self):
+ reader = FakeReader(FakeMetrics(cpu_total_mj=1_000, gpu_mj=1_000, ane_mj=2_000))
+ monitor = _monitor_with(reader)
+
+ with monitor.sample() as result:
+ time.sleep(0.05)
+
+ assert result.duration_seconds > 0
+ assert result.mean_power_watts == pytest.approx(
+ result.soc_energy_joules / result.duration_seconds
+ )
+ assert result.ane_power_watts == pytest.approx(
+ result.ane_energy_joules / result.duration_seconds
+ )
+
+ def test_body_error_survives_a_failing_end_window(self):
+ """The caller's exception must not be swallowed by cleanup.
+
+ `return` inside a `finally` discards an in-flight exception, so a
+ failed end_window would have hidden whatever the body raised.
+ """
+ reader = FakeReader(FakeMetrics(cpu_total_mj=1_000))
+
+ def _explode(key):
+ raise RuntimeError("IOReport went away")
+
+ reader.end_window = _explode
+ monitor = _monitor_with(reader)
+
+ with pytest.raises(ValueError, match="original"):
+ with monitor.sample():
+ raise ValueError("original failure")
+
+ def test_window_closes_even_when_body_raises(self):
+ reader = FakeReader(FakeMetrics(cpu_total_mj=1_000))
+ monitor = _monitor_with(reader)
+
+ with pytest.raises(ValueError):
+ with monitor.sample():
+ raise ValueError("boom")
+
+ assert reader.ended == reader.begun
# ---------------------------------------------------------------------------
-# Tests: sample() with uninitialized monitor
+# sample() — no backend
# ---------------------------------------------------------------------------
-class TestSampleUninitialized:
- def test_uninitialized_monitor_empty_result(self):
- """When monitor is not initialized, sample yields empty result."""
- from openjarvis.telemetry.energy_apple import AppleEnergyMonitor
+class TestSampleWithoutBackend:
+ def test_reports_zero_rather_than_fabricating(self):
+ """Without a backend and without opt-in, no numbers are invented."""
+ monitor = _monitor_with(None)
+
+ with monitor.sample() as result:
+ time.sleep(0.01)
+
+ assert result.energy_joules == 0.0
+ assert result.soc_energy_joules == 0.0
+ assert result.ane_energy_joules == 0.0
+ assert result.vendor == "apple"
+ assert result.energy_method == "tdp_estimate"
- monitor = AppleEnergyMonitor.__new__(AppleEnergyMonitor)
- monitor._poll_interval_ms = 50
- monitor._monitor = None
- monitor._zeus_ok = False
- monitor._chip_name = "Apple Silicon"
- monitor._tdp_watts = 20.0
+ def test_estimate_is_produced_only_when_opted_in(self):
+ monitor = _monitor_with(None, allow_estimates=True, tdp=30.0)
with monitor.sample() as result:
- pass
+ time.sleep(0.05)
+
+ assert result.energy_joules > 0.0
+ assert result.mean_power_watts == pytest.approx(30.0 * 0.60, rel=1e-6)
+ assert result.energy_method == "tdp_estimate"
+
+
+# ---------------------------------------------------------------------------
+# snapshot()
+# ---------------------------------------------------------------------------
+
+
+class TestSnapshot:
+ def test_reports_cumulative_energy_and_derived_power(self):
+ """Energy fields are cumulative counters; power comes from the delta.
- # CPU-time fallback produces small but non-zero estimates
- assert result.energy_joules >= 0.0
- assert result.cpu_energy_joules >= 0.0
- assert result.gpu_energy_joules >= 0.0
- assert result.dram_energy_joules >= 0.0
- assert result.ane_energy_joules >= 0.0
- assert_sample_result_basics(
- result, vendor="apple", energy_method="cpu_time_estimate"
+ The eval runners difference these across a window
+ (``_compute_energy_delta`` is last-minus-first), so a snapshot must
+ carry the running counter, not a per-interval delta.
+ """
+ reader = FakeReader(
+ cumulative=[
+ FakeMetrics(cpu_total_mj=1_000, gpu_mj=500, ane_mj=100),
+ FakeMetrics(cpu_total_mj=3_000, gpu_mj=700, ane_mj=900),
+ ]
)
+ monitor = _monitor_with(reader)
+
+ first = monitor.snapshot()
+ assert first.cpu_energy_joules == pytest.approx(1.0)
+ # No previous reading to difference against yet.
+ assert first.soc_power_watts == 0.0
+
+ time.sleep(0.05)
+ second = monitor.snapshot()
+ assert second.cpu_energy_joules == pytest.approx(3.0)
+ assert second.soc_power_watts > 0.0
+ assert second.ane_power_watts > 0.0
+ assert second.basis == "soc"
+
+ def test_counter_reset_reports_no_power_rather_than_negative(self):
+ reader = FakeReader(
+ cumulative=[
+ FakeMetrics(cpu_total_mj=5_000),
+ FakeMetrics(cpu_total_mj=10),
+ ]
+ )
+ monitor = _monitor_with(reader)
+
+ monitor.snapshot()
+ second = monitor.snapshot()
+
+ assert second.cpu_power_watts == 0.0
+
+ def test_empty_without_backend(self):
+ result = _monitor_with(None).snapshot()
+
+ assert result.energy_joules == 0.0
+ assert result.vendor == "apple"
+
+
+# ---------------------------------------------------------------------------
+# On-hardware smoke test
+# ---------------------------------------------------------------------------
+
+
+class TestOnRealHardware:
+ def test_measures_more_energy_under_load_than_idle(self):
+ from openjarvis.telemetry.energy_apple import AppleEnergyMonitor
+
+ if not AppleEnergyMonitor.available():
+ pytest.skip("zeus-apple-silicon not installed")
+
+ monitor = AppleEnergyMonitor()
+
+ with monitor.sample() as idle:
+ time.sleep(0.5)
+
+ with monitor.sample() as busy:
+ t = time.time()
+ x = 0
+ while time.time() - t < 0.5:
+ x = (x * 31 + 7) % 1000003
+
+ assert busy.soc_energy_joules > 0
+ # The whole point of measuring rather than modelling: load must move
+ # the number. The old wall-clock × TDP estimate could not tell these
+ # two windows apart.
+ assert busy.mean_power_watts > idle.mean_power_watts
+ monitor.close()
diff --git a/tests/telemetry/test_energy_wiring.py b/tests/telemetry/test_energy_wiring.py
index 06ec1cb6..27ad5a37 100644
--- a/tests/telemetry/test_energy_wiring.py
+++ b/tests/telemetry/test_energy_wiring.py
@@ -19,6 +19,7 @@ from openjarvis.core.config import JarvisConfig
from openjarvis.core.events import EventBus, EventType
from openjarvis.core.types import Message, Role, TelemetryRecord
from openjarvis.telemetry.aggregator import AggregatedStats, TelemetryAggregator
+from openjarvis.telemetry.energy_monitor import BASIS_GPU, EnergySample
from openjarvis.telemetry.instrumented_engine import InstrumentedEngine
from openjarvis.telemetry.store import TelemetryStore
@@ -51,27 +52,40 @@ def _mock_engine(content="Test response"):
def _mock_energy_monitor():
- """Return a mock energy monitor with realistic sample data."""
+ """Return a mock energy monitor with realistic sample data.
+
+ The yielded sample is a real ``EnergySample``, not a ``MagicMock``. A mock
+ returns a ``MagicMock`` for any attribute the test forgot to set, which
+ sqlite then refuses to bind — so a newly added telemetry field would make
+ every record silently fail to persist and surface here as an unrelated
+ "0 records" assertion. A real dataclass picks up new fields with their
+ real defaults.
+ """
monitor = MagicMock()
monitor.close = MagicMock()
- sample = MagicMock()
- sample.energy_joules = 42.5
- sample.mean_power_watts = 250.0
- sample.peak_power_watts = 350.0
- sample.mean_utilization_pct = 78.0
- sample.peak_utilization_pct = 95.0
- sample.mean_memory_used_gb = 16.0
- sample.peak_memory_used_gb = 20.0
- sample.mean_temperature_c = 65.0
- sample.peak_temperature_c = 72.0
- sample.duration_seconds = 0.5
- sample.num_snapshots = 10
- sample.energy_method = "hw_counter"
- sample.vendor = "nvidia"
- sample.cpu_energy_joules = 0.0
- sample.gpu_energy_joules = 42.5
- sample.dram_energy_joules = 0.0
+ sample = EnergySample(
+ energy_joules=42.5,
+ mean_power_watts=250.0,
+ peak_power_watts=350.0,
+ mean_utilization_pct=78.0,
+ peak_utilization_pct=95.0,
+ mean_memory_used_gb=16.0,
+ peak_memory_used_gb=20.0,
+ mean_temperature_c=65.0,
+ peak_temperature_c=72.0,
+ duration_seconds=0.5,
+ num_snapshots=10,
+ energy_method="hw_counter",
+ vendor="nvidia",
+ cpu_energy_joules=0.0,
+ gpu_energy_joules=42.5,
+ dram_energy_joules=0.0,
+ gpu_power_watts=250.0,
+ soc_energy_joules=42.5,
+ soc_power_watts=250.0,
+ basis=BASIS_GPU,
+ )
@contextmanager
def _sample():
diff --git a/uv.lock b/uv.lock
index 5443be5e..3846aa7e 100644
--- a/uv.lock
+++ b/uv.lock
@@ -44,7 +44,7 @@ name = "aiofile"
version = "3.9.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "caio" },
+ { name = "caio", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/67/e2/d7cb819de8df6b5c1968a2756c3cb4122d4fa2b8fc768b53b7c9e5edb646/aiofile-3.9.0.tar.gz", hash = "sha256:e5ad718bb148b265b6df1b3752c4d1d83024b93da9bd599df74b9d9ffcf7919b", size = 17943, upload-time = "2024-10-08T10:39:35.846Z" }
wheels = [
@@ -285,6 +285,17 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/b5/23/1b7dc5f0807f83098183a57db6ee85b2c93b646d74a6e03781c9208aaeb0/apache_tvm_ffi-0.1.9-cp312-abi3-win_amd64.whl", hash = "sha256:d1dcf4c041d5ec05e3da1d545800c33cdbb95c113baa7705085ff79fa262752b", size = 1973200, upload-time = "2026-02-27T19:27:32.367Z" },
]
+[[package]]
+name = "apple-fm-sdk"
+version = "0.2.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "build", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
+ { name = "setuptools", version = "80.10.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'" },
+ { name = "setuptools", version = "82.0.0", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (python_full_version < '3.12' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/91/02/5cfad13c6ab9acd22e81bc083789db02f3551069c10a1551ec60dd04a273/apple_fm_sdk-0.2.1.tar.gz", hash = "sha256:e1801d68ae0517f8524c31723b1dc1a662bf9d06b7a9cc765c9dddb77e647980", size = 107506, upload-time = "2026-06-29T23:30:06.383Z" }
+
[[package]]
name = "astor"
version = "0.8.1"
@@ -369,7 +380,7 @@ name = "authlib"
version = "1.6.8"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "cryptography" },
+ { name = "cryptography", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/6b/6c/c88eac87468c607f88bc24df1f3b31445ee6fc9ba123b09e666adf687cd9/authlib-1.6.8.tar.gz", hash = "sha256:41ae180a17cf672bc784e4a518e5c82687f1fe1e98b0cafaeda80c8e4ab2d1cb", size = 165074, upload-time = "2026-02-14T04:02:17.941Z" }
wheels = [
@@ -617,6 +628,22 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/bd/dc/cadbf64b335a2ee0f31a84d05f34551c2199caa6f639a90c9157b564d0d6/blurhash-1.1.5-py2.py3-none-any.whl", hash = "sha256:96a8686e8b9fced1676550b814e59256214e2d4033202b16c91271ed4d317fec", size = 6632, upload-time = "2025-08-17T10:36:11.404Z" },
]
+[[package]]
+name = "build"
+version = "1.5.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+ { name = "colorama", marker = "(python_full_version < '3.11' and os_name == 'nt' and sys_platform == 'emscripten') or (python_full_version < '3.11' and os_name == 'nt' and sys_platform == 'win32') or (os_name == 'nt' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
+ { name = "importlib-metadata", marker = "python_full_version < '3.10.2' and sys_platform != 'linux'" },
+ { name = "packaging", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
+ { name = "pyproject-hooks", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
+ { name = "tomli", marker = "python_full_version < '3.11' and sys_platform != 'linux'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/78/e0/df5e171f685f82f37b12e1f208064e24244911079d7b767447d1af7e0d70/build-1.5.0.tar.gz", hash = "sha256:302c22c3ba2a0fd5f3911918651341ebb3896176cbdec15bd421f80b1afc7647", size = 89796, upload-time = "2026-04-30T03:18:25.17Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/0d/fe/6bea5c9162869c5beba5d9c8abbed835ec85bf1ec1fba05a3822325c45f3/build-1.5.0-py3-none-any.whl", hash = "sha256:13f3eecb844759ab66efec90ca17639bbf14dc06cb2fdf37a9010322d9c50a6f", size = 26018, upload-time = "2026-04-30T03:18:23.644Z" },
+]
+
[[package]]
name = "cachetools"
version = "7.0.1"
@@ -1157,7 +1184,7 @@ resolution-markers = [
"python_full_version < '3.11' and sys_platform == 'linux'",
]
dependencies = [
- { name = "cuda-pathfinder" },
+ { name = "cuda-pathfinder", marker = "sys_platform == 'linux'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/37/31/bfcc870f69c6a017c4ad5c42316207fc7551940db6f3639aa4466ec5faf3/cuda_bindings-12.9.4-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a022c96b8bd847e8dc0675523431149a4c3e872f440e3002213dbb9e08f0331a", size = 11800959, upload-time = "2025-10-21T14:51:26.458Z" },
@@ -1189,7 +1216,7 @@ resolution-markers = [
"python_full_version < '3.11' and sys_platform != 'linux'",
]
dependencies = [
- { name = "cuda-pathfinder" },
+ { name = "cuda-pathfinder", marker = "sys_platform != 'linux'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/ec/ea/81999d01375645f34596c76eb046b4b36d58cc6fe2bddb2410f8a7b7a827/cuda_bindings-13.2.0-cp310-cp310-win_amd64.whl", hash = "sha256:845025438a1b9e20718b9fb42add3e0eb72e85458bcab3eeb80bfd8f0a9dab33", size = 5600047, upload-time = "2026-03-11T00:12:34.848Z" },
@@ -1217,7 +1244,7 @@ resolution-markers = [
"python_full_version < '3.11' and sys_platform == 'linux'",
]
dependencies = [
- { name = "cuda-bindings", version = "12.9.4", source = { registry = "https://pypi.org/simple" } },
+ { name = "cuda-bindings", version = "12.9.4", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform == 'linux'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/af/f3/6b032a554019cfb3447e671798c1bd3e79b5f1af20d10253f56cea269ef2/cuda_python-12.9.4-py3-none-any.whl", hash = "sha256:d2cacea882a69863f1e7d27ee71d75f0684f4c76910aff839067e4f89c902279", size = 7594, upload-time = "2025-10-21T14:55:12.846Z" },
@@ -1240,8 +1267,8 @@ resolution-markers = [
"python_full_version < '3.11' and sys_platform != 'linux'",
]
dependencies = [
- { name = "cuda-bindings", version = "13.2.0", source = { registry = "https://pypi.org/simple" } },
- { name = "cuda-pathfinder" },
+ { name = "cuda-bindings", version = "13.2.0", source = { registry = "https://pypi.org/simple" }, marker = "sys_platform != 'linux'" },
+ { name = "cuda-pathfinder", marker = "sys_platform != 'linux'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/4a/da/b4dbe129f941afe1c24a09ba53521b78875626763d96414798a74763282f/cuda_python-13.2.0-py3-none-any.whl", hash = "sha256:2f092b0ec13a860115fa595411889ee939ad203450ea4f91e9461b174ea7b084", size = 8145, upload-time = "2026-03-11T13:55:19.143Z" },
@@ -1252,10 +1279,10 @@ name = "cyclopts"
version = "4.6.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "attrs" },
- { name = "docstring-parser" },
- { name = "rich" },
- { name = "rich-rst" },
+ { name = "attrs", marker = "python_full_version >= '3.12'" },
+ { name = "docstring-parser", marker = "python_full_version >= '3.12'" },
+ { name = "rich", marker = "python_full_version >= '3.12'" },
+ { name = "rich-rst", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/49/5c/88a4068c660a096bbe87efc5b7c190080c9e86919c36ec5f092cb08d852f/cyclopts-4.6.0.tar.gz", hash = "sha256:483c4704b953ea6da742e8de15972f405d2e748d19a848a4d61595e8e5360ee5", size = 162724, upload-time = "2026-02-23T15:44:49.286Z" }
wheels = [
@@ -1344,7 +1371,7 @@ name = "deprecation"
version = "2.1.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "packaging" },
+ { name = "packaging", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/5a/d3/8ae2869247df154b64c1884d7346d412fed0c49df84db635aab2d1c40e62/deprecation-2.1.0.tar.gz", hash = "sha256:72b3bde64e5d778694b0cf68178aed03d15e15477116add3fb773e581f9518ff", size = 173788, upload-time = "2020-04-20T14:23:38.738Z" }
wheels = [
@@ -1515,7 +1542,7 @@ name = "exceptiongroup"
version = "1.3.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "typing-extensions", marker = "python_full_version < '3.13'" },
+ { name = "typing-extensions", marker = "python_full_version < '3.11' or python_full_version == '3.12.*'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" }
wheels = [
@@ -1735,26 +1762,26 @@ name = "fastmcp"
version = "3.0.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "authlib" },
- { name = "cyclopts" },
- { name = "exceptiongroup" },
- { name = "httpx" },
- { name = "jsonref" },
- { name = "jsonschema-path" },
- { name = "mcp" },
- { name = "openapi-pydantic" },
- { name = "opentelemetry-api" },
- { name = "packaging" },
- { name = "platformdirs" },
- { name = "py-key-value-aio", extra = ["filetree", "keyring", "memory"] },
- { name = "pydantic", extra = ["email"] },
- { name = "pyperclip" },
- { name = "python-dotenv" },
- { name = "pyyaml" },
- { name = "rich" },
- { name = "uvicorn" },
- { name = "watchfiles" },
- { name = "websockets" },
+ { name = "authlib", marker = "python_full_version >= '3.12'" },
+ { name = "cyclopts", marker = "python_full_version >= '3.12'" },
+ { name = "exceptiongroup", marker = "python_full_version >= '3.12'" },
+ { name = "httpx", marker = "python_full_version >= '3.12'" },
+ { name = "jsonref", marker = "python_full_version >= '3.12'" },
+ { name = "jsonschema-path", marker = "python_full_version >= '3.12'" },
+ { name = "mcp", marker = "python_full_version >= '3.12'" },
+ { name = "openapi-pydantic", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-api", marker = "python_full_version >= '3.12'" },
+ { name = "packaging", marker = "python_full_version >= '3.12'" },
+ { name = "platformdirs", marker = "python_full_version >= '3.12'" },
+ { name = "py-key-value-aio", extra = ["filetree", "keyring", "memory"], marker = "python_full_version >= '3.12'" },
+ { name = "pydantic", extra = ["email"], marker = "python_full_version >= '3.12'" },
+ { name = "pyperclip", marker = "python_full_version >= '3.12'" },
+ { name = "python-dotenv", marker = "python_full_version >= '3.12'" },
+ { name = "pyyaml", marker = "python_full_version >= '3.12'" },
+ { name = "rich", marker = "python_full_version >= '3.12'" },
+ { name = "uvicorn", marker = "python_full_version >= '3.12'" },
+ { name = "watchfiles", marker = "python_full_version >= '3.12'" },
+ { name = "websockets", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/11/6b/1a7ec89727797fb07ec0928e9070fa2f45e7b35718e1fe01633a34c35e45/fastmcp-3.0.2.tar.gz", hash = "sha256:6bd73b4a3bab773ee6932df5249dcbcd78ed18365ed0aeeb97bb42702a7198d7", size = 17239351, upload-time = "2026-02-22T16:32:28.843Z" }
wheels = [
@@ -2555,7 +2582,7 @@ name = "jaraco-classes"
version = "3.4.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "more-itertools" },
+ { name = "more-itertools", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/06/c0/ed4a27bc5571b99e3cff68f8a9fa5b56ff7df1c2251cc715a652ddd26402/jaraco.classes-3.4.0.tar.gz", hash = "sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd", size = 11780, upload-time = "2024-03-31T07:27:36.643Z" }
wheels = [
@@ -2576,7 +2603,7 @@ name = "jaraco-functools"
version = "4.4.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "more-itertools" },
+ { name = "more-itertools", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/0f/27/056e0638a86749374d6f57d0b0db39f29509cce9313cf91bdc0ac4d91084/jaraco_functools-4.4.0.tar.gz", hash = "sha256:da21933b0417b89515562656547a77b4931f98176eb173644c0d35032a33d6bb", size = 19943, upload-time = "2025-12-21T09:29:43.6Z" }
wheels = [
@@ -2732,9 +2759,9 @@ name = "jsonschema-path"
version = "0.4.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "pathable" },
- { name = "pyyaml" },
- { name = "referencing" },
+ { name = "pathable", marker = "python_full_version >= '3.12'" },
+ { name = "pyyaml", marker = "python_full_version >= '3.12'" },
+ { name = "referencing", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b3/da/1ebeb1c0ff579c330e200e8b06e6200653e3d0758136d8bd86762d63e7de/jsonschema_path-0.4.2.tar.gz", hash = "sha256:5f5ff183150030ea24bb51cf1ddac9bf5dbf030272e2792a7ffe8262f7eea2a5", size = 13417, upload-time = "2026-02-23T16:21:36.602Z" }
wheels = [
@@ -2758,12 +2785,12 @@ name = "keyring"
version = "25.7.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "jaraco-classes" },
- { name = "jaraco-context" },
- { name = "jaraco-functools" },
- { name = "jeepney", marker = "sys_platform == 'linux'" },
- { name = "pywin32-ctypes", marker = "sys_platform == 'win32'" },
- { name = "secretstorage", marker = "sys_platform == 'linux'" },
+ { name = "jaraco-classes", marker = "python_full_version >= '3.12'" },
+ { name = "jaraco-context", marker = "python_full_version >= '3.12'" },
+ { name = "jaraco-functools", marker = "python_full_version >= '3.12'" },
+ { name = "jeepney", marker = "python_full_version >= '3.12' and sys_platform == 'linux'" },
+ { name = "pywin32-ctypes", marker = "python_full_version >= '3.12' and sys_platform == 'win32'" },
+ { name = "secretstorage", marker = "python_full_version >= '3.12' and sys_platform == 'linux'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/43/4b/674af6ef2f97d56f0ab5153bf0bfa28ccb6c3ed4d1babf4305449668807b/keyring-25.7.0.tar.gz", hash = "sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b", size = 63516, upload-time = "2025-11-16T16:26:09.482Z" }
wheels = [
@@ -2901,22 +2928,22 @@ name = "lmnr"
version = "0.7.42"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "grpcio" },
- { name = "httpx" },
- { name = "opentelemetry-api" },
- { name = "opentelemetry-exporter-otlp-proto-grpc" },
- { name = "opentelemetry-exporter-otlp-proto-http" },
- { name = "opentelemetry-instrumentation" },
- { name = "opentelemetry-instrumentation-threading" },
- { name = "opentelemetry-sdk" },
- { name = "opentelemetry-semantic-conventions" },
- { name = "opentelemetry-semantic-conventions-ai" },
- { name = "orjson" },
- { name = "packaging" },
- { name = "pydantic" },
- { name = "python-dotenv" },
- { name = "tenacity" },
- { name = "tqdm" },
+ { name = "grpcio", marker = "python_full_version >= '3.12'" },
+ { name = "httpx", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-api", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-exporter-otlp-proto-grpc", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-exporter-otlp-proto-http", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-instrumentation", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-instrumentation-threading", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-sdk", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-semantic-conventions", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-semantic-conventions-ai", marker = "python_full_version >= '3.12'" },
+ { name = "orjson", marker = "python_full_version >= '3.12'" },
+ { name = "packaging", marker = "python_full_version >= '3.12'" },
+ { name = "pydantic", marker = "python_full_version >= '3.12'" },
+ { name = "python-dotenv", marker = "python_full_version >= '3.12'" },
+ { name = "tenacity", marker = "python_full_version >= '3.12'" },
+ { name = "tqdm", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/0a/95/5b9dac7022e81494871aa228ccb0578668781cbc4241b59bd89ebe37536e/lmnr-0.7.42.tar.gz", hash = "sha256:2bea334ca201470120f1c5dd18eccf6aee503c8d962addda2761dfa681f74845", size = 240415, upload-time = "2026-02-23T14:23:24.39Z" }
wheels = [
@@ -3402,7 +3429,7 @@ name = "mlx"
version = "0.31.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "mlx-metal" },
+ { name = "mlx-metal", marker = "sys_platform == 'darwin'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/29/7c/c16d52494a1ba6d90443f31fa26bc810bf878d532dfa9a7a13f49ef9542d/mlx-0.31.2-cp310-cp310-macosx_14_0_arm64.whl", hash = "sha256:b29cf940f34205f09bb552ac60465ae833c4ae640b52777c6d725ddbad8461ca", size = 586942, upload-time = "2026-04-22T03:14:21.97Z" },
@@ -3424,13 +3451,13 @@ name = "mlx-lm"
version = "0.31.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "jinja2" },
- { name = "mlx" },
- { name = "numpy" },
- { name = "protobuf" },
- { name = "pyyaml" },
- { name = "sentencepiece" },
- { name = "transformers" },
+ { name = "jinja2", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
+ { name = "mlx", marker = "sys_platform == 'darwin'" },
+ { name = "numpy", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
+ { name = "protobuf", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
+ { name = "pyyaml", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
+ { name = "sentencepiece", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
+ { name = "transformers", marker = "(python_full_version < '3.11' and sys_platform == 'emscripten') or (python_full_version < '3.11' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32')" },
]
sdist = { url = "https://files.pythonhosted.org/packages/84/94/9a38d6b0c6fcca995b9136c94eb7da1e9c5165652edf228b96b29960fa7a/mlx_lm-0.31.3.tar.gz", hash = "sha256:61eb0e3ba09444f77f874aff295401d7ccd20b39495cbbce0c782a15474ce733", size = 304318, upload-time = "2026-04-22T07:37:27.922Z" }
wheels = [
@@ -3851,7 +3878,7 @@ name = "nvidia-cudnn-cu12"
version = "9.10.2.21"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "nvidia-cublas-cu12" },
+ { name = "nvidia-cublas-cu12", marker = "sys_platform == 'linux'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/ba/51/e123d997aa098c61d029f76663dedbfb9bc8dcf8c60cbd6adbe42f76d049/nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:949452be657fa16687d0930933f032835951ef0892b37d2d53824d1a84dc97a8", size = 706758467, upload-time = "2025-06-06T21:54:08.597Z" },
@@ -3881,7 +3908,7 @@ name = "nvidia-cufft-cu12"
version = "11.3.3.83"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "nvidia-nvjitlink-cu12" },
+ { name = "nvidia-nvjitlink-cu12", marker = "sys_platform == 'linux'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/1f/13/ee4e00f30e676b66ae65b4f08cb5bcbb8392c03f54f2d5413ea99a5d1c80/nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4d2dd21ec0b88cf61b62e6b43564355e5222e4a3fb394cac0db101f2dd0d4f74", size = 193118695, upload-time = "2025-03-07T01:45:27.821Z" },
@@ -3908,9 +3935,9 @@ name = "nvidia-cusolver-cu12"
version = "11.7.3.90"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "nvidia-cublas-cu12" },
- { name = "nvidia-cusparse-cu12" },
- { name = "nvidia-nvjitlink-cu12" },
+ { name = "nvidia-cublas-cu12", marker = "sys_platform == 'linux'" },
+ { name = "nvidia-cusparse-cu12", marker = "sys_platform == 'linux'" },
+ { name = "nvidia-nvjitlink-cu12", marker = "sys_platform == 'linux'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/85/48/9a13d2975803e8cf2777d5ed57b87a0b6ca2cc795f9a4f59796a910bfb80/nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:4376c11ad263152bd50ea295c05370360776f8c3427b30991df774f9fb26c450", size = 267506905, upload-time = "2025-03-07T01:47:16.273Z" },
@@ -3921,7 +3948,7 @@ name = "nvidia-cusparse-cu12"
version = "12.5.8.93"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "nvidia-nvjitlink-cu12" },
+ { name = "nvidia-nvjitlink-cu12", marker = "sys_platform == 'linux'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/c2/f5/e1854cb2f2bcd4280c44736c93550cc300ff4b8c95ebe370d0aa7d2b473d/nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1ec05d76bbbd8b61b06a80e1eaf8cf4959c3d4ce8e711b65ebd0443bb0ebb13b", size = 288216466, upload-time = "2025-03-07T01:48:13.779Z" },
@@ -4095,7 +4122,7 @@ name = "openapi-pydantic"
version = "0.5.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "pydantic" },
+ { name = "pydantic", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/02/2e/58d83848dd1a79cb92ed8e63f6ba901ca282c5f09d04af9423ec26c56fd7/openapi_pydantic-0.5.1.tar.gz", hash = "sha256:ff6835af6bde7a459fb93eb93bb92b8749b754fc6e51b2f1590a19dc3005ee0d", size = 60892, upload-time = "2025-01-08T19:29:27.083Z" }
wheels = [
@@ -4125,17 +4152,17 @@ name = "openhands-sdk"
version = "1.11.5"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "deprecation" },
- { name = "fastmcp" },
- { name = "filelock" },
- { name = "httpx" },
- { name = "litellm" },
- { name = "lmnr" },
- { name = "pydantic" },
- { name = "python-frontmatter" },
- { name = "python-json-logger" },
- { name = "tenacity" },
- { name = "websockets" },
+ { name = "deprecation", marker = "python_full_version >= '3.12'" },
+ { name = "fastmcp", marker = "python_full_version >= '3.12'" },
+ { name = "filelock", marker = "python_full_version >= '3.12'" },
+ { name = "httpx", marker = "python_full_version >= '3.12'" },
+ { name = "litellm", marker = "python_full_version >= '3.12'" },
+ { name = "lmnr", marker = "python_full_version >= '3.12'" },
+ { name = "pydantic", marker = "python_full_version >= '3.12'" },
+ { name = "python-frontmatter", marker = "python_full_version >= '3.12'" },
+ { name = "python-json-logger", marker = "python_full_version >= '3.12'" },
+ { name = "tenacity", marker = "python_full_version >= '3.12'" },
+ { name = "websockets", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/3a/fc/cfb73768099be94c9f92b2e160f29d1f6d3a4dd84fea5e33a2b0984449cc/openhands_sdk-1.11.5.tar.gz", hash = "sha256:dd6225876b7b8dbb6c608559f2718c3d0bf44d0bb741e990b185c6cdc5150c5a", size = 295069, upload-time = "2026-02-20T22:16:47.102Z" }
wheels = [
@@ -4161,6 +4188,9 @@ dependencies = [
]
[package.optional-dependencies]
+afm = [
+ { name = "apple-fm-sdk", marker = "sys_platform == 'darwin'" },
+]
browser = [
{ name = "playwright" },
]
@@ -4249,13 +4279,13 @@ docs = [
energy-all = [
{ name = "amdsmi" },
{ name = "pynvml" },
- { name = "zeus-ml" },
+ { name = "zeus-apple-silicon", marker = "sys_platform == 'darwin'" },
]
energy-amd = [
{ name = "amdsmi" },
]
energy-apple = [
- { name = "zeus-ml" },
+ { name = "zeus-apple-silicon", marker = "sys_platform == 'darwin'" },
]
eval-sheets = [
{ name = "google-auth" },
@@ -4372,6 +4402,7 @@ requires-dist = [
{ name = "amdsmi", marker = "extra == 'energy-all'", specifier = ">=6.1" },
{ name = "amdsmi", marker = "extra == 'energy-amd'", specifier = ">=6.1" },
{ name = "anthropic", marker = "extra == 'inference-cloud'", specifier = ">=0.30" },
+ { name = "apple-fm-sdk", marker = "sys_platform == 'darwin' and extra == 'afm'", specifier = ">=0.2.1" },
{ name = "click", specifier = ">=8" },
{ name = "colbert-ai", marker = "extra == 'memory-colbert'", specifier = ">=0.2" },
{ name = "croniter", marker = "extra == 'scheduler'", specifier = ">=2.0" },
@@ -4467,11 +4498,11 @@ requires-dist = [
{ name = "wandb", marker = "extra == 'eval-wandb'", specifier = ">=0.17" },
{ name = "wasmtime", marker = "extra == 'sandbox-wasm'", specifier = ">=25" },
{ name = "websockets", specifier = ">=15.0.1" },
- { name = "zeus-ml", extras = ["apple"], marker = "extra == 'energy-all'" },
- { name = "zeus-ml", extras = ["apple"], marker = "extra == 'energy-apple'" },
+ { name = "zeus-apple-silicon", marker = "sys_platform == 'darwin' and extra == 'energy-all'", specifier = ">=1.1.0" },
+ { name = "zeus-apple-silicon", marker = "sys_platform == 'darwin' and extra == 'energy-apple'", specifier = ">=1.1.0" },
{ name = "zulip", marker = "extra == 'channel-zulip'", specifier = ">=0.9" },
]
-provides-extras = ["browser", "channel-discord", "channel-gmail", "channel-line", "channel-mastodon", "channel-messenger", "channel-nostr", "channel-reddit", "channel-rocketchat", "channel-slack", "channel-telegram", "channel-twilio", "channel-twitch", "channel-twitter", "channel-viber", "channel-xmpp", "channel-zulip", "dashboard", "desktop", "dev", "docs", "energy-all", "energy-amd", "energy-apple", "eval-sheets", "eval-wandb", "framework-comparison", "gpu-metrics", "inference-cloud", "inference-gemma", "inference-google", "inference-litellm", "inference-mlx", "inference-vllm", "learning-dspy", "learning-gepa", "media", "memory-bm25", "memory-colbert", "memory-faiss", "memory-pdf", "mining-pearl-cpu", "mining-pearl-vllm", "openhands", "orchestrator-training", "pdf", "sandbox-docker", "sandbox-wasm", "scheduler", "security-signing", "server", "speech", "speech-deepgram", "tools-search"]
+provides-extras = ["afm", "browser", "channel-discord", "channel-gmail", "channel-line", "channel-mastodon", "channel-messenger", "channel-nostr", "channel-reddit", "channel-rocketchat", "channel-slack", "channel-telegram", "channel-twilio", "channel-twitch", "channel-twitter", "channel-viber", "channel-xmpp", "channel-zulip", "dashboard", "desktop", "dev", "docs", "energy-all", "energy-amd", "energy-apple", "eval-sheets", "eval-wandb", "framework-comparison", "gpu-metrics", "inference-cloud", "inference-gemma", "inference-google", "inference-litellm", "inference-mlx", "inference-vllm", "learning-dspy", "learning-gepa", "media", "memory-bm25", "memory-colbert", "memory-faiss", "memory-pdf", "mining-pearl-cpu", "mining-pearl-vllm", "openhands", "orchestrator-training", "pdf", "sandbox-docker", "sandbox-wasm", "scheduler", "security-signing", "server", "speech", "speech-deepgram", "tools-search"]
[package.metadata.requires-dev]
desktop-native = [{ name = "openjarvis-rust", directory = "rust/crates/openjarvis-python" }]
@@ -4561,10 +4592,10 @@ name = "opentelemetry-instrumentation"
version = "0.60b1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "opentelemetry-api" },
- { name = "opentelemetry-semantic-conventions" },
- { name = "packaging" },
- { name = "wrapt" },
+ { name = "opentelemetry-api", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-semantic-conventions", marker = "python_full_version >= '3.12'" },
+ { name = "packaging", marker = "python_full_version >= '3.12'" },
+ { name = "wrapt", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/41/0f/7e6b713ac117c1f5e4e3300748af699b9902a2e5e34c9cf443dde25a01fa/opentelemetry_instrumentation-0.60b1.tar.gz", hash = "sha256:57ddc7974c6eb35865af0426d1a17132b88b2ed8586897fee187fd5b8944bd6a", size = 31706, upload-time = "2025-12-11T13:36:42.515Z" }
wheels = [
@@ -4576,9 +4607,9 @@ name = "opentelemetry-instrumentation-threading"
version = "0.60b1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "opentelemetry-api" },
- { name = "opentelemetry-instrumentation" },
- { name = "wrapt" },
+ { name = "opentelemetry-api", marker = "python_full_version >= '3.12'" },
+ { name = "opentelemetry-instrumentation", marker = "python_full_version >= '3.12'" },
+ { name = "wrapt", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/9b/0a/e36123ec4c0910a3936b92982545a53e9bca5b26a28df06883751a783f84/opentelemetry_instrumentation_threading-0.60b1.tar.gz", hash = "sha256:20b18a68abe5801fa9474336b7c27487d4af3e00b66f6a8734e4fdd75c8b0b43", size = 8768, upload-time = "2025-12-11T13:37:16.29Z" }
wheels = [
@@ -4784,10 +4815,10 @@ resolution-markers = [
"python_full_version < '3.11' and sys_platform != 'linux'",
]
dependencies = [
- { name = "numpy" },
- { name = "python-dateutil" },
- { name = "pytz" },
- { name = "tzdata" },
+ { name = "numpy", marker = "python_full_version < '3.11'" },
+ { name = "python-dateutil", marker = "python_full_version < '3.11'" },
+ { name = "pytz", marker = "python_full_version < '3.11'" },
+ { name = "tzdata", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/33/01/d40b85317f86cf08d853a4f495195c73815fdf205eef3993821720274518/pandas-2.3.3.tar.gz", hash = "sha256:e05e1af93b977f7eafa636d043f9f94c7ee3ac81af99c13508215942e64c993b", size = 4495223, upload-time = "2025-09-29T23:34:51.853Z" }
wheels = [
@@ -4846,9 +4877,9 @@ resolution-markers = [
"python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'",
]
dependencies = [
- { name = "numpy" },
- { name = "python-dateutil" },
- { name = "tzdata", marker = "sys_platform == 'emscripten' or sys_platform == 'win32'" },
+ { name = "numpy", marker = "python_full_version >= '3.11'" },
+ { name = "python-dateutil", marker = "python_full_version >= '3.11'" },
+ { name = "tzdata", marker = "(python_full_version >= '3.11' and sys_platform == 'emscripten') or (python_full_version >= '3.11' and sys_platform == 'win32')" },
]
sdist = { url = "https://files.pythonhosted.org/packages/de/da/b1dc0481ab8d55d0f46e343cfe67d4551a0e14fcee52bd38ca1bd73258d8/pandas-3.0.0.tar.gz", hash = "sha256:0facf7e87d38f721f0af46fe70d97373a37701b1c09f7ed7aeeb292ade5c050f", size = 4633005, upload-time = "2026-01-21T15:52:04.726Z" }
wheels = [
@@ -5305,8 +5336,8 @@ name = "py-key-value-aio"
version = "0.4.4"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "beartype" },
- { name = "typing-extensions" },
+ { name = "beartype", marker = "python_full_version >= '3.12'" },
+ { name = "typing-extensions", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/04/3c/0397c072a38d4bc580994b42e0c90c5f44f679303489e4376289534735e5/py_key_value_aio-0.4.4.tar.gz", hash = "sha256:e3012e6243ed7cc09bb05457bd4d03b1ba5c2b1ca8700096b3927db79ffbbe55", size = 92300, upload-time = "2026-02-16T21:21:43.245Z" }
wheels = [
@@ -5315,14 +5346,14 @@ wheels = [
[package.optional-dependencies]
filetree = [
- { name = "aiofile" },
- { name = "anyio" },
+ { name = "aiofile", marker = "python_full_version >= '3.12'" },
+ { name = "anyio", marker = "python_full_version >= '3.12'" },
]
keyring = [
- { name = "keyring" },
+ { name = "keyring", marker = "python_full_version >= '3.12'" },
]
memory = [
- { name = "cachetools" },
+ { name = "cachetools", marker = "python_full_version >= '3.12'" },
]
[[package]]
@@ -5884,6 +5915,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/df/80/fc9d01d5ed37ba4c42ca2b55b4339ae6e200b456be3a1aaddf4a9fa99b8c/pyperclip-1.11.0-py3-none-any.whl", hash = "sha256:299403e9ff44581cb9ba2ffeed69c7aa96a008622ad0c46cb575ca75b5b84273", size = 11063, upload-time = "2025-09-26T14:40:36.069Z" },
]
+[[package]]
+name = "pyproject-hooks"
+version = "1.2.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/e7/82/28175b2414effca1cdac8dc99f76d660e7a4fb0ceefa4b4ab8f5f6742925/pyproject_hooks-1.2.0.tar.gz", hash = "sha256:1e859bd5c40fae9448642dd871adf459e5e2084186e8d2c2a79a824c970da1f8", size = 19228, upload-time = "2024-09-29T09:24:13.293Z" }
+wheels = [
+ { url = "https://files.pythonhosted.org/packages/bd/24/12818598c362d7f300f18e74db45963dbcb85150324092410c8b49405e42/pyproject_hooks-1.2.0-py3-none-any.whl", hash = "sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913", size = 10216, upload-time = "2024-09-29T09:24:11.978Z" },
+]
+
[[package]]
name = "pytest"
version = "9.0.2"
@@ -5982,7 +6022,7 @@ name = "python-frontmatter"
version = "1.1.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "pyyaml" },
+ { name = "pyyaml", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/96/de/910fa208120314a12f9a88ea63e03707261692af782c99283f1a2c8a5e6f/python-frontmatter-1.1.0.tar.gz", hash = "sha256:7118d2bd56af9149625745c58c9b51fb67e8d1294a0c76796dafdc72c36e5f6d", size = 16256, upload-time = "2024-01-16T18:50:04.052Z" }
wheels = [
@@ -6395,8 +6435,8 @@ name = "rich-rst"
version = "1.3.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "docutils" },
- { name = "rich" },
+ { name = "docutils", marker = "python_full_version >= '3.12'" },
+ { name = "rich", marker = "python_full_version >= '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/bc/6d/a506aaa4a9eaa945ed8ab2b7347859f53593864289853c5d6d62b77246e0/rich_rst-1.3.2.tar.gz", hash = "sha256:a1196fdddf1e364b02ec68a05e8ff8f6914fee10fbca2e6b6735f166bb0da8d4", size = 14936, upload-time = "2025-10-14T16:49:45.332Z" }
wheels = [
@@ -6686,10 +6726,10 @@ resolution-markers = [
"python_full_version < '3.11' and sys_platform != 'linux'",
]
dependencies = [
- { name = "joblib" },
- { name = "numpy" },
- { name = "scipy", version = "1.15.3", source = { registry = "https://pypi.org/simple" } },
- { name = "threadpoolctl" },
+ { name = "joblib", marker = "python_full_version < '3.11'" },
+ { name = "numpy", marker = "python_full_version < '3.11'" },
+ { name = "scipy", version = "1.15.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
+ { name = "threadpoolctl", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/98/c2/a7855e41c9d285dfe86dc50b250978105dce513d6e459ea66a6aeb0e1e0c/scikit_learn-1.7.2.tar.gz", hash = "sha256:20e9e49ecd130598f1ca38a1d85090e1a600147b9c02fa6f15d69cb53d968fda", size = 7193136, upload-time = "2025-09-09T08:21:29.075Z" }
wheels = [
@@ -6739,10 +6779,10 @@ resolution-markers = [
"python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'",
]
dependencies = [
- { name = "joblib" },
- { name = "numpy" },
- { name = "scipy", version = "1.17.0", source = { registry = "https://pypi.org/simple" } },
- { name = "threadpoolctl" },
+ { name = "joblib", marker = "python_full_version >= '3.11'" },
+ { name = "numpy", marker = "python_full_version >= '3.11'" },
+ { name = "scipy", version = "1.17.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
+ { name = "threadpoolctl", marker = "python_full_version >= '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/0e/d4/40988bf3b8e34feec1d0e6a051446b1f66225f8529b9309becaeef62b6c4/scikit_learn-1.8.0.tar.gz", hash = "sha256:9bccbb3b40e3de10351f8f5068e105d0f4083b1a65fa07b6634fbc401a6287fd", size = 7335585, upload-time = "2025-12-10T07:08:53.618Z" }
wheels = [
@@ -6781,7 +6821,7 @@ resolution-markers = [
"python_full_version < '3.11' and sys_platform != 'linux'",
]
dependencies = [
- { name = "numpy" },
+ { name = "numpy", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/0f/37/6964b830433e654ec7485e45a00fc9a27cf868d622838f6b6d9c5ec0d532/scipy-1.15.3.tar.gz", hash = "sha256:eae3cf522bc7df64b42cad3925c876e1b0b6c35c1337c93e12c0f366f55b0eaf", size = 59419214, upload-time = "2025-05-08T16:13:05.955Z" }
wheels = [
@@ -6851,7 +6891,7 @@ resolution-markers = [
"python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'",
]
dependencies = [
- { name = "numpy" },
+ { name = "numpy", marker = "python_full_version >= '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/56/3e/9cca699f3486ce6bc12ff46dc2031f1ec8eb9ccc9a320fdaf925f1417426/scipy-1.17.0.tar.gz", hash = "sha256:2591060c8e648d8b96439e111ac41fd8342fdeff1876be2e19dea3fe8930454e", size = 30396830, upload-time = "2026-01-10T21:34:23.009Z" }
wheels = [
@@ -6902,8 +6942,8 @@ name = "secretstorage"
version = "3.5.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
- { name = "cryptography" },
- { name = "jeepney" },
+ { name = "cryptography", marker = "python_full_version >= '3.12' and sys_platform == 'linux'" },
+ { name = "jeepney", marker = "python_full_version >= '3.12' and sys_platform == 'linux'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz", hash = "sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", size = 19884, upload-time = "2025-11-23T19:02:53.191Z" }
wheels = [
@@ -7128,9 +7168,9 @@ resolution-markers = [
"python_full_version < '3.11' and sys_platform != 'linux'",
]
dependencies = [
- { name = "aiodns" },
- { name = "pyasn1" },
- { name = "pyasn1-modules" },
+ { name = "aiodns", marker = "python_full_version < '3.11'" },
+ { name = "pyasn1", marker = "python_full_version < '3.11'" },
+ { name = "pyasn1-modules", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/d9/03/925a18ce8aced370a5135b3e0ef870ec71a30da005f747891c83df98ca53/slixmpp-1.12.0.tar.gz", hash = "sha256:7469f6dcaf5742fd62e0e66ee447c5338a74520c1982a70784e7b790def2fdd5", size = 715300, upload-time = "2025-10-19T17:50:18.975Z" }
wheels = [
@@ -7171,9 +7211,9 @@ resolution-markers = [
"python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'",
]
dependencies = [
- { name = "aiodns" },
- { name = "pyasn1" },
- { name = "pyasn1-modules" },
+ { name = "aiodns", marker = "python_full_version >= '3.11'" },
+ { name = "pyasn1", marker = "python_full_version >= '3.11'" },
+ { name = "pyasn1-modules", marker = "python_full_version >= '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/8b/1c/6ce021fb5524b41330d583956d1ff8e508c95d6c1bb0ed34790e754cc8d2/slixmpp-1.13.2.tar.gz", hash = "sha256:1b6f7c4c273ae5d34c4e54f6d66984a27467de225312f19f942f971f38b78da2", size = 757698, upload-time = "2026-02-08T19:05:21.165Z" }
wheels = [
@@ -7765,9 +7805,9 @@ resolution-markers = [
"python_full_version < '3.11' and sys_platform != 'linux'",
]
dependencies = [
- { name = "aiohttp" },
- { name = "iso8601" },
- { name = "typing-extensions" },
+ { name = "aiohttp", marker = "python_full_version < '3.11'" },
+ { name = "iso8601", marker = "python_full_version < '3.11'" },
+ { name = "typing-extensions", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/de/61/3f177d7e48af2816bd7bc639a7d5c4f2fa0ff3ae46faf36754b3c3676de9/twitchio-2.10.0.tar.gz", hash = "sha256:3ae5e17b3764eff24ad7d12decb473c9d34f18510b5d705e0bbe6fcf0b06fd75", size = 114393, upload-time = "2024-06-25T14:47:22.463Z" }
wheels = [
@@ -7793,25 +7833,13 @@ resolution-markers = [
"python_full_version == '3.11.*' and sys_platform != 'emscripten' and sys_platform != 'linux' and sys_platform != 'win32'",
]
dependencies = [
- { name = "aiohttp" },
+ { name = "aiohttp", marker = "python_full_version >= '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/16/55/d0b8f1c6552f23994d925301fa7d5743109568cc8f079fee9866c5a52499/twitchio-3.2.1.tar.gz", hash = "sha256:a1be858a4028625173978db91224326b910a0aaa4c7db879a8867003642a0115", size = 248627, upload-time = "2026-01-29T14:14:44.803Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/7e/a1/2f7e31066eb2c6e78cecd95cf1dd66773cf8f90738339320e12df5e2da2d/twitchio-3.2.1-py3-none-any.whl", hash = "sha256:576d131fdfb0378ee0bfd7f35f8381636f9a911c3961589add97055a578da555", size = 320563, upload-time = "2026-01-29T14:14:43.252Z" },
]
-[[package]]
-name = "typeguard"
-version = "4.5.1"
-source = { registry = "https://pypi.org/simple" }
-dependencies = [
- { name = "typing-extensions" },
-]
-sdist = { url = "https://files.pythonhosted.org/packages/2b/e8/66e25efcc18542d58706ce4e50415710593721aae26e794ab1dec34fb66f/typeguard-4.5.1.tar.gz", hash = "sha256:f6f8ecbbc819c9bc749983cc67c02391e16a9b43b8b27f15dc70ed7c4a007274", size = 80121, upload-time = "2026-02-19T16:09:03.392Z" }
-wheels = [
- { url = "https://files.pythonhosted.org/packages/91/88/b55b3117287a8540b76dbdd87733808d4d01c8067a3b339408c250bb3600/typeguard-4.5.1-py3-none-any.whl", hash = "sha256:44d2bf329d49a244110a090b55f5f91aa82d9a9834ebfd30bcc73651e4a8cc40", size = 36745, upload-time = "2026-02-19T16:09:01.6Z" },
-]
-
[[package]]
name = "typer"
version = "0.24.0"
@@ -7848,20 +7876,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/dc/9b/47798a6c91d8bdb567fe2698fe81e0c6b7cb7ef4d13da4114b41d239f65d/typing_inspection-0.4.2-py3-none-any.whl", hash = "sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7", size = 14611, upload-time = "2025-10-01T02:14:40.154Z" },
]
-[[package]]
-name = "tyro"
-version = "1.0.8"
-source = { registry = "https://pypi.org/simple" }
-dependencies = [
- { name = "docstring-parser" },
- { name = "typeguard" },
- { name = "typing-extensions" },
-]
-sdist = { url = "https://files.pythonhosted.org/packages/9e/89/f4d76ba5ac4ca627c3a0413fdc8d90f202929c45d9c25deec29c9a86a0ae/tyro-1.0.8.tar.gz", hash = "sha256:c22bf238ce029b8cc262759129fccfd968300c3761f00ce186570af3891e14bb", size = 460626, upload-time = "2026-02-25T00:18:45.388Z" }
-wheels = [
- { url = "https://files.pythonhosted.org/packages/61/7c/9c5a4ab9e24f1428fd5ab2728b7f68878f54d549a11fa431ead76d3d127b/tyro-1.0.8-py3-none-any.whl", hash = "sha256:abb6a4054f20616d9ca5181ba0c6ae0e35cf8f35ae0c78d3d4dbd1f678e0643d", size = 181935, upload-time = "2026-02-25T00:18:43.689Z" },
-]
-
[[package]]
name = "tzdata"
version = "2025.3"
@@ -8670,26 +8684,16 @@ wheels = [
]
[[package]]
-name = "zeus-ml"
-version = "0.11.0.post1"
+name = "zeus-apple-silicon"
+version = "1.1.0"
source = { registry = "https://pypi.org/simple" }
-dependencies = [
- { name = "amdsmi" },
- { name = "httpx" },
- { name = "numpy" },
- { name = "nvidia-ml-py" },
- { name = "pandas", version = "2.3.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
- { name = "pandas", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
- { name = "pydantic" },
- { name = "python-dateutil" },
- { name = "rich" },
- { name = "scikit-learn", version = "1.7.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
- { name = "scikit-learn", version = "1.8.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" },
- { name = "tyro" },
-]
-sdist = { url = "https://files.pythonhosted.org/packages/c7/26/e7e70121272fffc4c9ce3ec232e457d33d02b490eefe1b7a2b022bf296aa/zeus_ml-0.11.0.post1.tar.gz", hash = "sha256:0bc061b6c34edcfc2e86b3de81000157e30227fe11e7d0c683f6f5cf7bc3be22", size = 178924, upload-time = "2025-02-03T01:17:07.93Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/64/a5/7da627c178365b90d2e4c83c78ed3ccb085251d4247584d8ca9929e86ffb/zeus_apple_silicon-1.1.0.tar.gz", hash = "sha256:ca99b33e9e375498b2f5b9671eee50258ed7483c10d62eac7595cbb882ce3a4e", size = 46173, upload-time = "2026-03-29T05:30:46.405Z" }
wheels = [
- { url = "https://files.pythonhosted.org/packages/37/f4/d787c837308a26d7db69154761ae8b7e7f0a0703952cd57dc40fd13471c8/zeus_ml-0.11.0.post1-py3-none-any.whl", hash = "sha256:d0554ff3a4b8a27d818f4ea8688daa500be5f10322fb5824365aef24c1682e43", size = 227090, upload-time = "2025-02-03T01:17:04.018Z" },
+ { url = "https://files.pythonhosted.org/packages/b9/2c/2d2cd9d53167b4ab951217d54fa13f3ac183808958193de7d83455903d10/zeus_apple_silicon-1.1.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:ed75237100da14552c50618ad8a18e16eb8d7fb1bdf31c43f5134bf2ab47830d", size = 62858, upload-time = "2026-03-29T05:30:37.676Z" },
+ { url = "https://files.pythonhosted.org/packages/4a/cd/69179ff6d638103440457cbe1c8c0b30d0a22a6bb6f2c043a42995fb99e4/zeus_apple_silicon-1.1.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:fca4ff0211b1440841fa8454ea6effedc55c83a17be770ad92bb39da961a2214", size = 62752, upload-time = "2026-03-29T05:30:38.966Z" },
+ { url = "https://files.pythonhosted.org/packages/4b/50/dae39d71cdfed918eea04596ca582f3a8428bb8e44d76e88020d02f04b75/zeus_apple_silicon-1.1.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:c862421e3477344260fad80276cb6b8f6ed7189b5472ef3fca9a54cd5a780aa7", size = 61615, upload-time = "2026-03-29T05:30:40.228Z" },
+ { url = "https://files.pythonhosted.org/packages/c8/0c/e162be99be598d9be1ee48732a95475a42473b311c4809fc01dcc13a68c7/zeus_apple_silicon-1.1.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:410ecf3f594fc94202ad778d274131a24a403553c9b56c7907344edc9107ec03", size = 61526, upload-time = "2026-03-29T05:30:41.568Z" },
+ { url = "https://files.pythonhosted.org/packages/a9/a9/ed1a74db7dbad082bb4ea8fc91660f96d9adcc52d4dea1afecc74015a96c/zeus_apple_silicon-1.1.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:e111ac377a08f2e06cb1303af8625927a8a9dcd5cf6d2796a655eddf18debe4b", size = 59941, upload-time = "2026-03-29T05:30:44.092Z" },
]
[[package]]