Skip to content

Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.

Add new environment modules for kitchen appliances, media player, productivity, smart home, and wearable devices

HenryNdubuaku4f57f65b238b
diff --git a/README.md b/README.md
index 0ee4db8..c032aac 100644
--- a/README.md
+++ b/README.md
@@ -74,6 +74,19 @@ needle playground --weights my.cact # a tuned model
The server downloads and initializes the model before serving, so the first query is instant. The **Finetune on these tools** button runs the fine-tuning pipeline below from the UI and hands back a downloadable `.cact`.
+## Environments
+
+Ready-made tool surfaces in `needle.environments`: `smart_home`, `media_player`, `productivity`, `wearable`, `kitchen_appliance`, and `data_capture`. Each is a hand-curated set of tools whose enums, bounds, and descriptions map cleanly onto Needle's constrained decoding, with a ready agent and a frozen acceptance suite.
+
+```python
+from needle.environments import smart_home
+
+smart_home.agent.complete("dim the study lights to 30 percent")
+smart_home.run_tests()
+```
+
+`python -m needle.environments.smart_home` runs a suite from the shell. To adapt an environment to your product, swap the `Literal` values (rooms, contacts, categories) for your own and keep the shapes: closed sets as enums, bounded numbers, verbatim copy for free text, five tools or fewer. The full tool surfaces and the suite contract are in [doc/environments.md](doc/environments.md).
+
## Fine-tuning
Needle fine-tunes with LoRA on the frozen base and merges the adapter at export, so a run is cheap and the tuned model is still a single `.cact` that runs on the same engine. The workflow is: (optionally) synthesize data, LoRA fine-tune, then build a tuned `.cact`. See [doc/finetuning.md](doc/finetuning.md) for dataset sizing, reading the loss curve, and troubleshooting.
diff --git a/doc/environments.md b/doc/environments.md
new file mode 100644
index 0000000..421eb70
--- /dev/null
+++ b/doc/environments.md
@@ -0,0 +1,118 @@
+# Environments
+
+Pre-configured tool surfaces in `needle.environments`. Each module is a hand-curated set of at most five tools whose enums, bounds, and descriptions map cleanly onto Needle's constrained decoding, plus a ready agent and a frozen acceptance suite.
+
+```python
+from needle.environments import smart_home
+
+smart_home.agent.complete("dim the study lights to 30 percent")
+smart_home.run_tests()
+```
+
+```sh
+python -m needle.environments.smart_home # run one frozen suite, exit 0 on pass
+```
+
+## Module contract
+
+Every environment module exposes the same surface:
+
+| Attribute | What it is |
+| --- | --- |
+| `TOOLS` | The decorated tool functions, each returning `{"ok": True, ...}` echoes of its arguments. |
+| `SYSTEM` | The system prompt the agent is built with. |
+| `agent` | A `needle.Needle(tools=TOOLS, system=SYSTEM)`, constructed lazily on first access so importing never fetches the engine. Construction sets `NEEDLE_STRICT_VALIDATE=1` (if unset) so out-of-bounds calls are suppressed to refusals. |
+| `TEST_CASES` | The frozen suite: dicts of `query`, expected `calls`, `category`, and optional `critical`. |
+| `run_tests(min_confidence=0.0, verbose=True)` | Runs the suite against the shipped engine; returns `True` at >=90% pass with zero critical failures. |
+
+`needle.environments.ENVIRONMENTS` maps name to module; `needle.environments.run_tests()` runs all six.
+
+## Available environments
+
+### smart_home
+
+Lights, thermostat, fans, blinds, robot vacuum, over four rooms (`kitchen`, `living_room`, `bedroom`, `study`).
+
+| Tool | Arguments |
+| --- | --- |
+| `control_lights` | `room`, `action` on/off/dim, `brightness_percent` 0-100, `color` from 5 names |
+| `set_thermostat` | `temperature` 10-30 C |
+| `control_fan` | `room` (no kitchen), `action` on/off, `speed` low/medium/high |
+| `control_blinds` | `room`, `action` open/close |
+| `start_robot_vacuum` | `action` start/stop/dock, `room` (no study) |
+
+### media_player
+
+Smart speaker playback and volume. `play_music` starts named content; the zero-argument playback tools control whatever is already playing.
+
+| Tool | Arguments |
+| --- | --- |
+| `play_music` | `query` verbatim, 1-80 chars |
+| `pause_media`, `resume_media`, `skip_track` | none |
+| `set_volume` | `level` 0-100 |
+
+### productivity
+
+Timers, reminders, calendar events, tasks, notes. Date and time phrases are copied verbatim for the host app to resolve; a reminder without a time is treated as incomplete, and undated to-dos belong to `add_task`.
+
+| Tool | Arguments |
+| --- | --- |
+| `set_timer` | `time_human` verbatim, 1-60 chars |
+| `create_reminder` | `message` 1-120 chars, `date_time_human` |
+| `create_calendar_event` | `title` 1-120 chars, `start_time_human`, `location` 1-80 chars |
+| `add_task` | `title` 1-120 chars, `priority` low/medium/high |
+| `create_note` | `text` 1-200 chars, `title` 1-60 chars |
+
+### wearable
+
+Watch notifications, workouts, find-my-phone. Replies only go to the four named senders (`Maya`, `Leo`, `Dr. Patel`, `Cactus Team`); workout types use gerund forms.
+
+| Tool | Arguments |
+| --- | --- |
+| `reply_to_notification` | `notification_match` sender enum, `text` verbatim, 1-240 chars |
+| `dismiss_notification` | `notification_match` sender enum |
+| `start_workout` | `workout_type` running/walking/cycling/swimming/strength/yoga |
+| `end_workout`, `find_my_phone` | none |
+
+### kitchen_appliance
+
+Oven, coffee maker, dishwasher, cooking timers. Hard numeric bounds make unsafe requests unrepresentable; the one read-only tool keeps checks separate from actions.
+
+| Tool | Arguments |
+| --- | --- |
+| `set_oven` | `temperature` 50-250 C |
+| `control_coffee_maker` | `action` brew/stop/warm |
+| `start_dishwasher` | `cycle` eco/heavy/quick |
+| `set_cooking_timer` | `label` 1-40 chars, `minutes` 1-360 |
+| `get_appliance_status` | `appliance` oven/coffee_maker/dishwasher, read-only |
+
+### data_capture
+
+Contacts, expenses, meals, water, weight: dictated facts into typed records. Every value is copied verbatim, a member of a closed set, or a bounded number; the phone pattern and email format are enforced during decoding.
+
+| Tool | Arguments |
+| --- | --- |
+| `create_contact` | `name` 1-60 chars, `phone` pattern `^\+?[0-9][0-9 -]{5,17}$`, `email` format email |
+| `log_expense` | `amount` 0-100000, `category` from 6, `merchant` 1-60 chars |
+| `log_meal` | `description` 1-120 chars, `meal_type` breakfast/lunch/dinner/snack |
+| `log_water_intake` | `amount_ml` 1-5000 |
+| `log_weight` | `weight_kg` 20-300 |
+
+## The frozen suites
+
+Each suite has 32 cases across six categories:
+
+| Category | Expectation |
+| --- | --- |
+| `positive` | The exact call, arguments included. |
+| `missing` | `[]`: a required value was not stated, do not guess it. |
+| `irrelevant` | `[]`: no declared tool covers the request. |
+| `negation` | `[]`: the request is negated ("don't", "do not", "never"). |
+| `invalid` | `[]`: a stated value is outside the declared bounds. |
+| `parallel` | Two calls from one query, order-insensitive. |
+
+Cases marked `critical: True` (all `missing`, `negation`, and `invalid` cases) fail the suite regardless of the overall rate. `run_tests()` scores raw model output; `run_tests(min_confidence=0.4)` applies the production contract, acting on a call only at or above the threshold and treating anything below as a refusal.
+
+## Adapting one
+
+Swap the `Literal` values (rooms, contacts, categories) for your own and keep the shapes: closed sets as enums, bounded numbers, verbatim copy for free text, five tools or fewer. One learned rule from smart_home: avoid enum values that hide inside likely query words (a room named office poisons an off action, so that home has a study).
diff --git a/llms.txt b/llms.txt
index 9c69520..9a0b6b3 100644
--- a/llms.txt
+++ b/llms.txt
@@ -184,6 +184,17 @@ needle playground --weights my.cact # a tuned model
Preset demos, editable tools/prompt, multi-turn follow-ups, and a "Finetune on these tools" button that runs the pipeline above and returns a downloadable `.cact`.
+## Environments (ready-made tool surfaces)
+
+`needle.environments` ships six: `smart_home`, `media_player`, `productivity`, `wearable`, `kitchen_appliance`, `data_capture`. Each module has `TOOLS`, `SYSTEM`, a lazily constructed `agent`, `TEST_CASES`, and `run_tests(min_confidence=0.0)`; `python -m needle.environments.smart_home` runs its frozen acceptance suite.
+
+```python
+from needle.environments import smart_home
+smart_home.agent.complete("dim the study lights to 30 percent")
+```
+
+Adapt by swapping the Literal values for your own and keeping the shapes: closed sets as enums, bounded numbers, verbatim copy for free text, five tools or fewer.
+
## CLI summary
- `needle run --checkpoint <base.pkl> --query "..." --tools tools.json` - JAX reference inference from a checkpoint (dev path; normal inference is the Python `Needle` API above).
diff --git a/needle/environments/__init__.py b/needle/environments/__init__.py
new file mode 100644
index 0000000..9f8f1bd
--- /dev/null
+++ b/needle/environments/__init__.py
@@ -0,0 +1,45 @@
+"""Pre-configured environments: hand-curated tool surfaces whose enums,
+bounds, and descriptions map cleanly onto Needle's constrained decoding.
+Each module ships TOOLS, SYSTEM, a ready agent, and a frozen acceptance
+suite runnable as `python -m needle.environments.<name>`.
+
+ from needle.environments import smart_home
+
+ smart_home.agent.complete("dim the study lights to 30 percent")
+ smart_home.run_tests()
+
+To adapt an environment to your product, swap the Literal values (rooms,
+contacts, categories) for your own and keep the shapes: closed sets as
+enums, bounded numbers, verbatim copy for free text, five tools or fewer.
+"""
+
+import importlib
+
+_NAMES = ("data_capture", "kitchen_appliance", "media_player", "productivity",
+ "smart_home", "wearable")
+
+__all__ = ["ENVIRONMENTS", "run_tests", *_NAMES]
+
+
+def _load(name):
+ return importlib.import_module(f"needle.environments.{name}")
+
+
+def __getattr__(name):
+ if name in _NAMES:
+ return _load(name)
+ if name == "ENVIRONMENTS":
+ return {n: _load(n) for n in _NAMES}
+ raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+def __dir__():
+ return sorted([*globals(), "ENVIRONMENTS", *_NAMES])
+
+
+def run_tests(min_confidence=0.0, verbose=True):
+ passed = True
+ for name in _NAMES:
+ print(name)
+ passed = _load(name).run_tests(min_confidence, verbose) and passed
+ return passed
diff --git a/needle/environments/_harness.py b/needle/environments/_harness.py
new file mode 100644
index 0000000..fb561f9
--- /dev/null
+++ b/needle/environments/_harness.py
@@ -0,0 +1,45 @@
+import json
+import os
+
+import needle
+
+_agents = {}
+
+
+def agent_for(module):
+ key = module.__name__
+ if key not in _agents:
+ os.environ.setdefault("NEEDLE_STRICT_VALIDATE", "1")
+ _agents[key] = needle.Needle(tools=module.TOOLS, system=module.SYSTEM)
+ return _agents[key]
+
+
+def run_tests(module, min_confidence=0.0, verbose=True):
+ """Run an environment's frozen acceptance suite against the shipped engine.
+ The default scores raw model output; passing e.g. min_confidence=0.4 applies
+ the production contract (act on a call only at or above the threshold,
+ otherwise treat it as a refusal)."""
+ agent = agent_for(module)
+ failures, critical_failures = [], []
+ for case in module.TEST_CASES:
+ agent.reset()
+ response = agent.complete(case["query"])
+ got = response.get("function_calls") or []
+ if got and response.get("confidence", 0.0) < min_confidence:
+ got = []
+ want = case["calls"]
+ ok = got == want or sorted(
+ json.dumps(c, sort_keys=True) for c in got
+ ) == sorted(json.dumps(c, sort_keys=True) for c in want)
+ if not ok:
+ failures.append(case)
+ if case.get("critical"):
+ critical_failures.append(case)
+ if verbose:
+ print(f"FAIL [{case['category']}] {case['query']}")
+ print(f" want {json.dumps(want)}")
+ print(f" got {json.dumps(got)}")
+ passed = len(module.TEST_CASES) - len(failures)
+ print(f"{passed}/{len(module.TEST_CASES)} passed, {len(critical_failures)} critical failures "
+ f"(confidence gate {min_confidence})")
+ return passed >= round(0.9 * len(module.TEST_CASES)) and not critical_failures
diff --git a/needle/environments/data_capture.py b/needle/environments/data_capture.py
new file mode 100644
index 0000000..2ab3aa5
--- /dev/null
+++ b/needle/environments/data_capture.py
@@ -0,0 +1,132 @@
+"""Structured capture: turn dictated facts into typed records.
+
+Every value is either copied verbatim, a member of a closed set, or a
+bounded number - the model records what was said and nothing else. The
+phone pattern and email format are enforced during decoding.
+"""
+
+import sys
+from typing import Annotated, Literal, Optional
+
+import needle
+from needle.environments import _harness
+
+
+def create_contact(
+ name: Annotated[str, needle.Field(min_length=1, max_length=60)],
+ phone: Optional[Annotated[str, needle.Field(pattern=r"^\+?[0-9][0-9 -]{5,17}$")]] = None,
+ email: Optional[Annotated[str, needle.Field(format="email")]] = None,
+):
+ """Save a new contact. Copy the name word for word and the number digit for digit; never invent or complete them.
+
+ Args:
+ name: Full name of the contact, copied word for word.
+ phone: The phone number exactly as given; include only when stated.
+ email: The email address; include only when stated.
+ """
+ return {"ok": True, "name": name, "phone": phone, "email": email}
+
+
+def log_expense(
+ amount: Annotated[float, needle.Field(ge=0, le=100000)],
+ category: Literal["food", "groceries", "transport", "entertainment", "utilities", "shopping"],
+ merchant: Optional[Annotated[str, needle.Field(min_length=1, max_length=60)]] = None,
+):
+ """Record money spent. Log only amounts the user stated; never estimate.
+
+ Args:
+ amount: The amount as stated, digits only.
+ category: The spending category the user named.
+ merchant: The store or vendor copied word for word; include only when named.
+ """
+ return {"ok": True, "amount": amount, "category": category, "merchant": merchant}
+
+
+def log_meal(
+ description: Annotated[str, needle.Field(min_length=1, max_length=120)],
+ meal_type: Optional[Literal["breakfast", "lunch", "dinner", "snack"]] = None,
+):
+ """Log a meal or food item. Use log_expense for money spent on food.
+
+ Args:
+ description: What was eaten, copied word for word.
+ meal_type: Which meal; include only when stated.
+ """
+ return {"ok": True, "description": description, "meal_type": meal_type}
+
+
+def log_water_intake(amount_ml: Annotated[int, needle.Field(ge=1, le=5000)]):
+ """Log water consumption in your water log.
+
+ Args:
+ amount_ml: Amount of water in milliliters.
+ """
+ return {"ok": True, "amount_ml": amount_ml}
+
+
+def log_weight(weight_kg: Annotated[float, needle.Field(ge=20, le=300)]):
+ """Record a body-weight measurement in kilograms.
+
+ Args:
+ weight_kg: The weight in kilograms as stated.
+ """
+ return {"ok": True, "weight_kg": weight_kg}
+
+
+TOOLS = [create_contact, log_expense, log_meal, log_water_intake, log_weight]
+SYSTEM = "Copy names, merchants, and descriptions verbatim from the user. Record only values the user stated; never estimate. Map each explicit supported record to exactly one declared call. Unsupported, incomplete, ambiguous, and negated requests return no call."
+
+
+TEST_CASES = [
+ {'query': 'add Maya Chen to my contacts', 'calls': [{'name': 'create_contact', 'arguments': {'name': 'Maya Chen'}}], 'category': 'positive'},
+ {'query': 'save Leo Park to contacts, phone 555-0134', 'calls': [{'name': 'create_contact', 'arguments': {'name': 'Leo Park', 'phone': '555-0134'}}], 'category': 'positive'},
+ {'query': 'create a contact for Nadia Osei, number 917-555-0188, email [email protected]', 'calls': [{'name': 'create_contact', 'arguments': {'name': 'Nadia Osei', 'phone': '917-555-0188', 'email': '[email protected]'}}], 'category': 'positive'},
+ {'query': 'put Tomás Rivera in my contacts', 'calls': [{'name': 'create_contact', 'arguments': {'name': 'Tomás Rivera'}}], 'category': 'positive'},
+ {'query': 'log an expense of 23 for transport', 'calls': [{'name': 'log_expense', 'arguments': {'amount': 23, 'category': 'transport'}}], 'category': 'positive'},
+ {'query': 'record an 85.99 groceries expense from FreshMart', 'calls': [{'name': 'log_expense', 'arguments': {'amount': 85.99, 'category': 'groceries', 'merchant': 'FreshMart'}}], 'category': 'positive'},
+ {'query': 'track 142.75 spent on utilities', 'calls': [{'name': 'log_expense', 'arguments': {'amount': 142.75, 'category': 'utilities'}}], 'category': 'positive'},
+ {'query': 'capture 260 in shopping spend from Uniqlo', 'calls': [{'name': 'log_expense', 'arguments': {'amount': 260, 'category': 'shopping', 'merchant': 'Uniqlo'}}], 'category': 'positive'},
+ {'query': 'log 18.75 for entertainment', 'calls': [{'name': 'log_expense', 'arguments': {'amount': 18.75, 'category': 'entertainment'}}], 'category': 'positive'},
+ {'query': 'log a meal, grilled salmon with rice', 'calls': [{'name': 'log_meal', 'arguments': {'description': 'grilled salmon with rice'}}], 'category': 'positive'},
+ {'query': 'log lunch, leftover pad thai', 'calls': [{'name': 'log_meal', 'arguments': {'description': 'leftover pad thai', 'meal_type': 'lunch'}}], 'category': 'positive'},
+ {'query': 'record oatmeal with berries for breakfast', 'calls': [{'name': 'log_meal', 'arguments': {'description': 'oatmeal with berries', 'meal_type': 'breakfast'}}], 'category': 'positive'},
+ {'query': 'jot down what i ate, rice cakes with peanut butter', 'calls': [{'name': 'log_meal', 'arguments': {'description': 'rice cakes with peanut butter'}}], 'category': 'positive'},
+ {'query': 'log 750 ml of water', 'calls': [{'name': 'log_water_intake', 'arguments': {'amount_ml': 750}}], 'category': 'positive'},
+ {'query': 'record 500 ml of water intake', 'calls': [{'name': 'log_water_intake', 'arguments': {'amount_ml': 500}}], 'category': 'positive'},
+ {'query': 'i drank 1200 ml of water, log it', 'calls': [{'name': 'log_water_intake', 'arguments': {'amount_ml': 1200}}], 'category': 'positive'},
+ {'query': 'log my weight at 82.5 kg', 'calls': [{'name': 'log_weight', 'arguments': {'weight_kg': 82.5}}], 'category': 'positive'},
+ {'query': 'record a weight of 74 kg', 'calls': [{'name': 'log_weight', 'arguments': {'weight_kg': 74}}], 'category': 'positive'},
+ {'query': 'log an expense of 45 from this afternoon', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'record my transport spending from today as an expense', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'log my water intake from this morning', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'log my current weight for me', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'text Marcus to ask about his new number', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'how much have i spent on groceries this month', 'calls': [], 'category': 'irrelevant'},
+ {'query': "delete yesterday's lunch entry from the log", 'calls': [], 'category': 'irrelevant'},
+ {'query': "don't log the 15.40 i spent on transport", 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'do not add Ravi Kumar at 415-555-0162 to my contacts', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'never log 79.4 kg as my weight', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'log my weight as 500 kg', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'log 9000 ml of water for today', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'log lunch, turkey sandwich on rye, and 600 ml of water', 'calls': [{'name': 'log_meal', 'arguments': {'description': 'turkey sandwich on rye', 'meal_type': 'lunch'}}, {'name': 'log_water_intake', 'arguments': {'amount_ml': 600}}], 'category': 'parallel'},
+ {'query': 'log my weight of 78.4 kg and 350 ml of water', 'calls': [{'name': 'log_weight', 'arguments': {'weight_kg': 78.4}}, {'name': 'log_water_intake', 'arguments': {'amount_ml': 350}}], 'category': 'parallel'},
+]
+
+
+def run_tests(min_confidence=0.0, verbose=True):
+ return _harness.run_tests(sys.modules[__name__], min_confidence, verbose)
+
+
+def __getattr__(name):
+ if name == "agent":
+ return _harness.agent_for(sys.modules[__name__])
+ raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+if __name__ == "__main__":
+ sys.exit(0 if run_tests() else 1)
diff --git a/needle/environments/kitchen_appliance.py b/needle/environments/kitchen_appliance.py
new file mode 100644
index 0000000..56fc05f
--- /dev/null
+++ b/needle/environments/kitchen_appliance.py
@@ -0,0 +1,121 @@
+"""Countertop appliance hub: oven, coffee maker, dishwasher.
+
+The shape that matters on a microcontroller: hard numeric bounds make unsafe
+requests unrepresentable, and the one read-only tool keeps checks separate
+from actions. Optional settings the model tends to guess (oven modes, cup
+sizes, default cycles) are deliberately absent.
+"""
+
+import sys
+from typing import Annotated, Literal, Optional
+
+import needle
+from needle.environments import _harness
+
+
+def set_oven(temperature: Annotated[int, needle.Field(ge=50, le=250)]):
+ """Preheat or set the oven to a temperature in degrees Celsius. Use get_appliance_status to check on it.
+
+ Args:
+ temperature: Target temperature in degrees Celsius.
+ """
+ return {"ok": True, "temperature": temperature}
+
+
+def control_coffee_maker(action: Literal["brew", "stop", "warm"]):
+ """Control the coffee maker. Never use this for the oven or the dishwasher.
+
+ Args:
+ action: brew, stop, or warm.
+ """
+ return {"ok": True, "action": action}
+
+
+def start_dishwasher(cycle: Optional[Literal["eco", "heavy", "quick"]] = None):
+ """Start the dishwasher, optionally on a stated wash cycle.
+
+ Args:
+ cycle: The wash cycle; include only when stated.
+ """
+ return {"ok": True, "cycle": cycle}
+
+
+def set_cooking_timer(
+ label: Annotated[str, needle.Field(min_length=1, max_length=40)],
+ minutes: Annotated[int, needle.Field(ge=1, le=360)],
+):
+ """Set a named cooking timer.
+
+ Args:
+ label: Label for the timer, copied from the user.
+ minutes: Timer duration in minutes.
+ """
+ return {"ok": True, "label": label, "minutes": minutes}
+
+
+def get_appliance_status(appliance: Literal["oven", "coffee_maker", "dishwasher"]):
+ """Read an appliance's current status. This changes nothing; any request to change settings uses the other tools.
+
+ Args:
+ appliance: Which appliance to check.
+ """
+ return {"ok": True, "appliance": appliance, "status": "idle"}
+
+
+TOOLS = [set_oven, control_coffee_maker, start_dishwasher, set_cooking_timer, get_appliance_status]
+SYSTEM = "Map each explicit supported appliance action to exactly one declared call; never duplicate an action. Do not guess missing values. Unsupported, invalid, ambiguous, and negated requests return no call."
+
+
+TEST_CASES = [
+ {'query': 'preheat the oven to 180 degrees', 'calls': [{'name': 'set_oven', 'arguments': {'temperature': 180}}], 'category': 'positive'},
+ {'query': 'set the oven to 220 degrees', 'calls': [{'name': 'set_oven', 'arguments': {'temperature': 220}}], 'category': 'positive'},
+ {'query': 'bake at 200 degrees', 'calls': [{'name': 'set_oven', 'arguments': {'temperature': 200}}], 'category': 'positive'},
+ {'query': 'bring the oven up to 160 degrees', 'calls': [{'name': 'set_oven', 'arguments': {'temperature': 160}}], 'category': 'positive'},
+ {'query': 'brew a fresh pot of coffee', 'calls': [{'name': 'control_coffee_maker', 'arguments': {'action': 'brew'}}], 'category': 'positive'},
+ {'query': 'start brewing another batch of coffee', 'calls': [{'name': 'control_coffee_maker', 'arguments': {'action': 'brew'}}], 'category': 'positive'},
+ {'query': 'stop the coffee maker', 'calls': [{'name': 'control_coffee_maker', 'arguments': {'action': 'stop'}}], 'category': 'positive'},
+ {'query': 'keep my coffee warm', 'calls': [{'name': 'control_coffee_maker', 'arguments': {'action': 'warm'}}], 'category': 'positive'},
+ {'query': 'run the dishwasher on the eco cycle', 'calls': [{'name': 'start_dishwasher', 'arguments': {'cycle': 'eco'}}], 'category': 'positive'},
+ {'query': 'start a heavy dishwasher cycle', 'calls': [{'name': 'start_dishwasher', 'arguments': {'cycle': 'heavy'}}], 'category': 'positive'},
+ {'query': 'run a quick cycle on the dishwasher', 'calls': [{'name': 'start_dishwasher', 'arguments': {'cycle': 'quick'}}], 'category': 'positive'},
+ {'query': 'set a pasta timer for 10 minutes', 'calls': [{'name': 'set_cooking_timer', 'arguments': {'label': 'pasta', 'minutes': 10}}], 'category': 'positive'},
+ {'query': 'start a 90 minute timer for the brisket', 'calls': [{'name': 'set_cooking_timer', 'arguments': {'label': 'brisket', 'minutes': 90}}], 'category': 'positive'},
+ {'query': 'give me a 5 minute timer for the eggs', 'calls': [{'name': 'set_cooking_timer', 'arguments': {'label': 'eggs', 'minutes': 5}}], 'category': 'positive'},
+ {'query': 'the cookies need a 12 minute timer', 'calls': [{'name': 'set_cooking_timer', 'arguments': {'label': 'cookies', 'minutes': 12}}], 'category': 'positive'},
+ {'query': 'check the oven', 'calls': [{'name': 'get_appliance_status', 'arguments': {'appliance': 'oven'}}], 'category': 'positive'},
+ {'query': 'check on the dishwasher', 'calls': [{'name': 'get_appliance_status', 'arguments': {'appliance': 'dishwasher'}}], 'category': 'positive'},
+ {'query': 'check the coffee maker for me', 'calls': [{'name': 'get_appliance_status', 'arguments': {'appliance': 'coffee_maker'}}], 'category': 'positive'},
+ {'query': 'heat up the oven for the garlic bread', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'set a cooking timer for the rice', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'start a timer for 25 minutes', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'change the coffee maker setting', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'defrost some chicken in the microwave', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'boil the kettle for tea', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'turn the fridge down to 3 degrees', 'calls': [], 'category': 'irrelevant'},
+ {'query': "don't preheat the oven to 220 degrees tonight", 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'do not run the dishwasher on heavy today', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'never brew coffee this late at night', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'get the oven up to 300 degrees', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'set a stew timer for 500 minutes', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'set the oven to 180 degrees and start a pizza timer for 15 minutes', 'calls': [{'name': 'set_oven', 'arguments': {'temperature': 180}}, {'name': 'set_cooking_timer', 'arguments': {'label': 'pizza', 'minutes': 15}}], 'category': 'parallel'},
+ {'query': 'brew coffee and run the dishwasher on quick', 'calls': [{'name': 'control_coffee_maker', 'arguments': {'action': 'brew'}}, {'name': 'start_dishwasher', 'arguments': {'cycle': 'quick'}}], 'category': 'parallel'},
+]
+
+
+def run_tests(min_confidence=0.0, verbose=True):
+ return _harness.run_tests(sys.modules[__name__], min_confidence, verbose)
+
+
+def __getattr__(name):
+ if name == "agent":
+ return _harness.agent_for(sys.modules[__name__])
+ raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+if __name__ == "__main__":
+ sys.exit(0 if run_tests() else 1)
diff --git a/needle/environments/media_player.py b/needle/environments/media_player.py
new file mode 100644
index 0000000..6366d6a
--- /dev/null
+++ b/needle/environments/media_player.py
@@ -0,0 +1,104 @@
+"""Music playback for a smart speaker.
+
+The split that matters: play_music starts named content, and the zero-argument
+playback tools control whatever is already playing. Volume is the only number,
+and it is bounded.
+"""
+
+import sys
+from typing import Annotated
+
+import needle
+from needle.environments import _harness
+
+
+def play_music(query: Annotated[str, needle.Field(min_length=1, max_length=80)]):
+ """Play music by song name, artist, album, or genre. Copy the request word for word. Use the playback tools for pausing, resuming, or skipping.
+
+ Args:
+ query: The song, artist, album, or genre, copied word for word.
+ """
+ return {"ok": True, "query": query}
+
+
+def pause_media():
+ """Pause the currently playing media. Takes no arguments."""
+ return {"ok": True, "state": "paused"}
+
+
+def resume_media():
+ """Resume the paused media playback. Starting something new is play_music. Takes no arguments."""
+ return {"ok": True, "state": "playing"}
+
+
+def skip_track():
+ """Skip to the next track. Takes no arguments."""
+ return {"ok": True, "state": "skipped"}
+
+
+def set_volume(level: Annotated[int, needle.Field(ge=0, le=100)]):
+ """Set the speaker volume. This never pauses or skips anything.
+
+ Args:
+ level: Volume level from 0 to 100.
+ """
+ return {"ok": True, "level": level}
+
+
+TOOLS = [play_music, pause_media, resume_media, skip_track, set_volume]
+SYSTEM = "Map each explicit supported media action to exactly one declared call; never duplicate an action. Do not guess missing values. Unsupported, invalid, ambiguous, and negated requests return no call."
+
+
+TEST_CASES = [
+ {'query': 'play Purple Rain by Prince', 'calls': [{'name': 'play_music', 'arguments': {'query': 'Purple Rain by Prince'}}], 'category': 'positive'},
+ {'query': 'put on some smooth jazz', 'calls': [{'name': 'play_music', 'arguments': {'query': 'smooth jazz'}}], 'category': 'positive'},
+ {'query': 'queue up Blinding Lights by The Weeknd', 'calls': [{'name': 'play_music', 'arguments': {'query': 'Blinding Lights by The Weeknd'}}], 'category': 'positive'},
+ {'query': 'play some lofi beats', 'calls': [{'name': 'play_music', 'arguments': {'query': 'lofi beats'}}], 'category': 'positive'},
+ {'query': 'put on Hotel California by the Eagles', 'calls': [{'name': 'play_music', 'arguments': {'query': 'Hotel California by the Eagles'}}], 'category': 'positive'},
+ {'query': 'play Bohemian Rhapsody by Queen', 'calls': [{'name': 'play_music', 'arguments': {'query': 'Bohemian Rhapsody by Queen'}}], 'category': 'positive'},
+ {'query': 'set the volume to 40', 'calls': [{'name': 'set_volume', 'arguments': {'level': 40}}], 'category': 'positive'},
+ {'query': 'turn the volume up to 85', 'calls': [{'name': 'set_volume', 'arguments': {'level': 85}}], 'category': 'positive'},
+ {'query': 'lower the volume to 15', 'calls': [{'name': 'set_volume', 'arguments': {'level': 15}}], 'category': 'positive'},
+ {'query': 'set the speaker volume to 60', 'calls': [{'name': 'set_volume', 'arguments': {'level': 60}}], 'category': 'positive'},
+ {'query': 'crank the volume up to 95', 'calls': [{'name': 'set_volume', 'arguments': {'level': 95}}], 'category': 'positive'},
+ {'query': 'pause the music', 'calls': [{'name': 'pause_media', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'hit pause on the song', 'calls': [{'name': 'pause_media', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'pause playback while I take this call', 'calls': [{'name': 'pause_media', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'resume the music', 'calls': [{'name': 'resume_media', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'resume playback where it left off', 'calls': [{'name': 'resume_media', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'skip this song', 'calls': [{'name': 'skip_track', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'skip to the next track', 'calls': [{'name': 'skip_track', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'turn up the volume a little', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'play something for me', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'turn the music down a bit', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': "play whatever you think I'd like", 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'put on the podcast Hard Fork', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'tune into the radio station Jazz FM', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'add this song to my favorites playlist', 'calls': [], 'category': 'irrelevant'},
+ {'query': "don't pause the song, I love this part", 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'do not turn the volume up to 90', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'never play Baby Shark on this speaker again', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'set the volume to 140', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'turn the volume up to 500', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'set the volume to 55 and play Clair de Lune by Debussy', 'calls': [{'name': 'set_volume', 'arguments': {'level': 55}}, {'name': 'play_music', 'arguments': {'query': 'Clair de Lune by Debussy'}}], 'category': 'parallel'},
+ {'query': 'pause the music and turn the volume down to 20', 'calls': [{'name': 'pause_media', 'arguments': {}}, {'name': 'set_volume', 'arguments': {'level': 20}}], 'category': 'parallel'},
+]
+
+
+def run_tests(min_confidence=0.0, verbose=True):
+ return _harness.run_tests(sys.modules[__name__], min_confidence, verbose)
+
+
+def __getattr__(name):
+ if name == "agent":
+ return _harness.agent_for(sys.modules[__name__])
+ raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+if __name__ == "__main__":
+ sys.exit(0 if run_tests() else 1)
diff --git a/needle/environments/productivity.py b/needle/environments/productivity.py
new file mode 100644
index 0000000..4920830
--- /dev/null
+++ b/needle/environments/productivity.py
@@ -0,0 +1,133 @@
+"""Personal organizer actions for a phone: timers, reminders, events, tasks, notes.
+
+Date and time phrases are passed verbatim for the host app to resolve; the
+model never rewrites them. Reminders require a time on purpose - an undated
+"remind me" is treated as incomplete rather than guessed at.
+"""
+
+import sys
+from typing import Annotated, Literal, Optional
+
+import needle
+from needle.environments import _harness
+
+Phrase = Annotated[str, needle.Field(min_length=1, max_length=60)]
+
+
+def set_timer(time_human: Phrase):
+ """Set a timer for the specified duration or end time.
+
+ Args:
+ time_human: The duration or target end time in human readable format e.g. '45 minutes', 'at 13:30'.
+ """
+ return {"ok": True, "time_human": time_human}
+
+
+def create_reminder(message: Annotated[str, needle.Field(min_length=1, max_length=120)], date_time_human: Phrase):
+ """Create a reminder that fires at a stated time. A reminder needs both a message and a time phrase; use add_task for undated to-dos.
+
+ Args:
+ message: What to be reminded about, copied word for word.
+ date_time_human: The user's date or time phrase copied word for word.
+ """
+ return {"ok": True, "message": message, "date_time_human": date_time_human}
+
+
+def create_calendar_event(
+ title: Annotated[str, needle.Field(min_length=1, max_length=120)],
+ start_time_human: Phrase,
+ location: Optional[Annotated[str, needle.Field(min_length=1, max_length=80)]] = None,
+):
+ """Create a calendar event with a title and start time. Copy both word for word; never rephrase or resolve them.
+
+ Args:
+ title: The event title copied word for word.
+ start_time_human: The start date or time phrase copied word for word.
+ location: The event location; include only when stated.
+ """
+ return {"ok": True, "title": title, "start_time_human": start_time_human, "location": location}
+
+
+def add_task(
+ title: Annotated[str, needle.Field(min_length=1, max_length=120)],
+ priority: Optional[Literal["low", "medium", "high"]] = None,
+):
+ """Add a task to the task manager. Tasks are undated; dated or timed requests belong to create_reminder or create_calendar_event.
+
+ Args:
+ title: The task copied word for word.
+ priority: low, medium, or high; include only when the user says a priority word.
+ """
+ return {"ok": True, "title": title, "priority": priority}
+
+
+def create_note(
+ text: Annotated[str, needle.Field(min_length=1, max_length=200)],
+ title: Optional[Phrase] = None,
+):
+ """Save a free-form note. Notes store information; they never fire alerts.
+
+ Args:
+ text: The note content copied word for word.
+ title: A short note title; include only when the user names one.
+ """
+ return {"ok": True, "text": text, "title": title}
+
+
+TOOLS = [set_timer, create_reminder, create_calendar_event, add_task, create_note]
+SYSTEM = "Copy titles, messages, and date or time phrases verbatim from the user; never rephrase or resolve them. Map each explicit supported request to exactly one declared call. Do not guess missing values. Unsupported, invalid, ambiguous, and negated requests return no call."
+
+
+TEST_CASES = [
+ {'query': 'start a timer for 25 minutes', 'calls': [{'name': 'set_timer', 'arguments': {'time_human': '25 minutes'}}], 'category': 'positive'},
+ {'query': 'run a timer for 90 seconds', 'calls': [{'name': 'set_timer', 'arguments': {'time_human': '90 seconds'}}], 'category': 'positive'},
+ {'query': 'set a timer that goes off at 9:15pm', 'calls': [{'name': 'set_timer', 'arguments': {'time_human': '9:15pm'}}], 'category': 'positive'},
+ {'query': 'remind me at 6pm to water the plants', 'calls': [{'name': 'create_reminder', 'arguments': {'message': 'water the plants', 'date_time_human': '6pm'}}], 'category': 'positive'},
+ {'query': 'set a reminder for Sunday morning to call Grandma', 'calls': [{'name': 'create_reminder', 'arguments': {'message': 'call Grandma', 'date_time_human': 'Sunday morning'}}], 'category': 'positive'},
+ {'query': 'remind me tomorrow at noon to renew the car insurance', 'calls': [{'name': 'create_reminder', 'arguments': {'message': 'renew the car insurance', 'date_time_human': 'tomorrow at noon'}}], 'category': 'positive'},
+ {'query': 'could you remind me at 8:15am to take my vitamins', 'calls': [{'name': 'create_reminder', 'arguments': {'message': 'take my vitamins', 'date_time_human': '8:15am'}}], 'category': 'positive'},
+ {'query': 'add dentist appointment to my calendar tomorrow at 3pm', 'calls': [{'name': 'create_calendar_event', 'arguments': {'title': 'dentist appointment', 'start_time_human': 'tomorrow at 3pm'}}], 'category': 'positive'},
+ {'query': 'schedule team standup for Monday at 9am', 'calls': [{'name': 'create_calendar_event', 'arguments': {'title': 'team standup', 'start_time_human': 'Monday at 9am'}}], 'category': 'positive'},
+ {'query': 'put book club on my calendar Wednesday at 7pm at Riverside Library', 'calls': [{'name': 'create_calendar_event', 'arguments': {'title': 'book club', 'start_time_human': 'Wednesday at 7pm', 'location': 'Riverside Library'}}], 'category': 'positive'},
+ {'query': 'schedule lunch with Sam for Friday at 1pm at Cafe Roma', 'calls': [{'name': 'create_calendar_event', 'arguments': {'title': 'lunch with Sam', 'start_time_human': 'Friday at 1pm', 'location': 'Cafe Roma'}}], 'category': 'positive'},
+ {'query': 'add renew my passport to my to-do list', 'calls': [{'name': 'add_task', 'arguments': {'title': 'renew my passport'}}], 'category': 'positive'},
+ {'query': 'add descale the kettle to my tasks', 'calls': [{'name': 'add_task', 'arguments': {'title': 'descale the kettle'}}], 'category': 'positive'},
+ {'query': 'add file the insurance claim as a high priority task', 'calls': [{'name': 'add_task', 'arguments': {'title': 'file the insurance claim', 'priority': 'high'}}], 'category': 'positive'},
+ {'query': 'add a low priority task to organize the garage', 'calls': [{'name': 'add_task', 'arguments': {'title': 'organize the garage', 'priority': 'low'}}], 'category': 'positive'},
+ {'query': 'make a note that sunflower42 is the wifi password', 'calls': [{'name': 'create_note', 'arguments': {'text': 'sunflower42 is the wifi password'}}], 'category': 'positive'},
+ {'query': 'save a note titled Packing List saying bring the hiking boots', 'calls': [{'name': 'create_note', 'arguments': {'text': 'bring the hiking boots', 'title': 'Packing List'}}], 'category': 'positive'},
+ {'query': 'write a note titled Garden that says plant the tulip bulbs in October', 'calls': [{'name': 'create_note', 'arguments': {'text': 'plant the tulip bulbs in October', 'title': 'Garden'}}], 'category': 'positive'},
+ {'query': 'remind me to take out the recycling', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'set a timer for the pasta', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'put dinner with Alex on my calendar', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'set a reminder about picking up the dry cleaning', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'email the shopping list to my roommate', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'check my calendar for this weekend', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'delete the reminder about the oil change', 'calls': [], 'category': 'irrelevant'},
+ {'query': "don't set a timer for 20 minutes", 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'do not put yoga class on my calendar for Saturday at 8am', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'never remind me about the water bill again', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'set a timer for negative 5 minutes', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'remind me yesterday at 4pm to submit the timesheet', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'set a timer for 10 minutes and remind me at 5pm to hydrate', 'calls': [{'name': 'set_timer', 'arguments': {'time_human': '10 minutes'}}, {'name': 'create_reminder', 'arguments': {'message': 'hydrate', 'date_time_human': '5pm'}}], 'category': 'parallel'},
+ {'query': 'add buy candles to my task list and note that rent is due Friday', 'calls': [{'name': 'add_task', 'arguments': {'title': 'buy candles'}}, {'name': 'create_note', 'arguments': {'text': 'rent is due Friday'}}], 'category': 'parallel'},
+]
+
+
+def run_tests(min_confidence=0.0, verbose=True):
+ return _harness.run_tests(sys.modules[__name__], min_confidence, verbose)
+
+
+def __getattr__(name):
+ if name == "agent":
+ return _harness.agent_for(sys.modules[__name__])
+ raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+if __name__ == "__main__":
+ sys.exit(0 if run_tests() else 1)
diff --git a/needle/environments/smart_home.py b/needle/environments/smart_home.py
new file mode 100644
index 0000000..36df8f4
--- /dev/null
+++ b/needle/environments/smart_home.py
@@ -0,0 +1,139 @@
+"""Smart home control for a bounded set of rooms and devices.
+
+Replace the Literal values with the rooms and devices your product exposes.
+Keep the shapes: every closed set is an enum, every number has bounds, and
+no correct call ever needs a value the user did not say. One learned rule:
+avoid enum values that hide inside likely query words (a room named office
+poisons an off action, so this home has a study).
+"""
+
+import sys
+from typing import Annotated, Literal, Optional
+
+import needle
+from needle.environments import _harness
+
+Room = Literal["kitchen", "living_room", "bedroom", "study"]
+
+
+def control_lights(
+ room: Room,
+ action: Literal["on", "off", "dim"],
+ brightness_percent: Annotated[Optional[int], needle.Field(ge=0, le=100)] = None,
+ color: Optional[Literal["warm white", "cool white", "red", "green", "blue"]] = None,
+):
+ """Turn lights on or off in a room, dim them to a brightness percentage, or set their color. A color request means action on. Never use this for blinds, fans, or any other device.
+
+ Args:
+ room: The room to control.
+ action: on, off, or dim.
+ brightness_percent: Brightness from 0 to 100. A dim request with a number must carry it in this same call.
+ color: The light color; include only when the user names one.
+ """
+ return {"ok": True, "room": room, "action": action, "brightness_percent": brightness_percent, "color": color}
+
+
+def set_thermostat(temperature: Annotated[int, needle.Field(ge=10, le=30)]):
+ """Set the home thermostat to a target temperature in degrees Celsius. This never controls fans, lights, or any other device.
+
+ Args:
+ temperature: Target temperature in degrees Celsius.
+ """
+ return {"ok": True, "temperature": temperature}
+
+
+def control_fan(
+ room: Literal["living_room", "bedroom", "study"],
+ action: Literal["on", "off"],
+ speed: Optional[Literal["low", "medium", "high"]] = None,
+):
+ """Turn a room fan on or off, optionally at a stated speed. This never changes the thermostat.
+
+ Args:
+ room: The room whose fan to control.
+ action: on or off.
+ speed: Fan speed; include only when stated.
+ """
+ return {"ok": True, "room": room, "action": action, "speed": speed}
+
+
+def control_blinds(room: Room, action: Literal["open", "close"]):
+ """Open or close the window blinds in one stated room. Never pick the room yourself. Never use this for lights or the vacuum.
+
+ Args:
+ room: The room whose blinds to move.
+ action: open or close.
+ """
+ return {"ok": True, "room": room, "action": action}
+
+
+def start_robot_vacuum(
+ action: Literal["start", "stop", "dock"],
+ room: Optional[Literal["kitchen", "living_room", "bedroom"]] = None,
+):
+ """Start or stop the robot vacuum, or send it back to its dock to charge.
+
+ Args:
+ action: start, stop, or dock.
+ room: The room to clean; include only when starting a run in a stated room.
+ """
+ return {"ok": True, "action": action, "room": room}
+
+
+TOOLS = [control_lights, set_thermostat, control_fan, control_blinds, start_robot_vacuum]
+SYSTEM = "Map each explicit supported home action to exactly one declared call; never duplicate an action. Do not guess missing targets or values. Unsupported, invalid, ambiguous, and negated requests return no call."
+
+
+TEST_CASES = [
+ {'query': 'turn on the kitchen lights', 'calls': [{'name': 'control_lights', 'arguments': {'room': 'kitchen', 'action': 'on'}}], 'category': 'positive'},
+ {'query': 'switch off the lights in the bedroom', 'calls': [{'name': 'control_lights', 'arguments': {'room': 'bedroom', 'action': 'off'}}], 'category': 'positive'},
+ {'query': 'dim the living room lights to 35 percent', 'calls': [{'name': 'control_lights', 'arguments': {'room': 'living_room', 'action': 'dim', 'brightness_percent': 35}}], 'category': 'positive'},
+ {'query': 'turn on the study lights in warm white', 'calls': [{'name': 'control_lights', 'arguments': {'room': 'study', 'action': 'on', 'color': 'warm white'}}], 'category': 'positive'},
+ {'query': 'put the bedroom lights on in blue', 'calls': [{'name': 'control_lights', 'arguments': {'room': 'bedroom', 'action': 'on', 'color': 'blue'}}], 'category': 'positive'},
+ {'query': 'set the thermostat to 22 degrees', 'calls': [{'name': 'set_thermostat', 'arguments': {'temperature': 22}}], 'category': 'positive'},
+ {'query': 'warm the house to 24 degrees', 'calls': [{'name': 'set_thermostat', 'arguments': {'temperature': 24}}], 'category': 'positive'},
+ {'query': 'cool the whole home down to 19 degrees', 'calls': [{'name': 'set_thermostat', 'arguments': {'temperature': 19}}], 'category': 'positive'},
+ {'query': 'turn on the bedroom fan', 'calls': [{'name': 'control_fan', 'arguments': {'room': 'bedroom', 'action': 'on'}}], 'category': 'positive'},
+ {'query': 'switch the study fan off', 'calls': [{'name': 'control_fan', 'arguments': {'room': 'study', 'action': 'off'}}], 'category': 'positive'},
+ {'query': 'turn on the living room fan at high speed', 'calls': [{'name': 'control_fan', 'arguments': {'room': 'living_room', 'action': 'on', 'speed': 'high'}}], 'category': 'positive'},
+ {'query': 'run the study fan on low', 'calls': [{'name': 'control_fan', 'arguments': {'room': 'study', 'action': 'on', 'speed': 'low'}}], 'category': 'positive'},
+ {'query': 'open the kitchen blinds', 'calls': [{'name': 'control_blinds', 'arguments': {'room': 'kitchen', 'action': 'open'}}], 'category': 'positive'},
+ {'query': 'close the blinds in the living room', 'calls': [{'name': 'control_blinds', 'arguments': {'room': 'living_room', 'action': 'close'}}], 'category': 'positive'},
+ {'query': 'open up the bedroom blinds', 'calls': [{'name': 'control_blinds', 'arguments': {'room': 'bedroom', 'action': 'open'}}], 'category': 'positive'},
+ {'query': 'start the robot vacuum', 'calls': [{'name': 'start_robot_vacuum', 'arguments': {'action': 'start'}}], 'category': 'positive'},
+ {'query': 'start vacuuming the bedroom', 'calls': [{'name': 'start_robot_vacuum', 'arguments': {'action': 'start', 'room': 'bedroom'}}], 'category': 'positive'},
+ {'query': 'stop the robot vacuum', 'calls': [{'name': 'start_robot_vacuum', 'arguments': {'action': 'stop'}}], 'category': 'positive'},
+ {'query': 'turn on the lights', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'set the thermostat to something comfortable', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'open the blinds', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'turn the fan on', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'lock the back door', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'check whether the robot vacuum is charging', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'play some jazz in the living room', 'calls': [], 'category': 'irrelevant'},
+ {'query': "don't turn on the study lights", 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'do not close the living room blinds', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'never run the vacuum while I am on a call', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'dim the bedroom lights to 150 percent', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'set the thermostat to 40 degrees', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'turn off the bedroom lights and set the thermostat to 18 degrees', 'calls': [{'name': 'control_lights', 'arguments': {'room': 'bedroom', 'action': 'off'}}, {'name': 'set_thermostat', 'arguments': {'temperature': 18}}], 'category': 'parallel'},
+ {'query': 'start the vacuum in the kitchen and open the living room blinds', 'calls': [{'name': 'start_robot_vacuum', 'arguments': {'action': 'start', 'room': 'kitchen'}}, {'name': 'control_blinds', 'arguments': {'room': 'living_room', 'action': 'open'}}], 'category': 'parallel'},
+]
+
+
+def run_tests(min_confidence=0.0, verbose=True):
+ return _harness.run_tests(sys.modules[__name__], min_confidence, verbose)
+
+
+def __getattr__(name):
+ if name == "agent":
+ return _harness.agent_for(sys.modules[__name__])
+ raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+if __name__ == "__main__":
+ sys.exit(0 if run_tests() else 1)
diff --git a/needle/environments/wearable.py b/needle/environments/wearable.py
new file mode 100644
index 0000000..63aa76f
--- /dev/null
+++ b/needle/environments/wearable.py
@@ -0,0 +1,111 @@
+"""Smartwatch companion actions: notifications, workouts, finding the phone.
+
+Replace the sender list with the contacts your integration exposes; the model
+only ever replies to senders it can name. Workout types use the gerund forms
+the model was trained on.
+"""
+
+import sys
+from typing import Annotated, Literal
+
+import needle
+from needle.environments import _harness
+
+Sender = Literal["Maya", "Leo", "Dr. Patel", "Cactus Team"]
+
+
+def reply_to_notification(notification_match: Sender, text: Annotated[str, needle.Field(min_length=1, max_length=240)]):
+ """Reply to a message notification from a known sender. Copy the reply text word for word, preserving capitalization. Use dismiss_notification to clear one without replying.
+
+ Args:
+ notification_match: Who the notification is from.
+ text: The reply message copied word for word.
+ """
+ return {"ok": True, "notification_match": notification_match, "text": text}
+
+
+def dismiss_notification(notification_match: Sender):
+ """Dismiss a sender's notification without replying.
+
+ Args:
+ notification_match: Whose notification to dismiss.
+ """
+ return {"ok": True, "notification_match": notification_match}
+
+
+def start_workout(workout_type: Literal["running", "walking", "cycling", "swimming", "strength", "yoga"]):
+ """Start tracking a workout of an explicitly named type: running, walking, cycling, swimming, strength, or yoga. Use end_workout to finish one already running.
+
+ Args:
+ workout_type: The type of workout the user named.
+ """
+ return {"ok": True, "workout_type": workout_type}
+
+
+def end_workout():
+ """End the current workout session. Starting a new one is start_workout. Takes no arguments."""
+ return {"ok": True, "state": "ended"}
+
+
+def find_my_phone():
+ """Trigger the phone to ring so you can find it. Takes no arguments."""
+ return {"ok": True, "ringing": True}
+
+
+TOOLS = [reply_to_notification, dismiss_notification, start_workout, end_workout, find_my_phone]
+SYSTEM = "Copy reply text verbatim, preserving capitalization. Map each explicit supported watch action to exactly one declared call. Do not guess missing values. Unsupported, invalid, ambiguous, and negated requests return no call."
+
+
+TEST_CASES = [
+ {'query': 'reply to Maya saying be there in 10 minutes', 'calls': [{'name': 'reply_to_notification', 'arguments': {'notification_match': 'Maya', 'text': 'be there in 10 minutes'}}], 'category': 'positive'},
+ {'query': 'reply to Dr. Patel with See you Thursday', 'calls': [{'name': 'reply_to_notification', 'arguments': {'notification_match': 'Dr. Patel', 'text': 'See you Thursday'}}], 'category': 'positive'},
+ {'query': 'answer the Cactus Team message saying the build is green', 'calls': [{'name': 'reply_to_notification', 'arguments': {'notification_match': 'Cactus Team', 'text': 'the build is green'}}], 'category': 'positive'},
+ {'query': 'send Leo a reply that says lunch works for me', 'calls': [{'name': 'reply_to_notification', 'arguments': {'notification_match': 'Leo', 'text': 'lunch works for me'}}], 'category': 'positive'},
+ {'query': 'dismiss the notification from Leo', 'calls': [{'name': 'dismiss_notification', 'arguments': {'notification_match': 'Leo'}}], 'category': 'positive'},
+ {'query': 'swipe away the alert from Maya', 'calls': [{'name': 'dismiss_notification', 'arguments': {'notification_match': 'Maya'}}], 'category': 'positive'},
+ {'query': 'clear the Dr. Patel notification', 'calls': [{'name': 'dismiss_notification', 'arguments': {'notification_match': 'Dr. Patel'}}], 'category': 'positive'},
+ {'query': 'get rid of the Cactus Team notification', 'calls': [{'name': 'dismiss_notification', 'arguments': {'notification_match': 'Cactus Team'}}], 'category': 'positive'},
+ {'query': 'start a running workout', 'calls': [{'name': 'start_workout', 'arguments': {'workout_type': 'running'}}], 'category': 'positive'},
+ {'query': 'begin a swimming session on my watch', 'calls': [{'name': 'start_workout', 'arguments': {'workout_type': 'swimming'}}], 'category': 'positive'},
+ {'query': 'kick off some strength training', 'calls': [{'name': 'start_workout', 'arguments': {'workout_type': 'strength'}}], 'category': 'positive'},
+ {'query': 'track a cycling workout', 'calls': [{'name': 'start_workout', 'arguments': {'workout_type': 'cycling'}}], 'category': 'positive'},
+ {'query': 'go ahead and start a walking workout', 'calls': [{'name': 'start_workout', 'arguments': {'workout_type': 'walking'}}], 'category': 'positive'},
+ {'query': 'end my workout', 'calls': [{'name': 'end_workout', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'finish the current workout session', 'calls': [{'name': 'end_workout', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'wrap up my workout now', 'calls': [{'name': 'end_workout', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'find my phone', 'calls': [{'name': 'find_my_phone', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'make my phone ring so I can locate it', 'calls': [{'name': 'find_my_phone', 'arguments': {}}], 'category': 'positive'},
+ {'query': 'send a reply to Maya', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'dismiss the latest notification', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'start a workout', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'reply saying I am stuck in a meeting', 'calls': [], 'category': 'missing', 'critical': True},
+ {'query': 'check my heart rate on the watch', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'call Maya from my watch', 'calls': [], 'category': 'irrelevant'},
+ {'query': 'show my step count for today', 'calls': [], 'category': 'irrelevant'},
+ {'query': "don't reply to the message from Leo", 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'do not start a running workout yet', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'never dismiss notifications from Dr. Patel', 'calls': [], 'category': 'negation', 'critical': True},
+ {'query': 'reply to Priya saying happy birthday', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'start a pilates workout', 'calls': [], 'category': 'invalid', 'critical': True},
+ {'query': 'dismiss the notification from Leo and start a strength workout', 'calls': [{'name': 'dismiss_notification', 'arguments': {'notification_match': 'Leo'}}, {'name': 'start_workout', 'arguments': {'workout_type': 'strength'}}], 'category': 'parallel'},
+ {'query': 'find my phone and reply to Maya saying almost home', 'calls': [{'name': 'reply_to_notification', 'arguments': {'notification_match': 'Maya', 'text': 'almost home'}}, {'name': 'find_my_phone', 'arguments': {}}], 'category': 'parallel'},
+]
+
+
+def run_tests(min_confidence=0.0, verbose=True):
+ return _harness.run_tests(sys.modules[__name__], min_confidence, verbose)
+
+
+def __getattr__(name):
+ if name == "agent":
+ return _harness.agent_for(sys.modules[__name__])
+ raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+if __name__ == "__main__":
+ sys.exit(0 if run_tests() else 1)
diff --git a/tests/test_environments.py b/tests/test_environments.py
new file mode 100644
index 0000000..e7b1d66
--- /dev/null
+++ b/tests/test_environments.py
@@ -0,0 +1,72 @@
+import re
+
+import pytest
+
+from conftest import requires_engine
+from needle import environments
+
+CATEGORIES = {"positive", "missing", "irrelevant", "negation", "invalid", "parallel"}
+
+
[email protected](params=sorted(environments.ENVIRONMENTS))
+def env(request):
+ return environments.ENVIRONMENTS[request.param]
+
+
+def _schemas(env):
+ return {fn._needle_tool["name"]: fn._needle_tool for fn in env.TOOLS}
+
+
+def test_registry_matches_modules():
+ for name, module in environments.ENVIRONMENTS.items():
+ assert module.__name__ == f"needle.environments.{name}"
+ assert module.SYSTEM and module.TOOLS and module.TEST_CASES
+
+
+def test_tool_surface(env):
+ schemas = _schemas(env)
+ assert 0 < len(schemas) <= 5
+ for schema in schemas.values():
+ assert schema["description"]
+ for prop in schema["parameters"]["properties"].values():
+ assert "type" in prop
+
+
+def test_cases_cover_all_categories(env):
+ assert {case["category"] for case in env.TEST_CASES} == CATEGORIES
+
+
+def test_cases_match_declared_tools(env):
+ schemas = _schemas(env)
+ for case in env.TEST_CASES:
+ for call in case["calls"]:
+ parameters = schemas[call["name"]]["parameters"]
+ arguments = call["arguments"]
+ assert set(arguments) <= set(parameters["properties"])
+ assert set(parameters.get("required", [])) <= set(arguments)
+ for key, value in arguments.items():
+ prop = parameters["properties"][key]
+ if "enum" in prop:
+ assert value in prop["enum"]
+ if "minimum" in prop:
+ assert value >= prop["minimum"]
+ if "maximum" in prop:
+ assert value <= prop["maximum"]
+ if "pattern" in prop:
+ assert re.match(prop["pattern"], value)
+
+
+def test_tools_execute(env):
+ schemas = {fn._needle_tool["name"]: fn for fn in env.TOOLS}
+ for case in env.TEST_CASES:
+ for call in case["calls"]:
+ result = schemas[call["name"]](**call["arguments"])
+ assert result["ok"] is True
+
+
+@requires_engine
+def test_smart_home_smoke():
+ from needle.environments import smart_home
+
+ response = smart_home.agent.complete("turn on the kitchen lights")
+ assert response.get("type") in ("call", "text", "refuse")