Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat(integrations): add LayaToolSelector for sub-35ms tool and function routing

LEVELING2108586e33a32957
diff --git a/docs/.nav.yml b/docs/.nav.yml
index a6893a69..6b67bbb5 100644
--- a/docs/.nav.yml
+++ b/docs/.nav.yml
@@ -9,6 +9,7 @@ nav:
- langchain.md
- llamaindex.md
- crewai.md
+ - tools.md
- glob: "*"
ignore_no_matches: true
- Deployment:
diff --git a/docs/tools.md b/docs/tools.md
new file mode 100644
index 00000000..3ccc7fd2
--- /dev/null
+++ b/docs/tools.md
@@ -0,0 +1,165 @@
+# Tool & Function Selection
+
+Laya provides sub-35ms, non-autoregressive tool routing and function selection for AI agent workflows across **LangChain**, **CrewAI**, **LlamaIndex**, and vanilla agent architectures (single-question latency measured at **32.8 ms** with `laya-multilingual` and **39.5 ms** with `laya` on a Tesla T4 GPU; 193–464 ms on CPU):
+
+* **`LayaToolSelector`**: Sub-35ms tool router evaluating user queries and agent states against candidate tools in a single forward pass.
+* **`ToolRouteDecision`**: Structured routing result carrying the selected tool, tool index, calibrated confidence, probability distribution across tools, and fallback status.
+
+Replaces slow, token-generating LLM tool-calling (1,000–2,500 ms) with deterministic, typed decisions executed in <35 ms with **zero token generation cost**.
+
+Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` instance without requiring PyTorch on edge clients.
+
+---
+
+## Installation
+
+```bash
+pip install laya
+```
+
+---
+
+## 1. Sub-35ms Tool Selection
+
+Autoregressive LLMs (such as GPT-4o or Claude 3.5 Sonnet) take 1–2 seconds generating reasoning tokens just to choose which tool to call. `LayaToolSelector` evaluates candidate tools against the query in **~33 ms**:
+
+```python
+from laya.integrations.tools import LayaToolSelector
+
+def search_web(query: str):
+ """Search Google or Bing for real-time news and websites."""
+ return f"Search results for: {query}"
+
+def calculate_mortgage(principal: float, rate: float, years: int):
+ """Calculate monthly mortgage amortization payments and interest."""
+ return "Monthly payment: $2,450"
+
+def query_database(sql: str):
+ """Execute read-only SQL queries against Postgres customer database."""
+ return "Rows returned: 12"
+
+tools = [search_web, calculate_mortgage, query_database]
+
+# Initialize sub-35ms tool selector
+selector = LayaToolSelector(tools=tools)
+
+# Select the matching tool in <35ms:
+decision = selector.select("What is the current mortgage rate on a 30-year fixed loan?")
+print(f"Selected Tool: {decision.tool_name}")
+print(f"Confidence: {decision.confidence:.3f}")
+print(f"Probabilities: {decision.probabilities}")
+```
+
+Both synchronous `select()` and non-blocking asynchronous `aselect()` are supported.
+
+---
+
+## 2. Direct Execution (`call`)
+
+`call()` selects the candidate tool and immediately executes it with the supplied arguments:
+
+```python
+result = selector.call("Calculate payments for 500k at 6.5% for 30 years",
+ principal=500000, rate=0.065, years=30)
+print(result) # "Monthly payment: $2,450"
+```
+
+If the tool is a LangChain tool with `.run()`, `call()` automatically invokes its execution method.
+
+---
+
+## 3. Direct Answer Detection (No Tool Needed)
+
+In conversational agent pipelines, many user queries do not require external tool invocation (e.g. greetings, general conversational questions). Setting `allow_direct_answer=True` enables Laya to recognize when to answer conversationally without calling any tool:
+
+```python
+selector = LayaToolSelector(
+ tools=[search_web, query_database],
+ allow_direct_answer=True,
+)
+
+decision = selector.select("Hello! How can you help me today?")
+if decision.is_direct_answer:
+ print("No tool needed: respond conversationally.")
+else:
+ print(f"Call tool: {decision.tool_name}")
+```
+
+---
+
+## 4. Calibrated Confidence Gating & Fallbacks
+
+`LayaToolSelector` uses Laya's calibrated confidence scores to gate routing decisions. When an ambiguous or out-of-domain query is presented, the selector avoids bad tool invocations by falling back to a general agent or raising `LayaLowConfidenceError`:
+
+```python
+from laya.integrations._errors import LayaLowConfidenceError
+
+def general_fallback_llm(prompt: str):
+ """Fallback handler for ambiguous or novel requests."""
+ return "Dispatching to general reasoning LLM..."
+
+selector = LayaToolSelector(
+ tools=[search_web, calculate_mortgage],
+ confidence_threshold=0.85,
+ fallback_tool=general_fallback_llm,
+)
+
+decision = selector.select("Compose a haiku about autumn leaves.")
+if decision.is_fallback:
+ print(f"Ambiguous request ({decision.confidence:.2f} < 0.85). Routed to fallback handler.")
+```
+
+If `fallback_tool` is omitted, requests below threshold raise `LayaLowConfidenceError`:
+
+```python
+try:
+ decision = selector.select("Unclear request")
+except LayaLowConfidenceError as e:
+ print(f"Abstained: confidence {e.confidence:.3f} below threshold {e.threshold}")
+```
+
+---
+
+## 5. LangChain LCEL Integration
+
+`LayaToolSelector` exports an LCEL-compatible Runnable via `.as_runnable()`, allowing it to be composed seamlessly into LangGraph nodes or LangChain pipelines:
+
+```python
+from laya.integrations.tools import LayaToolSelector
+
+selector = LayaToolSelector(tools=[search_web, query_database])
+
+# Compose with pipe operator |
+chain = selector.as_runnable() | (lambda decision: decision.tool_name)
+tool_to_call = chain.invoke("Look up order #45920 in the database")
+print(f"Dispatching to: {tool_to_call}")
+```
+
+---
+
+## 6. Remote HTTP Mode (`laya-serve`)
+
+In serverless or microservice architectures, `LayaToolSelector` connects directly to a self-hosted `laya-serve` cluster using standard library `urllib` (no external HTTP client required):
+
+```python
+selector = LayaToolSelector(
+ tools=[search_web, query_database],
+ base_url="http://laya-cluster.internal:8000",
+ api_key="your-cluster-secret",
+ model="laya-multilingual",
+)
+
+decision = selector.select("Search for latest updates")
+```
+
+---
+
+## 7. Supported Tool Formats
+
+`LayaToolSelector` automatically parses candidate tools from all standard formats:
+* **Python functions**: extracts `__name__` and first line of docstring `__doc__`.
+* **LangChain & CrewAI tools**: reads `.name` and `.description`.
+* **LlamaIndex tools**: reads `tool.metadata.name` and `tool.metadata.description`.
+* **JSON Schema / OpenAI specs**: parses `{"type": "function", "function": {"name": ..., "description": ...}}`.
+* **Dictionaries**: reads `{"name": ..., "description": ...}`.
+* **Plain strings**: parses `"tool_name: tool description"`.
diff --git a/examples/tool_selection_quickstart.py b/examples/tool_selection_quickstart.py
new file mode 100644
index 00000000..12a2201a
--- /dev/null
+++ b/examples/tool_selection_quickstart.py
@@ -0,0 +1,152 @@
+"""Laya System 1 decision engine: Tool and Function Selection Quickstart.
+
+Demonstrates:
+1. Sub-35ms tool selection for AI agent workflows (replaces 1-2s LLM tool calling).
+2. Direct execution via .call() with function arguments.
+3. Direct answer detection when no tool is needed (allow_direct_answer=True).
+4. Calibrated confidence threshold fallback gating for ambiguous requests.
+5. LangChain LCEL Runnable integration with pipe (|) operator.
+"""
+import os
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from laya.integrations.tools import (
+ LayaLowConfidenceError,
+ LayaToolSelector,
+)
+
+
+# Optional lightweight demo agent for running without local model checkpoints
+class _DemoAgent:
+ def predict(self, state, questions, **kwargs):
+ text = str(state).lower()
+ q_id = next(iter(questions.keys()))
+
+ words = set(text.replace("?", "").replace(",", "").replace("!", "").split())
+ if any(w in words for w in ("hello", "hi", "greetings")) or "how are you" in text:
+ chosen = "direct_answer"
+ conf = 0.98
+ elif any(w in words for w in ("search", "weather", "news", "president", "tokyo", "paris")):
+ chosen = "tool_0"
+ conf = 0.95
+ elif any(w in words for w in ("sql", "database", "orders", "users", "postgres", "query", "select")):
+ chosen = "tool_2"
+ conf = 0.92
+ elif any(w in words for w in ("tax", "calculate", "mortgage", "+", "multiply", "math", "sum")):
+ chosen = "tool_1"
+ conf = 0.94
+ else:
+ chosen = "tool_0"
+ conf = 0.40 # Low confidence on ambiguous input
+
+ return {
+ "model": "laya-demo",
+ "answers": {
+ q_id: {
+ "type": "choice",
+ "choice": chosen,
+ "confidence": conf,
+ "answer_confidence": conf,
+ "probabilities": {chosen: conf},
+ }
+ },
+ }
+
+
+# Define tool candidates
+def search_web(query: str) -> str:
+ """Search Google or Bing for real-time news and websites."""
+ return f"[Web Results] Found 10 results for query: {query}"
+
+
+def calculate_tax(amount: float, rate: float = 0.08) -> float:
+ """Calculate sales and state taxes on purchase amount."""
+ return amount * rate
+
+
+def query_database(sql: str) -> str:
+ """Execute read-only SQL queries against Postgres customer database."""
+ return f"[Postgres] Returned 5 rows for: {sql}"
+
+
+def general_reasoning_fallback(prompt: str) -> str:
+ """Fallback handler for ambiguous or complex reasoning requests."""
+ return f"[Fallback LLM] Processing complex request: {prompt}"
+
+
+def main():
+ print("=" * 70)
+ print(" Laya System 1: Sub-35ms Tool and Function Selection")
+ print("=" * 70)
+
+ tools = [search_web, calculate_tax, query_database]
+ agent = _DemoAgent()
+
+ # 1. Standard Tool Selection
+ print("\n[1] Sub-35ms Tool Selection:")
+ selector = LayaToolSelector(tools=tools, agent=agent)
+
+ queries = [
+ "What is the current weather forecast in Tokyo?",
+ "Calculate the 8% tax on a $250 purchase",
+ "SELECT * FROM orders WHERE status = 'shipped'",
+ ]
+
+ for q in queries:
+ decision = selector.select(q)
+ print(f" Query: {q}")
+ print(f" -> Tool: {decision.tool_name}")
+ print(f" -> Conf: {decision.confidence:.3f}")
+ print(f" -> Reason: {decision.reason}\n")
+
+ # 2. Direct Execution via .call()
+ print("[2] Direct Tool Execution via selector.call():")
+ tax_value = selector.call("Calculate sales tax on $500", tools=None, amount=500.0, rate=0.08)
+ print(f" Calculated Tax: ${tax_value:.2f}\n")
+
+ # 3. Direct Answer Detection (No Tool Required)
+ print("[3] Direct Conversational Answer Detection:")
+ selector_direct = LayaToolSelector(tools=tools, allow_direct_answer=True, agent=agent)
+ decision = selector_direct.select("Hello! Can you help me today?")
+ if decision.is_direct_answer:
+ print(" -> Detected conversational query: Answer directly without calling tools!\n")
+
+ # 4. Confidence Threshold & Fallbacks
+ print("[4] Confidence Gating & Fallback Dispatch:")
+ ambiguous_query = "Summarize the philosophical differences between Kant and Hegel"
+
+ # With fallback tool
+ selector_fb = LayaToolSelector(
+ tools=tools,
+ confidence_threshold=0.80,
+ fallback_tool=general_reasoning_fallback,
+ agent=agent,
+ )
+ decision = selector_fb.select(ambiguous_query)
+ print(f" Query: {ambiguous_query}")
+ print(f" -> Selected: {decision.tool_name}")
+ print(f" -> Confidence: {decision.confidence:.3f} (< 0.80 threshold)")
+ print(f" -> Fallback: {decision.is_fallback}\n")
+
+ # Without fallback tool: raises LayaLowConfidenceError
+ selector_gate = LayaToolSelector(tools=tools, confidence_threshold=0.80, agent=agent)
+ try:
+ selector_gate.select(ambiguous_query)
+ except LayaLowConfidenceError as e:
+ print(f" -> Successfully caught LayaLowConfidenceError: {e}\n")
+
+ # 5. LangChain LCEL Composition
+ print("[5] LangChain LCEL Runnable Composition:")
+ chain = selector.as_runnable() | (lambda d: f"Chain routed to: {d.tool_name}")
+ routed_output = chain.invoke("Search for the latest tech news")
+ print(f" -> {routed_output}\n")
+
+ print("=" * 70)
+ print(" Laya Tool Selection Quickstart completed successfully!")
+ print("=" * 70)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/laya/__init__.py b/laya/__init__.py
index 16e4ef2e..bb85bbd5 100644
--- a/laya/__init__.py
+++ b/laya/__init__.py
@@ -77,6 +77,8 @@ _LAZY_ATTRS = {
"LayaTriage": (".integrations", "LayaTriage"),
"LayaEvaluator": (".integrations", "LayaEvaluator"),
"LayaDecision": (".integrations", "LayaDecision"),
+ "LayaToolSelector": (".integrations", "LayaToolSelector"),
+ "ToolRouteDecision": (".integrations", "ToolRouteDecision"),
}
@@ -144,6 +146,8 @@ __all__ = [
"LayaTriage",
"LayaEvaluator",
"LayaDecision",
+ "LayaToolSelector",
+ "ToolRouteDecision",
"PredictContext",
"PredictHook",
"Hook",
diff --git a/laya/integrations/__init__.py b/laya/integrations/__init__.py
index f99ed903..0279b01f 100644
--- a/laya/integrations/__init__.py
+++ b/laya/integrations/__init__.py
@@ -19,6 +19,10 @@ from .llamaindex import (
LayaQueryRouter,
LayaSingleSelector,
)
+from .tools import (
+ LayaToolSelector,
+ ToolRouteDecision,
+)
__all__ = [
"LayaRouter",
@@ -34,5 +38,7 @@ __all__ = [
"LayaTaskGuard",
"LayaTaskGuardError",
"CrewRouteDecision",
+ "LayaToolSelector",
+ "ToolRouteDecision",
"LayaLowConfidenceError",
]
diff --git a/laya/integrations/tools.py b/laya/integrations/tools.py
new file mode 100644
index 00000000..dbd474f5
--- /dev/null
+++ b/laya/integrations/tools.py
@@ -0,0 +1,451 @@
+"""Tool and function selection for agent frameworks using Laya System 1 decision engine.
+
+Provides sub-35ms, non-autoregressive tool routing and function selection for AI agent
+workflows (LangChain, CrewAI, LlamaIndex, AutoGen, and custom agent loops).
+
+Replaces slow, token-generating LLM tool calling (1,000-2,500ms) with calibrated, typed
+decisions executed in a single forward pass without token generation cost.
+
+Supports Python functions, LangChain/CrewAI/LlamaIndex tools, and JSON Schema specifications.
+Operates seamlessly with local in-process models (`Agent` / `Router`) and remote HTTP
+deployments (self-hosted `laya-serve`) without heavy client dependencies.
+"""
+from __future__ import annotations
+
+import asyncio
+import json
+import threading
+import urllib.error
+import urllib.parse
+import urllib.request
+from dataclasses import dataclass
+from typing import Any, Dict, Optional, Sequence, Tuple, Union
+
+# Shared per-call control rules across integrations
+from ._controls import budget_kwargs as _budget_kwargs
+from ._controls import decision_kwargs as _decision_kwargs
+from ._controls import hook_kwargs as _hook_kwargs
+from ._controls import predict_kwargs as _predict_kwargs
+from ._controls import reject_remote_hooks as _reject_remote_hooks
+from ._errors import LayaLowConfidenceError
+from ..confidence import _gate_confidence
+
+
+def _gated_confidence(answer: Dict[str, Any]) -> float:
+ """Extract calibrated confidence score from a Laya decision answer."""
+ conf = _gate_confidence(answer or {})
+ return conf if conf is not None else 1.0
+
+
+@dataclass
+class ToolRouteDecision:
+ """Result of a Laya sub-35ms tool routing decision."""
+
+ tool: Any
+ tool_name: str
+ tool_index: int
+ confidence: float
+ probabilities: Dict[str, float]
+ reason: str
+ is_direct_answer: bool
+ is_fallback: bool
+ raw_decision: Dict[str, Any]
+
+
+def _extract_tool_metadata(tool: Any, default_index: int) -> Tuple[str, str]:
+ """Extract (name, description) from any tool representation."""
+ if isinstance(tool, str):
+ if ":" in tool:
+ parts = tool.split(":", 1)
+ return parts[0].strip(), parts[1].strip()
+ return tool.strip(), tool.strip()
+
+ if isinstance(tool, dict):
+ if "function" in tool and isinstance(tool["function"], dict):
+ fn = tool["function"]
+ name = fn.get("name") or f"tool_{default_index}"
+ desc = fn.get("description") or name
+ return str(name), str(desc)
+ name = tool.get("name") or f"tool_{default_index}"
+ desc = tool.get("description") or tool.get("doc") or name
+ return str(name), str(desc)
+
+ # Check for LlamaIndex tool.metadata
+ if hasattr(tool, "metadata"):
+ meta = tool.metadata
+ name = getattr(meta, "name", None) or f"tool_{default_index}"
+ desc = getattr(meta, "description", None) or name
+ return str(name), str(desc)
+
+ # Check for .name and .description attributes (LangChain, CrewAI, custom classes)
+ name = getattr(tool, "name", None)
+ desc = getattr(tool, "description", None)
+ if name is not None:
+ return str(name), str(desc or name)
+
+ # Check callable / function
+ if callable(tool):
+ fn_name = getattr(tool, "__name__", None) or f"tool_{default_index}"
+ doc = getattr(tool, "__doc__", None)
+ desc = doc.strip().split("\n")[0] if doc else fn_name
+ return str(fn_name), str(desc)
+
+ return f"tool_{default_index}", f"Tool {default_index}"
+
+
+def _format_tools_criteria(
+ tools: Sequence[Any],
+ allow_direct_answer: bool = False,
+ direct_answer_key: str = "__direct_answer__",
+ direct_answer_description: str = "Answer the user request directly without calling any tools.",
+) -> Tuple[Dict[str, str], Dict[str, Tuple[int, Any, str]]]:
+ """Format candidate tools into Laya choice criteria and reverse lookup map."""
+ criteria: Dict[str, str] = {}
+ lookup: Dict[str, Tuple[int, Any, str]] = {}
+
+ for i, tool in enumerate(tools):
+ name, desc = _extract_tool_metadata(tool, i)
+ key = f"tool_{i}"
+ criteria[key] = f"{name}: {desc}" if desc and desc != name else name
+ lookup[key] = (i, tool, name)
+
+ if allow_direct_answer:
+ direct_key = "direct_answer"
+ criteria[direct_key] = f"{direct_answer_key}: {direct_answer_description}"
+ lookup[direct_key] = (-1, None, direct_answer_key)
+
+ return criteria, lookup
+
+
+class _SameOriginRedirectHandler(urllib.request.HTTPRedirectHandler):
+ """Refuse redirects that switch origins or downgrade from HTTPS to HTTP."""
+
+ def redirect_request(self, req, fp, code, msg, headers, newurl):
+ old = urllib.parse.urlparse(req.full_url)
+ new = urllib.parse.urlparse(newurl)
+ old_port = old.port or (443 if old.scheme.lower() == "https" else 80)
+ new_port = new.port or (443 if new.scheme.lower() == "https" else 80)
+ if (
+ old.scheme.lower() != new.scheme.lower()
+ or (old.hostname or "").lower() != (new.hostname or "").lower()
+ or old_port != new_port
+ ):
+ raise urllib.error.URLError("refusing cross-origin or HTTPS-downgrade redirect")
+ return super().redirect_request(req, fp, code, msg, headers, newurl)
+
+
+def _call_remote(
+ base_url: str,
+ state: Any,
+ questions: Dict[str, Any],
+ api_key: Optional[str] = None,
+ model: Optional[str] = None,
+ timeout: float = 10.0,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
+ lang: Optional[str] = None,
+ min_confidence: Optional[float] = None,
+) -> Dict[str, Any]:
+ """Send decision request to a remote laya-serve HTTP instance using standard library urllib."""
+ url = base_url.rstrip("/")
+ if not url.endswith("/v1/systemone"):
+ url = f"{url}/v1/systemone"
+
+ payload: Dict[str, Any] = {"state": state, "questions": questions}
+ if model:
+ payload["model"] = model
+ if max_len is not None:
+ payload["max_len"] = max_len
+ if head_max_len is not None:
+ payload["head_max_len"] = head_max_len
+ if lang is not None:
+ payload["lang"] = lang
+ if min_confidence is not None:
+ payload["min_confidence"] = min_confidence
+
+ data = json.dumps(payload).encode("utf-8")
+ headers = {"Content-Type": "application/json"}
+ if api_key:
+ headers["Authorization"] = f"Bearer {api_key}"
+
+ req = urllib.request.Request(url, data=data, headers=headers, method="POST")
+ opener = urllib.request.build_opener(_SameOriginRedirectHandler())
+ try:
+ with opener.open(req, timeout=timeout) as resp:
+ return json.loads(resp.read().decode("utf-8"))
+ except urllib.error.HTTPError as e:
+ body = e.read().decode("utf-8", errors="replace")
+ raise RuntimeError(f"Laya server error {e.code}: {body}") from e
+ except Exception as e:
+ raise RuntimeError(f"Failed to connect to Laya server at {url}: {e}") from e
+
+
+_DEFAULT_ROUTER = None
+_DEFAULT_ROUTER_LOCK = threading.Lock()
+
+
+def _get_default_router():
+ global _DEFAULT_ROUTER
+ if _DEFAULT_ROUTER is None:
+ with _DEFAULT_ROUTER_LOCK:
+ if _DEFAULT_ROUTER is None:
+ from ..router import Router
+ _DEFAULT_ROUTER = Router()
+ return _DEFAULT_ROUTER
+
+
+def _execute_decision(
+ state: Any,
+ questions: Dict[str, Any],
+ agent: Optional[Any] = None,
+ base_url: Optional[str] = None,
+ api_key: Optional[str] = None,
+ model: Optional[str] = None,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
+ lang: Optional[str] = None,
+ min_confidence: Optional[float] = None,
+ hooks: Optional[Any] = None,
+ on_predict_start: Optional[Any] = None,
+ on_predict_end: Optional[Any] = None,
+ hooks_raise: Optional[bool] = None,
+ hooks_timeout: Optional[float] = None,
+) -> Dict[str, Any]:
+ """Execute decision on local runner or remote laya-serve instance."""
+ hook_kwargs = _hook_kwargs(hooks, on_predict_start, on_predict_end, hooks_raise, hooks_timeout)
+ if base_url:
+ _reject_remote_hooks(hook_kwargs, base_url)
+ budget = _budget_kwargs(max_len, head_max_len)
+ decision = _decision_kwargs(lang, min_confidence)
+ return _call_remote(base_url, state, questions, api_key=api_key, model=model,
+ **budget, **decision)
+ runner = agent if agent is not None else _get_default_router()
+ kwargs = _predict_kwargs(model, max_len, head_max_len, lang, min_confidence)
+ kwargs.update(hook_kwargs)
+ return runner.predict(state, questions, **kwargs)
+
+
+class LayaToolSelector:
+ """Sub-35ms tool and function router for AI agent workflows.
+
+ Evaluates user queries and agent states against candidate tools in a single
+ non-autoregressive forward pass, eliminating 1-2 second LLM tool-calling latency.
+
+ Supports Python functions, LangChain/CrewAI/LlamaIndex tools, and JSON Schema specs.
+ Works seamlessly with local in-process models (`Agent` / `Router`) and remote
+ HTTP `laya-serve` instances.
+ """
+
+ def __init__(
+ self,
+ tools: Optional[Sequence[Any]] = None,
+ instructions: str = "Which tool should be called to handle the user request?",
+ allow_direct_answer: bool = False,
+ direct_answer_description: str = "Answer the user request directly without calling any tools.",
+ confidence_threshold: Optional[float] = None,
+ fallback_tool: Optional[Any] = None,
+ agent: Optional[Any] = None,
+ base_url: Optional[str] = None,
+ api_key: Optional[str] = None,
+ model: Optional[str] = None,
+ max_len: Optional[int] = None,
+ head_max_len: Optional[int] = None,
+ lang: Optional[str] = None,
+ min_confidence: Optional[float] = None,
+ hooks: Optional[Any] = None,
+ on_predict_start: Optional[Any] = None,
+ on_predict_end: Optional[Any] = None,
+ hooks_raise: Optional[bool] = None,
+ hooks_timeout: Optional[float] = None,
+ ):
+ self.tools = list(tools) if tools is not None else None
+ self.instructions = instructions
+ self.allow_direct_answer = bool(allow_direct_answer)
+ self.direct_answer_description = direct_answer_description
+ self.confidence_threshold = confidence_threshold
+ self.fallback_tool = fallback_tool
+ self.agent = agent
+ self.base_url = base_url
+ self.api_key = api_key
+ self.model = model
+ self.max_len = max_len
+ self.head_max_len = head_max_len
+ self.lang = lang
+ self.min_confidence = min_confidence
+ self.hooks = hooks
+ self.on_predict_start = on_predict_start
+ self.on_predict_end = on_predict_end
+ self.hooks_raise = hooks_raise
+ self.hooks_timeout = hooks_timeout
+
+ def select(
+ self,
+ state: Union[str, dict, list, Any],
+ tools: Optional[Sequence[Any]] = None,
+ instructions: Optional[str] = None,
+ ) -> ToolRouteDecision:
+ """Select the best candidate tool for the given query/state in sub-35ms."""
+ candidate_tools = tools if tools is not None else self.tools
+ if not candidate_tools and not self.allow_direct_answer:
+ raise ValueError("No tools provided to LayaToolSelector.")
+
+ candidate_tools = candidate_tools or []
+ instruct = instructions or self.instructions
+
+ criteria, lookup = _format_tools_criteria(
+ candidate_tools,
+ allow_direct_answer=self.allow_direct_answer,
+ direct_answer_description=self.direct_answer_description,
+ )
+
+ question = {
+ "type": "choice",
+ "instructions": instruct,
+ "criteria": criteria,
+ }
+ questions = {"tool_select": question}
+
+ state_payload = state if isinstance(state, (str, dict, list)) else str(state)
+ result = _execute_decision(
+ state_payload,
+ questions,
+ agent=self.agent,
+ base_url=self.base_url,
+ api_key=self.api_key,
+ model=self.model,
+ max_len=self.max_len,
+ head_max_len=self.head_max_len,
+ lang=self.lang,
+ min_confidence=self.min_confidence,
+ hooks=self.hooks,
+ on_predict_start=self.on_predict_start,
+ on_predict_end=self.on_predict_end,
+ hooks_raise=self.hooks_raise,
+ hooks_timeout=self.hooks_timeout,
+ )
+
+ answer = result.get("answers", {}).get("tool_select", {})
+ chosen_key = answer.get("choice")
+ conf = _gated_confidence(answer)
+ raw_probs = answer.get("probabilities", {})
+
+ # Map internal keys back to tool names in probabilities
+ probabilities: Dict[str, float] = {}
+ for k, prob in raw_probs.items():
+ if k in lookup:
+ tool_info = lookup[k]
+ probabilities[tool_info[2]] = float(prob)
+ else:
+ probabilities[k] = float(prob)
+
+ # Check confidence gating
+ threshold = self.confidence_threshold if self.confidence_threshold is not None else self.min_confidence
+ is_low_confidence = bool(answer.get("low_confidence")) or (threshold is not None and conf < threshold)
+
+ if is_low_confidence:
+ if self.fallback_tool is not None:
+ fb_name, _ = _extract_tool_metadata(self.fallback_tool, -1)
+ return ToolRouteDecision(
+ tool=self.fallback_tool,
+ tool_name=fb_name,
+ tool_index=-1,
+ confidence=conf,
+ probabilities=probabilities,
+ reason=f"Confidence {conf:.3f} below threshold {threshold}; routed to fallback tool",
+ is_direct_answer=False,
+ is_fallback=True,
+ raw_decision=result,
+ )
+ raise LayaLowConfidenceError(
+ f"Laya tool selection confidence {conf:.3f} is below threshold {threshold}",
+ confidence=conf,
+ threshold=float(threshold) if threshold is not None else 0.0,
+ raw_decision=result,
+ )
+
+ if chosen_key not in lookup:
+ # Fallback to first available tool if unmapped key returned
+ first_key = next(iter(lookup))
+ idx, tool_obj, name = lookup[first_key]
+ else:
+ idx, tool_obj, name = lookup[chosen_key]
+
+ is_direct = (idx == -1)
+ reason = (
+ "Direct answer requested; no tool required"
+ if is_direct
+ else f"Selected tool '{name}' with confidence {conf:.3f}"
+ )
+
+ return ToolRouteDecision(
+ tool=tool_obj,
+ tool_name=name,
+ tool_index=idx,
+ confidence=conf,
+ probabilities=probabilities,
+ reason=reason,
+ is_direct_answer=is_direct,
+ is_fallback=False,
+ raw_decision=result,
+ )
+
+ async def aselect(
+ self,
+ state: Union[str, dict, list, Any],
+ tools: Optional[Sequence[Any]] = None,
+ instructions: Optional[str] = None,
+ ) -> ToolRouteDecision:
+ """Asynchronously select the best candidate tool."""
+ loop = asyncio.get_running_loop()
+ return await loop.run_in_executor(None, self.select, state, tools, instructions)
+
+ def call(
+ self,
+ state: Union[str, dict, list, Any],
+ tools: Optional[Sequence[Any]] = None,
+ *args: Any,
+ **kwargs: Any,
+ ) -> Any:
+ """Select the appropriate tool and invoke it directly with the provided arguments."""
+ decision = self.select(state, tools=tools)
+ if decision.is_direct_answer:
+ return None
+ tool = decision.tool
+ if callable(tool):
+ return tool(*args, **kwargs)
+ if hasattr(tool, "run") and callable(tool.run):
+ return tool.run(*args, **kwargs)
+ if hasattr(tool, "invoke") and callable(tool.invoke):
+ return tool.invoke(*args, **kwargs)
+ raise TypeError(f"Selected tool '{decision.tool_name}' ({type(tool).__name__}) is not callable.")
+
+ def as_runnable(self) -> Any:
+ """Return a LangChain-compatible Runnable wrapper for use in LCEL chains."""
+ try:
+ from langchain_core.runnables import RunnableLambda
+ return RunnableLambda(lambda inp: self.select(inp))
+ except ImportError:
+ class _SimpleRunnable:
+ def __init__(self, selector: LayaToolSelector):
+ self.selector = selector
+
+ def invoke(self, inp: Any) -> ToolRouteDecision:
+ return self.selector.select(inp)
+
+ def __or__(self, other: Any) -> Any:
+ class _ChainedRunnable:
+ def __init__(self, first: Any, second: Any):
+ self.first = first
+ self.second = second
+
+ def invoke(self, inp: Any) -> Any:
+ res = self.first.invoke(inp)
+ if hasattr(self.second, "invoke"):
+ return self.second.invoke(res)
+ if callable(self.second):
+ return self.second(res)
+ return res
+
+ return _ChainedRunnable(self, other)
+
+ return _SimpleRunnable(self)
diff --git a/tests/test_tools.py b/tests/test_tools.py
new file mode 100644
index 00000000..19705f2f
--- /dev/null
+++ b/tests/test_tools.py
@@ -0,0 +1,306 @@
+"""Unit tests for Laya Tool and Function Selection integration.
+
+Tests verify tool metadata extraction, criteria formatting, confidence threshold
+fallback gating, direct execution, remote HTTP payload construction, and async
+selection without requiring model weights, GPU, or external network access.
+"""
+import asyncio
+import inspect
+import json
+import os
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+import laya
+from laya.integrations._controls import (
+ DECISION_CONTROLS,
+ HOOK_CONTROLS,
+ PREDICT_CONTROLS,
+)
+from laya.integrations.tools import (
+ LayaLowConfidenceError,
+ LayaToolSelector,
+ _extract_tool_metadata,
+ _format_tools_criteria,
+)
+
+PASS, FAIL = [], []
+
+
+def check(name, got, want):
+ if got == want:
+ PASS.append(name)
+ else:
+ FAIL.append(f"{name}:\n got {got!r}\n want {want!r}")
+
+
+def check_true(name, cond, detail=""):
+ if cond:
+ PASS.append(name)
+ else:
+ FAIL.append(f"{name} {detail}")
+
+
+# --------------------------------------------------------------- Mock Agent
+class MockLayaAgent:
+ """Mock agent returning deterministic responses for testing."""
+
+ def __init__(self, response_fn):
+ self.response_fn = response_fn
+ self.calls = []
+
+ def predict(self, state, questions, **kwargs):
+ self.calls.append({"state": state, "questions": questions, "kwargs": kwargs})
+ return self.response_fn(state, questions)
+
+
+# --------------------------------------------------------------- 1. Tool Metadata Extraction
+# String tools
+check("extract/str_colon", _extract_tool_metadata("search: Search web", 0), ("search", "Search web"))
+check("extract/str_plain", _extract_tool_metadata("calculator", 1), ("calculator", "calculator"))
+
+# Dict tools
+check("extract/dict", _extract_tool_metadata({"name": "sql", "description": "Run SQL query"}, 0),
+ ("sql", "Run SQL query"))
+check("extract/dict_function",
+ _extract_tool_metadata({"type": "function", "function": {"name": "get_stock", "description": "Fetch stock price"}}, 0),
+ ("get_stock", "Fetch stock price"))
+
+# Class / Object tools (LangChain / CrewAI style)
+class DummyLangChainTool:
+ name = "web_search"
+ description = "Searches Google for query"
+
+check("extract/obj_attrs", _extract_tool_metadata(DummyLangChainTool(), 0),
+ ("web_search", "Searches Google for query"))
+
+# LlamaIndex style with metadata
+class DummyLlamaIndexMeta:
+ name = "rag_search"
+ description = "Search vector index"
+
+class DummyLlamaIndexTool:
+ metadata = DummyLlamaIndexMeta()
+
+check("extract/llama_meta", _extract_tool_metadata(DummyLlamaIndexTool(), 0),
+ ("rag_search", "Search vector index"))
+
+# Python callable function
+def calculate_tax(amount: float) -> float:
+ """Calculate standard state tax."""
+ return amount * 0.08
+
+check("extract/function", _extract_tool_metadata(calculate_tax, 0),
+ ("calculate_tax", "Calculate standard state tax."))
+
+
+# --------------------------------------------------------------- 2. Criteria Formatting
+tools_list = [
+ DummyLangChainTool(),
+ calculate_tax,
+ {"name": "db_lookup", "description": "Query Postgres database"},
+]
+
+criteria_no_direct, lookup_no_direct = _format_tools_criteria(tools_list, allow_direct_answer=False)
+check("criteria/count", len(criteria_no_direct), 3)
+check("criteria/tool_0", criteria_no_direct["tool_0"], "web_search: Searches Google for query")
+check("criteria/tool_1", criteria_no_direct["tool_1"], "calculate_tax: Calculate standard state tax.")
+check("criteria/tool_2", criteria_no_direct["tool_2"], "db_lookup: Query Postgres database")
+check("lookup/tool_0", lookup_no_direct["tool_0"][2], "web_search")
+
+criteria_direct, lookup_direct = _format_tools_criteria(tools_list, allow_direct_answer=True)
+check("criteria/direct_count", len(criteria_direct), 4)
+check_true("criteria/has_direct", "direct_answer" in criteria_direct)
+check("lookup/direct_index", lookup_direct["direct_answer"][0], -1)
+
+
+# --------------------------------------------------------------- 3. Tool Selection Logic
+def make_mock_response(chosen_key, confidence=0.92, probabilities=None):
+ if probabilities is None:
+ probabilities = {chosen_key: confidence, "tool_1": 0.05, "tool_2": 0.03}
+ return {
+ "model": "laya-multilingual",
+ "answers": {
+ "tool_select": {
+ "type": "choice",
+ "choice": chosen_key,
+ "confidence": confidence,
+ "answer_confidence": confidence,
+ "probabilities": probabilities,
+ }
+ },
+ }
+
+mock_agent = MockLayaAgent(lambda s, q: make_mock_response("tool_0", confidence=0.95))
+selector = LayaToolSelector(tools=tools_list, agent=mock_agent)
+
+decision = selector.select("Find latest news on quantum computing")
+check("select/tool_name", decision.tool_name, "web_search")
+check("select/tool_index", decision.tool_index, 0)
+check("select/confidence", decision.confidence, 0.95)
+check("select/is_direct_answer", decision.is_direct_answer, False)
+check("select/is_fallback", decision.is_fallback, False)
+check_true("select/reason", "Selected tool 'web_search'" in decision.reason)
+check_true("select/probabilities_mapped", "web_search" in decision.probabilities)
+
+# Direct answer selection
+mock_agent_direct = MockLayaAgent(
+ lambda s, q: make_mock_response("direct_answer", confidence=0.99,
+ probabilities={"direct_answer": 0.99, "tool_0": 0.01})
+)
+selector_direct = LayaToolSelector(
+ tools=tools_list,
+ allow_direct_answer=True,
+ agent=mock_agent_direct,
+)
+decision_direct = selector_direct.select("Hello, how are you today?")
+check("select/direct_name", decision_direct.tool_name, "__direct_answer__")
+check("select/direct_index", decision_direct.tool_index, -1)
+check("select/direct_tool_is_none", decision_direct.tool, None)
+check("select/direct_flag", decision_direct.is_direct_answer, True)
+
+
+# --------------------------------------------------------------- 4. Confidence Threshold & Fallbacks
+# Below threshold without fallback raises LayaLowConfidenceError
+mock_agent_low = MockLayaAgent(lambda s, q: make_mock_response("tool_0", confidence=0.45))
+selector_low = LayaToolSelector(tools=tools_list, confidence_threshold=0.80, agent=mock_agent_low)
+
+try:
+ selector_low.select("Obscure question")
+ FAIL.append("threshold/should_raise: expected LayaLowConfidenceError")
+except LayaLowConfidenceError as e:
+ PASS.append("threshold/raises_error")
+ check("threshold/error_conf", e.confidence, 0.45)
+ check("threshold/error_thresh", e.threshold, 0.80)
+
+# Below threshold with fallback tool
+fallback_tool = {"name": "general_llm_fallback", "description": "General conversational fallback"}
+selector_fb = LayaToolSelector(
+ tools=tools_list,
+ confidence_threshold=0.80,
+ fallback_tool=fallback_tool,
+ agent=mock_agent_low,
+)
+decision_fb = selector_fb.select("Obscure question")
+check("threshold/fallback_used", decision_fb.is_fallback, True)
+check("threshold/fallback_tool_name", decision_fb.tool_name, "general_llm_fallback")
+check("threshold/fallback_tool_obj", decision_fb.tool, fallback_tool)
+check("threshold/fallback_index", decision_fb.tool_index, -1)
+
+
+# --------------------------------------------------------------- 5. Direct Execution (call)
+def add_numbers(a: int, b: int) -> int:
+ """Add two numbers together."""
+ return a + b
+
+mock_agent_calc = MockLayaAgent(lambda s, q: make_mock_response("tool_0", confidence=0.99))
+selector_calc = LayaToolSelector(tools=[add_numbers], agent=mock_agent_calc)
+calc_result = selector_calc.call("Calculate 10 + 25", None, 10, 25)
+check("call/callable_execution", calc_result, 35)
+
+# Direct answer returns None on call()
+selector_call_direct = LayaToolSelector(tools=[add_numbers], allow_direct_answer=True, agent=mock_agent_direct)
+direct_call_res = selector_call_direct.call("Hello", None, 1, 2)
+check("call/direct_answer_returns_none", direct_call_res, None)
+
+
+# --------------------------------------------------------------- 6. Async Selection (aselect)
+async def test_async_select():
+ sel = LayaToolSelector(tools=tools_list, agent=mock_agent)
+ res = await sel.aselect("Search quantum computing")
+ check("async/tool_name", res.tool_name, "web_search")
+
+asyncio.run(test_async_select())
+
+
+# --------------------------------------------------------------- 7. LCEL / Runnable Interface
+runnable = selector.as_runnable()
+runnable_res = runnable.invoke("Search quantum computing")
+check("runnable/tool_name", runnable_res.tool_name, "web_search")
+
+# Chain with pipe operator
+chained = runnable | (lambda d: f"Calling: {d.tool_name}")
+check("runnable/chain", chained.invoke("Search quantum computing"), "Calling: web_search")
+
+
+# --------------------------------------------------------------- 8. Remote HTTP Mode & Hook Rejection
+mock_http_response = json.dumps(make_mock_response("tool_0", confidence=0.93)).encode("utf-8")
+
+class MockHTTPResponse:
+ def read(self):
+ return mock_http_response
+ def __enter__(self):
+ return self
+ def __exit__(self, *args):
+ pass
+
+class MockHTTPHandler:
+ def open(self, req, timeout=10.0):
+ body = json.loads(req.data.decode("utf-8"))
+ if "questions" not in body or "state" not in body:
+ raise ValueError("Malformed request body")
+ return MockHTTPResponse()
+
+original_build_opener = laya.integrations.tools.urllib.request.build_opener
+laya.integrations.tools.urllib.request.build_opener = lambda *args: MockHTTPHandler()
+
+try:
+ remote_selector = LayaToolSelector(
+ tools=tools_list,
+ base_url="http://localhost:8000",
+ api_key="test-key",
+ max_len=1024,
+ head_max_len=256,
+ )
+ remote_decision = remote_selector.select("Search quantum computing")
+ check("remote/tool_name", remote_decision.tool_name, "web_search")
+ check("remote/confidence", remote_decision.confidence, 0.93)
+
+ # Rejection of local hooks over remote
+ remote_hooks_selector = LayaToolSelector(
+ tools=tools_list,
+ base_url="http://localhost:8000",
+ hooks=["mock_hook"],
+ )
+ try:
+ remote_hooks_selector.select("Query")
+ FAIL.append("remote/hooks: expected ValueError")
+ except ValueError as e:
+ PASS.append("remote/hooks_rejected")
+ check_true("remote/hooks_msg", "run in the local runner and cannot be sent" in str(e))
+finally:
+ laya.integrations.tools.urllib.request.build_opener = original_build_opener
+
+
+# --------------------------------------------------------------- 9. Controls Signatures Check
+params = inspect.signature(LayaToolSelector.__init__).parameters
+for ctrl in PREDICT_CONTROLS:
+ check_true(f"controls/predict_{ctrl}", ctrl in params)
+for ctrl in DECISION_CONTROLS:
+ check_true(f"controls/decision_{ctrl}", ctrl in params)
+for ctrl in HOOK_CONTROLS:
+ check_true(f"controls/hook_{ctrl}", ctrl in params)
+
+
+# --------------------------------------------------------------- 10. Public Module Exports
+integrations_mod = __import__("laya.integrations", fromlist=["__all__"])
+check_true("export/integrations_selector", "LayaToolSelector" in integrations_mod.__all__)
+check_true("export/integrations_decision", "ToolRouteDecision" in integrations_mod.__all__)
+check_true("export/laya_selector", "LayaToolSelector" in laya.__all__)
+check_true("export/laya_decision", "ToolRouteDecision" in laya.__all__)
+
+
+# --------------------------------------------------------------- Report
+print("\n" + "=" * 50)
+print(f"LayaToolSelector Test Suite: {len(PASS)} passed, {len(FAIL)} failed")
+print("=" * 50)
+
+if FAIL:
+ print("\nFAILURES:")
+ for f in FAIL:
+ print(f" - {f}")
+ sys.exit(1)
+else:
+ print("All tests passed cleanly!")
+ sys.exit(0)