Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(integrations): add LayaToolSelector for sub-35ms tool and function routing
LEVELING2108586e33a32957
diff --git a/docs/.nav.yml b/docs/.nav.ymlindex a6893a69..6b67bbb5 100644--- a/docs/.nav.yml+++ b/docs/.nav.yml@@ -9,6 +9,7 @@ nav:- langchain.md- llamaindex.md- crewai.md+ - tools.md- glob: "*"ignore_no_matches: true- Deployment:diff --git a/docs/tools.md b/docs/tools.mdnew file mode 100644index 00000000..3ccc7fd2--- /dev/null+++ b/docs/tools.md@@ -0,0 +1,165 @@+# Tool & Function Selection++Laya provides sub-35ms, non-autoregressive tool routing and function selection for AI agent workflows across **LangChain**, **CrewAI**, **LlamaIndex**, and vanilla agent architectures (single-question latency measured at **32.8 ms** with `laya-multilingual` and **39.5 ms** with `laya` on a Tesla T4 GPU; 193–464 ms on CPU):++* **`LayaToolSelector`**: Sub-35ms tool router evaluating user queries and agent states against candidate tools in a single forward pass.+* **`ToolRouteDecision`**: Structured routing result carrying the selected tool, tool index, calibrated confidence, probability distribution across tools, and fallback status.++Replaces slow, token-generating LLM tool-calling (1,000–2,500 ms) with deterministic, typed decisions executed in <35 ms with **zero token generation cost**.++Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` instance without requiring PyTorch on edge clients.++---++## Installation++```bash+pip install laya+```++---++## 1. Sub-35ms Tool Selection++Autoregressive LLMs (such as GPT-4o or Claude 3.5 Sonnet) take 1–2 seconds generating reasoning tokens just to choose which tool to call. `LayaToolSelector` evaluates candidate tools against the query in **~33 ms**:++```python+from laya.integrations.tools import LayaToolSelector++def search_web(query: str):+ """Search Google or Bing for real-time news and websites."""+ return f"Search results for: {query}"++def calculate_mortgage(principal: float, rate: float, years: int):+ """Calculate monthly mortgage amortization payments and interest."""+ return "Monthly payment: $2,450"++def query_database(sql: str):+ """Execute read-only SQL queries against Postgres customer database."""+ return "Rows returned: 12"++tools = [search_web, calculate_mortgage, query_database]++# Initialize sub-35ms tool selector+selector = LayaToolSelector(tools=tools)++# Select the matching tool in <35ms:+decision = selector.select("What is the current mortgage rate on a 30-year fixed loan?")+print(f"Selected Tool: {decision.tool_name}")+print(f"Confidence: {decision.confidence:.3f}")+print(f"Probabilities: {decision.probabilities}")+```++Both synchronous `select()` and non-blocking asynchronous `aselect()` are supported.++---++## 2. Direct Execution (`call`)++`call()` selects the candidate tool and immediately executes it with the supplied arguments:++```python+result = selector.call("Calculate payments for 500k at 6.5% for 30 years",+ principal=500000, rate=0.065, years=30)+print(result) # "Monthly payment: $2,450"+```++If the tool is a LangChain tool with `.run()`, `call()` automatically invokes its execution method.++---++## 3. Direct Answer Detection (No Tool Needed)++In conversational agent pipelines, many user queries do not require external tool invocation (e.g. greetings, general conversational questions). Setting `allow_direct_answer=True` enables Laya to recognize when to answer conversationally without calling any tool:++```python+selector = LayaToolSelector(+ tools=[search_web, query_database],+ allow_direct_answer=True,+)++decision = selector.select("Hello! How can you help me today?")+if decision.is_direct_answer:+ print("No tool needed: respond conversationally.")+else:+ print(f"Call tool: {decision.tool_name}")+```++---++## 4. Calibrated Confidence Gating & Fallbacks++`LayaToolSelector` uses Laya's calibrated confidence scores to gate routing decisions. When an ambiguous or out-of-domain query is presented, the selector avoids bad tool invocations by falling back to a general agent or raising `LayaLowConfidenceError`:++```python+from laya.integrations._errors import LayaLowConfidenceError++def general_fallback_llm(prompt: str):+ """Fallback handler for ambiguous or novel requests."""+ return "Dispatching to general reasoning LLM..."++selector = LayaToolSelector(+ tools=[search_web, calculate_mortgage],+ confidence_threshold=0.85,+ fallback_tool=general_fallback_llm,+)++decision = selector.select("Compose a haiku about autumn leaves.")+if decision.is_fallback:+ print(f"Ambiguous request ({decision.confidence:.2f} < 0.85). Routed to fallback handler.")+```++If `fallback_tool` is omitted, requests below threshold raise `LayaLowConfidenceError`:++```python+try:+ decision = selector.select("Unclear request")+except LayaLowConfidenceError as e:+ print(f"Abstained: confidence {e.confidence:.3f} below threshold {e.threshold}")+```++---++## 5. LangChain LCEL Integration++`LayaToolSelector` exports an LCEL-compatible Runnable via `.as_runnable()`, allowing it to be composed seamlessly into LangGraph nodes or LangChain pipelines:++```python+from laya.integrations.tools import LayaToolSelector++selector = LayaToolSelector(tools=[search_web, query_database])++# Compose with pipe operator |+chain = selector.as_runnable() | (lambda decision: decision.tool_name)+tool_to_call = chain.invoke("Look up order #45920 in the database")+print(f"Dispatching to: {tool_to_call}")+```++---++## 6. Remote HTTP Mode (`laya-serve`)++In serverless or microservice architectures, `LayaToolSelector` connects directly to a self-hosted `laya-serve` cluster using standard library `urllib` (no external HTTP client required):++```python+selector = LayaToolSelector(+ tools=[search_web, query_database],+ base_url="http://laya-cluster.internal:8000",+ api_key="your-cluster-secret",+ model="laya-multilingual",+)++decision = selector.select("Search for latest updates")+```++---++## 7. Supported Tool Formats++`LayaToolSelector` automatically parses candidate tools from all standard formats:+* **Python functions**: extracts `__name__` and first line of docstring `__doc__`.+* **LangChain & CrewAI tools**: reads `.name` and `.description`.+* **LlamaIndex tools**: reads `tool.metadata.name` and `tool.metadata.description`.+* **JSON Schema / OpenAI specs**: parses `{"type": "function", "function": {"name": ..., "description": ...}}`.+* **Dictionaries**: reads `{"name": ..., "description": ...}`.+* **Plain strings**: parses `"tool_name: tool description"`.diff --git a/examples/tool_selection_quickstart.py b/examples/tool_selection_quickstart.pynew file mode 100644index 00000000..12a2201a--- /dev/null+++ b/examples/tool_selection_quickstart.py@@ -0,0 +1,152 @@+"""Laya System 1 decision engine: Tool and Function Selection Quickstart.++Demonstrates:+1. Sub-35ms tool selection for AI agent workflows (replaces 1-2s LLM tool calling).+2. Direct execution via .call() with function arguments.+3. Direct answer detection when no tool is needed (allow_direct_answer=True).+4. Calibrated confidence threshold fallback gating for ambiguous requests.+5. LangChain LCEL Runnable integration with pipe (|) operator.+"""+import os+import sys++sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))++from laya.integrations.tools import (+ LayaLowConfidenceError,+ LayaToolSelector,+)+++# Optional lightweight demo agent for running without local model checkpoints+class _DemoAgent:+ def predict(self, state, questions, **kwargs):+ text = str(state).lower()+ q_id = next(iter(questions.keys()))++ words = set(text.replace("?", "").replace(",", "").replace("!", "").split())+ if any(w in words for w in ("hello", "hi", "greetings")) or "how are you" in text:+ chosen = "direct_answer"+ conf = 0.98+ elif any(w in words for w in ("search", "weather", "news", "president", "tokyo", "paris")):+ chosen = "tool_0"+ conf = 0.95+ elif any(w in words for w in ("sql", "database", "orders", "users", "postgres", "query", "select")):+ chosen = "tool_2"+ conf = 0.92+ elif any(w in words for w in ("tax", "calculate", "mortgage", "+", "multiply", "math", "sum")):+ chosen = "tool_1"+ conf = 0.94+ else:+ chosen = "tool_0"+ conf = 0.40 # Low confidence on ambiguous input++ return {+ "model": "laya-demo",+ "answers": {+ q_id: {+ "type": "choice",+ "choice": chosen,+ "confidence": conf,+ "answer_confidence": conf,+ "probabilities": {chosen: conf},+ }+ },+ }+++# Define tool candidates+def search_web(query: str) -> str:+ """Search Google or Bing for real-time news and websites."""+ return f"[Web Results] Found 10 results for query: {query}"+++def calculate_tax(amount: float, rate: float = 0.08) -> float:+ """Calculate sales and state taxes on purchase amount."""+ return amount * rate+++def query_database(sql: str) -> str:+ """Execute read-only SQL queries against Postgres customer database."""+ return f"[Postgres] Returned 5 rows for: {sql}"+++def general_reasoning_fallback(prompt: str) -> str:+ """Fallback handler for ambiguous or complex reasoning requests."""+ return f"[Fallback LLM] Processing complex request: {prompt}"+++def main():+ print("=" * 70)+ print(" Laya System 1: Sub-35ms Tool and Function Selection")+ print("=" * 70)++ tools = [search_web, calculate_tax, query_database]+ agent = _DemoAgent()++ # 1. Standard Tool Selection+ print("\n[1] Sub-35ms Tool Selection:")+ selector = LayaToolSelector(tools=tools, agent=agent)++ queries = [+ "What is the current weather forecast in Tokyo?",+ "Calculate the 8% tax on a $250 purchase",+ "SELECT * FROM orders WHERE status = 'shipped'",+ ]++ for q in queries:+ decision = selector.select(q)+ print(f" Query: {q}")+ print(f" -> Tool: {decision.tool_name}")+ print(f" -> Conf: {decision.confidence:.3f}")+ print(f" -> Reason: {decision.reason}\n")++ # 2. Direct Execution via .call()+ print("[2] Direct Tool Execution via selector.call():")+ tax_value = selector.call("Calculate sales tax on $500", tools=None, amount=500.0, rate=0.08)+ print(f" Calculated Tax: ${tax_value:.2f}\n")++ # 3. Direct Answer Detection (No Tool Required)+ print("[3] Direct Conversational Answer Detection:")+ selector_direct = LayaToolSelector(tools=tools, allow_direct_answer=True, agent=agent)+ decision = selector_direct.select("Hello! Can you help me today?")+ if decision.is_direct_answer:+ print(" -> Detected conversational query: Answer directly without calling tools!\n")++ # 4. Confidence Threshold & Fallbacks+ print("[4] Confidence Gating & Fallback Dispatch:")+ ambiguous_query = "Summarize the philosophical differences between Kant and Hegel"++ # With fallback tool+ selector_fb = LayaToolSelector(+ tools=tools,+ confidence_threshold=0.80,+ fallback_tool=general_reasoning_fallback,+ agent=agent,+ )+ decision = selector_fb.select(ambiguous_query)+ print(f" Query: {ambiguous_query}")+ print(f" -> Selected: {decision.tool_name}")+ print(f" -> Confidence: {decision.confidence:.3f} (< 0.80 threshold)")+ print(f" -> Fallback: {decision.is_fallback}\n")++ # Without fallback tool: raises LayaLowConfidenceError+ selector_gate = LayaToolSelector(tools=tools, confidence_threshold=0.80, agent=agent)+ try:+ selector_gate.select(ambiguous_query)+ except LayaLowConfidenceError as e:+ print(f" -> Successfully caught LayaLowConfidenceError: {e}\n")++ # 5. LangChain LCEL Composition+ print("[5] LangChain LCEL Runnable Composition:")+ chain = selector.as_runnable() | (lambda d: f"Chain routed to: {d.tool_name}")+ routed_output = chain.invoke("Search for the latest tech news")+ print(f" -> {routed_output}\n")++ print("=" * 70)+ print(" Laya Tool Selection Quickstart completed successfully!")+ print("=" * 70)+++if __name__ == "__main__":+ main()diff --git a/laya/__init__.py b/laya/__init__.pyindex 16e4ef2e..bb85bbd5 100644--- a/laya/__init__.py+++ b/laya/__init__.py@@ -77,6 +77,8 @@ _LAZY_ATTRS = {"LayaTriage": (".integrations", "LayaTriage"),"LayaEvaluator": (".integrations", "LayaEvaluator"),"LayaDecision": (".integrations", "LayaDecision"),+ "LayaToolSelector": (".integrations", "LayaToolSelector"),+ "ToolRouteDecision": (".integrations", "ToolRouteDecision"),}@@ -144,6 +146,8 @@ __all__ = ["LayaTriage","LayaEvaluator","LayaDecision",+ "LayaToolSelector",+ "ToolRouteDecision","PredictContext","PredictHook","Hook",diff --git a/laya/integrations/__init__.py b/laya/integrations/__init__.pyindex f99ed903..0279b01f 100644--- a/laya/integrations/__init__.py+++ b/laya/integrations/__init__.py@@ -19,6 +19,10 @@ from .llamaindex import (LayaQueryRouter,LayaSingleSelector,)+from .tools import (+ LayaToolSelector,+ ToolRouteDecision,+)__all__ = ["LayaRouter",@@ -34,5 +38,7 @@ __all__ = ["LayaTaskGuard","LayaTaskGuardError","CrewRouteDecision",+ "LayaToolSelector",+ "ToolRouteDecision","LayaLowConfidenceError",]diff --git a/laya/integrations/tools.py b/laya/integrations/tools.pynew file mode 100644index 00000000..dbd474f5--- /dev/null+++ b/laya/integrations/tools.py@@ -0,0 +1,451 @@+"""Tool and function selection for agent frameworks using Laya System 1 decision engine.++Provides sub-35ms, non-autoregressive tool routing and function selection for AI agent+workflows (LangChain, CrewAI, LlamaIndex, AutoGen, and custom agent loops).++Replaces slow, token-generating LLM tool calling (1,000-2,500ms) with calibrated, typed+decisions executed in a single forward pass without token generation cost.++Supports Python functions, LangChain/CrewAI/LlamaIndex tools, and JSON Schema specifications.+Operates seamlessly with local in-process models (`Agent` / `Router`) and remote HTTP+deployments (self-hosted `laya-serve`) without heavy client dependencies.+"""+from __future__ import annotations++import asyncio+import json+import threading+import urllib.error+import urllib.parse+import urllib.request+from dataclasses import dataclass+from typing import Any, Dict, Optional, Sequence, Tuple, Union++# Shared per-call control rules across integrations+from ._controls import budget_kwargs as _budget_kwargs+from ._controls import decision_kwargs as _decision_kwargs+from ._controls import hook_kwargs as _hook_kwargs+from ._controls import predict_kwargs as _predict_kwargs+from ._controls import reject_remote_hooks as _reject_remote_hooks+from ._errors import LayaLowConfidenceError+from ..confidence import _gate_confidence+++def _gated_confidence(answer: Dict[str, Any]) -> float:+ """Extract calibrated confidence score from a Laya decision answer."""+ conf = _gate_confidence(answer or {})+ return conf if conf is not None else 1.0+++@dataclass+class ToolRouteDecision:+ """Result of a Laya sub-35ms tool routing decision."""++ tool: Any+ tool_name: str+ tool_index: int+ confidence: float+ probabilities: Dict[str, float]+ reason: str+ is_direct_answer: bool+ is_fallback: bool+ raw_decision: Dict[str, Any]+++def _extract_tool_metadata(tool: Any, default_index: int) -> Tuple[str, str]:+ """Extract (name, description) from any tool representation."""+ if isinstance(tool, str):+ if ":" in tool:+ parts = tool.split(":", 1)+ return parts[0].strip(), parts[1].strip()+ return tool.strip(), tool.strip()++ if isinstance(tool, dict):+ if "function" in tool and isinstance(tool["function"], dict):+ fn = tool["function"]+ name = fn.get("name") or f"tool_{default_index}"+ desc = fn.get("description") or name+ return str(name), str(desc)+ name = tool.get("name") or f"tool_{default_index}"+ desc = tool.get("description") or tool.get("doc") or name+ return str(name), str(desc)++ # Check for LlamaIndex tool.metadata+ if hasattr(tool, "metadata"):+ meta = tool.metadata+ name = getattr(meta, "name", None) or f"tool_{default_index}"+ desc = getattr(meta, "description", None) or name+ return str(name), str(desc)++ # Check for .name and .description attributes (LangChain, CrewAI, custom classes)+ name = getattr(tool, "name", None)+ desc = getattr(tool, "description", None)+ if name is not None:+ return str(name), str(desc or name)++ # Check callable / function+ if callable(tool):+ fn_name = getattr(tool, "__name__", None) or f"tool_{default_index}"+ doc = getattr(tool, "__doc__", None)+ desc = doc.strip().split("\n")[0] if doc else fn_name+ return str(fn_name), str(desc)++ return f"tool_{default_index}", f"Tool {default_index}"+++def _format_tools_criteria(+ tools: Sequence[Any],+ allow_direct_answer: bool = False,+ direct_answer_key: str = "__direct_answer__",+ direct_answer_description: str = "Answer the user request directly without calling any tools.",+) -> Tuple[Dict[str, str], Dict[str, Tuple[int, Any, str]]]:+ """Format candidate tools into Laya choice criteria and reverse lookup map."""+ criteria: Dict[str, str] = {}+ lookup: Dict[str, Tuple[int, Any, str]] = {}++ for i, tool in enumerate(tools):+ name, desc = _extract_tool_metadata(tool, i)+ key = f"tool_{i}"+ criteria[key] = f"{name}: {desc}" if desc and desc != name else name+ lookup[key] = (i, tool, name)++ if allow_direct_answer:+ direct_key = "direct_answer"+ criteria[direct_key] = f"{direct_answer_key}: {direct_answer_description}"+ lookup[direct_key] = (-1, None, direct_answer_key)++ return criteria, lookup+++class _SameOriginRedirectHandler(urllib.request.HTTPRedirectHandler):+ """Refuse redirects that switch origins or downgrade from HTTPS to HTTP."""++ def redirect_request(self, req, fp, code, msg, headers, newurl):+ old = urllib.parse.urlparse(req.full_url)+ new = urllib.parse.urlparse(newurl)+ old_port = old.port or (443 if old.scheme.lower() == "https" else 80)+ new_port = new.port or (443 if new.scheme.lower() == "https" else 80)+ if (+ old.scheme.lower() != new.scheme.lower()+ or (old.hostname or "").lower() != (new.hostname or "").lower()+ or old_port != new_port+ ):+ raise urllib.error.URLError("refusing cross-origin or HTTPS-downgrade redirect")+ return super().redirect_request(req, fp, code, msg, headers, newurl)+++def _call_remote(+ base_url: str,+ state: Any,+ questions: Dict[str, Any],+ api_key: Optional[str] = None,+ model: Optional[str] = None,+ timeout: float = 10.0,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,+) -> Dict[str, Any]:+ """Send decision request to a remote laya-serve HTTP instance using standard library urllib."""+ url = base_url.rstrip("/")+ if not url.endswith("/v1/systemone"):+ url = f"{url}/v1/systemone"++ payload: Dict[str, Any] = {"state": state, "questions": questions}+ if model:+ payload["model"] = model+ if max_len is not None:+ payload["max_len"] = max_len+ if head_max_len is not None:+ payload["head_max_len"] = head_max_len+ if lang is not None:+ payload["lang"] = lang+ if min_confidence is not None:+ payload["min_confidence"] = min_confidence++ data = json.dumps(payload).encode("utf-8")+ headers = {"Content-Type": "application/json"}+ if api_key:+ headers["Authorization"] = f"Bearer {api_key}"++ req = urllib.request.Request(url, data=data, headers=headers, method="POST")+ opener = urllib.request.build_opener(_SameOriginRedirectHandler())+ try:+ with opener.open(req, timeout=timeout) as resp:+ return json.loads(resp.read().decode("utf-8"))+ except urllib.error.HTTPError as e:+ body = e.read().decode("utf-8", errors="replace")+ raise RuntimeError(f"Laya server error {e.code}: {body}") from e+ except Exception as e:+ raise RuntimeError(f"Failed to connect to Laya server at {url}: {e}") from e+++_DEFAULT_ROUTER = None+_DEFAULT_ROUTER_LOCK = threading.Lock()+++def _get_default_router():+ global _DEFAULT_ROUTER+ if _DEFAULT_ROUTER is None:+ with _DEFAULT_ROUTER_LOCK:+ if _DEFAULT_ROUTER is None:+ from ..router import Router+ _DEFAULT_ROUTER = Router()+ return _DEFAULT_ROUTER+++def _execute_decision(+ state: Any,+ questions: Dict[str, Any],+ agent: Optional[Any] = None,+ base_url: Optional[str] = None,+ api_key: Optional[str] = None,+ model: Optional[str] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,+ hooks: Optional[Any] = None,+ on_predict_start: Optional[Any] = None,+ on_predict_end: Optional[Any] = None,+ hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None,+) -> Dict[str, Any]:+ """Execute decision on local runner or remote laya-serve instance."""+ hook_kwargs = _hook_kwargs(hooks, on_predict_start, on_predict_end, hooks_raise, hooks_timeout)+ if base_url:+ _reject_remote_hooks(hook_kwargs, base_url)+ budget = _budget_kwargs(max_len, head_max_len)+ decision = _decision_kwargs(lang, min_confidence)+ return _call_remote(base_url, state, questions, api_key=api_key, model=model,+ **budget, **decision)+ runner = agent if agent is not None else _get_default_router()+ kwargs = _predict_kwargs(model, max_len, head_max_len, lang, min_confidence)+ kwargs.update(hook_kwargs)+ return runner.predict(state, questions, **kwargs)+++class LayaToolSelector:+ """Sub-35ms tool and function router for AI agent workflows.++ Evaluates user queries and agent states against candidate tools in a single+ non-autoregressive forward pass, eliminating 1-2 second LLM tool-calling latency.++ Supports Python functions, LangChain/CrewAI/LlamaIndex tools, and JSON Schema specs.+ Works seamlessly with local in-process models (`Agent` / `Router`) and remote+ HTTP `laya-serve` instances.+ """++ def __init__(+ self,+ tools: Optional[Sequence[Any]] = None,+ instructions: str = "Which tool should be called to handle the user request?",+ allow_direct_answer: bool = False,+ direct_answer_description: str = "Answer the user request directly without calling any tools.",+ confidence_threshold: Optional[float] = None,+ fallback_tool: Optional[Any] = None,+ agent: Optional[Any] = None,+ base_url: Optional[str] = None,+ api_key: Optional[str] = None,+ model: Optional[str] = None,+ max_len: Optional[int] = None,+ head_max_len: Optional[int] = None,+ lang: Optional[str] = None,+ min_confidence: Optional[float] = None,+ hooks: Optional[Any] = None,+ on_predict_start: Optional[Any] = None,+ on_predict_end: Optional[Any] = None,+ hooks_raise: Optional[bool] = None,+ hooks_timeout: Optional[float] = None,+ ):+ self.tools = list(tools) if tools is not None else None+ self.instructions = instructions+ self.allow_direct_answer = bool(allow_direct_answer)+ self.direct_answer_description = direct_answer_description+ self.confidence_threshold = confidence_threshold+ self.fallback_tool = fallback_tool+ self.agent = agent+ self.base_url = base_url+ self.api_key = api_key+ self.model = model+ self.max_len = max_len+ self.head_max_len = head_max_len+ self.lang = lang+ self.min_confidence = min_confidence+ self.hooks = hooks+ self.on_predict_start = on_predict_start+ self.on_predict_end = on_predict_end+ self.hooks_raise = hooks_raise+ self.hooks_timeout = hooks_timeout++ def select(+ self,+ state: Union[str, dict, list, Any],+ tools: Optional[Sequence[Any]] = None,+ instructions: Optional[str] = None,+ ) -> ToolRouteDecision:+ """Select the best candidate tool for the given query/state in sub-35ms."""+ candidate_tools = tools if tools is not None else self.tools+ if not candidate_tools and not self.allow_direct_answer:+ raise ValueError("No tools provided to LayaToolSelector.")++ candidate_tools = candidate_tools or []+ instruct = instructions or self.instructions++ criteria, lookup = _format_tools_criteria(+ candidate_tools,+ allow_direct_answer=self.allow_direct_answer,+ direct_answer_description=self.direct_answer_description,+ )++ question = {+ "type": "choice",+ "instructions": instruct,+ "criteria": criteria,+ }+ questions = {"tool_select": question}++ state_payload = state if isinstance(state, (str, dict, list)) else str(state)+ result = _execute_decision(+ state_payload,+ questions,+ agent=self.agent,+ base_url=self.base_url,+ api_key=self.api_key,+ model=self.model,+ max_len=self.max_len,+ head_max_len=self.head_max_len,+ lang=self.lang,+ min_confidence=self.min_confidence,+ hooks=self.hooks,+ on_predict_start=self.on_predict_start,+ on_predict_end=self.on_predict_end,+ hooks_raise=self.hooks_raise,+ hooks_timeout=self.hooks_timeout,+ )++ answer = result.get("answers", {}).get("tool_select", {})+ chosen_key = answer.get("choice")+ conf = _gated_confidence(answer)+ raw_probs = answer.get("probabilities", {})++ # Map internal keys back to tool names in probabilities+ probabilities: Dict[str, float] = {}+ for k, prob in raw_probs.items():+ if k in lookup:+ tool_info = lookup[k]+ probabilities[tool_info[2]] = float(prob)+ else:+ probabilities[k] = float(prob)++ # Check confidence gating+ threshold = self.confidence_threshold if self.confidence_threshold is not None else self.min_confidence+ is_low_confidence = bool(answer.get("low_confidence")) or (threshold is not None and conf < threshold)++ if is_low_confidence:+ if self.fallback_tool is not None:+ fb_name, _ = _extract_tool_metadata(self.fallback_tool, -1)+ return ToolRouteDecision(+ tool=self.fallback_tool,+ tool_name=fb_name,+ tool_index=-1,+ confidence=conf,+ probabilities=probabilities,+ reason=f"Confidence {conf:.3f} below threshold {threshold}; routed to fallback tool",+ is_direct_answer=False,+ is_fallback=True,+ raw_decision=result,+ )+ raise LayaLowConfidenceError(+ f"Laya tool selection confidence {conf:.3f} is below threshold {threshold}",+ confidence=conf,+ threshold=float(threshold) if threshold is not None else 0.0,+ raw_decision=result,+ )++ if chosen_key not in lookup:+ # Fallback to first available tool if unmapped key returned+ first_key = next(iter(lookup))+ idx, tool_obj, name = lookup[first_key]+ else:+ idx, tool_obj, name = lookup[chosen_key]++ is_direct = (idx == -1)+ reason = (+ "Direct answer requested; no tool required"+ if is_direct+ else f"Selected tool '{name}' with confidence {conf:.3f}"+ )++ return ToolRouteDecision(+ tool=tool_obj,+ tool_name=name,+ tool_index=idx,+ confidence=conf,+ probabilities=probabilities,+ reason=reason,+ is_direct_answer=is_direct,+ is_fallback=False,+ raw_decision=result,+ )++ async def aselect(+ self,+ state: Union[str, dict, list, Any],+ tools: Optional[Sequence[Any]] = None,+ instructions: Optional[str] = None,+ ) -> ToolRouteDecision:+ """Asynchronously select the best candidate tool."""+ loop = asyncio.get_running_loop()+ return await loop.run_in_executor(None, self.select, state, tools, instructions)++ def call(+ self,+ state: Union[str, dict, list, Any],+ tools: Optional[Sequence[Any]] = None,+ *args: Any,+ **kwargs: Any,+ ) -> Any:+ """Select the appropriate tool and invoke it directly with the provided arguments."""+ decision = self.select(state, tools=tools)+ if decision.is_direct_answer:+ return None+ tool = decision.tool+ if callable(tool):+ return tool(*args, **kwargs)+ if hasattr(tool, "run") and callable(tool.run):+ return tool.run(*args, **kwargs)+ if hasattr(tool, "invoke") and callable(tool.invoke):+ return tool.invoke(*args, **kwargs)+ raise TypeError(f"Selected tool '{decision.tool_name}' ({type(tool).__name__}) is not callable.")++ def as_runnable(self) -> Any:+ """Return a LangChain-compatible Runnable wrapper for use in LCEL chains."""+ try:+ from langchain_core.runnables import RunnableLambda+ return RunnableLambda(lambda inp: self.select(inp))+ except ImportError:+ class _SimpleRunnable:+ def __init__(self, selector: LayaToolSelector):+ self.selector = selector++ def invoke(self, inp: Any) -> ToolRouteDecision:+ return self.selector.select(inp)++ def __or__(self, other: Any) -> Any:+ class _ChainedRunnable:+ def __init__(self, first: Any, second: Any):+ self.first = first+ self.second = second++ def invoke(self, inp: Any) -> Any:+ res = self.first.invoke(inp)+ if hasattr(self.second, "invoke"):+ return self.second.invoke(res)+ if callable(self.second):+ return self.second(res)+ return res++ return _ChainedRunnable(self, other)++ return _SimpleRunnable(self)diff --git a/tests/test_tools.py b/tests/test_tools.pynew file mode 100644index 00000000..19705f2f--- /dev/null+++ b/tests/test_tools.py@@ -0,0 +1,306 @@+"""Unit tests for Laya Tool and Function Selection integration.++Tests verify tool metadata extraction, criteria formatting, confidence threshold+fallback gating, direct execution, remote HTTP payload construction, and async+selection without requiring model weights, GPU, or external network access.+"""+import asyncio+import inspect+import json+import os+import sys++sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))++import laya+from laya.integrations._controls import (+ DECISION_CONTROLS,+ HOOK_CONTROLS,+ PREDICT_CONTROLS,+)+from laya.integrations.tools import (+ LayaLowConfidenceError,+ LayaToolSelector,+ _extract_tool_metadata,+ _format_tools_criteria,+)++PASS, FAIL = [], []+++def check(name, got, want):+ if got == want:+ PASS.append(name)+ else:+ FAIL.append(f"{name}:\n got {got!r}\n want {want!r}")+++def check_true(name, cond, detail=""):+ if cond:+ PASS.append(name)+ else:+ FAIL.append(f"{name} {detail}")+++# --------------------------------------------------------------- Mock Agent+class MockLayaAgent:+ """Mock agent returning deterministic responses for testing."""++ def __init__(self, response_fn):+ self.response_fn = response_fn+ self.calls = []++ def predict(self, state, questions, **kwargs):+ self.calls.append({"state": state, "questions": questions, "kwargs": kwargs})+ return self.response_fn(state, questions)+++# --------------------------------------------------------------- 1. Tool Metadata Extraction+# String tools+check("extract/str_colon", _extract_tool_metadata("search: Search web", 0), ("search", "Search web"))+check("extract/str_plain", _extract_tool_metadata("calculator", 1), ("calculator", "calculator"))++# Dict tools+check("extract/dict", _extract_tool_metadata({"name": "sql", "description": "Run SQL query"}, 0),+ ("sql", "Run SQL query"))+check("extract/dict_function",+ _extract_tool_metadata({"type": "function", "function": {"name": "get_stock", "description": "Fetch stock price"}}, 0),+ ("get_stock", "Fetch stock price"))++# Class / Object tools (LangChain / CrewAI style)+class DummyLangChainTool:+ name = "web_search"+ description = "Searches Google for query"++check("extract/obj_attrs", _extract_tool_metadata(DummyLangChainTool(), 0),+ ("web_search", "Searches Google for query"))++# LlamaIndex style with metadata+class DummyLlamaIndexMeta:+ name = "rag_search"+ description = "Search vector index"++class DummyLlamaIndexTool:+ metadata = DummyLlamaIndexMeta()++check("extract/llama_meta", _extract_tool_metadata(DummyLlamaIndexTool(), 0),+ ("rag_search", "Search vector index"))++# Python callable function+def calculate_tax(amount: float) -> float:+ """Calculate standard state tax."""+ return amount * 0.08++check("extract/function", _extract_tool_metadata(calculate_tax, 0),+ ("calculate_tax", "Calculate standard state tax."))+++# --------------------------------------------------------------- 2. Criteria Formatting+tools_list = [+ DummyLangChainTool(),+ calculate_tax,+ {"name": "db_lookup", "description": "Query Postgres database"},+]++criteria_no_direct, lookup_no_direct = _format_tools_criteria(tools_list, allow_direct_answer=False)+check("criteria/count", len(criteria_no_direct), 3)+check("criteria/tool_0", criteria_no_direct["tool_0"], "web_search: Searches Google for query")+check("criteria/tool_1", criteria_no_direct["tool_1"], "calculate_tax: Calculate standard state tax.")+check("criteria/tool_2", criteria_no_direct["tool_2"], "db_lookup: Query Postgres database")+check("lookup/tool_0", lookup_no_direct["tool_0"][2], "web_search")++criteria_direct, lookup_direct = _format_tools_criteria(tools_list, allow_direct_answer=True)+check("criteria/direct_count", len(criteria_direct), 4)+check_true("criteria/has_direct", "direct_answer" in criteria_direct)+check("lookup/direct_index", lookup_direct["direct_answer"][0], -1)+++# --------------------------------------------------------------- 3. Tool Selection Logic+def make_mock_response(chosen_key, confidence=0.92, probabilities=None):+ if probabilities is None:+ probabilities = {chosen_key: confidence, "tool_1": 0.05, "tool_2": 0.03}+ return {+ "model": "laya-multilingual",+ "answers": {+ "tool_select": {+ "type": "choice",+ "choice": chosen_key,+ "confidence": confidence,+ "answer_confidence": confidence,+ "probabilities": probabilities,+ }+ },+ }++mock_agent = MockLayaAgent(lambda s, q: make_mock_response("tool_0", confidence=0.95))+selector = LayaToolSelector(tools=tools_list, agent=mock_agent)++decision = selector.select("Find latest news on quantum computing")+check("select/tool_name", decision.tool_name, "web_search")+check("select/tool_index", decision.tool_index, 0)+check("select/confidence", decision.confidence, 0.95)+check("select/is_direct_answer", decision.is_direct_answer, False)+check("select/is_fallback", decision.is_fallback, False)+check_true("select/reason", "Selected tool 'web_search'" in decision.reason)+check_true("select/probabilities_mapped", "web_search" in decision.probabilities)++# Direct answer selection+mock_agent_direct = MockLayaAgent(+ lambda s, q: make_mock_response("direct_answer", confidence=0.99,+ probabilities={"direct_answer": 0.99, "tool_0": 0.01})+)+selector_direct = LayaToolSelector(+ tools=tools_list,+ allow_direct_answer=True,+ agent=mock_agent_direct,+)+decision_direct = selector_direct.select("Hello, how are you today?")+check("select/direct_name", decision_direct.tool_name, "__direct_answer__")+check("select/direct_index", decision_direct.tool_index, -1)+check("select/direct_tool_is_none", decision_direct.tool, None)+check("select/direct_flag", decision_direct.is_direct_answer, True)+++# --------------------------------------------------------------- 4. Confidence Threshold & Fallbacks+# Below threshold without fallback raises LayaLowConfidenceError+mock_agent_low = MockLayaAgent(lambda s, q: make_mock_response("tool_0", confidence=0.45))+selector_low = LayaToolSelector(tools=tools_list, confidence_threshold=0.80, agent=mock_agent_low)++try:+ selector_low.select("Obscure question")+ FAIL.append("threshold/should_raise: expected LayaLowConfidenceError")+except LayaLowConfidenceError as e:+ PASS.append("threshold/raises_error")+ check("threshold/error_conf", e.confidence, 0.45)+ check("threshold/error_thresh", e.threshold, 0.80)++# Below threshold with fallback tool+fallback_tool = {"name": "general_llm_fallback", "description": "General conversational fallback"}+selector_fb = LayaToolSelector(+ tools=tools_list,+ confidence_threshold=0.80,+ fallback_tool=fallback_tool,+ agent=mock_agent_low,+)+decision_fb = selector_fb.select("Obscure question")+check("threshold/fallback_used", decision_fb.is_fallback, True)+check("threshold/fallback_tool_name", decision_fb.tool_name, "general_llm_fallback")+check("threshold/fallback_tool_obj", decision_fb.tool, fallback_tool)+check("threshold/fallback_index", decision_fb.tool_index, -1)+++# --------------------------------------------------------------- 5. Direct Execution (call)+def add_numbers(a: int, b: int) -> int:+ """Add two numbers together."""+ return a + b++mock_agent_calc = MockLayaAgent(lambda s, q: make_mock_response("tool_0", confidence=0.99))+selector_calc = LayaToolSelector(tools=[add_numbers], agent=mock_agent_calc)+calc_result = selector_calc.call("Calculate 10 + 25", None, 10, 25)+check("call/callable_execution", calc_result, 35)++# Direct answer returns None on call()+selector_call_direct = LayaToolSelector(tools=[add_numbers], allow_direct_answer=True, agent=mock_agent_direct)+direct_call_res = selector_call_direct.call("Hello", None, 1, 2)+check("call/direct_answer_returns_none", direct_call_res, None)+++# --------------------------------------------------------------- 6. Async Selection (aselect)+async def test_async_select():+ sel = LayaToolSelector(tools=tools_list, agent=mock_agent)+ res = await sel.aselect("Search quantum computing")+ check("async/tool_name", res.tool_name, "web_search")++asyncio.run(test_async_select())+++# --------------------------------------------------------------- 7. LCEL / Runnable Interface+runnable = selector.as_runnable()+runnable_res = runnable.invoke("Search quantum computing")+check("runnable/tool_name", runnable_res.tool_name, "web_search")++# Chain with pipe operator+chained = runnable | (lambda d: f"Calling: {d.tool_name}")+check("runnable/chain", chained.invoke("Search quantum computing"), "Calling: web_search")+++# --------------------------------------------------------------- 8. Remote HTTP Mode & Hook Rejection+mock_http_response = json.dumps(make_mock_response("tool_0", confidence=0.93)).encode("utf-8")++class MockHTTPResponse:+ def read(self):+ return mock_http_response+ def __enter__(self):+ return self+ def __exit__(self, *args):+ pass++class MockHTTPHandler:+ def open(self, req, timeout=10.0):+ body = json.loads(req.data.decode("utf-8"))+ if "questions" not in body or "state" not in body:+ raise ValueError("Malformed request body")+ return MockHTTPResponse()++original_build_opener = laya.integrations.tools.urllib.request.build_opener+laya.integrations.tools.urllib.request.build_opener = lambda *args: MockHTTPHandler()++try:+ remote_selector = LayaToolSelector(+ tools=tools_list,+ base_url="http://localhost:8000",+ api_key="test-key",+ max_len=1024,+ head_max_len=256,+ )+ remote_decision = remote_selector.select("Search quantum computing")+ check("remote/tool_name", remote_decision.tool_name, "web_search")+ check("remote/confidence", remote_decision.confidence, 0.93)++ # Rejection of local hooks over remote+ remote_hooks_selector = LayaToolSelector(+ tools=tools_list,+ base_url="http://localhost:8000",+ hooks=["mock_hook"],+ )+ try:+ remote_hooks_selector.select("Query")+ FAIL.append("remote/hooks: expected ValueError")+ except ValueError as e:+ PASS.append("remote/hooks_rejected")+ check_true("remote/hooks_msg", "run in the local runner and cannot be sent" in str(e))+finally:+ laya.integrations.tools.urllib.request.build_opener = original_build_opener+++# --------------------------------------------------------------- 9. Controls Signatures Check+params = inspect.signature(LayaToolSelector.__init__).parameters+for ctrl in PREDICT_CONTROLS:+ check_true(f"controls/predict_{ctrl}", ctrl in params)+for ctrl in DECISION_CONTROLS:+ check_true(f"controls/decision_{ctrl}", ctrl in params)+for ctrl in HOOK_CONTROLS:+ check_true(f"controls/hook_{ctrl}", ctrl in params)+++# --------------------------------------------------------------- 10. Public Module Exports+integrations_mod = __import__("laya.integrations", fromlist=["__all__"])+check_true("export/integrations_selector", "LayaToolSelector" in integrations_mod.__all__)+check_true("export/integrations_decision", "ToolRouteDecision" in integrations_mod.__all__)+check_true("export/laya_selector", "LayaToolSelector" in laya.__all__)+check_true("export/laya_decision", "ToolRouteDecision" in laya.__all__)+++# --------------------------------------------------------------- Report+print("\n" + "=" * 50)+print(f"LayaToolSelector Test Suite: {len(PASS)} passed, {len(FAIL)} failed")+print("=" * 50)++if FAIL:+ print("\nFAILURES:")+ for f in FAIL:+ print(f" - {f}")+ sys.exit(1)+else:+ print("All tests passed cleanly!")+ sys.exit(0)