Skip to content

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

feat(blocks): add TypeSafe Jev judgment blocks (#14639)

Toran Bruce Richards591fac9374b6
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/README.md b/autogpt_platform/backend/backend/blocks/typesafe/README.md
new file mode 100644
index 0000000000..8e45ffa835
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/README.md
@@ -0,0 +1,138 @@
+# TypeSafe Jev blocks
+
+[Jev](https://docs.typesafe.ai/concepts/system-one) is TypeSafe's System One
+model for typed judgments: Choice selects an option, Score evaluates a defined
+scale, and Noul evaluates a true/false statement. Choice and Score include
+probabilities; Noul returns a scalar judgment. Jev does not generate explanations
+or other free text. Every call
+is stateless: put all relevant evidence and context in `state` each time.
+The async SDK uses its default model, `jev-latest`.
+
+Select a **TypeSafe** API-key credential in each block's credentials field.
+Keys belong to the user; the provider does not install a shared environment-key
+fallback. These blocks charge zero platform credits because they use your own
+key. TypeSafe may charge your account: the API reports token usage, not a price
+or cost field, and no TypeSafe price is assumed here.
+
+## State and transparency
+
+All seven blocks accept `credentials` and `state: Any = None`. Strings are sent as
+text; JSON values become compact JSON text. Jev's roughly 32k-token context is
+shared by the **whole request**, including every question and rubric. Since
+the SDK does not supply a Jev tokenizer, these blocks conservatively bound
+the complete serialized request to 30,976 UTF-8 bytes, reserving 1,024 of the
+32,000-byte bound. This is a safety bound, not an exact token count. When
+necessary, the state is shortened to a prefix and `truncated` becomes true;
+`truncation_note` explains how much was kept. Question/rubric text alone that
+cannot fit raises a clear error. Truncated JSON state is a text prefix and
+may no longer be a complete JSON document.
+
+Every call emits `request: str` and `response: str | None` as the verbatim
+HTTP body strings, plus `latency_ms: float`, `input_tokens: int | None`,
+`output_tokens: int | None`, `request_id: str`, `truncated: bool`, and
+`truncation_note: str`. On failure, unavailable response/usage values are null
+and an `error: str` pin explains the failure; no decision or routing pins fire.
+Valid UTF-8 responses remain verbatim. If a response contains invalid UTF-8,
+`response` preserves its exact bytes as a `data:application/octet-stream;base64,`
+URL, and `error` explains this encoding. Decode the portion after the comma
+with Base64 to recover the original bytes; no replacement characters are used.
+Timing is measured around the
+call, and token counts come from the API. Headers containing credentials are
+never outputs. Inspect these pins to see precisely which evidence and rubric
+were evaluated.
+
+## Blocks
+
+The following inputs and outputs are in addition to the shared fields above.
+The platform also supplies its standard `error` output.
+
+| Block | Inputs | Outputs |
+| --- | --- | --- |
+| `JevChoiceBlock` | `question: str`, `options: dict[str, str]` | `choice: str`, `probabilities: dict[str, float]`, `confidence: float` |
+| `JevScoreBlock` | `question: str`, `levels: list[str]` (lowest first) | `score: float`, `legend: dict[str, str]`, `probabilities: dict[str, float]`, `confidence: float`, `max_score: int` |
+| `JevAskManyBlock` | `questions: dict[str, typed question]` (nonempty) | `answers: dict[str, dict[str, Any]]`, one typed answer per key |
+| `JevRouteBlock` | `question: str`, `options: dict[str, str]` (2–5), `data: Any = None` | Winning `option_1`…`option_5: Any` forwards `data`; `choice: str`, `probabilities: dict[str, float]` |
+| `JevYesNoBlock` | `question: str`, `data: Any = None`, `min_confidence: float = 0.0` (0–1) | `yes: Any` or `no: Any` forwards `data`; `unsure: Any` forwards it instead below the threshold; `confidence: float` |
+| `JevPickBestBlock` | `candidates: list[Any]`, `question: str` | `best: Any`, `best_index: int` (zero-based), `ranked: list[Any]`, `probabilities: dict[str, float]` |
+| `JevFilterBlock` | `items: list[Any]`, `question: str`, `levels: list[str]`, `min_score: float`, `max_items: int = 100` | `passed: list[Any]`, `rejected: list[Any]`, `scores: list[float]` aligned with items; per-call `item_index: int` |
+
+Route pin numbering follows option insertion order. PickBest uses labels
+`candidate_1`, `candidate_2`, etc.; its ranking comes directly from returned
+probabilities, returning the original candidates in descending probability
+order (ties keep input order). Each candidate description is compact JSON
+limited to 2,000 characters, with any truncation reported on the common pins.
+Choice/Route require at least two options, Score/Filter at least two levels,
+and PickBest at least two candidates. Questions must be nonempty strings.
+Filter accepts `max_items` from 1 to
+1,000, and `min_score` from 0 to `len(levels) - 1`.
+
+Filter evaluates `{"item": item, "context": state}` sequentially,
+one request per item, and rejects inputs above `max_items` before making any
+calls. For each item it emits `item_index` followed by the transparency pins;
+connect them together when recording a per-item audit. Its final result lists
+preserve input order. An empty item list yields empty result lists and makes
+no API call, so there are no per-call transparency outputs in that case.
+If an item call fails, earlier transcripts remain available, but Filter emits
+no partial result lists that could be mistaken for a complete evaluation.
+
+AskMany accepts these question shapes and sends them together in one request:
+
+```json
+{
+ "team": {"type": "choice", "question": "Which team owns this?", "options": {"billing": "Payment problems", "support": "Other product help"}},
+ "urgency": {"type": "score", "question": "How urgent is this?", "levels": ["No blocked work", "One person blocked", "An entire team blocked"]},
+ "refund": {"type": "noul", "question": "Is a refund requested?"}
+}
+```
+
+Noul also accepts `criteria: {"true": "...", "false": "..."}`. Its answer is
+`{"type": "noul", "noul": float}`, without probabilities or confidence.
+Choice option descriptions
+define each answer; Score level descriptions define what earns each tier.
+
+## Calibrate the scale with evidence
+
+For hiring, avoid a bare scale such as "bad / okay / good." Describe concrete
+evidence at every level so that a very good resume does not automatically earn
+the maximum. For example: **0 terrible** — no demonstrated programming;
+**1 bad** — introductory exercises only; **2 okay** — delivered a tested
+backend feature under guidance; **3 excellent** — independently owns production
+Python services; **4 brilliant** — leads complex systems with measurable
+improvements across teams; **5 one-in-a-thousand** — sustained, externally
+verifiable, field-leading contributions with exceptional impact. A Score can
+be fractional; `max_score` is always `len(levels) - 1`.
+
+## Example agent and optional live check
+
+Import [`resume-review.example.json`](resume-review.example.json), which uses
+the repository's agent graph format:
+
+```text
+Resume file → FileRead (text) → JevAskMany → JevRoute
+ ├─ option_1 → format → reviewer email
+ └─ option_2 → format → row → Google Sheet
+Results spreadsheet (Google picker) ────────────────────────────────┘
+```
+
+Use a UTF-8 `.txt` resume; PDF/DOCX need an appropriate extraction block first.
+The example has no saved credentials. Choose TypeSafe credentials on both Jev
+nodes, configure SMTP and replace the deliberate `.invalid` recipient/server,
+and choose a spreadsheet through the Google picker. The platform must have
+Google OAuth configured and agent input subtype blocks enabled (the default).
+After setup, running the graph sends an email or appends a row. AskMany's
+typed ratings are both Route's state and its forwarded data; the two formatting
+steps only adapt that data for the destination blocks.
+
+All automated tests mock the API. To explicitly make seven real requests,
+run from `autogpt_platform/backend`:
+
+```powershell
+poetry run python -m backend.blocks.typesafe._integration --env-file "C:\path\to\.env" --output "jev-integration.json"
+```
+
+Alternatively, set `TYPESAFE_API_KEY` and omit `--env-file`. The script runs
+each block once with synthetic data, uses one item for Filter, prints every
+output pin (including verbatim bodies), and optionally saves them as JSON.
+It stops on a failed block, preserving completed blocks in the optional report.
+It reads the key internally, never prints it or rewrites the dotenv file, and
+does nothing on import. It does not execute the example email/sheet graph.
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/__init__.py b/autogpt_platform/backend/backend/blocks/typesafe/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_base.py b/autogpt_platform/backend/backend/blocks/typesafe/_base.py
new file mode 100644
index 0000000000..85396028a4
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_base.py
@@ -0,0 +1,47 @@
+from collections.abc import Iterator
+from typing import Any
+
+from typesafe_sdk import Choice, Noul, Score
+
+from backend.blocks._base import Block
+from backend.data.model import NodeExecutionStats
+
+from ._client import JevCallResult, call_jev
+from ._config import TypeSafeCredentials
+
+
+class JevBlockBase(Block):
+ async def call_jev(
+ self,
+ credentials: TypeSafeCredentials,
+ state: Any,
+ questions: dict[str, Choice | Score | Noul],
+ ) -> JevCallResult:
+ return await call_jev(
+ api_key=credentials.api_key.get_secret_value(),
+ state=state,
+ questions=questions,
+ )
+
+ def transparency(self, result: JevCallResult) -> Iterator[tuple[str, Any]]:
+ if result.input_tokens is not None and result.output_tokens is not None:
+ self.merge_stats(
+ NodeExecutionStats(
+ input_token_count=result.input_tokens,
+ output_token_count=result.output_tokens,
+ )
+ )
+ data = result.model_dump()
+ for pin in (
+ "request",
+ "response",
+ "latency_ms",
+ "input_tokens",
+ "output_tokens",
+ "request_id",
+ "truncated",
+ "truncation_note",
+ ):
+ yield pin, data[pin]
+ if result.error:
+ yield "error", result.error
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_budget.py b/autogpt_platform/backend/backend/blocks/typesafe/_budget.py
new file mode 100644
index 0000000000..21bb3380d3
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_budget.py
@@ -0,0 +1,96 @@
+import json
+import os
+from typing import Any
+
+from pydantic import BaseModel
+from typesafe_sdk import Choice, Noul, Score
+from typesafe_sdk._core.json import serialize
+from typesafe_sdk.constants import DEFAULT_MODEL, DEFAULT_MODEL_ENV
+
+INPUT_TOKEN_BUDGET = 32_000
+TOKEN_RESERVE = 1_024
+MAX_REQUEST_BYTES = INPUT_TOKEN_BUDGET - TOKEN_RESERVE
+
+
+class PreparedState(BaseModel):
+ state: str
+ truncated: bool
+ truncation_note: str
+
+
+def prepare_state(
+ state: Any, questions: dict[str, Choice | Score | Noul]
+) -> PreparedState:
+ if not questions:
+ raise ValueError("At least one Jev question is required.")
+ text, json_state = _state_text(state)
+ framing_bytes = len(
+ serialize(
+ {
+ "state": "",
+ "model": os.environ.get(DEFAULT_MODEL_ENV, "").strip() or DEFAULT_MODEL,
+ "questions": questions,
+ }
+ )
+ )
+ if framing_bytes > MAX_REQUEST_BYTES:
+ raise ValueError(
+ "Jev questions and request framing exceed the shared request budget "
+ f"of {MAX_REQUEST_BYTES:,} UTF-8 bytes. Shorten the questions or criteria."
+ )
+ available = MAX_REQUEST_BYTES - framing_bytes
+ if len(serialize(text)) - 2 <= available:
+ return PreparedState(state=text, truncated=False, truncation_note="")
+ prefix = _fitting_prefix(text, available)
+ return PreparedState(
+ state=prefix,
+ truncated=True,
+ truncation_note=_truncation_note(text, prefix, json_state),
+ )
+
+
+def _state_text(state: Any) -> tuple[str, bool]:
+ match state:
+ case str():
+ return state, False
+ case _:
+ try:
+ return (
+ json.dumps(
+ state,
+ ensure_ascii=False,
+ separators=(",", ":"),
+ allow_nan=False,
+ ),
+ True,
+ )
+ except (TypeError, ValueError) as error:
+ raise ValueError(
+ "Jev state must be text or JSON-serializable data."
+ ) from error
+
+
+def _fitting_prefix(text: str, available: int) -> str:
+ low, high = 0, len(text)
+ while low < high:
+ middle = (low + high + 1) // 2
+ if len(serialize(text[:middle])) - 2 <= available:
+ low = middle
+ else:
+ high = middle - 1
+ return text[:low]
+
+
+def _truncation_note(original: str, prefix: str, json_state: bool) -> str:
+ note = (
+ f"State truncated from {len(original):,} to {len(prefix):,} characters "
+ f"({len(original.encode('utf-8')):,} to {len(prefix.encode('utf-8')):,} UTF-8 bytes). "
+ f"The entire serialized request is limited to {MAX_REQUEST_BYTES:,} UTF-8 bytes, "
+ f"including all questions and JSON framing, with a {TOKEN_RESERVE:,}-token reserve "
+ f"against the approximately {INPUT_TOKEN_BUDGET:,}-token shared context. "
+ "This conservative byte proxy is not an exact Jev token count; "
+ "no tokenizer is documented."
+ )
+ if json_state:
+ note += " Structured state was compact JSON text; its retained prefix may be incomplete JSON."
+ return note
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_budget_test.py b/autogpt_platform/backend/backend/blocks/typesafe/_budget_test.py
new file mode 100644
index 0000000000..09b748f73b
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_budget_test.py
@@ -0,0 +1,94 @@
+import json
+import os
+
+import pytest
+from typesafe_sdk import Choice, Noul
+from typesafe_sdk._core.json import serialize
+from typesafe_sdk.constants import DEFAULT_MODEL, DEFAULT_MODEL_ENV
+
+from backend.blocks.typesafe._budget import MAX_REQUEST_BYTES, prepare_state
+
+
+def request_size(state: str, questions: dict) -> int:
+ return len(
+ serialize(
+ {
+ "state": state,
+ "model": os.environ.get(DEFAULT_MODEL_ENV, "").strip() or DEFAULT_MODEL,
+ "questions": questions,
+ }
+ )
+ )
+
+
+def test_json_state_is_compact_and_plain_text_is_unchanged():
+ questions = {"q": Noul(instructions="Is it relevant?")}
+ structured = prepare_state({"label": "é", "items": [1, True]}, questions)
+ assert structured.state == '{"label":"é","items":[1,true]}'
+ assert not structured.truncated
+ assert structured.truncation_note == ""
+ assert prepare_state(" unchanged\n", questions).state == " unchanged\n"
+
+
+ "text",
+ ["x" * 40_000, "😀" * 20_000, '"\n' * 40_000],
+ ids=["ascii", "unicode", "json-escaping"],
+)
+def test_state_truncation_accounts_for_full_wire_json_and_unicode(text: str):
+ questions = {"q": Noul(instructions="Is it relevant?")}
+ result = prepare_state(text, questions)
+ assert result.truncated
+ assert text.startswith(result.state)
+ assert request_size(result.state, questions) <= MAX_REQUEST_BYTES
+ assert request_size(text[: len(result.state) + 1], questions) > MAX_REQUEST_BYTES
+ assert "UTF-8" in result.truncation_note
+ assert "token" in result.truncation_note
+
+
+def test_questions_reduce_the_budget_available_to_state():
+ short = {"q": Noul(instructions="Short?")}
+ long = {"q": Noul(instructions="Long?" * 1_000)}
+ assert len(prepare_state("x" * 40_000, long).state) < len(
+ prepare_state("x" * 40_000, short).state
+ )
+
+
+def test_oversized_question_or_criteria_is_rejected_without_truncation():
+ questions = {
+ "q": Choice(instructions="Pick", criteria={"a": "x" * 33_000, "b": "B"})
+ }
+ with pytest.raises(ValueError, match="questions.*budget"):
+ prepare_state("state", questions)
+ assert questions["q"].criteria["a"] == "x" * 33_000
+
+
+def test_no_questions_is_rejected():
+ with pytest.raises(ValueError, match="At least one"):
+ prepare_state("state", {})
+
+
[email protected]("state", [object(), float("nan"), {"bad": float("inf")}])
+def test_non_json_state_is_rejected(state: object):
+ with pytest.raises(ValueError, match="JSON"):
+ prepare_state(state, {"q": Noul(instructions="Is it relevant?")})
+
+
+def test_truncated_structured_state_is_an_explicit_text_prefix():
+ original = {"content": "a" * 40_000}
+ result = prepare_state(original, {"q": Noul(instructions="Relevant?")})
+ assert result.truncated
+ assert json.dumps(original, separators=(",", ":")).startswith(result.state)
+ assert "JSON text" in result.truncation_note
+
+
+ "configured_model", ["", " ", "x", " x ", "jev-custom-" + "x" * 100]
+)
+def test_boundary_helper_uses_configured_model(monkeypatch, configured_model):
+ monkeypatch.setenv("TYPESAFE_DEFAULT_MODEL", configured_model)
+ questions = {"q": Noul(instructions="Relevant?")}
+ text = "x" * 40_000
+ result = prepare_state(text, questions)
+ assert request_size(result.state, questions) <= MAX_REQUEST_BYTES
+ assert request_size(text[: len(result.state) + 1], questions) > MAX_REQUEST_BYTES
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_client.py b/autogpt_platform/backend/backend/blocks/typesafe/_client.py
new file mode 100644
index 0000000000..d1b9659c06
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_client.py
@@ -0,0 +1,174 @@
+import base64
+import json
+from time import perf_counter
+from typing import Any
+
+from pydantic import BaseModel
+from typesafe_sdk import AsyncTypeSafeClient, Choice, Noul, RetryPolicy, Score
+from typesafe_sdk._core.errors import (
+ TypeSafeAPIConnectionError,
+ TypeSafeAPIError,
+ TypeSafeAPIResponseValidationError,
+)
+from typesafe_sdk._core.json import serialize
+from typesafe_sdk._core.response_types import SystemOneResponse
+from typesafe_sdk.constants import DEFAULT_BASE_URL
+
+from ._budget import PreparedState, prepare_state
+from ._wire import WireCapture, capture_wire
+
+
+class JevCallResult(BaseModel):
+ answers: dict[str, dict[str, Any]]
+ request: str
+ response: str | None
+ latency_ms: float
+ input_tokens: int | None
+ output_tokens: int | None
+ request_id: str
+ truncated: bool
+ truncation_note: str
+ error: str = ""
+
+
+async def call_jev(
+ api_key: str, state: Any, questions: dict[str, Choice | Score | Noul]
+) -> JevCallResult:
+ prepared = prepare_state(state, questions)
+ async with AsyncTypeSafeClient(
+ api_key=api_key, base_url=DEFAULT_BASE_URL, retry=RetryPolicy(max_retries=0)
+ ) as client:
+ with capture_wire() as wire:
+ started = perf_counter()
+ try:
+ result = await client.system_one(
+ state=prepared.state, questions=questions
+ )
+ except TypeSafeAPIResponseValidationError as error:
+ return _failure(
+ wire,
+ prepared,
+ started,
+ f"Jev returned an invalid response (HTTP {error.status}).",
+ error.request_id or "",
+ )
+ except TypeSafeAPIError as error:
+ return _failure(
+ wire,
+ prepared,
+ started,
+ f"Jev API request failed (HTTP {error.status}).",
+ error.request_id or "",
+ )
+ except TypeSafeAPIConnectionError:
+ return _failure(
+ wire,
+ prepared,
+ started,
+ "Jev connection failed or timed out; no HTTP response was received.",
+ )
+ except UnicodeDecodeError:
+ if wire.response is None:
+ raise
+ return _failure(
+ wire,
+ prepared,
+ started,
+ "Jev returned an invalid response.",
+ wire.request_id,
+ )
+ return _success(result, wire, prepared, started)
+
+
+def _success(
+ result: SystemOneResponse,
+ wire: WireCapture,
+ prepared: PreparedState,
+ started: float,
+) -> JevCallResult:
+ request_id = result.raw_http_response.headers.get("x-typesafe-request-id", "")
+ errors = [
+ message
+ for missing, message in (
+ (
+ result.usage.input_tokens is None or result.usage.output_tokens is None,
+ "Jev response did not report token usage.",
+ ),
+ (not request_id, "Jev response did not include a request ID."),
+ )
+ if missing
+ ]
+ return _result(
+ wire,
+ prepared,
+ started,
+ answers=json.loads(serialize(result.answers)),
+ input_tokens=result.usage.input_tokens,
+ output_tokens=result.usage.output_tokens,
+ request_id=request_id,
+ error=" ".join(errors),
+ )
+
+
+def _failure(
+ wire: WireCapture,
+ prepared: PreparedState,
+ started: float,
+ error: str,
+ request_id: str = "",
+) -> JevCallResult:
+ return _result(
+ wire,
+ prepared,
+ started,
+ answers={},
+ input_tokens=None,
+ output_tokens=None,
+ request_id=request_id,
+ error=error,
+ )
+
+
+def _result(
+ wire: WireCapture,
+ prepared: PreparedState,
+ started: float,
+ *,
+ answers: dict[str, dict[str, Any]],
+ input_tokens: int | None,
+ output_tokens: int | None,
+ request_id: str,
+ error: str,
+) -> JevCallResult:
+ if wire.request is None or (wire.response is None and not error):
+ raise RuntimeError(
+ "Jev SDK wire capture failed; request transparency is unavailable."
+ )
+ response, encoded = _response_body(wire.response)
+ if encoded:
+ error = (
+ f"{error} Response body is not valid UTF-8; "
+ "response contains the exact bytes as a Base64 data URL."
+ ).strip()
+ return JevCallResult(
+ answers=answers,
+ request=wire.request.decode("utf-8"),
+ response=response,
+ latency_ms=(perf_counter() - started) * 1_000,
+ input_tokens=input_tokens,
+ output_tokens=output_tokens,
+ request_id=request_id,
+ truncated=prepared.truncated,
+ truncation_note=prepared.truncation_note,
+ error=error,
+ )
+
+
+def _response_body(body: bytes | None) -> tuple[str | None, bool]:
+ if body is None:
+ return None, False
+ try:
+ return body.decode("utf-8"), False
+ except UnicodeDecodeError:
+ encoded = base64.b64encode(body).decode("ascii")
+ return f"data:application/octet-stream;base64,{encoded}", True
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_client_test.py b/autogpt_platform/backend/backend/blocks/typesafe/_client_test.py
new file mode 100644
index 0000000000..4e8f732505
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_client_test.py
@@ -0,0 +1,290 @@
+import asyncio
+import json
+import logging
+from collections.abc import Awaitable, Callable, Iterator
+
+import httpx2
+import pytest
+from typesafe_sdk import AsyncTypeSafeClient, Choice, Noul, RetryPolicy, Score
+from typesafe_sdk._core import transport as sdk_transport
+
+from backend.blocks.typesafe import _client
+
+RESPONSE = """ { "model": "jev-latest", "answers": {
+ "choice": {"type":"choice","choice":"a","probabilities":{"a":0.9,"b":0.1},"confidence":0.8},
+ "score": {"type":"score","score":0.75,"legend":{"0":"Low","1":"High"},"probabilities":{"0":0.25,"1":0.75},"confidence":0.6},
+ "noul": {"type":"noul","noul":0.82}
+}, "usage": {"input_tokens":123,"output_tokens":45} }\n"""
+
+
[email protected](params=[False, True], ids=["logging-enabled", "logging-disabled"])
+def logging_mode(request: pytest.FixtureRequest) -> Iterator[None]:
+ previous = logging.root.manager.disable
+ if request.param:
+ logging.disable(logging.INFO)
+ try:
+ yield
+ assert logging.root.manager.disable == (
+ logging.INFO if request.param else previous
+ )
+ finally:
+ logging.disable(previous)
+
+
+def mock_transport(
+ monkeypatch: pytest.MonkeyPatch,
+ handler: Callable[[httpx2.Request], Awaitable[httpx2.Response]],
+) -> list[AsyncTypeSafeClient]:
+ clients = []
+
+ def factory(
+ *, api_key: str, base_url: str, retry: RetryPolicy
+ ) -> AsyncTypeSafeClient:
+ client = AsyncTypeSafeClient(
+ api_key=api_key,
+ base_url=base_url,
+ transport=httpx2.MockTransport(handler),
+ retry=retry,
+ )
+ clients.append(client)
+ return client
+
+ monkeypatch.setattr(_client, "AsyncTypeSafeClient", factory)
+ monkeypatch.delenv("TYPESAFE_DEFAULT_MODEL", raising=False)
+ monkeypatch.delenv("TYPESAFE_BASE_URL", raising=False)
+ return clients
+
+
+async def test_actual_async_sdk_captures_verbatim_bodies_and_typed_answers(
+ monkeypatch: pytest.MonkeyPatch,
+ caplog: pytest.LogCaptureFixture,
+ logging_mode: None,
+):
+ sent: list[httpx2.Request] = []
+
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ sent.append(request)
+ await asyncio.sleep(0.01)
+ return httpx2.Response(
+ 200,
+ content=RESPONSE.encode(),
+ headers={"x-typesafe-request-id": "req-123"},
+ )
+
+ clients = mock_transport(monkeypatch, handler)
+ questions = {
+ "choice": Choice(instructions="Pick", criteria={"a": "A", "b": "B"}),
+ "score": Score(instructions="Rate", criteria=["Low", "High"]),
+ "noul": Noul(instructions="Yes?"),
+ }
+ with caplog.at_level(logging.DEBUG, logger="typesafe_sdk"):
+ result = await _client.call_jev(
+ "test-secret", {"secret": "private-state"}, questions
+ )
+ assert result.request.encode() == sent[0].content
+ assert result.response == RESPONSE
+ assert result.answers == json.loads(RESPONSE)["answers"]
+ assert result.request_id == "req-123"
+ assert result.input_tokens == 123
+ assert result.output_tokens == 45
+ assert result.latency_ms >= 10
+ assert not result.truncated
+ assert result.truncation_note == ""
+ assert result.error == ""
+ assert sent[0].method == "POST"
+ assert str(sent[0].url) == "https://api.typesafe.ai/v1/systemone"
+ assert sent[0].headers["authorization"] == "Bearer test-secret"
+ assert json.loads(result.request)["model"] == "jev-latest"
+ assert json.loads(result.request)["state"] == '{"secret":"private-state"}'
+ assert clients[0]._http_client.is_closed
+ assert "private-state" not in caplog.text
+ assert "test-secret" not in caplog.text
+ assert '"noul":0.82' not in caplog.text
+
+
+async def test_concurrent_calls_do_not_mix_wire_bodies(
+ monkeypatch: pytest.MonkeyPatch,
+ caplog: pytest.LogCaptureFixture,
+ logging_mode: None,
+):
+ arrived = 0
+ both_arrived = asyncio.Event()
+
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ nonlocal arrived
+ state = json.loads(request.content)["state"]
+ arrived += 1
+ if arrived == 2:
+ both_arrived.set()
+ await asyncio.wait_for(both_arrived.wait(), timeout=2)
+ return httpx2.Response(
+ 200,
+ content=RESPONSE.replace("0.82", "0.81" if state == "first" else "0.83"),
+ headers={"x-typesafe-request-id": state},
+ )
+
+ mock_transport(monkeypatch, handler)
+ original_logger = sdk_transport.logger
+ questions = {"noul": Noul(instructions="Yes?")}
+ first, second = await asyncio.gather(
+ _client.call_jev("test", "first", questions),
+ _client.call_jev("test", "second", questions),
+ )
+ assert json.loads(first.request)["state"] == first.request_id == "first"
+ assert json.loads(second.request)["state"] == second.request_id == "second"
+ assert first.answers["noul"]["noul"] == 0.81
+ assert second.answers["noul"]["noul"] == 0.83
+ assert "body=" not in caplog.text
+ assert sdk_transport.logger is original_logger
+
+
+async def test_http_failure_restores_logger_and_closes_client(
+ monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
+):
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ return httpx2.Response(
+ 401,
+ content=' {"error":"private-response"}\n',
+ headers={"x-typesafe-request-id": "error-id"},
+ )
+
+ clients = mock_transport(monkeypatch, handler)
+ logger = logging.getLogger("typesafe_sdk")
+ before = (logger.level, logger.disabled, list(logger.filters))
+ original_logger = sdk_transport.logger
+ result = await _client.call_jev(
+ "test", "private-state", {"q": Noul(instructions="Yes?")}
+ )
+ assert result.error == "Jev API request failed (HTTP 401)."
+ assert result.response == ' {"error":"private-response"}\n'
+ assert json.loads(result.request)["state"] == "private-state"
+ assert result.input_tokens is None and result.output_tokens is None
+ assert result.answers == {}
+ assert result.request_id == "error-id"
+ assert sdk_transport.logger is original_logger
+ assert (logger.level, logger.disabled, logger.filters) == before
+ assert clients[0]._http_client.is_closed
+ assert "private-state" not in caplog.text
+ assert "private-response" not in caplog.text
+
+
+async def test_cancellation_restores_logger_and_closes_client(
+ monkeypatch: pytest.MonkeyPatch, logging_mode: None
+):
+ waiting = asyncio.Event()
+
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ waiting.set()
+ await asyncio.Future()
+ raise AssertionError("Unreachable")
+
+ clients = mock_transport(monkeypatch, handler)
+ logger = logging.getLogger("typesafe_sdk")
+ before = (logger.level, logger.disabled, list(logger.filters))
+ original_logger = sdk_transport.logger
+ task = asyncio.create_task(
+ _client.call_jev("test", "state", {"q": Noul(instructions="Yes?")})
+ )
+ await asyncio.wait_for(waiting.wait(), timeout=2)
+ task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await task
+ assert (logger.level, logger.disabled, logger.filters) == before
+ assert clients[0]._http_client.is_closed
+ assert sdk_transport.logger is original_logger
+
+
+async def test_truncation_is_visible_in_result(monkeypatch: pytest.MonkeyPatch):
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ return httpx2.Response(
+ 200, content=RESPONSE, headers={"x-typesafe-request-id": "truncated"}
+ )
+
+ mock_transport(monkeypatch, handler)
+ result = await _client.call_jev(
+ "test", "x" * 40_000, {"q": Noul(instructions="Yes?")}
+ )
+ assert result.truncated
+ assert result.truncation_note
+ assert len(json.loads(result.request)["state"]) < 40_000
+
+
+async def test_missing_token_usage_is_not_invented(monkeypatch: pytest.MonkeyPatch):
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ return httpx2.Response(
+ 200,
+ content=RESPONSE.replace('"input_tokens":123,"output_tokens":45', ""),
+ headers={"x-typesafe-request-id": "missing-usage"},
+ )
+
+ mock_transport(monkeypatch, handler)
+ result = await _client.call_jev("test", "state", {"q": Noul(instructions="Yes?")})
+ assert result.error == "Jev response did not report token usage."
+ assert result.input_tokens is None and result.output_tokens is None
+ assert result.response == RESPONSE.replace(
+ '"input_tokens":123,"output_tokens":45', ""
+ )
+
+
+async def test_overload_is_not_retried_and_environment_cannot_redirect_credentials(
+ monkeypatch: pytest.MonkeyPatch,
+):
+ sent: list[httpx2.Request] = []
+
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ sent.append(request)
+ return httpx2.Response(529, json={"error": "overloaded"})
+
+ mock_transport(monkeypatch, handler)
+ monkeypatch.setenv("TYPESAFE_BASE_URL", "https://untrusted.invalid")
+ result = await _client.call_jev("test", "state", {"q": Noul(instructions="Yes?")})
+ assert result.error == "Jev API request failed (HTTP 529)."
+ assert len(sent) == 1
+ assert str(sent[0].url) == "https://api.typesafe.ai/v1/systemone"
+
+
[email protected]("response", ["{invalid-json", '{"model":"jev-latest"}'])
+async def test_invalid_response_remains_visible(
+ monkeypatch: pytest.MonkeyPatch,
+ response: str,
+):
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ return httpx2.Response(
+ 200, content=response, headers={"x-typesafe-request-id": "bad"}
+ )
+
+ mock_transport(monkeypatch, handler)
+ result = await _client.call_jev("test", "state", {"q": Noul(instructions="Yes?")})
+ assert result.response == response
+ assert result.error == "Jev returned an invalid response (HTTP 200)."
+ assert result.input_tokens is None and result.output_tokens is None
+ assert result.request_id == "bad"
+
+
+async def test_connection_failure_has_no_fabricated_response(
+ monkeypatch: pytest.MonkeyPatch,
+):
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ raise httpx2.ConnectError("test connection failure", request=request)
+
+ clients = mock_transport(monkeypatch, handler)
+ result = await _client.call_jev("test", "state", {"q": Noul(instructions="Yes?")})
+ assert result.response is None
+ assert result.request and result.request_id == ""
+ assert result.input_tokens is None and result.output_tokens is None
+ assert (
+ result.error
+ == "Jev connection failed or timed out; no HTTP response was received."
+ )
+ assert clients[0]._http_client.is_closed
+
+
+async def test_missing_request_id_preserves_response(monkeypatch: pytest.MonkeyPatch):
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ return httpx2.Response(200, content=RESPONSE)
+
+ mock_transport(monkeypatch, handler)
+ result = await _client.call_jev("test", "state", {"q": Noul(instructions="Yes?")})
+ assert result.response == RESPONSE
+ assert result.request_id == ""
+ assert result.error == "Jev response did not include a request ID."
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_config.py b/autogpt_platform/backend/backend/blocks/typesafe/_config.py
new file mode 100644
index 0000000000..152f04e9bc
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_config.py
@@ -0,0 +1,43 @@
+"""TypeSafe API-key registration for Jev blocks."""
+
+from typing import Literal
+
+from pydantic import SecretStr
+
+from backend.data.model import APIKeyCredentials, CredentialsField, CredentialsMetaInput
+from backend.integrations.providers import ProviderName
+from backend.sdk import BlockCostType, ProviderBuilder
+
+typesafe = (
+ ProviderBuilder("typesafe")
+ .with_description("Jev: fast, typed judgments with probabilities")
+ .with_supported_auth_types("api_key")
+ # User-supplied keys are billed by TypeSafe; no provider price is available.
+ .with_base_cost(0, BlockCostType.RUN)
+ .build()
+)
+
+TypeSafeCredentials = APIKeyCredentials
+TypeSafeCredentialsInput = CredentialsMetaInput[
+ Literal[ProviderName.TYPESAFE], Literal["api_key"]
+]
+
+TEST_CREDENTIALS = APIKeyCredentials(
+ id="01234567-89ab-cdef-0123-456789abcdef",
+ provider="typesafe",
+ api_key=SecretStr("mock-typesafe-api-key"),
+ title="Mock TypeSafe API key",
+ expires_at=None,
+)
+TEST_CREDENTIALS_INPUT = {
+ "provider": TEST_CREDENTIALS.provider,
+ "id": TEST_CREDENTIALS.id,
+ "type": TEST_CREDENTIALS.type,
+ "title": TEST_CREDENTIALS.title,
+}
+
+
+def TypeSafeCredentialsField() -> TypeSafeCredentialsInput:
+ return CredentialsField(
+ description="Your TypeSafe API key. TypeSafe bills usage directly to your account."
+ )
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_integration.py b/autogpt_platform/backend/backend/blocks/typesafe/_integration.py
new file mode 100644
index 0000000000..96506d1db7
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_integration.py
@@ -0,0 +1,174 @@
+"""Opt-in live smoke check: seven blocks, seven requests, synthetic data only."""
+
+import argparse
+import asyncio
+import json
+import os
+import sys
+from pathlib import Path
+from typing import Any, cast
+
+from dotenv import dotenv_values
+from pydantic import SecretStr
+
+from backend.blocks._base import Block
+from backend.blocks.typesafe._config import TypeSafeCredentials
+from backend.blocks.typesafe.ask_many import JevAskManyBlock
+from backend.blocks.typesafe.choice import JevChoiceBlock
+from backend.blocks.typesafe.filter import JevFilterBlock
+from backend.blocks.typesafe.pick_best import JevPickBestBlock
+from backend.blocks.typesafe.route import JevRouteBlock
+from backend.blocks.typesafe.score import JevScoreBlock
+from backend.blocks.typesafe.yes_no import JevYesNoBlock
+
+STATE = "The customer reports being charged twice for the same order."
+OPTIONS = {"billing": "Charges and payments", "technical": "Software failures"}
+LEVELS = [
+ "No evidence of a duplicate charge.",
+ "A duplicate charge is suspected but not explicitly reported.",
+ "The customer explicitly reports a duplicate charge for one order.",
+]
+CASES: tuple[tuple[type[Block], dict[str, Any]], ...] = (
+ (
+ JevChoiceBlock,
+ {"question": "Which team should handle this?", "options": OPTIONS},
+ ),
+ (
+ JevScoreBlock,
+ {"question": "How clear is the duplicate-charge report?", "levels": LEVELS},
+ ),
+ (
+ JevAskManyBlock,
+ {
+ "questions": {
+ "team": {
+ "type": "choice",
+ "question": "Which team should handle this?",
+ "options": OPTIONS,
+ },
+ "evidence": {
+ "type": "score",
+ "question": "How clear is the duplicate-charge report?",
+ "levels": LEVELS,
+ },
+ "duplicate": {
+ "type": "noul",
+ "question": "Does the customer report a duplicate charge?",
+ },
+ }
+ },
+ ),
+ (
+ JevRouteBlock,
+ {
+ "question": "Which team should handle this?",
+ "options": OPTIONS,
+ "data": {"ticket": "synthetic-1"},
+ },
+ ),
+ (
+ JevYesNoBlock,
+ {"question": "Does this concern billing?", "data": {"ticket": "synthetic-1"}},
+ ),
+ (
+ JevPickBestBlock,
+ {
+ "question": "Which action best addresses the customer's report?",
+ "candidates": [
+ "Investigate the duplicate charge.",
+ "Ask the customer to reinstall the app.",
+ ],
+ },
+ ),
+ (
+ JevFilterBlock,
+ {
+ "items": ["The customer reports two charges for the same order."],
+ "question": "How clear is the duplicate-charge report in the item?",
+ "levels": LEVELS,
+ "min_score": 1.0,
+ "max_items": 1,
+ },
+ ),
+)
+
+
+def main() -> int:
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument(
+ "--env-file",
+ type=Path,
+ help="Read only TYPESAFE_API_KEY from this dotenv file.",
+ )
+ parser.add_argument(
+ "--output", type=Path, help="Also save every emitted pin to a JSON report."
+ )
+ args = parser.parse_args()
+ api_key = _read_key(args.env_file)
+ if not api_key:
+ parser.error("Set TYPESAFE_API_KEY or pass --env-file containing that key.")
+ credentials = TypeSafeCredentials(
+ id="5ed76f4a-ccde-4758-8060-27d2a2718f01",
+ provider="typesafe",
+ api_key=SecretStr(api_key),
+ title="Local Jev integration check",
+ )
+ try:
+ asyncio.run(_run_cases(credentials, args.output))
+ except Exception as error:
+ print(
+ f"Integration check failed ({type(error).__name__}); "
+ "exception details omitted to protect credentials.",
+ file=sys.stderr,
+ )
+ return 1
+ return 0
+
+
+def _read_key(env_file: Path | None) -> str | None:
+ if env_file is not None:
+ if not env_file.is_file():
+ raise SystemExit("The supplied dotenv file does not exist.")
+ return dotenv_values(env_file).get("TYPESAFE_API_KEY")
+ return os.environ.get("TYPESAFE_API_KEY")
+
+
+async def _run_cases(credentials: TypeSafeCredentials, output: Path | None) -> None:
+ metadata = {
+ "id": credentials.id,
+ "provider": credentials.provider,
+ "type": credentials.type,
+ "title": credentials.title,
+ }
+ request_count = 0
+ reports = []
+ for block_type, fields in CASES:
+ block = block_type()
+ input_data = block.input_schema.model_validate(
+ {"state": STATE, "credentials": metadata, **fields}
+ )
+ print(f"\n=== {block_type.__name__} ===", flush=True)
+ events = []
+ async for pin, value in block.run(input_data, credentials=credentials):
+ events.append({"pin": pin, "value": value})
+ print(f"{pin}:", flush=True)
+ if pin in {"request", "response"}:
+ print(cast(str, value), flush=True)
+ else:
+ print(json.dumps(value, ensure_ascii=False, default=str), flush=True)
+ request_count += pin == "request"
+ reports.append({"block": block_type.__name__, "outputs": events})
+ if output is not None:
+ output.parent.mkdir(parents=True, exist_ok=True)
+ output.write_text(
+ json.dumps(reports, ensure_ascii=False, indent=2), encoding="utf-8"
+ )
+ if any(event["pin"] == "error" for event in events):
+ raise RuntimeError(f"{block_type.__name__} returned an error.")
+ if request_count != len(CASES):
+ raise RuntimeError("Expected exactly one request per block.")
+ print(f"\nPASS: {len(CASES)} blocks, {request_count} requests.", flush=True)
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_schemas.py b/autogpt_platform/backend/backend/blocks/typesafe/_schemas.py
new file mode 100644
index 0000000000..8952dac438
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_schemas.py
@@ -0,0 +1,96 @@
+from typing import Annotated, Any, Literal
+
+from pydantic import BaseModel, Field
+from typesafe_sdk import Choice, Noul
+from typesafe_sdk import NoulCriteria as SDKNoulCriteria
+from typesafe_sdk import Score
+
+from backend.blocks._base import BlockSchemaInput, BlockSchemaOutput
+from backend.data.model import SchemaField
+
+from ._config import TypeSafeCredentialsField, TypeSafeCredentialsInput
+
+CALIBRATION = (
+ "Levels define the scale: describe concrete evidence for each tier so a very "
+ "good case does not reach the top. For hiring, distinguish terrible, bad, "
+ "okay, excellent, brilliant, and one-in-a-thousand by what qualifies."
+)
+
+
+class JevInput(BlockSchemaInput):
+ credentials: TypeSafeCredentialsInput = TypeSafeCredentialsField()
+ state: Any = SchemaField(
+ default=None,
+ advanced=False,
+ description=(
+ "Text or JSON context shared by every question in this stateless call. "
+ "JSON is compactly serialized; oversized context is truncated with a note."
+ ),
+ )
+
+
+class JevOutput(BlockSchemaOutput):
+ request: str = SchemaField(description="Verbatim JSON request body sent to Jev.")
+ response: str | None = SchemaField(
+ description=(
+ "Verbatim UTF-8 response body from Jev, or null if no response arrived. "
+ "Invalid UTF-8 is preserved as a lossless Base64 data URL with an error."
+ )
+ )
+ latency_ms: float = SchemaField(description="Wall-clock API call duration in ms.")
+ input_tokens: int | None = SchemaField(
+ description="Input tokens reported by the API, or null if unavailable on failure."
+ )
+ output_tokens: int | None = SchemaField(
+ description="Output tokens reported by the API, or null if unavailable on failure."
+ )
+ request_id: str = SchemaField(description="TypeSafe request ID response header.")
+ truncated: bool = SchemaField(description="Whether any input text was truncated.")
+ truncation_note: str = SchemaField(
+ description="Truncation details, or an empty string."
+ )
+ error: str = SchemaField(
+ default="", description="API failure details; emitted only when a call fails."
+ )
+
+
+class ChoiceQuestion(BaseModel):
+ type: Literal["choice"]
+ question: str = Field(min_length=1)
+ options: dict[str, str] = Field(min_length=2)
+
+ def to_question(self) -> Choice:
+ return Choice(instructions=self.question, criteria=self.options)
+
+
+class ScoreQuestion(BaseModel):
+ type: Literal["score"]
+ question: str = Field(min_length=1)
+ levels: list[str] = Field(min_length=2, description=CALIBRATION)
+
+ def to_question(self) -> Score:
+ return Score(instructions=self.question, criteria=self.levels)
+
+
+class NoulCriteria(BaseModel):
+ true: str
+ false: str
+
+
+class NoulQuestion(BaseModel):
+ type: Literal["noul"]
+ question: str = Field(min_length=1)
+ criteria: NoulCriteria | None = None
+
+ def to_question(self) -> Noul:
+ criteria: SDKNoulCriteria | None = (
+ {"true": self.criteria.true, "false": self.criteria.false}
+ if self.criteria is not None
+ else None
+ )
+ return Noul(instructions=self.question, criteria=criteria)
+
+
+Question = Annotated[
+ ChoiceQuestion | ScoreQuestion | NoulQuestion, Field(discriminator="type")
+]
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_test.py b/autogpt_platform/backend/backend/blocks/typesafe/_test.py
new file mode 100644
index 0000000000..4d6fee8d70
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_test.py
@@ -0,0 +1,33 @@
+from typing import Any
+
+from ._client import JevCallResult
+
+TEST_OPTIONS = {"yes": "Evidence is present", "no": "Evidence is absent"}
+TEST_LEVELS = ["No relevant evidence", "Concrete evidence meets the requirement"]
+CHOICE_ANSWER = {
+ "type": "choice",
+ "choice": "yes",
+ "probabilities": {"yes": 0.8, "no": 0.2},
+ "confidence": 0.8,
+}
+SCORE_ANSWER = {
+ "type": "score",
+ "score": 0.8,
+ "legend": {"0": TEST_LEVELS[0], "1": TEST_LEVELS[1]},
+ "probabilities": {"0": 0.2, "1": 0.8},
+ "confidence": 0.8,
+}
+TEST_TRANSPARENCY = [
+ ("request", '{"state":"test","questions":{}}'),
+ ("response", '{"answers":{}}'),
+ ("latency_ms", 12.5),
+ ("input_tokens", 10),
+ ("output_tokens", 3),
+ ("request_id", "test-request"),
+ ("truncated", False),
+ ("truncation_note", ""),
+]
+
+
+def mock_result(answers: dict[str, dict[str, Any]]) -> JevCallResult:
+ return JevCallResult(answers=answers, **dict(TEST_TRANSPARENCY))
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_wire.py b/autogpt_platform/backend/backend/blocks/typesafe/_wire.py
new file mode 100644
index 0000000000..3aadc48cba
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_wire.py
@@ -0,0 +1,94 @@
+import logging
+from collections.abc import Iterator
+from contextlib import contextmanager
+from contextvars import ContextVar
+from threading import Lock
+from typing import Protocol, TypedDict, cast
+
+from pydantic import BaseModel
+from typesafe_sdk._core import transport as sdk_transport
+
+_SDK_WIRE_MESSAGE = "%(method)s %(url)s %(arrow)s headers=%(headers)s body=%(body)r"
+
+
+class WireCapture(BaseModel):
+ request: bytes | None = None
+ response: bytes | None = None
+ request_id: str = ""
+
+
+@contextmanager
+def capture_wire() -> Iterator[WireCapture]:
+ """Capture SDK 0.6.0 _log_wire records without changing global logging."""
+ capture = WireCapture()
+ token = _current_capture.set(capture)
+ try:
+ _enable_capture()
+ try:
+ yield capture
+ finally:
+ _disable_capture()
+ finally:
+ _current_capture.reset(token)
+
+
+class _WireArguments(TypedDict):
+ arrow: str
+ body: bytes | None
+ headers: dict[str, str]
+
+
+class _SDKTransportLogging(Protocol):
+ logger: logging.Logger
+
+
+class _CaptureLogger(logging.Logger):
+ def __init__(self, delegate: logging.Logger):
+ super().__init__(delegate.name)
+ self.delegate = delegate
+
+ def isEnabledFor(self, level: int) -> bool:
+ if level == logging.DEBUG and _current_capture.get() is not None:
+ return True
+ return self.delegate.isEnabledFor(level)
+
+ def handle(self, record: logging.LogRecord) -> None:
+ if record.msg != _SDK_WIRE_MESSAGE:
+ self.delegate.handle(record)
+ return
+ capture = _current_capture.get()
+ if capture is not None:
+ args = cast(_WireArguments, record.args)
+ if args["arrow"] == "->":
+ capture.request = args["body"]
+ elif args["arrow"] == "<-":
+ capture.response = args["body"]
+ capture.request_id = args["headers"].get("x-typesafe-request-id", "")
+ # Wire bodies stay in this context and never reach logging handlers.
+
+
+_current_capture: ContextVar[WireCapture | None] = ContextVar(
+ "jev_wire_capture", default=None
+)
+_lock = Lock()
+_active = 0
+_previous_logger: logging.Logger | None = None
+_transport_logging = cast(_SDKTransportLogging, sdk_transport)
+
+
+def _enable_capture() -> None:
+ global _active, _previous_logger
+ with _lock:
+ if _active == 0:
+ _previous_logger = _transport_logging.logger
+ _transport_logging.logger = _CaptureLogger(_previous_logger)
+ _active += 1
+
+
+def _disable_capture() -> None:
+ global _active, _previous_logger
+ with _lock:
+ _active -= 1
+ if _active == 0 and _previous_logger is not None:
+ _transport_logging.logger = _previous_logger
+ _previous_logger = None
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/_wire_test.py b/autogpt_platform/backend/backend/blocks/typesafe/_wire_test.py
new file mode 100644
index 0000000000..a82fef0f2b
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/_wire_test.py
@@ -0,0 +1,63 @@
+import logging
+from contextvars import Context
+
+import pytest
+from typesafe_sdk._core import transport as sdk_transport
+
+from backend.blocks.typesafe._wire import capture_wire
+
+WIRE_MESSAGE = "%(method)s %(url)s %(arrow)s headers=%(headers)s body=%(body)r"
+
+
+def emit_wire(arrow: str, body: bytes) -> None:
+ sdk_transport.logger.debug(
+ WIRE_MESSAGE,
+ {"method": "POST", "url": "/test", "arrow": arrow, "headers": {}, "body": body},
+ )
+
+
+def test_wire_capture_suppresses_bodies_from_unrelated_contexts(
+ caplog: pytest.LogCaptureFixture,
+):
+ with caplog.at_level(logging.DEBUG):
+ with capture_wire() as capture:
+ emit_wire("->", b"own-private-request")
+ Context().run(emit_wire, "<-", b"unrelated-private-response")
+ emit_wire("<-", b"own-private-response")
+ assert capture.request == b"own-private-request"
+ assert capture.response == b"own-private-response"
+ assert "private" not in caplog.text
+
+
+def test_nested_captures_restore_the_outer_context():
+ with capture_wire() as outer:
+ emit_wire("->", b"outer-request")
+ with capture_wire() as inner:
+ emit_wire("->", b"inner-request")
+ emit_wire("<-", b"inner-response")
+ emit_wire("<-", b"outer-response")
+ assert outer.request == b"outer-request"
+ assert outer.response == b"outer-response"
+ assert inner.request == b"inner-request"
+ assert inner.response == b"inner-response"
+
+
+def test_previously_disabled_logger_is_restored(monkeypatch: pytest.MonkeyPatch):
+ logger = logging.getLogger("typesafe_sdk")
+ monkeypatch.setattr(logger, "disabled", True)
+ with capture_wire() as capture:
+ emit_wire("->", b"request")
+ assert capture.request == b"request"
+ assert logger.disabled
+
+
+def test_global_logging_disable_is_preserved_without_losing_capture():
+ previous = logging.root.manager.disable
+ try:
+ logging.disable(logging.INFO)
+ with capture_wire() as capture:
+ emit_wire("->", b"request")
+ assert capture.request == b"request"
+ assert logging.root.manager.disable == logging.INFO
+ finally:
+ logging.disable(previous)
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/ask_many.py b/autogpt_platform/backend/backend/blocks/typesafe/ask_many.py
new file mode 100644
index 0000000000..a9afdab2ef
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/ask_many.py
@@ -0,0 +1,79 @@
+from typing import Any
+
+from backend.blocks._base import BlockCategory, BlockOutput
+from backend.data.model import SchemaField
+
+from ._base import JevBlockBase
+from ._config import TEST_CREDENTIALS, TEST_CREDENTIALS_INPUT, TypeSafeCredentials
+from ._schemas import CALIBRATION, JevInput, JevOutput, Question
+from ._test import (
+ CHOICE_ANSWER,
+ SCORE_ANSWER,
+ TEST_LEVELS,
+ TEST_OPTIONS,
+ TEST_TRANSPARENCY,
+ mock_result,
+)
+
+
+class JevAskManyBlock(JevBlockBase):
+ class Input(JevInput):
+ questions: dict[str, Question] = SchemaField(
+ description=(
+ "Named typed judgments sharing one state and one API call: "
+ "choice uses question/options, score uses question/levels, "
+ "noul uses question and optional true/false criteria. " + CALIBRATION
+ ),
+ min_length=1,
+ )
+
+ class Output(JevOutput):
+ answers: dict[str, dict[str, Any]] = SchemaField(
+ description="Typed answers as plain dictionaries, keyed by question name."
+ )
+
+ def __init__(self):
+ answers = {"evidence": CHOICE_ANSWER, "quality": SCORE_ANSWER}
+ super().__init__(
+ id="840fa616-d3a1-4384-855c-657ff9e8a7e6",
+ description=f"Ask multiple Choice, Score, or Noul questions of one shared state with Jev in a single call. {CALIBRATION}",
+ categories={BlockCategory.AI},
+ input_schema=self.Input,
+ output_schema=self.Output,
+ test_input={
+ "credentials": TEST_CREDENTIALS_INPUT,
+ "state": "test",
+ "questions": {
+ "evidence": {
+ "type": "choice",
+ "question": "Is evidence present?",
+ "options": TEST_OPTIONS,
+ },
+ "quality": {
+ "type": "score",
+ "question": "How strong is the evidence?",
+ "levels": TEST_LEVELS,
+ },
+ },
+ },
+ test_credentials=TEST_CREDENTIALS,
+ test_output=TEST_TRANSPARENCY + [("answers", answers)],
+ test_mock={"call_jev": lambda *args, **kwargs: mock_result(answers)},
+ )
+
+ async def run(
+ self, input_data: Input, *, credentials: TypeSafeCredentials, **kwargs
+ ) -> BlockOutput:
+ result = await self.call_jev(
+ credentials,
+ input_data.state,
+ {
+ name: question.to_question()
+ for name, question in input_data.questions.items()
+ },
+ )
+ for output in self.transparency(result):
+ yield output
+ if result.error:
+ return
+ yield "answers", result.answers
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/choice.py b/autogpt_platform/backend/backend/blocks/typesafe/choice.py
new file mode 100644
index 0000000000..4fa9614e28
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/choice.py
@@ -0,0 +1,73 @@
+from typesafe_sdk import Choice
+
+from backend.blocks._base import BlockCategory, BlockOutput
+from backend.data.model import SchemaField
+
+from ._base import JevBlockBase
+from ._config import TEST_CREDENTIALS, TEST_CREDENTIALS_INPUT, TypeSafeCredentials
+from ._schemas import JevInput, JevOutput
+from ._test import CHOICE_ANSWER, TEST_OPTIONS, TEST_TRANSPARENCY, mock_result
+
+
+class JevChoiceBlock(JevBlockBase):
+ class Input(JevInput):
+ question: str = SchemaField(
+ description="Plain-language judgment to make.", min_length=1
+ )
+ options: dict[str, str] = SchemaField(
+ description="Option names mapped to concrete descriptions of what qualifies.",
+ min_length=2,
+ )
+
+ class Output(JevOutput):
+ choice: str = SchemaField(description="Winning option name returned by Jev.")
+ probabilities: dict[str, float] = SchemaField(
+ description="Probability for each option."
+ )
+ confidence: float = SchemaField(description="Confidence returned by Jev.")
+
+ def __init__(self):
+ super().__init__(
+ id="d2b65a78-6b22-44a1-90f8-b8fedfdf85af",
+ description="Make a typed choice with Jev. Describe what qualifies for each option in plain language.",
+ categories={BlockCategory.AI},
+ input_schema=self.Input,
+ output_schema=self.Output,
+ test_input={
+ "credentials": TEST_CREDENTIALS_INPUT,
+ "state": "test",
+ "question": "Is evidence present?",
+ "options": TEST_OPTIONS,
+ },
+ test_credentials=TEST_CREDENTIALS,
+ test_output=TEST_TRANSPARENCY
+ + [
+ (key, CHOICE_ANSWER[key])
+ for key in ("choice", "probabilities", "confidence")
+ ],
+ test_mock={
+ "call_jev": lambda *args, **kwargs: mock_result(
+ {"judgment": CHOICE_ANSWER}
+ )
+ },
+ )
+
+ async def run(
+ self, input_data: Input, *, credentials: TypeSafeCredentials, **kwargs
+ ) -> BlockOutput:
+ result = await self.call_jev(
+ credentials,
+ input_data.state,
+ {
+ "judgment": Choice(
+ instructions=input_data.question, criteria=input_data.options
+ )
+ },
+ )
+ for output in self.transparency(result):
+ yield output
+ if result.error:
+ return
+ answer = result.answers["judgment"]
+ for pin in ("choice", "probabilities", "confidence"):
+ yield pin, answer[pin]
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/filter.py b/autogpt_platform/backend/backend/blocks/typesafe/filter.py
new file mode 100644
index 0000000000..366d45d7b7
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/filter.py
@@ -0,0 +1,118 @@
+from typing import Any
+
+from pydantic import model_validator
+from typesafe_sdk import Score
+
+from backend.blocks._base import BlockCategory, BlockOutput
+from backend.data.model import SchemaField
+
+from ._base import JevBlockBase
+from ._config import TEST_CREDENTIALS, TEST_CREDENTIALS_INPUT, TypeSafeCredentials
+from ._schemas import CALIBRATION, JevInput, JevOutput
+from ._test import SCORE_ANSWER, TEST_LEVELS, TEST_TRANSPARENCY, mock_result
+
+
+class JevFilterBlock(JevBlockBase):
+ class Input(JevInput):
+ items: list[Any] = SchemaField(
+ description="Items to score, one sequential API call per item; oversized lists are rejected."
+ )
+ question: str = SchemaField(
+ description="Plain-language question to score for each item, using state as context.",
+ min_length=1,
+ )
+ levels: list[str] = SchemaField(
+ description=f"Ordered level descriptions, lowest first. {CALIBRATION}",
+ min_length=2,
+ )
+ min_score: float = SchemaField(
+ description="Keep items with returned score at or above this value.", ge=0
+ )
+ max_items: int = SchemaField(
+ description="Maximum number of items allowed in this execution.",
+ default=100,
+ ge=1,
+ le=1000,
+ )
+
+ @model_validator(mode="after")
+ def validate_bounds(self):
+ if len(self.items) > self.max_items:
+ raise ValueError(
+ "items exceeds max_items; split the list before filtering"
+ )
+ if self.min_score > len(self.levels) - 1:
+ raise ValueError("min_score exceeds the highest configured score level")
+ return self
+
+ class Output(JevOutput):
+ item_index: int = SchemaField(
+ description="Zero-based item index emitted before that item's transparency outputs."
+ )
+ passed: list[Any] = SchemaField(
+ description="Items meeting min_score, in original order."
+ )
+ rejected: list[Any] = SchemaField(
+ description="Items below min_score, in original order."
+ )
+ scores: list[float] = SchemaField(
+ description="Returned scores aligned with the complete input items list."
+ )
+
+ def __init__(self):
+ super().__init__(
+ id="76d7ffff-48de-4bbe-a890-d2ced69b1be3",
+ description=f"Filter items with sequential Jev scores, one call per item. Each call receives JSON context and item and emits its own transparency outputs. {CALIBRATION}",
+ categories={BlockCategory.AI, BlockCategory.LOGIC},
+ input_schema=self.Input,
+ output_schema=self.Output,
+ test_input={
+ "credentials": TEST_CREDENTIALS_INPUT,
+ "state": "test",
+ "question": "How strong is the evidence?",
+ "levels": TEST_LEVELS,
+ "items": ["evidence"],
+ "min_score": 0.5,
+ },
+ test_credentials=TEST_CREDENTIALS,
+ test_output=[("item_index", 0)]
+ + TEST_TRANSPARENCY
+ + [("passed", ["evidence"]), ("rejected", []), ("scores", [0.8])],
+ test_mock={
+ "call_jev": lambda *args, **kwargs: mock_result(
+ {"judgment": SCORE_ANSWER}
+ )
+ },
+ )
+
+ async def run(
+ self, input_data: Input, *, credentials: TypeSafeCredentials, **kwargs
+ ) -> BlockOutput:
+ scores = []
+ for index, item in enumerate(input_data.items):
+ result = await self.call_jev(
+ credentials,
+ {"item": item, "context": input_data.state},
+ {
+ "judgment": Score(
+ instructions=input_data.question, criteria=input_data.levels
+ )
+ },
+ )
+ yield "item_index", index
+ for output in self.transparency(result):
+ yield output
+ if result.error:
+ return
+ scores.append(result.answers["judgment"]["score"])
+ yield "passed", [
+ item
+ for item, score in zip(input_data.items, scores)
+ if score >= input_data.min_score
+ ]
+ yield "rejected", [
+ item
+ for item, score in zip(input_data.items, scores)
+ if score < input_data.min_score
+ ]
+ yield "scores", scores
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/pick_best.py b/autogpt_platform/backend/backend/blocks/typesafe/pick_best.py
new file mode 100644
index 0000000000..7159afc963
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/pick_best.py
@@ -0,0 +1,122 @@
+import json
+from typing import Any
+
+from typesafe_sdk import Choice
+
+from backend.blocks._base import BlockCategory, BlockOutput
+from backend.data.model import SchemaField
+
+from ._base import JevBlockBase
+from ._config import TEST_CREDENTIALS, TEST_CREDENTIALS_INPUT, TypeSafeCredentials
+from ._schemas import JevInput, JevOutput
+from ._test import TEST_TRANSPARENCY, mock_result
+
+CANDIDATE_DESCRIPTION_LIMIT = 2000
+PICK_ANSWER = {
+ "type": "choice",
+ "choice": "candidate_2",
+ "probabilities": {"candidate_1": 0.2, "candidate_2": 0.8},
+ "confidence": 0.8,
+}
+
+
+class JevPickBestBlock(JevBlockBase):
+ class Input(JevInput):
+ candidates: list[Any] = SchemaField(
+ description="Candidates to compare; descriptions are compact JSON capped at 2,000 characters each.",
+ min_length=2,
+ )
+ question: str = SchemaField(
+ description="Plain-language comparison, for example: Which candidate best meets the requirements in the state?",
+ min_length=1,
+ )
+
+ class Output(JevOutput):
+ best: Any = SchemaField(description="Original candidate selected by Jev.")
+ best_index: int = SchemaField(
+ description="Zero-based index of the selected candidate."
+ )
+ ranked: list[Any] = SchemaField(
+ description="Original candidates sorted by returned probability, highest first; ties retain input order."
+ )
+ probabilities: dict[str, float] = SchemaField(
+ description="Probabilities keyed by candidate_1, candidate_2, and so on."
+ )
+
+ def __init__(self):
+ super().__init__(
+ id="fdae49f8-685a-41ef-80a0-597566b5d4d7",
+ description="Choose the best candidate using Jev and rank candidates directly by its probabilities. State the comparison criteria in plain language.",
+ categories={BlockCategory.AI},
+ input_schema=self.Input,
+ output_schema=self.Output,
+ test_input={
+ "credentials": TEST_CREDENTIALS_INPUT,
+ "state": "test",
+ "question": "Which candidate meets the requirement?",
+ "candidates": ["first", "second"],
+ },
+ test_credentials=TEST_CREDENTIALS,
+ test_output=TEST_TRANSPARENCY
+ + [
+ ("best", "second"),
+ ("best_index", 1),
+ ("ranked", ["second", "first"]),
+ ("probabilities", PICK_ANSWER["probabilities"]),
+ ],
+ test_mock={
+ "call_jev": lambda *args, **kwargs: mock_result(
+ {"judgment": PICK_ANSWER}
+ )
+ },
+ )
+
+ async def run(
+ self, input_data: Input, *, credentials: TypeSafeCredentials, **kwargs
+ ) -> BlockOutput:
+ options, note = candidate_options(input_data.candidates)
+ result = await self.call_jev(
+ credentials,
+ input_data.state,
+ {"judgment": Choice(instructions=input_data.question, criteria=options)},
+ )
+ if note:
+ result.truncated = True
+ result.truncation_note = " ".join(
+ filter(None, [result.truncation_note, note])
+ )
+ for output in self.transparency(result):
+ yield output
+ if result.error:
+ return
+ answer = result.answers["judgment"]
+ labels = list(options)
+ best_index = labels.index(answer["choice"])
+ indices = sorted(
+ range(len(labels)),
+ key=lambda index: answer["probabilities"][labels[index]],
+ reverse=True,
+ )
+ yield "best", input_data.candidates[best_index]
+ yield "best_index", best_index
+ yield "ranked", [input_data.candidates[index] for index in indices]
+ yield "probabilities", answer["probabilities"]
+
+
+def candidate_options(candidates: list[Any]) -> tuple[dict[str, str], str]:
+ options: dict[str, str] = {}
+ shortened: list[str] = []
+ for index, candidate in enumerate(candidates):
+ label = f"candidate_{index + 1}"
+ text = json.dumps(
+ candidate, ensure_ascii=False, separators=(",", ":"), allow_nan=False
+ )
+ options[label] = text[:CANDIDATE_DESCRIPTION_LIMIT]
+ if len(text) > CANDIDATE_DESCRIPTION_LIMIT:
+ shortened.append(label)
+ note = (
+ f"Candidate descriptions capped at {CANDIDATE_DESCRIPTION_LIMIT} characters: {', '.join(shortened)}."
+ if shortened
+ else ""
+ )
+ return options, note
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/resume-review.example.json b/autogpt_platform/backend/backend/blocks/typesafe/resume-review.example.json
new file mode 100644
index 0000000000..641d7ba2b7
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/resume-review.example.json
@@ -0,0 +1,594 @@
+{
+ "id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "version": 1,
+ "is_active": false,
+ "name": "Jev resume review — configure before running",
+ "description": "UTF-8 resume file → file-to-text → typed Jev questions → Jev route → reviewer email for top scores, Google Sheet for remaining scores.",
+ "instructions": "Before running, select TypeSafe credentials on both Jev blocks; configure SMTP credentials, server, and reviewer address; connect Google and select a results spreadsheet. The .invalid email/server are deliberate placeholders. This example has no stored credentials. Upload a synthetic UTF-8 .txt resume for an initial check. PDF/DOCX require a suitable text-extraction step. Running after configuration sends an email or appends a row.",
+ "user_id": "",
+ "created_at": "2026-09-16T00:00:00Z",
+ "nodes": [
+ {
+ "id": "db522e24-718b-421c-92e2-000000000001",
+ "block_id": "95ead23f-8283-4654-aef3-10c053b74a31",
+ "input_default": {
+ "name": "Resume file",
+ "description": "Upload a UTF-8 plain-text resume (.txt). Convert PDF/DOCX to text first.",
+ "base_64": false
+ },
+ "metadata": {
+ "position": {
+ "x": 0,
+ "y": 100
+ },
+ "customized_name": "Resume file"
+ },
+ "input_links": [],
+ "output_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000001",
+ "source_id": "db522e24-718b-421c-92e2-000000000001",
+ "sink_id": "db522e24-718b-421c-92e2-000000000002",
+ "source_name": "result",
+ "sink_name": "file_input",
+ "is_static": true
+ }
+ ],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000002",
+ "block_id": "3735a31f-7e18-4aca-9e90-08a7120674bc",
+ "input_default": {
+ "delimiter": "",
+ "size_limit": 0
+ },
+ "metadata": {
+ "position": {
+ "x": 320,
+ "y": 100
+ },
+ "customized_name": "File to text"
+ },
+ "input_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000001",
+ "source_id": "db522e24-718b-421c-92e2-000000000001",
+ "sink_id": "db522e24-718b-421c-92e2-000000000002",
+ "source_name": "result",
+ "sink_name": "file_input",
+ "is_static": true
+ }
+ ],
+ "output_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000002",
+ "source_id": "db522e24-718b-421c-92e2-000000000002",
+ "sink_id": "db522e24-718b-421c-92e2-000000000003",
+ "source_name": "content",
+ "sink_name": "state",
+ "is_static": false
+ }
+ ],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000003",
+ "block_id": "840fa616-d3a1-4384-855c-657ff9e8a7e6",
+ "input_default": {
+ "questions": {
+ "fit": {
+ "type": "score",
+ "question": "Rate documented fit for a senior Python backend role using only evidence in this resume. Missing evidence does not qualify for a tier.",
+ "levels": [
+ "Terrible: no demonstrated programming or backend experience.",
+ "Bad: only introductory programming exercises; no delivered backend work.",
+ "Okay: has delivered a Python backend feature with testing under guidance.",
+ "Excellent: independently shipped and maintained production Python services, with tests and operational ownership.",
+ "Brilliant: led complex production systems and demonstrates measurable reliability or performance improvements across teams.",
+ "One-in-a-thousand: sustained, externally verifiable, field-leading backend contributions with exceptional impact far beyond normal senior-role expectations."
+ ]
+ },
+ "python_evidence": {
+ "type": "choice",
+ "question": "Does the resume document Python backend work in production?",
+ "options": {
+ "documented": "Explicitly identifies production Python backend work and concrete responsibilities.",
+ "not_documented": "Does not supply explicit evidence of production Python backend work."
+ }
+ }
+ }
+ },
+ "metadata": {
+ "position": {
+ "x": 640,
+ "y": 100
+ },
+ "customized_name": "Jev resume questions"
+ },
+ "input_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000002",
+ "source_id": "db522e24-718b-421c-92e2-000000000002",
+ "sink_id": "db522e24-718b-421c-92e2-000000000003",
+ "source_name": "content",
+ "sink_name": "state",
+ "is_static": false
+ }
+ ],
+ "output_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000003",
+ "source_id": "db522e24-718b-421c-92e2-000000000003",
+ "sink_id": "db522e24-718b-421c-92e2-000000000004",
+ "source_name": "answers",
+ "sink_name": "state",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000004",
+ "source_id": "db522e24-718b-421c-92e2-000000000003",
+ "sink_id": "db522e24-718b-421c-92e2-000000000004",
+ "source_name": "answers",
+ "sink_name": "data",
+ "is_static": false
+ }
+ ],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000004",
+ "block_id": "529b586b-38e7-4ba8-a564-89542075f633",
+ "input_default": {
+ "question": "Choose shortlist only if fit.score is at least 4 and python_evidence.choice is documented. Choose remaining in every other case.",
+ "options": {
+ "shortlist": "The stated shortlist rule is met; request a human review by email.",
+ "remaining": "The stated shortlist rule is not met; record the typed ratings in the sheet."
+ }
+ },
+ "metadata": {
+ "position": {
+ "x": 1000,
+ "y": 100
+ },
+ "customized_name": "Route top resumes"
+ },
+ "input_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000003",
+ "source_id": "db522e24-718b-421c-92e2-000000000003",
+ "sink_id": "db522e24-718b-421c-92e2-000000000004",
+ "source_name": "answers",
+ "sink_name": "state",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000004",
+ "source_id": "db522e24-718b-421c-92e2-000000000003",
+ "sink_id": "db522e24-718b-421c-92e2-000000000004",
+ "source_name": "answers",
+ "sink_name": "data",
+ "is_static": false
+ }
+ ],
+ "output_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000005",
+ "source_id": "db522e24-718b-421c-92e2-000000000004",
+ "sink_id": "db522e24-718b-421c-92e2-000000000005",
+ "source_name": "option_1",
+ "sink_name": "values",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000007",
+ "source_id": "db522e24-718b-421c-92e2-000000000004",
+ "sink_id": "db522e24-718b-421c-92e2-000000000007",
+ "source_name": "option_2",
+ "sink_name": "values",
+ "is_static": false
+ }
+ ],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000005",
+ "block_id": "db7d8f02-2f44-4c55-ab7a-eae0941f0c30",
+ "input_default": {
+ "format": "Resume marked for human review by Jev.\nFit score: {{ fit.score }} / 5\nFit confidence: {{ fit.confidence }}\nProduction Python evidence: {{ python_evidence.choice }}\nReview the uploaded resume and the Jev request/response before deciding.",
+ "escape_html": false
+ },
+ "metadata": {
+ "position": {
+ "x": 1350,
+ "y": -120
+ },
+ "customized_name": "Format email"
+ },
+ "input_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000005",
+ "source_id": "db522e24-718b-421c-92e2-000000000004",
+ "sink_id": "db522e24-718b-421c-92e2-000000000005",
+ "source_name": "option_1",
+ "sink_name": "values",
+ "is_static": false
+ }
+ ],
+ "output_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000006",
+ "source_id": "db522e24-718b-421c-92e2-000000000005",
+ "sink_id": "db522e24-718b-421c-92e2-000000000006",
+ "source_name": "output",
+ "sink_name": "body",
+ "is_static": false
+ }
+ ],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000006",
+ "block_id": "4335878a-394e-4e67-adf2-919877ff49ae",
+ "input_default": {
+ "to_email": "[email protected]",
+ "subject": "Resume requires human review",
+ "config": {
+ "smtp_server": "smtp.example.invalid",
+ "smtp_port": 587
+ }
+ },
+ "metadata": {
+ "position": {
+ "x": 1690,
+ "y": -120
+ },
+ "customized_name": "Email reviewer — configure SMTP"
+ },
+ "input_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000006",
+ "source_id": "db522e24-718b-421c-92e2-000000000005",
+ "sink_id": "db522e24-718b-421c-92e2-000000000006",
+ "source_name": "output",
+ "sink_name": "body",
+ "is_static": false
+ }
+ ],
+ "output_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000011",
+ "source_id": "db522e24-718b-421c-92e2-000000000006",
+ "sink_id": "db522e24-718b-421c-92e2-000000000011",
+ "source_name": "status",
+ "sink_name": "value",
+ "is_static": false
+ }
+ ],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000007",
+ "block_id": "db7d8f02-2f44-4c55-ab7a-eae0941f0c30",
+ "input_default": {
+ "format": "Fit={{ fit.score }}/5; confidence={{ fit.confidence }}; production_python={{ python_evidence.choice }}",
+ "escape_html": false
+ },
+ "metadata": {
+ "position": {
+ "x": 1350,
+ "y": 300
+ },
+ "customized_name": "Format remaining resume row"
+ },
+ "input_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000007",
+ "source_id": "db522e24-718b-421c-92e2-000000000004",
+ "sink_id": "db522e24-718b-421c-92e2-000000000007",
+ "source_name": "option_2",
+ "sink_name": "values",
+ "is_static": false
+ }
+ ],
+ "output_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000008",
+ "source_id": "db522e24-718b-421c-92e2-000000000007",
+ "sink_id": "db522e24-718b-421c-92e2-000000000008",
+ "source_name": "output",
+ "sink_name": "entry",
+ "is_static": false
+ }
+ ],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000008",
+ "block_id": "aeb08fc1-2fc1-4141-bc8e-f758f183a822",
+ "input_default": {
+ "list": [
+ "Remaining resume"
+ ]
+ },
+ "metadata": {
+ "position": {
+ "x": 1690,
+ "y": 300
+ },
+ "customized_name": "Build sheet row"
+ },
+ "input_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000008",
+ "source_id": "db522e24-718b-421c-92e2-000000000007",
+ "sink_id": "db522e24-718b-421c-92e2-000000000008",
+ "source_name": "output",
+ "sink_name": "entry",
+ "is_static": false
+ }
+ ],
+ "output_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000009",
+ "source_id": "db522e24-718b-421c-92e2-000000000008",
+ "sink_id": "db522e24-718b-421c-92e2-000000000009",
+ "source_name": "updated_list",
+ "sink_name": "row",
+ "is_static": false
+ }
+ ],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000009",
+ "block_id": "531d50c0-d6b9-4cf9-a013-7bf783d313c7",
+ "input_default": {
+ "sheet_name": "",
+ "value_input_option": "RAW"
+ },
+ "metadata": {
+ "position": {
+ "x": 2020,
+ "y": 300
+ },
+ "customized_name": "Append remaining resume"
+ },
+ "input_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000009",
+ "source_id": "db522e24-718b-421c-92e2-000000000008",
+ "sink_id": "db522e24-718b-421c-92e2-000000000009",
+ "source_name": "updated_list",
+ "sink_name": "row",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000010",
+ "source_id": "db522e24-718b-421c-92e2-000000000010",
+ "sink_id": "db522e24-718b-421c-92e2-000000000009",
+ "source_name": "result",
+ "sink_name": "spreadsheet",
+ "is_static": true
+ }
+ ],
+ "output_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000012",
+ "source_id": "db522e24-718b-421c-92e2-000000000009",
+ "sink_id": "db522e24-718b-421c-92e2-000000000012",
+ "source_name": "result",
+ "sink_name": "value",
+ "is_static": false
+ }
+ ],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000010",
+ "block_id": "d3b32f15-6fd7-40e3-be52-e083f51b19a2",
+ "input_default": {
+ "name": "Results spreadsheet",
+ "description": "Connect Google and select the destination spreadsheet. The picker carries your credential.",
+ "allowed_views": [
+ "SPREADSHEETS"
+ ],
+ "allow_folder_selection": false
+ },
+ "metadata": {
+ "position": {
+ "x": 1690,
+ "y": 650
+ },
+ "customized_name": "Select results spreadsheet"
+ },
+ "input_links": [],
+ "output_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000010",
+ "source_id": "db522e24-718b-421c-92e2-000000000010",
+ "sink_id": "db522e24-718b-421c-92e2-000000000009",
+ "source_name": "result",
+ "sink_name": "spreadsheet",
+ "is_static": true
+ }
+ ],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000011",
+ "block_id": "363ae599-353e-4804-937e-b2ee3cef3da4",
+ "input_default": {
+ "name": "Email status"
+ },
+ "metadata": {
+ "position": {
+ "x": 2020,
+ "y": -120
+ },
+ "customized_name": "Email status"
+ },
+ "input_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000011",
+ "source_id": "db522e24-718b-421c-92e2-000000000006",
+ "sink_id": "db522e24-718b-421c-92e2-000000000011",
+ "source_name": "status",
+ "sink_name": "value",
+ "is_static": false
+ }
+ ],
+ "output_links": [],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ },
+ {
+ "id": "db522e24-718b-421c-92e2-000000000012",
+ "block_id": "363ae599-353e-4804-937e-b2ee3cef3da4",
+ "input_default": {
+ "name": "Sheet result"
+ },
+ "metadata": {
+ "position": {
+ "x": 2350,
+ "y": 300
+ },
+ "customized_name": "Sheet result"
+ },
+ "input_links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000012",
+ "source_id": "db522e24-718b-421c-92e2-000000000009",
+ "sink_id": "db522e24-718b-421c-92e2-000000000012",
+ "source_name": "result",
+ "sink_name": "value",
+ "is_static": false
+ }
+ ],
+ "output_links": [],
+ "graph_id": "a8a925ad-8de1-4529-b716-933b28f35b08",
+ "graph_version": 1,
+ "webhook_id": null
+ }
+ ],
+ "links": [
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000001",
+ "source_id": "db522e24-718b-421c-92e2-000000000001",
+ "sink_id": "db522e24-718b-421c-92e2-000000000002",
+ "source_name": "result",
+ "sink_name": "file_input",
+ "is_static": true
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000002",
+ "source_id": "db522e24-718b-421c-92e2-000000000002",
+ "sink_id": "db522e24-718b-421c-92e2-000000000003",
+ "source_name": "content",
+ "sink_name": "state",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000003",
+ "source_id": "db522e24-718b-421c-92e2-000000000003",
+ "sink_id": "db522e24-718b-421c-92e2-000000000004",
+ "source_name": "answers",
+ "sink_name": "state",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000004",
+ "source_id": "db522e24-718b-421c-92e2-000000000003",
+ "sink_id": "db522e24-718b-421c-92e2-000000000004",
+ "source_name": "answers",
+ "sink_name": "data",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000005",
+ "source_id": "db522e24-718b-421c-92e2-000000000004",
+ "sink_id": "db522e24-718b-421c-92e2-000000000005",
+ "source_name": "option_1",
+ "sink_name": "values",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000006",
+ "source_id": "db522e24-718b-421c-92e2-000000000005",
+ "sink_id": "db522e24-718b-421c-92e2-000000000006",
+ "source_name": "output",
+ "sink_name": "body",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000007",
+ "source_id": "db522e24-718b-421c-92e2-000000000004",
+ "sink_id": "db522e24-718b-421c-92e2-000000000007",
+ "source_name": "option_2",
+ "sink_name": "values",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000008",
+ "source_id": "db522e24-718b-421c-92e2-000000000007",
+ "sink_id": "db522e24-718b-421c-92e2-000000000008",
+ "source_name": "output",
+ "sink_name": "entry",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000009",
+ "source_id": "db522e24-718b-421c-92e2-000000000008",
+ "sink_id": "db522e24-718b-421c-92e2-000000000009",
+ "source_name": "updated_list",
+ "sink_name": "row",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000010",
+ "source_id": "db522e24-718b-421c-92e2-000000000010",
+ "sink_id": "db522e24-718b-421c-92e2-000000000009",
+ "source_name": "result",
+ "sink_name": "spreadsheet",
+ "is_static": true
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000011",
+ "source_id": "db522e24-718b-421c-92e2-000000000006",
+ "sink_id": "db522e24-718b-421c-92e2-000000000011",
+ "source_name": "status",
+ "sink_name": "value",
+ "is_static": false
+ },
+ {
+ "id": "5393bf41-ef64-4207-9b48-000000000012",
+ "source_id": "db522e24-718b-421c-92e2-000000000009",
+ "sink_id": "db522e24-718b-421c-92e2-000000000012",
+ "source_name": "result",
+ "sink_name": "value",
+ "is_static": false
+ }
+ ],
+ "sub_graphs": []
+}
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/route.py b/autogpt_platform/backend/backend/blocks/typesafe/route.py
new file mode 100644
index 0000000000..23c120e842
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/route.py
@@ -0,0 +1,99 @@
+from typing import Any
+
+from typesafe_sdk import Choice
+
+from backend.blocks._base import BlockCategory, BlockOutput
+from backend.data.model import SchemaField
+
+from ._base import JevBlockBase
+from ._config import TEST_CREDENTIALS, TEST_CREDENTIALS_INPUT, TypeSafeCredentials
+from ._schemas import JevInput, JevOutput
+from ._test import CHOICE_ANSWER, TEST_OPTIONS, TEST_TRANSPARENCY, mock_result
+
+
+class JevRouteBlock(JevBlockBase):
+ class Input(JevInput):
+ question: str = SchemaField(
+ description="Plain-language judgment deciding the route.", min_length=1
+ )
+ options: dict[str, str] = SchemaField(
+ description="Two to five option names and qualifying descriptions; dictionary order assigns option_1 to option_5.",
+ min_length=2,
+ max_length=5,
+ )
+ data: Any = SchemaField(
+ default=None,
+ advanced=False,
+ description="Value forwarded unchanged on the winning route only.",
+ )
+
+ class Output(JevOutput):
+ option_1: Any = SchemaField(
+ description="Data when the first configured option wins."
+ )
+ option_2: Any = SchemaField(
+ description="Data when the second configured option wins."
+ )
+ option_3: Any = SchemaField(
+ description="Data when the third configured option wins."
+ )
+ option_4: Any = SchemaField(
+ description="Data when the fourth configured option wins."
+ )
+ option_5: Any = SchemaField(
+ description="Data when the fifth configured option wins."
+ )
+ choice: str = SchemaField(description="Winning option name returned by Jev.")
+ probabilities: dict[str, float] = SchemaField(
+ description="Probability for each configured option."
+ )
+
+ def __init__(self):
+ super().__init__(
+ id="529b586b-38e7-4ba8-a564-89542075f633",
+ description="Route data using Jev's typed choice. Plain-language option descriptions define each route; only the winning pin fires.",
+ categories={BlockCategory.AI, BlockCategory.LOGIC},
+ input_schema=self.Input,
+ output_schema=self.Output,
+ test_input={
+ "credentials": TEST_CREDENTIALS_INPUT,
+ "state": "test",
+ "question": "Is evidence present?",
+ "options": TEST_OPTIONS,
+ "data": {"id": 7},
+ },
+ test_credentials=TEST_CREDENTIALS,
+ test_output=TEST_TRANSPARENCY
+ + [
+ ("choice", "yes"),
+ ("probabilities", CHOICE_ANSWER["probabilities"]),
+ ("option_1", {"id": 7}),
+ ],
+ test_mock={
+ "call_jev": lambda *args, **kwargs: mock_result(
+ {"judgment": CHOICE_ANSWER}
+ )
+ },
+ )
+
+ async def run(
+ self, input_data: Input, *, credentials: TypeSafeCredentials, **kwargs
+ ) -> BlockOutput:
+ result = await self.call_jev(
+ credentials,
+ input_data.state,
+ {
+ "judgment": Choice(
+ instructions=input_data.question, criteria=input_data.options
+ )
+ },
+ )
+ for output in self.transparency(result):
+ yield output
+ if result.error:
+ return
+ answer = result.answers["judgment"]
+ yield "choice", answer["choice"]
+ yield "probabilities", answer["probabilities"]
+ index = list(input_data.options).index(answer["choice"]) + 1
+ yield f"option_{index}", input_data.data
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/score.py b/autogpt_platform/backend/backend/blocks/typesafe/score.py
new file mode 100644
index 0000000000..78c6afb892
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/score.py
@@ -0,0 +1,83 @@
+from typesafe_sdk import Score
+
+from backend.blocks._base import BlockCategory, BlockOutput
+from backend.data.model import SchemaField
+
+from ._base import JevBlockBase
+from ._config import TEST_CREDENTIALS, TEST_CREDENTIALS_INPUT, TypeSafeCredentials
+from ._schemas import CALIBRATION, JevInput, JevOutput
+from ._test import SCORE_ANSWER, TEST_LEVELS, TEST_TRANSPARENCY, mock_result
+
+
+class JevScoreBlock(JevBlockBase):
+ class Input(JevInput):
+ question: str = SchemaField(
+ description="Plain-language question to score.", min_length=1
+ )
+ levels: list[str] = SchemaField(
+ description=f"Ordered level descriptions, lowest first. {CALIBRATION}",
+ min_length=2,
+ )
+
+ class Output(JevOutput):
+ score: float = SchemaField(
+ description="Score returned by Jev on the 0..max_score scale."
+ )
+ legend: dict[str, str] = SchemaField(
+ description="Level numbers mapped to their descriptions."
+ )
+ probabilities: dict[str, float] = SchemaField(
+ description="Probability for each score level."
+ )
+ confidence: float = SchemaField(description="Confidence returned by Jev.")
+ max_score: int = SchemaField(
+ description="Highest possible score: number of levels minus one."
+ )
+
+ def __init__(self):
+ super().__init__(
+ id="8dcc4bd7-09d7-4c52-a70a-d09f5ca044af",
+ description=f"Score evidence with Jev using an explicit ordered scale. {CALIBRATION}",
+ categories={BlockCategory.AI},
+ input_schema=self.Input,
+ output_schema=self.Output,
+ test_input={
+ "credentials": TEST_CREDENTIALS_INPUT,
+ "state": "test",
+ "question": "How strong is the evidence?",
+ "levels": TEST_LEVELS,
+ },
+ test_credentials=TEST_CREDENTIALS,
+ test_output=TEST_TRANSPARENCY
+ + [
+ (key, SCORE_ANSWER[key])
+ for key in ("score", "legend", "probabilities", "confidence")
+ ]
+ + [("max_score", 1)],
+ test_mock={
+ "call_jev": lambda *args, **kwargs: mock_result(
+ {"judgment": SCORE_ANSWER}
+ )
+ },
+ )
+
+ async def run(
+ self, input_data: Input, *, credentials: TypeSafeCredentials, **kwargs
+ ) -> BlockOutput:
+ result = await self.call_jev(
+ credentials,
+ input_data.state,
+ {
+ "judgment": Score(
+ instructions=input_data.question, criteria=input_data.levels
+ )
+ },
+ )
+ for output in self.transparency(result):
+ yield output
+ if result.error:
+ return
+ answer = result.answers["judgment"]
+ for pin in ("score", "legend", "probabilities", "confidence"):
+ yield pin, answer[pin]
+ yield "max_score", len(input_data.levels) - 1
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/test_blocks.py b/autogpt_platform/backend/backend/blocks/typesafe/test_blocks.py
new file mode 100644
index 0000000000..ced9307dd9
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/test_blocks.py
@@ -0,0 +1,242 @@
+from unittest.mock import AsyncMock
+
+import pytest
+
+from ._config import TEST_CREDENTIALS, TEST_CREDENTIALS_INPUT
+from ._test import CHOICE_ANSWER, SCORE_ANSWER, TEST_LEVELS, TEST_OPTIONS, mock_result
+from .ask_many import JevAskManyBlock
+from .choice import JevChoiceBlock
+from .filter import JevFilterBlock
+from .pick_best import CANDIDATE_DESCRIPTION_LIMIT, JevPickBestBlock
+from .route import JevRouteBlock
+from .score import JevScoreBlock
+from .yes_no import JevYesNoBlock
+
+BLOCKS = (
+ JevChoiceBlock,
+ JevScoreBlock,
+ JevAskManyBlock,
+ JevRouteBlock,
+ JevYesNoBlock,
+ JevPickBestBlock,
+ JevFilterBlock,
+)
+COMMON = {"credentials": TEST_CREDENTIALS_INPUT, "state": {"context": "evidence"}}
+
+
[email protected]("block_type", BLOCKS)
+async def test_embedded_registry_cases_are_mocked(block_type, monkeypatch):
+ block = block_type()
+ mock = AsyncMock(side_effect=block.test_mock["call_jev"])
+ monkeypatch.setattr(block, "call_jev", mock)
+ inputs = block.input_schema.model_validate(block.test_input)
+ outputs = [
+ output async for output in block.run(inputs, credentials=TEST_CREDENTIALS)
+ ]
+ assert outputs == block.test_output
+ mock.assert_awaited_once()
+
+
+async def test_choice_preserves_judgment_and_state(monkeypatch):
+ block = JevChoiceBlock()
+ answer = {**CHOICE_ANSWER, "confidence": 0.37}
+ mock = AsyncMock(return_value=mock_result({"judgment": answer}))
+ monkeypatch.setattr(block, "call_jev", mock)
+ inputs = block.Input(**COMMON, question="Evidence?", options=TEST_OPTIONS)
+ output = dict(
+ [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ )
+ assert output["choice"] == answer["choice"]
+ assert output["probabilities"] == answer["probabilities"]
+ assert output["confidence"] == 0.37
+ assert mock.call_args.args[1] == COMMON["state"]
+ question = mock.call_args.args[2]["judgment"]
+ assert question.instructions == "Evidence?"
+ assert question.criteria == TEST_OPTIONS
+
+
+async def test_score_returns_fractional_score_without_calibration_heuristics(
+ monkeypatch,
+):
+ block = JevScoreBlock()
+ answer = {**SCORE_ANSWER, "score": 0.73}
+ mock = AsyncMock(return_value=mock_result({"judgment": answer}))
+ monkeypatch.setattr(block, "call_jev", mock)
+ inputs = block.Input(**COMMON, question="Evidence?", levels=TEST_LEVELS)
+ output = dict(
+ [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ )
+ assert output["score"] == 0.73
+ assert output["max_score"] == 1
+ assert output["legend"] == answer["legend"]
+ assert mock.call_args.args[2]["judgment"].criteria == TEST_LEVELS
+
+
+async def test_ask_many_shares_one_call_and_supports_noul(monkeypatch):
+ block = JevAskManyBlock()
+ answers = {
+ "choice": CHOICE_ANSWER,
+ "score": SCORE_ANSWER,
+ "noul": {"type": "noul", "noul": 0.65},
+ }
+ mock = AsyncMock(return_value=mock_result(answers))
+ monkeypatch.setattr(block, "call_jev", mock)
+ questions = {
+ "choice": {"type": "choice", "question": "Evidence?", "options": TEST_OPTIONS},
+ "score": {
+ "type": "score",
+ "question": "Evidence strength?",
+ "levels": TEST_LEVELS,
+ },
+ "noul": {
+ "type": "noul",
+ "question": "Evidence present?",
+ "criteria": {"true": "Present", "false": "Absent"},
+ },
+ }
+ inputs = block.Input.model_validate({**COMMON, "questions": questions})
+ output = dict(
+ [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ )
+ assert output["answers"] == answers
+ mock.assert_awaited_once()
+ sent = mock.call_args.args[2]
+ assert list(sent) == list(questions)
+ assert sent["noul"].criteria == {"true": "Present", "false": "Absent"}
+
+
[email protected]("winner", range(5))
+async def test_route_only_fires_winning_pin_in_dictionary_order(winner, monkeypatch):
+ block = JevRouteBlock()
+ options = {name: name for name in ("z", "a", "q", "b", "r")}
+ answer = {**CHOICE_ANSWER, "choice": list(options)[winner]}
+ monkeypatch.setattr(
+ block, "call_jev", AsyncMock(return_value=mock_result({"judgment": answer}))
+ )
+ data = {"original": [1, 2]}
+ inputs = block.Input(**COMMON, question="Route?", options=options, data=data)
+ output = dict(
+ [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ )
+ branches = {
+ key: value for key, value in output.items() if key.startswith("option_")
+ }
+ assert branches == {f"option_{winner + 1}": data}
+
+
+ "choice,confidence,threshold,pin",
+ [
+ ("yes", 0.8, 0.8, "yes"),
+ ("no", 0.8, 0.0, "no"),
+ ("yes", 0.79, 0.8, "unsure"),
+ ("no", 0.0, 0.0, "no"),
+ ],
+)
+async def test_yes_no_uses_exact_confidence_threshold(
+ choice, confidence, threshold, pin, monkeypatch
+):
+ block = JevYesNoBlock()
+ answer = {**CHOICE_ANSWER, "choice": choice, "confidence": confidence}
+ monkeypatch.setattr(
+ block, "call_jev", AsyncMock(return_value=mock_result({"judgment": answer}))
+ )
+ inputs = block.Input(
+ **COMMON, question="Evidence?", data=None, min_confidence=threshold
+ )
+ output = dict(
+ [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ )
+ assert {
+ key: value for key, value in output.items() if key in {"yes", "no", "unsure"}
+ } == {pin: None}
+ assert output["confidence"] == confidence
+
+
+async def test_pick_best_uses_choice_for_winner_and_probability_for_ranking(
+ monkeypatch,
+):
+ block = JevPickBestBlock()
+ candidates = [{"id": 1}, {"id": 2}, {"id": 3}]
+ answer = {
+ "choice": "candidate_2",
+ "probabilities": {"candidate_3": 0.5, "candidate_2": 0.25, "candidate_1": 0.25},
+ }
+ mock = AsyncMock(return_value=mock_result({"judgment": answer}))
+ monkeypatch.setattr(block, "call_jev", mock)
+ inputs = block.Input(**COMMON, question="Best?", candidates=candidates)
+ output = dict(
+ [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ )
+ assert output["best"] == candidates[1]
+ assert output["best_index"] == 1
+ assert output["ranked"] == [candidates[2], candidates[0], candidates[1]]
+ assert output["probabilities"] == answer["probabilities"]
+ assert mock.call_args.args[2]["judgment"].criteria["candidate_1"] == '{"id":1}'
+
+
+async def test_pick_best_reports_description_truncation(monkeypatch):
+ block = JevPickBestBlock()
+ answer = {
+ "choice": "candidate_1",
+ "probabilities": {"candidate_1": 0.8, "candidate_2": 0.2},
+ }
+ mock = AsyncMock(return_value=mock_result({"judgment": answer}))
+ monkeypatch.setattr(block, "call_jev", mock)
+ inputs = block.Input(**COMMON, question="Best?", candidates=["x" * 3000, "short"])
+ output = dict(
+ [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ )
+ assert (
+ len(mock.call_args.args[2]["judgment"].criteria["candidate_1"])
+ == CANDIDATE_DESCRIPTION_LIMIT
+ )
+ assert output["truncated"] is True
+ assert "candidate_1" in output["truncation_note"]
+ assert output["best"] == "x" * 3000
+
+
+async def test_filter_preserves_order_alignment_and_per_call_transparency(monkeypatch):
+ block = JevFilterBlock()
+ results = [
+ mock_result({"judgment": {**SCORE_ANSWER, "score": score}})
+ for score in (0.2, 0.5, 0.9)
+ ]
+ for index, result in enumerate(results):
+ result.request_id = f"request-{index}"
+ mock = AsyncMock(side_effect=results)
+ monkeypatch.setattr(block, "call_jev", mock)
+ inputs = block.Input(
+ **COMMON,
+ items=["low", "equal", "high"],
+ question="Quality?",
+ levels=TEST_LEVELS,
+ min_score=0.5,
+ )
+ outputs = [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ output = dict(outputs)
+ assert output["passed"] == ["equal", "high"]
+ assert output["rejected"] == ["low"]
+ assert output["scores"] == [0.2, 0.5, 0.9]
+ assert [value for pin, value in outputs if pin == "item_index"] == [0, 1, 2]
+ assert [value for pin, value in outputs if pin == "request_id"] == [
+ "request-0",
+ "request-1",
+ "request-2",
+ ]
+ assert [call.args[1] for call in mock.call_args_list] == [
+ {"item": item, "context": COMMON["state"]} for item in inputs.items
+ ]
+ assert list(mock.call_args_list[0].args[1]) == ["item", "context"]
+
+
+async def test_filter_empty_input_makes_no_calls(monkeypatch):
+ block = JevFilterBlock()
+ mock = AsyncMock()
+ monkeypatch.setattr(block, "call_jev", mock)
+ inputs = block.Input(
+ **COMMON, items=[], question="Quality?", levels=TEST_LEVELS, min_score=0.5
+ )
+ outputs = [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ assert outputs == [("passed", []), ("rejected", []), ("scores", [])]
+ mock.assert_not_awaited()
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/test_candidates.py b/autogpt_platform/backend/backend/blocks/typesafe/test_candidates.py
new file mode 100644
index 0000000000..865de5686f
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/test_candidates.py
@@ -0,0 +1,36 @@
+import gc
+import weakref
+from typing import Any
+
+from backend.blocks.typesafe import pick_best
+
+
+class TrackedDescription(str):
+ pass
+
+
+def test_candidate_options_releases_full_descriptions_during_iteration(monkeypatch):
+ serialize = pick_best.json.dumps
+ descriptions: list[weakref.ReferenceType[TrackedDescription]] = []
+ candidates = ["é" * 3000, {"text": "b" * 4000}, "short", [1, 2]]
+ expected = {
+ f"candidate_{index + 1}": serialize(
+ candidate, ensure_ascii=False, separators=(",", ":"), allow_nan=False
+ )[: pick_best.CANDIDATE_DESCRIPTION_LIMIT]
+ for index, candidate in enumerate(candidates)
+ }
+
+ def track_description(candidate: Any, **kwargs) -> str:
+ gc.collect()
+ assert sum(reference() is not None for reference in descriptions) <= 1
+ description = TrackedDescription(serialize(candidate, **kwargs))
+ descriptions.append(weakref.ref(description))
+ return description
+
+ monkeypatch.setattr(pick_best.json, "dumps", track_description)
+ options, note = pick_best.candidate_options(candidates)
+ assert list(options.items()) == list(expected.items())
+ assert (
+ note
+ == "Candidate descriptions capped at 2000 characters: candidate_1, candidate_2."
+ )
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/test_client_utf8.py b/autogpt_platform/backend/backend/blocks/typesafe/test_client_utf8.py
new file mode 100644
index 0000000000..9ee8092cae
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/test_client_utf8.py
@@ -0,0 +1,72 @@
+import base64
+import json
+import logging
+
+import httpx2
+import pytest
+from typesafe_sdk import Noul
+
+from backend.blocks.typesafe import _client
+from backend.blocks.typesafe._client_test import mock_transport
+
+BASE64_DATA_URL_PREFIX = "data:application/octet-stream;base64,"
+
+
[email protected]("status", [200, 401, 502])
+ "body",
+ [b"\xff private-response \\xff", b'{"error":"private-response\xff\\xff"}'],
+ ids=["non-json", "inside-json-string"],
+)
+async def test_malformed_utf8_preserves_exact_bytes_and_request_metadata(
+ monkeypatch: pytest.MonkeyPatch,
+ caplog: pytest.LogCaptureFixture,
+ status: int,
+ body: bytes,
+):
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ return httpx2.Response(
+ status, content=body, headers={"x-typesafe-request-id": "invalid-utf8-id"}
+ )
+
+ clients = mock_transport(monkeypatch, handler)
+ with caplog.at_level(logging.DEBUG, logger="typesafe_sdk"):
+ result = await _client.call_jev(
+ "test-secret", "test-state", {"q": Noul(instructions="Yes?")}
+ )
+ assert result.response is not None
+ assert result.response.startswith(BASE64_DATA_URL_PREFIX)
+ assert (
+ base64.b64decode(
+ result.response.removeprefix(BASE64_DATA_URL_PREFIX), validate=True
+ )
+ == body
+ )
+ assert result.request_id == "invalid-utf8-id"
+ assert json.loads(result.request)["state"] == "test-state"
+ assert result.answers == {}
+ assert result.input_tokens is None and result.output_tokens is None
+ assert "UTF-8" in result.error and "Base64" in result.error
+ assert result.latency_ms >= 0
+ assert json.loads(result.model_dump_json())["response"] == result.response
+ assert clients[0]._http_client.is_closed
+ assert "private-response" not in caplog.text
+
+
+ "body", [b'{"error":"literal \\xff stays literal"}', '{"error":"é"}'.encode()]
+)
+async def test_valid_utf8_error_response_remains_verbatim(
+ monkeypatch: pytest.MonkeyPatch, body: bytes
+):
+ async def handler(request: httpx2.Request) -> httpx2.Response:
+ return httpx2.Response(
+ 502, content=body, headers={"x-typesafe-request-id": "valid-utf8-id"}
+ )
+
+ mock_transport(monkeypatch, handler)
+ result = await _client.call_jev(
+ "test", "test-state", {"q": Noul(instructions="Yes?")}
+ )
+ assert result.response == body.decode("utf-8")
+ assert result.error == "Jev API request failed (HTTP 502)."
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/test_errors.py b/autogpt_platform/backend/backend/blocks/typesafe/test_errors.py
new file mode 100644
index 0000000000..0081abdcac
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/test_errors.py
@@ -0,0 +1,82 @@
+from unittest.mock import AsyncMock
+
+import pytest
+
+from ._config import TEST_CREDENTIALS, TEST_CREDENTIALS_INPUT
+from ._test import SCORE_ANSWER, TEST_LEVELS, mock_result
+from .ask_many import JevAskManyBlock
+from .choice import JevChoiceBlock
+from .filter import JevFilterBlock
+from .pick_best import JevPickBestBlock
+from .route import JevRouteBlock
+from .score import JevScoreBlock
+from .yes_no import JevYesNoBlock
+
+
+ "block_type",
+ [
+ JevChoiceBlock,
+ JevScoreBlock,
+ JevAskManyBlock,
+ JevRouteBlock,
+ JevYesNoBlock,
+ JevPickBestBlock,
+ JevFilterBlock,
+ ],
+)
+async def test_failed_calls_emit_transparency_and_no_judgment(block_type, monkeypatch):
+ block = block_type()
+ failure = mock_result({}).model_copy(
+ update={
+ "error": "Jev API returned HTTP 401",
+ "response": '{"error":"unauthorized"}',
+ "input_tokens": None,
+ "output_tokens": None,
+ }
+ )
+ monkeypatch.setattr(block, "call_jev", AsyncMock(return_value=failure))
+ inputs = block.input_schema.model_validate(block.test_input)
+ outputs = dict(
+ [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ )
+ expected = failure.model_dump(exclude={"answers"})
+ if block_type == JevFilterBlock:
+ expected["item_index"] = 0
+ assert outputs == expected
+
+
+async def test_filter_failure_keeps_transcripts_without_partial_result_lists(
+ monkeypatch,
+):
+ block = JevFilterBlock()
+ success = mock_result({"judgment": SCORE_ANSWER})
+ failure = mock_result({}).model_copy(
+ update={
+ "error": "Jev connection failed",
+ "response": None,
+ "input_tokens": None,
+ "output_tokens": None,
+ }
+ )
+ mock = AsyncMock(side_effect=[success, failure])
+ monkeypatch.setattr(block, "call_jev", mock)
+ inputs = block.Input.model_validate(
+ {
+ "credentials": TEST_CREDENTIALS_INPUT,
+ "state": "context",
+ "items": ["first", "failed", "never"],
+ "question": "Quality?",
+ "levels": TEST_LEVELS,
+ "min_score": 0.5,
+ }
+ )
+ outputs = [item async for item in block.run(inputs, credentials=TEST_CREDENTIALS)]
+ assert [value for pin, value in outputs if pin == "item_index"] == [0, 1]
+ assert [value for pin, value in outputs if pin == "response"] == [
+ success.response,
+ None,
+ ]
+ assert outputs[-1] == ("error", "Jev connection failed")
+ assert not {"passed", "rejected", "scores"}.intersection(dict(outputs))
+ assert mock.await_count == 2
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/test_null_input.py b/autogpt_platform/backend/backend/blocks/typesafe/test_null_input.py
new file mode 100644
index 0000000000..0daeeff799
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/test_null_input.py
@@ -0,0 +1,33 @@
+from unittest.mock import AsyncMock
+
+import pytest
+
+from backend.data.execution import ExecutionContext
+
+from ._config import TEST_CREDENTIALS
+from .route import JevRouteBlock
+from .yes_no import JevYesNoBlock
+
+
+ "block_type,pin", [(JevRouteBlock, "option_1"), (JevYesNoBlock, "yes")]
+)
+async def test_null_state_and_data_survive_platform_execution(
+ block_type, pin, monkeypatch
+):
+ block = block_type()
+ mock = AsyncMock(side_effect=block.test_mock["call_jev"])
+ monkeypatch.setattr(block, "call_jev", mock)
+ inputs = {**block.test_input, "state": None, "data": None}
+ outputs = dict(
+ [
+ item
+ async for item in block.execute(
+ inputs,
+ credentials=TEST_CREDENTIALS,
+ execution_context=ExecutionContext(),
+ )
+ ]
+ )
+ assert outputs[pin] is None
+ assert mock.call_args.args[1] is None
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/test_provider.py b/autogpt_platform/backend/backend/blocks/typesafe/test_provider.py
new file mode 100644
index 0000000000..e812eee432
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/test_provider.py
@@ -0,0 +1,47 @@
+"""Verify Jev credential discovery and explicit BYOK cost registration."""
+
+import pytest
+
+from backend.blocks._base import BlockCostType
+from backend.blocks.typesafe._config import typesafe
+from backend.blocks.typesafe.ask_many import JevAskManyBlock
+from backend.blocks.typesafe.choice import JevChoiceBlock
+from backend.blocks.typesafe.filter import JevFilterBlock
+from backend.blocks.typesafe.pick_best import JevPickBestBlock
+from backend.blocks.typesafe.route import JevRouteBlock
+from backend.blocks.typesafe.score import JevScoreBlock
+from backend.blocks.typesafe.yes_no import JevYesNoBlock
+from backend.data.block_cost_config import BLOCK_COSTS
+from backend.integrations.providers import ProviderName
+from backend.sdk.cost_integration import get_block_costs
+from backend.sdk.registry import AutoRegistry
+
+
+def test_typesafe_provider_is_discoverable_without_a_shared_key():
+ assert AutoRegistry.get_provider(ProviderName.TYPESAFE.value) is typesafe
+ assert typesafe.supported_auth_types == {"api_key"}
+ assert typesafe.default_credentials == []
+
+
+ "block_type",
+ [
+ JevChoiceBlock,
+ JevScoreBlock,
+ JevAskManyBlock,
+ JevRouteBlock,
+ JevYesNoBlock,
+ JevPickBestBlock,
+ JevFilterBlock,
+ ],
+)
+def test_every_jev_block_has_a_typed_key_and_explicit_cost_entry(block_type):
+ block = block_type()
+ credentials = block.input_schema.get_credentials_fields_info()["credentials"]
+ assert credentials.provider == {ProviderName.TYPESAFE}
+ assert credentials.supported_types == {"api_key"}
+ costs = get_block_costs(block_type)
+ assert BLOCK_COSTS[block_type] == costs
+ assert len(costs) == 1
+ assert costs[0].cost_type == BlockCostType.RUN
+ assert costs[0].cost_amount == 0
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/test_validation.py b/autogpt_platform/backend/backend/blocks/typesafe/test_validation.py
new file mode 100644
index 0000000000..c3f1bb39e1
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/test_validation.py
@@ -0,0 +1,75 @@
+import pytest
+from pydantic import ValidationError
+
+from ._config import TEST_CREDENTIALS_INPUT
+from ._test import TEST_LEVELS
+from .ask_many import JevAskManyBlock
+from .choice import JevChoiceBlock
+from .filter import JevFilterBlock
+from .pick_best import JevPickBestBlock
+from .route import JevRouteBlock
+from .score import JevScoreBlock
+from .yes_no import JevYesNoBlock
+
+BLOCKS = (
+ JevChoiceBlock,
+ JevScoreBlock,
+ JevAskManyBlock,
+ JevRouteBlock,
+ JevYesNoBlock,
+ JevPickBestBlock,
+ JevFilterBlock,
+)
+COMMON = {"credentials": TEST_CREDENTIALS_INPUT, "state": {"context": "evidence"}}
+
+
+ "block_type,inputs",
+ [
+ (JevChoiceBlock, {"question": "Pick?", "options": {"one": "Only"}}),
+ (JevScoreBlock, {"question": "Rate?", "levels": ["Only"]}),
+ (
+ JevRouteBlock,
+ {
+ "question": "Route?",
+ "options": {str(i): str(i) for i in range(6)},
+ "data": 1,
+ },
+ ),
+ (JevYesNoBlock, {"question": "Yes?", "min_confidence": 1.1, "data": 1}),
+ (JevPickBestBlock, {"question": "Best?", "candidates": []}),
+ (
+ JevFilterBlock,
+ {
+ "question": "Rate?",
+ "levels": TEST_LEVELS,
+ "items": [1, 2],
+ "min_score": 0.5,
+ "max_items": 1,
+ },
+ ),
+ (
+ JevFilterBlock,
+ {"question": "Rate?", "levels": TEST_LEVELS, "items": [], "min_score": 2},
+ ),
+ (
+ JevAskManyBlock,
+ {"questions": {"bad": {"type": "text", "question": "Generate?"}}},
+ ),
+ (
+ JevAskManyBlock,
+ {
+ "questions": {
+ "bad": {
+ "type": "noul",
+ "question": "True?",
+ "criteria": {"true": "True"},
+ }
+ }
+ },
+ ),
+ ],
+)
+def test_invalid_inputs_are_rejected_before_api_calls(block_type, inputs):
+ with pytest.raises(ValidationError):
+ block_type.Input(**COMMON, **inputs)
diff --git a/autogpt_platform/backend/backend/blocks/typesafe/yes_no.py b/autogpt_platform/backend/backend/blocks/typesafe/yes_no.py
new file mode 100644
index 0000000000..045b9046a6
--- /dev/null
+++ b/autogpt_platform/backend/backend/blocks/typesafe/yes_no.py
@@ -0,0 +1,92 @@
+from typing import Any
+
+from typesafe_sdk import Choice
+
+from backend.blocks._base import BlockCategory, BlockOutput
+from backend.data.model import SchemaField
+
+from ._base import JevBlockBase
+from ._config import TEST_CREDENTIALS, TEST_CREDENTIALS_INPUT, TypeSafeCredentials
+from ._schemas import JevInput, JevOutput
+from ._test import CHOICE_ANSWER, TEST_TRANSPARENCY, mock_result
+
+
+class JevYesNoBlock(JevBlockBase):
+ class Input(JevInput):
+ question: str = SchemaField(
+ description="Plain-language question answerable yes or no.", min_length=1
+ )
+ data: Any = SchemaField(
+ default=None,
+ advanced=False,
+ description="Value forwarded unchanged on yes, no, or unsure.",
+ )
+ min_confidence: float = SchemaField(
+ description="Emit unsure when Jev's confidence is below this threshold.",
+ default=0.0,
+ ge=0.0,
+ le=1.0,
+ )
+
+ class Output(JevOutput):
+ yes: Any = SchemaField(
+ description="Data when Jev chooses yes and meets the confidence threshold."
+ )
+ no: Any = SchemaField(
+ description="Data when Jev chooses no and meets the confidence threshold."
+ )
+ unsure: Any = SchemaField(
+ description="Data when confidence is below min_confidence."
+ )
+ confidence: float = SchemaField(description="Confidence returned by Jev.")
+
+ def __init__(self):
+ super().__init__(
+ id="71f7d92c-a895-407f-8966-ede134a854b7",
+ description="Ask Jev a plain-language yes/no question and forward data to the chosen pin, or unsure below your confidence threshold.",
+ categories={BlockCategory.AI, BlockCategory.LOGIC},
+ input_schema=self.Input,
+ output_schema=self.Output,
+ test_input={
+ "credentials": TEST_CREDENTIALS_INPUT,
+ "state": "test",
+ "question": "Is evidence present?",
+ "data": {"id": 7},
+ },
+ test_credentials=TEST_CREDENTIALS,
+ test_output=TEST_TRANSPARENCY + [("confidence", 0.8), ("yes", {"id": 7})],
+ test_mock={
+ "call_jev": lambda *args, **kwargs: mock_result(
+ {"judgment": CHOICE_ANSWER}
+ )
+ },
+ )
+
+ async def run(
+ self, input_data: Input, *, credentials: TypeSafeCredentials, **kwargs
+ ) -> BlockOutput:
+ result = await self.call_jev(
+ credentials,
+ input_data.state,
+ {
+ "judgment": Choice(
+ instructions=input_data.question,
+ criteria={
+ "yes": "The answer to the question is yes.",
+ "no": "The answer to the question is no.",
+ },
+ )
+ },
+ )
+ for output in self.transparency(result):
+ yield output
+ if result.error:
+ return
+ answer = result.answers["judgment"]
+ yield "confidence", answer["confidence"]
+ pin = (
+ "unsure"
+ if answer["confidence"] < input_data.min_confidence
+ else answer["choice"]
+ )
+ yield pin, input_data.data
diff --git a/autogpt_platform/backend/backend/integrations/providers.py b/autogpt_platform/backend/backend/integrations/providers.py
index d11a12fc28..974cae3901 100644
--- a/autogpt_platform/backend/backend/integrations/providers.py
+++ b/autogpt_platform/backend/backend/integrations/providers.py
@@ -57,6 +57,7 @@ class ProviderName(str, Enum):
TELEGRAM = "telegram"
TWITTER = "twitter"
TODOIST = "todoist"
+ TYPESAFE = "typesafe"
UNREAL_SPEECH = "unreal_speech"
V0 = "v0"
WEBSHARE_PROXY = "webshare_proxy"
diff --git a/autogpt_platform/backend/poetry.lock b/autogpt_platform/backend/poetry.lock
index ab36660890..d949c9e756 100644
--- a/autogpt_platform/backend/poetry.lock
+++ b/autogpt_platform/backend/poetry.lock
@@ -2723,6 +2723,28 @@ http2 = ["h2 (>=3,<5)"]
socks = ["socksio (==1.*)"]
trio = ["trio (>=0.22.0,<1.0)"]
+[[package]]
+name = "httpcore2"
+version = "2.3.0"
+description = "A minimal low-level HTTP client."
+optional = false
+python-versions = ">=3.10"
+groups = ["main"]
+files = [
+ {file = "httpcore2-2.3.0-py3-none-any.whl", hash = "sha256:477e9e334f74e5240dcac002e890580f36a57d40ff0fb14cc9655731d23b8415"},
+ {file = "httpcore2-2.3.0.tar.gz", hash = "sha256:07327e251560960eea8e969d92d4c6a325feb13cca39e25340731336c3baf924"},
+]
+
+[package.dependencies]
+h11 = ">=0.16"
+truststore = ">=0.10"
+
+[package.extras]
+asyncio = ["anyio (>=4.5.0,<5.0)"]
+http2 = ["h2 (>=3,<5)"]
+socks = ["socksio (==1.*)"]
+trio = ["trio (>=0.22.0,<1.0)"]
+
[[package]]
name = "httplib2"
version = "0.31.2"
@@ -2829,6 +2851,31 @@ files = [
{file = "httpx_sse-0.4.3.tar.gz", hash = "sha256:9b1ed0127459a66014aec3c56bebd93da3c1bc8bb6618c8082039a44889a755d"},
]
+[[package]]
+name = "httpx2"
+version = "2.3.0"
+description = "The next generation HTTP client."
+optional = false
+python-versions = ">=3.10"
+groups = ["main"]
+files = [
+ {file = "httpx2-2.3.0-py3-none-any.whl", hash = "sha256:6f393663bdf6dbe7fe90118e3eb5b2bd024a675cae0390ac08cec9198812d8b7"},
+ {file = "httpx2-2.3.0.tar.gz", hash = "sha256:227e7c41d95a76d4077a52640564132777215fc3394e07b66a3116c33d668fa9"},
+]
+
+[package.dependencies]
+anyio = "*"
+httpcore2 = "2.3.0"
+idna = "*"
+truststore = ">=0.10"
+
+[package.extras]
+brotli = ["brotli ; platform_python_implementation == \"CPython\"", "brotlicffi ; platform_python_implementation != \"CPython\""]
+cli = ["click (==8.*)", "pygments (==2.*)", "rich (>=10,<15)"]
+http2 = ["h2 (>=3,<5)"]
+socks = ["socksio (==1.*)"]
+zstd = ["zstandard (>=0.18.0) ; python_version <= \"3.13\""]
+
[[package]]
name = "hyperframe"
version = "6.1.0"
@@ -3694,6 +3741,69 @@ files = [
{file = "msgpack-1.1.2.tar.gz", hash = "sha256:3b60763c1373dd60f398488069bcdc703cd08a711477b5d480eecc9f9626f47e"},
]
+[[package]]
+name = "msgspec"
+version = "0.21.1"
+description = "A fast serialization and validation library, with builtin support for JSON, MessagePack, YAML, and TOML."
+optional = false
+python-versions = ">=3.10"
+groups = ["main"]
+files = [
+ {file = "msgspec-0.21.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:72d9cd03241b8b2edb2e12dcc66c500fa480d8cbd71a8bac105809d468882064"},
+ {file = "msgspec-0.21.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:ed2ab278200e743a1d2610a4e0c8fc74f6cecb8548544cdec43f927bd9265238"},
+ {file = "msgspec-0.21.1-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:dd677e3001fdfed9186de72eab434da2976303cd5eb9550921d3d0c3e3e168ce"},
+ {file = "msgspec-0.21.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f667b90b37fad734a91671abd68e0d7f4d066862771b87e91c53996dcb7a9027"},
+ {file = "msgspec-0.21.1-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:49880fd20fdbcfe1b793f07dd83f12572bab679c9800352c8b2240289aa46a06"},
+ {file = "msgspec-0.21.1-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:ae0162e22849a5e91eaad907766525107523b0daea3df267a9fcb5ba4e0936ae"},
+ {file = "msgspec-0.21.1-cp310-cp310-win_amd64.whl", hash = "sha256:f041a2279f31e3a53319005e4d60ba77c085cfcbe394cdc7ce803c2d01fe9449"},
+ {file = "msgspec-0.21.1-cp310-cp310-win_arm64.whl", hash = "sha256:1bf17cbd7b28a5dffc7e764c654eed8ccde5e0f1de7970628608304640d4ce4e"},
+ {file = "msgspec-0.21.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:b504b6e7f7a22a24b27232b73034421692147865162daaec9f3bf62439007c87"},
+ {file = "msgspec-0.21.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4692b7c1609155708c4418f88e92f63c13fdf08aa095c84bae82bad75b53389b"},
+ {file = "msgspec-0.21.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d3124010b3815451494c85ff345e693cb9fe5889cfcbbef39ed8622e0e72319c"},
+ {file = "msgspec-0.21.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6badc03b9725352219cca017bfe71c61f2fbd0fb5982b410ac17c97c213deb30"},
+ {file = "msgspec-0.21.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:5d2d4116ebe3035a78d9ec76e99a9d64e5fa6d44fe61a9c5de7fd1acf54bcc69"},
+ {file = "msgspec-0.21.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:0d1009f6715f5bff3b54d4ff5c7428ad96197e0534e1645b8e9b955890c84664"},
+ {file = "msgspec-0.21.1-cp311-cp311-win_amd64.whl", hash = "sha256:c6faffe5bb644ec884052679af4dfd776d4b5ca90e4a7ec7e7e319e4e6b93a6e"},
+ {file = "msgspec-0.21.1-cp311-cp311-win_arm64.whl", hash = "sha256:ee9e3f11fa94603f7d673bf795cfa31b549c4a2c723bc39b45beb1e7f5a3fb99"},
+ {file = "msgspec-0.21.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d4248cf0b6129b7d230eacd493c17cc2d4f3989f3bb7f633a928a85b7dcfa251"},
+ {file = "msgspec-0.21.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:5102c7e9b3acff82178449b85006d96310e690291bb1ea0142f1b24bcb8aabcb"},
+ {file = "msgspec-0.21.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:846758412e9518252b2ac9bffd6f0e54d9ff614f5f9488df7749f81ff5c80920"},
+ {file = "msgspec-0.21.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:21995e74b5c598c2e004110ad66ec7f1b8c20bf2bcf3b2de8fd9a3094422d3ff"},
+ {file = "msgspec-0.21.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:6129f0cca52992e898fd5344187f7c8127b63d810b2fd73e36fca73b4c6475ee"},
+ {file = "msgspec-0.21.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:ef3ec2296248d1f8b9231acb051b6d471dfde8f21819e86c9adaaa9f42918521"},
+ {file = "msgspec-0.21.1-cp312-cp312-win_amd64.whl", hash = "sha256:d4ab834a054c6f0cbeef6df9e7e1b33d5f1bc7b86dea1d2fd7cad003873e783d"},
+ {file = "msgspec-0.21.1-cp312-cp312-win_arm64.whl", hash = "sha256:628aaa35c74950a8c59da330d7e98917e1c7188f983745782027748ee4ca573e"},
+ {file = "msgspec-0.21.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:764173717a01743f007e9f74520ed281f24672c604514f7d76c1c3a10e8edb66"},
+ {file = "msgspec-0.21.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:344c7cd0eaed1fb81d7959f99100ef71ec9b536881a376f11b9a6c4803365697"},
+ {file = "msgspec-0.21.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:48943e278b3854c2f89f955ddc6f9f430d3f0784b16e47d10604ee0463cd21f5"},
+ {file = "msgspec-0.21.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a9aa659ebb0101b1cbc31461212b87e341d961f0ab0772aaf068a99e001ec4aa"},
+ {file = "msgspec-0.21.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7b27d1a8ead2b6f5b0c4f2d07b8be1ccfcc041c8a0e704781edebe3ae13c484"},
+ {file = "msgspec-0.21.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:38fe93e86b61328fe544cb7fd871fad5a27c8734bfda90f65e5dbe288ae50f61"},
+ {file = "msgspec-0.21.1-cp313-cp313-win_amd64.whl", hash = "sha256:8bc666331c35fcce05a7cd2d6221adbe0f6058f8e750711413d22793c080ac6a"},
+ {file = "msgspec-0.21.1-cp313-cp313-win_arm64.whl", hash = "sha256:42bb1241e0750c1a4346f2aa84db26c5ffd99a4eb3a954927d9f149ff2f42898"},
+ {file = "msgspec-0.21.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:fab48eb45fdbfbdb2c0edfec00ffc53b6b6085beefc6b50b61e01659f9f8757f"},
+ {file = "msgspec-0.21.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:3cb779ea0c35bc807ff941d415875c1f69ca0be91a2e907ab99a171811d86a9a"},
+ {file = "msgspec-0.21.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:68604db36b3b4dd9bf160e436e12798a4738848144cea1aca1cb984011eb160f"},
+ {file = "msgspec-0.21.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3d6b9dc50948eaf65df54d2fd0ff66e6d8c32f116037209ee861810eb9b676cb"},
+ {file = "msgspec-0.21.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:52c5e21930942302394429c5a582ce7e6b62c7f983b3760834c2ce107e0dd6df"},
+ {file = "msgspec-0.21.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:abbb39d65681fa24ed394e01af3d59d869068324f900c61d06062b7fb9980f2f"},
+ {file = "msgspec-0.21.1-cp314-cp314-win_amd64.whl", hash = "sha256:5666b1b560b97b6ec2eb3fca8a502298ebac56e13bbca1f88523538ce83d01ea"},
+ {file = "msgspec-0.21.1-cp314-cp314-win_arm64.whl", hash = "sha256:d8b8578e4c83b14ceea4cef0d0b747e31d9330fe4b03b2b2ad4063866a178f93"},
+ {file = "msgspec-0.21.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:15f523d51c00ebad412213bfe9f06f0a50ec2b93e0c19e824a2d267cabb48ea2"},
+ {file = "msgspec-0.21.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:4e47390360583ba3d5c6cb44cf0a9f61b0a06a899d3c2c00627cedebb2e2884b"},
+ {file = "msgspec-0.21.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f60800e6299b798142dc40b0644da77ceac5ea0568be58228417eae14135c847"},
+ {file = "msgspec-0.21.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5f8e9dfcd98419cf7568808470c4317a3fb30bef0e3715b568730a2b272a20d7"},
+ {file = "msgspec-0.21.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:92d89dfad13bd1ea640dc3e37e724ed380da1030b272bdf5ecafb983c3ad7c75"},
+ {file = "msgspec-0.21.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:0d03867786e5d7ba25d666df4b11320c27170f4aeafcb8e3a8b0a50a4fb742ca"},
+ {file = "msgspec-0.21.1-cp314-cp314t-win_amd64.whl", hash = "sha256:740fbf1c9d59992ca3537d6fbe9ebbf9eaf726a65fbf31448e0ecbc710697a63"},
+ {file = "msgspec-0.21.1-cp314-cp314t-win_arm64.whl", hash = "sha256:0d2cc73df6058d811a126ac3a8ad63a4dfa210c82f9cf5a004802eaf4712de90"},
+ {file = "msgspec-0.21.1.tar.gz", hash = "sha256:2313508e394b0d208f8f56892ca9b2799e2561329de9763b19619595a6c0f72c"},
+]
+
+[package.extras]
+toml = ["tomli ; python_version < \"3.11\"", "tomli_w"]
+yaml = ["pyyaml"]
+
[[package]]
name = "multidict"
version = "6.7.1"
@@ -7557,6 +7667,18 @@ files = [
{file = "trove_classifiers-2026.1.14.14.tar.gz", hash = "sha256:00492545a1402b09d4858605ba190ea33243d361e2b01c9c296ce06b5c3325f3"},
]
+[[package]]
+name = "truststore"
+version = "0.10.4"
+description = "Verify certificates using native system trust stores"
+optional = false
+python-versions = ">=3.10"
+groups = ["main"]
+files = [
+ {file = "truststore-0.10.4-py3-none-any.whl", hash = "sha256:adaeaecf1cbb5f4de3b1959b42d41f6fab57b2b1666adb59e89cb0b53361d981"},
+ {file = "truststore-0.10.4.tar.gz", hash = "sha256:9d91bd436463ad5e4ee4aba766628dd6cd7010cf3e2461756b3303710eebc301"},
+]
+
[[package]]
name = "tweepy"
version = "4.16.0"
@@ -7579,6 +7701,24 @@ async = ["aiohttp (>=3.7.3,<4)", "async-lru (>=1.0.3,<3)"]
dev = ["coverage (>=4.4.2)", "coveralls (>=2.1.0)", "tox (>=3.21.0)"]
test = ["urllib3 (<2)", "vcrpy (>=1.10.3)"]
+[[package]]
+name = "typesafe-sdk"
+version = "0.6.0"
+description = "Python SDK for TypeSafe AI API."
+optional = false
+python-versions = ">=3.10"
+groups = ["main"]
+files = [
+ {file = "typesafe_sdk-0.6.0-py3-none-any.whl", hash = "sha256:21559d5558e95a7ee008d0a044a60bb5bb2d3b1ec426fe158a7f37af35ba3232"},
+ {file = "typesafe_sdk-0.6.0.tar.gz", hash = "sha256:78540639bde10a56d378f35690264799c00d1ffd7c0fa56b27838d0aa1a4bb34"},
+]
+
+[package.dependencies]
+httpx2 = ">=2.0.0"
+msgspec = ">=0.21.1"
+tenacity = ">=9.0.0"
+typing-extensions = ">=4.13.0"
+
[[package]]
name = "typing-extensions"
version = "4.15.0"
@@ -8761,4 +8901,4 @@ cffi = ["cffi (>=1.17,<2.0) ; platform_python_implementation != \"PyPy\" and pyt
[metadata]
lock-version = "2.1"
python-versions = ">=3.10,<3.14"
-content-hash = "79c0fee02fecaddc1610ffbe81c5a45170e7986b93a44e75f293af0cb7409448"
+content-hash = "0186546e1c40c78c66ecd8780f1ea23b3031ca4b3c7a646edd276e6ddb208bc5"
diff --git a/autogpt_platform/backend/pyproject.toml b/autogpt_platform/backend/pyproject.toml
index 029218fb2c..4ce5ded50d 100644
--- a/autogpt_platform/backend/pyproject.toml
+++ b/autogpt_platform/backend/pyproject.toml
@@ -78,6 +78,7 @@ stripe = "^11.5.0"
tenacity = "^9.1.4"
todoist-api-python = "^2.1.7"
tweepy = "^4.16.0"
+typesafe-sdk = "0.6.0"
uvicorn = { extras = ["standard"], version = "^0.40.0" }
websockets = "^15.0"
youtube-transcript-api = "^1.2.1"
diff --git a/docs/integrations/README.md b/docs/integrations/README.md
index 550e6df32a..3ed42eea6d 100644
--- a/docs/integrations/README.md
+++ b/docs/integrations/README.md
@@ -261,6 +261,13 @@ Below is a comprehensive list of all available blocks, categorized by their prim
| [Exa Create Enrichment](block-integrations/exa/websets_enrichment.md#exa-create-enrichment) | Create enrichments to extract additional structured data from webset items |
| [Exa Create Research](block-integrations/exa/research.md#exa-create-research) | Create research task with optional waiting - explores web and synthesizes findings with citations |
| [Ideogram Model](block-integrations/llm.md#ideogram-model) | This block runs Ideogram models with both simple and advanced settings |
+| [Jev Ask Many](block-integrations/typesafe/ask_many.md#jev-ask-many) | Ask multiple Choice, Score, or Noul questions of one shared state with Jev in a single call |
+| [Jev Choice](block-integrations/typesafe/choice.md#jev-choice) | Make a typed choice with Jev |
+| [Jev Filter](block-integrations/typesafe/filter.md#jev-filter) | Filter items with sequential Jev scores, one call per item |
+| [Jev Pick Best](block-integrations/typesafe/pick_best.md#jev-pick-best) | Choose the best candidate using Jev and rank candidates directly by its probabilities |
+| [Jev Route](block-integrations/typesafe/route.md#jev-route) | Route data using Jev's typed choice |
+| [Jev Score](block-integrations/typesafe/score.md#jev-score) | Score evidence with Jev using an explicit ordered scale |
+| [Jev Yes No](block-integrations/typesafe/yes_no.md#jev-yes-no) | Ask Jev a plain-language yes/no question and forward data to the chosen pin, or unsure below your confidence threshold |
| [Jina Chunking](block-integrations/jina/chunking.md#jina-chunking) | Chunks texts using Jina AI's segmentation service |
| [Jina Embedding](block-integrations/jina/embeddings.md#jina-embedding) | Generates embeddings using Jina AI |
| [Orchestrator](block-integrations/llm.md#orchestrator) | Uses AI to intelligently decide what tool to use |
diff --git a/docs/integrations/SUMMARY.md b/docs/integrations/SUMMARY.md
index 23a4198556..3277509326 100644
--- a/docs/integrations/SUMMARY.md
+++ b/docs/integrations/SUMMARY.md
@@ -172,6 +172,13 @@
* [Twitter Timeline](block-integrations/twitter/timeline.md)
* [Twitter Tweet Lookup](block-integrations/twitter/tweet_lookup.md)
* [Twitter User Lookup](block-integrations/twitter/user_lookup.md)
+* [Typesafe Ask Many](block-integrations/typesafe/ask_many.md)
+* [Typesafe Choice](block-integrations/typesafe/choice.md)
+* [Typesafe Filter](block-integrations/typesafe/filter.md)
+* [Typesafe Pick Best](block-integrations/typesafe/pick_best.md)
+* [Typesafe Route](block-integrations/typesafe/route.md)
+* [Typesafe Score](block-integrations/typesafe/score.md)
+* [Typesafe Yes No](block-integrations/typesafe/yes_no.md)
* [Video Add Audio](block-integrations/video/add_audio.md)
* [Video Clip](block-integrations/video/clip.md)
* [Video Concat](block-integrations/video/concat.md)
diff --git a/docs/integrations/block-integrations/typesafe/ask_many.md b/docs/integrations/block-integrations/typesafe/ask_many.md
new file mode 100644
index 0000000000..6c30417f17
--- /dev/null
+++ b/docs/integrations/block-integrations/typesafe/ask_many.md
@@ -0,0 +1,49 @@
+# Typesafe Ask Many
+<!-- MANUAL: file_description -->
+Ask several typed questions about one shared state with TypeSafe Jev.
+<!-- END MANUAL -->
+
+## Jev Ask Many
+
+### What it is
+Ask multiple Choice, Score, or Noul questions of one shared state with Jev in a single call. Levels define the scale: describe concrete evidence for each tier so a very good case does not reach the top. For hiring, distinguish terrible, bad, okay, excellent, brilliant, and one-in-a-thousand by what qualifies.
+
+### How it works
+<!-- MANUAL: how_it_works -->
+The block submits every entry in `questions` in one API request with the same state. Each named entry selects Choice (`question` and `options`), Score (`question` and ordered `levels`), or Noul (`question` and optional `true`/`false` criteria). The `answers` output preserves the question keys and returns each typed answer as a plain dictionary. Noul returns a scalar value without invented confidence or probability fields.
+
+At least one question is required. Describe concrete evidence for every Score tier, reserving the highest tier for exceptional evidence. Questions share the request budget; oversized state is truncated with an explicit note. The transparency outputs record the actual request and response, and a failed call emits an error instead of answers.
+<!-- END MANUAL -->
+
+### Inputs
+
+| Input | Description | Type | Required |
+|-------|-------------|------|----------|
+| state | Text or JSON context shared by every question in this stateless call. JSON is compactly serialized; oversized context is truncated with a note. | State | No |
+| questions | Named typed judgments sharing one state and one API call: choice uses question/options, score uses question/levels, noul uses question and optional true/false criteria. Levels define the scale: describe concrete evidence for each tier so a very good case does not reach the top. For hiring, distinguish terrible, bad, okay, excellent, brilliant, and one-in-a-thousand by what qualifies. | Dict[str, Any] | Yes |
+
+### Outputs
+
+| Output | Description | Type |
+|--------|-------------|------|
+| error | API failure details; emitted only when a call fails. | str |
+| request | Verbatim JSON request body sent to Jev. | str |
+| response | Verbatim UTF-8 response body from Jev, or null if no response arrived. Invalid UTF-8 is preserved as a lossless Base64 data URL with an error. | str |
+| latency_ms | Wall-clock API call duration in ms. | float |
+| input_tokens | Input tokens reported by the API, or null if unavailable on failure. | int |
+| output_tokens | Output tokens reported by the API, or null if unavailable on failure. | int |
+| request_id | TypeSafe request ID response header. | str |
+| truncated | Whether any input text was truncated. | bool |
+| truncation_note | Truncation details, or an empty string. | str |
+| answers | Typed answers as plain dictionaries, keyed by question name. | Dict[str, Dict[str, Any]] |
+
+### Possible use case
+<!-- MANUAL: use_case -->
+**Resume Review**: Evaluate production experience and a calibrated role-fit score together before routing the ratings to a reviewer.
+
+**Support Triage**: Classify the owning team, score urgency, and check whether a refund was requested in one call.
+
+**Document Screening**: Ask several independent evidence questions about the same extracted document text.
+<!-- END MANUAL -->
+
+---
diff --git a/docs/integrations/block-integrations/typesafe/choice.md b/docs/integrations/block-integrations/typesafe/choice.md
new file mode 100644
index 0000000000..e86068f415
--- /dev/null
+++ b/docs/integrations/block-integrations/typesafe/choice.md
@@ -0,0 +1,52 @@
+# Typesafe Choice
+<!-- MANUAL: file_description -->
+Choose among explicitly described options using TypeSafe Jev.
+<!-- END MANUAL -->
+
+## Jev Choice
+
+### What it is
+Make a typed choice with Jev. Describe what qualifies for each option in plain language.
+
+### How it works
+<!-- MANUAL: how_it_works -->
+Provide a question, the relevant state, and at least two named options. Each option description explains what qualifies for that answer. Jev selects one configured name and returns its probabilities and confidence; the block forwards those values without changing the model's judgment.
+
+Each call is stateless, so include all needed context in `state`. Structured state becomes compact JSON text, and oversized state is truncated with a visible flag and note. Request and response bodies are exposed alongside timing and token usage. Failed calls emit available transparency and an error without a choice.
+<!-- END MANUAL -->
+
+### Inputs
+
+| Input | Description | Type | Required |
+|-------|-------------|------|----------|
+| state | Text or JSON context shared by every question in this stateless call. JSON is compactly serialized; oversized context is truncated with a note. | State | No |
+| question | Plain-language judgment to make. | str | Yes |
+| options | Option names mapped to concrete descriptions of what qualifies. | Dict[str, str] | Yes |
+
+### Outputs
+
+| Output | Description | Type |
+|--------|-------------|------|
+| error | API failure details; emitted only when a call fails. | str |
+| request | Verbatim JSON request body sent to Jev. | str |
+| response | Verbatim UTF-8 response body from Jev, or null if no response arrived. Invalid UTF-8 is preserved as a lossless Base64 data URL with an error. | str |
+| latency_ms | Wall-clock API call duration in ms. | float |
+| input_tokens | Input tokens reported by the API, or null if unavailable on failure. | int |
+| output_tokens | Output tokens reported by the API, or null if unavailable on failure. | int |
+| request_id | TypeSafe request ID response header. | str |
+| truncated | Whether any input text was truncated. | bool |
+| truncation_note | Truncation details, or an empty string. | str |
+| choice | Winning option name returned by Jev. | str |
+| probabilities | Probability for each option. | Dict[str, float] |
+| confidence | Confidence returned by Jev. | float |
+
+### Possible use case
+<!-- MANUAL: use_case -->
+**Support Classification**: Choose the team responsible for a ticket from clear ownership descriptions.
+
+**Content Tagging**: Assign a document to a controlled set of topics using the supplied definitions.
+
+**Workflow Selection**: Choose which named processing strategy best fits an incoming record.
+<!-- END MANUAL -->
+
+---
diff --git a/docs/integrations/block-integrations/typesafe/filter.md b/docs/integrations/block-integrations/typesafe/filter.md
new file mode 100644
index 0000000000..7ba042a1ec
--- /dev/null
+++ b/docs/integrations/block-integrations/typesafe/filter.md
@@ -0,0 +1,56 @@
+# Typesafe Filter
+<!-- MANUAL: file_description -->
+Filter a bounded list using item-by-item Jev scores and an explicit threshold.
+<!-- END MANUAL -->
+
+## Jev Filter
+
+### What it is
+Filter items with sequential Jev scores, one call per item. Each call receives JSON context and item and emits its own transparency outputs. Levels define the scale: describe concrete evidence for each tier so a very good case does not reach the top. For hiring, distinguish terrible, bad, okay, excellent, brilliant, and one-in-a-thousand by what qualifies.
+
+### How it works
+<!-- MANUAL: how_it_works -->
+The block scores items sequentially, making one API call per item with an item-first state containing `item` and `context`. Ordered `levels` define the scale, and a returned score at or above `min_score` places the original item in `passed`; lower scores place it in `rejected`. The final `scores` list stays aligned with the input order. Each attempted item emits its zero-based `item_index` followed by its transparency outputs.
+
+`max_items` defaults to 100 and accepts 1–1,000; larger input lists are rejected before any call. The threshold must fit the configured scale, and level descriptions should define concrete evidence for each tier. Empty input yields three empty lists without calling the API. A failed item stops processing and retains prior transcripts without emitting partial result lists.
+<!-- END MANUAL -->
+
+### Inputs
+
+| Input | Description | Type | Required |
+|-------|-------------|------|----------|
+| state | Text or JSON context shared by every question in this stateless call. JSON is compactly serialized; oversized context is truncated with a note. | State | No |
+| items | Items to score, one sequential API call per item; oversized lists are rejected. | List[Any] | Yes |
+| question | Plain-language question to score for each item, using state as context. | str | Yes |
+| levels | Ordered level descriptions, lowest first. Levels define the scale: describe concrete evidence for each tier so a very good case does not reach the top. For hiring, distinguish terrible, bad, okay, excellent, brilliant, and one-in-a-thousand by what qualifies. | List[str] | Yes |
+| min_score | Keep items with returned score at or above this value. | float | Yes |
+| max_items | Maximum number of items allowed in this execution. | int | No |
+
+### Outputs
+
+| Output | Description | Type |
+|--------|-------------|------|
+| error | API failure details; emitted only when a call fails. | str |
+| request | Verbatim JSON request body sent to Jev. | str |
+| response | Verbatim UTF-8 response body from Jev, or null if no response arrived. Invalid UTF-8 is preserved as a lossless Base64 data URL with an error. | str |
+| latency_ms | Wall-clock API call duration in ms. | float |
+| input_tokens | Input tokens reported by the API, or null if unavailable on failure. | int |
+| output_tokens | Output tokens reported by the API, or null if unavailable on failure. | int |
+| request_id | TypeSafe request ID response header. | str |
+| truncated | Whether any input text was truncated. | bool |
+| truncation_note | Truncation details, or an empty string. | str |
+| item_index | Zero-based item index emitted before that item's transparency outputs. | int |
+| passed | Items meeting min_score, in original order. | List[Any] |
+| rejected | Items below min_score, in original order. | List[Any] |
+| scores | Returned scores aligned with the complete input items list. | List[float] |
+
+### Possible use case
+<!-- MANUAL: use_case -->
+**Evidence Screening**: Keep extracted passages that meet a defined relevance threshold for a research question.
+
+**Ticket Prioritization**: Separate reports that meet a concrete impact rubric from lower-impact reports.
+
+**Catalog Review**: Select product records with sufficiently complete specifications for a downstream workflow.
+<!-- END MANUAL -->
+
+---
diff --git a/docs/integrations/block-integrations/typesafe/pick_best.md b/docs/integrations/block-integrations/typesafe/pick_best.md
new file mode 100644
index 0000000000..9a5e3d2951
--- /dev/null
+++ b/docs/integrations/block-integrations/typesafe/pick_best.md
@@ -0,0 +1,53 @@
+# Typesafe Pick Best
+<!-- MANUAL: file_description -->
+Select an original candidate and rank alternatives using Jev's returned probabilities.
+<!-- END MANUAL -->
+
+## Jev Pick Best
+
+### What it is
+Choose the best candidate using Jev and rank candidates directly by its probabilities. State the comparison criteria in plain language.
+
+### How it works
+<!-- MANUAL: how_it_works -->
+The block turns at least two candidates into Choice options named `candidate_1`, `candidate_2`, and so on. Candidate descriptions are compact JSON text capped at 2,000 characters each; any truncation is reported. Jev evaluates those options against the question and state, and the winning label maps back to the unchanged original candidate and its zero-based `best_index`.
+
+`ranked` contains the original candidates ordered directly by descending returned probability, retaining input order for ties. It does not adjust the model's answers. The shared transparency outputs show the exact candidate descriptions that were sent. A failed call emits an error without selecting or ranking candidates.
+<!-- END MANUAL -->
+
+### Inputs
+
+| Input | Description | Type | Required |
+|-------|-------------|------|----------|
+| state | Text or JSON context shared by every question in this stateless call. JSON is compactly serialized; oversized context is truncated with a note. | State | No |
+| candidates | Candidates to compare; descriptions are compact JSON capped at 2,000 characters each. | List[Any] | Yes |
+| question | Plain-language comparison, for example: Which candidate best meets the requirements in the state? | str | Yes |
+
+### Outputs
+
+| Output | Description | Type |
+|--------|-------------|------|
+| error | API failure details; emitted only when a call fails. | str |
+| request | Verbatim JSON request body sent to Jev. | str |
+| response | Verbatim UTF-8 response body from Jev, or null if no response arrived. Invalid UTF-8 is preserved as a lossless Base64 data URL with an error. | str |
+| latency_ms | Wall-clock API call duration in ms. | float |
+| input_tokens | Input tokens reported by the API, or null if unavailable on failure. | int |
+| output_tokens | Output tokens reported by the API, or null if unavailable on failure. | int |
+| request_id | TypeSafe request ID response header. | str |
+| truncated | Whether any input text was truncated. | bool |
+| truncation_note | Truncation details, or an empty string. | str |
+| best | Original candidate selected by Jev. | Best |
+| best_index | Zero-based index of the selected candidate. | int |
+| ranked | Original candidates sorted by returned probability, highest first; ties retain input order. | List[Any] |
+| probabilities | Probabilities keyed by candidate_1, candidate_2, and so on. | Dict[str, float] |
+
+### Possible use case
+<!-- MANUAL: use_case -->
+**Response Selection**: Choose which prepared support reply best addresses the customer's stated problem.
+
+**Search Result Selection**: Rank retrieved passages by how well they support a specific question.
+
+**Action Comparison**: Select among proposed workflow actions using requirements supplied in the state.
+<!-- END MANUAL -->
+
+---
diff --git a/docs/integrations/block-integrations/typesafe/route.md b/docs/integrations/block-integrations/typesafe/route.md
new file mode 100644
index 0000000000..0343f22181
--- /dev/null
+++ b/docs/integrations/block-integrations/typesafe/route.md
@@ -0,0 +1,57 @@
+# Typesafe Route
+<!-- MANUAL: file_description -->
+Route unchanged data to one of five fixed output pins using a Jev choice.
+<!-- END MANUAL -->
+
+## Jev Route
+
+### What it is
+Route data using Jev's typed choice. Plain-language option descriptions define each route; only the winning pin fires.
+
+### How it works
+<!-- MANUAL: how_it_works -->
+Configure two to five options in the desired order. The first option corresponds to `option_1`, the second to `option_2`, and so on. Jev evaluates the question against the state; only the winning numbered pin emits the original `data`. The block also emits the selected option name, probabilities, and common request transparency.
+
+Option descriptions define the decision criteria in plain language. The numbered pins are fixed even when fewer than five options are configured, and unused pins do not emit data. Calls are stateless and report any truncation of the supplied state. An API failure emits an error without firing a routing pin.
+<!-- END MANUAL -->
+
+### Inputs
+
+| Input | Description | Type | Required |
+|-------|-------------|------|----------|
+| state | Text or JSON context shared by every question in this stateless call. JSON is compactly serialized; oversized context is truncated with a note. | State | No |
+| question | Plain-language judgment deciding the route. | str | Yes |
+| options | Two to five option names and qualifying descriptions; dictionary order assigns option_1 to option_5. | Dict[str, str] | Yes |
+| data | Value forwarded unchanged on the winning route only. | Data | No |
+
+### Outputs
+
+| Output | Description | Type |
+|--------|-------------|------|
+| error | API failure details; emitted only when a call fails. | str |
+| request | Verbatim JSON request body sent to Jev. | str |
+| response | Verbatim UTF-8 response body from Jev, or null if no response arrived. Invalid UTF-8 is preserved as a lossless Base64 data URL with an error. | str |
+| latency_ms | Wall-clock API call duration in ms. | float |
+| input_tokens | Input tokens reported by the API, or null if unavailable on failure. | int |
+| output_tokens | Output tokens reported by the API, or null if unavailable on failure. | int |
+| request_id | TypeSafe request ID response header. | str |
+| truncated | Whether any input text was truncated. | bool |
+| truncation_note | Truncation details, or an empty string. | str |
+| option_1 | Data when the first configured option wins. | Option 1 |
+| option_2 | Data when the second configured option wins. | Option 2 |
+| option_3 | Data when the third configured option wins. | Option 3 |
+| option_4 | Data when the fourth configured option wins. | Option 4 |
+| option_5 | Data when the fifth configured option wins. | Option 5 |
+| choice | Winning option name returned by Jev. | str |
+| probabilities | Probability for each configured option. | Dict[str, float] |
+
+### Possible use case
+<!-- MANUAL: use_case -->
+**Team Routing**: Forward a support ticket to the branch corresponding to the responsible team.
+
+**Resume Review Routing**: Send high-rated resume evaluations toward an email review branch and the remaining evaluations toward a sheet.
+
+**Document Processing**: Send a classified document to its configured extraction or review workflow.
+<!-- END MANUAL -->
+
+---
diff --git a/docs/integrations/block-integrations/typesafe/score.md b/docs/integrations/block-integrations/typesafe/score.md
new file mode 100644
index 0000000000..edf4902981
--- /dev/null
+++ b/docs/integrations/block-integrations/typesafe/score.md
@@ -0,0 +1,54 @@
+# Typesafe Score
+<!-- MANUAL: file_description -->
+Score evidence on a concrete, ordered scale with TypeSafe Jev.
+<!-- END MANUAL -->
+
+## Jev Score
+
+### What it is
+Score evidence with Jev using an explicit ordered scale. Levels define the scale: describe concrete evidence for each tier so a very good case does not reach the top. For hiring, distinguish terrible, bad, okay, excellent, brilliant, and one-in-a-thousand by what qualifies.
+
+### How it works
+<!-- MANUAL: how_it_works -->
+Provide a question and at least two level descriptions, ordered from lowest to highest. Jev returns a potentially fractional score, the level legend, probabilities, and confidence. The block emits these values directly and computes only `max_score` as `len(levels) - 1`.
+
+The level text defines what earns each tier. For example, distinguish delivering a tested feature, independently operating production services, and leading measurable improvements across teams; reserve the top tier for evidence beyond ordinary excellent performance. Each call exposes its actual bodies, usage, timing, and truncation status. A failed call emits an error without a score.
+<!-- END MANUAL -->
+
+### Inputs
+
+| Input | Description | Type | Required |
+|-------|-------------|------|----------|
+| state | Text or JSON context shared by every question in this stateless call. JSON is compactly serialized; oversized context is truncated with a note. | State | No |
+| question | Plain-language question to score. | str | Yes |
+| levels | Ordered level descriptions, lowest first. Levels define the scale: describe concrete evidence for each tier so a very good case does not reach the top. For hiring, distinguish terrible, bad, okay, excellent, brilliant, and one-in-a-thousand by what qualifies. | List[str] | Yes |
+
+### Outputs
+
+| Output | Description | Type |
+|--------|-------------|------|
+| error | API failure details; emitted only when a call fails. | str |
+| request | Verbatim JSON request body sent to Jev. | str |
+| response | Verbatim UTF-8 response body from Jev, or null if no response arrived. Invalid UTF-8 is preserved as a lossless Base64 data URL with an error. | str |
+| latency_ms | Wall-clock API call duration in ms. | float |
+| input_tokens | Input tokens reported by the API, or null if unavailable on failure. | int |
+| output_tokens | Output tokens reported by the API, or null if unavailable on failure. | int |
+| request_id | TypeSafe request ID response header. | str |
+| truncated | Whether any input text was truncated. | bool |
+| truncation_note | Truncation details, or an empty string. | str |
+| score | Score returned by Jev on the 0..max_score scale. | float |
+| legend | Level numbers mapped to their descriptions. | Dict[str, str] |
+| probabilities | Probability for each score level. | Dict[str, float] |
+| confidence | Confidence returned by Jev. | float |
+| max_score | Highest possible score: number of levels minus one. | int |
+
+### Possible use case
+<!-- MANUAL: use_case -->
+**Role-Fit Review**: Rate documented experience against concrete hiring tiers before a human reviews the result.
+
+**Incident Impact**: Score an incident using levels tied to the number of users or workflows affected.
+
+**Specification Completeness**: Evaluate whether a document provides the evidence required by each completeness tier.
+<!-- END MANUAL -->
+
+---
diff --git a/docs/integrations/block-integrations/typesafe/yes_no.md b/docs/integrations/block-integrations/typesafe/yes_no.md
new file mode 100644
index 0000000000..149555e585
--- /dev/null
+++ b/docs/integrations/block-integrations/typesafe/yes_no.md
@@ -0,0 +1,54 @@
+# Typesafe Yes No
+<!-- MANUAL: file_description -->
+Make a Jev yes/no judgment with an optional confidence threshold for an unsure branch.
+<!-- END MANUAL -->
+
+## Jev Yes No
+
+### What it is
+Ask Jev a plain-language yes/no question and forward data to the chosen pin, or unsure below your confidence threshold.
+
+### How it works
+<!-- MANUAL: how_it_works -->
+The block asks Jev a Choice question with fixed `yes` and `no` options. It emits the returned confidence and forwards the original `data` on the chosen pin when that confidence meets `min_confidence`. If confidence is below the threshold, only `unsure` receives the data instead. The default threshold is 0.0 and valid thresholds range from 0.0 to 1.0.
+
+Supply the evidence needed for the judgment in the stateless `state` input. The confidence threshold determines the output branch without changing Jev's answer or confidence. Shared outputs expose the exact request, response, usage, timing, and truncation details. Failed calls emit an error without firing `yes`, `no`, or `unsure`.
+<!-- END MANUAL -->
+
+### Inputs
+
+| Input | Description | Type | Required |
+|-------|-------------|------|----------|
+| state | Text or JSON context shared by every question in this stateless call. JSON is compactly serialized; oversized context is truncated with a note. | State | No |
+| question | Plain-language question answerable yes or no. | str | Yes |
+| data | Value forwarded unchanged on yes, no, or unsure. | Data | No |
+| min_confidence | Emit unsure when Jev's confidence is below this threshold. | float | No |
+
+### Outputs
+
+| Output | Description | Type |
+|--------|-------------|------|
+| error | API failure details; emitted only when a call fails. | str |
+| request | Verbatim JSON request body sent to Jev. | str |
+| response | Verbatim UTF-8 response body from Jev, or null if no response arrived. Invalid UTF-8 is preserved as a lossless Base64 data URL with an error. | str |
+| latency_ms | Wall-clock API call duration in ms. | float |
+| input_tokens | Input tokens reported by the API, or null if unavailable on failure. | int |
+| output_tokens | Output tokens reported by the API, or null if unavailable on failure. | int |
+| request_id | TypeSafe request ID response header. | str |
+| truncated | Whether any input text was truncated. | bool |
+| truncation_note | Truncation details, or an empty string. | str |
+| yes | Data when Jev chooses yes and meets the confidence threshold. | Yes |
+| no | Data when Jev chooses no and meets the confidence threshold. | No |
+| unsure | Data when confidence is below min_confidence. | Unsure |
+| confidence | Confidence returned by Jev. | float |
+
+### Possible use case
+<!-- MANUAL: use_case -->
+**Human Review Gate**: Forward uncertain judgments to a reviewer while routing confident yes/no decisions separately.
+
+**Request Detection**: Decide whether a support message explicitly requests a particular action.
+
+**Evidence Check**: Test whether a document supplies the evidence for a required criterion.
+<!-- END MANUAL -->
+
+---