Skip to content
Python

feat(sdk): preserve Gemini signatures across streamed tool conversations

Rohit P6c40f67ad656
diff --git a/docs/harness/guides/providers.md b/docs/harness/guides/providers.md
index 051d2a79..22839167 100644
--- a/docs/harness/guides/providers.md
+++ b/docs/harness/guides/providers.md
@@ -8,6 +8,7 @@ The session saves its resolved model. Unavailable models fail explicitly.
|---|---|---|
| OpenAI | Responses | `name: "OpenAI", api: "responses"` |
| Anthropic | Native Messages | `name: "Anthropic", api: "anthropic"` |
+| Google Gemini | Google OpenAI-compatible Chat Completions | Google's `/v1beta/openai` base URL and a Gemini API key |
| OpenRouter or a compatible proxy | Chat Completions | Custom `baseUrl` and credentials |
| Ollama / vLLM | Chat Completions | Local base URL and an installed model ID |
@@ -101,3 +102,49 @@ Anthropic Messages. There is no silent fallback or automatic request replay.
include both. Actual hit rates and cost depend on model, prefix stability,
minimum prompt size and provider pricing. See
[Anthropic's prompt caching guide](https://platform.claude.com/docs/en/build-with-claude/prompt-caching).
+
+## Gemini tool conversations
+
+Register Google's compatible endpoint and choose a model available to your
+Google account. The SDK manages streamed tool calls and their follow-up requests:
+
+```typescript
+await harness.setModelProvider({
+ name: "Gemini",
+ api: "chat-completions",
+ baseUrl: "https://generativelanguage.googleapis.com/v1beta/openai",
+ credentials: { apiKeyEnv: "GEMINI_API_KEY", storage: "memory" },
+});
+const agent = harness.createAgent({
+ name: "assistant",
+ instructions: "Use the application's lookup tool to answer the user.",
+ tools: ["app.lookup"], // Register this application tool first.
+ defaultModel: { provider: "Gemini", name: "gemini-3.8-flash" },
+});
+```
+
+```rust,ignore
+harness.set_model_provider(
+ ModelProvider::chat_completions("Gemini")
+ .base_url("https://generativelanguage.googleapis.com/v1beta/openai")
+ .credentials(Credentials::from_env("GEMINI_API_KEY").memory_only()),
+).await?;
+```
+
+Google's opaque thought signatures are preserved in durable provider-private
+history and replayed unchanged on the corresponding tool call or assistant
+message. They survive host restart and credential rotation. They do not appear
+in application tool arguments, display history, reasoning text, or telemetry
+content—even when full-content telemetry is enabled.
+
+Private replay state is bound to the connection identity, API, endpoint and
+model. Sending it to another connection/model is rejected; the harness does not
+invent signatures or bypass Google's signature validation. Use a fresh session
+when changing providers. A provider-side proxy must preserve Google's
+`extra_content.google.thought_signature` fields for this flow to work.
+
+The compatible adapter accepts Google's complete tool calls without an `index`,
+as well as indexed argument fragments. It never guesses how to combine partial
+unindexed calls, and incomplete streams cannot authorize tool execution.
+See [Google's compatibility guide](https://ai.google.dev/gemini-api/docs/openai)
+and [thought signature requirements](https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures).
diff --git a/harness/harness-sdk/README.md b/harness/harness-sdk/README.md
index 80342498..030aa0ac 100644
--- a/harness/harness-sdk/README.md
+++ b/harness/harness-sdk/README.md
@@ -87,6 +87,11 @@ that rejects `stream_options`, add `.stream_usage(false)` to its `ModelProvider`
The setting persists across restart and key rotation. Missing usage is unknown,
not zero; see [streaming token usage](../../docs/harness/guides/providers.md#streaming-token-usage).
+Anthropic Messages enables automatic prompt caching by default. Set
+`.prompt_caching(false)` on its `ModelProvider` to opt out. Google Gemini uses
+the Chat Completions adapter with private thought-signature replay; see
+[models and credentials](../../docs/harness/guides/providers.md).
+
Windows uses DPAPI and macOS uses Keychain for secure credentials. On Linux,
choose `.memory_only()` explicitly or use server-managed credentials; there is no
plaintext fallback. Keys never belong in agent definitions. Re-register callback
diff --git a/harness/runtime/src/chat_completions.rs b/harness/runtime/src/chat_completions.rs
index eb941962..61aab61c 100644
--- a/harness/runtime/src/chat_completions.rs
+++ b/harness/runtime/src/chat_completions.rs
@@ -10,6 +10,45 @@ fn config() -> Error {
Error::new(Failure::Configuration)
}
+const GOOGLE_PRIVATE: &str = "google-chat-completions";
+const MAX_SIGNATURE_BYTES: usize = 256_000;
+
+/// Google signatures are opaque replay state, never user-visible reasoning.
+/// Scope them to the authenticated connection, transport, endpoint and model.
+pub fn replay_binding(endpoint: &str, model: &str, connection_identity: Option<&str>) -> String {
+ crate::bundle::digest(
+ json!(["chat-completions", endpoint, model, connection_identity])
+ .to_string()
+ .as_bytes(),
+ )
+}
+
+fn google_signature(value: &Value) -> Result<Option<String>, Error> {
+ let Some(value) = value
+ .get("extra_content")
+ .and_then(|v| v.get("google"))
+ .and_then(|v| v.get("thought_signature"))
+ else {
+ return Ok(None);
+ };
+ let signature = value.as_str().ok_or_else(invalid)?;
+ if signature.is_empty() || signature.len() > MAX_SIGNATURE_BYTES {
+ return Err(invalid());
+ }
+ Ok(Some(signature.to_owned()))
+}
+
+fn private_signature(binding: &str, target: &str, id: &str, signature: String) -> Value {
+ json!({"type":"reasoning","provider":GOOGLE_PRIVATE,"binding":binding,
+ "block":{"type":"thought_signature","target":target,"id":id,"signature":signature}})
+}
+
+pub fn message_signature(item: &Value) -> bool {
+ item["type"] == "reasoning"
+ && item["provider"] == GOOGLE_PRIVATE
+ && item["block"]["target"] == "message"
+}
+
fn content(value: &Value) -> Result<Value, Error> {
if value.is_string() {
return Ok(value.clone());
@@ -31,13 +70,59 @@ pub fn body(
tools: &[Value],
max_tokens: usize,
format: Option<&Value>,
+) -> Result<Value, Error> {
+ body_with_binding(
+ model,
+ instructions,
+ history,
+ tools,
+ max_tokens,
+ format,
+ None,
+ )
+}
+
+#[allow(clippy::too_many_arguments)]
+pub fn body_with_binding(
+ model: &str,
+ instructions: &str,
+ history: &[Value],
+ tools: &[Value],
+ max_tokens: usize,
+ format: Option<&Value>,
+ binding: Option<&str>,
) -> Result<Value, Error> {
let mut messages = vec![json!({"role":"system","content":instructions})];
let mut pending = BTreeSet::new();
let mut seen = BTreeSet::new();
+ let mut signatures = BTreeMap::new();
+ for item in history.iter().filter(|item| item["type"] == "reasoning") {
+ let block = &item["block"];
+ let target = block["target"].as_str().ok_or_else(config)?;
+ let id = block["id"]
+ .as_str()
+ .filter(|id| !id.is_empty())
+ .ok_or_else(config)?;
+ let signature = block["signature"]
+ .as_str()
+ .filter(|s| !s.is_empty() && s.len() <= MAX_SIGNATURE_BYTES)
+ .ok_or_else(config)?;
+ if item["provider"] != GOOGLE_PRIVATE
+ || binding.is_none()
+ || item["binding"].as_str() != binding
+ || block["type"] != "thought_signature"
+ || !["tool_call", "message"].contains(&target)
+ || signatures.insert((target, id), signature).is_some()
+ {
+ return Err(config());
+ }
+ }
for item in history {
+ if item["type"] == "reasoning" {
+ continue;
+ }
// Encrypted/signed reasoning must never cross to another API.
- if item.get("provider").is_some() || item["type"] == "reasoning" {
+ if item.get("provider").is_some() {
return Err(config());
}
match item["type"].as_str() {
@@ -54,7 +139,10 @@ pub fn body(
{
return Err(config());
}
- let call = json!({"id":id,"type":"function","function":{"name":item["name"].as_str().ok_or_else(config)?,"arguments":arguments}});
+ let mut call = json!({"id":id,"type":"function","function":{"name":item["name"].as_str().ok_or_else(config)?,"arguments":arguments}});
+ if let Some(signature) = signatures.remove(&("tool_call", id)) {
+ call["extra_content"] = json!({"google":{"thought_signature":signature}});
+ }
if messages.last().is_some_and(|m| m["role"] == "assistant") {
let last = messages.last_mut().unwrap();
if last.get("tool_calls").is_none() {
@@ -82,12 +170,22 @@ pub fn body(
if !["assistant", "user"].contains(&role) {
return Err(config());
}
- messages.push(json!({"role":role,"content":content(&item["content"])?}));
+ let mut message = json!({"role":role,"content":content(&item["content"])?});
+ if let Some(id) = item["id"].as_str() {
+ if let Some(signature) = signatures.remove(&("message", id)) {
+ if role != "assistant" {
+ return Err(config());
+ }
+ message["extra_content"] =
+ json!({"google":{"thought_signature":signature}});
+ }
+ }
+ messages.push(message);
}
_ => return Err(config()),
}
}
- if !pending.is_empty() {
+ if !pending.is_empty() || !signatures.is_empty() {
return Err(config());
}
let definitions = tools.iter().map(|tool| {
@@ -115,6 +213,9 @@ struct Call {
id: String,
name: String,
arguments: String,
+ signature: Option<String>,
+ /// Unindexed Gemini calls must be complete, self-contained objects.
+ complete: bool,
}
#[derive(Default)]
pub struct Decoder {
@@ -126,19 +227,20 @@ pub struct Decoder {
accounting_tail: bool,
usage: Option<Usage>,
bytes: usize,
+ binding: Option<String>,
+ google_compat: bool,
+ message_signature: Option<String>,
+ failed: bool,
}
-impl StreamDecoder for Decoder {
- fn usage_report(&self) -> Option<crate::telemetry::TokenUsage> {
- self.usage.as_ref().map(Usage::reported)
- }
- fn output_started(&self) -> bool {
- !self.text.is_empty()
- || self
- .calls
- .values()
- .any(|c| !c.name.is_empty() || !c.arguments.is_empty())
+impl Decoder {
+ pub fn with_binding(binding: String, google_compat: bool) -> Self {
+ Self {
+ binding: Some(binding),
+ google_compat,
+ ..Self::default()
+ }
}
- fn event(&mut self, event: &Value) -> Result<Option<String>, Error> {
+ fn accept(&mut self, event: &Value) -> Result<Option<String>, Error> {
if self.done || self.accounting_tail || event.get("error").is_some() {
return Err(invalid());
}
@@ -223,6 +325,20 @@ impl StreamDecoder for Decoder {
{
return Err(Error::new(Failure::SafetyBlocked));
}
+ if let Some(signature) = google_signature(delta)? {
+ if self.binding.is_none() {
+ return Err(config());
+ }
+ if self
+ .message_signature
+ .as_ref()
+ .is_some_and(|old| old != &signature)
+ {
+ return Err(invalid());
+ }
+ self.message_signature = Some(signature);
+ self.google_compat = true;
+ }
let text = match delta.get("content").filter(|v| !v.is_null()) {
Some(value) => {
let s = value.as_str().ok_or_else(invalid)?;
@@ -233,28 +349,75 @@ impl StreamDecoder for Decoder {
};
if let Some(calls) = delta.get("tool_calls") {
for c in calls.as_array().ok_or_else(invalid)? {
- let index = c["index"].as_u64().ok_or_else(invalid)?;
+ let signature = google_signature(c)?;
+ if signature.is_some() {
+ if self.binding.is_none() {
+ return Err(config());
+ }
+ self.google_compat = true;
+ }
+ let unindexed = c.get("index").is_none();
+ let index = if unindexed && self.google_compat {
+ // Google's compatible endpoint may emit complete calls
+ // without indexes. Never guess how to join partial calls.
+ let id = c["id"]
+ .as_str()
+ .filter(|id| !id.is_empty())
+ .ok_or_else(invalid)?;
+ c["function"]["name"]
+ .as_str()
+ .filter(|name| !name.is_empty())
+ .ok_or_else(invalid)?;
+ let arguments = c["function"]["arguments"].as_str().ok_or_else(invalid)?;
+ if !serde_json::from_str::<Value>(arguments)
+ .map_err(|_| invalid())?
+ .is_object()
+ || self.calls.values().any(|call| call.id == id)
+ {
+ return Err(invalid());
+ }
+ self.calls.keys().next_back().map_or(0, |index| index + 1)
+ } else {
+ c["index"].as_u64().ok_or_else(invalid)?
+ };
if index >= 64 || c.get("type").is_some_and(|v| v != "function") {
return Err(invalid());
}
let call = self.calls.entry(index).or_default();
+ if call.complete {
+ return Err(invalid());
+ }
if let Some(id) = c.get("id") {
- if !call.id.is_empty() {
+ let id = id
+ .as_str()
+ .filter(|id| !id.is_empty())
+ .ok_or_else(invalid)?;
+ if !call.id.is_empty() && (!self.google_compat || call.id != id) {
return Err(invalid());
}
- call.id = id.as_str().ok_or_else(invalid)?.into();
+ call.id = id.into();
}
if let Some(name) = c["function"].get("name") {
- call.name.push_str(name.as_str().ok_or_else(invalid)?);
+ let name = name.as_str().ok_or_else(invalid)?;
+ if !self.google_compat || call.name != name {
+ call.name.push_str(name);
+ }
}
if let Some(args) = c["function"].get("arguments") {
call.arguments.push_str(args.as_str().ok_or_else(invalid)?);
}
+ if let Some(signature) = signature {
+ if call.signature.as_ref().is_some_and(|old| old != &signature) {
+ return Err(invalid());
+ }
+ call.signature = Some(signature);
+ }
+ call.complete = unindexed;
}
}
if let Some(reason) = choice.get("finish_reason").filter(|v| !v.is_null()) {
match reason.as_str() {
- Some("stop") if self.calls.is_empty() => {}
+ Some("stop") if self.calls.is_empty() || self.google_compat => {}
Some("tool_calls") if !self.calls.is_empty() => {}
Some("length") => return Err(Error::new(Failure::Truncated)),
Some("content_filter") => return Err(Error::new(Failure::SafetyBlocked)),
@@ -264,25 +427,69 @@ impl StreamDecoder for Decoder {
}
Ok(text)
}
+}
+impl StreamDecoder for Decoder {
+ fn usage_report(&self) -> Option<crate::telemetry::TokenUsage> {
+ self.usage.as_ref().map(Usage::reported)
+ }
+ fn output_started(&self) -> bool {
+ !self.text.is_empty()
+ || self
+ .calls
+ .values()
+ .any(|c| !c.name.is_empty() || !c.arguments.is_empty())
+ }
+ fn event(&mut self, event: &Value) -> Result<Option<String>, Error> {
+ if self.failed {
+ return Err(invalid());
+ }
+ let result = self.accept(event);
+ if result.is_err() {
+ self.failed = true;
+ }
+ result
+ }
fn done(&mut self) -> Result<(), Error> {
- if self.done || self.reason.is_none() {
+ if self.failed || self.done || self.reason.is_none() {
return Err(invalid());
}
self.done = true;
Ok(())
}
fn finish(self) -> Result<Completion, Error> {
+ if self.failed {
+ return Err(invalid());
+ }
if !self.done || self.reason.is_none() {
return Err(Error::new(Failure::IncompleteStream));
}
let mut output = vec![];
- if !self.text.is_empty() {
- output.push(json!({"type":"message","role":"assistant","content":[{"type":"output_text","text":self.text}]}));
+ if !self.text.is_empty() || self.message_signature.is_some() {
+ let mut message = json!({"type":"message","role":"assistant","content":[{"type":"output_text","text":self.text}]});
+ if let Some(signature) = self.message_signature {
+ let id = crate::id("chat_message");
+ output.push(private_signature(
+ self.binding.as_deref().ok_or_else(config)?,
+ "message",
+ &id,
+ signature,
+ ));
+ message["id"] = json!(id);
+ }
+ output.push(message);
}
for (index, call) in self.calls {
if call.id.is_empty() || call.name.is_empty() {
return Err(invalid());
}
+ if let Some(signature) = call.signature {
+ output.push(private_signature(
+ self.binding.as_deref().ok_or_else(config)?,
+ "tool_call",
+ &call.id,
+ signature,
+ ));
+ }
output.push(json!({"type":"function_call","id":format!("chat_call_{index}"),"call_id":call.id,"name":call.name,"arguments":call.arguments}));
}
if output.is_empty() || crate::model::validate_output(&output).is_err() {
@@ -298,6 +505,209 @@ impl StreamDecoder for Decoder {
#[cfg(test)]
mod tests {
use super::*;
+
+ fn signed_call(id: &str) -> Value {
+ json!({"id":id,"type":"function","function":{"name":"app__lookup","arguments":"{\"name\":\"example\"}"},"extra_content":{"google":{"thought_signature":"OPAQUE_SIGNATURE"}}})
+ }
+
+ #[test]
+ fn gemini_complete_unindexed_calls_preserve_signatures_and_parallel_order() {
+ let binding = replay_binding(
+ "https://example.test/openai/chat/completions",
+ "gemini-fixture",
+ Some("identity-1"),
+ );
+ let mut d = Decoder::with_binding(binding.clone(), false);
+ let second = json!({"id":"call-2","type":"function","function":{"name":"app__lookup","arguments":"{\"name\":\"other\"}"}});
+ assert_eq!(
+ d.event(&chunk(
+ json!({"role":"assistant","tool_calls":[signed_call("call-1"),second]}),
+ Value::Null
+ ))
+ .unwrap(),
+ None
+ );
+ d.event(&chunk(json!({"role":"assistant"}), json!("stop")))
+ .unwrap();
+ d.done().unwrap();
+ let completion = d.finish().unwrap();
+ assert_eq!(completion.output[0]["type"], "reasoning");
+ assert_eq!(completion.output[1]["call_id"], "call-1");
+ assert_eq!(completion.output[2]["call_id"], "call-2");
+ let mut history = vec![json!({"role":"user","content":"Look up both"})];
+ history.extend(completion.output);
+ for id in ["call-1", "call-2"] {
+ history.push(json!({"type":"function_call_output","call_id":id,"output":"available"}));
+ }
+ let request = body_with_binding(
+ "gemini-fixture",
+ "Instructions",
+ &history,
+ &[],
+ 100,
+ None,
+ Some(&binding),
+ )
+ .unwrap();
+ assert_eq!(
+ request["messages"][2]["tool_calls"][0]["extra_content"]["google"]["thought_signature"],
+ "OPAQUE_SIGNATURE"
+ );
+ assert_eq!(request["messages"][2]["tool_calls"][1]["id"], "call-2");
+ assert!(request["messages"][2]["tool_calls"][1]
+ .get("extra_content")
+ .is_none());
+ assert_eq!(request["messages"][3]["tool_call_id"], "call-1");
+ assert_eq!(request["messages"][4]["tool_call_id"], "call-2");
+ assert!(
+ !serde_json::to_string(&crate::model::public_items(&history))
+ .unwrap()
+ .contains("OPAQUE_SIGNATURE")
+ );
+ assert!(
+ crate::anthropic::body("claude", "Instructions", &history, &[], 100, None).is_err()
+ );
+ assert!(body("gemini-fixture", "Instructions", &history, &[], 100, None).is_err());
+ for changed in [
+ replay_binding(
+ "https://other.test/openai/chat/completions",
+ "gemini-fixture",
+ Some("identity-1"),
+ ),
+ replay_binding(
+ "https://example.test/openai/chat/completions",
+ "other-model",
+ Some("identity-1"),
+ ),
+ replay_binding(
+ "https://example.test/openai/chat/completions",
+ "gemini-fixture",
+ Some("identity-2"),
+ ),
+ ] {
+ assert!(body_with_binding(
+ "gemini-fixture",
+ "Instructions",
+ &history,
+ &[],
+ 100,
+ None,
+ Some(&changed)
+ )
+ .is_err());
+ }
+ }
+
+ #[test]
+ fn gemini_indexed_fragments_and_terminal_text_signatures_roundtrip() {
+ let mut d = Decoder::with_binding("binding".into(), true);
+ d.event(&chunk(json!({"tool_calls":[{"index":0,"id":"call-1","type":"function","function":{"name":"app__lookup","arguments":"{\"name\":"}}]}), Value::Null)).unwrap();
+ d.event(&chunk(json!({"tool_calls":[{"index":0,"id":"call-1","function":{"arguments":"\"example\"}"},"extra_content":{"google":{"thought_signature":"OPAQUE_SIGNATURE"}}}]}), json!("stop"))).unwrap();
+ d.done().unwrap();
+ assert_eq!(
+ d.finish().unwrap().output[1]["arguments"],
+ "{\"name\":\"example\"}"
+ );
+
+ let mut text = Decoder::with_binding("binding".into(), true);
+ assert_eq!(
+ text.event(&chunk(json!({"content":"The answer"}), Value::Null))
+ .unwrap(),
+ Some("The answer".into())
+ );
+ assert_eq!(text.event(&chunk(json!({"content":"","extra_content":{"google":{"thought_signature":"TEXT_SIGNATURE"}}}), json!("stop"))).unwrap(), Some("".into()));
+ text.done().unwrap();
+ let mut history = vec![json!({"role":"user","content":"Question"})];
+ history.extend(text.finish().unwrap().output);
+ history.push(json!({"role":"user","content":"Follow up"}));
+ let replay = body_with_binding(
+ "fixture",
+ "Instructions",
+ &history,
+ &[],
+ 100,
+ None,
+ Some("binding"),
+ )
+ .unwrap();
+ assert_eq!(replay["messages"][2]["content"][0]["text"], "The answer");
+ assert_eq!(
+ replay["messages"][2]["extra_content"]["google"]["thought_signature"],
+ "TEXT_SIGNATURE"
+ );
+ assert!(
+ !serde_json::to_string(&crate::model::public_items(&history))
+ .unwrap()
+ .contains("TEXT_SIGNATURE")
+ );
+ }
+
+ #[test]
+ fn gemini_metadata_does_not_authorize_ambiguous_or_incomplete_calls() {
+ let mut partial = signed_call("call-1");
+ partial["function"]["arguments"] = json!("{");
+ let mut no_id = signed_call("call-1");
+ no_id.as_object_mut().unwrap().remove("id");
+ let mut bad_signature = signed_call("call-1");
+ bad_signature["extra_content"]["google"]["thought_signature"] = json!(42);
+ for call in [partial, no_id, bad_signature] {
+ let mut d = Decoder::with_binding("binding".into(), true);
+ assert!(d
+ .event(&chunk(json!({"tool_calls":[call]}), json!("stop")))
+ .is_err());
+ assert!(d.done().is_err());
+ assert!(d.finish().is_err());
+ }
+ let mut duplicate = Decoder::with_binding("binding".into(), true);
+ assert!(duplicate
+ .event(&chunk(
+ json!({"tool_calls":[signed_call("same"),signed_call("same")]}),
+ json!("stop")
+ ))
+ .is_err());
+ let mut incomplete = Decoder::with_binding("binding".into(), true);
+ incomplete
+ .event(&chunk(
+ json!({"tool_calls":[signed_call("call-1")]}),
+ json!("stop"),
+ ))
+ .unwrap();
+ assert_eq!(
+ incomplete.finish().unwrap_err().kind,
+ Failure::IncompleteStream
+ );
+ let mut unbound = Decoder::default();
+ assert!(unbound
+ .event(&chunk(
+ json!({"tool_calls":[signed_call("call-1")]}),
+ json!("stop")
+ ))
+ .is_err());
+ // Generic chat streams still require an explicit tool finish reason.
+ let mut ordinary = Decoder::with_binding("binding".into(), false);
+ assert!(ordinary.event(&chunk(json!({"tool_calls":[{"index":0,"id":"x","type":"function","function":{"name":"app__lookup","arguments":"{}"}}]}), json!("stop"))).is_err());
+ }
+
+ #[test]
+ fn gemini_conflicting_or_late_signatures_fail_closed() {
+ let mut d = Decoder::with_binding("binding".into(), true);
+ let mut call = signed_call("call-1");
+ call["index"] = json!(0);
+ d.event(&chunk(json!({"tool_calls":[call]}), Value::Null))
+ .unwrap();
+ assert!(d.event(&chunk(json!({"tool_calls":[{"index":0,"extra_content":{"google":{"thought_signature":"DIFFERENT"}}}]}), Value::Null)).is_err());
+ assert!(d.done().is_err());
+ let mut late = Decoder::with_binding("binding".into(), true);
+ late.event(&chunk(json!({"content":"Hi"}), json!("stop")))
+ .unwrap();
+ assert!(late
+ .event(&chunk(
+ json!({"extra_content":{"google":{"thought_signature":"LATE"}}}),
+ json!("stop")
+ ))
+ .is_err());
+ assert!(late.finish().is_err());
+ }
#[test]
fn requests_usage_for_streamed_completions() {
let request = body(
diff --git a/harness/runtime/src/model.rs b/harness/runtime/src/model.rs
index 77cff84d..6a191b05 100644
--- a/harness/runtime/src/model.rs
+++ b/harness/runtime/src/model.rs
@@ -259,7 +259,7 @@ impl Model {
span.metadata("gen_ai.request.model", &self.name);
span.model_started();
if !self.content_withheld {
- span.content_tail("gen_ai.prompt",||serde_json::to_string(&json!({"instructions":instructions,"history":&history[history.len().saturating_sub(20)..]})).unwrap_or_default());
+ span.content_tail("gen_ai.prompt",||serde_json::to_string(&json!({"instructions":instructions,"history":public_items(&history[history.len().saturating_sub(20)..])})).unwrap_or_default());
}
let (inner, mut rx) = mpsc::channel(32);
let future = request(inner);
@@ -301,7 +301,8 @@ impl Model {
}
if !self.content_withheld {
span.content("gen_ai.completion", || {
- serde_json::to_string(&completion.output).unwrap_or_default()
+ serde_json::to_string(&public_items(&completion.output))
+ .unwrap_or_default()
});
}
}
@@ -359,13 +360,25 @@ impl Model {
.timeout(std::time::Duration::from_secs(180))
.build()?;
if api == crate::model_api::Api::ChatCompletions {
- let mut body = crate::chat_completions::body(
+ let binding = crate::chat_completions::replay_binding(
+ &endpoint,
+ &self.name,
+ self.connection
+ .as_ref()
+ .map(|(_, selection)| selection.identity.as_str()),
+ );
+ let google_compat = reqwest::Url::parse(&endpoint).is_ok_and(|url| {
+ url.host_str() == Some("generativelanguage.googleapis.com")
+ && url.path().ends_with("/openai/chat/completions")
+ });
+ let mut body = crate::chat_completions::body_with_binding(
&self.name,
instructions,
history,
tools,
max_tokens,
format.as_ref(),
+ Some(&binding),
)?;
// Some compatible/local servers reject stream_options. This is an
// explicit provider capability override, never an automatic retry
@@ -381,7 +394,7 @@ impl Model {
key,
body,
tx,
- crate::chat_completions::Decoder::default(),
+ crate::chat_completions::Decoder::with_binding(binding, google_compat),
)
.await?;
return Ok(());
@@ -403,7 +416,7 @@ impl Model {
crate::anthropic::request(&client, &endpoint, key, body, tx).await?;
return Ok(());
}
- if history.iter().any(|v| v["provider"] == "anthropic") {
+ if history.iter().any(|v| v.get("provider").is_some()) {
bail!("Provider-private history cannot be sent to another provider");
}
let input = history
@@ -944,6 +957,16 @@ impl Model {
}
}
+/// Provider-private reasoning/signatures belong only to replay storage and the
+/// originating model. Even full-content exports must not expose this state.
+pub fn public_items(items: &[Value]) -> Vec<Value> {
+ items
+ .iter()
+ .filter(|item| item["type"] != "reasoning")
+ .cloned()
+ .collect()
+}
+
pub fn validate_output(output: &[Value]) -> Result<()> {
if output.len() > 64 || serde_json::to_vec(output)?.len() > 512000 {
bail!("Provider output exceeds checkpoint budget");
diff --git a/harness/runtime/src/runtime.rs b/harness/runtime/src/runtime.rs
index 016e55f5..37cba965 100644
--- a/harness/runtime/src/runtime.rs
+++ b/harness/runtime/src/runtime.rs
@@ -645,8 +645,9 @@ impl Runtime {
} else if summary {
instructions.push_str("\nThe configured turn budget is exhausted. No tools may execute. Return a short plain-language response (at most 120 words) telling the user the configured limit was reached, what was actually completed, and what is incomplete. Do not claim failed/unknown tools succeeded. Do not request tools or continue the task.");
}
- let request_text =
- serde_json::to_string(&json!({"instructions":instructions,"history":history}))?;
+ let request_text = serde_json::to_string(
+ &json!({"instructions":instructions,"history":crate::model::public_items(history)}),
+ )?;
let (_, blocked) = self
.check_content(
sid,
diff --git a/harness/runtime/src/runtime_safety.rs b/harness/runtime/src/runtime_safety.rs
index c0e541bf..fe504498 100644
--- a/harness/runtime/src/runtime_safety.rs
+++ b/harness/runtime/src/runtime_safety.rs
@@ -224,8 +224,13 @@ impl Runtime {
});
continue;
}
- if protects_output || sanitized != text {
- completion.output.retain(|v| v["type"] != "message");
+ if sanitized != text {
+ // Preserve untouched message boundaries and opaque replay
+ // metadata. A rewritten text has no valid original signature;
+ // text signatures are optional and must not be fabricated.
+ completion.output.retain(|v| {
+ v["type"] != "message" && !crate::chat_completions::message_signature(v)
+ });
if !sanitized.is_empty() {
completion.output.insert(0,json!({"type":"message","role":"assistant","content":[{"type":"output_text","text":sanitized}]}));
}
diff --git a/harness/runtime/tests/core.rs b/harness/runtime/tests/core.rs
index c3f8e5c6..8292371b 100644
--- a/harness/runtime/tests/core.rs
+++ b/harness/runtime/tests/core.rs
@@ -330,6 +330,40 @@ fn history_pages_advance_across_private_items_without_disclosing_them() {
);
}
+#[test]
+fn google_signatures_survive_storage_but_never_enter_display_history() {
+ use openworker_runtime::provider::StreamDecoder;
+ let dir = tempfile::tempdir().unwrap();
+ let mut db = Store::open(dir.path()).unwrap();
+ let mut s = session();
+ let mut decoder =
+ openworker_runtime::chat_completions::Decoder::with_binding("binding".into(), true);
+ decoder.event(&json!({"choices":[{"index":0,"delta":{"tool_calls":[{"id":"call-1","type":"function","function":{"name":"app__lookup","arguments":"{}"},"extra_content":{"google":{"thought_signature":"OPAQUE_PRIVATE"}}}]},"finish_reason":"stop"}]})).unwrap();
+ decoder.done().unwrap();
+ s.history.extend(decoder.finish().unwrap().output);
+ s.history
+ .push(json!({"type":"function_call_output","call_id":"call-1","output":"available"}));
+ db.save(&s, "created", json!({})).unwrap();
+ drop(db);
+ let db = Store::open(dir.path()).unwrap();
+ let raw = db.get(&s.id).unwrap().history;
+ assert!(serde_json::to_string(&raw)
+ .unwrap()
+ .contains("OPAQUE_PRIVATE"));
+ assert!(!db
+ .history_page(&s.id, 0)
+ .unwrap()
+ .to_string()
+ .contains("OPAQUE_PRIVATE"));
+ let excerpt = db.transcript_excerpt(&s.id, 0, 0).unwrap();
+ assert!(!excerpt.to_string().contains("OPAQUE_PRIVATE"));
+ assert!(excerpt.to_string().contains("app__lookup"));
+ let summary = openworker_runtime::compaction::logical(&raw);
+ assert!(!serde_json::to_string(&summary)
+ .unwrap()
+ .contains("OPAQUE_PRIVATE"));
+}
+
#[test]
fn transcript_excerpts_are_scoped_bounded_and_preserve_unicode() {
let dir = tempfile::tempdir().unwrap();
diff --git a/harness/runtime/tests/model_accounting.rs b/harness/runtime/tests/model_accounting.rs
index f3e3e12e..8f347e8f 100644
--- a/harness/runtime/tests/model_accounting.rs
+++ b/harness/runtime/tests/model_accounting.rs
@@ -115,6 +115,60 @@ fn telemetry() -> (Telemetry, Controller, Arc<Mutex<Vec<Record>>>) {
.unwrap();
(t, c, rows)
}
+
+#[tokio::test]
+async fn full_content_exports_exclude_google_replay_signatures() {
+ use openworker_runtime::{model::Progress, provider::Completion};
+ let rows = Arc::new(Mutex::new(vec![]));
+ let (t, c) = Telemetry::start_with_content(
+ CaptureSink(rows.clone()),
+ 128,
+ ContentPolicy {
+ capture: Capture::Full,
+ ..Default::default()
+ },
+ )
+ .unwrap();
+ let root = tempfile::tempdir().unwrap();
+ let manager = Arc::new(Manager::open(root.path()).unwrap());
+ let signed_tool = sse(vec![
+ json!({"choices":[{"index":0,"delta":{"tool_calls":[{"id":"call-1","type":"function","function":{"name":"app__lookup","arguments":"{}"},"extra_content":{"google":{"thought_signature":"OPAQUE_TOOL_SIGNATURE"}}}]},"finish_reason":"stop"}]}),
+ ]) + "data: [DONE]\n\n";
+ let signed_text = sse(vec![
+ json!({"choices":[{"index":0,"delta":{"content":"Public answer","extra_content":{"google":{"thought_signature":"OPAQUE_TEXT_SIGNATURE"}}},"finish_reason":"stop"}]}),
+ ]) + "data: [DONE]\n\n";
+ let (url, server) = endpoint(vec![(200, signed_tool), (200, signed_text)]).await;
+ let m = model(&manager, "chat-completions", &url, &t, ModelPurpose::Main);
+ async fn complete(model: &Model, history: &[Value]) -> Completion {
+ let (tx, mut rx) = tokio::sync::mpsc::channel(32);
+ let collect = async {
+ let mut completion = None;
+ while let Some(progress) = rx.recv().await {
+ if let Progress::Complete(value) = progress {
+ completion = Some(value);
+ }
+ }
+ completion.unwrap()
+ };
+ let (result, completion) =
+ tokio::join!(model.next("Instructions", history, &[], tx), collect);
+ result.unwrap();
+ completion
+ }
+ let mut history = vec![json!({"role":"user","content":"Public question"})];
+ history.extend(complete(&m, &history).await.output);
+ history.push(json!({"type":"function_call_output","call_id":"call-1","output":"available"}));
+ let second = complete(&m, &history).await;
+ assert!(serde_json::to_string(&second.output)
+ .unwrap()
+ .contains("OPAQUE_TEXT_SIGNATURE"));
+ server.await.unwrap();
+ assert!(c.shutdown(Duration::from_secs(3)));
+ let exported = format!("{:?}", rows.lock().unwrap());
+ assert!(!exported.contains("OPAQUE_"));
+ assert!(exported.contains("Public answer"));
+ assert!(exported.contains("Public question"));
+}
#[tokio::test]
async fn all_real_model_purposes_keep_usage_once_and_share_turn() {
let (t, c, rows) = telemetry();
diff --git a/harness/typescript-sdk/README.md b/harness/typescript-sdk/README.md
index 8ae21745..08535cb7 100644
--- a/harness/typescript-sdk/README.md
+++ b/harness/typescript-sdk/README.md
@@ -117,6 +117,12 @@ authorize and constrain what they capture. Plain JSON return values are unchange
## Chat Completions streaming compatibility
+Google Gemini's compatible endpoint supports signed tool conversations; the
+harness preserves provider-private thought signatures across tool results and
+restart. See [Gemini tool conversations](../../docs/harness/guides/providers.md#gemini-tool-conversations).
+Anthropic Messages enables automatic prompt caching by default; use
+`promptCaching: false` to opt out. See [Anthropic prompt caching](../../docs/harness/guides/providers.md#anthropic-prompt-caching).
+
The harness requests streaming token usage by default. Set `streamUsage: false`
on `setModelProvider({ api: "chat-completions", ... })` only when a compatible
server rejects `stream_options`. This persisted setting survives credential
diff --git a/harness/typescript-sdk/package.json b/harness/typescript-sdk/package.json
index 8a384bb9..e9304c02 100644
--- a/harness/typescript-sdk/package.json
+++ b/harness/typescript-sdk/package.json
@@ -31,6 +31,7 @@
"test:openrouter-streaming": "npm run build && node --test test/openrouter-streaming.test.mjs",
"test:chat-usage": "npm run build && node --test --test-concurrency=1 test/chat-usage.test.mjs",
"test:anthropic-caching": "npm run build && node --test --test-concurrency=1 test/anthropic-caching.test.mjs",
+ "test:gemini-compat": "npm run build && node --test --test-concurrency=1 test/gemini-compat.test.mjs",
"test:provider-auth": "npm run build && node --test test/provider-auth.test.mjs",
"prepack": "npm run build",
"test:service": "npm run build && node --test test/service.test.mjs",
diff --git a/harness/typescript-sdk/test/gemini-compat.test.mjs b/harness/typescript-sdk/test/gemini-compat.test.mjs
new file mode 100644
index 00000000..52afbb04
--- /dev/null
+++ b/harness/typescript-sdk/test/gemini-compat.test.mjs
@@ -0,0 +1,126 @@
+// Gemini-compatible synthetic SSE + a real SDK host. No Google credentials.
+// The fixture verifies both required signature replay and observed indexless
+// complete-call framing. It is not the partner's captured failing response.
+import test from "node:test";
+import assert from "node:assert/strict";
+import { createServer } from "node:http";
+import { mkdtemp, rm } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { Harness } from "../dist/index.js";
+
+const options = { skip: !process.env.OW_TS_TEST_HOST, timeout: 90000 };
+const signatureA = "OPAQUE_GOOGLE_SIGNATURE_A";
+const signatureB = "OPAQUE_GOOGLE_SIGNATURE_B";
+const signatureText = "OPAQUE_GOOGLE_TEXT_SIGNATURE";
+const extra = (signature) => ({ google: { thought_signature: signature } });
+
+for (const mode of ["normal", "partial", "conflicting", "missing-done"]) {
+ test(`Gemini compatible signed tool loop: ${mode}`, options, async (t) => {
+ const root = await mkdtemp(join(tmpdir(), "ow-gemini-compat-"));
+ const requests = [];
+ const failures = [];
+ const server = createServer(async (req, res) => {
+ try {
+ let raw = "";
+ for await (const chunk of req) raw += chunk;
+ const input = JSON.parse(raw);
+ requests.push(input);
+ assert.equal(input.stream, true);
+ assert.deepEqual(input.stream_options, { include_usage: true });
+ const n = requests.length;
+ if (mode === "normal" && n > 1) {
+ const calls = input.messages.flatMap((m) => m.tool_calls ?? []);
+ assert.equal(calls[0].extra_content.google.thought_signature, signatureA);
+ assert.deepEqual(JSON.parse(calls[0].function.arguments), { name: "first" });
+ assert.ok(input.messages.some((m) => m.role === "tool" && m.tool_call_id === "google-call-a"));
+ if (n > 2) {
+ assert.equal(calls[1].extra_content.google.thought_signature, signatureB);
+ assert.deepEqual(JSON.parse(calls[1].function.arguments), { name: "second" });
+ assert.ok(input.messages.some((m) => m.role === "tool" && m.tool_call_id === "google-call-b"));
+ }
+ if (n > 3) {
+ assert.ok(input.messages.some((m) => m.role === "assistant" && m.extra_content?.google?.thought_signature === signatureText));
+ }
+ }
+ res.writeHead(200, { "Content-Type": "text/event-stream" });
+ const chunk = (delta, finish_reason = null) => ({ id: `response-${n}`, choices: [{ index: 0, delta, finish_reason }] });
+ const write = (event) => {
+ const frame = Buffer.from(`data: ${JSON.stringify(event)}\n\n`);
+ // Exercise SSE and UTF-8 parser boundaries independently of objects.
+ res.write(frame.subarray(0, 13)); res.write(frame.subarray(13));
+ };
+ if (mode !== "normal" || n === 1) {
+ const call = { id: "google-call-a", type: "function", function: { name: "app__lookup", arguments: mode === "partial" ? "{" : '{"name":"first"}' }, extra_content: extra(signatureA) };
+ if (mode === "conflicting") call.index = 0;
+ write(chunk({ role: "assistant", tool_calls: [call] }));
+ if (mode === "conflicting") write(chunk({ tool_calls: [{ index: 0, extra_content: extra("CHANGED_SIGNATURE") }] }));
+ write(chunk({ role: "assistant" }, "stop"));
+ } else if (n === 2) {
+ write(chunk({ role: "assistant", tool_calls: [{ index: 0, id: "google-call-b", type: "function", function: { name: "app__lookup", arguments: '{"name":' }, extra_content: extra(signatureB) }] }));
+ write(chunk({ tool_calls: [{ index: 0, function: { arguments: '"second"}' } }] }, "tool_calls"));
+ } else {
+ write(chunk({ role: "assistant", content: "Both checked." }));
+ write(chunk({ role: "assistant", content: "", extra_content: extra(signatureText) }, "stop"));
+ }
+ if (mode !== "missing-done") {
+ write({ id: `response-${n}`, choices: [], usage: { prompt_tokens: 42, completion_tokens: 7, total_tokens: 49 } });
+ res.end("data: [DONE]\n\n");
+ } else res.end();
+ } catch (error) {
+ failures.push(error);
+ if (!res.headersSent) res.writeHead(400);
+ res.end('{"error":{"message":"fixture request validation failed"}}');
+ }
+ });
+ await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
+ let harness;
+ t.after(async () => {
+ await harness?.shutdown(); server.closeAllConnections();
+ await new Promise((resolve) => server.close(resolve));
+ await rm(root, { recursive: true, force: true });
+ });
+ const startOptions = { executable: process.env.OW_TS_TEST_HOST, stateDir: root, sandbox: "off" };
+ const provider = { name: "GeminiFixture", api: "chat-completions", baseUrl: `http://127.0.0.1:${server.address().port}/v1/openai` };
+ const definition = { name: "gemini-check", instructions: "Check both names, then explain.", tools: ["app.lookup"], approvals: { mode: "only", tools: [] }, planning: false, memory: false, scheduling: false, defaultModel: { provider: "GeminiFixture", name: "gemini-fixture" } };
+ let effects = 0;
+ const tools = { namespace: "app", tools: { lookup: {
+ description: "Read a synthetic name status.", inputSchema: { type: "object", properties: { name: { type: "string" } }, required: ["name"], additionalProperties: false },
+ execute: (args) => { effects++; assert.ok(["first", "second"].includes(args.name)); return { available: true }; },
+ } } };
+ async function start() {
+ harness = await Harness.start(startOptions);
+ await harness.setModelProvider(provider);
+ await harness.registerTools(tools);
+ return harness.createAgent(definition);
+ }
+ const agent = await start();
+ const session = await agent.newSession();
+ const run = await session.send("Check first and second.");
+ const events = [];
+ for await (const item of run.events()) events.push(item);
+ const result = await run.result();
+ assert.deepEqual(failures, []);
+ if (mode !== "normal") {
+ assert.equal(result.status, "failed");
+ assert.equal(effects, 0, "Invalid or incomplete signed output cannot execute tools");
+ return;
+ }
+ assert.equal(result.status, "completed");
+ assert.equal(result.finalOutput, "Both checked.");
+ assert.equal(effects, 2);
+ assert.equal(requests.length, 3);
+ for (const signature of [signatureA, signatureB, signatureText]) {
+ assert.ok(!JSON.stringify(events).includes(signature), "Private signatures must not appear in public events");
+ assert.ok(!JSON.stringify(await session.snapshot()).includes(signature), "Private signatures must not appear in display snapshots");
+ }
+ const reference = session.reference();
+ await harness.shutdown();
+ const restartedAgent = await start();
+ const restored = await restartedAgent.resumeSession(reference);
+ assert.equal((await (await restored.send("Explain again briefly.")).result()).status, "completed");
+ assert.deepEqual(failures, []);
+ assert.equal(effects, 2);
+ assert.equal(requests.length, 4, "Stored signatures must survive host restart");
+ });
+}