Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

fix(llm): route Luna through Responses API

Lucas do Amaral Saboya05dc99f0d129
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c6418a9a..fb78c51e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
+- The OpenCode provider now sends `gpt-5.6-luna` requests to OpenCode Go's
+ Responses endpoint. Luna is not served through Chat Completions, where plain
+ and structured ai-memory calls returned HTTP 500 (#618).
- `bootstrap` no longer aborts the whole multi-chunk run when one chunk
returns no `pages` key (#614). Later chunks are told which paths
earlier ones wrote, so a model that judges the material already
diff --git a/crates/ai-memory-llm/src/lib.rs b/crates/ai-memory-llm/src/lib.rs
index 134266dc..16eed176 100644
--- a/crates/ai-memory-llm/src/lib.rs
+++ b/crates/ai-memory-llm/src/lib.rs
@@ -16,6 +16,8 @@
//! * **OpenAI**: `response_format = { type: "json_schema", strict: true }`.
//! * **OpenAI OAuth/Codex**: ChatGPT/Codex Responses API with
//! `text.format = { type: "json_schema", strict: true }`.
+//! * **OpenCode Go**: Responses API for GPT-5.6 Luna; OpenAI-compatible Chat
+//! Completions for the rest of the catalogue.
//! * **GitHub Copilot**: GitHub token exchange to a short-lived Copilot API
//! token, then OpenAI-style Chat Completions with JSON schema format.
//! * **Gemini**: `generationConfig.responseMimeType = "application/json"`
diff --git a/crates/ai-memory-llm/src/opencode.rs b/crates/ai-memory-llm/src/opencode.rs
index ba614e0b..7e772a1f 100644
--- a/crates/ai-memory-llm/src/opencode.rs
+++ b/crates/ai-memory-llm/src/opencode.rs
@@ -1,16 +1,24 @@
-//! OpenCode Zen/Go provider.
+//! OpenCode Go provider.
//!
-//! Thin wrapper around [`OpenAiCompatProvider`] that bakes in the OpenCode
-//! Zen API base URL (`https://opencode.ai/zen/go/v1`) and names the provider
-//! `"opencode"`. Accepts an `sk-...` API key from `OPENCODE_API_KEY`.
+//! Selects the wire API published for each model under
+//! `https://opencode.ai/zen/go/v1`: GPT-5.6 Luna uses the Responses API and
+//! other models use OpenAI-compatible Chat Completions. Accepts an `sk-...`
+//! API key from `OPENCODE_API_KEY`.
+
+use std::time::Duration;
use async_trait::async_trait;
-use secrecy::SecretString;
+use secrecy::{ExposeSecret, SecretString};
+use serde::{Deserialize, Serialize};
-use crate::error::LlmResult;
+use crate::error::{LlmError, LlmResult};
+use crate::openai::{
+ STRUCTURED_OUTPUT_SCHEMA_NAME, enforce_strict_object_schemas, normalize_openai_base,
+};
use crate::openai_compat::OpenAiCompatProvider;
use crate::provider::LlmProvider;
-use crate::types::{ChatRequest, ChatResponse, LlmOperationId};
+use crate::response::{provider_error_body, response_json_limited};
+use crate::types::{ChatRequest, ChatResponse, LlmOperationId, ReasoningEffort, Usage};
/// Public OpenCode Zen/Go OpenAI-compatible base URL.
pub const OPENCODE_ZEN_BASE_URL: &str = "https://opencode.ai/zen/go/v1";
@@ -18,13 +26,21 @@ pub const OPENCODE_ZEN_BASE_URL: &str = "https://opencode.ai/zen/go/v1";
/// Default model when `AI_MEMORY_LLM_MODEL` is not set.
pub const OPENCODE_DEFAULT_MODEL: &str = "claude-sonnet-4-6";
-/// OpenCode Zen/Go LLM provider.
+const OPENCODE_USER_AGENT: &str = concat!("ai-memory/", env!("CARGO_PKG_VERSION"));
+const OPENCODE_SESSION_HEADER: &str = "x-opencode-session";
+
+/// OpenCode Go LLM provider.
///
-/// Routes through `https://opencode.ai/zen/go/v1` using the OpenAI chat
-/// completions wire format. Authenticate with the `sk-...` key obtained
-/// from <https://opencode.ai/auth>.
+/// Routes through `https://opencode.ai/zen/go/v1` using the model's published
+/// Responses or Chat Completions wire format. Authenticate with the `sk-...`
+/// key obtained from <https://opencode.ai/auth>.
pub struct OpenCodeProvider {
- inner: OpenAiCompatProvider,
+ transport: OpenCodeTransport,
+}
+
+enum OpenCodeTransport {
+ ChatCompletions(OpenAiCompatProvider),
+ Responses(OpenCodeResponsesProvider),
}
impl OpenCodeProvider {
@@ -41,17 +57,27 @@ impl OpenCodeProvider {
model: impl Into<String>,
base_url: impl Into<String>,
) -> LlmResult<Self> {
- let inner = OpenAiCompatProvider::new(base_url, Some(api_key), model.into())?
- .with_client_headers(
- concat!("ai-memory/", env!("CARGO_PKG_VERSION")),
- "x-opencode-session",
- );
- Ok(Self { inner })
+ let model = model.into();
+ let base_url = base_url.into();
+ let transport = if model_uses_responses_api(&model) {
+ OpenCodeTransport::Responses(OpenCodeResponsesProvider::new(api_key, model, base_url)?)
+ } else {
+ OpenCodeTransport::ChatCompletions(
+ OpenAiCompatProvider::new(base_url, Some(api_key), model)?
+ .with_client_headers(OPENCODE_USER_AGENT, OPENCODE_SESSION_HEADER),
+ )
+ };
+ Ok(Self { transport })
}
#[cfg(test)]
fn with_strict(mut self, strict: bool) -> Self {
- self.inner = self.inner.with_strict(strict);
+ self.transport = match self.transport {
+ OpenCodeTransport::ChatCompletions(provider) => {
+ OpenCodeTransport::ChatCompletions(provider.with_strict(strict))
+ }
+ responses => responses,
+ };
self
}
@@ -60,14 +86,28 @@ impl OpenCodeProvider {
/// `ProviderConfig::request_timeout_secs`.
#[must_use]
pub fn with_timeout_secs(mut self, secs: u64) -> Self {
- self.inner = self.inner.with_timeout_secs(secs);
+ self.transport = match self.transport {
+ OpenCodeTransport::ChatCompletions(provider) => {
+ OpenCodeTransport::ChatCompletions(provider.with_timeout_secs(secs))
+ }
+ OpenCodeTransport::Responses(provider) => {
+ OpenCodeTransport::Responses(provider.with_timeout_secs(secs))
+ }
+ };
self
}
/// Forward reasoning effort to the OpenAI-compatible Zen/Go client.
#[must_use]
- pub fn with_reasoning_effort(mut self, effort: Option<crate::ReasoningEffort>) -> Self {
- self.inner = self.inner.with_reasoning_effort(effort);
+ pub fn with_reasoning_effort(mut self, effort: Option<ReasoningEffort>) -> Self {
+ self.transport = match self.transport {
+ OpenCodeTransport::ChatCompletions(provider) => {
+ OpenCodeTransport::ChatCompletions(provider.with_reasoning_effort(effort))
+ }
+ OpenCodeTransport::Responses(provider) => {
+ OpenCodeTransport::Responses(provider.with_reasoning_effort(effort))
+ }
+ };
self
}
}
@@ -79,7 +119,10 @@ impl LlmProvider for OpenCodeProvider {
}
fn model(&self) -> &str {
- self.inner.model()
+ match &self.transport {
+ OpenCodeTransport::ChatCompletions(provider) => provider.model(),
+ OpenCodeTransport::Responses(provider) => &provider.model,
+ }
}
async fn complete(&self, request: ChatRequest) -> LlmResult<ChatResponse> {
@@ -92,9 +135,16 @@ impl LlmProvider for OpenCodeProvider {
request: ChatRequest,
operation_id: LlmOperationId,
) -> LlmResult<ChatResponse> {
- self.inner
- .complete_with_operation_id(request, operation_id)
- .await
+ match &self.transport {
+ OpenCodeTransport::ChatCompletions(provider) => {
+ provider
+ .complete_with_operation_id(request, operation_id)
+ .await
+ }
+ OpenCodeTransport::Responses(provider) => {
+ provider.complete(request, operation_id).await
+ }
+ }
}
async fn complete_structured_raw(
@@ -112,10 +162,237 @@ impl LlmProvider for OpenCodeProvider {
schema: serde_json::Value,
operation_id: LlmOperationId,
) -> LlmResult<serde_json::Value> {
- self.inner
- .complete_structured_raw_with_operation_id(request, schema, operation_id)
- .await
+ match &self.transport {
+ OpenCodeTransport::ChatCompletions(provider) => {
+ provider
+ .complete_structured_raw_with_operation_id(request, schema, operation_id)
+ .await
+ }
+ OpenCodeTransport::Responses(provider) => {
+ provider
+ .complete_structured(request, schema, operation_id)
+ .await
+ }
+ }
+ }
+}
+
+fn model_uses_responses_api(model: &str) -> bool {
+ // OpenCode Go publishes Luna only through /responses. Its
+ // /chat/completions route returns an internal-server error for these calls.
+ model.eq_ignore_ascii_case("gpt-5.6-luna")
+}
+
+struct OpenCodeResponsesProvider {
+ client: reqwest::Client,
+ api_key: SecretString,
+ base_url: String,
+ model: String,
+ timeout: Duration,
+ reasoning_effort: Option<ReasoningEffort>,
+}
+
+impl OpenCodeResponsesProvider {
+ fn new(api_key: SecretString, model: String, base_url: String) -> LlmResult<Self> {
+ Ok(Self {
+ client: reqwest::Client::builder().build()?,
+ api_key,
+ base_url,
+ model,
+ timeout: Duration::from_secs(crate::DEFAULT_REQUEST_TIMEOUT_SECS),
+ reasoning_effort: None,
+ })
}
+
+ fn with_timeout_secs(mut self, secs: u64) -> Self {
+ self.timeout = Duration::from_secs(secs);
+ self
+ }
+
+ fn with_reasoning_effort(mut self, effort: Option<ReasoningEffort>) -> Self {
+ self.reasoning_effort = effort;
+ self
+ }
+
+ async fn complete(
+ &self,
+ request: ChatRequest,
+ operation_id: LlmOperationId,
+ ) -> LlmResult<ChatResponse> {
+ let response = self
+ .post(&self.build_request(&request, None), operation_id)
+ .await?;
+ Ok(ChatResponse {
+ text: extract_output_text(&response).unwrap_or_default(),
+ usage: response.usage.map(|usage| Usage {
+ input_tokens: usage.input_tokens,
+ output_tokens: usage.output_tokens,
+ }),
+ model: response.model,
+ })
+ }
+
+ async fn complete_structured(
+ &self,
+ request: ChatRequest,
+ mut schema: serde_json::Value,
+ operation_id: LlmOperationId,
+ ) -> LlmResult<serde_json::Value> {
+ enforce_strict_object_schemas(&mut schema);
+ let text = ResponsesText {
+ format: ResponsesTextFormat::JsonSchema {
+ name: STRUCTURED_OUTPUT_SCHEMA_NAME.into(),
+ schema,
+ strict: true,
+ },
+ };
+ let response = self
+ .post(&self.build_request(&request, Some(text)), operation_id)
+ .await?;
+ let text = extract_output_text(&response).unwrap_or_default();
+ serde_json::from_str(&text).map_err(LlmError::from)
+ }
+
+ fn build_request<'a>(
+ &'a self,
+ request: &'a ChatRequest,
+ text: Option<ResponsesText>,
+ ) -> ResponsesRequest<'a> {
+ ResponsesRequest {
+ model: &self.model,
+ instructions: request.system.as_deref(),
+ input: request
+ .messages
+ .iter()
+ .map(|message| ResponsesInputMessage {
+ role: message.role.as_str(),
+ content: vec![ResponsesInputContent {
+ kind: "input_text",
+ text: &message.content,
+ }],
+ })
+ .collect(),
+ max_output_tokens: request.max_tokens,
+ store: false,
+ text,
+ reasoning: self.reasoning_effort.map(|effort| ResponsesReasoning {
+ effort: effort.openai_wire_effort(),
+ }),
+ }
+ }
+
+ async fn post<B: Serialize>(
+ &self,
+ body: &B,
+ operation_id: LlmOperationId,
+ ) -> LlmResult<ResponsesResponse> {
+ let url = normalize_openai_base(&self.base_url, "responses");
+ let response = self
+ .client
+ .post(url)
+ .timeout(self.timeout)
+ .bearer_auth(self.api_key.expose_secret())
+ .header("content-type", "application/json")
+ .header(reqwest::header::USER_AGENT, OPENCODE_USER_AGENT)
+ .header(OPENCODE_SESSION_HEADER, operation_id.to_string())
+ .json(body)
+ .send()
+ .await?;
+ let status = response.status();
+ if !status.is_success() {
+ return Err(LlmError::Provider {
+ status: status.as_u16(),
+ body: provider_error_body(response).await,
+ });
+ }
+ response_json_limited(response).await
+ }
+}
+
+#[derive(Serialize)]
+struct ResponsesRequest<'a> {
+ model: &'a str,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ instructions: Option<&'a str>,
+ input: Vec<ResponsesInputMessage<'a>>,
+ max_output_tokens: u32,
+ store: bool,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ text: Option<ResponsesText>,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ reasoning: Option<ResponsesReasoning>,
+}
+
+#[derive(Serialize)]
+struct ResponsesText {
+ format: ResponsesTextFormat,
+}
+
+#[derive(Serialize)]
+#[serde(tag = "type", rename_all = "snake_case")]
+enum ResponsesTextFormat {
+ JsonSchema {
+ name: String,
+ schema: serde_json::Value,
+ strict: bool,
+ },
+}
+
+#[derive(Serialize)]
+struct ResponsesInputMessage<'a> {
+ role: &'a str,
+ content: Vec<ResponsesInputContent<'a>>,
+}
+
+#[derive(Serialize)]
+struct ResponsesInputContent<'a> {
+ #[serde(rename = "type")]
+ kind: &'a str,
+ text: &'a str,
+}
+
+#[derive(Serialize)]
+struct ResponsesReasoning {
+ effort: ReasoningEffort,
+}
+
+#[derive(Deserialize)]
+struct ResponsesResponse {
+ model: String,
+ #[serde(default)]
+ output: Vec<ResponsesOutputItem>,
+ #[serde(default)]
+ usage: Option<ResponsesUsage>,
+}
+
+#[derive(Deserialize)]
+struct ResponsesOutputItem {
+ #[serde(default)]
+ content: Vec<ResponsesOutputContent>,
+}
+
+#[derive(Deserialize)]
+struct ResponsesOutputContent {
+ #[serde(default)]
+ text: Option<String>,
+}
+
+#[derive(Deserialize)]
+struct ResponsesUsage {
+ #[serde(default)]
+ input_tokens: u32,
+ #[serde(default)]
+ output_tokens: u32,
+}
+
+fn extract_output_text(response: &ResponsesResponse) -> Option<String> {
+ response
+ .output
+ .iter()
+ .flat_map(|item| item.content.iter())
+ .filter_map(|content| content.text.as_deref())
+ .find(|text| !text.is_empty())
+ .map(ToOwned::to_owned)
}
#[cfg(test)]
@@ -135,6 +412,17 @@ mod tests {
})
}
+ fn responses_response_with_content(content: &str) -> serde_json::Value {
+ json!({
+ "model": "gpt-5.6-luna",
+ "output": [{
+ "type": "message",
+ "content": [{ "type": "output_text", "text": content }],
+ }],
+ "usage": { "input_tokens": 1, "output_tokens": 1 },
+ })
+ }
+
fn header_value<'a>(request: &'a Request, name: &str) -> Option<&'a str> {
request
.headers
@@ -156,7 +444,85 @@ mod tests {
}
#[tokio::test]
- async fn completion_identifies_ai_memory_and_its_logical_operation() {
+ async fn luna_completion_uses_responses_api() {
+ let server = MockServer::start().await;
+ Mock::given(method("POST"))
+ .and(path("/v1/responses"))
+ .respond_with(
+ ResponseTemplate::new(200).set_body_json(responses_response_with_content("ok")),
+ )
+ .mount(&server)
+ .await;
+
+ let provider = OpenCodeProvider::new_with_base_url(
+ SecretString::from("sk-test"),
+ "gpt-5.6-luna",
+ server.uri(),
+ )
+ .unwrap();
+ let operation_id = LlmOperationId::new();
+ let response = provider
+ .complete_with_operation_id(ChatRequest::user_prompt("hello"), operation_id)
+ .await
+ .unwrap();
+
+ assert_eq!(response.text, "ok");
+ assert_eq!(response.model, "gpt-5.6-luna");
+ assert_eq!(response.usage.unwrap().input_tokens, 1);
+ let requests = server.received_requests().await.unwrap();
+ let body: serde_json::Value = serde_json::from_slice(&requests[0].body).unwrap();
+ assert_eq!(body["input"][0]["content"][0]["type"], "input_text");
+ assert_eq!(body["max_output_tokens"], 1024);
+ assert!(body.get("temperature").is_none());
+ assert_eq!(
+ header_value(&requests[0], "user-agent"),
+ Some(OPENCODE_USER_AGENT)
+ );
+ assert_eq!(
+ header_value(&requests[0], OPENCODE_SESSION_HEADER),
+ Some(operation_id.to_string().as_str())
+ );
+ }
+
+ #[tokio::test]
+ async fn luna_structured_completion_uses_responses_api() {
+ let server = MockServer::start().await;
+ Mock::given(method("POST"))
+ .and(path("/v1/responses"))
+ .respond_with(
+ ResponseTemplate::new(200)
+ .set_body_json(responses_response_with_content(r#"{"ok":true}"#)),
+ )
+ .mount(&server)
+ .await;
+
+ let provider = OpenCodeProvider::new_with_base_url(
+ SecretString::from("sk-test"),
+ "gpt-5.6-luna",
+ server.uri(),
+ )
+ .unwrap();
+ let value = provider
+ .complete_structured_raw(
+ ChatRequest::user_prompt("emit JSON"),
+ json!({
+ "type": "object",
+ "properties": { "ok": { "type": "boolean" } },
+ "required": ["ok"],
+ }),
+ )
+ .await
+ .unwrap();
+
+ assert_eq!(value, json!({ "ok": true }));
+ let requests = server.received_requests().await.unwrap();
+ let body: serde_json::Value = serde_json::from_slice(&requests[0].body).unwrap();
+ assert_eq!(body["text"]["format"]["type"], "json_schema");
+ assert_eq!(body["text"]["format"]["schema"]["type"], "object");
+ }
+
+ #[tokio::test]
+ async fn deepseek_completion_keeps_chat_completions_and_operation_headers() {
let server = MockServer::start().await;
Mock::given(method("POST"))
.and(path("/v1/chat/completions"))
@@ -166,7 +532,7 @@ mod tests {
let provider = OpenCodeProvider::new_with_base_url(
SecretString::from("sk-test"),
- "model-x",
+ "deepseek-v4-flash",
server.uri(),
)
.unwrap();
diff --git a/docs/install.md b/docs/install.md
index 4bfb2032..711c2820 100644
--- a/docs/install.md
+++ b/docs/install.md
@@ -1499,7 +1499,7 @@ If you set only the provider, ai-memory picks a sensible default:
| `AI_MEMORY_LLM_PROVIDER=openai-oauth` | `gpt-5.5` | ChatGPT/Codex backend. Run `ai-memory auth login openai-oauth` once; ai-memory stores the refresh token in `<data_dir>/auth.json` and refreshes access tokens automatically. Optional `AI_MEMORY_LLM_REASONING_EFFORT` (`none`/`minimal`/`low`/`medium`/`high`/`xhigh`/`max`/`ultra`/`persistent`) is mapped to each provider's native reasoning field; omit it to keep the model default. |
| `AI_MEMORY_LLM_PROVIDER=copilot` | `gpt-5.5` | GitHub Copilot Chat backend. ai-memory stores a GitHub user token in `<data_dir>/auth.json`, exchanges it for a short-lived Copilot API token, and refreshes before expiry. |
| `AI_MEMORY_LLM_PROVIDER=gemini` | `gemini-3.5-flash` | Google's hosted option with a generous free tier. ai-memory disables Gemini 3.5 Flash's default dynamic thinking so hidden thought tokens do not truncate strict JSON. Set `GEMINI_API_KEY` (or `GOOGLE_API_KEY`). |
-| `AI_MEMORY_LLM_PROVIDER=opencode` | `claude-sonnet-4-6` | [OpenCode Zen/Go](https://opencode.ai) cloud API at the OpenAI-compatible `opencode.ai/zen/go/v1` endpoint. Requests identify ai-memory by version and reuse one session header across related attempts. Set `OPENCODE_API_KEY` (key from `opencode.ai/auth`). Alias: `opencode-zen`. |
+| `AI_MEMORY_LLM_PROVIDER=opencode` | `claude-sonnet-4-6` | [OpenCode Go](https://opencode.ai/docs/go#endpoints) cloud API at `opencode.ai/zen/go/v1`. GPT-5.6 Luna uses the Responses endpoint; other models use Chat Completions. Requests identify ai-memory by version and reuse one session header across related attempts. Set `OPENCODE_API_KEY` (key from `opencode.ai/auth`). Alias: `opencode-zen`. |
| `AI_MEMORY_EMBEDDING_PROVIDER=openai` | `text-embedding-3-small` (1536-dim) | 5× cheaper than `-3-large` with marginal recall loss. |
| `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `AI_MEMORY_EMBEDDING_BASE_URL=https://openrouter.ai/api/v1` | `openai/text-embedding-3-small` via [OpenRouter](https://openrouter.ai) | Uses `EMBEDDING_API_KEY`, else reuses `LLM_API_KEY` or `OPENAI_API_KEY`, with the OpenAI-compatible embedding client. |
| `AI_MEMORY_EMBEDDING_PROVIDER=openai` + `AI_MEMORY_EMBEDDING_BASE_URL=https://api.orcarouter.ai/v1` | `openai/text-embedding-3-small` via [OrcaRouter](https://www.orcarouter.ai) | Uses `EMBEDDING_API_KEY`, else reuses `LLM_API_KEY`, with the OpenAI-compatible embedding client. |