Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
docs: finalize provider comparison — Kimi is reasoning-model, ineligible
AkitaOnRails9e384b0707ad
diff --git a/docs/llm-provider-comparison.md b/docs/llm-provider-comparison.mdindex 391b567b..8df88d23 100644--- a/docs/llm-provider-comparison.md+++ b/docs/llm-provider-comparison.md@@ -234,13 +234,53 @@ Both models **correctly restrained themselves** on fixturenon-trivial test the original schema-broken Run 1 couldn'teven reach.-### Kimi-K2.6 (OpenRouter) vs qwen3:32b (Ollama)+### Kimi-K2.6 (OpenRouter) — INELIGIBLE for this task-_(Run in flight at the time of writing — will update with-numbers once it lands. The expectation is that Kimi's "prose-slips" issue from Run 1 is now mitigated by the explicit-"NO ``` code fences, NO prose preamble, first char must be `{`"-directives; we expect Kimi to also reach 4-5/5 parse rate.)_+After the prompt + schema fixes, the Kimi rerun **hung for+16+ minutes on the first fixture** and never returned a parseable+response. Direct probing of the OpenRouter endpoint showed+why:++```+$ curl … -d '{"model":"moonshotai/kimi-k2.6", "max_tokens": 50, ...}'+{+ "choices": [{+ "message": {+ "content": null, ← no actual content+ "reasoning": "...208 chars..."+ }+ }],+ "usage": { "completion_tokens": 50, "reasoning_tokens": 50 }+}+```++Kimi-K2.6 is a **reasoning model**: it consumes the+`max_tokens` budget internally as "thinking" before emitting+visible `content`. For a short probe with `max_tokens: 50`,+all 50 tokens went to reasoning and content stayed `null`.++For the consolidation prompt with `max_tokens: 4000`, Kimi+would happily reason for many minutes against the strict-JSON+instructions before *either* emitting JSON or running out of+budget with no content. The eval observed 16 minutes of no+progress on fixture 1 before being killed.++This is **not a fixable prompt or schema issue** — it's a+property of the model's response style. Run 1 only "worked"+on Kimi (in the sense of producing *something*) because the+loose prompt let Kimi emit prose markdown, which used `content`+naturally. The post-fix strict-JSON prompt provokes Kimi's+reasoning mode and starves the visible response.++**Kimi-K2.6 is not a suitable provider for ai-memory's+consolidation workload.** It would work for the broader+"summarise this for me" use case where formatted prose is+fine — just not for our JSON-schema-validated path.++Other reasoning-mode models (Claude with extended thinking,+GPT-o3, Gemini "thinking" variants) would need the same+caveat: turn off reasoning mode, or budget tokens with+reasoning consumption in mind.## Qualitative read (Run 2)@@ -301,12 +341,20 @@ this project.** With the prompt + schema fixes applied:comprehensive.**Recommendation: keep Ollama qwen3:32b as the default-production LLM** (where ai-memory now points). Sonnet remains-a useful escape hatch for one-off complex consolidations,-batch lint runs against very large wikis, or situations where-the home server is unreachable. Kimi sits in an awkward-middle: comparable cost-per-call to Sonnet on OpenRouter, less-output discipline.+production LLM** (where ai-memory now points). Sonnet 4.5+remains a useful escape hatch for one-off complex+consolidations, batch lint runs against very large wikis, or+situations where the home server is unreachable. **Avoid+Kimi-K2.6** for this workload — its reasoning-model response+style is incompatible with strict-JSON output.++Cost comparison (rough order-of-magnitude, per consolidation):++| Provider | $/run | latency | notes |+|---|---|---|---|+| Ollama qwen3:32b (local) | $0 | ~100 s | electricity not modeled |+| Sonnet 4.5 (OpenRouter) | ~$0.06| ~26 s | depends on output size |+| Kimi-K2.6 (OpenRouter) | — | ✗ | inappropriate task fit |### When to revisit