Awesome list for AI agent harness engineering: tools, patterns, evals, memory, MCP, permissions, observability, and orchestration.
Python agent-harnessagent-memoryagent-orchestrationai-agent-harnessai-agentsawesome-listcontext-engineeringharness-engineeringmcp
Add SBCO (arXiv 2608.10157) to Generators & Meta-Harnesses
ai-boost412076a49185
diff --git a/README.md b/README.mdindex 087de02..61ed88f 100644--- a/README.md+++ b/README.md@@ -462,6 +462,7 @@ Real repositories worth studying — each with a note on *why* it's worth your t- [Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories](https://arxiv.org/abs/2608.02276) — August 2026 method that makes harness editing a *learned capability* rather than a fixed editor agent: a dedicated 9B harness engineer is post-trained with online RL — batches of target-agent failures become validated executable patches, fresh reruns of the frozen target supply outcome rewards, and only the engineer is updated. Raising vanilla Qwen3.5-9B from 44.3% to 53.6% across WebShop, ALFWorld, and DBBench — with gains persisting even after the target agent is fine-tuned — it points toward co-evolving the harness engineer and the target agent rather than treating the harness as static scaffolding.- [MemoHarness: Agent Harnesses That Learn from Experience](https://arxiv.org/abs/2607.14159) — July 2026 framework that decomposes the harness into six control dimensions along the inference flow (context assembly, tool interaction, generation control, orchestration, memory management, output processing) and adapts the configuration per case by retrieving from a dual-layer experience bank of per-case failure diagnoses and distilled global patterns — no labels, feedback, or search rounds at test time. Beats Codex on Terminal-Bench (0.806 vs 0.722) at lower cost ($6.89 vs $10.28, with ~94% of input tokens cacheable), transfers +0.098 on average across six base models, and ships an [open-source implementation](https://github.com/HowieHwong/MemoHarness). The clearest demonstration that a harness can be a retrieval-adapted artifact rather than a fixed configuration or an expensive optimization loop — and the six-dimension decomposition is a reusable taxonomy for auditing any harness's editable surface.- [Harness-Zero: Harness Distillation via Agent-as-Harness](https://arxiv.org/abs/2609.24974) — September 2026 method (Peking University, Google, HKUST) for making harness gains portable: a harnessing agent guided by an evolved reference harness reviews and corrects the student's *own* trajectories in the target harness's action space before execution, turning harness guidance into SFT demonstrations. The critical findings — the distilled model under a minimal harness beats the base model with the optimized harness attached (44.3% vs 41.7% macro-average across SpreadsheetBench, AppWorld, and USPTO), and correcting the student's own trajectories transfers far better than training on stronger models' trajectories — define the boundary conditions for when harness intelligence belongs in weights versus in scaffolding. [Open-source implementation](https://github.com/metaevo-ai/harness-zero). +- [SBCO: Self-Supervised, Verifier-Grounded Harness Optimization for Planning Agents](https://arxiv.org/abs/2608.10157) — August 2026 (Samsung Research America) method for domains where self-referential harness improvement breaks down because task competence doesn't imply self-modification competence (unlike coding). SBCO jointly learns a decomposed bank of per-constraint verifiers and a harness policy via block coordinate ascent — a fixed meta-agent grades its own outputs in a closed loop with no human labels — matching or beating the customized Huxley Gödel Machine baseline on travel planning and shopping while using 4–5.5× less compute. The 86%-of-gains-from-high-F1-verifiers analysis is a concrete diagnostic for which constraints repay verifier investment, and the cross-model policy transfer (GPT-5.4-mini → Mistral) shows the learned verifier bank is a portable harness asset rather than model-specific tuning.### Demo Harnesses