Awesome list for AI agent harness engineering: tools, patterns, evals, memory, MCP, permissions, observability, and orchestration.
Python agent-harnessagent-memoryagent-orchestrationai-agent-harnessai-agentsawesome-listcontext-engineeringharness-engineeringmcp
Add Driving the Agent Quality Flywheel to Verification & CI Integration section
ai-boost4364dabca2d4
diff --git a/README.md b/README.mdindex 4c912a2..ff8eafa 100644--- a/README.md+++ b/README.md@@ -304,6 +304,7 @@ Harness components organized by the problem they solve, not by vendor.- [Agent Evaluation Framework 2026: Metrics, Rubrics & Benchmarks](https://galileo.ai/blog/agent-evaluation-framework-metrics-rubrics-benchmarks) — Comprehensive framework combining multi-environment baselines (AgentBench), domain-specific benchmarks (Terminal Bench 2.0, WebArena, SWE-bench Verified), and industry standards (NIST AI Agent Standards Initiative, February 2026). Provides reference metrics and rubrics for evaluating coding agents, chatbots, and specialized agents across dimensions (correctness, efficiency, safety). Essential for building eval harnesses that measure across standardized dimensions.- [The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems](https://arxiv.org/abs/2602.17753) — Systematic index analyzing deployed agent safety documentation, guardrails, and third-party testing across 30 production systems; identifies critical gaps in agentic safety disclosure and documentation. Useful as a checklist for production harness safeguards before deployment.- [sentrux](https://github.com/sentrux/sentrux) — Real-time architectural sensor that closes the feedback loop for coding agents: scans codebases, scores structural health, and surfaces degradation via MCP so agents can self-correct before entropy compounds. The `gate --save` / `gate` pair makes architectural regression detection CI-friendly — filling the verification gap between linters (style) and tests (behavior) for code produced by autonomous harnesses. +- [Driving the Agent Quality Flywheel from Your Coding Agent](https://developers.googleblog.com/en/driving-the-agent-quality-flywheel-from-your-coding-agent/) — Google's June 2026 account of automating the eval-optimize loop for coding agents: independent AutoRaters grade agent outputs so the optimizer cannot game its own metrics, and custom rubrics isolate specific behaviors like stale-message echoing that blended scores miss. Shows how to close the verification loop continuously against both synthetic dev scenarios and production traces.### Observability & Tracing