Awesome list for AI agent harness engineering: tools, patterns, evals, memory, MCP, permissions, observability, and orchestration.
Python agent-harnessagent-memoryagent-orchestrationai-agent-harnessai-agentsawesome-listcontext-engineeringharness-engineeringmcp
Add three high-quality 2026 harness engineering resources
ai-boost2d34f7f07f46
diff --git a/README.md b/README.mdindex 90a2307..9637a1a 100644--- a/README.md+++ b/README.md@@ -82,6 +82,7 @@ Canonical essays that define what harness engineering is and why it matters.- [How We Build Azure SRE Agent with Agentic Workflows](https://techcommunity.microsoft.com/blog/appsonazureblog/how-we-build-azure-sre-agent-with-agentic-workflows/4508753) — Architecture walkthrough of Microsoft's agent that has handled 35,000+ production incidents autonomously, reducing Azure App Service time-to-mitigation from 40.5 hours to 3 minutes. Documents the integration of MCP tools, telemetry, code repositories, and incident management platforms into a single agent harness with human-in-the-loop governance. The most data-backed production harness case study published in 2026.- [Context Engineering for Reliable AI Agents: Lessons from Building Azure SRE Agent](https://techcommunity.microsoft.com/blog/appsonazureblog/context-engineering-lessons-from-building-azure-sre-agent/4481200/) — Microsoft's account of shifting from 100+ bespoke tools and a prescriptive prompt to a filesystem-based context engineering system for their SRE agent. Key finding: exposing everything (source code, runbooks, query schemas, past investigation notes) as files and letting the agent use `read_file`, `grep`, `find`, and `shell` outperformed specialized tooling — "Intent Met" score rose from 45% to 75% on novel incidents.- [Harness Engineering: Structured Workflows for AI-Assisted Development](https://developers.redhat.com/articles/2026/04/07/harness-engineering-structured-workflows-ai-assisted-development) — Red Hat's enterprise perspective on harness engineering (April 7, 2026): AI writes better code when you design the environment it works in. Emphasizes structured context over free-form tickets, expanding the agent's toolbox through MCP integrations (CI status, deployment logs, runtime metrics) as real data sources, and a four-pillar model (vibes, specs, skills, agents) for organizing how humans and agents collaborate.+- [Harness engineering for coding agent users](https://martinfowler.com/articles/harness-engineering.html) — Birgitta Böckeler's systematic mental model (April 2026) for coding-agent harnesses, framing them as feedforward guides plus feedback sensors that self-correct before output reaches human eyes. Distinguishes computational controls (linters, tests) from inferential ones (LLM-as-judge), and argues that harnessability should become a first-class criterion in technology and architecture decisions.---@@ -325,6 +326,7 @@ Real repositories worth studying — each with a note on *why* it's worth your t- [Community-Powered Security with AI: An Open Source Framework for Security Research](https://github.blog/security/community-powered-security-with-ai-an-open-source-framework-for-security-research/) — GitHub Security Lab's January 14, 2026 launch of Taskflow Agent is a strong example of security-specific harness engineering: encode expert workflows as reusable tasks, keep the framework open for audit, and let AI scale established security practice instead of improvising ad hoc scans. Worth including because it turns security research itself into a sharable harness layer rather than a one-off internal workflow.- [AnonymAI: Integrating Differential Privacy with LLM Agents](https://www.mdpi.com/1999-5903/18/1/41) — Framework for automating data anonymization in agent workflows. Directly addresses the harness problem of unintentional PII leakage through tool calls and memory writes — privacy enforcement moves from the agent (prompt-level trust) to the harness boundary (structural enforcement). Essential for regulatory compliance in EU, Canada, and emerging state-level privacy regimes.- [Fault Tolerance Patterns: OpenClaw Journey Six—Core Retry Loop and Seven-Layer Fault-Tolerance](https://tonylixu.medium.com/openclaw-journey-six-core-retry-loop-and-seven-layer-fault-tolerance-7a9ce03147e2) — Four-layer fault tolerance (retry with backoff → model fallback chains → error classification → checkpoint recovery) reduces unrecoverable failures from 23% to under 2% across agent systems. The layered approach is essential reading for hardening production agent harnesses against the combinatorial failure modes introduced by multiple tool calls and external dependencies.+- [Sandbox Agents | OpenAI API Docs](https://developers.openai.com/api/docs/guides/agents/sandboxes) — OpenAI's authoritative April 2026 guide to sandbox architecture in the Agents SDK. The core principle is strict separation between the harness control plane (auth, billing, orchestration) and the sandbox compute plane (files, shell, ports), with manifest contracts, resumable session state, and sandbox-native memory.- [NVIDIA OpenShell](https://github.com/NVIDIA/OpenShell) — Open-source policy-driven sandbox runtime for autonomous AI agents, announced at GTC 2026. Enforces security constraints at the kernel level via Landlock LSM (filesystem), seccomp BPF (syscalls), and an OPA/Rego-evaluated HTTP CONNECT proxy (network) — constraints are enforced on the environment itself, so even a compromised agent cannot override them. Supports Claude Code, Codex, Cursor, and OpenCode inside the sandbox. - [Microsoft Agent Governance Toolkit](https://github.com/microsoft/agent-governance-toolkit) — Seven-package, multi-language (Python, Rust, TypeScript, Go, .NET) runtime security toolkit that addresses all 10 OWASP Agentic AI risks with deterministic, sub-millisecond policy enforcement. Includes Agent OS (policy engine intercepting every action), Agent Mesh (secure agent-to-agent communication), and Agent Runtime (dynamic execution rings). Hooks into LangChain callbacks, CrewAI task decorators, Google ADK plugins, and Microsoft Agent Framework middleware. - [Cloudflare Dynamic Workers](https://blog.cloudflare.com/dynamic-workers/) — V8 isolate-based sandboxing for AI-agent-generated code execution, now in open beta. Isolates start in milliseconds using megabytes of memory — 100x faster and up to 100x more memory-efficient than containers. The sandbox intercepts outbound HTTP requests for credential injection so agent code never touches secrets directly. A fundamentally different architectural option from container-based sandboxes (E2B, Daytona).@@ -377,6 +379,7 @@ Reusable starting points for harness artifacts. Copy and adapt.- [FinOps for Agents: Loop Limits, Tool-Call Caps, and the New Unit Economics of Agentic SaaS](https://www.infoworld.com/article/4138748/finops-for-agents-loop-limits-tool-call-caps-and-the-new-unit-economics-of-agentic-saas.html) — Defines five concrete budget guardrails enforced at the infrastructure gateway: loop/step limits, tool-call caps, per-run token budgets, wall-clock timeouts, and per-tenant budgets with anomaly alerts. Introduces Cost-per-Accepted-Outcome (CAPO) as the right unit economic metric for agent harnesses — shifting cost measurement from tokens consumed to business value delivered.- [Backtesting AI Agents: How SRE Teams Prove Reliability Before Production](https://drdroid.io/blog/backtesting-ai-agents-how-sre-teams-prove-reliability-before-production) — Formalizes agent validation as infrastructure-grade testing with pass^k reliability (all 20+ trials must succeed) rather than pass@k (one success). Defines five measurable dimensions (consistency, robustness, predictability, safety, cost stability) with specific SLO thresholds. Recommends dataset composition of 20% golden paths, 30% edge cases, 20% adversarial, 30% regression from production incidents.- [How My Agents Self-Heal in Production](https://blog.langchain.com/production-agents-self-heal/) — A concrete April 3, 2026 production pattern for closing the post-deploy loop: detect regressions, attribute whether the last deploy caused them, then dispatch a coding agent to open a fix PR automatically. This belongs here because it turns evals and observability into an active remediation harness, not just a dashboard humans are expected to watch.+- [Minions: Stripe's one-shot, end-to-end coding agents—Part 2](https://stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents-part-2) — Stripe's deep-dive into their unattended minion harness shipping 1,300+ PRs/week: "blueprints" interleave deterministic code nodes with agentic subtasks, a centralized 500-tool MCP server (Toolshed) serves the whole fleet, and pre-warmed devboxes prove that investments in human developer productivity pay equal dividends for agents.- [Amazon Bedrock AgentCore](https://aws.amazon.com/bedrock/agentcore/) — AWS's fully managed agent deployment platform providing serverless runtime with session isolation, built-in memory (session + long-term), secure gateway for tool access, browser runtime, and code interpreter — all framework-agnostic. Now supports AG-UI protocol for real-time agent-to-frontend streaming, VPC/PrivateLink for enterprise security, and CloudFormation for infrastructure-as-code deployments. The reference cloud-native agent hosting platform for teams that need managed infrastructure rather than building their own.- [AWS Agent Registry for Centralized Agent Discovery and Governance](https://aws.amazon.com/about-aws/whats-new/2026/04/aws-agent-registry-in-agentcore-preview/) — AWS's April 9, 2026 preview adds a missing production primitive: a governed catalog for agents, tools, skills, MCP servers, and custom resources with approval workflows, audit trails, and MCP-accessible discovery. Worth including because large organizations do not just need to run agents safely; they need to know what agent capabilities already exist so teams stop rebuilding the same scaffolding in silos.- [A Dev's Guide to Production-Ready AI Agents](https://cloud.google.com/blog/products/ai-machine-learning/a-devs-guide-to-production-ready-ai-agents) — Google Cloud's developer guide (April 2026) for moving AI agents from prototype to production using ADK, Vertex AI Agent Engine for managed hosting, and Cloud Run for serverless deployment. Covers the full production stack: agent development patterns, scaling considerations, security and identity requirements, and operational monitoring — the most concrete first-party guide for deploying agents on Google Cloud infrastructure.