Harness engineering beginner tutorial, from 0 to 1
TypeScript agentagenticagentic-aiaiai-agentai-agentsdshdsh-pluginharnessharness-engineeringharness-frameworkllm
refactor: Add Portuguese continuity lecture resources and session simulator
João Paulo Fricks5651cdc4f7d6
diff --git a/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/continuity-checklist.md b/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/continuity-checklist.mdnew file mode 100644index 0000000..65216b0--- /dev/null+++ b/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/continuity-checklist.md@@ -0,0 +1,6 @@+# Checklist de Continuidade++- Um agente iniciando do zero consegue identificar o trabalho mais recente em menos de cinco minutos?+- O fluxo atual de inicialização estável da aplicação está documentado?+- O trabalho inacabado está claramente identificado?+- A próxima melhor tarefa está visível sem a necessidade de ler conversas antigas?diff --git a/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/index.md b/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/index.mdnew file mode 100644index 0000000..04058ea--- /dev/null+++ b/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/index.md@@ -0,0 +1,7 @@+# Código da Aula 05++Use esta pasta para exemplos de:++- tarefas interrompidas entre múltiplas sessões;+- ausência de artefatos de continuidade;+- padrões de recuperação de continuidade.diff --git a/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/session-handoff.md b/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/session-handoff.mdnew file mode 100644index 0000000..b5e3612--- /dev/null+++ b/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/session-handoff.md@@ -0,0 +1,17 @@+# Exemplo de Handoff de Sessão++## Concluído++- Adicionado suporte à importação de arquivos Markdown (`.md`)+- Adicionada uma lista básica de documentos na interface (*renderer*)++## Quebrado ou Não Verificado++- A importação funciona para arquivos `.md`, mas falha para arquivos `.txt` grandes+- A aplicação inicia corretamente, porém a visualização de detalhes do documento ainda não foi integrada++## Próximo Melhor Passo++- Corrigir o fluxo de importação para arquivos `.txt`+- Validar o processo completo de importação de ponta a ponta+- Em seguida, implementar o painel de detalhes do documentodiff --git a/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/session-simulator.ts b/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/session-simulator.tsnew file mode 100644index 0000000..e6c63a3--- /dev/null+++ b/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/session-simulator.ts@@ -0,0 +1,221 @@+/**+ * session-simulator.ts+ *+ * Simula duas sessões trabalhando em uma tarefa de múltiplas etapas.+ * Execução 1: Sem arquivo de handoff — a Sessão B começa do zero e duplica trabalho.+ * Execução 2: Com arquivo de handoff — a Sessão B continua de onde a Sessão A parou.+ *+ * Executar:+ * npx tsx docs/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/session-simulator.ts+ */++// ---------------------------------------------------------------------------+// Tipos+// ---------------------------------------------------------------------------++interface TaskStep {+ id: number;+ name: string;+ durationMs: number;+}++interface SessionResult {+ session: string;+ stepsCompleted: number;+ totalDurationMs: number;+ duplicatedSteps: number;+ output: string[];+}++// ---------------------------------------------------------------------------+// Definição da tarefa+// ---------------------------------------------------------------------------++const taskSteps: TaskStep[] = [+ { id: 1, name: "Ler a estrutura do projeto", durationMs: 50 },+ { id: 2, name: "Compreender o módulo de autenticação existente", durationMs: 80 },+ { id: 3, name: "Projetar novo endpoint de busca", durationMs: 60 },+ { id: 4, name: "Implementar endpoint de busca", durationMs: 100 },+ { id: 5, name: "Escrever testes de integração", durationMs: 70 },+ { id: 6, name: "Atualizar documentação", durationMs: 40 },+];++// ---------------------------------------------------------------------------+// Executor de sessão simulada+// ---------------------------------------------------------------------------++/** Simula uma sessão executando trabalho. */+function runSession(+ sessionName: string,+ startStep: number,+ endStep: number+): SessionResult {+ const output: string[] = [];+ let totalDuration = 0;++ for (let i = startStep; i <= endStep; i++) {+ const step = taskSteps.find((s) => s.id === i)!;+ totalDuration += step.durationMs;+ output.push(+ ` [${sessionName}] Etapa ${step.id}: ${step.name} (${step.durationMs}ms)`+ );+ }++ return {+ session: sessionName,+ stepsCompleted: endStep - startStep + 1,+ totalDurationMs: totalDuration,+ duplicatedSteps: 0,+ output,+ };+}++// ---------------------------------------------------------------------------+// Execução 1: Sem arquivo de handoff+// ---------------------------------------------------------------------------++function simulateNoHandoff(): {+ sessionA: SessionResult;+ sessionB: SessionResult;+} {+ // A Sessão A executa as etapas 1-3 antes de atingir o limite de contexto+ const sessionA = runSession("Sessão A", 1, 3);++ // A Sessão B não possui contexto e começa do zero+ const sessionB = runSession("Sessão B", 1, 6);+ sessionB.duplicatedSteps = 3; // Refaz as etapas 1-3 já concluídas pela Sessão A++ return { sessionA, sessionB };+}++// ---------------------------------------------------------------------------+// Execução 2: Com arquivo de handoff+// ---------------------------------------------------------------------------++function simulateWithHandoff(): {+ sessionA: SessionResult;+ sessionB: SessionResult;+} {+ // A Sessão A executa as etapas 1-3 e grava um arquivo de handoff+ const sessionA = runSession("Sessão A", 1, 3);++ // A Sessão B lê o handoff e continua a partir da etapa 4+ const sessionB = runSession("Sessão B", 4, 6);+ sessionB.duplicatedSteps = 0;++ return { sessionA, sessionB };+}++// ---------------------------------------------------------------------------+// Relatórios+// ---------------------------------------------------------------------------++function pad(s: string, len: number): string {+ return s.length >= len ? s : s + " ".repeat(len - s.length);+}++function printRun(+ title: string,+ result: { sessionA: SessionResult; sessionB: SessionResult }+): void {+ console.log("\n" + "=".repeat(80));+ console.log(` ${title}`);+ console.log("=".repeat(80));++ console.log("\n Saída da Sessão A:");+ for (const line of result.sessionA.output) {+ console.log(line);+ }++ console.log("\n Saída da Sessão B:");+ for (const line of result.sessionB.output) {+ console.log(line);+ }++ console.log("\n Observações da Sessão B:");+ if (result.sessionB.duplicatedSteps > 0) {+ console.log(+ ` - Refez ${result.sessionB.duplicatedSteps} etapas já concluídas pela Sessão A`+ );+ console.log(+ ` - Desperdiçou ${+ result.sessionB.totalDurationMs -+ taskSteps.slice(3).reduce((s, t) => s + t.durationMs, 0)+ }ms em trabalho duplicado`+ );+ } else {+ console.log(" - Leu o arquivo de handoff e continuou a partir da etapa 4");+ console.log(" - Nenhum trabalho duplicado foi realizado");+ }+}++function printComparison(): void {+ const noHandoff = simulateNoHandoff();+ const withHandoff = simulateWithHandoff();++ printRun("EXECUÇÃO 1: SEM ARQUIVO DE HANDOFF", noHandoff);+ printRun("EXECUÇÃO 2: COM ARQUIVO DE HANDOFF", withHandoff);++ // Tabela comparativa+ console.log("\n" + "=".repeat(80));+ console.log(" TABELA COMPARATIVA");+ console.log("=".repeat(80) + "\n");++ const header = `| ${pad("Métrica", 35)}| ${pad(+ "Sem Handoff",+ 15+ )}| ${pad("Com Handoff", 15)}|`;+ const sep = `|${"-".repeat(37)}|${"-".repeat(17)}|${"-".repeat(17)}|`;++ console.log(header);+ console.log(sep);++ const noTotal =+ noHandoff.sessionA.totalDurationMs ++ noHandoff.sessionB.totalDurationMs;++ const withTotal =+ withHandoff.sessionA.totalDurationMs ++ withHandoff.sessionB.totalDurationMs;++ console.log(+ `| ${pad("Etapas concluídas pela Sessão A", 35)}| ${pad(+ String(noHandoff.sessionA.stepsCompleted),+ 15+ )}| ${pad(String(withHandoff.sessionA.stepsCompleted), 15)}|`+ );++ console.log(+ `| ${pad("Etapas concluídas pela Sessão B", 35)}| ${pad(+ String(noHandoff.sessionB.stepsCompleted),+ 15+ )}| ${pad(String(withHandoff.sessionB.stepsCompleted), 15)}|`+ );++ console.log(+ `| ${pad("Etapas duplicadas", 35)}| ${pad(+ String(noHandoff.sessionB.duplicatedSteps),+ 15+ )}| ${pad(String(withHandoff.sessionB.duplicatedSteps), 15)}|`+ );++ console.log(+ `| ${pad("Trabalho total (ms)", 35)}| ${pad(+ String(noTotal),+ 15+ )}| ${pad(String(withTotal), 15)}|`+ );++ console.log(+ `| ${pad("Tempo economizado com handoff", 35)}| ${pad(+ "-",+ 15+ )}| ${pad(String(noTotal - withTotal) + "ms", 15)}|`+ );++ console.log(+ "\n Um arquivo de handoff elimina trabalho duplicado e garante continuidade entre sessões.\n"+ );+}++printComparison();\ No newline at end of filediff --git a/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md b/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.mdnew file mode 100644index 0000000..7cbbc3c--- /dev/null+++ b/docs/pt-BR/lectures/lecture-05-why-long-running-tasks-lose-continuity/index.md@@ -0,0 +1,179 @@+[中文版 →](../../../zh/lectures/lecture-05-why-long-running-tasks-lose-continuity/)++> Exemplos de código: [code/](https://github.com/walkinglabs/learn-harness-engineering/blob/main/docs/en/lectures/lecture-05-why-long-running-tasks-lose-continuity/code/)+> Projeto prático: [Projeto 03. Continuidade entre múltiplas sessões](./../../projects/project-03-multi-session-continuity/index.md)++# Aula 05. Mantendo o Contexto Vivo Entre Sessões++Você pede ao Claude Code para implementar uma funcionalidade completa. Ele trabalha por 30 minutos, realiza a maior parte da tarefa, mas o contexto está se esgotando. Você inicia uma nova sessão para continuar — e descobre que ele não se lembra das decisões tomadas anteriormente, por que a opção A foi escolhida em vez da opção B, quais arquivos já foram modificados ou em que estado os testes se encontram. Ele gasta mais 15 minutos explorando novamente o projeto e pode até seguir uma abordagem diferente da utilizada anteriormente.++Esse é o dilema real que agentes de IA enfrentam em tarefas que se estendem por múltiplas sessões. Nesta aula, veremos por que os agentes "perdem o fio da meada" durante tarefas longas e como a persistência estruturada de estado permite que uma nova sessão retome rapidamente o trabalho de onde a anterior parou.++## Janelas de Contexto Não São Infinitas++As janelas de contexto são finitas. Esse não é um problema que será resolvido apenas com a evolução dos modelos — mesmo que o tamanho das janelas cresça para 1 milhão de tokens, tarefas complexas continuarão sendo capazes de esgotá-las. Os agentes não apenas geram código; eles também precisam compreender bases de código, acompanhar seu próprio histórico de decisões, processar saídas de ferramentas e manter o contexto da conversa. Toda essa informação cresce mais rápido do que a expansão das janelas de contexto.++Existe um problema ainda mais profundo: as informações produzidas pelos agentes não possuem a mesma importância. Os passos intermediários de raciocínio contêm o **"porquê"** das decisões — por que a opção A foi escolhida em vez da B, por que determinada biblioteca foi selecionada, por que uma otimização específica foi descartada. O resultado final contém apenas o **"o quê"** — o código em si. Estratégias de compactação normalmente preservam o segundo, mas perdem o primeiro. A próxima sessão consegue ver o código, mas não entende por que ele foi escrito daquela forma e pode acabar "otimizando" uma decisão de design que foi tomada deliberadamente.++A Anthropic observou algo interessante em suas pesquisas sobre agentes de longa duração: quando os agentes percebem que o contexto está ficando escasso, eles tendem a apresentar um comportamento de **"finalização apressada"** (*rushed finish*). Eles tentam concluir o trabalho rapidamente, pulam etapas de verificação ou escolhem soluções mais simples em vez das melhores opções disponíveis. A Anthropic chama esse fenômeno de **"ansiedade de contexto"** (*context anxiety*).++## Fluxo de Continuidade Entre Sessões++Sem arquivos de persistência de estado, toda nova sessão precisa começar do zero:+++```mermaid+flowchart LR+ S1["Session 1<br/>feature is half done"] --> End1["Context is nearly full<br/>session ends"]+ End1 --> S2["Session 2 starts fresh"]+ S2 --> Guess["Re-read folders, rerun tests,<br/>guess why the code was written this way"]+ Guess --> Drift["Work gets repeated<br/>and recovery is slow"]+```++With state persistence files, new sessions can pick up quickly:++```mermaid+flowchart LR+ Work["Session 1 work"] --> Progress["PROGRESS.md<br/>done / in progress / next step"]+ Work --> Decisions["DECISIONS.md<br/>why this approach was chosen"]+ Work --> Verify["Verification notes<br/>which tests pass and fail"]+ Work --> Commit["Git checkpoint<br/>exact repo state"]++ Progress --> Rebuild["Session 2 rebuild"]+ Decisions --> Rebuild+ Verify --> Rebuild+ Commit --> Rebuild++ Rebuild --> Resume["New session picks up quickly"]+```++## Core Concepts++- **Context windows are finite**: No matter what window size is claimed (128K, 200K, 1M), long tasks will eventually exhaust them. After exhaustion, either compaction (losing information) or reset (starting a new session) is required — both lose something.+- **State persistence files**: Persisted state files that let a new session unambiguously resume where the last one left off. The most basic form includes progress logs, verification records, and next actions.+- **Rebuild cost**: The time a new session needs to reach an executable state. A good harness can compress rebuild cost from 15 minutes to 3 minutes.+- **Drift**: The gap between the agent's understanding and the actual state of the code repository. Every session boundary introduces drift; without control, it compounds session after session.+- **Context anxiety**: A phenomenon observed by Anthropic — agents exhibit rushed finish behavior when approaching context limits, ending tasks early to avoid information loss. At its core, it's an irrational resource anxiety.+- **Compaction vs. reset**: Compaction summarizes context within the same session (keeps "what," may lose "why"); reset opens a new session rebuilding from persisted state (clean but depends on artifact completeness).++## What Happens When Continuity Breaks++The previous session spent significant context budget analyzing three approaches and choosing option B. This session's agent doesn't know about that analysis and might re-decide based on incomplete information — potentially choosing option A. Same information, different conclusion, because the decision-making context is gone.++Even worse is duplicate work. The agent isn't sure whether certain work was already completed and does it again. Or worse — does half of it, discovers a conflict with the existing implementation, and has to rework. Without progress records, the new session has no idea what's already been done.++Over several sessions, the implementation direction may have silently drifted from the original requirements. Each new session has a slightly different understanding of the project goals. Each deviation compounds on the last, and the final result may be far removed from the original intent.++There's also the verification gap. The previous session's verification results (which tests pass, which fail, why they fail) weren't recorded. The new session has to re-run all verification to understand the current state. Every session re-diagnoses from scratch, every time wasting precious context.++Both OpenAI and Anthropic emphasize structured state persistence in their documentation. OpenAI's harness engineering article treats the repository as an "operational record" — every operation's results should leave traceable evidence in the repo. Anthropic's long-running agents documentation specifically recommends "handoff files" — structured documents containing current state, known issues, and next actions.++## Practical Approaches to State Persistence++Core approach: **Treat the agent like an engineer whose short-term memory gets wiped at every session.** Before it "clocks out," it must write down critical information so the next "shift" agent can pick up quickly.++**Tool 1: Progress file (PROGRESS.md).** The most basic state persistence file:++```markdown+# Project Progress++## Current State+- Latest commit: abc1234 (feat: add user preferences endpoint)+- Test status: 42/43 passing (test_pagination_edge_case failing)+- Lint: passing++## Completed+- [x] User model and database migration+- [x] Basic CRUD endpoints+- [x] Auth middleware integration++## In Progress+- [ ] Pagination feature (90% - edge case test failing)++## Known Issues+- test_pagination_edge_case returns 500 on empty result sets+- Need to confirm whether deleted users should appear in listings++## Next Steps+1. Fix pagination edge case bug+2. Add "include deleted users" query parameter+3. Update API documentation+```++**Tool 2: Decision log (DECISIONS.md).** Record important design decisions and reasons. No need for detailed design documents — just "what decision, why, when":++```markdown+# Design Decisions++## 2024-01-15: Use Redis for user preferences caching+- Reason: High read frequency (every API call), small data size+- Rejected alternative: PostgreSQL materialized view (high change frequency makes maintenance cost not worthwhile)+- Constraint: Cache TTL of 5 minutes, active invalidation on write+```++**Tool 3: Git commits as checkpoints.** Commit after completing each atomic unit of work. Commit messages should explain what was done and why. These are free, automatically versioned state snapshots.++**Tool 4: init.sh or harness initialization flow.** Specify in `AGENTS.md` the "clock-in" and "clock-out" routines:++```markdown+## At session start (clock in)+1. Read PROGRESS.md for current state+2. Read DECISIONS.md for important decisions+3. Run make check to confirm repo is in consistent state+4. Continue from PROGRESS.md "Next Steps" section++## Before session end (clock out)+1. Update PROGRESS.md+2. Run make check to confirm consistent state+3. Commit all completed work+```++**Mixed strategy**: Not every task needs a context reset. Short tasks (under 30 minutes) can complete within one session. Long tasks (spanning sessions) must use progress files and decision logs for continuity. Decision criterion: if a task needs more than 60% of the window, start preparing the handoff.++### A Deeper Look at Context Anxiety++Anthropic's March 2026 research further revealed the specific manifestations of context anxiety: on Sonnet 4.5, when context approaches the window limit, the agent shows strong "rushed finish" behavior.++Two strategies address this:++**Compaction**: Summarizing early conversation within the same session. Advantage: maintains continuity, the agent can see "what." Disadvantage: "why" is often lost in summaries — why option B was chosen over A, why a particular optimization was skipped. More critically, compaction doesn't eliminate context anxiety — the agent knows context was once large, and psychologically still tends to rush to finish.++**Context reset**: Completely clearing context, opening a new session, rebuilding from persisted artifacts. Advantage: clean mental state — the new session has no "I'm running out of time" anxiety. Disadvantage: depends on the completeness of handoff artifacts. If the progress file is missing critical information, the new session may waste time going in the wrong direction.++Anthropic's actual data: for Sonnet 4.5, context anxiety is severe enough that compaction alone isn't sufficient — context reset becomes a critical component of harness design. But for Opus 4.5, this behavior is greatly diminished, and compaction can manage context without relying on resets. This means: **harness design needs specific understanding of the target model, not a one-size-fits-all template.**++> Source: [Anthropic: Harness design for long-running application development](https://www.anthropic.com/engineering/harness-design-long-running-apps)++## Real-World Example++An agent was tasked with implementing a blog system with user authentication — 12 feature points, estimated 5 sessions needed.++**Baseline without state persistence files**: Session 1 implemented the user model and basic routes. Session 2 started without the agent remembering the auth middleware's interface contract, spending ~15 minutes inferring the previous design intent. By session 3, accumulated drift caused the agent to start reimplementing already-completed features. By session 5, the repo contained lots of redundant code but the core auth feature still hadn't passed end-to-end tests. Only 7 of 12 feature points completed, 3 with hidden correctness issues.++**With state persistence files**: Using progress files, decision logs, verification records, and git checkpoints. State report updated automatically at each session end. Session 2's rebuild cost dropped to ~3 minutes. By session 5, all 12 feature points completed and verified.++Quantitative comparison: rebuild time reduced ~78%, feature completion rate from 58% to 100%, hidden defect rate from 43% down to 8%.++## Key Takeaways++- Context windows are a finite resource. Long tasks will span sessions, and sessions will lose information — this is objective reality.+- The solution isn't bigger windows — it's better state persistence. Progress files, decision logs, and git checkpoints work together to let new sessions pick up where previous ones left off.+- Treat the agent like an engineer whose short-term memory gets wiped every session: before "clocking out," write down what was done, why, and what's next.+- Rebuild cost is the key metric. A good harness should get new sessions to an executable state within 3 minutes.+- Mixed strategy: short tasks within sessions, long tasks with structured artifacts for continuity.++## Further Reading++- [Anthropic: Effective Harnesses for Long-Running Agents](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents)+- [OpenAI: Harness Engineering](https://openai.com/index/harness-engineering/)+- [Lost in the Middle: How Language Models Use Long Contexts](https://arxiv.org/abs/2307.03172)+- [Claude Code Documentation](https://docs.anthropic.com/en/docs/claude-code)+- [HumanLayer: Harness Engineering for Coding Agents](https://humanlayer.dev/articles/harness-engineering-for-coding-agents/)++## Exercises++1. **State persistence measurement**: Pick a development task needing at least 3 sessions. Without providing any state persistence files, record at each session start how much context the agent spends "figuring out what happened last time." After each session, create a progress file and let the next session start from it. Compare rebuild costs with and without progress files.++2. **Handoff template design**: Design a minimal handoff template with four fields: repo state (commit hash), runtime state (test pass rate), blockers, next actions. Let a completely fresh agent session restore project state using only this template. Record ambiguities encountered during restoration, iterate to improve the template.++3. **Mixed strategy experiment**: In a 5-session development task, compare three strategies: (a) always start fresh sessions + progress files, (b) do as much as possible in one session (context compaction), (c) mixed strategy (short tasks in-session, long tasks across sessions + progress files). Compare rebuild time, feature completion rate, and decision consistency.