The one and only agent harness for complex codebases. Project memory, planning, execution, and verified completion inside Codex.
Rust Debugging
Covers cargo, tokio, panics, and the fact that you usually don't actually need a debugger — Rust's type system, dbg!, and logging cover 80% of sessions faster than gdb would.
Environment detection (Phase 0)
cargo --version
rustc --version
cat rust-toolchain.toml rust-toolchain 2>/dev/null
cat Cargo.toml | head -30
# Debuggers available
which rust-gdb
which rust-lldb
which lldb
# Async infrastructure
grep -E '"(tokio|async-std|smol)"' Cargo.toml
# Profile flags
grep -E '^\[profile' Cargo.toml
The default cargo run builds with dev profile which includes debug symbols. cargo run --release strips them. For debugging, stay in dev unless the bug only manifests under optimization.
The Rust debugging hierarchy (use in this order)
Rust's ecosystem has a specific order that's faster than reaching for gdb first:
dbg!(expr)macro — for a single value at a specific spot. Prints file:line + value, returns the value unchanged so you can inline it. Faster than a debugger for 60% of bugs.RUST_LOG=tracewithtracing/env_logger— for flow and state across an operation. Zero code change in dev-time.RUST_BACKTRACE=1/=full— for crashes. Almost always sufficient; you rarely need a live debugger for a panic.rust-gdb/rust-lldb— when you need to pause execution and inspect memory, especially for unsafe code or FFI.tokio-console— for async deadlocks, stuck tasks, hot loops.cargo-expand— when a macro is doing something weird.
Reach for the lightest tool that answers the hypothesis.
dbg! — the underused macro
let x = 5;
let y = dbg!(x * 2); // prints: [src/main.rs:2] x * 2 = 10
Inside a complex expression:
let total = items.iter().filter(|i| i.active).map(|i| dbg!(i.cost)).sum::<u64>();
Multiple values at once:
dbg!(&user, &request, elapsed.as_millis());
dbg! writes to stderr, so it won't corrupt stdout-based pipelines. Journal each dbg! you add; revert at Phase 9.
RUST_LOG for flow-level debugging
If the codebase uses tracing or env_logger:
RUST_LOG=debug cargo run
RUST_LOG=trace cargo run # very verbose
RUST_LOG=my_crate=trace,hyper=info cargo run # per-module level
RUST_LOG=debug,tokio=off cargo run # silence noisy crates
For tracing-based apps, instrument with spans:
#[tracing::instrument]
fn handle_request(req: &Request) -> Response { ... }
This gives you structured per-call entry/exit logs with args and timing, zero additional code in the body.
RUST_BACKTRACE for panics
RUST_BACKTRACE=1 cargo run # backtrace on panic
RUST_BACKTRACE=full cargo run # include libstd/tokio frames
The panic itself usually tells you the file:line. The backtrace tells you how it got there. Between the two, most crash bugs are solved without a debugger.
rust-gdb / rust-lldb
Launch
# Build with debug symbols (default dev profile)
cargo build
# Attach gdb wrapper (applies Rust type pretty-printers)
rust-gdb ./target/debug/my_binary
# Or lldb:
rust-lldb ./target/debug/my_binary
# With args
rust-gdb --args ./target/debug/my_binary arg1 arg2
# Attach to running process
rust-gdb -p $(pgrep my_binary)
Breakpoints
Rust symbols are mangled. Use either:
(gdb) b main # main function
(gdb) b my_crate::module::function # canonical path
(gdb) b src/handler.rs:42 # file:line
(gdb) info functions my_function # find mangled name
State inspection
(gdb) p x # print value (uses Rust pretty-printer for Vec, Option, HashMap, etc.)
(gdb) p *ptr # deref
(gdb) info locals # all locals in current frame
(gdb) info args # function args
(gdb) bt # backtrace
(gdb) frame <n> # switch to stack frame n
(gdb) watch my_var # stop when my_var changes
(gdb) rbreak regex # breakpoint all functions matching regex
Pair with pwndbg for better layout on native bugs — see tools/pwndbg.md. Pwndbg works with rust-gdb too.
Sanitizers — memory and race bugs
Nightly Rust exposes LLVM sanitizers through -Z sanitizer. Use them when a crash,
data race, or leak is otherwise too timing- or state-dependent to explain from a
backtrace; they replace guesswork with a report at the invalid operation.
# AddressSanitizer: out-of-bounds accesses and use-after-free
cargo +nightly run -Z sanitizer=address
# ThreadSanitizer: data races between threads
cargo +nightly test -Z sanitizer=thread
# LeakSanitizer: allocations that remain reachable at process exit
cargo +nightly run -Z sanitizer=leak
address catches invalid memory access and use-after-free, thread catches unsynchronized
concurrent accesses, and leak reports memory that was allocated but not released. Run
the invocation that matches the suspected class; sanitizer diagnostics are especially
useful before attaching a debugger to optimized or timing-sensitive code.
rr — time-travel debugging
On supported Linux systems, record the failure once, then replay it deterministically and
run backwards from the failure with reverse-continue or reverse-step:
rr record ./target/debug/my_binary
rr replay
# (rr) reverse-continue
# (rr) reverse-step
rr is Linux/x86-64 only. It does not apply on Apple Silicon, including arm64 macOS;
use rust-lldb or a sanitizer there instead.
tokio-console — async task debugging
For tokio-based async apps, this is essential when tasks are stuck or leaking.
# Add tokio-console instrumentation to the target binary
# In Cargo.toml:
# [dependencies]
# console-subscriber = "0.2"
# In main.rs:
# console_subscriber::init();
# Build with tokio_unstable:
RUSTFLAGS="--cfg tokio_unstable" cargo run
# In another terminal:
tokio-console # connects to default port 6669
Shows live tasks, their state, wake counts, poll durations, parent tasks. The single fastest way to find "why is my async thing stuck".
cargo-expand — when a macro is suspect
cargo install cargo-expand
cargo expand # expand all macros in the crate
cargo expand my::module::path # scope to one item
If you suspect a macro (especially #[derive], #[tokio::main], #[async_trait]) is generating code that doesn't match your mental model, this shows you exactly what the compiler sees.
Release-build gotcha
cargo build --release # no debug symbols by default
If the bug only shows up in --release:
# Cargo.toml
[profile.release]
debug = true # add symbols, keep optimizations
Now rust-gdb ./target/release/my_binary works on release builds. This is required when optimization-enabled codegen bugs (inlining, LLVM folding) are suspected.
Silent-failure patterns in Rust
| Pattern | Why it's silent |
|---|---|
.unwrap_or_default() |
Masks errors as the zero value |
.unwrap_or(fallback) |
Same, with a specific fallback |
let _ = fallible_operation() |
Explicitly discards the Result, no compiler warning |
if let Ok(x) = ... { use(x); } // no else |
Silent on Err |
.ok() chaining |
Converts Result to Option, throwing the error away |
Panic inside a tokio task not .awaited |
Task dies silently; runtime usually logs but it's quiet if logs are off |
eprintln! that goes to a redirected-null stderr |
Looks like nothing happened |
Drop impl that panics under specific condition |
Double-panic aborts process silently if no logging configured |
Phase 9 cleanup specifics
# Revert dbg! macro additions
git diff | grep -E 'dbg!\('
# For any file with dbg! additions:
git checkout <file>
# Unset env vars
unset RUST_LOG RUST_BACKTRACE
# Kill any rust-gdb/lldb sessions
pkill -f 'rust-gdb' || true
pkill -f 'rust-lldb' || true
pkill -f '^lldb ' || true
# tokio-console binds 6669 by default
lsof -iTCP:6669 -sTCP:LISTEN -nP 2>/dev/null