Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

fix(code): stop TS export duplication + comment displacement (#1906)

Hafiz Ismailadf8fed9bd75
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 718cd844c..0f017f9ce 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -105,6 +105,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* **learn:** fix three Windows-specific failures in `headroom learn --verbosity` and CLI-backed analysis ([#1624](https://github.com/headroomlabs-ai/headroom/issues/1624)). `verbosity.py` read transcripts and profiles with the platform-default text codec instead of UTF-8, so non-ASCII content raised a silently-caught `UnicodeDecodeError`, producing `Sessions: 0, human turns: 0` for every project. `_greedy_path_decode` listed a directory's children with `is_dir()` inline in the same expression as `iterdir()`, so a single `PermissionError` on an inaccessible sibling (e.g. the `AppData\Local\Temporary Internet Files` junction present on most Windows profiles) aborted the whole listing and silently mis-decoded any project path that walked through it, causing `--project <path>` to report "No matching project" or resolve the wrong directory. `_call_cli_llm` launched CLI backends via `Popen`/`run`, which use `CreateProcess` on Windows and don't apply the shell's `PATHEXT` extension search, so an npm-installed `.cmd` shim (e.g. `claude`, `codex`) raised `FileNotFoundError` even though it was on `PATH`; a `shutil.which`-based retry now resolves the shim.
* **proxy:** The Anthropic Messages route (`POST /v1/messages`) now honors the `x-headroom-base-url` per-request upstream override. It previously ignored the header and always forwarded to `api.anthropic.com`, so clients that speak the Anthropic Messages wire format while authenticating against a non-Anthropic gateway (e.g. OpenCode Zen) were rejected upstream with `401 invalid x-api-key`. The route now forwards to `<x-headroom-base-url>/v1/messages`, consistent with the OpenAI-compatible and passthrough routes ([#1760](https://github.com/headroomlabs-ai/headroom/issues/1760)).
* **proxy:** the savings store now fsyncs its parent directory after the atomic rename, so the most recent `proxy_savings.json` write survives a power-loss or crash. `_save_locked` fsynced the temp file's contents but never the directory entry the rename created, leaving the rename itself non-durable on POSIX. Best-effort — a no-op on Windows and virtual filesystems where directory fsync is unsupported.
+- **code:** fix two `CodeAwareCompressor` AST-reassembly bugs: an exported JS/TS function or class (`export function foo() {`) produced a duplicated `export export` keyword and invalid syntax, because line-based node slicing (used to preserve indentation) pulled in the preceding `export` sibling's text on top of the `export_statement` handler's own prefix reconstruction. Separately, in every supported language, a doc comment immediately above a top-level function, class, or type was detached from its declaration during extraction and re-emitted in a cluster at the end of the compressed output instead of staying attached to what it documents.
## [0.31.0](https://github.com/headroomlabs-ai/headroom/compare/v0.30.0...v0.31.0) (2026-07-09)
diff --git a/headroom/transforms/code_compressor.py b/headroom/transforms/code_compressor.py
index 0a3e0c0d9..8f006ea27 100644
--- a/headroom/transforms/code_compressor.py
+++ b/headroom/transforms/code_compressor.py
@@ -1235,18 +1235,21 @@ class CodeAwareCompressor(Transform):
# Package declarations (Go, Java)
if lang_config.package_node and node_type == lang_config.package_node:
- structure.imports.insert(0, _get_node_text(node, code))
+ leading = _get_leading_comment_text(node, code, captured_byte_ranges)
+ structure.imports.insert(0, leading + _get_node_text(node, code))
captured_byte_ranges.append((node.start_byte, node.end_byte))
return
# Import statements
if node_type in lang_config.import_nodes:
- structure.imports.append(_get_node_text(node, code))
+ leading = _get_leading_comment_text(node, code, captured_byte_ranges)
+ structure.imports.append(leading + _get_node_text(node, code))
captured_byte_ranges.append((node.start_byte, node.end_byte))
return
# Export statements (JS/TS) — may contain functions or re-exports
if node_type == "export_statement":
+ leading = _get_leading_comment_text(node, code, captured_byte_ranges)
text = _get_node_text(node, code)
# Check if this export wraps a function or class
has_func_or_class = False
@@ -1263,16 +1266,17 @@ class CodeAwareCompressor(Transform):
export_prefix = _slice_code_bytes(code, node.start_byte, child.start_byte)
export_suffix = _slice_code_bytes(code, child.end_byte, node.end_byte)
structure.function_signatures.append(
- export_prefix + compressed + export_suffix
+ leading + export_prefix + compressed + export_suffix
)
break
if not has_func_or_class:
- structure.imports.append(text)
+ structure.imports.append(leading + text)
captured_byte_ranges.append((node.start_byte, node.end_byte))
return
# Decorated definitions (Python)
if lang_config.decorator_node and node_type == lang_config.decorator_node:
+ leading = _get_leading_comment_text(node, code, captured_byte_ranges)
decorator_text = []
definition_compressed = None
for child in node.children:
@@ -1287,7 +1291,7 @@ class CodeAwareCompressor(Transform):
child, code, language, lang_config, body_limits, analysis
)
if decorator_text and definition_compressed:
- full_def = "\n".join(decorator_text) + "\n" + definition_compressed
+ full_def = leading + "\n".join(decorator_text) + "\n" + definition_compressed
# Route to correct list based on inner definition type
for child in node.children:
if child.type in lang_config.class_nodes:
@@ -1296,25 +1300,27 @@ class CodeAwareCompressor(Transform):
else:
structure.function_signatures.append(full_def)
elif definition_compressed:
- structure.function_signatures.append(definition_compressed)
+ structure.function_signatures.append(leading + definition_compressed)
captured_byte_ranges.append((node.start_byte, node.end_byte))
return
# Function/method definitions
if node_type in lang_config.function_nodes:
+ leading = _get_leading_comment_text(node, code, captured_byte_ranges)
compressed = self._compress_function_ast(
node, code, language, lang_config, body_limits, analysis
)
- structure.function_signatures.append(compressed)
+ structure.function_signatures.append(leading + compressed)
captured_byte_ranges.append((node.start_byte, node.end_byte))
return
# Class definitions — compress each method individually
if node_type in lang_config.class_nodes:
+ leading = _get_leading_comment_text(node, code, captured_byte_ranges)
compressed = self._compress_class_ast(
node, code, language, lang_config, body_limits, analysis
)
- structure.class_definitions.append(compressed)
+ structure.class_definitions.append(leading + compressed)
captured_byte_ranges.append((node.start_byte, node.end_byte))
trailing_semicolon = _get_same_line_trailing_semicolon(node)
if trailing_semicolon is not None:
@@ -1325,7 +1331,8 @@ class CodeAwareCompressor(Transform):
# Type definitions
if node_type in lang_config.type_nodes:
- structure.type_definitions.append(_get_node_text(node, code))
+ leading = _get_leading_comment_text(node, code, captured_byte_ranges)
+ structure.type_definitions.append(leading + _get_node_text(node, code))
captured_byte_ranges.append((node.start_byte, node.end_byte))
return
@@ -1374,8 +1381,7 @@ class CodeAwareCompressor(Transform):
# (methods inside classes).
code_lines = code.split("\n")
start_row = node.start_point[0]
- end_row = node.end_point[0]
- node_lines = code_lines[start_row : end_row + 1]
+ node_lines = _get_node_lines(node, code_lines)
node_text = "\n".join(node_lines)
func_name = _get_definition_name(node)
@@ -1640,8 +1646,7 @@ class CodeAwareCompressor(Transform):
# Use line-based extraction to preserve indentation
code_lines = code.split("\n")
start_row = node.start_point[0]
- end_row = node.end_point[0]
- node_lines = code_lines[start_row : end_row + 1]
+ node_lines = _get_node_lines(node, code_lines)
node_text = "\n".join(node_lines)
# Find the class/member container. For some languages this is not the
@@ -2029,6 +2034,58 @@ def _get_node_text(node: Any, code: str) -> str:
return _slice_code_bytes(code, node.start_byte, node.end_byte)
+_COMMENT_NODE_TYPES = frozenset({"comment", "line_comment", "block_comment"})
+
+
+def _get_leading_comment_text(
+ node: Any, code: str, captured_byte_ranges: list[tuple[int, int]]
+) -> str:
+ """Collect contiguous doc-comment siblings immediately preceding a node.
+
+ Doc comments are top-level siblings of the declaration they document, not
+ children of it. Left uncaptured, they fall through to the leftover
+ top-level sweep and get grouped separately from the declarations they
+ document instead of staying attached. Only comments with no blank line
+ before the node (or the next comment) are treated as attached.
+ """
+ comments: list[Any] = []
+ sibling = getattr(node, "prev_sibling", None)
+ anchor_row = node.start_point[0]
+ while (
+ sibling is not None
+ and sibling.type in _COMMENT_NODE_TYPES
+ and (anchor_row - sibling.end_point[0] <= 1)
+ ):
+ comments.append(sibling)
+ anchor_row = sibling.start_point[0]
+ sibling = getattr(sibling, "prev_sibling", None)
+ if not comments:
+ return ""
+ comments.reverse()
+ captured_byte_ranges.extend((c.start_byte, c.end_byte) for c in comments)
+ return "\n".join(_get_node_text(c, code) for c in comments) + "\n"
+
+
+def _get_node_lines(node: Any, code_lines: list[str]) -> list[str]:
+ """Line-based slice of a node's source, preserving original indentation.
+
+ Line-based (not byte-offset) slicing is used deliberately so leading
+ whitespace survives for indented nested definitions (e.g. methods inside
+ a class). But when a node shares its first line with a preceding sibling
+ (e.g. the `export` keyword in `export function foo() {`), a naive
+ full-line slice pulls in that sibling's text too — and callers that
+ reconstruct the wrapper (re-adding the sibling text themselves) end up
+ duplicating it. Trim the sibling prefix from the first line when it's not
+ pure whitespace; keep the whole line (indentation intact) otherwise.
+ """
+ start_row, start_col = node.start_point
+ end_row = node.end_point[0]
+ node_lines = list(code_lines[start_row : end_row + 1])
+ if node_lines and node_lines[0][:start_col].strip():
+ node_lines[0] = node_lines[0][start_col:]
+ return node_lines
+
+
def _get_same_line_trailing_semicolon(node: Any) -> Any | None:
"""Return a trailing semicolon sibling that belongs to this declaration."""
next_sibling = getattr(node, "next_sibling", None)
diff --git a/tests/test_code_aware_brace_comment_regressions.py b/tests/test_code_aware_brace_comment_regressions.py
new file mode 100644
index 000000000..cee8f5c5a
--- /dev/null
+++ b/tests/test_code_aware_brace_comment_regressions.py
@@ -0,0 +1,125 @@
+"""Regression tests for two CodeAwareCompressor AST-reassembly bugs found
+while investigating a reported Go brace-duplication issue.
+
+1. `export` keyword duplication (TS/JS): `_compress_function_ast` and
+ `_compress_class_ast` use LINE-based slicing (not byte-offset) to
+ preserve indentation for nested definitions. When a function/class shares
+ its first line with a preceding sibling — e.g. the `export` keyword in
+ `export function foo() {`, a sibling of the function inside
+ `export_statement`, not part of the function node itself — a naive
+ full-line slice pulled that sibling's text in too. The `export_statement`
+ handler then re-prepended the same `export` text, producing
+ `export export function foo() {` (invalid syntax, silently discarded by
+ the `_verify_syntax` fallback).
+2. Doc-comment displacement (all languages): doc comments are top-level
+ siblings of the declaration they document, not children of it. Left
+ unattached during AST extraction, they fell through to a "leftover
+ top-level code" bucket that `_assemble_compressed` emits as one block
+ after every function signature — detaching every doc comment from what it
+ documents and dumping them all at the end of the file.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+from headroom.transforms.code_compressor import (
+ CodeAwareCompressor,
+ CodeCompressorConfig,
+ CodeLanguage,
+ _check_tree_sitter_available,
+)
+
+pytestmark = pytest.mark.skipif(
+ not _check_tree_sitter_available(),
+ reason="tree-sitter not installed (pip install headroom-ai[code])",
+)
+
+TS_EXPORTED = """export interface User {
+ id: string;
+ name: string;
+}
+
+/**
+ * Fetches a user by id.
+ */
+export function getUser(id: string): User {
+ return { id, name: "test" };
+}
+
+/**
+ * Greets a user by name.
+ */
+export function greet(user: User): string {
+ return `Hello, ${user.name}!`;
+}
+
+export class UserStore {
+ private users: User[] = [];
+
+ add(user: User): void {
+ this.users.push(user);
+ }
+}
+"""
+
+GO_DOC_COMMENTS = """package main
+
+import "fmt"
+
+// Add adds two integers together and returns the sum.
+func Add(a, b int) int {
+\treturn a + b
+}
+
+// Greet returns a friendly greeting for the given name.
+func Greet(name string) string {
+\treturn fmt.Sprintf("Hello, %s!", name)
+}
+"""
+
+
+def _compress_ast(code: str, language: CodeLanguage):
+ compressor = CodeAwareCompressor(CodeCompressorConfig())
+ compressed, _, _ = compressor._compress_with_ast(code, language, "", None)
+ return compressor, compressed
+
+
+def test_ts_export_keyword_not_duplicated() -> None:
+ """`export function`/`export class` must not become `export export ...`."""
+ compressor, compressed = _compress_ast(TS_EXPORTED, CodeLanguage.TYPESCRIPT)
+
+ assert "export export" not in compressed, compressed
+ assert compressor._verify_syntax(compressed, CodeLanguage.TYPESCRIPT) is True
+
+
+def test_ts_doc_comments_stay_attached_to_declaration() -> None:
+ """A `/** ... */` doc comment must stay immediately before the function it
+ documents, not get dumped in a cluster at the end of the output."""
+ _, compressed = _compress_ast(TS_EXPORTED, CodeLanguage.TYPESCRIPT)
+
+ assert "/**\n * Fetches a user by id.\n */\nexport function getUser" in compressed
+ assert "/**\n * Greets a user by name.\n */\nexport function greet" in compressed
+
+
+def test_go_doc_comments_stay_attached_to_function() -> None:
+ """Same doc-comment-attachment bug, Go's `//` line-comment form."""
+ compressor, compressed = _compress_ast(GO_DOC_COMMENTS, CodeLanguage.GO)
+
+ assert "// Add adds two integers together and returns the sum.\nfunc Add" in compressed
+ assert "// Greet returns a friendly greeting for the given name.\nfunc Greet" in compressed
+ assert compressor._verify_syntax(compressed, CodeLanguage.GO) is True
+
+
+def test_actual_typescript_compression() -> None:
+ """Parity with the existing JS/Python/Go 'actual compression' tests —
+ real TS input must actually compress, not silently no-op."""
+ config = CodeCompressorConfig(min_tokens_for_compression=10, enable_ccr=False)
+ compressor = CodeAwareCompressor(config)
+ code = TS_EXPORTED * 3 # large enough to trigger body elision
+
+ result = compressor.compress(code, language="typescript")
+
+ assert result.compression_ratio < 1.0
+ assert result.syntax_valid is True
+ assert result.language == CodeLanguage.TYPESCRIPT
diff --git a/tests/test_transforms/test_code_compressor.py b/tests/test_transforms/test_code_compressor.py
index 774d9b6eb..059e35cac 100644
--- a/tests/test_transforms/test_code_compressor.py
+++ b/tests/test_transforms/test_code_compressor.py
@@ -753,7 +753,7 @@ class TestTreeSitterIntegration:
assert result.language == CodeLanguage.JAVASCRIPT
def test_actual_go_compression(self):
- """Test Go code is processed (compression may fall back due to nested structures)."""
+ """Test actual compression of Go code."""
config = CodeCompressorConfig(
min_tokens_for_compression=10,
enable_ccr=False,
@@ -763,12 +763,9 @@ class TestTreeSitterIntegration:
result = compressor.compress(code, language="go")
- # Go code is processed and returns valid output
- # Note: compression_ratio may be 1.0 if compression produces invalid syntax
- # and falls back to original (Go has complex nested brace handling)
+ assert result.compression_ratio < 1.0
assert result.syntax_valid is True
assert result.language == CodeLanguage.GO
- assert result.compressed # Some output is produced
@pytest.mark.parametrize(
(