Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

fix(code): pin tree-sitter-language-pack <1.0 so code compression works (#1234)

Tufan ALINf11a27122997
diff --git a/pyproject.toml b/pyproject.toml
index 30b520806..56b9dc6cd 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -86,8 +86,14 @@ proxy-prod = [
"gunicorn>=21.0.0; sys_platform != 'win32'",
]
# AST-based code compression (tree-sitter)
+# NOTE: cap below 1.0. tree-sitter-language-pack 1.x is a breaking rewrite whose
+# get_language()/get_parser() return the pack's own binding types instead of
+# standalone tree_sitter.Language/Parser, so _get_parser() in
+# transforms/code_compressor.py fails and code compression silently no-ops.
+# The 0.x line (>=0.10,<1.0) returns standalone tree_sitter objects as expected.
code = [
- "tree-sitter-language-pack>=0.10.0",
+ "tree-sitter-language-pack>=0.10.0,<1.0",
+ "tree-sitter>=0.25.2,<0.26",
]
# ML-based compression with Kompress (ModernBERT).
# (The legacy [llmlingua] extra was removed in 0.9.x — no live code path used it.