Skip to content

armgit / avibe

Public Imported from GitHub
0

Make your computer a home for your AI partner. Avibe is the local-first Agent OS: Vibey, its built-in AI partner, works out of the box, and Claude Code, Codex & OpenCode run unchanged beside it — reach them from your browser, phone, or any chat app.

perf(reply): omit discarded inline parsing in block scans (#1914)

Call Whitefcebe92a2c6b
diff --git a/core/reply_enhancer.py b/core/reply_enhancer.py
index bb81b1e23..4fad4b676 100644
--- a/core/reply_enhancer.py
+++ b/core/reply_enhancer.py
@@ -36,7 +36,8 @@ from markdown_it.rules_inline.link import link as _commonmark_link
from markdown_it.rules_inline.state_inline import StateInline
logger = logging.getLogger(__name__)
-_BLOCK_MARKDOWN = MarkdownIt("commonmark")
+# Block consumers use source maps/content/levels, never parsed inline children.
+_BLOCK_MARKDOWN = MarkdownIt("commonmark").disable("inline")
_INLINE_MARKDOWN = MarkdownIt("commonmark")
_INLINE_CODE_RANGES_KEY = "avibe_inline_code_ranges"
_INLINE_ANGLE_RANGES_KEY = "avibe_inline_angle_ranges"
diff --git a/docs/plans/ci-reply-block-parsing.md b/docs/plans/ci-reply-block-parsing.md
new file mode 100644
index 000000000..81c01f97a
--- /dev/null
+++ b/docs/plans/ci-reply-block-parsing.md
@@ -0,0 +1,63 @@
+# Reply Block Parsing CI Repair
+
+## Evidence and Scope
+
+The first-attempt master run 34018535654 at `e72b9870a` failed only the
+reply-enhancer opener stress case and its unit aggregate. Parsing 100,000 open
+brackets followed by one valid file link took 11.609 seconds against the existing
+10-second assertion. The other 164 tests and 40 subtests in that process passed.
+The full file consumed 16.03 seconds of launcher-thread CPU in a 16.19-second
+observation interval. This supports inspecting computation, not attributing the
+failure to unproven host contention. The failed run is not performance evidence
+for a successful workflow.
+
+A local profile of the original input found four CommonMark inline parses:
+two attachment scans, each preceded by a block scan which unnecessarily parsed
+inline children. The two block consumers inspect only top-level token source
+maps, content, type and nesting level. Neither reads children. The profile spent
+4.28 of 9.07 seconds in those discarded inline passes. A separate unprofiled
+in-process probe measured 3.07 seconds originally and 1.62 seconds with the block
+parser's inline rule disabled, with identical reply output. These local timings
+are not CI speedup claims.
+
+The orchestrator selects only `core/reply_enhancer.py`, its existing platform
+test module, and this plan. Configure the existing dedicated CommonMark block
+parser to omit inline children. Keep the actual attachment/code parsers and all
+syntax ownership, file policy, destination normalization, masks and offsets.
+There is no replacement parser, dependency, retry, timeout or workflow change.
+
+## Contracts and Validation
+
+- Block tokens retain all fields except intentionally unused children, compared
+ against the original CommonMark parser across existing block shapes.
+- Both block consumers must work while inline parsing on their dedicated parser
+ is forbidden. Public reply/strip and media replacement paths remain real.
+- Preserve every original test body and the 100,000-opener, 10-second assertion.
+- Run the full reply platform and secret-request tests plus dispatcher/media
+ consumers, workflow contracts and lint. Require all 17 exact-head CI checks,
+ current-head bot PASS, zero unresolved whole-PR threads and clean integration.
+- The merged-source CI must independently pass before ordered delivery.
+
+Local validation completed: 520 cases across 11 separate test processes,
+including all 167 reply-platform cases and 46 subtests, plus real media and
+message-dispatch consumers. Ruff 0.4.9 and the 84-package dependency check pass.
+All 181 original function definitions and 455 original unittest assertions are
+AST-identical. Re-enabling only the old inline rule makes the new consumer guard
+and all six child-elimination subcases fail, confirming that the regressions
+detect the discarded computation. The revised profile takes 4.55 seconds with
+two inline parses, against the original four in 9.07 seconds.
+
+Complete archived logs of failed run 34018535654 confirm all 405 selected files,
+metrics and exit records exactly once, with only the named reply file nonzero.
+The separate distribution 21 cases and installer 198 cases passed (96.48 and
+104.68 seconds), including the released 3.0.13 upgrade. No CI rerun was requested.
+
+## Review and Residuals
+
+Initial findings-bearing review heads: zero. This is a narrow production
+computation repair, not a change to the stress test's acceptance policy.
+CommonMark attachment scans remain necessary; further parser rewrites and
+arbitrary performance threshold increases are out of scope. CI must confirm
+the benefit on hosted runners. Existing installation-chain no-change decision
+and all real migration, private database, wheel/sdist, Docker and Windows gates
+remain intact.
diff --git a/tests/test_reply_enhancer_platform.py b/tests/test_reply_enhancer_platform.py
index 13332b75f..3fb792be7 100644
--- a/tests/test_reply_enhancer_platform.py
+++ b/tests/test_reply_enhancer_platform.py
@@ -2387,6 +2387,53 @@ class ReplyEnhancerPlatformTests(unittest.IsolatedAsyncioTestCase):
self.assertEqual(enhanced.text, text)
self.assertEqual(enhanced.files, [])
+ def test_block_parser_preserves_commonmark_structure_without_inline_children(self):
+ specimens = [
+ "> - [report](<file:///tmp/report.md>) and `code`\n> continuation\n",
+ "heading\n=======\n\n~~~markdown\n[file](file:///tmp/code.md)\n~~~\n",
+ " indented code\n\n<div>\nraw HTML\n</div>\n\nvisible\n",
+ "[reference]: /target \"title\"\n\n[reference] and ![image](/image.png)\n",
+ "1. first\r\n - second\r\n\r\n---\r\n[one] | [two]\r\n",
+ "[" * 1000 + "[report](<file:///tmp/report.md>)",
+ ]
+ reference = reply_enhancer.MarkdownIt("commonmark")
+ for text in specimens:
+ with self.subTest(text=text[:80]):
+ expected = reference.parse(text)
+ actual = reply_enhancer._BLOCK_MARKDOWN.parse(text)
+ self.assertEqual(len(actual), len(expected))
+ for actual_token, expected_token in zip(actual, expected):
+ actual_fields = actual_token.as_dict()
+ expected_fields = expected_token.as_dict()
+ actual_fields.pop("children")
+ expected_fields.pop("children")
+ self.assertEqual(actual_fields, expected_fields)
+ self.assertFalse(actual_token.children)
+
+ def test_block_range_consumers_do_not_run_discarded_inline_parsing(self):
+ text = "> [report](<file:///tmp/report.md>)\n\n```\ncode\n```\n"
+ with patch.object(
+ reply_enhancer._BLOCK_MARKDOWN.inline,
+ "parse",
+ side_effect=AssertionError("block-only consumers must not parse inline children"),
+ ):
+ code, inline, blocking = reply_enhancer._markdown_block_ranges(text)
+ level = reply_enhancer._markdown_container_level_at(
+ "> body\n> [one] | [two]", len("> body")
+ )
+ enhanced = process_reply(text)
+ stripped = reply_enhancer.strip_file_links(text)
+ replaced = reply_enhancer._replace_file_links(text, lambda match: "/media/report")
+
+ self.assertEqual(code, [(text.index("```"), len(text))])
+ self.assertEqual(blocking, code)
+ self.assertEqual(inline, [(0, text.index("\n") + 1, "[report](<file:///tmp/report.md>)")])
+ self.assertEqual(level, 2)
+ self.assertEqual([file.path for file in enhanced.files], ["/tmp/report.md"])
+ self.assertEqual(enhanced.text, "> report\n\n```\ncode\n```")
+ self.assertEqual(stripped, "> report\n\n```\ncode\n```\n")
+ self.assertEqual(replaced, "> [report](</media/report>)\n\n```\ncode\n```\n")
+
def test_file_link_parser_handles_many_openers_in_bounded_time(self):
text = "[" * 100000 + "[report](<file:///tmp/report.md>)"