Skip to content

armgit / ECC

Public Imported from GitHub
0

The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

fix(gateguard): treat directive comments and continuations as code

Jeffrey Montoya82420f061a4c
diff --git a/docs/gateguard/design-notes.md b/docs/gateguard/design-notes.md
index 8baf6bc59..cfbb9812f 100644
--- a/docs/gateguard/design-notes.md
+++ b/docs/gateguard/design-notes.md
@@ -385,8 +385,43 @@ Anything the lexer cannot read with confidence makes the entry non-trivial:
- in JS/TS, any `/` outside a comment (division and regex literals cannot be
told apart), JSX-like `<x`, `</`, `<>`, `<!`, and `-->`;
- a line comment ending in `\` (it continues onto the next line in C and
- Make), the trigraph `??/`, whitespace after a line-continuation `\`, a `/*`
- inside a block comment, and an unterminated block comment or string.
+ Make), the trigraph `??/`, a `/*` inside a block comment, and an
+ unterminated block comment or string;
+- a code line ending in a line continuation `\` (trailing blanks included):
+ a C macro, a Python explicit continuation or a string continued onto the
+ next line joins that line, so adding or removing a comment-only line after
+ it changes code.
+
+## Directive comments
+
+Some comments are read by the language, the build or a tool, so their text is
+code. A directive comment is kept in the compared code verbatim: changing,
+adding or removing one makes the entry non-trivial, while an unchanged
+directive next to an edited ordinary comment does not.
+
+- A comment whose text starts with `!` (shebangs, Rust inner docs, `/*!`),
+ `/` (`///` doc comments and TypeScript triple-slash references), `go:`,
+ `export` or `extern` followed by a blank (cgo), or `line` followed by a
+ blank (Go line directives), with no blank after the comment marker.
+- A comment whose first word, after blanks, `*` and `/`, starts with `@`, `#`,
+ `<`, `+`, `!`, `type:` or `requires`, or is `global`, `globals` or
+ `exported` followed by a blank
+ (JSDoc and TypeScript pragmas, `//# sourceMappingURL`, `// +build`, Python
+ type comments, ESLint globals, PowerShell `#Requires`).
+- A comment containing a tool or encoding marker such as `lint`, `ts-`,
+ `noqa`, `nosec`, `pragma`, `coding:`, `fmt:`, `istanbul`, `prettier-`,
+ `webpack`, `__PURE__`, `fallthrough`, `shellcheck`, `suppress`, `sonar`,
+ `gitleaks`, `allowlist`, `vim:`, `-*-`, `DO NOT EDIT`, `#compdef` or Go's
+ example `// Output:`, or a JSDoc type
+ (`@` together with `{`). The list is in `DIRECTIVE_WORDS`; matching is
+ case-insensitive and substring-based, so it over-matches prose that happens
+ to contain a marker, which only keeps an edit gated.
+- Every Rust doc comment (`///`, `//!`, `/**`, `/*!`): doc tests compile and
+ run.
+
+Suppression markers (`nosec`, `NOSONAR`, `gitleaks:allow`,
+`pragma: allowlist secret`, `eslint-disable`) matter most: removing a finding
+from a security scanner is not a comment edit.
## Shell scripts
@@ -399,13 +434,14 @@ is not trivial) wherever a comment could be code:
`>` the entry is not trivial (zsh glob flags such as `(#i)`). Single quotes
are raw and must close on the line; `$'...'` is not trivial; double quotes
may hold `\` escapes and a `${...}` without quotes, but `$(` inside them,
- any backtick, any `<<` (heredocs, here-strings, shifts), a `#!` line, a
+ any backtick, any `<<` (heredocs, here-strings, shifts), a
backslash before a line break, and a lone carriage return (bash reads it
as a word character) make the entry not trivial. Only spaces and tabs
between words are folded; `\` escapes stay code.
- PowerShell: `#` and `<# ... #>` are comments only at the start of a line or
after a space, tab or `;`; elsewhere the entry is not trivial. `#Requires`
- is a directive and never a comment. Here-strings (`@"`, `@'`), typographic
+ and `#!` are [directive comments](#directive-comments), as in every
+ language. Here-strings (`@"`, `@'`), typographic
quotes (PowerShell accepts them as quote marks), `$(` inside a double-quoted
string, a backtick before a line break, and a nested `<#` are not trivial.
- Batch: only `REM` lines (after optional blanks and `@`, followed by a blank
diff --git a/scripts/lib/gateguard-change-profile.js b/scripts/lib/gateguard-change-profile.js
index cb680bfca..ddc0b8337 100644
--- a/scripts/lib/gateguard-change-profile.js
+++ b/scripts/lib/gateguard-change-profile.js
@@ -31,7 +31,7 @@ const C_FAMILY = { lineComment: '//', blockComment: true, quotes: '"\'', indentS
const LEX_SPECS = {
js: { ...C_FAMILY, forbiddenCode: '`/', jsx: true },
go: { ...C_FAMILY, forbiddenCode: '`' },
- rust: { ...C_FAMILY, quotes: '"', rustChars: true, rawPrefixes: 'r#' },
+ rust: { ...C_FAMILY, quotes: '"', rustChars: true, rawPrefixes: 'r#', rustDocs: true },
java: { ...C_FAMILY, tripleQuotes: true },
kotlin: { ...C_FAMILY, tripleQuotes: true, noDollarInStrings: true },
csharp: { ...C_FAMILY, tripleQuotes: true, rawPrefixes: '@$' },
@@ -75,35 +75,86 @@ function stringPrefixIsUnsafe(code, spec) {
return prefix.toLowerCase().includes('f');
}
+// --- Directive comments ---
+// see docs/gateguard/design-notes.md#directive-comments
+
+const DIRECTIVE_LEADS = ['@', '#', '<', '+', '!', 'type:', 'global ', 'globals ', 'exported ', 'requires'];
+const DIRECTIVE_RAW_LEADS = ['!', '/', 'go:', 'export ', 'extern ', 'line '];
+const DIRECTIVE_WORDS = [
+ 'lint', 'jshint', 'ts-', 'noqa', 'nosec', 'semgrep', 'sonar', 'noinspection', 'pragma', 'coding:', 'coding=', 'fmt:', 'isort:',
+ 'mypy:', 'pyright:', 'pyre-', 'yapf:', 'ruff:', 'flake8', 'bandit', 'no cover', 'nocover', 'istanbul', 'c8 ', 'v8 ',
+ 'prettier-', 'biome-', 'deno-', 'dprint-', 'rome-', 'webpack', 'vite-', '__pure__', '__no_side_effects__',
+ '__inline__', '@preserve', '@license', 'sourcemappingurl', 'sourceurl', '+build', 'cgo', 'clang-', 'cppcheck',
+ 'coverity', 'iwyu', 'fallthrough', 'fall through', 'fall-through', 'fallthru', 'fall thru', 'shellcheck',
+ 'psscriptanalyzer', 'suppress', 'jscs', 'checkstyle', 'spotbugs', 'findbugs', 'nopmd', 'codeql', 'lgtm', 'gitleaks',
+ 'trufflehog', 'detect-secrets', 'allowlist', 'vim:', 'vi:', ' ex:', '-*-', 'code generated', 'do not edit',
+ 'rubocop', 'swiftlint', 'resharper', '@ts-', '@type', '@typedef', '@template', '@satisfies', '@import', '@callback',
+ '@overload', '@enum', '@this', '@implements', '@extends', '@augments', '@jsx', '@flow', '@noflow', '@generated',
+ 'compdef', 'autoload', 'output:'
+];
+
+function isDirectiveComment(body, spec) {
+ if (spec.rustDocs && (body[0] === '/' || body[0] === '!' || body[0] === '*')) return true;
+ if (DIRECTIVE_RAW_LEADS.some(lead => body.startsWith(lead))) return true;
+ const lower = body.toLowerCase();
+ let start = 0;
+ while (start < lower.length && (isSpace(lower[start]) || lower[start] === '*' || lower[start] === '/')) start++;
+ const lead = lower.slice(start);
+ if (DIRECTIVE_LEADS.some(word => lead.startsWith(word))) return true;
+ if (DIRECTIVE_WORDS.some(word => lower.includes(word))) return true;
+ return lower.includes('@') && lower.includes('{');
+}
+
+function directiveMarker(body, spec) {
+ return isDirectiveComment(body, spec) ? ` \u0001${Buffer.from(body, 'utf8').toString('hex')}\u0001 ` : '';
+}
+
+function endsWithContinuation(code) {
+ let end = code.length;
+ while (end > 0 && isSpace(code[end - 1])) end--;
+ return code[end - 1] === '\\';
+}
+
function lexCodeLines(text, spec) {
const lines = [];
let code = '';
let state = 'code';
let quote = '';
let stringBody = '';
- let commentTail = '';
+ let commentStart = 0;
+ const endComment = end => {
+ const marker = directiveMarker(text.slice(commentStart, end), spec);
+ if (marker) code += marker;
+ };
const endLine = () => {
+ if (endsWithContinuation(code)) return false;
lines.push(code);
code = '';
+ return true;
};
+ if (spec.jsx !== undefined && text.startsWith('#!')) state = 'line';
for (let i = 0; i < text.length; i++) {
const ch = text[i];
if (ch === '\n' || ch === '\r') {
if (state === 'string') return null;
if (state === 'line') {
- if (commentTail.endsWith('\\')) return null;
+ if (text.slice(commentStart, i).trimEnd().endsWith('\\')) return null;
+ endComment(i);
state = 'code';
}
- endLine();
+ if (state === 'block') {
+ lines.push(code);
+ code = '';
+ } else if (!endLine()) {
+ return null;
+ }
if (ch === '\r' && text[i + 1] === '\n') i++;
continue;
}
- if (state === 'line') {
- if (!isSpace(ch)) commentTail = commentTail.length > 8 ? commentTail.slice(-4) + ch : commentTail + ch;
- continue;
- }
+ if (state === 'line') continue;
if (state === 'block') {
if (ch === '*' && text[i + 1] === '/') {
+ endComment(i);
state = 'code';
code += ' ';
i++;
@@ -129,23 +180,24 @@ function lexCodeLines(text, spec) {
}
if (spec.lineComment === '//' && ch === '/' && text[i + 1] === '/') {
state = 'line';
- commentTail = '';
+ commentStart = i + 2;
i++;
continue;
}
if (spec.lineComment === '#' && ch === '#') {
state = 'line';
- commentTail = '';
+ commentStart = i + 1;
continue;
}
if (spec.blockComment && ch === '/' && text[i + 1] === '*') {
state = 'block';
+ commentStart = i + 2;
i++;
continue;
}
if (spec.forbiddenCode && spec.forbiddenCode.includes(ch)) return null;
if (spec.jsx && ch === '<' && (isLetter(text[i + 1]) || text[i + 1] === '/' || text[i + 1] === '>' || text[i + 1] === '!')) return null;
- if (spec.jsx && ch === '-' && text[i + 1] === '-' && text[i + 2] === '>') return null;
+ if (spec.jsx !== undefined && ch === '-' && text[i + 1] === '-' && text[i + 2] === '>') return null;
if (ch === "'" && spec.rustChars) {
const end = rustCharEnd(text, i);
code += end === -1 ? ch : text.slice(i, end + 1);
@@ -164,9 +216,11 @@ function lexCodeLines(text, spec) {
code += ch;
}
if (state === 'string' || state === 'block') return null;
- if (state === 'line' && commentTail.endsWith('\\')) return null;
- endLine();
- return lines;
+ if (state === 'line') {
+ if (text.slice(commentStart).trimEnd().endsWith('\\')) return null;
+ endComment(text.length);
+ }
+ return endLine() ? lines : null;
}
function normalizeCodeLine(line, spec) {
@@ -235,6 +289,8 @@ function isTrivialEdit(oldString, newString, spec) {
// --- Shell lexing ---
// see docs/gateguard/design-notes.md#shell-scripts
+const SHELL_SPEC = Object.freeze({});
+
function isLineStart(text, i) {
return i === 0 || text[i - 1] === '\n' || text[i - 1] === '\r';
}
@@ -319,8 +375,9 @@ function shCodeLines(text) {
const starts = hashStartsComment(text, i);
if (starts === null) return null;
if (starts) {
- if (isLineStart(text, i) && text[i + 1] === '!') return null;
+ const start = i + 1;
while (i + 1 < text.length && text[i + 1] !== '\n' && text[i + 1] !== '\r') i++;
+ code += directiveMarker(text.slice(start, i + 1), SHELL_SPEC);
continue;
}
}
@@ -361,6 +418,7 @@ function psCodeLines(text) {
const lines = [];
let code = '';
let block = false;
+ let blockStart = 0;
for (let i = 0; i < text.length; i++) {
const ch = text[i];
if (ch === '\n' || ch === '\r') {
@@ -373,7 +431,7 @@ function psCodeLines(text) {
if (ch === '<' && text[i + 1] === '#') return null;
if (ch === '#' && text[i + 1] === '>') {
block = false;
- code += ' ';
+ code += directiveMarker(text.slice(blockStart, i), SHELL_SPEC) || ' ';
i++;
}
continue;
@@ -395,13 +453,15 @@ function psCodeLines(text) {
if (ch === '<' && text[i + 1] === '#') {
if (!psHashStartsComment(text, i)) return null;
block = true;
+ blockStart = i + 2;
i++;
continue;
}
if (ch === '#') {
if (!psHashStartsComment(text, i)) return null;
- if (text.slice(i + 1, i + 9).toLowerCase() === 'requires') return null;
+ const start = i + 1;
while (i + 1 < text.length && text[i + 1] !== '\n' && text[i + 1] !== '\r') i++;
+ code += directiveMarker(text.slice(start, i + 1), SHELL_SPEC);
continue;
}
code += ch;
@@ -423,7 +483,7 @@ function batchCodeLines(text) {
const after = line[start + 3];
if (word === 'rem' && (after === undefined || isBlank(after))) {
if (/[%^&|<>()]/.test(line)) return null;
- lines.push('');
+ lines.push(directiveMarker(line.slice(start + 4), SHELL_SPEC));
} else {
lines.push(line.trim() ? line : '');
}
diff --git a/tests/lib/gateguard-change-profile.test.js b/tests/lib/gateguard-change-profile.test.js
index a540830b4..0fbc0ea3e 100644
--- a/tests/lib/gateguard-change-profile.test.js
+++ b/tests/lib/gateguard-change-profile.test.js
@@ -456,6 +456,79 @@ test('comment tricks that can hide code are never trivial', () => {
assert.strictEqual(trivial('a.py', 'x = a \\\n + b', 'x = a \\ \n + b'), false, 'whitespace after a line continuation');
});
+test('a code line ending in a line continuation is never trivial', () => {
+ const cases = [
+ ['a.c', '#define A 1 \\\n// c\nint x;', '#define A 1 \\\nint x;'],
+ ['a.c', '#define A 1 \\\nint x;', '#define A 1 \\\n// c\nint x;'],
+ ['a.cpp', '#define F(x) \\\n /* a */ (x)', '#define F(x) \\\n /* b */ (x)'],
+ ['a.h', 'int x; \\', 'int x; \\ '],
+ ['a.py', 'y = 1 + \\\n 2\n# c', 'y = 1 + \\\n 2'],
+ ['a.py', 'y = 1 + \\', 'y = 1 + \\'],
+ ['a.js', "s = 'a' + \\\n// c", "s = 'a' + \\\n// d"]
+ ];
+ for (const [file, before, after] of cases) {
+ assert.strictEqual(trivial(file, before, after), false, `${file}: ${JSON.stringify(before)} -> ${JSON.stringify(after)}`);
+ }
+});
+
+test('directive comments are code: shebangs, encodings, type, lint and build directives', () => {
+ const cases = [
+ ['a.py', '#!/usr/bin/env python3\nprint(1)', '#!/usr/bin/env -S python3 -X dev\nprint(1)'],
+ ['a.py', '# -*- coding: utf-8 -*-\ns = 1', '# -*- coding: latin-1 -*-\ns = 1'],
+ ['a.py', '# vim: set fileencoding=utf-8 :\ns = 1', '# vim: set fileencoding=latin-1 :\ns = 1'],
+ ['a.py', 'x = f() # type: ignore', 'x = f() # ok'],
+ ['a.py', 'import os # noqa: F401', 'import os # unused'],
+ ['a.py', 'x = 1 # pylint: disable=invalid-name', 'x = 1 # fine'],
+ ['a.py', 'call(cmd, shell=True) # nosec', 'call(cmd, shell=True) # reviewed'],
+ ['a.py', 'def f(): # pragma: no cover\n pass', 'def f(): # rarely hit\n pass'],
+ ['a.py', 'x = 1', 'x = 1 # fmt: skip'],
+ ['a.ts', '// @ts-expect-error\nconst n: number = s;', '// expected\nconst n: number = s;'],
+ ['a.ts', 'const n = s; // @ts-ignore', 'const n = s; // ignore'],
+ ['a.ts', '/// <reference types="node" />\nx();', '/// <reference types="deno" />\nx();'],
+ ['a.js', '/* eslint-disable no-eval */\neval(x);', '/* note */\neval(x);'],
+ ['a.js', 'eval(x); // eslint-disable-line', 'eval(x); // reviewed'],
+ ['a.js', 'const m = import(/* webpackChunkName: "a" */ "./a");', 'const m = import(/* webpackChunkName: "b" */ "./a");'],
+ ['a.js', 'const v = /*#__PURE__*/ make();', 'const v = /* pure */ make();'],
+ ['a.js', '// @flow\nx();', '// flow\nx();'],
+ ['a.js', '/** @type {number} */\nlet n;', '/** @type {string} */\nlet n;'],
+ ['a.js', '/* global jQuery */\njQuery();', '/* jQuery */\njQuery();'],
+ ['a.js', 'x(); // istanbul ignore next', 'x(); // rarely'],
+ ['a.js', 'x(); // prettier-ignore', 'x(); // keep'],
+ ['a.go', '//go:build linux\n\npackage o', '//go:build ignore\n\npackage o'],
+ ['a.go', '// +build linux\n\npackage o', '// +build ignore\n\npackage o'],
+ ['a.go', '//go:embed a.txt\nvar s string', '//go:embed b.txt\nvar s string'],
+ ['a.go', '//go:generate stringer -type=A\ntype A int', '//go:generate rm -rf .\ntype A int'],
+ ['a.go', '//export Add\nfunc Add() {}', '// Add adds\nfunc Add() {}'],
+ ['a.go', 'x := f() //nolint:errcheck', 'x := f() // fine'],
+ ['a.go', '// Code generated by protoc. DO NOT EDIT.\npackage p', '// Written by hand.\npackage p'],
+ ['a.rs', '/// ```\n/// assert!(true);\n/// ```\nfn f() {}', '/// ```\n/// assert!(false);\n/// ```\nfn f() {}'],
+ ['a.rs', '//! assert!(true);\nfn f() {}', '//! assert!(false);\nfn f() {}'],
+ ['a.rs', '/** assert!(true); */\nfn f() {}', '/** assert!(false); */\nfn f() {}'],
+ ['a.c', 'case 1: x(); /* fallthrough */\ncase 2: y();', 'case 1: x(); /* next */\ncase 2: y();'],
+ ['a.cpp', 'int x = f(); // NOLINT', 'int x = f(); // ok'],
+ ['a.java', 'int x = f(); // NOSONAR', 'int x = f(); // ok'],
+ ['a.cs', 'var x = f(); // ReSharper disable once All', 'var x = f(); // ok'],
+ ['a.kt', 'val x = f() // ktlint-disable', 'val x = f() // ok'],
+ ['a.sh', '# shellcheck disable=SC2086\necho $x', '# quoting is fine\necho $x'],
+ ['a.sh', 'echo a\n#!/bin/sh', 'echo a\n#!/bin/bash'],
+ ['a.ps1', '# PSScriptAnalyzer suppression\nGet-Item a', '# note\nGet-Item a'],
+ ['a.bat', 'REM vim: set ff=dos :\necho a', 'REM note\necho a'],
+ ['a.py', 'API_KEY = load() # pragma: allowlist secret', 'API_KEY = load() # loaded'],
+ ['a.js', 'const k = "x"; // gitleaks:allow', 'const k = "x"; // ok'],
+ ['a.go', 'fmt.Println(42)\n// Output: 42', 'fmt.Println(42)\n// Output: 43'],
+ ['a.zsh', '#compdef _git git\nx', '#compdef _hg hg\nx']
+ ];
+ for (const [file, before, after] of cases) {
+ assert.strictEqual(trivial(file, before, after), false, `${file}: ${JSON.stringify(before)} -> ${JSON.stringify(after)}`);
+ }
+});
+
+test('an unchanged directive does not stop a nearby comment edit from being trivial', () => {
+ assert.strictEqual(trivial('a.py', '#!/usr/bin/env python3\n# old\nx = 1', '#!/usr/bin/env python3\n# new\nx = 1'), true);
+ assert.strictEqual(trivial('a.ts', '// @ts-check\n// old\nx();', '// @ts-check\n// new\nx();'), true);
+ assert.strictEqual(trivial('a.sh', '#!/bin/sh\n# old\nmake', '#!/bin/sh\n# new\nmake'), true);
+});
+
test('Write, unknown languages and MultiEdit with any non-trivial entry are not trivial', () => {
assert.strictEqual(write('a.js', '// only a comment\n').trivial, false);
assert.strictEqual(trivial('a.rb', '# a', '# b'), false);