The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
fix(gateguard): treat directive comments and continuations as code
Jeffrey Montoya82420f061a4c
diff --git a/docs/gateguard/design-notes.md b/docs/gateguard/design-notes.mdindex 8baf6bc59..cfbb9812f 100644--- a/docs/gateguard/design-notes.md+++ b/docs/gateguard/design-notes.md@@ -385,8 +385,43 @@ Anything the lexer cannot read with confidence makes the entry non-trivial:- in JS/TS, any `/` outside a comment (division and regex literals cannot betold apart), JSX-like `<x`, `</`, `<>`, `<!`, and `-->`;- a line comment ending in `\` (it continues onto the next line in C and- Make), the trigraph `??/`, whitespace after a line-continuation `\`, a `/*`- inside a block comment, and an unterminated block comment or string.+ Make), the trigraph `??/`, a `/*` inside a block comment, and an+ unterminated block comment or string;+- a code line ending in a line continuation `\` (trailing blanks included):+ a C macro, a Python explicit continuation or a string continued onto the+ next line joins that line, so adding or removing a comment-only line after+ it changes code.++## Directive comments++Some comments are read by the language, the build or a tool, so their text is+code. A directive comment is kept in the compared code verbatim: changing,+adding or removing one makes the entry non-trivial, while an unchanged+directive next to an edited ordinary comment does not.++- A comment whose text starts with `!` (shebangs, Rust inner docs, `/*!`),+ `/` (`///` doc comments and TypeScript triple-slash references), `go:`,+ `export` or `extern` followed by a blank (cgo), or `line` followed by a+ blank (Go line directives), with no blank after the comment marker.+- A comment whose first word, after blanks, `*` and `/`, starts with `@`, `#`,+ `<`, `+`, `!`, `type:` or `requires`, or is `global`, `globals` or+ `exported` followed by a blank+ (JSDoc and TypeScript pragmas, `//# sourceMappingURL`, `// +build`, Python+ type comments, ESLint globals, PowerShell `#Requires`).+- A comment containing a tool or encoding marker such as `lint`, `ts-`,+ `noqa`, `nosec`, `pragma`, `coding:`, `fmt:`, `istanbul`, `prettier-`,+ `webpack`, `__PURE__`, `fallthrough`, `shellcheck`, `suppress`, `sonar`,+ `gitleaks`, `allowlist`, `vim:`, `-*-`, `DO NOT EDIT`, `#compdef` or Go's+ example `// Output:`, or a JSDoc type+ (`@` together with `{`). The list is in `DIRECTIVE_WORDS`; matching is+ case-insensitive and substring-based, so it over-matches prose that happens+ to contain a marker, which only keeps an edit gated.+- Every Rust doc comment (`///`, `//!`, `/**`, `/*!`): doc tests compile and+ run.++Suppression markers (`nosec`, `NOSONAR`, `gitleaks:allow`,+`pragma: allowlist secret`, `eslint-disable`) matter most: removing a finding+from a security scanner is not a comment edit.## Shell scripts@@ -399,13 +434,14 @@ is not trivial) wherever a comment could be code:`>` the entry is not trivial (zsh glob flags such as `(#i)`). Single quotesare raw and must close on the line; `$'...'` is not trivial; double quotesmay hold `\` escapes and a `${...}` without quotes, but `$(` inside them,- any backtick, any `<<` (heredocs, here-strings, shifts), a `#!` line, a+ any backtick, any `<<` (heredocs, here-strings, shifts), abackslash before a line break, and a lone carriage return (bash reads itas a word character) make the entry not trivial. Only spaces and tabsbetween words are folded; `\` escapes stay code.- PowerShell: `#` and `<# ... #>` are comments only at the start of a line orafter a space, tab or `;`; elsewhere the entry is not trivial. `#Requires`- is a directive and never a comment. Here-strings (`@"`, `@'`), typographic+ and `#!` are [directive comments](#directive-comments), as in every+ language. Here-strings (`@"`, `@'`), typographicquotes (PowerShell accepts them as quote marks), `$(` inside a double-quotedstring, a backtick before a line break, and a nested `<#` are not trivial.- Batch: only `REM` lines (after optional blanks and `@`, followed by a blankdiff --git a/scripts/lib/gateguard-change-profile.js b/scripts/lib/gateguard-change-profile.jsindex cb680bfca..ddc0b8337 100644--- a/scripts/lib/gateguard-change-profile.js+++ b/scripts/lib/gateguard-change-profile.js@@ -31,7 +31,7 @@ const C_FAMILY = { lineComment: '//', blockComment: true, quotes: '"\'', indentSconst LEX_SPECS = {js: { ...C_FAMILY, forbiddenCode: '`/', jsx: true },go: { ...C_FAMILY, forbiddenCode: '`' },- rust: { ...C_FAMILY, quotes: '"', rustChars: true, rawPrefixes: 'r#' },+ rust: { ...C_FAMILY, quotes: '"', rustChars: true, rawPrefixes: 'r#', rustDocs: true },java: { ...C_FAMILY, tripleQuotes: true },kotlin: { ...C_FAMILY, tripleQuotes: true, noDollarInStrings: true },csharp: { ...C_FAMILY, tripleQuotes: true, rawPrefixes: '@$' },@@ -75,35 +75,86 @@ function stringPrefixIsUnsafe(code, spec) {return prefix.toLowerCase().includes('f');}+// --- Directive comments ---+// see docs/gateguard/design-notes.md#directive-comments++const DIRECTIVE_LEADS = ['@', '#', '<', '+', '!', 'type:', 'global ', 'globals ', 'exported ', 'requires'];+const DIRECTIVE_RAW_LEADS = ['!', '/', 'go:', 'export ', 'extern ', 'line '];+const DIRECTIVE_WORDS = [+ 'lint', 'jshint', 'ts-', 'noqa', 'nosec', 'semgrep', 'sonar', 'noinspection', 'pragma', 'coding:', 'coding=', 'fmt:', 'isort:',+ 'mypy:', 'pyright:', 'pyre-', 'yapf:', 'ruff:', 'flake8', 'bandit', 'no cover', 'nocover', 'istanbul', 'c8 ', 'v8 ',+ 'prettier-', 'biome-', 'deno-', 'dprint-', 'rome-', 'webpack', 'vite-', '__pure__', '__no_side_effects__',+ '__inline__', '@preserve', '@license', 'sourcemappingurl', 'sourceurl', '+build', 'cgo', 'clang-', 'cppcheck',+ 'coverity', 'iwyu', 'fallthrough', 'fall through', 'fall-through', 'fallthru', 'fall thru', 'shellcheck',+ 'psscriptanalyzer', 'suppress', 'jscs', 'checkstyle', 'spotbugs', 'findbugs', 'nopmd', 'codeql', 'lgtm', 'gitleaks',+ 'trufflehog', 'detect-secrets', 'allowlist', 'vim:', 'vi:', ' ex:', '-*-', 'code generated', 'do not edit',+ 'rubocop', 'swiftlint', 'resharper', '@ts-', '@type', '@typedef', '@template', '@satisfies', '@import', '@callback',+ '@overload', '@enum', '@this', '@implements', '@extends', '@augments', '@jsx', '@flow', '@noflow', '@generated',+ 'compdef', 'autoload', 'output:'+];++function isDirectiveComment(body, spec) {+ if (spec.rustDocs && (body[0] === '/' || body[0] === '!' || body[0] === '*')) return true;+ if (DIRECTIVE_RAW_LEADS.some(lead => body.startsWith(lead))) return true;+ const lower = body.toLowerCase();+ let start = 0;+ while (start < lower.length && (isSpace(lower[start]) || lower[start] === '*' || lower[start] === '/')) start++;+ const lead = lower.slice(start);+ if (DIRECTIVE_LEADS.some(word => lead.startsWith(word))) return true;+ if (DIRECTIVE_WORDS.some(word => lower.includes(word))) return true;+ return lower.includes('@') && lower.includes('{');+}++function directiveMarker(body, spec) {+ return isDirectiveComment(body, spec) ? ` \u0001${Buffer.from(body, 'utf8').toString('hex')}\u0001 ` : '';+}++function endsWithContinuation(code) {+ let end = code.length;+ while (end > 0 && isSpace(code[end - 1])) end--;+ return code[end - 1] === '\\';+}+function lexCodeLines(text, spec) {const lines = [];let code = '';let state = 'code';let quote = '';let stringBody = '';- let commentTail = '';+ let commentStart = 0;+ const endComment = end => {+ const marker = directiveMarker(text.slice(commentStart, end), spec);+ if (marker) code += marker;+ };const endLine = () => {+ if (endsWithContinuation(code)) return false;lines.push(code);code = '';+ return true;};+ if (spec.jsx !== undefined && text.startsWith('#!')) state = 'line';for (let i = 0; i < text.length; i++) {const ch = text[i];if (ch === '\n' || ch === '\r') {if (state === 'string') return null;if (state === 'line') {- if (commentTail.endsWith('\\')) return null;+ if (text.slice(commentStart, i).trimEnd().endsWith('\\')) return null;+ endComment(i);state = 'code';}- endLine();+ if (state === 'block') {+ lines.push(code);+ code = '';+ } else if (!endLine()) {+ return null;+ }if (ch === '\r' && text[i + 1] === '\n') i++;continue;}- if (state === 'line') {- if (!isSpace(ch)) commentTail = commentTail.length > 8 ? commentTail.slice(-4) + ch : commentTail + ch;- continue;- }+ if (state === 'line') continue;if (state === 'block') {if (ch === '*' && text[i + 1] === '/') {+ endComment(i);state = 'code';code += ' ';i++;@@ -129,23 +180,24 @@ function lexCodeLines(text, spec) {}if (spec.lineComment === '//' && ch === '/' && text[i + 1] === '/') {state = 'line';- commentTail = '';+ commentStart = i + 2;i++;continue;}if (spec.lineComment === '#' && ch === '#') {state = 'line';- commentTail = '';+ commentStart = i + 1;continue;}if (spec.blockComment && ch === '/' && text[i + 1] === '*') {state = 'block';+ commentStart = i + 2;i++;continue;}if (spec.forbiddenCode && spec.forbiddenCode.includes(ch)) return null;if (spec.jsx && ch === '<' && (isLetter(text[i + 1]) || text[i + 1] === '/' || text[i + 1] === '>' || text[i + 1] === '!')) return null;- if (spec.jsx && ch === '-' && text[i + 1] === '-' && text[i + 2] === '>') return null;+ if (spec.jsx !== undefined && ch === '-' && text[i + 1] === '-' && text[i + 2] === '>') return null;if (ch === "'" && spec.rustChars) {const end = rustCharEnd(text, i);code += end === -1 ? ch : text.slice(i, end + 1);@@ -164,9 +216,11 @@ function lexCodeLines(text, spec) {code += ch;}if (state === 'string' || state === 'block') return null;- if (state === 'line' && commentTail.endsWith('\\')) return null;- endLine();- return lines;+ if (state === 'line') {+ if (text.slice(commentStart).trimEnd().endsWith('\\')) return null;+ endComment(text.length);+ }+ return endLine() ? lines : null;}function normalizeCodeLine(line, spec) {@@ -235,6 +289,8 @@ function isTrivialEdit(oldString, newString, spec) {// --- Shell lexing ---// see docs/gateguard/design-notes.md#shell-scripts+const SHELL_SPEC = Object.freeze({});+function isLineStart(text, i) {return i === 0 || text[i - 1] === '\n' || text[i - 1] === '\r';}@@ -319,8 +375,9 @@ function shCodeLines(text) {const starts = hashStartsComment(text, i);if (starts === null) return null;if (starts) {- if (isLineStart(text, i) && text[i + 1] === '!') return null;+ const start = i + 1;while (i + 1 < text.length && text[i + 1] !== '\n' && text[i + 1] !== '\r') i++;+ code += directiveMarker(text.slice(start, i + 1), SHELL_SPEC);continue;}}@@ -361,6 +418,7 @@ function psCodeLines(text) {const lines = [];let code = '';let block = false;+ let blockStart = 0;for (let i = 0; i < text.length; i++) {const ch = text[i];if (ch === '\n' || ch === '\r') {@@ -373,7 +431,7 @@ function psCodeLines(text) {if (ch === '<' && text[i + 1] === '#') return null;if (ch === '#' && text[i + 1] === '>') {block = false;- code += ' ';+ code += directiveMarker(text.slice(blockStart, i), SHELL_SPEC) || ' ';i++;}continue;@@ -395,13 +453,15 @@ function psCodeLines(text) {if (ch === '<' && text[i + 1] === '#') {if (!psHashStartsComment(text, i)) return null;block = true;+ blockStart = i + 2;i++;continue;}if (ch === '#') {if (!psHashStartsComment(text, i)) return null;- if (text.slice(i + 1, i + 9).toLowerCase() === 'requires') return null;+ const start = i + 1;while (i + 1 < text.length && text[i + 1] !== '\n' && text[i + 1] !== '\r') i++;+ code += directiveMarker(text.slice(start, i + 1), SHELL_SPEC);continue;}code += ch;@@ -423,7 +483,7 @@ function batchCodeLines(text) {const after = line[start + 3];if (word === 'rem' && (after === undefined || isBlank(after))) {if (/[%^&|<>()]/.test(line)) return null;- lines.push('');+ lines.push(directiveMarker(line.slice(start + 4), SHELL_SPEC));} else {lines.push(line.trim() ? line : '');}diff --git a/tests/lib/gateguard-change-profile.test.js b/tests/lib/gateguard-change-profile.test.jsindex a540830b4..0fbc0ea3e 100644--- a/tests/lib/gateguard-change-profile.test.js+++ b/tests/lib/gateguard-change-profile.test.js@@ -456,6 +456,79 @@ test('comment tricks that can hide code are never trivial', () => {assert.strictEqual(trivial('a.py', 'x = a \\\n + b', 'x = a \\ \n + b'), false, 'whitespace after a line continuation');});+test('a code line ending in a line continuation is never trivial', () => {+ const cases = [+ ['a.c', '#define A 1 \\\n// c\nint x;', '#define A 1 \\\nint x;'],+ ['a.c', '#define A 1 \\\nint x;', '#define A 1 \\\n// c\nint x;'],+ ['a.cpp', '#define F(x) \\\n /* a */ (x)', '#define F(x) \\\n /* b */ (x)'],+ ['a.h', 'int x; \\', 'int x; \\ '],+ ['a.py', 'y = 1 + \\\n 2\n# c', 'y = 1 + \\\n 2'],+ ['a.py', 'y = 1 + \\', 'y = 1 + \\'],+ ['a.js', "s = 'a' + \\\n// c", "s = 'a' + \\\n// d"]+ ];+ for (const [file, before, after] of cases) {+ assert.strictEqual(trivial(file, before, after), false, `${file}: ${JSON.stringify(before)} -> ${JSON.stringify(after)}`);+ }+});++test('directive comments are code: shebangs, encodings, type, lint and build directives', () => {+ const cases = [+ ['a.py', '#!/usr/bin/env python3\nprint(1)', '#!/usr/bin/env -S python3 -X dev\nprint(1)'],+ ['a.py', '# -*- coding: utf-8 -*-\ns = 1', '# -*- coding: latin-1 -*-\ns = 1'],+ ['a.py', '# vim: set fileencoding=utf-8 :\ns = 1', '# vim: set fileencoding=latin-1 :\ns = 1'],+ ['a.py', 'x = f() # type: ignore', 'x = f() # ok'],+ ['a.py', 'import os # noqa: F401', 'import os # unused'],+ ['a.py', 'x = 1 # pylint: disable=invalid-name', 'x = 1 # fine'],+ ['a.py', 'call(cmd, shell=True) # nosec', 'call(cmd, shell=True) # reviewed'],+ ['a.py', 'def f(): # pragma: no cover\n pass', 'def f(): # rarely hit\n pass'],+ ['a.py', 'x = 1', 'x = 1 # fmt: skip'],+ ['a.ts', '// @ts-expect-error\nconst n: number = s;', '// expected\nconst n: number = s;'],+ ['a.ts', 'const n = s; // @ts-ignore', 'const n = s; // ignore'],+ ['a.ts', '/// <reference types="node" />\nx();', '/// <reference types="deno" />\nx();'],+ ['a.js', '/* eslint-disable no-eval */\neval(x);', '/* note */\neval(x);'],+ ['a.js', 'eval(x); // eslint-disable-line', 'eval(x); // reviewed'],+ ['a.js', 'const m = import(/* webpackChunkName: "a" */ "./a");', 'const m = import(/* webpackChunkName: "b" */ "./a");'],+ ['a.js', 'const v = /*#__PURE__*/ make();', 'const v = /* pure */ make();'],+ ['a.js', '// @flow\nx();', '// flow\nx();'],+ ['a.js', '/** @type {number} */\nlet n;', '/** @type {string} */\nlet n;'],+ ['a.js', '/* global jQuery */\njQuery();', '/* jQuery */\njQuery();'],+ ['a.js', 'x(); // istanbul ignore next', 'x(); // rarely'],+ ['a.js', 'x(); // prettier-ignore', 'x(); // keep'],+ ['a.go', '//go:build linux\n\npackage o', '//go:build ignore\n\npackage o'],+ ['a.go', '// +build linux\n\npackage o', '// +build ignore\n\npackage o'],+ ['a.go', '//go:embed a.txt\nvar s string', '//go:embed b.txt\nvar s string'],+ ['a.go', '//go:generate stringer -type=A\ntype A int', '//go:generate rm -rf .\ntype A int'],+ ['a.go', '//export Add\nfunc Add() {}', '// Add adds\nfunc Add() {}'],+ ['a.go', 'x := f() //nolint:errcheck', 'x := f() // fine'],+ ['a.go', '// Code generated by protoc. DO NOT EDIT.\npackage p', '// Written by hand.\npackage p'],+ ['a.rs', '/// ```\n/// assert!(true);\n/// ```\nfn f() {}', '/// ```\n/// assert!(false);\n/// ```\nfn f() {}'],+ ['a.rs', '//! assert!(true);\nfn f() {}', '//! assert!(false);\nfn f() {}'],+ ['a.rs', '/** assert!(true); */\nfn f() {}', '/** assert!(false); */\nfn f() {}'],+ ['a.c', 'case 1: x(); /* fallthrough */\ncase 2: y();', 'case 1: x(); /* next */\ncase 2: y();'],+ ['a.cpp', 'int x = f(); // NOLINT', 'int x = f(); // ok'],+ ['a.java', 'int x = f(); // NOSONAR', 'int x = f(); // ok'],+ ['a.cs', 'var x = f(); // ReSharper disable once All', 'var x = f(); // ok'],+ ['a.kt', 'val x = f() // ktlint-disable', 'val x = f() // ok'],+ ['a.sh', '# shellcheck disable=SC2086\necho $x', '# quoting is fine\necho $x'],+ ['a.sh', 'echo a\n#!/bin/sh', 'echo a\n#!/bin/bash'],+ ['a.ps1', '# PSScriptAnalyzer suppression\nGet-Item a', '# note\nGet-Item a'],+ ['a.bat', 'REM vim: set ff=dos :\necho a', 'REM note\necho a'],+ ['a.py', 'API_KEY = load() # pragma: allowlist secret', 'API_KEY = load() # loaded'],+ ['a.js', 'const k = "x"; // gitleaks:allow', 'const k = "x"; // ok'],+ ['a.go', 'fmt.Println(42)\n// Output: 42', 'fmt.Println(42)\n// Output: 43'],+ ['a.zsh', '#compdef _git git\nx', '#compdef _hg hg\nx']+ ];+ for (const [file, before, after] of cases) {+ assert.strictEqual(trivial(file, before, after), false, `${file}: ${JSON.stringify(before)} -> ${JSON.stringify(after)}`);+ }+});++test('an unchanged directive does not stop a nearby comment edit from being trivial', () => {+ assert.strictEqual(trivial('a.py', '#!/usr/bin/env python3\n# old\nx = 1', '#!/usr/bin/env python3\n# new\nx = 1'), true);+ assert.strictEqual(trivial('a.ts', '// @ts-check\n// old\nx();', '// @ts-check\n// new\nx();'), true);+ assert.strictEqual(trivial('a.sh', '#!/bin/sh\n# old\nmake', '#!/bin/sh\n# new\nmake'), true);+});+test('Write, unknown languages and MultiEdit with any non-trivial entry are not trivial', () => {assert.strictEqual(write('a.js', '// only a comment\n').trivial, false);assert.strictEqual(trivial('a.rb', '# a', '# b'), false);