Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

gitlink: read shell commands with a real shell parser (#1092)

kyle-elliott-asymptote928bc8698a7c
diff --git a/cli/beacon/go.mod b/cli/beacon/go.mod
index 58021a1c..5db9f7e0 100644
--- a/cli/beacon/go.mod
+++ b/cli/beacon/go.mod
@@ -11,9 +11,10 @@ require (
github.com/pelletier/go-toml/v2 v2.3.1
github.com/spf13/cobra v1.8.1
golang.org/x/sys v0.47.0
- golang.org/x/term v0.33.0
+ golang.org/x/term v0.45.0
gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v1.59.0
+ mvdan.cc/sh/v3 v3.14.1
)
replace github.com/asymptote-labs/agent-beacon/pkg/asymptoteobserve => ../../pkg/asymptoteobserve
diff --git a/cli/beacon/go.sum b/cli/beacon/go.sum
index df69c8ff..be15e5d8 100644
--- a/cli/beacon/go.sum
+++ b/cli/beacon/go.sum
@@ -22,6 +22,8 @@ github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkp
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f h1:Y/CXytFA4m6baUTXGLOoWe4PQhGxaX0KpnayAqC48p4=
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM=
+github.com/go-quicktest/qt v1.102.0 h1:HSQxCeh5YZH3EL3W39ixjtyaEhcWSXQHtHnMBzSs474=
+github.com/go-quicktest/qt v1.102.0/go.mod h1:p4lGIVX+8Wa6ZPNDvqcxq36XpUDLh42FLetFU7odllI=
github.com/google/cel-go v0.30.0 h1:ll54AkzKunWkBn9wSoiUXbFZXYZTkdJGNXTBXUoolGo=
github.com/google/cel-go v0.30.0/go.mod h1:X0bD6iVNR8pkROSOoHVdgTkzmRcosof7WQqCD6wcMc8=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
@@ -34,10 +36,12 @@ github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
-github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
-github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/klauspost/compress v1.18.7 h1:aUyZsS4kH3QTKurYhAOwAHxllVPnOthb3vPfnF1Ehjw=
github.com/klauspost/compress v1.18.7/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
+github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
+github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69Aj6K7nkY=
github.com/lucasb-eyer/go-colorful v1.2.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
@@ -61,6 +65,8 @@ github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qq
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
+github.com/rogpeppe/go-internal v1.15.0 h1:D0RCU5rMAp+SpgkiNdrjfJ+LX4J1M32V2NeCY7EJ6hc=
+github.com/rogpeppe/go-internal v1.15.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=
@@ -80,10 +86,8 @@ golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
-golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
-golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
-golang.org/x/text v0.39.0 h1:UbZz4pLOvn600D6Oh6GGEI6VAmndrEBLv8/6BEXzyus=
-golang.org/x/text v0.39.0/go.mod h1:3UwRclnC2g0TU9x8PZiyfOajCd1zaUNHF9cvqcQZ+ZM=
+golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0=
+golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE=
@@ -126,3 +130,5 @@ modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+mvdan.cc/sh/v3 v3.14.1 h1:bXkhQWNHCs0KZEChF8hYS6FC+T2N9mUZLbQv9blditI=
+mvdan.cc/sh/v3 v3.14.1/go.mod h1:syYCoFET8w9tvevxiXUtY8/ICrU+l26jHmhJDra3Vwo=
diff --git a/cli/beacon/internal/gitlink/shell.go b/cli/beacon/internal/gitlink/shell.go
index 3f9c9802..1e9b83d7 100644
--- a/cli/beacon/internal/gitlink/shell.go
+++ b/cli/beacon/internal/gitlink/shell.go
@@ -2,293 +2,444 @@ package gitlink
import (
"path/filepath"
+ "slices"
"strings"
+
+ "mvdan.cc/sh/v3/expand"
+ "mvdan.cc/sh/v3/syntax"
)
// ShellWrittenPaths returns the files a shell command line evidently writes: the operands of rm,
-// touch, truncate, mv, cp, install, tee, `git mv`/`git rm`, in-place sed/perl, and the targets of
-// output redirections. It is deliberately shallow. Anything it cannot read plainly -- variables,
-// subshells, a script that writes files itself -- contributes nothing, because a wrong path would
-// link a session to a commit it had no part in, and a missed one only leaves a link out.
+// touch, truncate, mv, cp, install, ln, tee, `git mv`/`git rm`, in-place sed/perl, and the targets
+// of output redirections. It is deliberately shallow. Anything it cannot read plainly -- variables,
+// command substitutions, globs, a script that writes files itself -- contributes nothing, because
+// a wrong path would link a session to a commit it had no part in, and a missed one only leaves a
+// link out. A command line that does not parse was captured incomplete and contributes nothing.
func ShellWrittenPaths(command string) []string {
+ file, err := syntax.NewParser(syntax.Variant(syntax.LangBash)).Parse(strings.NewReader(command), "")
+ if err != nil {
+ return nil
+ }
var paths []string
moved := false
- for _, segment := range splitShellSegments(command) {
- words, redirected := stripRedirections(shellWords(segment))
- found := append(redirected, commandTargets(words)...)
+ syntax.Walk(file, func(node syntax.Node) bool {
+ stmt, ok := node.(*syntax.Stmt)
+ if !ok {
+ return true
+ }
+ var found []string
+ for _, r := range stmt.Redirs {
+ if target, ok := redirectTarget(r); ok {
+ found = append(found, target)
+ }
+ }
+ call, _ := stmt.Cmd.(*syntax.CallExpr)
+ var args []arg
+ if call != nil {
+ args = callArgs(call)
+ found = append(found, commandTargets(args)...)
+ }
if moved {
// After a cd, a relative path is relative to somewhere this parser does not track.
found = absoluteOnly(found)
}
paths = append(paths, found...)
- if command := unwrappedCommand(words); len(command) > 0 && (command[0] == "cd" || command[0] == "pushd" || command[0] == "popd") {
- moved = true
+ if command := unwrapped(args); len(command) > 0 && command[0].known {
+ switch command[0].text {
+ case "cd", "pushd", "popd":
+ moved = true
+ }
}
- }
+ return true
+ })
return dedupe(paths)
}
-func unwrappedCommand(words []string) []string {
- // Skip environment assignments and the wrappers that run the real command.
- for len(words) > 0 {
- w := words[0]
- if strings.Contains(w, "=") && !strings.HasPrefix(w, "=") && !strings.HasPrefix(w, "-") {
- words = words[1:]
+// arg is one word of a simple command after brace expansion. known is false when its value
+// depends on something this parser does not evaluate: a parameter, a command or arithmetic
+// substitution, a glob, or a leading tilde. An unknown word is never reported as a path.
+type arg struct {
+ text string
+ known bool
+}
+
+func callArgs(call *syntax.CallExpr) []arg {
+ var args []arg
+ for _, word := range call.Args {
+ // SplitBraces rewrites the word's parts, so it works on a copy: the tree is still being walked.
+ split := &syntax.Word{Parts: slices.Clone(word.Parts)}
+ if !syntax.SplitBraces(split) {
+ args = append(args, wordArg(word))
continue
}
- if w == "sudo" || w == "command" || w == "builtin" || w == "exec" || w == "nohup" || w == "time" {
- words = words[1:]
- continue
+ for _, expanded := range expand.Braces(split) {
+ args = append(args, wordArg(expanded))
}
- break
}
- return words
+ return args
}
-func commandTargets(words []string) []string {
- words = unwrappedCommand(words)
- if len(words) == 0 {
- return nil
- }
- name := filepath.Base(words[0])
- args := words[1:]
- if name == "git" && len(args) > 0 && (args[0] == "mv" || args[0] == "rm") {
- name, args = "git-"+args[0], args[1:]
- }
- operands, flags := splitFlags(args)
- switch name {
- case "rm", "unlink", "touch", "truncate", "git-rm":
- return operands
- case "mv", "git-mv":
- // Every operand: the sources stop existing and the destination starts to.
- return operands
- case "cp", "install", "ln":
- if len(operands) >= 2 {
- return operands[len(operands)-1:]
- }
- case "tee":
- return operands
- case "sed", "perl", "gsed":
- if hasInPlaceFlag(flags) && len(operands) >= 2 {
- // The first operand is the script, unless it came in with -e.
- return operands[1:]
+func wordArg(word *syntax.Word) arg {
+ var b strings.Builder
+ for i, part := range word.Parts {
+ switch p := part.(type) {
+ case *syntax.Lit:
+ if i == 0 && strings.HasPrefix(p.Value, "~") {
+ return arg{}
+ }
+ if !unescape(&b, p.Value, false) {
+ return arg{}
+ }
+ case *syntax.SglQuoted:
+ if p.Dollar {
+ return arg{}
+ }
+ b.WriteString(p.Value)
+ case *syntax.DblQuoted:
+ if p.Dollar {
+ return arg{}
+ }
+ for _, inner := range p.Parts {
+ lit, ok := inner.(*syntax.Lit)
+ if !ok {
+ return arg{}
+ }
+ unescape(&b, lit.Value, true)
+ }
+ default:
+ return arg{}
}
}
- return nil
+ return arg{text: b.String(), known: true}
}
-func hasInPlaceFlag(flags []string) bool {
- for _, f := range flags {
- if f == "--in-place" || strings.HasPrefix(f, "--in-place=") {
- return true
+// unescape writes a literal's value as the shell passes it on, removing the backslashes it
+// removes. Outside quotes it reports false for an unescaped glob character, whose value depends on
+// the file system.
+func unescape(b *strings.Builder, s string, quoted bool) bool {
+ for i := 0; i < len(s); i++ {
+ c := s[i]
+ if c == '\\' && i+1 < len(s) && (!quoted || strings.IndexByte("$`\"\\\n", s[i+1]) >= 0) {
+ i++
+ if s[i] != '\n' {
+ b.WriteByte(s[i])
+ }
+ continue
}
- if strings.HasPrefix(f, "-") && !strings.HasPrefix(f, "--") && strings.Contains(f[1:], "i") {
- return true
+ if !quoted && (c == '*' || c == '?' || c == '[') {
+ return false
}
+ b.WriteByte(c)
}
- return false
+ return true
}
-// splitFlags separates operands from flags. Everything after `--` is an operand.
-func splitFlags(args []string) (operands, flags []string) {
- for i, a := range args {
- if a == "--" {
- return append(operands, args[i+1:]...), flags
+// redirectTarget returns the file an output redirection writes. Input redirections, here-documents
+// and here-strings write nothing, and neither does a duplicated descriptor (2>&1) or a device.
+func redirectTarget(r *syntax.Redirect) (string, bool) {
+ switch r.Op {
+ case syntax.RdrOut, syntax.AppOut, syntax.ClbOut, syntax.RdrAll, syntax.AppAll:
+ case syntax.DplOut:
+ // >&word writes a file unless word names a descriptor.
+ if target := wordArg(r.Word); !target.known || target.text == "-" || strings.Trim(target.text, "0123456789") == "" {
+ return "", false
}
- if strings.HasPrefix(a, "-") && a != "-" {
- flags = append(flags, a)
- continue
- }
- operands = append(operands, a)
+ default:
+ return "", false
}
- return operands, flags
+ target := wordArg(r.Word)
+ if !target.known || target.text == "" || strings.HasPrefix(target.text, "/dev/") {
+ return "", false
+ }
+ return target.text, true
}
-// stripRedirections removes output redirections from words and returns the remaining words and
-// the redirections' targets. Only an unquoted word can be a redirection: `echo '> x'` prints.
-func stripRedirections(words []shellWord) (rest, targets []string) {
- for i := 0; i < len(words); i++ {
- w := words[i]
- op, target := "", ""
- if !w.quoted {
- op, target = redirection(w.text)
- }
- if op == "" {
- text := w.text
- if w.expands {
- // Keep unknown words in place: deleting one can promote an argument
- // to the command, or a copy source to the destination.
- text = ""
- if eq := strings.IndexByte(w.text, '='); w.assignment && len(unwrappedCommand(rest)) == 0 {
- // An assignment's value is irrelevant when unwrapping the command.
- text = w.text[:eq+1]
- }
- }
- rest = append(rest, text)
- continue
- }
- if target == "" && i+1 < len(words) {
- i++
- if words[i].expands {
- continue
- }
- target = words[i].text
- }
- if target != "" && !strings.HasPrefix(target, "&") && target != "/dev/null" && !strings.HasPrefix(target, "/dev/") {
- targets = append(targets, target)
+// unwrapped skips the wrappers that run the real command in the same directory. A wrapper option
+// that changes the directory or the command itself (sudo -i, env -C, env -S) leaves no command.
+func unwrapped(args []arg) []arg {
+ for len(args) > 0 && args[0].known {
+ switch args[0].text {
+ case "command", "builtin", "exec", "nohup", "time":
+ args = args[1:]
+ case "sudo":
+ args = afterAssignments(afterOptions(args[1:], wrapperSudo))
+ case "env":
+ args = afterAssignments(afterOptions(args[1:], wrapperEnv))
+ default:
+ return args
}
}
- return rest, targets
+ return args
}
-// redirection recognizes `>`, `>>`, `>|`, `1>`, `2>`, `&>`, with the target attached or not. Input
-// redirection is not a write and is left in place.
-func redirection(word string) (op, target string) {
- for _, prefix := range []string{"&>>", "&>", "1>>", "2>>", "1>", "2>", ">>", ">|", ">"} {
- if strings.HasPrefix(word, prefix) {
- return prefix, word[len(prefix):]
- }
- }
- return "", ""
+type wrapperSpec struct {
+ optionSpec
+ blocking string // short options that move or replace the command
+ blockingLong []string // long forms of the same
}
-// splitShellSegments splits a command line into simple commands at ;, &&, ||, |, & and newlines,
-// outside quotes.
-func splitShellSegments(command string) []string {
- var segments []string
- var cur strings.Builder
- quote := byte(0)
- flush := func() {
- if s := strings.TrimSpace(cur.String()); s != "" {
- segments = append(segments, s)
- }
- cur.Reset()
+var (
+ wrapperSudo = wrapperSpec{
+ optionSpec: optionSpec{short: "CghpRTtUu", long: []string{"--close-from", "--group", "--host", "--prompt", "--role", "--command-timeout", "--type", "--other-user", "--user"}},
+ blocking: "DRi",
+ blockingLong: []string{"--chdir", "--chroot", "--login"},
}
- for i := 0; i < len(command); i++ {
- c := command[i]
+ wrapperEnv = wrapperSpec{
+ optionSpec: optionSpec{short: "u", long: []string{"--unset"}},
+ blocking: "CS",
+ blockingLong: []string{"--chdir", "--split-string"},
+ }
+)
+
+// afterOptions returns the words after a wrapper's own options, or nothing when one of them
+// moves or replaces the command.
+func afterOptions(args []arg, spec wrapperSpec) []arg {
+ for i := 0; i < len(args); i++ {
+ a := args[i]
switch {
- case quote != 0:
- if c == quote {
- quote = 0
- } else if c == '\\' && quote == '"' && i+1 < len(command) {
- cur.WriteByte(c)
- i++
- c = command[i]
+ case !a.known || a.text == "-" || !strings.HasPrefix(a.text, "-"):
+ return args[i:]
+ case a.text == "--":
+ return args[i+1:]
+ case strings.HasPrefix(a.text, "--"):
+ name, _, attached := strings.Cut(a.text, "=")
+ if slices.Contains(spec.blockingLong, name) {
+ return nil
}
- cur.WriteByte(c)
- case c == '\'' || c == '"':
- quote = c
- cur.WriteByte(c)
- case c == '\\' && i+1 < len(command):
- cur.WriteByte(c)
- i++
- cur.WriteByte(command[i])
- case c == '|' && i > 0 && command[i-1] == '>':
- // `>|` is a clobbering redirection, not a pipe.
- cur.WriteByte(c)
- case c == ';' || c == '\n' || c == '|':
- flush()
- case c == '&':
- // `&>` is a redirection, not a separator.
- if i+1 < len(command) && command[i+1] == '>' {
- cur.WriteByte(c)
- continue
- }
- if i > 0 && command[i-1] == '>' {
- cur.WriteByte(c)
- continue
+ if slices.Contains(spec.long, name) && !attached {
+ i++
}
- flush()
default:
- cur.WriteByte(c)
+ for j := 1; j < len(a.text); j++ {
+ if strings.IndexByte(spec.blocking, a.text[j]) >= 0 {
+ return nil
+ }
+ if strings.IndexByte(spec.short, a.text[j]) >= 0 {
+ if j+1 == len(a.text) {
+ i++
+ }
+ break
+ }
+ }
}
}
- flush()
- return segments
+ return nil
}
-// shellWord is one word of a simple command. quoted records that some of it was quoted, which
-// stops it being read as an operator; expands records that its value depends on an expansion ($,
-// `, a glob), which makes it unknown here -- such a word is never reported as a path.
-// assignment means an unquoted shell identifier and equals sign preceded its value.
-type shellWord struct {
- text string
- quoted bool
- expands bool
- assignment bool
+func afterAssignments(args []arg) []arg {
+ for len(args) > 0 && args[0].known {
+ name, _, ok := strings.Cut(args[0].text, "=")
+ if !ok || !isShellName(name) {
+ break
+ }
+ args = args[1:]
+ }
+ return args
}
-// shellWords splits a simple command into words, honoring quotes and backslashes.
-func shellWords(segment string) []shellWord {
- var words []shellWord
- var cur strings.Builder
- inWord, expands, quoted, assignment := false, false, false, false
- quote := byte(0)
- flush := func() {
- if inWord {
- words = append(words, shellWord{text: cur.String(), quoted: quoted, expands: expands, assignment: assignment})
+func isShellName(name string) bool {
+ if name == "" {
+ return false
+ }
+ for i, c := range name {
+ if c != '_' && !(c >= 'a' && c <= 'z') && !(c >= 'A' && c <= 'Z') && !(i > 0 && c >= '0' && c <= '9') {
+ return false
}
- cur.Reset()
- inWord, expands, quoted, assignment = false, false, false, false
}
- for i := 0; i < len(segment); i++ {
- c := segment[i]
+ return true
+}
+
+// optionSpec names a command's options that take a value: a short one takes the rest of its
+// cluster or the next word, a long one what follows `=` or the next word.
+type optionSpec struct {
+ short string
+ long []string
+}
+
+var valueOptions = map[string]optionSpec{
+ "touch": {short: "dtr", long: []string{"--date", "--reference"}},
+ "truncate": {short: "rs", long: []string{"--reference", "--size"}},
+ "cp": {short: "St", long: []string{"--suffix", "--target-directory"}},
+ "mv": {short: "St", long: []string{"--suffix", "--target-directory"}},
+ "ln": {short: "St", long: []string{"--suffix", "--target-directory"}},
+ "install": {short: "gmoSt", long: []string{"--group", "--mode", "--owner", "--suffix", "--target-directory"}},
+}
+
+// parseOptions splits args into operands and the values of spec's options, keyed by the short
+// letter or the long name. Everything after `--` is an operand, and so is an unknown word.
+func parseOptions(args []arg, spec optionSpec) (operands []arg, values map[string]arg) {
+ values = map[string]arg{}
+ for i := 0; i < len(args); i++ {
+ a := args[i]
switch {
- case quote == '\'':
- if c == '\'' {
- quote = 0
- } else {
- cur.WriteByte(c)
+ case !a.known || a.text == "-" || !strings.HasPrefix(a.text, "-"):
+ operands = append(operands, a)
+ case a.text == "--":
+ return append(operands, args[i+1:]...), values
+ case strings.HasPrefix(a.text, "--"):
+ name, value, attached := strings.Cut(a.text, "=")
+ if !slices.Contains(spec.long, name) {
+ continue
}
- case quote == '"':
switch {
- case c == '"':
- quote = 0
- case c == '\\' && i+1 < len(segment):
+ case attached:
+ values[name] = arg{text: value, known: true}
+ case i+1 < len(args):
i++
- cur.WriteByte(segment[i])
- case c == '$' || c == '`':
- expands = true
- cur.WriteByte(c)
+ values[name] = args[i]
default:
- cur.WriteByte(c)
+ values[name] = arg{}
}
- case c == '\'' || c == '"':
- quote, inWord, quoted = c, true, true
- case c == '\\' && i+1 < len(segment):
- i++
- cur.WriteByte(segment[i])
- inWord, quoted = true, true
- case c == ' ' || c == '\t':
- flush()
- case c == '(' || c == ')' || c == '{' || c == '}':
- flush()
default:
- if c == '=' && !quoted && shellAssignmentName(cur.String()) {
- assignment = true
+ for j := 1; j < len(a.text); j++ {
+ c := a.text[j]
+ if strings.IndexByte(spec.short, c) < 0 {
+ continue
+ }
+ switch {
+ case j+1 < len(a.text):
+ values[string(c)] = arg{text: a.text[j+1:], known: true}
+ case i+1 < len(args):
+ i++
+ values[string(c)] = args[i]
+ default:
+ values[string(c)] = arg{}
+ }
+ break
+ }
+ }
+ }
+ return operands, values
+}
+
+func commandTargets(args []arg) []string {
+ args = unwrapped(args)
+ if len(args) == 0 || !args[0].known {
+ return nil
+ }
+ name := filepath.Base(args[0].text)
+ args = args[1:]
+ if name == "git" && len(args) > 0 && args[0].known && (args[0].text == "mv" || args[0].text == "rm") {
+ name, args = "git-"+args[0].text, args[1:]
+ }
+ switch name {
+ case "rm", "unlink", "touch", "truncate", "git-rm", "tee":
+ operands, _ := parseOptions(args, valueOptions[name])
+ return knownTexts(operands)
+ case "mv", "git-mv", "cp", "install", "ln":
+ operands, values := parseOptions(args, valueOptions[name])
+ // mv's sources stop existing, so they are written too.
+ moves := name == "mv" || name == "git-mv"
+ dir, ok := values["t"]
+ if !ok {
+ dir, ok = values["--target-directory"]
+ }
+ if ok {
+ // -t DIR makes every operand a source, written as DIR/<base name>.
+ var written []string
+ if moves {
+ written = knownTexts(operands)
}
- if c == '$' || c == '`' || c == '*' || c == '?' || c == '[' || c == '~' {
- expands = true
+ if !dir.known {
+ return written
}
- cur.WriteByte(c)
- inWord = true
+ for _, src := range operands {
+ if src.known {
+ written = append(written, filepath.Join(dir.text, filepath.Base(src.text)))
+ }
+ }
+ return written
+ }
+ if moves {
+ return knownTexts(operands)
+ }
+ if len(operands) >= 2 {
+ return knownTexts(operands[len(operands)-1:])
}
+ case "sed", "perl", "gsed":
+ return inPlaceFiles(name, args)
}
- flush()
- return words
+ return nil
}
-// shellAssignmentName recognizes the unquoted name before an assignment's equals sign.
-func shellAssignmentName(name string) bool {
- if name == "" {
- return false
+// inPlaceFiles returns the files sed or perl edits in place, and nothing without -i/--in-place.
+// They are every operand when the script came in with -e/-f (attached, separate, or
+// --expression/--file), and every operand after the first, the script, otherwise.
+func inPlaceFiles(name string, args []arg) []string {
+ // A script switch brings the script, attached or as the next word. -i and a valued switch take
+ // the rest of the cluster as their value, so an i or e after one (-i.bak, perl -Mwarnings,
+ // -Ilib) is part of that value.
+ script, valued := "ef", "l"
+ if name == "perl" {
+ script, valued = "eE", "CDFIMdmx0"
}
- for i, c := range name {
- if c != '_' && !(c >= 'a' && c <= 'z') && !(c >= 'A' && c <= 'Z') && !(i > 0 && c >= '0' && c <= '9') {
- return false
+ inPlace, scripted := false, false
+ var operands []arg
+ for i := 0; i < len(args); i++ {
+ a := args[i]
+ switch {
+ case !a.known || a.text == "-" || !strings.HasPrefix(a.text, "-"):
+ operands = append(operands, a)
+ case a.text == "--":
+ operands = append(operands, args[i+1:]...)
+ i = len(args)
+ case a.text == "--in-place" || strings.HasPrefix(a.text, "--in-place="):
+ inPlace = true
+ case a.text == "--expression" || a.text == "--file":
+ scripted = true
+ i++
+ case strings.HasPrefix(a.text, "--expression=") || strings.HasPrefix(a.text, "--file="):
+ scripted = true
+ case a.text == "--line-length":
+ i++
+ case strings.HasPrefix(a.text, "--"):
+ default:
+ for j := 1; j < len(a.text); j++ {
+ c := a.text[j]
+ if c == 'i' {
+ inPlace = true
+ // BSD sed reads its backup suffix from the next word; macOS agents pass ''.
+ if name == "sed" && j+1 == len(a.text) && i+1 < len(args) && args[i+1].known && args[i+1].text == "" {
+ i++
+ }
+ break
+ }
+ if strings.IndexByte(script, c) >= 0 {
+ scripted = true
+ if j+1 == len(a.text) {
+ i++ // the script is the next word
+ }
+ break
+ }
+ if strings.IndexByte(valued, c) >= 0 {
+ if name != "perl" && j+1 == len(a.text) {
+ i++ // sed -l N
+ }
+ break
+ }
+ }
}
}
- return true
+ if !inPlace {
+ return nil
+ }
+ if !scripted {
+ if len(operands) < 2 {
+ return nil
+ }
+ operands = operands[1:]
+ }
+ return knownTexts(operands)
+}
+
+func knownTexts(args []arg) []string {
+ var out []string
+ for _, a := range args {
+ if a.known {
+ out = append(out, a.text)
+ }
+ }
+ return out
}
// PatchPaths returns the files an apply_patch envelope touches: Add, Update, Delete, and the
diff --git a/cli/beacon/internal/gitlink/shell_test.go b/cli/beacon/internal/gitlink/shell_test.go
index a5aac50d..64844b76 100644
--- a/cli/beacon/internal/gitlink/shell_test.go
+++ b/cli/beacon/internal/gitlink/shell_test.go
@@ -17,6 +17,18 @@ func TestShellWrittenPaths(t *testing.T) {
{"git mv src/a.go src/b.go", []string{"src/a.go", "src/b.go"}},
{"git rm --cached secret.env", []string{"secret.env"}},
{"cp -r template out/dir", []string{"out/dir"}},
+ {"cp -t out a.txt src/b.txt", []string{"out/a.txt", "out/b.txt"}},
+ {"cp --target-directory=out a.txt", []string{"out/a.txt"}},
+ {"install -Dt bin tool", []string{"bin/tool"}},
+ {"ln -s -t links ../real.txt", []string{"links/real.txt"}},
+ {"install -m 0755 -t /usr/local/bin build/tool", []string{"/usr/local/bin/tool"}},
+ {"install -o root -g wheel -t /etc/app conf.yaml", []string{"/etc/app/conf.yaml"}},
+ {"install -m 0644 conf.yaml /etc/app/conf.yaml", []string{"/etc/app/conf.yaml"}},
+ {`cp -t "$DEST" a.txt`, nil},
+ {`cp -t out "$SRC" a.txt`, []string{"out/a.txt"}},
+ {"mv -t archive a.txt b.txt", []string{"a.txt", "b.txt", "archive/a.txt", "archive/b.txt"}},
+ {"truncate -s 0 app.log", []string{"app.log"}},
+ {"touch -d '2 hours ago' stamp", []string{"stamp"}},
{"echo hi > out.txt", []string{"out.txt"}},
{"echo hi >out.txt 2>&1", []string{"out.txt"}},
{"printf x >> log.txt; cat log.txt", []string{"log.txt"}},
@@ -28,6 +40,39 @@ func TestShellWrittenPaths(t *testing.T) {
{"sed -i.bak -e 's/a/b/' x.go", []string{"x.go"}},
{"sed 's/a/b/' x.go", nil}, // not in place: prints, writes nothing
{"perl -pi -e 's/a/b/' z.pl", []string{"z.pl"}},
+ {"perl -Mwarnings -ne 'print if /x/' z.pl", nil}, // -M's module name is not -i
+ {"perl -Ilib -ne 'print' z.pl", nil},
+ {"perl -MList::Util=sum -lane 'print sum @F' data.txt", nil},
+ {"perl -Mstrict -pi -e 's/a/b/' z.pl", []string{"z.pl"}},
+ {"perl -0777 -pi -e 's/a/b/' z.pl", []string{"z.pl"}},
+ {"perl -lpi -e 's/a/b/' z.pl", []string{"z.pl"}},
+ {"sed -n -es/i/j/p x.go y.go", nil}, // -e's script is not -i
+ {"perl -pi -e's/a/b/' z.pl", []string{"z.pl"}},
+ {"sed -i -e's/a/b/' x.go", []string{"x.go"}},
+ {"sed -i --expression=s/a/b/ x.go", []string{"x.go"}},
+ {"sed -i -e 's/a/b/' -e 's/c/d/' x.go", []string{"x.go"}},
+ {"sed -i -f fix.sed x.go", []string{"x.go"}},
+ {"sed -ie 's/a/b/' x.go", []string{"x.go"}}, // GNU: the suffix is "e"
+ {"sed -E -i 's/a/b/' x.go", []string{"x.go"}},
+ {"sed -i 's/x/y/' f", []string{"f"}},
+ {"sed -i '' 's/x/y/' f", []string{"f"}}, // BSD sed: '' is the backup suffix
+ {"sed -i '' 's/x/y/' f g", []string{"f", "g"}},
+ {"sed -i '' -e 's/a/b/' a.go", []string{"a.go"}},
+ {`sed -i "s/$old/$new/" f`, []string{"f"}}, // an unknown script is not a suffix
+ {`sed -i "s/$a/$b/" x y`, []string{"x", "y"}},
+ {"sudo -u deploy rm /srv/app/old.txt", []string{"/srv/app/old.txt"}},
+ {"sudo -E tee /etc/app.conf < in.txt", []string{"/etc/app.conf"}},
+ {"sudo -i rm old.txt", nil}, // a login shell starts in the target user's home
+ {"env FOO=1 rm a.txt", []string{"a.txt"}},
+ {"env -C sub rm a.txt", nil},
+ {"rm a.txt # cleanup b.txt", []string{"a.txt"}},
+ {"rm {a,b}.txt", []string{"a.txt", "b.txt"}},
+ {"if true; then rm a.txt; fi", []string{"a.txt"}},
+ {"for f in *.tmp; do rm \"$f\"; done", nil},
+ {"echo $((1<<3)) > n.txt\nrm old.txt", []string{"n.txt", "old.txt"}},
+ {"grep x <<< \"$v\"\nrm a.txt", []string{"a.txt"}},
+ {"git commit -m \"$(cat <<'EOF'\nrm -rf dist\nEOF\n)\"", nil},
+ {"rm a.txt 'unterminated", nil}, // a command that does not parse was captured incomplete
{"FOO=1 sudo rm build/out", []string{"build/out"}},
{`rm "file with spaces.txt"`, []string{"file with spaces.txt"}},
{`rm file\ two.txt`, []string{"file two.txt"}},
@@ -58,7 +103,20 @@ func TestShellWrittenPaths(t *testing.T) {
{"touch before.txt; command pushd sub; rm after.txt", []string{"before.txt"}},
{"command popd; rm after.txt", nil},
{"ls -la; go test ./...", nil},
+ {"tee -a log.txt < output.txt", []string{"log.txt"}},
+ {"tee log.txt <output.txt", []string{"log.txt"}},
+ {"tee copy.txt <<< hello", []string{"copy.txt"}},
+ {"echo '<' > lt.txt", []string{"lt.txt"}},
+ {"tee '<' out.txt", []string{"<", "out.txt"}},
+ {"cp a.txt b.txt < /dev/null", []string{"b.txt"}},
+ {"sudo tee /etc/app.conf > /dev/null <<'EOF'\nkey: value\nEOF", []string{"/etc/app.conf"}},
{"cat <<EOF > gen.txt\nhello\nEOF", []string{"gen.txt"}},
+ {"cat > deploy.sh <<'EOF'\nrm -rf dist\necho done > status.txt\nEOF\nchmod +x deploy.sh", []string{"deploy.sh"}},
+ {"cat <<-END > a.txt\n\trm b.txt\n\tEND\nrm c.txt", []string{"a.txt", "c.txt"}},
+ {"cat <<A <<\"B\" > out.txt\ntouch x.txt\nA\ntouch y.txt\nB\ntouch z.txt", []string{"out.txt", "z.txt"}},
+ {"cat > a.md <<'EOF'\nDon't touch\nEOF\nrm b.txt", []string{"a.md", "b.txt"}},
+ {"grep -c x <<< 'rm here.txt'", nil},
+ {"tee x.txt <<<\"hello\"\nrm y.txt", []string{"x.txt", "y.txt"}},
} {
got := ShellWrittenPaths(tc.command)
if len(got) == 0 && len(tc.want) == 0 {
diff --git a/docs/cli/git.mdx b/docs/cli/git.mdx
index 8b6a238a..11ce64ee 100644
--- a/docs/cli/git.mdx
+++ b/docs/cli/git.mdx
@@ -26,7 +26,7 @@ A link names a session by its harness and session id, for example `beacon:claude
Beacon already records every file a supported runtime writes, so matching a commit to its sessions is a read of the local runtime log. No state file is kept and nothing touches the network.
1. **Window.** Beacon reads the log from the parent commit's time up to the commit. The window always covers at least `--min-lookback` (2 hours) and never more than `--max-lookback` (24 hours) before the commit. Events after the commit are ignored, because they cannot have produced it.
-2. **Evidence.** A session wrote a file when the log has, for that session, a `file.created`, `file.modified` or `file.deleted` event, a tool event whose `file.operation` is a write, a shell command whose target is plain to read (`rm`, `mv`, `cp`, `touch`, `tee`, `sed -i`, `git mv`, output redirection), or an `apply_patch` envelope. Reads do not count.
+2. **Evidence.** A session wrote a file when the log has, for that session, a `file.created`, `file.modified` or `file.deleted` event, a tool event whose `file.operation` is a write, a shell command whose target is plain to read (`rm`, `mv`, `cp`, `install`, `ln`, `touch`, `truncate`, `tee`, `sed -i`, `perl -pi`, `git mv`, `git rm`, output redirection), or an `apply_patch` envelope. Reads do not count, and neither does a path that depends on a variable, a glob or a command substitution.
3. **Overlap.** Beacon compares those paths with the files the commit changes, counting both sides of a rename. A session that wrote at least one of them is linked. Sessions are ordered by how many of the commit's files they wrote, then by how recently.
For a merge commit, only files where the result differs from every parent count: conflict resolutions and edits made while merging. The files a merge brings in were written by the other side's commits, so a clean merge or a `git pull` links nothing.