The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
gitlink: read shell commands with a real shell parser (#1092)
kyle-elliott-asymptote928bc8698a7c
diff --git a/cli/beacon/go.mod b/cli/beacon/go.modindex 58021a1c..5db9f7e0 100644--- a/cli/beacon/go.mod+++ b/cli/beacon/go.mod@@ -11,9 +11,10 @@ require (github.com/pelletier/go-toml/v2 v2.3.1github.com/spf13/cobra v1.8.1golang.org/x/sys v0.47.0- golang.org/x/term v0.33.0+ golang.org/x/term v0.45.0gopkg.in/yaml.v3 v3.0.1modernc.org/sqlite v1.59.0+ mvdan.cc/sh/v3 v3.14.1)replace github.com/asymptote-labs/agent-beacon/pkg/asymptoteobserve => ../../pkg/asymptoteobservediff --git a/cli/beacon/go.sum b/cli/beacon/go.sumindex df69c8ff..be15e5d8 100644--- a/cli/beacon/go.sum+++ b/cli/beacon/go.sum@@ -22,6 +22,8 @@ github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpgithub.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f h1:Y/CXytFA4m6baUTXGLOoWe4PQhGxaX0KpnayAqC48p4=github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM=+github.com/go-quicktest/qt v1.102.0 h1:HSQxCeh5YZH3EL3W39ixjtyaEhcWSXQHtHnMBzSs474=+github.com/go-quicktest/qt v1.102.0/go.mod h1:p4lGIVX+8Wa6ZPNDvqcxq36XpUDLh42FLetFU7odllI=github.com/google/cel-go v0.30.0 h1:ll54AkzKunWkBn9wSoiUXbFZXYZTkdJGNXTBXUoolGo=github.com/google/cel-go v0.30.0/go.mod h1:X0bD6iVNR8pkROSOoHVdgTkzmRcosof7WQqCD6wcMc8=github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=@@ -34,10 +36,12 @@ github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFsgithub.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=-github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=-github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=github.com/klauspost/compress v1.18.7 h1:aUyZsS4kH3QTKurYhAOwAHxllVPnOthb3vPfnF1Ehjw=github.com/klauspost/compress v1.18.7/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=+github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=+github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69Aj6K7nkY=github.com/lucasb-eyer/go-colorful v1.2.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=@@ -61,6 +65,8 @@ github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqgithub.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=+github.com/rogpeppe/go-internal v1.15.0 h1:D0RCU5rMAp+SpgkiNdrjfJ+LX4J1M32V2NeCY7EJ6hc=+github.com/rogpeppe/go-internal v1.15.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs=github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=@@ -80,10 +86,8 @@ golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=-golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=-golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=-golang.org/x/text v0.39.0 h1:UbZz4pLOvn600D6Oh6GGEI6VAmndrEBLv8/6BEXzyus=-golang.org/x/text v0.39.0/go.mod h1:3UwRclnC2g0TU9x8PZiyfOajCd1zaUNHF9cvqcQZ+ZM=+golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0=+golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE=@@ -126,3 +130,5 @@ modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=+mvdan.cc/sh/v3 v3.14.1 h1:bXkhQWNHCs0KZEChF8hYS6FC+T2N9mUZLbQv9blditI=+mvdan.cc/sh/v3 v3.14.1/go.mod h1:syYCoFET8w9tvevxiXUtY8/ICrU+l26jHmhJDra3Vwo=diff --git a/cli/beacon/internal/gitlink/shell.go b/cli/beacon/internal/gitlink/shell.goindex 3f9c9802..1e9b83d7 100644--- a/cli/beacon/internal/gitlink/shell.go+++ b/cli/beacon/internal/gitlink/shell.go@@ -2,293 +2,444 @@ package gitlinkimport ("path/filepath"+ "slices""strings"++ "mvdan.cc/sh/v3/expand"+ "mvdan.cc/sh/v3/syntax")// ShellWrittenPaths returns the files a shell command line evidently writes: the operands of rm,-// touch, truncate, mv, cp, install, tee, `git mv`/`git rm`, in-place sed/perl, and the targets of-// output redirections. It is deliberately shallow. Anything it cannot read plainly -- variables,-// subshells, a script that writes files itself -- contributes nothing, because a wrong path would-// link a session to a commit it had no part in, and a missed one only leaves a link out.+// touch, truncate, mv, cp, install, ln, tee, `git mv`/`git rm`, in-place sed/perl, and the targets+// of output redirections. It is deliberately shallow. Anything it cannot read plainly -- variables,+// command substitutions, globs, a script that writes files itself -- contributes nothing, because+// a wrong path would link a session to a commit it had no part in, and a missed one only leaves a+// link out. A command line that does not parse was captured incomplete and contributes nothing.func ShellWrittenPaths(command string) []string {+ file, err := syntax.NewParser(syntax.Variant(syntax.LangBash)).Parse(strings.NewReader(command), "")+ if err != nil {+ return nil+ }var paths []stringmoved := false- for _, segment := range splitShellSegments(command) {- words, redirected := stripRedirections(shellWords(segment))- found := append(redirected, commandTargets(words)...)+ syntax.Walk(file, func(node syntax.Node) bool {+ stmt, ok := node.(*syntax.Stmt)+ if !ok {+ return true+ }+ var found []string+ for _, r := range stmt.Redirs {+ if target, ok := redirectTarget(r); ok {+ found = append(found, target)+ }+ }+ call, _ := stmt.Cmd.(*syntax.CallExpr)+ var args []arg+ if call != nil {+ args = callArgs(call)+ found = append(found, commandTargets(args)...)+ }if moved {// After a cd, a relative path is relative to somewhere this parser does not track.found = absoluteOnly(found)}paths = append(paths, found...)- if command := unwrappedCommand(words); len(command) > 0 && (command[0] == "cd" || command[0] == "pushd" || command[0] == "popd") {- moved = true+ if command := unwrapped(args); len(command) > 0 && command[0].known {+ switch command[0].text {+ case "cd", "pushd", "popd":+ moved = true+ }}- }+ return true+ })return dedupe(paths)}-func unwrappedCommand(words []string) []string {- // Skip environment assignments and the wrappers that run the real command.- for len(words) > 0 {- w := words[0]- if strings.Contains(w, "=") && !strings.HasPrefix(w, "=") && !strings.HasPrefix(w, "-") {- words = words[1:]+// arg is one word of a simple command after brace expansion. known is false when its value+// depends on something this parser does not evaluate: a parameter, a command or arithmetic+// substitution, a glob, or a leading tilde. An unknown word is never reported as a path.+type arg struct {+ text string+ known bool+}++func callArgs(call *syntax.CallExpr) []arg {+ var args []arg+ for _, word := range call.Args {+ // SplitBraces rewrites the word's parts, so it works on a copy: the tree is still being walked.+ split := &syntax.Word{Parts: slices.Clone(word.Parts)}+ if !syntax.SplitBraces(split) {+ args = append(args, wordArg(word))continue}- if w == "sudo" || w == "command" || w == "builtin" || w == "exec" || w == "nohup" || w == "time" {- words = words[1:]- continue+ for _, expanded := range expand.Braces(split) {+ args = append(args, wordArg(expanded))}- break}- return words+ return args}-func commandTargets(words []string) []string {- words = unwrappedCommand(words)- if len(words) == 0 {- return nil- }- name := filepath.Base(words[0])- args := words[1:]- if name == "git" && len(args) > 0 && (args[0] == "mv" || args[0] == "rm") {- name, args = "git-"+args[0], args[1:]- }- operands, flags := splitFlags(args)- switch name {- case "rm", "unlink", "touch", "truncate", "git-rm":- return operands- case "mv", "git-mv":- // Every operand: the sources stop existing and the destination starts to.- return operands- case "cp", "install", "ln":- if len(operands) >= 2 {- return operands[len(operands)-1:]- }- case "tee":- return operands- case "sed", "perl", "gsed":- if hasInPlaceFlag(flags) && len(operands) >= 2 {- // The first operand is the script, unless it came in with -e.- return operands[1:]+func wordArg(word *syntax.Word) arg {+ var b strings.Builder+ for i, part := range word.Parts {+ switch p := part.(type) {+ case *syntax.Lit:+ if i == 0 && strings.HasPrefix(p.Value, "~") {+ return arg{}+ }+ if !unescape(&b, p.Value, false) {+ return arg{}+ }+ case *syntax.SglQuoted:+ if p.Dollar {+ return arg{}+ }+ b.WriteString(p.Value)+ case *syntax.DblQuoted:+ if p.Dollar {+ return arg{}+ }+ for _, inner := range p.Parts {+ lit, ok := inner.(*syntax.Lit)+ if !ok {+ return arg{}+ }+ unescape(&b, lit.Value, true)+ }+ default:+ return arg{}}}- return nil+ return arg{text: b.String(), known: true}}-func hasInPlaceFlag(flags []string) bool {- for _, f := range flags {- if f == "--in-place" || strings.HasPrefix(f, "--in-place=") {- return true+// unescape writes a literal's value as the shell passes it on, removing the backslashes it+// removes. Outside quotes it reports false for an unescaped glob character, whose value depends on+// the file system.+func unescape(b *strings.Builder, s string, quoted bool) bool {+ for i := 0; i < len(s); i++ {+ c := s[i]+ if c == '\\' && i+1 < len(s) && (!quoted || strings.IndexByte("$`\"\\\n", s[i+1]) >= 0) {+ i+++ if s[i] != '\n' {+ b.WriteByte(s[i])+ }+ continue}- if strings.HasPrefix(f, "-") && !strings.HasPrefix(f, "--") && strings.Contains(f[1:], "i") {- return true+ if !quoted && (c == '*' || c == '?' || c == '[') {+ return false}+ b.WriteByte(c)}- return false+ return true}-// splitFlags separates operands from flags. Everything after `--` is an operand.-func splitFlags(args []string) (operands, flags []string) {- for i, a := range args {- if a == "--" {- return append(operands, args[i+1:]...), flags+// redirectTarget returns the file an output redirection writes. Input redirections, here-documents+// and here-strings write nothing, and neither does a duplicated descriptor (2>&1) or a device.+func redirectTarget(r *syntax.Redirect) (string, bool) {+ switch r.Op {+ case syntax.RdrOut, syntax.AppOut, syntax.ClbOut, syntax.RdrAll, syntax.AppAll:+ case syntax.DplOut:+ // >&word writes a file unless word names a descriptor.+ if target := wordArg(r.Word); !target.known || target.text == "-" || strings.Trim(target.text, "0123456789") == "" {+ return "", false}- if strings.HasPrefix(a, "-") && a != "-" {- flags = append(flags, a)- continue- }- operands = append(operands, a)+ default:+ return "", false}- return operands, flags+ target := wordArg(r.Word)+ if !target.known || target.text == "" || strings.HasPrefix(target.text, "/dev/") {+ return "", false+ }+ return target.text, true}-// stripRedirections removes output redirections from words and returns the remaining words and-// the redirections' targets. Only an unquoted word can be a redirection: `echo '> x'` prints.-func stripRedirections(words []shellWord) (rest, targets []string) {- for i := 0; i < len(words); i++ {- w := words[i]- op, target := "", ""- if !w.quoted {- op, target = redirection(w.text)- }- if op == "" {- text := w.text- if w.expands {- // Keep unknown words in place: deleting one can promote an argument- // to the command, or a copy source to the destination.- text = ""- if eq := strings.IndexByte(w.text, '='); w.assignment && len(unwrappedCommand(rest)) == 0 {- // An assignment's value is irrelevant when unwrapping the command.- text = w.text[:eq+1]- }- }- rest = append(rest, text)- continue- }- if target == "" && i+1 < len(words) {- i++- if words[i].expands {- continue- }- target = words[i].text- }- if target != "" && !strings.HasPrefix(target, "&") && target != "/dev/null" && !strings.HasPrefix(target, "/dev/") {- targets = append(targets, target)+// unwrapped skips the wrappers that run the real command in the same directory. A wrapper option+// that changes the directory or the command itself (sudo -i, env -C, env -S) leaves no command.+func unwrapped(args []arg) []arg {+ for len(args) > 0 && args[0].known {+ switch args[0].text {+ case "command", "builtin", "exec", "nohup", "time":+ args = args[1:]+ case "sudo":+ args = afterAssignments(afterOptions(args[1:], wrapperSudo))+ case "env":+ args = afterAssignments(afterOptions(args[1:], wrapperEnv))+ default:+ return args}}- return rest, targets+ return args}-// redirection recognizes `>`, `>>`, `>|`, `1>`, `2>`, `&>`, with the target attached or not. Input-// redirection is not a write and is left in place.-func redirection(word string) (op, target string) {- for _, prefix := range []string{"&>>", "&>", "1>>", "2>>", "1>", "2>", ">>", ">|", ">"} {- if strings.HasPrefix(word, prefix) {- return prefix, word[len(prefix):]- }- }- return "", ""+type wrapperSpec struct {+ optionSpec+ blocking string // short options that move or replace the command+ blockingLong []string // long forms of the same}-// splitShellSegments splits a command line into simple commands at ;, &&, ||, |, & and newlines,-// outside quotes.-func splitShellSegments(command string) []string {- var segments []string- var cur strings.Builder- quote := byte(0)- flush := func() {- if s := strings.TrimSpace(cur.String()); s != "" {- segments = append(segments, s)- }- cur.Reset()+var (+ wrapperSudo = wrapperSpec{+ optionSpec: optionSpec{short: "CghpRTtUu", long: []string{"--close-from", "--group", "--host", "--prompt", "--role", "--command-timeout", "--type", "--other-user", "--user"}},+ blocking: "DRi",+ blockingLong: []string{"--chdir", "--chroot", "--login"},}- for i := 0; i < len(command); i++ {- c := command[i]+ wrapperEnv = wrapperSpec{+ optionSpec: optionSpec{short: "u", long: []string{"--unset"}},+ blocking: "CS",+ blockingLong: []string{"--chdir", "--split-string"},+ }+)++// afterOptions returns the words after a wrapper's own options, or nothing when one of them+// moves or replaces the command.+func afterOptions(args []arg, spec wrapperSpec) []arg {+ for i := 0; i < len(args); i++ {+ a := args[i]switch {- case quote != 0:- if c == quote {- quote = 0- } else if c == '\\' && quote == '"' && i+1 < len(command) {- cur.WriteByte(c)- i++- c = command[i]+ case !a.known || a.text == "-" || !strings.HasPrefix(a.text, "-"):+ return args[i:]+ case a.text == "--":+ return args[i+1:]+ case strings.HasPrefix(a.text, "--"):+ name, _, attached := strings.Cut(a.text, "=")+ if slices.Contains(spec.blockingLong, name) {+ return nil}- cur.WriteByte(c)- case c == '\'' || c == '"':- quote = c- cur.WriteByte(c)- case c == '\\' && i+1 < len(command):- cur.WriteByte(c)- i++- cur.WriteByte(command[i])- case c == '|' && i > 0 && command[i-1] == '>':- // `>|` is a clobbering redirection, not a pipe.- cur.WriteByte(c)- case c == ';' || c == '\n' || c == '|':- flush()- case c == '&':- // `&>` is a redirection, not a separator.- if i+1 < len(command) && command[i+1] == '>' {- cur.WriteByte(c)- continue- }- if i > 0 && command[i-1] == '>' {- cur.WriteByte(c)- continue+ if slices.Contains(spec.long, name) && !attached {+ i++}- flush()default:- cur.WriteByte(c)+ for j := 1; j < len(a.text); j++ {+ if strings.IndexByte(spec.blocking, a.text[j]) >= 0 {+ return nil+ }+ if strings.IndexByte(spec.short, a.text[j]) >= 0 {+ if j+1 == len(a.text) {+ i+++ }+ break+ }+ }}}- flush()- return segments+ return nil}-// shellWord is one word of a simple command. quoted records that some of it was quoted, which-// stops it being read as an operator; expands records that its value depends on an expansion ($,-// `, a glob), which makes it unknown here -- such a word is never reported as a path.-// assignment means an unquoted shell identifier and equals sign preceded its value.-type shellWord struct {- text string- quoted bool- expands bool- assignment bool+func afterAssignments(args []arg) []arg {+ for len(args) > 0 && args[0].known {+ name, _, ok := strings.Cut(args[0].text, "=")+ if !ok || !isShellName(name) {+ break+ }+ args = args[1:]+ }+ return args}-// shellWords splits a simple command into words, honoring quotes and backslashes.-func shellWords(segment string) []shellWord {- var words []shellWord- var cur strings.Builder- inWord, expands, quoted, assignment := false, false, false, false- quote := byte(0)- flush := func() {- if inWord {- words = append(words, shellWord{text: cur.String(), quoted: quoted, expands: expands, assignment: assignment})+func isShellName(name string) bool {+ if name == "" {+ return false+ }+ for i, c := range name {+ if c != '_' && !(c >= 'a' && c <= 'z') && !(c >= 'A' && c <= 'Z') && !(i > 0 && c >= '0' && c <= '9') {+ return false}- cur.Reset()- inWord, expands, quoted, assignment = false, false, false, false}- for i := 0; i < len(segment); i++ {- c := segment[i]+ return true+}++// optionSpec names a command's options that take a value: a short one takes the rest of its+// cluster or the next word, a long one what follows `=` or the next word.+type optionSpec struct {+ short string+ long []string+}++var valueOptions = map[string]optionSpec{+ "touch": {short: "dtr", long: []string{"--date", "--reference"}},+ "truncate": {short: "rs", long: []string{"--reference", "--size"}},+ "cp": {short: "St", long: []string{"--suffix", "--target-directory"}},+ "mv": {short: "St", long: []string{"--suffix", "--target-directory"}},+ "ln": {short: "St", long: []string{"--suffix", "--target-directory"}},+ "install": {short: "gmoSt", long: []string{"--group", "--mode", "--owner", "--suffix", "--target-directory"}},+}++// parseOptions splits args into operands and the values of spec's options, keyed by the short+// letter or the long name. Everything after `--` is an operand, and so is an unknown word.+func parseOptions(args []arg, spec optionSpec) (operands []arg, values map[string]arg) {+ values = map[string]arg{}+ for i := 0; i < len(args); i++ {+ a := args[i]switch {- case quote == '\'':- if c == '\'' {- quote = 0- } else {- cur.WriteByte(c)+ case !a.known || a.text == "-" || !strings.HasPrefix(a.text, "-"):+ operands = append(operands, a)+ case a.text == "--":+ return append(operands, args[i+1:]...), values+ case strings.HasPrefix(a.text, "--"):+ name, value, attached := strings.Cut(a.text, "=")+ if !slices.Contains(spec.long, name) {+ continue}- case quote == '"':switch {- case c == '"':- quote = 0- case c == '\\' && i+1 < len(segment):+ case attached:+ values[name] = arg{text: value, known: true}+ case i+1 < len(args):i++- cur.WriteByte(segment[i])- case c == '$' || c == '`':- expands = true- cur.WriteByte(c)+ values[name] = args[i]default:- cur.WriteByte(c)+ values[name] = arg{}}- case c == '\'' || c == '"':- quote, inWord, quoted = c, true, true- case c == '\\' && i+1 < len(segment):- i++- cur.WriteByte(segment[i])- inWord, quoted = true, true- case c == ' ' || c == '\t':- flush()- case c == '(' || c == ')' || c == '{' || c == '}':- flush()default:- if c == '=' && !quoted && shellAssignmentName(cur.String()) {- assignment = true+ for j := 1; j < len(a.text); j++ {+ c := a.text[j]+ if strings.IndexByte(spec.short, c) < 0 {+ continue+ }+ switch {+ case j+1 < len(a.text):+ values[string(c)] = arg{text: a.text[j+1:], known: true}+ case i+1 < len(args):+ i+++ values[string(c)] = args[i]+ default:+ values[string(c)] = arg{}+ }+ break+ }+ }+ }+ return operands, values+}++func commandTargets(args []arg) []string {+ args = unwrapped(args)+ if len(args) == 0 || !args[0].known {+ return nil+ }+ name := filepath.Base(args[0].text)+ args = args[1:]+ if name == "git" && len(args) > 0 && args[0].known && (args[0].text == "mv" || args[0].text == "rm") {+ name, args = "git-"+args[0].text, args[1:]+ }+ switch name {+ case "rm", "unlink", "touch", "truncate", "git-rm", "tee":+ operands, _ := parseOptions(args, valueOptions[name])+ return knownTexts(operands)+ case "mv", "git-mv", "cp", "install", "ln":+ operands, values := parseOptions(args, valueOptions[name])+ // mv's sources stop existing, so they are written too.+ moves := name == "mv" || name == "git-mv"+ dir, ok := values["t"]+ if !ok {+ dir, ok = values["--target-directory"]+ }+ if ok {+ // -t DIR makes every operand a source, written as DIR/<base name>.+ var written []string+ if moves {+ written = knownTexts(operands)}- if c == '$' || c == '`' || c == '*' || c == '?' || c == '[' || c == '~' {- expands = true+ if !dir.known {+ return written}- cur.WriteByte(c)- inWord = true+ for _, src := range operands {+ if src.known {+ written = append(written, filepath.Join(dir.text, filepath.Base(src.text)))+ }+ }+ return written+ }+ if moves {+ return knownTexts(operands)+ }+ if len(operands) >= 2 {+ return knownTexts(operands[len(operands)-1:])}+ case "sed", "perl", "gsed":+ return inPlaceFiles(name, args)}- flush()- return words+ return nil}-// shellAssignmentName recognizes the unquoted name before an assignment's equals sign.-func shellAssignmentName(name string) bool {- if name == "" {- return false+// inPlaceFiles returns the files sed or perl edits in place, and nothing without -i/--in-place.+// They are every operand when the script came in with -e/-f (attached, separate, or+// --expression/--file), and every operand after the first, the script, otherwise.+func inPlaceFiles(name string, args []arg) []string {+ // A script switch brings the script, attached or as the next word. -i and a valued switch take+ // the rest of the cluster as their value, so an i or e after one (-i.bak, perl -Mwarnings,+ // -Ilib) is part of that value.+ script, valued := "ef", "l"+ if name == "perl" {+ script, valued = "eE", "CDFIMdmx0"}- for i, c := range name {- if c != '_' && !(c >= 'a' && c <= 'z') && !(c >= 'A' && c <= 'Z') && !(i > 0 && c >= '0' && c <= '9') {- return false+ inPlace, scripted := false, false+ var operands []arg+ for i := 0; i < len(args); i++ {+ a := args[i]+ switch {+ case !a.known || a.text == "-" || !strings.HasPrefix(a.text, "-"):+ operands = append(operands, a)+ case a.text == "--":+ operands = append(operands, args[i+1:]...)+ i = len(args)+ case a.text == "--in-place" || strings.HasPrefix(a.text, "--in-place="):+ inPlace = true+ case a.text == "--expression" || a.text == "--file":+ scripted = true+ i+++ case strings.HasPrefix(a.text, "--expression=") || strings.HasPrefix(a.text, "--file="):+ scripted = true+ case a.text == "--line-length":+ i+++ case strings.HasPrefix(a.text, "--"):+ default:+ for j := 1; j < len(a.text); j++ {+ c := a.text[j]+ if c == 'i' {+ inPlace = true+ // BSD sed reads its backup suffix from the next word; macOS agents pass ''.+ if name == "sed" && j+1 == len(a.text) && i+1 < len(args) && args[i+1].known && args[i+1].text == "" {+ i+++ }+ break+ }+ if strings.IndexByte(script, c) >= 0 {+ scripted = true+ if j+1 == len(a.text) {+ i++ // the script is the next word+ }+ break+ }+ if strings.IndexByte(valued, c) >= 0 {+ if name != "perl" && j+1 == len(a.text) {+ i++ // sed -l N+ }+ break+ }+ }}}- return true+ if !inPlace {+ return nil+ }+ if !scripted {+ if len(operands) < 2 {+ return nil+ }+ operands = operands[1:]+ }+ return knownTexts(operands)+}++func knownTexts(args []arg) []string {+ var out []string+ for _, a := range args {+ if a.known {+ out = append(out, a.text)+ }+ }+ return out}// PatchPaths returns the files an apply_patch envelope touches: Add, Update, Delete, and thediff --git a/cli/beacon/internal/gitlink/shell_test.go b/cli/beacon/internal/gitlink/shell_test.goindex a5aac50d..64844b76 100644--- a/cli/beacon/internal/gitlink/shell_test.go+++ b/cli/beacon/internal/gitlink/shell_test.go@@ -17,6 +17,18 @@ func TestShellWrittenPaths(t *testing.T) {{"git mv src/a.go src/b.go", []string{"src/a.go", "src/b.go"}},{"git rm --cached secret.env", []string{"secret.env"}},{"cp -r template out/dir", []string{"out/dir"}},+ {"cp -t out a.txt src/b.txt", []string{"out/a.txt", "out/b.txt"}},+ {"cp --target-directory=out a.txt", []string{"out/a.txt"}},+ {"install -Dt bin tool", []string{"bin/tool"}},+ {"ln -s -t links ../real.txt", []string{"links/real.txt"}},+ {"install -m 0755 -t /usr/local/bin build/tool", []string{"/usr/local/bin/tool"}},+ {"install -o root -g wheel -t /etc/app conf.yaml", []string{"/etc/app/conf.yaml"}},+ {"install -m 0644 conf.yaml /etc/app/conf.yaml", []string{"/etc/app/conf.yaml"}},+ {`cp -t "$DEST" a.txt`, nil},+ {`cp -t out "$SRC" a.txt`, []string{"out/a.txt"}},+ {"mv -t archive a.txt b.txt", []string{"a.txt", "b.txt", "archive/a.txt", "archive/b.txt"}},+ {"truncate -s 0 app.log", []string{"app.log"}},+ {"touch -d '2 hours ago' stamp", []string{"stamp"}},{"echo hi > out.txt", []string{"out.txt"}},{"echo hi >out.txt 2>&1", []string{"out.txt"}},{"printf x >> log.txt; cat log.txt", []string{"log.txt"}},@@ -28,6 +40,39 @@ func TestShellWrittenPaths(t *testing.T) {{"sed -i.bak -e 's/a/b/' x.go", []string{"x.go"}},{"sed 's/a/b/' x.go", nil}, // not in place: prints, writes nothing{"perl -pi -e 's/a/b/' z.pl", []string{"z.pl"}},+ {"perl -Mwarnings -ne 'print if /x/' z.pl", nil}, // -M's module name is not -i+ {"perl -Ilib -ne 'print' z.pl", nil},+ {"perl -MList::Util=sum -lane 'print sum @F' data.txt", nil},+ {"perl -Mstrict -pi -e 's/a/b/' z.pl", []string{"z.pl"}},+ {"perl -0777 -pi -e 's/a/b/' z.pl", []string{"z.pl"}},+ {"perl -lpi -e 's/a/b/' z.pl", []string{"z.pl"}},+ {"sed -n -es/i/j/p x.go y.go", nil}, // -e's script is not -i+ {"perl -pi -e's/a/b/' z.pl", []string{"z.pl"}},+ {"sed -i -e's/a/b/' x.go", []string{"x.go"}},+ {"sed -i --expression=s/a/b/ x.go", []string{"x.go"}},+ {"sed -i -e 's/a/b/' -e 's/c/d/' x.go", []string{"x.go"}},+ {"sed -i -f fix.sed x.go", []string{"x.go"}},+ {"sed -ie 's/a/b/' x.go", []string{"x.go"}}, // GNU: the suffix is "e"+ {"sed -E -i 's/a/b/' x.go", []string{"x.go"}},+ {"sed -i 's/x/y/' f", []string{"f"}},+ {"sed -i '' 's/x/y/' f", []string{"f"}}, // BSD sed: '' is the backup suffix+ {"sed -i '' 's/x/y/' f g", []string{"f", "g"}},+ {"sed -i '' -e 's/a/b/' a.go", []string{"a.go"}},+ {`sed -i "s/$old/$new/" f`, []string{"f"}}, // an unknown script is not a suffix+ {`sed -i "s/$a/$b/" x y`, []string{"x", "y"}},+ {"sudo -u deploy rm /srv/app/old.txt", []string{"/srv/app/old.txt"}},+ {"sudo -E tee /etc/app.conf < in.txt", []string{"/etc/app.conf"}},+ {"sudo -i rm old.txt", nil}, // a login shell starts in the target user's home+ {"env FOO=1 rm a.txt", []string{"a.txt"}},+ {"env -C sub rm a.txt", nil},+ {"rm a.txt # cleanup b.txt", []string{"a.txt"}},+ {"rm {a,b}.txt", []string{"a.txt", "b.txt"}},+ {"if true; then rm a.txt; fi", []string{"a.txt"}},+ {"for f in *.tmp; do rm \"$f\"; done", nil},+ {"echo $((1<<3)) > n.txt\nrm old.txt", []string{"n.txt", "old.txt"}},+ {"grep x <<< \"$v\"\nrm a.txt", []string{"a.txt"}},+ {"git commit -m \"$(cat <<'EOF'\nrm -rf dist\nEOF\n)\"", nil},+ {"rm a.txt 'unterminated", nil}, // a command that does not parse was captured incomplete{"FOO=1 sudo rm build/out", []string{"build/out"}},{`rm "file with spaces.txt"`, []string{"file with spaces.txt"}},{`rm file\ two.txt`, []string{"file two.txt"}},@@ -58,7 +103,20 @@ func TestShellWrittenPaths(t *testing.T) {{"touch before.txt; command pushd sub; rm after.txt", []string{"before.txt"}},{"command popd; rm after.txt", nil},{"ls -la; go test ./...", nil},+ {"tee -a log.txt < output.txt", []string{"log.txt"}},+ {"tee log.txt <output.txt", []string{"log.txt"}},+ {"tee copy.txt <<< hello", []string{"copy.txt"}},+ {"echo '<' > lt.txt", []string{"lt.txt"}},+ {"tee '<' out.txt", []string{"<", "out.txt"}},+ {"cp a.txt b.txt < /dev/null", []string{"b.txt"}},+ {"sudo tee /etc/app.conf > /dev/null <<'EOF'\nkey: value\nEOF", []string{"/etc/app.conf"}},{"cat <<EOF > gen.txt\nhello\nEOF", []string{"gen.txt"}},+ {"cat > deploy.sh <<'EOF'\nrm -rf dist\necho done > status.txt\nEOF\nchmod +x deploy.sh", []string{"deploy.sh"}},+ {"cat <<-END > a.txt\n\trm b.txt\n\tEND\nrm c.txt", []string{"a.txt", "c.txt"}},+ {"cat <<A <<\"B\" > out.txt\ntouch x.txt\nA\ntouch y.txt\nB\ntouch z.txt", []string{"out.txt", "z.txt"}},+ {"cat > a.md <<'EOF'\nDon't touch\nEOF\nrm b.txt", []string{"a.md", "b.txt"}},+ {"grep -c x <<< 'rm here.txt'", nil},+ {"tee x.txt <<<\"hello\"\nrm y.txt", []string{"x.txt", "y.txt"}},} {got := ShellWrittenPaths(tc.command)if len(got) == 0 && len(tc.want) == 0 {diff --git a/docs/cli/git.mdx b/docs/cli/git.mdxindex 8b6a238a..11ce64ee 100644--- a/docs/cli/git.mdx+++ b/docs/cli/git.mdx@@ -26,7 +26,7 @@ A link names a session by its harness and session id, for example `beacon:claudeBeacon already records every file a supported runtime writes, so matching a commit to its sessions is a read of the local runtime log. No state file is kept and nothing touches the network.1. **Window.** Beacon reads the log from the parent commit's time up to the commit. The window always covers at least `--min-lookback` (2 hours) and never more than `--max-lookback` (24 hours) before the commit. Events after the commit are ignored, because they cannot have produced it.-2. **Evidence.** A session wrote a file when the log has, for that session, a `file.created`, `file.modified` or `file.deleted` event, a tool event whose `file.operation` is a write, a shell command whose target is plain to read (`rm`, `mv`, `cp`, `touch`, `tee`, `sed -i`, `git mv`, output redirection), or an `apply_patch` envelope. Reads do not count.+2. **Evidence.** A session wrote a file when the log has, for that session, a `file.created`, `file.modified` or `file.deleted` event, a tool event whose `file.operation` is a write, a shell command whose target is plain to read (`rm`, `mv`, `cp`, `install`, `ln`, `touch`, `truncate`, `tee`, `sed -i`, `perl -pi`, `git mv`, `git rm`, output redirection), or an `apply_patch` envelope. Reads do not count, and neither does a path that depends on a variable, a glob or a command substitution.3. **Overlap.** Beacon compares those paths with the files the commit changes, counting both sides of a rename. A session that wrote at least one of them is linked. Sessions are ordered by how many of the commit's files they wrote, then by how recently.For a merge commit, only files where the result differs from every parent count: conflict resolutions and edits made while merging. The files a merge brings in were written by the other side's commits, so a clean merge or a `git pull` links nothing.