Self-hosted workflows for coding-agent delivery. Orchestrate CodeBuddy, Codex and Claude Code with scripts and human approval gates. Includes Local workflows for coding assistants.
JavaScript agent-orchestrationagentic-workflowai-agentsai-codingclaude-codecode-reviewcodebuddycodexcoding-agentscursordeveloper-toolshuman-on-the-loopllmmcpmulti-agentsdlcself-hostedsopworkflow-automationworkflow-engine
feat(sandbox): mount the platform toolchain into custom-image sandboxes (#604)
crispinhu271dde97530d09
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 57ed7c12..88f9bcb9 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -6,6 +6,7 @@ Platform releases use `vX.Y.Z` tags and publish `ghcr.io/tencent/loopforge`. Loo- The platform image bundles DeepSeek Harness `@deepseek-ai/[email protected]` next to Codex and CodeBuddy, so a DeepSeek key works in conversations and SOP roles without an install step. 0.3.8 had no `dsh` and answered a DeepSeek Harness chat with “AI 助手生成失败,请稍后重试”. The image download grows by about 180 MB. An installation without the image installs the same pinned version from the conversation: every dsh release is a prerelease requiring Node 24.9, and npm on Node 22 found no version for the bare package name.- A conversation workspace whose image lacks the chosen CLI installs Cursor CLI and Antigravity again. The sandbox drops every capability, so the installer's `tar` running as root could not restore the archive owner (`Cannot change ownership to uid 2000`), and the turn failed with “准备 cursor 原生CLI失败”. The workspace install of DeepSeek Harness uses the pinned `@deepseek-ai/[email protected]` with relaxed engines, as the platform install already did; the bare package name resolved to no version.+- A conversation workspace on a custom Dockerfile without Node.js, npm or Python 3 runs the CLIs bundled in the platform image (Codex, CodeBuddy, DeepSeek Harness). The platform copies its Node.js, npm and a relocatable Python 3.12 into `${LOOPFORGE_DATA_DIR}/cli/toolchain/<build>/` on first use, and a bundled CLI the first time a sandbox needs it; every sandbox mounts that directory read-only at the end of `PATH`, so tools shipped by the image still win. The platform image gains python-build-standalone (about 30 MB to download, checked against a pinned SHA-256). Other CLIs are still installed inside the sandbox, now with the mounted npm. A musl (Alpine) or foreign-architecture image, which lacks the loader the copied `node` needs, keeps its own `PATH` and installs with its package manager as before. Each platform build gets its own directory of about 200 MB plus the CLIs it copied; directories of earlier builds are not removed, so delete the other `cli/toolchain/<build>/` directories after an upgrade.- A DeepSeek Harness reply longer than 8 KB (about 2,700 Chinese characters) is shown in full. `dsh --json` cuts every event string at 8 KiB and only its `final` event carries the whole answer, so the chat stopped mid-sentence while the saved reply was complete.- SOP sandboxes of the published image install OpenCode, Kimi Code and Pi on first use, as they already did for Claude Code. An SOP role on one of these CLIs used to fail with `exec: <cli>: not found` (exit 127) even after a conversation installed it.- Run logs and telemetry redact every environment variable ending in `_API_KEY`, `_TOKEN`, `_SECRET` or `_PASSWORD`. Agent CLI keys such as `DEEPSEEK_API_KEY`, and `AGENTFLOW_CLI_API_KEY` used with a custom Base URL, used to appear in clear text in the `docker exec` line of an SOP run log, readable by project members. Rotate a key that ran an SOP role before this release.diff --git a/Dockerfile b/Dockerfileindex b8641017..c33b826e 100644--- a/Dockerfile+++ b/Dockerfile@@ -31,6 +31,26 @@ FROM node:22-bookwormRUN --mount=type=cache,target=/root/.npm apt-get update && apt-get install -y --no-install-recommends ca-certificates git openssh-client bash curl python3 ripgrep \&& rm -rf /var/lib/apt/lists/* \&& npm install -g @openai/[email protected] @tencent-ai/[email protected] @deepseek-ai/[email protected]+# A relocatable Python that sandboxes of images without python3 mount for the+# native CLI hooks (internal/runtime/toolchain.go). Its terminfo database has+# names differing only in case, which a case-insensitive data directory (macOS)+# cannot hold; the hooks never use curses.+ARG TARGETARCH+ARG PYTHON_STANDALONE=20261110/cpython-3.12.16+20261110+ARG PYTHON_STANDALONE_SHA256_AMD64=6a9e62dad033790aef722dbd370e0ce76ce17b73627ba3ae679f537049350b39+ARG PYTHON_STANDALONE_SHA256_ARM64=b6a41defec4975d94b4ff1950236e7383e5c02ba40ea48c9ec33c66087a35aab+RUN case "${TARGETARCH:-amd64}" in \+ amd64) arch=x86_64 sum="${PYTHON_STANDALONE_SHA256_AMD64}" ;; \+ arm64) arch=aarch64 sum="${PYTHON_STANDALONE_SHA256_ARM64}" ;; \+ *) echo "unsupported arch ${TARGETARCH}" >&2; exit 1 ;; \+ esac \+ && curl -fsSL -o /tmp/python.tar.gz "https://github.com/astral-sh/python-build-standalone/releases/download/${PYTHON_STANDALONE}-${arch}-unknown-linux-gnu-install_only_stripped.tar.gz" \+ && echo "${sum} /tmp/python.tar.gz" | sha256sum -c - \+ && mkdir -p /app \+ && tar -xzf /tmp/python.tar.gz -C /app \+ && rm -f /tmp/python.tar.gz \+ && rm -rf /app/python/share/terminfo \+ && /app/python/bin/python3 -c 'import json,urllib.request,ctypes'COPY --from=docker:27-cli /usr/local/bin/docker /usr/local/bin/dockerCOPY --from=docker:27-cli /usr/local/libexec/docker/cli-plugins/docker-buildx /usr/local/libexec/docker/cli-plugins/docker-buildxWORKDIR /appdiff --git a/docs/deployment.md b/docs/deployment.mdindex b1a3f16b..e50aea6f 100644--- a/docs/deployment.md+++ b/docs/deployment.md@@ -15,6 +15,8 @@ npm install --global @openai/[email protected] @anthropic-ai/[email protected]Ensure the service can find `codebuddy` and Node.js on its PATH, or set `AGENTFLOW_CODEBUDDY_BIN` to the installed executable. Configure the model key and its network environment in personal settings. The no-key script demo does not need the CLI. The production Docker image includes CodeBuddy, Codex and DeepSeek Harness (`dsh`). Claude Code and the other agent CLIs are not bundled. When a conversation fails because one is missing, the error says it is not installed and offers an install button. That downloads the pinned `@anthropic-ai/claude-code` release, or the public package for OpenCode, Kimi Code and Pi, into `$LOOPFORGE_DATA_DIR/cli`; an installation outside the image gets the pinned `@deepseek-ai/dsh` release the same way. Cursor CLI and Antigravity use their providers' installers and land in the service user's `~/.local/bin`. Closing or reloading the browser does not stop an install that has already started, and the button stays on “自动安装” until it finishes. A finished install replaces the earlier generation failure with a prompt to send the message again. The next message reuses the installation. An SOP sandbox of the published image that selects Claude Code, OpenCode, Kimi Code or Pi installs the same package on first use and shares it with conversations. Cursor CLI and Antigravity are not installed into SOP sandboxes. Connected runners keep their own CLI requirements.+A conversation workspace on a custom Dockerfile without Node.js, npm or Python 3 uses the platform's own copies. On first use the platform copies its Node.js, npm and a relocatable Python into `$LOOPFORGE_DATA_DIR/cli/toolchain/<build>/`, plus each bundled CLI (CodeBuddy, Codex, DeepSeek Harness) the first time a sandbox needs it, and mounts the directory read-only at the end of the sandbox `PATH`. Tools the image ships take precedence. Only glibc images of the platform's architecture use it; Alpine (musl) images need their own Node.js, npm and Python 3. A new platform build creates a new directory and leaves the previous ones in place: after an upgrade, delete the other `cli/toolchain/<build>/` directories (about 200 MB each plus the CLIs copied into them).+Stop the foreground service with Ctrl-C. Start it again with the same command. MySQL data stays in its Compose volume. `docker compose stop mysql` stops the database without deleting it.Use `AGENTFLOW_MYSQL_DSN` for your own MySQL deployment. A fresh account command or server connection creates missing schema objects. Apply existing-installation data migrations individually after reviewing their prerequisites; do not blindly replay all historical migrations against a new database.diff --git a/internal/runtime/conversation_docker_process.go b/internal/runtime/conversation_docker_process.goindex 190e937e..5a22750b 100644--- a/internal/runtime/conversation_docker_process.go+++ b/internal/runtime/conversation_docker_process.go@@ -1,11 +1,11 @@package runtimefunc dockerConversationProcessArgv(file string, argv []string) []string {- return append([]string{"python3", "-c", dockerConversationProcessSupervisor, file}, argv...)+ return withToolchainPath(append([]string{"python3", "-c", dockerConversationProcessSupervisor, file}, argv...))}func dockerConversationStopArgv(file string) []string {- return []string{"python3", "-c", dockerConversationProcessStop, file}+ return withToolchainPath([]string{"python3", "-c", dockerConversationProcessStop, file})}// A CLI may create new sessions for its native shell tools. Killing only itsdiff --git a/internal/runtime/conversation_sandbox.go b/internal/runtime/conversation_sandbox.goindex 78930adb..0b8b4cc1 100644--- a/internal/runtime/conversation_sandbox.go+++ b/internal/runtime/conversation_sandbox.go@@ -113,11 +113,17 @@ func ensureConversationCLI(ctx context.Context, sandbox Sandbox, provider, home}// Provision outside the private workspace as root even when the container// runs as the service UID. Native CLI execution keeps the container's user.- output, err := exec.CommandContext(ctx, "docker", "exec", "--user", "0:0", "--workdir", "/", s.name, "sh", "-lc", setup, "loopforge-cli-setup", ids[0], ids[1], home).CombinedOutput()+ toolchain, err := s.toolchainCLIExecEnv(binary)+ if err != nil {+ return "", err+ }+ setupArgs := append(append([]string{"exec", "--user", "0:0", "--workdir", "/"}, toolchain...), s.name, "sh", "-lc", setup, "loopforge-cli-setup", ids[0], ids[1], home)+ output, err := exec.CommandContext(ctx, "docker", setupArgs...).CombinedOutput()if err != nil {return "", fmt.Errorf("准备 %s 原生CLI失败(工作区镜像需支持 Node.js/npm 和 Python3):%w: %s", provider, err, conversationSetupOutput(output))}- if check, err := exec.CommandContext(ctx, "docker", "exec", "--workdir", "/", s.name, "python3", "-c", "import os,pwd; pwd.getpwuid(os.geteuid())").CombinedOutput(); err != nil {+ checkArgs := append(append(append([]string{"exec", "--workdir", "/"}, toolchain...), s.name), withToolchainPath([]string{"python3", "-c", "import os,pwd; pwd.getpwuid(os.geteuid())"})...)+ if check, err := exec.CommandContext(ctx, "docker", checkArgs...).CombinedOutput(); err != nil {return "", fmt.Errorf("验证工作区 CLI 用户失败: %w: %s", err, conversationSetupOutput(check))}lines := strings.Split(strings.TrimSpace(string(output)), "\n")@@ -160,7 +166,7 @@ func conversationCLISetup(provider, binary string) (string, error) {`PATH=/opt/loopforge/conversation-cli/bin:/opt/loopforge/conversation-cli/.local/bin:$PATHTAR_OPTIONS=--no-same-ownerexport PATH TAR_OPTIONS-if ! command -v python3 >/dev/null 2>&1; then+` + toolchainPathSetup + `if ! command -v python3 >/dev/null 2>&1; thenif command -v apt-get >/dev/null 2>&1; then apt-get update >&2 && apt-get install -y python3 >&2elif command -v apk >/dev/null 2>&1; then apk add --no-cache python3 >&2else echo 'Python3 is required for native CLI hooks and process supervision' >&2; exit 1; fidiff --git a/internal/runtime/sandbox_docker.go b/internal/runtime/sandbox_docker.goindex 55d5000f..feaa67b5 100644--- a/internal/runtime/sandbox_docker.go+++ b/internal/runtime/sandbox_docker.go@@ -167,6 +167,8 @@ func (s *dockerSandbox) execOnce(ctx context.Context, command CommandSpec, sinkif cliPath != "" {args = append(args, "-e", "AGENTFLOW_RUNTIME_CLI_PATH="+cliPath)}+ toolchain := s.toolchainExecEnv()+ args = append(args, toolchain...)args = append(args, s.name)argv := dockerWorkspaceExecArgv(command.Argv)if command.WorkingDir != "" {@@ -180,7 +182,7 @@ cd "$1" && shift && exec "$@"`, "conversation", command.WorkingDir}, dropShellLodefer func() {stopCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)defer cancel()- stopArgs := append([]string{"exec", s.name}, dockerConversationStopArgv(processFile)...)+ stopArgs := append(append(append([]string{"exec"}, toolchain...), s.name), dockerConversationStopArgv(processFile)...)if stopErr := exec.CommandContext(stopCtx, "docker", stopArgs...).Run(); stopErr != nil {err = errors.Join(err, fmt.Errorf("停止 CLI 子进程失败: %w", stopErr))}@@ -214,7 +216,7 @@ if [ -n "${AGENTFLOW_RUNTIME_CLI_PATH:-}" ] && [ -d "$AGENTFLOW_RUNTIME_CLI_PATHPATH=$PATH:$AGENTFLOW_RUNTIME_CLI_PATHexport PATHfi-`+` + toolchainPathSetup// onDemandCLIPath installs a CLI the published runtime image does not bundle// (Claude Code and the npm agent CLIs) for sandboxes of that image, and returns@@ -719,7 +721,7 @@ func (s *dockerSandbox) ensureStarted(ctx context.Context) error {return err}args = append(args, stateArgs...)- if cliDir := RuntimeCLIDir(); cliDir != "" && image == isolatedRuntimeImage(s.dockerfile) {+ if cliDir := RuntimeCLIDir(); cliDir != "" {if err := os.MkdirAll(cliDir, 0o755); err != nil {return fmt.Errorf("prepare runtime CLI directory: %w", err)}diff --git a/internal/runtime/toolchain.go b/internal/runtime/toolchain.gonew file mode 100644index 00000000..fa71b270--- /dev/null+++ b/internal/runtime/toolchain.go@@ -0,0 +1,347 @@+package runtime++import (+ "context"+ "debug/elf"+ "fmt"+ "io"+ "os"+ "os/exec"+ "path/filepath"+ "strconv"+ "strings"+ "time"+)++// A sandbox of a custom image may lack Node.js, npm or Python 3, which the+// native CLIs and their hooks need. The platform container cannot bind paths+// of its own image into a sibling container, so it copies its Node.js prefix,+// the CLIs bundled there and a relocatable Python into the host-visible CLI+// directory once per platform build. Sandboxes mount it read-only and append+// it to PATH, so a tool shipped by the image still wins.+const toolchainEnv = "AGENTFLOW_TOOLCHAIN"++// Paths inside the platform image; tests point them at a fixture.+var (+ toolchainNodePrefix = "/usr/local"+ toolchainPythonDir = "/app/python"+ toolchainCommitFile = "/app/loopforge-commit"+)++// toolchainPathSetup appends the mounted toolchain to PATH when exec passed it+// and the container has the ELF interpreter its node needs. A musl (Alpine)+// or foreign-architecture image keeps its PATH and installs CLIs and python3+// with its own package manager, because `command -v` cannot tell that the+// toolchain binaries would not start there.+const toolchainPathSetup = `if [ -n "${AGENTFLOW_TOOLCHAIN:-}" ] && [ -r "$AGENTFLOW_TOOLCHAIN/.interp" ]; then+ toolchain_interp=+ read -r toolchain_interp < "$AGENTFLOW_TOOLCHAIN/.interp" || :+ if [ -n "$toolchain_interp" ] && [ -e "$toolchain_interp" ]; then+ PATH=$PATH:$AGENTFLOW_TOOLCHAIN/bin:$AGENTFLOW_TOOLCHAIN/python/bin+ export PATH+ fi+ unset toolchain_interp+fi+`++// toolchainLockTimeout bounds the wait for another process copying the+// toolchain; a cold copy onto a slow data directory takes minutes.+const toolchainLockTimeout = 15 * time.Minute++// toolchainBeforeLock lets a test observe a caller that passed the unlocked+// fast path and is about to wait for the lock.+var toolchainBeforeLock = func() {}++// lockToolchain serializes toolchain writers across every platform process+// sharing the CLI directory. Writers recheck under the lock, so a directory+// another process published is never replaced.+func lockToolchain(parent string) (func(), error) {+ toolchainBeforeLock()+ if err := os.MkdirAll(parent, 0o755); err != nil {+ return nil, fmt.Errorf("准备工具链目录失败:%w", err)+ }+ f, err := os.OpenFile(filepath.Join(parent, ".lock"), os.O_CREATE|os.O_RDWR, 0o644)+ if err != nil {+ return nil, err+ }+ ctx, cancel := context.WithTimeout(context.Background(), toolchainLockTimeout)+ defer cancel()+ if err := flockExclusive(ctx, f); err != nil {+ _ = f.Close()+ return nil, fmt.Errorf("等待工具链锁失败:%w", err)+ }+ return func() { _ = unlockFile(f) }, nil+}++// RuntimeToolchainDir returns this build's toolchain copy, creating it on+// first use. It returns "" when the CLI directory is not configured or the+// process does not run in the platform image.+func RuntimeToolchainDir() (string, error) {+ dir, ok := toolchainLocation()+ if !ok {+ return "", nil+ }+ parent := filepath.Dir(dir)+ ready := filepath.Join(dir, ".ready")+ if _, err := os.Stat(ready); err == nil {+ return dir, nil+ }+ unlock, err := lockToolchain(parent)+ if err != nil {+ return "", err+ }+ defer unlock()+ if _, err := os.Stat(ready); err == nil {+ return dir, nil+ }+ stage, err := os.MkdirTemp(parent, ".stage-")+ if err != nil {+ return "", fmt.Errorf("准备工具链目录失败:%w", err)+ }+ defer os.RemoveAll(stage)+ if err := os.Chmod(stage, 0o755); err != nil {+ return "", err+ }+ if err := copyToolchain(stage); err != nil {+ return "", fmt.Errorf("复制工具链失败:%w", err)+ }+ if err := os.WriteFile(filepath.Join(stage, ".ready"), nil, 0o644); err != nil {+ return "", err+ }+ // Only a copy that never got its .ready marker can be here.+ if err := os.RemoveAll(dir); err != nil {+ return "", err+ }+ if err := os.Rename(stage, dir); err != nil {+ return "", fmt.Errorf("启用工具链失败:%w", err)+ }+ return dir, nil+}++// toolchainLocation is where this build's toolchain lives; ok is false when+// the CLI directory is not configured or the process does not run in the+// platform image.+func toolchainLocation() (string, bool) {+ root := RuntimeCLIDir()+ if root == "" || !fileIsExecutable(filepath.Join(toolchainNodePrefix, "bin", "node")) {+ return "", false+ }+ return filepath.Join(root, "toolchain", toolchainKey()), true+}++// readyToolchainDir returns the toolchain only when it is already published.+// It never copies, so a sandbox command does not wait for or fail on it.+func readyToolchainDir() string {+ dir, ok := toolchainLocation()+ if !ok {+ return ""+ }+ if _, err := os.Stat(filepath.Join(dir, ".ready")); err != nil {+ return ""+ }+ return dir+}++func toolchainKey() string {+ if data, err := os.ReadFile(toolchainCommitFile); err == nil {+ if key := strings.TrimSpace(string(data)); key != "" && !strings.ContainsAny(key, "/\\ \t\r\n") {+ return key+ }+ }+ if info, err := os.Stat(filepath.Join(toolchainNodePrefix, "lib", "node_modules")); err == nil {+ return "mtime-" + strconv.FormatInt(info.ModTime().Unix(), 10)+ }+ return "default"+}++// copyToolchain copies node, npm and Python. The bundled CLIs are hundreds of+// megabytes each, so EnsureToolchainCLI copies one only when a sandbox needs it.+// Other files in the prefix (such as the docker CLI) stay out.+func copyToolchain(stage string) error {+ bin := filepath.Join(stage, "bin")+ modules := filepath.Join(stage, "lib", "node_modules")+ if err := os.MkdirAll(bin, 0o755); err != nil {+ return err+ }+ if err := os.MkdirAll(modules, 0o755); err != nil {+ return err+ }+ prefixBin := filepath.Join(toolchainNodePrefix, "bin")+ if err := copyPreserving(bin, filepath.Join(prefixBin, "node"), filepath.Join(prefixBin, "npm"), filepath.Join(prefixBin, "npx")); err != nil {+ return err+ }+ if err := copyPreserving(modules, filepath.Join(toolchainNodePrefix, "lib", "node_modules", "npm")); err != nil {+ return err+ }+ // Without .interp the sandbox launch never adds the toolchain to PATH.+ if interp := elfInterpreter(filepath.Join(prefixBin, "node")); interp != "" {+ if err := os.WriteFile(filepath.Join(stage, ".interp"), []byte(interp+"\n"), 0o644); err != nil {+ return err+ }+ }+ if info, err := os.Stat(toolchainPythonDir); err == nil && info.IsDir() {+ return copyPreserving(stage, toolchainPythonDir)+ }+ return nil+}++// EnsureToolchainCLI copies the package behind a CLI bundled in the platform+// image, with every launcher pointing into it, into the toolchain at dir. A+// binary the platform image does not bundle is left to the sandbox installer.+func EnsureToolchainCLI(dir, binary string) error {+ if dir == "" || binary == "" || strings.ContainsAny(binary, "/\\") {+ return nil+ }+ prefixBin := filepath.Join(toolchainNodePrefix, "bin")+ target, err := os.Readlink(filepath.Join(prefixBin, binary))+ pkg, ok := toolchainPackage(target)+ if err != nil || !ok {+ return nil+ }+ launcher := filepath.Join(dir, "bin", binary)+ if fileIsExecutable(launcher) {+ return nil+ }+ unlock, err := lockToolchain(filepath.Dir(dir))+ if err != nil {+ return err+ }+ defer unlock()+ if fileIsExecutable(launcher) {+ return nil+ }+ modules := filepath.Join(dir, "lib", "node_modules")+ dest := filepath.Join(modules, filepath.FromSlash(pkg))+ if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil {+ return err+ }+ stage, err := os.MkdirTemp(filepath.Dir(dest), ".stage-")+ if err != nil {+ return err+ }+ defer os.RemoveAll(stage)+ if err := os.Chmod(stage, 0o755); err != nil {+ return err+ }+ if err := copyPreserving(stage, filepath.Join(toolchainNodePrefix, "lib", "node_modules", filepath.FromSlash(pkg))); err != nil {+ return fmt.Errorf("复制 %s 失败:%w", binary, err)+ }+ if err := os.RemoveAll(dest); err != nil {+ return err+ }+ if err := os.Rename(filepath.Join(stage, filepath.Base(dest)), dest); err != nil {+ return err+ }+ entries, err := os.ReadDir(prefixBin)+ if err != nil {+ return err+ }+ for _, entry := range entries {+ link, err := os.Readlink(filepath.Join(prefixBin, entry.Name()))+ if other, ok := toolchainPackage(link); err != nil || !ok || other != pkg {+ continue+ }+ path := filepath.Join(dir, "bin", entry.Name())+ _ = os.Remove(path)+ if err := os.Symlink(link, path); err != nil {+ return err+ }+ }+ return nil+}++// toolchainPackage returns the package ("name" or "@scope/name") an npm+// launcher in the prefix bin points into.+func toolchainPackage(link string) (string, bool) {+ rest, ok := strings.CutPrefix(link, "../lib/node_modules/")+ if !ok {+ return "", false+ }+ parts := strings.Split(rest, "/")+ if strings.HasPrefix(parts[0], "@") {+ if len(parts) < 3 {+ return "", false+ }+ return parts[0] + "/" + parts[1], true+ }+ if len(parts) < 2 || parts[0] == "npm" {+ return "", false+ }+ return parts[0], true+}++// elfInterpreter returns the dynamic loader an ELF executable requests, such+// as /lib/ld-linux-aarch64.so.1, or "" when path is not a dynamic ELF file.+func elfInterpreter(path string) string {+ f, err := elf.Open(path)+ if err != nil {+ return ""+ }+ defer f.Close()+ for _, prog := range f.Progs {+ if prog.Type != elf.PT_INTERP {+ continue+ }+ data, err := io.ReadAll(prog.Open())+ if err != nil {+ return ""+ }+ interp := strings.TrimRight(string(data), "\x00")+ if !strings.HasPrefix(interp, "/") || strings.ContainsAny(interp, "\r\n") {+ return ""+ }+ return interp+ }+ return ""+}++func copyPreserving(dst string, sources ...string) error {+ args := append([]string{"-a"}, sources...)+ out, err := exec.Command("cp", append(args, dst+"/")...).CombinedOutput()+ if err != nil {+ return fmt.Errorf("%w: %s", err, strings.TrimSpace(string(out)))+ }+ return nil+}++// sandboxToolchain is the toolchain a sandbox of a custom image should see.+// Sandboxes of the platform image already carry these tools.+func (s *dockerSandbox) sandboxToolchain() (string, error) {+ if isolatedRuntimeImage(s.dockerfile) != "" {+ return "", nil+ }+ return RuntimeToolchainDir()+}++// toolchainExecEnv exposes an already published toolchain to a sandbox+// command. The toolchain is only a fallback, so a command never copies it and+// never fails because it is missing; native CLI preparation creates it.+func (s *dockerSandbox) toolchainExecEnv() []string {+ if isolatedRuntimeImage(s.dockerfile) != "" {+ return nil+ }+ dir := readyToolchainDir()+ if dir == "" {+ return nil+ }+ return []string{"-e", toolchainEnv + "=" + dir}+}++// toolchainCLIExecEnv creates the toolchain, with binary when the platform+// image bundles it, for native CLI preparation.+func (s *dockerSandbox) toolchainCLIExecEnv(binary string) ([]string, error) {+ dir, err := s.sandboxToolchain()+ if err != nil || dir == "" {+ return nil, err+ }+ if err := EnsureToolchainCLI(dir, binary); err != nil {+ return nil, err+ }+ return []string{"-e", toolchainEnv + "=" + dir}, nil+}++// withToolchainPath runs argv after toolchainPathSetup, for commands that+// docker exec starts directly instead of through a login shell.+func withToolchainPath(argv []string) []string {+ return append([]string{"sh", "-c", toolchainPathSetup + `exec "$0" "$@"`}, argv...)+}diff --git a/internal/runtime/toolchain_helpers_test.go b/internal/runtime/toolchain_helpers_test.gonew file mode 100644index 00000000..432cd1cf--- /dev/null+++ b/internal/runtime/toolchain_helpers_test.go@@ -0,0 +1,91 @@+package runtime++import (+ "os"+ "path/filepath"+ "strings"+ "testing"++ "agentflow/internal/domain"+)++// toolchain_paths_test.go covers the happy paths of the toolchain helpers;+// these cover the error branches and that a published copy is never redone.++func TestRuntimeToolchainDirReportsUnusableCLIDir(t *testing.T) {+ prefix := writeToolchainFixture(t)+ file := filepath.Join(t.TempDir(), "not-a-dir")+ if err := os.WriteFile(file, nil, 0o644); err != nil {+ t.Fatal(err)+ }+ useToolchainFixture(t, prefix, file)+ if dir, err := RuntimeToolchainDir(); err == nil || dir != "" || !strings.Contains(err.Error(), "准备工具链目录失败") {+ t.Fatalf("RuntimeToolchainDir() = (%q, %v), want the directory error", dir, err)+ }+ sandbox := &dockerSandbox{dockerfile: domain.DockerfileDefinition{Content: "FROM debian:bookworm\n"}}+ // A sandbox command only uses a published toolchain and never fails on it.+ if env := sandbox.toolchainExecEnv(); env != nil {+ t.Fatalf("toolchainExecEnv() = %v, want none", env)+ }+ if env, err := sandbox.toolchainCLIExecEnv("fakecli"); err == nil || env != nil {+ t.Fatalf("toolchainCLIExecEnv() = (%v, %v), want the error", env, err)+ }+}++func TestEnsureToolchainCLIReportsCopyFailure(t *testing.T) {+ prefix := writeToolchainFixture(t)+ useToolchainFixture(t, prefix, t.TempDir())+ if err := os.Symlink("../lib/node_modules/gone/bin/gone.js", filepath.Join(prefix, "bin", "gonecli")); err != nil {+ t.Fatal(err)+ }+ dir, err := RuntimeToolchainDir()+ if err != nil || dir == "" {+ t.Fatalf("RuntimeToolchainDir() = (%q, %v)", dir, err)+ }+ if err := EnsureToolchainCLI(dir, "gonecli"); err == nil || !strings.Contains(err.Error(), "复制 gonecli 失败") {+ t.Fatalf("EnsureToolchainCLI() = %v, want the copy error", err)+ }+ if _, err := os.Lstat(filepath.Join(dir, "bin", "gonecli")); !os.IsNotExist(err) {+ t.Fatalf("launcher linked after a failed copy: %v", err)+ }+ stages, err := filepath.Glob(filepath.Join(dir, "lib", "node_modules", ".stage-*"))+ if err != nil || len(stages) != 0 {+ t.Fatalf("leftover stages %v (%v)", stages, err)+ }+}++func TestToolchainPublishedCopiesAreReused(t *testing.T) {+ prefix := writeToolchainFixture(t)+ useToolchainFixture(t, prefix, t.TempDir())+ dir, err := RuntimeToolchainDir()+ if err != nil || dir == "" {+ t.Fatalf("RuntimeToolchainDir() = (%q, %v)", dir, err)+ }+ if info, err := os.Stat(dir); err != nil || info.Mode().Perm() != 0o755 {+ t.Fatalf("toolchain dir mode = %v, %v; want 0755 so sandbox users can read it", info.Mode().Perm(), err)+ }+ sentinel := filepath.Join(dir, "sentinel")+ if err := os.WriteFile(sentinel, nil, 0o644); err != nil {+ t.Fatal(err)+ }+ if again, err := RuntimeToolchainDir(); err != nil || again != dir {+ t.Fatalf("second RuntimeToolchainDir() = (%q, %v), want %q", again, err, dir)+ }+ if _, err := os.Stat(sentinel); err != nil {+ t.Fatalf("published toolchain was recopied: %v", err)+ }++ if err := EnsureToolchainCLI(dir, "fakecli"); err != nil {+ t.Fatal(err)+ }+ marker := filepath.Join(dir, "lib", "node_modules", "fakepkg", "marker")+ if err := os.WriteFile(marker, nil, 0o644); err != nil {+ t.Fatal(err)+ }+ if err := EnsureToolchainCLI(dir, "fakecli"); err != nil {+ t.Fatal(err)+ }+ if _, err := os.Stat(marker); err != nil {+ t.Fatalf("published CLI package was recopied: %v", err)+ }+}diff --git a/internal/runtime/toolchain_paths_test.go b/internal/runtime/toolchain_paths_test.gonew file mode 100644index 00000000..fdcded9f--- /dev/null+++ b/internal/runtime/toolchain_paths_test.go@@ -0,0 +1,417 @@+package runtime++import (+ "os"+ "os/exec"+ "path/filepath"+ "slices"+ "strconv"+ "strings"+ "testing"+ "time"++ "agentflow/internal/domain"+)++// The CLIs the platform image bundles are scoped npm packages+// (@openai/codex, @tencent-ai/codebuddy-code, @deepseek-ai/dsh), so the+// launcher parser must keep the scope, and npm itself stays in the base copy.+func TestToolchainPackageParsesLauncherTargets(t *testing.T) {+ for _, tc := range []struct {+ link string+ want string+ ok bool+ }{+ {"../lib/node_modules/@openai/codex/bin/codex.js", "@openai/codex", true},+ {"../lib/node_modules/@tencent-ai/codebuddy-code/bin/codebuddy", "@tencent-ai/codebuddy-code", true},+ {"../lib/node_modules/fakepkg/bin/fake.js", "fakepkg", true},+ {"../lib/node_modules/npm/bin/npm-cli.js", "", false},+ {"../lib/node_modules/@scope/only", "", false},+ {"../lib/node_modules/pkg", "", false},+ {"/usr/local/lib/node_modules/fakepkg/bin/fake.js", "", false},+ {"node", "", false},+ {"", "", false},+ } {+ got, ok := toolchainPackage(tc.link)+ if got != tc.want || ok != tc.ok {+ t.Errorf("toolchainPackage(%q) = %q, %v; want %q, %v", tc.link, got, ok, tc.want, tc.ok)+ }+ }+}++// The toolchain directory is keyed by the platform build; an unusable commit+// file falls back to the node_modules mtime and then to "default".+func TestToolchainKeyFallbacks(t *testing.T) {+ prefix := writeToolchainFixture(t)+ useToolchainFixture(t, prefix, t.TempDir())+ if got := toolchainKey(); got != "testbuild" {+ t.Fatalf("commit key = %q, want testbuild", got)+ }+ stamp := time.Unix(1700000000, 0)+ if err := os.Chtimes(filepath.Join(prefix, "lib", "node_modules"), stamp, stamp); err != nil {+ t.Fatal(err)+ }+ mtimeKey := "mtime-" + strconv.FormatInt(stamp.Unix(), 10)+ for _, commit := range []string{"", " \n", "abc def", "../escape", `a\b`, "two\nlines"} {+ if err := os.WriteFile(toolchainCommitFile, []byte(commit), 0o644); err != nil {+ t.Fatal(err)+ }+ if got := toolchainKey(); got != mtimeKey {+ t.Errorf("commit %q: key = %q, want %q", commit, got, mtimeKey)+ }+ }+ if err := os.Remove(toolchainCommitFile); err != nil {+ t.Fatal(err)+ }+ if got := toolchainKey(); got != mtimeKey {+ t.Fatalf("missing commit file: key = %q, want %q", got, mtimeKey)+ }+ toolchainNodePrefix = t.TempDir()+ if got := toolchainKey(); got != "default" {+ t.Fatalf("no commit and no node_modules: key = %q, want default", got)+ }+}++// Outside the platform image (no executable node in the prefix) or without a+// CLI directory there is nothing to mount, and nothing may be written.+func TestRuntimeToolchainDirDisabledOutsidePlatformImage(t *testing.T) {+ prefix := writeToolchainFixture(t)+ cliDir := t.TempDir()+ useToolchainFixture(t, prefix, "")+ if dir, err := RuntimeToolchainDir(); dir != "" || err != nil {+ t.Fatalf("no CLI dir: got %q, %v", dir, err)+ }+ t.Setenv("AGENTFLOW_RUNTIME_CLI_DIR", cliDir)+ if err := os.Chmod(filepath.Join(prefix, "bin", "node"), 0o644); err != nil {+ t.Fatal(err)+ }+ if dir, err := RuntimeToolchainDir(); dir != "" || err != nil {+ t.Fatalf("non-executable node: got %q, %v", dir, err)+ }+ if err := os.Remove(filepath.Join(prefix, "bin", "node")); err != nil {+ t.Fatal(err)+ }+ if dir, err := RuntimeToolchainDir(); dir != "" || err != nil {+ t.Fatalf("missing node: got %q, %v", dir, err)+ }+ if _, err := os.Stat(filepath.Join(cliDir, "toolchain")); !os.IsNotExist(err) {+ t.Fatalf("toolchain directory created outside the platform image: %v", err)+ }+}++// The base copy holds node, npm, npx and Python only; bundled CLIs are copied+// on demand. A half-written copy without .ready is replaced, a ready one reused.+func TestRuntimeToolchainDirCopiesBaseToolsAndPython(t *testing.T) {+ prefix := writeToolchainFixture(t)+ cliDir := t.TempDir()+ useToolchainFixture(t, prefix, cliDir)+ toolchainPythonDir = filepath.Join(prefix, "python")+ if err := os.MkdirAll(filepath.Join(toolchainPythonDir, "bin"), 0o755); err != nil {+ t.Fatal(err)+ }+ if err := os.WriteFile(filepath.Join(toolchainPythonDir, "bin", "python3"), []byte("#!/bin/sh\n"), 0o755); err != nil {+ t.Fatal(err)+ }+ want := filepath.Join(cliDir, "toolchain", "testbuild")+ if err := os.MkdirAll(filepath.Join(want, "bin"), 0o755); err != nil {+ t.Fatal(err)+ }+ stale := filepath.Join(want, "bin", "stale")+ if err := os.WriteFile(stale, nil, 0o644); err != nil {+ t.Fatal(err)+ }++ dir, err := RuntimeToolchainDir()+ if err != nil || dir != want {+ t.Fatalf("RuntimeToolchainDir() = %q, %v; want %q", dir, err, want)+ }+ if _, err := os.Stat(stale); !os.IsNotExist(err) {+ t.Fatalf("unfinished copy without .ready was kept: %v", err)+ }+ for _, path := range []string{"bin/node", "bin/npm", "bin/npx", "lib/node_modules/npm/bin/npm-cli.js", "python/bin/python3"} {+ if !fileIsExecutable(filepath.Join(dir, path)) {+ t.Errorf("%s missing or not executable in the toolchain", path)+ }+ }+ if link, err := os.Readlink(filepath.Join(dir, "bin", "npm")); err != nil || link != "../lib/node_modules/npm/bin/npm-cli.js" {+ t.Errorf("npm launcher = %q, %v; want the relative link kept", link, err)+ }+ for _, path := range []string{"bin/fakecli", "lib/node_modules/fakepkg"} {+ if _, err := os.Lstat(filepath.Join(dir, path)); !os.IsNotExist(err) {+ t.Errorf("bundled CLI %s copied eagerly: %v", path, err)+ }+ }+ if _, err := os.Stat(filepath.Join(dir, ".ready")); err != nil {+ t.Fatalf("ready marker: %v", err)+ }++ marker := filepath.Join(dir, "bin", "kept")+ if err := os.WriteFile(marker, nil, 0o644); err != nil {+ t.Fatal(err)+ }+ if again, err := RuntimeToolchainDir(); err != nil || again != dir {+ t.Fatalf("second call = %q, %v; want %q", again, err, dir)+ }+ if _, err := os.Stat(marker); err != nil {+ t.Fatalf("ready toolchain was recopied: %v", err)+ }+}++func TestRuntimeToolchainDirWithoutPythonDir(t *testing.T) {+ prefix := writeToolchainFixture(t)+ cliDir := t.TempDir()+ useToolchainFixture(t, prefix, cliDir)+ dir, err := RuntimeToolchainDir()+ if err != nil || dir == "" {+ t.Fatalf("RuntimeToolchainDir() = %q, %v", dir, err)+ }+ if _, err := os.Stat(filepath.Join(dir, "python")); !os.IsNotExist(err) {+ t.Fatalf("python directory appeared without a platform Python: %v", err)+ }+}++// A scoped package with several launchers is copied once and every launcher+// pointing into it is linked, so `codex` and its aliases all resolve.+func TestEnsureToolchainCLICopiesScopedPackageAndAllItsLaunchers(t *testing.T) {+ prefix := writeToolchainFixture(t)+ cliDir := t.TempDir()+ useToolchainFixture(t, prefix, cliDir)+ pkg := filepath.Join(prefix, "lib", "node_modules", "@scope", "agent")+ if err := os.MkdirAll(filepath.Join(pkg, "bin"), 0o755); err != nil {+ t.Fatal(err)+ }+ for _, file := range []string{"bin/agent.js", "bin/agent-mcp.js"} {+ if err := os.WriteFile(filepath.Join(pkg, file), []byte("#!/bin/sh\n"), 0o755); err != nil {+ t.Fatal(err)+ }+ }+ for name, target := range map[string]string{+ "agent": "../lib/node_modules/@scope/agent/bin/agent.js",+ "agent-mcp": "../lib/node_modules/@scope/agent/bin/agent-mcp.js",+ } {+ if err := os.Symlink(target, filepath.Join(prefix, "bin", name)); err != nil {+ t.Fatal(err)+ }+ }+ dir, err := RuntimeToolchainDir()+ if err != nil || dir == "" {+ t.Fatalf("RuntimeToolchainDir() = %q, %v", dir, err)+ }+ if err := EnsureToolchainCLI(dir, "agent"); err != nil {+ t.Fatal(err)+ }+ for _, name := range []string{"agent", "agent-mcp"} {+ if !fileIsExecutable(filepath.Join(dir, "bin", name)) {+ t.Errorf("launcher %s does not resolve inside the toolchain", name)+ }+ }+ if !fileIsExecutable(filepath.Join(dir, "lib", "node_modules", "@scope", "agent", "bin", "agent.js")) {+ t.Fatal("scoped package not copied under its scope")+ }+ for _, path := range []string{"bin/fakecli", "lib/node_modules/fakepkg"} {+ if _, err := os.Lstat(filepath.Join(dir, path)); !os.IsNotExist(err) {+ t.Errorf("unrelated package %s copied: %v", path, err)+ }+ }+ entries, err := os.ReadDir(filepath.Join(dir, "lib", "node_modules", "@scope"))+ if err != nil {+ t.Fatal(err)+ }+ for _, entry := range entries {+ if strings.HasPrefix(entry.Name(), ".stage-") {+ t.Fatalf("leftover stage %s", entry.Name())+ }+ }++ // Once the launcher exists, a later call must not recopy the package.+ kept := filepath.Join(dir, "lib", "node_modules", "@scope", "agent", "kept")+ if err := os.WriteFile(kept, nil, 0o644); err != nil {+ t.Fatal(err)+ }+ if err := EnsureToolchainCLI(dir, "agent-mcp"); err != nil {+ t.Fatal(err)+ }+ if _, err := os.Stat(kept); err != nil {+ t.Fatalf("installed package was recopied: %v", err)+ }+}++// Binaries the platform image does not bundle as an npm launcher are left to+// the sandbox installer: nothing is copied and no lock is taken.+func TestEnsureToolchainCLIIgnoresUnbundledBinaries(t *testing.T) {+ prefix := writeToolchainFixture(t)+ cliDir := t.TempDir()+ useToolchainFixture(t, prefix, cliDir)+ dir := filepath.Join(cliDir, "toolchain", "testbuild")+ for _, tc := range []struct{ dir, binary string }{+ {"", "fakecli"},+ {dir, ""},+ {dir, "../bin/fakecli"},+ {dir, "bin/fakecli"},+ {dir, `bin\fakecli`},+ {dir, "npm"},+ {dir, "node"},+ {dir, "missing"},+ } {+ if err := EnsureToolchainCLI(tc.dir, tc.binary); err != nil {+ t.Errorf("EnsureToolchainCLI(%q, %q) = %v, want nil", tc.dir, tc.binary, err)+ }+ }+ if _, err := os.Stat(filepath.Join(cliDir, "toolchain")); !os.IsNotExist(err) {+ t.Fatalf("unbundled binary touched the toolchain directory: %v", err)+ }+}++// withToolchainPath wraps commands docker exec starts directly (the Python+// process supervisor): the toolchain is appended to PATH, so a tool the custom+// image ships still wins, and the original argv runs unchanged.+func TestWithToolchainPathAppendsMountedToolchain(t *testing.T) {+ sh, err := exec.LookPath("sh")+ if err != nil {+ t.Skip("sh not available")+ }+ image := filepath.Dir(sh)+ toolchain := t.TempDir()+ if err := os.WriteFile(filepath.Join(toolchain, ".interp"), []byte(sh+"\n"), 0o644); err != nil {+ t.Fatal(err)+ }+ run := func(env ...string) []string {+ t.Helper()+ argv := withToolchainPath([]string{"sh", "-c", `printf '%s|%s|%s' "$PATH" "$1" "$2"`, "argv0", "two words", "--flag"})+ cmd := exec.Command(argv[0], argv[1:]...)+ cmd.Env = append([]string{"PATH=" + image}, env...)+ out, err := cmd.CombinedOutput()+ if err != nil {+ t.Fatalf("%v: %s", err, out)+ }+ return strings.Split(string(out), "|")+ }+ got := run(toolchainEnv + "=" + toolchain)+ if want := image + ":" + toolchain + "/bin:" + toolchain + "/python/bin"; got[0] != want {+ t.Fatalf("PATH = %q, want %q", got[0], want)+ }+ if got[1] != "two words" || got[2] != "--flag" {+ t.Fatalf("arguments not passed through: %q", got[1:])+ }+ for _, env := range [][]string{nil, {toolchainEnv + "="}, {toolchainEnv + "=" + filepath.Join(toolchain, "missing")}} {+ if got := run(env...); got[0] != image {+ t.Errorf("env %v: PATH = %q, want unchanged", env, got[0])+ }+ }+ for _, argv := range [][]string{dockerConversationProcessArgv("/tmp/p", []string{"codex"}), dockerConversationStopArgv("/tmp/p")} {+ if len(argv) < 4 || argv[0] != "sh" || argv[1] != "-c" || !strings.HasPrefix(argv[2], toolchainPathSetup) || argv[3] != "python3" {+ t.Fatalf("supervisor argv does not set up the toolchain PATH first: %q", argv)+ }+ }+}++// The base copy records node's ELF loader in .interp, which the sandbox PATH+// setup requires; a node that is not a dynamic ELF file leaves no .interp.+func TestRuntimeToolchainDirRecordsNodeLoader(t *testing.T) {+ sh, err := exec.LookPath("sh")+ if err != nil {+ t.Skip("sh not available")+ }+ sh, err = filepath.EvalSymlinks(sh)+ if err != nil {+ t.Fatal(err)+ }+ interp := elfInterpreter(sh)+ if interp == "" {+ t.Skipf("%s is not a dynamic ELF executable", sh)+ }+ if !strings.HasPrefix(interp, "/") || strings.ContainsAny(interp, "\x00\r\n") {+ t.Fatalf("elfInterpreter(%s) = %q", sh, interp)+ }+ if _, err := os.Stat(interp); err != nil {+ t.Fatalf("loader %q of %s does not exist: %v", interp, sh, err)+ }+ for _, path := range []string{filepath.Join(t.TempDir(), "missing"), writeScript(t, "#!/bin/sh\n"), writeScript(t, "\x7fELF but truncated")} {+ if got := elfInterpreter(path); got != "" {+ t.Errorf("elfInterpreter(%s) = %q, want empty", path, got)+ }+ }++ prefix := writeToolchainFixture(t)+ useToolchainFixture(t, prefix, t.TempDir())+ dir, err := RuntimeToolchainDir()+ if err != nil || dir == "" {+ t.Fatalf("RuntimeToolchainDir() = %q, %v", dir, err)+ }+ if _, err := os.Stat(filepath.Join(dir, ".interp")); !os.IsNotExist(err) {+ t.Fatalf("script node produced .interp: %v", err)+ }++ raw, err := os.ReadFile(sh)+ if err != nil {+ t.Fatal(err)+ }+ prefix = writeToolchainFixture(t)+ if err := os.WriteFile(filepath.Join(prefix, "bin", "node"), raw, 0o755); err != nil {+ t.Fatal(err)+ }+ useToolchainFixture(t, prefix, t.TempDir())+ dir, err = RuntimeToolchainDir()+ if err != nil || dir == "" {+ t.Fatalf("RuntimeToolchainDir() = %q, %v", dir, err)+ }+ if got, err := os.ReadFile(filepath.Join(dir, ".interp")); err != nil || string(got) != interp+"\n" {+ t.Fatalf(".interp = %q, %v; want %q", got, err, interp+"\n")+ }+}++func writeScript(t *testing.T, content string) string {+ t.Helper()+ path := filepath.Join(t.TempDir(), "script")+ if err := os.WriteFile(path, []byte(content), 0o755); err != nil {+ t.Fatal(err)+ }+ return path+}++// Native CLI preparation creates the toolchain for custom images only:+// sandboxes of the platform's isolated image already carry these tools.+func TestToolchainCLIExecEnvOnlyForCustomImages(t *testing.T) {+ prefix := writeToolchainFixture(t)+ cliDir := t.TempDir()+ useToolchainFixture(t, prefix, cliDir)+ t.Setenv("AGENTFLOW_ISOLATED_IMAGE", "loopforge-runtime:test")++ platform := &dockerSandbox{dockerfile: domain.DockerfileDefinition{Content: IsolatedDockerfile}}+ if env, err := platform.toolchainCLIExecEnv("fakecli"); err != nil || env != nil {+ t.Fatalf("platform image CLI exec env = %v, %v; want none", env, err)+ }+ if _, err := os.Stat(filepath.Join(cliDir, "toolchain")); !os.IsNotExist(err) {+ t.Fatalf("platform image sandbox copied the toolchain: %v", err)+ }++ custom := &dockerSandbox{dockerfile: domain.DockerfileDefinition{Content: "FROM debian:bookworm\n"}}+ dir := filepath.Join(cliDir, "toolchain", "testbuild")+ env, err := custom.toolchainCLIExecEnv("fakecli")+ if err != nil || !slices.Equal(env, []string{"-e", toolchainEnv + "=" + dir}) {+ t.Fatalf("custom image CLI exec env = %v, %v", env, err)+ }+ if !fileIsExecutable(filepath.Join(dir, "bin", "fakecli")) {+ t.Fatal("CLI preparation did not make the bundled CLI available")+ }+ if got := custom.toolchainExecEnv(); !slices.Equal(got, env) {+ t.Fatalf("sandbox commands after preparation: env = %v, want %v", got, env)+ }+ // A dockerfile with extra files is a custom image even with the same content.+ withFiles := &dockerSandbox{dockerfile: domain.DockerfileDefinition{Content: IsolatedDockerfile, Files: map[string]string{"extra": "x"}}}+ if got := withFiles.toolchainExecEnv(); !slices.Equal(got, env) {+ t.Fatalf("isolated content with extra files: env = %v, want %v", got, env)+ }++ // Outside the platform image even a custom-image sandbox gets no mount.+ if err := os.Remove(filepath.Join(prefix, "bin", "node")); err != nil {+ t.Fatal(err)+ }+ useToolchainFixture(t, prefix, t.TempDir())+ if env, err := custom.toolchainCLIExecEnv("fakecli"); err != nil || env != nil {+ t.Fatalf("no platform toolchain: env = %v, %v; want none", env, err)+ }+ if got := custom.toolchainExecEnv(); got != nil {+ t.Fatalf("no platform toolchain: exec env = %v, want none", got)+ }+}diff --git a/internal/runtime/toolchain_test.go b/internal/runtime/toolchain_test.gonew file mode 100644index 00000000..073835a4--- /dev/null+++ b/internal/runtime/toolchain_test.go@@ -0,0 +1,232 @@+package runtime++import (+ "os"+ "os/exec"+ "path/filepath"+ "strings"+ "sync"+ "testing"+ "time"++ "agentflow/internal/domain"+)++func writeToolchainFixture(t *testing.T) string {+ t.Helper()+ prefix := t.TempDir()+ for _, dir := range []string{"bin", "lib/node_modules/npm/bin", "lib/node_modules/fakepkg/bin"} {+ if err := os.MkdirAll(filepath.Join(prefix, dir), 0o755); err != nil {+ t.Fatal(err)+ }+ }+ for _, file := range []string{"bin/node", "lib/node_modules/npm/bin/npm-cli.js", "lib/node_modules/fakepkg/bin/fake.js"} {+ if err := os.WriteFile(filepath.Join(prefix, file), []byte("#!/bin/sh\n"), 0o755); err != nil {+ t.Fatal(err)+ }+ }+ for name, target := range map[string]string{+ "npm": "../lib/node_modules/npm/bin/npm-cli.js",+ "npx": "../lib/node_modules/npm/bin/npm-cli.js",+ "fakecli": "../lib/node_modules/fakepkg/bin/fake.js",+ } {+ if err := os.Symlink(target, filepath.Join(prefix, "bin", name)); err != nil {+ t.Fatal(err)+ }+ }+ return prefix+}++func useToolchainFixture(t *testing.T, prefix, cliDir string) {+ t.Helper()+ oldPrefix, oldPython, oldCommit := toolchainNodePrefix, toolchainPythonDir, toolchainCommitFile+ toolchainNodePrefix = prefix+ toolchainPythonDir = filepath.Join(prefix, "no-python")+ toolchainCommitFile = filepath.Join(prefix, "commit")+ t.Cleanup(func() {+ toolchainNodePrefix, toolchainPythonDir, toolchainCommitFile = oldPrefix, oldPython, oldCommit+ })+ if err := os.WriteFile(toolchainCommitFile, []byte("testbuild\n"), 0o644); err != nil {+ t.Fatal(err)+ }+ t.Setenv("AGENTFLOW_RUNTIME_CLI_DIR", cliDir)+}++func startToolchainChild(t *testing.T, prefix, cliDir, marker string) (*exec.Cmd, *strings.Builder) {+ t.Helper()+ cmd := exec.Command(os.Args[0], "-test.run=^TestToolchainChildProcess$", "-test.count=1", "-test.v")+ cmd.Env = append(os.Environ(), "LF_TOOLCHAIN_CHILD=1", "LF_TOOLCHAIN_PREFIX="+prefix, "LF_TOOLCHAIN_CLI="+cliDir, "LF_TOOLCHAIN_MARKER="+marker)+ out := &strings.Builder{}+ cmd.Stdout, cmd.Stderr = out, out+ if err := cmd.Start(); err != nil {+ t.Fatal(err)+ }+ return cmd, out+}++// TestToolchainChildProcess is the second platform process of the tests below.+func TestToolchainChildProcess(t *testing.T) {+ if os.Getenv("LF_TOOLCHAIN_CHILD") != "1" {+ t.Skip("helper process")+ }+ useToolchainFixture(t, os.Getenv("LF_TOOLCHAIN_PREFIX"), os.Getenv("LF_TOOLCHAIN_CLI"))+ if marker := os.Getenv("LF_TOOLCHAIN_MARKER"); marker != "" {+ toolchainBeforeLock = func() { _ = os.WriteFile(marker, nil, 0o644) }+ }+ dir, err := RuntimeToolchainDir()+ if err != nil {+ t.Fatal(err)+ }+ if err := EnsureToolchainCLI(dir, "fakecli"); err != nil {+ t.Fatal(err)+ }+ t.Log("TOOLCHAIN_DIR=" + dir)+}++// A process that passed the unlocked .ready check while another process+// published the toolchain must reuse it, not replace it.+func TestToolchainLatePublisherKeepsPublishedCLI(t *testing.T) {+ prefix := writeToolchainFixture(t)+ cliDir := t.TempDir()+ useToolchainFixture(t, prefix, cliDir)+ parent := filepath.Join(cliDir, "toolchain")+ dir := filepath.Join(parent, "testbuild")+ unlock, err := lockToolchain(parent)+ if err != nil {+ t.Fatal(err)+ }+ marker := filepath.Join(t.TempDir(), "waiting")+ child, out := startToolchainChild(t, prefix, cliDir, marker)+ deadline := time.Now().Add(30 * time.Second)+ for {+ if _, err := os.Stat(marker); err == nil {+ break+ }+ if time.Now().After(deadline) {+ unlock()+ t.Fatalf("child never reached the toolchain lock: %s", out)+ }+ time.Sleep(20 * time.Millisecond)+ }+ // Publish as the other process would while the child waits on the lock.+ if err := os.MkdirAll(filepath.Join(dir, "bin"), 0o755); err != nil {+ t.Fatal(err)+ }+ published := filepath.Join(dir, "bin", "published-cli")+ if err := os.WriteFile(published, []byte("#!/bin/sh\n"), 0o755); err != nil {+ t.Fatal(err)+ }+ if err := os.WriteFile(filepath.Join(dir, ".ready"), nil, 0o644); err != nil {+ t.Fatal(err)+ }+ unlock()+ if err := child.Wait(); err != nil {+ t.Fatalf("child: %v\n%s", err, out)+ }+ if !strings.Contains(out.String(), "TOOLCHAIN_DIR="+dir) {+ t.Fatalf("child used another directory:\n%s", out)+ }+ if !fileIsExecutable(published) {+ t.Fatal("the late process replaced the published toolchain")+ }+ if !fileIsExecutable(filepath.Join(dir, "bin", "fakecli")) {+ t.Fatal("the late process did not add its CLI to the published toolchain")+ }+}++func TestToolchainConcurrentProcessesShareOneCopy(t *testing.T) {+ prefix := writeToolchainFixture(t)+ cliDir := t.TempDir()+ useToolchainFixture(t, prefix, cliDir)+ var wg sync.WaitGroup+ errs := make([]string, 4)+ for i := range errs {+ wg.Add(1)+ go func(i int) {+ defer wg.Done()+ child, out := startToolchainChild(t, prefix, cliDir, "")+ if err := child.Wait(); err != nil {+ errs[i] = err.Error() + "\n" + out.String()+ }+ }(i)+ }+ wg.Wait()+ for _, err := range errs {+ if err != "" {+ t.Fatal(err)+ }+ }+ dir := filepath.Join(cliDir, "toolchain", "testbuild")+ for _, name := range []string{"node", "npm", "npx", "fakecli"} {+ if _, err := os.Lstat(filepath.Join(dir, "bin", name)); err != nil {+ t.Fatalf("missing %s: %v", name, err)+ }+ }+ entries, err := os.ReadDir(filepath.Join(cliDir, "toolchain"))+ if err != nil {+ t.Fatal(err)+ }+ for _, entry := range entries {+ if strings.HasPrefix(entry.Name(), ".stage-") {+ t.Fatalf("leftover stage %s", entry.Name())+ }+ }+}++// A sandbox command only uses a published toolchain: it never copies one and+// never fails without it.+func TestToolchainExecEnvUsesOnlyPublishedToolchain(t *testing.T) {+ prefix := writeToolchainFixture(t)+ cliDir := t.TempDir()+ useToolchainFixture(t, prefix, cliDir)+ t.Setenv("AGENTFLOW_ISOLATED_IMAGE", "loopforge:test")+ custom := &dockerSandbox{dockerfile: domain.DockerfileDefinition{Content: "FROM debian:bookworm\n"}}+ if env := custom.toolchainExecEnv(); env != nil {+ t.Fatalf("unpublished toolchain exposed: %q", env)+ }+ if _, err := os.Stat(filepath.Join(cliDir, "toolchain")); !os.IsNotExist(err) {+ t.Fatalf("a sandbox command started a toolchain copy: %v", err)+ }+ dir, err := RuntimeToolchainDir()+ if err != nil || dir == "" {+ t.Fatalf("dir=%q err=%v", dir, err)+ }+ if env := custom.toolchainExecEnv(); strings.Join(env, " ") != "-e "+toolchainEnv+"="+dir {+ t.Fatalf("published toolchain env = %q", env)+ }+ platform := &dockerSandbox{dockerfile: domain.DockerfileDefinition{Content: IsolatedDockerfile}}+ if env := platform.toolchainExecEnv(); env != nil {+ t.Fatalf("platform image sandbox got the toolchain: %q", env)+ }+}++func TestToolchainPathSetupRequiresItsLoader(t *testing.T) {+ toolchain := t.TempDir()+ run := func(interp string) string {+ t.Helper()+ if err := os.WriteFile(filepath.Join(toolchain, ".interp"), []byte(interp+"\n"), 0o644); err != nil {+ t.Fatal(err)+ }+ cmd := exec.Command("sh", "-c", "set -eu\nPATH=/usr/bin:/bin\n"+toolchainPathSetup+`printf %s "$PATH"`)+ cmd.Env = append(os.Environ(), toolchainEnv+"="+toolchain)+ out, err := cmd.CombinedOutput()+ if err != nil {+ t.Fatalf("setup failed: %v: %s", err, out)+ }+ return string(out)+ }+ if got, want := run("/bin/sh"), "/usr/bin:/bin:"+toolchain+"/bin:"+toolchain+"/python/bin"; got != want {+ t.Fatalf("PATH with loader present = %q, want %q", got, want)+ }+ if got := run(filepath.Join(toolchain, "missing-ld.so")); got != "/usr/bin:/bin" {+ t.Fatalf("PATH without loader = %q", got)+ }+ if err := os.Remove(filepath.Join(toolchain, ".interp")); err != nil {+ t.Fatal(err)+ }+ cmd := exec.Command("sh", "-c", "set -eu\nPATH=/usr/bin:/bin\n"+toolchainPathSetup+`printf %s "$PATH"`)+ cmd.Env = append(os.Environ(), toolchainEnv+"="+toolchain)+ if out, err := cmd.CombinedOutput(); err != nil || string(out) != "/usr/bin:/bin" {+ t.Fatalf("PATH without .interp = %q, %v", out, err)+ }+}