Fully autonomous AI Agents system capable of performing complex penetration testing tasks
Go ai-agentsai-security-toolanthropicautonomous-agentsgolanggptgraphqlmulti-agent-systemoffensive-securityopen-sourceopenaipenetration-testingpenetration-testing-toolsreactsecurity-automationsecurity-testingsecurity-toolsself-hosted
fix: installer stack detection, tests that can fail, proxy and port docs
Dmitry Ngb611fbebcf55
diff --git a/.env.example b/.env.exampleindex e30f7e34..cd9914b9 100644--- a/.env.example+++ b/.env.example@@ -204,7 +204,8 @@ MAX_LIMITED_AGENT_TOOL_CALLS=## Agent planning step for pentester, coder, installerAGENT_PLANNING_STEP_ENABLED=-## HTTP proxy to use it in isolation environment+## HTTP proxy for every backend LLM, embedding, search and update-check call (local endpoints included);+## sandboxes, the scraper and Graphiti do not use itPROXY_URL=## SSL/TLS Certificate Configurationdiff --git a/README.md b/README.mdindex 3bfd587c..ef526309 100644--- a/README.md+++ b/README.md@@ -882,13 +882,16 @@ For multi-user setups, an authenticated administrator can manage local users thr>> `LLM_SERVER_*` environment variables are experimental feature and will be changed in the future. Right now you can use them to specify custom LLM server URL and one model for all agent types.>-> `PROXY_URL` is a global proxy URL for all LLM providers and external search systems. You can use it for isolation from external networks.+> `PROXY_URL` routes the backend's own HTTP calls through a proxy: every request to LLM and embedding providers and to search engines — self-hosted ones such as Ollama or SearXNG included, as there is no `NO_PROXY` exemption — and the update check. Sandbox containers (where agents run their commands), the scraper, the Graphiti service, OAuth sign-in, and Langfuse/OpenTelemetry export do not use it, so if you need isolation from external networks, restrict their outbound traffic at the network level.>> The `docker-compose.yml` file runs the PentAGI service as root user because it needs access to docker.sock for container management. If you're using TCP/IP network connection to Docker instead of socket file, you can remove root privileges and use the default `pentagi` user for better security.### Accessing PentAGI from External Networks-By default, PentAGI binds to `127.0.0.1` (localhost only) for security. To access PentAGI from other machines on your network, you need to configure external access.+By default, the PentAGI web interface and the other compose services bind to `127.0.0.1` (localhost only) for security. To access PentAGI from other machines on your network, you need to configure external access.++> [!IMPORTANT]+> Flow sandboxes are not bound to localhost. Each flow publishes two TCP ports for out-of-band callbacks such as reverse shells, taken from the 2000-port window that starts at `DOCKER_PORTS_BASE` (`28000`–`29999` by default), on `DOCKER_PUBLIC_IP` — `0.0.0.0` by default, that is every interface of the Docker host. Docker routes published ports around `ufw` rules and lets them through `firewalld` with its own `docker` zone, so restrict this range with a firewall in front of the host or, with Docker's default iptables backend, with rules in the `DOCKER-USER` chain. With `DOCKER_NETWORK=host`, sandboxes share the host's network stack, so their listeners are exposed without any port mapping.#### Configuration Steps@@ -3100,7 +3103,7 @@ Example Docker daemon mirror configuration:}```-On Linux, this is typically configured in `/etc/docker/daemon.json`. On Docker Desktop, use the equivalent Docker Engine or proxy settings. A Docker Hub mirror covers Docker Hub-hosted images such as `vxcontrol/*`, but the main Compose stack already includes `quay.io/prometheuscommunity/postgres-exporter`, and the optional observability stack includes `gcr.io/cadvisor/cadvisor`. Those registries still need direct access or individually approved proxy/mirror paths.+On Linux, this is typically configured in `/etc/docker/daemon.json`. On Docker Desktop, use the equivalent Docker Engine or proxy settings. A Docker Hub mirror covers Docker Hub-hosted images such as `vxcontrol/*`, but the main Compose stack already includes `quay.io/prometheuscommunity/postgres-exporter`, the optional observability stack includes `gcr.io/cadvisor/cadvisor`, and the optional Langfuse stack includes `cgr.dev/chainguard/minio`. Those registries still need direct access or individually approved proxy/mirror paths.See the official Docker documentation for [registry mirrors](https://docs.docker.com/docker-hub/image-library/mirror/) and [daemon proxy configuration](https://docs.docker.com/engine/daemon/proxy/).@@ -3706,7 +3709,7 @@ EMBEDDING_STRIP_NEW_LINES=true # Whether to remove new lines from text before eEMBEDDING_MAX_TEXT_BYTES=8192 # Max bytes of text sent to embedding model per document (byte proxy for token limit)# Advanced settings-PROXY_URL= # Optional proxy for all API calls+PROXY_URL= # Optional proxy for all backend LLM, embedding, search and update-check callsHTTP_CLIENT_TIMEOUT=600 # Timeout in seconds for external API calls (default: 600, 0 = no timeout)TERMINAL_TOOL_TIMEOUT=1200 # Default timeout in seconds for terminal tool commands when timeout=0 or negative (range: 1–10800; values <= 0 or above 10800 are clamped to 10800 = 3 hours)diff --git a/backend/cmd/installer/checker/checker.go b/backend/cmd/installer/checker/checker.goindex 19425712..7b43a778 100644--- a/backend/cmd/installer/checker/checker.go+++ b/backend/cmd/installer/checker/checker.go@@ -427,8 +427,8 @@ func (h *defaultCheckHandler) GatherPentagiInfo(ctx context.Context, c *CheckRes// would count as not extracted, and extraction overwrites the examples the user// edited when the update is forced. Verification of an extracted stack writes it.c.PentagiExtracted = checkFileExists(dockerComposeFile) &&- checkFileExists(ExampleCustomConfigLLMFile) &&- checkFileExists(ExampleOllamaConfigLLMFile)+ checkFileExists(filepath.Join(envDir, ExampleCustomConfigLLMFile)) &&+ checkFileExists(filepath.Join(envDir, ExampleOllamaConfigLLMFile))c.PentagiScriptInstalled = checkFileExists(PentagiScriptFile)if h.dockerClient != nil {diff --git a/backend/cmd/installer/checker/checker_test.go b/backend/cmd/installer/checker/checker_test.goindex 358d8bff..10781500 100644--- a/backend/cmd/installer/checker/checker_test.go+++ b/backend/cmd/installer/checker/checker_test.go@@ -1,6 +1,7 @@package checkerimport (+ "os""path/filepath""sync""testing"@@ -63,3 +64,48 @@ func TestChecker_NeverClaimsExternalForAStackThatIsNotConnected(t *testing.T) {})}}++func TestChecker_GatherPentagiInfo_LooksForTheExtractedStackBesideTheEnvFile(t *testing.T) {+ tests := []struct {+ name string+ beside, elsewhere []string // files beside the env file, and in the working directory+ want bool+ }{+ {+ "the compose file and both examples beside the env file",+ []string{"docker-compose.yml", "example.custom.provider.yml", "example.ollama.provider.yml"}, nil, true,+ },+ {"the compose file alone", []string{"docker-compose.yml"}, nil, false},+ {"the ollama example missing", []string{"docker-compose.yml", "example.custom.provider.yml"}, nil, false},+ {"the custom example missing", []string{"docker-compose.yml", "example.ollama.provider.yml"}, nil, false},+ {"the compose file missing", []string{"example.custom.provider.yml", "example.ollama.provider.yml"}, nil, false},+ {+ "the examples in the working directory only",+ []string{"docker-compose.yml"}, []string{"example.custom.provider.yml", "example.ollama.provider.yml"}, false,+ },+ }++ for _, tt := range tests {+ t.Run(tt.name, func(t *testing.T) {+ envDir, workDir := t.TempDir(), t.TempDir()+ for dir, names := range map[string][]string{envDir: tt.beside, workDir: tt.elsewhere} {+ for _, name := range names {+ if err := os.WriteFile(filepath.Join(dir, name), []byte("x"), 0o644); err != nil {+ t.Fatalf("write %s: %v", name, err)+ }+ }+ }+ t.Chdir(workDir)++ handler := &defaultCheckHandler{mx: &sync.Mutex{}, appState: &mockState{envPath: filepath.Join(envDir, ".env")}}+ got := &CheckResult{}+ if err := handler.GatherPentagiInfo(t.Context(), got); err != nil {+ t.Fatalf("gather: %v", err)+ }++ if got.PentagiExtracted != tt.want {+ t.Errorf("PentagiExtracted = %t, want %t", got.PentagiExtracted, tt.want)+ }+ })+ }+}diff --git a/backend/cmd/installer/checker/helpers.go b/backend/cmd/installer/checker/helpers.goindex bcea99a7..df3fdb0d 100644--- a/backend/cmd/installer/checker/helpers.go+++ b/backend/cmd/installer/checker/helpers.go@@ -767,25 +767,9 @@ func getImageInfo(ctx context.Context, cli *client.Client, imageName string) *Imreturn imageInfo}-// parseImageRef splits a Docker image reference into the repository and tag the update-// server matches on, plus whatever digest the reference or the daemon carried.-//-// The shape is [registry[:port]/]path[:tag][@digest], and the two colons are the trap: the-// one that separates a registry port and the one that separates a tag look identical. They-// are told apart by position, not by content — a tag colon is the LAST colon after the last-// slash. The previous rule guessed by content, treating any tag containing a dot as a port-// number, which misread every version-numbered tag the compose files use: "grafana/grafana"-// at "11.4.0" became a repository literally named "grafana/grafana:11.4.0" at tag "latest".-//-// That failure is silent where it matters. The update server answers on the repository and-// tag pair, so a mangled pair simply gets no answer — and no answer is indistinguishable-// from "nothing to update".// isDockerHubHost reports whether a leading path element is one of the names-// Docker Hub answers to.-//-// A closed list rather than a shape test. "Contains a dot or a colon" describes-// every registry there is, and treating every registry as Docker Hub is exactly-// how the host came to be dropped from references that need it.+// Docker Hub answers to. It is a closed list and not a shape test: "contains a+// dot or a colon" describes every registry there is.func isDockerHubHost(head string) bool {switch head {case "docker.io", "index.docker.io", "registry-1.docker.io", "registry.hub.docker.com":@@ -795,6 +779,14 @@ func isDockerHubHost(head string) bool {}}+// parseImageRef splits a Docker image reference into the repository and tag the update+// server matches on, plus whatever digest the reference or the daemon carried.+//+// The shape is [registry[:port]/]path[:tag][@digest]. The colon of a registry port and the+// colon of a tag are told apart by position, not by content: a tag colon is the last colon+// after the last slash, so a version-numbered tag such as "11.4.0" is not taken for a port.+// A mangled pair fails silently: the update server matches on the repository and tag pair,+// and a pair it does not track is offered nothing, which reads as "nothing to update".func parseImageRef(imageRef, imageID string) *ImageInfo {if imageRef == "" {return nil@@ -820,14 +812,10 @@ func parseImageRef(imageRef, imageID string) *ImageInfo {// The service stores a reference the way compose writes it: bare for Docker// Hub, host and all for anywhere else — `gcr.io/cadvisor/cadvisor`, not// `cadvisor/cadvisor`. Those two are different images: the second is a Docker- // Hub repository that has nothing to do with cAdvisor. Dropping the host for- // every registry made this client report the second while running the first,- // and since the match is an exact string comparison, the component simply- // resolved to nothing — answered `repository_not_tracked` forever, and- // dragging its whole stack's resolution down to `not_tracked` with it.- //- // It stayed invisible until cadvisor and pgexporter — the only two components- // not on Docker Hub — started being reported at all.+ // Hub repository that has nothing to do with cAdvisor. The match is an exact+ // string comparison, so a reference of another registry reported without its+ // host is answered `repository_not_tracked` and takes its whole stack's+ // resolution down to `not_tracked`.if head, rest, found := strings.Cut(imageRef, "/"); found {if isDockerHubHost(head) {imageRef = restdiff --git a/backend/cmd/installer/processor/fs.go b/backend/cmd/installer/processor/fs.goindex fefe02ad..97ea22f6 100644--- a/backend/cmd/installer/processor/fs.go+++ b/backend/cmd/installer/processor/fs.go@@ -161,7 +161,8 @@ func (fs *fileSystemOperationsImpl) verifyStackIntegrity(ctx context.Context, st}}-// checkStackIntegrity is a silent version of verifyStackIntegrity, used for getting files statuses+// checkStackIntegrity is the read-only counterpart of verifyStackIntegrity; it does not report+// the provider examples verifyStackIntegrity restores for pentagi.func (fs *fileSystemOperationsImpl) checkStackIntegrity(ctx context.Context, stack ProductStack) (FilesCheckResult, error) {result := make(FilesCheckResult)diff --git a/backend/cmd/installer/processor/fs_test.go b/backend/cmd/installer/processor/fs_test.goindex 0472e1b0..3d27be20 100644--- a/backend/cmd/installer/processor/fs_test.go+++ b/backend/cmd/installer/processor/fs_test.go@@ -155,8 +155,7 @@ func TestFs_VerifyStackIntegrity_RestoresAProviderExampleThatIsNotAFile(t *testi}}-// Over the embedded files and the compose files beside them: Docker creates a directory in-// place of a file a compose file mounts and the stack does not extract.+// Over the embedded files and the compose files at the repository root.func TestFs_EnsureStackIntegrity_ExtractsEveryFileItsComposeFileMounts(t *testing.T) {withDefault := regexp.MustCompile(`\$\{[A-Za-z0-9_]+:?-([^}]*)\}`)for _, tc := range []struct {@@ -179,7 +178,7 @@ func TestFs_EnsureStackIntegrity_ExtractsEveryFileItsComposeFileMounts(t *testin"./observability/jaeger", "./observability/loki/config.yml", "./observability/otel",}},} {- t.Run(string(tc.stack), func(t *testing.T) {+ t.Run("the "+string(tc.stack)+" stack", func(t *testing.T) {content, err := os.ReadFile(filepath.Join(repositoryRoot(t), tc.composeFile))require.NoError(t, err)var compose struct {@@ -221,17 +220,15 @@ func TestFs_EnsureStackIntegrity_ExtractsEveryFileItsComposeFileMounts(t *testin}}-// Over the embedded files themselves. Subtests are keyed by unit: both replace the directory-// with the example, and only the verification of an extracted stack spares an edited example-// from a forced update.+// Over the embedded files themselves; subtests are keyed by unit.func TestFs_ADirectoryInPlaceOfAProviderExampleBecomesTheExample(t *testing.T) {for _, tc := range []struct {name stringrun func(fileSystemOperations, context.Context, ProductStack, *operationState) errorwantCustom string}{- {"ensureStackIntegrity", fileSystemOperations.ensureStackIntegrity, "the example"},- {"verifyStackIntegrity", fileSystemOperations.verifyStackIntegrity, "edited: true\n"},+ {"a forced extraction overwrites the edited example", fileSystemOperations.ensureStackIntegrity, "the example"},+ {"a forced verification keeps the edited example", fileSystemOperations.verifyStackIntegrity, "edited: true\n"},} {t.Run(tc.name, func(t *testing.T) {embedded := files.NewFiles()@@ -312,21 +309,21 @@ func TestFs_CleanupStackFiles_RemovesEveryFileOfTheStackAndNothingElse(t *testinfunc TestFs_EnsureFileFromEmbed_CopiesOnlyWhatIsMissingUnlessForced(t *testing.T) {for _, tc := range []struct {- name string- onDisk, force bool- want []string+ name string+ onDisk, directory, force bool+ want []string}{- {"a missing file is extracted", false, false, []string{"test.yml"}},- {"a file on disk is kept", true, false, nil},- {"a file on disk is overwritten when forced", true, true, []string{"test.yml"}},- {"a directory in its place is replaced", false, false, []string{"test.yml"}},+ {"a missing file is extracted", false, false, false, []string{"test.yml"}},+ {"a file on disk is kept", true, false, false, nil},+ {"a file on disk is overwritten when forced", true, false, true, []string{"test.yml"}},+ {"a directory in its place is replaced", false, true, false, []string{"test.yml"}},} {t.Run(tc.name, func(t *testing.T) {ops, embedded, dir := fsOperations(t)embedded.AddFile("test.yml", []byte("test content"))if tc.onDisk {require.NoError(t, os.WriteFile(filepath.Join(dir, "test.yml"), []byte("existing"), 0o644))- } else if strings.HasPrefix(tc.name, "a directory") {+ } else if tc.directory {require.NoError(t, os.Mkdir(filepath.Join(dir, "test.yml"), 0o755))}state := testOperationState(t)diff --git a/backend/cmd/installer/processor/logic.go b/backend/cmd/installer/processor/logic.goindex ae835176..ef5ac755 100644--- a/backend/cmd/installer/processor/logic.go+++ b/backend/cmd/installer/processor/logic.go@@ -339,9 +339,10 @@ func (p *processor) applyPentagiChanges(ctx context.Context, state *operationStareturn nil}-// checkFiles computes file statuses for a given stack, honoring the same-// rules as verifyStackIntegrity: active stacks only and excluded files policy.-// It serves as a dry-run for file operations without performing any writes.+// checkFiles computes file statuses for a given stack under the rules the apply+// path follows: embedded stacks only, and the excluded files policy of+// verifyStackIntegrity. It writes nothing; the provider examples+// verifyStackIntegrity restores for pentagi are not reported.func (p *processor) checkFiles(ctx context.Context, stack ProductStack, state *operationState,) (result map[string]files.FileStatus, err error) {diff --git a/backend/cmd/installer/wizard/controller/controller_test.go b/backend/cmd/installer/wizard/controller/controller_test.goindex 988f4e3a..29eb5d1f 100644--- a/backend/cmd/installer/wizard/controller/controller_test.go+++ b/backend/cmd/installer/wizard/controller/controller_test.go@@ -309,7 +309,7 @@ func TestController_GetApplyChangesConfig_DescribesAConfigPathChangeAndCallsItCr"OLLAMA_SERVER_CONFIG_PATH", "PENTAGI_OLLAMA_SERVER_CONFIG_PATH","LLM_SERVER_CONFIG_PATH", "PENTAGI_LLM_SERVER_CONFIG_PATH",} {- t.Run(name, func(t *testing.T) {+ t.Run("a change of "+name, func(t *testing.T) {c, _ := controllerOverExampleEnv(t)require.NoError(t, c.Commit())require.False(t, c.GetApplyChangesConfig().HasCritical, "the example env is critical before anything changed")diff --git a/backend/cmd/installer/wizard/locale/locale.go b/backend/cmd/installer/wizard/locale/locale.goindex dd241add..fa9927b9 100644--- a/backend/cmd/installer/wizard/locale/locale.go+++ b/backend/cmd/installer/wizard/locale/locale.go@@ -1468,7 +1468,7 @@ Examples:ServerSettingsTrustedProxiesDesc = "Comma-separated proxy IPs/CIDRs whose X-Forwarded-For is trusted (empty: trust none)"ServerSettingsProxyURL = "HTTP/HTTPS Proxy"- ServerSettingsProxyURLDesc = "Proxy for outbound requests to LLMs and external tools (not used for Docker API access)"+ ServerSettingsProxyURLDesc = "Proxy for backend LLM, embedding and search requests (not for sandboxes, scraper, Graphiti)"ServerSettingsProxyUsername = "Proxy Username"ServerSettingsProxyUsernameDesc = "Username for proxy authentication (optional)"@@ -1571,7 +1571,7 @@ Examples:Set this only when PentAGI sits behind a reverse proxy or load balancer you control. Leave empty to trust none. Example: 10.0.0.0/8,172.16.0.0/12`- ServerSettingsProxyURLHelp = `HTTP or HTTPS proxy for outbound requests to LLM providers and external tools. Not used for Docker API communication.`+ ServerSettingsProxyURLHelp = `HTTP or HTTPS proxy for every backend request to LLM, embedding and search providers, local endpoints included, and for update checks; the installer's own update and download calls use it too. Sandbox containers, the scraper, Graphiti and Docker API communication do not use it.`ServerSettingsHTTPClientTimeoutHelp = `Timeout in seconds for all external HTTP/HTTPS API calls including:• LLM provider requests (OpenAI, Anthropic, Bedrock, etc.)@@ -1838,7 +1838,7 @@ const (Critical for penetration testing workflows requiring network scanning, custom tools, and secure task isolation.`// General help text- ToolsDockerGeneralHelp = `Each AI agent task runs in an isolated Docker container with two ports (28000-32000 range) automatically allocated per flow. Worker containers are created on-demand from default images or agent-selected ones.+ ToolsDockerGeneralHelp = `Each AI agent task runs in an isolated Docker container with two ports allocated per flow from the 2000-port window that starts at DOCKER_PORTS_BASE (28000-29999 by default). Worker containers are created on-demand from default images or agent-selected ones.Basic setup requires enabling capabilities: Docker Access allows spawning additional containers for specialized tools, while Network Admin grants low-level network permissions essential for scanning tools like nmap.@@ -1934,7 +1934,7 @@ Security Note: Host network mode reduces container isolation. Only use when neceToolsDockerPublicIPHelp = `Public IP Address enables out-of-band (OOB) attack techniques by providing workers with a reachable address for reverse connections.-Workers automatically receive two random ports (28000-32000 range) mapped to this IP for receiving callbacks from exploited targets.+Each flow's worker gets two ports from the 2000-port window that starts at DOCKER_PORTS_BASE (28000-29999 by default), published on this IP for receiving callbacks from exploited targets. The default 0.0.0.0 publishes them on every interface of the host.By default agents will try to get public address from the services api.ipify.org, ipinfo.io/ip or ifconfig.me.`diff --git a/backend/cmd/installer/wizard/models/llm_provider_form.go b/backend/cmd/installer/wizard/models/llm_provider_form.goindex da54a17d..e8292c2b 100644--- a/backend/cmd/installer/wizard/models/llm_provider_form.go+++ b/backend/cmd/installer/wizard/models/llm_provider_form.go@@ -317,8 +317,8 @@ func (m *LLMProviderFormModel) createConfigPathField(config *controller.LLMProviif config.HostConfigPath.Default == "" {input.Placeholder = controller.LLMConfigMountPath(string(m.providerID))}- // the path shown may come from the container variable while the host one is not in- // the env file at all, and an input takes its value only from a variable that is+ // the path shown may come from the container variable while the host one is neither in+ // the env file nor changed, and NewTextInput leaves the input of such a variable emptyinput.SetValue(config.HostConfigPath.Value)return FormField{diff --git a/backend/cmd/installer/wizard/models/llm_provider_form_test.go b/backend/cmd/installer/wizard/models/llm_provider_form_test.goindex 519744d8..5baa0301 100644--- a/backend/cmd/installer/wizard/models/llm_provider_form_test.go+++ b/backend/cmd/installer/wizard/models/llm_provider_form_test.go@@ -78,8 +78,7 @@ func TestLLMProviderForm_GivesEveryOfferedDoorItsOwnNameDescriptionAndURL(t *tes}}-// BEDROCK_CONFIG_PATH is read inside the container, where a path on the host does not exist.-// The env file has no PENTAGI_BEDROCK_CONFIG_PATH line, as every host installed from a release.+// The env file has no PENTAGI_BEDROCK_CONFIG_PATH line, as on a host whose .env predates the variable.func TestLLMProviderForm_HandleSave_MountsABedrockConfigTypedAsAFileOnTheHost(t *testing.T) {onHost := filepath.Join(t.TempDir(), "my-bedrock.provider.yml")if err := os.WriteFile(onHost, []byte("simple:\n model: zai.glm-4.7-flash\n"), 0o600); err != nil {diff --git a/backend/docs/config.md b/backend/docs/config.mdindex 4dc244df..cb659944 100644--- a/backend/docs/config.md+++ b/backend/docs/config.md@@ -1894,7 +1894,7 @@ googleSearch: &functions.GoogleSearchFunc{```The proxy setting is essential for:-- Routing all outbound API requests through a controlled proxy+- Routing the backend's LLM, embedding, search and update-check requests through a controlled proxy (sandbox containers, the scraper and Graphiti do not use it)- Implementing network-level security policies- Enabling access to external services from restricted networks- Monitoring and auditing external API usagediff --git a/backend/docs/installer/checker.md b/backend/docs/installer/checker.mdindex a4e04313..a0316299 100644--- a/backend/docs/installer/checker.md+++ b/backend/docs/installer/checker.md@@ -90,7 +90,7 @@ Recommended remediation order:3. If Docker Hub is blocked or rate-limited, configure an organization-approved Docker registry mirror or registry proxy at the Docker daemon / Docker Desktop level4. Restart Docker and rerun the installer checks-PentAGI variables such as `PENTAGI_IMAGE`, `DOCKER_DEFAULT_IMAGE`, and `DOCKER_DEFAULT_IMAGE_FOR_PENTEST` do not replace Docker daemon registry configuration. They only influence the PentAGI application image or worker image selection after Docker is already able to pull the required images. Note that the main Compose stack already includes a service from `quay.io` (`postgres-exporter`), and the optional observability stack includes an image from `gcr.io`. A Docker Hub mirror alone is therefore not sufficient for a full deployment — those registries also need to be reachable or individually mirrored.+PentAGI variables such as `PENTAGI_IMAGE`, `DOCKER_DEFAULT_IMAGE`, and `DOCKER_DEFAULT_IMAGE_FOR_PENTEST` do not replace Docker daemon registry configuration. They only influence the PentAGI application image or worker image selection after Docker is already able to pull the required images. Note that the main Compose stack already includes a service from `quay.io` (`postgres-exporter`), the optional observability stack includes an image from `gcr.io`, and the optional Langfuse stack includes one from `cgr.dev` (`minio`). A Docker Hub mirror alone is therefore not sufficient for a full deployment — those registries also need to be reachable or individually mirrored.See Docker's official documentation for [registry mirrors](https://docs.docker.com/docker-hub/image-library/mirror/) and [daemon proxy configuration](https://docs.docker.com/engine/daemon/proxy/).@@ -111,7 +111,7 @@ For every container image the request carries:- **Component** — the artefact's own name, taken from an explicit component → container table. Nothing here is derived: for four of the sixteen images the container name is neither the compose service name nor the component name — Langfuse's `postgres`, `clickhouse`, `redis` and `minio` services run in containers called `langfuse-postgres`, `langfuse-clickhouse`, `langfuse-redis` and `langfuse-minio`, while the component sent stays the bare `postgres`, `clickhouse`, `redis`, `minio` — and a wrong container name is indistinguishable from an absent one, so a guess would silently drop the component instead of failing. The container name is what the table holds and what `ContainerInspect` is given; the compose `hostname:` field (`otelcol` for the `otel` service) is never read.- **Status** — `running` when the container is up, `installed` when it exists but is stopped.- **OS and Arch** — the **image's**, read from `ImageInspect` (`.Os`, `.Architecture`), never the host's. A container image is `linux` even on macOS and Windows, where Docker runs in a VM, and on Apple silicon an `amd64` image may well be running under emulation. Substituting `runtime.GOOS`/`runtime.GOARCH` is not caught by validation — `darwin` is a valid value, it just matches no published artefact — so the installation would silently be told "no updates" forever. An image on a platform the contract has no name for (say `arm/v7`) is dropped with a log line rather than rounded to the nearest value: rounding asks about one image and gets an answer about another.-- **Repository and Tag** — parsed from the reference the container was configured with, with the registry prefix and the implicit `library/` removed, and the tag `docker pull` would apply filled in when the reference omits one. Both halves are required: an update is the newest image *under a reference*, so a component that cannot name its reference cannot be resolved at all.+- **Repository and Tag** — parsed from the reference the container was configured with, with a Docker Hub host and the implicit `library/` removed and the host of any other registry kept, and the tag `docker pull` would apply filled in when the reference omits one. Both halves are required: an update is the newest image *under a reference*, so a component that cannot name its reference cannot be resolved at all.- **Image hash** — the image's **config digest**, i.e. what `docker inspect` returns as `Id`.Note the order: `ContainerInspect` first, then `ImageInspect` **by the config digest that inspection returned**, not by the tag. A tag can have been moved onto a different image since the container started, and inspecting by tag would describe something other than what is actually running.diff --git a/backend/pkg/controller/prompter.go b/backend/pkg/controller/prompter.goindex d442b306..189d20ef 100644--- a/backend/pkg/controller/prompter.go+++ b/backend/pkg/controller/prompter.go@@ -30,11 +30,9 @@ func newUserPrompter(ctx context.Context, db database.Querier, userID int64) (tereturn buildUserPrompter(defaults, userPrompts), nil}-// buildUserPrompter is the pure merge step extracted from newUserPrompter so-// it can be unit-tested without a database fake or filesystem access. It-// mutates the supplied defaults map by overlaying each non-empty user-// override that still validates, then returns a Prompter backed by that map.-// Callers must pass a fresh map (e.g., from templates.LoadDefaultPromptsMap)+// buildUserPrompter overlays each non-empty user override that still validates+// onto defaults and returns a Prompter backed by that map. It mutates defaults:+// callers must pass a fresh map (e.g., from templates.LoadDefaultPromptsMap)// so the embedded defaults are not modified.func buildUserPrompter(defaults templates.PromptsMap, userPrompts []database.Prompt) templates.Prompter {for _, p := range userPrompts {@@ -47,10 +45,10 @@ func buildUserPrompter(defaults templates.PromptsMap, userPrompts []database.Procontinue}- // Validation runs only on save, so an override written before a variable- // was removed still loads, and text/template renders the missing key as- // "<no value>" without an error — the agent would be pointed at a tool- // that does not exist.+ // Saving validates against the variables declared at that time, so a stored+ // override can name one that was removed since; text/template would render+ // the missing key as "<no value>" without an error and point the agent at a+ // tool that does not exist.promptType := templates.PromptType(p.Type)if err := validator.ValidatePrompt(promptType, p.Prompt); err != nil {logrus.WithError(err).WithField("prompt_type", p.Type).diff --git a/backend/pkg/graph/context.go b/backend/pkg/graph/context.goindex 5a623d40..1c6064e0 100644--- a/backend/pkg/graph/context.go+++ b/backend/pkg/graph/context.go@@ -230,8 +230,8 @@ const (maxFlowTemplateTextLen = 65536)-// The title is one line and is stored trimmed. The text is a document and is stored as it was sent, the line-// break it ends with included; only one that holds nothing but whitespace is refused.+// validateFlowTemplateFields returns the title trimmed, as the one line it is, and the text as it was sent,+// the line break it ends with included: the text is a document. A text of nothing but whitespace is refused.func validateFlowTemplateFields(title, text string) (string, string, error) {title = strings.TrimSpace(title)diff --git a/backend/pkg/providers/bedrock/bedrock_test.go b/backend/pkg/providers/bedrock/bedrock_test.goindex 8e3019c0..954d95bb 100644--- a/backend/pkg/providers/bedrock/bedrock_test.go+++ b/backend/pkg/providers/bedrock/bedrock_test.go@@ -69,17 +69,16 @@ func TestBedrock_DefaultProviderConfig_ReadsAnExternalFileInsteadOfTheEmbeddedOnassert.Equal(t, "zai.glm-4.7-flash", prov.Model(pconfig.OptionsTypeSimple))}-// The installer extracts the example as the file BEDROCK_CONFIG_PATH is mounted from, so-// what a host starts editing is the assignment the binary runs without it.+// The installer extracts the example as the default Bedrock config mount, so it has to equal the built-in config.func TestBedrock_DefaultProviderConfig_IsWhatTheShippedExampleHolds(t *testing.T) {t.Parallel()example, err := os.ReadFile(filepath.Join("..", "..", "..", "..", "examples", "configs", "bedrock.provider.yml"))require.NoError(t, err)- builtIn, err := configFS.ReadFile("config.yml")+ providerConfig, err := DefaultProviderConfig(&config.Config{})require.NoError(t, err)- assert.Equal(t, string(builtIn), string(example),+ assert.Equal(t, string(providerConfig.GetRawConfig()), string(example),"copy pkg/providers/bedrock/config.yml over examples/configs/bedrock.provider.yml")}diff --git a/backend/pkg/providers/provider/failover_test.go b/backend/pkg/providers/provider/failover_test.goindex 5591695f..2ee1ffb2 100644--- a/backend/pkg/providers/provider/failover_test.go+++ b/backend/pkg/providers/provider/failover_test.go@@ -67,8 +67,16 @@ func (f *fakeProvider) CallWithExtraOptions(return f.chainAnswer(chain)}-func (f *fakeProvider) GetToolCallIDTemplate(ctx context.Context, _ templates.Prompter) (string, error) {- return f.Call(ctx, pconfig.OptionsTypeSimple, "hi")+// failoverPrompter marks the prompter a path hands in.+type failoverPrompter struct{ templates.Prompter }++func (f *fakeProvider) GetToolCallIDTemplate(ctx context.Context, prompter templates.Prompter) (string, error) {+ input := "no prompter"+ if _, isMarked := prompter.(failoverPrompter); isMarked {+ input = "hi"+ }++ return f.Call(ctx, pconfig.OptionsTypeSimple, input)}func failoverContent(resp *llms.ContentResponse, err error) (string, error) {@@ -99,7 +107,7 @@ var failoverPaths = []struct {[]llms.MessageContent{llms.TextParts(llms.ChatMessageTypeHuman, "hi")}, nil, nil))}},{"the tool call id template", func(ctx context.Context, prv Provider) (string, error) {- return prv.GetToolCallIDTemplate(ctx, nil)+ return prv.GetToolCallIDTemplate(ctx, failoverPrompter{})}},}diff --git a/backend/pkg/providers/providers_test.go b/backend/pkg/providers/providers_test.goindex a56872c9..2444355f 100644--- a/backend/pkg/providers/providers_test.go+++ b/backend/pkg/providers/providers_test.go@@ -535,7 +535,7 @@ func TestProviders_StoresEveryConfigTheProjectShips(t *testing.T) {switch {case strings.HasPrefix(name, "ollama-"):ptype, env = provider.ProviderOllama, &config.Config{OllamaServerConfig: path}- case strings.HasPrefix(name, "bedrock-"):+ case strings.HasPrefix(name, "bedrock"):ptype, env = provider.ProviderBedrock, &config.Config{BedrockConfig: path}}shipped = append(shipped, shippedConfig{"the example " + name, ptype, env})diff --git a/backend/pkg/server/services/users_test.go b/backend/pkg/server/services/users_test.goindex dbc357c5..9e47ce10 100644--- a/backend/pkg/server/services/users_test.go+++ b/backend/pkg/server/services/users_test.go@@ -476,6 +476,7 @@ func TestUsers_ChangePassword_ThroughTheEditorTakesUsersEdit(t *testing.T) {rec := h.changeThroughTheEditor(t)assert.Equal(t, http.StatusForbidden, rec.Code, rec.Body.String())+ assert.Contains(t, rec.Body.String(), "NotPermitted", "the permission gate refuses, not the session check")afterGeneration, afterPassword := h.state(t)assert.Equal(t, beforeGeneration, afterGeneration)assert.Equal(t, beforePassword, afterPassword)diff --git a/frontend/docs/live-testing.md b/frontend/docs/live-testing.mdindex a5779cef..0622931e 100644--- a/frontend/docs/live-testing.md+++ b/frontend/docs/live-testing.md@@ -579,27 +579,33 @@ branch that cannot be reached.**What was not touched is saved as it was written.** A body that was loaded keeps itsown layout through a save: the blank lines between its blocks and how many there were,the spaces on a blank line, the bullet character, the numbers and the leading spaces of-its lists, every row of a table as the line it was (its padding, its pipes — bare or escaped-inside a code span — and a line of text written right under the table), a code block as it-stood (its fence, its indentation, a block written by indentation and not by a fence),-the fence of a code span, a bare URL, the link around an image, the underline of a setext-heading, the spaces a heading or a list ended with, a line break that is not a hard break.-A case that types one character and compares the stored body with the original should find-that character and nothing else outside the block it was typed into; inside that block-the editor writes its own form — a table row with one space around each cell where the table-was written that way and padded to its column otherwise, `\|` for a pipe in a code span of that-row, a code block between fences or four spaces in. What is kept gives way wherever it would be-read as something else: under a list, a table or a fenced block written a few spaces in goes-to the margin, text goes there when the list would take it for its last item's, and inside a list-item a code block is written between fences at the item's column. The body is stored as it was sent,-the line break it ends with included: neither the forms nor the backend trim it, and only-the single-line fields (title, question, description, code language) are trimmed. A Go template-action `{{ … }}` is literal text to the editor — nothing inside it is read as markdown and a pipe-in it is never escaped — and a line that opens with a control action (`{{ range }}`, `{{ end }}`)-right under a table is the template's line, not a row. What still differs is not a defect of such a case:-a link typed into the editor, as opposed to one that was loaded, is stored as `[url](url)`;-a named HTML entity outside code and outside an action is decoded (`&` becomes `&`);-a list item that opens with indented code is written with a fence; and a document with CRLF line endings is saved with LF.+its lists, every row of a table as the line it was (its padding, its pipes — bare or+escaped inside a code span — and a line of text written right under the table),+a code block as it stood (its fence, its indentation, a block written by indentation and+not by a fence), the fence of a code span, a bare URL, the link around an image,+the underline of a setext heading, the spaces a heading or a list ended with, a line break+that is not a hard break. A case that types one character and compares the stored body+with the original should find that character and nothing else outside the block+it was typed into; inside that block the editor writes its own form — a table row+with one space around each cell where the table was written that way and padded+to its column otherwise, `\|` for a pipe in a code span of that row, a code block between+fences or four spaces in. What is kept gives way wherever it would be read as something else:+under a list, a table or a fenced block written a few spaces in goes to the margin,+text goes there when the list would take it for its last item's, and+inside a list item a code block is written between fences at the item's column.+The body is stored as it was sent, the line break it ends with included:+neither the forms nor the backend trim it, and only the single-line fields+(title, question, description, code language) are trimmed.++A Go template action `{{ … }}` is literal text to the editor — nothing inside+it is read as markdown and a pipe in it is never escaped — and a line that+opens with a control action (`{{ range }}`, `{{ end }}`) right under a table is+the template's line, not a row. What still differs is not a defect of such a case:+a link typed into the editor, as opposed to one that was loaded,+is stored as `[url](url)`; a named HTML entity outside code and+outside an action is decoded (`&` becomes `&`);+a list item that opens with indented code is written with a fence;+and a document with CRLF line endings is saved with LF.A template's body, a knowledge document's content and both halves of a prompt arethe same component. The document tells you to "edit" those entities; this is whatdiff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-code-fence.test.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-code-fence.test.tsindex 9285200b..310b6f70 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-code-fence.test.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-code-fence.test.ts@@ -124,8 +124,7 @@ describe('code-block highlighting', () => {});});-// Guards the two paired seams of the code-span fix: serializeCodeSpan (markdown-editor-marked.ts) and the-// code-mark renderMarkdown override that zeroes the placeholder fence. A regression in either turns these red.+// Pins serializeCodeSpan (markdown-editor-marked.ts) and the code-mark renderMarkdown override that zeroes the fence.describe('inline code containing a backtick round-trips exactly', () => {it('preserves and converges a backtick-containing code span', () => {const out = roundTrip('x ``a `b` c`` y');diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-content-integrity.test.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-content-integrity.test.tsindex e0163b39..6854bbe5 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-content-integrity.test.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-content-integrity.test.ts@@ -2,7 +2,7 @@ import { Editor } from '@tiptap/core';import { beforeAll, describe, expect, it } from 'vitest';import { createMarkdownExtensions } from './markdown-editor-extensions';-import { roundTrip, setupEditorJsdom, structuralCounts } from './markdown-editor-test-setup';+import { cellsOf, roundTrip, setupEditorJsdom, structuralCounts } from './markdown-editor-test-setup';beforeAll(setupEditorJsdom);@@ -138,7 +138,7 @@ describe('generative content-integrity — atoms survive load↔serialize across// A raw `|` inside a table cell is the one context the single-atom oracle above cannot generate: marked// splits the row on it before inline-tokenizing, so an unprotected pipe drops the trailing cells. Build// tables whose cells carry pipe-bearing atoms inside code spans / Go actions, plus a sentinel trailing- // cell; assert the sentinel and the atom's words survive and the table converges.+ // cell; assert the cells read back, that the sentinel and the atom's words survive and the table converges.it('pipe-bearing atoms inside table cells keep their row intact and converge', { timeout: 30000 }, () => {const rng = mulberry32(0x7ab1e);const pipeAtoms = ['x | y', '{{.Host | lower}}', 'a || b', 'grep foo | wc -l', 'no-pipe-here'];@@ -165,13 +165,17 @@ describe('generative content-integrity — atoms survive load↔serialize acrossconst out = roundTrip(doc);expect(roundTrip(out), `did not converge (i=${i}):\n${doc}\n-->\n${out}`).toBe(out);+ // The saved text alone cannot show a row that was read as a line of text: it is saved as it was.+ expect(cellsOf(out), `cells changed (i=${i}):\n${doc}\n-->\n${out}`).toEqual([+ 'code',+ 'note',+ ...atoms.flatMap((atom, row) => [atom, sentinels[row]]),+ ]);for (const sentinel of sentinels) {expect(out.includes(sentinel), `cell "${sentinel}" dropped (i=${i}):\n${doc}\n-->\n${out}`).toBe(true);}- // The atom itself must survive as the code span it was written as — the sentinel guards the row's- // cell count, this guards the pipe-bearing content from silent corruption.for (const atom of atoms) {const span = `\`${atom}\``;diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-extensions.test.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-extensions.test.tsindex 72851034..2867ab17 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-extensions.test.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-extensions.test.ts@@ -589,8 +589,7 @@ describe('typing matches load — underscore emphasis literal, a bare URL is a lexpect(md).toContain('[https://evil.example.com/x](https://evil.example.com/x)');});- // Typed at the end of a loaded document, where an action is literal text and holds its own delimiters. A- // mark writes its delimiters back, so the bytes alone would not show one that took in half an action.+ // A mark writes its delimiters back, so the bytes alone would not show one that took in half an action.it.each([['a backtick after a raw string', 'x {{ printf `%s` .A }} y', ' `z`', 'x {{ printf `%s` .A }} y `z`', ['code']],['an asterisk after a comment', '{{/* a */}} b', ' c*', '{{/* a */}} b c*', []],@@ -663,7 +662,7 @@ describe('typing matches load — underscore emphasis literal, a bare URL is a l});});-describe('line-leading # / > in a paragraph stay body text on round-trip (ENCODER-LEADING)', () => {+describe('line-leading # / > in a paragraph stay body text on round-trip', () => {const paraDoc = (...content: unknown[]) => ({ content: [{ content, type: 'paragraph' }], type: 'doc' });const save = (doc: unknown) => {@@ -1005,8 +1004,6 @@ describe('a heading that holds a line break', () => {const hard = (marker?: string) => ({ attrs: { marker }, type: 'hardBreak' });const soft = hard('');- // Setext has two levels only; below them the ATX form is all there is, and it is one line. Setext cannot- // hold a break at either end of the heading, or one above a line that reads as a block of its own.it.each([['between two lines at level 1', 1, [text('a'), hard(), text('b')], 'a \nb\n==='],['between two lines at level 2', 2, [text('a'), hard(), text('b')], 'a \nb\n---'],@@ -1663,9 +1660,6 @@ describe('line-leading block markers are escaped to marked grammar, inline backsexpect(roundTrip('####### seven')).toBe('####### seven');});- // The load side used to unescape `\#`/`\>` at ANY inline position while the serializer only re-escaped them- // line-leading, so a backslash in the middle of a line was eaten. `\<root\>` is a GNU-grep word boundary;- // dropping the second backslash changes what the command matches.it.each(['use \\# for comments and \\> for redirect',"grep '\\<root\\>' /etc/passwd",@@ -1784,7 +1778,6 @@ describe('brackets in a link label or an image alt do not break the node on reloexpect(reload(first.markdown).markdown).toBe(first.markdown);});- // An unbalanced bracket in alt broke `` outright: the reload parsed ZERO image nodes.it('leaves the brackets of a Go action in an image alt to the template', () => {const md = '![{{ index .M "[k]" }} of [x]](http://x.test/i.png)';const saved = roundTrip(md);@@ -1803,10 +1796,6 @@ describe('brackets in a link label or an image alt do not break the node on relo});describe('a URL inserted as its own link label settles instead of growing', () => {- // The Link popover inserts the URL as the visible text when there is no selection. A `]` in that text- // closed the markdown label early — `[https://example.com/a]b](…)` — and the serialized form grew on- // every load+save cycle (measured 50 → 103 → 156 → 260). The form keeps brackets out of the label; this- // pins the property that matters, that the document converges.it.each(['https://example.com/a]b', 'http://[::1]:8080/x', 'https://example.com/plain'])('inserting %s converges',(url) => {@@ -1844,8 +1833,7 @@ describe('a heading applied over the whole document', () => {return out;};- // toggleHeading had no whole-document wrapper, so Ctrl+A + "Heading 2" converted every table cell paragraph- // to a heading; saved as `| ## h | ## i |`, it reloads as literal cell text and escalates on each save.+ // The selection covers the table too, and a GFM table cell cannot hold a heading.it('leaves table cells alone', () => {const source = ['lead', '', '| h | i |', '| --- | --- |', '| x | y |'].join('\n');const out = apply(source, 2);diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-extensions.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-extensions.tsindex 2b4227fb..17009382 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-extensions.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-extensions.ts@@ -974,7 +974,7 @@ const TunedStarterKit = StarterKit.extend({// ({{vars}} / <tags>) that don't affect serialization.// • underline: false — its `++text++` markdown corrupts `C++ … C++` prose on load and Ctrl+U emits `++`.// • link autolink/linkOnPaste: true — a bare URL/email becomes a link on load, paste, AND typing, kept-// symmetric with the marked layer (which no longer neutralises autolink/url). Do NOT set false: it+// symmetric with the marked layer (which reads a web address or an e-mail as a link). Do NOT set false: it// diverges typing from load and re-freezes bare URLs as text.// • link openOnClick: false — a click seats the caret in the link instead of navigating away, so LinkHandle// (markdown-editor-link-handle.tsx) can show the edit popover; opening still works via that popover's button.diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-heading-autoformat.test.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-heading-autoformat.test.tsindex a2977249..8a7d7acd 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-heading-autoformat.test.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-heading-autoformat.test.ts@@ -183,8 +183,7 @@ describe('multi-line blocks (hardBreak / Shift+Enter) stay body text', () => {expect(heading).toBe(0);});- // A heading is single-line; promoting `# a`⏎`# b` would emit `# a \n# b`, which re-parses as TWO headings on- // reload. A block that contains a hardBreak stays a paragraph and round-trips as escaped body text instead.+ // A block that holds a line break stays a paragraph and round-trips as escaped body text.it('a multi-line block whose FIRST line starts with "# " is NOT promoted (would be a lossy heading)', () => {const { heading, md, reloadHeading } = promote(doc(para(t('z# A'), hb, t('# B'))), 'z# A');diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-heading-autoformat.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-heading-autoformat.tsindex 2d010460..c68b89d2 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-heading-autoformat.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-heading-autoformat.ts@@ -27,10 +27,10 @@ const containsHardBreak = (node: PMNode, hardBreak: null | PMNode['type']): bool// Two guards keep the promotion sound:// • it keys off the FIRST child, not textContent: a `# ` AFTER a hardBreak (Shift+Enter) must stay body text,// and there the first child is the pre-break text, so the block does not read as starting with `#`;-// • it skips a paragraph that CONTAINS a line break, hard or soft: a heading is single-line, so promoting a-// multi-line block (e.g. `# a`⏎`# b`) would emit `# a \n# b`, which re-parses as TWO headings on the next-// load. Such a block stays a paragraph; escapeLineLeadingBlockMarkers then escapes its leading `# ` so it-// round-trips.+// • it skips a paragraph that CONTAINS a line break, hard or soft: an ATX heading is one line, so a promoted+// multi-line block would be saved as a setext heading or, where setext cannot hold it, with the break as a+// space (`# a`⏎`# b` as `# a # b`). Such a block stays a paragraph; escapeLineLeadingBlockMarkers then+// escapes its leading `# ` so it round-trips.export const HeadingAutoformat = Extension.create({addProseMirrorPlugins() {return [diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-marked.test.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-marked.test.tsindex aba4e496..e76a1332 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-marked.test.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-marked.test.ts@@ -202,8 +202,6 @@ describe('a Go template action is literal text', () => {});describe('a pipe inside a code span of a table cell', () => {- // The loader reads a pipe inside a code span as content whether it is escaped or not, so a row is saved- // as the line it was; one that has been edited, or was made in the editor, is written the way GFM asks.it.each([['written escaped', '| a | b |\n| --- | --- |\n| `x \\| y` | z |'],['written bare', '| a | b |\n| --- | --- |\n| `x | y` | z |'],@@ -299,8 +297,6 @@ describe('a pipe inside a code span of a table cell', () => {});describe('a line under a table that is not a row of it', () => {- // marked takes any line under a table for a row, pads it into cells and drops what is past the header's- // count. The table ends above such a line and the line is the text it was written as.it.each([["a template's range around the rows",diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-marked.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-marked.tsindex 4b7c55a0..d272d549 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-marked.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-marked.ts@@ -71,7 +71,7 @@ const createTunedMarked = () => {// • escape — keep `\`+punct literal (`\.` `\*` `\|` `\\`); marked's default DROPS the backslash// (CommonMark unescape), silently corrupting regex/paths on the first load. The// `\#`/`\>` counterpart to the paragraph serializer is NOT here: an inline tokenizer- // fires at any position, so it also ate the backslash mid-line (`grep '\<root\>'`).+ // fires at any position, so it would also eat the backslash mid-line (`grep '\<root\>'`).// It lives in the Lexer's inlineTokens override below, which sees line starts.// • html/tag — keep `<xml-like>` tags literal (marked swallows real-HTML-element names)// • def — keep `[label]: target` lines as the text they are. marked lifts a definition out@@ -83,8 +83,9 @@ const createTunedMarked = () => {// shown and saved without its brackets. Only a web address or an e-mail is a// link; anything else is the text it was written as, brackets and all — taken// whole, or the bare-URL rule would find a link inside `<git+https://…>`.- // NB: autolink/url are intentionally NOT neutralised — a bare `https://…`, `<url>` or email is- // meant to become a link (see markdown-editor-extensions.ts link config, kept symmetric with typing).+ // NB: beyond the `autolink` case above and the `url` override below, autolink/url are intentionally+ // NOT neutralised — a bare `https://…`, `<url>` or email is meant to become a link (see+ // markdown-editor-extensions.ts link config, kept symmetric with typing).// NB: named HTML entities (`< > & "`) are decoded downstream — @tiptap/markdown's// token parsing runs @tiptap/core's decodeHtmlEntities, so a bare-prose `<` becomes `<` (fixes// HTML-encoding artifacts from ingestion). Numeric refs (`{`) and anything inside code are@@ -230,7 +231,7 @@ const createTunedMarked = () => {}// Undo the paragraph serializer's line-leading `\#`/`\>` here rather than in an inline tokenizer or in- // `lex`. An inline tokenizer has no notion of position and ate the backslash mid-line; `lex` runs+ // `lex`. An inline tokenizer has no notion of position and would eat the backslash mid-line; `lex` runs// BEFORE block tokenization, so unescaping there hands marked a live `#`/`>` and the paragraph the// escape exists to protect becomes a heading or a quote again.// A table cell is left alone: it cannot open a heading or a quote, cellText never escapes one, and a@@ -820,9 +821,9 @@ export const renderInline = (nodes: JSONContent[], helpers: MarkdownRendererHelp// cells on the next load. Pad an odd run by one backslash — the cell gains a `\`, the table keeps its cells,// and the padded form is byte-stable from the first save.// A pipe inside a Go action is the template's pipeline operator. Escaping it makes text/template reject the-// whole prompt (`unexpected "\" in operand`), so the file cannot be saved at all — and every save re-added the+// whole prompt (`unexpected "\" in operand`), so the file cannot be saved at all — and every save would re-add the// backslash, leaving no way out from rich mode. The loader counts cells with action pipes masked, so leaving-// them raw no longer skews the header/delimiter comparison.+// them raw does not skew the header/delimiter comparison.// A pipe inside a code span is escaped as well, as GFM asks. Whichever opens first is what marked reads// there: an action that holds a raw string is an action, a code span that holds an action is code, and the// action inside it still keeps its pipes. This is how a row is written once it has been edited; one that@@ -891,8 +892,8 @@ export const escapeCellPipes = (text: string): string => {};// GFM has no headerless table, and renderTableToMarkdown answers that by emitting an EMPTY header row above-// the demoted rows — so every header-off + save + reload cycle grew the table by one blank row and the header-// switch silently flipped back on. Promote the first row instead: the row count and every cell survive, and a+// the demoted rows — so every header-off + save + reload cycle would grow the table by one blank row and the header+// switch would silently flip back on. Promote the first row instead: the row count and every cell survive, and a// second save is a no-op because the promoted table already has a header.const withPromotedHeaderRow = (node: JSONContent): JSONContent => {const rows = node.content ?? [];diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-paste.test.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-paste.test.tsindex b519ed1a..25aaf001 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-paste.test.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-paste.test.ts@@ -124,7 +124,6 @@ describe('MarkdownPaste — the parsed payload matches load (same tuned markdownexpect(out).toBe(saved);});- // A rich clipboard is left to ProseMirror, whose link rule ends a URL inside the action it runs into.// jsdom has neither ClipboardEvent nor DataTransfer, which the paste rules build for their handlers.it.each([['a URL that runs into an action', 'https://e.x/{{.P}}'],diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-table-pipes.test.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-table-pipes.test.tsindex 684d5522..c05a0677 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-table-pipes.test.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-table-pipes.test.ts@@ -5,7 +5,7 @@ import { GROWTH_IF_QUADRATIC, slowdownWhenInputQuadruples } from '@/test-utils/cimport { createMarkdownExtensions } from './markdown-editor-extensions';import { escapeTablePipes } from './markdown-editor-table-pipes';-import { roundTrip, setupEditorJsdom, structuralCounts } from './markdown-editor-test-setup';+import { cellsOf, roundTrip, setupEditorJsdom, structuralCounts } from './markdown-editor-test-setup';setupEditorJsdom();@@ -69,20 +69,11 @@ describe('escapeTablePipes — pure pre-lex pipe protection', () => {});});-describe('table cell with a piped code span — content survives load and converges (H1)', () => {- it('keeps the trailing cell and preserves the code content', () => {- const src = '| Op | Meaning |\n| --- | --- |\n| `x | y` | z |';-- expect(roundTrip(src)).toBe(src);- });-- // This used to pin `{{.X \| upper}}` — the escaped form, which Go text/template rejects outright- // (`unexpected "\" in operand`), so the prompt could not be saved at all. The pipe is the template's- // pipeline operator and stays raw; the cell count survives because the loader masks action pipes.+describe('table cell with a piped code span — content survives load and converges', () => {it('keeps a Go-template action with a pipe inside a cell', () => {const out = roundTrip('| Var | Out |\n| --- | --- |\n| {{.X | upper}} | done |');- expect(out).toContain('done');+ expect(cellsOf(out)).toEqual(['Var', 'Out', '{{.X | upper}}', 'done']);expect(out).toContain('{{.X | upper}}');expect(out).not.toContain('\\|');expect(roundTrip(out)).toBe(out);@@ -91,7 +82,7 @@ describe('table cell with a piped code span — content survives load and converit('keeps a code span with a backtick and a pipe inside a cell', () => {const out = roundTrip('| Op | Meaning |\n| --- | --- |\n| `` `x` | y `` | kept |');- expect(out).toContain('kept');+ expect(cellsOf(out)).toEqual(['Op', 'Meaning', '`x` | y', 'kept']);expect(out).toContain('`` `x` | y ``');expect(roundTrip(out)).toBe(out);});@@ -107,7 +98,7 @@ describe('pipe-less GFM tables (no outer pipe) — cells survive too', () => {it('keeps the trailing cell of a no-leading-pipe table on round-trip', () => {const out = roundTrip('A | B | C\n--- | --- | ---\n`git log | head` | notes | done');- expect(out).toContain('done');+ expect(cellsOf(out)).toEqual(['A', 'B', 'C', 'git log | head', 'notes', 'done']);expect(out).toContain('`git log | head`');expect(roundTrip(out)).toBe(out);});@@ -115,15 +106,14 @@ describe('pipe-less GFM tables (no outer pipe) — cells survive too', () => {it('protects a template action in a no-leading-pipe body row', () => {const out = roundTrip('Var | Out\n--- | ---\n{{.Host | lower}} | done');- expect(out).toContain('done');+ expect(cellsOf(out)).toEqual(['Var', 'Out', '{{.Host | lower}}', 'done']);expect(out).toContain('{{.Host | lower}}');});it('protects rows whether or not each has a leading pipe (mixed)', () => {const out = roundTrip('| A | B |\n| --- | --- |\n| `p | q` | one |\n`r | s` | two');- expect(out).toContain('one');- expect(out).toContain('two');+ expect(cellsOf(out)).toEqual(['A', 'B', 'p | q', 'one', 'r | s', 'two']);});it('stops at a block boundary — a heading after the table is not escaped', () => {@@ -199,6 +189,7 @@ describe('tables inside a blockquote — prefix-stripped and protected', () => {const src = '> | a | b |\n> | --- | --- |\n> | `x | y` | z |';expect(roundTrip(src)).toBe(src);+ expect(cellsOf(src)).toEqual(['a', 'b', 'x | y', 'z']);});it('leaves blockquote prose (no table) untouched', () => {@@ -225,6 +216,7 @@ describe('fence length tracking — a longer fence is not closed by a shorter inconst src = '````\n```\ninner\n````\n\n| a | b |\n| --- | --- |\n| `x | y` | z |';expect(roundTrip(src)).toBe(src);+ expect(cellsOf(src)).toEqual(['a', 'b', 'x | y', 'z']);});});@@ -236,9 +228,10 @@ describe('CRLF line endings — tables still protected', () => {});it('keeps the trailing cell of a CRLF table on round-trip', () => {- const out = roundTrip('| a | b |\r\n| --- | --- |\r\n| `x | y` | z |\r\n');+ const src = '| a | b |\r\n| --- | --- |\r\n| `x | y` | z |\r\n';- expect(out).toBe('| a | b |\n| --- | --- |\n| `x | y` | z |\n');+ expect(roundTrip(src)).toBe('| a | b |\n| --- | --- |\n| `x | y` | z |\n');+ expect(cellsOf(src)).toEqual(['a', 'b', 'x | y', 'z']);});it('leaves CRLF bytes untouched when there is no table to escape', () => {@@ -260,26 +253,6 @@ describe('TABLE_DELIMITER_LINE is linear (ReDoS guard)', () => {});describe('a backtick in a backtick fence info string is not a fence opener', () => {- const cellsOf = (markdown: string) => {- const editor = new Editor({- content: markdown,- contentType: 'markdown',- extensions: createMarkdownExtensions(),- });- const cells: string[] = [];-- editor.state.doc.descendants((node) => {- if (node.type.name === 'tableCell' || node.type.name === 'tableHeader') {- cells.push(node.textContent);- }-- return true;- });- editor.destroy();-- return cells;- };-const TABLE = ['| Op | Meaning |', '| --- | --- |', '| `x | y` | KEEP |'].join('\n');// marked's fence rule is /^ {0,3}(`{3,}(?=[^`\n]*(?:\n|$))|~{3,})…/ — the no-backtick lookahead applies to@@ -298,23 +271,11 @@ describe('a backtick in a backtick fence info string is not a fence opener', ()expect(cellsOf(TABLE)).toEqual(['Op', 'Meaning', 'x | y', 'KEEP']);});- // The opposite direction of the same desynchronisation: once the phantom fence closes, the scanner's parity- // is inverted and it escapes pipes INSIDE a real code block, injecting a backslash into code content.+ // The opposite direction: once the phantom fence closes, the scan takes the lines of a real code block for a table.it('leaves a real code block byte-identical, injecting no escape into its content', () => {const source = ['```a`b', '```', '', TABLE].join('\n');- const editor = new Editor({ content: source, contentType: 'markdown', extensions: createMarkdownExtensions() });- const codeBlocks: string[] = [];-- editor.state.doc.descendants((node) => {- if (node.type.name === 'codeBlock') {- codeBlocks.push(node.textContent);- }- return true;- });- editor.destroy();-- expect(codeBlocks.join('')).not.toContain('\\|');+ expect(escapeTablePipes(source)).toBe(source);});it('still treats a genuine fence as a fence', () => {@@ -325,26 +286,6 @@ describe('a backtick in a backtick fence info string is not a fence opener', ()});describe('tables nested inside list items keep their pipe protection', () => {- const cellsOf = (markdown: string) => {- const editor = new Editor({- content: markdown,- contentType: 'markdown',- extensions: createMarkdownExtensions(),- });- const cells: string[] = [];-- editor.state.doc.descendants((node) => {- if (node.type.name === 'tableCell' || node.type.name === 'tableHeader') {- cells.push(node.textContent);- }-- return true;- });- editor.destroy();-- return cells;- };-const table = (indent: string) =>[`${indent}| Op | Meaning |`, `${indent}| --- | --- |`, `${indent}| \`x | y\` | KEEP |`].join('\n');@@ -391,26 +332,6 @@ describe('tables nested inside list items keep their pipe protection', () => {});describe('a Go template pipeline in a table cell keeps its own pipe', () => {- const cellsOf = (markdown: string) => {- const editor = new Editor({- content: markdown,- contentType: 'markdown',- extensions: createMarkdownExtensions(),- });- const cells: string[] = [];-- editor.state.doc.descendants((node) => {- if (node.type.name === 'tableCell' || node.type.name === 'tableHeader') {- cells.push(node.textContent);- }-- return true;- });- editor.destroy();-- return cells;- };-// `{{.Host | urlquery}}` is a Go text/template pipeline. Escaping its pipe makes text/template reject the// whole file with `unexpected "\" in operand`, so the prompt cannot be saved at all.it('does not escape a pipeline in a body cell on save', () => {@@ -469,8 +390,6 @@ describe('the scan takes for a row only what marked takes for one', () => {expect(escapeTablePipes(md)).toBe(escaped);});- // The scan may take a line for a row that marked reads as something else. The stand-in it leaves there is- // a pipe again in whatever the line turned out to be.it.each([['a heading over a row of dashes',@@ -565,9 +484,6 @@ describe('a table with no header row keeps its row count across a save', () => {return { rows, saved };};- // GFM has no headerless table, so the serializer used to emit an EMPTY header row above the demoted rows:- // every header-off + save + reload cycle grew the table by one blank row (2 → 3 → 4) and the switch- // silently flipped back on. Promoting the first row keeps the row count and every cell.it('does not grow when the header row is toggled off', () => {const editor = new Editor({content: ['| a | b |', '| --- | --- |', '| 1 | 2 |'].join('\n'),@@ -606,9 +522,6 @@ describe('adjacent tables do not accumulate blank paragraphs', () => {return kinds.join(',');};- // The table renderer emits a newline of its own on top of the block separator, so between two adjacent- // tables that extra line reloaded as an empty paragraph — which serialised to another blank line on the- // next save. Measured before the fix: 68 → 88 → 90 → 92 → 94 bytes, one paragraph per cycle, no fixed point.it('keeps two tables adjacent across repeated saves', () => {const source = ['| a | b |', '| --- | --- |', '| 1 | 2 |', '', '| c | d |', '| --- | --- |', '| 3 | 4 |'].join('\n',diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-table-pipes.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-table-pipes.tsindex 09b5c316..21a5f53f 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-table-pipes.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-table-pipes.ts@@ -278,8 +278,8 @@ export const escapeTablePipes = (markdown: string, escaped = '\\|'): string => {const run = lines.slice(index, end);// The run is the item's, with or without a table in it: walked line by line here, a fence the- // item opens on its marker line would be closed by this scan where the item closes it, and- // every table below would go unprotected.+ // item opens on its marker line is not seen, the line that closes it opens one in this scan,+ // and every table below would go unprotected.if (end > index + 1) {if (run.some((row) => row.includes('|'))) {const inside = escapeTablePipes(diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-test-setup.ts b/frontend/src/components/shared/markdown-editor/markdown-editor-test-setup.tsindex fd8704f2..7d243b26 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-test-setup.ts+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-test-setup.ts@@ -57,6 +57,21 @@ export const structuralCounts = (markdown: string): Record<string, number> => {return counts;};+// The text of every table cell the editor reads, header cells included, in document order.+export const cellsOf = (markdown: string): string[] => {+ const editor = new Editor({ content: markdown, contentType: 'markdown', extensions: createMarkdownExtensions() });+ const cells: string[] = [];++ editor.state.doc.descendants((node) => {+ if (node.type.name === 'tableCell' || node.type.name === 'tableHeader') {+ cells.push(node.textContent);+ }+ });+ editor.destroy();++ return cells;+};+// The two halves of a round trip, apart, for a test that measures how the cost of one grows.export const markdownCodec = (): {destroy: () => void;diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor-view-mode.tsx b/frontend/src/components/shared/markdown-editor/markdown-editor-view-mode.tsxindex c9866929..4faebf00 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor-view-mode.tsx+++ b/frontend/src/components/shared/markdown-editor/markdown-editor-view-mode.tsx@@ -2,7 +2,7 @@ import { SquareMenu, Type } from 'lucide-react';import { Tabs, TabsList, TabsTrigger } from '@/components/ui/tabs';-// 'rich' reflows whitespace on save (tiptap); 'raw' is a byte-exact textarea over the source.+// 'rich' (tiptap) saves what it parsed, which can differ from the source; 'raw' is a byte-exact textarea over it.export type EditorViewMode = 'raw' | 'rich';interface EditorViewModeToggleProps {diff --git a/frontend/src/components/shared/markdown-editor/markdown-editor.tsx b/frontend/src/components/shared/markdown-editor/markdown-editor.tsxindex 2ff3aba6..6f76876e 100644--- a/frontend/src/components/shared/markdown-editor/markdown-editor.tsx+++ b/frontend/src/components/shared/markdown-editor/markdown-editor.tsx@@ -169,8 +169,8 @@ function useMarkdownEditor({placeholder = 'Write something…',value,}: UseMarkdownEditorOptions): Editor | null {- // Suppress echoes of our own output: the markdown round-trip re-serializes slightly (whitespace/list- // markers/blank lines), and those normalizations must not flip RHF's isDirty as if the user had edited.+ // Suppress echoes of our own output: the markdown round-trip can re-serialize a document (a decoded HTML+ // entity, CRLF line endings), and that must not flip RHF's isDirty as if the user had edited.const lastEmittedRef = useRef<string>(value);// Ignore the onUpdate echoes tiptap fires for the initial parse during view construction — forwarding@@ -279,9 +279,10 @@ function useMarkdownEditor({}// On mount the construction-time parse already reflects `initialContent`, so compare against THAT, not- // getMarkdown(): serialization normalizes most real documents, so `getMarkdown() !== value` on mount- // would re-parse the identical doc — doubling mount cost for exactly the large documents where parse- // dominates. After mount, getMarkdown() is the right divergence check for external form.reset syncs.+ // getMarkdown(): serialization can differ from the source (a decoded HTML entity, CRLF line endings), so+ // `getMarkdown() !== value` on mount would re-parse the identical doc — doubling mount cost for exactly+ // the large documents where parse dominates. After mount, getMarkdown() is the right divergence check+ // for external form.reset syncs.const isExternalChange = hasResetInitialHistoryRef.current? editor.getMarkdown() !== value: value !== initialContent;diff --git a/frontend/src/features/knowledges/knowledge-form.test.tsx b/frontend/src/features/knowledges/knowledge-form.test.tsxindex 1db72c84..15f4e666 100644--- a/frontend/src/features/knowledges/knowledge-form.test.tsx+++ b/frontend/src/features/knowledges/knowledge-form.test.tsx@@ -173,9 +173,7 @@ describe('KnowledgeForm — update', () => {it('resets untouched fields to the server document returned by onSubmit', async () => {const user = userEvent.setup();- // The server's document carries another `content` (a field the user never edits here). The form's- // resetOptions keep DIRTY fields at their typed value, so asserting the reset- // applied the server document means asserting on an untouched field.+ // The server's document carries another `content`, a field this test never edits.const serverDocument = {answerType: KnowledgeAnswerType.Other,codeLang: null,