Modern Docker Management, Designed for Everyone
Go composecontainer-managementcontainersdockerdocker-composedocker-managementgoself-hostedsveltekittypescriptweb-ui
refactor: reorganize services into domain packages (#3546)
Kyle Mendell1cea5f48575f
diff --git a/.gitignore b/.gitignoreindex c98753455..ed2921528 100644--- a/.gitignore+++ b/.gitignore@@ -8,7 +8,7 @@ node_modules.svelte-kitfrontend/.svelte-kitfrontend/build-build+/builddata/dist//arcanediff --git a/AGENTS.md b/AGENTS.mdindex 13c27a623..d9502dd68 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -1,213 +1,150 @@-# Arcane AI Agent Instructions--> **All AI agents must conform to [AI_POLICY.md](./AI_POLICY.md)**--Arcane is a Docker management UI: **Go backend** (Echo + Huma v2), **SvelteKit v3 frontend** (Svelte 5), optional headless agent, Cobra CLI. Three Go modules via `go.work`: `backend/`, `cli/`, `types/`. Domain docs: `CONTEXT.md`.--## Development Environment--```bash-./scripts/development/dev.sh start|stop|restart|rebuild|clean|logs-# Frontend: http://localhost:3000 (Vite HMR) | Backend: http://localhost:3552 (Air)+# Arcane Agent Guide++All AI-assisted work must follow [AI_POLICY.md](./AI_POLICY.md). Keep changes focused,+verify them locally, and disclose AI assistance when contributing.++Arcane is a Docker management platform with a Go backend, SvelteKit frontend,+headless agent modes, and a Cobra CLI.++## Non-negotiable rules++- Update existing code in place. Search before adding functions, services, wrappers,+ API clients, components, or utilities.+- Never add stubs, shims, pass-through helpers, or duplicate implementations.+- Call existing `pkg/` helpers directly instead of wrapping them.+- Add new code only for genuinely new behavior and integrate it with the owning domain.+- Keep comments short. If code needs a paragraph to explain its structure, simplify it.+- Never run state-changing Git commands. Do not stage, commit, push, tag, stash,+ create branches, or create worktrees.+- Always use the `golang-master` skill when writing Go.+- Name every unexported Go function with an `Internal` suffix.+- Put public/shared Go types in the top-level `types/` module.+- Put reusable helper utilities under `backend/pkg/utils/` in the appropriate package.+- After any change, run `just format all`, then `just lint all`, and fix every issue.+ Never revert formatter output.++## Repository architecture++The Go workspace contains three modules:++```text+backend/ Go application, HTTP API, domain logic, jobs, and embedded frontend+cli/ Cobra CLI and its API client+types/ Public domain and API contracts shared by backend and CLI```-## ⚠️ Golden Rules — READ FIRST--1. **ALWAYS update existing code in place.** Never create a new function, service, helper, or API wrapper when one already exists that does the same thing. Find the existing code and modify it.-2. **NEVER create stub, shim, or pass-through helper functions.** If a `pkg/` helper or service method exists, call it directly. Do not write a thin wrapper that just forwards to it. This includes refactors: never hollow out an existing exported function into a one-line forwarder to a new `*Internal` variant with a mode flag — keep the real body in the existing function and let the other callsite handle its own difference in behavior.-3. **NEVER duplicate functionality.** Before writing anything, search `pkg/`, `internal/services/`, and `frontend/src/lib/services/` for existing implementations.-4. **NEVER create a new API standard.** If the codebase uses Huma v2 typed handlers on Echo, do not introduce Gin, raw `http.HandlerFunc`, or untyped patterns. Extend the existing standard.-5. If you need a paragraph comment in order to justify a function, it is wrong. Fix the code.--## Architecture Overview--### Backend (`backend/`)--```-cmd/ # entrypoint-api/ # HTTP API surface-├── api.go # Huma v2 on Echo via humaecho — register handlers here-├── handlers/ # Thin Huma handlers → call services-├── middleware/ # Huma auth bridge-└── ws/ # Echo WebSocket handlers-frontend/ # embedded SvelteKit build-internal/-├── bootstrap/ # DI wiring, router setup — START HERE-├── config/ # env config-├── database/ # GORM + migrations-├── middleware/ # Echo middleware (auth, CORS, rate limit)-├── models/ # GORM models (embed BaseModel)-└── services/ # Business logic — *_service.go-pkg/-├── authz/ # Permissions + access policy-├── dockerutil/ # Docker helpers (names, labels, logs, mounts)-├── libarcane/ # Core libraries (compose, edge, image update, build, swarm)-├── pagination/ # Search/filter/sort/pagination-├── projects/ # Compose parsing, image refs, discovery-├── scheduler/ # Cron jobs-└── utils/ # Shared helpers (strings, cache, ptr, httpx)-resources/ # migrations, images, email templates+The frontend lives in `frontend/`. End-to-end tests live in `tests/`.++### Backend++The backend uses domain-oriented vertical slices:++```text+backend/+├── cmd/ process entrypoint+├── api/ API assembly and exceptional HTTP/stream/WebSocket routes+├── internal/+│ ├── <domain>/ domain module, handler, service, and related files+│ ├── bootstrap/ application lifecycle, router, jobs, and startup wiring+│ ├── di/ Fx dependency graph and providers+│ ├── config/ environment configuration+│ ├── database/ database setup and migrations+│ ├── middleware/ authentication, authorization, and environment proxying+│ └── models/ private GORM persistence models+├── pkg/ reusable infrastructure and domain-independent libraries+├── resources/ migrations, email templates, and runtime assets+└── frontend/ embedded frontend build```-**Key patterns:**--- Echo is the HTTP router. Huma v2 is the typed REST/OpenAPI layer mounted via `humaecho.NewWithGroup`.-- Handlers are thin: extract typed input → call service → return typed response.-- Services use constructor injection (see `bootstrap.go`).-- Direct Echo routes only for WebSockets, streaming, diagnostics, webhooks, Playwright routes, and embedded frontend.-- Use `slog` for logging, `fmt.Errorf("context: %w", err)` for errors.--**Reuse `pkg/` helpers — call directly, never wrap:**--- Docker → `pkg/dockerutil`: `ContainerNameFromNames`, `ComposeProjectLabel`/`ComposeServiceLabel`, `StreamContainerLogs`/`ReadAllLogs`-- Compose → `pkg/projects`: `ImageRefsFromComposeServices`/`ImageRefsFromRuntimeServices`-- Pagination → `pkg/pagination`: `SearchOrderAndPaginate`, `PaginateAndSortDB`-- Shared → `pkg/utils`--### Frontend (`frontend/src/`)--SvelteKit v3 — config lives in `vite.config.ts` (no `svelte.config.js`). Routes use `+page.svelte`/`+page.ts`/`+layout.svelte`/`+layout.ts` naming.--```-routes/(app)/ # App pages (dashboard, containers, images, etc.)-routes/(auth)/ # Auth pages-lib/components/ # Reusable components (shadcn-svelte)-lib/services/ # API services extending BaseAPIService-lib/stores/ # Svelte stores (*.store.svelte using runes)-lib/types/ # TypeScript types-../messages/en.json # i18n source strings (Paraglide)-```--### Shared Types (`types/`)--Domain types shared between backend and CLI. Each domain has its own package.--### CLI (`cli/`)--Cobra app. Commands in `cli/pkg/<domain>/`, helpers in `cli/internal/`, types from `types/`.--## Critical Patterns--### Svelte 5 Runes ONLY--```svelte-<script lang="ts">- let { name }: { name: string } = $props();- let count = $state(0);- let doubled = $derived(count * 2);- $effect(() => { /* ... */ });-</script>-<button onclick={handleClick}>Click</button>+Most domains under `backend/internal/<domain>/` follow this shape:++- `module.go` constructs the domain and exposes `Service()` or `Handler()` only when+ another domain needs them.+- `handler.go` owns Huma input/output types, route registration, and thin handlers.+- `service.go` and focused sibling files own business logic.+- Tests stay beside the code they cover.++Wire dependencies in `internal/di`; keep startup order and lifecycle hooks in+`internal/bootstrap`. Do not turn `api/` back into a global handlers directory or+recreate a global `internal/services` layer.++Use Echo v5 as the router and Huma v2 for typed REST/OpenAPI operations. Register+permissioned endpoints with `middleware.RegisterWithPermission`. Direct Echo routes+are reserved for WebSockets, streams, diagnostics, webhooks, Playwright support,+the environment proxy, and embedded frontend delivery.++Handlers translate typed HTTP data and call services. They do not contain business+logic. Services receive dependencies through constructors/Fx. Use `slog` for+structured logging and the existing `emperror.dev/errors` and `internal/common`+patterns for wrapped or semantic errors.++Before adding backend logic, search the owning domain plus:++- `backend/pkg/dockerutil` for Docker names, labels, clients, logs, and stream helpers.+- `backend/pkg/projects` for Compose parsing, discovery, and image references.+- `backend/pkg/pagination` for in-memory and database pagination.+- `backend/pkg/libarcane` for reusable Arcane engines and transport behavior.+- `backend/pkg/utils` for shared infrastructure utilities.++Persistence models embed `models.BaseModel`. Use existing GORM model helpers and+`Preload` relationships where appropriate; do not expose persistence models as API+contracts.++### Frontend++The frontend is SvelteKit v3 on Svelte 5. Configuration lives in+`frontend/vite.config.ts`.++```text+frontend/src/+├── routes/ SvelteKit pages and layouts+└── lib/+ ├── components/ shared UI components+ ├── config/ navigation and access-surface configuration+ ├── services/ API clients extending BaseAPIService+ ├── stores/ rune-based application state+ ├── types/ frontend-only TypeScript types+ └── utils/ frontend utilities```-**NEVER:** `export let`, `on:click`, `$:`, `$$props`, `$$restProps`, old slot syntax.--### API Service Pattern--```typescript-export class ContainerService extends BaseAPIService {- async getContainers(options?: SearchPaginationSortRequest) {- const envId = await environmentStore.getCurrentEnvironmentId();- const params = transformPaginationParams(options);- return this.api.get(`/environments/${envId}/containers`, { params });- }-}-export const containerService = new ContainerService();-```--### Huma Handler Pattern--```go-// 1. Define typed input/output structs-type ListContainersInput struct {- EnvironmentID string `path:"id" doc:"Environment ID"`- Search string `query:"search" doc:"Search query"`- Limit int `query:"limit" default:"20"`-}--// 2. Register in handlers via huma.Register (in registerHandlers in api.go)-// 3. Handler body: call service, return response-```--### GORM Model Pattern--```go-type Stack struct {- models.BaseModel- Name string `json:"name" gorm:"column:name" sortable:"true"`-}-func (Stack) TableName() string { return "stacks" }-```--Use `Preload` for relationships. Use `models.JSON` for JSON fields, `models.StringSlice` for string arrays.--### i18n — No Raw Strings--All user-facing text goes in `frontend/messages/en.json`, accessed via Paraglide messages:--```svelte-<Button>{m.common_save_changes()}</Button>-```--**AI Agent Rule for i18n**:--- Never use hardcoded English strings in frontend UI components (`.svelte` files). This includes text inside HTML tags and text assigned to variables, props, or state that is rendered in the UI.-- Before adding a page-specific message key, search `frontend/messages/en.json` for the same user-facing text and reuse an existing shared key when the grammatical purpose matches.-- Always define new keys ONLY in `frontend/messages/en.json` and use `m.*()`. Do NOT edit other language files (e.g., `fr.json`, `es.json`) because translations are handled automatically by Crowdin via pull requests.--## RBAC Architecture--Three layers — keep them separate:--- **Permission catalog** (`backend/pkg/authz/catalog.go`, `permissions.go`): raw capabilities, scope taxonomy, constants.-- **Access-surface registry** (`backend/pkg/authz/access_policy.go`): page/route/landing reachability metadata.-- **Frontend UX gates**: navigation config with `accessSurfaceId` references. Backend enforcement via `RequirePermission`/`RequireGlobalAdmin` remains authoritative.--Admin semantics from `PermissionSet.IsGlobalAdmin()` (backend) / `isGlobalAdmin` in user DTOs (frontend). Do not infer admin from role IDs in frontend auth checks.--## Multi-Environment Support--- Environment ID `"0"` = local Docker socket-- All API calls include `/environments/{id}/` prefix-- `environmentStore.ready` is a Promise — await before use-- On environment change, detail pages redirect to list pages--## Background Jobs+- Use Svelte 5 runes: `$props`, `$state`, `$derived`, and `$effect`.+- Do not use `export let`, `$:`, `on:event`, `$$props`, `$$restProps`, or legacy slots.+- Extend `BaseAPIService`; reuse existing services and query/mutation patterns.+- Use precise TypeScript types. Do not introduce `any`.+- Reuse shared components before creating page-local variants.+- Put every rendered string behind Paraglide messages.+- Reuse a matching key from `frontend/messages/en.json` before adding one.+- Add new keys only to `en.json`; Crowdin manages every other locale.-1. Implement `Job` interface in `backend/pkg/scheduler/` (`Name()`, `Schedule()` → 6-field cron, `Run()`)-2. Register in `jobs_bootstrap.go`+### Multi-environment and authorization-## Image & Container Updates+- Environment ID `"0"` is the local Docker environment.+- Environment-scoped API paths use `/environments/{id}/...`.+- Await `environmentStore.ready` or `getCurrentEnvironmentId()` before requests.+- Redirect environment-specific detail pages when the selected environment changes.+- Backend permission middleware is authoritative; frontend gates are UX only.+- Keep the permission catalog, access-surface registry, and frontend navigation gates+ as separate layers.+- Determine global admin status from `PermissionSet.IsGlobalAdmin()` or the user DTO's+ `isGlobalAdmin`; never infer it from a role ID.-- `backend/pkg/libarcane/imageupdate/` — digest checks, registry queries, label checks, sorter-- `backend/internal/services/image_update_service.go` — check + persist update records-- `backend/internal/services/updater_service.go` — apply updates-- `backend/api/handlers/image_updates.go` + `updater.go` — API endpoints-- `backend/pkg/scheduler/` — `image_polling_job.go`, `auto_update_job.go`, `auto_heal_job.go`+### Runtime modes and jobs-## Agent Modes+- Manager mode serves the UI and manages environments.+- Direct agent mode uses `AGENT_MODE=true` and accepts manager connections.+- Edge agent mode uses `EDGE_AGENT=true` with `MANAGER_API_URL` and dials the manager.+- Background jobs implement the scheduler job contract and are wired through+ `internal/di` and registered in `internal/bootstrap/jobs_bootstrap.go`.-- **Edge**: Agent dials out to manager via WebSocket/gRPC tunnel. Config: `EDGE_AGENT=true`, `MANAGER_API_URL`, `AGENT_TOKEN`.-- **Direct**: Agent runs HTTP server on TCP 3553, manager dials in. Config: `AGENT_MODE=true`, `AGENT_TOKEN`. No `MANAGER_API_URL`.+## Validation-## Testing+Use the narrowest relevant test first, then the repository gates:```bash-cd backend && go test ./... # unit tests (in-memory SQLite, testify)-just test e2e # Playwright E2E-just lint frontend # Svelte type checking+just format all+just lint all+just test backend|cli|types|e2e|all```-## Anti-Patterns--| ❌ Don't | ✅ Do |-| ------------------------------------------------ | -------------------------------------------------- |-| Business logic in handlers | Thin handlers → services |-| Gin router/middleware | Huma on Echo |-| Svelte 4 syntax (`export let`, `on:click`, `$:`) | Svelte 5 runes (`$props()`, `onclick`, `$derived`) |-| Hardcoded API paths without env ID | Include `/environments/{id}/` |-| Models without `BaseModel` | Embed `models.BaseModel` |-| TypeScript `any` | Proper types from `#lib/types` |-| Hardcoded UI strings | Paraglide messages via `m.*()` |-| Wrapping `pkg/` helpers | Call existing helpers directly |-| New API standard (raw HTTP, untyped) | Extend existing Huma typed handlers |+For AI-assisted code contributions, also run the development environment and manually+exercise the changed frontend and backend behavior as required by `AI_POLICY.md`.diff --git a/backend/api/api.go b/backend/api/api.goindex 5cb512b65..a31a62a6f 100644--- a/backend/api/api.go+++ b/backend/api/api.go@@ -9,12 +9,51 @@ import ("reflect""strings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/appimages"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth"+ "github.com/getarcaneapp/arcane/backend/v2/internal/build"+ "github.com/getarcaneapp/arcane/backend/v2/internal/diagnostics"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/federated"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitops"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitrepo"+ "github.com/getarcaneapp/arcane/backend/v2/internal/health"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/job"+ "github.com/getarcaneapp/arcane/backend/v2/internal/network"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/oidc"+ "github.com/getarcaneapp/arcane/backend/v2/internal/passkey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/port"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/search"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/swarm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/template"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user"+ "github.com/getarcaneapp/arcane/backend/v2/internal/variable"+ "github.com/getarcaneapp/arcane/backend/v2/internal/version"+ "github.com/getarcaneapp/arcane/backend/v2/internal/vulnerability"+"github.com/danielgtaylor/huma/v2""github.com/danielgtaylor/huma/v2/adapters/humaecho""github.com/getarcaneapp/arcane/backend/v2/api/handlers"- "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/container"+ "github.com/getarcaneapp/arcane/backend/v2/internal/dashboard"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/system"+ "github.com/getarcaneapp/arcane/backend/v2/internal/updater"+ "github.com/getarcaneapp/arcane/backend/v2/internal/volume"+ "github.com/getarcaneapp/arcane/backend/v2/internal/webhook"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/labstack/echo/v5""go.uber.org/fx")@@ -171,54 +210,53 @@ func qualifyGenericArcaneArgumentsInternal(pkgPath, typeName, schemaName string)// HandlerDeps contains the services required to register HTTP API handlers.//// It intentionally contains only dependencies consumed by SetupAPI,-// registerHandlersInternal, and the authentication bridge.+// registerHandlersInternal, and the authentication middleware.type HandlerDeps struct {fx.In- AppImages *services.ApplicationImagesService- User *services.UserService- Project *services.ProjectService- Environment *services.EnvironmentService- Settings *services.SettingsService- JobSchedule *services.JobService- SettingsSearch *services.SettingsSearchService- CustomizeSearch *services.CustomizeSearchService- Container *services.ContainerService- Image *services.ImageService- Build *services.BuildService- BuildWorkspace *services.BuildWorkspaceService- Volume *services.VolumeService- Network *services.NetworkService- Port *services.PortService- Swarm *services.SwarmService- ImageUpdate *services.ImageUpdateService- Auth *services.AuthService- Passkey *services.PasskeyService- Oidc *services.OidcService- Docker *services.DockerClientService- Template *services.TemplateService- ContainerRegistry *services.ContainerRegistryService- System *services.SystemService- SystemUpgrade *services.SystemUpgradeService- Diagnostics *services.DiagnosticsService- Updater *services.UpdaterService- Event *services.EventService- Activity *services.ActivityService- Version *services.VersionService- Notification *services.NotificationService- ApiKey *services.ApiKeyService- Federated *services.FederatedCredentialService- GitRepository *services.GitRepositoryService- GitOpsSync *services.GitOpsSyncService- Webhook *services.WebhookService- Vulnerability *services.VulnerabilityService- Dashboard *services.DashboardService- Role *services.RoleService- Variable *services.VariableService+ AppImages *appimages.ApplicationImagesService+ User *user.Module+ Project *project.Module+ Environment *environment.Module+ Settings *settings.Module+ JobSchedule *job.Module+ Search *search.Module+ Container *container.Module+ Image *image.Module+ Build *build.BuildService+ BuildWorkspace *build.BuildWorkspaceService+ Volume *volume.Module+ Network *network.NetworkService+ Port *port.PortService+ Swarm *swarm.Module+ ImageUpdate *imageupdate.Module+ Auth *auth.Module+ Passkey *passkey.PasskeyService+ Oidc *oidc.OidcService+ Docker *docker.DockerClientService+ Template *template.Module+ ContainerRegistry *registry.Module+ System *system.Module+ SystemUpgrade *system.SystemUpgradeService+ Diagnostics *diagnostics.DiagnosticsService+ Updater *updater.Module+ Event *event.Module+ Activity *activity.Module+ Version *version.VersionService+ Notification *notification.Module+ ApiKey *apikey.Module+ Federated *federated.FederatedCredentialService+ GitRepository *gitrepo.Module+ GitOpsSync *gitops.Module+ Webhook *webhook.Module+ Vulnerability *vulnerability.Module+ Dashboard *dashboard.Module+ Role *role.Module+ Variable *variable.Module}// SetupAPI creates and configures the Huma API attached to the Echo router.-func SetupAPI(e *echo.Echo, apiGroup *echo.Group, appCtx handlers.ActivityAppContext, cfg *config.Config, deps HandlerDeps) huma.API {+func SetupAPI(e *echo.Echo, apiGroup *echo.Group, appCtx handlerutil.ActivityAppContext, cfg *config.Config, deps HandlerDeps) huma.API {e.JSONSerializer = jsonV2Serializer{}humaConfig := huma.DefaultConfig("Arcane API", config.Version)@@ -269,7 +307,7 @@ func SetupAPI(e *echo.Echo, apiGroup *echo.Group, appCtx handlers.ActivityAppConapi := humaecho.NewWithGroup(e, apiGroup, humaConfig)// Add authentication middleware- api.UseMiddleware(middleware.NewAuthBridge(api, deps.Auth, deps.ApiKey, deps.Role, deps.Environment, cfg))+ api.UseMiddleware(auth.NewHumaMiddleware(api, deps.Auth.Service(), deps.ApiKey.Service(), deps.Role.Service(), deps.Environment.Service(), cfg))api.UseMiddleware(middleware.NewActivityBatchID())// Register all Huma handlers@@ -350,54 +388,49 @@ func SetupAPIForSpec() huma.API {api := humaecho.NewWithGroup(e, apiGroup, humaConfig)// Register handlers with zero-value dependencies for schema discovery only.- registerHandlersInternal(api, HandlerDeps{}, handlers.NewActivityAppContext(context.Background()), nil)+ registerHandlersInternal(api, HandlerDeps{}, handlerutil.NewActivityAppContext(context.Background()), nil)return api}-// registerHandlers registers all Huma-based API handlers.-// Add new handlers here as they are migrated from Gin.-func registerHandlersInternal(api huma.API, deps HandlerDeps, handlerAppCtx handlers.ActivityAppContext, cfg *config.Config) {- handlers.RegisterHealth(api)- handlers.RegisterAuth(api, deps.User, deps.Auth, deps.Oidc, deps.Settings, deps.Passkey)- handlers.RegisterPasskeys(api, deps.Passkey, deps.Auth, deps.User)- handlers.RegisterApiKeys(api, deps.ApiKey)- handlers.RegisterFederatedCredentials(api, deps.Federated)- handlers.RegisterRoles(api, deps.Role)- handlers.RegisterAppImages(api, deps.AppImages)- handlers.RegisterUsers(api, deps.User, deps.Auth, deps.Settings)- handlers.RegisterProjects(api, deps.Project, deps.Activity, handlerAppCtx)- handlers.RegisterVersion(api, deps.Version)- handlers.RegisterEvents(api, deps.Event)- handlers.RegisterActivities(api, deps.Activity, deps.Environment)- handlers.RegisterOidc(api, deps.Auth, deps.Passkey, deps.Oidc, deps.Role, deps.User, cfg)- handlers.RegisterEnvironments(api, deps.Environment, deps.Settings, deps.ApiKey, deps.Event, cfg)- handlers.RegisterContainerRegistries(api, deps.ContainerRegistry, deps.Environment)- handlers.RegisterTemplates(api, deps.Template)- if cfg != nil && cfg.AgentMode {- handlers.RegisterMaterializedVariables(api, deps.Variable, deps.Environment)- } else {- handlers.RegisterVariables(api, deps.Variable, deps.Environment)- }- handlers.RegisterImages(api, deps.Docker, deps.Image, deps.ImageUpdate, deps.Settings, deps.Build, deps.Activity, handlerAppCtx)- handlers.RegisterBuildWorkspaces(api, deps.BuildWorkspace)- handlers.RegisterImageUpdates(api, deps.ImageUpdate, deps.Image, handlerAppCtx)- handlers.RegisterSettings(api, deps.Settings, deps.SettingsSearch, deps.Environment, cfg)- handlers.RegisterJobSchedules(api, deps.JobSchedule, deps.Environment)- handlers.RegisterVolumes(api, deps.Docker, deps.Volume, deps.Activity, handlerAppCtx)- handlers.RegisterContainers(api, deps.Container, deps.Docker, deps.Settings, deps.Activity, handlerAppCtx)- handlers.RegisterPorts(api, deps.Port)- handlers.RegisterNetworks(api, deps.Network, deps.Docker, deps.Activity, handlerAppCtx)- handlers.RegisterSwarm(api, deps.Swarm, deps.Environment, deps.Event, cfg)- handlers.RegisterNotifications(api, deps.Notification, cfg)- handlers.RegisterUpdater(api, deps.Updater, handlerAppCtx)- handlers.RegisterCustomize(api, deps.CustomizeSearch)- handlers.RegisterSystem(api, deps.Docker, deps.System, deps.SystemUpgrade, deps.Environment, cfg, deps.Activity, handlerAppCtx)+// registerHandlersInternal registers all Huma-based API handlers.+func registerHandlersInternal(api huma.API, deps HandlerDeps, handlerAppCtx handlerutil.ActivityAppContext, cfg *config.Config) {+ health.RegisterRoutes(api)+ deps.Auth.RegisterRoutes(api)+ passkey.RegisterPasskeys(api, deps.Passkey, deps.Auth.Service(), deps.User.Service())+ deps.ApiKey.RegisterRoutes(api)+ federated.RegisterFederatedCredentials(api, deps.Federated)+ deps.Role.RegisterRoutes(api)+ appimages.RegisterAppImages(api, deps.AppImages)+ deps.User.RegisterRoutes(api)+ deps.Project.RegisterRoutes(api, handlerAppCtx)+ version.RegisterVersion(api, deps.Version)+ deps.Event.RegisterRoutes(api)+ deps.Activity.RegisterRoutes(api)+ oidc.RegisterOidc(api, deps.Auth.Service(), deps.Passkey, deps.Oidc, deps.Role.Service(), deps.User.Service(), cfg)+ deps.Environment.RegisterRoutes(api)+ deps.ContainerRegistry.RegisterRoutes(api)+ deps.Template.RegisterRoutes(api)+ deps.Variable.RegisterRoutes(api, cfg)+ deps.Image.RegisterRoutes(api, handlerAppCtx)+ build.RegisterBuildWorkspaces(api, deps.BuildWorkspace)+ deps.ImageUpdate.RegisterRoutes(api, handlerAppCtx)+ deps.Settings.RegisterRoutes(api)+ deps.JobSchedule.RegisterRoutes(api)+ deps.Volume.RegisterRoutes(api, handlerAppCtx)+ deps.Container.RegisterRoutes(api, handlerAppCtx)+ port.RegisterPorts(api, deps.Port)+ network.RegisterNetworks(api, deps.Network, deps.Docker, deps.Activity.Service(), handlerAppCtx)+ deps.Swarm.RegisterRoutes(api)+ deps.Notification.RegisterRoutes(api)+ deps.Updater.RegisterRoutes(api, handlerAppCtx)+ deps.Search.RegisterRoutes(api)+ deps.System.RegisterRoutes(api, handlerAppCtx)handlers.RegisterDiagnostics(api, deps.Diagnostics)- handlers.RegisterGitRepositories(api, deps.GitRepository)- handlers.RegisterGitOpsSyncs(api, deps.GitOpsSync)- handlers.RegisterWebhooks(api, deps.Webhook)- handlers.RegisterVulnerability(api, deps.Vulnerability, handlerAppCtx)- handlers.RegisterDashboard(api, deps.Dashboard, deps.Environment)- handlers.RegisterStream(api, deps.Dashboard, deps.Activity, deps.Environment)+ deps.GitRepository.RegisterRoutes(api)+ deps.GitOpsSync.RegisterRoutes(api)+ deps.Webhook.RegisterRoutes(api)+ deps.Vulnerability.RegisterRoutes(api, handlerAppCtx)+ deps.Dashboard.RegisterRoutes(api)+ handlers.RegisterStream(api, deps.Dashboard.Handler(), deps.Activity.Handler(), deps.Environment.Handler())}diff --git a/backend/api/api_test.go b/backend/api/api_test.goindex 5bf22f7a4..07923dd46 100644--- a/backend/api/api_test.go+++ b/backend/api/api_test.go@@ -3,15 +3,17 @@ package apiimport ("bytes""context"+ "net/http""reflect"+ "sort""strings""testing""time"humav2 "github.com/danielgtaylor/huma/v2"- "github.com/getarcaneapp/arcane/backend/v2/api/handlers""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"basetypes "github.com/getarcaneapp/arcane/types/v2/base"containertypes "github.com/getarcaneapp/arcane/types/v2/container"envtypes "github.com/getarcaneapp/arcane/types/v2/env"@@ -22,6 +24,7 @@ import ("github.com/labstack/echo/v5"dockercontainer "github.com/moby/moby/api/types/container"dockernetwork "github.com/moby/moby/api/types/network"+ "github.com/stretchr/testify/assert""github.com/stretchr/testify/require")@@ -297,7 +300,7 @@ func TestVariableMaterializationRoutesAreAgentOnly(t *testing.T) {agentAPI := SetupAPI(router,router.Group("/api"),- handlers.NewActivityAppContext(context.Background()),+ handlerutil.NewActivityAppContext(context.Background()),&config.Config{AgentMode: true},HandlerDeps{},)@@ -341,3 +344,51 @@ func TestEasyJoinRoutesDeclareSwarmJoinPermission(t *testing.T) {})}}++// TestEnvScopedOperationsDeclarePermission guards the remote-environment proxy+// authorization model. Every authenticated environment-scoped operation must+// declare the permission that the proxy enforces before forwarding it.+func TestEnvScopedOperationsDeclarePermission(t *testing.T) {+ api := SetupAPIForSpec()+ oapi := api.OpenAPI()++ require.False(t, oapi == nil || oapi.Paths == nil,+ "expected an OpenAPI document with paths")++ var missing []string+ for path, item := range oapi.Paths {+ if !strings.HasPrefix(path, "/environments/{id}/") {+ continue+ }+ for method, op := range envScopedTestOperationsInternal(item) {+ if op == nil {+ continue+ }+ if op.Security != nil && len(op.Security) == 0 {+ continue+ }+ permission, ok := op.Metadata[authz.MetaRequiredPermission].(string)+ if !ok || permission == "" {+ missing = append(missing, method+" "+path)+ }+ }+ }++ if len(missing) > 0 {+ sort.Strings(missing)+ assert.Failf(t, "unexpected failure", "%d env-scoped operation(s) missing required-permission metadata; register them with middleware.RegisterWithPermission:\n %s",+ len(missing), strings.Join(missing, "\n "))+ }+}++func envScopedTestOperationsInternal(item *humav2.PathItem) map[string]*humav2.Operation {+ return map[string]*humav2.Operation{+ http.MethodGet: item.Get,+ http.MethodPost: item.Post,+ http.MethodPut: item.Put,+ http.MethodDelete: item.Delete,+ http.MethodPatch: item.Patch,+ http.MethodHead: item.Head,+ http.MethodOptions: item.Options,+ }+}diff --git a/backend/api/buildables.go b/backend/api/buildables.goindex 3ba854a58..999aadd82 100644--- a/backend/api/buildables.go+++ b/backend/api/buildables.go@@ -9,23 +9,23 @@ import ("emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/buildables"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/cookie""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper"- "github.com/getarcaneapp/arcane/types/v2/auth"+ authtypes "github.com/getarcaneapp/arcane/types/v2/auth""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/user""github.com/labstack/echo/v5")// SetupBuildablesRoutes registers buildable feature routes based on EnabledFeatures.-func SetupBuildablesRoutes(apiGroup *echo.Group, authService *services.AuthService) {+func SetupBuildablesRoutes(apiGroup *echo.Group, authService *auth.AuthService) {if buildables.HasBuildFeature("autologin") {registerAutoLoginRoutes(apiGroup, authService)}}-func registerAutoLoginRoutes(apiGroup *echo.Group, authService *services.AuthService) {+func registerAutoLoginRoutes(apiGroup *echo.Group, authService *auth.AuthService) {authGroup := apiGroup.Group("/auth")authGroup.GET("/auto-login-config", func(c *echo.Context) error {@@ -38,7 +38,7 @@ func registerAutoLoginRoutes(apiGroup *echo.Group, authService *services.AuthSerreturn c.JSON(http.StatusInternalServerError, base.ErrorResponse{Error: "failed to get auto-login config"})}- return c.JSON(http.StatusOK, base.ApiResponse[auth.AutoLoginConfig]{+ return c.JSON(http.StatusOK, base.ApiResponse[authtypes.AutoLoginConfig]{Success: true,Data: *config,})@@ -63,17 +63,17 @@ func registerAutoLoginRoutes(apiGroup *echo.Group, authService *services.AuthSerreturn c.JSON(http.StatusInternalServerError, base.ErrorResponse{Error: "auto-login password not configured"})}- userModel, tokenPair, err := authService.Login(c.Request().Context(), autoLoginConfig.Username, password, auth.SessionMeta{+ userModel, tokenPair, err := authService.Login(c.Request().Context(), autoLoginConfig.Username, password, authtypes.SessionMeta{UserAgent: c.Request().UserAgent(),IPAddress: c.RealIP(),})if err != nil {switch {- case errors.Is(err, services.ErrInvalidCredentials):+ case errors.Is(err, auth.ErrInvalidCredentials):return c.JSON(http.StatusUnauthorized, base.ErrorResponse{Error: "Invalid username or password"})- case errors.Is(err, services.ErrLocalAuthDisabled):+ case errors.Is(err, auth.ErrLocalAuthDisabled):return c.JSON(http.StatusBadRequest, base.ErrorResponse{Error: "Local authentication is disabled"})- case errors.Is(err, services.ErrMFARequired):+ case errors.Is(err, auth.ErrMFARequired):return c.JSON(http.StatusForbidden, base.ErrorResponse{Error: "passkey MFA is required; auto-login cannot bypass MFA"})default:return c.JSON(http.StatusInternalServerError, base.ErrorResponse{Error: "Authentication failed"})@@ -90,10 +90,10 @@ func registerAutoLoginRoutes(apiGroup *echo.Group, authService *services.AuthSerexpiresAt := tokenPair.ExpiresAtc.Response().Header().Set("Set-Cookie", cookie.BuildTokenCookieStringFor(maxAge, tokenPair.AccessToken, cookie.SecureCookieFromRequest(c.Request())))- return c.JSON(http.StatusOK, base.ApiResponse[auth.AuthenticationResponse]{+ return c.JSON(http.StatusOK, base.ApiResponse[authtypes.AuthenticationResponse]{Success: true,- Data: auth.AuthenticationResponse{- Status: auth.AuthenticationStatusAuthenticated,+ Data: authtypes.AuthenticationResponse{+ Status: authtypes.AuthenticationStatusAuthenticated,Token: tokenPair.AccessToken,RefreshToken: tokenPair.RefreshToken,ExpiresAt: &expiresAt,diff --git a/backend/api/env_scoped_permission_test.go b/backend/api/env_scoped_permission_test.godeleted file mode 100644index 157f13a7a..000000000--- a/backend/api/env_scoped_permission_test.go+++ /dev/null@@ -1,71 +0,0 @@-package api--import (- "net/http"- "sort"- "strings"- "testing"-- "github.com/danielgtaylor/huma/v2"-- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/stretchr/testify/assert"- "github.com/stretchr/testify/require"-)--// TestEnvScopedOperationsDeclarePermission guards the remote-environment proxy-// authorization model: every operation served under /environments/{id}/<sub>-// must declare its required permission via middleware.RegisterWithPermission,-// which records it in the operation metadata. The proxy reads this metadata to-// enforce the same permission it enforces locally before forwarding a request-// to an agent. An operation missing this metadata would be denied by the proxy-// (default-deny), so this test fails loudly if any env-scoped operation is-// registered with a bare huma.Register instead of RegisterWithPermission.--func TestEnvScopedOperationsDeclarePermission(t *testing.T) {- api := SetupAPIForSpec()- oapi := api.OpenAPI()-- require.False(t, oapi == nil || oapi.Paths == nil,- "expected an OpenAPI document with paths")-- var missing []string- for path, item := range oapi.Paths {- if !strings.HasPrefix(path, "/environments/{id}/") {- continue- }- for method, op := range envScopedTestOperations(item) {- if op == nil {- continue- }- // Operations that explicitly opt out of authentication (an empty,- // non-nil Security list) are intentionally public and require no- // permission metadata.- if op.Security != nil && len(op.Security) == 0 {- continue- }- perm, ok := op.Metadata[authz.MetaRequiredPermission].(string)- if !ok || perm == "" {- missing = append(missing, method+" "+path)- }- }- }-- if len(missing) > 0 {- sort.Strings(missing)- assert.Failf(t, "unexpected failure", "%d env-scoped operation(s) missing required-permission metadata; register them with middleware.RegisterWithPermission:\n %s",- len(missing), strings.Join(missing, "\n "))- }-}--func envScopedTestOperations(item *huma.PathItem) map[string]*huma.Operation {- return map[string]*huma.Operation{- http.MethodGet: item.Get,- http.MethodPost: item.Post,- http.MethodPut: item.Put,- http.MethodDelete: item.Delete,- http.MethodPatch: item.Patch,- http.MethodHead: item.Head,- http.MethodOptions: item.Options,- }-}diff --git a/backend/api/handlers/activities_test.go b/backend/api/handlers/activities_test.godeleted file mode 100644index 7e5434cd1..000000000--- a/backend/api/handlers/activities_test.go+++ /dev/null@@ -1,393 +0,0 @@-package handlers--import (- "bufio"- "context"- "encoding/json"- "fmt"- "io"- "net/http"- "net/http/httptest"- "sync/atomic"- "testing"- "time"-- "github.com/libtnb/sqlite"- "github.com/stretchr/testify/require"- "gorm.io/gorm"-- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/getarcaneapp/arcane/types/v2/activity"- streamtypes "github.com/getarcaneapp/arcane/types/v2/stream"- "github.com/stretchr/testify/assert"-)--func setupActivityHandlerTestDBInternal(t *testing.T) *database.DB {- t.Helper()-- db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, db.AutoMigrate(- &models.Activity{},- &models.ActivityMessage{},- &models.Environment{},- &models.SettingVariable{},- ))- return &database.DB{DB: db}-}--func TestActivitySnapshotFingerprintDetectsChangesInternal(t *testing.T) {- progress := 10- items := []activity.Activity{- {ID: "a-1", Status: activity.StatusRunning, Progress: &progress, Step: "pull"},- {ID: "a-2", Status: activity.StatusSuccess},- }- base := activitySnapshotFingerprintInternal(items)- require.Equal(t, base, activitySnapshotFingerprintInternal(items))-- bumped := 20- items[0].Progress = &bumped- require.NotEqual(t, base, activitySnapshotFingerprintInternal(items))-}--func TestActivityHandlerClearHistoryDeletesSelectedEnvironmentOnlyInternal(t *testing.T) {- ctx := context.Background()- db := setupActivityHandlerTestDBInternal(t)- activityService := services.NewActivityService(db, nil)- handler := &ActivityHandler{activityService: activityService}-- completed, err := activityService.StartActivity(ctx, services.StartActivityRequest{EnvironmentID: "0", Type: models.ActivityTypeResourceAction})- require.NoError(t, err)- _, err = activityService.CompleteActivity(ctx, completed.ID, models.ActivityStatusSuccess, "done", nil)- require.NoError(t, err)-- running, err := activityService.StartActivity(ctx, services.StartActivityRequest{EnvironmentID: "0", Type: models.ActivityTypeResourceAction})- require.NoError(t, err)- remoteCompleted, err := activityService.StartActivity(ctx, services.StartActivityRequest{EnvironmentID: "remote-1", Type: models.ActivityTypeResourceAction})- require.NoError(t, err)- _, err = activityService.CompleteActivity(ctx, remoteCompleted.ID, models.ActivityStatusSuccess, "done", nil)- require.NoError(t, err)-- out, err := handler.ClearHistory(ctx, &ClearActivityHistoryInput{EnvironmentID: "0"})- require.NoError(t, err)- require.EqualValues(t, 1, out.Body.Data.Deleted)-- var remaining []models.Activity- require.NoError(t, db.Find(&remaining).Error)- require.Len(t, remaining, 2)- require.ElementsMatch(t, []string{running.ID, remoteCompleted.ID}, []string{remaining[0].ID, remaining[1].ID})-}--func TestActivityHandlerClearHistoryProxiesRemoteEnvironmentInternal(t *testing.T) {- ctx := context.Background()- db := setupActivityHandlerTestDBInternal(t)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)-- token := "remote-token"- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- if !assert.Equal(t, http.MethodDelete, r.Method) {- return- }- if !assert.Equal(t, "/api/environments/0/activities/history", r.URL.Path) {- return- }- if !assert.Equal(t, token, r.Header.Get("X-API-Key")) {- return- }- if !assert.Equal(t, token, r.Header.Get("X-Arcane-Agent-Token")) {- return- }- w.Header().Set("Content-Type", "application/json")- _, _ = w.Write([]byte(`{"success":true,"data":{"deleted":7}}`))- }))- defer server.Close()-- now := time.Now()- require.NoError(t, db.Create(&models.Environment{- BaseModel: models.BaseModel{- ID: "remote-1",- CreatedAt: now,- UpdatedAt: &now,- },- Name: "Remote",- ApiUrl: server.URL,- Status: string(models.EnvironmentStatusOnline),- Enabled: true,- AccessToken: &token,- }).Error)-- handler := &ActivityHandler{- environmentService: services.NewEnvironmentService(db, server.Client(), nil, nil, settingsService, nil),- }-- out, err := handler.ClearHistory(ctx, &ClearActivityHistoryInput{EnvironmentID: "remote-1"})- require.NoError(t, err)- require.EqualValues(t, 7, out.Body.Data.Deleted)-}--// limitStreamTestDBToSingleConnInternal serializes DB access: the aggregated-// stream queries from concurrent goroutines, and every extra pooled-// connection to a :memory: SQLite database is a fresh empty database.-func limitStreamTestDBToSingleConnInternal(t *testing.T, db *database.DB) {- t.Helper()- sqlDB, err := db.DB.DB()- require.NoError(t, err)- sqlDB.SetMaxOpenConns(1)-}--func createStreamTestRemoteEnvironmentInternal(t *testing.T, db *database.DB, environmentID, name, apiURL, token string) {- t.Helper()- now := time.Now()- require.NoError(t, db.Create(&models.Environment{- BaseModel: models.BaseModel{- ID: environmentID,- CreatedAt: now,- UpdatedAt: &now,- },- Name: name,- ApiUrl: apiURL,- Status: string(models.EnvironmentStatusOnline),- Enabled: true,- AccessToken: &token,- }).Error)-}--// runActivityClientStreamInternal drives the activities channel through a pipe-// and returns each decoded activity event to onEvent until it reports done.-func runActivityClientStreamInternal(t *testing.T, ctx context.Context, cancel context.CancelFunc, handler *ActivityHandler, ps *authz.PermissionSet, onEvent func(activity.StreamEvent) bool) {- t.Helper()-- pr, pw := io.Pipe()- done := make(chan struct{})- go func() {- defer close(done)- defer func() { _ = pw.Close() }()- streamHandler := &StreamHandler{activity: handler}- streamHandler.streamClientInternal(ctx, ps, &StreamClientInput{- Channels: streamtypes.ChannelActivities,- Limit: 50,- }, pw, func() {})- }()-- scanner := bufio.NewScanner(pr)- for scanner.Scan() {- var event streamtypes.Event- require.NoError(t, json.Unmarshal(scanner.Bytes(), &event))- if event.Activity != nil && onEvent(*event.Activity) {- cancel()- break- }- }-- go func() {- _, _ = io.Copy(io.Discard, pr)- }()- select {- case <-done:- case <-time.After(5 * time.Second):- require.FailNow(t, "stream did not terminate after cancel")- }-}--func TestActivityHandlerStreamAllEmitsEnvironmentScopedEventsInternal(t *testing.T) {- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)- defer cancel()-- db := setupActivityHandlerTestDBInternal(t)- limitStreamTestDBToSingleConnInternal(t, db)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)- activityService := services.NewActivityService(db, settingsService)-- local, err := activityService.StartActivity(ctx, services.StartActivityRequest{EnvironmentID: "0", Type: models.ActivityTypeResourceAction})- require.NoError(t, err)-- token := "remote-token"- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- w.Header().Set("Content-Type", "application/json")- _, _ = w.Write([]byte(`{"success":true,"data":[{"id":"remote-activity-1"}],"pagination":{"totalPages":1,"totalItems":1,"currentPage":1,"itemsPerPage":50}}`))- }))- defer server.Close()- createStreamTestRemoteEnvironmentInternal(t, db, "remote-1", "Remote", server.URL, token)-- handler := &ActivityHandler{- activityService: activityService,- environmentService: services.NewEnvironmentService(db, server.Client(), nil, nil, settingsService, nil),- }-- var localSnapshot, remoteSnapshot bool- runActivityClientStreamInternal(t, ctx, cancel, handler, authz.SudoPermissionSet(), func(event activity.StreamEvent) bool {- if event.Type == "snapshot" && event.EnvironmentID == "0" && len(event.Activities) == 1 {- require.Equal(t, local.ID, event.Activities[0].ID)- require.Equal(t, "0", event.Activities[0].SourceEnvironmentID)- localSnapshot = true- }- if event.Type == "snapshot" && event.EnvironmentID == "remote-1" && len(event.Activities) == 1 {- require.Equal(t, "remote-activity-1", event.Activities[0].ID)- require.Equal(t, "remote-1", event.Activities[0].SourceEnvironmentID)- require.Equal(t, "Remote", event.Activities[0].SourceEnvironmentName)- remoteSnapshot = true- }- return localSnapshot && remoteSnapshot- })-- require.True(t, localSnapshot)- require.True(t, remoteSnapshot)-}--func TestActivityHandlerStreamAllReusesRemoteEnvironmentAfterInitialPollInternal(t *testing.T) {- ctx, cancel := context.WithTimeout(context.Background(), 12*time.Second)- defer cancel()-- db := setupActivityHandlerTestDBInternal(t)- limitStreamTestDBToSingleConnInternal(t, db)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)- activityService := services.NewActivityService(db, settingsService)-- var countEnvironmentQueries atomic.Bool- var environmentQueryCount atomic.Int64- require.NoError(t, db.Callback().Query().After("gorm:query").Register("arcane_test_count_environment_queries", func(tx *gorm.DB) {- if countEnvironmentQueries.Load() && activityTestQueryLoadsEnvironmentIDInternal(tx, "remote-1") {- environmentQueryCount.Add(1)- }- }))-- token := "remote-token"- // Vary the payload per poll so snapshot fingerprint suppression does not- // swallow the follow-up snapshots this test waits for.- var pollCount atomic.Int64- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- w.Header().Set("Content-Type", "application/json")- payload := fmt.Sprintf(`{"success":true,"data":[{"id":"remote-activity-1","latestMessage":"poll-%d"}],"pagination":{"totalPages":1,"totalItems":1,"currentPage":1,"itemsPerPage":50}}`, pollCount.Add(1))- _, _ = w.Write([]byte(payload))- }))- defer server.Close()- createStreamTestRemoteEnvironmentInternal(t, db, "remote-1", "Remote", server.URL, token)-- handler := &ActivityHandler{- activityService: activityService,- environmentService: services.NewEnvironmentService(db, server.Client(), nil, nil, settingsService, nil),- }-- remoteSnapshotCount := 0- runActivityClientStreamInternal(t, ctx, cancel, handler, authz.SudoPermissionSet(), func(event activity.StreamEvent) bool {- if event.Type != "snapshot" || event.EnvironmentID != "remote-1" {- return false- }-- remoteSnapshotCount++- if remoteSnapshotCount == 1 {- countEnvironmentQueries.Store(true)- return false- }-- require.Zero(t, environmentQueryCount.Load(), "steady-state remote activity poll should not reload environment rows")- return true- })-- require.GreaterOrEqual(t, remoteSnapshotCount, 2)-}--func activityTestQueryLoadsEnvironmentIDInternal(tx *gorm.DB, environmentID string) bool {- if tx.Statement == nil || tx.Statement.Table != "environments" {- return false- }- for _, arg := range tx.Statement.Vars {- if arg == environmentID {- return true- }- }- return false-}--func TestActivityHandlerStreamAllRemoteFailureEmitsErrorAndKeepsStreamingInternal(t *testing.T) {- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)- defer cancel()-- db := setupActivityHandlerTestDBInternal(t)- limitStreamTestDBToSingleConnInternal(t, db)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)- activityService := services.NewActivityService(db, settingsService)-- token := "remote-token"- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- http.Error(w, "boom", http.StatusInternalServerError)- }))- defer server.Close()- createStreamTestRemoteEnvironmentInternal(t, db, "remote-1", "Remote", server.URL, token)-- handler := &ActivityHandler{- activityService: activityService,- environmentService: services.NewEnvironmentService(db, server.Client(), nil, nil, settingsService, nil),- }-- var localSnapshot, remoteError bool- runActivityClientStreamInternal(t, ctx, cancel, handler, authz.SudoPermissionSet(), func(event activity.StreamEvent) bool {- if event.Type == "snapshot" && event.EnvironmentID == "0" {- localSnapshot = true- }- if event.Type == "error" && event.EnvironmentID == "remote-1" && event.Error != "" {- remoteError = true- }- return localSnapshot && remoteError- })-- require.True(t, localSnapshot)- require.True(t, remoteError)-}--func TestActivityHandlerStreamAllFiltersUnauthorizedEnvironmentsInternal(t *testing.T) {- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)- defer cancel()-- db := setupActivityHandlerTestDBInternal(t)- limitStreamTestDBToSingleConnInternal(t, db)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)- activityService := services.NewActivityService(db, settingsService)- _, err = activityService.StartActivity(ctx, services.StartActivityRequest{EnvironmentID: "0", Type: models.ActivityTypeResourceAction})- require.NoError(t, err)-- allowedServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {- w.Header().Set("Content-Type", "application/json")- _, _ = w.Write([]byte(`{"success":true,"data":[{"id":"allowed-activity"}],"pagination":{"totalPages":1,"totalItems":1,"currentPage":1,"itemsPerPage":50}}`))- }))- defer allowedServer.Close()-- var deniedRequests atomic.Int64- deniedServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {- deniedRequests.Add(1)- w.Header().Set("Content-Type", "application/json")- _, _ = w.Write([]byte(`{"success":true,"data":[],"pagination":{"totalPages":1,"totalItems":0,"currentPage":1,"itemsPerPage":50}}`))- }))- defer deniedServer.Close()-- createStreamTestRemoteEnvironmentInternal(t, db, "remote-allowed", "Allowed", allowedServer.URL, "allowed-token")- createStreamTestRemoteEnvironmentInternal(t, db, "remote-denied", "Denied", deniedServer.URL, "denied-token")-- handler := &ActivityHandler{- activityService: activityService,- environmentService: services.NewEnvironmentService(db, allowedServer.Client(), nil, nil, settingsService, nil),- }- ps := authz.NewPermissionSet()- ps.AddEnv("remote-allowed", authz.PermActivitiesRead)-- seenEnvironments := make(map[string]struct{})- runActivityClientStreamInternal(t, ctx, cancel, handler, ps, func(event activity.StreamEvent) bool {- if event.EnvironmentID != "" {- seenEnvironments[event.EnvironmentID] = struct{}{}- }- return event.Type == "snapshot" && event.EnvironmentID == "remote-allowed"- })-- require.Contains(t, seenEnvironments, "remote-allowed")- require.NotContains(t, seenEnvironments, "0")- require.NotContains(t, seenEnvironments, "remote-denied")- require.Zero(t, deniedRequests.Load())-}diff --git a/backend/api/handlers/appimages_test.go b/backend/api/handlers/appimages_test.godeleted file mode 100644index 48f33122c..000000000--- a/backend/api/handlers/appimages_test.go+++ /dev/null@@ -1,109 +0,0 @@-package handlers--import (- "context"- "strings"- "testing"-- "github.com/danielgtaylor/huma/v2"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/resources"- "github.com/stretchr/testify/require"-)--func TestGetPWAIconReturnsPNGFromEmbeddedBackendAssets(t *testing.T) {- handler := &AppImagesHandler{- appImagesService: services.NewApplicationImagesService(resources.FS, nil),- }-- filenames := []string{- "icon-72x72.png",- "icon-96x96.png",- "icon-128x128.png",- "icon-144x144.png",- "icon-152x152.png",- "icon-192x192.png",- "icon-384x384.png",- "icon-512x512.png",- }-- for _, filename := range filenames {- t.Run(filename, func(t *testing.T) {- resp, err := handler.GetPWAIcon(context.Background(), &GetPWAIconInput{- Filename: filename,- })- require.NoError(t, err)- require.Equal(t, "image/png", resp.ContentType)- require.Equal(t, "public, max-age=900, stale-while-revalidate=86400", resp.CacheControl)- require.NotEmpty(t, resp.Body)- })- }-}--func TestGetLogoRejectsXSSPayloadInColorParam(t *testing.T) {- handler := &AppImagesHandler{- appImagesService: services.NewApplicationImagesService(resources.FS, nil),- }-- resp, err := handler.GetLogo(context.Background(), &GetLogoInput{- Color: "red}</style><script>alert(1)</script><style>x{",- })- require.NoError(t, err)- require.Equal(t, "image/svg+xml", resp.ContentType)- require.Equal(t, "nosniff", resp.XContentTypeOptions)-- body := string(resp.Body)- require.NotContains(t, body, "<script>", "injected <script> tag must not appear in response body")- require.NotContains(t, body, "</style><script", "style block must not be broken out of")- require.Contains(t, body, "fill:oklch(0.606 0.25 292.717)", "should fall back to default accent color")-}--func TestGetLogoAcceptsValidOklchOverride(t *testing.T) {- handler := &AppImagesHandler{- appImagesService: services.NewApplicationImagesService(resources.FS, nil),- }-- resp, err := handler.GetLogo(context.Background(), &GetLogoInput{- Color: "oklch(0.65 0.2 150)",- })- require.NoError(t, err)- require.Contains(t, string(resp.Body), "fill:oklch(0.65 0.2 150)")-}--func TestGetLogoAcceptsValidHexOverride(t *testing.T) {- handler := &AppImagesHandler{- appImagesService: services.NewApplicationImagesService(resources.FS, nil),- }-- resp, err := handler.GetLogo(context.Background(), &GetLogoInput{- Color: "#abcdef",- })- require.NoError(t, err)- require.Contains(t, string(resp.Body), "fill:#abcdef")-}--func TestGetLogoSetsNoSniffHeader(t *testing.T) {- handler := &AppImagesHandler{- appImagesService: services.NewApplicationImagesService(resources.FS, nil),- }-- resp, err := handler.GetLogo(context.Background(), &GetLogoInput{})- require.NoError(t, err)- require.Equal(t, "nosniff", resp.XContentTypeOptions)- require.True(t, strings.HasPrefix(resp.ContentType, "image/svg"))-}--func TestGetPWAIconRejectsNonPWAAssets(t *testing.T) {- handler := &AppImagesHandler{- appImagesService: services.NewApplicationImagesService(resources.FS, nil),- }-- resp, err := handler.GetPWAIcon(context.Background(), &GetPWAIconInput{- Filename: "logo.png",- })- require.Nil(t, resp)- require.Error(t, err)-- statusErr := huma.Error400BadRequest("invalid PWA icon filename")- require.Equal(t, statusErr.GetStatus(), err.(interface{ GetStatus() int }).GetStatus())-}diff --git a/backend/api/handlers/auth_time_format_test.go b/backend/api/handlers/auth_time_format_test.godeleted file mode 100644index a611b90e1..000000000--- a/backend/api/handlers/auth_time_format_test.go+++ /dev/null@@ -1,43 +0,0 @@-package handlers--import (- "reflect"- "testing"-- "github.com/danielgtaylor/huma/v2"- "github.com/stretchr/testify/require"-)--func TestUpdateMyProfileTimeFormatValidation(t *testing.T) {- registry := huma.NewMapRegistry("#/components/schemas/", huma.DefaultSchemaNamer)- schema := registry.Schema(reflect.TypeFor[UpdateMyProfileBody](), true, "UpdateMyProfileBody")-- for _, test := range []struct {- name string- value string- wantError bool- }{- {name: "auto", value: "auto"},- {name: "12 hour", value: "12h"},- {name: "24 hour", value: "24h"},- {name: "invalid", value: "military", wantError: true},- } {- t.Run(test.name, func(t *testing.T) {- result := &huma.ValidateResult{}- huma.Validate(- registry,- schema,- huma.NewPathBuffer(nil, 0),- huma.ModeWriteToServer,- map[string]any{"timeFormat": test.value},- result,- )-- if test.wantError {- require.NotEmpty(t, result.Errors)- return- }- require.Empty(t, result.Errors)- })- }-}diff --git a/backend/api/handlers/dashboard_test.go b/backend/api/handlers/dashboard_test.godeleted file mode 100644index f0fb1e81f..000000000--- a/backend/api/handlers/dashboard_test.go+++ /dev/null@@ -1,408 +0,0 @@-package handlers--import (- "bufio"- "context"- "encoding/json"- "io"- "net/http"- "net/http/httptest"- "strings"- "sync/atomic"- "testing"- "time"-- "github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- dashboardtypes "github.com/getarcaneapp/arcane/types/v2/dashboard"- streamtypes "github.com/getarcaneapp/arcane/types/v2/stream"- "github.com/libtnb/sqlite"- dockercontainer "github.com/moby/moby/api/types/container"- dockerimage "github.com/moby/moby/api/types/image"- "github.com/stretchr/testify/assert"- "github.com/stretchr/testify/require"- "gorm.io/gorm"-)--func setupDashboardHandlerTestDB(t *testing.T) (*database.DB, *services.SettingsService) {- t.Helper()-- db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, db.AutoMigrate(&models.ApiKey{}, &models.Environment{}, &models.ImageUpdateRecord{}, &models.Project{}, &models.SettingVariable{}))-- databaseDB := &database.DB{DB: db}- settingsSvc, err := newSettingsServiceForTestInternal(t, context.Background(), databaseDB)- require.NoError(t, err)-- return databaseDB, settingsSvc-}--func newDashboardHandlerTestDockerService(- t *testing.T,- settingsSvc *services.SettingsService,- containers []dockercontainer.Summary,- images []dockerimage.Summary,-) *services.DockerClientService {- t.Helper()-- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- w.Header().Set("Content-Type", "application/json")-- switch {- case strings.HasSuffix(r.URL.Path, "/_ping"):- w.Header().Set("API-Version", "1.41")- w.WriteHeader(http.StatusOK)- case strings.HasSuffix(r.URL.Path, "/containers/json"):- if !assert.NoError(t, json.NewEncoder(w).Encode(containers)) {- return- }- case strings.HasSuffix(r.URL.Path, "/images/json"):- if !assert.NoError(t, json.NewEncoder(w).Encode(images)) {- return- }- default:- http.NotFound(w, r)- }- }))- t.Cleanup(server.Close)-- return services.NewDockerClientService(- context.Background(),- nil,- &config.Config{DockerHost: server.URL},- settingsSvc,- )-}--func TestDashboardHandlerGetDashboardReturnsSnapshot(t *testing.T) {- db, settingsSvc := setupDashboardHandlerTestDB(t)-- containers := []dockercontainer.Summary{- {- ID: "container-running",- Names: []string{"/running-app"},- Image: "repo/app:stable",- ImageID: "sha256:image-a",- Created: 1700000000,- State: "running",- Status: "Up 2 hours",- Labels: map[string]string{},- },- {- ID: "container-stopped",- Names: []string{"/stopped-app"},- Image: "repo/worker:latest",- ImageID: "sha256:image-b",- Created: 1800000000,- State: "exited",- Status: "Exited (0) 1 hour ago",- Labels: map[string]string{},- },- }- images := []dockerimage.Summary{- {ID: "sha256:image-a", RepoTags: []string{"repo/app:stable"}, Created: 1710000000, Size: 100},- {ID: "sha256:image-b", RepoTags: []string{"repo/worker:latest"}, Created: 1720000000, Size: 250},- }-- require.NoError(t, db.WithContext(context.Background()).Create(&models.ImageUpdateRecord{- ID: "sha256:image-b",- HasUpdate: true,- }).Error)- require.NoError(t, db.WithContext(context.Background()).Create(&models.ApiKey{- Name: "expiring-soon",- KeyHash: "hash-soon",- KeyPrefix: "arc_test_handler",- UserID: new("user-1"),- ExpiresAt: new(time.Now().Add(12 * time.Hour)),- }).Error)-- dockerSvc := newDashboardHandlerTestDockerService(t, settingsSvc, containers, images)- handler := &DashboardHandler{- dashboardService: services.NewDashboardService(db, dockerSvc, nil, nil, nil, settingsSvc, nil, nil, nil, nil),- }-- output, err := handler.GetDashboard(context.Background(), &GetDashboardInput{EnvironmentID: "0"})- require.NoError(t, err)- require.NotNil(t, output)- require.True(t, output.Body.Success)-- snapshot := output.Body.Data- require.Len(t, snapshot.Containers.Data, 2)- require.Len(t, snapshot.Images.Data, 2)- require.Equal(t, 1, snapshot.Containers.Counts.RunningContainers)- require.Equal(t, 1, snapshot.Containers.Counts.StoppedContainers)- require.Equal(t, dashboardtypes.SnapshotSettings{}, snapshot.Settings)- require.ElementsMatch(t, []dashboardtypes.ActionItem{- {Kind: dashboardtypes.ActionItemKindStoppedContainers, Count: 1, Severity: dashboardtypes.ActionItemSeverityWarning},- {Kind: dashboardtypes.ActionItemKindImageUpdates, Count: 1, Severity: dashboardtypes.ActionItemSeverityWarning},- {Kind: dashboardtypes.ActionItemKindExpiringKeys, Count: 1, Severity: dashboardtypes.ActionItemSeverityWarning},- }, snapshot.ActionItems.Items)-}--// runDashboardStreamAllInternal drives the dashboard channel of the multiplexed-// client stream through a pipe and returns each decoded event to onEvent until-// it reports done or the stream ends; remaining output is drained so a blocked-// writer can finish.-func runDashboardStreamAllInternal(t *testing.T, ctx context.Context, cancel context.CancelFunc, handler *DashboardHandler, ps *authz.PermissionSet, onEvent func(dashboardtypes.StreamEvent) bool) {- t.Helper()-- streamHandler := &StreamHandler{dashboard: handler}- input := &StreamClientInput{Channels: streamtypes.ChannelDashboard}-- pr, pw := io.Pipe()- done := make(chan struct{})- go func() {- defer close(done)- defer func() { _ = pw.Close() }()- streamHandler.streamClientInternal(ctx, ps, input, pw, func() {})- }()-- scanner := bufio.NewScanner(pr)- for scanner.Scan() {- var envelope streamtypes.Event- require.NoError(t, json.Unmarshal(scanner.Bytes(), &envelope))- // Heartbeats are connection keepalives, not dashboard events.- if envelope.Dashboard == nil {- continue- }- if onEvent(*envelope.Dashboard) {- cancel()- break- }- }- require.NoError(t, scanner.Err())-- go func() {- _, _ = io.Copy(io.Discard, pr)- }()- select {- case <-done:- case <-time.After(5 * time.Second):- require.FailNow(t, "stream did not terminate after cancel")- }-}--func TestDashboardHandlerStreamAllEmitsRemoteSnapshotInternal(t *testing.T) {- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)- defer cancel()-- db := setupActivityHandlerTestDBInternal(t)- limitStreamTestDBToSingleConnInternal(t, db)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)-- token := "remote-token"- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- if !assert.Equal(t, "/api/environments/0/dashboard", r.URL.Path) {- return- }- w.Header().Set("Content-Type", "application/json")- _, _ = w.Write([]byte(`{"success":true,"data":{- "containers":{"data":[{"id":"c1"}],"counts":{"runningContainers":2,"stoppedContainers":1,"totalContainers":3},"pagination":{"totalPages":1,"totalItems":1,"currentPage":1,"itemsPerPage":20}},- "images":{"data":[],"pagination":{"totalPages":1,"totalItems":0,"currentPage":1,"itemsPerPage":20}},- "imageUsageCounts":{"imagesInuse":4,"imagesUnused":1,"totalImages":5,"totalImageSize":0},- "actionItems":{"items":[]},- "settings":{}- }}`))- }))- defer server.Close()- createStreamTestRemoteEnvironmentInternal(t, db, "remote-1", "Remote", server.URL, token)-- handler := &DashboardHandler{- dashboardService: services.NewDashboardService(db, nil, nil, nil, nil, nil, nil, nil, nil, nil),- environmentService: services.NewEnvironmentService(db, server.Client(), nil, nil, settingsService, nil),- }-- var remoteSnapshot bool- runDashboardStreamAllInternal(t, ctx, cancel, handler, authz.SudoPermissionSet(), func(event dashboardtypes.StreamEvent) bool {- if event.Type == "snapshot" && event.EnvironmentID == "remote-1" && event.Snapshot != nil {- require.Equal(t, 2, event.Snapshot.Containers.Counts.RunningContainers)- require.Equal(t, 3, event.Snapshot.Containers.Counts.TotalContainers)- require.Equal(t, 5, event.Snapshot.ImageUsageCounts.Total)- require.Empty(t, event.Snapshot.Containers.Data, "first-page table rows must be trimmed from stream events")- remoteSnapshot = true- }- return remoteSnapshot- })-- require.True(t, remoteSnapshot)-}--func TestDashboardHandlerStreamAllLegacyAgentComposesSnapshotFromGranularEndpointsInternal(t *testing.T) {- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)- defer cancel()-- db := setupActivityHandlerTestDBInternal(t)- limitStreamTestDBToSingleConnInternal(t, db)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)-- token := "remote-token"- // Older agents have no /dashboard route but do expose the granular- // counts/version endpoints the fallback composes a snapshot from.- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- w.Header().Set("Content-Type", "application/json")- switch r.URL.Path {- case "/api/environments/0/containers/counts":- _, _ = w.Write([]byte(`{"success":true,"data":{"runningContainers":4,"stoppedContainers":2,"totalContainers":6}}`))- case "/api/environments/0/images/counts":- _, _ = w.Write([]byte(`{"success":true,"data":{"imagesInuse":3,"imagesUnused":1,"totalImages":4,"totalImageSize":1024}}`))- case "/api/app-version":- _, _ = w.Write([]byte(`{"currentVersion":"1.9.0","displayVersion":"1.9.0","updateAvailable":true}`))- default:- w.WriteHeader(http.StatusNotFound)- _, _ = w.Write([]byte(`{"success":false,"error":"API endpoint not found: ` + r.URL.Path + `"}`))- }- }))- defer server.Close()- createStreamTestRemoteEnvironmentInternal(t, db, "remote-1", "Remote", server.URL, token)-- handler := &DashboardHandler{- dashboardService: services.NewDashboardService(db, nil, nil, nil, nil, nil, nil, nil, nil, nil),- environmentService: services.NewEnvironmentService(db, server.Client(), nil, nil, settingsService, nil),- }-- var composedSnapshot bool- runDashboardStreamAllInternal(t, ctx, cancel, handler, authz.SudoPermissionSet(), func(event dashboardtypes.StreamEvent) bool {- if event.Type == "snapshot" && event.EnvironmentID == "remote-1" && event.Snapshot != nil {- require.Equal(t, 4, event.Snapshot.Containers.Counts.RunningContainers)- require.Equal(t, 2, event.Snapshot.Containers.Counts.StoppedContainers)- require.Equal(t, 4, event.Snapshot.ImageUsageCounts.Total)- require.Len(t, event.Snapshot.ActionItems.Items, 1)- require.Equal(t, dashboardtypes.ActionItemKindStoppedContainers, event.Snapshot.ActionItems.Items[0].Kind)- require.NotNil(t, event.Snapshot.VersionInfo)- require.Equal(t, "1.9.0", event.Snapshot.VersionInfo.CurrentVersion)- composedSnapshot = true- }- return composedSnapshot- })-- require.True(t, composedSnapshot)-}--func TestDashboardHandlerStreamAllLegacyAgent404EmitsIncompatibleErrorInternal(t *testing.T) {- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)- defer cancel()-- db := setupActivityHandlerTestDBInternal(t)- limitStreamTestDBToSingleConnInternal(t, db)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)-- token := "remote-token"- // An agent so incompatible that even the granular fallback endpoints 404.- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- w.Header().Set("Content-Type", "application/json")- w.WriteHeader(http.StatusNotFound)- _, _ = w.Write([]byte(`{"success":false,"error":"API endpoint not found: ` + r.URL.Path + `"}`))- }))- defer server.Close()- createStreamTestRemoteEnvironmentInternal(t, db, "remote-1", "Remote", server.URL, token)-- handler := &DashboardHandler{- dashboardService: services.NewDashboardService(db, nil, nil, nil, nil, nil, nil, nil, nil, nil),- environmentService: services.NewEnvironmentService(db, server.Client(), nil, nil, settingsService, nil),- }-- var incompatibleError, localEvent bool- runDashboardStreamAllInternal(t, ctx, cancel, handler, authz.SudoPermissionSet(), func(event dashboardtypes.StreamEvent) bool {- if event.Type == "error" && event.EnvironmentID == "remote-1" {- require.Equal(t, dashboardtypes.StreamErrorCodeAgentIncompatible, event.ErrorCode)- require.NotEmpty(t, event.Error)- incompatibleError = true- }- // The local producer still emits (an error here — no docker in tests),- // proving one failing environment doesn't end the stream.- if event.EnvironmentID == "0" {- localEvent = true- }- return incompatibleError && localEvent- })-- require.True(t, incompatibleError)- require.True(t, localEvent)-}--func TestDashboardHandlerStreamAllFiltersUnauthorizedEnvironmentsInternal(t *testing.T) {- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)- defer cancel()-- db := setupActivityHandlerTestDBInternal(t)- limitStreamTestDBToSingleConnInternal(t, db)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)-- allowedServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {- w.Header().Set("Content-Type", "application/json")- _, _ = w.Write([]byte(`{"success":true,"data":{"containers":{"counts":{"runningContainers":1,"totalContainers":1}},"images":{},"imageUsageCounts":{"totalImages":1},"actionItems":{"items":[]},"settings":{}}}`))- }))- defer allowedServer.Close()-- var deniedRequests atomic.Int64- deniedServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {- deniedRequests.Add(1)- w.Header().Set("Content-Type", "application/json")- _, _ = w.Write([]byte(`{"success":true,"data":{}}`))- }))- defer deniedServer.Close()-- createStreamTestRemoteEnvironmentInternal(t, db, "remote-allowed", "Allowed", allowedServer.URL, "allowed-token")- createStreamTestRemoteEnvironmentInternal(t, db, "remote-denied", "Denied", deniedServer.URL, "denied-token")-- handler := &DashboardHandler{- dashboardService: services.NewDashboardService(db, nil, nil, nil, nil, nil, nil, nil, nil, nil),- environmentService: services.NewEnvironmentService(db, allowedServer.Client(), nil, nil, settingsService, nil),- }- ps := authz.NewPermissionSet()- ps.AddEnv("remote-allowed", authz.PermDashboardRead)-- seenEnvironments := make(map[string]struct{})- runDashboardStreamAllInternal(t, ctx, cancel, handler, ps, func(event dashboardtypes.StreamEvent) bool {- if event.EnvironmentID != "" {- seenEnvironments[event.EnvironmentID] = struct{}{}- }- return event.Type == "snapshot" && event.EnvironmentID == "remote-allowed"- })-- require.Contains(t, seenEnvironments, "remote-allowed")- require.NotContains(t, seenEnvironments, "0")- require.NotContains(t, seenEnvironments, "remote-denied")- require.Zero(t, deniedRequests.Load())-}--func TestDashboardHandlerRemoteFetchReusesLoadedEnvironmentInternal(t *testing.T) {- ctx := context.Background()- db := setupActivityHandlerTestDBInternal(t)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)-- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {- w.Header().Set("Content-Type", "application/json")- _, _ = w.Write([]byte(`{"success":true,"data":{"containers":{},"images":{},"imageUsageCounts":{},"actionItems":{"items":[]},"settings":{}}}`))- }))- defer server.Close()- createStreamTestRemoteEnvironmentInternal(t, db, "remote-1", "Remote", server.URL, "remote-token")-- environmentService := services.NewEnvironmentService(db, server.Client(), nil, nil, settingsService, nil)- environments, err := environmentService.ListRemoteEnvironments(ctx)- require.NoError(t, err)- require.Len(t, environments, 1)-- var environmentQueryCount atomic.Int64- require.NoError(t, db.Callback().Query().After("gorm:query").Register("arcane_test_count_dashboard_environment_queries", func(tx *gorm.DB) {- if activityTestQueryLoadsEnvironmentIDInternal(tx, "remote-1") {- environmentQueryCount.Add(1)- }- }))-- handler := &DashboardHandler{environmentService: environmentService}- for range 2 {- _, err := handler.fetchRemoteDashboardSnapshotInternal(ctx, environments[0], false)- require.NoError(t, err)- }- require.Zero(t, environmentQueryCount.Load())-}diff --git a/backend/api/handlers/diagnostics.go b/backend/api/handlers/diagnostics.goindex faf211495..d93351c33 100644--- a/backend/api/handlers/diagnostics.go+++ b/backend/api/handlers/diagnostics.go@@ -6,10 +6,11 @@ import ("github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/api/ws"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/diagnostics"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/system""go.getarcane.app/streams/logs")@@ -18,7 +19,7 @@ import (// streams and pprof routes live in the api/ws package alongside the other// streaming endpoints; the snapshot is assembled there too (ws.BuildDiagnostics).type DiagnosticsHandler struct {- diag *services.DiagnosticsService+ diag *diagnostics.DiagnosticsService}type DiagnosticsInput struct{}@@ -32,7 +33,7 @@ type GetDiagnosticsLogsOutput struct {}// RegisterDiagnostics registers the Huma diagnostics REST endpoints.-func RegisterDiagnostics(api huma.API, diag *services.DiagnosticsService) {+func RegisterDiagnostics(api huma.API, diag *diagnostics.DiagnosticsService) {h := &DiagnosticsHandler{diag: diag}huma.Register(api, huma.Operation{@@ -42,8 +43,8 @@ func RegisterDiagnostics(api huma.API, diag *services.DiagnosticsService) {Summary: "Get runtime diagnostics",Description: "Returns Go runtime, memory, garbage-collector, and WebSocket connection statistics.",Tags: []string{"Diagnostics"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermDiagnosticsRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermDiagnosticsRead),}, h.GetDiagnostics)huma.Register(api, huma.Operation{@@ -53,8 +54,8 @@ func RegisterDiagnostics(api huma.API, diag *services.DiagnosticsService) {Summary: "Get recent backend logs",Description: "Returns the most recent buffered backend log entries (oldest first).",Tags: []string{"Diagnostics"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermDiagnosticsRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermDiagnosticsRead),}, h.GetRecentLogs)}diff --git a/backend/api/handlers/edge_mtls_ca_test.go b/backend/api/handlers/edge_mtls_ca_test.godeleted file mode 100644index 49d9e0f90..000000000--- a/backend/api/handlers/edge_mtls_ca_test.go+++ /dev/null@@ -1,117 +0,0 @@-package handlers--import (- "context"- "os"- "path/filepath"- "testing"- "time"-- "github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge"- "github.com/stretchr/testify/require"- libcrypto "go.getarcane.app/sys/crypto"-)--func TestMain(m *testing.M) {- libcrypto.InitEncryption(&libcrypto.Config{- EncryptionKey: "test-encryption-key-for-edge-mtls-32bytes-min",- Environment: "test",- })- os.Exit(m.Run())-}--func TestGeneratedEdgeMTLSCAPath(t *testing.T) {- t.Run("returns error when edge mTLS is disabled", func(t *testing.T) {- cfg := &config.Config{EdgeMTLSMode: edge.EdgeMTLSModeDisabled}-- _, err := generatedEdgeMTLSCAPathInternal(cfg)-- require.Error(t, err)- })-- t.Run("returns configured CA path when CA is externally managed", func(t *testing.T) {- caPath := t.TempDir() + "/custom-ca.crt"- require.NoError(t, os.WriteFile(caPath, []byte("pem"), 0o644))-- cfg := &config.Config{- EdgeMTLSMode: edge.EdgeMTLSModeRequired,- EdgeMTLSCAFile: caPath,- }-- path, err := generatedEdgeMTLSCAPathInternal(cfg)-- require.NoError(t, err)- require.Equal(t, caPath, path)- })-- t.Run("returns generated CA path when Arcane manages the CA", func(t *testing.T) {- cfg := &config.Config{- EdgeMTLSMode: edge.EdgeMTLSModeRequired,- EdgeMTLSAssetsDir: t.TempDir(),- }- require.NoError(t, edge.PrepareManagerMTLSAssetsWithContext(context.Background(), &edge.Config{- EdgeMTLSMode: cfg.EdgeMTLSMode,- EdgeMTLSAssetsDir: cfg.EdgeMTLSAssetsDir,- AppURL: cfg.GetAppURL(),- }))-- path, err := generatedEdgeMTLSCAPathInternal(cfg)-- require.NoError(t, err)- require.FileExists(t, path)-- pemBytes, readErr := os.ReadFile(path)- require.NoError(t, readErr)- require.Contains(t, string(pemBytes), "BEGIN CERTIFICATE")- })-- t.Run("does not generate missing Arcane-managed CA", func(t *testing.T) {- cfg := &config.Config{- EdgeMTLSMode: edge.EdgeMTLSModeRequired,- EdgeMTLSAssetsDir: t.TempDir(),- }-- _, err := generatedEdgeMTLSCAPathInternal(cfg)-- require.Error(t, err)- require.NoFileExists(t, filepath.Join(cfg.EdgeMTLSAssetsDir, "ca.crt"))- require.NoFileExists(t, filepath.Join(cfg.EdgeMTLSAssetsDir, "ca.key"))- })-}--func TestReadGeneratedEdgeMTLSCertificateInfo(t *testing.T) {- cfg := &config.Config{- EdgeMTLSMode: edge.EdgeMTLSModeRequired,- EdgeMTLSAssetsDir: t.TempDir(),- }-- _, err := edge.GenerateManagerClientMTLSAssetsWithContext(context.Background(), &edge.Config{- EdgeMTLSMode: cfg.EdgeMTLSMode,- EdgeMTLSAssetsDir: cfg.EdgeMTLSAssetsDir,- AppURL: cfg.GetAppURL(),- }, "env-123", "Lab Server")- require.NoError(t, err)-- info, err := readGeneratedEdgeMTLSCertificateInfoInternal(cfg, "env-123")-- require.NoError(t, err)- require.NotNil(t, info)- require.NotNil(t, info.CommonName)- require.Equal(t, "Lab-Server-env-123", *info.CommonName)- require.NotNil(t, info.ExpiresAt)- require.True(t, info.ExpiresAt.After(time.Now().UTC()))- require.NotNil(t, info.DaysRemaining)- require.Positive(t, *info.DaysRemaining)- require.False(t, info.Expired)- require.False(t, info.ExpiringSoon)-}--func TestEdgeMTLSCertificateDaysRemaining(t *testing.T) {- now := time.Date(2026, 4, 30, 12, 0, 0, 0, time.UTC)-- require.Equal(t, 2, edgeMTLSCertificateDaysRemainingInternal(now, now.Add(48*time.Hour)))- require.Equal(t, 3, edgeMTLSCertificateDaysRemainingInternal(now, now.Add(48*time.Hour+time.Second)))- require.Equal(t, 0, edgeMTLSCertificateDaysRemainingInternal(now, now))- require.Equal(t, 0, edgeMTLSCertificateDaysRemainingInternal(now, now.Add(-time.Second)))-}diff --git a/backend/api/handlers/environments_test.go b/backend/api/handlers/environments_test.godeleted file mode 100644index 1176fb86b..000000000--- a/backend/api/handlers/environments_test.go+++ /dev/null@@ -1,289 +0,0 @@-package handlers--import (- "context"- "net/http"- "net/http/httptest"- "os"- "reflect"- "testing"- "time"-- "github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge"- envtypes "github.com/getarcaneapp/arcane/types/v2/environment"- "github.com/stretchr/testify/assert"- "github.com/stretchr/testify/require"-)--func TestEnvironmentHandlerTestConnectionRequiresUpdatePermissionForCustomURLInternal(t *testing.T) {- permissions := authz.NewPermissionSet()- permissions.AddGlobal(authz.PermEnvironmentsRead)- ctx := context.WithValue(context.Background(), humamw.ContextKeyUserPermissions, permissions)-- output, err := (&EnvironmentHandler{}).TestConnection(ctx, &TestConnectionInput{- ID: "0",- Body: &envtypes.TestConnectionRequest{- ApiUrl: new("http://10.0.0.2:3553"),- },- })-- require.Nil(t, output)- require.ErrorContains(t, err, "permission denied: "+authz.PermEnvironmentsUpdate)-}--func TestEnvironmentHandlerTestConnectionReturnsOpaqueBadRequestForCustomURLInternal(t *testing.T) {- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {- w.WriteHeader(http.StatusTeapot)- }))- t.Cleanup(server.Close)-- db := setupActivityHandlerTestDBInternal(t)- env := &models.Environment{Name: "Test", ApiUrl: "http://stored.example", Enabled: true}- env.ID = "env-1"- require.NoError(t, db.Create(env).Error)-- permissions := authz.NewPermissionSet()- permissions.AddGlobal(authz.PermEnvironmentsUpdate)- ctx := context.WithValue(context.Background(), humamw.ContextKeyUserPermissions, permissions)- handler := &EnvironmentHandler{- environmentService: services.NewEnvironmentService(db, server.Client(), nil, nil, nil, nil),- }-- output, err := handler.TestConnection(ctx, &TestConnectionInput{- ID: "env-1",- Body: &envtypes.TestConnectionRequest{- ApiUrl: new(server.URL),- },- })-- require.NotNil(t, output)- require.False(t, output.Body.Success)- require.Nil(t, output.Body.Data.Message)- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- require.Equal(t, http.StatusBadRequest, statusErr.GetStatus())- require.Equal(t, "Environment connection test failed", statusErr.Error())-}--func TestEnvironmentSecretDeploymentRoutesRequirePairPermission(t *testing.T) {- testCases := []struct {- name string- path string- }{- {name: "deployment snippets", path: "/api/environments/env-1/deployment"},- {name: "mtls bundle", path: "/api/environments/env-1/deployment/mtls/bundle"},- {name: "mtls file", path: "/api/environments/env-1/deployment/mtls/agent.key"},- {name: "edge ca", path: "/api/edge-mtls/ca"},- }-- for _, testCase := range testCases {- t.Run(testCase.name+" read denied", func(t *testing.T) {- ps := authz.NewPermissionSet()- ps.AddGlobal(authz.PermEnvironmentsRead)- router, api := newPermissionGatingRouterInternal(t, ps)- RegisterEnvironments(api, nil, nil, nil, nil, nil)-- req := httptest.NewRequest(http.MethodGet, testCase.path, nil)- rec := httptest.NewRecorder()- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusForbidden, rec.Code)- require.Contains(t, rec.Body.String(), authz.PermEnvironmentsPair)- })-- }-}--func TestEnvironmentMTLSAssetDownloadName(t *testing.T) {- env := &models.Environment{Name: "Lab Server"}- env.ID = "env-123"-- assert.Equal(t, "Lab-Server-env-123.pem", environmentMTLSAssetDownloadNameInternal(env, "agent.crt"))- assert.Equal(t, "Lab-Server-env-123.key", environmentMTLSAssetDownloadNameInternal(env, "agent.key"))- assert.Equal(t, "ca.crt", environmentMTLSAssetDownloadNameInternal(env, "ca.crt"))-}--func TestEnvironmentMTLSAssetFileModeInternal(t *testing.T) {- assert.Equal(t, os.FileMode(0o600), environmentMTLSAssetFileModeInternal(services.DeploymentSnippetFile{- Name: "agent.key",- Permissions: "0600",- }))- assert.Equal(t, os.FileMode(0o644), environmentMTLSAssetFileModeInternal(services.DeploymentSnippetFile{- Name: "ca.crt",- }))- assert.Equal(t, os.FileMode(0o600), environmentMTLSAssetFileModeInternal(services.DeploymentSnippetFile{- Name: "agent.key",- }))-}--func TestEnvironmentHandlerApplyEdgeRuntimeState(t *testing.T) {- t.Run("leaves non-edge environments unchanged", func(t *testing.T) {- handler := &EnvironmentHandler{}- env := envtypes.Environment{- ID: "0",- Status: string(models.EnvironmentStatusOnline),- IsEdge: false,- }-- handler.applyEdgeRuntimeStateInternal(&env)-- assert.Equal(t, string(models.EnvironmentStatusOnline), env.Status)- assert.Nil(t, env.EdgeTransport)- assert.Nil(t, env.EdgeSecurityMode)- assert.Nil(t, env.EdgeSessionID)- assert.Nil(t, env.EdgeAgentInstance)- assert.Nil(t, env.EdgeCapabilities)- assert.Nil(t, env.Connected)- assert.Nil(t, env.ConnectedAt)- assert.Nil(t, env.LastHeartbeat)- })-- t.Run("marks stale edge status offline when no live tunnel exists", func(t *testing.T) {- handler := &EnvironmentHandler{}- envID := "env-edge-offline"- edge.GetRegistry().Unregister(envID)- t.Cleanup(func() { edge.GetRegistry().Unregister(envID) })-- env := envtypes.Environment{- ID: envID,- Status: string(models.EnvironmentStatusOnline),- IsEdge: true,- }-- handler.applyEdgeRuntimeStateInternal(&env)-- assert.Equal(t, string(models.EnvironmentStatusOffline), env.Status)- assert.Nil(t, env.EdgeTransport)- assert.Nil(t, env.EdgeSecurityMode)- assert.Nil(t, env.EdgeSessionID)- assert.Nil(t, env.EdgeAgentInstance)- assert.Nil(t, env.EdgeCapabilities)- if assert.NotNil(t, env.Connected) {- assert.False(t, *env.Connected)- }- assert.Nil(t, env.ConnectedAt)- assert.Nil(t, env.LastHeartbeat)- })-- t.Run("preserves pending edge environments until they connect", func(t *testing.T) {- handler := &EnvironmentHandler{}- envID := "env-edge-pending"- edge.GetRegistry().Unregister(envID)- t.Cleanup(func() { edge.GetRegistry().Unregister(envID) })-- env := envtypes.Environment{- ID: envID,- Status: string(models.EnvironmentStatusPending),- IsEdge: true,- }-- handler.applyEdgeRuntimeStateInternal(&env)-- assert.Equal(t, string(models.EnvironmentStatusPending), env.Status)- assert.Nil(t, env.EdgeTransport)- assert.Nil(t, env.EdgeSecurityMode)- assert.Nil(t, env.EdgeSessionID)- assert.Nil(t, env.EdgeAgentInstance)- assert.Nil(t, env.EdgeCapabilities)- if assert.NotNil(t, env.Connected) {- assert.False(t, *env.Connected)- }- assert.Nil(t, env.ConnectedAt)- assert.Nil(t, env.LastHeartbeat)- })-- t.Run("reports live tunnel status as online", func(t *testing.T) {- handler := &EnvironmentHandler{}- envID := "env-edge-live"- edge.GetRegistry().Unregister(envID)- t.Cleanup(func() { edge.GetRegistry().Unregister(envID) })-- tunnel := edge.NewAgentTunnelWithConn(envID, edge.NewGRPCManagerTunnelConn(nil))- tunnel.SessionID = "session-live"- tunnel.AgentInstance = "agent-live"- tunnel.SecurityMode = "mtls"- tunnel.Capabilities = []string{"container.list", "container.inspect"}- edge.GetRegistry().Register(envID, tunnel)-- env := envtypes.Environment{- ID: envID,- Status: string(models.EnvironmentStatusOffline),- IsEdge: true,- }-- handler.applyEdgeRuntimeStateInternal(&env)-- assert.Equal(t, string(models.EnvironmentStatusOnline), env.Status)- if assert.NotNil(t, env.EdgeTransport) {- assert.Equal(t, edge.EdgeTransportGRPC, *env.EdgeTransport)- }- if assert.NotNil(t, env.EdgeSecurityMode) {- assert.Equal(t, "mtls", *env.EdgeSecurityMode)- }- if assert.NotNil(t, env.EdgeSessionID) {- assert.Equal(t, "session-live", *env.EdgeSessionID)- }- if assert.NotNil(t, env.EdgeAgentInstance) {- assert.Equal(t, "agent-live", *env.EdgeAgentInstance)- }- assert.Equal(t, []string{"container.list", "container.inspect"}, env.EdgeCapabilities)- if assert.NotNil(t, env.Connected) {- assert.True(t, *env.Connected)- }- assert.NotNil(t, env.ConnectedAt)- assert.NotNil(t, env.LastHeartbeat)- })-- t.Run("marks recently polled edge environments standby without a live tunnel", func(t *testing.T) {- handler := &EnvironmentHandler{}- envID := "env-edge-polled"- edge.GetRegistry().Unregister(envID)- t.Cleanup(func() { edge.GetRegistry().Unregister(envID) })-- edge.GetPollRuntimeRegistry().Update(envID, edge.DefaultTunnelPollInterval, time.Now())-- env := envtypes.Environment{- ID: envID,- Status: string(models.EnvironmentStatusOffline),- IsEdge: true,- }-- handler.applyEdgeRuntimeStateInternal(&env)-- assert.Equal(t, string(models.EnvironmentStatusStandby), env.Status)- if assert.NotNil(t, env.Connected) {- assert.False(t, *env.Connected)- }- assert.Nil(t, env.EdgeTransport)- assert.Nil(t, env.EdgeSecurityMode)- assert.Nil(t, env.EdgeSessionID)- assert.Nil(t, env.EdgeAgentInstance)- assert.Nil(t, env.EdgeCapabilities)- assert.Nil(t, env.ConnectedAt)- assert.Nil(t, env.LastHeartbeat)- assert.NotNil(t, env.LastPollAt)- })-}--// The environment stream suppresses redundant sends by comparing a fingerprint-// of the visible list. A field added to Environment but not to the fingerprint-// would stop propagating to connected clients until the 30s floor, so pin the-// field count: if this fails, add the new field to-// fingerprintEnvironmentsInternal and bump the constant.-func TestEnvironmentFingerprintCoversEveryField(t *testing.T) {- const environmentFingerprintFieldCount = 19- require.Equal(t, environmentFingerprintFieldCount, reflect.TypeOf(envtypes.Environment{}).NumField(),- "environment.Environment gained or lost a field; update fingerprintEnvironmentsInternal to match")-}--func TestFingerprintEnvironmentsDetectsFieldChange(t *testing.T) {- base := []envtypes.Environment{{ID: "1", Name: "prod", Status: "online", Enabled: true}}- renamed := []envtypes.Environment{{ID: "1", Name: "production", Status: "online", Enabled: true}}-- assert.NotEqual(t, fingerprintEnvironmentsInternal(base), fingerprintEnvironmentsInternal(renamed))-}diff --git a/backend/api/handlers/events_test.go b/backend/api/handlers/events_test.godeleted file mode 100644index a6c56dbd0..000000000--- a/backend/api/handlers/events_test.go+++ /dev/null@@ -1,137 +0,0 @@-package handlers--import (- "bytes"- "context"- "encoding/json"- "net/http"- "net/http/httptest"- "testing"-- "github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"- "github.com/labstack/echo/v5"- "github.com/libtnb/sqlite"- "github.com/stretchr/testify/require"- "gorm.io/gorm"-)--func TestEventDeleteRequiresEventsDeletePermission(t *testing.T) {- testCases := []struct {- name string- permission string- }{- {- name: "events read cannot delete",- permission: authz.PermEventsRead,- },- }-- for _, testCase := range testCases {- t.Run(testCase.name, func(t *testing.T) {- ps := authz.NewPermissionSet()- ps.AddGlobal(testCase.permission)- router, api := newPermissionGatingRouterInternal(t, ps)- RegisterEvents(api, nil)-- req := httptest.NewRequest(http.MethodDelete, "/api/events/event-1", nil)- rec := httptest.NewRecorder()- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusForbidden, rec.Code)- require.Contains(t, rec.Body.String(), authz.PermEventsDelete)- })- }-}--func TestAgentEventIngestionRequiresAgentTokenAndPersists(t *testing.T) {- ctx := context.Background()- db := setupEventIngestionTestDBInternal(t)- eventSvc := services.NewEventService(db, &config.Config{}, nil)- router := echo.New()- RegisterAgentEventIngestion(router.Group("/api"), eventSvc, &config.Config{AgentToken: "agent-token"})-- payload := services.CreateEventRequest{- Type: models.EventTypeContainerStart,- Severity: models.EventSeverityInfo,- Title: "Container started: web",- Description: "Container 'web' has been started",- EnvironmentID: new("0"),- Metadata: models.JSON{"source": "agent"},- }- body, err := json.Marshal(payload)- require.NoError(t, err)-- t.Run("unconfigured manager token is service unavailable", func(t *testing.T) {- unconfiguredRouter := echo.New()- RegisterAgentEventIngestion(unconfiguredRouter.Group("/api"), eventSvc, &config.Config{})-- req := httptest.NewRequest(http.MethodPost, "/api/events", bytes.NewReader(body))- req.Header.Set("Content-Type", "application/json")- req.Header.Set(pkgutils.HeaderAgentToken, "agent-token")- rec := httptest.NewRecorder()- unconfiguredRouter.ServeHTTP(rec, req)-- require.Equal(t, http.StatusServiceUnavailable, rec.Code)- require.Contains(t, rec.Body.String(), "agent event ingestion is not configured")- require.Equal(t, int64(0), countPersistedEventsInternal(t, ctx, db))- })-- t.Run("missing token rejected", func(t *testing.T) {- req := httptest.NewRequest(http.MethodPost, "/api/events", bytes.NewReader(body))- req.Header.Set("Content-Type", "application/json")- rec := httptest.NewRecorder()- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusUnauthorized, rec.Code)- require.Equal(t, int64(0), countPersistedEventsInternal(t, ctx, db))- })-- t.Run("api key header is not accepted", func(t *testing.T) {- req := httptest.NewRequest(http.MethodPost, "/api/events", bytes.NewReader(body))- req.Header.Set("Content-Type", "application/json")- req.Header.Set(pkgutils.HeaderApiKey, "agent-token")- rec := httptest.NewRecorder()- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusUnauthorized, rec.Code)- require.Equal(t, int64(0), countPersistedEventsInternal(t, ctx, db))- })-- t.Run("agent token persists event", func(t *testing.T) {- req := httptest.NewRequest(http.MethodPost, "/api/events", bytes.NewReader(body))- req.Header.Set("Content-Type", "application/json")- req.Header.Set(pkgutils.HeaderAgentToken, "agent-token")- rec := httptest.NewRecorder()- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusAccepted, rec.Code)-- var saved models.Event- require.NoError(t, db.WithContext(ctx).First(&saved).Error)- require.Equal(t, models.EventTypeContainerStart, saved.Type)- require.Equal(t, "Container started: web", saved.Title)- require.NotNil(t, saved.EnvironmentID)- require.Equal(t, "0", *saved.EnvironmentID)- require.Equal(t, "agent", saved.Metadata["source"])- })-}--func setupEventIngestionTestDBInternal(t *testing.T) *database.DB {- t.Helper()- db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, db.AutoMigrate(&models.Event{}))- return &database.DB{DB: db}-}--func countPersistedEventsInternal(t *testing.T, ctx context.Context, db *database.DB) int64 {- t.Helper()- var count int64- require.NoError(t, db.WithContext(ctx).Model(&models.Event{}).Count(&count).Error)- return count-}diff --git a/backend/api/handlers/git_repositories_test.go b/backend/api/handlers/git_repositories_test.godeleted file mode 100644index f9b1b4155..000000000--- a/backend/api/handlers/git_repositories_test.go+++ /dev/null@@ -1,130 +0,0 @@-package handlers--import (- "context"- "net/http"- "net/http/httptest"- "testing"-- "github.com/danielgtaylor/huma/v2"- "github.com/danielgtaylor/huma/v2/adapters/humaecho"- "github.com/labstack/echo/v5"- "github.com/stretchr/testify/require"-- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"-)--// TestGitRepositoryHandlers_PermissionGating verifies that the mutating-// git-repository operations are gated by their RequirePermission middleware:-// a caller without the required permission gets 403 before the handler runs.-//-// We exercise the full Huma operation stack (registration → middleware → handler)-// so the same path that runs in production is what's under test.-func TestGitRepositoryHandlers_PermissionGating(t *testing.T) {- tests := []struct {- name string- // register installs one operation on the API with its real middleware.- register func(api huma.API)- method string- path string- }{- {- name: "create repository",- register: func(api huma.API) {- huma.Register(api, huma.Operation{- OperationID: "test-create-git-repo",- Method: http.MethodPost,- Path: "/customize/git-repositories",- Middlewares: humamw.RequirePermission(api, authz.PermGitReposCreate),- }, func(_ context.Context, _ *struct{}) (*struct{}, error) {- require.FailNow(t, "handler must not run when permission is missing")- return nil, nil- })- },- method: http.MethodPost,- path: "/api/customize/git-repositories",- },- {- name: "update repository",- register: func(api huma.API) {- huma.Register(api, huma.Operation{- OperationID: "test-update-git-repo",- Method: http.MethodPut,- Path: "/customize/git-repositories/{id}",- Middlewares: humamw.RequirePermission(api, authz.PermGitReposUpdate),- }, func(_ context.Context, _ *struct {- ID string `path:"id"`- }) (*struct{}, error) {- require.FailNow(t, "handler must not run when permission is missing")- return nil, nil- })- },- method: http.MethodPut,- path: "/api/customize/git-repositories/repo-1",- },- {- name: "delete repository",- register: func(api huma.API) {- huma.Register(api, huma.Operation{- OperationID: "test-delete-git-repo",- Method: http.MethodDelete,- Path: "/customize/git-repositories/{id}",- Middlewares: humamw.RequirePermission(api, authz.PermGitReposDelete),- }, func(_ context.Context, _ *struct {- ID string `path:"id"`- }) (*struct{}, error) {- require.FailNow(t, "handler must not run when permission is missing")- return nil, nil- })- },- method: http.MethodDelete,- path: "/api/customize/git-repositories/repo-1",- },- {- name: "test repository",- register: func(api huma.API) {- huma.Register(api, huma.Operation{- OperationID: "test-test-git-repo",- Method: http.MethodPost,- Path: "/customize/git-repositories/{id}/test",- Middlewares: humamw.RequirePermission(api, authz.PermGitReposTest),- }, func(_ context.Context, _ *struct {- ID string `path:"id"`- }) (*struct{}, error) {- require.FailNow(t, "handler must not run when permission is missing")- return nil, nil- })- },- method: http.MethodPost,- path: "/api/customize/git-repositories/repo-1/test",- },- }-- for _, tt := range tests {- t.Run(tt.name, func(t *testing.T) {- router := echo.New()- api := humaecho.NewWithGroup(router, router.Group("/api"), huma.DefaultConfig("test", "1.0.0"))-- // Attach an empty (deny-all) permission set, simulating an- // authenticated caller who lacks git-repository permissions.- api.UseMiddleware(func(ctx huma.Context, next func(huma.Context)) {- next(huma.WithContext(ctx, context.WithValue(ctx.Context(), humamw.ContextKeyUserPermissions, authz.NewPermissionSet())))- })-- tt.register(api)-- req := httptest.NewRequest(tt.method, tt.path, nil)- rec := httptest.NewRecorder()- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusForbidden, rec.Code)- require.Contains(t, rec.Body.String(), "permission denied")- })- }-}--// Compile-time assertion that GitRepositoryService is the type the production-// handler expects; keeps this test honest if the field is ever renamed.-var _ = &GitRepositoryHandler{repoService: &services.GitRepositoryService{}}diff --git a/backend/api/handlers/helpers.go b/backend/api/handlers/helpers.godeleted file mode 100644index 4610f8ff0..000000000--- a/backend/api/handlers/helpers.go+++ /dev/null@@ -1,194 +0,0 @@-package handlers--import (- "context"- "encoding/json/v2"- stderrors "errors"- "mime/multipart"- "strings"-- "emperror.dev/errors"-- "github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"- "github.com/getarcaneapp/arcane/backend/v2/pkg/remenv"- "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper"- "github.com/getarcaneapp/arcane/types/v2/base"-)--// ActivityAppContext carries the app lifecycle context through handler registration.-type ActivityAppContext struct {- ctx context.Context-}--// NewActivityAppContext wraps the app lifecycle context for handler constructors.-func NewActivityAppContext(ctx context.Context) ActivityAppContext {- return ActivityAppContext{ctx: ctx}-}--// ContextInternal returns the wrapped app lifecycle context.-func (c ActivityAppContext) ContextInternal() context.Context {- return c.contextInternal()-}--func (c ActivityAppContext) contextInternal() context.Context {- return c.ctx-}--// buildPaginationParamsInternal converts query parameters to pagination.QueryParams.-// A limit of -1 means "show all items" (no pagination).-func buildPaginationParamsInternal(start, limit int, sortCol, sortDir, search string) pagination.QueryParams {- // limit = -1 means "show all" and 0 means "no pagination"; only invalid values default to 20- if limit < -1 {- limit = 20- }-- return pagination.QueryParams{- SearchQuery: pagination.SearchQuery{- Search: strings.TrimSpace(search),- },- SortParams: pagination.SortParams{- Sort: strings.TrimSpace(sortCol),- Order: pagination.SortOrder(sortDir),- },- Params: pagination.Params{- Start: start,- Limit: limit,- },- Filters: make(map[string]string),- }-}--// toPaginationResponseInternal converts the pagination package's response into the API response shape.-func toPaginationResponseInternal(p pagination.Response) base.PaginationResponse {- return base.PaginationResponse{- TotalPages: p.TotalPages,- TotalItems: p.TotalItems,- CurrentPage: p.CurrentPage,- ItemsPerPage: p.ItemsPerPage,- GrandTotalItems: p.GrandTotalItems,- }-}--// requireUserInternal returns the authenticated user from the request context or a 401 error.-func requireUserInternal(ctx context.Context) (*models.User, error) {- user, exists := models.CurrentUserFromContext(ctx)- if !exists || user == nil {- return nil, huma.Error401Unauthorized("Not authenticated")- }- return user, nil-}--func openUploadedFileInternal(form multipart.Form) (multipart.File, *multipart.FileHeader, error) {- files := form.File["file"]- if len(files) == 0 {- return nil, nil, huma.Error400BadRequest("No file uploaded")- }-- fileHeader := files[0]- file, err := fileHeader.Open()- if err != nil {- return nil, nil, huma.Error500InternalServerError(errors.WithMessage(err, "Failed to read upload").Error())- }-- return file, fileHeader, nil-}--func registerSecuredInternal[I, O any](- api huma.API,- op huma.Operation,- permission string,- handler func(context.Context, *I) (*O, error),-) {- op.Security = defaultOperationSecurityInternal()- humamw.RegisterWithPermission(api, op, permission, handler)-}--func operationInternal(operationID, method, path, summary, description string, tags ...string) huma.Operation {- return huma.Operation{- OperationID: operationID,- Method: method,- Path: path,- Summary: summary,- Description: description,- Tags: tags,- }-}--func currentActorInternal(ctx context.Context) models.User {- actor := models.User{}- if currentUser, exists := models.CurrentUserFromContext(ctx); exists && currentUser != nil {- actor = *currentUser- }- return actor-}--func mapOneAPIResponseInternal[S any, D any](source S, mappingMessage func(error) string) (base.ApiResponse[D], error) {- out, err := mapper.MapOne[S, D](source)- if err != nil {- return base.ApiResponse[D]{}, huma.Error500InternalServerError(mappingMessage(err))- }-- return base.ApiResponse[D]{- Success: true,- Data: out,- }, nil-}--func defaultOperationSecurityInternal() []map[string][]string {- return []map[string][]string{- {"BearerAuth": {}},- {"ApiKeyAuth": {}},- }-}--func proxyRemoteJSONInternal[T any](- ctx context.Context,- environmentService *services.EnvironmentService,- environmentID string,- method string,- path string,- requestBody any,-) (*T, error) {- body, err := marshalRemoteRequestBodyInternal(requestBody)- if err != nil {- return nil, huma.Error500InternalServerError("failed to marshal request body: " + err.Error())- }-- var output T- if err := environmentService.ProxyJSONRequest(ctx, environmentID, method, path, body, &output); err != nil {- return nil, translateRemoteProxyErrorInternal(err)- }-- return &output, nil-}--func marshalRemoteRequestBodyInternal(requestBody any) ([]byte, error) {- switch value := requestBody.(type) {- case nil:- return nil, nil- case []byte:- return value, nil- default:- return json.Marshal(value)- }-}--func translateRemoteProxyErrorInternal(err error) error {- if transportErr, ok := stderrors.AsType[*remenv.TransportError](err); ok {- return huma.Error502BadGateway("failed to proxy request to environment: " + transportErr.Error())- }-- if statusErr, ok := stderrors.AsType[*remenv.StatusError](err); ok {- return huma.NewError(statusErr.StatusCode, "environment returned error: "+string(statusErr.Body), nil)- }-- if decodeErr, ok := stderrors.AsType[*remenv.DecodeError](err); ok {- return huma.Error500InternalServerError("failed to decode environment response: " + decodeErr.Error())- }-- return huma.Error500InternalServerError("failed to proxy request to environment: " + err.Error())-}diff --git a/backend/api/handlers/notifications_test.go b/backend/api/handlers/notifications_test.godeleted file mode 100644index dca1c427a..000000000--- a/backend/api/handlers/notifications_test.go+++ /dev/null@@ -1,235 +0,0 @@-package handlers--import (- "bytes"- "context"- "encoding/json"- "net/http"- "net/http/httptest"- "reflect"- "strings"- "testing"- "time"-- "github.com/danielgtaylor/huma/v2"- "github.com/danielgtaylor/huma/v2/adapters/humaecho"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/types/v2/base"- "github.com/getarcaneapp/arcane/types/v2/imageupdate"- notificationdto "github.com/getarcaneapp/arcane/types/v2/notification"- "github.com/labstack/echo/v5"- "github.com/libtnb/sqlite"- "github.com/stretchr/testify/assert"- "github.com/stretchr/testify/require"- "go.getarcane.app/sys/crypto"- "gorm.io/gorm"-)--func setupNotificationHandlerTestService(t *testing.T) (*database.DB, *services.NotificationService) {- t.Helper()-- db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, db.AutoMigrate(&models.NotificationSettings{}, &models.SettingVariable{}, &models.Environment{}))-- databaseDB := &database.DB{DB: db}- envSvc := services.NewEnvironmentService(databaseDB, nil, nil, nil, nil, nil)-- return databaseDB, services.NewNotificationService(databaseDB, &config.Config{}, envSvc, nil)-}--func TestIsSupportedNotificationTestType(t *testing.T) {- expected := []string{- "simple",- "image-update",- "batch-image-update",- "vulnerability-found",- "prune-report",- "auto-heal",- }-- for _, tt := range expected {- assert.True(t, isSupportedNotificationTestType(tt), "expected %q to be supported", tt)- }-- assert.False(t, isSupportedNotificationTestType("bogus"))- assert.False(t, isSupportedNotificationTestType(""))-}--func TestNormalizeNotificationTestType(t *testing.T) {- assert.Equal(t, "simple", normalizeNotificationTestType(""))- assert.Equal(t, "simple", normalizeNotificationTestType(" "))- assert.Equal(t, "auto-heal", normalizeNotificationTestType("auto-heal"))- assert.Equal(t, "auto-heal", normalizeNotificationTestType(" auto-heal "))-}--func TestNotificationHandlerGetAllNotificationSettingsRedactsCredentialsInternal(t *testing.T) {- ctx := context.Background()- crypto.InitEncryption(&crypto.Config{- EncryptionKey: "test-encryption-key-for-testing-32bytes-min",- Environment: "test",- })-- db, svc := setupNotificationHandlerTestService(t)- h := &NotificationHandler{- notificationService: svc,- config: &config.Config{},- }-- _, err := svc.CreateOrUpdateSettings(ctx, models.NotificationProviderDiscord, true, models.JSON{- "webhookId": "123456789",- "token": "discord-secret-token",- "username": "Arcane",- })- require.NoError(t, err)-- output, err := h.GetAllNotificationSettings(ctx, &GetAllNotificationSettingsInput{EnvironmentID: "0"})- require.NoError(t, err)- require.Len(t, output.Body, 1)- require.Equal(t, "123456789", output.Body[0].Config["webhookId"])- require.Equal(t, "Arcane", output.Body[0].Config["username"])- require.Empty(t, output.Body[0].Config["token"])-- var stored models.NotificationSettings- require.NoError(t, db.WithContext(ctx).Where("provider = ?", models.NotificationProviderDiscord).First(&stored).Error)- require.NotEqual(t, "discord-secret-token", stored.Config["token"])- decrypted, err := crypto.Decrypt(stored.Config["token"].(string))- require.NoError(t, err)- require.Equal(t, "discord-secret-token", decrypted)-}--func TestDispatchNotification_RejectsAgentModeInternal(t *testing.T) {- h := &NotificationHandler{config: &config.Config{AgentMode: true}}-- resp, err := h.DispatchNotification(context.Background(), &DispatchNotificationInput{})- require.Nil(t, resp)- require.Error(t, err)- assert.Contains(t, err.Error(), "notifications are managed on the Arcane manager")-}--func TestDispatchNotification_ReturnsBadRequestForUnsupportedDispatchKind(t *testing.T) {- ctx := context.Background()- db, svc := setupNotificationHandlerTestService(t)- h := &NotificationHandler{- notificationService: svc,- config: &config.Config{},- }-- token := "remote-token"- now := time.Now()- require.NoError(t, db.WithContext(ctx).Create(&models.Environment{- BaseModel: models.BaseModel{ID: "env-remote", CreatedAt: now, UpdatedAt: &now},- Name: "Remote Edge",- ApiUrl: "http://remote.example",- Enabled: true,- Status: string(models.EnvironmentStatusOnline),- AccessToken: &token,- }).Error)-- resp, err := h.DispatchNotification(ctx, &DispatchNotificationInput{- APIKey: token,- Body: notificationdto.DispatchRequest{- Kind: "bogus_kind",- },- })- require.Nil(t, resp)- require.Error(t, err)-- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- require.Equal(t, http.StatusBadRequest, statusErr.GetStatus())- require.Contains(t, statusErr.Error(), "unsupported dispatch kind")-}--func TestDispatchNotificationRoute_AuthenticatesEnvironmentAccessToken(t *testing.T) {- ctx := context.Background()- db, svc := setupNotificationHandlerTestService(t)- cfg := &config.Config{}- envSvc := services.NewEnvironmentService(db, nil, nil, nil, nil, nil)-- token := "remote-token"- now := time.Now()- require.NoError(t, db.WithContext(ctx).Create(&models.Environment{- BaseModel: models.BaseModel{ID: "env-remote", CreatedAt: now, UpdatedAt: &now},- Name: "Remote Edge",- ApiUrl: "http://remote.example",- Enabled: true,- Status: string(models.EnvironmentStatusOnline),- AccessToken: &token,- }).Error)-- router := echo.New()- humaConfig := huma.DefaultConfig("test", "1.0.0")- humaConfig.Components.SecuritySchemes = map[string]*huma.SecurityScheme{- "BearerAuth": {Type: "http", Scheme: "bearer"},- "ApiKeyAuth": {Type: "apiKey", In: "header", Name: "X-API-Key"},- }- humaConfig.Components.Schemas = huma.NewMapRegistry("#/components/schemas/", func(t reflect.Type, hint string) string {- for t.Kind() == reflect.Pointer {- t = t.Elem()- }- name := huma.DefaultSchemaNamer(t, hint)- if t.PkgPath() == "" {- return name- }- return strings.NewReplacer("/", "_", ".", "_").Replace(t.PkgPath()) + "_" + name- })- api := humaecho.NewWithGroup(router, router.Group("/api"), humaConfig)- api.UseMiddleware(humamw.NewAuthBridge(api, &services.AuthService{}, nil, nil, envSvc, cfg))- RegisterNotifications(api, svc, cfg)-- payload, err := json.Marshal(notificationdto.DispatchRequest{- Kind: notificationdto.DispatchKindImageUpdate,- ImageUpdate: ¬ificationdto.DispatchImageUpdate{- ImageRef: "nginx:latest",- UpdateInfo: imageupdate.Response{- HasUpdate: true,- UpdateType: "digest",- CurrentDigest: "sha256:current",- LatestDigest: "sha256:latest",- },- },- })- require.NoError(t, err)-- t.Run("accepts stored environment token", func(t *testing.T) {- req := httptest.NewRequest(http.MethodPost, "/api/notifications/dispatch", bytes.NewReader(payload))- req.Header.Set("Content-Type", "application/json")- req.Header.Set("X-API-Key", token)- rec := httptest.NewRecorder()-- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusOK, rec.Code, rec.Body.String())- var response base.ApiResponse[notificationdto.DispatchResponse]- require.NoError(t, json.NewDecoder(rec.Body).Decode(&response))- require.True(t, response.Success)- require.Equal(t, "Notification dispatched successfully", response.Data.Message)- require.Zero(t, response.Data.Delivered)- })-- for _, testCase := range []struct {- name string- token string- }{- {name: "rejects missing token"},- {name: "rejects unknown token", token: "unknown-token"},- } {- t.Run(testCase.name, func(t *testing.T) {- req := httptest.NewRequest(http.MethodPost, "/api/notifications/dispatch", bytes.NewReader(payload))- req.Header.Set("Content-Type", "application/json")- if testCase.token != "" {- req.Header.Set("X-API-Key", testCase.token)- }- rec := httptest.NewRecorder()-- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusUnauthorized, rec.Code, rec.Body.String())- })- }-}diff --git a/backend/api/handlers/permission_gating_test.go b/backend/api/handlers/permission_gating_test.godeleted file mode 100644index 92e6ebd50..000000000--- a/backend/api/handlers/permission_gating_test.go+++ /dev/null@@ -1,26 +0,0 @@-package handlers--import (- "context"- "testing"-- "github.com/danielgtaylor/huma/v2"- "github.com/danielgtaylor/huma/v2/adapters/humaecho"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/labstack/echo/v5"-)--func newPermissionGatingRouterInternal(t *testing.T, ps *authz.PermissionSet) (*echo.Echo, huma.API) {- t.Helper()- if ps == nil {- ps = authz.NewPermissionSet()- }-- router := echo.New()- api := humaecho.NewWithGroup(router, router.Group("/api"), huma.DefaultConfig("test", "1.0.0"))- api.UseMiddleware(func(ctx huma.Context, next func(huma.Context)) {- next(huma.WithContext(ctx, context.WithValue(ctx.Context(), humamw.ContextKeyUserPermissions, ps)))- })- return router, api-}diff --git a/backend/api/handlers/remenv_handlers_test.go b/backend/api/handlers/remenv_handlers_test.godeleted file mode 100644index 4bd379191..000000000--- a/backend/api/handlers/remenv_handlers_test.go+++ /dev/null@@ -1,221 +0,0 @@-package handlers--import (- "context"- "encoding/json"- "net/http"- "net/http/httptest"- "testing"- "time"-- "github.com/danielgtaylor/huma/v2"- humamiddleware "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/getarcaneapp/arcane/types/v2/base"- "github.com/getarcaneapp/arcane/types/v2/env"- "github.com/getarcaneapp/arcane/types/v2/jobschedule"- settingstypes "github.com/getarcaneapp/arcane/types/v2/settings"- "github.com/libtnb/sqlite"- "github.com/stretchr/testify/assert"- "github.com/stretchr/testify/require"- "gorm.io/gorm"-)--// adminTestContextInternal returns a context with a sudo PermissionSet attached,-// suitable for unit-testing handlers that gate via RequirePermission middleware.--func adminTestContextInternal() context.Context {- return context.WithValue(context.Background(), humamiddleware.ContextKeyUserPermissions, authz.SudoPermissionSet())-}--func setupRemoteHandlerEnvironmentServiceInternal(t *testing.T, server *httptest.Server) *services.EnvironmentService {- t.Helper()-- db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, db.AutoMigrate(&models.Environment{}))-- now := time.Now()- envRecord := &models.Environment{- BaseModel: models.BaseModel{- ID: "env-remote",- CreatedAt: now,- UpdatedAt: &now,- },- Name: "Remote Env",- ApiUrl: server.URL,- Status: string(models.EnvironmentStatusOnline),- Enabled: true,- IsEdge: false,- }- require.NoError(t, db.WithContext(context.Background()).Create(envRecord).Error)-- return services.NewEnvironmentService(&database.DB{DB: db}, server.Client(), nil, nil, nil, nil)-}--func TestProxyRemoteJSONInternal_MapsStatusError(t *testing.T) {- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- http.Error(w, "bad request from remote", http.StatusBadRequest)- }))- defer server.Close()-- envSvc := setupRemoteHandlerEnvironmentServiceInternal(t, server)-- _, err := proxyRemoteJSONInternal[jobschedule.JobListResponse](context.Background(), envSvc, "env-remote", http.MethodGet, "/api/environments/0/jobs", nil)- require.Error(t, err)-- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- require.Equal(t, http.StatusBadRequest, statusErr.GetStatus())- require.Contains(t, statusErr.Error(), "bad request from remote")-}--func TestProxyRemoteJSONInternal_MapsDecodeError(t *testing.T) {- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- w.WriteHeader(http.StatusOK)- _, _ = w.Write([]byte(`{"broken"`))- }))- defer server.Close()-- envSvc := setupRemoteHandlerEnvironmentServiceInternal(t, server)-- _, err := proxyRemoteJSONInternal[jobschedule.JobListResponse](context.Background(), envSvc, "env-remote", http.MethodGet, "/api/environments/0/jobs", nil)- require.Error(t, err)-- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- require.Equal(t, http.StatusInternalServerError, statusErr.GetStatus())- require.Contains(t, statusErr.Error(), "failed to decode environment response")-}--func TestJobSchedulesHandler_ListJobs_RemoteSuccess(t *testing.T) {- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- if !assert.Equal(t, "/api/environments/0/jobs", r.URL.Path) {- return- }- if !assert.NoError(t, json.NewEncoder(w).Encode(jobschedule.JobListResponse{- Jobs: []jobschedule.JobStatus{{ID: "job-1", Name: "Test Job"}},- })) {- return- }- }))- defer server.Close()-- handler := &JobSchedulesHandler{- jobService: &services.JobService{},- environmentService: setupRemoteHandlerEnvironmentServiceInternal(t, server),- }-- output, err := handler.ListJobs(context.Background(), &ListJobsInput{ID: "env-remote"})- require.NoError(t, err)- require.Len(t, output.Body.Jobs, 1)- require.Equal(t, "job-1", output.Body.Jobs[0].ID)-}--func TestSettingsHandler_GetPublicSettings_RemoteSuccess(t *testing.T) {- expected := []settingstypes.PublicSetting{{Key: "theme", Type: "string", Value: "dark"}}- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- if !assert.Equal(t, "/api/environments/0/settings/public", r.URL.Path) {- return- }- if !assert.NoError(t, json.NewEncoder(w).Encode(expected)) {- return- }- }))- defer server.Close()-- handler := &SettingsHandler{- settingsService: &services.SettingsService{},- environmentService: setupRemoteHandlerEnvironmentServiceInternal(t, server),- }-- output, err := handler.GetPublicSettings(context.Background(), &GetPublicSettingsInput{EnvironmentID: "env-remote"})- require.NoError(t, err)- require.Equal(t, expected, output.Body)-}--func TestSettingsHandler_GetSettings_RemoteFiltersNonAdminVisibility(t *testing.T) {- remoteSettings := []settingstypes.PublicSetting{- {Key: "dockerHost", Type: "string", Value: "unix:///var/run/docker.sock"},- {Key: "baseServerUrl", Type: "string", Value: "https://manager.example"},- {Key: "defaultShell", Type: "string", Value: "/bin/bash"},- {Key: "futureAdminSetting", Type: "string", Value: "hidden"},- }- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- if !assert.Equal(t, "/api/environments/0/settings", r.URL.Path) {- return- }- if !assert.NoError(t, json.NewEncoder(w).Encode(remoteSettings)) {- return- }- }))- defer server.Close()-- db := setupActivityHandlerTestDBInternal(t)- settingsService, err := newSettingsServiceForTestInternal(t, context.Background(), db)- require.NoError(t, err)- handler := &SettingsHandler{- settingsService: settingsService,- environmentService: setupRemoteHandlerEnvironmentServiceInternal(t, server),- }- ps := authz.NewPermissionSet()- ps.AddEnv("env-remote", authz.PermSettingsRead)- ctx := context.WithValue(context.Background(), humamiddleware.ContextKeyUserPermissions, ps)-- output, err := handler.GetSettings(ctx, &GetSettingsInput{EnvironmentID: "env-remote"})- require.NoError(t, err)- require.Equal(t, []settingstypes.PublicSetting{remoteSettings[0]}, output.Body)-}--func TestSettingsHandler_GetSettings_RemotePreservesAdminResponse(t *testing.T) {- remoteSettings := []settingstypes.PublicSetting{- {Key: "baseServerUrl", Type: "string", Value: "https://manager.example"},- {Key: "futureAdminSetting", Type: "string", Value: "visible-to-admin"},- }- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {- if !assert.NoError(t, json.NewEncoder(w).Encode(remoteSettings)) {- return- }- }))- defer server.Close()-- db := setupActivityHandlerTestDBInternal(t)- settingsService, err := newSettingsServiceForTestInternal(t, context.Background(), db)- require.NoError(t, err)- handler := &SettingsHandler{- settingsService: settingsService,- environmentService: setupRemoteHandlerEnvironmentServiceInternal(t, server),- }-- output, err := handler.GetSettings(adminTestContextInternal(), &GetSettingsInput{EnvironmentID: "env-remote"})- require.NoError(t, err)- require.Equal(t, remoteSettings, output.Body)-}--func TestVariableHandler_GetMaterializedVariables_RemoteSuccess(t *testing.T) {- expected := base.ApiResponse[[]env.Variable]{- Success: true,- Data: []env.Variable{{Key: "FOO", Value: "bar"}},- }- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- if !assert.Equal(t, "/api/environments/0/templates/variables", r.URL.Path) {- return- }- if !assert.NoError(t, json.NewEncoder(w).Encode(expected)) {- return- }- }))- defer server.Close()-- handler := &VariableHandler{- variableService: &services.VariableService{},- environmentService: setupRemoteHandlerEnvironmentServiceInternal(t, server),- }-- output, err := handler.GetMaterializedVariables(adminTestContextInternal(), &GetGlobalVariablesInput{EnvironmentID: "env-remote"})- require.NoError(t, err)- require.Equal(t, expected, output.Body)-}diff --git a/backend/api/handlers/roles_access_policy_test.go b/backend/api/handlers/roles_access_policy_test.godeleted file mode 100644index be2b6b3d2..000000000--- a/backend/api/handlers/roles_access_policy_test.go+++ /dev/null@@ -1,83 +0,0 @@-package handlers--import (- "testing"-- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/stretchr/testify/require"-)--func TestBuildPermissionsManifestIncludesAccessSurfaces(t *testing.T) {- manifest := buildPermissionsManifestInternal()-- require.NotEmpty(t, manifest.Resources)- require.NotEmpty(t, manifest.AccessSurfaces)-- surfacesByID := make(map[string]struct{}, len(manifest.AccessSurfaces))- for _, surface := range manifest.AccessSurfaces {- surfacesByID[surface.ID] = struct{}{}- }-- for _, id := range []string{- "landing.settings",- "landing.customize",- "settings.category.webhooks",- "customize.category.templates",- "route.dashboard",- } {- _, ok := surfacesByID[id]- require.True(t, ok, "expected manifest to include access surface %s", id)- }-- registrySurfaces := authz.AccessSurfaces()- require.Len(t, manifest.AccessSurfaces, len(registrySurfaces))-}--func TestBuildPermissionsManifestIncludesEventsDelete(t *testing.T) {- manifest := buildPermissionsManifestInternal()-- for _, resource := range manifest.Resources {- if resource.Key != "events" {- continue- }- for _, action := range resource.Actions {- if action.Permission == authz.PermEventsDelete {- return- }- }- require.FailNowf(t, "unexpected failure", "events resource did not include %s", authz.PermEventsDelete)- }- require.FailNow(t, "events resource not found")-}--func TestBuildPermissionsManifestKeepsVariablesSeparateFromTemplates(t *testing.T) {- manifest := buildPermissionsManifestInternal()-- var variablePermissions []string- for _, resource := range manifest.Resources {- if resource.Key != "variables" {- continue- }- for _, action := range resource.Actions {- variablePermissions = append(variablePermissions, action.Permission)- }- }- require.ElementsMatch(t, []string{- authz.PermVariablesRead,- authz.PermVariablesCreate,- authz.PermVariablesUpdate,- authz.PermVariablesDelete,- authz.PermVariablesSync,- }, variablePermissions)- for _, permission := range variablePermissions {- require.NotEqual(t, authz.PermTemplatesRead, permission)- }-- surfaces := make(map[string][]string, len(manifest.AccessSurfaces))- for _, surface := range manifest.AccessSurfaces {- surfaces[surface.ID] = surface.Permissions- }- require.Equal(t, []string{authz.PermVariablesRead}, surfaces["customize.category.variables"])- require.Equal(t, []string{authz.PermActivitiesRead}, surfaces["route.activities"])- require.ElementsMatch(t, authz.AllPermissions(), surfaces["route.oidc-role-mappings"])-}diff --git a/backend/api/handlers/settings_actor_test.go b/backend/api/handlers/settings_actor_test.godeleted file mode 100644index a06452800..000000000--- a/backend/api/handlers/settings_actor_test.go+++ /dev/null@@ -1,32 +0,0 @@-package handlers--import (- "context"- "testing"- "time"-- "github.com/getarcaneapp/arcane/backend/v2/internal/actors"- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/stretchr/testify/require"- "go.uber.org/fx/fxtest"-)--func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*services.SettingsService, error) {- t.Helper()- lifecycle := fxtest.NewLifecycle(t)- runtime, err := actors.NewRuntime(t.Context(), lifecycle)- require.NoError(t, err)- executor, err := actors.NewExecutor(t.Context(), runtime, "settings-test", t.Name(), 3)- require.NoError(t, err)- effects, err := actors.NewExecutor(t.Context(), runtime, "settings-effects-test", t.Name(), 3)- require.NoError(t, err)- t.Cleanup(func() {- stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)- defer cancel()- require.NoError(t, executor.Stop(stopCtx))- require.NoError(t, effects.Stop(stopCtx))- require.NoError(t, lifecycle.Stop(stopCtx))- })- return services.NewSettingsService(ctx, db, executor, effects)-}diff --git a/backend/api/handlers/settings_test.go b/backend/api/handlers/settings_test.godeleted file mode 100644index d1ace33ae..000000000--- a/backend/api/handlers/settings_test.go+++ /dev/null@@ -1,106 +0,0 @@-package handlers--import (- "context"- "os"- "path/filepath"- "runtime"- "testing"-- "github.com/danielgtaylor/huma/v2"- "github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge"- apitypes "github.com/getarcaneapp/arcane/types/v2/settings"- "github.com/libtnb/sqlite"- "github.com/stretchr/testify/require"- "gorm.io/gorm"-)--func TestSettingsHandler_AppendRuntimeSettings(t *testing.T) {- handler := &SettingsHandler{- cfg: &config.Config{- UIConfigurationDisabled: true,- BackupVolumeName: "custom-backups",- },- }-- publicSettings := handler.appendRuntimeSettingsInternal(nil, false)- publicKeys := runtimeSettingKeysInternal(publicSettings)- require.NotContains(t, publicKeys, "uiConfigDisabled")- require.NotContains(t, publicKeys, "backupVolumeName")- require.NotContains(t, publicKeys, "depotConfigured")- require.NotContains(t, publicKeys, "edgeMTLSManagerCAAvailable")-- authenticatedSettings := handler.appendRuntimeSettingsInternal(nil, true)- authenticatedKeys := runtimeSettingKeysInternal(authenticatedSettings)- require.Contains(t, authenticatedKeys, "uiConfigDisabled")- require.Contains(t, authenticatedKeys, "backupVolumeName")- require.Contains(t, authenticatedKeys, "edgeMTLSManagerCAAvailable")- require.Equal(t, "true", authenticatedKeys["uiConfigDisabled"])- require.Equal(t, "custom-backups", authenticatedKeys["backupVolumeName"])- require.Equal(t, "false", authenticatedKeys["edgeMTLSManagerCAAvailable"])-}--func TestSettingsHandler_AppendRuntimeSettings_DoesNotGenerateEdgeMTLSCA(t *testing.T) {- assetsDir := t.TempDir()- handler := &SettingsHandler{- cfg: &config.Config{- EdgeMTLSMode: edge.EdgeMTLSModeRequired,- EdgeMTLSAssetsDir: assetsDir,- },- }-- authenticatedSettings := handler.appendRuntimeSettingsInternal(nil, true)- authenticatedKeys := runtimeSettingKeysInternal(authenticatedSettings)-- require.Equal(t, "false", authenticatedKeys["edgeMTLSManagerCAAvailable"])- require.NoFileExists(t, filepath.Join(assetsDir, "ca.crt"))- require.NoFileExists(t, filepath.Join(assetsDir, "ca.key"))-}--func TestSettingsHandler_UpdateLocalEnvironment_RejectsUnreadableProjectsDirectory(t *testing.T) {- if runtime.GOOS == "windows" {- t.Skip("POSIX permission-denied behavior is not portable to Windows")- }- if os.Geteuid() == 0 {- t.Skip("test requires a non-root UID to trigger permission-denied on ReadDir")- }-- ctx := context.Background()-- db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, db.AutoMigrate(&models.SettingVariable{}))-- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, &database.DB{DB: db})- require.NoError(t, err)-- originalDir := settingsSvc.GetSettingsConfig().ProjectsDirectory.Value-- unreadable := t.TempDir()- require.NoError(t, os.Chmod(unreadable, 0))- t.Cleanup(func() { _ = os.Chmod(unreadable, 0o700) })-- handler := &SettingsHandler{settingsService: settingsSvc, cfg: &config.Config{}}-- _, err = handler.updateSettingsForLocalEnvironment(ctx, apitypes.Update{ProjectsDirectory: new(unreadable)})- require.Error(t, err)-- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr, "expected a huma status error")- require.Equal(t, 400, statusErr.GetStatus())- require.Contains(t, err.Error(), "cannot read projects directory", "error message should explain the failure: %s", err.Error())-- require.Equal(t, originalDir, settingsSvc.GetSettingsConfig().ProjectsDirectory.Value, "projectsDirectory must not be persisted on validation failure")-}--func runtimeSettingKeysInternal(settings []apitypes.PublicSetting) map[string]string {- keys := make(map[string]string, len(settings))- for _, setting := range settings {- keys[setting.Key] = setting.Value- }-- return keys-}diff --git a/backend/api/handlers/settings_validation_test.go b/backend/api/handlers/settings_validation_test.godeleted file mode 100644index b4d415458..000000000--- a/backend/api/handlers/settings_validation_test.go+++ /dev/null@@ -1,104 +0,0 @@-package handlers--import (- "strings"- "testing"-- "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"- "github.com/stretchr/testify/assert"-)--func TestProjectsDirectoryValidation(t *testing.T) {- tests := []struct {- name string- projectsDir string- expectValid bool- }{- {- name: "valid absolute path",- projectsDir: "/app/data/projects",- expectValid: true,- },- {- name: "valid absolute path root",- projectsDir: "/projects",- expectValid: true,- },- {- name: "invalid relative path",- projectsDir: "data/projects",- expectValid: false,- },- {- name: "invalid relative path with dot",- projectsDir: "./data/projects",- expectValid: false,- },- {- name: "invalid relative path parent",- projectsDir: "../projects",- expectValid: false,- },- {- name: "valid mapping format",- projectsDir: "/app/data/projects:D:/host/path",- expectValid: true,- },- {- name: "valid Windows drive path",- projectsDir: "C:/projects",- expectValid: true,- },- {- name: "valid Windows drive path backslashes",- projectsDir: "C:\\projects",- expectValid: true,- },- {- name: "valid mapping with Windows backslashes",- projectsDir: "/app/data/projects:D:\\host\\path",- expectValid: true,- },- {- name: "valid mapping with Windows container path",- projectsDir: "C:/container:D:/host",- expectValid: true,- },- {- name: "invalid single colon not Windows",- projectsDir: "invalid:path",- expectValid: false,- },- }-- for _, tt := range tests {- t.Run(tt.name, func(t *testing.T) {- // Replicate the validation logic from UpdateSettings handler- path := tt.projectsDir- isValid := true-- switch {- case projects.IsWindowsDrivePath(path):- // Valid Windows path- case strings.Contains(path, ":"):- // Mapping format (container:host)- parts := strings.SplitN(path, ":", 2)- if len(parts) != 2 {- isValid = false- break- }- container := parts[0]- if !strings.HasPrefix(container, "/") && !projects.IsWindowsDrivePath(container) {- isValid = false- }- default:- if !strings.HasPrefix(path, "/") {- // No colon and doesn't start with / - must be relative- isValid = false- }- }-- assert.Equal(t, tt.expectValid, isValid, "path: %s", tt.projectsDir)- })- }-}diff --git a/backend/api/handlers/stream.go b/backend/api/handlers/stream.goindex ee1f6f8bf..f16fa2e77 100644--- a/backend/api/handlers/stream.go+++ b/backend/api/handlers/stream.go@@ -9,10 +9,13 @@ import ("time""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/dashboard"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- httputils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"activitytypes "github.com/getarcaneapp/arcane/types/v2/activity"dashboardtypes "github.com/getarcaneapp/arcane/types/v2/dashboard"environmenttypes "github.com/getarcaneapp/arcane/types/v2/environment"@@ -33,9 +36,9 @@ const (// StreamHandler multiplexes every live feed onto one connection.type StreamHandler struct {- dashboard *DashboardHandler- activity *ActivityHandler- environment *EnvironmentHandler+ dashboard *dashboard.DashboardHandler+ activity *activity.ActivityHandler+ environment *environment.EnvironmentHandler}type StreamClientInput struct {@@ -49,22 +52,14 @@ type StreamClientInput struct {// each channel's producer stays the single implementation of that feed.func RegisterStream(api huma.API,- dashboardService *services.DashboardService,- activityService *services.ActivityService,- environmentService *services.EnvironmentService,+ dashboardHandler *dashboard.DashboardHandler,+ activityHandler *activity.ActivityHandler,+ environmentHandler *environment.EnvironmentHandler,) {h := &StreamHandler{- dashboard: &DashboardHandler{- dashboardService: dashboardService,- environmentService: environmentService,- },- activity: &ActivityHandler{- activityService: activityService,- environmentService: environmentService,- },- // Only the environment service is needed: the channel producer lists- // environments and derives liveness, nothing more.- environment: &EnvironmentHandler{environmentService: environmentService},+ dashboard: dashboardHandler,+ activity: activityHandler,+ environment: environmentHandler,}huma.Register(api, huma.Operation{@@ -74,7 +69,7 @@ func RegisterStream(Summary: "Multiplexed client stream",Description: "Streams the requested channels (environments, dashboard, activities) over a single JSON-lines connection",Tags: []string{"Stream"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),// Ungated: each channel applies its own permission check below, and a// caller with access to none simply gets a heartbeat-only stream.}, h.StreamClient)@@ -83,7 +78,7 @@ func RegisterStream(func (h *StreamHandler) StreamClient(ctx context.Context, input *StreamClientInput) (*huma.StreamResponse, error) {return &huma.StreamResponse{Body: func(humaCtx huma.Context) { //nolint:contextcheck // streaming work must use humaCtx.Context()- httputils.SetJSONStreamHeaders(humaCtx)+ httpx.SetJSONStreamHeaders(humaCtx)writer := humaCtx.BodyWriter()flush := func() {@@ -92,14 +87,14 @@ func (h *StreamHandler) StreamClient(ctx context.Context, input *StreamClientInp}}- ps, _ := humamw.PermissionsFromContext(humaCtx.Context())+ ps, _ := middleware.PermissionsFromContext(humaCtx.Context())h.streamClientInternal(humaCtx.Context(), ps, input, writer, flush)},}, nil}func (h *StreamHandler) streamClientInternal(ctx context.Context, ps *authz.PermissionSet, input *StreamClientInput, writer io.Writer, flush func()) {- releaseDeadPeerTimeout, timeoutErr := httputils.AcquireDeadPeerTimeout(ctx, clientStreamDeadPeerTimeout)+ releaseDeadPeerTimeout, timeoutErr := httpx.AcquireDeadPeerTimeout(ctx, clientStreamDeadPeerTimeout)if timeoutErr != nil {slog.DebugContext(ctx, "could not bound client stream dead-peer timeout", "error", timeoutErr)}@@ -133,7 +128,7 @@ func (h *StreamHandler) producersForInternal(ps *authz.PermissionSet, input *Strreturn streamtypes.Event{Channel: channel, Environment: &event, Timestamp: event.Timestamp}},func(producerCtx context.Context, events chan<- environmenttypes.StreamEvent) {- h.environment.runEnvironmentStreamProducerInternal(producerCtx, ps, events)+ h.environment.RunStreamProducer(producerCtx, ps, events)},))}@@ -142,11 +137,11 @@ func (h *StreamHandler) producersForInternal(ps *authz.PermissionSet, input *StrwrapDashboard := func(channel string, event dashboardtypes.StreamEvent) streamtypes.Event {return streamtypes.Event{Channel: channel, Dashboard: &event, Timestamp: event.Timestamp}}- if ps.Allows(authz.PermDashboardRead, localDockerEnvironmentID) {+ if ps.Allows(authz.PermDashboardRead, environment.LocalEnvironmentID) {producers = append(producers, forwardStreamChannelInternal(streamtypes.ChannelDashboard, wrapDashboard,func(producerCtx context.Context, events chan<- dashboardtypes.StreamEvent) {- h.dashboard.runLocalDashboardStreamProducerInternal(producerCtx, input.DebugAllGood, events)+ h.dashboard.RunLocalStreamProducer(producerCtx, input.DebugAllGood, events)},))}@@ -154,7 +149,7 @@ func (h *StreamHandler) producersForInternal(ps *authz.PermissionSet, input *Strproducers = append(producers, forwardStreamChannelInternal(streamtypes.ChannelDashboard, wrapDashboard,func(producerCtx context.Context, events chan<- dashboardtypes.StreamEvent) {- h.dashboard.runRemoteDashboardStreamPollersInternal(producerCtx, ps, input.DebugAllGood, events)+ h.dashboard.RunRemoteStreamPollers(producerCtx, ps, input.DebugAllGood, events)},))}@@ -164,11 +159,11 @@ func (h *StreamHandler) producersForInternal(ps *authz.PermissionSet, input *StrwrapActivity := func(channel string, event activitytypes.StreamEvent) streamtypes.Event {return streamtypes.Event{Channel: channel, Activity: &event, Timestamp: event.Timestamp}}- if ps.Allows(authz.PermActivitiesRead, localDockerEnvironmentID) {+ if ps.Allows(authz.PermActivitiesRead, environment.LocalEnvironmentID) {producers = append(producers, forwardStreamChannelInternal(streamtypes.ChannelActivities, wrapActivity,func(producerCtx context.Context, events chan<- activitytypes.StreamEvent) {- h.activity.runLocalActivityStreamProducerInternal(producerCtx, input.Limit, events)+ h.activity.RunLocalStreamProducer(producerCtx, input.Limit, events)},))}@@ -176,7 +171,7 @@ func (h *StreamHandler) producersForInternal(ps *authz.PermissionSet, input *Strproducers = append(producers, forwardStreamChannelInternal(streamtypes.ChannelActivities, wrapActivity,func(producerCtx context.Context, events chan<- activitytypes.StreamEvent) {- h.activity.runRemoteActivityStreamPollersInternal(producerCtx, ps, input.Limit, events)+ h.activity.RunRemoteStreamPollers(producerCtx, ps, input.Limit, events)},))}diff --git a/backend/api/handlers/stream_test.go b/backend/api/handlers/stream_test.godeleted file mode 100644index 4e87ef0a4..000000000--- a/backend/api/handlers/stream_test.go+++ /dev/null@@ -1,83 +0,0 @@-package handlers--import (- "bufio"- "context"- "encoding/json"- "io"- "testing"- "time"-- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- streamtypes "github.com/getarcaneapp/arcane/types/v2/stream"- "github.com/stretchr/testify/require"-)--// The environments channel must report an environment the manager cannot reach.-//-// This is the property the whole live-status fix rests on, and the one neither-// of the other channels can provide: the dashboard channel only emits when it-// gets a snapshot back from an agent, so an environment that is down produces-// nothing. Without a pushed status an offline agent stays offline in the UI-// forever, which is exactly what happens after a fleet update.-func TestStreamHandlerEnvironmentsChannelEmitsUnreachableEnvironmentInternal(t *testing.T) {- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)- defer cancel()-- db := setupActivityHandlerTestDBInternal(t)- limitStreamTestDBToSingleConnInternal(t, db)-- // An edge environment with no tunnel registered: unreachable by construction.- createStreamTestRemoteEnvironmentInternal(t, db, "remote-1", "Remote", "edge://remote-1", "token")- require.NoError(t, db.Exec(`UPDATE environments SET is_edge = ? WHERE id = ?`, true, "remote-1").Error)-- handler := &StreamHandler{- environment: &EnvironmentHandler{- environmentService: services.NewEnvironmentService(db, nil, nil, nil, nil, nil),- },- }- ps := authz.NewPermissionSet()- ps.Sudo = true-- pr, pw := io.Pipe()- done := make(chan struct{})- go func() {- defer close(done)- defer func() { _ = pw.Close() }()- handler.streamClientInternal(ctx, ps, &StreamClientInput{Channels: streamtypes.ChannelEnvironments}, pw, func() {})- }()-- var seen *streamtypes.Event- scanner := bufio.NewScanner(pr)- for scanner.Scan() {- var envelope streamtypes.Event- require.NoError(t, json.Unmarshal(scanner.Bytes(), &envelope))- if envelope.Environment == nil {- continue- }- seen = &envelope- cancel()- break- }-- go func() { _, _ = io.Copy(io.Discard, pr) }()- select {- case <-done:- case <-time.After(5 * time.Second):- require.FailNow(t, "stream did not terminate after cancel")- }-- require.NotNil(t, seen, "expected an environments snapshot for an unreachable environment")- require.Equal(t, streamtypes.ChannelEnvironments, seen.Channel)-- var remote *string- for _, env := range seen.Environment.Environments {- if env.ID == "remote-1" {- status := env.Status- remote = &status- }- }- require.NotNil(t, remote, "unreachable environment must still appear in the snapshot")- require.Equal(t, "offline", *remote)-}diff --git a/backend/api/handlers/swarm_easy_join_permissions_test.go b/backend/api/handlers/swarm_easy_join_permissions_test.godeleted file mode 100644index fb015e51d..000000000--- a/backend/api/handlers/swarm_easy_join_permissions_test.go+++ /dev/null@@ -1,44 +0,0 @@-package handlers--import (- "context"- "testing"-- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- swarmtypes "github.com/getarcaneapp/arcane/types/v2/swarm"- "github.com/stretchr/testify/require"-)--func TestRequireEasyJoinManagerPermissionsUsesSwarmJoinOnly(t *testing.T) {- legacyPermissions := authz.NewPermissionSet()- legacyPermissions.AddEnv("manager", authz.PermSwarmNodes, authz.PermSwarmUnlock)- legacyContext := context.WithValue(context.Background(), humamw.ContextKeyUserPermissions, legacyPermissions)- require.Error(t, requireEasyJoinManagerPermissionsInternal(legacyContext, "manager"))-- joinPermissions := authz.NewPermissionSet()- joinPermissions.AddEnv("manager", authz.PermSwarmJoin)- joinContext := context.WithValue(context.Background(), humamw.ContextKeyUserPermissions, joinPermissions)- require.NoError(t, requireEasyJoinManagerPermissionsInternal(joinContext, "manager"))-}--func TestJoinEnvironmentsRequiresSwarmJoinOnEveryTarget(t *testing.T) {- permissions := authz.NewPermissionSet()- permissions.AddEnv("manager", authz.PermSwarmJoin)- ctx := context.WithValue(context.Background(), humamw.ContextKeyUserPermissions, permissions)- handler := &SwarmHandler{swarmService: &services.SwarmService{}}-- _, err := handler.JoinEnvironments(ctx, &JoinSwarmEnvironmentsInput{- EnvironmentID: "manager",- Body: swarmtypes.SwarmJoinEnvironmentsRequest{- RemoteAddrs: []string{"manager:2377"},- Targets: []swarmtypes.SwarmJoinEnvironmentTarget{- {EnvironmentID: "target", Role: swarmtypes.SwarmJoinEnvironmentRoleWorker},- },- },- })-- require.Error(t, err)- require.Contains(t, err.Error(), "swarm:join permission is required for every target environment")-}diff --git a/backend/api/handlers/system_test.go b/backend/api/handlers/system_test.godeleted file mode 100644index c586773f0..000000000--- a/backend/api/handlers/system_test.go+++ /dev/null@@ -1,79 +0,0 @@-package handlers--import (- "context"- "strings"- "testing"-- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/types/v2/system"- "github.com/stretchr/testify/require"-)--func TestSystemHandlerConvertDockerRunUsesAlignedYAMLFormat(t *testing.T) {- handler := &SystemHandler{systemService: &services.SystemService{}}-- output, err := handler.ConvertDockerRun(context.Background(), &ConvertDockerRunInput{- Body: system.ConvertDockerRunRequest{- DockerRunCommand: "docker run --name web -p 8080:80 -v data:/data -e FOO=bar -e BAZ=qux " +- "--restart unless-stopped -w /srv/app -u 1000:1000 --entrypoint /entrypoint.sh " +- "-it --privileged --label com.example.role=frontend " +- "--health-cmd 'curl -f http://localhost || exit 1' -m 512m --cpus 0.5 " +- "nginx:1.27-alpine nginx -g 'daemon off;'",- },- })- require.NoError(t, err)-- require.True(t, output.Body.Success)- require.Equal(t, "web", output.Body.ServiceName)- require.Equal(t, "FOO=bar\nBAZ=qux", output.Body.EnvVars)- require.Equal(t, strings.TrimPrefix(`-services:- web:- image: nginx:1.27-alpine- container_name: web- ports:- - 8080:80- volumes:- - data:/data- environment:- - FOO=bar- - BAZ=qux- restart: unless-stopped- working_dir: /srv/app- user: 1000:1000- entrypoint: /entrypoint.sh- command: nginx -g 'daemon off;'- stdin_open: true- tty: true- privileged: true- labels:- - com.example.role=frontend- healthcheck:- test: curl -f http://localhost || exit 1- deploy:- resources:- limits:- memory: 512m- cpus: "0.5"-volumes:- data:- external: true-`, "\n"), output.Body.DockerCompose)-}--func TestSystemHandlerConvertDockerRunSupportsEnvFileAndUlimit(t *testing.T) {- handler := &SystemHandler{systemService: &services.SystemService{}}-- output, err := handler.ConvertDockerRun(context.Background(), &ConvertDockerRunInput{- Body: system.ConvertDockerRunRequest{- DockerRunCommand: "docker run --name worker --env-file .env --ulimit nofile=1024:2048 alpine:3.20 sleep 30",- },- })- require.NoError(t, err)-- require.Equal(t, "worker", output.Body.ServiceName)- require.Empty(t, output.Body.EnvVars)- require.Contains(t, output.Body.DockerCompose, "env_file:\n - .env")- require.Contains(t, output.Body.DockerCompose, "ulimits:\n nofile:\n soft: 1024\n hard: 2048")-}diff --git a/backend/api/handlers/templates_test.go b/backend/api/handlers/templates_test.godeleted file mode 100644index f137cf741..000000000--- a/backend/api/handlers/templates_test.go+++ /dev/null@@ -1,228 +0,0 @@-package handlers--import (- "context"- "net"- "net/http"- "net/http/httptest"- "net/url"- "os"- "testing"- "time"-- "github.com/danielgtaylor/huma/v2"- "github.com/danielgtaylor/huma/v2/adapters/humaecho"- humamiddleware "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/golang-jwt/jwt/v5"- "github.com/labstack/echo/v5"- "github.com/libtnb/sqlite"- "github.com/stretchr/testify/require"- "gorm.io/gorm"-)--func TestFetchTemplateRegistryRequiresAuthentication(t *testing.T) {- router := newTemplateFetchTestRouter(t, http.DefaultClient)-- req := httptest.NewRequest(http.MethodGet, "/api/templates/fetch?url=http://example.com/registry.json", nil)- rec := httptest.NewRecorder()-- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusUnauthorized, rec.Code)- require.Contains(t, rec.Body.String(), "Unauthorized")-}--func TestFetchTemplateRegistryAuthenticatedSuccess(t *testing.T) {- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- w.Header().Set("Content-Type", "application/json")- _, _ = w.Write([]byte(`{"name":"Registry","description":"Demo","version":"1.0.0","author":"Arcane","templates":[]}`))- }))- defer server.Close()-- client, registryURL := makeTemplateFetchRemote(t, server)- router := newTemplateFetchTestRouter(t, client)-- req := httptest.NewRequest(http.MethodGet, "/api/templates/fetch?url="+url.QueryEscape(registryURL), nil)- req.Header.Set("Authorization", "Bearer "+makeTemplateFetchToken(t, "test-secret", "user-1", "alice"))- rec := httptest.NewRecorder()-- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusOK, rec.Code)- require.Contains(t, rec.Body.String(), `"success":true`)- require.Contains(t, rec.Body.String(), `"name":"Registry"`)-}--func TestFetchTemplateRegistryAuthenticatedUnsafeTargetIsSanitized(t *testing.T) {- router := newTemplateFetchTestRouter(t, http.DefaultClient)-- req := httptest.NewRequest(http.MethodGet, "/api/templates/fetch?url="+url.QueryEscape("http://127.0.0.1:8080/registry.json"), nil)- req.Header.Set("Authorization", "Bearer "+makeTemplateFetchToken(t, "test-secret", "user-1", "alice"))- rec := httptest.NewRecorder()-- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusBadGateway, rec.Code)- require.Contains(t, rec.Body.String(), "Failed to fetch registry")- require.NotContains(t, rec.Body.String(), "127.0.0.1")- require.NotContains(t, rec.Body.String(), "connection refused")- require.NotContains(t, rec.Body.String(), "Invalid JSON response")-}--func TestTemplateReadEndpointsRequireAuthentication(t *testing.T) {- router := newTemplateFetchTestRouter(t, http.DefaultClient)-- testCases := []struct {- name string- path string- }{- {name: "list templates", path: "/api/templates"},- {name: "list all templates", path: "/api/templates/all"},- {name: "get template", path: "/api/templates/test-template"},- {name: "get template content", path: "/api/templates/test-template/content"},- }-- for _, testCase := range testCases {- t.Run(testCase.name, func(t *testing.T) {- req := httptest.NewRequest(http.MethodGet, testCase.path, nil)- rec := httptest.NewRecorder()-- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusUnauthorized, rec.Code)- require.Contains(t, rec.Body.String(), "Unauthorized")- })- }-}--func TestHealthEndpointRemainsPublic(t *testing.T) {- router := newTemplateFetchTestRouter(t, http.DefaultClient)-- req := httptest.NewRequest(http.MethodGet, "/api/health", nil)- rec := httptest.NewRecorder()-- router.ServeHTTP(rec, req)-- require.Equal(t, http.StatusOK, rec.Code)- require.Contains(t, rec.Body.String(), "UP")-}--func newTemplateFetchTestRouter(t *testing.T, httpClient *http.Client) *echo.Echo {- t.Helper()-- originalDir, err := os.Getwd()- require.NoError(t, err)- require.NoError(t, os.Chdir(t.TempDir()))- t.Cleanup(func() {- require.NoError(t, os.Chdir(originalDir))- })-- db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, db.AutoMigrate(&models.User{}, &models.UserSession{}))-- databaseDB := &database.DB{DB: db}- userService := services.NewUserService(databaseDB)- _, err = userService.CreateUser(context.Background(), &models.User{- BaseModel: models.BaseModel{ID: "user-1"},- Username: "alice",- })- require.NoError(t, err)- require.NoError(t, db.Create(&models.UserSession{- BaseModel: models.BaseModel{ID: "session-1"},- UserID: "user-1",- RefreshTokenHash: "template-test-refresh-hash",- LastUsedAt: time.Now(),- ExpiresAt: time.Now().Add(time.Hour),- }).Error)-- authService := services.NewAuthService(userService, nil, nil, services.NewSessionService(databaseDB), nil, "test-secret", &config.Config{}, nil)- templateService := services.NewTemplateService(context.Background(), nil, httpClient, nil)-- router := echo.New()- apiGroup := router.Group("/api")-- humaConfig := huma.DefaultConfig("test", "1.0.0")- humaConfig.Components.SecuritySchemes = map[string]*huma.SecurityScheme{- "BearerAuth": {- Type: "http",- Scheme: "bearer",- },- "ApiKeyAuth": {- Type: "apiKey",- In: "header",- Name: "X-API-Key",- },- }- humaConfig.Security = []map[string][]string{- {"BearerAuth": {}},- {"ApiKeyAuth": {}},- }-- api := humaecho.NewWithGroup(router, apiGroup, humaConfig)- api.UseMiddleware(humamiddleware.NewAuthBridge(api, authService, nil, sudoPermResolver{}, nil, &config.Config{}))- RegisterHealth(api)- RegisterTemplates(api, templateService)-- return router-}--// sudoPermResolver is a test stub satisfying humamiddleware.PermissionResolver-// that grants every permission. Used in tests that don't care about RBAC-// gating — they want auth to succeed and permissions to be unrestricted.-type sudoPermResolver struct{}--func (sudoPermResolver) ResolvePermissions(_ context.Context, _ *models.User) (*authz.PermissionSet, error) {- return authz.SudoPermissionSet(), nil-}-func (sudoPermResolver) ResolveApiKeyPermissions(_ context.Context, _ string) (*authz.PermissionSet, error) {- return authz.SudoPermissionSet(), nil-}--func makeTemplateFetchToken(t *testing.T, secret string, userID, username string) string {- t.Helper()-- claims := jwt.MapClaims{- "jti": userID,- "sub": "access",- "iat": jwt.NewNumericDate(time.Now()).Unix(),- "exp": jwt.NewNumericDate(time.Now().Add(5 * time.Minute)).Unix(),- "user_id": userID,- "sid": "session-1",- "username": username,- "roles": []string{"admin"},- "app_version": config.Version,- }-- token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)- signed, err := token.SignedString([]byte(secret))- require.NoError(t, err)-- return signed-}--func makeTemplateFetchRemote(t *testing.T, server *httptest.Server) (*http.Client, string) {- t.Helper()-- parsedURL, err := url.Parse(server.URL)- require.NoError(t, err)-- listenerAddr := server.Listener.Addr().String()- transport := http.DefaultTransport.(*http.Transport).Clone()- transport.DialContext = func(ctx context.Context, network, address string) (net.Conn, error) {- dialer := &net.Dialer{}- return dialer.DialContext(ctx, network, listenerAddr)- }-- client := &http.Client{- Transport: transport,- Timeout: 10 * time.Second,- }-- return client, "http://93.184.216.34:" + parsedURL.Port() + "/registry.json"-}diff --git a/backend/api/handlers/users_test.go b/backend/api/handlers/users_test.godeleted file mode 100644index b504f033a..000000000--- a/backend/api/handlers/users_test.go+++ /dev/null@@ -1,214 +0,0 @@-package handlers--import (- "context"- "net/http"- "testing"-- "github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/libtnb/sqlite"- "github.com/stretchr/testify/require"- "gorm.io/gorm"-- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/types/v2/user"-)--func setupUserHandlerTestDB(t *testing.T) *database.DB {- t.Helper()-- db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, db.AutoMigrate(&models.User{}))-- return &database.DB{DB: db}-}--func createHandlerTestUser(t *testing.T, svc *services.UserService, id, username string, _ models.StringSlice) *models.User {- t.Helper()-- user := &models.User{- BaseModel: models.BaseModel{ID: id},- Username: username,- }-- created, err := svc.CreateUser(context.Background(), user)- require.NoError(t, err)-- return created-}--func adminContext() context.Context {- return context.WithValue(context.Background(), humamw.ContextKeyUserPermissions, authz.SudoPermissionSet())-}--func TestDeleteUserReturnsConflictForLastAdmin(t *testing.T) {- db := setupUserHandlerTestDB(t)- require.NoError(t, db.AutoMigrate(&models.Role{}, &models.UserRoleAssignment{}, &models.Environment{}))- roleSvc := services.NewRoleService(db)- require.NoError(t, roleSvc.EnsureBuiltInRoles(context.Background()))- userSvc := services.NewUserService(db).WithRoleService(roleSvc)- handler := &UserHandler{userService: userSvc}- admin := createHandlerTestUser(t, userSvc, "admin-1", "arcane", models.StringSlice{})- require.NoError(t, roleSvc.SetUserAssignments(context.Background(), admin.ID, []models.UserRoleAssignment{- {RoleID: authz.BuiltInRoleAdmin, EnvironmentID: nil},- }))-- _, err := handler.DeleteUser(adminContext(), &DeleteUserInput{UserID: admin.ID})- require.Error(t, err)-- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- require.Equal(t, http.StatusConflict, statusErr.GetStatus())- require.Contains(t, statusErr.Error(), services.ErrCannotRemoveLastAdmin.Error())-}--// callerContext returns a context carrying the given user and their resolved-// PermissionSet, matching what the auth bridge attaches in production.-func callerContext(u *models.User, ps *authz.PermissionSet) context.Context {- ctx := context.WithValue(context.Background(), humamw.ContextKeyUserPermissions, ps)- return context.WithValue(ctx, models.CurrentUserContextKey{}, u)-}--func grantRole(t *testing.T, roleSvc *services.RoleService, userID, roleID string) {- t.Helper()- require.NoError(t, roleSvc.SetUserAssignments(context.Background(), userID, []models.UserRoleAssignment{- {RoleID: roleID, EnvironmentID: nil},- }))-}--func setupPrivilegedUserHandler(t *testing.T) (*services.UserService, *services.RoleService, *UserHandler) {- t.Helper()- db := setupUserHandlerTestDB(t)- require.NoError(t, db.AutoMigrate(&models.Role{}, &models.UserRoleAssignment{}, &models.Environment{}))- roleSvc := services.NewRoleService(db)- require.NoError(t, roleSvc.EnsureBuiltInRoles(context.Background()))- userSvc := services.NewUserService(db).WithRoleService(roleSvc)- return userSvc, roleSvc, &UserHandler{userService: userSvc}-}--func TestUpdateUserRejectsPasswordResetOnAdminByDelegatedManager(t *testing.T) {- userSvc, roleSvc, handler := setupPrivilegedUserHandler(t)- ctx := context.Background()-- admin := createHandlerTestUser(t, userSvc, "admin-1", "arcane", models.StringSlice{})- grantRole(t, roleSvc, admin.ID, authz.BuiltInRoleAdmin)- manager := createHandlerTestUser(t, userSvc, "mgr-1", "manager", models.StringSlice{})- managerRole, err := roleSvc.CreateRole(ctx, "User Manager", nil, []string{authz.PermUsersUpdate})- require.NoError(t, err)- grantRole(t, roleSvc, manager.ID, managerRole.ID)- managerPerms, err := roleSvc.ResolvePermissions(ctx, manager)- require.NoError(t, err)-- password := "attacker1234"- _, err = handler.UpdateUser(callerContext(manager, managerPerms), &UpdateUserInput{- UserID: admin.ID,- Body: user.UpdateUser{Password: &password},- })- require.Error(t, err)- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- require.Equal(t, http.StatusForbidden, statusErr.GetStatus())-- reloaded, getErr := userSvc.GetUserByID(ctx, admin.ID)- require.NoError(t, getErr)- require.NotEqual(t, password, reloaded.PasswordHash)-}--func TestUpdateUserRejectsProfileEditOnAdminByDelegatedManager(t *testing.T) {- userSvc, roleSvc, handler := setupPrivilegedUserHandler(t)- ctx := context.Background()-- admin := createHandlerTestUser(t, userSvc, "admin-1", "arcane", models.StringSlice{})- grantRole(t, roleSvc, admin.ID, authz.BuiltInRoleAdmin)- manager := createHandlerTestUser(t, userSvc, "mgr-1", "manager", models.StringSlice{})- managerRole, err := roleSvc.CreateRole(ctx, "User Manager", nil, []string{authz.PermUsersUpdate})- require.NoError(t, err)- grantRole(t, roleSvc, manager.ID, managerRole.ID)- managerPerms, err := roleSvc.ResolvePermissions(ctx, manager)- require.NoError(t, err)-- displayName := "pwned"- _, err = handler.UpdateUser(callerContext(manager, managerPerms), &UpdateUserInput{- UserID: admin.ID,- Body: user.UpdateUser{DisplayName: &displayName},- })- require.Error(t, err)- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- require.Equal(t, http.StatusForbidden, statusErr.GetStatus())-}--func TestUpdateUserRejectsPasswordChangeOnOtherUserByNonAdmin(t *testing.T) {- userSvc, roleSvc, handler := setupPrivilegedUserHandler(t)- ctx := context.Background()-- admin := createHandlerTestUser(t, userSvc, "admin-1", "arcane", models.StringSlice{})- grantRole(t, roleSvc, admin.ID, authz.BuiltInRoleAdmin)- victim := createHandlerTestUser(t, userSvc, "user-1", "victim", models.StringSlice{})- manager := createHandlerTestUser(t, userSvc, "mgr-1", "manager", models.StringSlice{})- managerRole, err := roleSvc.CreateRole(ctx, "User Manager", nil, []string{authz.PermUsersUpdate})- require.NoError(t, err)- grantRole(t, roleSvc, manager.ID, managerRole.ID)- managerPerms, err := roleSvc.ResolvePermissions(ctx, manager)- require.NoError(t, err)-- password := "newpass123"- _, err = handler.UpdateUser(callerContext(manager, managerPerms), &UpdateUserInput{- UserID: victim.ID,- Body: user.UpdateUser{Password: &password},- })- require.Error(t, err)- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- require.Equal(t, http.StatusForbidden, statusErr.GetStatus())-}--func TestUpdateUserAllowsGlobalAdminEditingAdmin(t *testing.T) {- userSvc, roleSvc, handler := setupPrivilegedUserHandler(t)- ctx := context.Background()-- admin := createHandlerTestUser(t, userSvc, "admin-1", "arcane", models.StringSlice{})- grantRole(t, roleSvc, admin.ID, authz.BuiltInRoleAdmin)- other := createHandlerTestUser(t, userSvc, "admin-2", "backup", models.StringSlice{})- grantRole(t, roleSvc, other.ID, authz.BuiltInRoleAdmin)- otherPerms, err := roleSvc.ResolvePermissions(ctx, other)- require.NoError(t, err)-- displayName := "Renamed Admin"- out, err := handler.UpdateUser(callerContext(other, otherPerms), &UpdateUserInput{- UserID: admin.ID,- Body: user.UpdateUser{DisplayName: &displayName},- })- require.NoError(t, err)- require.NotNil(t, out)-}--func TestDeleteUserRejectsDelegatedManagerDeletingAdmin(t *testing.T) {- userSvc, roleSvc, handler := setupPrivilegedUserHandler(t)- ctx := context.Background()-- admin := createHandlerTestUser(t, userSvc, "admin-1", "arcane", models.StringSlice{})- grantRole(t, roleSvc, admin.ID, authz.BuiltInRoleAdmin)- backup := createHandlerTestUser(t, userSvc, "admin-2", "backup", models.StringSlice{})- grantRole(t, roleSvc, backup.ID, authz.BuiltInRoleAdmin)- manager := createHandlerTestUser(t, userSvc, "mgr-1", "manager", models.StringSlice{})- managerRole, err := roleSvc.CreateRole(ctx, "User Manager", nil, []string{authz.PermUsersDelete})- require.NoError(t, err)- grantRole(t, roleSvc, manager.ID, managerRole.ID)- managerPerms, err := roleSvc.ResolvePermissions(ctx, manager)- require.NoError(t, err)-- _, err = handler.DeleteUser(callerContext(manager, managerPerms), &DeleteUserInput{UserID: admin.ID})- require.Error(t, err)- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- require.Equal(t, http.StatusForbidden, statusErr.GetStatus())-- _, getErr := userSvc.GetUserByID(ctx, admin.ID)- require.NoError(t, getErr)-}diff --git a/backend/api/handlers/variables_test.go b/backend/api/handlers/variables_test.godeleted file mode 100644index e9e3aaa4a..000000000--- a/backend/api/handlers/variables_test.go+++ /dev/null@@ -1,146 +0,0 @@-package handlers--import (- "context"- "io"- "net/http"- "net/http/httptest"- "path/filepath"- "strings"- "testing"-- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/labstack/echo/v5"- "github.com/libtnb/sqlite"- "github.com/stretchr/testify/require"- "gorm.io/gorm"-)--func TestVariableRoutesRequireTheirExactPermissions(t *testing.T) {- tests := []struct {- name string- method string- path string- body string- permission string- }{- {name: "list", method: http.MethodGet, path: "/api/variables", permission: authz.PermVariablesRead},- {name: "create", method: http.MethodPost, path: "/api/variables", body: `{"key":"API_URL","value":"https://example.test","allEnvironments":true}`, permission: authz.PermVariablesCreate},- {name: "update", method: http.MethodPut, path: "/api/variables/variable-1", body: `{}`, permission: authz.PermVariablesUpdate},- {name: "delete", method: http.MethodDelete, path: "/api/variables/variable-1", permission: authz.PermVariablesDelete},- {name: "sync", method: http.MethodPost, path: "/api/variables/sync", permission: authz.PermVariablesSync},- {name: "sync status", method: http.MethodGet, path: "/api/variables/sync-status", permission: authz.PermVariablesRead},- }-- allVariablePermissions := []string{- authz.PermVariablesRead,- authz.PermVariablesCreate,- authz.PermVariablesUpdate,- authz.PermVariablesDelete,- authz.PermVariablesSync,- }-- for _, tt := range tests {- t.Run(tt.name+" rejects other grants", func(t *testing.T) {- permissions := authz.NewPermissionSet()- permissions.AddGlobal(authz.PermTemplatesRead, authz.PermTemplatesCreate, authz.PermTemplatesUpdate, authz.PermTemplatesDelete)- for _, permission := range allVariablePermissions {- if permission != tt.permission {- permissions.AddGlobal(permission)- }- }-- router, api := newPermissionGatingRouterInternal(t, permissions)- RegisterVariables(api, nil, nil)- response := performVariableRequestInternal(router, tt.method, tt.path, tt.body)-- require.Equal(t, http.StatusForbidden, response.Code)- require.Contains(t, response.Body.String(), "permission denied: "+tt.permission)- })-- t.Run(tt.name+" allows exact grant", func(t *testing.T) {- permissions := authz.NewPermissionSet()- permissions.AddGlobal(tt.permission)-- router, api := newPermissionGatingRouterInternal(t, permissions)- variableService, environmentService := setupVariableHandlerServicesInternal(t)- RegisterVariables(api, variableService, environmentService)- response := performVariableRequestInternal(router, tt.method, tt.path, tt.body)-- require.NotEqual(t, http.StatusForbidden, response.Code)- require.NotContains(t, response.Body.String(), "permission denied")- })- }-}--func TestMaterializedVariableRoutesRequireSudo(t *testing.T) {- tests := []struct {- name string- method string- body string- }{- {name: "get", method: http.MethodGet},- {name: "put", method: http.MethodPut, body: `{"variables":[{"key":"TOKEN","value":"plaintext"}]}`},- }-- for _, tt := range tests {- t.Run(tt.name+" rejects human admin", func(t *testing.T) {- permissions := authz.NewPermissionSet()- permissions.AddGlobal(authz.AllPermissions()...)- router, api := newPermissionGatingRouterInternal(t, permissions)- RegisterMaterializedVariables(api, nil, nil)-- response := performVariableRequestInternal(router, tt.method, "/api/environments/0/templates/variables", tt.body)-- require.Equal(t, http.StatusForbidden, response.Code)- require.Contains(t, response.Body.String(), "agent authentication required")- })-- t.Run(tt.name+" allows agent", func(t *testing.T) {- router, api := newPermissionGatingRouterInternal(t, authz.SudoPermissionSet())- variableService, environmentService := setupVariableHandlerServicesInternal(t)- RegisterMaterializedVariables(api, variableService, environmentService)-- response := performVariableRequestInternal(router, tt.method, "/api/environments/0/templates/variables", tt.body)-- require.Equal(t, http.StatusOK, response.Code)- })- }-}--func setupVariableHandlerServicesInternal(t *testing.T) (*services.VariableService, *services.EnvironmentService) {- t.Helper()-- databaseConnection, err := gorm.Open(sqlite.Open(filepath.Join(t.TempDir(), "variables.db")), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, databaseConnection.AutoMigrate(- &models.GlobalVariable{},- &models.Environment{},- &models.KVEntry{},- &models.SettingVariable{},- ))- databaseDB := &database.DB{DB: databaseConnection}- settingsService, err := newSettingsServiceForTestInternal(t, context.Background(), databaseDB)- require.NoError(t, err)- require.NoError(t, settingsService.UpdateSetting(context.Background(), "projectsDirectory", t.TempDir()))- environmentService := services.NewEnvironmentService(databaseDB, nil, nil, nil, settingsService, nil)- variableService := services.NewVariableService(databaseDB, environmentService, settingsService, services.NewKVService(databaseDB))- return variableService, environmentService-}--func performVariableRequestInternal(router *echo.Echo, method, path, body string) *httptest.ResponseRecorder {- var reader io.Reader- if body != "" {- reader = strings.NewReader(body)- }- request := httptest.NewRequest(method, path, reader)- if body != "" {- request.Header.Set(echo.HeaderContentType, echo.MIMEApplicationJSON)- }- response := httptest.NewRecorder()- router.ServeHTTP(response, request)- return response-}diff --git a/backend/api/handlers/volumes_upload_test.go b/backend/api/handlers/volumes_upload_test.godeleted file mode 100644index 880e15b62..000000000--- a/backend/api/handlers/volumes_upload_test.go+++ /dev/null@@ -1,49 +0,0 @@-package handlers--import (- "context"- "mime/multipart"- "net/http"- "testing"-- "github.com/danielgtaylor/huma/v2"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/stretchr/testify/assert"- "github.com/stretchr/testify/require"-)--func TestUploadFileReturnsBadRequestWhenNoFileProvided(t *testing.T) {- h := &VolumeHandler{volumeService: &services.VolumeService{}}-- _, err := h.UploadFile(adminTestContextInternal(), &UploadFileInput{- EnvironmentID: "0",- VolumeName: "vol-1",- Path: "/",- RawBody: multipart.Form{},- })-- require.Error(t, err)-- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- assert.Equal(t, http.StatusBadRequest, statusErr.GetStatus())-}--func TestUploadAndRestoreReturnsBadRequestWhenNoFileProvided(t *testing.T) {- h := &VolumeHandler{volumeService: &services.VolumeService{}}-- ctx := context.WithValue(adminTestContextInternal(), models.CurrentUserContextKey{}, &models.User{BaseModel: models.BaseModel{ID: "u-1"}})-- _, err := h.UploadAndRestore(ctx, &UploadAndRestoreInput{- EnvironmentID: "0",- VolumeName: "vol-1",- RawBody: multipart.Form{},- })-- require.Error(t, err)-- var statusErr huma.StatusError- require.ErrorAs(t, err, &statusErr)- assert.Equal(t, http.StatusBadRequest, statusErr.GetStatus())-}diff --git a/backend/api/playwright.go b/backend/api/playwright.godeleted file mode 100644index bb89edf44..000000000--- a/backend/api/playwright.go+++ /dev/null@@ -1,87 +0,0 @@-//go:build playwright--package api--import (- "net/http"-- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/labstack/echo/v5"-)--type PlaywrightHandler struct {- PlaywrightService *services.PlaywrightService-}--func NewPlaywrightHandler(playwrightService *services.PlaywrightService) *PlaywrightHandler {- return &PlaywrightHandler{PlaywrightService: playwrightService}-}--func SetupPlaywrightRoutes(api *echo.Group, playwrightService *services.PlaywrightService) {- playwright := api.Group("/playwright")-- playwrightHandler := NewPlaywrightHandler(playwrightService)-- playwright.POST("/create-test-api-keys", playwrightHandler.CreateTestApiKeysHandler)- playwright.POST("/delete-test-api-keys", playwrightHandler.DeleteTestApiKeysHandler)- playwright.POST("/create-test-federated-credential", playwrightHandler.CreateTestFederatedCredentialHandler)-}--type CreateTestApiKeysRequest struct {- Count int `json:"count"`-}--type CreateTestFederatedCredentialRequest struct {- IssuerURL string `json:"issuerUrl"`- Audiences []string `json:"audiences"`- Subject string `json:"subject"`- RoleID string `json:"roleId"`- TokenTTLSeconds int `json:"tokenTtlSeconds"`-}--func (ph *PlaywrightHandler) CreateTestApiKeysHandler(c *echo.Context) error {- var req CreateTestApiKeysRequest- if err := c.Bind(&req); err != nil {- req.Count = 2- }-- if req.Count <= 0 {- req.Count = 2- }-- apiKeys, err := ph.PlaywrightService.CreateTestApiKeys(c.Request().Context(), req.Count)- if err != nil {- return c.JSON(http.StatusInternalServerError, map[string]any{"error": err.Error()})- }-- return c.JSON(http.StatusCreated, map[string]any{"apiKeys": apiKeys})-}--func (ph *PlaywrightHandler) DeleteTestApiKeysHandler(c *echo.Context) error {- if err := ph.PlaywrightService.DeleteAllTestApiKeys(c.Request().Context()); err != nil {- return c.JSON(http.StatusInternalServerError, map[string]any{"error": err.Error()})- }-- return c.NoContent(http.StatusNoContent)-}--func (ph *PlaywrightHandler) CreateTestFederatedCredentialHandler(c *echo.Context) error {- var req CreateTestFederatedCredentialRequest- if err := c.Bind(&req); err != nil {- return c.JSON(http.StatusBadRequest, map[string]any{"error": "invalid request body"})- }-- credentialID, err := ph.PlaywrightService.CreateTestFederatedCredential(- c.Request().Context(),- req.IssuerURL,- req.Audiences,- req.Subject,- req.RoleID,- req.TokenTTLSeconds,- )- if err != nil {- return c.JSON(http.StatusInternalServerError, map[string]any{"error": err.Error()})- }-- return c.JSON(http.StatusCreated, map[string]any{"credential": map[string]string{"id": credentialID}})-}diff --git a/backend/api/webhooks_trigger.go b/backend/api/webhooks_trigger.goindex 24cddab2a..96857bff6 100644--- a/backend/api/webhooks_trigger.go+++ b/backend/api/webhooks_trigger.go@@ -5,28 +5,28 @@ import ("emperror.dev/errors"- "github.com/getarcaneapp/arcane/backend/v2/api/handlers"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/webhook""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/labstack/echo/v5")// RegisterWebhookTrigger registers the public (unauthenticated) trigger endpoint.// The token in the URL is the sole authentication mechanism.// Rate-limited via PerIPRateLimitForPaths in router_bootstrap.go.-func RegisterWebhookTrigger(g *echo.Group, webhookService *services.WebhookService, appCtx handlers.ActivityAppContext) {+func RegisterWebhookTrigger(g *echo.Group, webhookService *webhook.WebhookService, appCtx handlerutil.ActivityAppContext) {g.POST("/webhooks/trigger/:token", func(c *echo.Context) error {if webhookService == nil {return c.JSON(http.StatusInternalServerError, map[string]any{"success": false, "error": "service not available"})}token := c.Param("token")- result, err := webhookService.TriggerByToken(utils.ActivityRuntimeContext(c.Request().Context(), appCtx.ContextInternal()), token)+ result, err := webhookService.TriggerByToken(utils.ActivityRuntimeContext(c.Request().Context(), appCtx.Context()), token)if err != nil {status := http.StatusInternalServerError- if errors.Is(err, services.ErrWebhookNotFound) || errors.Is(err, services.ErrWebhookInvalid) {+ if errors.Is(err, webhook.ErrWebhookNotFound) || errors.Is(err, webhook.ErrWebhookInvalid) {status = http.StatusNotFound- } else if errors.Is(err, services.ErrWebhookDisabled) {+ } else if errors.Is(err, webhook.ErrWebhookDisabled) {status = http.StatusForbidden}msg := err.Error()diff --git a/backend/api/ws/diagnostics.go b/backend/api/ws/diagnostics.goindex ac47e16ae..ad06f474a 100644--- a/backend/api/ws/diagnostics.go+++ b/backend/api/ws/diagnostics.go@@ -8,9 +8,11 @@ import ("net/http/pprof""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/diagnostics"+"github.com/coder/websocket"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth""github.com/getarcaneapp/arcane/backend/v2/internal/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"wshub "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/ws"systemtypes "github.com/getarcaneapp/arcane/types/v2/system"@@ -34,7 +36,7 @@ const (// BuildDiagnostics assembles a full diagnostics snapshot: runtime/memory/GC from// the DiagnosticsService plus this package's WebSocket metrics and worker-goroutine// count. Shared by the REST endpoint (via handlers) and the live WebSocket stream.-func BuildDiagnostics(diag *services.DiagnosticsService) systemtypes.Diagnostics {+func BuildDiagnostics(diag *diagnostics.DiagnosticsService) systemtypes.Diagnostics {d := systemtypes.Diagnostics{Timestamp: time.Now().UTC()}if diag != nil {d.Runtime, d.Memory, d.GC = diag.Collect()@@ -50,10 +52,10 @@ func BuildDiagnostics(diag *services.DiagnosticsService) systemtypes.Diagnostics// registerDiagnosticsRoutesInternal wires the global diagnostics WebSocket streams and the// net/http/pprof debug endpoints. Streams require the diagnostics permission;// pprof keeps the stricter admin-required gate. Called from NewWebSocketHandler.-func (h *WebSocketHandler) registerDiagnosticsRoutesInternal(group *echo.Group, authMiddleware *middleware.AuthMiddleware) {+func (h *WebSocketHandler) registerDiagnosticsRoutesInternal(group *echo.Group, authMiddleware *auth.AuthMiddleware) {diag := group.Group("/diagnostics",authMiddleware.WithAdminNotRequired().Add(),- middleware.RequirePermission(authz.PermDiagnosticsRead),+ middleware.RequireEchoPermission(authz.PermDiagnosticsRead),)diag.GET("/stream", h.DiagnosticsStream)diag.GET("/logs/stream", h.ServerLogsStream)diff --git a/backend/api/ws/handler.go b/backend/api/ws/handler.goindex 7d08f4578..85a2ed1d2 100644--- a/backend/api/ws/handler.go+++ b/backend/api/ws/handler.go@@ -13,6 +13,13 @@ import ("sync/atomic""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth"+ "github.com/getarcaneapp/arcane/backend/v2/internal/diagnostics"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/swarm"++ "github.com/getarcaneapp/arcane/backend/v2/internal/container"+"emperror.dev/errors""github.com/coder/websocket"@@ -25,11 +32,12 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/system"+ systemlib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/system"wshub "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/ws"httputil "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"systemtypes "github.com/getarcaneapp/arcane/types/v2/system"++ "github.com/getarcaneapp/arcane/backend/v2/internal/system""go.getarcane.app/sys/cgroup")@@ -44,11 +52,11 @@ var defaultWebSocketMetrics = wshub.NewWebSocketMetrics()// WebSocketHandler consolidates all WebSocket and streaming endpoints.// REST endpoints are handled by Huma handlers.type WebSocketHandler struct {- projectService *services.ProjectService- containerService *services.ContainerService- swarmService *services.SwarmService- systemService *services.SystemService- diagnosticsService *services.DiagnosticsService+ projectService *project.ProjectService+ containerService *container.ContainerService+ swarmService *swarm.SwarmService+ systemService *system.SystemService+ diagnosticsService *diagnostics.DiagnosticsServicecheckWSOrigin func(*http.Request) boolwsMetrics *wshub.WebSocketMetricsactiveConnections sync.Map@@ -77,7 +85,7 @@ type WebSocketHandler struct {}containerStatsHubs sync.MapcgroupCache *cgroup.Cache- gpuMonitor *system.GPUMonitor+ gpuMonitor *systemlib.GPUMonitordiskUsagePathCache *hot.HotCache[struct{}, string]projectLogStreamer func(ctx context.Context, projectID string, logsChan chan<- string, follow bool, tail, since string, timestamps bool) error@@ -211,12 +219,12 @@ func (h *WebSocketHandler) markLogStreamDoneInternal(key string, stream *wsLogStfunc NewWebSocketHandler(group *echo.Group,- projectService *services.ProjectService,- containerService *services.ContainerService,- swarmService *services.SwarmService,- systemService *services.SystemService,- diagnosticsService *services.DiagnosticsService,- authMiddleware *middleware.AuthMiddleware,+ projectService *project.ProjectService,+ containerService *container.ContainerService,+ swarmService *swarm.SwarmService,+ systemService *system.SystemService,+ diagnosticsService *diagnostics.DiagnosticsService,+ authMiddleware *auth.AuthMiddleware,cfg *config.Config,) {handler := &WebSocketHandler{@@ -228,7 +236,7 @@ func NewWebSocketHandler(wsMetrics: defaultWebSocketMetrics,logStreams: make(map[string]*wsLogStream),cgroupCache: cgroup.NewCache(cgroupCacheTTL),- gpuMonitor: system.NewGPUMonitor(cfg.GPUMonitoringEnabled, cfg.GPUType),+ gpuMonitor: systemlib.NewGPUMonitor(cfg.GPUMonitoringEnabled, cfg.GPUType),diskUsagePathCache: hot.NewHotCache[struct{}, string](hot.LRU, 1).WithTTL(5 * time.Minute).Build(),@@ -236,7 +244,7 @@ func NewWebSocketHandler(}wsGroup := group.Group("/environments/:id/ws", authMiddleware.WithAdminNotRequired().Add())for _, r := range handler.proxiedRoutes() {- wsGroup.GET(r.path, r.handler, middleware.RequirePermission(r.perm))+ wsGroup.GET(r.path, r.handler, middleware.RequireEchoPermission(r.perm))}handler.registerDiagnosticsRoutesInternal(group, authMiddleware)}@@ -882,7 +890,7 @@ func (h *WebSocketHandler) writeExecErrorInternal(ctx context.Context, conn *web_ = conn.Write(wctx, websocket.MessageText, []byte(err.Error()+"\r\n"))}-func (h *WebSocketHandler) execCleanupFuncInternal(ctx context.Context, execSession *services.ExecSession, execID, containerID string) func() {+func (h *WebSocketHandler) execCleanupFuncInternal(ctx context.Context, execSession *container.ExecSession, execID, containerID string) func() {return func() {slog.Debug("Cleaning up exec session", "execID", execID, "containerID", containerID, "contextErr", ctx.Err())// Cleanup must proceed even if parent ctx is canceled.diff --git a/backend/api/ws/handler_test.go b/backend/api/ws/handler_test.godeleted file mode 100644index 32fe7f6e0..000000000--- a/backend/api/ws/handler_test.go+++ /dev/null@@ -1,627 +0,0 @@-package ws--import (- "context"- "encoding/json"- "errors"- "net/http"- "net/http/httptest"- "strings"- "sync"- "sync/atomic"- "testing"- "time"-- "github.com/coder/websocket"- "github.com/labstack/echo/v5"- "github.com/stretchr/testify/require"-- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/system"- wshub "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/ws"- systemtypes "github.com/getarcaneapp/arcane/types/v2/system"- "go.getarcane.app/sys/cgroup"-)--func TestBroadcastLogStreamErrorInternal_JSON(t *testing.T) {- hub := wshub.NewHub(10)- ctx := t.Context()- go hub.Run(ctx)-- clientConn, serverConn, cleanup := newTestWSPairInternal(t)- t.Cleanup(cleanup)-- wshub.ServeClientWithOnRemove(ctx, hub, serverConn, nil)-- stream := &wsLogStream{- hub: hub,- format: "json",- }-- broadcastLogStreamErrorInternal("container log stream", "Failed to stream container logs: ", "container-1", "json", errors.New("boom"), stream)-- payload, err := readMessageInternal(t, clientConn)- require.NoError(t, err)-- var message wshub.LogMessage- require.NoError(t, json.Unmarshal(payload, &message))- require.Equal(t, "error", message.Level)- require.Equal(t, "Failed to stream container logs: boom", message.Message)- require.Empty(t, message.ContainerID)- require.NotEmpty(t, message.Timestamp)-}--func TestBroadcastLogStreamErrorInternal_Text(t *testing.T) {- hub := wshub.NewHub(10)- ctx := t.Context()- go hub.Run(ctx)-- clientConn, serverConn, cleanup := newTestWSPairInternal(t)- t.Cleanup(cleanup)-- wshub.ServeClientWithOnRemove(ctx, hub, serverConn, nil)-- stream := &wsLogStream{- hub: hub,- format: "text",- }-- broadcastLogStreamErrorInternal("container log stream", "Failed to stream container logs: ", "container-1", "text", errors.New("boom"), stream)-- payload, err := readMessageInternal(t, clientConn)- require.NoError(t, err)- require.Equal(t, "Failed to stream container logs: boom", string(payload))-}--func newTestWSPairInternal(t *testing.T) (clientConn *websocket.Conn, serverConn *websocket.Conn, cleanup func()) {- t.Helper()- serverReady := make(chan *websocket.Conn, 1)-- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- conn, err := websocket.Accept(w, r, &websocket.AcceptOptions{InsecureSkipVerify: true})- if err != nil {- return- }- serverReady <- conn- }))-- url := "ws" + strings.TrimPrefix(server.URL, "http")- clientConn, _, err := websocket.Dial(t.Context(), url, nil)- require.NoError(t, err)-- serverConn = <-serverReady-- return clientConn, serverConn, func() {- _ = clientConn.CloseNow()- _ = serverConn.CloseNow()- server.Close()- }-}--func newTestWebSocketHandler() *WebSocketHandler {- return &WebSocketHandler{- wsMetrics: wshub.NewWebSocketMetrics(),- logStreams: make(map[string]*wsLogStream),- cgroupCache: cgroup.NewCache(cgroupCacheTTL),- gpuMonitor: system.NewGPUMonitor(false, ""),- checkWSOrigin: func(*http.Request) bool { return true },- }-}--func dialWebSocket(t *testing.T, serverURL, path string) *websocket.Conn {- t.Helper()-- wsURL := "ws" + strings.TrimPrefix(serverURL, "http") + path- conn, _, err := websocket.Dial(t.Context(), wsURL, nil)- require.NoError(t, err)-- return conn-}--func readMessageInternal(t *testing.T, conn *websocket.Conn) ([]byte, error) {- t.Helper()-- ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)- defer cancel()- _, payload, err := conn.Read(ctx)- return payload, err-}--func TestWebSocketHandler_ProjectLogs_SharedStreamPerTarget(t *testing.T) {-- handler := newTestWebSocketHandler()- var starts atomic.Int32- var cancels atomic.Int32-- handler.projectLogStreamer = func(ctx context.Context, projectID string, logsChan chan<- string, follow bool, tail, since string, timestamps bool) error {- starts.Add(1)- ticker := time.NewTicker(20 * time.Millisecond)- defer ticker.Stop()- defer cancels.Add(1)-- for {- select {- case <-ctx.Done():- return ctx.Err()- case <-ticker.C:- select {- case <-ctx.Done():- return ctx.Err()- case logsChan <- "api | shared project log":- }- }- }- }-- router := echo.New()- router.GET("/api/environments/:id/ws/projects/:projectId/logs", handler.ProjectLogs)- server := httptest.NewServer(router)- defer server.Close()-- conn1 := dialWebSocket(t, server.URL, "/api/environments/0/ws/projects/project-1/logs")- conn2 := dialWebSocket(t, server.URL, "/api/environments/0/ws/projects/project-1/logs")-- _, err := readMessageInternal(t, conn1)- require.NoError(t, err)-- _, err = readMessageInternal(t, conn2)- require.NoError(t, err)-- require.Eventually(t, func() bool {- return starts.Load() == 1- }, 2*time.Second, 20*time.Millisecond)-- require.Eventually(t, func() bool {- handler.logStreamsMu.Lock()- defer handler.logStreamsMu.Unlock()- return len(handler.logStreams) == 1- }, time.Second, 20*time.Millisecond)-- require.NoError(t, conn1.CloseNow())-- require.Eventually(t, func() bool {- handler.logStreamsMu.Lock()- defer handler.logStreamsMu.Unlock()- return len(handler.logStreams) == 1- }, 2*time.Second, 10*time.Millisecond)-- handler.logStreamsMu.Lock()- activeAfterFirstClose := len(handler.logStreams)- handler.logStreamsMu.Unlock()- require.Equal(t, 1, activeAfterFirstClose)- require.Equal(t, int32(0), cancels.Load())-- require.NoError(t, conn2.CloseNow())-- require.Eventually(t, func() bool {- handler.logStreamsMu.Lock()- defer handler.logStreamsMu.Unlock()- return len(handler.logStreams) == 0- }, 2*time.Second, 20*time.Millisecond)- require.Eventually(t, func() bool {- return cancels.Load() == 1- }, 2*time.Second, 20*time.Millisecond)-}--func TestWebSocketHandler_ProjectLogs_CompletedSourceStartsFreshStream(t *testing.T) {-- handler := newTestWebSocketHandler()- var starts atomic.Int32- firstDone := make(chan struct{})-- handler.projectLogStreamer = func(ctx context.Context, projectID string, logsChan chan<- string, follow bool, tail, since string, timestamps bool) error {- call := starts.Add(1)- select {- case <-ctx.Done():- return ctx.Err()- case logsChan <- "api | finite project log":- }- if call == 1 {- close(firstDone)- }- return nil- }-- router := echo.New()- router.GET("/api/environments/:id/ws/projects/:projectId/logs", handler.ProjectLogs)- server := httptest.NewServer(router)- defer server.Close()-- path := "/api/environments/0/ws/projects/project-1/logs?follow=false"- conn1 := dialWebSocket(t, server.URL, path)- defer func() {- _ = conn1.CloseNow()- }()-- msg1, err := readMessageInternal(t, conn1)- require.NoError(t, err)- require.Equal(t, "finite project log", string(msg1))-- select {- case <-firstDone:- case <-time.After(2 * time.Second):- require.FailNow(t, "first finite log stream did not complete")- }-- conn2 := dialWebSocket(t, server.URL, path)- defer func() {- _ = conn2.CloseNow()- }()-- msg2, err := readMessageInternal(t, conn2)- require.NoError(t, err)- require.Equal(t, "finite project log", string(msg2))-- require.Eventually(t, func() bool {- return starts.Load() == 2- }, 2*time.Second, 20*time.Millisecond)-}--func TestWebSocketHandler_ContainerLogs_BroadcastsStreamErrors(t *testing.T) {-- handler := newTestWebSocketHandler()- handler.containerLogStreamer = func(ctx context.Context, containerID string, logsChan chan<- string, follow bool, tail, since string, timestamps bool) error {- select {- case <-ctx.Done():- return ctx.Err()- case logsChan <- "api | container log":- }- return errors.New("stream failed")- }-- router := echo.New()- router.GET("/api/environments/:id/ws/containers/:containerId/logs", handler.ContainerLogs)- server := httptest.NewServer(router)- defer server.Close()-- conn := dialWebSocket(t, server.URL, "/api/environments/0/ws/containers/container-1/logs")- defer func() {- _ = conn.CloseNow()- }()-- var got []string-- msg, err := readMessageInternal(t, conn)- require.NoError(t, err)- got = append(got, string(msg))-- msg, err = readMessageInternal(t, conn)- require.NoError(t, err)- got = append(got, string(msg))-- require.ElementsMatch(t, []string{- "api | container log",- "Failed to stream container logs: stream failed",- }, got)-}--func TestWebSocketHandler_ContainerLogs_ErrorStartsFreshStreamForNewSubscribers(t *testing.T) {-- handler := newTestWebSocketHandler()- var starts atomic.Int32- handler.containerLogStreamer = func(ctx context.Context, containerID string, logsChan chan<- string, follow bool, tail, since string, timestamps bool) error {- starts.Add(1)- select {- case <-ctx.Done():- return ctx.Err()- case logsChan <- "api | container log":- }- return errors.New("stream failed")- }-- router := echo.New()- router.GET("/api/environments/:id/ws/containers/:containerId/logs", handler.ContainerLogs)- server := httptest.NewServer(router)- defer server.Close()-- path := "/api/environments/0/ws/containers/container-1/logs"- conn1 := dialWebSocket(t, server.URL, path)- defer func() {- _ = conn1.CloseNow()- }()-- got1 := make([]string, 0, 2)- for range 2 {- msg, err := readMessageInternal(t, conn1)- require.NoError(t, err)- got1 = append(got1, string(msg))- }- require.ElementsMatch(t, []string{- "api | container log",- "Failed to stream container logs: stream failed",- }, got1)-- conn2 := dialWebSocket(t, server.URL, path)- defer func() {- _ = conn2.CloseNow()- }()-- got2 := make([]string, 0, 2)- for range 2 {- msg, err := readMessageInternal(t, conn2)- require.NoError(t, err)- got2 = append(got2, string(msg))- }- require.ElementsMatch(t, []string{- "api | container log",- "Failed to stream container logs: stream failed",- }, got2)-- require.Eventually(t, func() bool {- return starts.Load() == 2- }, 2*time.Second, 20*time.Millisecond)-}--func TestWebSocketHandler_SystemStats_UsesSharedSampler(t *testing.T) {-- handler := newTestWebSocketHandler()- var collects atomic.Int32-- handler.systemStatsCollector = func(ctx context.Context) systemtypes.SystemStats {- n := collects.Add(1)- return systemtypes.SystemStats{- CPUUsage: float64(n),- }- }-- router := echo.New()- router.GET("/api/environments/:id/ws/system/stats", handler.SystemStats)- server := httptest.NewServer(router)- defer server.Close()-- conn1 := dialWebSocket(t, server.URL, "/api/environments/0/ws/system/stats?interval=1")- conn2 := dialWebSocket(t, server.URL, "/api/environments/0/ws/system/stats?interval=1")-- _, err := readMessageInternal(t, conn1)- require.NoError(t, err)-- _, err = readMessageInternal(t, conn2)- require.NoError(t, err)-- require.Eventually(t, func() bool {- return collects.Load() >= 1- }, 2*time.Second, 50*time.Millisecond)-- require.NoError(t, conn1.CloseNow())- require.NoError(t, conn2.CloseNow())-- require.Eventually(t, func() bool {- handler.systemStatsSampler.lifecycleMu.Lock()- defer handler.systemStatsSampler.lifecycleMu.Unlock()- return !handler.systemStatsSampler.running && handler.systemStatsSampler.clients == 0- }, 2*time.Second, 20*time.Millisecond)-- stoppedAt := collects.Load()- require.Never(t, func() bool {- return collects.Load() != stoppedAt- }, 1200*time.Millisecond, 100*time.Millisecond)-}--func TestWebSocketHandler_AcquireSystemStatsSampler_WaitsForInitialSnapshot(t *testing.T) {- handler := newTestWebSocketHandler()- handler.systemStatsCollector = func(ctx context.Context) systemtypes.SystemStats {- return systemtypes.SystemStats{CPUUsage: 42}- }-- firstDone := make(chan struct{})- go func() {- handler.acquireSystemStatsSamplerInternal(context.Background())- close(firstDone)- }()-- require.Eventually(t, func() bool {- handler.systemStatsSampler.lifecycleMu.Lock()- defer handler.systemStatsSampler.lifecycleMu.Unlock()- return handler.systemStatsSampler.running && handler.systemStatsSampler.ready != nil- }, 500*time.Millisecond, 10*time.Millisecond)-- secondDone := make(chan struct{})- go func() {- handler.acquireSystemStatsSamplerInternal(context.Background())- close(secondDone)- }()-- require.Never(t, func() bool {- select {- case <-secondDone:- return true- default:- return false- }- }, 200*time.Millisecond, 20*time.Millisecond)-- select {- case <-firstDone:- case <-time.After(2 * time.Second):- require.FailNow(t, "first sampler acquisition did not finish")- }-- select {- case <-secondDone:- case <-time.After(2 * time.Second):- require.FailNow(t, "second sampler acquisition did not wait for readiness")- }-- stats := handler.latestSystemStatsSnapshotInternal()- require.InDelta(t, 42.0, stats.CPUUsage, 0.000001)-- handler.releaseSystemStatsSamplerInternal()- handler.releaseSystemStatsSamplerInternal()-- require.Eventually(t, func() bool {- handler.systemStatsSampler.lifecycleMu.Lock()- defer handler.systemStatsSampler.lifecycleMu.Unlock()- return !handler.systemStatsSampler.running && handler.systemStatsSampler.clients == 0 && handler.systemStatsSampler.ready == nil- }, 2*time.Second, 20*time.Millisecond)-}--func TestWebSocketHandler_AcquireSystemStatsSampler_StopsWaitingWhenCallerCancels(t *testing.T) {- handler := newTestWebSocketHandler()-- readyToCancel := make(chan struct{})- handler.systemStatsCollector = func(ctx context.Context) systemtypes.SystemStats {- return systemtypes.SystemStats{CPUUsage: 42}- }-- handler.cpuUsageReader = func(interval time.Duration) (float64, bool) {- close(readyToCancel)- time.Sleep(200 * time.Millisecond)- return 42, true- }-- ctx, cancel := context.WithCancel(context.Background())- done := make(chan bool, 1)- go func() {- done <- handler.acquireSystemStatsSamplerInternal(ctx)- }()-- select {- case <-readyToCancel:- case <-time.After(time.Second):- require.FailNow(t, "sampler did not start CPU initialization")- }-- cancel()-- select {- case ok := <-done:- require.False(t, ok)- case <-time.After(100 * time.Millisecond):- require.FailNow(t, "sampler acquisition did not stop waiting after cancellation")- }-- handler.releaseSystemStatsSamplerInternal()-- require.Eventually(t, func() bool {- handler.systemStatsSampler.lifecycleMu.Lock()- defer handler.systemStatsSampler.lifecycleMu.Unlock()- return !handler.systemStatsSampler.running && handler.systemStatsSampler.clients == 0- }, time.Second, 20*time.Millisecond)-}--func TestWebSocketHandler_LogStream_ReplacesDoneStreamAndCleansStaleRefs(t *testing.T) {- handler := newTestWebSocketHandler()- key := "env|project|resource|text|false|true|100||false"-- var staleCancels atomic.Int32- stale := &wsLogStream{- hub: wshub.NewHub(1),- cancel: func() { staleCancels.Add(1) },- refs: 2,- done: true,- }- handler.logStreams[key] = stale-- var freshCancels atomic.Int32- fresh := handler.getOrCreateLogStreamInternal(key, func(onEmpty func(*wsLogStream)) *wsLogStream {- return &wsLogStream{- hub: wshub.NewHub(1),- cancel: func() { freshCancels.Add(1) },- }- })-- require.NotSame(t, stale, fresh)- handler.logStreamsMu.Lock()- require.Same(t, fresh, handler.logStreams[key])- handler.logStreamsMu.Unlock()-- handler.markLogStreamDoneInternal(key, stale)- handler.logStreamsMu.Lock()- require.Same(t, fresh, handler.logStreams[key])- handler.logStreamsMu.Unlock()-- handler.releaseLogStreamInternal(key, stale)- require.Equal(t, 1, stale.refs)- handler.logStreamsMu.Lock()- require.Same(t, fresh, handler.logStreams[key])- handler.logStreamsMu.Unlock()-- handler.releaseLogStreamInternal(key, stale)- require.Equal(t, int32(1), staleCancels.Load())- handler.logStreamsMu.Lock()- require.Same(t, fresh, handler.logStreams[key])- handler.logStreamsMu.Unlock()-- handler.releaseLogStreamInternal(key, fresh)- require.Equal(t, int32(1), freshCancels.Load())- handler.logStreamsMu.Lock()- _, ok := handler.logStreams[key]- handler.logStreamsMu.Unlock()- require.False(t, ok)-}--func TestWebSocketHandler_LogStream_CancelsOnlyOnce(t *testing.T) {- handler := newTestWebSocketHandler()- key := "env|project|resource|text|false|true|100||false"-- var cancels atomic.Int32- stream := &wsLogStream{- hub: wshub.NewHub(1),- cancel: func() { cancels.Add(1) },- refs: 1,- }- handler.logStreams[key] = stream-- handler.markLogStreamDoneInternal(key, stream)- handler.releaseLogStreamInternal(key, stream)- handler.markLogStreamDoneInternal(key, stream)-- require.Equal(t, int32(1), cancels.Load())-}--func TestWebSocketHandler_GetCachedCgroupLimitsInternal_DeduplicatesRefresh(t *testing.T) {- handler := newTestWebSocketHandler()-- var calls atomic.Int32- start := make(chan struct{})- var startOnce sync.Once- release := make(chan struct{})- // A fresh cache has a zero timestamp, so the first Get takes the refresh path- // and subsequent concurrent Gets dedupe behind the write lock. A goroutine- // can miss both the cached value and the in-flight refresh at the flight- // boundary and run the detector a second time after the first completes, so- // close(start) must be idempotent.- handler.cgroupCache = cgroup.NewCacheWithDetector(cgroupCacheTTL, func() (*cgroup.Limits, error) {- calls.Add(1)- startOnce.Do(func() { close(start) })- <-release- return &cgroup.Limits{CPUCount: 2}, nil- })-- const goroutines = 8- results := make(chan *cgroup.Limits, goroutines)- ready := make(chan struct{})- var entered sync.WaitGroup- entered.Add(goroutines)- var wg sync.WaitGroup- for range goroutines {- wg.Go(func() {- <-ready- entered.Done()- results <- handler.getCachedCgroupLimitsInternal()- })- }-- close(ready)- entered.Wait()-- select {- case <-start:- case <-time.After(2 * time.Second):- require.FailNow(t, "detector was not called")- }-- require.Equal(t, int32(1), calls.Load())-- close(release)- wg.Wait()- close(results)-- for result := range results {- require.NotNil(t, result)- require.Equal(t, 2, result.CPUCount)- }- // Dedup guarantees no concurrent detector calls (asserted above while the- // refresh was in flight); a single boundary straggler may legitimately- // trigger one extra refresh after the first flight completes.- require.LessOrEqual(t, calls.Load(), int32(2))-}diff --git a/backend/cli/admin/reset_mfa.go b/backend/cli/admin/reset_mfa.goindex b128172af..de1859356 100644--- a/backend/cli/admin/reset_mfa.go+++ b/backend/cli/admin/reset_mfa.go@@ -10,9 +10,11 @@ import ("emperror.dev/errors""github.com/spf13/cobra"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/passkey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user")var resetMFAUsername string@@ -55,16 +57,16 @@ func runResetMFACommandInternal(cmd *cobra.Command, _ []string) error {}}()- userService := services.NewUserService(db)+ userService := user.NewUserService(db)user, err := userService.GetUserByUsername(cmd.Context(), username)if err != nil {- if errors.Is(err, services.ErrUserNotFound) {+ if errors.Is(err, common.ErrUserNotFound) {return errors.Errorf("user %q not found", username)}return errors.WrapIf(err, "failed to find user")}- passkeyService := services.NewPasskeyService(db, cfg)+ passkeyService := passkey.NewPasskeyService(db, cfg)if err := passkeyService.ResetMFAForUser(cmd.Context(), user.ID); err != nil {return errors.WrapIf(err, "failed to reset passkey MFA")}diff --git a/backend/cli/admin/reset_mfa_test.go b/backend/cli/admin/reset_mfa_test.goindex b2cf10ce7..cc004390f 100644--- a/backend/cli/admin/reset_mfa_test.go+++ b/backend/cli/admin/reset_mfa_test.go@@ -6,6 +6,9 @@ import ("testing""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/passkey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session"+"github.com/libtnb/sqlite""github.com/stretchr/testify/require""gorm.io/gorm"@@ -13,7 +16,6 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/types/v2/auth")@@ -60,7 +62,7 @@ func TestResetMFACommandServiceStatePreservesPasskey(t *testing.T) {PasskeyMFAEnabled: true,}require.NoError(t, db.Create(user).Error)- passkey := &models.Passkey{+ passkeyRecord := &models.Passkey{BaseModel: models.BaseModel{ID: "mfa-reset-passkey"},UserID: user.ID,RPID: "arcane.example.test",@@ -68,8 +70,8 @@ func TestResetMFACommandServiceStatePreservesPasskey(t *testing.T) {PublicKey: []byte("public-key"),Name: "Alice's key",}- require.NoError(t, db.Create(passkey).Error)- sessionService := services.NewSessionService(db)+ require.NoError(t, db.Create(passkeyRecord).Error)+ sessionService := session.NewSessionService(db)firstSession, _, err := sessionService.CreateSession(ctx, user.ID, time.Now().Add(time.Hour), auth.SessionMeta{})require.NoError(t, err)secondSession, _, err := sessionService.CreateSession(ctx, user.ID, time.Now().Add(time.Hour), auth.SessionMeta{})@@ -93,12 +95,12 @@ func TestResetMFACommandServiceStatePreservesPasskey(t *testing.T) {Purpose: "mfa",UserID: new(user.ID),AuthTransactionID: new(transaction.ID),- RPID: passkey.RPID,+ RPID: passkeyRecord.RPID,SessionData: "{}",ExpiresAt: time.Now().Add(time.Minute),}).Error)- service := services.NewPasskeyService(db, &config.Config{AppUrl: "https://arcane.example.test"})+ service := passkey.NewPasskeyService(db, &config.Config{AppUrl: "https://arcane.example.test"})require.NoError(t, service.ResetMFAForUser(ctx, user.ID))var updatedUser models.User@@ -113,8 +115,8 @@ func TestResetMFACommandServiceStatePreservesPasskey(t *testing.T) {require.Zero(t, count)var preserved models.Passkey- require.NoError(t, db.Where("id = ?", passkey.ID).First(&preserved).Error)- require.Equal(t, passkey.ID, preserved.ID)+ require.NoError(t, db.Where("id = ?", passkeyRecord.ID).First(&preserved).Error)+ require.Equal(t, passkeyRecord.ID, preserved.ID)first, err := sessionService.GetSessionByID(ctx, firstSession.ID)require.NoError(t, err)require.NotNil(t, first.RevokedAt)diff --git a/backend/cli/admin/reset_password.go b/backend/cli/admin/reset_password.goindex af420a40e..264d960f8 100644--- a/backend/cli/admin/reset_password.go+++ b/backend/cli/admin/reset_password.go@@ -12,10 +12,12 @@ import ("github.com/spf13/cobra""golang.org/x/term"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/validation")@@ -157,11 +159,11 @@ func resetPasswordInternal(ctx context.Context, db *database.DB, username, passwreturn err}- roleService := services.NewRoleService(db)- userService := services.NewUserService(db).WithRoleService(roleService)+ roleService := role.NewRoleService(db)+ userService := user.NewUserService(db).WithRoleService(roleService)target, err := userService.GetUserByUsername(ctx, username)if err != nil {- if errors.Is(err, services.ErrUserNotFound) {+ if errors.Is(err, common.ErrUserNotFound) {return errors.Errorf("global administrator %q not found", username)}return errors.WrapIf(err, "failed to find user")diff --git a/backend/cli/admin/reset_password_test.go b/backend/cli/admin/reset_password_test.goindex 05fbb129b..5548bbd77 100644--- a/backend/cli/admin/reset_password_test.go+++ b/backend/cli/admin/reset_password_test.go@@ -12,7 +12,9 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/validation""github.com/getarcaneapp/arcane/types/v2/auth"@@ -37,12 +39,12 @@ func newResetPasswordTestDBInternal(t *testing.T) *database.DB {return db}-func createGlobalAdminInternal(t *testing.T, db *database.DB) (*models.User, *services.RoleService, *services.UserService) {+func createGlobalAdminInternal(t *testing.T, db *database.DB) (*models.User, *role.RoleService, *user.UserService) {t.Helper()ctx := context.Background()- roleService := services.NewRoleService(db)+ roleService := role.NewRoleService(db)require.NoError(t, roleService.EnsureBuiltInRoles(ctx))- userService := services.NewUserService(db).WithRoleService(roleService)+ userService := user.NewUserService(db).WithRoleService(roleService)passwordHash, err := userService.HashPassword("old-password")require.NoError(t, err)adminUser, err := userService.CreateUser(ctx, &models.User{@@ -63,7 +65,7 @@ func TestResetPasswordInternalResetsPasswordAndRevokesSessions(t *testing.T) {db := newResetPasswordTestDBInternal(t)ctx := context.Background()adminUser, _, userService := createGlobalAdminInternal(t, db)- sessionService := services.NewSessionService(db)+ sessionService := session.NewSessionService(db)firstSession, _, err := sessionService.CreateSession(ctx, adminUser.ID, time.Now().Add(time.Hour), auth.SessionMeta{})require.NoError(t, err)secondSession, _, err := sessionService.CreateSession(ctx, adminUser.ID, time.Now().Add(time.Hour), auth.SessionMeta{})@@ -89,7 +91,7 @@ func TestResetPasswordInternalRollsBackWhenSessionRevocationFails(t *testing.T)db := newResetPasswordTestDBInternal(t)ctx := context.Background()adminUser, _, userService := createGlobalAdminInternal(t, db)- sessionService := services.NewSessionService(db)+ sessionService := session.NewSessionService(db)session, _, err := sessionService.CreateSession(ctx, adminUser.ID, time.Now().Add(time.Hour), auth.SessionMeta{})require.NoError(t, err)require.NoError(t, db.Exec(`diff --git a/backend/cli/config_schema.go b/backend/cli/config_schema.goindex ae09acbf8..f8e2c809c 100644--- a/backend/cli/config_schema.go+++ b/backend/cli/config_schema.go@@ -3,7 +3,7 @@ package cliimport ("os"- "github.com/getarcaneapp/arcane/backend/v2/internal/configschema"+ "github.com/getarcaneapp/arcane/backend/v2/internal/config/schema""github.com/spf13/cobra")@@ -24,13 +24,13 @@ var configSchemaCmd = &cobra.Command{os.Exit(1)}- doc, err := configschema.GenerateWithSourceRoot(sourceRoot)+ doc, err := schema.GenerateWithSourceRoot(sourceRoot)if err != nil {cmd.PrintErrf("Error generating config schema: %v\n", err)os.Exit(1)}- output, err := configschema.MarshalJSON(doc)+ output, err := schema.MarshalJSON(doc)if err != nil {cmd.PrintErrf("Error encoding config schema: %v\n", err)os.Exit(1)diff --git a/backend/api/handlers/activities.go b/backend/internal/activity/handler.gosimilarity index 76%rename from backend/api/handlers/activities.gorename to backend/internal/activity/handler.goindex c99f99bc2..df37125d5 100644--- a/backend/api/handlers/activities.go+++ b/backend/internal/activity/handler.go@@ -1,4 +1,4 @@-package handlers+package activityimport ("context"@@ -13,20 +13,30 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"- "github.com/getarcaneapp/arcane/types/v2/activity"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+ activitytypes "github.com/getarcaneapp/arcane/types/v2/activity""github.com/getarcaneapp/arcane/types/v2/base"+ "github.com/samber/mo""go.getarcane.app/streams/agg""gorm.io/gorm")type ActivityHandler struct {- activityService *services.ActivityService- environmentService *services.EnvironmentService+ activityService *ActivityService+ environment EnvironmentDependencies+}++type EnvironmentDependencies struct {+ ProxyJSONRequest handlerutil.RemoteJSONProxy+ ListRemoteEnvironments func(context.Context) ([]models.Environment, error)+ GetActiveRemoteEnvironment func(string) mo.Option[models.Environment]+ ProxyJSONRequestForEnvironment func(context.Context, models.Environment, string, string, []byte, any) error+ ResolveEnvironmentName func(context.Context, string) string}type ListActivitiesInput struct {@@ -42,7 +52,7 @@ type ListActivitiesInput struct {}type ListActivitiesOutput struct {- Body base.Paginated[activity.Activity]+ Body base.Paginated[activitytypes.Activity]}type GetActivityInput struct {@@ -52,7 +62,7 @@ type GetActivityInput struct {}type GetActivityOutput struct {- Body base.ApiResponse[activity.Detail]+ Body base.ApiResponse[activitytypes.Detail]}type ClearActivityHistoryInput struct {@@ -60,7 +70,7 @@ type ClearActivityHistoryInput struct {}type ClearActivityHistoryOutput struct {- Body base.ApiResponse[activity.ClearHistoryResult]+ Body base.ApiResponse[activitytypes.ClearHistoryResult]}type StreamAllActivitiesInput struct {@@ -81,54 +91,55 @@ type CancelActivityInput struct {}type CancelActivityOutput struct {- Body base.ApiResponse[activity.Activity]+ Body base.ApiResponse[activitytypes.Activity]}-//nolint:dupl // Huma operation registrations are intentionally explicit.-func RegisterActivities(api huma.API, activityService *services.ActivityService, environmentService *services.EnvironmentService) {- h := &ActivityHandler{- activityService: activityService,- environmentService: environmentService,+func NewHandler(activityService *ActivityService, environment EnvironmentDependencies) *ActivityHandler {+ return &ActivityHandler{+ activityService: activityService,+ environment: environment,}+}- humamw.RegisterWithPermission(api, huma.Operation{+func RegisterActivities(api huma.API, h *ActivityHandler) {+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-activities",Method: http.MethodGet,Path: "/environments/{id}/activities",Summary: "List background activities",Description: "Get current and recent background activities for an environment",Tags: []string{"Activities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermActivitiesRead, h.ListActivities)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-activity",Method: http.MethodGet,Path: "/environments/{id}/activities/{activityId}",Summary: "Get background activity",Description: "Get a background activity with its recent output messages",Tags: []string{"Activities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermActivitiesRead, h.GetActivity)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "cancel-activity",Method: http.MethodPost,Path: "/environments/{id}/activities/{activityId}/cancel",Summary: "Cancel a background activity",Description: "Request cancellation of a running or queued background activity",Tags: []string{"Activities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermActivitiesCancel, h.CancelActivity)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "clear-activity-history",Method: http.MethodDelete,Path: "/environments/{id}/activities/history",Summary: "Clear background activity history",Description: "Delete completed background activity history for an environment",Tags: []string{"Activities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermActivitiesDelete, h.ClearHistory)}@@ -137,7 +148,7 @@ func (h *ActivityHandler) ListActivities(ctx context.Context, input *ListActivitreturn h.proxyListActivitiesInternal(ctx, input)}- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if input.Status != "" {params.Filters["status"] = input.Status}@@ -155,10 +166,10 @@ func (h *ActivityHandler) ListActivities(ctx context.Context, input *ListActivith.applyActivitySourceLabelsInternal(ctx, input.EnvironmentID, activities)return &ListActivitiesOutput{- Body: base.Paginated[activity.Activity]{+ Body: base.Paginated[activitytypes.Activity]{Success: true,Data: activities,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -181,7 +192,7 @@ func (h *ActivityHandler) GetActivity(ctx context.Context, input *GetActivityInph.applyActivitySourceLabelInternal(ctx, input.EnvironmentID, &detail.Activity)return &GetActivityOutput{- Body: base.ApiResponse[activity.Detail]{+ Body: base.ApiResponse[activitytypes.Detail]{Success: true,Data: *detail,},@@ -199,9 +210,9 @@ func (h *ActivityHandler) ClearHistory(ctx context.Context, input *ClearActivity}return &ClearActivityHistoryOutput{- Body: base.ApiResponse[activity.ClearHistoryResult]{+ Body: base.ApiResponse[activitytypes.ClearHistoryResult]{Success: true,- Data: activity.ClearHistoryResult{Deleted: deleted},+ Data: activitytypes.ClearHistoryResult{Deleted: deleted},},}, nil}@@ -220,7 +231,7 @@ func (h *ActivityHandler) CancelActivity(ctx context.Context, input *CancelActivswitch {case errors.Is(err, gorm.ErrRecordNotFound):return nil, huma.Error404NotFound("activity not found")- case errors.Is(err, services.ErrActivityNotCancelable):+ case errors.Is(err, ErrActivityNotCancelable):return nil, huma.Error409Conflict("activity is not running")default:return nil, huma.Error500InternalServerError(err.Error())@@ -229,7 +240,7 @@ func (h *ActivityHandler) CancelActivity(ctx context.Context, input *CancelActivh.applyActivitySourceLabelInternal(ctx, input.EnvironmentID, cancelled)return &CancelActivityOutput{- Body: base.ApiResponse[activity.Activity]{+ Body: base.ApiResponse[activitytypes.Activity]{Success: true,Data: *cancelled,},@@ -241,7 +252,7 @@ func (h *ActivityHandler) proxyCancelActivityInternal(ctx context.Context, inputif requestedBy := h.cancelRequestedByInternal(ctx, input.RequestedBy); requestedBy != "" {path += "?requestedBy=" + url.QueryEscape(requestedBy)}- out, err := proxyRemoteJSONInternal[base.ApiResponse[activity.Activity]](ctx, h.environmentService, input.EnvironmentID, http.MethodPost, path, nil)+ out, err := handlerutil.ProxyRemoteJSON[base.ApiResponse[activitytypes.Activity]](ctx, h.environment.ProxyJSONRequest, input.EnvironmentID, http.MethodPost, path, nil)if err != nil {return nil, err}@@ -264,14 +275,14 @@ func (h *ActivityHandler) cancelRequestedByInternal(ctx context.Context, forwardreturn strings.TrimSpace(forwarded)}-func (h *ActivityHandler) runLocalActivityStreamProducerInternal(ctx context.Context, limit int, events chan<- activity.StreamEvent) {+func (h *ActivityHandler) RunLocalStreamProducer(ctx context.Context, limit int, events chan<- activitytypes.StreamEvent) {sendSnapshot := func() bool {activities, _, err := h.activityService.ListActivitiesPaginated(ctx, "0", pagination.QueryParams{Params: pagination.Params{Limit: resolveActivityStreamLimitInternal(limit)},})if err != nil {if ctx.Err() == nil {- agg.Send(ctx, events, activity.StreamEvent{+ agg.Send(ctx, events, activitytypes.StreamEvent{Type: "error",EnvironmentID: "0",Error: err.Error(),@@ -281,7 +292,7 @@ func (h *ActivityHandler) runLocalActivityStreamProducerInternal(ctx context.Conreturn false}h.applyActivitySourceLabelsInternal(ctx, "0", activities)- return agg.Send(ctx, events, activity.StreamEvent{+ return agg.Send(ctx, events, activitytypes.StreamEvent{Type: "snapshot",EnvironmentID: "0",Activities: activities,@@ -318,13 +329,13 @@ func (h *ActivityHandler) runLocalActivityStreamProducerInternal(ctx context.Con}}-// runRemoteActivityStreamPollersInternal keeps one poller goroutine per+// RunRemoteStreamPollers keeps one poller goroutine per// enabled remote environment, re-listing periodically so environments added// or removed while the stream is open are picked up without a reconnect.-func (h *ActivityHandler) runRemoteActivityStreamPollersInternal(ctx context.Context, ps *authz.PermissionSet, limit int, events chan<- activity.StreamEvent) {+func (h *ActivityHandler) RunRemoteStreamPollers(ctx context.Context, ps *authz.PermissionSet, limit int, events chan<- activitytypes.StreamEvent) {agg.ReconcilePollersByKey(ctx,func(ctx context.Context) ([]models.Environment, error) {- environments, err := h.environmentService.ListRemoteEnvironments(ctx)+ environments, err := h.environment.ListRemoteEnvironments(ctx)if err != nil {return nil, err}@@ -357,7 +368,7 @@ func activityStreamEnvironmentVersionInternal(environment models.Environment) st// activitySnapshotFingerprintInternal hashes the fields that affect what the// client renders, so a poller can skip re-sending a snapshot identical to the// previous one.-func activitySnapshotFingerprintInternal(items []activity.Activity) string {+func activitySnapshotFingerprintInternal(items []activitytypes.Activity) string {hash := fnv.New64a()writeField := func(value string) {_, _ = hash.Write([]byte(value))@@ -382,7 +393,7 @@ func activitySnapshotFingerprintInternal(items []activity.Activity) string {return strconv.FormatUint(hash.Sum64(), 16)}-func (h *ActivityHandler) runRemoteActivityStreamPollerInternal(ctx context.Context, environment models.Environment, limit int, events chan<- activity.StreamEvent) {+func (h *ActivityHandler) runRemoteActivityStreamPollerInternal(ctx context.Context, environment models.Environment, limit int, events chan<- activitytypes.StreamEvent) {environmentID := environment.IDlastError := ""lastFingerprint := ""@@ -392,9 +403,9 @@ func (h *ActivityHandler) runRemoteActivityStreamPollerInternal(ctx context.Contdefer cancelPoll()currentEnvironment := environment- if h.environmentService != nil {+ if h.environment.GetActiveRemoteEnvironment != nil {var ok bool- currentEnvironment, ok = h.environmentService.GetActiveRemoteEnvironmentSnapshot(environmentID).Get()+ currentEnvironment, ok = h.environment.GetActiveRemoteEnvironment(environmentID).Get()if !ok {return}@@ -413,7 +424,7 @@ func (h *ActivityHandler) runRemoteActivityStreamPollerInternal(ctx context.Cont// once per distinct message and keep polling.if msg := err.Error(); msg != lastError {lastError = msg- agg.Send(ctx, events, activity.StreamEvent{+ agg.Send(ctx, events, activitytypes.StreamEvent{Type: "error",EnvironmentID: environmentID,Error: msg,@@ -429,7 +440,7 @@ func (h *ActivityHandler) runRemoteActivityStreamPollerInternal(ctx context.Contif fingerprint == lastFingerprint {return}- if agg.Send(ctx, events, activity.StreamEvent{+ if agg.Send(ctx, events, activitytypes.StreamEvent{Type: "snapshot",EnvironmentID: environmentID,Activities: output.Body.Data,@@ -456,7 +467,7 @@ func (h *ActivityHandler) runRemoteActivityStreamPollerInternal(ctx context.Contfunc (h *ActivityHandler) proxyListActivitiesInternal(ctx context.Context, input *ListActivitiesInput) (*ListActivitiesOutput, error) {path := "/api/environments/0/activities?" + activityListQueryInternal(input).Encode()- out, err := proxyRemoteJSONInternal[base.Paginated[activity.Activity]](ctx, h.environmentService, input.EnvironmentID, http.MethodGet, path, nil)+ out, err := handlerutil.ProxyRemoteJSON[base.Paginated[activitytypes.Activity]](ctx, h.environment.ProxyJSONRequest, input.EnvironmentID, http.MethodGet, path, nil)if err != nil {return nil, err}@@ -466,9 +477,9 @@ func (h *ActivityHandler) proxyListActivitiesInternal(ctx context.Context, inputfunc (h *ActivityHandler) proxyListActivitiesForEnvironmentInternal(ctx context.Context, environment models.Environment, input *ListActivitiesInput) (*ListActivitiesOutput, error) {path := "/api/environments/0/activities?" + activityListQueryInternal(input).Encode()- var out base.Paginated[activity.Activity]- if err := h.environmentService.ProxyJSONRequestForEnvironment(ctx, environment, http.MethodGet, path, nil, &out); err != nil {- return nil, translateRemoteProxyErrorInternal(err)+ var out base.Paginated[activitytypes.Activity]+ if err := h.environment.ProxyJSONRequestForEnvironment(ctx, environment, http.MethodGet, path, nil, &out); err != nil {+ return nil, handlerutil.TranslateRemoteProxyError(err)}applyActivitySourceLabelsForEnvironmentInternal(environment, out.Data)return &ListActivitiesOutput{Body: out}, nil@@ -476,7 +487,7 @@ func (h *ActivityHandler) proxyListActivitiesForEnvironmentInternal(ctx context.func (h *ActivityHandler) proxyGetActivityInternal(ctx context.Context, input *GetActivityInput) (*GetActivityOutput, error) {path := fmt.Sprintf("/api/environments/0/activities/%s?limit=%d", url.PathEscape(input.ActivityID), input.Limit)- out, err := proxyRemoteJSONInternal[base.ApiResponse[activity.Detail]](ctx, h.environmentService, input.EnvironmentID, http.MethodGet, path, nil)+ out, err := handlerutil.ProxyRemoteJSON[base.ApiResponse[activitytypes.Detail]](ctx, h.environment.ProxyJSONRequest, input.EnvironmentID, http.MethodGet, path, nil)if err != nil {return nil, err}@@ -485,26 +496,26 @@ func (h *ActivityHandler) proxyGetActivityInternal(ctx context.Context, input *G}func (h *ActivityHandler) proxyClearHistoryInternal(ctx context.Context, input *ClearActivityHistoryInput) (*ClearActivityHistoryOutput, error) {- out, err := proxyRemoteJSONInternal[base.ApiResponse[activity.ClearHistoryResult]](ctx, h.environmentService, input.EnvironmentID, http.MethodDelete, "/api/environments/0/activities/history", nil)+ out, err := handlerutil.ProxyRemoteJSON[base.ApiResponse[activitytypes.ClearHistoryResult]](ctx, h.environment.ProxyJSONRequest, input.EnvironmentID, http.MethodDelete, "/api/environments/0/activities/history", nil)if err != nil {return nil, err}return &ClearActivityHistoryOutput{Body: *out}, nil}-func (h *ActivityHandler) applyActivitySourceLabelsInternal(ctx context.Context, environmentID string, activities []activity.Activity) {+func (h *ActivityHandler) applyActivitySourceLabelsInternal(ctx context.Context, environmentID string, activities []activitytypes.Activity) {sourceID, sourceName := h.resolveActivitySourceInternal(ctx, environmentID)for i := range activities {applyActivitySourceInternal(&activities[i], sourceID, sourceName)}}-func (h *ActivityHandler) applyActivitySourceLabelInternal(ctx context.Context, environmentID string, item *activity.Activity) {+func (h *ActivityHandler) applyActivitySourceLabelInternal(ctx context.Context, environmentID string, item *activitytypes.Activity) {sourceID, sourceName := h.resolveActivitySourceInternal(ctx, environmentID)applyActivitySourceInternal(item, sourceID, sourceName)}-func (h *ActivityHandler) applyActivityStreamEventSourceLabelInternal(ctx context.Context, environmentID string, event *activity.StreamEvent) {+func (h *ActivityHandler) applyActivityStreamEventSourceLabelInternal(ctx context.Context, environmentID string, event *activitytypes.StreamEvent) {if event == nil {return}@@ -517,32 +528,32 @@ func (h *ActivityHandler) applyActivityStreamEventSourceLabelInternal(ctx contex}}-func applyActivitySourceLabelsForEnvironmentInternal(environment models.Environment, activities []activity.Activity) {- sourceID, sourceName := activitySourceFromEnvironmentInternal(environment)+func applyActivitySourceLabelsForEnvironmentInternal(environmentModel models.Environment, activities []activitytypes.Activity) {+ sourceID, sourceName := activitySourceFromEnvironmentInternal(environmentModel)for i := range activities {applyActivitySourceInternal(&activities[i], sourceID, sourceName)}}-func activitySourceFromEnvironmentInternal(environment models.Environment) (string, string) {- environmentID := environment.ID+func activitySourceFromEnvironmentInternal(environmentModel models.Environment) (string, string) {+ environmentID := environmentModel.IDif environmentID == "" {- environmentID = services.LocalEnvironmentID+ environmentID = environment.LocalEnvironmentID}- return environmentID, services.EnvironmentDisplayName(environmentID, environment.Name)+ return environmentID, environment.DisplayName(environmentID, environmentModel.Name)}func (h *ActivityHandler) resolveActivitySourceInternal(ctx context.Context, environmentID string) (string, string) {if environmentID == "" {- environmentID = services.LocalEnvironmentID+ environmentID = environment.LocalEnvironmentID}- if h.environmentService != nil {- return environmentID, h.environmentService.ResolveEnvironmentName(ctx, environmentID)+ if h.environment.ResolveEnvironmentName != nil {+ return environmentID, h.environment.ResolveEnvironmentName(ctx, environmentID)}- return environmentID, services.EnvironmentDisplayName(environmentID, "")+ return environmentID, environment.DisplayName(environmentID, "")}-func applyActivitySourceInternal(item *activity.Activity, sourceID, sourceName string) {+func applyActivitySourceInternal(item *activitytypes.Activity, sourceID, sourceName string) {if item == nil {return}diff --git a/backend/internal/services/activity_limiter.go b/backend/internal/activity/limiter.gosimilarity index 94%rename from backend/internal/services/activity_limiter.gorename to backend/internal/activity/limiter.goindex d95528895..dfcd13fe9 100644--- a/backend/internal/services/activity_limiter.go+++ b/backend/internal/activity/limiter.go@@ -1,10 +1,11 @@-package services+package activityimport ("context""sync""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/samber/mo")@@ -23,7 +24,7 @@ const (// occupying capacity until they release, and waiters are re-evaluated against// the new limit.type activitySlotLimiter struct {- settings *SettingsService+ settings *settings.SettingsServicemu sync.Mutexstate map[string]*activityEnvironmentSlots@@ -37,7 +38,7 @@ type activityEnvironmentSlots struct {wake chan struct{}}-func newActivitySlotLimiterInternal(settings *SettingsService) *activitySlotLimiter {+func newActivitySlotLimiterInternal(settings *settings.SettingsService) *activitySlotLimiter {return &activitySlotLimiter{settings: settings,state: map[string]*activityEnvironmentSlots{},diff --git a/backend/internal/activity/module.go b/backend/internal/activity/module.gonew file mode 100644index 000000000..735d39fbc--- /dev/null+++ b/backend/internal/activity/module.go@@ -0,0 +1,48 @@+// Package activity owns background activity persistence, execution tracking,+// streaming, and the HTTP surface used to inspect and cancel work.+package activity++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+)++type Dependencies struct {+ DB *database.DB+ Settings *settings.SettingsService+ Environment EnvironmentDependencies+}++type Module struct {+ service *ActivityService+ handler *ActivityHandler+}++func New(deps Dependencies) *Module {+ service := NewActivityService(deps.DB, deps.Settings)+ return &Module{service: service, handler: NewHandler(service, deps.Environment)}+}++func (m *Module) Service() *ActivityService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) Handler() *ActivityHandler {+ if m == nil {+ return nil+ }+ return m.handler+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterActivities(api, NewHandler(nil, EnvironmentDependencies{}))+ return+ }+ RegisterActivities(api, m.handler)+}diff --git a/backend/internal/services/activity_service.go b/backend/internal/activity/service.gosimilarity index 99%rename from backend/internal/services/activity_service.gorename to backend/internal/activity/service.goindex 5fef0a781..e017f0d9e 100644--- a/backend/internal/services/activity_service.go+++ b/backend/internal/activity/service.go@@ -1,4 +1,4 @@-package services+package activityimport ("context"@@ -14,10 +14,12 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/dbutil"activitytypes "github.com/getarcaneapp/arcane/types/v2/activity""github.com/samber/mo""gorm.io/gorm"@@ -183,7 +185,7 @@ type StartActivityRequest = activitylib.StartRequesttype UpdateActivityRequest = activitylib.UpdateRequesttype AppendActivityMessageRequest = activitylib.AppendMessageRequest-func NewActivityService(db *database.DB, settingsService *SettingsService) *ActivityService {+func NewActivityService(db *database.DB, settingsService *settings.SettingsService) *ActivityService {return &ActivityService{db: db,subscribers: map[int]*activitySubscriber{},@@ -593,7 +595,7 @@ func (s *ActivityService) CompleteActivity(ctx context.Context, activityID strinif writeErr == nil {break}- if attempt == completeWriteAttempts || !isRetryableDBWriteErrorInternal(writeErr) {+ if attempt == completeWriteAttempts || !dbutil.IsRetryableWriteError(writeErr) {return nil, writeErr}slog.WarnContext(ctx, "retrying activity terminal status write", "activityId", activityID, "attempt", attempt, "error", writeErr)diff --git a/backend/internal/services/activity_service_test.go b/backend/internal/activity/service_test.gosimilarity index 96%rename from backend/internal/services/activity_service_test.gorename to backend/internal/activity/service_test.goindex eec93654c..823f02cb0 100644--- a/backend/internal/services/activity_service_test.go+++ b/backend/internal/activity/service_test.go@@ -1,4 +1,4 @@-package services+package activityimport ("context"@@ -7,10 +7,13 @@ import ("github.com/libtnb/sqlite""github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest""gorm.io/gorm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"@@ -379,6 +382,25 @@ func setupQueuedActivityServiceInternal(t *testing.T) (*ActivityService, contextreturn NewActivityService(wrapped, settings), ctx}+func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ executor, err := actors.NewExecutor(t.Context(), runtime, "activity-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "activity-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, executor.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, executor, effects)+}+func TestActivityServiceQueuedActivityFlipsToRunningWhenSlotFreesInternal(t *testing.T) {service, ctx := setupQueuedActivityServiceInternal(t)diff --git a/backend/api/handlers/apikeys.go b/backend/internal/apikey/handler.gosimilarity index 78%rename from backend/api/handlers/apikeys.gorename to backend/internal/apikey/handler.goindex 88b82e633..e9c9f6214 100644--- a/backend/api/handlers/apikeys.go+++ b/backend/internal/apikey/handler.go@@ -1,4 +1,4 @@-package handlers+package apikeyimport ("context"@@ -7,16 +7,16 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/getarcaneapp/arcane/types/v2/apikey"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+ apikeytypes "github.com/getarcaneapp/arcane/types/v2/apikey""github.com/getarcaneapp/arcane/types/v2/base")// ApiKeyHandler provides Huma-based API key management endpoints.type ApiKeyHandler struct {- apiKeyService *services.ApiKeyService+ apiKeyService *ApiKeyService}// --- Huma Input/Output Wrappers ---@@ -30,19 +30,19 @@ type ListApiKeysInput struct {}type ListApiKeysOutput struct {- Body base.Paginated[apikey.ApiKey]+ Body base.Paginated[apikeytypes.ApiKey]}type CreateApiKeyInput struct {- Body apikey.CreateApiKey+ Body apikeytypes.CreateApiKey}type CreateMyApiKeyInput struct {- Body apikey.CreateUserApiKey+ Body apikeytypes.CreateUserApiKey}type CreateApiKeyOutput struct {- Body base.ApiResponse[apikey.ApiKeyCreatedDto]+ Body base.ApiResponse[apikeytypes.ApiKeyCreatedDto]}type GetApiKeyInput struct {@@ -50,16 +50,16 @@ type GetApiKeyInput struct {}type GetApiKeyOutput struct {- Body base.ApiResponse[apikey.ApiKey]+ Body base.ApiResponse[apikeytypes.ApiKey]}type UpdateApiKeyInput struct {ID string `path:"id" doc:"API key ID"`- Body apikey.UpdateApiKey+ Body apikeytypes.UpdateApiKey}type UpdateApiKeyOutput struct {- Body base.ApiResponse[apikey.ApiKey]+ Body base.ApiResponse[apikeytypes.ApiKey]}type DeleteApiKeyInput struct {@@ -71,11 +71,11 @@ type DeleteApiKeyOutput struct {}type ListMyApiKeysOutput struct {- Body base.ApiResponse[[]apikey.ApiKey]+ Body base.ApiResponse[[]apikeytypes.ApiKey]}// RegisterApiKeys registers API key management routes using Huma.-func RegisterApiKeys(api huma.API, apiKeyService *services.ApiKeyService) {+func RegisterApiKeys(api huma.API, apiKeyService *ApiKeyService) {h := &ApiKeyHandler{apiKeyService: apiKeyService,}@@ -87,8 +87,8 @@ func RegisterApiKeys(api huma.API, apiKeyService *services.ApiKeyService) {Summary: "List API keys",Description: "Get a paginated list of API keys",Tags: []string{"API Keys"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermApiKeysList),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermApiKeysList),}, h.ListApiKeys)huma.Register(api, huma.Operation{@@ -98,8 +98,8 @@ func RegisterApiKeys(api huma.API, apiKeyService *services.ApiKeyService) {Summary: "Create an API key",Description: "Create a new API key for programmatic access",Tags: []string{"API Keys"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermApiKeysCreate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermApiKeysCreate),}, h.CreateApiKey)huma.Register(api, huma.Operation{@@ -109,8 +109,8 @@ func RegisterApiKeys(api huma.API, apiKeyService *services.ApiKeyService) {Summary: "Get an API key",Description: "Get details of a specific API key by ID",Tags: []string{"API Keys"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermApiKeysRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermApiKeysRead),}, h.GetApiKey)huma.Register(api, huma.Operation{@@ -120,8 +120,8 @@ func RegisterApiKeys(api huma.API, apiKeyService *services.ApiKeyService) {Summary: "Update an API key",Description: "Update an existing API key's details",Tags: []string{"API Keys"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermApiKeysUpdate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermApiKeysUpdate),}, h.UpdateApiKey)huma.Register(api, huma.Operation{@@ -131,8 +131,8 @@ func RegisterApiKeys(api huma.API, apiKeyService *services.ApiKeyService) {Summary: "Delete an API key",Description: "Delete an API key by ID",Tags: []string{"API Keys"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermApiKeysDelete),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermApiKeysDelete),}, h.DeleteApiKey)// Self-service endpoints — no admin permission required, scoped to the@@ -144,7 +144,7 @@ func RegisterApiKeys(api huma.API, apiKeyService *services.ApiKeyService) {Summary: "List my API keys",Description: "List API keys owned by the current user",Tags: []string{"API Keys"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.ListMyApiKeys)// Personal keys inherit the owner's permissions, so creating or deleting@@ -177,7 +177,7 @@ func RegisterApiKeys(api huma.API, apiKeyService *services.ApiKeyService) {// ListApiKeys returns a paginated list of API keys.func (h *ApiKeyHandler) ListApiKeys(ctx context.Context, input *ListApiKeysInput) (*ListApiKeysOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)apiKeys, paginationResp, err := h.apiKeyService.ListApiKeys(ctx, params)if err != nil {@@ -185,10 +185,10 @@ func (h *ApiKeyHandler) ListApiKeys(ctx context.Context, input *ListApiKeysInput}return &ListApiKeysOutput{- Body: base.Paginated[apikey.ApiKey]{+ Body: base.Paginated[apikeytypes.ApiKey]{Success: true,Data: apiKeys,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -196,22 +196,22 @@ func (h *ApiKeyHandler) ListApiKeys(ctx context.Context, input *ListApiKeysInput// CreateApiKey creates a new scoped API key. Requested grants are capped by// the calling credential's effective permissions.func (h *ApiKeyHandler) CreateApiKey(ctx context.Context, input *CreateApiKeyInput) (*CreateApiKeyOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}- callerPerms, _ := humamw.PermissionsFromContext(ctx)+ callerPerms, _ := middleware.PermissionsFromContext(ctx)apiKey, err := h.apiKeyService.CreateApiKey(ctx, user.ID, callerPerms, input.Body)if err != nil {- if errors.Is(err, services.ErrApiKeyPermissionEscalation) {+ if errors.Is(err, ErrApiKeyPermissionEscalation) {return nil, huma.Error403Forbidden(err.Error())}return nil, huma.Error500InternalServerError("Failed to create API key")}return &CreateApiKeyOutput{- Body: base.ApiResponse[apikey.ApiKeyCreatedDto]{+ Body: base.ApiResponse[apikeytypes.ApiKeyCreatedDto]{Success: true,Data: *apiKey,},@@ -226,7 +226,7 @@ func (h *ApiKeyHandler) GetApiKey(ctx context.Context, input *GetApiKeyInput) (*}return &GetApiKeyOutput{- Body: base.ApiResponse[apikey.ApiKey]{+ Body: base.ApiResponse[apikeytypes.ApiKey]{Success: true,Data: *apiKey,},@@ -235,30 +235,30 @@ func (h *ApiKeyHandler) GetApiKey(ctx context.Context, input *GetApiKeyInput) (*// UpdateApiKey updates an existing API key.func (h *ApiKeyHandler) UpdateApiKey(ctx context.Context, input *UpdateApiKeyInput) (*UpdateApiKeyOutput, error) {- if _, err := requireUserInternal(ctx); err != nil {+ if _, err := handlerutil.RequireUser(ctx); err != nil {return nil, err}- callerPerms, _ := humamw.PermissionsFromContext(ctx)+ callerPerms, _ := middleware.PermissionsFromContext(ctx)apiKey, err := h.apiKeyService.UpdateApiKey(ctx, callerPerms, input.ID, input.Body)if err != nil {- if errors.Is(err, services.ErrApiKeyNotFound) {+ if errors.Is(err, ErrApiKeyNotFound) {return nil, huma.Error404NotFound("API key not found")}- if errors.Is(err, services.ErrApiKeyProtected) {+ if errors.Is(err, ErrApiKeyProtected) {return nil, huma.Error403Forbidden("static API keys cannot be updated")}- if errors.Is(err, services.ErrApiKeyPermissionEscalation) {+ if errors.Is(err, ErrApiKeyPermissionEscalation) {return nil, huma.Error403Forbidden(err.Error())}- if errors.Is(err, services.ErrApiKeyPersonalNoGrants) {+ if errors.Is(err, ErrApiKeyPersonalNoGrants) {return nil, huma.Error400BadRequest(err.Error())}return nil, huma.Error500InternalServerError("Failed to update API key")}return &UpdateApiKeyOutput{- Body: base.ApiResponse[apikey.ApiKey]{+ Body: base.ApiResponse[apikeytypes.ApiKey]{Success: true,Data: *apiKey,},@@ -268,10 +268,10 @@ func (h *ApiKeyHandler) UpdateApiKey(ctx context.Context, input *UpdateApiKeyInp// DeleteApiKey deletes an API key.func (h *ApiKeyHandler) DeleteApiKey(ctx context.Context, input *DeleteApiKeyInput) (*DeleteApiKeyOutput, error) {if err := h.apiKeyService.DeleteApiKey(ctx, input.ID); err != nil {- if errors.Is(err, services.ErrApiKeyNotFound) {+ if errors.Is(err, ErrApiKeyNotFound) {return nil, huma.Error404NotFound("API key not found")}- if errors.Is(err, services.ErrApiKeyProtected) {+ if errors.Is(err, ErrApiKeyProtected) {return nil, huma.Error403Forbidden("static API keys cannot be deleted")}return nil, huma.Error500InternalServerError("Failed to delete API key")@@ -289,7 +289,7 @@ func (h *ApiKeyHandler) DeleteApiKey(ctx context.Context, input *DeleteApiKeyInp// ListMyApiKeys lists API keys owned by the current user (self-service).func (h *ApiKeyHandler) ListMyApiKeys(ctx context.Context, input *struct{}) (*ListMyApiKeysOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -300,7 +300,7 @@ func (h *ApiKeyHandler) ListMyApiKeys(ctx context.Context, input *struct{}) (*Li}return &ListMyApiKeysOutput{- Body: base.ApiResponse[[]apikey.ApiKey]{+ Body: base.ApiResponse[[]apikeytypes.ApiKey]{Success: true,Data: keys,},@@ -313,11 +313,11 @@ func (h *ApiKeyHandler) ListMyApiKeys(ctx context.Context, input *struct{}) (*Lifunc (h *ApiKeyHandler) CreateMyApiKey(ctx context.Context, input *CreateMyApiKeyInput) (*CreateApiKeyOutput, error) {// Defense in depth alongside the BearerAuth-only Security requirement:// only session auth sets a session ID, so API-key and sudo callers stop here.- if _, ok := humamw.GetCurrentSessionIDFromContext(ctx); !ok {+ if _, ok := middleware.GetCurrentSessionIDFromContext(ctx); !ok {return nil, huma.Error403Forbidden("personal API keys can only be managed from an interactive session")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -328,7 +328,7 @@ func (h *ApiKeyHandler) CreateMyApiKey(ctx context.Context, input *CreateMyApiKe}return &CreateApiKeyOutput{- Body: base.ApiResponse[apikey.ApiKeyCreatedDto]{+ Body: base.ApiResponse[apikeytypes.ApiKeyCreatedDto]{Success: true,Data: *apiKey,},@@ -341,11 +341,11 @@ func (h *ApiKeyHandler) CreateMyApiKey(ctx context.Context, input *CreateMyApiKefunc (h *ApiKeyHandler) DeleteMyApiKey(ctx context.Context, input *DeleteApiKeyInput) (*DeleteApiKeyOutput, error) {// Defense in depth alongside the BearerAuth-only Security requirement:// only session auth sets a session ID, so API-key and sudo callers stop here.- if _, ok := humamw.GetCurrentSessionIDFromContext(ctx); !ok {+ if _, ok := middleware.GetCurrentSessionIDFromContext(ctx); !ok {return nil, huma.Error403Forbidden("personal API keys can only be managed from an interactive session")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -359,10 +359,10 @@ func (h *ApiKeyHandler) DeleteMyApiKey(ctx context.Context, input *DeleteApiKeyI}if err := h.apiKeyService.DeleteApiKey(ctx, input.ID); err != nil {- if errors.Is(err, services.ErrApiKeyNotFound) {+ if errors.Is(err, ErrApiKeyNotFound) {return nil, huma.Error404NotFound("API key not found")}- if errors.Is(err, services.ErrApiKeyProtected) {+ if errors.Is(err, ErrApiKeyProtected) {return nil, huma.Error403Forbidden("this API key cannot be deleted")}return nil, huma.Error500InternalServerError("Failed to delete API key")diff --git a/backend/internal/apikey/module.go b/backend/internal/apikey/module.gonew file mode 100644index 000000000..8a82c5b0a--- /dev/null+++ b/backend/internal/apikey/module.go@@ -0,0 +1,40 @@+// Package apikey owns API-key persistence, permission grants, validation, and+// the API-key HTTP surface.+package apikey++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user"+)++type Dependencies struct {+ DB *database.DB+ User *user.UserService+ Role *role.RoleService+}++type Module struct {+ service *ApiKeyService+}++func New(deps Dependencies) *Module {+ return &Module{service: NewApiKeyService(deps.DB, deps.User).WithRoleService(deps.Role)}+}++func (m *Module) Service() *ApiKeyService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterApiKeys(api, nil)+ return+ }+ RegisterApiKeys(api, m.service)+}diff --git a/backend/internal/services/api_key_service.go b/backend/internal/apikey/service.gosimilarity index 96%rename from backend/internal/services/api_key_service.gorename to backend/internal/apikey/service.goindex a7a90dcbf..a7fa00d53 100644--- a/backend/internal/services/api_key_service.go+++ b/backend/internal/apikey/service.go@@ -1,4 +1,4 @@-package services+package apikeyimport ("context"@@ -10,8 +10,11 @@ import ("emperror.dev/errors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/dbutil"@@ -43,25 +46,25 @@ var defaultAdminAPIKeyDescription = func() *string {type ApiKeyService struct {db *database.DB- userService *UserService- roleService *RoleService- argon2Params *Argon2Params+ userService *user.UserService+ roleService *role.RoleService+ argon2Params *user.Argon2Params}-func NewApiKeyService(db *database.DB, userService *UserService) *ApiKeyService {+func NewApiKeyService(db *database.DB, userService *user.UserService) *ApiKeyService {return &ApiKeyService{db: db,userService: userService,- argon2Params: DefaultArgon2Params(),+ argon2Params: user.DefaultArgon2Params(),}}-// WithRoleService wires the RoleService dependency. Separated from the+// WithRoleService wires the role.RoleService dependency. Separated from the// constructor to break the bootstrap-ordering cycle between ApiKeyService and-// RoleService (RoleService.BackfillApiKeyPermissions needs ApiKeyService to+// role.RoleService (role.RoleService.BackfillApiKeyPermissions needs ApiKeyService to// exist when it runs, while permission-validated CreateApiKey needs the-// RoleService).-func (s *ApiKeyService) WithRoleService(roleService *RoleService) *ApiKeyService {+// role.RoleService).+func (s *ApiKeyService) WithRoleService(roleService *role.RoleService) *ApiKeyService {s.roleService = roleServicereturn s}@@ -149,7 +152,7 @@ const ErrApiKeyPersonalNoGrants = errors.Sentinel("personal API keys inherit the// permissions, so no holder of apikeys:create/update — sudo included — can// push a key above what its owner is allowed to do. Complements the caller-set// check: the caller cap uses the credential's live context permissions, this-// one re-resolves the owner's roles from the DB. Skipped when RoleService is+// one re-resolves the owner's roles from the DB. Skipped when role.RoleService is// unavailable (bootstrap path only, where no human caller exists).func (s *ApiKeyService) validateGrantsAgainstOwnerInternal(ctx context.Context, ownerID string, grants []apikey.PermissionGrant) error {if s.roleService == nil || len(grants) == 0 {@@ -340,7 +343,7 @@ func (s *ApiKeyService) CreateDefaultAdminAPIKey(ctx context.Context, userID, rafunc (s *ApiKeyService) getDefaultAdminUser(ctx context.Context) (*models.User, error) {adminUser, err := s.userService.GetUserByUsername(ctx, defaultAdminUsername)if err != nil {- if errors.Is(err, ErrUserNotFound) {+ if errors.Is(err, common.ErrUserNotFound) {slog.WarnContext(ctx, "Default admin user not found, skipping default admin API key reconciliation", "username", defaultAdminUsername)return nil, nil}@@ -670,7 +673,7 @@ func (s *ApiKeyService) UpdateApiKey(ctx context.Context, callerPerms *authz.Perreturn errors.WrapIf(err, "failed to update API key")}if req.Permissions != nil && s.roleService != nil {- if err := s.roleService.setApiKeyPermissionsInternal(ctx, tx, ak.ID, toApiKeyPermissionRowsInternal(ak.ID, req.Permissions)); err != nil {+ if err := s.roleService.SetApiKeyPermissionsInDB(ctx, tx, ak.ID, toApiKeyPermissionRowsInternal(ak.ID, req.Permissions)); err != nil {return errors.WrapIf(err, "failed to update api key permissions")}permissionsUpdated = true@@ -680,7 +683,7 @@ func (s *ApiKeyService) UpdateApiKey(ctx context.Context, callerPerms *authz.Perreturn nil, err}if permissionsUpdated {- s.roleService.apiKeyCache.Delete(ak.ID)+ s.roleService.InvalidateApiKey(ak.ID)}return &apikey.ApiKey{diff --git a/backend/internal/services/api_key_service_test.go b/backend/internal/apikey/service_test.gosimilarity index 93%rename from backend/internal/services/api_key_service_test.gorename to backend/internal/apikey/service_test.goindex 30c570e13..ba56b390f 100644--- a/backend/internal/services/api_key_service_test.go+++ b/backend/internal/apikey/service_test.go@@ -1,4 +1,4 @@-package services+package apikeyimport ("context"@@ -15,6 +15,8 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/types/v2/apikey"@@ -26,7 +28,17 @@ func setupAPIKeyServiceTestDB(t *testing.T) *database.DB {dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", strings.NewReplacer("/", "_", " ", "_").Replace(t.Name()))db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})require.NoError(t, err)- require.NoError(t, db.AutoMigrate(&models.User{}, &models.ApiKey{}, &models.Environment{}))+ require.NoError(t, db.AutoMigrate(+ &models.SettingVariable{},+ &models.User{},+ &models.UserSession{},+ &models.Environment{},+ &models.Role{},+ &models.UserRoleAssignment{},+ &models.ApiKey{},+ &models.ApiKeyPermission{},+ &models.OidcRoleMapping{},+ ))sqlDB, err := db.DB()require.NoError(t, err)@@ -36,20 +48,24 @@ func setupAPIKeyServiceTestDB(t *testing.T) *database.DB {return &database.DB{DB: db}}-func setupAPIKeyService(t *testing.T) (*ApiKeyService, *database.DB, *UserService) {+func setupAPIKeyService(t *testing.T) (*ApiKeyService, *database.DB, *user.UserService) {t.Helper()db := setupAPIKeyServiceTestDB(t)- userService := NewUserService(db)+ userService := user.NewUserService(db)return NewApiKeyService(db, userService), db, userService}-func createTestAPIKeyUser(t *testing.T, ctx context.Context, userService *UserService, id string) *models.User {+func createTestAPIKeyUser(t *testing.T, ctx context.Context, userService *user.UserService, id string, usernames ...string) *models.User {t.Helper()+ username := fmt.Sprintf("user-%s", id)+ if len(usernames) > 0 {+ username = usernames[0]+ }user := &models.User{BaseModel: models.BaseModel{ID: id},- Username: fmt.Sprintf("user-%s", id),+ Username: username,}created, err := userService.CreateUser(ctx, user)@@ -87,7 +103,7 @@ func invalidateAPIKey(rawKey string) string {return rawKey[:len(rawKey)-1] + "0"}-func createDefaultAdminUser(t *testing.T, ctx context.Context, userService *UserService) *models.User {+func createDefaultAdminUser(t *testing.T, ctx context.Context, userService *user.UserService) *models.User {t.Helper()user := &models.User{@@ -106,8 +122,8 @@ func TestListApiKeysPermissionQueryCountIsConstant(t *testing.T) {for _, keyCount := range []int{1, 5} {t.Run(fmt.Sprintf("%d_keys", keyCount), func(t *testing.T) {- db := setupAuthServiceTestDB(t)- service := NewApiKeyService(db, NewUserService(db)).WithRoleService(NewRoleService(db))+ db := setupAPIKeyServiceTestDB(t)+ service := NewApiKeyService(db, user.NewUserService(db)).WithRoleService(role.NewRoleService(db))userID := "query-count-user"apiKeys := make([]models.ApiKey, keyCount)@@ -246,15 +262,17 @@ func TestUpdateApiKeyRejectsStaticKey(t *testing.T) {func TestUpdateApiKeyRollsBackMetadataWhenPermissionUpdateFails(t *testing.T) {ctx := context.Background()- db := setupAuthServiceTestDB(t)+ db := setupAPIKeyServiceTestDB(t)require.NoError(t, db.Exec("CREATE UNIQUE INDEX IF NOT EXISTS idx_akp_uniq ON api_key_permissions(api_key_id, permission, COALESCE(environment_id, ''))").Error)- roleSvc := NewRoleService(db)+ roleSvc := role.NewRoleService(db)require.NoError(t, roleSvc.EnsureBuiltInRoles(ctx))- userSvc := NewUserService(db).WithRoleService(roleSvc)+ userSvc := user.NewUserService(db).WithRoleService(roleSvc)service := NewApiKeyService(db, userSvc).WithRoleService(roleSvc)- admin := createTestUser(t, userSvc, "admin-update-rollback", "admin-update-rollback")- grantGlobalAdmin(t, roleSvc, admin.ID)+ admin := createTestAPIKeyUser(t, ctx, userSvc, "admin-update-rollback", "admin-update-rollback")+ require.NoError(t, roleSvc.SetUserAssignments(ctx, admin.ID, []models.UserRoleAssignment{+ {RoleID: authz.BuiltInRoleAdmin, EnvironmentID: nil},+ }))created, err := service.CreateApiKey(ctx, admin.ID, authz.SudoPermissionSet(), apikey.CreateApiKey{Name: "original"})require.NoError(t, err)@@ -299,14 +317,14 @@ func TestCreateApiKeyRejectsGrantsBeyondCallerPermissions(t *testing.T) {func TestApiKeyGrantsAreCappedByOwnerRoles(t *testing.T) {ctx := context.Background()- db := setupAuthServiceTestDB(t)+ db := setupAPIKeyServiceTestDB(t)- roleSvc := NewRoleService(db)+ roleSvc := role.NewRoleService(db)require.NoError(t, roleSvc.EnsureBuiltInRoles(ctx))- userSvc := NewUserService(db).WithRoleService(roleSvc)+ userSvc := user.NewUserService(db).WithRoleService(roleSvc)service := NewApiKeyService(db, userSvc).WithRoleService(roleSvc)// Owner has no roles at all — their permission ceiling is empty.- owner := createTestUser(t, userSvc, "roleless-owner", "roleless-owner")+ owner := createTestAPIKeyUser(t, ctx, userSvc, "roleless-owner", "roleless-owner")// Create path: even a sudo caller cannot mint a key above the owner's roles._, err := service.CreateApiKey(ctx, owner.ID, authz.SudoPermissionSet(), apikey.CreateApiKey{@@ -690,15 +708,17 @@ func TestGetEnvironmentByAPIKeyExpiredDoesNotUpdateLastUsedAt(t *testing.T) {func TestCreateEnvironmentApiKeySeedsAllPermissionsScopedToEnv(t *testing.T) {ctx := context.Background()- db := setupAuthServiceTestDB(t)+ db := setupAPIKeyServiceTestDB(t)require.NoError(t, db.Exec("CREATE UNIQUE INDEX IF NOT EXISTS idx_akp_uniq ON api_key_permissions(api_key_id, permission, COALESCE(environment_id, ''))").Error)- roleSvc := NewRoleService(db)+ roleSvc := role.NewRoleService(db)require.NoError(t, roleSvc.EnsureBuiltInRoles(ctx))- userSvc := NewUserService(db).WithRoleService(roleSvc)+ userSvc := user.NewUserService(db).WithRoleService(roleSvc)service := NewApiKeyService(db, userSvc).WithRoleService(roleSvc)- admin := createTestUser(t, userSvc, "admin-env-bootstrap", "admin-env-bootstrap")- grantGlobalAdmin(t, roleSvc, admin.ID)+ admin := createTestAPIKeyUser(t, ctx, userSvc, "admin-env-bootstrap", "admin-env-bootstrap")+ require.NoError(t, roleSvc.SetUserAssignments(ctx, admin.ID, []models.UserRoleAssignment{+ {RoleID: authz.BuiltInRoleAdmin, EnvironmentID: nil},+ }))envID := "env-bootstrap-test"created, err := service.CreateEnvironmentApiKey(ctx, envID)@@ -719,10 +739,10 @@ func TestCreateEnvironmentApiKeySeedsAllPermissionsScopedToEnv(t *testing.T) {func TestBackfillApiKeyPermissionsRepairsExistingBootstrapKey(t *testing.T) {ctx := context.Background()- db := setupAuthServiceTestDB(t)+ db := setupAPIKeyServiceTestDB(t)require.NoError(t, db.Exec("CREATE UNIQUE INDEX IF NOT EXISTS idx_akp_uniq ON api_key_permissions(api_key_id, permission, COALESCE(environment_id, ''))").Error)- roleSvc := NewRoleService(db)+ roleSvc := role.NewRoleService(db)require.NoError(t, roleSvc.EnsureBuiltInRoles(ctx))// Simulate a pre-existing env-bootstrap key with NO permission grantsdiff --git a/backend/api/handlers/appimages.go b/backend/internal/appimages/handler.gosimilarity index 95%rename from backend/api/handlers/appimages.gorename to backend/internal/appimages/handler.goindex f251c6d6b..8ea28e622 100644--- a/backend/api/handlers/appimages.go+++ b/backend/internal/appimages/handler.go@@ -1,4 +1,4 @@-package handlers+package appimagesimport ("context"@@ -8,12 +8,11 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- "github.com/getarcaneapp/arcane/backend/v2/internal/services")// AppImagesHandler provides Huma-based application image endpoints.type AppImagesHandler struct {- appImagesService *services.ApplicationImagesService+ appImagesService *ApplicationImagesService}// --- Huma Input/Output Wrappers ---@@ -46,7 +45,7 @@ var allowedPWAIconFilenames = map[string]struct{}{}// RegisterAppImages registers application image routes using Huma.-func RegisterAppImages(api huma.API, appImagesService *services.ApplicationImagesService) {+func RegisterAppImages(api huma.API, appImagesService *ApplicationImagesService) {h := &AppImagesHandler{appImagesService: appImagesService,}diff --git a/backend/internal/services/app_images_service.go b/backend/internal/appimages/service.gosimilarity index 84%rename from backend/internal/services/app_images_service.gorename to backend/internal/appimages/service.goindex b03826a34..ea7c5d8a3 100644--- a/backend/internal/services/app_images_service.go+++ b/backend/internal/appimages/service.go@@ -1,4 +1,4 @@-package services+package appimagesimport ("embed"@@ -8,18 +8,19 @@ import ("sync""emperror.dev/errors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/image"- "github.com/getarcaneapp/arcane/types/v2/settings"+ settingstypes "github.com/getarcaneapp/arcane/types/v2/settings")type ApplicationImagesService struct {mu sync.RWMuteximageData map[string][]bytemimeTypes map[string]string- settingsService *SettingsService+ settingsService *settings.SettingsService}-func NewApplicationImagesService(embeddedFS embed.FS, settingsService *SettingsService) *ApplicationImagesService {+func NewApplicationImagesService(embeddedFS embed.FS, settingsService *settings.SettingsService) *ApplicationImagesService {service := &ApplicationImagesService{imageData: make(map[string][]byte),mimeTypes: make(map[string]string),@@ -78,8 +79,8 @@ func (s *ApplicationImagesService) GetImageWithColor(name string, colorOverride}func (s *ApplicationImagesService) applyAccentColorToSVGInternal(svgData []byte, colorOverride string) []byte {- accentColor := settings.DefaultAccentColor- if settings.SafeAccentColor.MatchString(colorOverride) {+ accentColor := settingstypes.DefaultAccentColor+ if settingstypes.SafeAccentColor.MatchString(colorOverride) {accentColor = colorOverride}diff --git a/backend/internal/services/auth_service_buildables.go b/backend/internal/auth/buildables.gosimilarity index 98%rename from backend/internal/services/auth_service_buildables.gorename to backend/internal/auth/buildables.goindex 8610fb8c5..8a9afb7a7 100644--- a/backend/internal/services/auth_service_buildables.go+++ b/backend/internal/auth/buildables.go@@ -1,6 +1,6 @@//go:build buildables-package services+package authimport ("context"diff --git a/backend/internal/services/auth_service_buildables_test.go b/backend/internal/auth/buildables_test.gosimilarity index 94%rename from backend/internal/services/auth_service_buildables_test.gorename to backend/internal/auth/buildables_test.goindex 333f2795c..86a1d35d6 100644--- a/backend/internal/services/auth_service_buildables_test.go+++ b/backend/internal/auth/buildables_test.go@@ -1,6 +1,6 @@//go:build buildables-package services+package authimport ("context"@@ -25,7 +25,7 @@ func TestGetAutoLoginConfig_Enabled(t *testing.T) {ctx := context.Background()db := setupAuthServiceTestDB(t)- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)+ settingsSvc, err := newSettingsServiceForAuthTestInternal(t, ctx, db)require.NoError(t, err)require.NoError(t, settingsSvc.EnsureDefaultSettings(ctx))require.NoError(t, settingsSvc.SetBoolSetting(ctx, "authLocalEnabled", true))@@ -51,7 +51,7 @@ func TestGetAutoLoginConfig_DisabledWhenLocalAuthDisabled(t *testing.T) {ctx := context.Background()db := setupAuthServiceTestDB(t)- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)+ settingsSvc, err := newSettingsServiceForAuthTestInternal(t, ctx, db)require.NoError(t, err)require.NoError(t, settingsSvc.EnsureDefaultSettings(ctx))require.NoError(t, settingsSvc.SetBoolSetting(ctx, "authLocalEnabled", false))diff --git a/backend/api/handlers/auth.go b/backend/internal/auth/handler.gosimilarity index 81%rename from backend/api/handlers/auth.gorename to backend/internal/auth/handler.goindex 9bceea776..507cc00c1 100644--- a/backend/api/handlers/auth.go+++ b/backend/internal/auth/handler.go@@ -1,4 +1,4 @@-package handlers+package authimport ("bytes"@@ -18,23 +18,24 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/cookie"- "github.com/getarcaneapp/arcane/types/v2/auth"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+ authtypes "github.com/getarcaneapp/arcane/types/v2/auth""github.com/getarcaneapp/arcane/types/v2/base"- "github.com/getarcaneapp/arcane/types/v2/settings"- "github.com/getarcaneapp/arcane/types/v2/user"+ settingstypes "github.com/getarcaneapp/arcane/types/v2/settings"+ usertypes "github.com/getarcaneapp/arcane/types/v2/user")type AuthHandler struct {- userService *services.UserService- authService *services.AuthService- passkeyService *services.PasskeyService- oidcService *services.OidcService- settingsService *services.SettingsService+ userService *user.UserService+ authService *AuthService+ beginMFAAuthentication func(context.Context, string, authtypes.SessionMeta, string) (*authtypes.MFAChallenge, error)+ settingsService *settings.SettingsService}// --- Huma Input/Output Wrappers ---@@ -42,12 +43,12 @@ type AuthHandler struct {type LoginInput struct {UserAgent string `header:"User-Agent"`- Body auth.Login+ Body authtypes.Login}type LoginOutput struct {SetCookie []string `header:"Set-Cookie" doc:"Session cookie"`- Body base.ApiResponse[auth.AuthenticationResponse]+ Body base.ApiResponse[authtypes.AuthenticationResponse]}type LogoutOutput struct {@@ -57,16 +58,16 @@ type LogoutOutput struct {type RefreshTokenInput struct {UserAgent string `header:"User-Agent"`- Body auth.Refresh+ Body authtypes.Refresh}type RefreshTokenOutput struct {SetCookie []string `header:"Set-Cookie" doc:"Updated session cookie"`- Body base.ApiResponse[auth.TokenRefreshResponse]+ Body base.ApiResponse[authtypes.TokenRefreshResponse]}type ChangePasswordInput struct {- Body auth.PasswordChange+ Body authtypes.PasswordChange}type ChangePasswordOutput struct {@@ -78,12 +79,12 @@ type LogoutAllOtherSessionsOutput struct {}type UpdateMyProfileBody struct {- DisplayName *string `json:"displayName,omitempty"`- Email *string `json:"email,omitempty"`- Locale *string `json:"locale,omitempty"`- TimeFormat *user.TimeFormat `json:"timeFormat,omitempty" enum:"auto,12h,24h"`- FontSize *int `json:"fontSize,omitempty" minimum:"12" maximum:"20"`- Preferences *user.Preferences `json:"preferences,omitempty"`+ DisplayName *string `json:"displayName,omitempty"`+ Email *string `json:"email,omitempty"`+ Locale *string `json:"locale,omitempty"`+ TimeFormat *usertypes.TimeFormat `json:"timeFormat,omitempty" enum:"auto,12h,24h"`+ FontSize *int `json:"fontSize,omitempty" minimum:"12" maximum:"20"`+ Preferences *usertypes.Preferences `json:"preferences,omitempty"`}type UpdateMyProfileInput struct {@@ -91,11 +92,11 @@ type UpdateMyProfileInput struct {}type UpdateMyProfileOutput struct {- Body base.ApiResponse[user.User]+ Body base.ApiResponse[usertypes.User]}type GetCurrentUserOutput struct {- Body base.ApiResponse[user.User]+ Body base.ApiResponse[usertypes.User]}type UploadMyAvatarInput struct {@@ -103,21 +104,20 @@ type UploadMyAvatarInput struct {}type UploadMyAvatarOutput struct {- Body base.ApiResponse[user.User]+ Body base.ApiResponse[usertypes.User]}type DeleteMyAvatarOutput struct {- Body base.ApiResponse[user.User]+ Body base.ApiResponse[usertypes.User]}// RegisterAuth registers authentication routes using Huma.-func RegisterAuth(api huma.API, userService *services.UserService, authService *services.AuthService, oidcService *services.OidcService, settingsService *services.SettingsService, passkeyService *services.PasskeyService) {+func RegisterAuth(api huma.API, userService *user.UserService, authService *AuthService, settingsService *settings.SettingsService, beginMFAAuthentication func(context.Context, string, authtypes.SessionMeta, string) (*authtypes.MFAChallenge, error)) {h := &AuthHandler{- userService: userService,- authService: authService,- passkeyService: passkeyService,- oidcService: oidcService,- settingsService: settingsService,+ userService: userService,+ authService: authService,+ beginMFAAuthentication: beginMFAAuthentication,+ settingsService: settingsService,}huma.Register(api, huma.Operation{@@ -147,7 +147,7 @@ func RegisterAuth(api huma.API, userService *services.UserService, authService *Summary: "Get current user",Description: "Get the currently authenticated user's information",Tags: []string{"Auth"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.GetCurrentUser)huma.Register(api, huma.Operation{@@ -167,7 +167,7 @@ func RegisterAuth(api huma.API, userService *services.UserService, authService *Summary: "Change password",Description: "Change the current user's password",Tags: []string{"Auth"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.ChangePassword)huma.Register(api, huma.Operation{@@ -177,7 +177,7 @@ func RegisterAuth(api huma.API, userService *services.UserService, authService *Summary: "Logout all other sessions",Description: "Revoke every session for the current user except the one making this request",Tags: []string{"Auth"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.LogoutAllOtherSessions)huma.Register(api, huma.Operation{@@ -187,7 +187,7 @@ func RegisterAuth(api huma.API, userService *services.UserService, authService *Summary: "Update own profile",Description: "Update the current user's display name and email. Forbidden for OIDC-managed accounts.",Tags: []string{"Auth"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.UpdateMyProfile)huma.Register(api, huma.Operation{@@ -197,7 +197,7 @@ func RegisterAuth(api huma.API, userService *services.UserService, authService *Summary: "Upload own avatar",Description: "Upload a custom profile picture (PNG, JPEG or WebP). Replaces any existing avatar.",Tags: []string{"Auth"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),RequestBody: &huma.RequestBody{Required: true,Content: map[string]*huma.MediaType{@@ -220,7 +220,7 @@ func RegisterAuth(api huma.API, userService *services.UserService, authService *Summary: "Delete own avatar",Description: "Remove the current user's custom profile picture, reverting to the default avatar.",Tags: []string{"Auth"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.DeleteMyAvatar)}@@ -234,13 +234,13 @@ func (h *AuthHandler) Login(ctx context.Context, input *LoginInput) (*LoginOutpureturn nil, huma.Error400BadRequest("Local authentication is disabled")}- meta := sessionMetaFromContextInternal(ctx, input.UserAgent)+ meta := handlerutil.SessionMetaFromContext(ctx, input.UserAgent)userModel, err := h.authService.AuthenticateLocalPrimary(ctx, input.Body.Username, input.Body.Password)if err != nil {switch {- case errors.Is(err, services.ErrInvalidCredentials):+ case errors.Is(err, ErrInvalidCredentials):return nil, huma.Error401Unauthorized("Invalid username or password")- case errors.Is(err, services.ErrLocalAuthDisabled):+ case errors.Is(err, ErrLocalAuthDisabled):return nil, huma.Error400BadRequest("Local authentication is disabled")default:return nil, huma.Error500InternalServerError("Authentication failed")@@ -248,16 +248,16 @@ func (h *AuthHandler) Login(ctx context.Context, input *LoginInput) (*LoginOutpu}if userModel.PasskeyMFAEnabled {- challenge, err := h.passkeyService.BeginMFAAuthentication(ctx, userModel.ID, meta, models.UserSessionSourceLocal)+ challenge, err := h.beginMFAAuthentication(ctx, userModel.ID, meta, models.UserSessionSourceLocal)if err != nil {return nil, huma.Error500InternalServerError("Authentication failed")}return &LoginOutput{- Body: base.ApiResponse[auth.AuthenticationResponse]{+ Body: base.ApiResponse[authtypes.AuthenticationResponse]{Success: true,- Data: auth.AuthenticationResponse{+ Data: authtypes.AuthenticationResponse{Success: true,- Status: auth.AuthenticationStatusMFARequired,+ Status: authtypes.AuthenticationStatusMFARequired,MFA: challenge,},},@@ -280,11 +280,11 @@ func (h *AuthHandler) Login(ctx context.Context, input *LoginInput) (*LoginOutpureturn &LoginOutput{SetCookie: []string{cookie.BuildTokenCookieStringFor(maxAge, tokenPair.AccessToken, cookie.SecureCookieFromContext(ctx))},- Body: base.ApiResponse[auth.AuthenticationResponse]{+ Body: base.ApiResponse[authtypes.AuthenticationResponse]{Success: true,- Data: auth.AuthenticationResponse{+ Data: authtypes.AuthenticationResponse{Success: true,- Status: auth.AuthenticationStatusAuthenticated,+ Status: authtypes.AuthenticationStatusAuthenticated,Token: tokenPair.AccessToken,RefreshToken: tokenPair.RefreshToken,ExpiresAt: &expiresAt,@@ -297,7 +297,7 @@ func (h *AuthHandler) Login(ctx context.Context, input *LoginInput) (*LoginOutpu// Logout clears the authentication session.func (h *AuthHandler) Logout(ctx context.Context, input *struct{}) (*LogoutOutput, error) {if h.authService != nil {- if sessionID, exists := humamw.GetCurrentSessionIDFromContext(ctx); exists {+ if sessionID, exists := middleware.GetCurrentSessionIDFromContext(ctx); exists {if err := h.authService.RevokeSession(ctx, sessionID); err != nil {slog.ErrorContext(ctx, "Failed to revoke session on logout; clearing cookie anyway", "sessionID", sessionID, "error", err)}@@ -322,7 +322,7 @@ func (h *AuthHandler) Logout(ctx context.Context, input *struct{}) (*LogoutOutpu// Uses ToUserResponseDto (not the generic struct mapper) so the RBAC fields// (RoleAssignments, PermissionsByEnv) are resolved via RoleService.func (h *AuthHandler) GetCurrentUser(ctx context.Context, input *struct{}) (*GetCurrentUserOutput, error) {- userID, exists := humamw.GetUserIDFromContext(ctx)+ userID, exists := middleware.GetUserIDFromContext(ctx)if !exists {return nil, huma.Error401Unauthorized("Not authenticated")}@@ -338,7 +338,7 @@ func (h *AuthHandler) GetCurrentUser(ctx context.Context, input *struct{}) (*Get}return &GetCurrentUserOutput{- Body: base.ApiResponse[user.User]{+ Body: base.ApiResponse[usertypes.User]{Success: true,Data: out,},@@ -347,10 +347,10 @@ func (h *AuthHandler) GetCurrentUser(ctx context.Context, input *struct{}) (*Get// RefreshToken obtains a new access token using a refresh token.func (h *AuthHandler) RefreshToken(ctx context.Context, input *RefreshTokenInput) (*RefreshTokenOutput, error) {- tokenPair, err := h.authService.RefreshToken(ctx, input.Body.RefreshToken, sessionMetaFromContextInternal(ctx, input.UserAgent))+ tokenPair, err := h.authService.RefreshToken(ctx, input.Body.RefreshToken, handlerutil.SessionMetaFromContext(ctx, input.UserAgent))if err != nil {switch {- case errors.Is(err, services.ErrInvalidToken), errors.Is(err, services.ErrExpiredToken), errors.Is(err, common.ErrTokenValidation), errors.Is(err, common.ErrSessionRevoked), errors.Is(err, services.ErrTokenVersionMismatch):+ case errors.Is(err, common.ErrInvalidToken), errors.Is(err, common.ErrExpiredToken), errors.Is(err, common.ErrTokenValidation), errors.Is(err, common.ErrSessionRevoked), errors.Is(err, common.ErrTokenVersionMismatch):return nil, huma.Error401Unauthorized("Invalid or expired refresh token")default:return nil, huma.Error500InternalServerError("Failed to refresh token")@@ -362,9 +362,9 @@ func (h *AuthHandler) RefreshToken(ctx context.Context, input *RefreshTokenInputreturn &RefreshTokenOutput{SetCookie: []string{cookie.BuildTokenCookieStringFor(maxAge, tokenPair.AccessToken, cookie.SecureCookieFromContext(ctx))},- Body: base.ApiResponse[auth.TokenRefreshResponse]{+ Body: base.ApiResponse[authtypes.TokenRefreshResponse]{Success: true,- Data: auth.TokenRefreshResponse{+ Data: authtypes.TokenRefreshResponse{Token: tokenPair.AccessToken,RefreshToken: tokenPair.RefreshToken,ExpiresAt: tokenPair.ExpiresAt,@@ -373,16 +373,9 @@ func (h *AuthHandler) RefreshToken(ctx context.Context, input *RefreshTokenInput}, nil}-func sessionMetaFromContextInternal(ctx context.Context, userAgent string) auth.SessionMeta {- return auth.SessionMeta{- UserAgent: userAgent,- IPAddress: humamw.GetRemoteAddrFromContext(ctx),- }-}-// ChangePassword changes the current user's password.func (h *AuthHandler) ChangePassword(ctx context.Context, input *ChangePasswordInput) (*ChangePasswordOutput, error) {- userModel, err := requireUserInternal(ctx)+ userModel, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -391,11 +384,11 @@ func (h *AuthHandler) ChangePassword(ctx context.Context, input *ChangePasswordIreturn nil, huma.Error400BadRequest("Current password is required")}- currentSessionID, _ := humamw.GetCurrentSessionIDFromContext(ctx)+ currentSessionID, _ := middleware.GetCurrentSessionIDFromContext(ctx)err = h.authService.ChangePassword(ctx, userModel.ID, input.Body.CurrentPassword, input.Body.NewPassword, currentSessionID)if err != nil {switch {- case errors.Is(err, services.ErrInvalidCredentials):+ case errors.Is(err, ErrInvalidCredentials):return nil, huma.Error401Unauthorized("Current password is incorrect")case errors.Is(err, common.ErrValidation):return nil, huma.Error400BadRequest(err.Error())@@ -417,12 +410,12 @@ func (h *AuthHandler) ChangePassword(ctx context.Context, input *ChangePasswordI// LogoutAllOtherSessions revokes every active session for the current user// except the session making this request.func (h *AuthHandler) LogoutAllOtherSessions(ctx context.Context, input *struct{}) (*LogoutAllOtherSessionsOutput, error) {- userModel, err := requireUserInternal(ctx)+ userModel, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}- currentSessionID, _ := humamw.GetCurrentSessionIDFromContext(ctx)+ currentSessionID, _ := middleware.GetCurrentSessionIDFromContext(ctx)if err := h.authService.LogoutAllOtherSessions(ctx, userModel.ID, currentSessionID); err != nil {return nil, huma.Error500InternalServerError("failed to revoke sessions: " + err.Error())}@@ -439,10 +432,10 @@ func (h *AuthHandler) LogoutAllOtherSessions(ctx context.Context, input *struct{// validatePreferencesInternal checks the preference values Huma's enum tags// cannot express. Everything else is constrained by the schema.-func validatePreferencesInternal(p *user.Preferences) error {+func validatePreferencesInternal(p *usertypes.Preferences) error {// The "default" sentinel means "no override"; anything else must be a color// literal safe to inject into CSS.- if v := p.AccentColor; v != nil && *v != "" && *v != "default" && !settings.SafeAccentColor.MatchString(*v) {+ if v := p.AccentColor; v != nil && *v != "" && *v != "default" && !settingstypes.SafeAccentColor.MatchString(*v) {return huma.Error400BadRequest("invalid accentColor value")}// Keep the landing page a same-origin relative path. The frontend@@ -458,7 +451,7 @@ func validatePreferencesInternal(p *user.Preferences) error {// Reflection keeps this merge-only update free of per-field branches, the same// way the settings update path walks its DTO, so a new preference needs no// handler change.-func mergePreferencesInternal(dst, src *user.Preferences) {+func mergePreferencesInternal(dst, src *usertypes.Preferences) {dstValue := reflect.ValueOf(dst).Elem()srcValue := reflect.ValueOf(src).Elem()@@ -473,7 +466,7 @@ func mergePreferencesInternal(dst, src *user.Preferences) {// UpdateMyProfile lets the current user update their own displayName and email.// OIDC-managed accounts are read-only here.func (h *AuthHandler) UpdateMyProfile(ctx context.Context, input *UpdateMyProfileInput) (*UpdateMyProfileOutput, error) {- currentUser, err := requireUserInternal(ctx)+ currentUser, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -493,7 +486,7 @@ func (h *AuthHandler) UpdateMyProfile(ctx context.Context, input *UpdateMyProfiluserModel.DisplayName = input.Body.DisplayName}if input.Body.Email != nil {- normalized, err := normalizeOptionalEmailInternal(input.Body.Email)+ normalized, err := user.NormalizeOptionalEmail(input.Body.Email)if err != nil {return nil, huma.Error400BadRequest(err.Error())}@@ -526,7 +519,7 @@ func (h *AuthHandler) UpdateMyProfile(ctx context.Context, input *UpdateMyProfil}return &UpdateMyProfileOutput{- Body: base.ApiResponse[user.User]{+ Body: base.ApiResponse[usertypes.User]{Success: true,Data: out,},@@ -536,7 +529,7 @@ func (h *AuthHandler) UpdateMyProfile(ctx context.Context, input *UpdateMyProfil// UploadMyAvatar lets the current user upload a custom profile picture.// Accepts PNG, JPEG, or WebP images up to the configured avatar upload limit.func (h *AuthHandler) UploadMyAvatar(ctx context.Context, input *UploadMyAvatarInput) (*UploadMyAvatarOutput, error) {- currentUser, err := requireUserInternal(ctx)+ currentUser, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -594,7 +587,7 @@ func (h *AuthHandler) UploadMyAvatar(ctx context.Context, input *UploadMyAvatarI}return &UploadMyAvatarOutput{- Body: base.ApiResponse[user.User]{+ Body: base.ApiResponse[usertypes.User]{Success: true,Data: out,},@@ -615,7 +608,7 @@ func (h *AuthHandler) avatarMaxUploadSizeMbInternal(ctx context.Context) int {// DeleteMyAvatar removes the current user's custom profile picture.func (h *AuthHandler) DeleteMyAvatar(ctx context.Context, input *struct{}) (*DeleteMyAvatarOutput, error) {- currentUser, err := requireUserInternal(ctx)+ currentUser, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -636,7 +629,7 @@ func (h *AuthHandler) DeleteMyAvatar(ctx context.Context, input *struct{}) (*Del}return &DeleteMyAvatarOutput{- Body: base.ApiResponse[user.User]{+ Body: base.ApiResponse[usertypes.User]{Success: true,Data: out,},diff --git a/backend/api/middleware/auth.go b/backend/internal/auth/huma_middleware.gosimilarity index 75%rename from backend/api/middleware/auth.gorename to backend/internal/auth/huma_middleware.goindex d651b093c..25d7d8e67 100644--- a/backend/api/middleware/auth.go+++ b/backend/internal/auth/huma_middleware.go@@ -1,63 +1,24 @@-package middleware+package authimport ("context"+ "errors""log/slog""net/http""strings"- "emperror.dev/errors"+ pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/danielgtaylor/huma/v2"+ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/cookie")-// ContextKey is a type for context keys used by Huma handlers.-type ContextKey string--const (- // ContextKeyUserID is the context key for the authenticated user's ID.- ContextKeyUserID ContextKey = "userID"- // ContextKeyCurrentSessionID is the context key for the authenticated session ID.- ContextKeyCurrentSessionID ContextKey = "currentSessionID"- // ContextKeyUserPermissions is the context key for the caller's resolved- // PermissionSet, attached by the auth bridge.- ContextKeyUserPermissions ContextKey = "userPermissions"- // ContextKeyRemoteAddr is the context key for the request remote address.- ContextKeyRemoteAddr ContextKey = "remoteAddr"-)--// GetUserIDFromContext retrieves the user ID from the context.-func GetUserIDFromContext(ctx context.Context) (string, bool) {- userID, ok := ctx.Value(ContextKeyUserID).(string)- return userID, ok-}--// GetCurrentSessionIDFromContext retrieves the current session ID from the context.-func GetCurrentSessionIDFromContext(ctx context.Context) (string, bool) {- sessionID, ok := ctx.Value(ContextKeyCurrentSessionID).(string)- return sessionID, ok-}--// PermissionsFromContext retrieves the caller's resolved PermissionSet.-// Returns nil, false on unauthenticated paths.-func PermissionsFromContext(ctx context.Context) (*authz.PermissionSet, bool) {- ps, ok := ctx.Value(ContextKeyUserPermissions).(*authz.PermissionSet)- return ps, ok-}--// GetRemoteAddrFromContext retrieves the request remote address from context.-func GetRemoteAddrFromContext(ctx context.Context) string {- remoteAddr, _ := ctx.Value(ContextKeyRemoteAddr).(string)- return remoteAddr-}-// securityRequirements holds parsed security requirements from an operation.type securityRequirements struct {isRequired bool@@ -69,17 +30,6 @@ type operationProvider interface {Operation() *huma.Operation}-type environmentAccessTokenResolver interface {- ResolveEnvironmentByAccessToken(ctx context.Context, token string) (*models.Environment, error)-}--// PermissionResolver resolves a caller's effective permission set. Implemented-// by services.RoleService; kept as an interface so tests can stub it.-type PermissionResolver interface {- ResolvePermissions(ctx context.Context, user *models.User) (*authz.PermissionSet, error)- ResolveApiKeyPermissions(ctx context.Context, apiKeyID string) (*authz.PermissionSet, error)-}-// parseSecurityRequirementsInternal extracts security requirements from a Huma operation.func parseSecurityRequirementsInternal(api huma.API, ctx operationProvider) securityRequirements {reqs := securityRequirements{}@@ -118,7 +68,7 @@ func parseSecurityRequirementsInternal(api huma.API, ctx operationProvider) secu// authenticated user on success, or the underlying error from VerifyToken so// the caller can distinguish a missing/invalid token from a token-version// mismatch (which requires clearing the stale cookie).-func tryBearerAuthInternal(ctx huma.Context, authService *services.AuthService) (*models.User, string, error) {+func tryBearerAuthInternal(ctx huma.Context, authService *AuthService) (*models.User, string, error) {token := extractBearerTokenInternal(ctx)if token == "" {return nil, "", nil@@ -133,7 +83,7 @@ func tryBearerAuthInternal(ctx huma.Context, authService *services.AuthService)// tryApiKeyAuthInternal checks if API key authentication should be allowed// through. Returns the resolved user plus the API key record so the caller// can resolve permissions according to the key's kind.-func tryApiKeyAuthInternal(ctx huma.Context, apiKeyService *services.ApiKeyService) (*models.User, *models.ApiKey, bool) {+func tryApiKeyAuthInternal(ctx huma.Context, apiKeyService *apikey.ApiKeyService) (*models.User, *models.ApiKey, bool) {apiKey := ctx.Header(pkgutils.HeaderApiKey)if apiKey == "" {return nil, nil, false@@ -147,7 +97,7 @@ func tryApiKeyAuthInternal(ctx huma.Context, apiKeyService *services.ApiKeyServireturn user, key, true}-func tryEnvironmentAccessTokenAuthInternal(ctx huma.Context, resolver environmentAccessTokenResolver, token string) (*models.User, *models.Environment, bool) {+func tryEnvironmentAccessTokenAuthInternal(ctx huma.Context, resolver EnvironmentAccessTokenResolver, token string) (*models.User, *models.Environment, bool) {if resolver == nil || strings.TrimSpace(token) == "" {return nil, nil, false}@@ -171,18 +121,17 @@ func tryAgentAuthInternal(ctx huma.Context, cfg *config.Config) (*models.User, b// Check for agent bootstrap pairingif strings.HasPrefix(path, pkgutils.AgentPairingPrefix) &&- cfg.AgentToken != "" &&- ctx.Header(pkgutils.HeaderAgentBootstrap) == cfg.AgentToken {+ AgentTokenMatches(ctx.Header(pkgutils.HeaderAgentBootstrap), cfg.AgentToken) {return createAgentSudoUserInternal(), true}// Check for agent token- if tok := ctx.Header(pkgutils.HeaderAgentToken); tok != "" && cfg.AgentToken != "" && tok == cfg.AgentToken {+ if AgentTokenMatches(ctx.Header(pkgutils.HeaderAgentToken), cfg.AgentToken) {return createAgentSudoUserInternal(), true}// Check for API key as agent token- if tok := ctx.Header(pkgutils.HeaderApiKey); tok != "" && cfg.AgentToken != "" && tok == cfg.AgentToken {+ if AgentTokenMatches(ctx.Header(pkgutils.HeaderApiKey), cfg.AgentToken) {return createAgentSudoUserInternal(), true}@@ -226,13 +175,13 @@ func createEnvironmentUserInternal(env *models.Environment) *models.User {}}-// NewAuthBridge creates a Huma middleware that validates credentials and+// NewHumaMiddleware creates middleware that validates credentials and// enforces security requirements defined on operations. It also resolves the// caller's effective PermissionSet via permResolver and stashes it on the-// request context for downstream RequirePermission checks.-func NewAuthBridge(api huma.API, authService *services.AuthService, apiKeyService *services.ApiKeyService, permResolver PermissionResolver, envTokenResolver environmentAccessTokenResolver, cfg *config.Config) func(ctx huma.Context, next func(huma.Context)) {+// request context for downstream middleware.RequirePermission checks.+func NewHumaMiddleware(api huma.API, authService *AuthService, apiKeyService *apikey.ApiKeyService, permResolver PermissionResolver, envTokenResolver EnvironmentAccessTokenResolver, cfg *config.Config) func(ctx huma.Context, next func(huma.Context)) {return func(ctx huma.Context, next func(huma.Context)) {- ctx = huma.WithContext(ctx, context.WithValue(ctx.Context(), ContextKeyRemoteAddr, ctx.RemoteAddr()))+ ctx = huma.WithContext(ctx, context.WithValue(ctx.Context(), middleware.ContextKeyRemoteAddr, ctx.RemoteAddr()))if authService == nil {next(ctx)return@@ -292,7 +241,7 @@ func tryAgentAuthCtxInternal(ctx huma.Context, cfg *config.Config) (huma.Context// opportunisticBearerAuthInternal populates the user/session context if a valid// bearer token is present, but never fails the request. Used for public routes// (e.g. logout) that still need to know who the caller is when a token exists.-func opportunisticBearerAuthInternal(ctx huma.Context, authService *services.AuthService, permResolver PermissionResolver) huma.Context {+func opportunisticBearerAuthInternal(ctx huma.Context, authService *AuthService, permResolver PermissionResolver) huma.Context {if extractBearerTokenInternal(ctx) == "" {return ctx}@@ -301,14 +250,14 @@ func opportunisticBearerAuthInternal(ctx huma.Context, authService *services.Autreturn ctx}newCtx := setUserInContextInternal(ctx.Context(), user, resolveUserPermissionsInternal(ctx.Context(), permResolver, user))- newCtx = context.WithValue(newCtx, ContextKeyCurrentSessionID, sessionID)+ newCtx = context.WithValue(newCtx, middleware.ContextKeyCurrentSessionID, sessionID)return huma.WithContext(ctx, newCtx)}// handleApiKeyAuthInternal handles the API-key-present branch. Invalid user// keys fail closed; only recognized environment tokens defer to bearer auth// when both credentials are present on a bearer-capable operation.-func handleApiKeyAuthInternal(api huma.API, ctx huma.Context, authService *services.AuthService, apiKeyService *services.ApiKeyService, permResolver PermissionResolver, envTokenResolver environmentAccessTokenResolver, allowBearerFallback bool, next func(huma.Context)) {+func handleApiKeyAuthInternal(api huma.API, ctx huma.Context, authService *AuthService, apiKeyService *apikey.ApiKeyService, permResolver PermissionResolver, envTokenResolver EnvironmentAccessTokenResolver, allowBearerFallback bool, next func(huma.Context)) {if user, key, ok := tryApiKeyAuthInternal(ctx, apiKeyService); ok {// Personal keys inherit the owner's role permissions (same resolution// as session auth); scoped keys are limited to their own grants.@@ -340,15 +289,15 @@ func handleApiKeyAuthInternal(api huma.API, ctx huma.Context, authService *servi_ = huma.WriteErr(api, ctx, http.StatusUnauthorized, "Unauthorized: invalid API key")}-func handleBearerAuthInternal(api huma.API, ctx huma.Context, authService *services.AuthService, permResolver PermissionResolver) (huma.Context, bool) {+func handleBearerAuthInternal(api huma.API, ctx huma.Context, authService *AuthService, permResolver PermissionResolver) (huma.Context, bool) {user, sessionID, err := tryBearerAuthInternal(ctx, authService)if err == nil && user != nil {ps := resolveUserPermissionsInternal(ctx.Context(), permResolver, user)newCtx := setUserInContextInternal(ctx.Context(), user, ps)- newCtx = context.WithValue(newCtx, ContextKeyCurrentSessionID, sessionID)+ newCtx = context.WithValue(newCtx, middleware.ContextKeyCurrentSessionID, sessionID)return huma.WithContext(ctx, newCtx), true}- if errors.Is(err, services.ErrTokenVersionMismatch) {+ if errors.Is(err, common.ErrTokenVersionMismatch) {// The app version changed (a self-update). The session is still valid — the// refresh path tolerates the version change and rotates the token — so do NOT// clear the auth cookies. Return a recoverable 401 the frontend refreshes from.@@ -432,8 +381,8 @@ func setUserInContextInternal(ctx context.Context, user *models.User, ps *authz.if ps == nil {ps = authz.NewPermissionSet()}- ctx = context.WithValue(ctx, ContextKeyUserID, user.ID)+ ctx = context.WithValue(ctx, middleware.ContextKeyUserID, user.ID)ctx = context.WithValue(ctx, models.CurrentUserContextKey{}, user)- ctx = context.WithValue(ctx, ContextKeyUserPermissions, ps)+ ctx = context.WithValue(ctx, middleware.ContextKeyUserPermissions, ps)return ctx}diff --git a/backend/api/middleware/auth_test.go b/backend/internal/auth/huma_middleware_test.gosimilarity index 86%rename from backend/api/middleware/auth_test.gorename to backend/internal/auth/huma_middleware_test.goindex 3d7308cf3..f32858741 100644--- a/backend/api/middleware/auth_test.go+++ b/backend/internal/auth/huma_middleware_test.go@@ -1,4 +1,4 @@-package middleware+package authimport ("context"@@ -11,11 +11,13 @@ import ("github.com/danielgtaylor/huma/v2/adapters/humaecho""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"- "github.com/getarcaneapp/arcane/types/v2/auth"+ authtypes "github.com/getarcaneapp/arcane/types/v2/auth""github.com/golang-jwt/jwt/v5""github.com/labstack/echo/v5""github.com/libtnb/sqlite"@@ -35,53 +37,7 @@ type testEnvironmentAccessResolver struct {env *models.Environment}-func (r testEnvironmentAccessResolver) ResolveEnvironmentByAccessToken(_ context.Context, token string) (*models.Environment, error) {- if r.env != nil && r.env.AccessToken != nil && *r.env.AccessToken == token {- return r.env, nil- }- return nil, context.Canceled-}--type staticPermissionResolverInternal struct {- ps *authz.PermissionSet-}--func (r staticPermissionResolverInternal) ResolvePermissions(_ context.Context, _ *models.User) (*authz.PermissionSet, error) {- return r.ps, nil-}--func (r staticPermissionResolverInternal) ResolveApiKeyPermissions(_ context.Context, _ string) (*authz.PermissionSet, error) {- return r.ps, nil-}--func mintAuthBridgeTestTokenInternal(t *testing.T, userSvc *services.UserService, sessionSvc *services.SessionService, jwtSecret string, userID string) string {- t.Helper()-- _, err := userSvc.CreateUser(context.Background(), &models.User{- BaseModel: models.BaseModel{ID: userID},- Username: userID,- })- require.NoError(t, err)-- exp := time.Now().Add(5 * time.Minute)- session, _, err := sessionSvc.CreateSession(context.Background(), userID, exp, auth.SessionMeta{})- require.NoError(t, err)-- claims := jwt.MapClaims{- "jti": userID,- "sub": "access",- "iat": time.Now().Unix(),- "exp": exp.Unix(),- "sid": session.ID,- "user_id": userID,- "username": userID,- }- token, err := jwt.NewWithClaims(jwt.SigningMethodHS256, claims).SignedString([]byte(jwtSecret))- require.NoError(t, err)- return token-}--func TestNewAuthBridge_AcceptsEnvironmentAccessTokenViaAPIKey(t *testing.T) {+func TestNewHumaMiddleware_AcceptsEnvironmentAccessTokenViaAPIKey(t *testing.T) {token := "env-access-token"router := echo.New()apiGroup := router.Group("/api")@@ -96,7 +52,7 @@ func TestNewAuthBridge_AcceptsEnvironmentAccessTokenViaAPIKey(t *testing.T) {}api := humaecho.NewWithGroup(router, apiGroup, humaConfig)- api.UseMiddleware(NewAuthBridge(api, &services.AuthService{}, nil, nil, testEnvironmentAccessResolver{+ api.UseMiddleware(NewHumaMiddleware(api, &AuthService{}, nil, nil, testEnvironmentAccessResolver{env: &models.Environment{BaseModel: models.BaseModel{ID: "env-self"},Name: "Self Target",@@ -115,7 +71,7 @@ func TestNewAuthBridge_AcceptsEnvironmentAccessTokenViaAPIKey(t *testing.T) {require.Equal(t, "environment:env-self", user.ID)require.Equal(t, "Self Target", user.Username)- ps, ok := PermissionsFromContext(ctx)+ ps, ok := middleware.PermissionsFromContext(ctx)require.True(t, ok)require.True(t, ps.Allows(authz.PermContainersStart, "env-self"))require.False(t, ps.Allows(authz.PermContainersStart, "env-other"))@@ -137,14 +93,14 @@ func TestNewAuthBridge_AcceptsEnvironmentAccessTokenViaAPIKey(t *testing.T) {require.Contains(t, rec.Body.String(), "environment:env-self")}-func TestNewAuthBridge_UsesBearerWhenLoopbackProxySendsEnvironmentAccessToken(t *testing.T) {+func TestNewHumaMiddleware_UsesBearerWhenLoopbackProxySendsEnvironmentAccessToken(t *testing.T) {db := setupAuthMiddlewareTestDBInternal(t)- userSvc := services.NewUserService(db)- sessionSvc := services.NewSessionService(db)+ userSvc := user.NewUserService(db)+ sessionSvc := session.NewSessionService(db)jwtSecret := "test-secret-please-do-not-use-in-prod"- authSvc := services.NewAuthService(userSvc, nil, nil, sessionSvc, nil, jwtSecret, &config.Config{JWTRefreshExpiry: 24 * time.Hour}, nil)- bearerToken := mintAuthBridgeTestTokenInternal(t, userSvc, sessionSvc, jwtSecret, "u-loopback")+ authSvc := NewAuthService(userSvc, nil, nil, sessionSvc, nil, jwtSecret, &config.Config{JWTRefreshExpiry: 24 * time.Hour}, nil)+ bearerToken := mintHumaMiddlewareTestTokenInternal(t, userSvc, sessionSvc, jwtSecret, "u-loopback")ps := authz.NewPermissionSet()ps.AddEnv("0", authz.PermContainersStart)@@ -160,7 +116,7 @@ func TestNewAuthBridge_UsesBearerWhenLoopbackProxySendsEnvironmentAccessToken(t}api := humaecho.NewWithGroup(router, apiGroup, humaConfig)- api.UseMiddleware(NewAuthBridge(api, authSvc, nil, staticPermissionResolverInternal{ps: ps}, testEnvironmentAccessResolver{+ api.UseMiddleware(NewHumaMiddleware(api, authSvc, nil, staticPermissionResolverInternal{ps: ps}, testEnvironmentAccessResolver{env: &models.Environment{BaseModel: models.BaseModel{ID: "remote-env"},Name: "Remote Env",@@ -176,7 +132,7 @@ func TestNewAuthBridge_UsesBearerWhenLoopbackProxySendsEnvironmentAccessToken(t{"BearerAuth": {}},{"ApiKeyAuth": {}},},- Middlewares: RequirePermission(api, authz.PermContainersStart),+ Middlewares: middleware.RequirePermission(api, authz.PermContainersStart),}, func(ctx context.Context, _ *struct {ID string `path:"id"`CID string `path:"cid"`@@ -205,7 +161,7 @@ func TestNewAuthBridge_UsesBearerWhenLoopbackProxySendsEnvironmentAccessToken(t// A valid API key presented to a BearerAuth-only operation must be rejected:// the bridge only attempts API-key auth when the operation declares ApiKeyAuth.// This is the gate that makes personal-key create/delete session-only.-func TestNewAuthBridge_RejectsApiKeyOnBearerOnlyOperation(t *testing.T) {+func TestNewHumaMiddleware_RejectsApiKeyOnBearerOnlyOperation(t *testing.T) {router := echo.New()apiGroup := router.Group("/api")@@ -216,7 +172,7 @@ func TestNewAuthBridge_RejectsApiKeyOnBearerOnlyOperation(t *testing.T) {}api := humaecho.NewWithGroup(router, apiGroup, humaConfig)- api.UseMiddleware(NewAuthBridge(api, &services.AuthService{}, nil, nil, nil, &config.Config{}))+ api.UseMiddleware(NewHumaMiddleware(api, &AuthService{}, nil, nil, nil, &config.Config{}))huma.Register(api, huma.Operation{OperationID: "bearer-only",@@ -237,14 +193,6 @@ func TestNewAuthBridge_RejectsApiKeyOnBearerOnlyOperation(t *testing.T) {require.Equal(t, http.StatusUnauthorized, rec.Code)}-type testOperationProvider struct {- operation *huma.Operation-}--func (p testOperationProvider) Operation() *huma.Operation {- return p.operation-}-func TestParseSecurityRequirements(t *testing.T) {router := echo.New()apiGroup := router.Group("/api")@@ -307,22 +255,14 @@ func TestParseSecurityRequirements(t *testing.T) {}}-func setupAuthMiddlewareTestDBInternal(t *testing.T) *database.DB {- t.Helper()- db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, db.AutoMigrate(&models.SettingVariable{}, &models.User{}, &models.UserSession{}))- return &database.DB{DB: db}-}--func TestNewAuthBridge_OpportunisticAuthOnPublicRoute(t *testing.T) {+func TestNewHumaMiddleware_OpportunisticAuthOnPublicRoute(t *testing.T) {db := setupAuthMiddlewareTestDBInternal(t)- userSvc := services.NewUserService(db)- sessionSvc := services.NewSessionService(db)+ userSvc := user.NewUserService(db)+ sessionSvc := session.NewSessionService(db)jwtSecret := "test-secret-please-do-not-use-in-prod"cfg := &config.Config{JWTRefreshExpiry: 24 * time.Hour}- authSvc := services.NewAuthService(userSvc, nil, nil, sessionSvc, nil, jwtSecret, cfg, nil)+ authSvc := NewAuthService(userSvc, nil, nil, sessionSvc, nil, jwtSecret, cfg, nil)_, err := userSvc.CreateUser(context.Background(), &models.User{BaseModel: models.BaseModel{ID: "u-logout"},@@ -331,7 +271,7 @@ func TestNewAuthBridge_OpportunisticAuthOnPublicRoute(t *testing.T) {require.NoError(t, err)exp := time.Now().Add(5 * time.Minute)- session, _, err := sessionSvc.CreateSession(context.Background(), "u-logout", exp, auth.SessionMeta{})+ session, _, err := sessionSvc.CreateSession(context.Background(), "u-logout", exp, authtypes.SessionMeta{})require.NoError(t, err)claims := jwt.MapClaims{@@ -354,7 +294,7 @@ func TestNewAuthBridge_OpportunisticAuthOnPublicRoute(t *testing.T) {"BearerAuth": {Type: "http", Scheme: "bearer"},}api := humaecho.NewWithGroup(router, apiGroup, humaConfig)- api.UseMiddleware(NewAuthBridge(api, authSvc, nil, nil, nil, &config.Config{}))+ api.UseMiddleware(NewHumaMiddleware(api, authSvc, nil, nil, nil, &config.Config{}))var sawSessionID stringhuma.Register(api, huma.Operation{@@ -363,7 +303,7 @@ func TestNewAuthBridge_OpportunisticAuthOnPublicRoute(t *testing.T) {Path: "/public",Security: []map[string][]string{},}, func(ctx context.Context, _ *secureInput) (*secureOutput, error) {- if sid, ok := GetCurrentSessionIDFromContext(ctx); ok {+ if sid, ok := middleware.GetCurrentSessionIDFromContext(ctx); ok {sawSessionID = sid}return &secureOutput{}, nil@@ -403,14 +343,14 @@ func TestNewAuthBridge_OpportunisticAuthOnPublicRoute(t *testing.T) {// check. That must be RECOVERABLE (the refresh path rotates the token), so the// middleware returns a refreshable 401 and must NOT clear the auth cookies — otherwise// the user is logged out on every update.-func TestNewAuthBridge_VersionMismatchIsRecoverable(t *testing.T) {+func TestNewHumaMiddleware_VersionMismatchIsRecoverable(t *testing.T) {db := setupAuthMiddlewareTestDBInternal(t)- userSvc := services.NewUserService(db)- sessionSvc := services.NewSessionService(db)+ userSvc := user.NewUserService(db)+ sessionSvc := session.NewSessionService(db)jwtSecret := "test-secret-please-do-not-use-in-prod"cfg := &config.Config{JWTRefreshExpiry: 24 * time.Hour}- authSvc := services.NewAuthService(userSvc, nil, nil, sessionSvc, nil, jwtSecret, cfg, nil)+ authSvc := NewAuthService(userSvc, nil, nil, sessionSvc, nil, jwtSecret, cfg, nil)_, err := userSvc.CreateUser(context.Background(), &models.User{BaseModel: models.BaseModel{ID: "u-ver"},@@ -419,7 +359,7 @@ func TestNewAuthBridge_VersionMismatchIsRecoverable(t *testing.T) {require.NoError(t, err)exp := time.Now().Add(5 * time.Minute)- session, _, err := sessionSvc.CreateSession(context.Background(), "u-ver", exp, auth.SessionMeta{})+ session, _, err := sessionSvc.CreateSession(context.Background(), "u-ver", exp, authtypes.SessionMeta{})require.NoError(t, err)// An empty appVersion omits the claim, which passes the version check (no pin).@@ -448,7 +388,7 @@ func TestNewAuthBridge_VersionMismatchIsRecoverable(t *testing.T) {"BearerAuth": {Type: "http", Scheme: "bearer"},}api := humaecho.NewWithGroup(router, apiGroup, humaConfig)- api.UseMiddleware(NewAuthBridge(api, authSvc, nil, nil, nil, &config.Config{}))+ api.UseMiddleware(NewHumaMiddleware(api, authSvc, nil, nil, nil, &config.Config{}))huma.Register(api, huma.Operation{OperationID: "protected",@@ -481,7 +421,7 @@ func TestNewAuthBridge_VersionMismatchIsRecoverable(t *testing.T) {})}-func TestNewAuthBridge_AgentAuthAppliesForwardedIconCatalog(t *testing.T) {+func TestNewHumaMiddleware_AgentAuthAppliesForwardedIconCatalog(t *testing.T) {const agentToken = "agent-token"router := echo.New()apiGroup := router.Group("/api")@@ -492,7 +432,7 @@ func TestNewAuthBridge_AgentAuthAppliesForwardedIconCatalog(t *testing.T) {}api := humaecho.NewWithGroup(router, apiGroup, humaConfig)- api.UseMiddleware(NewAuthBridge(api, &services.AuthService{}, nil, nil, nil, &config.Config{+ api.UseMiddleware(NewHumaMiddleware(api, &AuthService{}, nil, nil, nil, &config.Config{AgentMode: true,AgentToken: agentToken,}))@@ -522,3 +462,65 @@ func TestNewAuthBridge_AgentAuthAppliesForwardedIconCatalog(t *testing.T) {require.Equal(t, http.StatusOK, rec.Code)}++func (r testEnvironmentAccessResolver) ResolveEnvironmentByAccessToken(_ context.Context, token string) (*models.Environment, error) {+ if r.env != nil && r.env.AccessToken != nil && *r.env.AccessToken == token {+ return r.env, nil+ }+ return nil, context.Canceled+}++func setupAuthMiddlewareTestDBInternal(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.SettingVariable{}, &models.User{}, &models.UserSession{}))+ return &database.DB{DB: db}+}++func mintHumaMiddlewareTestTokenInternal(t *testing.T, userSvc *user.UserService, sessionSvc *session.SessionService, jwtSecret string, userID string) string {+ t.Helper()++ _, err := userSvc.CreateUser(context.Background(), &models.User{+ BaseModel: models.BaseModel{ID: userID},+ Username: userID,+ })+ require.NoError(t, err)++ exp := time.Now().Add(5 * time.Minute)+ session, _, err := sessionSvc.CreateSession(context.Background(), userID, exp, authtypes.SessionMeta{})+ require.NoError(t, err)++ claims := jwt.MapClaims{+ "jti": userID,+ "sub": "access",+ "iat": time.Now().Unix(),+ "exp": exp.Unix(),+ "sid": session.ID,+ "user_id": userID,+ "username": userID,+ }+ token, err := jwt.NewWithClaims(jwt.SigningMethodHS256, claims).SignedString([]byte(jwtSecret))+ require.NoError(t, err)+ return token+}++func (r staticPermissionResolverInternal) ResolvePermissions(_ context.Context, _ *models.User) (*authz.PermissionSet, error) {+ return r.ps, nil+}++func (r staticPermissionResolverInternal) ResolveApiKeyPermissions(_ context.Context, _ string) (*authz.PermissionSet, error) {+ return r.ps, nil+}++type staticPermissionResolverInternal struct {+ ps *authz.PermissionSet+}++type testOperationProvider struct {+ operation *huma.Operation+}++func (p testOperationProvider) Operation() *huma.Operation {+ return p.operation+}diff --git a/backend/internal/middleware/auth_middleware.go b/backend/internal/auth/middleware.gosimilarity index 77%rename from backend/internal/middleware/auth_middleware.gorename to backend/internal/auth/middleware.goindex 5cc07f26a..8b4f06af5 100644--- a/backend/internal/middleware/auth_middleware.go+++ b/backend/internal/auth/middleware.go@@ -1,4 +1,4 @@-package middleware+package authimport ("github.com/samber/mo"@@ -13,24 +13,14 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/cookie""github.com/labstack/echo/v5")-// Echo context keys, kept aligned with api/middleware/auth.go constants so-// shared handlers can read them regardless of which auth layer attached them.-const (- echoCtxKeyUserID = "userID"- echoCtxKeyCurrentUser = "currentUser"- echoCtxKeyUserPermissions = "userPermissions"- echoCtxKeyAuthMethod = "authMethod"- echoCtxKeySessionID = "currentSessionID"-)-type AuthOptions struct {AdminRequired boolSuccessOptional bool@@ -45,14 +35,14 @@ type EnvironmentAccessTokenResolver interface {}// PermissionResolver resolves a caller's effective permission set. Implemented-// by services.RoleService; kept as an interface so tests can stub it.+// by role.RoleService; kept as an interface so tests can stub it.type PermissionResolver interface {ResolvePermissions(ctx context.Context, user *models.User) (*authz.PermissionSet, error)ResolveApiKeyPermissions(ctx context.Context, apiKeyID string) (*authz.PermissionSet, error)}type AuthMiddleware struct {- authService *services.AuthService+ authService *AuthServiceapiKeyValidator ApiKeyValidatorenvTokenResolver EnvironmentAccessTokenResolverroleResolver PermissionResolver@@ -60,7 +50,7 @@ type AuthMiddleware struct {options AuthOptions}-func NewAuthMiddleware(authService *services.AuthService, cfg *config.Config) *AuthMiddleware {+func NewAuthMiddleware(authService *AuthService, cfg *config.Config) *AuthMiddleware {return &AuthMiddleware{authService: authService,cfg: cfg,@@ -98,30 +88,6 @@ func (m *AuthMiddleware) WithAdminRequired() *AuthMiddleware {return &clone}-// RequirePermission returns an Echo middleware that rejects callers lacking-// `perm` for the environment ID in the request path (or globally for org-level-// permissions). Use on streaming/WS routes that aren't served by Huma. Expects-// the caller's PermissionSet to already be on the Echo context via the-// AuthMiddleware (i.e., chain it AFTER auth).-func RequirePermission(perm string) echo.MiddlewareFunc {- return func(next echo.HandlerFunc) echo.HandlerFunc {- return func(c *echo.Context) error {- ps, _ := c.Get(echoCtxKeyUserPermissions).(*authz.PermissionSet)- envID := ""- if authz.IsEnvScoped(perm) {- envID = authz.EnvIDFromPath(c.Request().URL.Path)- }- if !ps.Allows(perm, envID) {- return c.JSON(http.StatusForbidden, models.APIError{- Code: "FORBIDDEN",- Message: "permission denied: " + perm,- })- }- return next(c)- }- }-}-func (m *AuthMiddleware) Add() echo.MiddlewareFunc {return func(next echo.HandlerFunc) echo.HandlerFunc {return func(c *echo.Context) error {@@ -141,19 +107,19 @@ func (m *AuthMiddleware) agentAuth(ctx context.Context, c *echo.Context, next ecreq := c.Request()if strings.HasPrefix(req.URL.Path, pkgutils.AgentPairingPrefix) &&- agentTokenMatchesInternal(req.Header.Get(pkgutils.HeaderAgentBootstrap), m.cfg.AgentToken) {+ AgentTokenMatches(req.Header.Get(pkgutils.HeaderAgentBootstrap), m.cfg.AgentToken) {slog.InfoContext(ctx, "Agent auth: bootstrap pairing accepted", "path", req.URL.Path, "method", req.Method)agentSudoInternal(c)return next(c)}- if agentTokenMatchesInternal(req.Header.Get(pkgutils.HeaderAgentToken), m.cfg.AgentToken) {+ if AgentTokenMatches(req.Header.Get(pkgutils.HeaderAgentToken), m.cfg.AgentToken) {agentSudoInternal(c)return next(c)}// Check for API key as agent token- if agentTokenMatchesInternal(req.Header.Get(pkgutils.HeaderApiKey), m.cfg.AgentToken) {+ if AgentTokenMatches(req.Header.Get(pkgutils.HeaderApiKey), m.cfg.AgentToken) {agentSudoInternal(c)return next(c)}@@ -170,9 +136,9 @@ func (m *AuthMiddleware) agentAuth(ctx context.Context, c *echo.Context, next ec})}-// agentTokenMatchesInternal compares a presented token against the configured+// AgentTokenMatches compares a presented token against the configured// agent token in constant time to avoid timing side channels.-func agentTokenMatchesInternal(presented, configured string) bool {+func AgentTokenMatches(presented, configured string) bool {if presented == "" || configured == "" {return false}@@ -209,7 +175,7 @@ func (m *AuthMiddleware) managerAuth(ctx context.Context, c *echo.Context, next// A version mismatch means the app self-updated; the session is still valid// (the refresh path tolerates the version change and rotates the token), so do// NOT clear the cookies. Return a recoverable 401 the frontend refreshes from.- if errors.Is(err, services.ErrTokenVersionMismatch) {+ if errors.Is(err, common.ErrTokenVersionMismatch) {return c.JSON(http.StatusUnauthorized, models.APIError{Code: models.APIErrorCodeUnauthorized,Message: "Application has been updated. Refreshing session.",@@ -241,10 +207,10 @@ func (m *AuthMiddleware) managerAuth(ctx context.Context, c *echo.Context, next})}- c.Set(echoCtxKeyUserID, user.ID)- c.Set(echoCtxKeyCurrentUser, user)- c.Set(echoCtxKeySessionID, sessionID)- c.Set(echoCtxKeyUserPermissions, ps)+ c.Set(string(middleware.ContextKeyUserID), user.ID)+ c.Set(string(middleware.ContextKeyCurrentUser), user)+ c.Set(string(middleware.ContextKeyCurrentSessionID), sessionID)+ c.Set(string(middleware.ContextKeyUserPermissions), ps)return next(c)}@@ -268,10 +234,10 @@ func (m *AuthMiddleware) apiKeyHeaderAuth(ctx context.Context, c *echo.Context,Message: "You don't have permission to access this resource",})}- c.Set(echoCtxKeyUserID, user.ID)- c.Set(echoCtxKeyCurrentUser, user)- c.Set(echoCtxKeyUserPermissions, ps)- c.Set(echoCtxKeyAuthMethod, "api_key")+ c.Set(string(middleware.ContextKeyUserID), user.ID)+ c.Set(string(middleware.ContextKeyCurrentUser), user)+ c.Set(string(middleware.ContextKeyUserPermissions), ps)+ c.Set(string(middleware.ContextKeyAuthMethod), "api_key")return next(c)}}@@ -338,10 +304,10 @@ func agentSudoInternal(c *echo.Context) {Email: new("[email protected]"),Username: "agent",}- c.Set(echoCtxKeyUserID, agentUser.ID)- c.Set(echoCtxKeyCurrentUser, agentUser)- c.Set(echoCtxKeyUserPermissions, authz.SudoPermissionSet())- c.Set(echoCtxKeyAuthMethod, "agent_token")+ c.Set(string(middleware.ContextKeyUserID), agentUser.ID)+ c.Set(string(middleware.ContextKeyCurrentUser), agentUser)+ c.Set(string(middleware.ContextKeyUserPermissions), authz.SudoPermissionSet())+ c.Set(string(middleware.ContextKeyAuthMethod), "agent_token")}func environmentScopedInternal(c *echo.Context, env *models.Environment) {@@ -349,10 +315,10 @@ func environmentScopedInternal(c *echo.Context, env *models.Environment) {BaseModel: models.BaseModel{ID: "environment:" + env.ID},Username: env.Name,}- c.Set(echoCtxKeyUserID, envUser.ID)- c.Set(echoCtxKeyCurrentUser, envUser)- c.Set(echoCtxKeyUserPermissions, authz.EnvironmentPermissionSet(env.ID))- c.Set(echoCtxKeyAuthMethod, "environment_access_token")+ c.Set(string(middleware.ContextKeyUserID), envUser.ID)+ c.Set(string(middleware.ContextKeyCurrentUser), envUser)+ c.Set(string(middleware.ContextKeyUserPermissions), authz.EnvironmentPermissionSet(env.ID))+ c.Set(string(middleware.ContextKeyAuthMethod), "environment_access_token")}func extractBearerOrCookieTokenInternal(c *echo.Context) string {diff --git a/backend/internal/middleware/auth_middleware_test.go b/backend/internal/auth/middleware_test.gosimilarity index 99%rename from backend/internal/middleware/auth_middleware_test.gorename to backend/internal/auth/middleware_test.goindex 98933140d..c62fe4e9c 100644--- a/backend/internal/middleware/auth_middleware_test.go+++ b/backend/internal/auth/middleware_test.go@@ -1,4 +1,4 @@-package middleware+package authimport ("context"diff --git a/backend/internal/auth/module.go b/backend/internal/auth/module.gonew file mode 100644index 000000000..978ad6097--- /dev/null+++ b/backend/internal/auth/module.go@@ -0,0 +1,43 @@+// Package auth owns authentication, token issuance and verification, login+// session policy, and the authentication HTTP surface.+package auth++import (+ "context"++ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user"+ authtypes "github.com/getarcaneapp/arcane/types/v2/auth"+)++type Dependencies struct {+ Service *AuthService+ User *user.UserService+ Settings *settings.SettingsService+ BeginMFAAuthentication func(context.Context, string, authtypes.SessionMeta, string) (*authtypes.MFAChallenge, error)+}++type Module struct {+ deps Dependencies+}++func New(deps Dependencies) *Module {+ return &Module{deps: deps}+}++func (m *Module) Service() *AuthService {+ if m == nil {+ return nil+ }+ return m.deps.Service+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterAuth(api, nil, nil, nil, nil)+ return+ }+ RegisterAuth(api, m.deps.User, m.deps.Service, m.deps.Settings, m.deps.BeginMFAAuthentication)+}diff --git a/backend/internal/services/auth_service.go b/backend/internal/auth/service.gosimilarity index 91%rename from backend/internal/services/auth_service.gorename to backend/internal/auth/service.goindex e93fda1b8..3283ce8af 100644--- a/backend/internal/services/auth_service.go+++ b/backend/internal/auth/service.go@@ -1,4 +1,4 @@-package services+package authimport ("context"@@ -15,7 +15,12 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/jwtclaims""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/validation""github.com/getarcaneapp/arcane/types/v2/auth"@@ -25,14 +30,10 @@ import ()const (- ErrInvalidCredentials = errors.Sentinel("invalid credentials")- ErrUserNotFound = errors.Sentinel("user not found")- ErrInvalidToken = errors.Sentinel("invalid token")- ErrExpiredToken = errors.Sentinel("token expired")- ErrTokenVersionMismatch = errors.Sentinel("token version mismatch")- ErrLocalAuthDisabled = errors.Sentinel("local authentication is disabled")- ErrOidcAuthDisabled = errors.Sentinel("OIDC authentication is disabled")- ErrMFARequired = errors.Sentinel("multi-factor authentication is required")+ ErrInvalidCredentials = errors.Sentinel("invalid credentials")+ ErrLocalAuthDisabled = errors.Sentinel("local authentication is disabled")+ ErrOidcAuthDisabled = errors.Sentinel("OIDC authentication is disabled")+ ErrMFARequired = errors.Sentinel("multi-factor authentication is required"))type TokenPair struct {@@ -75,11 +76,11 @@ type verifiedTokenEntry struct {}type AuthService struct {- userService *UserService- settingsService *SettingsService- eventService *EventService- sessionService *SessionService- roleService *RoleService+ userService *user.UserService+ settingsService *settings.SettingsService+ eventService *event.EventService+ sessionService *session.SessionService+ roleService *role.RoleServicejwtSecret []byterefreshExpiry time.Durationconfig *config.Config@@ -90,7 +91,7 @@ type AuthService struct {tokenCache *hot.HotCache[string, verifiedTokenEntry]}-func NewAuthService(userService *UserService, settingsService *SettingsService, eventService *EventService, sessionService *SessionService, roleService *RoleService, jwtSecret string, cfg *config.Config, errorHandler emperror.ErrorHandler) *AuthService {+func NewAuthService(userService *user.UserService, settingsService *settings.SettingsService, eventService *event.EventService, sessionService *session.SessionService, roleService *role.RoleService, jwtSecret string, cfg *config.Config, errorHandler emperror.ErrorHandler) *AuthService {// Production managers must supply an explicit, non-default JWT_SECRET (fail// closed, mirroring the ENCRYPTION_KEY guard). Dev and agent mode auto-generate.requireExplicitSecret := cfg.Environment == config.AppEnvironmentProduction && !cfg.AgentMode@@ -152,7 +153,7 @@ func (s *AuthService) getAuthSettings(ctx context.Context) (*AuthSettings, error}func (s *AuthService) GetOidcConfigurationStatus(ctx context.Context) (*auth.OidcStatusInfo, error) {- oidcEnvForced := s.settingsService != nil && s.settingsService.isEnvOverrideActiveInternal("oidcEnabled")+ oidcEnvForced := s.settingsService != nil && s.settingsService.IsEnvOverrideActive("oidcEnabled")mergeAccounts := falseproviderName := ""@@ -160,7 +161,7 @@ func (s *AuthService) GetOidcConfigurationStatus(ctx context.Context) (*auth.Oidif s.settingsService != nil {func() {defer func() {- // In tests, a zero-valued SettingsService may panic; treat as merge disabled+ // In tests, a zero-valued settings.SettingsService may panic; treat as merge disabled_ = recover()}()if settings, err := s.settingsService.GetSettings(ctx); err == nil {@@ -287,13 +288,13 @@ func (s *AuthService) PrepareOidcLogin(ctx context.Context, userInfo auth.OidcUs// have succeeded. Source is server-selected and is persisted with the session.func (s *AuthService) CompleteLogin(ctx context.Context, user *models.User, meta auth.SessionMeta, source, mfaMethod string, eventMetadata ...models.JSON) (*TokenPair, error) {if user == nil {- return nil, ErrUserNotFound+ return nil, common.ErrUserNotFound}if strings.TrimSpace(source) == "" {source = models.UserSessionSourceLocal}mfaMethod = strings.TrimSpace(mfaMethod)- if user.PasskeyMFAEnabled && source != models.UserSessionSourcePasskey && mfaMethod != PasskeyMFAMethod && mfaMethod != RecoveryCodeMFAMethod {+ if user.PasskeyMFAEnabled && source != models.UserSessionSourcePasskey && mfaMethod != models.PasskeyMFAMethod && mfaMethod != models.RecoveryCodeMFAMethod {return nil, ErrMFARequired}meta.Source = source@@ -377,7 +378,7 @@ func (s *AuthService) LogLogout(ctx context.Context, user *models.User) {func (s *AuthService) findOrCreateOidcUser(ctx context.Context, userInfo auth.OidcUserInfo, tokenResp *auth.OidcTokenResponse) (*models.User, bool, error) {user, err := s.userService.GetUserByOidcSubjectId(ctx, userInfo.Subject)- if err != nil && !errors.Is(err, ErrUserNotFound) {+ if err != nil && !errors.Is(err, common.ErrUserNotFound) {return nil, false, err}@@ -414,7 +415,7 @@ func (s *AuthService) tryMergeOidcUser(ctx context.Context, userInfo auth.OidcUsexistingUser, emailErr := s.userService.GetUserByEmail(ctx, userInfo.Email)if emailErr != nil {- if errors.Is(emailErr, ErrUserNotFound) {+ if errors.Is(emailErr, common.ErrUserNotFound) {return nil, false, nil}return nil, false, emailErr@@ -664,11 +665,11 @@ func (s *AuthService) RefreshToken(ctx context.Context, refreshToken string, metreturn s.jwtSecret, nil})if err != nil {- return nil, ErrInvalidToken+ return nil, common.ErrInvalidToken}if !token.Valid {- return nil, ErrInvalidToken+ return nil, common.ErrInvalidToken}claims, ok := token.Claims.(*refreshClaims)@@ -697,14 +698,14 @@ func (s *AuthService) RefreshToken(ctx context.Context, refreshToken string, metreturn nil, common.Classify(common.ErrUnavailable, errors.New("Session service is not configured"))}- session, err := s.sessionService.GetSessionByID(ctx, claims.SessionID)+ userSession, err := s.sessionService.GetSessionByID(ctx, claims.SessionID)if err != nil {return nil, err}- if session.UserID != claims.UserID {- return nil, ErrInvalidToken+ if userSession.UserID != claims.UserID {+ return nil, common.ErrInvalidToken}- if err := validateSessionActiveInternal(session); err != nil {+ if err := session.ValidateActive(userSession); err != nil {return nil, err}@@ -731,13 +732,13 @@ func (s *AuthService) VerifyToken(ctx context.Context, accessToken string) (*mod})if err != nil {if strings.Contains(err.Error(), "token is expired") {- return nil, "", ErrExpiredToken+ return nil, "", common.ErrExpiredToken}- return nil, "", ErrInvalidToken+ return nil, "", common.ErrInvalidToken}if !token.Valid {- return nil, "", ErrInvalidToken+ return nil, "", common.ErrInvalidToken}claims, ok := token.Claims.(*userClaims)@@ -755,7 +756,7 @@ func (s *AuthService) VerifyToken(ctx context.Context, accessToken string) (*modif claims.AppVersion != "" && claims.AppVersion != config.Version {slog.InfoContext(ctx, "Token version mismatch detected", "tokenVersion", claims.AppVersion, "currentVersion", config.Version, "user", claims.Username)- return nil, "", ErrTokenVersionMismatch+ return nil, "", common.ErrTokenVersionMismatch}if claims.SessionID == "" {return nil, "", common.Classify(common.ErrTokenValidation, errors.New("Missing session ID in token"))@@ -768,19 +769,19 @@ func (s *AuthService) VerifyToken(ctx context.Context, accessToken string) (*modif cached, ok, _ := s.tokenCache.Get(tokenHash); ok {if cached.User.ID != claims.ID || cached.SessionID != claims.SessionID {s.tokenCache.Delete(tokenHash)- return nil, "", ErrInvalidToken+ return nil, "", common.ErrInvalidToken}- session, err := s.sessionService.GetSessionByID(ctx, cached.SessionID)+ userSession, err := s.sessionService.GetSessionByID(ctx, cached.SessionID)if err != nil {s.tokenCache.Delete(tokenHash)return nil, "", err}- if session.UserID != cached.User.ID {+ if userSession.UserID != cached.User.ID {s.tokenCache.Delete(tokenHash)- return nil, "", ErrInvalidToken+ return nil, "", common.ErrInvalidToken}- if err := validateSessionActiveInternal(session); err != nil {+ if err := session.ValidateActive(userSession); err != nil {s.tokenCache.Delete(tokenHash)return nil, "", err}@@ -793,26 +794,26 @@ func (s *AuthService) VerifyToken(ctx context.Context, accessToken string) (*mod// even if the JWT signature is still valid (e.g. same JWT_SECRET).dbUser, err := s.userService.GetUserByID(ctx, claims.ID)if err != nil {- if errors.Is(err, ErrUserNotFound) {- return nil, "", ErrInvalidToken+ if errors.Is(err, common.ErrUserNotFound) {+ return nil, "", common.ErrInvalidToken}return nil, "", err}- session, err := s.sessionService.GetSessionByID(ctx, claims.SessionID)+ userSession, err := s.sessionService.GetSessionByID(ctx, claims.SessionID)if err != nil {return nil, "", err}- if session.UserID != dbUser.ID {- return nil, "", ErrInvalidToken+ if userSession.UserID != dbUser.ID {+ return nil, "", common.ErrInvalidToken}- if err := validateSessionActiveInternal(session); err != nil {+ if err := session.ValidateActive(userSession); err != nil {return nil, "", err}- s.tokenCache.Set(tokenHash, verifiedTokenEntry{User: *dbUser, SessionID: session.ID})+ s.tokenCache.Set(tokenHash, verifiedTokenEntry{User: *dbUser, SessionID: userSession.ID})- return dbUser, session.ID, nil+ return dbUser, userSession.ID, nil}func hashTokenInternal(token string) string {@@ -981,10 +982,10 @@ func (s *AuthService) IssueFederatedToken(ctx context.Context, user *models.Userreturn nil, common.Classify(common.ErrUnavailable, errors.New("Session service is not configured"))}if user == nil {- return nil, ErrUserNotFound+ return nil, common.ErrUserNotFound}- ttlSeconds = clampFederatedTokenTTLSecondsInternal(ttlSeconds)+ ttlSeconds = ClampFederatedTokenTTLSeconds(ttlSeconds)now := time.Now()accessTokenExpiry := now.Add(time.Duration(ttlSeconds) * time.Second)@@ -1027,32 +1028,6 @@ func (s *AuthService) IssueFederatedToken(ctx context.Context, user *models.User}, nil}-func clampFederatedTokenTTLSecondsInternal(ttlSeconds int) int {- if ttlSeconds <= 0 {- return 900- }- if ttlSeconds < 60 {- return 60- }- if ttlSeconds > 3600 {- return 3600- }- return ttlSeconds-}--func validateSessionActiveInternal(session *models.UserSession) error {- if session == nil {- return ErrInvalidToken- }- if session.RevokedAt != nil {- return common.Classify(common.ErrSessionRevoked, errors.New("Session has been revoked"))- }- if time.Now().After(session.ExpiresAt) {- return ErrExpiredToken- }- return nil-}-func generateUsernameFromEmail(email, subject string) string {if email != "" {parts := strings.Split(email, "@")@@ -1092,3 +1067,18 @@ func (s *AuthService) runInBackground(ctx context.Context, name string, fn func(}}()}++// ClampFederatedTokenTTLSeconds bounds a requested federated-token lifetime to+// the range Arcane will mint.+func ClampFederatedTokenTTLSeconds(ttlSeconds int) int {+ if ttlSeconds <= 0 {+ return 900+ }+ if ttlSeconds < 60 {+ return 60+ }+ if ttlSeconds > 3600 {+ return 3600+ }+ return ttlSeconds+}diff --git a/backend/internal/services/auth_service_test.go b/backend/internal/auth/service_test.gosimilarity index 85%rename from backend/internal/services/auth_service_test.gorename to backend/internal/auth/service_test.goindex 1c3ce8bfa..04670a5d3 100644--- a/backend/internal/services/auth_service_test.go+++ b/backend/internal/auth/service_test.go@@ -1,4 +1,4 @@-package services+package authimport ("context"@@ -8,12 +8,17 @@ import ("github.com/libtnb/sqlite""github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest""gorm.io/gorm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/types/v2/auth""github.com/golang-jwt/jwt/v5""github.com/samber/hot"@@ -38,6 +43,25 @@ func setupAuthServiceTestDB(t *testing.T) *database.DB {return &database.DB{DB: db}}+func newSettingsServiceForAuthTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ writes, err := actors.NewExecutor(t.Context(), runtime, "auth-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "auth-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, writes.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, writes, effects)+}+func newTestAuthService(secret string) *AuthService {if secret == "" {b := make([]byte, 32)@@ -126,7 +150,7 @@ func makeRefreshToken(t *testing.T, secret []byte, subject string, id string, exfunc createTestSession(t *testing.T, db *database.DB, userID string, expiresAt time.Time) (*models.UserSession, string) {t.Helper()- sessionSvc := NewSessionService(db)+ sessionSvc := session.NewSessionService(db)session, refreshJTI, err := sessionSvc.CreateSession(context.Background(), userID, expiresAt, auth.SessionMeta{UserAgent: "test-agent",IPAddress: "127.0.0.1",@@ -148,10 +172,10 @@ func makeUnsignedToken(t *testing.T, claims jwt.Claims) string {func TestVerifyToken_ValidClaims(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)s := newTestAuthService("")s.userService = userSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)// Create user in DBuser := &models.User{@@ -203,17 +227,17 @@ func TestVerifyToken_RejectsNonHMACAlg(t *testing.T) {_, _, err := s.VerifyToken(context.Background(), token)- assert.ErrorIs(t, err, ErrInvalidToken,- "want ErrInvalidToken, got %v", err)+ assert.ErrorIs(t, err, common.ErrInvalidToken,+ "want common.ErrInvalidToken, got %v", err)}func TestVerifyToken_Expired(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)s := newTestAuthService("")s.userService = userSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)// Create user in DBuser := &models.User{@@ -228,7 +252,7 @@ func TestVerifyToken_Expired(t *testing.T) {_, _, err = s.VerifyToken(context.Background(), token)- assert.ErrorIs(t, err, ErrExpiredToken,+ assert.ErrorIs(t, err, common.ErrExpiredToken,"want ErrExpiredToken, got %v", err)}@@ -256,8 +280,8 @@ func TestVerifyToken_InvalidSignature(t *testing.T) {_, _, err := s.VerifyToken(context.Background(), token)- assert.ErrorIs(t, err, ErrInvalidToken,- "want ErrInvalidToken, got %v", err)+ assert.ErrorIs(t, err, common.ErrInvalidToken,+ "want common.ErrInvalidToken, got %v", err)}@@ -329,7 +353,7 @@ func TestVerifyToken_VersionMismatch(t *testing.T) {_, _, err := s.VerifyToken(context.Background(), token)- assert.ErrorIs(t, err, ErrTokenVersionMismatch,+ assert.ErrorIs(t, err, common.ErrTokenVersionMismatch,"want ErrTokenVersionMismatch, got %v", err)config.Version = oldVersion@@ -337,13 +361,13 @@ func TestVerifyToken_VersionMismatch(t *testing.T) {func TestRefreshToken_Valid(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)- settingsSvc, err := newSettingsServiceForTestInternal(t, context.Background(), db)+ userSvc := user.NewUserService(db)+ settingsSvc, err := newSettingsServiceForAuthTestInternal(t, context.Background(), db)require.NoError(t, err)s := newTestAuthService("")s.userService = userSvcs.settingsService = settingsSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)user := &models.User{BaseModel: models.BaseModel{ID: "u-refresh"},@@ -368,13 +392,13 @@ func TestRefreshToken_Valid(t *testing.T) {// users logged in across backend releases — see plans/how-can-we-make-compiled-quilt.md.func TestRefreshToken_VersionMismatchRotates(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)- settingsSvc, err := newSettingsServiceForTestInternal(t, context.Background(), db)+ userSvc := user.NewUserService(db)+ settingsSvc, err := newSettingsServiceForAuthTestInternal(t, context.Background(), db)require.NoError(t, err)s := newTestAuthService("")s.userService = userSvcs.settingsService = settingsSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)user := &models.User{BaseModel: models.BaseModel{ID: "u-versionmismatch"},@@ -417,10 +441,10 @@ func TestRefreshToken_VersionMismatchRotates(t *testing.T) {func TestVerifyToken_RejectsRevokedSession(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)s := newTestAuthService("")s.userService = userSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)user := &models.User{BaseModel: models.BaseModel{ID: "u-revoked"},@@ -440,10 +464,10 @@ func TestVerifyToken_RejectsRevokedSession(t *testing.T) {func TestVerifyToken_RejectsMissingSessionID(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)s := newTestAuthService("")s.userService = userSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)user := &models.User{BaseModel: models.BaseModel{ID: "u-no-sid"},@@ -460,10 +484,10 @@ func TestVerifyToken_RejectsMissingSessionID(t *testing.T) {func TestRevokeSessionThenVerifyTokenFails(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)s := newTestAuthService("")s.userService = userSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)user := &models.User{BaseModel: models.BaseModel{ID: "u-logout"},@@ -483,10 +507,10 @@ func TestRevokeSessionThenVerifyTokenFails(t *testing.T) {func TestVerifyToken_RejectsRevokedCachedSession(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)s := newTestAuthService("")s.userService = userSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)user := &models.User{BaseModel: models.BaseModel{ID: "u-cached-revoked"},@@ -496,12 +520,12 @@ func TestVerifyToken_RejectsRevokedCachedSession(t *testing.T) {require.NoError(t, err)exp := time.Now().Add(5 * time.Minute)- session, _ := createTestSession(t, db, user.ID, exp)- token := makeAccessToken(t, s.jwtSecret, "access", user.ID, user.Username, []string{"user"}, "", "", exp, session.ID)+ sessionRecord, _ := createTestSession(t, db, user.ID, exp)+ token := makeAccessToken(t, s.jwtSecret, "access", user.ID, user.Username, []string{"user"}, "", "", exp, sessionRecord.ID)_, _, err = s.VerifyToken(context.Background(), token)require.NoError(t, err)- require.NoError(t, NewSessionService(db).RevokeSession(context.Background(), session.ID))+ require.NoError(t, session.NewSessionService(db).RevokeSession(context.Background(), sessionRecord.ID))_, _, err = s.VerifyToken(context.Background(), token)require.ErrorIs(t, err, common.ErrSessionRevoked, "expected cached access token to be rejected after cross-process revocation, got %v", err)@@ -509,13 +533,13 @@ func TestVerifyToken_RejectsRevokedCachedSession(t *testing.T) {func TestRefreshToken_RotatesJTI(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)- settingsSvc, err := newSettingsServiceForTestInternal(t, context.Background(), db)+ userSvc := user.NewUserService(db)+ settingsSvc, err := newSettingsServiceForAuthTestInternal(t, context.Background(), db)require.NoError(t, err)s := newTestAuthService("")s.userService = userSvcs.settingsService = settingsSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)user := &models.User{BaseModel: models.BaseModel{ID: "u-rotate"},@@ -533,18 +557,18 @@ func TestRefreshToken_RotatesJTI(t *testing.T) {require.NotEmpty(t, tokenPair.RefreshToken)_, err = s.RefreshToken(context.Background(), token, auth.SessionMeta{})- require.ErrorIs(t, err, ErrInvalidToken)+ require.ErrorIs(t, err, common.ErrInvalidToken)}func TestRefreshToken_RejectsRevokedSession(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)- settingsSvc, err := newSettingsServiceForTestInternal(t, context.Background(), db)+ userSvc := user.NewUserService(db)+ settingsSvc, err := newSettingsServiceForAuthTestInternal(t, context.Background(), db)require.NoError(t, err)s := newTestAuthService("")s.userService = userSvcs.settingsService = settingsSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)user := &models.User{BaseModel: models.BaseModel{ID: "u-refresh-revoked"},@@ -564,10 +588,10 @@ func TestRefreshToken_RejectsRevokedSession(t *testing.T) {func TestChangePassword_RevokesAllSessions(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)s := newTestAuthService("")s.userService = userSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)passwordHash, err := userSvc.HashPassword("old-password")require.NoError(t, err)@@ -594,10 +618,10 @@ func TestChangePassword_RevokesAllSessions(t *testing.T) {func TestChangePassword_KeepsCurrentSessionAlive(t *testing.T) {db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)s := newTestAuthService("")s.userService = userSvc- s.sessionService = NewSessionService(db)+ s.sessionService = session.NewSessionService(db)passwordHash, err := userSvc.HashPassword("old-password")require.NoError(t, err)@@ -634,18 +658,19 @@ func TestRefreshToken_RejectsNonHMACAlg(t *testing.T) {_, err := s.RefreshToken(context.Background(), token, auth.SessionMeta{})- assert.ErrorIs(t, err, ErrInvalidToken,- "want ErrInvalidToken, got %v", err)+ assert.ErrorIs(t, err, common.ErrInvalidToken,+ "want common.ErrInvalidToken, got %v", err)}func TestGetOidcConfigurationStatus(t *testing.T) {+ db := setupAuthServiceTestDB(t)// Disableds := newTestAuthService("")s.config = &config.Config{}- // Set a non-nil settingsService to prevent nil pointer dereference- // GetSettings will fail gracefully and mergeAccounts will default to false- s.settingsService = &SettingsService{}+ settingsSvc, err := newSettingsServiceForAuthTestInternal(t, context.Background(), db)+ require.NoError(t, err)+ s.settingsService = settingsSvcstatus, err := s.GetOidcConfigurationStatus(context.Background())@@ -662,7 +687,9 @@ func TestGetOidcConfigurationStatus(t *testing.T) {// Explicit env override to false should still be treated as forcedt.Setenv("OIDC_ENABLED", "false")- s.settingsService.envOverrides = resolveSettingsEnvOverridesInternal()+ settingsSvc, err = newSettingsServiceForAuthTestInternal(t, context.Background(), db)+ require.NoError(t, err)+ s.settingsService = settingsSvcs.config.OidcEnabled = falsestatus, err = s.GetOidcConfigurationStatus(context.Background())@@ -674,7 +701,9 @@ func TestGetOidcConfigurationStatus(t *testing.T) {// Enabled but missing fieldst.Setenv("OIDC_ENABLED", "true")- s.settingsService.envOverrides = resolveSettingsEnvOverridesInternal()+ settingsSvc, err = newSettingsServiceForAuthTestInternal(t, context.Background(), db)+ require.NoError(t, err)+ s.settingsService = settingsSvcs.config.OidcEnabled = truestatus, err = s.GetOidcConfigurationStatus(context.Background())@@ -701,12 +730,12 @@ func TestFindOrCreateOidcUser_MergeEnabled_EmailNotVerified_NoExistingUser_Creatctx := context.Background()db := setupAuthServiceTestDB(t)- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)+ settingsSvc, err := newSettingsServiceForAuthTestInternal(t, ctx, db)require.NoError(t, err)require.NoError(t, settingsSvc.EnsureDefaultSettings(ctx))require.NoError(t, settingsSvc.SetBoolSetting(ctx, "oidcMergeAccounts", true))- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)authSvc := newTestAuthService("")authSvc.userService = userSvcauthSvc.settingsService = settingsSvc@@ -737,12 +766,12 @@ func TestFindOrCreateOidcUser_MergeEnabled_EmailNotVerified_WithExistingUser_Retctx := context.Background()db := setupAuthServiceTestDB(t)- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)+ settingsSvc, err := newSettingsServiceForAuthTestInternal(t, ctx, db)require.NoError(t, err)require.NoError(t, settingsSvc.EnsureDefaultSettings(ctx))require.NoError(t, settingsSvc.SetBoolSetting(ctx, "oidcMergeAccounts", true))- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)// Seed an existing local user with matching emailemail := "[email protected]"existing := &models.User{@@ -779,12 +808,12 @@ func TestFindOrCreateOidcUser_MergeEnabled_EmailVerificationMissing_WithExistingctx := context.Background()db := setupAuthServiceTestDB(t)- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)+ settingsSvc, err := newSettingsServiceForAuthTestInternal(t, ctx, db)require.NoError(t, err)require.NoError(t, settingsSvc.EnsureDefaultSettings(ctx))require.NoError(t, settingsSvc.SetBoolSetting(ctx, "oidcMergeAccounts", true))- userSvc := NewUserService(db)+ userSvc := user.NewUserService(db)// Seed an existing local user with matching emailemail := "[email protected]"existing := &models.User{diff --git a/backend/internal/bootstrap/bootstrap.go b/backend/internal/bootstrap/bootstrap.goindex 4260c609f..e9f2452e1 100644--- a/backend/internal/bootstrap/bootstrap.go+++ b/backend/internal/bootstrap/bootstrap.go@@ -16,14 +16,24 @@ import ("github.com/getarcaneapp/arcane/backend/v2/api/ws""github.com/getarcaneapp/arcane/backend/v2/internal/actors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/di"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitops"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/swarm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user"+ "github.com/getarcaneapp/arcane/backend/v2/internal/variable"+ "github.com/getarcaneapp/arcane/backend/v2/internal/volume""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/startup""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"- httputils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx""github.com/labstack/echo/v5""go.getarcane.app/streams/logs"libcrypto "go.getarcane.app/sys/crypto"@@ -141,9 +151,9 @@ func isWeakProductionEncryptionKeyInternal(encryptionKey, environment string, agfunc newConfiguredHTTPClient(cfg *config.Config) *http.Client {if cfg.HTTPClientTimeout > 0 {- return httputils.NewHTTPClientWithTimeout(time.Duration(cfg.HTTPClientTimeout) * time.Second)+ return httpx.NewHTTPClientWithTimeout(time.Duration(cfg.HTTPClientTimeout) * time.Second)}- return httputils.NewHTTPClient()+ return httpx.NewHTTPClient()}type initializeStartupStateParams struct {@@ -153,17 +163,17 @@ type initializeStartupStateParams struct {Config *config.ConfigHTTPClient *http.Client- Volume *services.VolumeService- Settings *services.SettingsService- Environment *services.EnvironmentService- GitOpsSync *services.GitOpsSyncService- Project *services.ProjectService- Variable *services.VariableService- Docker *services.DockerClientService- Swarm *services.SwarmService- Role *services.RoleService- User *services.UserService- ApiKey *services.ApiKeyService+ Volume *volume.Module+ Settings *settings.SettingsService+ Environment *environment.EnvironmentService+ GitOpsSync *gitops.GitOpsSyncService+ Project *project.ProjectService+ Variable *variable.VariableService+ Docker *docker.DockerClientService+ Swarm *swarm.SwarmService+ Role *role.RoleService+ User *user.UserService+ ApiKey *apikey.ApiKeyService}func initializeStartupState(p initializeStartupStateParams) {@@ -171,8 +181,8 @@ func initializeStartupState(p initializeStartupStateParams) {cfg := p.ConfighttpClient := p.HTTPClient- if p.Volume != nil {- startup.CleanupOrphanedVolumeHelpers(appCtx, p.Volume.CleanupOrphanedVolumeHelpers)+ if p.Volume.Service() != nil {+ startup.CleanupOrphanedVolumeHelpers(appCtx, p.Volume.Service().CleanupOrphanedVolumeHelpers)}runtimeCfg := &startup.RuntimeConfig{@@ -294,7 +304,7 @@ func initializeStartupState(p initializeStartupStateParams) {}}-func initializeGitOpsStartupStateInternal(appCtx context.Context, gitOpsSync *services.GitOpsSyncService) {+func initializeGitOpsStartupStateInternal(appCtx context.Context, gitOpsSync *gitops.GitOpsSyncService) {if gitOpsSync == nil {return}@@ -311,7 +321,7 @@ func initializeGitOpsStartupStateInternal(appCtx context.Context, gitOpsSync *se}}-func runRoleStartupTasks(ctx context.Context, roleService *services.RoleService, cfg *config.Config, agentMode bool) {+func runRoleStartupTasks(ctx context.Context, roleService *role.RoleService, cfg *config.Config, agentMode bool) {if roleService == nil {return}diff --git a/backend/internal/bootstrap/bootstrap_test.go b/backend/internal/bootstrap/bootstrap_test.goindex 191c3f2ab..01b0814a7 100644--- a/backend/internal/bootstrap/bootstrap_test.go+++ b/backend/internal/bootstrap/bootstrap_test.go@@ -13,9 +13,9 @@ import ("github.com/getarcaneapp/arcane/backend/v2/api""github.com/getarcaneapp/arcane/backend/v2/internal/actors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge"tunnelpb "github.com/getarcaneapp/arcane/backend/v2/proto/tunnel/v1""github.com/labstack/echo/v5"@@ -182,7 +182,7 @@ func TestHTTP2APIResponsesDoNotUseAPIGzipInternal(t *testing.T) {Context: context.Background(),Config: cfg,HandlerDeps: api.HandlerDeps{},- AuthMiddleware: middleware.NewAuthMiddleware(nil, cfg),+ AuthMiddleware: auth.NewAuthMiddleware(nil, cfg),TunnelRegistry: edge.NewTunnelRegistry(),})handler, protocols := configureHTTPProtocolsInternal(false, router)diff --git a/backend/internal/bootstrap/buildables_router_bootstrap.go b/backend/internal/bootstrap/buildables_router_bootstrap.goindex 0969c5099..a718c1fa8 100644--- a/backend/internal/bootstrap/buildables_router_bootstrap.go+++ b/backend/internal/bootstrap/buildables_router_bootstrap.go@@ -9,6 +9,6 @@ import (func init() {registerBuildableRoutes = append(registerBuildableRoutes, func(apiGroup *echo.Group, deps api.HandlerDeps) {- api.SetupBuildablesRoutes(apiGroup, deps.Auth)+ api.SetupBuildablesRoutes(apiGroup, deps.Auth.Service())})}diff --git a/backend/internal/bootstrap/edge_bootstrap.go b/backend/internal/bootstrap/edge_bootstrap.goindex 8816c56cc..f1f18f6e9 100644--- a/backend/internal/bootstrap/edge_bootstrap.go+++ b/backend/internal/bootstrap/edge_bootstrap.go@@ -10,9 +10,10 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge""github.com/labstack/echo/v5""go.uber.org/fx"@@ -27,8 +28,8 @@ func registerEdgeTunnelRoutes(actorRuntime *actors.Runtime,cfg *config.Config,apiGroup *echo.Group,- environmentService *services.EnvironmentService,- eventService *services.EventService,+ environmentService *environment.EnvironmentService,+ eventService *event.EventService,registry *edge.TunnelRegistry,) *edge.TunnelServer {// Resolver that validates API key and returns the environment ID@@ -75,7 +76,7 @@ func registerEdgeTunnelRoutes(}}- req := services.CreateEventRequest{+ req := event.CreateEventRequest{Type: models.EventType(evt.Type),Severity: models.EventSeverity(evt.Severity),Title: evt.Title,@@ -127,7 +128,7 @@ func registerEdgeTunnelRoutes(}envIDCopy := envIDenvNameCopy := envName- _, _ = eventService.CreateEvent(ctx, services.CreateEventRequest{+ _, _ = eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeEnvironmentMTLSEnroll,Severity: edgeMTLSEnrollmentSeverityInternal(reenrolled),Title: "Edge mTLS enrollment",@@ -168,12 +169,12 @@ func registerEdgeTunnelRoutes(return server}-func createEdgeMTLSIssueEventsInternal(ctx context.Context, eventService *services.EventService, envID string, envName string, remoteAddr string, certIssued bool, caGenerated bool, reenrolled bool) {+func createEdgeMTLSIssueEventsInternal(ctx context.Context, eventService *event.EventService, envID string, envName string, remoteAddr string, certIssued bool, caGenerated bool, reenrolled bool) {if eventService == nil {return}if caGenerated {- _, _ = eventService.CreateEvent(ctx, services.CreateEventRequest{+ _, _ = eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeEnvironmentMTLSCAGenerated,Severity: models.EventSeverityInfo,Title: "Edge mTLS CA generated",@@ -182,7 +183,7 @@ func createEdgeMTLSIssueEventsInternal(ctx context.Context, eventService *servic})}if certIssued {- _, _ = eventService.CreateEvent(ctx, services.CreateEventRequest{+ _, _ = eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeEnvironmentMTLSCertIssued,Severity: edgeMTLSCertIssuedSeverityInternal(reenrolled),Title: "Edge mTLS certificate issued",@@ -217,7 +218,7 @@ func optionalStringPtr(value string) *string {return &value}-func createEdgeConnectionEvent(ctx context.Context, eventService *services.EventService, envID, envName string, connected bool) error {+func createEdgeConnectionEvent(ctx context.Context, eventService *event.EventService, envID, envName string, connected bool) error {if eventService == nil {return nil}@@ -234,7 +235,7 @@ func createEdgeConnectionEvent(ctx context.Context, eventService *services.Eventseverity = models.EventSeveritySuccess}- _, err := eventService.CreateEvent(ctx, services.CreateEventRequest{+ _, err := eventService.CreateEvent(ctx, event.CreateEventRequest{Type: eventType,Severity: severity,Title: title,diff --git a/backend/internal/bootstrap/jobs_bootstrap.go b/backend/internal/bootstrap/jobs_bootstrap.goindex 141df9fe6..4e2113da9 100644--- a/backend/internal/bootstrap/jobs_bootstrap.go+++ b/backend/internal/bootstrap/jobs_bootstrap.go@@ -7,17 +7,22 @@ import ("net/http""slices"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity""github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitops"+ "github.com/getarcaneapp/arcane/backend/v2/internal/job""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/system""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""github.com/getarcaneapp/arcane/backend/v2/pkg/scheduler"schedulertypes "github.com/getarcaneapp/arcane/types/v2/scheduler""go.uber.org/fx")-func newJobScheduler(appCtx context.Context, lc fx.Lifecycle, cfg *config.Config, runtime *actors.Runtime, _ *actors.Gate[actors.AdmissionKey], imageUpdateWatcher *scheduler.ImageUpdateWatcher, analytics *scheduler.AnalyticsJob, systemUpgrade *services.SystemUpgradeService) (schedulertypes.JobScheduler, error) {+func newJobScheduler(appCtx context.Context, lc fx.Lifecycle, cfg *config.Config, runtime *actors.Runtime, _ *actors.Gate[actors.AdmissionKey], imageUpdateWatcher *scheduler.ImageUpdateWatcher, analytics *scheduler.AnalyticsJob, systemUpgrade *system.SystemUpgradeService) (schedulertypes.JobScheduler, error) {schedulerCtx, cancelScheduler := context.WithCancel(appCtx)jobScheduler, err := scheduler.NewJobScheduler(schedulerCtx, runtime, cfg.GetLocation())if err != nil {@@ -66,11 +71,11 @@ type registerJobsParams struct {Scheduler schedulertypes.JobSchedulerActorRuntime *actors.Runtime- Activity *services.ActivityService- GitOpsSync *services.GitOpsSyncService- Environment *services.EnvironmentService- JobSchedule *services.JobService- Settings *services.SettingsService+ Activity *activity.ActivityService+ GitOpsSync *gitops.GitOpsSyncService+ Environment *environment.EnvironmentService+ JobSchedule *job.JobService+ Settings *settings.SettingsServiceAdmission *actors.Gate[actors.AdmissionKey]AutoUpdate *scheduler.AutoUpdateJob@@ -173,9 +178,9 @@ type dynamicJobsParams struct {AppCtx context.ContextConfig *config.ConfigScheduler schedulertypes.JobScheduler- GitOpsSync *services.GitOpsSyncService- Environment *services.EnvironmentService- JobSchedule *services.JobService+ GitOpsSync *gitops.GitOpsSyncService+ Environment *environment.EnvironmentService+ JobSchedule *job.JobServiceAdmission *actors.Gate[actors.AdmissionKey]}diff --git a/backend/internal/bootstrap/playwright_router_bootstrap.go b/backend/internal/bootstrap/playwright_router_bootstrap.goindex 16666a934..da4a8fab4 100644--- a/backend/internal/bootstrap/playwright_router_bootstrap.go+++ b/backend/internal/bootstrap/playwright_router_bootstrap.go@@ -6,20 +6,22 @@ import ("log/slog""github.com/getarcaneapp/arcane/backend/v2/api"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/playwright""github.com/labstack/echo/v5")func init() {registerPlaywrightRoutes = []func(apiGroup *echo.Group, deps api.HandlerDeps){func(apiGroup *echo.Group, deps api.HandlerDeps) {- playwrightService := services.NewPlaywrightService(deps.ApiKey, deps.User, deps.Federated)- if playwrightService == nil {+ apiKeyService := deps.ApiKey.Service()+ userService := deps.User.Service()+ if apiKeyService == nil || userService == nil || deps.Federated == nil {slog.Warn("Playwright service not available, skipping playwright routes")return}- api.SetupPlaywrightRoutes(apiGroup, playwrightService)+ playwrightService := playwright.NewPlaywrightService(apiKeyService, userService)+ playwright.SetupRoutes(apiGroup, playwrightService, deps.Federated)slog.Info("Playwright routes registered for E2E testing")},}diff --git a/backend/internal/bootstrap/router_bootstrap.go b/backend/internal/bootstrap/router_bootstrap.goindex 084b2d2eb..06ce77c51 100644--- a/backend/internal/bootstrap/router_bootstrap.go+++ b/backend/internal/bootstrap/router_bootstrap.go@@ -15,17 +15,19 @@ import (slogecho "github.com/samber/slog-echo/v2""github.com/getarcaneapp/arcane/backend/v2/api"- "github.com/getarcaneapp/arcane/backend/v2/api/handlers""github.com/getarcaneapp/arcane/backend/v2/api/ws""github.com/getarcaneapp/arcane/backend/v2/frontend""github.com/getarcaneapp/arcane/backend/v2/internal/actors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/federated""github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/cookie"- httputil "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx""github.com/getarcaneapp/arcane/types/v2""go.uber.org/fx")@@ -102,7 +104,7 @@ func requestLoggerMiddlewareInternal() echo.MiddlewareFunc {func createAuthValidatorInternal(deps api.HandlerDeps) middleware.AuthValidator {resolveUser := func(ctx context.Context, user *models.User) *authz.PermissionSet {- ps, err := deps.Role.ResolvePermissions(ctx, user)+ ps, err := deps.Role.Service().ResolvePermissions(ctx, user)if err != nil || ps == nil {slog.WarnContext(ctx, "failed to resolve user permissions for env proxy", "error", err)return authz.NewPermissionSet()@@ -110,7 +112,7 @@ func createAuthValidatorInternal(deps api.HandlerDeps) middleware.AuthValidatorreturn ps}resolveKey := func(ctx context.Context, keyID string) *authz.PermissionSet {- ps, err := deps.Role.ResolveApiKeyPermissions(ctx, keyID)+ ps, err := deps.Role.Service().ResolveApiKeyPermissions(ctx, keyID)if err != nil || ps == nil {slog.WarnContext(ctx, "failed to resolve api key permissions for env proxy", "error", err)return authz.NewPermissionSet()@@ -123,7 +125,7 @@ func createAuthValidatorInternal(deps api.HandlerDeps) middleware.AuthValidatorif apiKey := req.Header.Get("X-Api-Key"); apiKey != "" {// User-owned API key: personal keys inherit the owner's role// permissions; scoped keys are limited to their own grants.- if user, key, err := deps.ApiKey.ValidateApiKeyWithID(ctx, apiKey); err == nil && user != nil {+ if user, key, err := deps.ApiKey.Service().ValidateApiKeyWithID(ctx, apiKey); err == nil && user != nil {if key != nil && key.Kind != models.ApiKeyKindPersonal {return resolveKey(ctx, key.ID), user, true}@@ -131,7 +133,7 @@ func createAuthValidatorInternal(deps api.HandlerDeps) middleware.AuthValidator}// Environment bootstrap key (user_id = NULL): used by the proxy when forwarding// requests to a remote env whose apiUrl resolves back to this manager.- if envID, err := deps.ApiKey.GetEnvironmentByApiKey(ctx, apiKey); err == nil && envID != nil {+ if envID, err := deps.ApiKey.Service().GetEnvironmentByApiKey(ctx, apiKey); err == nil && envID != nil {return authz.EnvironmentPermissionSet(*envID), nil, true}return nil, nil, false@@ -149,7 +151,7 @@ func createAuthValidatorInternal(deps api.HandlerDeps) middleware.AuthValidatorreturn nil, nil, false}- user, _, err := deps.Auth.VerifyToken(ctx, token)+ user, _, err := deps.Auth.Service().VerifyToken(ctx, token)if err != nil || user == nil {return nil, nil, false}@@ -165,7 +167,7 @@ type RouterParams struct {ActorRuntime *actors.RuntimeConfig *config.ConfigHandlerDeps api.HandlerDeps- AuthMiddleware *middleware.AuthMiddleware+ AuthMiddleware *auth.AuthMiddlewareTunnelRegistry *edge.TunnelRegistry}@@ -226,10 +228,10 @@ func newRouter(p RouterParams) (*echo.Echo, *edge.TunnelServer) {apiGroup.Use(middleware.PerIPRateLimitForPaths([]string{"/api/events"}, 60, 30,))- handlerAppCtx := handlers.NewActivityAppContext(ctx)+ handlerAppCtx := handlerutil.NewActivityAppContext(ctx)envResolver := func(ctx context.Context, id string) (string, *string, bool, error) {- env, err := deps.Environment.GetEnvironmentByID(ctx, id)+ env, err := deps.Environment.Service().GetEnvironmentByID(ctx, id)if err != nil || env == nil {return "", nil, false, err}@@ -237,9 +239,9 @@ func newRouter(p RouterParams) (*echo.Echo, *edge.TunnelServer) {}// Register public webhook trigger endpoint before auth middleware (token in URL is the sole auth)- api.RegisterWebhookTrigger(apiGroup, deps.Webhook, handlerAppCtx)- handlers.RegisterFederatedTokenExchange(apiGroup, deps.Federated)- handlers.RegisterAgentEventIngestion(apiGroup, deps.Event, cfg)+ api.RegisterWebhookTrigger(apiGroup, deps.Webhook.Service(), handlerAppCtx)+ federated.RegisterFederatedTokenExchange(apiGroup, deps.Federated)+ deps.Event.RegisterAgentRoutes(apiGroup)permissionMatcher := authz.NewPermissionMatcher()@@ -253,7 +255,7 @@ func newRouter(p RouterParams) (*echo.Echo, *edge.TunnelServer) {// Proxied WebSocket upgrades enforce the same Origin policy as the local// endpoints, so a cross-origin page cannot ride a session cookie into a// remote environment.- httputil.ValidateWebSocketOrigin(cfg.GetAppURL()),+ httpx.ValidateWebSocketOrigin(cfg.GetAppURL()),)apiGroup.Use(envProxyMiddleware)@@ -271,13 +273,13 @@ func newRouter(p RouterParams) (*echo.Echo, *edge.TunnelServer) {}// Remaining echo handlers (WebSocket/streaming)- ws.NewWebSocketHandler(apiGroup, deps.Project, deps.Container, deps.Swarm, deps.System, deps.Diagnostics, authMiddleware, cfg)+ ws.NewWebSocketHandler(apiGroup, deps.Project.Service(), deps.Container.Service(), deps.Swarm.Service(), deps.System.Service(), deps.Diagnostics, authMiddleware, cfg)// Register edge tunnel endpoint for manager to accept agent connections// This is only registered when NOT in agent mode (i.e., running as manager)var tunnelServer *edge.TunnelServerif !cfg.AgentMode {- tunnelServer = registerEdgeTunnelRoutes(ctx, p.Lifecycle, p.ActorRuntime, cfg, apiGroup, deps.Environment, deps.Event, p.TunnelRegistry)+ tunnelServer = registerEdgeTunnelRoutes(ctx, p.Lifecycle, p.ActorRuntime, cfg, apiGroup, deps.Environment.Service(), deps.Event.Service(), p.TunnelRegistry)}if cfg.Environment != "production" {diff --git a/backend/internal/bootstrap/server_bootstrap.go b/backend/internal/bootstrap/server_bootstrap.goindex 120706299..348294c90 100644--- a/backend/internal/bootstrap/server_bootstrap.go+++ b/backend/internal/bootstrap/server_bootstrap.go@@ -188,7 +188,7 @@ func newHTTPServerInternal(baseCtx context.Context, listenAddr string, handler hIdleTimeout: 120 * time.Second,BaseContext: func(net.Listener) context.Context { return baseCtx },// Long-lived stream handlers tune dead-peer detection for their own- // connection; see httpx.SetDeadPeerTimeout.+ // connection; see handlerutil.SetDeadPeerTimeout.ConnContext: httpx.WithConn,}if !useTLS {diff --git a/backend/internal/services/build_service.go b/backend/internal/build/service.gosimilarity index 93%rename from backend/internal/services/build_service.gorename to backend/internal/build/service.goindex 81c6b7bd5..5801dfcf7 100644--- a/backend/internal/services/build_service.go+++ b/backend/internal/build/service.go@@ -1,4 +1,4 @@-package services+package buildimport ("context"@@ -11,10 +11,16 @@ import ("strings""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+"emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitrepo""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"buildgit "github.com/getarcaneapp/arcane/backend/v2/pkg/gitutil""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"@@ -27,11 +33,11 @@ import (type BuildService struct {db *database.DB- settings *SettingsService- dockerService *DockerClientService- registryService *ContainerRegistryService- gitRepository *GitRepositoryService- eventService *EventService+ settings *settings.SettingsService+ dockerService *docker.DockerClientService+ registryService *registry.ContainerRegistryService+ gitRepository *gitrepo.GitRepositoryService+ eventService *event.EventServicebuilder buildtypes.BuildergitProbeFn func(context.Context, string, buildgit.AuthConfig) errorgitCloneFn func(context.Context, string, string, buildgit.AuthConfig) (string, error)@@ -42,11 +48,11 @@ const buildHistoryOutputLimitBytes = 2 * 1024 * 1024func NewBuildService(db *database.DB,- settings *SettingsService,- dockerService *DockerClientService,- registryService *ContainerRegistryService,- gitRepository *GitRepositoryService,- eventService *EventService,+ settings *settings.SettingsService,+ dockerService *docker.DockerClientService,+ registryService *registry.ContainerRegistryService,+ gitRepository *gitrepo.GitRepositoryService,+ eventService *event.EventService,) *BuildService {svc := &BuildService{db: db,@@ -61,7 +67,7 @@ func NewBuildService(registryAuthProvider = registryService}- // ContainerRegistryService already implements buildtypes.RegistryAuthProvider,+ // registry.ContainerRegistryService already implements buildtypes.RegistryAuthProvider,// so the builder consumes it directly instead of through forwarding methods on BuildService.svc.builder = buildapi.NewService(buildapi.Config{SettingsProvider: svc,@@ -345,8 +351,8 @@ func (s *BuildService) probeGitContextInternal(ctx context.Context, repositoryURreturn s.gitProbeFn(ctx, repositoryURL, authConfig)}- if s.gitRepository != nil && s.gitRepository.gitClient != nil {- return s.gitRepository.gitClient.ProbeRemote(ctx, repositoryURL, authConfig)+ if s.gitRepository != nil && s.gitRepository.Client != nil {+ return s.gitRepository.ProbeRemote(ctx, repositoryURL, authConfig)}return errors.New("git repository service not available")@@ -357,8 +363,8 @@ func (s *BuildService) cloneGitContextInternal(ctx context.Context, repositoryURreturn s.gitCloneFn(ctx, repositoryURL, ref, authConfig)}- if s.gitRepository != nil && s.gitRepository.gitClient != nil {- return s.gitRepository.gitClient.Clone(ctx, repositoryURL, ref, authConfig)+ if s.gitRepository != nil && s.gitRepository.Client != nil {+ return s.gitRepository.Clone(ctx, repositoryURL, ref, authConfig)}return "", errors.New("git repository service not available")@@ -371,8 +377,8 @@ func (s *BuildService) cleanupGitContextInternal(repoPath string) error {if s.gitCleanupFn != nil {return s.gitCleanupFn(repoPath)}- if s.gitRepository != nil && s.gitRepository.gitClient != nil {- return s.gitRepository.gitClient.Cleanup(repoPath)+ if s.gitRepository != nil && s.gitRepository.Client != nil {+ return s.gitRepository.Cleanup(repoPath)}return errors.New("git repository service not available")}diff --git a/backend/internal/services/build_service_test.go b/backend/internal/build/service_test.gosimilarity index 95%rename from backend/internal/services/build_service_test.gorename to backend/internal/build/service_test.goindex e4ff84bf5..62a225fc5 100644--- a/backend/internal/services/build_service_test.go+++ b/backend/internal/build/service_test.go@@ -1,4 +1,4 @@-package services+package buildimport ("bytes"@@ -12,7 +12,10 @@ import ("strings""testing"+ "github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitrepo""github.com/getarcaneapp/arcane/backend/v2/internal/models"buildgit "github.com/getarcaneapp/arcane/backend/v2/pkg/gitutil""github.com/libtnb/sqlite"@@ -169,10 +172,16 @@ func TestBuildService_ResolveBuildRequest_RejectsNonGitHTTPContextViaProbeFailur}func TestBuildService_ResolveBuildRequest_UsesSavedGitCredentials(t *testing.T) {- _, db := setupImageServiceAuthTest(t)- require.NoError(t, db.AutoMigrate(&models.GitRepository{}))+ gormDB, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, gormDB.AutoMigrate(&models.GitRepository{}))+ crypto.InitEncryption(&crypto.Config{+ Environment: string(config.AppEnvironmentTest),+ EncryptionKey: "test-encryption-key-for-testing-32bytes-min",+ })+ db := &database.DB{DB: gormDB}- repoService := NewGitRepositoryService(db, t.TempDir(), nil, nil)+ repoService := gitrepo.NewGitRepositoryService(db, t.TempDir(), nil, nil)createTestGitRepository(t, db, models.GitRepository{Name: "private-http",URL: "https://github.com/getarcaneapp/private-build.git",@@ -292,7 +301,7 @@ func TestBuildService_BuildImage_FailureRecordsHistoryAndEvent(t *testing.T) {buildErr := errors.New("BuildKit could not load the image with the docker exporter")svc := &BuildService{db: db,- eventService: NewEventService(db, nil, nil),+ eventService: event.NewEventService(db, nil, nil),builder: testBuildRecorder{err: buildErr,},@@ -344,7 +353,7 @@ func TestBuildService_BuildImage_FailureExporterErrorsAppearInOutputHistoryAndEvsvc := &BuildService{db: db,- eventService: NewEventService(db, nil, nil),+ eventService: event.NewEventService(db, nil, nil),builder: testBuildRecorder{err: buildErr,onProgress: func(_ buildtypes.BuildRequest, w io.Writer) {diff --git a/backend/api/handlers/build_workspaces.go b/backend/internal/build/workspace_handler.gosimilarity index 87%rename from backend/api/handlers/build_workspaces.gorename to backend/internal/build/workspace_handler.goindex f46897ecd..497325a38 100644--- a/backend/api/handlers/build_workspaces.go+++ b/backend/internal/build/workspace_handler.go@@ -1,4 +1,4 @@-package handlers+package buildimport ("context"@@ -7,60 +7,60 @@ import ("path""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"volumetypes "github.com/getarcaneapp/arcane/types/v2/volume")// BuildWorkspaceHandler provides file browsing endpoints for manual build workspaces.type BuildWorkspaceHandler struct {- service *services.BuildWorkspaceService+ service *BuildWorkspaceService}// RegisterBuildWorkspaces registers build workspace file browser routes.-func RegisterBuildWorkspaces(api huma.API, workspaceService *services.BuildWorkspaceService) {+func RegisterBuildWorkspaces(api huma.API, workspaceService *BuildWorkspaceService) {h := &BuildWorkspaceHandler{service: workspaceService}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "builds-browse",Method: "GET",Path: "/environments/{id}/builds/browse",Summary: "Browse build workspace files",Description: "List files and directories under the builds workspace root",Tags: []string{"Builds"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermBuildWorkspacesManage, h.BrowseDirectory)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "builds-browse-content",Method: "GET",Path: "/environments/{id}/builds/browse/content",Summary: "Get build workspace file content",Description: "Read file content under the builds workspace root",Tags: []string{"Builds"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermBuildWorkspacesManage, h.GetFileContent)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "builds-browse-download",Method: "GET",Path: "/environments/{id}/builds/browse/download",Summary: "Download build workspace file",Description: "Download a file from the builds workspace root",Tags: []string{"Builds"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermBuildWorkspacesManage, h.DownloadFile)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "builds-browse-upload",Method: "POST",Path: "/environments/{id}/builds/browse/upload",Summary: "Upload build workspace file",Description: "Upload a file into the builds workspace root",Tags: []string{"Builds"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),RequestBody: &huma.RequestBody{Content: map[string]*huma.MediaType{"multipart/form-data": {@@ -80,24 +80,24 @@ func RegisterBuildWorkspaces(api huma.API, workspaceService *services.BuildWorks},}, authz.PermBuildWorkspacesManage, h.UploadFile)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "builds-browse-mkdir",Method: "POST",Path: "/environments/{id}/builds/browse/mkdir",Summary: "Create build workspace directory",Description: "Create a directory under the builds workspace root",Tags: []string{"Builds"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermBuildWorkspacesManage, h.CreateDirectory)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "builds-browse-delete",Method: "DELETE",Path: "/environments/{id}/builds/browse",Summary: "Delete build workspace file",Description: "Delete a file or directory under the builds workspace root",Tags: []string{"Builds"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermBuildWorkspacesManage, h.DeleteFile)}@@ -186,7 +186,7 @@ func (h *BuildWorkspaceHandler) DownloadFile(ctx context.Context, input *Downloa}func (h *BuildWorkspaceHandler) UploadFile(ctx context.Context, input *UploadBuildFileInput) (*base.ApiResponse[base.MessageResponse], error) {- file, fileHeader, err := openUploadedFileInternal(input.RawBody)+ file, fileHeader, err := handlerutil.OpenUploadedFile(input.RawBody)if err != nil {return nil, err}diff --git a/backend/internal/services/build_workspace_service.go b/backend/internal/build/workspace_service.gosimilarity index 97%rename from backend/internal/services/build_workspace_service.gorename to backend/internal/build/workspace_service.goindex 8b582263d..918b072a8 100644--- a/backend/internal/services/build_workspace_service.go+++ b/backend/internal/build/workspace_service.go@@ -1,4 +1,4 @@-package services+package buildimport ("context"@@ -13,6 +13,7 @@ import ("log/slog"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"volumetypes "github.com/getarcaneapp/arcane/types/v2/volume")@@ -21,10 +22,10 @@ const defaultBuildsDirectory = "/builds"// BuildWorkspaceService provides file operations for the manual build workspace.type BuildWorkspaceService struct {- settings *SettingsService+ settings *settings.SettingsService}-func NewBuildWorkspaceService(settings *SettingsService) *BuildWorkspaceService {+func NewBuildWorkspaceService(settings *settings.SettingsService) *BuildWorkspaceService {return &BuildWorkspaceService{settings: settings}}diff --git a/backend/internal/build/workspace_service_test.go b/backend/internal/build/workspace_service_test.gonew file mode 100644index 000000000..ca32362a9--- /dev/null+++ b/backend/internal/build/workspace_service_test.go@@ -0,0 +1,63 @@+package build++import (+ "context"+ "os"+ "path/filepath"+ "testing"+ "time"++ "github.com/getarcaneapp/arcane/backend/v2/internal/actors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/libtnb/sqlite"+ "github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest"+ "gorm.io/gorm"+)++func TestBuildWorkspaceServiceDeleteFileRejectsCurrentDirectoryInternal(t *testing.T) {+ ctx := context.Background()+ db := setupSettingsTestDB(t)+ settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)+ require.NoError(t, err)++ root := t.TempDir()+ require.NoError(t, settingsService.UpdateSetting(ctx, "buildsDirectory", root))+ sentinel := filepath.Join(root, "keep.txt")+ require.NoError(t, os.WriteFile(sentinel, []byte("keep"), 0o600))++ err = NewBuildWorkspaceService(settingsService).DeleteFile(ctx, ".")+ require.ErrorContains(t, err, "cannot delete root directory")+ require.FileExists(t, sentinel)+}++// Test fixtures shared by this package's tests.++func setupSettingsTestDB(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.SettingVariable{}))+ return &database.DB{DB: db}+}++func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ executor, err := actors.NewExecutor(t.Context(), runtime, "settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, executor.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, executor, effects)+}diff --git a/backend/internal/common/errors.go b/backend/internal/common/errors.goindex e97e0a091..23396a707 100644--- a/backend/internal/common/errors.go+++ b/backend/internal/common/errors.go@@ -55,6 +55,10 @@ func (e *classified) Format(s fmt.State, verb rune) {}var (+ ErrInvalidToken = errors.Sentinel("invalid token")+ ErrExpiredToken = errors.Sentinel("token expired")+ ErrTokenVersionMismatch = errors.Sentinel("token version mismatch")+ ErrUserNotFound = errors.Sentinel("user not found")ErrTokenValidation = Classify(ErrUnauthorized, errors.Sentinel("Invalid token claims"))ErrSessionRevoked = Classify(ErrUnauthorized, errors.Sentinel("Session has been revoked"))ErrUpgradeInProgress = Classify(ErrConflict, errors.Sentinel("an upgrade is already in progress"))diff --git a/backend/internal/common/perms.go b/backend/internal/common/perms.godeleted file mode 100644index 3cb14281d..000000000--- a/backend/internal/common/perms.go+++ /dev/null@@ -1,8 +0,0 @@-package common--import "os"--var (- FilePerm os.FileMode = 0o644- DirPerm os.FileMode = 0o755-)diff --git a/backend/internal/config/config.go b/backend/internal/config/config.goindex 2e987bad8..b56d82c73 100644--- a/backend/internal/config/config.go+++ b/backend/internal/config/config.go@@ -11,7 +11,6 @@ import ("strings""time"- "github.com/getarcaneapp/arcane/backend/v2/internal/common"pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils")@@ -121,8 +120,6 @@ func Load() *Config {applyProxyDefaults(cfg)// Set global file permissions- common.FilePerm = cfg.FilePerm- common.DirPerm = cfg.DirPermpkgutils.FilePerm = cfg.FilePermpkgutils.DirPerm = cfg.DirPermdiff --git a/backend/internal/config/config_test.go b/backend/internal/config/config_test.goindex 412e1557f..8eddfc3e6 100644--- a/backend/internal/config/config_test.go+++ b/backend/internal/config/config_test.go@@ -7,23 +7,23 @@ import ("testing""time"- "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/stretchr/testify/assert""github.com/stretchr/testify/require")func TestConfig_LoadPermissions(t *testing.T) {- // Save original env and common perms+ // Save original env and global permsorigFilePerm := os.Getenv("FILE_PERM")origDirPerm := os.Getenv("DIR_PERM")- origCommonFilePerm := common.FilePerm- origCommonDirPerm := common.DirPerm+ origPkgFilePerm := pkgutils.FilePerm+ origPkgDirPerm := pkgutils.DirPermdefer func() {restoreEnv("FILE_PERM", origFilePerm)restoreEnv("DIR_PERM", origDirPerm)- common.FilePerm = origCommonFilePerm- common.DirPerm = origCommonDirPerm+ pkgutils.FilePerm = origPkgFilePerm+ pkgutils.DirPerm = origPkgDirPerm}()t.Run("Default permissions", func(t *testing.T) {@@ -33,8 +33,8 @@ func TestConfig_LoadPermissions(t *testing.T) {cfg := Load()assert.Equal(t, os.FileMode(0o644), cfg.FilePerm)assert.Equal(t, os.FileMode(0o755), cfg.DirPerm)- assert.Equal(t, os.FileMode(0o644), common.FilePerm)- assert.Equal(t, os.FileMode(0o755), common.DirPerm)+ assert.Equal(t, os.FileMode(0o644), pkgutils.FilePerm)+ assert.Equal(t, os.FileMode(0o755), pkgutils.DirPerm)})t.Run("Custom permissions", func(t *testing.T) {@@ -44,8 +44,8 @@ func TestConfig_LoadPermissions(t *testing.T) {cfg := Load()assert.Equal(t, os.FileMode(0o664), cfg.FilePerm)assert.Equal(t, os.FileMode(0o775), cfg.DirPerm)- assert.Equal(t, os.FileMode(0o664), common.FilePerm)- assert.Equal(t, os.FileMode(0o775), common.DirPerm)+ assert.Equal(t, os.FileMode(0o664), pkgutils.FilePerm)+ assert.Equal(t, os.FileMode(0o775), pkgutils.DirPerm)})t.Run("Restrictive permissions", func(t *testing.T) {@@ -55,8 +55,8 @@ func TestConfig_LoadPermissions(t *testing.T) {cfg := Load()assert.Equal(t, os.FileMode(0o600), cfg.FilePerm)assert.Equal(t, os.FileMode(0o700), cfg.DirPerm)- assert.Equal(t, os.FileMode(0o600), common.FilePerm)- assert.Equal(t, os.FileMode(0o700), common.DirPerm)+ assert.Equal(t, os.FileMode(0o600), pkgutils.FilePerm)+ assert.Equal(t, os.FileMode(0o700), pkgutils.DirPerm)})}diff --git a/backend/internal/configschema/schema.go b/backend/internal/config/schema/schema.gosimilarity index 98%rename from backend/internal/configschema/schema.gorename to backend/internal/config/schema/schema.goindex 224f22933..46dd4cea4 100644--- a/backend/internal/configschema/schema.go+++ b/backend/internal/config/schema/schema.go@@ -1,4 +1,4 @@-package configschema+package schemaimport ("bytes"@@ -19,7 +19,7 @@ import ("emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/utils")@@ -278,7 +278,7 @@ func parseStructEnvFieldsInternal(filename, structName string, opts envFieldOpti}func collectSettingEnvOverridesInternal() ([]SettingOverrideEntry, error) {- defaults := services.DefaultSettingsConfig()+ defaults := settings.DefaultSettingsConfig()settingsType := reflect.TypeFor[models.Settings]()entries := make([]SettingOverrideEntry, 0, settingsType.NumField())@@ -331,7 +331,7 @@ func collectSettingEnvOverridesInternal() ([]SettingOverrideEntry, error) {Public: isPublic,Requires: requires,Note: rule.note,- Source: "models.Settings + services.DefaultSettingsConfig",+ Source: "models.Settings + settings.DefaultSettingsConfig",SourceFile: sourceFileSettings,SourceSymbol: "models.Settings." + field.Name,})diff --git a/backend/internal/configschema/schema_test.go b/backend/internal/config/schema/schema_test.gosimilarity index 99%rename from backend/internal/configschema/schema_test.gorename to backend/internal/config/schema/schema_test.goindex 9bffae1eb..2a78e98a9 100644--- a/backend/internal/configschema/schema_test.go+++ b/backend/internal/config/schema/schema_test.go@@ -1,4 +1,4 @@-package configschema+package schemaimport ("reflect"@@ -64,7 +64,7 @@ func TestGenerate_SettingEnvOverridesMatchModelMetadata(t *testing.T) {assert.Equal(t, "30", dockerTimeout.DefaultValue)assert.Equal(t, "number", dockerTimeout.Type)assert.Equal(t, "timeouts", dockerTimeout.Category)- assert.Equal(t, "models.Settings + services.DefaultSettingsConfig", dockerTimeout.Source)+ assert.Equal(t, "models.Settings + settings.DefaultSettingsConfig", dockerTimeout.Source)oidcSecret, ok := entries["oidcClientSecret"]require.True(t, ok)diff --git a/backend/api/handlers/containers.go b/backend/internal/container/handler.gosimilarity index 91%rename from backend/api/handlers/containers.gorename to backend/internal/container/handler.goindex 4cd831100..c4abd3b71 100644--- a/backend/api/handlers/containers.go+++ b/backend/internal/container/handler.go@@ -1,4 +1,4 @@-package handlers+package containerimport ("context"@@ -11,14 +11,17 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"containertypes "github.com/getarcaneapp/arcane/types/v2/container"dockercontainer "github.com/moby/moby/api/types/container"@@ -27,10 +30,10 @@ import ()type ContainerHandler struct {- containerService *services.ContainerService- dockerService *services.DockerClientService- settingsService *services.SettingsService- activityService *services.ActivityService+ containerService *ContainerService+ dockerService *docker.DockerClientService+ settingsService *settings.SettingsService+ activityService *activity.ActivityServiceappCtx context.Context}@@ -137,149 +140,149 @@ type CommitContainerOutput struct {Body base.ApiResponse[containertypes.CommitResult]}-func RegisterContainers(api huma.API, containerSvc *services.ContainerService, dockerSvc *services.DockerClientService, settingsSvc *services.SettingsService, activitySvc *services.ActivityService, appCtx ActivityAppContext) {+func RegisterContainers(api huma.API, containerSvc *ContainerService, dockerSvc *docker.DockerClientService, settingsSvc *settings.SettingsService, activitySvc *activity.ActivityService, appCtx handlerutil.ActivityAppContext) {h := &ContainerHandler{containerService: containerSvc,dockerService: dockerSvc,settingsService: settingsSvc,activityService: activitySvc,- appCtx: appCtx.contextInternal(),+ appCtx: appCtx.Context(),}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-containers",Method: http.MethodGet,Path: "/environments/{id}/containers",Summary: "List containers",Description: "Paginated list of containers",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersList, h.ListContainers)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "container-status-counts",Method: http.MethodGet,Path: "/environments/{id}/containers/counts",Summary: "Container status counts",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersList, h.GetContainerStatusCounts)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-container",Method: http.MethodPost,Path: "/environments/{id}/containers",Summary: "Create container",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersCreate, h.CreateContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-container",Method: http.MethodGet,Path: "/environments/{id}/containers/{containerId}",Summary: "Get container",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersRead, h.GetContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "start-container",Method: http.MethodPost,Path: "/environments/{id}/containers/{containerId}/start",Summary: "Start container",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersStart, h.StartContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "stop-container",Method: http.MethodPost,Path: "/environments/{id}/containers/{containerId}/stop",Summary: "Stop container",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersStop, h.StopContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "restart-container",Method: http.MethodPost,Path: "/environments/{id}/containers/{containerId}/restart",Summary: "Restart container",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersRestart, h.RestartContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "kill-container",Method: http.MethodPost,Path: "/environments/{id}/containers/{containerId}/kill",Summary: "Kill container",Description: "Send a signal to the container's main process (default SIGKILL)",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersKill, h.KillContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "pause-container",Method: http.MethodPost,Path: "/environments/{id}/containers/{containerId}/pause",Summary: "Pause container",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersPause, h.PauseContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "unpause-container",Method: http.MethodPost,Path: "/environments/{id}/containers/{containerId}/unpause",Summary: "Unpause container",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersPause, h.UnpauseContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "commit-container",Method: http.MethodPost,Path: "/environments/{id}/containers/{containerId}/commit",Summary: "Commit container",Description: "Create an image from a container",Tags: []string{"Containers", "Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesCommit, h.CommitContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "redeploy-container",Method: http.MethodPost,Path: "/environments/{id}/containers/{containerId}/redeploy",Summary: "Redeploy container",Description: "Pull latest image and recreate container",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersRedeploy, h.RedeployContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-container",Method: http.MethodDelete,Path: "/environments/{id}/containers/{containerId}",Summary: "Delete container",Tags: []string{"Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersDelete, h.DeleteContainer)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "set-container-auto-update",Method: http.MethodPut,Path: "/environments/{id}/containers/{containerId}/auto-update",Summary: "Set container auto-update",Description: "Enable or disable auto-update for a specific container",Tags: []string{"Containers", "Updater"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersAutoUpdate, h.SetAutoUpdate)}func (h *ContainerHandler) ListContainers(ctx context.Context, input *ListContainersInput) (*ListContainersOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if input.Updates != "" {params.Filters["updates"] = input.Updates}@@ -298,7 +301,7 @@ func (h *ContainerHandler) ListContainers(ctx context.Context, input *ListContaiData: result.Items,Groups: result.Groups,Counts: result.Counts,- Pagination: toPaginationResponseInternal(result.Pagination),+ Pagination: handlerutil.PaginationResponse(result.Pagination),},}, nil}@@ -544,7 +547,7 @@ func buildNetworkingConfig(body containertypes.Create) *network.NetworkingConfig}func (h *ContainerHandler) CreateContainer(ctx context.Context, input *CreateContainerInput) (*CreateContainerOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -709,7 +712,7 @@ type containerActionConfigInternal struct {}func (h *ContainerHandler) runContainerActionInternal(ctx context.Context, input *ContainerActionInput, cfg containerActionConfigInternal) (*ContainerActionOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -735,7 +738,7 @@ func (h *ContainerHandler) CommitContainer(ctx context.Context, input *CommitConreturn nil, huma.Error400BadRequest("container ID is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -754,7 +757,7 @@ func (h *ContainerHandler) CommitContainer(ctx context.Context, input *CommitCon}func (h *ContainerHandler) RedeployContainer(ctx context.Context, input *ContainerActionInput) (*GetContainerOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -800,7 +803,7 @@ func (h *ContainerHandler) RedeployContainer(ctx context.Context, input *Contain}func (h *ContainerHandler) DeleteContainer(ctx context.Context, input *DeleteContainerInput) (*DeleteContainerOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}diff --git a/backend/internal/container/module.go b/backend/internal/container/module.gonew file mode 100644index 000000000..d679c6465--- /dev/null+++ b/backend/internal/container/module.go@@ -0,0 +1,59 @@+// Package container owns Docker container lifecycle, inspection, listing and+// stats, plus the HTTP surface that exposes them.+package container++import (+ "context"++ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+)++// Dependencies are the collaborators the container domain needs.+type Dependencies struct {+ Event *event.EventService+ Docker *docker.DockerClientService+ Image *image.ImageService+ Settings *settings.SettingsService+ Project *project.ProjectService+ Activity *activity.ActivityService+}++// Module wires the container domain and mounts its routes.+type Module struct {+ service *ContainerService+ deps Dependencies+}++// New builds the container domain from its dependencies.+func New(ctx context.Context, deps Dependencies) *Module {+ return &Module{+ service: NewContainerService(ctx, deps.Event, deps.Docker, deps.Image, deps.Settings, deps.Project),+ deps: deps,+ }+}++// Service exposes the container service to collaborators that use it directly.+func (m *Module) Service() *ContainerService {+ if m == nil {+ return nil+ }+ return m.service+}++// RegisterRoutes mounts the container endpoints. A nil module still registers,+// so OpenAPI spec generation can discover the routes without a service graph.+func (m *Module) RegisterRoutes(api huma.API, appCtx handlerutil.ActivityAppContext) {+ if m == nil {+ RegisterContainers(api, nil, nil, nil, nil, appCtx)+ return+ }+ RegisterContainers(api, m.service, m.deps.Docker, m.deps.Settings, m.deps.Activity, appCtx)+}diff --git a/backend/internal/services/container_service.go b/backend/internal/container/service.gosimilarity index 95%rename from backend/internal/services/container_service.gorename to backend/internal/container/service.goindex 7f56802f0..a2df0363a 100644--- a/backend/internal/services/container_service.go+++ b/backend/internal/container/service.go@@ -1,4 +1,4 @@-package services+package containerimport ("context"@@ -20,8 +20,12 @@ import ("github.com/moby/moby/api/types/network""github.com/moby/moby/client"- "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"@@ -39,12 +43,11 @@ import ()type ContainerService struct {- db *database.DB- dockerService *DockerClientService- eventService *EventService- imageService *ImageService- settingsService *SettingsService- projectService *ProjectService+ dockerService *docker.DockerClientService+ eventService *event.EventService+ imageService *image.ImageService+ settingsService *settings.SettingsService+ projectService *project.ProjectServicestatsHistory containerstats.StoreupdateInfoCache *hot.HotCache[string, *imagetypes.UpdateInfo]iconMetaCache *hot.HotCache[string, projects.ArcaneComposeMetadata]@@ -63,9 +66,8 @@ type ContainerListResult struct {Counts containertypes.StatusCounts}-func NewContainerService(ctx context.Context, db *database.DB, eventService *EventService, dockerService *DockerClientService, imageService *ImageService, settingsService *SettingsService, projectService *ProjectService) *ContainerService {+func NewContainerService(ctx context.Context, eventService *event.EventService, dockerService *docker.DockerClientService, imageService *image.ImageService, settingsService *settings.SettingsService, projectService *project.ProjectService) *ContainerService {svc := &ContainerService{- db: db,eventService: eventService,dockerService: dockerService,imageService: imageService,@@ -160,7 +162,7 @@ func (s *ContainerService) pullRedeployImageInternal(ctx context.Context, dockerpullCtx, pullCancel := context.WithTimeout(ctx, timeouts.GetDuration(settings.DockerImagePullTimeout.AsInt(), timeouts.DefaultDockerImagePull))defer pullCancel()- pullOptions, authErr := s.imageService.getPullOptionsWithAuth(ctx, imageName, nil)+ pullOptions, authErr := s.imageService.PullOptionsWithAuth(ctx, imageName, nil)if authErr != nil {slog.WarnContext(ctx, "failed to get registry authentication for container redeploy pull; proceeding without auth","image", imageName,@@ -170,7 +172,7 @@ func (s *ContainerService) pullRedeployImageInternal(ctx context.Context, docker}reader, pullErr := dockerClient.ImagePull(pullCtx, imageName, pullOptions)- if pullErr != nil && shouldRetryAnonymousPullInternal(pullOptions, pullErr) {+ if pullErr != nil && image.ShouldRetryAnonymousPull(pullOptions, pullErr) {slog.WarnContext(ctx, "container redeploy image pull failed with registry auth; retrying anonymously","image", imageName,"error", pullErr.Error(),@@ -435,7 +437,7 @@ func (s *ContainerService) CommitContainer(ctx context.Context, containerID stri}// tryRedeployViaComposeProjectInternal attempts to redeploy a compose-managed-// container by delegating to ProjectService.UpdateProjectServices, which loads+// container by delegating to project.ProjectService.UpdateProjectServices, which loads// the compose project with full project_directory / env-file / include context// and runs pull/stop/up for just the target service.//@@ -807,7 +809,7 @@ func (s *ContainerService) CreateContainer(ctx context.Context, config *containe_, err = dockerClient.ImageInspect(ctx, config.Image)if err != nil {// Image not found locally, need to pull it- pullOptions, authErr := s.imageService.getPullOptionsWithAuth(ctx, config.Image, credentials)+ pullOptions, authErr := s.imageService.PullOptionsWithAuth(ctx, config.Image, credentials)if authErr != nil {slog.WarnContext(ctx, "Failed to get registry authentication for container image; proceeding without auth","image", config.Image,@@ -971,7 +973,7 @@ func (s *ContainerService) ListContainersPaginated(var dockerContainers []container.Summaryif includeAll {var err error- dockerContainers, err = s.dockerService.listContainersInternal(ctx)+ dockerContainers, err = s.dockerService.ListContainers(ctx)if err != nil {return ContainerListResult{}, err}@@ -988,14 +990,14 @@ func (s *ContainerService) ListContainersPaginated(dockerContainers = containerList.Items}- dockerContainers = filterInternalContainers(dockerContainers, includeInternal)- imageIDs := collectImageIDs(dockerContainers)+ dockerContainers = FilterInternalContainers(dockerContainers, includeInternal)+ imageIDs := CollectImageIDs(dockerContainers)updateInfoMap := s.getUpdateInfoMap(ctx, imageIDs)currentContainerID, currentContainerErr := cgroup.CurrentContainerID()- items := s.buildContainerSummaries(dockerContainers, updateInfoMap, currentContainerID, currentContainerErr)+ items := s.BuildSummaries(dockerContainers, updateInfoMap, currentContainerID, currentContainerErr)config := s.buildContainerPaginationConfig()- counts := s.calculateContainerStatusCounts(items)+ counts := s.CalculateStatusCounts(items)if groupBy == containerGroupByProject {ungroupedParams := params@@ -1009,7 +1011,7 @@ func (s *ContainerService) ListContainersPaginated(// so the flattened items only carry icons applied to the groups first.metadataByProject := map[string]projects.ArcaneComposeMetadata{}for gi := range groups {- s.applyContainerSummaryIconsInternal(ctx, groups[gi].Items, metadataByProject)+ s.ApplySummaryIcons(ctx, groups[gi].Items, metadataByProject)}return ContainerListResult{@@ -1021,8 +1023,8 @@ func (s *ContainerService) ListContainersPaginated(}result := pagination.SearchOrderAndPaginate(items, params, config)- s.applyContainerSummaryIconsInternal(ctx, result.Items, nil)- paginationResp := pagination.BuildResponseFromFilterResult(result, params)+ s.ApplySummaryIcons(ctx, result.Items, nil)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)return ContainerListResult{Items: result.Items,@@ -1039,13 +1041,7 @@ func paginateContainerProjectGroupsInternal(totalCount := len(result.Items)if params.Limit <= 0 {- return groups, pagination.Response{- TotalPages: 1,- TotalItems: int64(totalCount),- CurrentPage: 1,- ItemsPerPage: totalCount,- GrandTotalItems: result.TotalAvailable,- }+ return groups, pagination.BuildResponse(int64(totalCount), result.TotalAvailable, params)}requestedPage := max((params.Start/params.Limit)+1, 1)@@ -1132,7 +1128,7 @@ func getContainerProjectNameInternal(container containertypes.Summary) string {return projectName}-func filterInternalContainers(containers []container.Summary, includeInternal bool) []container.Summary {+func FilterInternalContainers(containers []container.Summary, includeInternal bool) []container.Summary {if includeInternal {return containers}@@ -1147,7 +1143,7 @@ func filterInternalContainers(containers []container.Summary, includeInternal boreturn filtered}-func collectImageIDs(containers []container.Summary) []string {+func CollectImageIDs(containers []container.Summary) []string {imageIDSet := make(map[string]struct{}, len(containers))for _, dc := range containers {if dc.ImageID != "" {@@ -1202,7 +1198,7 @@ func (s *ContainerService) getUpdateInfoMap(ctx context.Context, imageIDs []strireturn updateInfoMap}-func (s *ContainerService) buildContainerSummaries(containers []container.Summary, updateInfoMap map[string]*imagetypes.UpdateInfo, currentContainerID string, currentContainerErr error) []containertypes.Summary {+func (s *ContainerService) BuildSummaries(containers []container.Summary, updateInfoMap map[string]*imagetypes.UpdateInfo, currentContainerID string, currentContainerErr error) []containertypes.Summary {items := make([]containertypes.Summary, 0, len(containers))for _, dc := range containers {summary := containertypes.NewSummary(dc)@@ -1215,10 +1211,10 @@ func (s *ContainerService) buildContainerSummaries(containers []container.Summarreturn items}-// applyContainerSummaryIconsInternal resolves icons for a page of summaries.+// ApplySummaryIcons resolves icons for a page of summaries.// Icon resolution is deferred until after pagination so the cost is bounded by// page size rather than the full container list.-func (s *ContainerService) applyContainerSummaryIconsInternal(ctx context.Context, summaries []containertypes.Summary, metadataByProject map[string]projects.ArcaneComposeMetadata) {+func (s *ContainerService) ApplySummaryIcons(ctx context.Context, summaries []containertypes.Summary, metadataByProject map[string]projects.ArcaneComposeMetadata) {if metadataByProject == nil {metadataByProject = map[string]projects.ArcaneComposeMetadata{}}@@ -1248,18 +1244,18 @@ func (s *ContainerService) applyContainerDetailsIconInternal(ctx context.Contextfunc (s *ContainerService) resolveContainerIconInternal(ctx context.Context, labels map[string]string, metadataByProject map[string]projects.ArcaneComposeMetadata) iconcatalog.ResolvedIconSet {explicitIcon := projects.FindArcaneIconSet(labels)if !explicitIcon.IsEmpty() {- return s.resolveIconSetInternal(ctx, explicitIcon)+ return iconcatalog.Resolve(project.IconCatalogForContext(ctx), explicitIcon)}projectName := dockerutils.ComposeProjectLabel(labels)if projectName == "" || s == nil || s.projectService == nil {- return s.resolveIconSetInternal(ctx, explicitIcon)+ return iconcatalog.Resolve(project.IconCatalogForContext(ctx), explicitIcon)}meta := s.getCachedProjectIconMetadataInternal(ctx, projectName, metadataByProject)serviceName := dockerutils.ComposeServiceLabel(labels)- return s.resolveIconSetInternal(ctx, iconcatalog.FirstNonEmpty(+ return iconcatalog.Resolve(project.IconCatalogForContext(ctx), iconcatalog.FirstNonEmpty(explicitIcon,meta.ServiceIconSets[serviceName],meta.ProjectIcon,@@ -1285,7 +1281,7 @@ func (s *ContainerService) getCachedProjectIconMetadataInternal(ctx context.Contmeta := projects.ArcaneComposeMetadata{ServiceIconSets: map[string]projects.IconSet{}}proj, err := s.projectService.GetProjectByComposeName(ctx, projectName)if err == nil && proj != nil {- meta = s.projectService.getProjectMetadataForProject(ctx, *proj, nil)+ meta = s.projectService.ProjectMetadata(ctx, *proj, nil)}if s.iconMetaCache != nil {s.iconMetaCache.Set(projectName, meta)@@ -1296,10 +1292,6 @@ func (s *ContainerService) getCachedProjectIconMetadataInternal(ctx context.Contreturn meta}-func (s *ContainerService) resolveIconSetInternal(ctx context.Context, iconSet iconcatalog.IconSet) iconcatalog.ResolvedIconSet {- return iconcatalog.Resolve(iconCatalogForContextInternal(ctx), iconSet)-}-func (s *ContainerService) buildContainerPaginationConfig() pagination.Config[containertypes.Summary] {return pagination.Config[containertypes.Summary]{SearchAccessors: []pagination.SearchAccessor[containertypes.Summary]{@@ -1484,7 +1476,7 @@ func (s *ContainerService) buildContainerFilterAccessors() []pagination.FilterAc}}-func (s *ContainerService) calculateContainerStatusCounts(items []containertypes.Summary) containertypes.StatusCounts {+func (s *ContainerService) CalculateStatusCounts(items []containertypes.Summary) containertypes.StatusCounts {counts := containertypes.StatusCounts{TotalContainers: len(items),}diff --git a/backend/internal/services/container_service_test.go b/backend/internal/container/service_test.gosimilarity index 84%rename from backend/internal/services/container_service_test.gorename to backend/internal/container/service_test.goindex 81de396ba..f31bfb494 100644--- a/backend/internal/services/container_service_test.go+++ b/backend/internal/container/service_test.go@@ -1,4 +1,4 @@-package services+package containerimport ("context"@@ -6,16 +6,23 @@ import ("net/http""net/http/httptest""net/netip"+ "regexp""testing"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/backend/v2/pkg/projects"containertypes "github.com/getarcaneapp/arcane/types/v2/container"imagetypes "github.com/getarcaneapp/arcane/types/v2/image"+ "github.com/libtnb/sqlite""github.com/moby/moby/api/types/container""github.com/moby/moby/api/types/network"+ "github.com/moby/moby/client""github.com/stretchr/testify/require"+ "gorm.io/gorm")func TestPaginateContainerProjectGroupsKeepsProjectWhole(t *testing.T) {@@ -109,7 +116,7 @@ func TestContainerSummaryIconsAppliedAfterGrouping(t *testing.T) {}},}- items := service.buildContainerSummaries(dockerContainers, nil, "", nil)+ items := service.BuildSummaries(dockerContainers, nil, "", nil)for _, item := range items {require.Empty(t, item.IconLightURL, "icons must be deferred until after pagination")}@@ -119,7 +126,7 @@ func TestContainerSummaryIconsAppliedAfterGrouping(t *testing.T) {pagination.QueryParams{Params: pagination.Params{Start: 0, Limit: 20}},)for gi := range groups {- service.applyContainerSummaryIconsInternal(t.Context(), groups[gi].Items, map[string]projects.ArcaneComposeMetadata{})+ service.ApplySummaryIcons(t.Context(), groups[gi].Items, map[string]projects.ArcaneComposeMetadata{})}flattened := flattenContainerProjectGroupsInternal(groups)@@ -214,7 +221,7 @@ func TestCompareContainerPortsForSortDesc_KeepsContainersWithoutPortsLast(t *tes}func TestContainerServiceCommitContainerCallsDockerAPIInternal(t *testing.T) {- db := setupProjectTestDB(t)+ db := setupProjectTestDBInternal(t)var gotRequest map[string]anyserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {if r.Method != http.MethodPost || dockerTestPathInternal(r.URL.Path) != "/commit" {@@ -234,9 +241,8 @@ func TestContainerServiceCommitContainerCallsDockerAPIInternal(t *testing.T) {svc := NewContainerService(context.Background(),- db,- NewEventService(db, nil, nil),- &DockerClientService{client: newTestDockerClient(t, server)},+ event.NewEventService(db, nil, nil),+ docker.NewDockerClientService(t.Context(), nil, nil, nil).WithClient(newTestDockerClientInternal(t, server)),nil,nil,nil,@@ -247,7 +253,7 @@ func TestContainerServiceCommitContainerCallsDockerAPIInternal(t *testing.T) {Tag: "snapshot",Comment: "manual snapshot",Author: "arcane",- }, systemUser)+ }, models.SystemUser)require.NoError(t, err)require.NotNil(t, out)require.Equal(t, "sha256:new-image", out.ID)@@ -265,7 +271,7 @@ func TestContainerServiceCommitContainerCallsDockerAPIInternal(t *testing.T) {}func TestContainerServiceCommitContainerOmitsReferenceWhenRepositoryEmptyInternal(t *testing.T) {- db := setupProjectTestDB(t)+ db := setupProjectTestDBInternal(t)var gotRequest map[string]anyserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {if r.Method != http.MethodPost || dockerTestPathInternal(r.URL.Path) != "/commit" {@@ -283,9 +289,8 @@ func TestContainerServiceCommitContainerOmitsReferenceWhenRepositoryEmptyInternasvc := NewContainerService(context.Background(),- db,- NewEventService(db, nil, nil),- &DockerClientService{client: newTestDockerClient(t, server)},+ event.NewEventService(db, nil, nil),+ docker.NewDockerClientService(t.Context(), nil, nil, nil).WithClient(newTestDockerClientInternal(t, server)),nil,nil,nil,@@ -293,7 +298,7 @@ func TestContainerServiceCommitContainerOmitsReferenceWhenRepositoryEmptyInternaout, err := svc.CommitContainer(context.Background(), "container-1", containertypes.CommitRequest{Tag: "latest",- }, systemUser)+ }, models.SystemUser)require.NoError(t, err)require.NotNil(t, out)require.Equal(t, "sha256:new-image", out.ID)@@ -317,3 +322,41 @@ func newGroupedContainerSummary(name string, project string) containertypes.SummState: "running",}}++// Test fixtures shared by this package's tests.++// dockerTestPathInternal strips the /v1.NN version prefix Docker clients prepend, so+// fake Docker servers can match on the bare path.+func dockerTestPathInternal(path string) string {+ return dockerAPIVersionPrefixInternal.ReplaceAllString(path, "")+}++// newTestDockerClientInternal builds a Docker client pointed at a fake Docker HTTP server.+func newTestDockerClientInternal(t *testing.T, server *httptest.Server) *client.Client {+ t.Helper()++ httpClient := server.Client()+ cli, err := client.New(+ client.WithHost(server.URL),+ client.WithAPIVersion("1.41"),+ client.WithHTTPClient(httpClient),+ )+ require.NoError(t, err)++ t.Cleanup(func() {+ _ = cli.Close()+ })++ return cli+}++// setupProjectTestDBInternal builds an in-memory DB migrated for project-related tests.+func setupProjectTestDBInternal(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.Project{}, &models.SettingVariable{}, &models.ImageUpdateRecord{}, &models.Event{}))+ return &database.DB{DB: db}+}++var dockerAPIVersionPrefixInternal = regexp.MustCompile(`^/v[0-9]+\.[0-9]+`)diff --git a/backend/api/handlers/dashboard.go b/backend/internal/dashboard/handler.gosimilarity index 89%rename from backend/api/handlers/dashboard.gorename to backend/internal/dashboard/handler.goindex 70c971892..8ae9cab55 100644--- a/backend/api/handlers/dashboard.go+++ b/backend/internal/dashboard/handler.go@@ -1,4 +1,4 @@-package handlers+package dashboardimport ("context"@@ -9,12 +9,13 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/remenv"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"containertypes "github.com/getarcaneapp/arcane/types/v2/container"dashboardtypes "github.com/getarcaneapp/arcane/types/v2/dashboard"@@ -25,8 +26,8 @@ import ()type DashboardHandler struct {- dashboardService *services.DashboardService- environmentService *services.EnvironmentService+ dashboardService *DashboardService+ environmentService *environment.EnvironmentService}type GetDashboardInput struct {@@ -47,20 +48,26 @@ const (dashboardStreamEventBuffer = 64)-func RegisterDashboard(api huma.API, dashboardService *services.DashboardService, environmentService *services.EnvironmentService) {- h := &DashboardHandler{+// NewHandler builds the dashboard HTTP handler. Cross-domain handlers (the+// multiplexed client stream) compose it rather than reaching into its fields.+func NewHandler(dashboardService *DashboardService, environmentService *environment.EnvironmentService) *DashboardHandler {+ return &DashboardHandler{dashboardService: dashboardService,environmentService: environmentService,}+}++func RegisterDashboard(api huma.API, dashboardService *DashboardService, environmentService *environment.EnvironmentService) {+ h := NewHandler(dashboardService, environmentService)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-dashboard",Method: http.MethodGet,Path: "/environments/{id}/dashboard",Summary: "Get dashboard snapshot",Description: "Returns the dashboard first-paint snapshot in a single response",Tags: []string{"Dashboard"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermDashboardRead, h.GetDashboard)}@@ -68,7 +75,7 @@ func (h *DashboardHandler) GetDashboard(ctx context.Context, input *GetDashboard// EnvironmentID is consumed by env proxy/auth middleware for routing/validation._ = input.EnvironmentID- snapshot, err := h.dashboardService.GetSnapshot(ctx, services.DashboardActionItemsOptions{+ snapshot, err := h.dashboardService.GetSnapshot(ctx, DashboardActionItemsOptions{DebugAllGood: input.DebugAllGood,})if err != nil {@@ -100,11 +107,11 @@ func trimDashboardStreamSnapshotInternal(snapshot *dashboardtypes.Snapshot) *dasreturn snapshot}-func (h *DashboardHandler) runLocalDashboardStreamProducerInternal(ctx context.Context, debugAllGood bool, events chan<- dashboardtypes.StreamEvent) {+func (h *DashboardHandler) RunLocalStreamProducer(ctx context.Context, debugAllGood bool, events chan<- dashboardtypes.StreamEvent) {lastError := ""poll := func() {- snapshot, err := h.dashboardService.GetSnapshot(ctx, services.DashboardActionItemsOptions{+ snapshot, err := h.dashboardService.GetSnapshot(ctx, DashboardActionItemsOptions{DebugAllGood: debugAllGood,})if err == nil && snapshot == nil {@@ -151,10 +158,10 @@ func (h *DashboardHandler) runLocalDashboardStreamProducerInternal(ctx context.C}}-// runRemoteDashboardStreamPollersInternal keeps one poller goroutine per+// RunRemoteStreamPollers keeps one poller goroutine per// enabled remote environment, re-listing periodically so environments added// or removed while the stream is open are picked up without a reconnect.-func (h *DashboardHandler) runRemoteDashboardStreamPollersInternal(ctx context.Context, ps *authz.PermissionSet, debugAllGood bool, events chan<- dashboardtypes.StreamEvent) {+func (h *DashboardHandler) RunRemoteStreamPollers(ctx context.Context, ps *authz.PermissionSet, debugAllGood bool, events chan<- dashboardtypes.StreamEvent) {agg.ReconcilePollersByKey(ctx,func(ctx context.Context) ([]models.Environment, error) {environments, err := h.environmentService.ListRemoteEnvironments(ctx)diff --git a/backend/internal/dashboard/module.go b/backend/internal/dashboard/module.gonew file mode 100644index 000000000..97a48a206--- /dev/null+++ b/backend/internal/dashboard/module.go@@ -0,0 +1,84 @@+// Package dashboard owns the dashboard snapshot: the service that assembles it+// from the container, project, image, volume and vulnerability domains, and the+// HTTP surface that serves it.+package dashboard++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/container"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/version"+ "github.com/getarcaneapp/arcane/backend/v2/internal/volume"+ "github.com/getarcaneapp/arcane/backend/v2/internal/vulnerability"+)++// Dependencies are the collaborators the dashboard domain needs.+type Dependencies struct {+ DB *database.DB+ Docker *docker.DockerClientService+ Container *container.ContainerService+ Project *project.ProjectService+ Image *image.ImageService+ Settings *settings.SettingsService+ Vulnerability *vulnerability.VulnerabilityService+ Environment *environment.EnvironmentService+ Version *version.VersionService+ Volume *volume.VolumeService+}++// Module is the dashboard domain's wiring seam: it owns the service and the+// handler, and mounts the domain's routes.+type Module struct {+ service *DashboardService+ handler *DashboardHandler+}++// New builds the dashboard domain from its dependencies.+func New(deps Dependencies) *Module {+ service := NewDashboardService(+ deps.DB,+ deps.Docker,+ deps.Container,+ deps.Project,+ deps.Image,+ deps.Settings,+ deps.Vulnerability,+ deps.Environment,+ deps.Version,+ deps.Volume,+ )+ return &Module{service: service, handler: NewHandler(service, deps.Environment)}+}++// Handler exposes the dashboard stream producer.+func (m *Module) Handler() *DashboardHandler {+ if m == nil {+ return nil+ }+ return m.handler+}++// Service exposes the dashboard service to collaborators that compose its+// producers, such as the multiplexed client stream.+func (m *Module) Service() *DashboardService {+ if m == nil {+ return nil+ }+ return m.service+}++// RegisterRoutes mounts the dashboard endpoints. A nil module still registers,+// so OpenAPI spec generation can discover the routes without a service graph.+func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterDashboard(api, nil, nil)+ return+ }+ RegisterDashboard(api, m.service, m.handler.environmentService)+}diff --git a/backend/internal/services/dashboard_service.go b/backend/internal/dashboard/service.gosimilarity index 81%rename from backend/internal/services/dashboard_service.gorename to backend/internal/dashboard/service.goindex 9affe1eed..756cb1ad5 100644--- a/backend/internal/services/dashboard_service.go+++ b/backend/internal/dashboard/service.go@@ -1,4 +1,4 @@-package services+package dashboardimport ("context"@@ -10,8 +10,17 @@ import (dockercontainer "github.com/moby/moby/api/types/container""golang.org/x/sync/errgroup"+ "github.com/getarcaneapp/arcane/backend/v2/internal/container""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/version"+ "github.com/getarcaneapp/arcane/backend/v2/internal/volume"+ "github.com/getarcaneapp/arcane/backend/v2/internal/vulnerability"dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/types/v2/base"@@ -24,21 +33,22 @@ import ("go.getarcane.app/updater/labels")-const defaultDashboardAPIKeyExpiryWindow = 14 * 24 * time.Hour-const dashboardSnapshotPreloadLimit = 50-const localEnvironmentID = "0"+const (+ defaultDashboardAPIKeyExpiryWindow = 14 * 24 * time.Hour+ dashboardSnapshotPreloadLimit = 50+)type DashboardService struct {db *database.DB- dockerService *DockerClientService- containerService *ContainerService- projectService *ProjectService- imageService *ImageService- settingsService *SettingsService- vulnerabilityService *VulnerabilityService- environmentService *EnvironmentService- versionService *VersionService- volumeService *VolumeService+ dockerService *docker.DockerClientService+ containerService *container.ContainerService+ projectService *project.ProjectService+ imageService *image.ImageService+ settingsService *settings.SettingsService+ vulnerabilityService *vulnerability.VulnerabilityService+ environmentService *environment.EnvironmentService+ versionService *version.VersionService+ volumeService *volume.VolumeService}type DashboardActionItemsOptions struct {@@ -47,15 +57,15 @@ type DashboardActionItemsOptions struct {func NewDashboardService(db *database.DB,- dockerService *DockerClientService,- containerService *ContainerService,- projectService *ProjectService,- imageService *ImageService,- settingsService *SettingsService,- vulnerabilityService *VulnerabilityService,- environmentService *EnvironmentService,- versionService *VersionService,- volumeService *VolumeService,+ dockerService *docker.DockerClientService,+ containerService *container.ContainerService,+ projectService *project.ProjectService,+ imageService *image.ImageService,+ settingsService *settings.SettingsService,+ vulnerabilityService *vulnerability.VulnerabilityService,+ environmentService *environment.EnvironmentService,+ versionService *version.VersionService,+ volumeService *volume.VolumeService,) *DashboardService {return &DashboardService{db: db,@@ -76,18 +86,18 @@ func (s *DashboardService) GetSnapshot(ctx context.Context, options DashboardActreturn nil, errors.New("docker service not available")}- dockerSnapshot, err := s.dockerService.GetSnapshot(ctx, localEnvironmentID)+ dockerSnapshot, err := s.dockerService.GetSnapshot(ctx, environment.LocalEnvironmentID)if err != nil {return nil, err}dockerContainers := dockerSnapshot.ContainersdockerImages := dockerSnapshot.Images- filteredContainers := filterInternalContainers(dockerContainers, false)+ filteredContainers := container.FilterInternalContainers(dockerContainers, false)containerItems := make([]containertypes.Summary, 0, len(filteredContainers))currentContainerID, currentContainerErr := cgroup.CurrentContainerID()if s.containerService != nil {- containerItems = s.containerService.buildContainerSummaries(filteredContainers, nil, currentContainerID, currentContainerErr)+ containerItems = s.containerService.BuildSummaries(filteredContainers, nil, currentContainerID, currentContainerErr)} else {for _, container := range filteredContainers {summary := containertypes.NewSummary(container)@@ -98,7 +108,7 @@ func (s *DashboardService) GetSnapshot(ctx context.Context, options DashboardActcontainerCounts := containertypes.StatusCounts{TotalContainers: len(containerItems)}if s.containerService != nil {- containerCounts = s.containerService.calculateContainerStatusCounts(containerItems)+ containerCounts = s.containerService.CalculateStatusCounts(containerItems)} else {for _, item := range containerItems {if item.State == "running" {@@ -117,7 +127,7 @@ func (s *DashboardService) GetSnapshot(ctx context.Context, options DashboardAct})containerPage := limitDashboardItemsInternal(containerItems, dashboardSnapshotPreloadLimit)if s.containerService != nil {- s.containerService.applyContainerSummaryIconsInternal(ctx, containerPage, nil)+ s.containerService.ApplySummaryIcons(ctx, containerPage, nil)}var projectIDByName map[string]string@@ -126,8 +136,8 @@ func (s *DashboardService) GetSnapshot(ctx context.Context, options DashboardAct} else {projectIDByName = map[string]string{}}- imageUsageMap := buildUsageMapInternal(filteredContainers, projectIDByName)- imageItems := mapDockerImagesToDTOs(dockerImages, imageUsageMap, nil, nil)+ imageUsageMap := image.BuildVolumeUsageMap(filteredContainers, projectIDByName)+ imageItems := image.MapDockerImagesToDTOs(dockerImages, imageUsageMap, nil, nil)sort.Slice(imageItems, func(i, j int) bool {if imageItems[i].Size == imageItems[j].Size {return imageItems[i].ID < imageItems[j].ID@@ -137,7 +147,7 @@ func (s *DashboardService) GetSnapshot(ctx context.Context, options DashboardActimagePage := limitDashboardItemsInternal(imageItems, dashboardSnapshotPreloadLimit)imageUsageCounts := imagetypes.UsageCounts{}- imageUsageCounts.Inuse, imageUsageCounts.Unused, imageUsageCounts.Total = countImageUsageInternal(dockerImages, filteredContainers)+ imageUsageCounts.Inuse, imageUsageCounts.Unused, imageUsageCounts.Total = docker.CountImageUsage(dockerImages, filteredContainers)for _, img := range dockerImages {imageUsageCounts.TotalSize += img.Size}@@ -146,7 +156,7 @@ func (s *DashboardService) GetSnapshot(ctx context.Context, options DashboardAct// container still counts as in use, matching the volumes page.var volumeUsageCounts *volumetypes.UsageCountsif s.volumeService != nil && dockerSnapshot.Volumes != nil {- counts := s.volumeService.countVolumeUsageFromSnapshotInternal(dockerSnapshot.Volumes.Items, dockerContainers)+ counts := s.volumeService.CountUsageFromSnapshot(dockerSnapshot.Volumes.Items, dockerContainers)volumeUsageCounts = &counts}@@ -298,9 +308,9 @@ func (s *DashboardService) getPendingResourceUpdatesCountInternal(ctx context.Coreturn 0, nil}- filteredContainers := filterInternalContainers(allContainers, false)+ filteredContainers := container.FilterInternalContainers(allContainers, false)standaloneContainers := filterStandaloneDockerContainersInternal(filteredContainers)- containerCount, err := s.getPendingContainerUpdatesCountForImageIDsInternal(ctx, collectImageIDs(standaloneContainers))+ containerCount, err := s.getPendingContainerUpdatesCountForImageIDsInternal(ctx, container.CollectImageIDs(standaloneContainers))if err != nil {return 0, err}@@ -346,7 +356,7 @@ func (s *DashboardService) getPendingProjectUpdatesCountInternal(ctx context.Conreturn 0, nil}- count, err := s.projectService.countProjectsWithPendingUpdatesInternal(ctx, allContainers)+ count, err := s.projectService.CountProjectsWithPendingUpdates(ctx, allContainers)if err != nil {return 0, errors.WrapIf(err, "failed to count projects with updates")}@@ -358,7 +368,7 @@ func (s *DashboardService) getActionableVulnerabilitiesCountInternal(ctx contextreturn 0, nil}- return s.vulnerabilityService.getActionableCountExcludingIgnoredInternal(ctx)+ return s.vulnerabilityService.ActionableCountExcludingIgnored(ctx)}func (s *DashboardService) getExpiringAPIKeysCountInternal(ctx context.Context) (int, error) {diff --git a/backend/internal/services/dashboard_service_test.go b/backend/internal/dashboard/service_test.gosimilarity index 72%rename from backend/internal/services/dashboard_service_test.gorename to backend/internal/dashboard/service_test.goindex 604b89c65..1a2a0515f 100644--- a/backend/internal/services/dashboard_service_test.go+++ b/backend/internal/dashboard/service_test.go@@ -1,4 +1,4 @@-package services+package dashboardimport ("context"@@ -11,9 +11,15 @@ import ("testing""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/volume"dashboardtypes "github.com/getarcaneapp/arcane/types/v2/dashboard"volumetypes "github.com/getarcaneapp/arcane/types/v2/volume""github.com/libtnb/sqlite"@@ -24,10 +30,11 @@ import ("github.com/moby/moby/client""github.com/stretchr/testify/assert""github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest""gorm.io/gorm")-func setupDashboardServiceTestDB(t *testing.T) (*database.DB, *SettingsService) {+func setupDashboardServiceTestDB(t *testing.T) (*database.DB, *settings.SettingsService) {t.Helper()db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{})@@ -35,7 +42,7 @@ func setupDashboardServiceTestDB(t *testing.T) (*database.DB, *SettingsService)require.NoError(t, db.AutoMigrate(&models.ApiKey{}, &models.Environment{}, &models.ImageUpdateRecord{}, &models.Project{}, &models.SettingVariable{}))databaseDB := &database.DB{DB: db}- settingsSvc, err := newSettingsServiceForTestInternal(t, context.Background(), databaseDB)+ settingsSvc, err := newSettingsServiceForTestInternal(context.Background(), t, databaseDB)require.NoError(t, err)return databaseDB, settingsSvc@@ -53,11 +60,11 @@ func createDashboardTestImageUpdateRecord(t *testing.T, db *database.DB, recordfunc newDashboardTestDockerService(t *testing.T,- settingsSvc *SettingsService,+ settingsSvc *settings.SettingsService,containers []dockercontainer.Summary,images []dockerimage.Summary,volumes []dockervolume.Volume,-) *DockerClientService {+) *docker.DockerClientService {t.Helper()server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {@@ -91,10 +98,7 @@ func newDashboardTestDockerService(_ = dockerClient.Close()})- return &DockerClientService{- client: dockerClient,- settingsService: settingsSvc,- }+ return docker.NewDockerClientService(t.Context(), nil, nil, settingsSvc).WithClient(dockerClient)}func TestDashboardService_GetSnapshot_ReturnsDashboardSnapshot(t *testing.T) {@@ -167,7 +171,7 @@ func TestDashboardService_GetSnapshot_ReturnsDashboardSnapshot(t *testing.T) {projectsDir := t.TempDir()t.Setenv("PROJECTS_DIRECTORY", projectsDir)require.NoError(t, settingsSvc.SetStringSetting(context.Background(), "projectsDirectory", projectsDir))- projectPath := createComposeProjectDir(t, projectsDir, "project-with-update")+ projectPath := createComposeProjectDirInternal(t, projectsDir, "project-with-update")require.NoError(t, os.WriteFile(filepath.Join(projectPath, "compose.yaml"), []byte("services:\n app:\n image: repo/worker:latest\n"), 0o644))dirName := "project-with-update"require.NoError(t, db.WithContext(context.Background()).Create(&models.Project{@@ -177,8 +181,8 @@ func TestDashboardService_GetSnapshot_ReturnsDashboardSnapshot(t *testing.T) {Path: projectPath,Status: models.ProjectStatusStopped,}).Error)- projectSvc := NewProjectService(db, settingsSvc, nil, &ImageService{db: db}, nil, nil, nil, nil, config.Load())- svc := NewDashboardService(db, dockerSvc, nil, projectSvc, nil, settingsSvc, nil, nil, nil, &VolumeService{})+ projectSvc := project.NewProjectService(db, settingsSvc, nil, image.NewImageService(db, nil, nil, nil, nil, nil), nil, nil, nil, nil, config.Load())+ svc := NewDashboardService(db, dockerSvc, nil, projectSvc, nil, settingsSvc, nil, nil, nil, volume.NewVolumeService(db, nil, nil, nil, nil, ""))snapshot, err := svc.GetSnapshot(context.Background(), DashboardActionItemsOptions{})require.NoError(t, err)@@ -244,3 +248,56 @@ func TestDashboardService_GetSnapshot_DebugAllGoodOnlyClearsActionItems(t *testirequire.Nil(t, snapshot.VolumeUsageCounts)require.Empty(t, snapshot.ActionItems.Items)}++// Test fixtures shared by this package's tests.++// createComposeProjectDirInternal writes a minimal single-service compose project under+// root and returns its path.+func createComposeProjectDirInternal(t *testing.T, root, name string) string {+ t.Helper()++ projectPath := filepath.Join(root, name)+ require.NoError(t, os.MkdirAll(projectPath, 0o755))+ require.NoError(t, os.WriteFile(filepath.Join(projectPath, "compose.yaml"), []byte("services:\n app:\n image: nginx:alpine\n"), 0o600))++ return projectPath+}++// createTestRemoteEnvironmentInternal inserts an enabled, online remote environment.+func createTestRemoteEnvironmentInternal(t *testing.T, db *database.DB, environmentID, name, apiURL, token string) {+ t.Helper()+ now := time.Now()+ require.NoError(t, db.Create(&models.Environment{+ BaseModel: models.BaseModel{+ ID: environmentID,+ CreatedAt: now,+ UpdatedAt: &now,+ },+ Name: name,+ ApiUrl: apiURL,+ Status: string(models.EnvironmentStatusOnline),+ Enabled: true,+ AccessToken: &token,+ }).Error)+}++// newSettingsServiceForTestInternal builds a SettingsService backed by its own actor runtime,+// stopped when the test ends.+func newSettingsServiceForTestInternal(ctx context.Context, t testing.TB, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(ctx, lifecycle)+ require.NoError(t, err)+ executor, err := actors.NewExecutor(ctx, runtime, "settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(ctx, runtime, "settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() { //nolint:contextcheck // cleanup must outlive ctx to stop the runtime+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, executor.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, executor, effects)+}diff --git a/backend/internal/di/di.go b/backend/internal/di/di.goindex d46834aba..5b81170b2 100644--- a/backend/internal/di/di.go+++ b/backend/internal/di/di.go@@ -3,9 +3,29 @@ package diimport ("github.com/getarcaneapp/arcane/backend/v2/internal/actors"- arcanelogging "github.com/getarcaneapp/arcane/backend/v2/internal/logging"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/appimages"+ "github.com/getarcaneapp/arcane/backend/v2/internal/build"+ "github.com/getarcaneapp/arcane/backend/v2/internal/diagnostics"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitops"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv"+ "github.com/getarcaneapp/arcane/backend/v2/internal/lifecycle"+ "github.com/getarcaneapp/arcane/backend/v2/internal/network"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/oidc"+ "github.com/getarcaneapp/arcane/backend/v2/internal/passkey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/port"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/search"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session"+ "github.com/getarcaneapp/arcane/backend/v2/internal/swarm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/system"+ "github.com/getarcaneapp/arcane/backend/v2/internal/variable"+ "github.com/getarcaneapp/arcane/backend/v2/internal/vulnerability""github.com/getarcaneapp/arcane/backend/v2/pkg/scheduler"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/logging""go.uber.org/fx")@@ -18,55 +38,74 @@ var ActorOptions = fx.Options(// ServiceOptions provides the backend service graph.var ServiceOptions = fx.Options(+ fx.Supply(new(logging.SlogErrorHandler)),fx.Provide(// Infrastructure values consumed by services.provideResourcesFSInternal,- arcanelogging.NewSlogErrorHandler,// Services constructed directly through their public constructors.- services.NewEventService,- services.NewActivityService,+ provideEventModuleInternal,+ provideEventServiceInternal,+ provideActivityModuleInternal,+ provideActivityServiceInternal,provideSettingsServiceInternal,- services.NewKVService,- services.NewJobService,- services.NewSettingsSearchService,- services.NewCustomizeSearchService,- services.NewApplicationImagesService,+ kv.NewKVService,+ provideJobModuleInternal,+ provideJobServiceInternal,+ search.New,+ appimages.NewApplicationImagesService,provideDockerClientServiceInternal,- services.NewRoleService,- services.NewSessionService,- services.NewPasskeyService,- services.NewEnvironmentService,- services.NewNotificationService,- services.NewVulnerabilityService,- services.NewImageUpdateService,- services.NewImageService,- services.NewBuildService,- services.NewBuildWorkspaceService,- services.NewLifecycleService,+ provideRoleModuleInternal,+ provideRoleServiceInternal,+ session.NewSessionService,+ passkey.NewPasskeyService,+ environment.NewEnvironmentService,+ provideEnvironmentModuleInternal,+ provideSettingsModuleInternal,+ notification.NewNotificationService,+ notification.New,+ vulnerability.NewVulnerabilityService,+ vulnerability.New,+ imageupdate.NewImageUpdateService,+ image.NewImageService,+ provideImageUpdateModuleInternal,+ provideImageModuleInternal,+ build.NewBuildService,+ build.NewBuildWorkspaceService,+ lifecycle.NewLifecycleService,provideProjectServiceInternal,- services.NewContainerService,- services.NewDashboardService,- services.NewNetworkService,- services.NewPortService,- services.NewSwarmService,- services.NewTemplateService,- services.NewOidcService,- services.NewSystemService,- services.NewSystemUpgradeService,- services.NewDiagnosticsService,- services.NewGitOpsSyncService,- services.NewWebhookService,- services.NewVariableService,+ project.New,+ provideContainerModuleInternal,+ provideDashboardModuleInternal,+ network.NewNetworkService,+ port.NewPortService,+ swarm.NewSwarmService,+ provideSwarmModuleInternal,+ provideTemplateModuleInternal,+ provideTemplateServiceInternal,+ oidc.NewOidcService,+ provideSystemModuleInternal,+ system.NewSystemUpgradeService,+ diagnostics.NewDiagnosticsService,+ gitops.NewGitOpsSyncService,+ gitops.New,+ provideWebhookModuleInternal,+ variable.NewVariableService,+ variable.New,// Adapters for scalar config fields, unexported parameters, builders, and lifecycle hooks.provideVersionServiceInternal,+ provideGitRepositoryModuleInternal,provideGitRepositoryServiceInternal,- provideVolumeServiceInternal,+ provideVolumeModuleInternal,provideAuthServiceInternal,+ provideAuthModuleInternal,+ provideContainerRegistryModuleInternal,provideContainerRegistryServiceInternal,- provideUpdaterServiceInternal,+ provideUpdaterModuleInternal,provideUserServiceInternal,+ provideUserModuleInternal,+ provideApiKeyModuleInternal,provideApiKeyServiceInternal,provideFederatedCredentialServiceInternal,provideAuthMiddlewareInternal,diff --git a/backend/internal/di/di_test.go b/backend/internal/di/di_test.goindex 10f3153b1..4031d84c6 100644--- a/backend/internal/di/di_test.go+++ b/backend/internal/di/di_test.go@@ -5,10 +5,48 @@ import ("net/http""testing"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/federated"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitops"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitrepo"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/job"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv"+ "github.com/getarcaneapp/arcane/backend/v2/internal/lifecycle"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/swarm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/template"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user"+ "github.com/getarcaneapp/arcane/backend/v2/internal/variable"+ "github.com/getarcaneapp/arcane/backend/v2/internal/vulnerability"++ "github.com/getarcaneapp/arcane/backend/v2/internal/appimages"+ "github.com/getarcaneapp/arcane/backend/v2/internal/build""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/container"+ "github.com/getarcaneapp/arcane/backend/v2/internal/dashboard""github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/diagnostics"+ "github.com/getarcaneapp/arcane/backend/v2/internal/network"+ "github.com/getarcaneapp/arcane/backend/v2/internal/oidc"+ "github.com/getarcaneapp/arcane/backend/v2/internal/port"+ "github.com/getarcaneapp/arcane/backend/v2/internal/search"+ "github.com/getarcaneapp/arcane/backend/v2/internal/system"+ "github.com/getarcaneapp/arcane/backend/v2/internal/updater"+ "github.com/getarcaneapp/arcane/backend/v2/internal/version"+ "github.com/getarcaneapp/arcane/backend/v2/internal/volume"+ "github.com/getarcaneapp/arcane/backend/v2/internal/webhook""github.com/getarcaneapp/arcane/backend/v2/pkg/scheduler""github.com/stretchr/testify/require""go.uber.org/fx"@@ -17,49 +55,48 @@ import (type graphParams struct {fx.In- AppImages *services.ApplicationImagesService- User *services.UserService- Project *services.ProjectService- Environment *services.EnvironmentService- Settings *services.SettingsService- KV *services.KVService- JobSchedule *services.JobService- SettingsSearch *services.SettingsSearchService- CustomizeSearch *services.CustomizeSearchService- Container *services.ContainerService- Image *services.ImageService- Build *services.BuildService- BuildWorkspace *services.BuildWorkspaceService- Lifecycle *services.LifecycleService- Volume *services.VolumeService- Network *services.NetworkService- Port *services.PortService- Swarm *services.SwarmService- ImageUpdate *services.ImageUpdateService- Session *services.SessionService- Auth *services.AuthService- Oidc *services.OidcService- Docker *services.DockerClientService- Template *services.TemplateService- ContainerRegistry *services.ContainerRegistryService- System *services.SystemService- SystemUpgrade *services.SystemUpgradeService- Diagnostics *services.DiagnosticsService- Updater *services.UpdaterService- Event *services.EventService- Activity *services.ActivityService- Version *services.VersionService- Notification *services.NotificationService- ApiKey *services.ApiKeyService- Federated *services.FederatedCredentialService- GitRepository *services.GitRepositoryService- GitOpsSync *services.GitOpsSyncService- Webhook *services.WebhookService- Vulnerability *services.VulnerabilityService- Dashboard *services.DashboardService- Role *services.RoleService- Variable *services.VariableService- AuthMiddleware *middleware.AuthMiddleware+ AppImages *appimages.ApplicationImagesService+ User *user.UserService+ Project *project.ProjectService+ Environment *environment.EnvironmentService+ Settings *settings.SettingsService+ KV *kv.KVService+ JobSchedule *job.JobService+ Search *search.Module+ Container *container.Module+ Image *image.ImageService+ Build *build.BuildService+ BuildWorkspace *build.BuildWorkspaceService+ Lifecycle *lifecycle.LifecycleService+ Volume *volume.Module+ Network *network.NetworkService+ Port *port.PortService+ Swarm *swarm.SwarmService+ ImageUpdate *imageupdate.ImageUpdateService+ Session *session.SessionService+ Auth *auth.AuthService+ Oidc *oidc.OidcService+ Docker *docker.DockerClientService+ Template *template.TemplateService+ ContainerRegistry *registry.ContainerRegistryService+ System *system.Module+ SystemUpgrade *system.SystemUpgradeService+ Diagnostics *diagnostics.DiagnosticsService+ Updater *updater.Module+ Event *event.EventService+ Activity *activity.ActivityService+ Version *version.VersionService+ Notification *notification.NotificationService+ ApiKey *apikey.ApiKeyService+ Federated *federated.FederatedCredentialService+ GitRepository *gitrepo.GitRepositoryService+ GitOpsSync *gitops.GitOpsSyncService+ Webhook *webhook.Module+ Vulnerability *vulnerability.VulnerabilityService+ Dashboard *dashboard.Module+ Role *role.RoleService+ Variable *variable.VariableService+ AuthMiddleware *auth.AuthMiddlewareAutoUpdate *scheduler.AutoUpdateJobImageUpdateWatcher *scheduler.ImageUpdateWatcherdiff --git a/backend/internal/di/providers.go b/backend/internal/di/providers.goindex 33362b35b..13cd241df 100644--- a/backend/internal/di/providers.go+++ b/backend/internal/di/providers.go@@ -6,15 +6,49 @@ import ("log/slog""net/http"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth"+ "github.com/getarcaneapp/arcane/backend/v2/internal/build"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/federated"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitops"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitrepo"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/job"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv"+ "github.com/getarcaneapp/arcane/backend/v2/internal/lifecycle"+ "github.com/getarcaneapp/arcane/backend/v2/internal/network"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/passkey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/search"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/swarm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/template"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user"+ "github.com/getarcaneapp/arcane/backend/v2/internal/version"+ "github.com/getarcaneapp/arcane/backend/v2/internal/vulnerability"+"emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/container"+ "github.com/getarcaneapp/arcane/backend/v2/internal/dashboard""github.com/getarcaneapp/arcane/backend/v2/internal/database"- arcanelogging "github.com/getarcaneapp/arcane/backend/v2/internal/logging"- "github.com/getarcaneapp/arcane/backend/v2/internal/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/system"+ "github.com/getarcaneapp/arcane/backend/v2/internal/updater"+ "github.com/getarcaneapp/arcane/backend/v2/internal/volume"+ "github.com/getarcaneapp/arcane/backend/v2/internal/webhook""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge""github.com/getarcaneapp/arcane/backend/v2/pkg/scheduler"+ arcanelogging "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/logging""github.com/getarcaneapp/arcane/backend/v2/resources""go.uber.org/fx")@@ -23,7 +57,74 @@ func provideResourcesFSInternal() embed.FS {return resources.FS}-func provideSettingsServiceInternal(ctx context.Context, lc fx.Lifecycle, db *database.DB, runtime *actors.Runtime) (*services.SettingsService, error) {+func provideEventModuleInternal(db *database.DB, cfg *config.Config, httpClient *http.Client) *event.Module {+ return event.New(event.Dependencies{DB: db, Config: cfg, HTTPClient: httpClient})+}++func provideEventServiceInternal(module *event.Module) *event.EventService {+ return module.Service()+}++func provideActivityModuleInternal(db *database.DB, settingsService *settings.SettingsService, environment *environment.EnvironmentService) *activity.Module {+ return activity.New(activity.Dependencies{+ DB: db,+ Settings: settingsService,+ Environment: activity.EnvironmentDependencies{+ ProxyJSONRequest: environment.ProxyJSONRequest,+ ListRemoteEnvironments: environment.ListRemoteEnvironments,+ GetActiveRemoteEnvironment: environment.GetActiveRemoteEnvironmentSnapshot,+ ProxyJSONRequestForEnvironment: environment.ProxyJSONRequestForEnvironment,+ ResolveEnvironmentName: environment.ResolveEnvironmentName,+ },+ })+}++func provideActivityServiceInternal(module *activity.Module) *activity.ActivityService {+ return module.Service()+}++func provideEnvironmentModuleInternal(service *environment.EnvironmentService, settingsService *settings.SettingsService, apiKey *apikey.ApiKeyService, eventService *event.EventService, cfg *config.Config) *environment.Module {+ return environment.New(service, environment.Dependencies{+ Settings: settingsService,+ ApiKey: apiKey,+ Event: eventService,+ Config: cfg,+ })+}++func provideSwarmModuleInternal(service *swarm.SwarmService, environmentService *environment.EnvironmentService, eventService *event.EventService, cfg *config.Config) *swarm.Module {+ return swarm.New(service, swarm.Dependencies{+ Environment: environmentService,+ Event: eventService,+ Config: cfg,+ })+}++func provideImageUpdateModuleInternal(service *imageupdate.ImageUpdateService, imageService *image.ImageService) *imageupdate.Module {+ return imageupdate.New(service, imageupdate.Dependencies{+ GetUpdateInfoByImageRefs: imageService.GetUpdateInfoByImageRefs,+ })+}++func provideImageModuleInternal(service *image.ImageService, dockerService *docker.DockerClientService, imageUpdate *imageupdate.ImageUpdateService, settingsService *settings.SettingsService, buildService *build.BuildService, activityService *activity.ActivityService) *image.Module {+ return image.New(service, image.Dependencies{+ Docker: dockerService,+ ImageUpdate: imageUpdate,+ Settings: settingsService,+ Build: buildService,+ Activity: activityService,+ })+}++func provideRoleModuleInternal(db *database.DB) *role.Module {+ return role.New(role.Dependencies{DB: db})+}++func provideRoleServiceInternal(module *role.Module) *role.RoleService {+ return module.Service()+}++func provideSettingsServiceInternal(ctx context.Context, lc fx.Lifecycle, db *database.DB, runtime *actors.Runtime) (*settings.SettingsService, error) {executor, err := actors.NewExecutor(ctx, runtime, "services", "settings", 3)if err != nil {return nil, err@@ -32,7 +133,7 @@ func provideSettingsServiceInternal(ctx context.Context, lc fx.Lifecycle, db *daif err != nil {return nil, errors.Combine(err, executor.Stop(ctx))}- service, err := services.NewSettingsService(ctx, db, executor, effects)+ service, err := settings.NewSettingsService(ctx, db, executor, effects)if err != nil {return nil, errors.Combine(err, executor.Stop(ctx), effects.Stop(ctx))}@@ -44,6 +145,14 @@ func provideSettingsServiceInternal(ctx context.Context, lc fx.Lifecycle, db *dareturn service, nil}+func provideSettingsModuleInternal(service *settings.SettingsService, searchModule *search.Module, environment *environment.EnvironmentService, cfg *config.Config) *settings.Module {+ return settings.New(service, settings.Dependencies{+ Search: searchModule.SettingsService(),+ ProxyRemoteJSON: environment.ProxyJSONRequest,+ Config: cfg,+ })+}+func provideAdmissionGateInternal(ctx context.Context, lc fx.Lifecycle, runtime *actors.Runtime) (*actors.Gate[actors.AdmissionKey], error) {gate, err := actors.NewGate[actors.AdmissionKey](ctx, runtime, "admission", "application")if err != nil {@@ -68,8 +177,8 @@ func provideTunnelRegistryInternal(ctx context.Context, lc fx.Lifecycle, runtimereturn registry, nil}-func provideDockerClientServiceInternal(ctx context.Context, lc fx.Lifecycle, actorRuntime *actors.Runtime, db *database.DB, cfg *config.Config, settings *services.SettingsService) *services.DockerClientService {- service := services.NewDockerClientService(ctx, db, cfg, settings)+func provideDockerClientServiceInternal(ctx context.Context, lc fx.Lifecycle, actorRuntime *actors.Runtime, db *database.DB, cfg *config.Config, settings *settings.SettingsService) *docker.DockerClientService {+ service := docker.NewDockerClientService(ctx, db, cfg, settings)var runner *actors.Runnerlc.Append(fx.Hook{OnStart: func(context.Context) error {@@ -89,37 +198,71 @@ func provideDockerClientServiceInternal(ctx context.Context, lc fx.Lifecycle, acreturn service}-func provideVersionServiceInternal(httpClient *http.Client, cfg *config.Config, registry *services.ContainerRegistryService, docker *services.DockerClientService, imageUpdate *services.ImageUpdateService) *services.VersionService {- return services.NewVersionService(httpClient, cfg.UpdateCheckDisabled, config.Version, config.Revision, registry, docker, imageUpdate)+func provideVersionServiceInternal(httpClient *http.Client, cfg *config.Config, registry *registry.ContainerRegistryService, docker *docker.DockerClientService, imageUpdate *imageupdate.ImageUpdateService) *version.VersionService {+ return version.NewVersionService(httpClient, cfg.UpdateCheckDisabled, config.Version, config.Revision, registry, docker, imageUpdate)+}++func provideGitRepositoryModuleInternal(db *database.DB, cfg *config.Config, eventService *event.EventService, settingsService *settings.SettingsService) *gitrepo.Module {+ return gitrepo.New(gitrepo.Dependencies{DB: db, WorkDir: cfg.GitWorkDir, Event: eventService, Settings: settingsService})}-func provideGitRepositoryServiceInternal(db *database.DB, cfg *config.Config, event *services.EventService, settings *services.SettingsService) *services.GitRepositoryService {- return services.NewGitRepositoryService(db, cfg.GitWorkDir, event, settings)+func provideGitRepositoryServiceInternal(module *gitrepo.Module) *gitrepo.GitRepositoryService {+ return module.Service()}-func provideVolumeServiceInternal(lc fx.Lifecycle, db *database.DB, docker *services.DockerClientService, event *services.EventService, settings *services.SettingsService, container *services.ContainerService, image *services.ImageService, cfg *config.Config) *services.VolumeService {- service := services.NewVolumeService(db, docker, event, settings, container, image, cfg.BackupVolumeName)+func provideVolumeModuleInternal(lc fx.Lifecycle, db *database.DB, docker *docker.DockerClientService, event *event.EventService, settings *settings.SettingsService, image *image.ImageService, activity *activity.ActivityService, cfg *config.Config) *volume.Module {+ module := volume.New(volume.Dependencies{+ DB: db,+ Docker: docker,+ Event: event,+ Settings: settings,+ Image: image,+ Activity: activity,+ BackupVolumeName: cfg.BackupVolumeName,+ })lc.Append(fx.Hook{OnStop: func(ctx context.Context) error {- service.CleanupHelperContainers(ctx)+ module.Service().CleanupHelperContainers(ctx)return nil},})- return service+ return module+}++func provideContainerModuleInternal(ctx context.Context, event *event.EventService, docker *docker.DockerClientService, image *image.ImageService, settings *settings.SettingsService, project *project.ProjectService, activity *activity.ActivityService) *container.Module {+ return container.New(ctx, container.Dependencies{+ Event: event,+ Docker: docker,+ Image: image,+ Settings: settings,+ Project: project,+ Activity: activity,+ })+}++func provideAuthServiceInternal(user *user.UserService, settings *settings.SettingsService, event *event.EventService, session *session.SessionService, role *role.RoleService, cfg *config.Config, errorHandler *arcanelogging.SlogErrorHandler) *auth.AuthService {+ return auth.NewAuthService(user, settings, event, session, role, cfg.JWTSecret, cfg, errorHandler)+}++func provideAuthModuleInternal(service *auth.AuthService, userService *user.UserService, settingsService *settings.SettingsService, passkeyService *passkey.PasskeyService) *auth.Module {+ return auth.New(auth.Dependencies{+ Service: service,+ User: userService,+ Settings: settingsService,+ BeginMFAAuthentication: passkeyService.BeginMFAAuthentication,+ })}-func provideAuthServiceInternal(user *services.UserService, settings *services.SettingsService, event *services.EventService, session *services.SessionService, role *services.RoleService, cfg *config.Config, errorHandler *arcanelogging.SlogErrorHandler) *services.AuthService {- return services.NewAuthService(user, settings, event, session, role, cfg.JWTSecret, cfg, errorHandler)+func provideContainerRegistryModuleInternal(db *database.DB, dockerService *docker.DockerClientService, kvService *kv.KVService, environment *environment.EnvironmentService) *registry.Module {+ return registry.New(registry.Dependencies{DB: db, Docker: dockerService, KV: kvService, SyncRemoteRegistries: environment.SyncRegistriesToRemoteEnvironments})}-func provideContainerRegistryServiceInternal(db *database.DB, docker *services.DockerClientService, kv *services.KVService) *services.ContainerRegistryService {- return services.NewContainerRegistryService(db, func(ctx context.Context) (services.RegistryDaemonClient, error) {- return docker.GetClient(ctx)- }, kv)+func provideContainerRegistryServiceInternal(module *registry.Module) *registry.ContainerRegistryService {+ return module.Service()}-func provideProjectServiceInternal(db *database.DB, settings *services.SettingsService, event *services.EventService, image *services.ImageService, docker *services.DockerClientService, build *services.BuildService, lifecycle *services.LifecycleService, kv *services.KVService, registry *services.ContainerRegistryService, environment *services.EnvironmentService, cfg *config.Config) *services.ProjectService {- return services.NewProjectService(db, settings, event, image, docker, build, lifecycle, registry, cfg).+func provideProjectServiceInternal(db *database.DB, settings *settings.SettingsService, event *event.EventService, image *image.ImageService, docker *docker.DockerClientService, build *build.BuildService, lifecycle *lifecycle.LifecycleService, kv *kv.KVService, registry *registry.ContainerRegistryService, environment *environment.EnvironmentService, cfg *config.Config) *project.ProjectService {+ return project.NewProjectService(db, settings, event, image, docker, build, lifecycle, registry, cfg).WithKVService(kv).WithRegistryCredentialsProvider(environment.GetEnabledRegistryCredentials)}@@ -132,42 +275,78 @@ type updaterServiceParams struct {fx.InDB *database.DB- Settings *services.SettingsService- Docker *services.DockerClientService- Project *services.ProjectService- ImageUpdate *services.ImageUpdateService- Registry *services.ContainerRegistryService- Event *services.EventService- Image *services.ImageService- Notification *services.NotificationService- SystemUpgrade *services.SystemUpgradeService- Activity *services.ActivityService+ Settings *settings.SettingsService+ Docker *docker.DockerClientService+ Project *project.ProjectService+ ImageUpdate *imageupdate.ImageUpdateService+ Registry *registry.ContainerRegistryService+ Event *event.EventService+ Image *image.ImageService+ Notification *notification.NotificationService+ SystemUpgrade *system.SystemUpgradeService+ Activity *activity.ActivityService+}++func provideUpdaterModuleInternal(p updaterServiceParams) (*updater.Module, error) {+ return updater.New(updater.Dependencies{+ DB: p.DB,+ Settings: p.Settings,+ Docker: p.Docker,+ Project: p.Project,+ ImageUpdate: p.ImageUpdate,+ Registry: p.Registry,+ Event: p.Event,+ Image: p.Image,+ Notification: p.Notification,+ SelfUpgrade: p.SystemUpgrade,+ Activity: p.Activity,+ })+}++func provideUserServiceInternal(db *database.DB, role *role.RoleService) *user.UserService {+ return user.NewUserService(db).WithRoleService(role)+}++func provideUserModuleInternal(service *user.UserService, auth *auth.AuthService, settingsService *settings.SettingsService) *user.Module {+ return user.New(user.Dependencies{Service: service, InvalidateUserTokenCache: auth.InvalidateUserTokenCache, Settings: settingsService})+}++func provideJobModuleInternal(db *database.DB, settingsService *settings.SettingsService, cfg *config.Config, environment *environment.EnvironmentService) *job.Module {+ return job.New(job.Dependencies{DB: db, Settings: settingsService, Config: cfg, Environment: environment})+}++func provideJobServiceInternal(module *job.Module) *job.JobService {+ return module.Service()+}++func provideTemplateModuleInternal(ctx context.Context, db *database.DB, httpClient *http.Client, settingsService *settings.SettingsService) *template.Module {+ return template.New(template.Dependencies{Context: ctx, DB: db, HTTPClient: httpClient, Settings: settingsService})}-func provideUpdaterServiceInternal(p updaterServiceParams) (*services.UpdaterService, error) {- return services.NewUpdaterService(p.DB, p.Settings, p.Docker, p.Project, p.ImageUpdate, p.Registry, p.Event, p.Image, p.Notification, p.SystemUpgrade, p.Activity)+func provideTemplateServiceInternal(module *template.Module) *template.TemplateService {+ return module.Service()}-func provideUserServiceInternal(db *database.DB, role *services.RoleService) *services.UserService {- return services.NewUserService(db).WithRoleService(role)+func provideApiKeyModuleInternal(db *database.DB, userService *user.UserService, roleService *role.RoleService) *apikey.Module {+ return apikey.New(apikey.Dependencies{DB: db, User: userService, Role: roleService})}-func provideApiKeyServiceInternal(db *database.DB, user *services.UserService, role *services.RoleService) *services.ApiKeyService {- return services.NewApiKeyService(db, user).WithRoleService(role)+func provideApiKeyServiceInternal(module *apikey.Module) *apikey.ApiKeyService {+ return module.Service()}-func provideFederatedCredentialServiceInternal(db *database.DB, auth *services.AuthService, user *services.UserService, settings *services.SettingsService, event *services.EventService, httpClient *http.Client, role *services.RoleService) *services.FederatedCredentialService {- return services.NewFederatedCredentialService(db, auth, user, settings, event, httpClient).WithRoleService(role)+func provideFederatedCredentialServiceInternal(db *database.DB, auth *auth.AuthService, user *user.UserService, settings *settings.SettingsService, event *event.EventService, httpClient *http.Client, role *role.RoleService) *federated.FederatedCredentialService {+ return federated.NewFederatedCredentialService(db, auth, user, settings, event, httpClient).WithRoleService(role)}-func provideAuthMiddlewareInternal(auth *services.AuthService, apiKey *services.ApiKeyService, env *services.EnvironmentService, role *services.RoleService, cfg *config.Config) *middleware.AuthMiddleware {- return middleware.NewAuthMiddleware(auth, cfg).+func provideAuthMiddlewareInternal(authService *auth.AuthService, apiKey *apikey.ApiKeyService, env *environment.EnvironmentService, role *role.RoleService, cfg *config.Config) *auth.AuthMiddleware {+ return auth.NewAuthMiddleware(authService, cfg).WithApiKeyValidator(apiKey).WithEnvironmentAccessTokenResolver(env).WithPermissionResolver(role)}-func provideFilesystemWatcherJobInternal(ctx context.Context, lc fx.Lifecycle, actorRuntime *actors.Runtime, project *services.ProjectService, template *services.TemplateService, settings *services.SettingsService, cfg *config.Config) (*scheduler.FilesystemWatcherJob, error) {+func provideFilesystemWatcherJobInternal(ctx context.Context, lc fx.Lifecycle, actorRuntime *actors.Runtime, project *project.ProjectService, template *template.TemplateService, settings *settings.SettingsService, cfg *config.Config) (*scheduler.FilesystemWatcherJob, error) {job, err := scheduler.NewFilesystemWatcherJob(ctx, actorRuntime, project, template, settings, cfg.ProjectScanMaxDepth)if err != nil {return nil, err@@ -194,3 +373,46 @@ func provideFilesystemWatcherJobInternal(ctx context.Context, lc fx.Lifecycle, a})return job, nil}++func provideDashboardModuleInternal(db *database.DB, docker *docker.DockerClientService, containerModule *container.Module, project *project.ProjectService, image *image.ImageService, settings *settings.SettingsService, vulnerability *vulnerability.VulnerabilityService, environment *environment.EnvironmentService, version *version.VersionService, volumeModule *volume.Module) *dashboard.Module {+ return dashboard.New(dashboard.Dependencies{+ DB: db,+ Docker: docker,+ Container: containerModule.Service(),+ Project: project,+ Image: image,+ Settings: settings,+ Vulnerability: vulnerability,+ Environment: environment,+ Version: version,+ Volume: volumeModule.Service(),+ })+}++func provideSystemModuleInternal(db *database.DB, cfg *config.Config, docker *docker.DockerClientService, containerModule *container.Module, image *image.ImageService, volumeModule *volume.Module, network *network.NetworkService, settings *settings.SettingsService, activity *activity.ActivityService, upgrade *system.SystemUpgradeService, environment *environment.EnvironmentService) *system.Module {+ return system.New(system.Dependencies{+ DB: db,+ Config: cfg,+ Docker: docker,+ Container: containerModule.Service(),+ Image: image,+ Volume: volumeModule.Service(),+ Network: network,+ Settings: settings,+ Activity: activity,+ SystemUpgrade: upgrade,+ Environment: environment,+ })+}++func provideWebhookModuleInternal(db *database.DB, containerModule *container.Module, updaterModule *updater.Module, project *project.ProjectService, gitOpsSync *gitops.GitOpsSyncService, event *event.EventService, environment *environment.EnvironmentService) *webhook.Module {+ return webhook.New(webhook.Dependencies{+ DB: db,+ Container: containerModule.Service(),+ Updater: updaterModule.Service(),+ Project: project,+ GitOpsSync: gitOpsSync,+ Event: event,+ Environment: environment,+ })+}diff --git a/backend/internal/services/diagnostics_service.go b/backend/internal/diagnostics/service.gosimilarity index 99%rename from backend/internal/services/diagnostics_service.gorename to backend/internal/diagnostics/service.goindex 429a0f1b4..496b6b461 100644--- a/backend/internal/services/diagnostics_service.go+++ b/backend/internal/diagnostics/service.go@@ -1,4 +1,4 @@-package services+package diagnosticsimport ("runtime"diff --git a/backend/internal/services/diagnostics_service_test.go b/backend/internal/diagnostics/service_test.gosimilarity index 96%rename from backend/internal/services/diagnostics_service_test.gorename to backend/internal/diagnostics/service_test.goindex b43d07293..accd21a5c 100644--- a/backend/internal/services/diagnostics_service_test.go+++ b/backend/internal/diagnostics/service_test.go@@ -1,4 +1,4 @@-package services+package diagnosticsimport ("testing"diff --git a/backend/internal/services/docker_client_service.go b/backend/internal/docker/service.gosimilarity index 89%rename from backend/internal/services/docker_client_service.gorename to backend/internal/docker/service.goindex 2b0cf49dc..1f7143d96 100644--- a/backend/internal/services/docker_client_service.go+++ b/backend/internal/docker/service.go@@ -1,4 +1,4 @@-package services+package dockerimport ("context"@@ -13,6 +13,7 @@ import ("github.com/cenkalti/backoff/v5""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"docker "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"@@ -32,17 +33,18 @@ import (const dockerClientNegotiationTimeout = 5 * time.Secondtype DockerClientService struct {+ *client.Client+db *database.DBconfig *config.Config- settingsService *SettingsService- client *client.Client+ settingsService *settings.SettingsServiceclientVersion stringclientLastProbe time.Timemu sync.MutexeventBus *bus.DockerEventBus}-func NewDockerClientService(_ context.Context, db *database.DB, cfg *config.Config, settingsService *SettingsService) *DockerClientService {+func NewDockerClientService(_ context.Context, db *database.DB, cfg *config.Config, settingsService *settings.SettingsService) *DockerClientService {return &DockerClientService{db: db,config: cfg,@@ -51,6 +53,18 @@ func NewDockerClientService(_ context.Context, db *database.DB, cfg *config.Conf}}+// WithClient returns a copy of the service bound to an already-connected Docker+// client, skipping host discovery. Used where the caller owns the connection.+func (s *DockerClientService) WithClient(dockerClient *client.Client) *DockerClientService {+ return &DockerClientService{+ db: s.db,+ config: s.config,+ settingsService: s.settingsService,+ Client: dockerClient,+ eventBus: s.eventBus,+ }+}+func newDockerClientInternal(ctx context.Context, host string) (*client.Client, error) {apiVersion, err := detectDockerAPIVersionInternal(ctx, host)if err != nil {@@ -117,8 +131,8 @@ func closeDockerClientInternal(cli *client.Client, message string) {// It initializes the client on the first call.func (s *DockerClientService) GetClient(ctx context.Context) (*client.Client, error) {s.mu.Lock()- if s.client != nil {- cli := s.client+ if s.Client != nil {+ cli := s.Clients.mu.Unlock()return cli, nil}@@ -130,14 +144,14 @@ func (s *DockerClientService) GetClient(ctx context.Context) (*client.Client, er}s.mu.Lock()- if s.client != nil {- existingClient := s.client+ if s.Client != nil {+ existingClient := s.Clients.mu.Unlock()closeDockerClientInternal(cli, "failed to close unused Docker client after concurrent initialization")return existingClient, nil}- s.client = cli+ s.Client = clis.clientVersion = cli.ClientVersion()s.clientLastProbe = time.Now()s.mu.Unlock()@@ -154,7 +168,7 @@ func (s *DockerClientService) RefreshClient(ctx context.Context) error {}s.mu.Lock()- if s.client != nil && apiVersion == s.clientVersion {+ if s.Client != nil && apiVersion == s.clientVersion {s.clientLastProbe = time.Now()s.mu.Unlock()return nil@@ -167,15 +181,15 @@ func (s *DockerClientService) RefreshClient(ctx context.Context) error {}s.mu.Lock()- if s.client != nil && apiVersion == s.clientVersion {+ if s.Client != nil && apiVersion == s.clientVersion {s.clientLastProbe = time.Now()s.mu.Unlock()closeDockerClientInternal(cli, "failed to close unused Docker client after concurrent refresh")return nil}- oldClient := s.client- s.client = cli+ oldClient := s.Client+ s.Client = clis.clientVersion = apiVersions.clientLastProbe = time.Now()s.mu.Unlock()@@ -187,14 +201,17 @@ func (s *DockerClientService) RefreshClient(ctx context.Context) error {// DockerHost returns the configured DOCKER_HOST value.func (s *DockerClientService) DockerHost() string {+ if s == nil || s.config == nil {+ return ""+ }return s.config.DockerHost}// Close closes the cached Docker client.func (s *DockerClientService) Close() {s.mu.Lock()- oldClient := s.client- s.client = nil+ oldClient := s.Client+ s.Client = nils.mu.Unlock()closeDockerClientInternal(oldClient, "failed to close Docker client")@@ -288,7 +305,7 @@ func sleepDockerEventBackoffInternal(ctx context.Context, eventBackoff *backoff.}}-func (s *DockerClientService) listContainersInternal(ctx context.Context) ([]container.Summary, error) {+func (s *DockerClientService) ListContainers(ctx context.Context) ([]container.Summary, error) {dockerClient, err := s.GetClient(ctx)if err != nil {return nil, errors.WrapIf(err, "failed to connect to Docker")@@ -305,7 +322,7 @@ func (s *DockerClientService) listContainersInternal(ctx context.Context) ([]conreturn containerList.Items, nil}-func (s *DockerClientService) listImagesInternal(ctx context.Context) ([]image.Summary, error) {+func (s *DockerClientService) ListImages(ctx context.Context) ([]image.Summary, error) {dockerClient, err := s.GetClient(ctx)if err != nil {return nil, errors.WrapIf(err, "failed to connect to Docker")@@ -368,14 +385,14 @@ func (s *DockerClientService) GetSnapshot(ctx context.Context, envID string) (*ddefer utils.RecoverToError(&workerErr, "docker snapshot worker")var err error- containers, err = s.listContainersInternal(groupCtx)+ containers, err = s.ListContainers(groupCtx)return err})g.Go(func() (workerErr error) {defer utils.RecoverToError(&workerErr, "docker snapshot worker")var err error- images, err = s.listImagesInternal(groupCtx)+ images, err = s.ListImages(groupCtx)return err})g.Go(func() (workerErr error) {@@ -411,7 +428,7 @@ func (s *DockerClientService) GetSnapshot(ctx context.Context, envID string) (*d}func (s *DockerClientService) GetAllContainers(ctx context.Context) ([]container.Summary, int, int, int, error) {- containers, err := s.listContainersInternal(ctx)+ containers, err := s.ListContainers(ctx)if err != nil {return nil, 0, 0, 0, err}@@ -430,22 +447,22 @@ func (s *DockerClientService) GetAllContainers(ctx context.Context) ([]container}func (s *DockerClientService) GetAllImages(ctx context.Context) ([]image.Summary, int, int, int, error) {- images, err := s.listImagesInternal(ctx)+ images, err := s.ListImages(ctx)if err != nil {return nil, 0, 0, 0, err}- containers, err := s.listContainersInternal(ctx)+ containers, err := s.ListContainers(ctx)if err != nil {return nil, 0, 0, 0, err}- inuse, unused, total := countImageUsageInternal(images, containers)+ inuse, unused, total := CountImageUsage(images, containers)return images, inuse, unused, total, nil}-func countImageUsageInternal(images []image.Summary, containers []container.Summary) (inuse int, unused int, total int) {+func CountImageUsage(images []image.Summary, containers []container.Summary) (inuse int, unused int, total int) {inUseImageIDs := make(map[string]struct{}, len(containers))for _, c := range containers {if c.ImageID == "" {@@ -467,7 +484,7 @@ func countImageUsageInternal(images []image.Summary, containers []container.Summ}func (s *DockerClientService) GetAllNetworks(ctx context.Context) ([]network.Summary, int, int, int, error) {- containers, err := s.listContainersInternal(ctx)+ containers, err := s.ListContainers(ctx)if err != nil {return nil, 0, 0, 0, err}@@ -510,7 +527,7 @@ func (s *DockerClientService) GetAllNetworks(ctx context.Context) ([]network.Sum}func (s *DockerClientService) GetAllVolumes(ctx context.Context) ([]*volume.Volume, int, int, int, error) {- containers, err := s.listContainersInternal(ctx)+ containers, err := s.ListContainers(ctx)if err != nil {return nil, 0, 0, 0, err}diff --git a/backend/internal/services/docker_client_service_test.go b/backend/internal/docker/service_test.gosimilarity index 97%rename from backend/internal/services/docker_client_service_test.gorename to backend/internal/docker/service_test.goindex ccb369692..115685bfa 100644--- a/backend/internal/services/docker_client_service_test.go+++ b/backend/internal/docker/service_test.go@@ -1,4 +1,4 @@-package services+package dockerimport ("context"@@ -96,7 +96,7 @@ func TestDockerClientService_RefreshClientRecreatesCachedClientAfterAPIVersionCherr = svc.RefreshClient(context.Background())require.NoError(t, err)- secondClient := svc.client+ secondClient := svc.Clientt.Cleanup(func() {_ = secondClient.Close()})@@ -129,7 +129,7 @@ func TestDockerClientService_RefreshClientClosesOldCachedClientWhenReplaced(t *terr = svc.RefreshClient(context.Background())require.NoError(t, err)- secondClient := svc.client+ secondClient := svc.Clientt.Cleanup(func() {_ = secondClient.Close()})@@ -167,7 +167,7 @@ func TestDockerClientService_RefreshClientProbeFailureKeepsCachedClient(t *testierr = svc.RefreshClient(context.Background())require.Error(t, err)- assert.Same(t, firstClient, svc.client)+ assert.Same(t, firstClient, svc.Client)assert.Equal(t, "1.41", svc.clientVersion)assert.False(t, svc.clientLastProbe.IsZero())}@@ -268,7 +268,7 @@ func TestCountImageUsage_UsesContainerImageIDs(t *testing.T) {{ImageID: ""},}- inuse, unused, total := countImageUsageInternal(images, containers)+ inuse, unused, total := CountImageUsage(images, containers)assert.Equal(t, 2, inuse)assert.Equal(t, 1, unused)@@ -276,7 +276,7 @@ func TestCountImageUsage_UsesContainerImageIDs(t *testing.T) {}func TestCountImageUsage_NoImages(t *testing.T) {- inuse, unused, total := countImageUsageInternal(nil, []container.Summary{{ImageID: "sha256:image-a"}})+ inuse, unused, total := CountImageUsage(nil, []container.Summary{{ImageID: "sha256:image-a"}})assert.Equal(t, 0, inuse)assert.Equal(t, 0, unused)diff --git a/backend/api/handlers/edge_mtls_ca.go b/backend/internal/environment/edge_mtls_handler.gosimilarity index 73%rename from backend/api/handlers/edge_mtls_ca.gorename to backend/internal/environment/edge_mtls_handler.goindex c20427c32..bdbf69a73 100644--- a/backend/api/handlers/edge_mtls_ca.go+++ b/backend/internal/environment/edge_mtls_handler.go@@ -1,4 +1,4 @@-package handlers+package environmentimport ("crypto/x509"@@ -17,36 +17,6 @@ import (const edgeMTLSCertificateExpiryWarningWindow = 30 * 24 * time.Hour-func generatedEdgeMTLSCAPathInternal(cfg *config.Config) (string, error) {- if cfg == nil {- return "", errors.New("config not available")- }- if edge.NormalizeEdgeMTLSMode(cfg.EdgeMTLSMode) == edge.EdgeMTLSModeDisabled {- return "", errors.New("edge mTLS is disabled")- }-- edgeCfg := &edge.Config{- EdgeMTLSMode: cfg.EdgeMTLSMode,- EdgeMTLSCAFile: cfg.EdgeMTLSCAFile,- EdgeMTLSAssetsDir: cfg.EdgeMTLSAssetsDir,- AppURL: cfg.GetAppURL(),- }- caPath, err := edge.GeneratedManagerMTLSCAPath(edgeCfg)- if err != nil {- return "", errors.WrapIf(err, "resolve edge mTLS CA path")- }- if _, err := os.Stat(caPath); err != nil {- return "", errors.WrapIf(err, "stat edge mTLS CA")- }-- return caPath, nil-}--func hasGeneratedEdgeMTLSCAInternal(cfg *config.Config) bool {- _, err := generatedEdgeMTLSCAPathInternal(cfg)- return err == nil-}-func generatedEdgeMTLSClientCertPathInternal(cfg *config.Config, envID string) (string, error) {if cfg == nil {return "", errors.New("config not available")diff --git a/backend/internal/environment/environment.go b/backend/internal/environment/environment.gonew file mode 100644index 000000000..17eb830b1--- /dev/null+++ b/backend/internal/environment/environment.go@@ -0,0 +1,484 @@+package environment++import (+ "context"+ "fmt"+ "log/slog"+ "net/http"+ "strings"+ "time"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/remenv"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/scheduler/entityjobs"+ httputils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/validation"+ "github.com/getarcaneapp/arcane/types/v2/containerregistry"+ "github.com/google/uuid"+ "go.getarcane.app/sys/crypto"+ "gorm.io/gorm"+)++type EnvironmentService struct {+ db *database.DB+ httpClient *http.Client+ dockerService *docker.DockerClientService+ eventService *event.EventService+ settingsService *settings.SettingsService+ apiKeyService *apikey.ApiKeyService+ remoteClient *remenv.Client+ edgeTokens *edgeTokenCacheInternal+ remoteEnvs *remoteEnvSnapshotCacheInternal++ // jobs carries the scheduler and app lifecycle context, injected+ // post-construction via SetScheduler (manager-only). Each enabled environment+ // gets its own health-check job; this replaces the single global+ // environment-health job.+ jobs *entityjobs.Registry++ // variableSyncer is injected post-construction via SetVariableSyncer+ // (manager-only) to avoid a wire cycle with variable.VariableService.+ variableSyncer VariableSyncer++ // runtimeWatchers receive a coalesced wake-up whenever an environment's+ // liveness changes. See environment_runtime_notify.go.+ runtimeWatchers runtimeWatchersInternal+}++// VariableSyncer pushes the effective global-variable set to one environment.+// Implemented by variable.VariableService.+type VariableSyncer interface {+ SyncEnvironment(ctx context.Context, envID string) error+}++const (+ ErrEnvironmentAccessTokenRequired = errors.Sentinel("environment access token required")+ ErrInvalidEnvironmentAccessToken = errors.Sentinel("invalid environment access token")+)++func NewEnvironmentService(db *database.DB, httpClient *http.Client, dockerService *docker.DockerClientService, eventService *event.EventService, settingsService *settings.SettingsService, apiKeyService *apikey.ApiKeyService) *EnvironmentService {+ if httpClient == nil {+ httpClient = http.DefaultClient+ }+ return &EnvironmentService{+ db: db,+ httpClient: httpClient,+ dockerService: dockerService,+ eventService: eventService,+ settingsService: settingsService,+ apiKeyService: apiKeyService,+ remoteClient: remenv.NewClient(httpClient, remenv.TunnelTransportFuncs{+ EnsureAvailableFunc: ensureRemoteEnvironmentTunnelAvailableInternal,+ DoFunc: doRemoteEnvironmentTunnelRequestInternal,+ }),+ edgeTokens: newEdgeTokenCacheInternal(),+ remoteEnvs: newRemoteEnvSnapshotCacheInternal(),+ jobs: entityjobs.New(environmentHealthJobPrefix, environmentHealthAdmissionScopeInternal),+ }+}++// SetVariableSyncer injects the global-variable syncer. Called during+// bootstrap on the manager only; agents leave it nil.+func (s *EnvironmentService) SetVariableSyncer(syncer VariableSyncer) {+ s.variableSyncer = syncer+}++func (s *EnvironmentService) ResolveEdgeEnvironmentByToken(ctx context.Context, token string) (string, error) {+ token = strings.TrimSpace(token)+ if token == "" {+ return "", errors.New("agent token required")+ }++ if envID, ok := s.edgeTokens.environmentID(token).Get(); ok {+ return envID, nil+ }++ var env models.Environment+ if err := s.db.WithContext(ctx).+ Select("id", "access_token").+ Where("is_edge = ?", true).+ Where("access_token = ?", token).+ First(&env).Error; err != nil {+ if errors.Is(err, gorm.ErrRecordNotFound) {+ s.logEdgeTokenResolveMissInternal(ctx, token)+ return "", errors.New("invalid agent token")+ }+ return "", errors.WrapIf(err, "failed to resolve edge environment by token")+ }++ s.edgeTokens.put(env.ID, token)+ return env.ID, nil+}++// logEdgeTokenResolveMissInternal emits a debug log diagnosing why an agent+// token failed to resolve to an edge environment. Counts existing edge+// environments (by access_token presence) so operators can distinguish+// "no edge envs configured" from "token does not match any configured env".+// Token contents are never logged in full — only length and a short+// fingerprint that cannot be reversed.+func (s *EnvironmentService) logEdgeTokenResolveMissInternal(ctx context.Context, token string) {+ if s == nil || s.db == nil {+ return+ }+ if !slog.Default().Enabled(ctx, slog.LevelDebug) {+ return+ }++ var totalEdgeEnvs int64+ var edgeEnvsWithToken int64+ totalEdgeEnvsErr := s.db.WithContext(ctx).Model(&models.Environment{}).Where("is_edge = ?", true).Count(&totalEdgeEnvs).Error+ edgeEnvsWithTokenErr := s.db.WithContext(ctx).Model(&models.Environment{}).+ Where("is_edge = ?", true).+ Where("access_token IS NOT NULL AND access_token != ?", "").+ Count(&edgeEnvsWithToken).Error++ args := []any{+ "token_length", len(token),+ "token_fingerprint", remenv.RedactedTokenFingerprint(token),+ }+ if totalEdgeEnvsErr == nil {+ args = append(args, "edge_envs_total", totalEdgeEnvs)+ }+ if edgeEnvsWithTokenErr == nil {+ args = append(args, "edge_envs_with_access_token", edgeEnvsWithToken)+ }++ slog.DebugContext(ctx, "Edge agent token did not match any environment", args...)+}++// ResolveEnvironmentName looks up an environment and returns the label to show for it.+// Use this instead of hardcoding a name for a known ID: names are user-editable, so+// even the local environment's is not fixed.+func (s *EnvironmentService) ResolveEnvironmentName(ctx context.Context, environmentID string) string {+ if strings.TrimSpace(environmentID) == "" {+ environmentID = LocalEnvironmentID+ }+ env, err := s.GetEnvironmentByID(ctx, environmentID)+ if err != nil || env == nil {+ slog.WarnContext(ctx, "failed to resolve environment name", "environmentID", environmentID, "error", err)+ return DisplayName(environmentID, "")+ }+ return DisplayName(env.ID, env.Name)+}++func (s *EnvironmentService) EnsureLocalEnvironment(ctx context.Context, appUrl string) error {+ var existingEnv models.Environment+ err := s.db.WithContext(ctx).Where("id = ?", LocalEnvironmentID).First(&existingEnv).Error++ if err == nil {+ // Local environment already exists, ensure ApiUrl matches current appUrl+ if existingEnv.ApiUrl != appUrl {+ if err := s.db.WithContext(ctx).Model(&existingEnv).Update("api_url", appUrl).Error; err != nil {+ return errors.WrapIf(err, "failed to update local environment api url")+ }+ slog.InfoContext(ctx, "updated local environment api url", "id", LocalEnvironmentID, "url", appUrl)+ }+ return nil+ }++ if !errors.Is(err, gorm.ErrRecordNotFound) {+ return errors.WrapIf(err, "failed to check for local environment")+ }++ // Create the local environment+ now := time.Now()+ localEnv := &models.Environment{+ BaseModel: models.BaseModel{+ ID: LocalEnvironmentID,+ CreatedAt: now,+ UpdatedAt: new(now),+ },+ Name: "Local Docker",+ ApiUrl: appUrl,+ Status: string(models.EnvironmentStatusOnline),+ Enabled: true,+ }++ if err := s.db.WithContext(ctx).Create(localEnv).Error; err != nil {+ return errors.WrapIf(err, "failed to create local environment")+ }++ slog.InfoContext(ctx, "created local environment record", "id", LocalEnvironmentID)+ return nil+}++func (s *EnvironmentService) CreateEnvironment(ctx context.Context, environment *models.Environment, userID, username *string) (*models.Environment, error) {+ environment.ID = uuid.New().String()++ // Only set status to offline if not already set (e.g., API key flow sets it to pending)+ if environment.Status == "" {+ environment.Status = string(models.EnvironmentStatusOffline)+ }++ now := time.Now()+ environment.CreatedAt = now+ environment.UpdatedAt = new(now)++ if err := s.db.WithContext(ctx).Create(environment).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to create environment")+ }++ // Create event in background+ go s.createEnvironmentEvent(context.WithoutCancel(ctx), environment.ID, environment.Name, models.EventTypeEnvironmentCreate, "Environment Created", fmt.Sprintf("Environment '%s' was created", environment.Name), models.EventSeveritySuccess, userID, username)++ if environment.Enabled {+ s.registerHealthJobInternal(ctx, environment.ID)+ }+ s.remoteEnvs.put(*environment)++ return environment, nil+}++func (s *EnvironmentService) GetEnvironmentByID(ctx context.Context, id string) (*models.Environment, error) {+ var envRecord models.Environment+ if err := s.db.WithContext(ctx).Where("id = ?", id).First(&envRecord).Error; err != nil {+ if errors.Is(err, gorm.ErrRecordNotFound) {+ return nil, errors.New("environment not found")+ }+ return nil, errors.WrapIf(err, "failed to get environment")+ }+ return &envRecord, nil+}++func (s *EnvironmentService) UpdateEnvironment(ctx context.Context, id string, updates map[string]any, userID, username *string) (*models.Environment, error) {+ current, err := s.GetEnvironmentByID(ctx, id)+ if err != nil {+ return nil, err+ }++ var nextAPIURL *string+ if rawAPIURL, ok := updates["api_url"]; ok {+ if apiURL, isString := rawAPIURL.(string); isString {+ nextAPIURL = new(apiURL)+ }+ }+ _, accessTokenUpdated := updates["access_token"]+ if err := validation.ValidateCredentialTargetChange(+ "environment API URL",+ current.ApiUrl,+ nextAPIURL,+ func(value string) string {+ normalized, normalizeErr := httputils.NormalizeBaseURL(value)+ if normalizeErr != nil {+ return strings.TrimSpace(value)+ }+ return normalized+ },+ map[string]bool{"accessToken": current.AccessToken != nil && *current.AccessToken != ""},+ map[string]bool{"accessToken": accessTokenUpdated},+ ); err != nil {+ return nil, err+ }++ updates["updated_at"] = new(time.Now())++ if err := s.db.WithContext(ctx).Model(&models.Environment{}).Where("id = ?", id).Updates(updates).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to update environment")+ }++ updated, err := s.GetEnvironmentByID(ctx, id)+ if err != nil {+ return nil, err+ }+ s.remoteEnvs.put(*updated)++ if rawAccessToken, ok := updates["access_token"]; ok {+ accessToken, _ := rawAccessToken.(string)+ s.edgeTokens.sync(id, accessToken)+ }++ // Reconcile the per-environment health job when the enabled flag is toggled.+ if rawEnabled, ok := updates["enabled"]; ok {+ if enabled, isBool := rawEnabled.(bool); isBool {+ if enabled {+ s.registerHealthJobInternal(ctx, id)+ } else {+ s.removeHealthJobInternal(ctx, id)+ }+ }+ }++ // Create event in background (skip for local environment)+ if id != "0" {+ go s.createEnvironmentEvent(context.WithoutCancel(ctx), id, updated.Name, models.EventTypeEnvironmentUpdate, "Environment Updated", fmt.Sprintf("Environment '%s' was updated", updated.Name), models.EventSeverityInfo, userID, username)+ }++ return updated, nil+}++func (s *EnvironmentService) DeleteEnvironment(ctx context.Context, id string, userID, username *string) error {+ // Get environment details before deletion+ env, err := s.GetEnvironmentByID(ctx, id)+ if err != nil {+ return err+ }++ // Stop the per-environment health job before the row is removed.+ s.removeHealthJobInternal(ctx, id)++ var syncIDs []string+ if err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {+ if err := tx.Model(&models.GitOpsSync{}).+ Where("environment_id = ?", id).+ Pluck("id", &syncIDs).Error; err != nil {+ return errors.WrapIf(err, "failed to list environment gitops syncs")+ }++ if len(syncIDs) > 0 {+ if err := tx.Model(&models.Project{}).+ Where("gitops_managed_by IN ?", syncIDs).+ Update("gitops_managed_by", nil).Error; err != nil {+ return errors.WrapIf(err, "failed to clear environment gitops project references")+ }+ if err := tx.Where("environment_id = ?", id).Delete(&models.GitOpsSync{}).Error; err != nil {+ return errors.WrapIf(err, "failed to delete environment gitops syncs")+ }+ }++ if err := tx.Delete(&models.Environment{}, "id = ?", id).Error; err != nil {+ return errors.WrapIf(err, "failed to delete environment")+ }++ return nil+ }); err != nil {+ if env.Enabled {+ s.registerHealthJobInternal(ctx, env.ID)+ }+ return err+ }++ // Deleting an environment orphans its GitOps syncs, whose jobs belong to+ // gitops.GitOpsSyncService's own registry — remove them by name here.+ if scheduler := s.jobs.Scheduler(); scheduler != nil {+ schedulerCtx := s.jobs.Context(ctx)+ for _, syncID := range syncIDs {+ scheduler.RemoveJob(schedulerCtx, entityjobs.GitOpsSyncJobPrefix+syncID)+ }+ }++ s.edgeTokens.invalidate(id)+ s.remoteEnvs.remove(id)++ // Create event in background+ go s.createEnvironmentEvent(context.WithoutCancel(ctx), id, env.Name, models.EventTypeEnvironmentDelete, "Environment Deleted", fmt.Sprintf("Environment '%s' was deleted", env.Name), models.EventSeverityWarning, userID, username)++ return nil+}++func (s *EnvironmentService) createEnvironmentEvent(ctx context.Context, envID, envName string, eventType models.EventType, title, description string, severity models.EventSeverity, userID, username *string) {+ if s == nil || s.eventService == nil {+ return+ }++ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{+ Type: eventType,+ Severity: severity,+ Title: title,+ Description: description,+ ResourceType: new("environment"),+ ResourceID: new(envID),+ ResourceName: new(envName),+ UserID: userID,+ Username: username,+ EnvironmentID: new(envID),+ })+}++func (s *EnvironmentService) RegenerateEnvironmentApiKey(ctx context.Context, envID string, newApiKeyID string, apiKey string, userID, username string, envName string) error {+ // Trim once at the boundary so the value persisted, the value cached,+ // and the value returned by callers (which already TrimSpace before+ // returning) all stay byte-identical. Any divergence here would surface+ // as a 401 "invalid agent token" because lookup is direct equality.+ apiKey = strings.TrimSpace(apiKey)++ updates := map[string]any{+ "api_key_id": newApiKeyID,+ "access_token": apiKey,+ "status": string(models.EnvironmentStatusPending),+ "last_seen": nil, // Clear last seen time+ }++ result := s.db.WithContext(ctx).Model(&models.Environment{}).Where("id = ?", envID).Updates(updates)+ if result.Error != nil {+ return errors.WrapIf(result.Error, "failed to update environment with new API key")+ }+ if result.RowsAffected == 0 {+ // A zero-row update would otherwise report a successful rotation while+ // the new key was never linked to anything.+ return errors.New("environment not found")+ }++ s.edgeTokens.sync(envID, apiKey)+ now := time.Now()+ s.remoteEnvs.update(envID, func(environment *models.Environment) {+ environment.ApiKeyID = &newApiKeyID+ environment.AccessToken = &apiKey+ environment.Status = string(models.EnvironmentStatusPending)+ environment.LastSeen = nil+ environment.UpdatedAt = &now+ })++ // Create event log in background+ go s.createEnvironmentEvent(context.WithoutCancel(ctx), envID, envName, models.EventTypeEnvironmentApiKeyRegenerated, "API Key Regenerated", "Environment API key was regenerated and status set to pending", models.EventSeverityInfo, new(userID), new(username))++ return nil+}++func (s *EnvironmentService) GetDB() *database.DB {+ return s.db+}++func (s *EnvironmentService) ResolveEnvironmentByAccessToken(ctx context.Context, token string) (*models.Environment, error) {+ token = strings.TrimSpace(token)+ if token == "" {+ return nil, ErrEnvironmentAccessTokenRequired+ }++ var env models.Environment+ if err := s.db.WithContext(ctx).+ Where("access_token = ?", token).+ First(&env).Error; err != nil {+ if errors.Is(err, gorm.ErrRecordNotFound) {+ return nil, ErrInvalidEnvironmentAccessToken+ }+ return nil, errors.WrapIf(err, "failed to resolve environment by access token")+ }++ return &env, nil+}++func (s *EnvironmentService) GetEnabledRegistryCredentials(ctx context.Context) ([]containerregistry.Credential, error) {+ var registries []models.ContainerRegistry+ if err := s.db.WithContext(ctx).Where("enabled = ?", true).Find(®istries).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to get enabled container registries")+ }++ var creds []containerregistry.Credential+ for _, reg := range registries {+ if !reg.Enabled || reg.Username == "" || reg.Token == "" {+ continue+ }++ decryptedToken, err := crypto.Decrypt(reg.Token)+ if err != nil {+ slog.WarnContext(ctx, "Failed to decrypt registry token", "registryURL", reg.URL, "error", err.Error())+ continue+ }++ creds = append(creds, containerregistry.Credential{+ URL: reg.URL,+ Username: reg.Username,+ Token: decryptedToken,+ Enabled: reg.Enabled,+ })+ }++ return creds, nil+}diff --git a/backend/internal/environment/environment_caches.go b/backend/internal/environment/environment_caches.gonew file mode 100644index 000000000..5555d4e5c--- /dev/null+++ b/backend/internal/environment/environment_caches.go@@ -0,0 +1,207 @@+package environment++import (+ "strings"+ "sync"+ "time"++ "github.com/samber/hot"+ "github.com/samber/mo"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+)++const edgeTokenCacheTTL = time.Minute++// edgeTokenCacheInternal maps an edge agent token to its environment ID. The+// reverse index lets an environment's entry be dropped when its token rotates,+// which the TTL cache alone cannot do.+type edgeTokenCacheInternal struct {+ mu sync.RWMutex+ byToken *hot.HotCache[string, string]+ byEnvID map[string]string+}++// remoteEnvSnapshotCacheInternal holds the latest in-process copy of every+// enabled, visible, non-local environment, so hot paths avoid a DB round trip.+type remoteEnvSnapshotCacheInternal struct {+ mu sync.RWMutex+ envs map[string]models.Environment+}++// runtimeWatchersInternal fans a coalesced wake-up out to everyone watching for+// environment liveness changes.+type runtimeWatchersInternal struct {+ mu sync.Mutex+ chans map[int]chan struct{}+ seq int+}++func newEdgeTokenCacheInternal() *edgeTokenCacheInternal {+ return &edgeTokenCacheInternal{+ byToken: hot.NewHotCache[string, string](hot.LRU, 1024).+ WithTTL(edgeTokenCacheTTL).+ WithJanitor().+ Build(),+ byEnvID: make(map[string]string),+ }+}++func newRemoteEnvSnapshotCacheInternal() *remoteEnvSnapshotCacheInternal {+ return &remoteEnvSnapshotCacheInternal{envs: make(map[string]models.Environment)}+}++func (c *edgeTokenCacheInternal) environmentID(token string) mo.Option[string] {+ if c == nil || c.byToken == nil || token == "" {+ return mo.None[string]()+ }++ staleEnvironmentID, wasCached := c.byToken.Peek(token)+ environmentID, ok, _ := c.byToken.Get(token)+ if ok {+ return mo.Some(environmentID)+ }+ if wasCached {+ c.mu.Lock()+ if currentToken, indexed := c.byEnvID[staleEnvironmentID]; indexed && currentToken == token {+ delete(c.byEnvID, staleEnvironmentID)+ }+ c.mu.Unlock()+ }+ return mo.None[string]()+}++func (c *edgeTokenCacheInternal) put(envID string, token string) {+ if c == nil || c.byToken == nil || envID == "" || token == "" {+ return+ }++ c.mu.Lock()+ defer c.mu.Unlock()++ if previousToken, ok := c.byEnvID[envID]; ok && previousToken != token {+ c.byToken.Delete(previousToken)+ }++ c.byEnvID[envID] = token+ c.byToken.Set(token, envID)+}++func (c *edgeTokenCacheInternal) invalidate(envID string) {+ if c == nil || envID == "" {+ return+ }++ c.mu.Lock()+ defer c.mu.Unlock()++ if token, ok := c.byEnvID[envID]; ok {+ delete(c.byEnvID, envID)+ if c.byToken != nil {+ c.byToken.Delete(token)+ }+ }+}++// sync replaces an environment's cached token, dropping the entry entirely when+// the new token is blank.+func (c *edgeTokenCacheInternal) sync(envID string, token string) {+ if c == nil || envID == "" {+ return+ }++ c.invalidate(envID)++ if resolvedToken := strings.TrimSpace(token); resolvedToken != "" {+ c.put(envID, resolvedToken)+ }+}++func isActiveRemoteEnvironmentInternal(environment models.Environment) bool {+ return environment.ID != "" && environment.ID != "0" && environment.Enabled && !environment.Hidden+}++func (c *remoteEnvSnapshotCacheInternal) get(environmentID string) mo.Option[models.Environment] {+ if c == nil || environmentID == "" {+ return mo.None[models.Environment]()+ }++ c.mu.RLock()+ envRecord, ok := c.envs[environmentID]+ c.mu.RUnlock()+ if !ok || !isActiveRemoteEnvironmentInternal(envRecord) {+ return mo.None[models.Environment]()+ }+ return mo.Some(envRecord)+}++func (c *remoteEnvSnapshotCacheInternal) replace(environments []models.Environment) {+ if c == nil {+ return+ }++ next := make(map[string]models.Environment, len(environments))+ for _, envRecord := range environments {+ if isActiveRemoteEnvironmentInternal(envRecord) {+ next[envRecord.ID] = envRecord+ }+ }++ c.mu.Lock()+ c.envs = next+ c.mu.Unlock()+}++func (c *remoteEnvSnapshotCacheInternal) put(environment models.Environment) {+ if c == nil {+ return+ }++ if !isActiveRemoteEnvironmentInternal(environment) {+ c.remove(environment.ID)+ return+ }++ c.mu.Lock()+ c.envs[environment.ID] = environment+ c.mu.Unlock()+}++func (c *remoteEnvSnapshotCacheInternal) remove(environmentID string) {+ if c == nil || environmentID == "" {+ return+ }++ c.mu.Lock()+ delete(c.envs, environmentID)+ c.mu.Unlock()+}++func (c *remoteEnvSnapshotCacheInternal) update(environmentID string, update func(*models.Environment)) {+ if c == nil || environmentID == "" || update == nil {+ return+ }++ c.mu.Lock()+ defer c.mu.Unlock()++ envRecord, ok := c.envs[environmentID]+ if !ok {+ return+ }+ update(&envRecord)+ if isActiveRemoteEnvironmentInternal(envRecord) {+ c.envs[environmentID] = envRecord+ } else {+ delete(c.envs, environmentID)+ }+}++// GetActiveRemoteEnvironmentSnapshot returns the latest in-process snapshot for+// an enabled, visible, non-local remote environment.+func (s *EnvironmentService) GetActiveRemoteEnvironmentSnapshot(environmentID string) mo.Option[models.Environment] {+ if s == nil {+ return mo.None[models.Environment]()+ }+ return s.remoteEnvs.get(environmentID)+}diff --git a/backend/internal/environment/environment_health.go b/backend/internal/environment/environment_health.gonew file mode 100644index 000000000..31b6dc95b--- /dev/null+++ b/backend/internal/environment/environment_health.go@@ -0,0 +1,436 @@+package environment++import (+ "context"+ "log/slog"+ "net/http"+ "time"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/actors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge"+ "github.com/getarcaneapp/arcane/types/v2/environment"+ schedulertypes "github.com/getarcaneapp/arcane/types/v2/scheduler"+ "github.com/moby/moby/client"+)++const (+ defaultEnvironmentHealthInterval = "0 */2 * * * *"+ environmentHealthCheckTimeout = 90 * time.Second+)++const environmentHealthJobPrefix = "environment-health:"++const environmentHealthAdmissionScopeInternal = "environment-health"++// SetScheduler injects the job scheduler and app lifecycle context. Called during+// bootstrap on the manager only (agent mode leaves scheduler nil, so all health-job+// registration becomes a no-op).+func (s *EnvironmentService) SetScheduler(ctx context.Context, scheduler schedulertypes.DynamicScheduler, admissionGate *actors.Gate[actors.AdmissionKey]) error {+ return s.jobs.SetScheduler(ctx, scheduler, admissionGate)+}++// registerHealthJobInternal schedules one environment's health check on the+// global environmentHealthInterval (environment health has no per-entity+// interval); the run body self-cancels cleanly when the environment is gone.+func (s *EnvironmentService) registerHealthJobInternal(ctx context.Context, envID string) {+ s.jobs.Register(ctx, envID,+ func(ctx context.Context) string {+ sched := s.settingsService.GetStringSetting(ctx, "environmentHealthInterval", defaultEnvironmentHealthInterval)+ if sched == "" {+ return defaultEnvironmentHealthInterval+ }+ return sched+ },+ func(ctx context.Context) { s.runHealthCheckInternal(ctx, envID) },+ )+}++func (s *EnvironmentService) removeHealthJobInternal(ctx context.Context, envID string) {+ s.jobs.Unregister(ctx, envID)+}++func (s *EnvironmentService) ListEnabledEnvironmentIDs(ctx context.Context) ([]string, error) {+ var ids []string+ if err := s.db.WithContext(ctx).+ Table("environments").+ Where("enabled = ?", true).+ Pluck("id", &ids).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to list enabled environments")+ }+ return ids, nil+}++func (s *EnvironmentService) registerAllEnabledHealthJobsInternal(ctx context.Context) int {+ ids, err := s.ListEnabledEnvironmentIDs(ctx)+ if err != nil {+ slog.ErrorContext(ctx, "failed to list environments for health jobs", "error", err)+ return 0+ }+ for _, id := range ids {+ s.registerHealthJobInternal(ctx, id)+ }+ return len(ids)+}++// RegisterHealthJobsOnStartup registers a health-check job for every enabled+// environment. Replaces the old global environment-health job.+func (s *EnvironmentService) RegisterHealthJobsOnStartup(ctx context.Context) {+ if !s.jobs.Enabled() {+ return+ }+ n := s.registerAllEnabledHealthJobsInternal(ctx)+ slog.InfoContext(ctx, "Registered environment health jobs on startup", "count", n)+}++// RescheduleHealthJobs re-registers all enabled environments' health jobs, picking+// up a changed global interval. Wired from the Jobs UI via job.JobService.+func (s *EnvironmentService) RescheduleHealthJobs(ctx context.Context) {+ if !s.jobs.Enabled() {+ return+ }+ s.registerAllEnabledHealthJobsInternal(ctx)+}++// RunHealthChecksNow runs every enabled environment's health check synchronously.+// Backs the "run now" button for the environment-health job in the Jobs UI.+func (s *EnvironmentService) RunHealthChecksNow(ctx context.Context) error {+ ids, err := s.ListEnabledEnvironmentIDs(ctx)+ if err != nil {+ return err+ }+ for _, id := range ids {+ s.runHealthCheckInternal(ctx, id)+ }+ return nil+}++// runHealthCheckInternal tests one environment's connection (updating its DB status)+// and, for online remotes, syncs registries and repositories to it.+func (s *EnvironmentService) runHealthCheckInternal(ctx context.Context, envID string) {+ lease, admitted, err := s.jobs.TryAcquire(ctx, envID)+ if err != nil {+ slog.ErrorContext(ctx, "environment health check admission failed", "environment_id", envID, "error", err)+ return+ }+ if !admitted {+ slog.WarnContext(ctx, "environment health check skipped; previous run still in progress", "environment_id", envID)+ return+ }+ defer lease.Release()++ status, err := s.TestConnection(ctx, envID, nil)+ switch {+ case err != nil:+ slog.WarnContext(ctx, "environment health check failed", "environment_id", envID, "status", status, "error", err)+ return+ case status != "online":+ return+ }++ // Local environment (ID "0") has no registries/repositories to push.+ if envID == "0" {+ return+ }++ syncCtx, cancel := context.WithTimeout(ctx, environmentHealthCheckTimeout)+ defer cancel()+ if err := s.SyncRegistriesToEnvironment(syncCtx, envID); err != nil {+ slog.WarnContext(syncCtx, "failed to sync registries during health check", "environment_id", envID, "error", err)+ }+ if err := s.SyncRepositoriesToEnvironment(syncCtx, envID); err != nil {+ slog.WarnContext(syncCtx, "failed to sync git repositories during health check", "environment_id", envID, "error", err)+ }+ if s.variableSyncer != nil {+ if err := s.variableSyncer.SyncEnvironment(syncCtx, envID); err != nil {+ slog.WarnContext(syncCtx, "failed to sync global variables during health check", "environment_id", envID, "error", err)+ }+ }+}++func (s *EnvironmentService) TestConnection(ctx context.Context, id string, customApiUrl *string) (string, error) {+ envRecord, err := s.GetEnvironmentByID(ctx, id)+ if err != nil {+ return "error", err+ }+ connectionFailure := func(status string, failure error) (string, error) {+ if customApiUrl == nil {+ return status, failure+ }+ slog.WarnContext(ctx, "Environment custom URL connection test failed", "environment_id", id, "error", failure)+ return "error", common.ErrEnvironmentConnectionTestFailed+ }++ // Special handling for local Docker environment (ID "0")+ if id == "0" && customApiUrl == nil {+ return s.testLocalDockerConnection(ctx, id)+ }++ // For edge environments, check if there's an active tunnel and route through it+ if envRecord.IsEdge && customApiUrl == nil {+ return s.testEdgeConnection(ctx, id)+ }++ apiUrl := envRecord.ApiUrl+ if customApiUrl != nil {+ apiUrl = *customApiUrl+ }++ healthURL, err := buildEnvironmentEndpointURLInternal(apiUrl, "/api/health")+ if err != nil {+ if customApiUrl == nil {+ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))+ }+ return connectionFailure("offline", errors.WrapIf(err, "invalid environment API URL"))+ }++ reqCtx, cancel := context.WithTimeout(ctx, 10*time.Second)+ defer cancel()+ req, err := http.NewRequestWithContext(reqCtx, http.MethodGet, healthURL, nil)+ if err != nil {+ if customApiUrl == nil {+ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))+ }+ return connectionFailure("offline", errors.WrapIf(err, "failed to create request"))+ }+ resp, err := s.httpClient.Do(req)+ if err != nil {+ if customApiUrl == nil {+ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))+ }+ return connectionFailure("offline", errors.WrapIf(err, "connection failed"))+ }+ defer func() { _ = resp.Body.Close() }()++ if resp.StatusCode == http.StatusOK {+ if customApiUrl == nil {+ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOnline))+ }+ return "online", nil+ }++ if customApiUrl == nil {+ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusError))+ }+ return connectionFailure("error", errors.Errorf("unexpected status code: %d", resp.StatusCode))+}++// testEdgeConnection tests connection to an edge agent via its tunnel+func (s *EnvironmentService) testEdgeConnection(ctx context.Context, id string) (string, error) {+ if !edge.HasActiveTunnel(id) {+ if _, ok := edge.RequestTunnelAndWait(ctx, id, edge.DefaultTunnelDemandTTL, edge.DefaultTunnelAcquireTimeout()).Get(); !ok {+ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))+ return "offline", errors.New("edge agent is not connected")+ }+ }++ reqCtx, cancel := context.WithTimeout(ctx, 10*time.Second)+ defer cancel()++ statusCode, _, err := edge.DoRequest(reqCtx, id, http.MethodGet, "/api/health", nil)+ if err != nil {+ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))+ return "offline", errors.WrapIf(err, "health check via tunnel failed")+ }++ if statusCode == http.StatusOK {+ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOnline))+ return "online", nil+ }++ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusError))+ return "error", errors.Errorf("unexpected status code: %d", statusCode)+}++func (s *EnvironmentService) testLocalDockerConnection(ctx context.Context, id string) (string, error) {+ // Test local Docker socket by pinging Docker+ reqCtx, cancel := context.WithTimeout(ctx, 5*time.Second)+ defer cancel()++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))+ return "offline", errors.WrapIf(err, "failed to connect to Docker")+ }++ _, err = dockerClient.Ping(reqCtx, client.PingOptions{})+ if err != nil {+ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))+ return "offline", errors.WrapIf(err, "docker ping failed")+ }++ _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOnline))+ return "online", nil+}++func (s *EnvironmentService) updateEnvironmentStatusInternal(ctx context.Context, id, status string) error {+ var currentEnv models.Environment+ if err := s.db.WithContext(ctx).Select("status", "is_edge").Where("id = ?", id).First(¤tEnv).Error; err != nil {+ return errors.WrapIf(err, "failed to check environment status")+ }++ if currentEnv.Status == string(models.EnvironmentStatusPending) {+ // Edge envs must complete pairing via the agent's outbound tunnel — manager+ // can't dial them directly, so a manager-side reachability check means nothing.+ // Direct envs are reachable from the manager, so a successful health check IS+ // the pairing signal. Don't promote on offline/error ticks though, or a transient+ // blip during initial setup would flip the env out of pending.+ if currentEnv.IsEdge || status != string(models.EnvironmentStatusOnline) {+ slog.DebugContext(ctx, "skipping status update for pending environment", "environment_id", id)+ return nil+ }+ slog.InfoContext(ctx, "promoted pending direct environment to online via reachability check", "environment_id", id)+ }++ now := time.Now()+ updates := map[string]any{+ "status": status,+ "last_seen": &now,+ "updated_at": &now,+ }+ if err := s.db.WithContext(ctx).Model(&models.Environment{}).Where("id = ?", id).Updates(updates).Error; err != nil {+ return errors.WrapIf(err, "failed to update environment status")+ }+ // Direct environments have no tunnel callback, so this health-check write is+ // the only moment their liveness changes.+ s.NotifyRuntimeStateChanged()+ return nil+}++func (s *EnvironmentService) UpdateEnvironmentHeartbeat(ctx context.Context, id string) error {+ now := time.Now()++ // Use Exec with raw SQL for better performance+ // Only update if last_seen is NULL or older than 30 seconds to reduce write frequency+ result := s.db.WithContext(ctx).Exec(`+ UPDATE environments+ SET last_seen = ?, status = ?, updated_at = ?+ WHERE id = ?+ AND (last_seen IS NULL OR last_seen < ?)+ `, new(now), string(models.EnvironmentStatusOnline), new(now), id, now.Add(-30*time.Second))++ if result.Error != nil {+ return errors.WrapIf(result.Error, "failed to update environment heartbeat")+ }++ // The 30s throttle above doubles as the notify throttle: a no-op heartbeat+ // changed nothing worth waking a stream for.+ if result.RowsAffected > 0 {+ s.NotifyRuntimeStateChanged()+ }++ return nil+}++// UpdateEnvironmentConnectionState updates runtime connectivity status without creating+// a generic "environment updated" event. This is used for edge tunnel connect/disconnect.+func (s *EnvironmentService) UpdateEnvironmentConnectionState(ctx context.Context, id string, connected bool) error {+ now := time.Now()++ updates := map[string]any{+ "updated_at": &now,+ }+ if connected {+ updates["status"] = string(models.EnvironmentStatusOnline)+ updates["last_seen"] = &now+ // Remember the tunnel transport so the UI can keep showing it after+ // the tunnel drops or while the agent is poll-only.+ if state, ok := edge.GetTunnelRuntimeState(id).Get(); ok && state.Transport != "" {+ updates["last_edge_transport"] = state.Transport+ }+ } else {+ updates["status"] = string(models.EnvironmentStatusOffline)+ }++ if err := s.db.WithContext(ctx).Model(&models.Environment{}).Where("id = ?", id).Updates(updates).Error; err != nil {+ return errors.WrapIf(err, "failed to update environment connection state")+ }++ s.NotifyRuntimeStateChanged()++ return nil+}++// ApplyEnvironmentRuntimeState normalizes edge environment runtime status using+// in-memory tunnel and poll registries without mutating persisted state.+func ApplyEnvironmentRuntimeState(env *environment.Environment) {+ if env == nil || !env.IsEdge {+ return+ }++ connected := false+ env.Connected = &connected+ env.ConnectedAt = nil+ env.LastHeartbeat = nil+ env.LastPollAt = nil+ env.EdgeTransport = nil+ env.EdgeSecurityMode = nil+ env.EdgeSessionID = nil+ env.EdgeAgentInstance = nil+ env.EdgeCapabilities = nil++ if pollState, ok := edge.GetPollRuntimeRegistry().Get(env.ID, time.Now()).Get(); ok {+ env.LastPollAt = pollState.LastPollAt+ }++ if runtimeState, ok := edge.GetTunnelRuntimeState(env.ID).Get(); ok {+ connected = true+ env.Connected = &connected+ env.Status = string(models.EnvironmentStatusOnline)+ env.ConnectedAt = runtimeState.ConnectedAt+ env.LastHeartbeat = runtimeState.LastHeartbeat+ if runtimeState.SecurityMode != "" {+ env.EdgeSecurityMode = &runtimeState.SecurityMode+ }+ if runtimeState.SessionID != "" {+ env.EdgeSessionID = &runtimeState.SessionID+ }+ if runtimeState.AgentInstance != "" {+ env.EdgeAgentInstance = &runtimeState.AgentInstance+ }+ if len(runtimeState.Capabilities) > 0 {+ env.EdgeCapabilities = append([]string(nil), runtimeState.Capabilities...)+ }+ if transport, ok := edge.GetActiveTunnelTransport(env.ID).Get(); ok {+ env.EdgeTransport = &transport+ } else if runtimeState.Transport != "" {+ env.EdgeTransport = &runtimeState.Transport+ }+ return+ }++ if env.LastPollAt != nil {+ env.Status = string(models.EnvironmentStatusStandby)+ return+ }++ if env.Status != string(models.EnvironmentStatusPending) {+ env.Status = string(models.EnvironmentStatusOffline)+ }+}++// ReconcileEdgeStatusesOnStartup resets edge environments to offline when the manager starts.+// Live edge tunnels are process-local runtime state, so persisted "online" flags can be stale+// after a restart until agents reconnect. Pending environments are left untouched.+func (s *EnvironmentService) ReconcileEdgeStatusesOnStartup(ctx context.Context) error {+ result := s.db.WithContext(ctx).Model(&models.Environment{}).+ Where("is_edge = ?", true).+ Where("status <> ?", string(models.EnvironmentStatusPending)).+ Where("status <> ?", string(models.EnvironmentStatusOffline)).+ Updates(map[string]any{+ "status": string(models.EnvironmentStatusOffline),+ "updated_at": new(time.Now()),+ })+ if result.Error != nil {+ return errors.WrapIf(result.Error, "failed to reconcile edge environment statuses")+ }++ if result.RowsAffected > 0 {+ slog.InfoContext(ctx, "Reconciled stale edge environment statuses on startup", "count", result.RowsAffected)+ }++ return nil+}diff --git a/backend/internal/environment/environment_listing.go b/backend/internal/environment/environment_listing.gonew file mode 100644index 000000000..57c3b1ad4--- /dev/null+++ b/backend/internal/environment/environment_listing.go@@ -0,0 +1,250 @@+package environment++import (+ "context"+ "strings"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper"+ "github.com/getarcaneapp/arcane/types/v2/environment"+)++func (s *EnvironmentService) ListEnvironmentsPaginated(ctx context.Context, params pagination.QueryParams, accessibleEnvIDs []string) ([]environment.Environment, pagination.Response, error) {+ if strings.TrimSpace(params.Filters["type"]) != "" {+ return s.listEnvironmentsPaginatedWithRuntimeFiltersInternal(ctx, params, accessibleEnvIDs)+ }++ var envs []models.Environment+ q := s.db.WithContext(ctx).Model(&models.Environment{}).Where("hidden = ?", false)+ // accessibleEnvIDs == nil means "no restriction". A non-nil slice limits the+ // result to those environment IDs; an empty slice therefore matches nothing.+ switch {+ case accessibleEnvIDs == nil:+ // no restriction+ case len(accessibleEnvIDs) == 0:+ q = q.Where("1 = 0")+ default:+ q = q.Where("id IN ?", accessibleEnvIDs)+ }++ if term := strings.TrimSpace(params.Search); term != "" {+ searchPattern := "%" + term + "%"+ q = q.Where(+ "name LIKE ? OR api_url LIKE ?",+ searchPattern, searchPattern,+ )+ }++ q = pagination.ApplyFilter(q, "status", params.Filters["status"])+ q = pagination.ApplyBooleanFilter(q, "enabled", params.Filters["enabled"])++ paginationResp, err := pagination.PaginateAndSortDB(params, q, &envs)+ if err != nil {+ return nil, pagination.Response{}, errors.WrapIf(err, "failed to paginate environments")+ }++ out, mapErr := mapper.MapSlice[models.Environment, environment.Environment](envs)+ if mapErr != nil {+ return nil, pagination.Response{}, errors.WrapIf(mapErr, "failed to map environments")+ }++ return out, paginationResp, nil+}++// filterEnvironmentsByIDInternal returns only the environments whose ID is in+// allowedIDs. A non-nil but empty allowedIDs yields an empty slice. Used to+// restrict the runtime-filtered list path to a caller's accessible environments.+func filterEnvironmentsByIDInternal(items []environment.Environment, allowedIDs []string) []environment.Environment {+ allowed := make(map[string]struct{}, len(allowedIDs))+ for _, id := range allowedIDs {+ allowed[id] = struct{}{}+ }+ out := make([]environment.Environment, 0, len(items))+ for _, item := range items {+ if _, ok := allowed[item.ID]; ok {+ out = append(out, item)+ }+ }+ return out+}++func (s *EnvironmentService) listEnvironmentsPaginatedWithRuntimeFiltersInternal(ctx context.Context, params pagination.QueryParams, accessibleEnvIDs []string) ([]environment.Environment, pagination.Response, error) {+ var envs []models.Environment+ if err := s.db.WithContext(ctx).+ Model(&models.Environment{}).+ Where("hidden = ?", false).+ Find(&envs).Error; err != nil {+ return nil, pagination.Response{}, errors.WrapIf(err, "failed to list environments")+ }++ items, mapErr := mapper.MapSlice[models.Environment, environment.Environment](envs)+ if mapErr != nil {+ return nil, pagination.Response{}, errors.WrapIf(mapErr, "failed to map environments")+ }++ // nil = no restriction; non-nil restricts to the caller's accessible envs.+ if accessibleEnvIDs != nil {+ items = filterEnvironmentsByIDInternal(items, accessibleEnvIDs)+ }++ for i := range items {+ ApplyEnvironmentRuntimeState(&items[i])+ }++ config := pagination.Config[environment.Environment]{+ SearchAccessors: []pagination.SearchAccessor[environment.Environment]{+ func(env environment.Environment) (string, error) { return env.Name, nil },+ func(env environment.Environment) (string, error) { return env.ApiUrl, nil },+ },+ SortBindings: []pagination.SortBinding[environment.Environment]{+ {+ Key: "id",+ Fn: func(a, b environment.Environment) int {+ return strings.Compare(strings.ToLower(a.ID), strings.ToLower(b.ID))+ },+ },+ {+ Key: "name",+ Fn: func(a, b environment.Environment) int {+ return strings.Compare(strings.ToLower(a.Name), strings.ToLower(b.Name))+ },+ },+ {+ Key: "status",+ Fn: func(a, b environment.Environment) int {+ return strings.Compare(strings.ToLower(a.Status), strings.ToLower(b.Status))+ },+ },+ {+ Key: "enabled",+ Fn: func(a, b environment.Environment) int {+ if a.Enabled == b.Enabled {+ return 0+ }+ if a.Enabled {+ return 1+ }+ return -1+ },+ },+ {+ Key: "apiUrl",+ Fn: func(a, b environment.Environment) int {+ return strings.Compare(strings.ToLower(a.ApiUrl), strings.ToLower(b.ApiUrl))+ },+ },+ },+ FilterAccessors: []pagination.FilterAccessor[environment.Environment]{+ {+ Key: "status",+ Fn: func(item environment.Environment, filterValue string) bool {+ return strings.EqualFold(item.Status, strings.TrimSpace(filterValue))+ },+ },+ {+ Key: "enabled",+ Fn: func(item environment.Environment, filterValue string) bool {+ switch strings.ToLower(strings.TrimSpace(filterValue)) {+ case "true", "1":+ return item.Enabled+ case "false", "0":+ return !item.Enabled+ default:+ return true+ }+ },+ },+ {+ Key: "type",+ Fn: environmentTypeMatchesInternal,+ },+ },+ }++ result := pagination.SearchOrderAndPaginate(items, params, config)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)++ return result.Items, paginationResp, nil+}++func environmentTypeMatchesInternal(env environment.Environment, filterValue string) bool {+ return environmentTypeKeyInternal(env) == strings.ToLower(strings.TrimSpace(filterValue))+}++func environmentTypeKeyInternal(env environment.Environment) string {+ if !env.IsEdge {+ return "http"+ }+ transport := ""+ if env.Connected != nil && *env.Connected && env.EdgeTransport != nil {+ transport = *env.EdgeTransport+ } else if env.LastEdgeTransport != nil {+ // Disconnected or poll-only agents classify by the transport they+ // last used rather than collapsing into the generic edge bucket.+ transport = *env.LastEdgeTransport+ }+ switch strings.ToLower(strings.TrimSpace(transport)) {+ case edge.EdgeTransportWebSocket:+ return "websocket"+ case edge.EdgeTransportGRPC:+ return "grpc"+ default:+ return "edge"+ }+}++func (s *EnvironmentService) ListVisibleEnvironments(ctx context.Context) ([]environment.Environment, error) {+ var envs []models.Environment+ if err := s.db.WithContext(ctx).+ Model(&models.Environment{}).+ Where("hidden = ?", false).+ Order("created_at asc, id asc").+ Find(&envs).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to list visible environments")+ }++ out, mapErr := mapper.MapSlice[models.Environment, environment.Environment](envs)+ if mapErr != nil {+ return nil, errors.WrapIf(mapErr, "failed to map environments")+ }++ for i := range out {+ ApplyEnvironmentRuntimeState(&out[i])+ }++ return out, nil+}++// ListRemoteEnvironmentIDs returns the IDs of enabled remote environments; it+// satisfies agg.RemoteEnvironmentLister for aggregated stream handlers.+func (s *EnvironmentService) ListRemoteEnvironmentIDs(ctx context.Context) ([]string, error) {+ envs, err := s.ListRemoteEnvironments(ctx)+ if err != nil {+ return nil, err+ }+ ids := make([]string, 0, len(envs))+ for _, env := range envs {+ ids = append(ids, env.ID)+ }+ return ids, nil+}++// ListRemoteEnvironments returns all non-local, enabled environments for syncing purposes.+func (s *EnvironmentService) ListRemoteEnvironments(ctx context.Context) ([]models.Environment, error) {+ var envs []models.Environment+ err := s.db.WithContext(ctx).+ Model(&models.Environment{}).+ Where("id != ?", "0").+ Where("enabled = ?", true).+ Where("hidden = ?", false).+ Find(&envs).Error+ if err != nil {+ return nil, errors.WrapIf(err, "failed to list remote environments")+ }+ s.remoteEnvs.replace(envs)+ return envs, nil+}diff --git a/backend/internal/environment/environment_proxy.go b/backend/internal/environment/environment_proxy.gonew file mode 100644index 000000000..8b4201ff2--- /dev/null+++ b/backend/internal/environment/environment_proxy.go@@ -0,0 +1,445 @@+package environment++import (+ "context"+ "encoding/json/v2"+ "log/slog"+ "maps"+ "net/http"+ "strings"+ "time"++ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/remenv"+ pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ httputils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"+ "github.com/getarcaneapp/arcane/types/v2/containerregistry"+ "github.com/getarcaneapp/arcane/types/v2/gitops"+ "go.getarcane.app/sys/crypto"+)++// SyncRegistriesToRemoteEnvironments syncs container registries to all eligible remote environments.+// Eligibility requires a non-local, enabled environment with a configured access token.+func (s *EnvironmentService) SyncRegistriesToRemoteEnvironments(ctx context.Context) error {+ envs, err := s.ListRemoteEnvironments(ctx)+ if err != nil {+ return errors.WrapIf(err, "failed to list remote environments for registry sync")+ }++ if len(envs) == 0 {+ return nil+ }++ var failedCount int+ for _, env := range envs {+ if env.AccessToken == nil || *env.AccessToken == "" {+ slog.DebugContext(ctx, "Skipping registry sync for environment without access token",+ "environmentID", env.ID,+ "environmentName", env.Name)+ continue+ }++ if err := s.SyncRegistriesToEnvironment(ctx, env.ID); err != nil {+ failedCount+++ slog.WarnContext(ctx, "Failed to sync registries to remote environment",+ "environmentID", env.ID,+ "environmentName", env.Name,+ "error", err.Error())+ }+ }++ if failedCount > 0 {+ return errors.Errorf("failed to sync registries to %d remote environment(s)", failedCount)+ }++ return nil+}++type remoteEnvironmentTargetInternal struct {+ ID string+ Name string+ IsEdge bool+ AccessToken *string+ TargetURL string+}++func (s *EnvironmentService) resolveRemoteEnvironmentTargetInternal(ctx context.Context, envID string) (*remoteEnvironmentTargetInternal, error) {+ envRecord, err := s.GetEnvironmentByID(ctx, envID)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to get environment")+ }++ return s.remoteEnvironmentTargetFromModelInternal(*envRecord)+}++func (s *EnvironmentService) remoteEnvironmentTargetFromModelInternal(environment models.Environment) (*remoteEnvironmentTargetInternal, error) {+ if environment.ID == "0" {+ return nil, errors.New("cannot proxy request to local environment")+ }++ targetURL := strings.TrimRight(environment.ApiUrl, "/")+ if !environment.IsEdge {+ validatedTargetURL, err := httputils.NormalizeBaseURL(environment.ApiUrl)+ if err != nil {+ return nil, errors.WrapIf(err, "invalid environment API URL")+ }+ targetURL = validatedTargetURL+ }++ return &remoteEnvironmentTargetInternal{+ ID: environment.ID,+ Name: environment.Name,+ IsEdge: environment.IsEdge,+ AccessToken: environment.AccessToken,+ TargetURL: targetURL,+ }, nil+}++func buildEnvironmentEndpointURLInternal(apiURL, endpointPath string) (string, error) {+ baseURL, err := httputils.NormalizeBaseURL(apiURL)+ if err != nil {+ return "", err+ }++ return strings.TrimRight(baseURL, "/") + endpointPath, nil+}++func (s *EnvironmentService) getProxyRequestContextInternal(ctx context.Context) (context.Context, context.CancelFunc) {+ if s != nil && s.settingsService != nil {+ settings := s.settingsService.GetSettingsConfig()+ return context.WithTimeout(ctx, timeouts.GetDuration(settings.ProxyRequestTimeout.AsInt(), timeouts.DefaultProxyRequest))+ }++ return context.WithTimeout(ctx, timeouts.DefaultProxyRequest)+}++func (s *EnvironmentService) buildRemoteRequestInternal(+ target *remoteEnvironmentTargetInternal,+ method string,+ path string,+ body []byte,+ headers map[string]string,+) (remenv.Request, error) {+ if target == nil {+ return remenv.Request{}, errors.New("remote environment target is required")+ }++ requestHeaders := make(map[string]string, len(headers)+2)+ maps.Copy(requestHeaders, headers)+ if len(body) > 0 && method != http.MethodGet && requestHeaders["Content-Type"] == "" {+ requestHeaders["Content-Type"] = "application/json"+ }+ remenv.ApplyAgentTokenHeaderMap(requestHeaders, target.AccessToken)++ return remenv.Request{+ EnvironmentID: target.ID,+ IsEdge: target.IsEdge,+ Method: method,+ URL: target.TargetURL + path,+ Path: path,+ Headers: requestHeaders,+ Body: body,+ }, nil+}++func (s *EnvironmentService) ExecuteRemoteRequest(ctx context.Context, envID string, method string, path string, body []byte) (*remenv.Response, error) {+ target, err := s.resolveRemoteEnvironmentTargetInternal(ctx, envID)+ if err != nil {+ return nil, err+ }++ return s.executeRemoteRequestForTargetInternal(ctx, target, method, path, body)+}++func (s *EnvironmentService) executeRemoteRequestForTargetInternal(+ ctx context.Context,+ target *remoteEnvironmentTargetInternal,+ method string,+ path string,+ body []byte,+) (*remenv.Response, error) {+ // Forward the activity batch ID so bulk actions proxied to a remote+ // environment group the same way they do locally.+ var headers map[string]string+ if batchID := pkgutils.ActivityBatchIDFromContext(ctx); batchID != "" {+ headers = map[string]string{pkgutils.HeaderActivityBatchID: batchID}+ }+ request, err := s.buildRemoteRequestInternal(target, method, path, body, headers)+ if err != nil {+ return nil, err+ }++ resp, err := s.remoteClient.Do(ctx, request)+ if err != nil {+ return nil, errors.WrapIff(err, "failed to send request to environment %s", target.Name)+ }++ return resp, nil+}++func (s *EnvironmentService) ProxyJSONRequest(ctx context.Context, envID string, method string, path string, body []byte, out any) error {+ proxyCtx, cancel := s.getProxyRequestContextInternal(ctx)+ defer cancel()++ target, err := s.resolveRemoteEnvironmentTargetInternal(proxyCtx, envID)+ if err != nil {+ return err+ }++ return s.proxyJSONRequestForTargetInternal(proxyCtx, target, method, path, body, out)+}++// ProxyJSONRequestForEnvironment sends a JSON request using an already-loaded+// environment row, avoiding an extra environment lookup on hot stream paths.+func (s *EnvironmentService) ProxyJSONRequestForEnvironment(ctx context.Context, environment models.Environment, method string, path string, body []byte, out any) error {+ proxyCtx, cancel := s.getProxyRequestContextInternal(ctx)+ defer cancel()++ target, err := s.remoteEnvironmentTargetFromModelInternal(environment)+ if err != nil {+ return err+ }++ return s.proxyJSONRequestForTargetInternal(proxyCtx, target, method, path, body, out)+}++func (s *EnvironmentService) proxyJSONRequestForTargetInternal(+ ctx context.Context,+ target *remoteEnvironmentTargetInternal,+ method string,+ path string,+ body []byte,+ out any,+) error {+ resp, err := s.executeRemoteRequestForTargetInternal(ctx, target, method, path, body)+ if err != nil {+ return err+ }+ if err := resp.RequireSuccess(); err != nil {+ return err+ }+ if err := resp.DecodeJSON(out); err != nil {+ return err+ }++ return nil+}++func ensureRemoteEnvironmentTunnelAvailableInternal(ctx context.Context, envID string) error {+ if edge.HasActiveTunnel(envID) {+ return nil+ }++ if _, ok := edge.RequestTunnelAndWait(ctx, envID, edge.DefaultTunnelDemandTTL, edge.DefaultTunnelAcquireTimeout()).Get(); ok {+ return nil+ }++ return errors.New("edge agent is not connected")+}++func doRemoteEnvironmentTunnelRequestInternal(+ ctx context.Context,+ envID string,+ method string,+ path string,+ headers map[string]string,+ body []byte,+) (*remenv.Response, error) {+ tunnel, ok := edge.GetRegistry().Get(envID).Get()+ if !ok {+ return nil, errors.Errorf("no active tunnel for environment %s", envID)+ }+ if tunnel.Conn.IsClosed() {+ return nil, errors.Errorf("tunnel for environment %s is closed", envID)+ }++ statusCode, respHeaders, respBody, err := edge.ProxyRequest(ctx, tunnel, method, path, "", headers, body)+ if err != nil {+ return nil, errors.WrapIf(err, "tunnel request failed")+ }++ return &remenv.Response{+ StatusCode: statusCode,+ Body: respBody,+ Headers: respHeaders,+ }, nil+}++// SyncRegistriesToEnvironment syncs all registries from this manager to a remote environment+func (s *EnvironmentService) SyncRegistriesToEnvironment(ctx context.Context, environmentID string) error {+ return fanOutSyncToEnvironmentInternal(ctx, s, environmentID, "registries", "/api/container-registries/sync",+ func(ctx context.Context, reg models.ContainerRegistry) (containerregistry.Sync, bool, error) {+ registryType, typeErr := registry.NormalizeRegistryType(reg.RegistryType)+ if typeErr != nil {+ return containerregistry.Sync{}, false, errors.WrapIff(typeErr, "normalize registry type for sync %s", reg.ID)+ }++ syncItem := containerregistry.Sync{+ ID: reg.ID,+ URL: reg.URL,+ Description: reg.Description,+ Insecure: reg.Insecure,+ Enabled: reg.Enabled,+ RegistryType: registryType,+ RepositoryNames: reg.RepositoryNames,+ CreatedAt: reg.CreatedAt,+ UpdatedAt: reg.UpdatedAt,+ }++ if registryType == registry.RegistryTypeECR {+ decryptedSecret, err := crypto.Decrypt(reg.AWSSecretAccessKey)+ if err != nil {+ slog.WarnContext(ctx, "Failed to decrypt ECR secret for sync", "registryID", reg.ID, "registryURL", reg.URL, "error", err.Error())+ return containerregistry.Sync{}, false, nil+ }++ syncItem.AWSAccessKeyID = reg.AWSAccessKeyID+ syncItem.AWSSecretAccessKey = decryptedSecret+ syncItem.AWSRegion = reg.AWSRegion+ } else {+ decryptedToken, err := crypto.Decrypt(reg.Token)+ if err != nil {+ slog.WarnContext(ctx, "Failed to decrypt registry token for sync", "registryID", reg.ID, "registryURL", reg.URL, "error", err.Error())+ return containerregistry.Sync{}, false, nil+ }++ syncItem.Username = reg.Username+ syncItem.Token = decryptedToken+ }++ return syncItem, true, nil+ },+ func(items []containerregistry.Sync) containerregistry.SyncRequest {+ return containerregistry.SyncRequest{Registries: items}+ },+ )+}++// SyncRepositoriesToEnvironment syncs all git repositories from this manager to a remote environment+func (s *EnvironmentService) SyncRepositoriesToEnvironment(ctx context.Context, environmentID string) error {+ return fanOutSyncToEnvironmentInternal(ctx, s, environmentID, "git repositories", "/api/git-repositories/sync",+ func(ctx context.Context, repo models.GitRepository) (gitops.RepositorySync, bool, error) {+ item := gitops.RepositorySync{+ ID: repo.ID,+ Name: repo.Name,+ URL: repo.URL,+ AuthType: repo.AuthType,+ Username: repo.Username,+ Description: repo.Description,+ Enabled: repo.Enabled,+ CreatedAt: repo.CreatedAt,+ }+ if repo.UpdatedAt != nil {+ item.UpdatedAt = *repo.UpdatedAt+ }++ if repo.Token != "" {+ decryptedToken, err := crypto.Decrypt(repo.Token)+ if err != nil {+ slog.WarnContext(ctx, "Failed to decrypt repository token for sync", "repositoryID", repo.ID, "repositoryName", repo.Name, "error", err.Error())+ return gitops.RepositorySync{}, false, nil+ }+ item.Token = decryptedToken+ }++ if repo.SSHKey != "" {+ decryptedSSHKey, err := crypto.Decrypt(repo.SSHKey)+ if err != nil {+ slog.WarnContext(ctx, "Failed to decrypt repository SSH key for sync", "repositoryID", repo.ID, "repositoryName", repo.Name, "error", err.Error())+ return gitops.RepositorySync{}, false, nil+ }+ item.SSHKey = decryptedSSHKey+ }++ return item, true, nil+ },+ func(items []gitops.RepositorySync) gitops.RepositorySyncRequest {+ return gitops.RepositorySyncRequest{Repositories: items}+ },+ )+}++// fanOutSyncToEnvironmentInternal pushes every row of Model held by this+// manager to one remote environment. toSyncItem maps a row to its wire form and+// reports whether to keep it — rows whose credentials fail to decrypt are+// skipped rather than failing the whole batch — and wrap builds the request+// envelope the target endpoint expects.+func fanOutSyncToEnvironmentInternal[Model any, Item any, Request any](+ ctx context.Context,+ s *EnvironmentService,+ environmentID string,+ kind string,+ path string,+ toSyncItem func(context.Context, Model) (Item, bool, error),+ wrap func([]Item) Request,+) error {+ target, err := s.resolveRemoteEnvironmentTargetInternal(ctx, environmentID)+ if err != nil {+ return err+ }++ slog.InfoContext(ctx, "Starting sync to environment", "kind", kind, "environmentID", environmentID, "environmentName", target.Name, "apiUrl", target.TargetURL)++ var records []Model+ if err := s.db.WithContext(ctx).Find(&records).Error; err != nil {+ return errors.WrapIff(err, "failed to get %s", kind)+ }++ syncItems := make([]Item, 0, len(records))+ for _, record := range records {+ item, keep, err := toSyncItem(ctx, record)+ if err != nil {+ return err+ }+ if keep {+ syncItems = append(syncItems, item)+ }+ }++ reqBody, err := json.Marshal(wrap(syncItems))+ if err != nil {+ return errors.WrapIf(err, "failed to marshal sync request")+ }++ reqCtx, cancel := context.WithTimeout(ctx, 30*time.Second)+ defer cancel()++ slog.InfoContext(ctx, "Sending sync request to agent", "kind", kind, "url", target.TargetURL+path, "count", len(syncItems), "isEdge", target.IsEdge)++ var result struct {+ Success bool `json:"success"`+ Data struct {+ Message string `json:"message"`+ } `json:"data"`+ }+ if err := s.proxyJSONRequestForTargetInternal(reqCtx, target, http.MethodPost, path, reqBody, &result); err != nil {+ return errors.WrapIf(err, "failed to send sync request")+ }++ if !result.Success {+ return errors.Errorf("sync failed: %s", result.Data.Message)+ }++ slog.InfoContext(ctx, "Successfully synced to environment", "kind", kind, "environmentID", environmentID, "environmentName", target.Name)++ return nil+}++// ProxyRequest sends a request to a remote environment's API.+func (s *EnvironmentService) ProxyRequest(ctx context.Context, envID string, method string, path string, body []byte) ([]byte, int, error) {+ proxyCtx, cancel := s.getProxyRequestContextInternal(ctx)+ defer cancel()++ resp, err := s.ExecuteRemoteRequest(proxyCtx, envID, method, path, body)+ if err != nil {+ return nil, 0, err+ }++ return resp.Body, resp.StatusCode, nil+}diff --git a/backend/internal/environment/environment_runtime_notify.go b/backend/internal/environment/environment_runtime_notify.gonew file mode 100644index 000000000..3aac08de3--- /dev/null+++ b/backend/internal/environment/environment_runtime_notify.go@@ -0,0 +1,58 @@+package environment++// subscribe returns a channel that receives a coalesced wake-up whenever+// environment liveness may have changed, plus a function to release it. The+// channel is never closed; callers select on it alongside their own context.+func (w *runtimeWatchersInternal) subscribe() (<-chan struct{}, func()) {+ w.mu.Lock()+ defer w.mu.Unlock()++ if w.chans == nil {+ w.chans = make(map[int]chan struct{})+ }++ w.seq+++ id := w.seq+ // Capacity 1: a second signal arriving before the watcher wakes is+ // redundant, because the watcher reads live state rather than a queue.+ ch := make(chan struct{}, 1)+ w.chans[id] = ch++ return ch, func() {+ w.mu.Lock()+ defer w.mu.Unlock()+ delete(w.chans, id)+ }+}++// notify wakes every watcher. It never blocks, so it is safe to call from+// connection callbacks on the tunnel hot path.+func (w *runtimeWatchersInternal) notify() {+ if w == nil {+ return+ }++ w.mu.Lock()+ defer w.mu.Unlock()++ for _, ch := range w.chans {+ select {+ case ch <- struct{}{}:+ default:+ }+ }+}++// SubscribeRuntimeChanges returns a channel that receives a coalesced wake-up+// whenever environment liveness may have changed, plus a function to release it.+func (s *EnvironmentService) SubscribeRuntimeChanges() (<-chan struct{}, func()) {+ return s.runtimeWatchers.subscribe()+}++// NotifyRuntimeStateChanged wakes every runtime watcher.+func (s *EnvironmentService) NotifyRuntimeStateChanged() {+ if s == nil {+ return+ }+ s.runtimeWatchers.notify()+}diff --git a/backend/internal/environment/environment_snippets.go b/backend/internal/environment/environment_snippets.gonew file mode 100644index 000000000..7419e66ba--- /dev/null+++ b/backend/internal/environment/environment_snippets.go@@ -0,0 +1,245 @@+package environment++import (+ "context"+ "fmt"+ "log/slog"+ "strings"++ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge"+)++// DeploymentSnippets contains deployment configuration snippets for an environment.+type DeploymentSnippets struct {+ DockerRun string+ DockerCompose string+ MTLS *DeploymentSnippetMTLS+}++type DeploymentSnippetFile struct {+ Name string `json:"name" doc:"Suggested filename"`+ Content string `json:"content,omitempty" doc:"PEM file contents. Omitted for sensitive files such as private keys; use downloadUrl instead."`+ DownloadURL string `json:"downloadUrl,omitempty" doc:"Pairing-permission endpoint to download this file when content is withheld"`+ Sensitive bool `json:"sensitive,omitempty" doc:"True when this file is sensitive and must be fetched via downloadUrl"`+ ContainerPath string `json:"containerPath" doc:"Container mount path expected by the mTLS snippet"`+ Permissions string `json:"permissions" doc:"Suggested file mode"`+}++type DeploymentSnippetMTLS struct {+ DockerRun string `json:"dockerRun" doc:"Docker run snippet using Arcane-generated mTLS assets"`+ DockerCompose string `json:"dockerCompose" doc:"Docker compose snippet using Arcane-generated mTLS assets"`+ Files []DeploymentSnippetFile `json:"files" doc:"Generated PEM files to place on the edge host"`+ HostDirHint string `json:"hostDirHint" doc:"Suggested host directory containing the generated PEM files"`+}++const (+ deploymentSnippetsDataPath = "/app/data"+ deploymentSnippetsMTLSPath = "/app/data/edge-mtls-agent"+)++// GenerateDeploymentSnippets generates Docker deployment snippets for an environment.+func (s *EnvironmentService) GenerateDeploymentSnippets(ctx context.Context, envID string, envAddress string, apiKey string) (*DeploymentSnippets, error) {+ managerURL := strings.TrimRight(envAddress, "/")++ dockerRun := strings.Join([]string{+ "docker run -d \\",+ " --name arcane-agent \\",+ " --restart unless-stopped \\",+ " -e AGENT_MODE=true \\",+ " -e EDGE_TRANSPORT=poll \\",+ fmt.Sprintf(" -e AGENT_TOKEN=%s \\", apiKey),+ fmt.Sprintf(" -e MANAGER_API_URL=%s \\", managerURL),+ " -p 3553:3553 \\",+ " -v /var/run/docker.sock:/var/run/docker.sock \\",+ fmt.Sprintf(" -v arcane-data:%s \\", deploymentSnippetsDataPath),+ " ghcr.io/getarcaneapp/agent:latest",+ }, "\n")++ dockerCompose := strings.Join([]string{+ "services:",+ " arcane-agent:",+ " image: ghcr.io/getarcaneapp/agent:latest",+ " container_name: arcane-agent",+ " restart: unless-stopped",+ " environment:",+ " - AGENT_MODE=true",+ " - EDGE_TRANSPORT=poll",+ " - AGENT_TOKEN=" + apiKey,+ " - MANAGER_API_URL=" + managerURL,+ " ports:",+ " - \"3553:3553\"",+ " volumes:",+ " - /var/run/docker.sock:/var/run/docker.sock",+ " - arcane-data:" + deploymentSnippetsDataPath,+ "",+ "volumes:",+ " arcane-data:",+ }, "\n")++ return &DeploymentSnippets{+ DockerRun: dockerRun,+ DockerCompose: dockerCompose,+ }, nil+}++// GenerateEdgeDeploymentSnippets generates Docker deployment snippets for an edge agent.+// Edge agents connect outbound to the manager and don't require exposed ports.+func (s *EnvironmentService) GenerateEdgeDeploymentSnippets(ctx context.Context, envID string, managerURL string, apiKey string, edgeCfg *edge.Config) (*DeploymentSnippets, error) {+ managerURL = strings.TrimRight(managerURL, "/")++ dockerRun := strings.Join([]string{+ "docker run -d \\",+ " --name arcane-edge-agent \\",+ " --restart unless-stopped \\",+ " -e EDGE_AGENT=true \\",+ " -e EDGE_TRANSPORT=poll \\",+ fmt.Sprintf(" -e AGENT_TOKEN=%s \\", apiKey),+ fmt.Sprintf(" -e MANAGER_API_URL=%s \\", managerURL),+ " -v /var/run/docker.sock:/var/run/docker.sock \\",+ fmt.Sprintf(" -v arcane-data:%s \\", deploymentSnippetsDataPath),+ " ghcr.io/getarcaneapp/agent:latest",+ }, "\n")++ dockerCompose := strings.Join([]string{+ "# Edge agent - connects outbound, no exposed ports required",+ "services:",+ " arcane-edge-agent:",+ " image: ghcr.io/getarcaneapp/agent:latest",+ " container_name: arcane-edge-agent",+ " restart: unless-stopped",+ " environment:",+ " - EDGE_AGENT=true",+ " - EDGE_TRANSPORT=poll",+ " - AGENT_TOKEN=" + apiKey,+ " - MANAGER_API_URL=" + managerURL,+ " volumes:",+ " - /var/run/docker.sock:/var/run/docker.sock",+ " - arcane-data:" + deploymentSnippetsDataPath,+ "",+ "volumes:",+ " arcane-data:",+ }, "\n")++ snippets := &DeploymentSnippets{+ DockerRun: dockerRun,+ DockerCompose: dockerCompose,+ }++ envName := ""+ if s != nil && s.db != nil {+ if env, getErr := s.GetEnvironmentByID(ctx, envID); getErr == nil && env != nil {+ envName = env.Name+ }+ }+ if edgeCfg != nil && strings.TrimSpace(edgeCfg.AppURL) == "" {+ edgeCfg.AppURL = managerURL+ }++ generatedAssets, err := edge.GenerateManagerClientMTLSAssetsWithContext(ctx, edgeCfg, envID, envName)+ if err != nil {+ slog.WarnContext(ctx, "Failed to generate edge mTLS assets; returning basic snippets only", "environment_id", envID, "error", err)+ return snippets, nil+ }+ if generatedAssets == nil {+ return snippets, nil+ }+ s.logGeneratedMTLSEventsInternal(ctx, envID, envName, generatedAssets)++ snippets.MTLS = buildMTLSDeploymentSnippetInternal(managerURL, apiKey, generatedAssets)+ return snippets, nil+}++func (s *EnvironmentService) logGeneratedMTLSEventsInternal(ctx context.Context, envID string, envName string, assets *edge.GeneratedMTLSAssets) {+ if s == nil || s.eventService == nil || assets == nil {+ return+ }+ if assets.CAGenerated {+ if _, err := s.eventService.CreateEvent(ctx, event.CreateEventRequest{+ Type: models.EventTypeEnvironmentMTLSCAGenerated,+ Severity: models.EventSeverityInfo,+ Title: "Edge mTLS CA generated",+ Description: "Arcane generated a new edge mTLS certificate authority",+ Metadata: models.JSON{"kind": "ca"},+ }); err != nil {+ slog.WarnContext(ctx, "Failed to create edge mTLS CA generation event", "error", err)+ }+ }+ if assets.CertIssued {+ envIDCopy := envID+ if _, err := s.eventService.CreateEvent(ctx, event.CreateEventRequest{+ Type: models.EventTypeEnvironmentMTLSCertIssued,+ Severity: models.EventSeverityInfo,+ Title: "Edge mTLS certificate issued",+ Description: fmt.Sprintf("Arcane issued an edge mTLS client certificate for environment '%s'", envName),+ ResourceType: new("environment"),+ ResourceID: &envIDCopy,+ ResourceName: new(envName),+ EnvironmentID: &envIDCopy,+ Metadata: models.JSON{"kind": "client"},+ }); err != nil {+ slog.WarnContext(ctx, "Failed to create edge mTLS certificate issuance event", "environment_id", envID, "error", err)+ }+ }+}++func buildMTLSDeploymentSnippetInternal(managerURL string, apiKey string, generatedAssets *edge.GeneratedMTLSAssets) *DeploymentSnippetMTLS {+ if generatedAssets == nil {+ return nil+ }++ mtlsDockerRun := strings.Join([]string{+ "docker run -d \\",+ " --name arcane-edge-agent \\",+ " --restart unless-stopped \\",+ " -e EDGE_AGENT=true \\",+ " -e EDGE_TRANSPORT=poll \\",+ " -e EDGE_MTLS_MODE=required \\",+ fmt.Sprintf(" -e EDGE_MTLS_ASSETS_DIR=%s \\", deploymentSnippetsMTLSPath),+ fmt.Sprintf(" -e AGENT_TOKEN=%s \\", apiKey),+ fmt.Sprintf(" -e MANAGER_API_URL=%s \\", managerURL),+ " -v /var/run/docker.sock:/var/run/docker.sock \\",+ fmt.Sprintf(" -v arcane-data:%s \\", deploymentSnippetsDataPath),+ " ghcr.io/getarcaneapp/agent:latest",+ }, "\n")++ mtlsDockerCompose := strings.Join([]string{+ "# Edge agent with automatic mTLS enrollment",+ "services:",+ " arcane-edge-agent:",+ " image: ghcr.io/getarcaneapp/agent:latest",+ " container_name: arcane-edge-agent",+ " restart: unless-stopped",+ " environment:",+ " - EDGE_AGENT=true",+ " - EDGE_TRANSPORT=poll",+ " - EDGE_MTLS_MODE=required",+ " - EDGE_MTLS_ASSETS_DIR=" + deploymentSnippetsMTLSPath,+ " - AGENT_TOKEN=" + apiKey,+ " - MANAGER_API_URL=" + managerURL,+ " volumes:",+ " - /var/run/docker.sock:/var/run/docker.sock",+ " - arcane-data:" + deploymentSnippetsDataPath,+ "",+ "volumes:",+ " arcane-data:",+ }, "\n")++ files := make([]DeploymentSnippetFile, 0, len(generatedAssets.Files))+ for _, file := range generatedAssets.Files {+ files = append(files, DeploymentSnippetFile{+ Name: file.Name,+ Content: file.Content,+ ContainerPath: file.ContainerPath,+ Permissions: file.Permissions,+ })+ }++ return &DeploymentSnippetMTLS{+ DockerRun: mtlsDockerRun,+ DockerCompose: mtlsDockerCompose,+ Files: files,+ HostDirHint: strings.TrimSpace(generatedAssets.HostDirHint),+ }+}diff --git a/backend/internal/environment/environment_swarm_nodes.go b/backend/internal/environment/environment_swarm_nodes.gonew file mode 100644index 000000000..5b4982010--- /dev/null+++ b/backend/internal/environment/environment_swarm_nodes.go@@ -0,0 +1,267 @@+package environment++import (+ "context"+ "log/slog"+ "strings"+ "time"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "gorm.io/gorm"+)++func (s *EnvironmentService) ListSwarmNodeAgentEnvironments(ctx context.Context, parentEnvironmentID string) ([]models.Environment, error) {+ var envs []models.Environment+ if err := s.db.WithContext(ctx).+ Model(&models.Environment{}).+ Where("parent_environment_id = ?", parentEnvironmentID).+ Find(&envs).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to list swarm node agent environments")+ }++ return envs, nil+}++// ListSwarmNodeCandidateEnvironments returns enabled visible environments that+// can provide swarm-node coverage for a manager environment.+func (s *EnvironmentService) ListSwarmNodeCandidateEnvironments(ctx context.Context) ([]models.Environment, error) {+ var envs []models.Environment+ if err := s.db.WithContext(ctx).+ Model(&models.Environment{}).+ Where("hidden = ?", false).+ Where("enabled = ?", true).+ Where("id <> ?", "0").+ Order("name ASC").+ Find(&envs).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to list swarm node candidate environments")+ }++ return envs, nil+}++// BindSwarmNodeEnvironment binds an existing visible environment to a swarm+// node without modifying its connection details or agent token.+func (s *EnvironmentService) BindSwarmNodeEnvironment(+ ctx context.Context,+ parentEnvironmentID, nodeID, environmentID string,+ rebind bool,+) (*models.Environment, error) {+ var envRecord models.Environment+ err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {+ if err := tx.Where("id = ?", environmentID).First(&envRecord).Error; err != nil {+ return errors.WrapIf(err, "failed to load environment for swarm node binding")+ }+ if envRecord.Hidden {+ return errors.New("dedicated agent environments cannot be attached as visible environments")+ }+ if !envRecord.Enabled {+ return errors.New("disabled environments cannot be attached to swarm nodes")+ }++ boundElsewhere := envRecord.ParentEnvironmentID != nil && envRecord.SwarmNodeID != nil &&+ (strings.TrimSpace(*envRecord.ParentEnvironmentID) != parentEnvironmentID || strings.TrimSpace(*envRecord.SwarmNodeID) != nodeID)+ if boundElsewhere && !rebind {+ return errors.New("environment is already bound to another swarm node; explicit rebinding is required")+ }++ var existingVisibleBindings int64+ if err := tx.Model(&models.Environment{}).+ Where("hidden = ? AND parent_environment_id = ? AND swarm_node_id = ? AND id <> ?", false, parentEnvironmentID, nodeID, environmentID).+ Count(&existingVisibleBindings).Error; err != nil {+ return errors.WrapIf(err, "failed to inspect existing swarm node binding")+ }+ if existingVisibleBindings > 0 && !rebind {+ return errors.New("swarm node already has a visible environment binding; explicit rebinding is required")+ }+ if rebind {+ if err := tx.Model(&models.Environment{}).+ Where("hidden = ? AND parent_environment_id = ? AND swarm_node_id = ? AND id <> ?", false, parentEnvironmentID, nodeID, environmentID).+ Updates(map[string]any{"parent_environment_id": nil, "swarm_node_id": nil, "updated_at": new(time.Now())}).Error; err != nil {+ return errors.WrapIf(err, "failed to clear previous swarm node binding")+ }+ }++ if err := tx.Model(&models.Environment{}).Where("id = ?", environmentID).Updates(map[string]any{+ "parent_environment_id": parentEnvironmentID,+ "swarm_node_id": nodeID,+ "updated_at": new(time.Now()),+ }).Error; err != nil {+ return errors.WrapIf(err, "failed to bind environment to swarm node")+ }++ return tx.Where("id = ?", environmentID).First(&envRecord).Error+ })+ if err != nil {+ return nil, err+ }++ s.remoteEnvs.put(envRecord)+ return &envRecord, nil+}++// DetachSwarmNodeEnvironment clears a visible environment binding from a node.+func (s *EnvironmentService) DetachSwarmNodeEnvironment(ctx context.Context, parentEnvironmentID, nodeID string) error {+ now := time.Now()+ if err := s.db.WithContext(ctx).Model(&models.Environment{}).+ Where("hidden = ? AND parent_environment_id = ? AND swarm_node_id = ?", false, parentEnvironmentID, nodeID).+ Updates(map[string]any{"parent_environment_id": nil, "swarm_node_id": nil, "updated_at": &now}).Error; err != nil {+ return errors.WrapIf(err, "failed to detach swarm node environment")+ }++ return nil+}++// DeleteSwarmNodeAgentDeployment removes a dedicated hidden agent registration+// while leaving visible remote environments untouched.+func (s *EnvironmentService) DeleteSwarmNodeAgentDeployment(ctx context.Context, parentEnvironmentID, nodeID string, userID, username *string) error {+ var envRecord models.Environment+ if err := s.db.WithContext(ctx).+ Where("hidden = ? AND parent_environment_id = ? AND swarm_node_id = ?", true, parentEnvironmentID, nodeID).+ First(&envRecord).Error; err != nil {+ if errors.Is(err, gorm.ErrRecordNotFound) {+ return nil+ }+ return errors.WrapIf(err, "failed to load swarm node agent deployment")+ }++ return s.DeleteEnvironment(ctx, envRecord.ID, userID, username)+}++func buildSwarmNodeAgentNameInternal(hostname, nodeID string) string {+ trimmedHostname := strings.TrimSpace(hostname)+ if trimmedHostname != "" {+ return trimmedHostname+ }+ if len(nodeID) > 12 {+ nodeID = nodeID[:12]+ }+ return "Swarm Node " + nodeID+}++func buildSwarmNodeAgentURLInternal(nodeID string) string {+ shortNodeID := nodeID+ if len(shortNodeID) > 12 {+ shortNodeID = shortNodeID[:12]+ }+ return "edge://swarm-node-" + shortNodeID+}++func (s *EnvironmentService) applySwarmNodeAgentApiKeyInternal(+ ctx context.Context,+ env *models.Environment,+ userID, username string,+ rotate bool,+) (string, error) {+ if env == nil {+ return "", errors.New("environment is required")+ }++ if !rotate && env.AccessToken != nil && strings.TrimSpace(*env.AccessToken) != "" {+ return strings.TrimSpace(*env.AccessToken), nil+ }++ if s.apiKeyService == nil {+ return "", errors.New("api key service not configured")+ }++ oldApiKeyID := env.ApiKeyID++ apiKeyDto, err := s.apiKeyService.CreateEnvironmentApiKey(ctx, env.ID)+ if err != nil {+ return "", errors.WrapIf(err, "failed to create environment API key")+ }++ if err := s.RegenerateEnvironmentApiKey(ctx, env.ID, apiKeyDto.ID, apiKeyDto.Key, userID, username, env.Name); err != nil {+ // The new key was never linked; remove it so a failed rotation does+ // not leave an orphaned valid credential behind.+ if delErr := s.apiKeyService.DeleteApiKey(ctx, apiKeyDto.ID); delErr != nil && !errors.Is(delErr, apikey.ErrApiKeyNotFound) {+ slog.ErrorContext(ctx, "Failed to clean up unlinked environment API key", "environmentID", env.ID, "error", delErr.Error())+ }+ return "", err+ }++ // Delete the previous key only after the environment points at the new+ // one — while still referenced it is protected and the delete would be+ // rejected. Failure is non-fatal for the rotation itself, but the old key+ // remains a valid credential until deleted, so log it as an error; the key+ // stays visible and deletable on the API Keys page.+ if oldApiKeyID != nil && *oldApiKeyID != apiKeyDto.ID {+ if err := s.apiKeyService.DeleteApiKey(ctx, *oldApiKeyID); err != nil && !errors.Is(err, apikey.ErrApiKeyNotFound) {+ slog.ErrorContext(ctx, "Failed to delete previous environment API key; the old key remains valid until deleted manually", "environmentID", env.ID, "error", err.Error())+ }+ }++ return apiKeyDto.Key, nil+}++func (s *EnvironmentService) EnsureSwarmNodeAgentEnvironment(+ ctx context.Context,+ parentEnvironmentID, nodeID, hostname, userID, username string,+ rotate bool,+) (*models.Environment, string, error) {+ if strings.TrimSpace(parentEnvironmentID) == "" {+ return nil, "", errors.New("parent environment ID is required")+ }+ if strings.TrimSpace(nodeID) == "" {+ return nil, "", errors.New("swarm node ID is required")+ }++ var env models.Environment+ // Prefer an existing visible binding. Legacy hidden registrations remain+ // reusable, but all newly provisioned node agents are normal Remote+ // Environments so one token and one agent can serve both use cases.+ err := s.db.WithContext(ctx).+ Where("parent_environment_id = ?", parentEnvironmentID).+ Where("swarm_node_id = ?", nodeID).+ Order("hidden ASC").+ First(&env).Error+ if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {+ return nil, "", errors.WrapIf(err, "failed to load swarm node agent environment")+ }++ if errors.Is(err, gorm.ErrRecordNotFound) {+ createdEnv := &models.Environment{+ Name: buildSwarmNodeAgentNameInternal(hostname, nodeID),+ ApiUrl: buildSwarmNodeAgentURLInternal(nodeID),+ Status: string(models.EnvironmentStatusPending),+ Enabled: true,+ IsEdge: true,+ Hidden: false,+ ParentEnvironmentID: new(parentEnvironmentID),+ SwarmNodeID: new(nodeID),+ }++ if _, createErr := s.CreateEnvironment(ctx, createdEnv, new(userID), new(username)); createErr != nil {+ return nil, "", errors.WrapIf(createErr, "failed to create swarm node agent environment")+ }+ env = *createdEnv+ }++ apiKey, err := s.applySwarmNodeAgentApiKeyInternal(ctx, &env, userID, username, rotate)+ if err != nil {+ return nil, "", err+ }++ refreshedEnv, err := s.GetEnvironmentByID(ctx, env.ID)+ if err != nil {+ return nil, "", errors.WrapIf(err, "failed to refresh swarm node agent environment")+ }++ return refreshedEnv, apiKey, nil+}++func (s *EnvironmentService) UpdateSwarmNodeIdentity(ctx context.Context, envID, swarmNodeID string) error {+ updates := map[string]any{+ "swarm_node_id": swarmNodeID,+ "updated_at": new(time.Now()),+ }++ if err := s.db.WithContext(ctx).Model(&models.Environment{}).Where("id = ?", envID).Updates(updates).Error; err != nil {+ return errors.WrapIf(err, "failed to update swarm node identity")+ }++ return nil+}diff --git a/backend/api/handlers/environments.go b/backend/internal/environment/handler.gosimilarity index 91%rename from backend/api/handlers/environments.gorename to backend/internal/environment/handler.goindex 93236781f..6ccdff38f 100644--- a/backend/api/handlers/environments.go+++ b/backend/internal/environment/handler.go@@ -1,4 +1,4 @@-package handlers+package environmentimport ("archive/zip"@@ -17,14 +17,17 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"- httputils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/environment"@@ -43,10 +46,10 @@ const (// EnvironmentHandler handles environment management endpoints.type EnvironmentHandler struct {- environmentService *services.EnvironmentService- settingsService *services.SettingsService- apiKeyService *services.ApiKeyService- eventService *services.EventService+ environmentService *EnvironmentService+ settingsService *settings.SettingsService+ apiKeyService *apikey.ApiKeyService+ eventService *event.EventServicecfg *config.Config}@@ -154,22 +157,6 @@ type DeploymentSnippet struct {MTLS *DeploymentSnippetMTLS `json:"mtls,omitempty" doc:"Optional Arcane-generated mTLS deployment assets for edge agents"`}-type DeploymentSnippetFile struct {- Name string `json:"name" doc:"Suggested filename"`- Content string `json:"content,omitempty" doc:"PEM file contents. Omitted for sensitive files such as private keys; use downloadUrl instead."`- DownloadURL string `json:"downloadUrl,omitempty" doc:"Pairing-permission endpoint to download this file when content is withheld"`- Sensitive bool `json:"sensitive,omitempty" doc:"True when this file is sensitive and must be fetched via downloadUrl"`- ContainerPath string `json:"containerPath" doc:"Container mount path expected by the mTLS snippet"`- Permissions string `json:"permissions" doc:"Suggested file mode"`-}--type DeploymentSnippetMTLS struct {- DockerRun string `json:"dockerRun" doc:"Docker run snippet using Arcane-generated mTLS assets"`- DockerCompose string `json:"dockerCompose" doc:"Docker compose snippet using Arcane-generated mTLS assets"`- Files []DeploymentSnippetFile `json:"files" doc:"Generated PEM files to place on the edge host"`- HostDirHint string `json:"hostDirHint" doc:"Suggested host directory containing the generated PEM files"`-}-type GetDeploymentSnippetsInput struct {ID string `path:"id" doc:"Environment ID"`}@@ -201,16 +188,19 @@ type DownloadEnvironmentMTLSFileInput struct {// Registration// ============================================================================-// RegisterEnvironments registers all environment management endpoints.-func RegisterEnvironments(api huma.API, environmentService *services.EnvironmentService, settingsService *services.SettingsService, apiKeyService *services.ApiKeyService, eventService *services.EventService, cfg *config.Config) {- h := &EnvironmentHandler{+// NewHandler builds the environment HTTP handler and its stream producer.+func NewHandler(environmentService *EnvironmentService, settingsService *settings.SettingsService, apiKeyService *apikey.ApiKeyService, eventService *event.EventService, cfg *config.Config) *EnvironmentHandler {+ return &EnvironmentHandler{environmentService: environmentService,settingsService: settingsService,apiKeyService: apiKeyService,eventService: eventService,cfg: cfg,}+}+// RegisterEnvironments registers all environment management endpoints.+func RegisterEnvironments(api huma.API, h *EnvironmentHandler) {huma.Register(api, huma.Operation{OperationID: "listEnvironments",Method: "GET",@@ -218,7 +208,7 @@ func RegisterEnvironments(api huma.API, environmentService *services.EnvironmentSummary: "List environments",Description: "Get a paginated list of Docker environments",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),// No global PermEnvironmentsList gate: this endpoint also backs the// environment switcher, so any authenticated caller may list. The handler// filters the result to the environments the caller can actually access.@@ -232,8 +222,8 @@ func RegisterEnvironments(api huma.API, environmentService *services.EnvironmentSummary: "Create an environment",Description: "Create a new Docker environment",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermEnvironmentsCreate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermEnvironmentsCreate),}, h.CreateEnvironment)huma.Register(api, huma.Operation{@@ -243,8 +233,8 @@ func RegisterEnvironments(api huma.API, environmentService *services.EnvironmentSummary: "Get an environment",Description: "Get a Docker environment by ID",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermEnvironmentsRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermEnvironmentsRead),}, h.GetEnvironment)huma.Register(api, huma.Operation{@@ -254,8 +244,8 @@ func RegisterEnvironments(api huma.API, environmentService *services.EnvironmentSummary: "Update an environment",Description: "Update a Docker environment",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermEnvironmentsUpdate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermEnvironmentsUpdate),}, h.UpdateEnvironment)huma.Register(api, huma.Operation{@@ -265,48 +255,48 @@ func RegisterEnvironments(api huma.API, environmentService *services.EnvironmentSummary: "Delete an environment",Description: "Delete a Arcane environment",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermEnvironmentsDelete),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermEnvironmentsDelete),}, h.DeleteEnvironment)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "testConnection",Method: "POST",Path: "/environments/{id}/test",Summary: "Test environment connection",Description: "Test connectivity to a Arcane environment",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermEnvironmentsRead, h.TestConnection)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "updateHeartbeat",Method: "POST",Path: "/environments/{id}/heartbeat",Summary: "Update environment heartbeat",Description: "Update the heartbeat timestamp for an environment",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermEnvironmentsSync, h.UpdateHeartbeat)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "pairAgent",Method: "POST",Path: "/environments/{id}/agent/pair",Summary: "Pair with local agent",Description: "Generate or rotate the local agent pairing token",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermEnvironmentsPair, h.PairAgent)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "syncEnvironment",Method: "POST",Path: "/environments/{id}/sync",Summary: "Sync environment",Description: "Sync container registries and git repositories to a remote environment",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermEnvironmentsSync, h.SyncEnvironment)huma.Register(api, huma.Operation{@@ -320,44 +310,44 @@ func RegisterEnvironments(api huma.API, environmentService *services.EnvironmentSecurity: []map[string][]string{},}, h.PairEnvironment)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "getDeploymentSnippets",Method: "GET",Path: "/environments/{id}/deployment",Summary: "Get deployment snippets",Description: "Get Docker run and compose snippets for environment deployment",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermEnvironmentsPair, h.GetDeploymentSnippets)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "downloadEnvironmentMTLSBundle",Method: "GET",Path: "/environments/{id}/deployment/mtls/bundle",Summary: "Download environment mTLS bundle",Description: "Download the generated mTLS client certificate bundle for an edge environment",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermEnvironmentsPair, h.DownloadEnvironmentMTLSBundle)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "downloadEnvironmentMTLSFile",Method: "GET",Path: "/environments/{id}/deployment/mtls/{fileName}",Summary: "Download environment mTLS asset",Description: "Download an individual generated mTLS client certificate asset for an edge environment",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermEnvironmentsPair, h.DownloadEnvironmentMTLSFile)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "getEnvironmentVersion",Method: "GET",Path: "/environments/{id}/version",Summary: "Get environment version",Description: "Get the version of a remote environment",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermEnvironmentsRead, h.GetEnvironmentVersion)huma.Register(api, huma.Operation{@@ -367,8 +357,8 @@ func RegisterEnvironments(api huma.API, environmentService *services.EnvironmentSummary: "Download Arcane-generated edge mTLS CA",Description: "Download the Arcane-managed certificate authority used for generated edge mTLS client certificates",Tags: []string{"Environments"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermEnvironmentsPair),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermEnvironmentsPair),}, h.DownloadEdgeMTLSCA)}@@ -383,7 +373,7 @@ func (h *EnvironmentHandler) ListEnvironments(ctx context.Context, input *ListEn// listers (sudo, global admins, or holders of the org-level// environments:list permission) see every environment; environment-scoped// callers see only the environments they hold at least one permission on.- ps, ok := humamw.PermissionsFromContext(ctx)+ ps, ok := middleware.PermissionsFromContext(ctx)if !ok {return nil, huma.Error403Forbidden("permission denied")}@@ -392,7 +382,7 @@ func (h *EnvironmentHandler) ListEnvironments(ctx context.Context, input *ListEnaccessibleEnvIDs = accessibleEnvironmentIDsInternal(ps)}- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if input.Type != "" {params.Filters["type"] = input.Type}@@ -409,7 +399,7 @@ func (h *EnvironmentHandler) ListEnvironments(ctx context.Context, input *ListEnBody: base.Paginated[environment.Environment]{Success: true,Data: envs,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -504,7 +494,7 @@ func fingerprintEnvironmentsInternal(envs []environment.Environment) uint64 {})}-func (h *EnvironmentHandler) runEnvironmentStreamProducerInternal(ctx context.Context, ps *authz.PermissionSet, events chan<- environment.StreamEvent) {+func (h *EnvironmentHandler) RunStreamProducer(ctx context.Context, ps *authz.PermissionSet, events chan<- environment.StreamEvent) {changes, unsubscribe := h.environmentService.SubscribeRuntimeChanges()defer unsubscribe()@@ -564,7 +554,7 @@ func (h *EnvironmentHandler) runEnvironmentStreamProducerInternal(ctx context.Co// CreateEnvironment creates a new environment.func (h *EnvironmentHandler) CreateEnvironment(ctx context.Context, input *CreateEnvironmentInput) (*CreateEnvironmentOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -624,7 +614,7 @@ func (h *EnvironmentHandler) createEnvironmentWithApiKeyInternal(ctx context.Con// update failed afterwards, so the key legitimately belongs to the// environment and must survive; deleting the environment cascades it.if delErr := h.apiKeyService.DeleteApiKey(ctx, apiKeyDto.ID); delErr != nil &&- !errors.Is(delErr, services.ErrApiKeyNotFound) && !errors.Is(delErr, services.ErrApiKeyProtected) {+ !errors.Is(delErr, apikey.ErrApiKeyNotFound) && !errors.Is(delErr, apikey.ErrApiKeyProtected) {slog.ErrorContext(ctx, "Failed to clean up unlinked environment API key", "environmentID", created.ID, "error", delErr.Error())}slog.ErrorContext(ctx, "Failed to link API key to environment", "environmentID", created.ID, "error", err.Error())@@ -739,7 +729,7 @@ func (h *EnvironmentHandler) UpdateEnvironment(ctx context.Context, input *Updat// If regenerating API key, return the new keyvar newApiKey *stringif input.Body.RegenerateApiKey != nil && *input.Body.RegenerateApiKey {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -759,7 +749,7 @@ func (h *EnvironmentHandler) UpdateEnvironment(ctx context.Context, input *Updatif err != nil {// The new key was never linked; remove it so a failed rotation does// not leave an orphaned valid credential behind.- if delErr := h.apiKeyService.DeleteApiKey(ctx, apiKeyDto.ID); delErr != nil && !errors.Is(delErr, services.ErrApiKeyNotFound) {+ if delErr := h.apiKeyService.DeleteApiKey(ctx, apiKeyDto.ID); delErr != nil && !errors.Is(delErr, apikey.ErrApiKeyNotFound) {slog.ErrorContext(ctx, "Failed to clean up unlinked environment API key", "environmentID", input.ID, "error", delErr.Error())}slog.ErrorContext(ctx, "Failed to regenerate API key", "environmentID", input.ID, "error", err.Error())@@ -773,7 +763,7 @@ func (h *EnvironmentHandler) UpdateEnvironment(ctx context.Context, input *Updat// it as an error; the key stays visible and deletable on the API Keys// page.if oldApiKeyID != nil && *oldApiKeyID != apiKeyDto.ID {- if err := h.apiKeyService.DeleteApiKey(ctx, *oldApiKeyID); err != nil && !errors.Is(err, services.ErrApiKeyNotFound) {+ if err := h.apiKeyService.DeleteApiKey(ctx, *oldApiKeyID); err != nil && !errors.Is(err, apikey.ErrApiKeyNotFound) {slog.ErrorContext(ctx, "Failed to delete previous environment API key; the old key remains valid until deleted manually", "environmentID", input.ID, "error", err.Error())}}@@ -807,7 +797,7 @@ func (h *EnvironmentHandler) UpdateEnvironment(ctx context.Context, input *Updat}func (h *EnvironmentHandler) applyEdgeRuntimeStateInternal(env *environment.Environment) {- services.ApplyEnvironmentRuntimeState(env)+ ApplyEnvironmentRuntimeState(env)}// DeleteEnvironment deletes an environment.@@ -843,7 +833,7 @@ func (h *EnvironmentHandler) TestConnection(ctx context.Context, input *TestConnapiUrl = input.Body.ApiUrl}if apiUrl != nil {- permissions, ok := humamw.PermissionsFromContext(ctx)+ permissions, ok := middleware.PermissionsFromContext(ctx)if !ok || !permissions.Allows(authz.PermEnvironmentsUpdate, "") {return nil, huma.Error403Forbidden("permission denied: " + authz.PermEnvironmentsUpdate)}@@ -1082,7 +1072,7 @@ func (h *EnvironmentHandler) GetDeploymentSnippets(ctx context.Context, input *G// Generate snippets with API key// Use edge snippets for edge environments- var snippets *services.DeploymentSnippets+ var snippets *DeploymentSnippetsif env.IsEdge {snippets, err = h.environmentService.GenerateEdgeDeploymentSnippets(ctx, env.ID, h.cfg.GetAppURL(), *env.AccessToken, &edge.Config{EdgeMTLSMode: h.cfg.EdgeMTLSMode,@@ -1169,7 +1159,7 @@ func (h *EnvironmentHandler) GetEnvironmentVersion(ctx context.Context, input *G}} else {// Direct HTTP request for non-edge environments- validatedURL, validateErr := httputils.ValidateOutboundHTTPURL(env.ApiUrl)+ validatedURL, validateErr := httpx.ValidateOutboundHTTPURL(env.ApiUrl)if validateErr != nil {return nil, huma.Error400BadRequest("Invalid environment API URL")}@@ -1214,7 +1204,15 @@ func (h *EnvironmentHandler) GetEnvironmentVersion(ctx context.Context, input *G// DownloadEdgeMTLSCA downloads the Arcane-managed edge mTLS CA certificate.func (h *EnvironmentHandler) DownloadEdgeMTLSCA(ctx context.Context, _ *DownloadEdgeMTLSCAInput) (*huma.StreamResponse, error) {- caPath, err := generatedEdgeMTLSCAPathInternal(h.cfg)+ var edgeCfg *edge.Config+ if h.cfg != nil {+ edgeCfg = &edge.Config{+ EdgeMTLSMode: h.cfg.EdgeMTLSMode,+ EdgeMTLSCAFile: h.cfg.EdgeMTLSCAFile,+ EdgeMTLSAssetsDir: h.cfg.EdgeMTLSAssetsDir,+ }+ }+ caPath, err := edge.AvailableManagerMTLSCAPath(edgeCfg)if err != nil {return nil, huma.Error404NotFound("Arcane-managed edge mTLS CA is not available")}@@ -1361,7 +1359,7 @@ func (h *EnvironmentHandler) loadEnvironmentMTLSEnvironmentInternal(ctx context.return env, nil}-func (h *EnvironmentHandler) loadEnvironmentMTLSFilesInternal(ctx context.Context, environmentID string) (*models.Environment, []services.DeploymentSnippetFile, error) {+func (h *EnvironmentHandler) loadEnvironmentMTLSFilesInternal(ctx context.Context, environmentID string) (*models.Environment, []DeploymentSnippetFile, error) {env, err := h.loadEnvironmentMTLSEnvironmentInternal(ctx, environmentID)if err != nil {return nil, nil, err@@ -1385,10 +1383,10 @@ func (h *EnvironmentHandler) loadEnvironmentMTLSFilesInternal(ctx context.Contexreturn env, snippets.MTLS.Files, nil}-func (h *EnvironmentHandler) loadEnvironmentMTLSFileInternal(ctx context.Context, environmentID string, fileName string) (*models.Environment, services.DeploymentSnippetFile, error) {+func (h *EnvironmentHandler) loadEnvironmentMTLSFileInternal(ctx context.Context, environmentID string, fileName string) (*models.Environment, DeploymentSnippetFile, error) {env, files, err := h.loadEnvironmentMTLSFilesInternal(ctx, environmentID)if err != nil {- return nil, services.DeploymentSnippetFile{}, err+ return nil, DeploymentSnippetFile{}, err}for _, file := range files {@@ -1397,7 +1395,7 @@ func (h *EnvironmentHandler) loadEnvironmentMTLSFileInternal(ctx context.Context}}- return nil, services.DeploymentSnippetFile{}, huma.Error404NotFound("Requested mTLS asset was not found")+ return nil, DeploymentSnippetFile{}, huma.Error404NotFound("Requested mTLS asset was not found")}// isSensitiveMTLSAssetNameInternal reports whether the given generated asset@@ -1449,7 +1447,7 @@ func environmentMTLSAssetDownloadNameInternal(env *models.Environment, fileName}}-func environmentMTLSAssetFileModeInternal(file services.DeploymentSnippetFile) os.FileMode {+func environmentMTLSAssetFileModeInternal(file DeploymentSnippetFile) os.FileMode {if parsed, err := strconv.ParseUint(strings.TrimSpace(file.Permissions), 8, 32); err == nil && parsed != 0 {return os.FileMode(parsed)}@@ -1477,11 +1475,11 @@ func (h *EnvironmentHandler) logMTLSAuditEventInternal(ctx context.Context, envif extra == nil {extra = models.JSON{}}- if remoteAddr := strings.TrimSpace(humamw.GetRemoteAddrFromContext(ctx)); remoteAddr != "" {+ if remoteAddr := strings.TrimSpace(middleware.GetRemoteAddrFromContext(ctx)); remoteAddr != "" {extra["remoteAddr"] = remoteAddr}- req := services.CreateEventRequest{+ req := event.CreateEventRequest{Type: eventType,Severity: models.EventSeverityInfo,Title: title,diff --git a/backend/internal/environment/module.go b/backend/internal/environment/module.gonew file mode 100644index 000000000..bc56277b3--- /dev/null+++ b/backend/internal/environment/module.go@@ -0,0 +1,53 @@+// Package environment owns environment persistence, remote runtime state, pairing,+// synchronization, and its HTTP and stream surfaces.+package environment++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+)++type Dependencies struct {+ Settings *settings.SettingsService+ ApiKey *apikey.ApiKeyService+ Event *event.EventService+ Config *config.Config+}++type Module struct {+ service *EnvironmentService+ handler *EnvironmentHandler+}++func New(service *EnvironmentService, deps Dependencies) *Module {+ return &Module{+ service: service,+ handler: NewHandler(service, deps.Settings, deps.ApiKey, deps.Event, deps.Config),+ }+}++func (m *Module) Service() *EnvironmentService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) Handler() *EnvironmentHandler {+ if m == nil {+ return nil+ }+ return m.handler+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterEnvironments(api, NewHandler(nil, nil, nil, nil, nil))+ return+ }+ RegisterEnvironments(api, m.handler)+}diff --git a/backend/internal/services/environment_runtime_notify_test.go b/backend/internal/environment/runtime_notify_test.gosimilarity index 94%rename from backend/internal/services/environment_runtime_notify_test.gorename to backend/internal/environment/runtime_notify_test.goindex 2288a4ec7..f36a5bfdb 100644--- a/backend/internal/services/environment_runtime_notify_test.go+++ b/backend/internal/environment/runtime_notify_test.go@@ -1,4 +1,4 @@-package services+package environmentimport ("testing"@@ -41,5 +41,5 @@ func TestEnvironmentServiceRuntimeChangeSignalCoalescesInternal(t *testing.T) {default:}- require.Empty(t, s.runtimeWatchers)+ require.Empty(t, s.runtimeWatchers.chans)}diff --git a/backend/internal/services/environment_service_test.go b/backend/internal/environment/service_test.gosimilarity index 92%rename from backend/internal/services/environment_service_test.gorename to backend/internal/environment/service_test.goindex 9f5b07bd1..b74075174 100644--- a/backend/internal/services/environment_service_test.go+++ b/backend/internal/environment/service_test.go@@ -1,4 +1,4 @@-package services+package environmentimport ("context"@@ -19,15 +19,20 @@ import ("github.com/coder/websocket""github.com/getarcaneapp/arcane/backend/v2/internal/actors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/scheduler/entityjobs""github.com/getarcaneapp/arcane/types/v2/containerregistry""github.com/getarcaneapp/arcane/types/v2/environment""github.com/getarcaneapp/arcane/types/v2/gitops"+ schedulertypes "github.com/getarcaneapp/arcane/types/v2/scheduler""github.com/stretchr/testify/assert""go.getarcane.app/sys/crypto""go.uber.org/fx/fxtest"@@ -45,7 +50,8 @@ func TestEnvironmentService_OverlappingHealthCheckIsSkippedInternal(t *testing.Trequire.NoError(t, err)require.True(t, admitted)- service := &EnvironmentService{admissionGate: gate}+ service := &EnvironmentService{jobs: entityjobs.New(environmentHealthJobPrefix, environmentHealthAdmissionScopeInternal)}+ require.NoError(t, service.SetScheduler(t.Context(), &environmentTestSchedulerInternal{}, gate))service.runHealthCheckInternal(t.Context(), "environment-id")lease.Release()@@ -55,6 +61,40 @@ func TestEnvironmentService_OverlappingHealthCheckIsSkippedInternal(t *testing.Trequire.NoError(t, lifecycle.Stop(stopCtx))}+type environmentTestSchedulerInternal struct {+ added []string+ removed []string+}++func (s *environmentTestSchedulerInternal) AddJob(_ context.Context, job schedulertypes.Job) error {+ s.added = append(s.added, job.Name())+ return nil+}++func (s *environmentTestSchedulerInternal) RemoveJob(_ context.Context, name string) {+ s.removed = append(s.removed, name)+}++func (s *environmentTestSchedulerInternal) HasJob(_ string) bool {+ return false+}++func newAdmissionGateForEnvironmentTestInternal(t testing.TB) *actors.Gate[actors.AdmissionKey] {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ gate, err := actors.NewGate[actors.AdmissionKey](t.Context(), runtime, "environment-test-admission", t.Name())+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, gate.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return gate+}+func setupEnvironmentServiceTestDB(t *testing.T) *database.DB {t.Helper()@@ -89,7 +129,7 @@ func setupEnvironmentServiceTestDB(t *testing.T) *database.DB {return &database.DB{DB: db}}-func createTestEnvironmentServiceUser(t *testing.T, ctx context.Context, userService *UserService, id string) *models.User {+func createTestEnvironmentServiceUser(t *testing.T, ctx context.Context, userService *user.UserService, id string) *models.User {t.Helper()user := &models.User{@@ -172,9 +212,9 @@ func TestEnvironmentService_DeleteEnvironment_CascadesGitOpsSyncs(t *testing.T)GitOpsManagedBy: &syncID,}).Error)- scheduler := &gitOpsSyncTestSchedulerInternal{}+ scheduler := &environmentTestSchedulerInternal{}svc := NewEnvironmentService(db, nil, nil, nil, nil, nil)- require.NoError(t, svc.SetScheduler(ctx, scheduler, newAdmissionGateForTestInternal(t)))+ require.NoError(t, svc.SetScheduler(ctx, scheduler, newAdmissionGateForEnvironmentTestInternal(t)))require.NoError(t, svc.DeleteEnvironment(ctx, "env-delete-gitops", nil, nil))@@ -185,7 +225,7 @@ func TestEnvironmentService_DeleteEnvironment_CascadesGitOpsSyncs(t *testing.T)var project models.Projectrequire.NoError(t, db.First(&project, "id = ?", "project-managed-by-deleted-env").Error)require.Nil(t, project.GitOpsManagedBy)- require.Contains(t, scheduler.removed, gitOpsSyncJobNameInternal(syncID))+ require.Contains(t, scheduler.removed, entityjobs.GitOpsSyncJobPrefix+syncID)}func createTestRegistry(t *testing.T, db *database.DB, id string) {@@ -206,7 +246,7 @@ func createTestRegistry(t *testing.T, db *database.DB, id string) {Token: encryptedToken,Enabled: true,Insecure: false,- RegistryType: registryTypeGeneric,+ RegistryType: registry.RegistryTypeGeneric,CreatedAt: now,UpdatedAt: now,}@@ -229,7 +269,7 @@ func createTestECRRegistry(t *testing.T, db *database.DB, id string) {},URL: "123456789012.dkr.ecr.us-east-1.amazonaws.com",Enabled: true,- RegistryType: registryTypeECR,+ RegistryType: registry.RegistryTypeECR,AWSAccessKeyID: "AKIA1234567890EXAMPLE",AWSSecretAccessKey: encryptedSecret,AWSRegion: "us-east-1",@@ -240,6 +280,18 @@ func createTestECRRegistry(t *testing.T, db *database.DB, id string) {require.NoError(t, db.WithContext(context.Background()).Create(registry).Error)}+func createTestGitRepository(t *testing.T, db *database.DB, repository models.GitRepository) {+ t.Helper()+ require.NoError(t, db.Create(&repository).Error)+}++func encryptSecretForTest(t *testing.T, value string) string {+ t.Helper()+ encrypted, err := crypto.Encrypt(value)+ require.NoError(t, err)+ return encrypted+}+func TestEnvironmentService_SyncRegistriesToRemoteEnvironments_SyncsEligibleRemotes(t *testing.T) {ctx := context.Background()db := setupEnvironmentServiceTestDB(t)@@ -340,26 +392,26 @@ func TestEnvironmentService_SyncRegistriesToEnvironment_IncludesECRFields(t *tesreturn}- registry := syncReq.Registries[0]- if !assert.Equal(t, registryTypeECR, registry.RegistryType) {+ syncedRegistry := syncReq.Registries[0]+ if !assert.Equal(t, registry.RegistryTypeECR, syncedRegistry.RegistryType) {return}- if !assert.Equal(t, "123456789012.dkr.ecr.us-east-1.amazonaws.com", registry.URL) {+ if !assert.Equal(t, "123456789012.dkr.ecr.us-east-1.amazonaws.com", syncedRegistry.URL) {return}- if !assert.Equal(t, "AKIA1234567890EXAMPLE", registry.AWSAccessKeyID) {+ if !assert.Equal(t, "AKIA1234567890EXAMPLE", syncedRegistry.AWSAccessKeyID) {return}- if !assert.Equal(t, "aws-secret", registry.AWSSecretAccessKey) {+ if !assert.Equal(t, "aws-secret", syncedRegistry.AWSSecretAccessKey) {return}- if !assert.Equal(t, "us-east-1", registry.AWSRegion) {+ if !assert.Equal(t, "us-east-1", syncedRegistry.AWSRegion) {return}- if !assert.Empty(t, registry.Username) {+ if !assert.Empty(t, syncedRegistry.Username) {return}- if !assert.Empty(t, registry.Token) {+ if !assert.Empty(t, syncedRegistry.Token) {return}@@ -630,14 +682,14 @@ func TestEnvironmentService_UpdateEnvironment_ClearingAccessTokenInvalidatesCach_, err = svc.UpdateEnvironment(ctx, "edge-auth-clear", map[string]any{"access_token": nil}, nil, nil)require.NoError(t, err)- cachedEnvID, ok := svc.getCachedEnvironmentIDForTokenInternal(oldToken).Get()+ cachedEnvID, ok := svc.edgeTokens.environmentID(oldToken).Get()require.False(t, ok)require.Empty(t, cachedEnvID)- _, tokenStillCached := svc.tokenCache.Peek(oldToken)- svc.tokenCacheMu.RLock()- _, reverseIndexStillCached := svc.tokenByEnvID["edge-auth-clear"]- svc.tokenCacheMu.RUnlock()+ _, tokenStillCached := svc.edgeTokens.byToken.Peek(oldToken)+ svc.edgeTokens.mu.RLock()+ _, reverseIndexStillCached := svc.edgeTokens.byEnvID["edge-auth-clear"]+ svc.edgeTokens.mu.RUnlock()require.False(t, tokenStillCached)require.False(t, reverseIndexStillCached)@@ -685,20 +737,20 @@ func TestEnvironmentServiceUpdateEnvironmentAllowsTargetChangeWithReplacementTokfunc TestEnvironmentService_getCachedEnvironmentIDForTokenInternal_ExpiresAndCleansReverseIndex(t *testing.T) {svc := NewEnvironmentService(nil, nil, nil, nil, nil, nil)- svc.tokenCacheMu.Lock()- svc.tokenByEnvID["env-expired"] = "expired-token"- svc.tokenCache.SetWithTTL("expired-token", "env-expired", time.Nanosecond)+ svc.edgeTokens.mu.Lock()+ svc.edgeTokens.byEnvID["env-expired"] = "expired-token"+ svc.edgeTokens.byToken.SetWithTTL("expired-token", "env-expired", time.Nanosecond)time.Sleep(time.Millisecond)- svc.tokenCacheMu.Unlock()+ svc.edgeTokens.mu.Unlock()- envID, ok := svc.getCachedEnvironmentIDForTokenInternal("expired-token").Get()+ envID, ok := svc.edgeTokens.environmentID("expired-token").Get()require.False(t, ok)require.Empty(t, envID)- _, tokenStillCached := svc.tokenCache.Peek("expired-token")- svc.tokenCacheMu.RLock()- _, reverseIndexStillCached := svc.tokenByEnvID["env-expired"]- svc.tokenCacheMu.RUnlock()+ _, tokenStillCached := svc.edgeTokens.byToken.Peek("expired-token")+ svc.edgeTokens.mu.RLock()+ _, reverseIndexStillCached := svc.edgeTokens.byEnvID["env-expired"]+ svc.edgeTokens.mu.RUnlock()require.False(t, tokenStillCached)require.False(t, reverseIndexStillCached)@@ -754,8 +806,8 @@ func TestEnvironmentService_GenerateDeploymentSnippets_ExplicitlyUsePollTransporfunc TestEnvironmentService_EnsureSwarmNodeAgentEnvironment_CreatesVisibleEnvironmentAndReusesToken(t *testing.T) {ctx := context.Background()db := setupEnvironmentServiceTestDB(t)- userService := NewUserService(db)- apiKeyService := NewApiKeyService(db, userService)+ userService := user.NewUserService(db)+ apiKeyService := apikey.NewApiKeyService(db, userService)svc := NewEnvironmentService(db, nil, nil, nil, nil, apiKeyService)user := createTestEnvironmentServiceUser(t, ctx, userService, "swarm-admin")@@ -895,8 +947,8 @@ func TestEnvironmentService_EnsureSwarmNodeAgentEnvironment_ReusesLegacyHiddenRefunc TestEnvironmentService_EnsureSwarmNodeAgentEnvironment_TokenResolvesEndToEnd(t *testing.T) {ctx := context.Background()db := setupEnvironmentServiceTestDB(t)- userService := NewUserService(db)- apiKeyService := NewApiKeyService(db, userService)+ userService := user.NewUserService(db)+ apiKeyService := apikey.NewApiKeyService(db, userService)svc := NewEnvironmentService(db, nil, nil, nil, nil, apiKeyService)user := createTestEnvironmentServiceUser(t, ctx, userService, "swarm-resolve-admin")diff --git a/backend/internal/environment/vocabulary.go b/backend/internal/environment/vocabulary.gonew file mode 100644index 000000000..5063d982d--- /dev/null+++ b/backend/internal/environment/vocabulary.go@@ -0,0 +1,20 @@+package environment++import "strings"++// LocalEnvironmentID is the reserved ID of the environment Arcane manages directly.+const LocalEnvironmentID = "0"++const localEnvironmentFallbackNameInternal = "Local"++// DisplayName returns the stored environment name or its readable fallback.+func DisplayName(environmentID, storedName string) string {+ if name := strings.TrimSpace(storedName); name != "" {+ return name+ }+ id := strings.TrimSpace(environmentID)+ if id == "" || id == LocalEnvironmentID {+ return localEnvironmentFallbackNameInternal+ }+ return id+}diff --git a/backend/api/handlers/events.go b/backend/internal/event/handler.gosimilarity index 84%rename from backend/api/handlers/events.gorename to backend/internal/event/handler.goindex 2d20363f5..5edd25b29 100644--- a/backend/api/handlers/events.go+++ b/backend/internal/event/handler.go@@ -1,4 +1,4 @@-package handlers+package eventimport ("context"@@ -10,19 +10,19 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"- "github.com/getarcaneapp/arcane/types/v2/event"+ eventtypes "github.com/getarcaneapp/arcane/types/v2/event""github.com/labstack/echo/v5")// EventHandler handles event management endpoints.type EventHandler struct {- eventService *services.EventService+ eventService *EventService}// ============================================================================@@ -40,13 +40,13 @@ type ListEventsInput struct {}type ListEventsOutput struct {- Body base.Paginated[event.Event]+ Body base.Paginated[eventtypes.Event]}type GetEventStatsInput struct{}type GetEventStatsOutput struct {- Body base.ApiResponse[services.EventSeverityCounts]+ Body base.ApiResponse[EventSeverityCounts]}type GetEventsByEnvironmentInput struct {@@ -61,7 +61,7 @@ type GetEventsByEnvironmentInput struct {}type GetEventsByEnvironmentOutput struct {- Body base.Paginated[event.Event]+ Body base.Paginated[eventtypes.Event]}type DeleteEventInput struct {@@ -79,7 +79,7 @@ type DeleteEventOutput struct {// RegisterAgentEventIngestion registers the manager ingestion endpoint used by// direct agents when no edge tunnel is active. This route is not part of the// Huma/OpenAPI surface and authenticates only with the configured agent token.-func RegisterAgentEventIngestion(g *echo.Group, eventService *services.EventService, cfg *config.Config) {+func RegisterAgentEventIngestion(g *echo.Group, eventService *EventService, cfg *config.Config) {g.POST("/events", func(c *echo.Context) error {if eventService == nil {return c.JSON(http.StatusInternalServerError, base.ApiResponse[base.MessageResponse]{@@ -101,7 +101,7 @@ func RegisterAgentEventIngestion(g *echo.Group, eventService *services.EventServ})}- var input services.CreateEventRequest+ var input CreateEventRequestif err := json.UnmarshalRead(http.MaxBytesReader(c.Response(), c.Request().Body, 1<<20), &input); err != nil {return c.JSON(http.StatusBadRequest, base.ApiResponse[base.MessageResponse]{Success: false,@@ -137,14 +137,14 @@ func validAgentEventIngestionTokenInternal(r *http.Request, cfg *config.Config)if token == "" || cfg.AgentToken == "" {return false}- // Constant time, matching agentTokenMatchesInternal in the auth middleware:+ // Constant time, matching auth.AgentTokenMatches in the auth middleware:// this endpoint is unauthenticated apart from the token, so a byte-by-byte// comparison is directly probeable.return subtle.ConstantTimeCompare([]byte(token), []byte(cfg.AgentToken)) == 1}// RegisterEvents registers all event management endpoints.-func RegisterEvents(api huma.API, eventService *services.EventService) {+func RegisterEvents(api huma.API, eventService *EventService) {h := &EventHandler{eventService: eventService,}@@ -156,8 +156,8 @@ func RegisterEvents(api huma.API, eventService *services.EventService) {Summary: "List events",Description: "Get a paginated list of system events",Tags: []string{"Events"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermEventsRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermEventsRead),}, h.ListEvents)huma.Register(api, huma.Operation{@@ -167,8 +167,8 @@ func RegisterEvents(api huma.API, eventService *services.EventService) {Summary: "Event severity counts",Description: "Get global event counts grouped by severity",Tags: []string{"Events"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermEventsRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermEventsRead),}, h.GetEventStats)huma.Register(api, huma.Operation{@@ -178,8 +178,8 @@ func RegisterEvents(api huma.API, eventService *services.EventService) {Summary: "Delete an event",Description: "Delete a system event by ID",Tags: []string{"Events"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermEventsDelete),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermEventsDelete),}, h.DeleteEvent)huma.Register(api, huma.Operation{@@ -189,8 +189,8 @@ func RegisterEvents(api huma.API, eventService *services.EventService) {Summary: "Get events by environment",Description: "Get a paginated list of events for a specific environment",Tags: []string{"Events"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermEventsRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermEventsRead),}, h.GetEventsByEnvironment)}@@ -200,7 +200,7 @@ func RegisterEvents(api huma.API, eventService *services.EventService) {// ListEvents returns a paginated list of events.func (h *EventHandler) ListEvents(ctx context.Context, input *ListEventsInput) (*ListEventsOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if input.Severity != "" {params.Filters["severity"] = input.Severity@@ -215,10 +215,10 @@ func (h *EventHandler) ListEvents(ctx context.Context, input *ListEventsInput) (}return &ListEventsOutput{- Body: base.Paginated[event.Event]{+ Body: base.Paginated[eventtypes.Event]{Success: true,Data: events,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -231,7 +231,7 @@ func (h *EventHandler) GetEventStats(ctx context.Context, _ *GetEventStatsInput)}return &GetEventStatsOutput{- Body: base.ApiResponse[services.EventSeverityCounts]{+ Body: base.ApiResponse[EventSeverityCounts]{Success: true,Data: counts,},@@ -244,7 +244,7 @@ func (h *EventHandler) GetEventsByEnvironment(ctx context.Context, input *GetEvereturn nil, huma.Error400BadRequest("Environment ID is required")}- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if input.Severity != "" {params.Filters["severity"] = input.Severity@@ -259,10 +259,10 @@ func (h *EventHandler) GetEventsByEnvironment(ctx context.Context, input *GetEve}return &GetEventsByEnvironmentOutput{- Body: base.Paginated[event.Event]{+ Body: base.Paginated[eventtypes.Event]{Success: true,Data: events,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}diff --git a/backend/internal/event/module.go b/backend/internal/event/module.gonew file mode 100644index 000000000..6bb514732--- /dev/null+++ b/backend/internal/event/module.go@@ -0,0 +1,58 @@+// Package event owns persisted system events, manager ingestion, and event HTTP routes.+package event++import (+ "net/http"++ "github.com/danielgtaylor/huma/v2"+ "github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/labstack/echo/v5"+)++// Dependencies are the collaborators the event domain needs.+type Dependencies struct {+ DB *database.DB+ Config *config.Config+ HTTPClient *http.Client+}++// Module owns event persistence and both of the domain's HTTP surfaces.+type Module struct {+ service *EventService+ config *config.Config+}++// New builds the event domain from its dependencies.+func New(deps Dependencies) *Module {+ return &Module{+ service: NewEventService(deps.DB, deps.Config, deps.HTTPClient),+ config: deps.Config,+ }+}++// Service exposes event operations to collaborating domains.+func (m *Module) Service() *EventService {+ if m == nil {+ return nil+ }+ return m.service+}++// RegisterRoutes mounts the typed event endpoints for runtime and schema discovery.+func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterEvents(api, nil)+ return+ }+ RegisterEvents(api, m.service)+}++// RegisterAgentRoutes mounts the token-authenticated direct-agent ingestion endpoint.+func (m *Module) RegisterAgentRoutes(group *echo.Group) {+ if m == nil {+ RegisterAgentEventIngestion(group, nil, nil)+ return+ }+ RegisterAgentEventIngestion(group, m.service, m.config)+}diff --git a/backend/internal/services/event_service.go b/backend/internal/event/service.gosimilarity index 98%rename from backend/internal/services/event_service.gorename to backend/internal/event/service.goindex 3b6312e2a..7b6cad3d6 100644--- a/backend/internal/services/event_service.go+++ b/backend/internal/event/service.go@@ -1,4 +1,4 @@-package services+package eventimport ("bytes"@@ -21,7 +21,7 @@ import ("github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper"- "github.com/getarcaneapp/arcane/types/v2/event"+ eventtypes "github.com/getarcaneapp/arcane/types/v2/event""github.com/samber/mo""github.com/samber/mo/option""golang.org/x/text/cases"@@ -285,7 +285,7 @@ func copyOptionalStringPtr(value *string) *string {return new(*value)}-func (s *EventService) ListEventsPaginated(ctx context.Context, params pagination.QueryParams) ([]event.Event, pagination.Response, error) {+func (s *EventService) ListEventsPaginated(ctx context.Context, params pagination.QueryParams) ([]eventtypes.Event, pagination.Response, error) {var events []models.Eventq := s.db.WithContext(ctx).Model(&models.Event{})@@ -308,7 +308,7 @@ func (s *EventService) ListEventsPaginated(ctx context.Context, params paginatioreturn nil, pagination.Response{}, errors.WrapIf(err, "failed to paginate events")}- eventDtos, mapErr := mapper.MapSlice[models.Event, event.Event](events)+ eventDtos, mapErr := mapper.MapSlice[models.Event, eventtypes.Event](events)if mapErr != nil {return nil, pagination.Response{}, errors.WrapIf(mapErr, "failed to map events")}@@ -316,7 +316,7 @@ func (s *EventService) ListEventsPaginated(ctx context.Context, params paginatioreturn eventDtos, paginationResp, nil}-func (s *EventService) GetEventsByEnvironmentPaginated(ctx context.Context, environmentID string, params pagination.QueryParams) ([]event.Event, pagination.Response, error) {+func (s *EventService) GetEventsByEnvironmentPaginated(ctx context.Context, environmentID string, params pagination.QueryParams) ([]eventtypes.Event, pagination.Response, error) {var events []models.Eventq := s.db.WithContext(ctx).Model(&models.Event{}).Where("environment_id = ?", environmentID)@@ -338,7 +338,7 @@ func (s *EventService) GetEventsByEnvironmentPaginated(ctx context.Context, envireturn nil, pagination.Response{}, errors.WrapIf(err, "failed to paginate events")}- eventDtos, mapErr := mapper.MapSlice[models.Event, event.Event](events)+ eventDtos, mapErr := mapper.MapSlice[models.Event, eventtypes.Event](events)if mapErr != nil {return nil, pagination.Response{}, errors.WrapIf(mapErr, "failed to map events")}diff --git a/backend/internal/services/event_service_test.go b/backend/internal/event/service_test.gosimilarity index 99%rename from backend/internal/services/event_service_test.gorename to backend/internal/event/service_test.goindex fa7b46d68..4bfe6f785 100644--- a/backend/internal/services/event_service_test.go+++ b/backend/internal/event/service_test.go@@ -1,4 +1,4 @@-package services+package eventimport ("context"diff --git a/backend/api/handlers/federated.go b/backend/internal/federated/handler.gosimilarity index 89%rename from backend/api/handlers/federated.gorename to backend/internal/federated/handler.goindex 625b8d513..769a0f6a3 100644--- a/backend/api/handlers/federated.go+++ b/backend/internal/federated/handler.go@@ -1,4 +1,4 @@-package handlers+package federatedimport ("context"@@ -6,10 +6,10 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/common"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"federatedtypes "github.com/getarcaneapp/arcane/types/v2/federated""github.com/labstack/echo/v5"@@ -18,7 +18,7 @@ import (// FederatedCredentialHandler provides Huma-based federated credential// management endpoints.type FederatedCredentialHandler struct {- federatedCredentialService *services.FederatedCredentialService+ federatedCredentialService *FederatedCredentialService}type federatedTokenExchangeError struct {@@ -74,7 +74,7 @@ type DeleteFederatedCredentialOutput struct {// RegisterFederatedTokenExchange registers the public RFC 8693 token exchange// endpoint. It intentionally uses Echo because the standard requires form// encoding while the rest of Arcane's Huma API is JSON-first.-func RegisterFederatedTokenExchange(g *echo.Group, federatedCredentialService *services.FederatedCredentialService) {+func RegisterFederatedTokenExchange(g *echo.Group, federatedCredentialService *FederatedCredentialService) {g.POST("/auth/federated/token", func(c *echo.Context) error {if federatedCredentialService == nil {return c.JSON(http.StatusInternalServerError, federatedTokenExchangeError{@@ -106,7 +106,7 @@ func RegisterFederatedTokenExchange(g *echo.Group, federatedCredentialService *s}// RegisterFederatedCredentials registers federated credential management routes.-func RegisterFederatedCredentials(api huma.API, federatedCredentialService *services.FederatedCredentialService) {+func RegisterFederatedCredentials(api huma.API, federatedCredentialService *FederatedCredentialService) {h := &FederatedCredentialHandler{federatedCredentialService: federatedCredentialService,}@@ -118,8 +118,8 @@ func RegisterFederatedCredentials(api huma.API, federatedCredentialService *servSummary: "List federated credentials",Description: "Get a paginated list of workload identity federation trust rules",Tags: []string{"Federated Credentials"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermFederatedList),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermFederatedList),}, h.ListFederatedCredentials)huma.Register(api, huma.Operation{@@ -129,8 +129,8 @@ func RegisterFederatedCredentials(api huma.API, federatedCredentialService *servSummary: "Create a federated credential",Description: "Create a workload identity federation trust rule",Tags: []string{"Federated Credentials"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.CreateFederatedCredential)huma.Register(api, huma.Operation{@@ -140,8 +140,8 @@ func RegisterFederatedCredentials(api huma.API, federatedCredentialService *servSummary: "Get a federated credential",Description: "Get details of a workload identity federation trust rule",Tags: []string{"Federated Credentials"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermFederatedRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermFederatedRead),}, h.GetFederatedCredential)huma.Register(api, huma.Operation{@@ -151,8 +151,8 @@ func RegisterFederatedCredentials(api huma.API, federatedCredentialService *servSummary: "Update a federated credential",Description: "Update a workload identity federation trust rule",Tags: []string{"Federated Credentials"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.UpdateFederatedCredential)huma.Register(api, huma.Operation{@@ -162,13 +162,13 @@ func RegisterFederatedCredentials(api huma.API, federatedCredentialService *servSummary: "Delete a federated credential",Description: "Delete a workload identity federation trust rule and its service user",Tags: []string{"Federated Credentials"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.DeleteFederatedCredential)}func (h *FederatedCredentialHandler) ListFederatedCredentials(ctx context.Context, input *ListFederatedCredentialsInput) (*ListFederatedCredentialsOutput, error) {- credentials, paginationResp, err := h.federatedCredentialService.List(ctx, buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search))+ credentials, paginationResp, err := h.federatedCredentialService.List(ctx, handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search))if err != nil {return nil, huma.Error500InternalServerError("failed to list federated credentials")}@@ -177,13 +177,13 @@ func (h *FederatedCredentialHandler) ListFederatedCredentials(ctx context.ContexBody: base.Paginated[federatedtypes.FederatedCredential]{Success: true,Data: credentials,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}func (h *FederatedCredentialHandler) CreateFederatedCredential(ctx context.Context, input *CreateFederatedCredentialInput) (*CreateFederatedCredentialOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -216,7 +216,7 @@ func (h *FederatedCredentialHandler) GetFederatedCredential(ctx context.Context,}func (h *FederatedCredentialHandler) UpdateFederatedCredential(ctx context.Context, input *UpdateFederatedCredentialInput) (*UpdateFederatedCredentialOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}diff --git a/backend/internal/federated/playwright.go b/backend/internal/federated/playwright.gonew file mode 100644index 000000000..73d20d5d0--- /dev/null+++ b/backend/internal/federated/playwright.go@@ -0,0 +1,70 @@+//go:build playwright++package federated++import (+ "context"+ "strings"++ "emperror.dev/errors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/google/uuid"+ "gorm.io/gorm"+)++// CreatePlaywrightCredential creates the service identity, credential, and role+// assignment required by the federated-auth end-to-end flow in one transaction.+func (s *FederatedCredentialService) CreatePlaywrightCredential(ctx context.Context, issuerURL string, audiences []string, subject string, roleID string, tokenTTLSeconds int) (string, error) {+ if s == nil || s.db == nil {+ return "", errors.New("federated credential service is not available")+ }+ if strings.TrimSpace(issuerURL) == "" || strings.TrimSpace(subject) == "" || strings.TrimSpace(roleID) == "" || len(audiences) == 0 {+ return "", errors.New("issuerUrl, subject, roleId, and audiences are required")+ }+ if tokenTTLSeconds <= 0 {+ tokenTTLSeconds = 600+ }++ var credentialID string+ err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {+ serviceUser := models.User{+ Username: "svc_federated_e2e_" + strings.ReplaceAll(uuid.NewString(), "-", ""),+ IsServiceAccount: true,+ }+ if err := tx.Create(&serviceUser).Error; err != nil {+ return errors.WrapIf(err, "failed to create federated e2e user")+ }++ credential := models.FederatedCredential{+ Name: "Playwright Federated Credential",+ Enabled: true,+ IssuerURL: strings.TrimRight(strings.TrimSpace(issuerURL), "/"),+ Audiences: models.StringSlice(audiences),+ SubjectClaim: "sub",+ SubjectMatch: strings.TrimSpace(subject),+ MatchType: models.FederatedCredentialMatchExact,+ RoleID: strings.TrimSpace(roleID),+ IdentityUserID: serviceUser.ID,+ TokenTTLSeconds: tokenTTLSeconds,+ }+ if err := tx.Create(&credential).Error; err != nil {+ return errors.WrapIf(err, "failed to create federated e2e credential")+ }++ assignment := models.UserRoleAssignment{+ UserID: serviceUser.ID,+ RoleID: credential.RoleID,+ Source: models.RoleAssignmentSourceManual,+ }+ if err := tx.Create(&assignment).Error; err != nil {+ return errors.WrapIf(err, "failed to create federated e2e role assignment")+ }++ credentialID = credential.ID+ return nil+ })+ if err != nil {+ return "", err+ }+ return credentialID, nil+}diff --git a/backend/internal/services/federated_credential_service.go b/backend/internal/federated/service.gosimilarity index 96%rename from backend/internal/services/federated_credential_service.gorename to backend/internal/federated/service.goindex 8f41aa1c4..d031507d0 100644--- a/backend/internal/services/federated_credential_service.go+++ b/backend/internal/federated/service.go@@ -1,4 +1,4 @@-package services+package federatedimport ("context"@@ -20,9 +20,14 @@ import ("golang.org/x/sync/singleflight""gorm.io/gorm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/dbutil"@@ -38,11 +43,11 @@ const (type FederatedCredentialService struct {db *database.DB- authService *AuthService- userService *UserService- settingsService *SettingsService- eventService *EventService- roleService *RoleService+ authService *auth.AuthService+ userService *user.UserService+ settingsService *settings.SettingsService+ eventService *event.EventService+ roleService *role.RoleServicehttpClient *http.ClientproviderMu sync.RWMutexproviders map[string]*oidc.Provider@@ -51,10 +56,10 @@ type FederatedCredentialService struct {func NewFederatedCredentialService(db *database.DB,- authService *AuthService,- userService *UserService,- settingsService *SettingsService,- eventService *EventService,+ authService *auth.AuthService,+ userService *user.UserService,+ settingsService *settings.SettingsService,+ eventService *event.EventService,httpClient *http.Client,) *FederatedCredentialService {if httpClient == nil {@@ -72,7 +77,7 @@ func NewFederatedCredentialService(}}-func (s *FederatedCredentialService) WithRoleService(roleService *RoleService) *FederatedCredentialService {+func (s *FederatedCredentialService) WithRoleService(roleService *role.RoleService) *FederatedCredentialService {s.roleService = roleServicereturn s}@@ -594,7 +599,7 @@ func normalizeCreateFederatedCredentialInternal(req federatedtypes.CreateFederatreq.MatchType = normalizeMatchTypeInternal(req.MatchType)req.Audiences = pkgutils.UniqueNonEmptyStrings(req.Audiences)req.EnvironmentID = mo.EmptyableToOption(strings.TrimSpace(mo.PointerToOption(req.EnvironmentID).OrEmpty())).ToPointer()- req.TokenTTLSeconds = clampFederatedTokenTTLSecondsInternal(req.TokenTTLSeconds)+ req.TokenTTLSeconds = auth.ClampFederatedTokenTTLSeconds(req.TokenTTLSeconds)if req.SubjectClaim == "" {req.SubjectClaim = defaultFederatedSubjectClaim@@ -664,7 +669,7 @@ func applyFederatedCredentialUpdateInternal(existing models.FederatedCredential,return existing, false, err}if req.TokenTTLSeconds != nil {- existing.TokenTTLSeconds = clampFederatedTokenTTLSecondsInternal(*req.TokenTTLSeconds)+ existing.TokenTTLSeconds = auth.ClampFederatedTokenTTLSeconds(*req.TokenTTLSeconds)}if req.ExpiresAt != nil {existing.ExpiresAt = req.ExpiresAt@@ -853,7 +858,7 @@ func (s *FederatedCredentialService) logExchangeInternal(ctx context.Context, rego func() {bgCtx := context.WithoutCancel(ctx)- _, err := s.eventService.CreateEvent(bgCtx, CreateEventRequest{+ _, err := s.eventService.CreateEvent(bgCtx, event.CreateEventRequest{Type: models.EventTypeFederatedExchange,Severity: severity,Title: title,diff --git a/backend/internal/services/federated_credential_service_test.go b/backend/internal/federated/service_test.gosimilarity index 87%rename from backend/internal/services/federated_credential_service_test.gorename to backend/internal/federated/service_test.goindex 2df7f09cf..adb43a94a 100644--- a/backend/internal/services/federated_credential_service_test.go+++ b/backend/internal/federated/service_test.go@@ -1,4 +1,4 @@-package services+package federatedimport ("context"@@ -18,12 +18,20 @@ import ("github.com/golang-jwt/jwt/v5""github.com/libtnb/sqlite""github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest""gorm.io/gorm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"federatedtypes "github.com/getarcaneapp/arcane/types/v2/federated""github.com/stretchr/testify/assert"@@ -130,18 +138,18 @@ func setupFederatedCredentialServiceTestDBInternal(t *testing.T) *database.DB {return &database.DB{DB: db}}-func setupFederatedCredentialServiceInternal(t *testing.T, issuer *federatedTestIssuerInternal) (*FederatedCredentialService, *AuthService, *database.DB) {+func setupFederatedCredentialServiceInternal(t *testing.T, issuer *federatedTestIssuerInternal) (*FederatedCredentialService, *auth.AuthService, *database.DB) {t.Helper()ctx := context.Background()db := setupFederatedCredentialServiceTestDBInternal(t)- roleSvc := NewRoleService(db)- userSvc := NewUserService(db).WithRoleService(roleSvc)- sessionSvc := NewSessionService(db)+ roleSvc := role.NewRoleService(db)+ userSvc := user.NewUserService(db).WithRoleService(roleSvc)+ sessionSvc := session.NewSessionService(db)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventSvc := NewEventService(db, &config.Config{}, nil)- authSvc := NewAuthService(userSvc, settingsSvc, eventSvc, sessionSvc, roleSvc, "test-federated-secret", &config.Config{+ eventSvc := event.NewEventService(db, &config.Config{}, nil)+ authSvc := auth.NewAuthService(userSvc, settingsSvc, eventSvc, sessionSvc, roleSvc, "test-federated-secret", &config.Config{JWTRefreshExpiry: 24 * time.Hour,}, nil)@@ -366,3 +374,24 @@ func TestFederatedCredentialServiceCreateRejectsBareWildcardGlob(t *testing.T) {require.Error(t, err)require.True(t, errors.Is(err, common.ErrFederatedCredentialInvalid), "unexpected error: %v", err)}++// Test fixtures shared by this package's tests.++func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ executor, err := actors.NewExecutor(t.Context(), runtime, "settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, executor.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, executor, effects)+}diff --git a/backend/internal/services/gitops_sync_service.go b/backend/internal/gitops/gitops_sync.gosimilarity index 87%rename from backend/internal/services/gitops_sync_service.gorename to backend/internal/gitops/gitops_sync.goindex 901e3a0e2..c8ba9976e 100644--- a/backend/internal/services/gitops_sync_service.go+++ b/backend/internal/gitops/gitops_sync.go@@ -1,4 +1,4 @@-package services+package gitopsimport ("context"@@ -17,31 +17,36 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitrepo"+ "github.com/getarcaneapp/arcane/backend/v2/internal/lifecycle""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/swarm""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/scheduler/entityjobs""github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper""github.com/getarcaneapp/arcane/types/v2/gitops"projecttypes "github.com/getarcaneapp/arcane/types/v2/project"schedulertypes "github.com/getarcaneapp/arcane/types/v2/scheduler"- "github.com/getarcaneapp/arcane/types/v2/swarm"- "github.com/moby/moby/client"+ swarmtypes "github.com/getarcaneapp/arcane/types/v2/swarm""gorm.io/gorm")type GitOpsSyncService struct {db *database.DB- repoService *GitRepositoryService- projectService *ProjectService- swarmService *SwarmService- eventService *EventService- settingsService *SettingsService+ repoService *gitrepo.GitRepositoryService+ projectService *project.ProjectService+ swarmService *swarm.SwarmService+ eventService *event.EventService+ settingsService *settings.SettingsService- // scheduler and lifecycleCtx are injected post-construction via SetScheduler.- scheduler schedulertypes.DynamicScheduler- lifecycleCtx context.Context- admissionGate *actors.Gate[actors.AdmissionKey]+ // jobs carries the scheduler and app lifecycle context, injected+ // post-construction via SetScheduler.+ jobs *entityjobs.Registry}const defaultGitSyncTimeout = 5 * time.Minute@@ -143,17 +148,17 @@ func (s *GitOpsSyncService) validateLifecycleConfigInternal(ctx context.Context,//nolint:revive // Preserve the existing API validation message.return common.Classify(common.ErrValidation, errors.WithDetails(errors.New("Timeout must be at least 1 second."), "field", "preDeployTimeoutSec"))}- maxTimeoutSec := s.settingsService.GetIntSetting(ctx, "lifecycleMaxTimeoutSec", lifecycleDefaultMaxTimeoutSec)+ maxTimeoutSec := s.settingsService.GetIntSetting(ctx, "lifecycleMaxTimeoutSec", lifecycle.DefaultMaxTimeoutSec)if maxTimeoutSec > 0 && *in.timeoutSec > maxTimeoutSec {//nolint:revive // Preserve the existing API validation message.return common.Classify(common.ErrValidation, errors.WithDetails(errors.Errorf("Timeout %ds exceeds the lifecycleMaxTimeoutSec setting (%ds).", *in.timeoutSec, maxTimeoutSec), "field", "preDeployTimeoutSec"))}}- if _, err := parseLifecycleEnvTextInternal(in.env); err != nil {+ if _, err := lifecycle.ParseEnvText(in.env); err != nil {return common.Classify(common.ErrValidation, errors.WithDetails(errors.New(err.Error()), "field", "preDeployEnv"))}- if _, err := parseLifecycleExtraMountsTextInternal(in.extraMounts); err != nil {+ if _, err := lifecycle.ParseExtraMountsText(in.extraMounts); err != nil {return common.Classify(common.ErrValidation, errors.WithDetails(errors.New(err.Error()), "field", "preDeployExtraMounts"))}@@ -254,7 +259,7 @@ func applyLifecycleFieldsToSyncInternal(sync *models.GitOpsSync, in lifecycleConif in.timeoutSec != nil {sync.PreDeployTimeoutSec = *in.timeoutSec} else {- sync.PreDeployTimeoutSec = lifecycleDefaultTimeoutSec+ sync.PreDeployTimeoutSec = lifecycle.DefaultTimeoutSec}if mode := normalizeLifecycleNetworkModeInternal(in.networkMode); mode != "" {sync.PreDeployNetworkMode = mode@@ -338,7 +343,7 @@ func hasEstablishedProjectBindingInternal(sync *models.GitOpsSync) bool {return sync != nil && sync.ProjectID != nil && strings.TrimSpace(*sync.ProjectID) != ""}-func NewGitOpsSyncService(db *database.DB, repoService *GitRepositoryService, projectService *ProjectService, swarmService *SwarmService, eventService *EventService, settingsService *SettingsService) *GitOpsSyncService {+func NewGitOpsSyncService(db *database.DB, repoService *gitrepo.GitRepositoryService, projectService *project.ProjectService, swarmService *swarm.SwarmService, eventService *event.EventService, settingsService *settings.SettingsService) *GitOpsSyncService {return &GitOpsSyncService{db: db,repoService: repoService,@@ -346,57 +351,18 @@ func NewGitOpsSyncService(db *database.DB, repoService *GitRepositoryService, prswarmService: swarmService,eventService: eventService,settingsService: settingsService,+ jobs: entityjobs.New(entityjobs.GitOpsSyncJobPrefix, gitOpsSyncAdmissionScopeInternal),}}-const gitOpsSyncJobPrefix = "gitops-sync:"const gitOpsSyncAdmissionScopeInternal = "gitops-sync"-func gitOpsSyncJobNameInternal(syncID string) string { return gitOpsSyncJobPrefix + syncID }-// SetScheduler injects the job scheduler and the app lifecycle context. It must be// called during bootstrap (after the service graph is built) before any per-sync// jobs are registered. The lifecycle context is used for background sync kicks so// they outlive the request/bootstrap goroutine that triggered them.-func (s *GitOpsSyncService) SetScheduler(ctx context.Context, scheduler schedulertypes.DynamicScheduler, admissionGate *actors.Gate[actors.AdmissionKey]) error { //nolint:contextcheck // background sync kicks must capture the app lifecycle context, not request contexts- if scheduler == nil || admissionGate == nil {- return errors.New("GitOps scheduler dependencies unavailable")- }- if ctx == nil {- ctx = context.Background()- }- s.lifecycleCtx = ctx- s.scheduler = scheduler- s.admissionGate = admissionGate- return nil-}--func (s *GitOpsSyncService) schedulerCtxInternal(ctx context.Context) context.Context {- if s.lifecycleCtx != nil {- return s.lifecycleCtx- }- if ctx != nil {- return context.WithoutCancel(ctx)- }- return context.Background()-}--// buildSyncJobInternal returns the dynamic job for a single sync. The schedule is a-// fixed "@every Nm" interval; the run body re-reads the sync each fire (so a row-// deleted or toggled to AutoSync=false self-cancels cleanly) and delegates to-// PerformSync, which owns its own timeout and the per-sync in-flight guard.-func (s *GitOpsSyncService) buildSyncJobInternal(syncID, environmentID string, intervalMinutes int) *schedulertypes.GenericJob {- interval := max(intervalMinutes, 1)- schedule := fmt.Sprintf("@every %dm", interval)- return &schedulertypes.GenericJob{- JobName: gitOpsSyncJobNameInternal(syncID),- ScheduleFn: func(_ context.Context) string {- return schedule- },- RunFn: func(ctx context.Context) {- s.runScheduledSyncInternal(ctx, environmentID, syncID)- },- }+func (s *GitOpsSyncService) SetScheduler(ctx context.Context, scheduler schedulertypes.DynamicScheduler, admissionGate *actors.Gate[actors.AdmissionKey]) error {+ return s.jobs.SetScheduler(ctx, scheduler, admissionGate)}// runScheduledSyncInternal is the body of a scheduled sync fire. It re-reads the@@ -417,36 +383,34 @@ func (s *GitOpsSyncService) runScheduledSyncInternal(ctx context.Context, enviroif !syncRecord.AutoSync {return}- if _, err := s.PerformSync(ctx, environmentID, syncID, systemUser); err != nil {+ if _, err := s.PerformSync(ctx, environmentID, syncID, models.SystemUser); err != nil {slog.ErrorContext(ctx, "gitops auto-sync run failed", "syncId", syncID, "error", err)}}+// registerSyncJobInternal schedules a single sync on a fixed "@every Nm"+// interval. The run body re-reads the sync each fire, so a row deleted or+// toggled to AutoSync=false self-cancels cleanly, and delegates to PerformSync,+// which owns its own timeout and the per-sync in-flight guard.func (s *GitOpsSyncService) registerSyncJobInternal(ctx context.Context, syncID, environmentID string, intervalMinutes int) {- if s.scheduler == nil {- return- }- job := s.buildSyncJobInternal(syncID, environmentID, intervalMinutes)- schedulerCtx := s.schedulerCtxInternal(ctx)- if err := s.scheduler.AddJob(schedulerCtx, job); err != nil {- slog.ErrorContext(schedulerCtx, "Failed to register gitops sync job", "syncId", syncID, "error", err)- }+ schedule := fmt.Sprintf("@every %dm", max(intervalMinutes, 1))+ s.jobs.Register(ctx, syncID,+ func(_ context.Context) string { return schedule },+ func(ctx context.Context) { s.runScheduledSyncInternal(ctx, environmentID, syncID) },+ )}func (s *GitOpsSyncService) unregisterSyncJobInternal(ctx context.Context, syncID string) {- if s.scheduler == nil {- return- }- s.scheduler.RemoveJob(s.schedulerCtxInternal(ctx), gitOpsSyncJobNameInternal(syncID))+ s.jobs.Unregister(ctx, syncID)}// kickSyncInternal runs a sync once in the background on the app lifecycle context.// Used when auto-sync is freshly enabled or when a sync is overdue at startup, so// the first run does not wait a full interval.func (s *GitOpsSyncService) kickSyncInternal(ctx context.Context, syncID, environmentID string) {- ctx = s.schedulerCtxInternal(ctx)+ ctx = s.jobs.Context(ctx)go func() {- if _, err := s.PerformSync(ctx, environmentID, syncID, systemUser); err != nil {+ if _, err := s.PerformSync(ctx, environmentID, syncID, models.SystemUser); err != nil {slog.ErrorContext(ctx, "gitops immediate sync kick failed", "syncId", syncID, "error", err)}}()@@ -456,7 +420,7 @@ func (s *GitOpsSyncService) kickSyncInternal(ctx context.Context, syncID, enviro// sync and kicks an immediate run for any that are overdue. This replaces the old// global polling job so existing syncs keep running after upgrade.func (s *GitOpsSyncService) RegisterAutoSyncJobsOnStartup(ctx context.Context) {- if s.scheduler == nil {+ if !s.jobs.Enabled() {return}var syncs []models.GitOpsSync@@ -521,7 +485,7 @@ func (s *GitOpsSyncService) getEffectiveSyncLimits(ctx context.Context, sync *mo}func (s *GitOpsSyncService) gitSyncLimitEnvOverrideActiveInternal(key string) bool {- return s.settingsService != nil && s.settingsService.isEnvOverrideActiveInternal(key)+ return s.settingsService != nil && s.settingsService.IsEnvOverrideActive(key)}func (s *GitOpsSyncService) GetSyncsPaginated(ctx context.Context, environmentID string, params pagination.QueryParams) ([]gitops.GitOpsSync, pagination.Response, gitops.SyncCounts, error) {@@ -700,7 +664,7 @@ func (s *GitOpsSyncService) CreateSync(ctx context.Context, environmentID stringslog.InfoContext(ctx, "GitOps sync created successfully", "syncID", syncRecord.ID, "name", syncRecord.Name)// Log event- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitSyncCreate,Severity: models.EventSeveritySuccess,Title: "Git sync created",@@ -812,7 +776,7 @@ func (s *GitOpsSyncService) UpdateSync(ctx context.Context, environmentID, id st}// Log event- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitSyncUpdate,Severity: models.EventSeveritySuccess,Title: "Git sync updated",@@ -883,7 +847,7 @@ func (s *GitOpsSyncService) DeleteSync(ctx context.Context, environmentID, id st}// Log event- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitSyncDelete,Severity: models.EventSeverityInfo,Title: "Git sync deleted",@@ -902,7 +866,7 @@ func (s *GitOpsSyncService) DeleteSync(ctx context.Context, environmentID, id stfunc (s *GitOpsSyncService) PerformSync(ctx context.Context, environmentID, id string, actor models.User) (*gitops.SyncResult, error) {// Coalesce overlapping runs for the same sync (scheduled fire, startup/enable// kick, manual trigger, webhook) so they don't race the clone/redeploy.- lease, admitted, err := s.admissionGate.TryAcquire(ctx, actors.AdmissionKey{Scope: gitOpsSyncAdmissionScopeInternal, ID: id})+ lease, admitted, err := s.jobs.TryAcquire(ctx, id)if err != nil {return nil, errors.WrapIf(err, "failed to admit GitOps sync")}@@ -928,7 +892,7 @@ func (s *GitOpsSyncService) PerformSync(ctx context.Context, environmentID, id ssource, err := s.prepareSyncSource(syncCtx, syncRecord, result, actor)if source != nil && source.repoPath != "" {defer func() {- if cleanupErr := s.repoService.gitClient.Cleanup(source.repoPath); cleanupErr != nil {+ if cleanupErr := s.repoService.Cleanup(source.repoPath); cleanupErr != nil {slog.WarnContext(syncCtx, "Failed to cleanup repository", "path", source.repoPath, "error", cleanupErr)}}()@@ -964,23 +928,23 @@ func (s *GitOpsSyncService) prepareSyncSource(ctx context.Context, sync *models.return nil, s.failSync(ctx, sync.ID, result, sync, actor, "Failed to get authentication config", err.Error())}- repoPath, err := s.repoService.gitClient.Clone(ctx, repository.URL, sync.Branch, authConfig)+ repoPath, err := s.repoService.Clone(ctx, repository.URL, sync.Branch, authConfig)if err != nil {return nil, s.failSync(ctx, sync.ID, result, sync, actor, "Failed to clone repository", err.Error())}- commitHash, err := s.repoService.gitClient.GetCurrentCommit(ctx, repoPath)+ commitHash, err := s.repoService.GetCurrentCommit(ctx, repoPath)if err != nil {slog.WarnContext(ctx, "Failed to get commit hash", "error", err)commitHash = ""}- if !s.repoService.gitClient.FileExists(ctx, repoPath, sync.ComposePath) {+ if !s.repoService.FileExists(ctx, repoPath, sync.ComposePath) {errMsg := "compose file not found: " + sync.ComposePathreturn &preparedSyncSource{repoPath: repoPath, commitHash: commitHash}, s.failSync(ctx, sync.ID, result, sync, actor, "Compose file not found at "+sync.ComposePath, errMsg)}- composeContent, err := s.repoService.gitClient.ReadFile(ctx, repoPath, sync.ComposePath)+ composeContent, err := s.repoService.ReadFile(ctx, repoPath, sync.ComposePath)if err != nil {return &preparedSyncSource{repoPath: repoPath, commitHash: commitHash}, s.failSync(ctx, sync.ID, result, sync, actor, "Failed to read compose file", err.Error())}@@ -992,8 +956,8 @@ func (s *GitOpsSyncService) prepareSyncSource(ctx context.Context, sync *models.}envPath := filepath.Join(filepath.Dir(sync.ComposePath), ".env")- if s.repoService.gitClient.FileExists(ctx, repoPath, envPath) {- content, err := s.repoService.gitClient.ReadFile(ctx, repoPath, envPath)+ if s.repoService.FileExists(ctx, repoPath, envPath) {+ content, err := s.repoService.ReadFile(ctx, repoPath, envPath)if err != nil {slog.WarnContext(ctx, "Failed to read .env file", "path", envPath, "error", err)} else {@@ -1010,10 +974,10 @@ func (s *GitOpsSyncService) prepareSyncSource(ctx context.Context, sync *models.composeDir := filepath.Dir(sync.ComposePath)overrideName, overrideContent, overrideFound, overrideErr := projects.ResolveComposeOverride(func(name string) bool {- return s.repoService.gitClient.FileExists(ctx, repoPath, filepath.Join(composeDir, name))+ return s.repoService.FileExists(ctx, repoPath, filepath.Join(composeDir, name))},func(name string) (string, error) {- return s.repoService.gitClient.ReadFile(ctx, repoPath, filepath.Join(composeDir, name))+ return s.repoService.ReadFile(ctx, repoPath, filepath.Join(composeDir, name))},)// Fail the sync on a read error instead of degrading: WriteComposeOverrideFile@@ -1132,17 +1096,17 @@ func (s *GitOpsSyncService) performSwarmStackSyncInternal(ctx context.Context, ssyncFiles = files}- swarmFiles := make([]swarm.SyncFile, len(syncFiles))+ swarmFiles := make([]swarmtypes.SyncFile, len(syncFiles))syncedFiles := make([]string, 0, len(syncFiles))for i, f := range syncFiles {- swarmFiles[i] = swarm.SyncFile{+ swarmFiles[i] = swarmtypes.SyncFile{RelativePath: f.RelativePath,Content: f.Content,}syncedFiles = append(syncedFiles, f.RelativePath)}- req := swarm.StackDeployRequest{+ req := swarmtypes.StackDeployRequest{Name: sync.ProjectName,ComposeContent: source.composeContent,OverrideContent: overrideContent,@@ -1164,7 +1128,7 @@ func (s *GitOpsSyncService) performSwarmStackSyncInternal(ctx context.Context, sresult.Message = fmt.Sprintf("Successfully deployed swarm stack %s from %s", sync.ProjectName, sync.ComposePath)// Log event- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitSyncRun,Severity: models.EventSeveritySuccess,Title: "Git sync completed for stack",@@ -1205,7 +1169,7 @@ func (s *GitOpsSyncService) redeployIfRunningAfterSync(ctx context.Context, proj// logSyncSuccess records the Git sync completion event once the filesystem and// sync-status updates have already succeeded.func (s *GitOpsSyncService) logSyncSuccess(ctx context.Context, sync *models.GitOpsSync, project *models.Project, actor models.User) {- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitSyncRun,Severity: models.EventSeveritySuccess,Title: "Git sync completed",@@ -1271,7 +1235,7 @@ func (s *GitOpsSyncService) GetSyncStatus(ctx context.Context, environmentID, id// discovery imports them as phantom projects. It is safe to run at startup because it// executes before any sync job is registered, so nothing is mid-stage.func (s *GitOpsSyncService) CleanupLeakedScratchDirsOnStartup(ctx context.Context) error {- projectsDir, err := s.projectService.getProjectsDirectoryInternal(ctx)+ projectsDir, err := s.projectService.GetProjectsDirectory(ctx)if err != nil {return errors.WrapIf(err, "failed to resolve projects directory for gitops scratch cleanup")}@@ -1383,18 +1347,18 @@ func (s *GitOpsSyncService) BrowseFiles(ctx context.Context, environmentID, id s}// Clone the repository- repoPath, err := s.repoService.gitClient.Clone(browseCtx, repository.URL, syncRecord.Branch, authConfig)+ repoPath, err := s.repoService.Clone(browseCtx, repository.URL, syncRecord.Branch, authConfig)if err != nil {return nil, errors.WrapIf(err, "failed to clone repository")}defer func() {- if cleanupErr := s.repoService.gitClient.Cleanup(repoPath); cleanupErr != nil {+ if cleanupErr := s.repoService.Cleanup(repoPath); cleanupErr != nil {slog.WarnContext(browseCtx, "Failed to cleanup repository", "path", repoPath, "error", cleanupErr)}}()// Browse the tree- files, err := s.repoService.gitClient.BrowseTree(browseCtx, repoPath, path)+ files, err := s.repoService.BrowseTree(browseCtx, repoPath, path)if err != nil {return nil, err}@@ -1448,7 +1412,7 @@ func (s *GitOpsSyncService) ImportSyncs(ctx context.Context, environmentID strin}func (s *GitOpsSyncService) logSyncError(ctx context.Context, sync *models.GitOpsSync, actor models.User, errorMsg string) {- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitSyncError,Severity: models.EventSeverityError,Title: "Git sync failed",@@ -1519,7 +1483,7 @@ func (s *GitOpsSyncService) createProjectForSyncInternal(ctx context.Context, sy// Use the non-suffixing create: a GitOps sync must never mint a "-N" duplicate.// A name collision means a project directory already exists for this name, so the// binding is broken — fail loudly and disable auto-sync instead of duplicating.- project, err := s.projectService.createProjectInternal(ctx, sync.ProjectName, composeContent, envContent, nil, actor, false)+ project, err := s.projectService.CreateProject(ctx, sync.ProjectName, composeContent, envContent, nil, actor, false)if err != nil {if errors.Is(err, projects.ErrProjectDirExists) {bindingErr := common.Classify(common.ErrGitOpsSyncProjectBindingBroken, errors.WrapIf(errors.Errorf("sync %s cannot create project %q: a directory with that name already exists; refusing to create a duplicate", sync.ID, projects.SanitizeProjectName(sync.ProjectName)), "GitOps sync project binding broken"))@@ -1651,7 +1615,7 @@ func (s *GitOpsSyncService) walkAndParseSyncDirectory(ctx context.Context, sync// Walk the directory to get all filesmaxFiles, maxTotalSize, maxBinarySize := s.getEffectiveSyncLimits(ctx, sync)- walkResult, err := s.repoService.gitClient.WalkDirectory(ctx, repoPath, sync.ComposePath, maxFiles, maxTotalSize, maxBinarySize)+ walkResult, err := s.repoService.WalkDirectory(ctx, repoPath, sync.ComposePath, maxFiles, maxTotalSize, maxBinarySize)if err != nil {return nil, errors.WrapIf(err, "failed to walk directory")}@@ -1722,13 +1686,13 @@ func (s *GitOpsSyncService) syncProjectDirectoryInternal(ctx context.Context, sy// stageDirectorySyncInternal builds a temporary project tree that reflects the exact// repo layout after sync, including cleanup of files removed from the repo.func (s *GitOpsSyncService) stageDirectorySyncInternal(ctx context.Context, sync *models.GitOpsSync, syncFiles []projects.SyncFile) (*stagedDirectorySync, error) {- projectsDir, err := s.projectService.getProjectsDirectoryInternal(ctx)+ projectsDir, err := s.projectService.GetProjectsDirectory(ctx)if err != nil {return nil, errors.WrapIf(err, "failed to get projects directory")}// The project-root env files are reserved for the three-file override- // merge applied below (applyGitSyncEnvToDirInternal) rather than a raw+ // merge applied below by the Project domain rather than a raw// overwrite — a raw .env write would silently wipe edits made in Arcane// on every sync.filteredSyncFiles, gitEnvContent := partitionReservedRootEnvFilesInternal(ctx, syncFiles)@@ -1772,7 +1736,7 @@ func (s *GitOpsSyncService) stageDirectorySyncInternal(ctx context.Context, sync// Syncs created before this fix may still have .env recorded as a tracked// file. Drop reserved root env files here too, or CleanupRemovedFiles would// treat .env as removed-by-git and delete the live-copied stage .env before- // applyGitSyncEnvToDirInternal gets a chance to read it for the merge.+ // ApplyGitSyncEnvToDirectory gets a chance to read it for the merge.oldSyncedFiles := filterReservedRootEnvFilesInternal(parseSyncedFiles(sync.SyncedFiles))if len(oldSyncedFiles) > 0 {if err := projects.CleanupRemovedFiles(projectsDir, stagePath, oldSyncedFiles, syncedFiles); err != nil {@@ -1807,7 +1771,7 @@ func (s *GitOpsSyncService) stageDirectorySyncInternal(ctx context.Context, sync// Route the project-root .env through the same three-file override merge// single-file git sync uses: git is source-of-truth, edits made in Arcane// become an override that wins, and new git-introduced keys still flow in.- preEnvContent, postEnvContent, err := s.applyGitSyncEnvToDirInternal(stagePath, projectsDir, gitEnvContent)+ preEnvContent, postEnvContent, err := s.projectService.ApplyGitSyncEnvToDirectory(stagePath, projectsDir, gitEnvContent)if err != nil {_ = os.RemoveAll(stagePath)return nil, err@@ -1816,7 +1780,7 @@ func (s *GitOpsSyncService) stageDirectorySyncInternal(ctx context.Context, synccontentsChanged = true}- serviceCount, err := s.validateDirectorySyncStageInternal(ctx, sync.ProjectName, stagePath, composeFileName)+ serviceCount, err := s.projectService.ValidateComposeDirectory(ctx, sync.ProjectName, stagePath, composeFileName)if err != nil {_ = os.RemoveAll(stagePath)return nil, errors.WrapIf(err, "invalid compose file")@@ -1836,7 +1800,7 @@ func (s *GitOpsSyncService) stageDirectorySyncInternal(ctx context.Context, sync// isReservedRootEnvFileInternal reports whether relPath is one of the// project-root env bookkeeping files the override-merge system owns (.env,// .env.git, project.env, .env.global). These are excluded from the raw-// directory-sync write and routed through applyGitSyncEnvToDirInternal+// directory-sync write and routed through ProjectService.ApplyGitSyncEnvToDirectory// instead. Exact-matching RelativePath intentionally leaves nested env files// (e.g. svc/.env) in the raw write set — only the project root is// merge-managed.@@ -1853,7 +1817,7 @@ func isReservedRootEnvFileInternal(relPath string) bool {// tracked synced-file list. Syncs created before the override-merge migration// may still have .env recorded from an earlier sync; leaving it in would make// CleanupRemovedFiles treat .env as removed-by-git and delete the live-copied-// stage .env before applyGitSyncEnvToDirInternal can read it for the merge.+// stage .env before ApplyGitSyncEnvToDirectory can read it for the merge.func filterReservedRootEnvFilesInternal(paths []string) []string {filtered := make([]string, 0, len(paths))for _, p := range paths {@@ -1895,32 +1859,6 @@ func partitionReservedRootEnvFilesInternal(ctx context.Context, syncFiles []projreturn filtered, gitEnvContent}-// applyGitSyncEnvToDirInternal routes a directory sync's project-root .env-// through the same three-file override merge single-file git sync uses-// (ProjectService.ApplyGitSyncProjectFiles), so edits made in Arcane become an-// override that wins over git while new git-introduced keys still flow in.-// dirPath already contains whatever .env/.env.git/project.env were-// copied/seeded from the live project (or a pre-existing candidate directory-// for a new project), so the existing single-file machinery works against it-// unchanged. Returns the effective .env content before and after the merge so-// the caller can decide whether the change should trigger a redeploy.-func (s *GitOpsSyncService) applyGitSyncEnvToDirInternal(dirPath, projectsDir string, gitEnvContent *string) (before, after string, err error) {- update, err := s.projectService.prepareGitSyncEnvUpdateInternal(dirPath, gitEnvContent)- if err != nil {- return "", "", errors.WrapIf(err, "failed to resolve git env state")- }- before = update.state.DirectContent- if update.effectiveContent != nil {- after = *update.effectiveContent- }-- if err := s.projectService.persistGitSyncEnvFilesInternal(dirPath, projectsDir, update); err != nil {- return "", "", errors.WrapIf(err, "failed to sync git env files")- }-- return before, after, nil-}-// seedStageEnvFromCandidateDirInternal copies env files from a pre-existing project// directory at the conventional path (projectsDir/<sanitized-sync-name>/) into the// staging directory before initial-sync validation. This lets users pre-seed@@ -2034,57 +1972,6 @@ func (s *GitOpsSyncService) lookupProjectByPathInternal(ctx context.Context, proreturn &project, true, nil}-func (s *GitOpsSyncService) ensureDirectorySyncProjectLinkedInternal(ctx context.Context, sync *models.GitOpsSync, project *models.Project) error {- if sync == nil || project == nil {- return nil- }-- if project.GitOpsManagedBy != nil && *project.GitOpsManagedBy != "" && *project.GitOpsManagedBy != sync.ID {- return errors.Errorf("project %s is already managed by a different GitOps sync", project.ID)- }-- if sync.ProjectID != nil && *sync.ProjectID == project.ID && project.GitOpsManagedBy != nil && *project.GitOpsManagedBy == sync.ID {- s.projectService.cacheComposeProjectIDInternal(projects.NormalizeProjectName(project.Name), project.ID)- return nil- }-- updatesSync := map[string]any{}- updatesProject := map[string]any{}- if sync.ProjectID == nil || *sync.ProjectID != project.ID {- updatesSync["project_id"] = project.ID- }- if project.GitOpsManagedBy == nil || *project.GitOpsManagedBy != sync.ID {- updatesProject["gitops_managed_by"] = sync.ID- }-- if len(updatesSync) == 0 && len(updatesProject) == 0 {- s.projectService.cacheComposeProjectIDInternal(projects.NormalizeProjectName(project.Name), project.ID)- return nil- }-- if err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {- if len(updatesSync) > 0 {- if err := tx.Model(&models.GitOpsSync{}).Where("id = ?", sync.ID).Updates(updatesSync).Error; err != nil {- return errors.WrapIff(err, "failed to relink GitOps sync %s", sync.ID)- }- }- if len(updatesProject) > 0 {- if err := tx.Model(&models.Project{}).Where("id = ?", project.ID).Updates(updatesProject).Error; err != nil {- return errors.WrapIff(err, "failed to relink project %s to GitOps sync %s", project.ID, sync.ID)- }- }- return nil- }); err != nil {- return err- }-- sync.ProjectID = &project.ID- project.GitOpsManagedBy = &sync.ID- s.projectService.cacheComposeProjectIDInternal(projects.NormalizeProjectName(project.Name), project.ID)-- return nil-}-func (s *GitOpsSyncService) findRecoverableManagedProjectInternal(ctx context.Context, sync *models.GitOpsSync) (*models.Project, error) {var managedProjects []models.Projectif err := s.db.WithContext(ctx).@@ -2096,10 +1983,10 @@ func (s *GitOpsSyncService) findRecoverableManagedProjectInternal(ctx context.Comatches := make([]models.Project, 0, len(managedProjects))for i := range managedProjects {project := managedProjects[i]- if err := s.projectService.ensureProjectPathUnderRoot(ctx, &project, true); err != nil {+ if err := s.projectService.EnsureProjectPathUnderRoot(ctx, &project, true); err != nil {return nil, err}- if _, err := s.projectService.resolveProjectComposeFileInternal(ctx, &project); err != nil {+ if _, err := s.projectService.ResolveProjectComposeFile(ctx, &project); err != nil {if errors.Is(err, common.ErrProjectComposeFileNotFound) {continue}@@ -2119,7 +2006,7 @@ func (s *GitOpsSyncService) findRecoverableManagedProjectInternal(ctx context.Co}func (s *GitOpsSyncService) findUniqueProjectDirectoryCandidateInternal(ctx context.Context, sync *models.GitOpsSync) (string, error) {- projectsDir, err := s.projectService.getProjectsDirectoryInternal(ctx)+ projectsDir, err := s.projectService.GetProjectsDirectory(ctx)if err != nil {return "", err}@@ -2181,29 +2068,16 @@ func (s *GitOpsSyncService) createRecoveredProjectFromDirectoryInternal(ctx contGitOpsManagedBy: &sync.ID,}- if serviceCount, err := s.projectService.countServicesFromCompose(ctx, *project); err == nil {+ if serviceCount, err := s.projectService.CountServicesFromCompose(ctx, *project); err == nil {project.ServiceCount = serviceCount} else {slog.WarnContext(ctx, "Failed to count services while recovering GitOps project", "syncId", sync.ID, "path", projectPath, "error", err)}- if err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {- if err := tx.Create(project).Error; err != nil {- return errors.WrapIff(err, "failed to create recovered project for sync %s", sync.ID)- }-- if err := tx.Model(&models.GitOpsSync{}).Where("id = ?", sync.ID).Update("project_id", project.ID).Error; err != nil {- return errors.WrapIff(err, "failed to relink sync %s to recovered project %s", sync.ID, project.ID)- }-- return nil- }); err != nil {+ if err := s.projectService.CreateGitOpsManagedProject(ctx, sync, project, models.User{}, false); err != nil {return nil, err}- sync.ProjectID = &project.ID- s.projectService.cacheComposeProjectIDInternal(projects.NormalizeProjectName(project.Name), project.ID)-return project, nil}@@ -2218,10 +2092,10 @@ func (s *GitOpsSyncService) recoverProjectFromDirectoryCandidateInternal(ctx conreturn nil, err}if found {- if err := s.projectService.ensureProjectPathUnderRoot(ctx, project, true); err != nil {+ if err := s.projectService.EnsureProjectPathUnderRoot(ctx, project, true); err != nil {return nil, err}- if err := s.ensureDirectorySyncProjectLinkedInternal(ctx, sync, project); err != nil {+ if err := s.projectService.EnsureGitOpsProjectLinked(ctx, sync, project); err != nil {return nil, err}return project, nil@@ -2244,10 +2118,10 @@ func (s *GitOpsSyncService) getDirectorySyncProjectInternal(ctx context.Context,return nil, err}if found {- if err := s.projectService.ensureProjectPathUnderRoot(ctx, project, true); err != nil {+ if err := s.projectService.EnsureProjectPathUnderRoot(ctx, project, true); err != nil {return nil, err}- if err := s.ensureDirectorySyncProjectLinkedInternal(ctx, sync, project); err != nil {+ if err := s.projectService.EnsureGitOpsProjectLinked(ctx, sync, project); err != nil {return nil, err}return project, nil@@ -2264,7 +2138,7 @@ func (s *GitOpsSyncService) getDirectorySyncProjectInternal(ctx context.Context,return nil, err}if project != nil {- if err := s.ensureDirectorySyncProjectLinkedInternal(ctx, sync, project); err != nil {+ if err := s.projectService.EnsureGitOpsProjectLinked(ctx, sync, project); err != nil {return nil, err}return project, nil@@ -2288,49 +2162,10 @@ func (s *GitOpsSyncService) getDirectorySyncProjectInternal(ctx context.Context,return nil, nil}-// validateDirectorySyncStageInternal loads the staged compose project using the real-// synced compose filename so include/env_file resolution happens against the-// fully copied directory contents.-func (s *GitOpsSyncService) validateDirectorySyncStageInternal(ctx context.Context, projectName, stagePath, composeFileName string) (int, error) {- projectsDir, err := s.projectService.getProjectsDirectoryInternal(ctx)- if err != nil {- return 0, err- }-- var dockerClient *client.Client- if s.projectService.dockerService != nil {- dockerClient, _ = s.projectService.dockerService.GetClient(ctx)- }- pathMapper := projects.NewPathMapperForConfiguredDirectory(- ctx,- s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"),- "/app/data/projects",- dockerClient,- )-- autoInjectEnv := s.settingsService.GetBoolSetting(ctx, "autoInjectEnv", false)- project, err := projects.LoadComposeProject(- ctx,- filepath.Join(stagePath, composeFileName),- projects.NormalizeProjectName(projectName),- projectsDir,- autoInjectEnv,- pathMapper,- nil,- nil,- true,- )- if err != nil {- return 0, err- }-- return len(project.Services), nil-}-// createDirectorySyncProjectInternal promotes a validated staged tree into a new// managed project directory and links it back to the Git sync record.func (s *GitOpsSyncService) createDirectorySyncProjectInternal(ctx context.Context, sync *models.GitOpsSync, stage *stagedDirectorySync, actor models.User) (*models.Project, error) {- projectsDir, err := s.projectService.getProjectsDirectoryInternal(ctx)+ projectsDir, err := s.projectService.GetProjectsDirectory(ctx)if err != nil {return nil, errors.WrapIf(err, "failed to get projects directory")}@@ -2366,35 +2201,11 @@ func (s *GitOpsSyncService) createDirectorySyncProjectInternal(ctx context.ConteRunningCount: 0,}- if err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {- if err := tx.Create(project).Error; err != nil {- return errors.WrapIf(err, "failed to create project")- }-- if err := tx.Model(&models.GitOpsSync{}).Where("id = ?", sync.ID).Update("project_id", project.ID).Error; err != nil {- return errors.WrapIf(err, "failed to update sync with project ID")- }-- if err := tx.Model(&models.Project{}).Where("id = ?", project.ID).Update("gitops_managed_by", sync.ID).Error; err != nil {- return errors.WrapIf(err, "failed to mark project as GitOps-managed")- }-- return nil- }); err != nil {+ if err := s.projectService.CreateGitOpsManagedProject(ctx, sync, project, actor); err != nil {_ = os.RemoveAll(projectPath)return nil, err}- sync.ProjectID = &project.ID- s.projectService.cacheComposeProjectIDInternal(projects.NormalizeProjectName(project.Name), project.ID)-- if s.projectService.eventService != nil {- metadata := models.JSON{"action": "create", "projectID": project.ID, "projectName": project.Name, "path": projectPath}- if logErr := s.projectService.eventService.LogProjectEvent(ctx, models.EventTypeProjectCreate, project.ID, project.Name, actor.ID, actor.Username, "0", metadata); logErr != nil {- slog.ErrorContext(ctx, "could not log project creation", "error", logErr)- }- }-return project, nil}@@ -2422,7 +2233,7 @@ func (s *GitOpsSyncService) updateDirectorySyncProjectInternal(ctx context.Contevar backupSkipped []stringif existed {- projectsDir, err := s.projectService.getProjectsDirectoryInternal(ctx)+ projectsDir, err := s.projectService.GetProjectsDirectory(ctx)if err != nil {return nil, errors.WrapIf(err, "failed to get projects directory")}diff --git a/backend/api/handlers/gitops_syncs.go b/backend/internal/gitops/handler.gosimilarity index 76%rename from backend/api/handlers/gitops_syncs.gorename to backend/internal/gitops/handler.goindex 7d8f55288..02e1e0c78 100644--- a/backend/api/handlers/gitops_syncs.go+++ b/backend/internal/gitops/handler.go@@ -1,21 +1,21 @@-package handlers+package gitopsimport ("context""emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/gitops")// GitOpsSyncHandler handles GitOps sync management endpoints.type GitOpsSyncHandler struct {- syncService *services.GitOpsSyncService+ syncService *GitOpsSyncService}// ============================================================================@@ -114,21 +114,21 @@ type ImportGitOpsSyncsOutput struct {// ============================================================================// RegisterGitOpsSyncs registers all GitOps sync endpoints.-func RegisterGitOpsSyncs(api huma.API, syncService *services.GitOpsSyncService) {+func RegisterGitOpsSyncs(api huma.API, syncService *GitOpsSyncService) {h := &GitOpsSyncHandler{syncService: syncService}const basePath = "/environments/{id}/gitops-syncs"const syncPath = basePath + "/{syncId}"- registerSecuredInternal(api, operationInternal("listGitOpsSyncs", "GET", basePath, "List GitOps syncs", "Get a paginated list of GitOps syncs for an environment", "GitOps Syncs"), authz.PermGitOpsList, h.ListSyncs)- registerSecuredInternal(api, operationInternal("createGitOpsSync", "POST", basePath, "Create a GitOps sync", "Create a new GitOps sync configuration for an environment", "GitOps Syncs"), authz.PermGitOpsCreate, h.CreateSync)- registerSecuredInternal(api, operationInternal("importGitOpsSyncs", "POST", basePath+"/import", "Import GitOps syncs", "Import multiple GitOps sync configurations from JSON", "GitOps Syncs"), authz.PermGitOpsCreate, h.ImportSyncs)- registerSecuredInternal(api, operationInternal("getGitOpsSync", "GET", syncPath, "Get a GitOps sync", "Get a GitOps sync by ID", "GitOps Syncs"), authz.PermGitOpsRead, h.GetSync)- registerSecuredInternal(api, operationInternal("updateGitOpsSync", "PUT", syncPath, "Update a GitOps sync", "Update an existing GitOps sync configuration", "GitOps Syncs"), authz.PermGitOpsUpdate, h.UpdateSync)- registerSecuredInternal(api, operationInternal("deleteGitOpsSync", "DELETE", syncPath, "Delete a GitOps sync", "Delete a GitOps sync configuration by ID", "GitOps Syncs"), authz.PermGitOpsDelete, h.DeleteSync)- registerSecuredInternal(api, operationInternal("performGitOpsSync", "POST", syncPath+"/sync", "Perform a GitOps sync", "Manually trigger a sync operation", "GitOps Syncs"), authz.PermGitOpsSync, h.PerformSync)- registerSecuredInternal(api, operationInternal("getGitOpsSyncStatus", "GET", syncPath+"/status", "Get GitOps sync status", "Get the current status of a GitOps sync", "GitOps Syncs"), authz.PermGitOpsRead, h.GetStatus)- registerSecuredInternal(api, operationInternal("browseGitOpsSyncFiles", "GET", syncPath+"/files", "Browse GitOps sync files", "Browse files in the synced repository", "GitOps Syncs"), authz.PermGitOpsRead, h.BrowseFiles)+ handlerutil.RegisterSecured(api, handlerutil.Operation("listGitOpsSyncs", "GET", basePath, "List GitOps syncs", "Get a paginated list of GitOps syncs for an environment", "GitOps Syncs"), authz.PermGitOpsList, h.ListSyncs)+ handlerutil.RegisterSecured(api, handlerutil.Operation("createGitOpsSync", "POST", basePath, "Create a GitOps sync", "Create a new GitOps sync configuration for an environment", "GitOps Syncs"), authz.PermGitOpsCreate, h.CreateSync)+ handlerutil.RegisterSecured(api, handlerutil.Operation("importGitOpsSyncs", "POST", basePath+"/import", "Import GitOps syncs", "Import multiple GitOps sync configurations from JSON", "GitOps Syncs"), authz.PermGitOpsCreate, h.ImportSyncs)+ handlerutil.RegisterSecured(api, handlerutil.Operation("getGitOpsSync", "GET", syncPath, "Get a GitOps sync", "Get a GitOps sync by ID", "GitOps Syncs"), authz.PermGitOpsRead, h.GetSync)+ handlerutil.RegisterSecured(api, handlerutil.Operation("updateGitOpsSync", "PUT", syncPath, "Update a GitOps sync", "Update an existing GitOps sync configuration", "GitOps Syncs"), authz.PermGitOpsUpdate, h.UpdateSync)+ handlerutil.RegisterSecured(api, handlerutil.Operation("deleteGitOpsSync", "DELETE", syncPath, "Delete a GitOps sync", "Delete a GitOps sync configuration by ID", "GitOps Syncs"), authz.PermGitOpsDelete, h.DeleteSync)+ handlerutil.RegisterSecured(api, handlerutil.Operation("performGitOpsSync", "POST", syncPath+"/sync", "Perform a GitOps sync", "Manually trigger a sync operation", "GitOps Syncs"), authz.PermGitOpsSync, h.PerformSync)+ handlerutil.RegisterSecured(api, handlerutil.Operation("getGitOpsSyncStatus", "GET", syncPath+"/status", "Get GitOps sync status", "Get the current status of a GitOps sync", "GitOps Syncs"), authz.PermGitOpsRead, h.GetStatus)+ handlerutil.RegisterSecured(api, handlerutil.Operation("browseGitOpsSyncFiles", "GET", syncPath+"/files", "Browse GitOps sync files", "Browse files in the synced repository", "GitOps Syncs"), authz.PermGitOpsRead, h.BrowseFiles)}// requireLifecyclePermissionInternal rejects callers lacking gitops:lifecycle@@ -144,7 +144,7 @@ func requireLifecyclePermissionInternal(ctx context.Context, environmentID strinif !lifecycleRequested {return nil}- if ps, _ := humamw.PermissionsFromContext(ctx); ps.Allows(authz.PermGitOpsLifecycle, environmentID) {+ if ps, _ := middleware.PermissionsFromContext(ctx); ps.Allows(authz.PermGitOpsLifecycle, environmentID) {return nil}return huma.Error403Forbidden("configuring a pre-deploy lifecycle hook requires the " + authz.PermGitOpsLifecycle + " permission")@@ -156,7 +156,7 @@ func requireLifecyclePermissionInternal(ctx context.Context, environmentID strin// ListSyncs returns a paginated list of GitOps syncs.func (h *GitOpsSyncHandler) ListSyncs(ctx context.Context, input *ListGitOpsSyncsInput) (*ListGitOpsSyncsOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)syncs, paginationResp, counts, err := h.syncService.GetSyncsPaginated(ctx, input.EnvironmentID, params)if err != nil {@@ -168,7 +168,7 @@ func (h *GitOpsSyncHandler) ListSyncs(ctx context.Context, input *ListGitOpsSyncSuccess: true,Data: syncs,Counts: counts,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -179,7 +179,7 @@ func (h *GitOpsSyncHandler) CreateSync(ctx context.Context, input *CreateGitOpsSreturn nil, err}- actor := currentActorInternal(ctx)+ actor := handlerutil.CurrentActor(ctx)sync, err := h.syncService.CreateSync(ctx, input.EnvironmentID, input.Body, actor)if err != nil {@@ -187,7 +187,7 @@ func (h *GitOpsSyncHandler) CreateSync(ctx context.Context, input *CreateGitOpsSreturn nil, huma.NewError(apiErr.HTTPStatus(), errors.WithMessage(err, "Failed to create GitOps sync").Error())}- body, mapErr := mapOneAPIResponseInternal[*models.GitOpsSync, gitops.GitOpsSync](sync, func(err error) string {+ body, mapErr := handlerutil.MapOneAPIResponse[*models.GitOpsSync, gitops.GitOpsSync](sync, func(err error) string {return "Failed to map GitOps sync"})if mapErr != nil {@@ -201,7 +201,7 @@ func (h *GitOpsSyncHandler) CreateSync(ctx context.Context, input *CreateGitOpsS// ImportSyncs imports multiple GitOps syncs.func (h *GitOpsSyncHandler) ImportSyncs(ctx context.Context, input *ImportGitOpsSyncsInput) (*ImportGitOpsSyncsOutput, error) {- actor := currentActorInternal(ctx)+ actor := handlerutil.CurrentActor(ctx)response, err := h.syncService.ImportSyncs(ctx, input.EnvironmentID, input.Body, actor)if err != nil {@@ -224,7 +224,7 @@ func (h *GitOpsSyncHandler) GetSync(ctx context.Context, input *GetGitOpsSyncInpreturn nil, huma.NewError(apiErr.HTTPStatus(), "Failed to retrieve GitOps sync")}- body, mapErr := mapOneAPIResponseInternal[*models.GitOpsSync, gitops.GitOpsSync](sync, func(err error) string {+ body, mapErr := handlerutil.MapOneAPIResponse[*models.GitOpsSync, gitops.GitOpsSync](sync, func(err error) string {return "Failed to map GitOps sync"})if mapErr != nil {@@ -242,7 +242,7 @@ func (h *GitOpsSyncHandler) UpdateSync(ctx context.Context, input *UpdateGitOpsSreturn nil, err}- actor := currentActorInternal(ctx)+ actor := handlerutil.CurrentActor(ctx)sync, err := h.syncService.UpdateSync(ctx, input.EnvironmentID, input.SyncID, input.Body, actor)if err != nil {@@ -250,7 +250,7 @@ func (h *GitOpsSyncHandler) UpdateSync(ctx context.Context, input *UpdateGitOpsSreturn nil, huma.NewError(apiErr.HTTPStatus(), errors.WithMessage(err, "Failed to update GitOps sync").Error())}- body, mapErr := mapOneAPIResponseInternal[*models.GitOpsSync, gitops.GitOpsSync](sync, func(err error) string {+ body, mapErr := handlerutil.MapOneAPIResponse[*models.GitOpsSync, gitops.GitOpsSync](sync, func(err error) string {return "Failed to map GitOps sync"})if mapErr != nil {@@ -264,7 +264,7 @@ func (h *GitOpsSyncHandler) UpdateSync(ctx context.Context, input *UpdateGitOpsS// DeleteSync deletes a GitOps sync by ID.func (h *GitOpsSyncHandler) DeleteSync(ctx context.Context, input *DeleteGitOpsSyncInput) (*DeleteGitOpsSyncOutput, error) {- actor := currentActorInternal(ctx)+ actor := handlerutil.CurrentActor(ctx)if err := h.syncService.DeleteSync(ctx, input.EnvironmentID, input.SyncID, actor); err != nil {apiErr := models.ToAPIError(err)@@ -283,7 +283,7 @@ func (h *GitOpsSyncHandler) DeleteSync(ctx context.Context, input *DeleteGitOpsS// PerformSync manually triggers a sync operation.func (h *GitOpsSyncHandler) PerformSync(ctx context.Context, input *PerformSyncInput) (*PerformSyncOutput, error) {- actor := currentActorInternal(ctx)+ actor := handlerutil.CurrentActor(ctx)result, err := h.syncService.PerformSync(ctx, input.EnvironmentID, input.SyncID, actor)if err != nil {diff --git a/backend/internal/gitops/module.go b/backend/internal/gitops/module.gonew file mode 100644index 000000000..85ecfcc10--- /dev/null+++ b/backend/internal/gitops/module.go@@ -0,0 +1,27 @@+// Package gitops owns GitOps synchronization, scheduling, persistence, and routes.+package gitops++import "github.com/danielgtaylor/huma/v2"++type Module struct {+ service *GitOpsSyncService+}++func New(service *GitOpsSyncService) *Module {+ return &Module{service: service}+}++func (m *Module) Service() *GitOpsSyncService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterGitOpsSyncs(api, nil)+ return+ }+ RegisterGitOpsSyncs(api, m.service)+}diff --git a/backend/internal/services/gitops_sync_service_test.go b/backend/internal/gitops/service_test.gosimilarity index 93%rename from backend/internal/services/gitops_sync_service_test.gorename to backend/internal/gitops/service_test.goindex 6d335b9fc..8084462d9 100644--- a/backend/internal/services/gitops_sync_service_test.go+++ b/backend/internal/gitops/service_test.go@@ -1,4 +1,4 @@-package services+package gitopsimport ("context"@@ -13,32 +13,82 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitrepo"+ "github.com/getarcaneapp/arcane/backend/v2/internal/lifecycle""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"git "github.com/getarcaneapp/arcane/backend/v2/pkg/gitutil""github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/scheduler/entityjobs""github.com/getarcaneapp/arcane/types/v2/gitops"schedulertypes "github.com/getarcaneapp/arcane/types/v2/scheduler"+ "github.com/libtnb/sqlite""github.com/stretchr/testify/assert""github.com/stretchr/testify/require""go.uber.org/fx/fxtest""gorm.io/gorm")+func setupGitOpsProjectTestDBInternal(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.Project{}, &models.SettingVariable{}, &models.ImageUpdateRecord{}, &models.Event{}))+ return &database.DB{DB: db}+}++func newGitOpsSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ fxLifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), fxLifecycle)+ require.NoError(t, err)+ executor, err := actors.NewExecutor(t.Context(), runtime, "gitops-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "gitops-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, executor.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, fxLifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, executor, effects)+}++func newGitOpsAdmissionGateForTestInternal(t testing.TB) *actors.Gate[actors.AdmissionKey] {+ t.Helper()+ fxLifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), fxLifecycle)+ require.NoError(t, err)+ gate, err := actors.NewGate[actors.AdmissionKey](t.Context(), runtime, "gitops-test-admission", t.Name())+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, gate.Stop(stopCtx))+ require.NoError(t, fxLifecycle.Stop(stopCtx))+ })+ return gate+}+func setupGitOpsSyncDirectoryTestService(t *testing.T) (*GitOpsSyncService, *database.DB, string) {t.Helper()ctx := context.Background()- db := setupProjectTestDB(t)+ db := setupGitOpsProjectTestDBInternal(t)require.NoError(t, db.AutoMigrate(&models.GitOpsSync{}))- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)+ settingsService, err := newGitOpsSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)projectsDir := t.TempDir()require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- eventService := NewEventService(db, config.Load(), nil)- projectService := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())+ eventService := event.NewEventService(db, config.Load(), nil)+ projectService := project.NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())return NewGitOpsSyncService(db, nil, projectService, nil, eventService, settingsService), db, projectsDir}@@ -55,7 +105,8 @@ func TestGitOpsSyncService_OverlappingSyncPreservesSuccessShapedSkipInternal(t *require.NoError(t, err)require.True(t, admitted)- service := &GitOpsSyncService{admissionGate: gate}+ service := &GitOpsSyncService{jobs: entityjobs.New(entityjobs.GitOpsSyncJobPrefix, gitOpsSyncAdmissionScopeInternal)}+ require.NoError(t, service.SetScheduler(t.Context(), &gitOpsSyncTestSchedulerInternal{}, gate))result, err := service.PerformSync(t.Context(), "0", "sync-id", models.User{})require.NoError(t, err)require.False(t, result.Success)@@ -99,7 +150,7 @@ func TestApplyLifecycleFieldsToSyncInternal_DefaultsPreDeployTimeout(t *testing.applyLifecycleFieldsToSyncInternal(&sync, lifecycleConfigInputInternal{})- require.Equal(t, lifecycleDefaultTimeoutSec, sync.PreDeployTimeoutSec)+ require.Equal(t, lifecycle.DefaultTimeoutSec, sync.PreDeployTimeoutSec)}func TestApplyLifecycleFieldsToSyncInternal_UsesExplicitPreDeployTimeout(t *testing.T) {@@ -205,10 +256,10 @@ func TestGitOpsSyncService_RegisterAutoSyncJobsOnStartup_SkipsOrphans(t *testing}).Error)scheduler := &gitOpsSyncTestSchedulerInternal{}- require.NoError(t, svc.SetScheduler(ctx, scheduler, newAdmissionGateForTestInternal(t)))+ require.NoError(t, svc.SetScheduler(ctx, scheduler, newGitOpsAdmissionGateForTestInternal(t)))svc.RegisterAutoSyncJobsOnStartup(ctx)- require.Equal(t, []string{gitOpsSyncJobNameInternal("sync-live")}, scheduler.added)+ require.Equal(t, []string{entityjobs.GitOpsSyncJobPrefix + "sync-live"}, scheduler.added)require.Empty(t, scheduler.removed)}@@ -243,7 +294,7 @@ func TestGitOpsSyncService_DeleteSync_SucceedsWhenEnvironmentMismatched(t *testictx := context.Background()svc, db, _ := setupGitOpsSyncDirectoryTestService(t)scheduler := &gitOpsSyncTestSchedulerInternal{}- require.NoError(t, svc.SetScheduler(ctx, scheduler, newAdmissionGateForTestInternal(t)))+ require.NoError(t, svc.SetScheduler(ctx, scheduler, newGitOpsAdmissionGateForTestInternal(t)))sync := &models.GitOpsSync{BaseModel: models.BaseModel{ID: "sync-env-mismatch"},@@ -259,7 +310,7 @@ func TestGitOpsSyncService_DeleteSync_SucceedsWhenEnvironmentMismatched(t *testivar count int64require.NoError(t, db.Model(&models.GitOpsSync{}).Where("id = ?", sync.ID).Count(&count).Error)assert.Zero(t, count)- assert.Contains(t, scheduler.removed, gitOpsSyncJobNameInternal(sync.ID))+ assert.Contains(t, scheduler.removed, entityjobs.GitOpsSyncJobPrefix+sync.ID)}// TestGitOpsSyncService_DeleteSync_ClearsOrphanedManagedFlag verifies the managed@@ -300,11 +351,11 @@ func TestGitOpsSyncService_RunScheduledSync_UnregistersMissingSync(t *testing.T)ctx := context.Background()svc, _, _ := setupGitOpsSyncDirectoryTestService(t)scheduler := &gitOpsSyncTestSchedulerInternal{}- require.NoError(t, svc.SetScheduler(ctx, scheduler, newAdmissionGateForTestInternal(t)))+ require.NoError(t, svc.SetScheduler(ctx, scheduler, newGitOpsAdmissionGateForTestInternal(t)))svc.runScheduledSyncInternal(ctx, "0", "ghost-sync")- assert.Contains(t, scheduler.removed, gitOpsSyncJobNameInternal("ghost-sync"))+ assert.Contains(t, scheduler.removed, entityjobs.GitOpsSyncJobPrefix+"ghost-sync")}// TestGitOpsSyncService_CleanupLeakedScratchDirsOnStartup_RemovesOrphans verifies the@@ -382,7 +433,7 @@ func TestGitOpsSyncService_SyncProjectDirectory_RefusesDuplicateOnNameCollision(func TestGitOpsSyncService_GetOrCreateProject_RefusesDuplicateOnNameCollision(t *testing.T) {ctx := context.Background()svc, db, projectsDir := setupGitOpsSyncDirectoryTestService(t)- require.NoError(t, svc.SetScheduler(ctx, &gitOpsSyncTestSchedulerInternal{}, newAdmissionGateForTestInternal(t)))+ require.NoError(t, svc.SetScheduler(ctx, &gitOpsSyncTestSchedulerInternal{}, newGitOpsAdmissionGateForTestInternal(t)))require.NoError(t, os.MkdirAll(filepath.Join(projectsDir, "Dozzle"), 0o755))@@ -809,7 +860,7 @@ func TestGitOpsSyncService_SyncProjectDirectory_IgnoresCommittedReservedEnvFilesfunc TestGitOpsSyncService_DirectorySync_RealWalkWithNestedConfig(t *testing.T) {ctx := context.Background()svc, db, _ := setupGitOpsSyncDirectoryTestService(t)- svc.repoService = &GitRepositoryService{gitClient: git.NewClient("")}+ svc.repoService = &gitrepo.GitRepositoryService{Client: git.NewClient("")}repoPath := t.TempDir()writeFileInternal(t, repoPath, "traefik (nl10)/docker-compose.yml", []byte(`services:@@ -875,7 +926,7 @@ func TestGitOpsSyncService_DirectorySync_RealWalkWithNestedConfig(t *testing.T)func TestGitOpsSyncService_DirectorySync_OverwritesExistingDirectoryAtFilePath(t *testing.T) {ctx := context.Background()svc, db, projectsDir := setupGitOpsSyncDirectoryTestService(t)- svc.repoService = &GitRepositoryService{gitClient: git.NewClient("")}+ svc.repoService = &gitrepo.GitRepositoryService{Client: git.NewClient("")}repoPath := t.TempDir()writeFileInternal(t, repoPath, "traefik (nl10)/docker-compose.yml", []byte(`services:@@ -1144,7 +1195,7 @@ func TestGitOpsSyncService_SyncProjectDirectory_FailsWhenBoundProjectMissing(t *ctx := context.Background()svc, db, projectsDir := setupGitOpsSyncDirectoryTestService(t)testScheduler := &gitOpsSyncTestSchedulerInternal{}- require.NoError(t, svc.SetScheduler(ctx, testScheduler, newAdmissionGateForTestInternal(t)))+ require.NoError(t, svc.SetScheduler(ctx, testScheduler, newGitOpsAdmissionGateForTestInternal(t)))missingProjectID := "missing-project"sync := &models.GitOpsSync{@@ -1192,14 +1243,14 @@ func TestGitOpsSyncService_SyncProjectDirectory_FailsWhenBoundProjectMissing(t *assert.Equal(t, "failed", *storedSync.LastSyncStatus)require.NotNil(t, storedSync.LastSyncError)assert.Contains(t, *storedSync.LastSyncError, "project binding")- assert.Contains(t, testScheduler.removed, gitOpsSyncJobNameInternal(sync.ID))+ assert.Contains(t, testScheduler.removed, entityjobs.GitOpsSyncJobPrefix+sync.ID)}func TestGitOpsSyncService_SyncProjectDirectory_DisablesAutoSyncWhenBoundProjectRecoveryAmbiguous(t *testing.T) {ctx := context.Background()svc, db, projectsDir := setupGitOpsSyncDirectoryTestService(t)testScheduler := &gitOpsSyncTestSchedulerInternal{}- require.NoError(t, svc.SetScheduler(ctx, testScheduler, newAdmissionGateForTestInternal(t)))+ require.NoError(t, svc.SetScheduler(ctx, testScheduler, newGitOpsAdmissionGateForTestInternal(t)))for _, dirName := range []string{"Radarr-3", "Radarr-30"} {projectPath := filepath.Join(projectsDir, dirName)@@ -1247,14 +1298,14 @@ func TestGitOpsSyncService_SyncProjectDirectory_DisablesAutoSyncWhenBoundProjectassert.Equal(t, "failed", *storedSync.LastSyncStatus)require.NotNil(t, storedSync.LastSyncError)assert.Contains(t, *storedSync.LastSyncError, "multiple candidate project directories")- assert.Contains(t, testScheduler.removed, gitOpsSyncJobNameInternal(sync.ID))+ assert.Contains(t, testScheduler.removed, entityjobs.GitOpsSyncJobPrefix+sync.ID)}func TestGitOpsSyncService_GetOrCreateProjectInternal_FailsWhenBoundProjectMissing(t *testing.T) {ctx := context.Background()svc, db, projectsDir := setupGitOpsSyncDirectoryTestService(t)testScheduler := &gitOpsSyncTestSchedulerInternal{}- require.NoError(t, svc.SetScheduler(ctx, testScheduler, newAdmissionGateForTestInternal(t)))+ require.NoError(t, svc.SetScheduler(ctx, testScheduler, newGitOpsAdmissionGateForTestInternal(t)))missingProjectID := "missing-project"sync := &models.GitOpsSync{@@ -1291,7 +1342,7 @@ func TestGitOpsSyncService_GetOrCreateProjectInternal_FailsWhenBoundProjectMissiassert.Equal(t, "failed", *storedSync.LastSyncStatus)require.NotNil(t, storedSync.LastSyncError)assert.Contains(t, *storedSync.LastSyncError, "project binding")- assert.Contains(t, testScheduler.removed, gitOpsSyncJobNameInternal(sync.ID))+ assert.Contains(t, testScheduler.removed, entityjobs.GitOpsSyncJobPrefix+sync.ID)}func TestEnvContentChangedInternal(t *testing.T) {@@ -1312,8 +1363,8 @@ func TestEnvContentChangedInternal(t *testing.T) {func TestGitOpsSyncService_GetEnvironmentSyncLimits(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)+ db := setupGitOpsProjectTestDBInternal(t)+ settingsSvc, err := newGitOpsSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)require.NoError(t, settingsSvc.SetIntSetting(ctx, "gitSyncMaxFiles", 123))@@ -1335,8 +1386,8 @@ func TestGitOpsSyncService_GetEffectiveSyncLimits(t *testing.T) {t.Setenv("GIT_SYNC_MAX_TOTAL_SIZE_MB", "")t.Setenv("GIT_SYNC_MAX_BINARY_SIZE_MB", "")- db := setupSettingsTestDB(t)- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)+ db := setupGitOpsProjectTestDBInternal(t)+ settingsSvc, err := newGitOpsSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)require.NoError(t, settingsSvc.SetIntSetting(ctx, "gitSyncMaxFiles", 200))@@ -1391,7 +1442,7 @@ func TestGitOpsSyncService_GetEffectiveSyncLimits(t *testing.T) {t.Setenv("GIT_SYNC_MAX_FILES", "10000")t.Setenv("GIT_SYNC_MAX_TOTAL_SIZE_MB", "1024")t.Setenv("GIT_SYNC_MAX_BINARY_SIZE_MB", "12")- settingsSvcEnv, svcErr := newSettingsServiceForTestInternal(t, ctx, db)+ settingsSvcEnv, svcErr := newGitOpsSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, svcErr)svcEnv := &GitOpsSyncService{settingsService: settingsSvcEnv}@@ -1412,7 +1463,7 @@ func TestGitOpsSyncService_GetEffectiveSyncLimits(t *testing.T) {t.Setenv("GIT_SYNC_MAX_FILES", "0")t.Setenv("GIT_SYNC_MAX_TOTAL_SIZE_MB", "0")t.Setenv("GIT_SYNC_MAX_BINARY_SIZE_MB", "0")- settingsSvcEnv, svcErr := newSettingsServiceForTestInternal(t, ctx, db)+ settingsSvcEnv, svcErr := newGitOpsSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, svcErr)svcEnv := &GitOpsSyncService{settingsService: settingsSvcEnv}@@ -1672,10 +1723,10 @@ func TestRedeployAfterSyncFailedError_FormatAndUnwrap(t *testing.T) {func TestMarkSyncRedeployFailedInternal_PersistsErrorOnSyncRow(t *testing.T) {ctx := context.Background()svc, db, _ := setupGitOpsSyncDirectoryTestService(t)- // Event logging requires a real EventService; the shared setup leaves it+ // Event logging requires a real event.EventService; the shared setup leaves it// nil since most tests don't exercise the event path.require.NoError(t, db.AutoMigrate(&models.Event{}))- svc.eventService = NewEventService(db, config.Load(), nil)+ svc.eventService = event.NewEventService(db, config.Load(), nil)sync := &models.GitOpsSync{BaseModel: models.BaseModel{ID: "sync-1"},diff --git a/backend/internal/services/gitops_sync_service_unix_test.go b/backend/internal/gitops/service_unix_test.gosimilarity index 99%rename from backend/internal/services/gitops_sync_service_unix_test.gorename to backend/internal/gitops/service_unix_test.goindex 3649d1a1b..85d43d853 100644--- a/backend/internal/services/gitops_sync_service_unix_test.go+++ b/backend/internal/gitops/service_unix_test.go@@ -1,6 +1,6 @@//go:build unix-package services+package gitopsimport ("context"diff --git a/backend/api/handlers/git_repositories.go b/backend/internal/gitrepo/handler.gosimilarity index 72%rename from backend/api/handlers/git_repositories.gorename to backend/internal/gitrepo/handler.goindex 43c9ed10b..ec1df5e07 100644--- a/backend/api/handlers/git_repositories.go+++ b/backend/internal/gitrepo/handler.go@@ -1,4 +1,4 @@-package handlers+package gitrepoimport ("context"@@ -6,15 +6,15 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/gitops")// GitRepositoryHandler handles git repository management endpoints.type GitRepositoryHandler struct {- repoService *services.GitRepositoryService+ repoService *GitRepositoryService}// ============================================================================@@ -106,18 +106,18 @@ type SyncGitRepositoriesOutput struct {// ============================================================================// RegisterGitRepositories registers all git repository endpoints.-func RegisterGitRepositories(api huma.API, repoService *services.GitRepositoryService) {+func RegisterGitRepositories(api huma.API, repoService *GitRepositoryService) {h := &GitRepositoryHandler{repoService: repoService}- registerSecuredInternal(api, operationInternal("listGitRepositories", "GET", "/customize/git-repositories", "List git repositories", "Get a paginated list of git repositories", "Customize"), authz.PermGitReposList, h.ListRepositories)- registerSecuredInternal(api, operationInternal("createGitRepository", "POST", "/customize/git-repositories", "Create a git repository", "Create a new git repository configuration", "Customize"), authz.PermGitReposCreate, h.CreateRepository)- registerSecuredInternal(api, operationInternal("getGitRepository", "GET", "/customize/git-repositories/{id}", "Get a git repository", "Get a git repository by ID", "Customize"), authz.PermGitReposRead, h.GetRepository)- registerSecuredInternal(api, operationInternal("updateGitRepository", "PUT", "/customize/git-repositories/{id}", "Update a git repository", "Update an existing git repository configuration", "Customize"), authz.PermGitReposUpdate, h.UpdateRepository)- registerSecuredInternal(api, operationInternal("deleteGitRepository", "DELETE", "/customize/git-repositories/{id}", "Delete a git repository", "Delete a git repository configuration by ID", "Customize"), authz.PermGitReposDelete, h.DeleteRepository)- registerSecuredInternal(api, operationInternal("testGitRepository", "POST", "/customize/git-repositories/{id}/test", "Test a git repository", "Test connectivity and authentication to a git repository", "Customize"), authz.PermGitReposTest, h.TestRepository)- registerSecuredInternal(api, operationInternal("listGitRepositoryBranches", "GET", "/customize/git-repositories/{id}/branches", "List repository branches", "Get all branches from a git repository with default branch detection", "Customize"), authz.PermGitReposRead, h.ListBranches)- registerSecuredInternal(api, operationInternal("browseGitRepositoryFiles", "GET", "/customize/git-repositories/{id}/files", "Browse repository files", "Browse files and directories in a git repository", "Customize"), authz.PermGitReposRead, h.BrowseFiles)- registerSecuredInternal(api, operationInternal("syncGitRepositories", "POST", "/git-repositories/sync", "Sync git repositories", "Sync git repositories from a manager to this agent instance", "Git Repositories"), authz.PermGitReposSync, h.SyncRepositories)+ handlerutil.RegisterSecured(api, handlerutil.Operation("listGitRepositories", "GET", "/customize/git-repositories", "List git repositories", "Get a paginated list of git repositories", "Customize"), authz.PermGitReposList, h.ListRepositories)+ handlerutil.RegisterSecured(api, handlerutil.Operation("createGitRepository", "POST", "/customize/git-repositories", "Create a git repository", "Create a new git repository configuration", "Customize"), authz.PermGitReposCreate, h.CreateRepository)+ handlerutil.RegisterSecured(api, handlerutil.Operation("getGitRepository", "GET", "/customize/git-repositories/{id}", "Get a git repository", "Get a git repository by ID", "Customize"), authz.PermGitReposRead, h.GetRepository)+ handlerutil.RegisterSecured(api, handlerutil.Operation("updateGitRepository", "PUT", "/customize/git-repositories/{id}", "Update a git repository", "Update an existing git repository configuration", "Customize"), authz.PermGitReposUpdate, h.UpdateRepository)+ handlerutil.RegisterSecured(api, handlerutil.Operation("deleteGitRepository", "DELETE", "/customize/git-repositories/{id}", "Delete a git repository", "Delete a git repository configuration by ID", "Customize"), authz.PermGitReposDelete, h.DeleteRepository)+ handlerutil.RegisterSecured(api, handlerutil.Operation("testGitRepository", "POST", "/customize/git-repositories/{id}/test", "Test a git repository", "Test connectivity and authentication to a git repository", "Customize"), authz.PermGitReposTest, h.TestRepository)+ handlerutil.RegisterSecured(api, handlerutil.Operation("listGitRepositoryBranches", "GET", "/customize/git-repositories/{id}/branches", "List repository branches", "Get all branches from a git repository with default branch detection", "Customize"), authz.PermGitReposRead, h.ListBranches)+ handlerutil.RegisterSecured(api, handlerutil.Operation("browseGitRepositoryFiles", "GET", "/customize/git-repositories/{id}/files", "Browse repository files", "Browse files and directories in a git repository", "Customize"), authz.PermGitReposRead, h.BrowseFiles)+ handlerutil.RegisterSecured(api, handlerutil.Operation("syncGitRepositories", "POST", "/git-repositories/sync", "Sync git repositories", "Sync git repositories from a manager to this agent instance", "Git Repositories"), authz.PermGitReposSync, h.SyncRepositories)}// ============================================================================@@ -126,7 +126,7 @@ func RegisterGitRepositories(api huma.API, repoService *services.GitRepositorySe// ListRepositories returns a paginated list of git repositories.func (h *GitRepositoryHandler) ListRepositories(ctx context.Context, input *ListGitRepositoriesInput) (*ListGitRepositoriesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)repositories, paginationResp, err := h.repoService.GetRepositoriesPaginated(ctx, params)if err != nil {@@ -137,14 +137,14 @@ func (h *GitRepositoryHandler) ListRepositories(ctx context.Context, input *ListBody: base.Paginated[gitops.GitRepository]{Success: true,Data: repositories,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}// CreateRepository creates a new git repository.func (h *GitRepositoryHandler) CreateRepository(ctx context.Context, input *CreateGitRepositoryInput) (*CreateGitRepositoryOutput, error) {- actor := currentActorInternal(ctx)+ actor := handlerutil.CurrentActor(ctx)repo, err := h.repoService.CreateRepository(ctx, input.Body, actor)if err != nil {@@ -152,7 +152,7 @@ func (h *GitRepositoryHandler) CreateRepository(ctx context.Context, input *Creareturn nil, huma.NewError(apiErr.HTTPStatus(), "Failed to create git repository")}- body, mapErr := mapOneAPIResponseInternal[*models.GitRepository, gitops.GitRepository](repo, func(err error) string {+ body, mapErr := handlerutil.MapOneAPIResponse[*models.GitRepository, gitops.GitRepository](repo, func(err error) string {return "Failed to map git repository"})if mapErr != nil {@@ -172,7 +172,7 @@ func (h *GitRepositoryHandler) GetRepository(ctx context.Context, input *GetGitRreturn nil, huma.NewError(apiErr.HTTPStatus(), "Failed to retrieve git repository")}- body, mapErr := mapOneAPIResponseInternal[*models.GitRepository, gitops.GitRepository](repo, func(err error) string {+ body, mapErr := handlerutil.MapOneAPIResponse[*models.GitRepository, gitops.GitRepository](repo, func(err error) string {return "Failed to map git repository"})if mapErr != nil {@@ -186,7 +186,7 @@ func (h *GitRepositoryHandler) GetRepository(ctx context.Context, input *GetGitR// UpdateRepository updates an existing git repository.func (h *GitRepositoryHandler) UpdateRepository(ctx context.Context, input *UpdateGitRepositoryInput) (*UpdateGitRepositoryOutput, error) {- actor := currentActorInternal(ctx)+ actor := handlerutil.CurrentActor(ctx)repo, err := h.repoService.UpdateRepository(ctx, input.ID, input.Body, actor)if err != nil {@@ -194,7 +194,7 @@ func (h *GitRepositoryHandler) UpdateRepository(ctx context.Context, input *Updareturn nil, huma.NewError(apiErr.HTTPStatus(), "Failed to update git repository")}- body, mapErr := mapOneAPIResponseInternal[*models.GitRepository, gitops.GitRepository](repo, func(err error) string {+ body, mapErr := handlerutil.MapOneAPIResponse[*models.GitRepository, gitops.GitRepository](repo, func(err error) string {return "Failed to map git repository"})if mapErr != nil {@@ -208,7 +208,7 @@ func (h *GitRepositoryHandler) UpdateRepository(ctx context.Context, input *Upda// DeleteRepository deletes a git repository by ID.func (h *GitRepositoryHandler) DeleteRepository(ctx context.Context, input *DeleteGitRepositoryInput) (*DeleteGitRepositoryOutput, error) {- actor := currentActorInternal(ctx)+ actor := handlerutil.CurrentActor(ctx)if err := h.repoService.DeleteRepository(ctx, input.ID, actor); err != nil {apiErr := models.ToAPIError(err)@@ -227,7 +227,7 @@ func (h *GitRepositoryHandler) DeleteRepository(ctx context.Context, input *Dele// TestRepository tests connectivity and authentication to a git repository.func (h *GitRepositoryHandler) TestRepository(ctx context.Context, input *TestGitRepositoryInput) (*TestGitRepositoryOutput, error) {- actor := currentActorInternal(ctx)+ actor := handlerutil.CurrentActor(ctx)if err := h.repoService.TestConnection(ctx, input.ID, input.Branch, actor); err != nil {return nil, huma.Error400BadRequest(errors.WithMessage(err, "Failed to test git repository connection").Error())diff --git a/backend/internal/gitrepo/module.go b/backend/internal/gitrepo/module.gonew file mode 100644index 000000000..dcc1012f6--- /dev/null+++ b/backend/internal/gitrepo/module.go@@ -0,0 +1,40 @@+// Package gitrepo owns configured Git repositories and their HTTP surface.+package gitrepo++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+)++type Dependencies struct {+ DB *database.DB+ WorkDir string+ Event *event.EventService+ Settings *settings.SettingsService+}++type Module struct {+ service *GitRepositoryService+}++func New(deps Dependencies) *Module {+ return &Module{service: NewGitRepositoryService(deps.DB, deps.WorkDir, deps.Event, deps.Settings)}+}++func (m *Module) Service() *GitRepositoryService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterGitRepositories(api, nil)+ return+ }+ RegisterGitRepositories(api, m.service)+}diff --git a/backend/internal/services/git_repository_service.go b/backend/internal/gitrepo/service.gosimilarity index 94%rename from backend/internal/services/git_repository_service.gorename to backend/internal/gitrepo/service.goindex 5cd3c0329..777aeb3df 100644--- a/backend/internal/services/git_repository_service.go+++ b/backend/internal/gitrepo/service.go@@ -1,4 +1,4 @@-package services+package gitrepoimport ("context"@@ -8,7 +8,9 @@ import ("emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"git "github.com/getarcaneapp/arcane/backend/v2/pkg/gitutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"@@ -22,16 +24,17 @@ import ()type GitRepositoryService struct {+ *git.Client+db *database.DB- gitClient *git.Client- eventService *EventService- settingsService *SettingsService+ eventService *event.EventService+ settingsService *settings.SettingsService}-func NewGitRepositoryService(db *database.DB, workDir string, eventService *EventService, settingsService *SettingsService) *GitRepositoryService {+func NewGitRepositoryService(db *database.DB, workDir string, eventService *event.EventService, settingsService *settings.SettingsService) *GitRepositoryService {return &GitRepositoryService{+ Client: git.NewClient(workDir),db: db,- gitClient: git.NewClient(workDir),eventService: eventService,settingsService: settingsService,}@@ -152,7 +155,7 @@ func (s *GitRepositoryService) CreateRepository(ctx context.Context, req models.}// Log event- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitRepositoryCreate,Severity: models.EventSeveritySuccess,Title: "Git repository created",@@ -244,7 +247,7 @@ func (s *GitRepositoryService) UpdateRepository(ctx context.Context, id string,}// Log event- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitRepositoryUpdate,Severity: models.EventSeveritySuccess,Title: "Git repository updated",@@ -282,7 +285,7 @@ func (s *GitRepositoryService) DeleteRepository(ctx context.Context, id string,}// Log event- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitRepositoryDelete,Severity: models.EventSeverityInfo,Title: "Git repository deleted",@@ -312,10 +315,10 @@ func (s *GitRepositoryService) TestConnection(ctx context.Context, id string, brreturn err}- err = s.gitClient.TestConnection(ctx, repository.URL, branch, authConfig)+ err = s.Client.TestConnection(ctx, repository.URL, branch, authConfig)if err != nil {// Log error event- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitRepositoryError,Severity: models.EventSeverityError,Title: "Git repository connection test failed",@@ -330,7 +333,7 @@ func (s *GitRepositoryService) TestConnection(ctx context.Context, id string, br}// Log success event- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeGitRepositoryTest,Severity: models.EventSeveritySuccess,Title: "Git repository connection successful",@@ -386,7 +389,7 @@ func (s *GitRepositoryService) ListBranches(ctx context.Context, id string) ([]greturn nil, err}- branches, err := s.gitClient.ListBranches(listCtx, repository.URL, authConfig)+ branches, err := s.Client.ListBranches(listCtx, repository.URL, authConfig)if err != nil {return nil, errors.WrapIf(err, "failed to list branches")}@@ -418,19 +421,19 @@ func (s *GitRepositoryService) BrowseFiles(ctx context.Context, id, branch, path}// Clone the repository- repoPath, err := s.gitClient.Clone(ctx, repository.URL, branch, authConfig)+ repoPath, err := s.Clone(ctx, repository.URL, branch, authConfig)if err != nil {return nil, errors.WrapIf(err, "failed to clone repository")}defer func() {- if cleanupErr := s.gitClient.Cleanup(repoPath); cleanupErr != nil {+ if cleanupErr := s.Cleanup(repoPath); cleanupErr != nil {// Log cleanup error but don't fail the operation_ = cleanupErr}}()// Browse the tree- files, err := s.gitClient.BrowseTree(ctx, repoPath, path)+ files, err := s.BrowseTree(ctx, repoPath, path)if err != nil {return nil, err}diff --git a/backend/internal/services/git_repository_service_test.go b/backend/internal/gitrepo/service_test.gosimilarity index 86%rename from backend/internal/services/git_repository_service_test.gorename to backend/internal/gitrepo/service_test.goindex 0b9fe7e5c..72106e1ff 100644--- a/backend/internal/services/git_repository_service_test.go+++ b/backend/internal/gitrepo/service_test.go@@ -1,21 +1,26 @@-package services+package gitrepoimport ("context""fmt""strings""testing"+ "time""emperror.dev/errors""github.com/libtnb/sqlite""github.com/stretchr/testify/assert""github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest""gorm.io/gorm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""go.getarcane.app/sys/crypto")@@ -46,11 +51,30 @@ func setupGitRepositoryServiceTestInternal(t *testing.T) (*GitRepositoryService,settingsService, err := newSettingsServiceForTestInternal(t, context.Background(), wrappedDB)require.NoError(t, err)- eventService := NewEventService(wrappedDB, &config.Config{}, nil)+ eventService := event.NewEventService(wrappedDB, &config.Config{}, nil)return NewGitRepositoryService(wrappedDB, t.TempDir(), eventService, settingsService), wrappedDB}+func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ executor, err := actors.NewExecutor(t.Context(), runtime, "gitrepo-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "gitrepo-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, executor.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, executor, effects)+}+func createGitRepositoryServiceTestRepoInternal(t *testing.T, svc *GitRepositoryService, req models.CreateGitRepositoryRequest) *models.GitRepository {t.Helper()diff --git a/backend/api/handlers/health.go b/backend/internal/health/handler.gosimilarity index 57%rename from backend/api/handlers/health.gorename to backend/internal/health/handler.goindex ef3aa36e8..decf9cfcc 100644--- a/backend/api/handlers/health.go+++ b/backend/internal/health/handler.go@@ -1,4 +1,4 @@-package handlers+package healthimport ("context"@@ -8,13 +8,16 @@ import ("github.com/getarcaneapp/arcane/types/v2/system")-// HealthOutput is the response for health check-type HealthOutput struct {+type healthHandler struct{}++type healthOutput struct {Body system.HealthResponse}-// RegisterHealth registers health check routes using Huma.-func RegisterHealth(api huma.API) {+// RegisterRoutes registers the health check routes.+func RegisterRoutes(api huma.API) {+ handler := &healthHandler{}+huma.Register(api, huma.Operation{OperationID: "health-check",Method: http.MethodGet,@@ -23,13 +26,7 @@ func RegisterHealth(api huma.API) {Description: "Check if the API is healthy",Tags: []string{"Health"},Security: []map[string][]string{},- }, func(ctx context.Context, input *struct{}) (*HealthOutput, error) {- return &HealthOutput{- Body: system.HealthResponse{- Status: "UP",- },- }, nil- })+ }, handler.getHealthInternal)huma.Register(api, huma.Operation{OperationID: "health-check-head",@@ -39,7 +36,17 @@ func RegisterHealth(api huma.API) {Description: "Check if the API is healthy (HEAD request)",Tags: []string{"Health"},Security: []map[string][]string{},- }, func(ctx context.Context, input *struct{}) (*struct{}, error) {- return nil, nil- })+ }, handler.headHealthInternal)+}++func (h *healthHandler) getHealthInternal(context.Context, *struct{}) (*healthOutput, error) {+ return &healthOutput{+ Body: system.HealthResponse{+ Status: "UP",+ },+ }, nil+}++func (h *healthHandler) headHealthInternal(context.Context, *struct{}) (*struct{}, error) {+ return nil, nil}diff --git a/backend/internal/services/image_attestations_service_test.go b/backend/internal/image/attestations_test.gosimilarity index 99%rename from backend/internal/services/image_attestations_service_test.gorename to backend/internal/image/attestations_test.goindex 609207ca5..799d7412d 100644--- a/backend/internal/services/image_attestations_service_test.go+++ b/backend/internal/image/attestations_test.go@@ -1,4 +1,4 @@-package services+package imageimport ("bytes"diff --git a/backend/api/handlers/images.go b/backend/internal/image/handler.gosimilarity index 89%rename from backend/api/handlers/images.gorename to backend/internal/image/handler.goindex 9a61f5cb9..e4ea47112 100644--- a/backend/api/handlers/images.go+++ b/backend/internal/image/handler.go@@ -1,4 +1,4 @@-package handlers+package imageimport ("context"@@ -13,12 +13,17 @@ import ("emperror.dev/errors""github.com/containerd/platforms""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/build"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/image"@@ -29,12 +34,12 @@ import (// ImageHandler provides Huma-based image management endpoints.type ImageHandler struct {- dockerService *services.DockerClientService- imageService *services.ImageService- imageUpdateService *services.ImageUpdateService- settingsService *services.SettingsService- buildService *services.BuildService- activityService *services.ActivityService+ dockerService *docker.DockerClientService+ imageService *ImageService+ imageUpdateService *imageupdate.ImageUpdateService+ settingsService *settings.SettingsService+ buildService *build.BuildService+ activityService *activity.ActivityServiceappCtx context.Context}@@ -186,7 +191,7 @@ type UploadImageOutput struct {}// RegisterImages registers image management routes using Huma.-func RegisterImages(api huma.API, dockerService *services.DockerClientService, imageService *services.ImageService, imageUpdateService *services.ImageUpdateService, settingsService *services.SettingsService, buildService *services.BuildService, activityService *services.ActivityService, appCtx ActivityAppContext) {+func RegisterImages(api huma.API, dockerService *docker.DockerClientService, imageService *ImageService, imageUpdateService *imageupdate.ImageUpdateService, settingsService *settings.SettingsService, buildService *build.BuildService, activityService *activity.ActivityService, appCtx handlerutil.ActivityAppContext) {h := &ImageHandler{dockerService: dockerService,imageService: imageService,@@ -194,157 +199,157 @@ func RegisterImages(api huma.API, dockerService *services.DockerClientService, isettingsService: settingsService,buildService: buildService,activityService: activityService,- appCtx: appCtx.contextInternal(),+ appCtx: appCtx.Context(),}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-images",Method: http.MethodGet,Path: "/environments/{id}/images",Summary: "List images",Description: "Get a paginated list of Docker images",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesList, h.ListImages)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-image-usage-counts",Method: http.MethodGet,Path: "/environments/{id}/images/counts",Summary: "Get image usage counts",Description: "Get counts of images in use, unused, total, and total size",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesList, h.GetImageUsageCounts)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-image-attestations",Method: http.MethodGet,Path: "/environments/{id}/images/{name}/attestations",Summary: "Get image attestations",Description: "Get in-toto attestation statements attached to a Docker image",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesRead, h.GetImageAttestations)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "search-images",Method: http.MethodGet,Path: "/environments/{id}/images/search",Summary: "Search images",Description: "Search Docker Hub images",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesRead, h.SearchImages)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "tag-image",Method: http.MethodPost,Path: "/environments/{id}/images/{name}/tag",Summary: "Tag image",Description: "Add a repository tag to an image",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesTag, h.TagImage)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-image-history",Method: http.MethodGet,Path: "/environments/{id}/images/{name}/history",Summary: "Get image history",Description: "Get Docker image layer history",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesRead, h.GetImageHistory)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "export-image",Method: http.MethodGet,Path: "/environments/{id}/images/{name}/export",Summary: "Export image",Description: "Download a Docker image as a tar archive",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesRead, h.ExportImage)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-image",Method: http.MethodGet,Path: "/environments/{id}/images/{imageId}",Summary: "Get image by ID",Description: "Get a Docker image by its ID",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesRead, h.GetImage)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "remove-image",Method: http.MethodDelete,Path: "/environments/{id}/images/{imageId}",Summary: "Remove an image",Description: "Remove a Docker image by ID",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesDelete, h.RemoveImage)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "pull-image",Method: http.MethodPost,Path: "/environments/{id}/images/pull",Summary: "Pull an image",Description: "Pull a Docker image from a registry with streaming progress output",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesPull, h.PullImage)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "build-image",Method: http.MethodPost,Path: "/environments/{id}/images/build",Summary: "Build an image",Description: "Build a Docker image using BuildKit with streaming progress output",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesBuild, h.BuildImage)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-image-builds",Method: http.MethodGet,Path: "/environments/{id}/images/builds",Summary: "List image builds",Description: "Get a paginated list of image build history for an environment",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesList, h.ListImageBuilds)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-image-build",Method: http.MethodGet,Path: "/environments/{id}/images/builds/{buildId}",Summary: "Get image build",Description: "Get a single image build history entry with output",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesRead, h.GetImageBuild)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "prune-images",Method: http.MethodPost,Path: "/environments/{id}/images/prune",Summary: "Prune unused images",Description: "Remove unused Docker images",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImagesPrune, h.PruneImages)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "upload-image",Method: http.MethodPost,Path: "/environments/{id}/images/upload",Summary: "Upload an image",Description: "Upload a Docker image from a tar archive",Tags: []string{"Images"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),RequestBody: &huma.RequestBody{Content: map[string]*huma.MediaType{"multipart/form-data": {@@ -367,7 +372,7 @@ func RegisterImages(api huma.API, dockerService *services.DockerClientService, i// ListImages returns a paginated list of images.func (h *ImageHandler) ListImages(ctx context.Context, input *ListImagesInput) (*ListImagesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if input.InUse != "" {params.Filters["inUse"] = input.InUse}@@ -392,7 +397,7 @@ func (h *ImageHandler) ListImages(ctx context.Context, input *ListImagesInput) (Body: base.Paginated[image.Summary]{Success: true,Data: images,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -429,7 +434,7 @@ func (h *ImageHandler) GetImageAttestations(ctx context.Context, input *GetImage}}- out, err := h.imageService.GetImageAttestations(ctx, imageName, services.ImageAttestationQuery{+ out, err := h.imageService.GetImageAttestations(ctx, imageName, ImageAttestationQuery{Platform: strings.TrimSpace(input.Platform),PredicateType: strings.TrimSpace(input.PredicateType),IncludeStatement: input.WithStatement,@@ -456,7 +461,7 @@ func (h *ImageHandler) TagImage(ctx context.Context, input *TagImageInput) (*Tagreturn nil, huma.Error400BadRequest("repository is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -564,7 +569,7 @@ func imageExportFileNameInternal(imageName string) string {// RemoveImage removes a Docker image.func (h *ImageHandler) RemoveImage(ctx context.Context, input *RemoveImageInput) (*RemoveImageOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -589,7 +594,7 @@ func (h *ImageHandler) PullImage(ctx context.Context, input *PullImageInput) (*hreturn nil, huma.Error400BadRequest("image name is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -647,7 +652,7 @@ func (h *ImageHandler) BuildImage(ctx context.Context, input *BuildImageInput) (return nil, huma.Error400BadRequest("contextDir is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -705,7 +710,7 @@ func (h *ImageHandler) ListImageBuilds(ctx context.Context, input *ListImageBuilreturn nil, huma.Error400BadRequest("Environment ID is required")}- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if input.Status != "" {params.Filters["status"] = input.Status}@@ -726,7 +731,7 @@ func (h *ImageHandler) ListImageBuilds(ctx context.Context, input *ListImageBuilBody: base.Paginated[image.BuildRecord]{Success: true,Data: builds,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -873,7 +878,7 @@ func (h *ImageHandler) GetImageUsageCounts(ctx context.Context, input *GetImageU// UploadImage uploads a Docker image from a tar archive.func (h *ImageHandler) UploadImage(ctx context.Context, input *UploadImageInput) (*UploadImageOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}diff --git a/backend/internal/services/image_service.go b/backend/internal/image/image.gosimilarity index 91%rename from backend/internal/services/image_service.gorename to backend/internal/image/image.goindex 7cd8d63d8..4eb0d96ca 100644--- a/backend/internal/services/image_service.go+++ b/backend/internal/image/image.go@@ -1,26 +1,32 @@-package services+package imageimport ("context"- "fmt""io""log/slog""strings""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+"emperror.dev/errors"ref "github.com/distribution/reference""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/vulnerability"dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"utilsregistry "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/registryauth""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/imageref""github.com/getarcaneapp/arcane/types/v2/containerregistry"imagetypes "github.com/getarcaneapp/arcane/types/v2/image"systemtypes "github.com/getarcaneapp/arcane/types/v2/system"- "github.com/getarcaneapp/arcane/types/v2/vulnerability"+ vulnerabilitytypes "github.com/getarcaneapp/arcane/types/v2/vulnerability""github.com/moby/moby/api/types/container""github.com/moby/moby/api/types/image""github.com/moby/moby/client"@@ -32,16 +38,16 @@ import (type ImageService struct {db *database.DB- dockerService *DockerClientService- imageUpdateService *ImageUpdateService- registryService *ContainerRegistryService- vulnerabilityService *VulnerabilityService- eventService *EventService+ dockerService *docker.DockerClientService+ imageUpdateService *imageupdate.ImageUpdateService+ registryService *registry.ContainerRegistryService+ vulnerabilityService *vulnerability.VulnerabilityService+ eventService *event.EventServiceprojectIDCache *hot.HotCache[struct{}, map[string]string]}-func NewImageService(db *database.DB, dockerService *DockerClientService, registryService *ContainerRegistryService, imageUpdateService *ImageUpdateService, vulnerabilityService *VulnerabilityService, eventService *EventService) *ImageService {+func NewImageService(db *database.DB, dockerService *docker.DockerClientService, registryService *registry.ContainerRegistryService, imageUpdateService *imageupdate.ImageUpdateService, vulnerabilityService *vulnerability.VulnerabilityService, eventService *event.EventService) *ImageService {return &ImageService{db: db,dockerService: dockerService,@@ -179,7 +185,7 @@ func (s *ImageService) PullImage(ctx context.Context, imageName string, progressslog.DebugContext(ctx, "Attempting to pull image", "image", imageName, "externalCredCount", len(externalCreds))- pullOptions, err := s.getPullOptionsWithAuth(ctx, imageName, externalCreds)+ pullOptions, err := s.PullOptionsWithAuth(ctx, imageName, externalCreds)if err != nil {slog.WarnContext(ctx, "Failed to get registry authentication for image; proceeding without auth", "image", imageName, "error", err.Error())pullOptions = client.ImagePullOptions{}@@ -189,7 +195,7 @@ func (s *ImageService) PullImage(ctx context.Context, imageName string, progressretriedWithoutAuth := falsereader, err := dockerClient.ImagePull(ctx, imageName, pullOptions)- if err != nil && shouldRetryAnonymousPullInternal(pullOptions, err) {+ if err != nil && ShouldRetryAnonymousPull(pullOptions, err) {retriedWithoutAuth = trueslog.WarnContext(ctx, "Docker ImagePull failed with registry auth; retrying anonymously", "image", imageName, "error", err.Error())pullOptions = client.ImagePullOptions{}@@ -424,7 +430,7 @@ func (s *ImageService) ImageExistsLocally(ctx context.Context, imageName string)return false, errors.WrapIff(err, "failed to inspect image %s", imageName)}-func (s *ImageService) getPullOptionsWithAuth(ctx context.Context, imageRef string, externalCreds []containerregistry.Credential) (client.ImagePullOptions, error) {+func (s *ImageService) PullOptionsWithAuth(ctx context.Context, imageRef string, externalCreds []containerregistry.Credential) (client.ImagePullOptions, error) {pullOptions := client.ImagePullOptions{}registryHost := utilsregistry.ExtractRegistryHost(imageRef)@@ -463,7 +469,7 @@ func (s *ImageService) getPullOptionsWithAuth(ctx context.Context, imageRef strireturn pullOptions, nil}-func shouldRetryAnonymousPullInternal(pullOptions client.ImagePullOptions, pullErr error) bool {+func ShouldRetryAnonymousPull(pullOptions client.ImagePullOptions, pullErr error) bool {if pullOptions.RegistryAuth == "" || pullErr == nil {return false}@@ -533,7 +539,7 @@ func (s *ImageService) PruneImages(ctx context.Context, options systemtypes.Prun"imagesDeleted": len(pruneReport.ImagesDeleted),"spaceReclaimed": pruneReport.SpaceReclaimed,}- if logErr := s.eventService.LogImageEvent(ctx, models.EventTypeImageDelete, "", "bulk_prune", systemUser.ID, systemUser.Username, "0", metadata); logErr != nil {+ if logErr := s.eventService.LogImageEvent(ctx, models.EventTypeImageDelete, "", "bulk_prune", models.SystemUser.ID, models.SystemUser.Username, "0", metadata); logErr != nil {slog.Warn("could not log image prune action", "err", logErr)}@@ -683,7 +689,7 @@ func (s *ImageService) ListImagesPaginated(ctx context.Context, params paginatiodefer utils.RecoverToError(&workerErr, "image worker")var err error- imageList, err := s.dockerService.listImagesInternal(groupCtx)+ imageList, err := s.dockerService.ListImages(groupCtx)if err != nil {return errors.WrapIf(err, "failed to list Docker images")}@@ -696,7 +702,7 @@ func (s *ImageService) ListImagesPaginated(ctx context.Context, params paginatiodefer utils.RecoverToError(&workerErr, "image worker")var err error- containerList, err := s.dockerService.listContainersInternal(groupCtx)+ containerList, err := s.dockerService.ListContainers(groupCtx)if err != nil {return errors.WrapIf(err, "failed to list containers")}@@ -720,10 +726,10 @@ func (s *ImageService) ListImagesPaginated(ctx context.Context, params paginatio}projectIDByName := s.BuildProjectIDMap(ctx, containers)- usageMap := buildUsageMapInternal(containers, projectIDByName)+ usageMap := BuildVolumeUsageMap(containers, projectIDByName)updateMap := buildUpdateMap(updateRecords)- items := mapDockerImagesToDTOs(dockerImages, usageMap, updateMap, nil)+ items := MapDockerImagesToDTOs(dockerImages, usageMap, updateMap, nil)config := s.getImagePaginationConfig()@@ -738,7 +744,7 @@ func (s *ImageService) ListImagesPaginated(ctx context.Context, params paginatioapplyVulnerabilitySummariesToItemsInternal(result.Items, vulnerabilityMap)}- paginationResp := pagination.BuildResponseFromFilterResult(result, params)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)return result.Items, paginationResp, nil}@@ -748,10 +754,11 @@ func buildImageRefUpdateLookupsInternal(imageRefs []string) []imageRefUpdateLookseen := make(map[string]struct{}, len(imageRefs))for _, rawRef := range imageRefs {- lookup, ok := parseImageRefUpdateLookupInternal(rawRef).Get()+ originalRef, tag, repositoryCandidates, ok := imageref.ParseUpdateLookup(rawRef)if !ok {continue}+ lookup := imageRefUpdateLookup{originalRef: originalRef, tag: tag, repositoryCandidates: repositoryCandidates}if _, exists := seen[lookup.originalRef]; exists {continue}@@ -762,53 +769,6 @@ func buildImageRefUpdateLookupsInternal(imageRefs []string) []imageRefUpdateLookreturn lookups}-func parseImageRefUpdateLookupInternal(imageRef string) mo.Option[imageRefUpdateLookup] {- trimmedRef := strings.TrimSpace(imageRef)- if trimmedRef == "" {- return mo.None[imageRefUpdateLookup]()- }-- named, err := ref.ParseNormalizedNamed(trimmedRef)- if err != nil {- return mo.None[imageRefUpdateLookup]()- }-- tag := "latest"- if tagged, ok := named.(ref.NamedTagged); ok {- tag = strings.TrimSpace(tagged.Tag())- }- if tag == "" {- tag = "latest"- }-- registryHost := utilsregistry.NormalizeRegistryForComparison(ref.Domain(named))- repositoryPath := strings.TrimSpace(ref.Path(named))- familiarRepository := strings.TrimSpace(ref.FamiliarName(named))-- repositoryCandidates := map[string]struct{}{}- for _, candidate := range []string{- repositoryPath,- familiarRepository,- fmt.Sprintf("%s/%s", registryHost, repositoryPath),- } {- candidate = strings.TrimSpace(candidate)- if candidate == "" {- continue- }- repositoryCandidates[candidate] = struct{}{}- }-- if registryHost == "docker.io" && strings.HasPrefix(repositoryPath, "library/") {- repositoryCandidates[strings.TrimPrefix(repositoryPath, "library/")] = struct{}{}- }-- return mo.Some(imageRefUpdateLookup{- originalRef: trimmedRef,- tag: tag,- repositoryCandidates: repositoryCandidates,- })-}-func selectLatestMatchingImageUpdateRecordInternal(lookup imageRefUpdateLookup,updateRecords []models.ImageUpdateRecord,@@ -843,7 +803,7 @@ func convertLabels(labels map[string]string) map[string]any {}func (s *ImageService) GetTotalImageSize(ctx context.Context) (int64, error) {- images, err := s.dockerService.listImagesInternal(ctx)+ images, err := s.dockerService.ListImages(ctx)if err != nil {return 0, errors.WrapIf(err, "failed to list images")}@@ -923,7 +883,7 @@ func (s *ImageService) BuildProjectIDMap(ctx context.Context, containers []contareturn projectIDs}-func buildUsageMapInternal(containers []container.Summary, projectIDByName map[string]string) map[string][]imagetypes.UsedBy {+func BuildVolumeUsageMap(containers []container.Summary, projectIDByName map[string]string) map[string][]imagetypes.UsedBy {usageMap := make(map[string][]imagetypes.UsedBy)projectSeen := make(map[string]map[string]bool)containerSeen := make(map[string]map[string]bool)@@ -1004,7 +964,7 @@ func getImageIDsFromSummariesInternal(items []imagetypes.Summary) []string {return ids}-func applyVulnerabilitySummariesToItemsInternal(items []imagetypes.Summary, vulnerabilityMap map[string]*vulnerability.ScanSummary) {+func applyVulnerabilitySummariesToItemsInternal(items []imagetypes.Summary, vulnerabilityMap map[string]*vulnerabilitytypes.ScanSummary) {if len(items) == 0 || len(vulnerabilityMap) == 0 {return}@@ -1080,7 +1040,7 @@ func buildUpdateInfo(updateRecord *models.ImageUpdateRecord) *imagetypes.UpdateI}}-func mapDockerImagesToDTOs(dockerImages []image.Summary, usageMap map[string][]imagetypes.UsedBy, updateMap map[string]*models.ImageUpdateRecord, vulnerabilityMap map[string]*vulnerability.ScanSummary) []imagetypes.Summary {+func MapDockerImagesToDTOs(dockerImages []image.Summary, usageMap map[string][]imagetypes.UsedBy, updateMap map[string]*models.ImageUpdateRecord, vulnerabilityMap map[string]*vulnerabilitytypes.ScanSummary) []imagetypes.Summary {items := make([]imagetypes.Summary, 0, len(dockerImages))for _, di := range dockerImages {repo, tag := determineRepoAndTag(di)diff --git a/backend/internal/services/image_attestations_service.go b/backend/internal/image/image_attestations.gosimilarity index 99%rename from backend/internal/services/image_attestations_service.gorename to backend/internal/image/image_attestations.goindex 552b33d1e..e90c45d9b 100644--- a/backend/internal/services/image_attestations_service.go+++ b/backend/internal/image/image_attestations.go@@ -1,4 +1,4 @@-package services+package imageimport ("bytes"diff --git a/backend/internal/image/module.go b/backend/internal/image/module.gonew file mode 100644index 000000000..93b50efe0--- /dev/null+++ b/backend/internal/image/module.go@@ -0,0 +1,45 @@+// Package image owns Docker image operations, metadata, attestations, and routes.+package image++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/build"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+)++type Dependencies struct {+ Docker *docker.DockerClientService+ ImageUpdate *imageupdate.ImageUpdateService+ Settings *settings.SettingsService+ Build *build.BuildService+ Activity *activity.ActivityService+}++type Module struct {+ service *ImageService+ deps Dependencies+}++func New(service *ImageService, deps Dependencies) *Module {+ return &Module{service: service, deps: deps}+}++func (m *Module) Service() *ImageService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API, appCtx handlerutil.ActivityAppContext) {+ if m == nil {+ RegisterImages(api, nil, nil, nil, nil, nil, nil, appCtx)+ return+ }+ RegisterImages(api, m.deps.Docker, m.service, m.deps.ImageUpdate, m.deps.Settings, m.deps.Build, m.deps.Activity, appCtx)+}diff --git a/backend/internal/services/image_service_test.go b/backend/internal/image/service_test.gosimilarity index 61%rename from backend/internal/services/image_service_test.gorename to backend/internal/image/service_test.goindex 4e1ff8e5c..f97888188 100644--- a/backend/internal/services/image_service_test.go+++ b/backend/internal/image/service_test.go@@ -1,4 +1,4 @@-package services+package imageimport ("context"@@ -8,27 +8,53 @@ import ("net/http""net/http/httptest""net/url"+ "regexp""strings""testing""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+"github.com/libtnb/sqlite"+ dockerauthconfig "github.com/moby/moby/api/pkg/authconfig""github.com/stretchr/testify/require""gorm.io/gorm""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/types/v2/containerregistry"imagetypes "github.com/getarcaneapp/arcane/types/v2/image""github.com/getarcaneapp/arcane/types/v2/vulnerability"- dockerauthconfig "github.com/moby/moby/api/pkg/authconfig"+ dockertypesimage "github.com/moby/moby/api/types/image"dockerregistry "github.com/moby/moby/api/types/registry""github.com/moby/moby/client""github.com/stretchr/testify/assert""go.getarcane.app/sys/crypto")+var imageDockerAPIVersionPrefixInternal = regexp.MustCompile(`^/v[0-9]+\.[0-9]+`)++func imageDockerTestPathInternal(path string) string {+ return imageDockerAPIVersionPrefixInternal.ReplaceAllString(path, "")+}++func setupImageProjectTestDBInternal(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(+ &models.Project{},+ &models.SettingVariable{},+ &models.ImageUpdateRecord{},+ &models.Event{},+ ))+ return &database.DB{DB: db}+}+func TestGetImageIDsFromSummariesInternal(t *testing.T) {items := []imagetypes.Summary{{ID: "img1"},@@ -123,7 +149,7 @@ func setupImageServiceAuthTest(t *testing.T) (*ImageService, *database.DB) {dbWrap := &database.DB{DB: db}svc := &ImageService{- registryService: NewContainerRegistryService(dbWrap, nil, NewKVService(dbWrap)),+ registryService: registry.NewContainerRegistryService(dbWrap, nil, kv.NewKVService(dbWrap)),}return svc, dbWrap@@ -140,24 +166,16 @@ func createTestPullRegistry(t *testing.T, db *database.DB, url, username, tokenUsername: username,Token: encryptedToken,Enabled: true,- RegistryType: registryTypeGeneric,+ RegistryType: registry.RegistryTypeGeneric,}require.NoError(t, db.WithContext(context.Background()).Create(reg).Error)}-func decodeRegistryAuth(t *testing.T, encoded string) dockerregistry.AuthConfig {- t.Helper()-- cfg, err := dockerauthconfig.Decode(encoded)- require.NoError(t, err)- return *cfg-}-func TestGetPullOptionsWithAuth_DBRegistrySkipsEmptyToken(t *testing.T) {svc, db := setupImageServiceAuthTest(t)createTestPullRegistry(t, db, "https://docker.io", "docker-user", " ")- pullOptions, err := svc.getPullOptionsWithAuth(context.Background(), "docker.io/library/nginx:latest", nil)+ pullOptions, err := svc.PullOptionsWithAuth(context.Background(), "docker.io/library/nginx:latest", nil)require.NoError(t, err)assert.Empty(t, pullOptions.RegistryAuth)}@@ -166,7 +184,7 @@ func TestGetPullOptionsWithAuth_DBRegistrySkipsEmptyUsername(t *testing.T) {svc, db := setupImageServiceAuthTest(t)createTestPullRegistry(t, db, "https://docker.io", " ", "docker-token")- pullOptions, err := svc.getPullOptionsWithAuth(context.Background(), "docker.io/library/nginx:latest", nil)+ pullOptions, err := svc.PullOptionsWithAuth(context.Background(), "docker.io/library/nginx:latest", nil)require.NoError(t, err)assert.Empty(t, pullOptions.RegistryAuth)}@@ -175,11 +193,11 @@ func TestGetPullOptionsWithAuth_DBRegistryUsesValidCredentials(t *testing.T) {svc, db := setupImageServiceAuthTest(t)createTestPullRegistry(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")- pullOptions, err := svc.getPullOptionsWithAuth(context.Background(), "docker.io/library/nginx:latest", nil)+ pullOptions, err := svc.PullOptionsWithAuth(context.Background(), "docker.io/library/nginx:latest", nil)require.NoError(t, err)require.NotEmpty(t, pullOptions.RegistryAuth)- authCfg := decodeRegistryAuth(t, pullOptions.RegistryAuth)+ authCfg := decodeRegistryAuthInternal(t, pullOptions.RegistryAuth)assert.Equal(t, "docker-user", authCfg.Username)assert.Equal(t, "docker-token", authCfg.Password)assert.Equal(t, "https://index.docker.io/v1/", authCfg.ServerAddress)@@ -189,20 +207,20 @@ func TestGetPullOptionsWithAuth_ExternalCredentialsOverrideDBRegistryInternal(tsvc, db := setupImageServiceAuthTest(t)createTestPullRegistry(t, db, "https://registry.example.com", "db-user", "db-token")- pullOptions, err := svc.getPullOptionsWithAuth(context.Background(), "registry.example.com/team/app:latest", []containerregistry.Credential{+ pullOptions, err := svc.PullOptionsWithAuth(context.Background(), "registry.example.com/team/app:latest", []containerregistry.Credential{{URL: "https://registry.example.com", Username: "external-user", Token: "external-token", Enabled: true},})require.NoError(t, err)require.NotEmpty(t, pullOptions.RegistryAuth)- authCfg := decodeRegistryAuth(t, pullOptions.RegistryAuth)+ authCfg := decodeRegistryAuthInternal(t, pullOptions.RegistryAuth)assert.Equal(t, "external-user", authCfg.Username)assert.Equal(t, "external-token", authCfg.Password)assert.Equal(t, "registry.example.com", authCfg.ServerAddress)}func TestImageServicePullImageRetriesAnonymouslyAfterAuthRejectedInternal(t *testing.T) {- db := setupProjectTestDB(t)+ db := setupImageProjectTestDBInternal(t)var authHeaders []stringserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {if !strings.HasSuffix(r.URL.Path, "/images/create") {@@ -218,10 +236,10 @@ func TestImageServicePullImageRetriesAnonymouslyAfterAuthRejectedInternal(t *tes}))t.Cleanup(server.Close)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- imageSvc := NewImageService(db, dockerService, nil, nil, nil, NewEventService(db, nil, nil))+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}+ imageSvc := NewImageService(db, dockerService, nil, nil, nil, event.NewEventService(db, nil, nil))- err := imageSvc.PullImage(context.Background(), "registry.example.com/team/app:latest", io.Discard, systemUser, []containerregistry.Credential{+ err := imageSvc.PullImage(context.Background(), "registry.example.com/team/app:latest", io.Discard, models.SystemUser, []containerregistry.Credential{{URL: "https://registry.example.com", Username: "external-user", Token: "external-token", Enabled: true},})require.NoError(t, err)@@ -231,10 +249,10 @@ func TestImageServicePullImageRetriesAnonymouslyAfterAuthRejectedInternal(t *tes}func TestImageServiceTagImageCallsDockerAPIInternal(t *testing.T) {- db := setupProjectTestDB(t)+ db := setupImageProjectTestDBInternal(t)var gotRepo, gotTag stringserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- if r.Method != http.MethodPost || dockerTestPathInternal(r.URL.Path) != "/images/source:latest/tag" {+ if r.Method != http.MethodPost || imageDockerTestPathInternal(r.URL.Path) != "/images/source:latest/tag" {http.NotFound(w, r)return}@@ -244,18 +262,18 @@ func TestImageServiceTagImageCallsDockerAPIInternal(t *testing.T) {}))t.Cleanup(server.Close)- imageSvc := NewImageService(db, &DockerClientService{client: newTestDockerClient(t, server)}, nil, nil, nil, NewEventService(db, nil, nil))+ imageSvc := NewImageService(db, &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}, nil, nil, nil, event.NewEventService(db, nil, nil))- err := imageSvc.TagImage(context.Background(), "source:latest", imagetypes.TagRequest{Repository: "registry.example.com/team/app", Tag: "v2"}, systemUser)+ err := imageSvc.TagImage(context.Background(), "source:latest", imagetypes.TagRequest{Repository: "registry.example.com/team/app", Tag: "v2"}, models.SystemUser)require.NoError(t, err)assert.Equal(t, "registry.example.com/team/app", gotRepo)assert.Equal(t, "v2", gotTag)}func TestImageServiceGetImageHistoryCallsDockerAPIInternal(t *testing.T) {- db := setupProjectTestDB(t)+ db := setupImageProjectTestDBInternal(t)server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- if r.Method != http.MethodGet || dockerTestPathInternal(r.URL.Path) != "/images/source:latest/history" {+ if r.Method != http.MethodGet || imageDockerTestPathInternal(r.URL.Path) != "/images/source:latest/history" {http.NotFound(w, r)return}@@ -265,7 +283,7 @@ func TestImageServiceGetImageHistoryCallsDockerAPIInternal(t *testing.T) {}))t.Cleanup(server.Close)- imageSvc := NewImageService(db, &DockerClientService{client: newTestDockerClient(t, server)}, nil, nil, nil, NewEventService(db, nil, nil))+ imageSvc := NewImageService(db, &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}, nil, nil, nil, event.NewEventService(db, nil, nil))history, err := imageSvc.GetImageHistory(context.Background(), "source:latest")require.NoError(t, err)@@ -275,7 +293,7 @@ func TestImageServiceGetImageHistoryCallsDockerAPIInternal(t *testing.T) {}func TestImageServiceSearchImagesRequiresTermInternal(t *testing.T) {- imageSvc := NewImageService(nil, &DockerClientService{}, nil, nil, nil, nil)+ imageSvc := NewImageService(nil, &docker.DockerClientService{}, nil, nil, nil, nil)_, err := imageSvc.SearchImages(context.Background(), " ")require.Error(t, err)@@ -283,9 +301,9 @@ func TestImageServiceSearchImagesRequiresTermInternal(t *testing.T) {}func TestImageServiceExportImageReturnsTarStreamInternal(t *testing.T) {- db := setupProjectTestDB(t)+ db := setupImageProjectTestDBInternal(t)server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- if r.Method != http.MethodGet || dockerTestPathInternal(r.URL.Path) != "/images/get" {+ if r.Method != http.MethodGet || imageDockerTestPathInternal(r.URL.Path) != "/images/get" {http.NotFound(w, r)return}@@ -297,7 +315,7 @@ func TestImageServiceExportImageReturnsTarStreamInternal(t *testing.T) {}))t.Cleanup(server.Close)- imageSvc := NewImageService(db, &DockerClientService{client: newTestDockerClient(t, server)}, nil, nil, nil, NewEventService(db, nil, nil))+ imageSvc := NewImageService(db, &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}, nil, nil, nil, event.NewEventService(db, nil, nil))reader, err := imageSvc.ExportImage(context.Background(), "source:latest")require.NoError(t, err)@@ -308,10 +326,10 @@ func TestImageServiceExportImageReturnsTarStreamInternal(t *testing.T) {}func TestImageServiceSearchImagesCallsDockerAPIInternal(t *testing.T) {- db := setupProjectTestDB(t)+ db := setupImageProjectTestDBInternal(t)var gotTerm stringserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- if r.Method != http.MethodGet || dockerTestPathInternal(r.URL.Path) != "/images/search" {+ if r.Method != http.MethodGet || imageDockerTestPathInternal(r.URL.Path) != "/images/search" {http.NotFound(w, r)return}@@ -322,7 +340,7 @@ func TestImageServiceSearchImagesCallsDockerAPIInternal(t *testing.T) {}))t.Cleanup(server.Close)- imageSvc := NewImageService(db, &DockerClientService{client: newTestDockerClient(t, server)}, nil, nil, nil, NewEventService(db, nil, nil))+ imageSvc := NewImageService(db, &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}, nil, nil, nil, event.NewEventService(db, nil, nil))results, err := imageSvc.SearchImages(context.Background(), "nginx")require.NoError(t, err)@@ -335,13 +353,99 @@ func TestImageServiceSearchImagesCallsDockerAPIInternal(t *testing.T) {func TestShouldRetryAnonymousPullInternal_UnauthorizedWithAuth(t *testing.T) {err := errors.New(`error response from daemon: Head "registry-1.docker.io/v2/library/nginx/manifests/latest": unauthorized: incorrect username or password`)- assert.True(t, shouldRetryAnonymousPullInternal(client.ImagePullOptions{RegistryAuth: "encoded-auth"}, err))+ assert.True(t, ShouldRetryAnonymousPull(client.ImagePullOptions{RegistryAuth: "encoded-auth"}, err))}func TestShouldRetryAnonymousPullInternal_SkipsRetryWithoutUnauthorizedOrAuth(t *testing.T) {nonAuthErr := errors.New("error response from daemon: i/o timeout")unauthorizedErr := errors.New("unauthorized: authentication required")- assert.False(t, shouldRetryAnonymousPullInternal(client.ImagePullOptions{RegistryAuth: "encoded-auth"}, nonAuthErr))- assert.False(t, shouldRetryAnonymousPullInternal(client.ImagePullOptions{}, unauthorizedErr))+ assert.False(t, ShouldRetryAnonymousPull(client.ImagePullOptions{RegistryAuth: "encoded-auth"}, nonAuthErr))+ assert.False(t, ShouldRetryAnonymousPull(client.ImagePullOptions{}, unauthorizedErr))+}++// Test fixtures shared by this package's tests.++// decodeRegistryAuthInternal decodes a base64 X-Registry-Auth header.+func decodeRegistryAuthInternal(t *testing.T, encoded string) dockerregistry.AuthConfig {+ t.Helper()++ cfg, err := dockerauthconfig.Decode(encoded)+ require.NoError(t, err)+ return *cfg+}++// newTestDockerClientInternal builds a Docker client pointed at a fake Docker HTTP server.+func newTestDockerClientInternal(t *testing.T, server *httptest.Server) *client.Client {+ t.Helper()++ httpClient := server.Client()+ cli, err := client.New(+ client.WithHost(server.URL),+ client.WithAPIVersion("1.41"),+ client.WithHTTPClient(httpClient),+ )+ require.NoError(t, err)++ t.Cleanup(func() {+ _ = cli.Close()+ })++ return cli+}++// newImagePullServerWithObserverInternal is NewImagePullServer with a pull callback.+func newImagePullServerWithObserverInternal(t *testing.T, inspectByRef map[string]dockertypesimage.InspectResponse, onPull func(fullRef string, authHeader string)) *httptest.Server {+ t.Helper()++ server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ switch {+ case strings.HasSuffix(r.URL.Path, "/images/create"):+ fullRef := strings.TrimSpace(r.URL.Query().Get("fromImage"))+ tag := strings.TrimSpace(r.URL.Query().Get("tag"))+ if fullRef != "" && tag != "" {+ lastSlash := strings.LastIndex(fullRef, "/")+ lastColon := strings.LastIndex(fullRef, ":")+ if lastColon <= lastSlash {+ fullRef += ":" + tag+ }+ }+ if onPull != nil {+ onPull(fullRef, strings.TrimSpace(r.Header.Get("X-Registry-Auth")))+ }++ w.Header().Set("Content-Type", "application/json")+ _ = json.NewEncoder(w).Encode(map[string]string{"status": "Pulled", "id": fullRef})+ return+ case strings.Contains(r.URL.Path, "/images/") && strings.HasSuffix(r.URL.Path, "/json"):+ path := r.URL.Path+ imagePathIndex := strings.Index(path, "/images/")+ if !assert.NotEqual(t, -1, imagePathIndex) {+ return+ }+ encodedRef := strings.TrimSuffix(path[imagePathIndex+len("/images/"):], "/json")+ imageRef, err := url.PathUnescape(encodedRef)+ if !assert.NoError(t, err) {+ return+ }++ inspect, ok := inspectByRef[imageRef]+ if !ok {+ http.Error(w, "not found", http.StatusNotFound)+ return+ }++ w.Header().Set("Content-Type", "application/json")+ if !assert.NoError(t, json.NewEncoder(w).Encode(inspect)) {+ return+ }+ return+ default:+ http.NotFound(w, r)+ }+ }))++ t.Cleanup(server.Close)++ return server}diff --git a/backend/api/handlers/image_updates.go b/backend/internal/imageupdate/handler.gosimilarity index 84%rename from backend/api/handlers/image_updates.gorename to backend/internal/imageupdate/handler.goindex 99f403bf2..fd8859316 100644--- a/backend/api/handlers/image_updates.go+++ b/backend/internal/imageupdate/handler.go@@ -1,4 +1,4 @@-package handlers+package imageupdateimport ("context"@@ -7,19 +7,19 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"imagetypes "github.com/getarcaneapp/arcane/types/v2/image""github.com/getarcaneapp/arcane/types/v2/imageupdate")type ImageUpdateHandler struct {- imageUpdateService *services.ImageUpdateService- imageService *services.ImageService- appCtx context.Context+ imageUpdateService *ImageUpdateService+ getUpdateInfoByImageRefs func(context.Context, []string) (map[string]*imagetypes.UpdateInfo, error)+ appCtx context.Context}type CheckImageUpdateInput struct {@@ -76,74 +76,74 @@ type GetUpdateSummaryOutput struct {}// RegisterImageUpdates registers image update endpoints.-func RegisterImageUpdates(api huma.API, imageUpdateSvc *services.ImageUpdateService, imageSvc *services.ImageService, appCtx ActivityAppContext) {+func RegisterImageUpdates(api huma.API, imageUpdateSvc *ImageUpdateService, getUpdateInfoByImageRefs func(context.Context, []string) (map[string]*imagetypes.UpdateInfo, error), appCtx handlerutil.ActivityAppContext) {h := &ImageUpdateHandler{- imageUpdateService: imageUpdateSvc,- imageService: imageSvc,- appCtx: appCtx.contextInternal(),+ imageUpdateService: imageUpdateSvc,+ getUpdateInfoByImageRefs: getUpdateInfoByImageRefs,+ appCtx: appCtx.Context(),}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "check-image-update",Method: http.MethodGet,Path: "/environments/{id}/image-updates/check",Summary: "Check image update by reference",Tags: []string{"Image Updates"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesCheck, h.CheckImageUpdate)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "check-image-update-by-id",Method: http.MethodGet,Path: "/environments/{id}/image-updates/check/{imageId}",Summary: "Check image update by ID",Tags: []string{"Image Updates"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesCheck, h.CheckImageUpdateByID)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "check-image-update-by-id-post",Method: http.MethodPost,Path: "/environments/{id}/image-updates/check/{imageId}",Summary: "Check image update by ID (POST)",Tags: []string{"Image Updates"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesCheck, h.CheckImageUpdateByID)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "check-multiple-images",Method: http.MethodPost,Path: "/environments/{id}/image-updates/check-batch",Summary: "Check multiple images",Tags: []string{"Image Updates"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesCheck, h.CheckMultipleImages)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "check-all-images",Method: http.MethodPost,Path: "/environments/{id}/image-updates/check-all",Summary: "Check all images",Tags: []string{"Image Updates"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesCheck, h.CheckAllImages)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-update-info-by-refs",Method: http.MethodGet,Path: "/environments/{id}/image-updates/by-refs",Summary: "Get persisted update info for image references",Tags: []string{"Image Updates"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesRead, h.GetUpdateInfoByRefs)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-update-summary",Method: http.MethodGet,Path: "/environments/{id}/image-updates/summary",Summary: "Get update summary",Tags: []string{"Image Updates"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesRead, h.GetUpdateSummary)}@@ -236,7 +236,7 @@ func (h *ImageUpdateHandler) GetUpdateInfoByRefs(ctx context.Context, input *Get}, nil}- result, err := h.imageService.GetUpdateInfoByImageRefs(ctx, imageRefs)+ result, err := h.getUpdateInfoByImageRefs(ctx, imageRefs)if err != nil {return nil, huma.Error500InternalServerError(errors.WithMessage(err, "Failed to check image updates").Error())}diff --git a/backend/internal/services/image_update_service.go b/backend/internal/imageupdate/image_update.gosimilarity index 95%rename from backend/internal/services/image_update_service.gorename to backend/internal/imageupdate/image_update.goindex bf7bed481..ada2d2822 100644--- a/backend/internal/services/image_update_service.go+++ b/backend/internal/imageupdate/image_update.go@@ -1,4 +1,4 @@-package services+package imageupdateimport ("context"@@ -9,18 +9,26 @@ import ("sync""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+"emperror.dev/emperror""emperror.dev/errors"ref "github.com/distribution/reference"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/ratelimit"- registry "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/registryauth"+ registryauth "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/registryauth""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"projectspkg "github.com/getarcaneapp/arcane/backend/v2/pkg/projects""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/imageref""github.com/getarcaneapp/arcane/types/v2/containerregistry""github.com/getarcaneapp/arcane/types/v2/imageupdate""github.com/moby/moby/api/types/container"@@ -37,13 +45,13 @@ import (type ImageUpdateService struct {db *database.DB- settingsService *SettingsService- registryService *ContainerRegistryService- dockerService *DockerClientService- eventService *EventService- notificationService *NotificationService+ settingsService *settings.SettingsService+ registryService *registry.ContainerRegistryService+ dockerService *docker.DockerClientService+ eventService *event.EventService+ notificationService *notification.NotificationServiceregistryLimiter *ratelimit.RegistryRateLimiter- activityService *ActivityService+ activityService *activity.ActivityServicenotifyMu sync.Mutex}@@ -62,7 +70,7 @@ type localImageSnapshot struct {IsLocalBuild bool}-func NewImageUpdateService(db *database.DB, settingsService *SettingsService, registryService *ContainerRegistryService, dockerService *DockerClientService, eventService *EventService, notificationService *NotificationService, activityService *ActivityService) *ImageUpdateService {+func NewImageUpdateService(db *database.DB, settingsService *settings.SettingsService, registryService *registry.ContainerRegistryService, dockerService *docker.DockerClientService, eventService *event.EventService, notificationService *notification.NotificationService, activityService *activity.ActivityService) *ImageUpdateService {return &ImageUpdateService{db: db,settingsService: settingsService,@@ -139,7 +147,7 @@ func (s *ImageUpdateService) startImageUpdateActivityInternal(ctx context.Contexif count > 1 {resourceType = "images"}- activity, err := s.activityService.StartActivity(ctx, StartActivityRequest{+ activity, err := s.activityService.StartActivity(ctx, activity.StartActivityRequest{EnvironmentID: "0",Type: models.ActivityTypeImageUpdateCheck,Queue: true,@@ -165,7 +173,7 @@ func (s *ImageUpdateService) appendImageUpdateActivityMessageInternal(ctx contexif level == "" {level = models.ActivityMessageLevelInfo}- if _, err := s.activityService.AppendMessage(ctx, activityID, AppendActivityMessageRequest{+ if _, err := s.activityService.AppendMessage(ctx, activityID, activity.AppendActivityMessageRequest{Level: level,Message: message,Progress: &progress,@@ -222,7 +230,7 @@ func (s *ImageUpdateService) CheckImageUpdate(ctx context.Context, imageRef stri"responseTimeMs": result.ResponseTimeMs,"skippedReason": "digest_pinned",}- if logErr := s.eventService.LogImageEvent(ctx, models.EventTypeImageScan, "", imageRef, systemUser.ID, systemUser.Username, "0", metadata); logErr != nil {+ if logErr := s.eventService.LogImageEvent(ctx, models.EventTypeImageScan, "", imageRef, models.SystemUser.ID, models.SystemUser.Username, "0", metadata); logErr != nil {slog.WarnContext(ctx, "Failed to log digest-pinned image update check event", "imageRef", imageRef, "error", logErr.Error())}}@@ -266,7 +274,7 @@ func (s *ImageUpdateService) CheckImageUpdate(ctx context.Context, imageRef stri"error": err.Error(),"checkType": "digest",}- if logErr := s.eventService.LogImageEvent(ctx, models.EventTypeImageScan, "", imageRef, systemUser.ID, systemUser.Username, "0", metadata); logErr != nil {+ if logErr := s.eventService.LogImageEvent(ctx, models.EventTypeImageScan, "", imageRef, models.SystemUser.ID, models.SystemUser.Username, "0", metadata); logErr != nil {slog.WarnContext(ctx, "Failed to log image update check error event", "imageRef", imageRef, "error", logErr.Error())}if saveErr := s.saveUpdateResultWithSnapshotInternal(ctx, imageRef, result, snapshot); saveErr != nil {@@ -290,7 +298,7 @@ func (s *ImageUpdateService) CheckImageUpdate(ctx context.Context, imageRef stri"latestDigest": digestResult.LatestDigest,"responseTimeMs": digestResult.ResponseTimeMs,}- if logErr := s.eventService.LogImageEvent(ctx, models.EventTypeImageScan, "", imageRef, systemUser.ID, systemUser.Username, "0", metadata); logErr != nil {+ if logErr := s.eventService.LogImageEvent(ctx, models.EventTypeImageScan, "", imageRef, models.SystemUser.ID, models.SystemUser.Username, "0", metadata); logErr != nil {slog.WarnContext(ctx, "Failed to log image update check event", "imageRef", imageRef, "error", logErr.Error())}if saveErr := s.saveUpdateResultWithSnapshotInternal(ctx, imageRef, digestResult, snapshot); saveErr != nil {@@ -390,7 +398,7 @@ func (s *ImageUpdateService) checkDigestUpdateWithSnapshotInternal(ctx context.C}registryCtx, registryCancel := s.registryContextInternal(ctx)- digestResult, err := s.registryService.inspectImageDigestInternal(registryCtx, imageRef, nil)+ digestResult, err := s.registryService.InspectImageDigest(registryCtx, imageRef, nil)registryCancel()elapsed := time.Since(start)if err != nil {@@ -482,7 +490,7 @@ func (s *ImageUpdateService) parseImageReference(imageRef string) *ImageParts {}return &ImageParts{- Registry: registry.NormalizeRegistryForComparison(registryHost),+ Registry: registryauth.NormalizeRegistryForComparison(registryHost),Repository: repository,Tag: tag,}@@ -545,7 +553,7 @@ func (s *ImageUpdateService) parseImageReferenceFallback(imageRef string) *Image}}}- return &ImageParts{Registry: registry.NormalizeRegistryForComparison(registryHost), Repository: repository, Tag: tag}+ return &ImageParts{Registry: registryauth.NormalizeRegistryForComparison(registryHost), Repository: repository, Tag: tag}}func (s *ImageUpdateService) getImageRefByIDInternal(ctx context.Context, imageID string) (string, error) {@@ -812,7 +820,7 @@ func (s *ImageUpdateService) CheckImageUpdateByID(ctx context.Context, imageID s"imageID": imageID,"error": err.Error(),}- if logErr := s.eventService.LogImageEvent(ctx, models.EventTypeImageScan, imageID, "", systemUser.ID, systemUser.Username, "0", metadata); logErr != nil {+ if logErr := s.eventService.LogImageEvent(ctx, models.EventTypeImageScan, imageID, "", models.SystemUser.ID, models.SystemUser.Username, "0", metadata); logErr != nil {slog.WarnContext(ctx, "Failed to log image update check by ID error event", "imageID", imageID, "error", logErr.Error())}return nil, errors.WrapIf(err, "failed to get image reference")@@ -1000,7 +1008,7 @@ func savePreparedUpdateResultWithTxInternal(tx *gorm.DB, imageID, repo, tag stri// instead of clobbering it with an error recordif hasExisting &&strings.TrimSpace(result.Error) != "" &&- isRateLimitErrorStringInternal(result.Error) &&+ registry.IsRateLimitErrorString(result.Error) &&strings.TrimSpace(mo.PointerToOption(existingRecord.LastError).OrEmpty()) == "" {slog.Debug("Preserving previous image update result; check hit a registry rate limit","imageID", imageID, "repository", repo, "tag", tag, "error", result.Error)@@ -1098,16 +1106,16 @@ func (s *ImageUpdateService) MarkImageRefUpToDateAfterPull(ctx context.Context,ResponseTimeMs: 0,}- lookup := parseImageRefUpdateLookupInternal(imageRef)+ _, tag, repositoryCandidates, hasLookup := imageref.ParseUpdateLookup(imageRef)return s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {- if lookup, hasLookup := lookup.Get(); hasLookup {- repositories := repositoryCandidatesSliceInternal(lookup.repositoryCandidates)+ if hasLookup {+ repositories := repositoryCandidatesSliceInternal(repositoryCandidates)if len(repositories) > 0 {// Only clear synthetic ref:: records, not real sha256 image ID records.// Clearing sha256 records would incorrectly mark containers that are still// running the old image as up-to-date (see: #2453).if err := tx.Model(&models.ImageUpdateRecord{}).- Where("id LIKE 'ref::%' AND tag = ? AND repository IN ?", lookup.tag, repositories).+ Where("id LIKE 'ref::%' AND tag = ? AND repository IN ?", tag, repositories).Update("has_update", false).Error; err != nil {return errors.WrapIf(err, "clear stale image updates")}@@ -1122,7 +1130,7 @@ func (s *ImageUpdateService) MarkImageRefUpToDateAfterPull(ctx context.Context,})}-func (s *ImageUpdateService) getStoredUpdateByImageIDInternal(ctx context.Context, imageID string) (*models.ImageUpdateRecord, bool, error) {+func (s *ImageUpdateService) StoredUpdateByImageID(ctx context.Context, imageID string) (*models.ImageUpdateRecord, bool, error) {imageID = strings.TrimSpace(imageID)if s == nil || s.db == nil || imageID == "" {return nil, false, nil@@ -1250,7 +1258,7 @@ func (s *ImageUpdateService) checkSingleImageInBatchInternal(ctx context.Context}registryCtx, registryCancel := s.registryContextInternal(ctx)- digestResult, digestErr := s.registryService.inspectImageDigestInternal(registryCtx, imageRef, externalCreds)+ digestResult, digestErr := s.registryService.InspectImageDigest(registryCtx, imageRef, externalCreds)registryCancel()if digestErr != nil {resp := &imageupdate.Response{@@ -1528,12 +1536,12 @@ func (s *ImageUpdateService) CheckMultipleImages(ctx context.Context, imageRefs"errorCount", errorCount,"duration", time.Since(startBatch))- s.sendBatchImageUpdateNotificationsInternal(ctx)+ s.SendBatchUpdateNotifications(ctx)return results, nil}-func (s *ImageUpdateService) sendBatchImageUpdateNotificationsInternal(ctx context.Context) {+func (s *ImageUpdateService) SendBatchUpdateNotifications(ctx context.Context) {if s.notificationService == nil {return}@@ -1676,7 +1684,7 @@ func (s *ImageUpdateService) GetUpdateSummary(ctx context.Context) (*imageupdatereturn nil, errors.New("docker service unavailable")}- dockerImages, err := s.dockerService.listImagesInternal(ctx)+ dockerImages, err := s.dockerService.ListImages(ctx)if err != nil {return nil, err}diff --git a/backend/internal/imageupdate/module.go b/backend/internal/imageupdate/module.gonew file mode 100644index 000000000..da66bb6f9--- /dev/null+++ b/backend/internal/imageupdate/module.go@@ -0,0 +1,39 @@+// Package imageupdate owns image update checks, persistence, and HTTP routes.+package imageupdate++import (+ "context"++ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+ imagetypes "github.com/getarcaneapp/arcane/types/v2/image"+)++type Dependencies struct {+ GetUpdateInfoByImageRefs func(context.Context, []string) (map[string]*imagetypes.UpdateInfo, error)+}++type Module struct {+ service *ImageUpdateService+ deps Dependencies+}++func New(service *ImageUpdateService, deps Dependencies) *Module {+ return &Module{service: service, deps: deps}+}++func (m *Module) Service() *ImageUpdateService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API, appCtx handlerutil.ActivityAppContext) {+ if m == nil {+ RegisterImageUpdates(api, nil, nil, appCtx)+ return+ }+ RegisterImageUpdates(api, m.service, m.deps.GetUpdateInfoByImageRefs, appCtx)+}diff --git a/backend/internal/services/image_update_service_test.go b/backend/internal/imageupdate/service_test.gosimilarity index 88%rename from backend/internal/services/image_update_service_test.gorename to backend/internal/imageupdate/service_test.goindex 5bea9da0f..c782b5b87 100644--- a/backend/internal/services/image_update_service_test.go+++ b/backend/internal/imageupdate/service_test.go@@ -1,4 +1,4 @@-package services+package imageupdateimport ("context"@@ -13,13 +13,22 @@ import ("testing""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+ref "github.com/distribution/reference"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/types/v2/containerregistry""github.com/getarcaneapp/arcane/types/v2/imageupdate""github.com/libtnb/sqlite"+ dockerauthconfig "github.com/moby/moby/api/pkg/authconfig"dockertypescontainer "github.com/moby/moby/api/types/container"dockertypesimage "github.com/moby/moby/api/types/image"dockerregistry "github.com/moby/moby/api/types/registry"@@ -29,10 +38,81 @@ import ("github.com/samber/mo""github.com/stretchr/testify/assert""github.com/stretchr/testify/require"+ "go.getarcane.app/sys/crypto""go.getarcane.app/updater/labels"+ "go.uber.org/fx/fxtest""gorm.io/gorm")+type fakeRegistryDaemonClient struct {+ registryLoginFn func(context.Context, client.RegistryLoginOptions) (client.RegistryLoginResult, error)+ distributionInspectFn func(context.Context, string, client.DistributionInspectOptions) (client.DistributionInspectResult, error)+}++func (f *fakeRegistryDaemonClient) RegistryLogin(ctx context.Context, options client.RegistryLoginOptions) (client.RegistryLoginResult, error) {+ if f.registryLoginFn == nil {+ return client.RegistryLoginResult{}, nil+ }+ return f.registryLoginFn(ctx, options)+}++func (f *fakeRegistryDaemonClient) DistributionInspect(ctx context.Context, imageRef string, options client.DistributionInspectOptions) (client.DistributionInspectResult, error) {+ if f.distributionInspectFn == nil {+ return client.DistributionInspectResult{}, nil+ }+ return f.distributionInspectFn(ctx, imageRef, options)+}++func newImageUpdateTestDockerClientInternal(t *testing.T, server *httptest.Server) *client.Client {+ t.Helper()+ cli, err := client.New(+ client.WithHost(server.URL),+ client.WithAPIVersion("1.41"),+ client.WithHTTPClient(server.Client()),+ )+ require.NoError(t, err)+ t.Cleanup(func() { _ = cli.Close() })+ return cli+}++func setupImageUpdateRegistryTestDBInternal(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(+ &models.ContainerRegistry{},+ &models.KVEntry{},+ &models.Project{},+ &models.ImageUpdateRecord{},+ &models.Event{},+ ))+ crypto.InitEncryption(&crypto.Config{+ Environment: "test",+ EncryptionKey: "test-encryption-key-for-testing-32bytes-min",+ })+ return &database.DB{DB: db}+}++func createImageUpdateTestPullRegistryInternal(t *testing.T, db *database.DB, url, username, token string) {+ t.Helper()+ encryptedToken, err := crypto.Encrypt(token)+ require.NoError(t, err)+ require.NoError(t, db.Create(&models.ContainerRegistry{+ URL: url,+ Username: username,+ Token: encryptedToken,+ Enabled: true,+ RegistryType: registry.RegistryTypeGeneric,+ }).Error)+}++func decodeImageUpdateRegistryAuthInternal(t *testing.T, encoded string) dockerregistry.AuthConfig {+ t.Helper()+ config, err := dockerauthconfig.Decode(encoded)+ require.NoError(t, err)+ return *config+}+// TestParseImageReference tests the parseImageReference function with various image formats// This is used for digest-based update checkingfunc TestImageUpdateService_ParseImageReference(t *testing.T) {@@ -359,7 +439,7 @@ func newComposeBuildImageUpdateServiceInternal(t *testing.T) (*ImageUpdateServict.Cleanup(dockerServer.Close)var registryCalls atomic.Int32- registryService := NewContainerRegistryService(db, func(context.Context) (RegistryDaemonClient, error) {+ registryService := registry.NewContainerRegistryService(db, func(context.Context) (registry.RegistryDaemonClient, error) {return &fakeRegistryDaemonClient{distributionInspectFn: func(context.Context, string, client.DistributionInspectOptions) (client.DistributionInspectResult, error) {registryCalls.Add(1)@@ -372,8 +452,8 @@ func newComposeBuildImageUpdateServiceInternal(t *testing.T) (*ImageUpdateServic}, nil}, nil)- dockerService := &DockerClientService{client: newTestDockerClient(t, dockerServer)}- eventService := NewEventService(db, nil, nil)+ dockerService := &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, dockerServer)}+ eventService := event.NewEventService(db, nil, nil)return NewImageUpdateService(db, nil, registryService, dockerService, eventService, nil, nil), ®istryCalls}@@ -429,7 +509,7 @@ func TestImageUpdateService_InspectLocalImageSnapshot_NoRepoDigestsRemainsLocal(}))t.Cleanup(server.Close)- svc := &ImageUpdateService{dockerService: &DockerClientService{client: newTestDockerClient(t, server)}}+ svc := &ImageUpdateService{dockerService: &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)}}snapshot, err := svc.inspectLocalImageSnapshotInternal(context.Background(), "local-only:latest", map[string]struct{}{})require.NoError(t, err)assert.True(t, snapshot.IsLocalBuild)@@ -554,7 +634,7 @@ func TestImageUpdateService_GetImageRefByIDInternal_UsesContainerFallback(t *tesdefer server.Close()svc := &ImageUpdateService{- dockerService: &DockerClientService{client: newTestDockerClient(t, server)},+ dockerService: &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)},}imageRef, err := svc.getImageRefByIDInternal(context.Background(), imageID)@@ -575,7 +655,7 @@ func TestImageUpdateService_CheckImageUpdate_SkipsDigestPinnedReferenceInternal(db := setupImageUpdateTestDB(t)pinnedDigest := digest.FromString("pinned-newt").String()imageRef := "ghcr.io/fosrl/newt@" + pinnedDigest- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewImageUpdateService(db, nil, nil, nil, eventService, nil, nil)result, err := svc.CheckImageUpdate(context.Background(), imageRef)@@ -655,7 +735,7 @@ func TestImageUpdateService_CheckMultipleImages_DigestPinnedTagPreservedWhenLocarepository := serverURL.Host + "/valkey/valkey"imageRef := repository + ":9@" + pinnedDigest- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ dockerService := &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)}svc := NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)results, err := svc.CheckMultipleImages(context.Background(), []string{imageRef}, nil)@@ -700,17 +780,16 @@ func TestImageUpdateService_CheckImageUpdate_UsesRegistryFallback(t *testing.T)require.NoError(t, err)imageRef := serverURL.Host + "/team/app:1.2.3"- registryService := NewContainerRegistryService(db, func(context.Context) (RegistryDaemonClient, error) {+ registryService := registry.NewContainerRegistryService(db, func(context.Context) (registry.RegistryDaemonClient, error) {return &fakeRegistryDaemonClient{distributionInspectFn: func(ctx context.Context, imageRef string, options client.DistributionInspectOptions) (client.DistributionInspectResult, error) {return client.DistributionInspectResult{}, errors.New("error response from daemon: Not Found")},}, nil- }, nil)- registryService.distributionHTTPClient = server.Client()+ }, nil, server.Client())- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- eventService := NewEventService(db, nil, nil)+ dockerService := &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)}+ eventService := event.NewEventService(db, nil, nil)svc := NewImageUpdateService(db, nil, registryService, dockerService, eventService, nil, nil)result, err := svc.CheckImageUpdate(context.Background(), imageRef)@@ -740,17 +819,16 @@ func TestImageUpdateService_CheckMultipleImages_UsesRegistryFallback(t *testing.require.NoError(t, err)imageRef := serverURL.Host + "/team/app:1.2.3"- registryService := NewContainerRegistryService(db, func(context.Context) (RegistryDaemonClient, error) {+ registryService := registry.NewContainerRegistryService(db, func(context.Context) (registry.RegistryDaemonClient, error) {return &fakeRegistryDaemonClient{distributionInspectFn: func(ctx context.Context, imageRef string, options client.DistributionInspectOptions) (client.DistributionInspectResult, error) {return client.DistributionInspectResult{}, errors.New("error response from daemon: <html><body><h1>403 Forbidden</h1> Request forbidden by administrative rules. </body></html>")},}, nil- }, nil)- registryService.distributionHTTPClient = server.Client()+ }, nil, server.Client())- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- eventService := NewEventService(db, nil, nil)+ dockerService := &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)}+ eventService := event.NewEventService(db, nil, nil)svc := NewImageUpdateService(db, nil, registryService, dockerService, eventService, nil, nil)results, err := svc.CheckMultipleImages(context.Background(), []string{imageRef}, nil)@@ -774,7 +852,7 @@ func TestImageUpdateService_CheckMultipleImagesCompletesActivityWhenRequestContedb := setupImageUpdateTestDB(t)require.NoError(t, db.AutoMigrate(&models.Activity{}, &models.ActivityMessage{}))- activityService := NewActivityService(db, nil)+ activityService := activity.NewActivityService(db, nil)svc := NewImageUpdateService(db, nil, nil, nil, nil, nil, activityService)for range 5 {@@ -821,11 +899,11 @@ func TestImageUpdateService_CheckMultipleImagesTimesOutStalledRegistryCheckInterdb := setupImageUpdateTestDB(t)require.NoError(t, db.AutoMigrate(&models.Activity{}, &models.ActivityMessage{}))- settingsService := newImageUpdateTestSettingsServiceInternal("1", "30")- activityService := NewActivityService(db, nil)+ settingsService := newImageUpdateTestSettingsServiceInternal(t, "1", "30")+ activityService := activity.NewActivityService(db, nil)dockerServer := newImageUpdateRegistryOnlyServer(t, "team/app:1.2.3", digest.FromString("unused").String())defer dockerServer.Close()- registryService := NewContainerRegistryService(db, func(context.Context) (RegistryDaemonClient, error) {+ registryService := registry.NewContainerRegistryService(db, func(context.Context) (registry.RegistryDaemonClient, error) {return &fakeRegistryDaemonClient{distributionInspectFn: func(ctx context.Context, imageRef string, options client.DistributionInspectOptions) (client.DistributionInspectResult, error) {<-ctx.Done()@@ -837,7 +915,7 @@ func TestImageUpdateService_CheckMultipleImagesTimesOutStalledRegistryCheckInterparentCtx, cancel := context.WithTimeout(context.Background(), 2500*time.Millisecond)defer cancel()- svc := NewImageUpdateService(db, settingsService, registryService, &DockerClientService{client: newTestDockerClient(t, dockerServer)}, nil, nil, activityService)+ svc := NewImageUpdateService(db, settingsService, registryService, &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, dockerServer)}, nil, nil, activityService)start := time.Now()results, err := svc.CheckMultipleImages(parentCtx, []string{"registry.example.com/team/app:1.2.3"}, nil)@@ -862,11 +940,11 @@ func TestImageUpdateService_CheckMultipleImagesPanicMarksActivityFailedInternal(db := setupImageUpdateTestDB(t)require.NoError(t, db.AutoMigrate(&models.Activity{}, &models.ActivityMessage{}))- settingsService := newImageUpdateTestSettingsServiceInternal("30", "30")- activityService := NewActivityService(db, nil)+ settingsService := newImageUpdateTestSettingsServiceInternal(t, "30", "30")+ activityService := activity.NewActivityService(db, nil)dockerServer := newImageUpdateRegistryOnlyServer(t, "team/app:1.2.3", digest.FromString("unused").String())defer dockerServer.Close()- registryService := NewContainerRegistryService(db, func(context.Context) (RegistryDaemonClient, error) {+ registryService := registry.NewContainerRegistryService(db, func(context.Context) (registry.RegistryDaemonClient, error) {return &fakeRegistryDaemonClient{distributionInspectFn: func(ctx context.Context, imageRef string, options client.DistributionInspectOptions) (client.DistributionInspectResult, error) {panic("registry check exploded")@@ -874,7 +952,7 @@ func TestImageUpdateService_CheckMultipleImagesPanicMarksActivityFailedInternal(}, nil}, nil)- svc := NewImageUpdateService(db, settingsService, registryService, &DockerClientService{client: newTestDockerClient(t, dockerServer)}, nil, nil, activityService)+ svc := NewImageUpdateService(db, settingsService, registryService, &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, dockerServer)}, nil, nil, activityService)_, err := svc.CheckMultipleImages(context.Background(), []string{"registry.example.com/team/app:1.2.3"}, nil)require.Error(t, err)@@ -890,9 +968,9 @@ func TestImageUpdateService_CheckMultipleImagesPanicMarksActivityFailedInternal(func TestImageUpdateService_GetAllImageRefsUsesDockerAPITimeoutInternal(t *testing.T) {db := setupImageUpdateTestDB(t)- settingsService := newImageUpdateTestSettingsServiceInternal("30", "1")+ settingsService := newImageUpdateTestSettingsServiceInternal(t, "30", "1")server := newBlockedDockerAPIServerInternal(t, "/images/json")- svc := NewImageUpdateService(db, settingsService, nil, &DockerClientService{client: newTestDockerClient(t, server)}, nil, nil, nil)+ svc := NewImageUpdateService(db, settingsService, nil, &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)}, nil, nil, nil)parentCtx, cancel := context.WithTimeout(context.Background(), 2500*time.Millisecond)defer cancel()@@ -908,9 +986,9 @@ func TestImageUpdateService_GetAllImageRefsUsesDockerAPITimeoutInternal(t *testifunc TestImageUpdateService_InspectLocalImageSnapshotUsesDockerAPITimeoutInternal(t *testing.T) {db := setupImageUpdateTestDB(t)- settingsService := newImageUpdateTestSettingsServiceInternal("30", "1")+ settingsService := newImageUpdateTestSettingsServiceInternal(t, "30", "1")server := newBlockedDockerAPIServerInternal(t, "/images/")- svc := NewImageUpdateService(db, settingsService, nil, &DockerClientService{client: newTestDockerClient(t, server)}, nil, nil, nil)+ svc := NewImageUpdateService(db, settingsService, nil, &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)}, nil, nil, nil)parentCtx, cancel := context.WithTimeout(context.Background(), 2500*time.Millisecond)defer cancel()@@ -925,9 +1003,9 @@ func TestImageUpdateService_InspectLocalImageSnapshotUsesDockerAPITimeoutInterna}func TestImageUpdateService_CheckMultipleImages_UsesDockerHubCredentialsOnFirstAttempt(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupImageUpdateRegistryTestDBInternal(t)require.NoError(t, db.AutoMigrate(&models.ImageUpdateRecord{}, &models.Event{}, &models.Project{}))- createTestPullRegistry(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")+ createImageUpdateTestPullRegistryInternal(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")localDigest := digest.FromString("batchlocal-rate-limit").String()remoteDigest := digest.FromString("batchremote-rate-limit").String()@@ -936,13 +1014,13 @@ func TestImageUpdateService_CheckMultipleImages_UsesDockerHubCredentialsOnFirstAdefer server.Close()var calls int- registryService := NewContainerRegistryService(db, func(context.Context) (RegistryDaemonClient, error) {+ registryService := registry.NewContainerRegistryService(db, func(context.Context) (registry.RegistryDaemonClient, error) {return &fakeRegistryDaemonClient{distributionInspectFn: func(ctx context.Context, imageRef string, options client.DistributionInspectOptions) (client.DistributionInspectResult, error) {calls++assert.Equal(t, "docker.io/library/registry:3", imageRef)- authCfg := decodeRegistryAuth(t, options.EncodedRegistryAuth)+ authCfg := decodeImageUpdateRegistryAuthInternal(t, options.EncodedRegistryAuth)assert.Equal(t, "docker-user", authCfg.Username)assert.Equal(t, "docker-token", authCfg.Password)assert.Equal(t, "https://index.docker.io/v1/", authCfg.ServerAddress)@@ -958,8 +1036,8 @@ func TestImageUpdateService_CheckMultipleImages_UsesDockerHubCredentialsOnFirstA}, nil}, nil)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- eventService := NewEventService(db, nil, nil)+ dockerService := &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)}+ eventService := event.NewEventService(db, nil, nil)svc := NewImageUpdateService(db, nil, registryService, dockerService, eventService, nil, nil)results, err := svc.CheckMultipleImages(context.Background(), []string{"docker.io/library/registry:3"}, nil)@@ -989,17 +1067,16 @@ func TestImageUpdateService_CheckMultipleImages_PersistsRefScopedErrorsWhenLocalrequire.NoError(t, err)imageRef := serverURL.Host + "/library/nginx:alpine"- registryService := NewContainerRegistryService(db, func(context.Context) (RegistryDaemonClient, error) {+ registryService := registry.NewContainerRegistryService(db, func(context.Context) (registry.RegistryDaemonClient, error) {return &fakeRegistryDaemonClient{distributionInspectFn: func(ctx context.Context, imageRef string, options client.DistributionInspectOptions) (client.DistributionInspectResult, error) {return client.DistributionInspectResult{}, errors.New("error response from daemon: Not Found")},}, nil- }, nil)- registryService.distributionHTTPClient = server.Client()+ }, nil, server.Client())- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- eventService := NewEventService(db, nil, nil)+ dockerService := &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)}+ eventService := event.NewEventService(db, nil, nil)svc := NewImageUpdateService(db, nil, registryService, dockerService, eventService, nil, nil)results, err := svc.CheckMultipleImages(context.Background(), []string{imageRef}, nil)@@ -1028,7 +1105,7 @@ func TestImageUpdateService_SaveUpdateResultWithSnapshotInternal_PersistsRegistrrequire.NoError(t, err)imageRef := serverURL.Host + "/library/nginx:alpine"- svc := NewImageUpdateService(db, nil, nil, &DockerClientService{client: newTestDockerClient(t, server)}, nil, nil, nil)+ svc := NewImageUpdateService(db, nil, nil, &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)}, nil, nil, nil)result := &imageupdate.Response{HasUpdate: true,UpdateType: "digest",@@ -1099,7 +1176,7 @@ func TestImageUpdateService_MarkImageRefUpToDateAfterPull_ClearsMatchingRecordsACheckTime: now,}).Error)- svc := NewImageUpdateService(db, nil, nil, &DockerClientService{client: newTestDockerClient(t, server)}, nil, nil, nil)+ svc := NewImageUpdateService(db, nil, nil, &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)}, nil, nil, nil)require.NoError(t, svc.MarkImageRefUpToDateAfterPull(context.Background(), imageRef))@@ -1525,7 +1602,7 @@ func TestImageUpdateService_SendBatchNotifications_DetachesCanceledContext(t *te},}).Error)- notif := NewNotificationService(db, nil, nil, nil)+ notif := notification.NewNotificationService(db, nil, nil, nil)svc := NewImageUpdateService(db, nil, nil, nil, nil, notif, nil)rec := models.ImageUpdateRecord{@@ -1544,7 +1621,7 @@ func TestImageUpdateService_SendBatchNotifications_DetachesCanceledContext(t *tectx, cancel := context.WithCancel(utils.WithAppLifecycleContext(context.Background()))cancel()- svc.sendBatchImageUpdateNotificationsInternal(ctx)+ svc.SendBatchUpdateNotifications(ctx)// The provider being reached, and the record being marked notified, both prove// GetUnnotifiedUpdates + the send + MarkUpdatesAsNotified ran despite the canceled@@ -1564,7 +1641,7 @@ func TestImageUpdateService_SendBatchNotifications_NoEligibleProviders_LeavesUnndb := setupImageUpdateTestDB(t)require.NoError(t, db.AutoMigrate(&models.NotificationSettings{}))- notif := NewNotificationService(db, nil, nil, nil)+ notif := notification.NewNotificationService(db, nil, nil, nil)svc := NewImageUpdateService(db, nil, nil, nil, nil, notif, nil)rec := models.ImageUpdateRecord{@@ -1576,7 +1653,7 @@ func TestImageUpdateService_SendBatchNotifications_NoEligibleProviders_LeavesUnn}require.NoError(t, db.Create(&rec).Error)- svc.sendBatchImageUpdateNotificationsInternal(context.Background())+ svc.SendBatchUpdateNotifications(context.Background())var reloaded models.ImageUpdateRecordrequire.NoError(t, db.First(&reloaded, "id = ?", "sha256:img-no-provider").Error)@@ -1616,7 +1693,7 @@ func TestImageUpdateService_SendBatchNotifications_PartialFailureStillMarksNotif}).Error)}- notif := NewNotificationService(db, nil, nil, nil)+ notif := notification.NewNotificationService(db, nil, nil, nil)svc := NewImageUpdateService(db, nil, nil, nil, nil, notif, nil)rec := models.ImageUpdateRecord{@@ -1628,7 +1705,7 @@ func TestImageUpdateService_SendBatchNotifications_PartialFailureStillMarksNotif}require.NoError(t, db.Create(&rec).Error)- svc.sendBatchImageUpdateNotificationsInternal(context.Background())+ svc.SendBatchUpdateNotifications(context.Background())var reloaded models.ImageUpdateRecordrequire.NoError(t, db.First(&reloaded, "id = ?", "sha256:img-partial").Error)@@ -1732,13 +1809,32 @@ func TestImageUpdateService_ParseAndGroupImages_DedupesNormalizedRefs(t *testingcontainsAll(firstRefSet, "redis:7", "docker.io/library/redis:7")))}-func newImageUpdateTestSettingsServiceInternal(registryTimeout, dockerAPITimeout string) *SettingsService {- settingsService := &SettingsService{}- cfg := DefaultSettingsConfig()- cfg.RegistryTimeout.Value = registryTimeout- cfg.DockerAPITimeout.Value = dockerAPITimeout- settingsService.config.Store(cfg)- return settingsService+func newImageUpdateTestSettingsServiceInternal(t *testing.T, registryTimeout, dockerAPITimeout string) *settings.SettingsService {+ t.Helper()+ t.Setenv("REGISTRY_TIMEOUT", registryTimeout)+ t.Setenv("DOCKER_API_TIMEOUT", dockerAPITimeout)+ ctx := context.Background()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.SettingVariable{}))+ dbWrap := &database.DB{DB: db}+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ writes, err := actors.NewExecutor(t.Context(), runtime, "image-update-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "image-update-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, writes.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ service, err := settings.NewSettingsService(ctx, dbWrap, writes, effects)+ require.NoError(t, err)+ return service}func newBlockedDockerAPIServerInternal(t *testing.T, pathContains string) *httptest.Server {@@ -1831,7 +1927,7 @@ func TestImageUpdateService_GetAllImageRefsHonorsExclusiveContainerOptOutInternanil,nil,nil,- &DockerClientService{client: newTestDockerClient(t, server)},+ &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)},nil,nil,nil,@@ -1877,7 +1973,7 @@ func TestImageUpdateService_GetAllImageRefsAppliesLimitAfterOptOutFilteringInternil,nil,nil,- &DockerClientService{client: newTestDockerClient(t, server)},+ &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)},nil,nil,nil,@@ -1928,7 +2024,7 @@ func TestImageUpdateService_GetAllImageRefsExcludesAliasesOfOptedOutImageInternanil,nil,nil,- &DockerClientService{client: newTestDockerClient(t, server)},+ &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)},nil,nil,nil,@@ -1976,7 +2072,7 @@ func TestImageUpdateService_GetAllImageRefsKeepsImageSharedByEligibleContainerInnil,nil,nil,- &DockerClientService{client: newTestDockerClient(t, server)},+ &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)},nil,nil,nil,@@ -2014,7 +2110,7 @@ func TestImageUpdateService_GetAllImageRefsFallsBackToRefWhenImageIDsDifferInternil,nil,nil,- &DockerClientService{client: newTestDockerClient(t, server)},+ &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)},nil,nil,nil,@@ -2059,7 +2155,7 @@ func TestImageUpdateService_GetAllImageRefsMergesIDAndReferenceEligibilityInternnil,nil,nil,- &DockerClientService{client: newTestDockerClient(t, server)},+ &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)},nil,nil,nil,@@ -2099,7 +2195,7 @@ func TestImageUpdateService_GetAllImageRefsFallsBackWhenContainerDiscoveryFailsInil,nil,nil,- &DockerClientService{client: newTestDockerClient(t, server)},+ &docker.DockerClientService{Client: newImageUpdateTestDockerClientInternal(t, server)},nil,nil,nil,diff --git a/backend/api/handlers/job_schedules.go b/backend/internal/job/handler.gosimilarity index 71%rename from backend/api/handlers/job_schedules.gorename to backend/internal/job/handler.goindex f1e60d81f..eed8850c3 100644--- a/backend/api/handlers/job_schedules.go+++ b/backend/internal/job/handler.go@@ -1,13 +1,14 @@-package handlers+package jobimport ("context""net/http""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/jobschedule")@@ -46,62 +47,61 @@ type RunJobOutput struct {Body jobschedule.JobRunResponse}-//nolint:dupl // Huma operation registrations are intentionally explicit.-func RegisterJobSchedules(api huma.API, jobSvc *services.JobService, envSvc *services.EnvironmentService) {+func RegisterJobSchedules(api huma.API, jobSvc *JobService, envSvc *environment.EnvironmentService) {h := &JobSchedulesHandler{jobService: jobSvc,environmentService: envSvc,}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-job-schedules",Method: http.MethodGet,Path: "/environments/{id}/job-schedules",Summary: "Get job schedules",Description: "Get configured cron schedules for background jobs",Tags: []string{"JobSchedules"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermJobsManage, h.Get)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-job-schedules",Method: http.MethodPut,Path: "/environments/{id}/job-schedules",Summary: "Update job schedules",Description: "Update background job cron schedules and reschedule running jobs",Tags: []string{"JobSchedules"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermJobsManage, h.Update)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-jobs",Method: http.MethodGet,Path: "/environments/{id}/jobs",Summary: "List all background jobs",Description: "Get status, schedule, and metadata for all background jobs",Tags: []string{"JobSchedules"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermJobsManage, h.ListJobs)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "run-job",Method: http.MethodPost,Path: "/environments/{id}/jobs/{jobId}/run",Summary: "Run a job now",Description: "Manually trigger a background job to run immediately",Tags: []string{"JobSchedules"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermJobsManage, h.RunJob)}type JobSchedulesHandler struct {- jobService *services.JobService- environmentService *services.EnvironmentService+ jobService *JobService+ environmentService *environment.EnvironmentService}func (h *JobSchedulesHandler) ListJobs(ctx context.Context, input *ListJobsInput) (*GetJobsOutput, error) {if input.ID != "0" {- jobs, err := proxyRemoteJSONInternal[jobschedule.JobListResponse](ctx, h.environmentService, input.ID, http.MethodGet, "/api/environments/0/jobs", nil)+ jobs, err := handlerutil.ProxyRemoteJSON[jobschedule.JobListResponse](ctx, h.environmentService.ProxyJSONRequest, input.ID, http.MethodGet, "/api/environments/0/jobs", nil)if err != nil {return nil, err}@@ -118,7 +118,7 @@ func (h *JobSchedulesHandler) ListJobs(ctx context.Context, input *ListJobsInputfunc (h *JobSchedulesHandler) RunJob(ctx context.Context, input *RunJobInput) (*RunJobOutput, error) {if input.ID != "0" {- runResp, err := proxyRemoteJSONInternal[jobschedule.JobRunResponse](ctx, h.environmentService, input.ID, http.MethodPost, "/api/environments/0/jobs/"+input.JobID+"/run", nil)+ runResp, err := handlerutil.ProxyRemoteJSON[jobschedule.JobRunResponse](ctx, h.environmentService.ProxyJSONRequest, input.ID, http.MethodPost, "/api/environments/0/jobs/"+input.JobID+"/run", nil)if err != nil {return nil, err}@@ -140,7 +140,7 @@ func (h *JobSchedulesHandler) RunJob(ctx context.Context, input *RunJobInput) (*func (h *JobSchedulesHandler) Get(ctx context.Context, input *GetJobSchedulesInput) (*GetJobSchedulesOutput, error) {if input.ID != "0" {- cfg, err := proxyRemoteJSONInternal[jobschedule.Config](ctx, h.environmentService, input.ID, http.MethodGet, "/api/environments/0/job-schedules", nil)+ cfg, err := handlerutil.ProxyRemoteJSON[jobschedule.Config](ctx, h.environmentService.ProxyJSONRequest, input.ID, http.MethodGet, "/api/environments/0/job-schedules", nil)if err != nil {return nil, err}@@ -153,7 +153,7 @@ func (h *JobSchedulesHandler) Get(ctx context.Context, input *GetJobSchedulesInpfunc (h *JobSchedulesHandler) Update(ctx context.Context, input *UpdateJobSchedulesInput) (*UpdateJobSchedulesOutput, error) {if input.ID != "0" {- apiResp, err := proxyRemoteJSONInternal[base.ApiResponse[jobschedule.Config]](ctx, h.environmentService, input.ID, http.MethodPut, "/api/environments/0/job-schedules", input.Body)+ apiResp, err := handlerutil.ProxyRemoteJSON[base.ApiResponse[jobschedule.Config]](ctx, h.environmentService.ProxyJSONRequest, input.ID, http.MethodPut, "/api/environments/0/job-schedules", input.Body)if err != nil {return nil, err}diff --git a/backend/internal/job/module.go b/backend/internal/job/module.gonew file mode 100644index 000000000..e5120bc8d--- /dev/null+++ b/backend/internal/job/module.go@@ -0,0 +1,45 @@+// Package job owns background-job schedule configuration and its HTTP surface.+package job++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+)++type Dependencies struct {+ DB *database.DB+ Settings *settings.SettingsService+ Config *config.Config+ Environment *environment.EnvironmentService+}++type Module struct {+ service *JobService+ environment *environment.EnvironmentService+}++func New(deps Dependencies) *Module {+ return &Module{+ service: NewJobService(deps.DB, deps.Settings, deps.Config),+ environment: deps.Environment,+ }+}++func (m *Module) Service() *JobService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterJobSchedules(api, nil, nil)+ return+ }+ RegisterJobSchedules(api, m.service, m.environment)+}diff --git a/backend/internal/services/job_service.go b/backend/internal/job/service.gosimilarity index 94%rename from backend/internal/services/job_service.gorename to backend/internal/job/service.goindex 5a3512db7..0d265bc29 100644--- a/backend/internal/services/job_service.go+++ b/backend/internal/job/service.go@@ -1,4 +1,4 @@-package services+package jobimport ("context"@@ -9,6 +9,7 @@ import ("emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""github.com/getarcaneapp/arcane/types/v2/jobschedule""github.com/getarcaneapp/arcane/types/v2/meta"@@ -19,28 +20,28 @@ import (// JobService manages configuration for background job schedules.//// Intervals are persisted in the existing settings table as individual keys.-// After updates, the SettingsService cache is reloaded and active jobs are+// After updates, the settings.SettingsService cache is reloaded and active jobs are// rescheduled through the configured scheduler.//-// NOTE: This is intentionally separate from SettingsService to keep the API+// NOTE: This is intentionally separate from settings.SettingsService to keep the API// surface job-focused and to centralize schedule validation/rescheduling.type JobService struct {db *database.DB- settings *SettingsService+ settings *settings.SettingsServicecfg *config.Configscheduler schedulertypes.JobControllerlifecycleCtx context.Contextlocation *time.Location // Timezone for cron schedule calculations// environment-health is no longer a single scheduler job — it fans out to one- // dynamic job per environment owned by EnvironmentService. These bridge the Jobs+ // dynamic job per environment owned by environment.EnvironmentService. These bridge the Jobs// UI (which addresses jobs by ID) back to that service. Set during bootstrap on// the manager only.OnEnvironmentHealthReschedule func(ctx context.Context)RunEnvironmentHealthNow func(ctx context.Context) error}-func NewJobService(db *database.DB, settings *SettingsService, cfg *config.Config) *JobService {+func NewJobService(db *database.DB, settings *settings.SettingsService, cfg *config.Config) *JobService {return &JobService{db: db,settings: settings,@@ -58,9 +59,9 @@ func (s *JobService) SetScheduler(ctx context.Context, scheduler schedulertypes.}func (s *JobService) GetJobSchedules(ctx context.Context) jobschedule.Config {- defaults := DefaultSettingsConfig()+ defaults := settings.DefaultSettingsConfig()- // Use SettingsService cache for fast reads.+ // Use settings.SettingsService cache for fast reads.return jobschedule.Config{EnvironmentHealthInterval: s.settings.GetStringSetting(ctx, "environmentHealthInterval", defaults.EnvironmentHealthInterval.Value),EventCleanupInterval: s.settings.GetStringSetting(ctx, "eventCleanupInterval", defaults.EventCleanupInterval.Value),@@ -187,7 +188,7 @@ func (s *JobService) jobRescheduleContextInternal(ctx context.Context) context.C// are rescheduled and their runtime schedule verified.func (s *JobService) rescheduleAffectedJobInternal(ctx context.Context, jobID string, jobMeta meta.JobMetadata) error {// environment-health fans out to per-environment dynamic jobs; delegate the- // reschedule to EnvironmentService instead of looking up a single job.+ // reschedule to environment.EnvironmentService instead of looking up a single job.if jobID == "environment-health" {if s.OnEnvironmentHealthReschedule == nil {return errors.New("environment-health rescheduler not initialized")@@ -302,7 +303,7 @@ func (s *JobService) ListJobs(ctx context.Context) (*jobschedule.JobListResponsefunc (s *JobService) RunJobNowInline(ctx context.Context, jobID string) error {// environment-health runs as per-environment dynamic jobs; "run now" fans out- // through EnvironmentService rather than a single registered job.+ // through environment.EnvironmentService rather than a single registered job.if jobID == "environment-health" && s.RunEnvironmentHealthNow != nil {return s.RunEnvironmentHealthNow(context.WithoutCancel(ctx))}@@ -366,7 +367,7 @@ func (s *JobService) getJobScheduleInternal(ctx context.Context, meta meta.JobMereturn ""}- defaultSchedule, _, _, err := DefaultSettingsConfig().FieldByKey(meta.SettingsKey)+ defaultSchedule, _, _, err := settings.DefaultSettingsConfig().FieldByKey(meta.SettingsKey)if err != nil || defaultSchedule == "" {defaultSchedule = "0 0 0 * * *"}diff --git a/backend/internal/services/job_service_test.go b/backend/internal/job/service_test.gosimilarity index 86%rename from backend/internal/services/job_service_test.gorename to backend/internal/job/service_test.goindex 75e432420..3b659e961 100644--- a/backend/internal/services/job_service_test.go+++ b/backend/internal/job/service_test.go@@ -1,4 +1,4 @@-package services+package jobimport ("context"@@ -6,16 +6,49 @@ import ("testing""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/types/v2/jobschedule"schedulertypes "github.com/getarcaneapp/arcane/types/v2/scheduler"+ "github.com/libtnb/sqlite""github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest"+ "gorm.io/gorm")+func setupSettingsTestDBInternal(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.SettingVariable{}))+ return &database.DB{DB: db}+}++func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ executor, err := actors.NewExecutor(t.Context(), runtime, "job-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "job-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, executor.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, executor, effects)+}+func TestJobService_GetJobSchedules_DefaultDockerClientRefreshInterval(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -29,7 +62,7 @@ func TestJobService_GetJobSchedules_DefaultDockerClientRefreshInterval(t *testinfunc TestJobService_ListJobs_AnalyticsHeartbeatIsManagedInternally(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -48,7 +81,7 @@ func TestJobService_ListJobs_AnalyticsHeartbeatIsManagedInternally(t *testing.T)func TestJobService_ListJobs_IncludesDisabledAutoHealJob(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -65,7 +98,7 @@ func TestJobService_ListJobs_IncludesDisabledAutoHealJob(t *testing.T) {func TestJobService_ListJobs_IncludesDockerClientRefreshJob(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -84,7 +117,7 @@ func TestJobService_ListJobs_IncludesDockerClientRefreshJob(t *testing.T) {func TestJobService_ListJobs_UsesRuntimeScheduleAndNextRun(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -109,7 +142,7 @@ func TestJobService_ListJobs_UsesRuntimeScheduleAndNextRun(t *testing.T) {func TestJobService_ListJobs_ImageUpdateWatcherIsContinuousAndRespectsEnabled(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -131,7 +164,7 @@ func TestJobService_ListJobs_ImageUpdateWatcherIsContinuousAndRespectsEnabled(tfunc TestJobService_UpdateJobSchedules_ReschedulesChangedJob(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -150,7 +183,7 @@ func TestJobService_UpdateJobSchedules_ReschedulesChangedJob(t *testing.T) {func TestJobService_UpdateJobSchedules_DeprecatedPollingIntervalDoesNotReschedule(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -169,7 +202,7 @@ func TestJobService_UpdateJobSchedules_DeprecatedPollingIntervalDoesNotReschedulfunc TestJobService_UpdateJobSchedules_UsesLifecycleContextForReschedule(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -196,7 +229,7 @@ func TestJobService_UpdateJobSchedules_UsesLifecycleContextForReschedule(t *testfunc TestJobService_UpdateJobSchedules_RejectsInvalidCronWithoutChangingSetting(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -217,7 +250,7 @@ func TestJobService_UpdateJobSchedules_RejectsInvalidCronWithoutChangingSetting(func TestJobService_UpdateJobSchedules_UnchangedScheduleDoesNotReschedule(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -232,7 +265,7 @@ func TestJobService_UpdateJobSchedules_UnchangedScheduleDoesNotReschedule(t *tesfunc TestJobService_UpdateJobSchedules_RestoresPreviousScheduleWhenRescheduleFails(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -257,7 +290,7 @@ func TestJobService_UpdateJobSchedules_RestoresPreviousScheduleWhenRescheduleFaifunc TestJobService_UpdateJobSchedules_SkipsManagerOnlyJobsInAgentMode(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)@@ -276,7 +309,7 @@ func TestJobService_UpdateJobSchedules_SkipsManagerOnlyJobsInAgentMode(t *testinfunc TestJobService_UpdateJobSchedules_DelegatesEnvironmentHealthReschedule(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSettingsTestDBInternal(t)settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)diff --git a/backend/internal/services/kv_service_prefix_test.go b/backend/internal/kv/prefix_test.gosimilarity index 98%rename from backend/internal/services/kv_service_prefix_test.gorename to backend/internal/kv/prefix_test.goindex 700247097..f1770bfa3 100644--- a/backend/internal/services/kv_service_prefix_test.go+++ b/backend/internal/kv/prefix_test.go@@ -1,4 +1,4 @@-package services+package kvimport ("context"diff --git a/backend/internal/services/kv_service.go b/backend/internal/kv/service.gosimilarity index 99%rename from backend/internal/services/kv_service.gorename to backend/internal/kv/service.goindex ff8c284ae..f9f6e2e3f 100644--- a/backend/internal/services/kv_service.go+++ b/backend/internal/kv/service.go@@ -1,4 +1,4 @@-package services+package kvimport ("context"diff --git a/backend/internal/services/kv_service_test.go b/backend/internal/kv/service_test.gosimilarity index 76%rename from backend/internal/services/kv_service_test.gorename to backend/internal/kv/service_test.goindex 4a18f5c13..cec121f79 100644--- a/backend/internal/services/kv_service_test.go+++ b/backend/internal/kv/service_test.go@@ -1,11 +1,14 @@-package services+package kvimport ("context""testing"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/libtnb/sqlite""github.com/stretchr/testify/require"+ "gorm.io/gorm")func setupKVServiceInternal(t *testing.T) *KVService {@@ -45,3 +48,11 @@ func TestKVService_Set_UpsertsValue(t *testing.T) {require.True(t, updatedOK)require.Equal(t, "2026-03-11T00:00:00Z", updatedValue)}++func setupSettingsTestDB(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.SettingVariable{}))+ return &database.DB{DB: db}+}diff --git a/backend/internal/services/lifecycle_path_diagnostics_other.go b/backend/internal/lifecycle/path_diagnostics_other.gosimilarity index 96%rename from backend/internal/services/lifecycle_path_diagnostics_other.gorename to backend/internal/lifecycle/path_diagnostics_other.goindex 7b3befdd1..0a9bf0491 100644--- a/backend/internal/services/lifecycle_path_diagnostics_other.go+++ b/backend/internal/lifecycle/path_diagnostics_other.go@@ -1,6 +1,6 @@//go:build !unix-package services+package lifecycleimport "fmt"diff --git a/backend/internal/services/lifecycle_path_diagnostics_unix.go b/backend/internal/lifecycle/path_diagnostics_unix.gosimilarity index 99%rename from backend/internal/services/lifecycle_path_diagnostics_unix.gorename to backend/internal/lifecycle/path_diagnostics_unix.goindex 326a57227..a6e4e982e 100644--- a/backend/internal/services/lifecycle_path_diagnostics_unix.go+++ b/backend/internal/lifecycle/path_diagnostics_unix.go@@ -1,6 +1,6 @@//go:build unix-package services+package lifecycleimport ("fmt"diff --git a/backend/internal/services/lifecycle_path_diagnostics_unix_test.go b/backend/internal/lifecycle/path_diagnostics_unix_test.gosimilarity index 99%rename from backend/internal/services/lifecycle_path_diagnostics_unix_test.gorename to backend/internal/lifecycle/path_diagnostics_unix_test.goindex c2b753a34..f83a91f21 100644--- a/backend/internal/services/lifecycle_path_diagnostics_unix_test.go+++ b/backend/internal/lifecycle/path_diagnostics_unix_test.go@@ -1,6 +1,6 @@//go:build unix-package services+package lifecycleimport ("fmt"diff --git a/backend/internal/services/lifecycle_service.go b/backend/internal/lifecycle/service.gosimilarity index 93%rename from backend/internal/services/lifecycle_service.gorename to backend/internal/lifecycle/service.goindex d79b57d95..3ecb09102 100644--- a/backend/internal/services/lifecycle_service.go+++ b/backend/internal/lifecycle/service.go@@ -1,4 +1,4 @@-package services+package lifecycleimport ("bufio"@@ -13,6 +13,8 @@ import ("strings""time"+ lifecycletype "github.com/getarcaneapp/arcane/types/v2/lifecycle"+"emperror.dev/errors"cerrdefs "github.com/containerd/errdefs"@@ -24,7 +26,10 @@ import ("gorm.io/gorm""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"@@ -40,15 +45,15 @@ const (// lifecycleMaxOutputBytes caps the stdout/stderr we retain on the// GitOpsSync row. The full stream is still consumed; the tail is dropped.lifecycleMaxOutputBytes = 16 * 1024- // lifecycleDefaultTimeoutSec is used when a sync has no per-sync timeout+ // DefaultTimeoutSec is used when a sync has no per-sync timeout// configured.- lifecycleDefaultTimeoutSec = 60- // lifecycleDefaultMaxTimeoutSec mirrors the settings default so callers+ DefaultTimeoutSec = 60+ // DefaultMaxTimeoutSec mirrors the settings default so callers// always have a sane upper bound even if settings can't be loaded.- lifecycleDefaultMaxTimeoutSec = 300+ DefaultMaxTimeoutSec = 300// lifecycleStreamDrainTimeout bounds how long we wait for the log copy// goroutine to drain after the container exits, before force-closing.- // Mirrors the value used by VulnerabilityService for the same purpose.+ // Mirrors the value used by vulnerability scanning for the same purpose.lifecycleStreamDrainTimeout = 30 * time.Second)@@ -63,15 +68,6 @@ const (// both in admin-configured Env and in scripts' KEY=VALUE stdout capture.var lifecycleEnvKeyRegex = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)-// lifecycleExtraMount is one admin-configured bind mount for the lifecycle-// runner container. Stored on the GitOpsSync row as docker-CLI-style-// "src:tgt[:ro|:rw]" entries, one per line; never sourced from repo data.-type lifecycleExtraMount struct {- Source string- Target string- Readonly bool-}-// LifecycleService runs pre-deploy lifecycle hooks declared on a project's// GitOps sync. A hook is a script in the synced repo executed in a throwaway// container immediately before the project is deployed, with optional capture@@ -83,15 +79,15 @@ type lifecycleExtraMount struct {// the GitOps sync, not each individual deploy.type LifecycleService struct {db *database.DB- settingsService *SettingsService- eventService *EventService- dockerService *DockerClientService+ settingsService *settings.SettingsService+ eventService *event.EventService+ dockerService *docker.DockerClientService}// NewLifecycleService constructs a LifecycleService wired against shared// infrastructure. The Docker client is obtained lazily on each hook run via// dockerService.GetClient so reconnects are transparent.-func NewLifecycleService(db *database.DB, settingsService *SettingsService, eventService *EventService, dockerService *DockerClientService) *LifecycleService {+func NewLifecycleService(db *database.DB, settingsService *settings.SettingsService, eventService *event.EventService, dockerService *docker.DockerClientService) *LifecycleService {return &LifecycleService{db: db,settingsService: settingsService,@@ -141,11 +137,11 @@ func (s *LifecycleService) executePreDeployInternal(ctx context.Context, projectreturn errors.WrapIf(err, "invalid pre-deploy script path")}- hookEnv, err := parseLifecycleEnvTextInternal(sync.PreDeployEnv)+ hookEnv, err := ParseEnvText(sync.PreDeployEnv)if err != nil {return errors.WrapIf(err, "invalid lifecycle env config")}- extraMounts, err := parseLifecycleExtraMountsTextInternal(sync.PreDeployExtraMounts)+ extraMounts, err := ParseExtraMountsText(sync.PreDeployExtraMounts)if err != nil {return errors.WrapIf(err, "invalid lifecycle extra mounts config")}@@ -212,7 +208,7 @@ func (s *LifecycleService) runScriptInContainerInternal(projectPath string,scriptPath string,hookEnv map[string]string,- extraMounts []lifecycleExtraMount,+ extraMounts []lifecycletype.ExtraMount,networkMode string,timeout time.Duration,) (stdoutContent string, stderrContent string, exitCode int64, err error) {@@ -350,7 +346,7 @@ func (s *LifecycleService) runScriptInContainerInternal(}// ensureRunnerImageInternal makes sure the runner image is available locally,-// pulling it on a miss. Mirrors the pattern used by VulnerabilityService for+// pulling it on a miss. Mirrors the pattern used by vulnerability scanning for// the Trivy scanner image, including reuse of the dockerImagePullTimeout// setting so operators on slow networks can tune it once.func (s *LifecycleService) ensureRunnerImageInternal(ctx context.Context, dockerClient *client.Client, image string) error {@@ -380,9 +376,9 @@ func (s *LifecycleService) ensureRunnerImageInternal(ctx context.Context, dockerfunc (s *LifecycleService) resolveTimeoutInternal(ctx context.Context, perSyncTimeoutSec int) time.Duration {timeoutSec := perSyncTimeoutSecif timeoutSec <= 0 {- timeoutSec = lifecycleDefaultTimeoutSec+ timeoutSec = DefaultTimeoutSec}- maxTimeoutSec := s.settingsService.GetIntSetting(ctx, "lifecycleMaxTimeoutSec", lifecycleDefaultMaxTimeoutSec)+ maxTimeoutSec := s.settingsService.GetIntSetting(ctx, "lifecycleMaxTimeoutSec", DefaultMaxTimeoutSec)if maxTimeoutSec > 0 && timeoutSec > maxTimeoutSec {timeoutSec = maxTimeoutSec}@@ -435,7 +431,7 @@ func (s *LifecycleService) emitLifecycleEventInternal("status": status,}- _, err := s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, err := s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeLifecycleExecute,Severity: severity,Title: title,@@ -525,11 +521,11 @@ func describeScriptStatErrorInternal(err error, projectPath, scriptPath, resolvereturn errors.WrapIff(err, "stat script %q", scriptPath)}-// parseLifecycleEnvTextInternal reads admin-configured env config as the+// ParseEnvText reads admin-configured env config as the// same KEY=VALUE text format used by .env files: one entry per line, blank// and "#"-prefixed lines ignored, keys must match POSIX identifier syntax.// Reuses the strict parser also used for stdout-capture.-func parseLifecycleEnvTextInternal(raw *string) (map[string]string, error) {+func ParseEnvText(raw *string) (map[string]string, error) {if raw == nil || strings.TrimSpace(*raw) == "" {return map[string]string{}, nil}@@ -540,16 +536,16 @@ func parseLifecycleEnvTextInternal(raw *string) (map[string]string, error) {return env, nil}-// parseLifecycleExtraMountsTextInternal reads admin-configured bind mounts+// ParseExtraMountsText reads admin-configured bind mounts// in docker-CLI "src:tgt[:ro|:rw]" form, one per line. Blank and// "#"-prefixed lines are ignored. Both source and target must be absolute// paths; mode defaults to read-write.-func parseLifecycleExtraMountsTextInternal(raw *string) ([]lifecycleExtraMount, error) {+func ParseExtraMountsText(raw *string) ([]lifecycletype.ExtraMount, error) {if raw == nil || strings.TrimSpace(*raw) == "" {return nil, nil}- var mounts []lifecycleExtraMount+ var mounts []lifecycletype.ExtraMountscanner := bufio.NewScanner(strings.NewReader(*raw))scanner.Buffer(make([]byte, 0, 4*1024), 64*1024)@@ -566,7 +562,7 @@ func parseLifecycleExtraMountsTextInternal(raw *string) ([]lifecycleExtraMount,return nil, errors.Errorf("line %d: expected src:tgt[:ro|:rw], got %q", lineNum, line)}- mount := lifecycleExtraMount{Source: parts[0], Target: parts[1]}+ mount := lifecycletype.ExtraMount{Source: parts[0], Target: parts[1]}if len(parts) == 3 {switch parts[2] {case "ro":@@ -642,7 +638,7 @@ func envMapToSliceInternal(env map[string]string) []string {return out}-func buildLifecycleMountsInternal(workspace *mounttypes.Mount, extras []lifecycleExtraMount) []mounttypes.Mount {+func buildLifecycleMountsInternal(workspace *mounttypes.Mount, extras []lifecycletype.ExtraMount) []mounttypes.Mount {mounts := make([]mounttypes.Mount, 0, 1+len(extras))// The project directory is mounted read-write so scripts can write// artifacts the deploy then consumes — e.g. `sops -d secrets.enc.env > .env`.diff --git a/backend/internal/services/lifecycle_service_test.go b/backend/internal/lifecycle/service_test.gosimilarity index 91%rename from backend/internal/services/lifecycle_service_test.gorename to backend/internal/lifecycle/service_test.goindex 1de4a7137..609e7caab 100644--- a/backend/internal/services/lifecycle_service_test.go+++ b/backend/internal/lifecycle/service_test.go@@ -1,4 +1,4 @@-package services+package lifecycleimport ("context"@@ -11,10 +11,14 @@ import ("github.com/libtnb/sqlite""github.com/stretchr/testify/assert""github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest""gorm.io/gorm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings")func setupLifecycleTestDB(t *testing.T) *database.DB {@@ -30,11 +34,25 @@ func setupLifecycleTestDB(t *testing.T) *database.DB {return &database.DB{DB: db}}-func newLifecycleTestService(t *testing.T, db *database.DB) (*LifecycleService, *SettingsService) {+func newLifecycleTestService(t *testing.T, db *database.DB) (*LifecycleService, *settings.SettingsService) {t.Helper()- settings, err := newSettingsServiceForTestInternal(t, context.Background(), db)+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)require.NoError(t, err)- events := NewEventService(db, nil, nil)+ writes, err := actors.NewExecutor(t.Context(), runtime, "lifecycle-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "lifecycle-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, writes.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ settings, err := settings.NewSettingsService(context.Background(), db, writes, effects)+ require.NoError(t, err)+ events := event.NewEventService(db, nil, nil)return NewLifecycleService(db, settings, events, nil), settings}@@ -50,19 +68,19 @@ func writeLifecycleProjectDirWithScript(t *testing.T, scriptRel, scriptBody stri}func TestParseLifecycleEnvText_Empty(t *testing.T) {- got, err := parseLifecycleEnvTextInternal(nil)+ got, err := ParseEnvText(nil)require.NoError(t, err)assert.Empty(t, got)empty := " "- got, err = parseLifecycleEnvTextInternal(&empty)+ got, err = ParseEnvText(&empty)require.NoError(t, err)assert.Empty(t, got)}func TestParseLifecycleEnvText_Valid(t *testing.T) {raw := "FOO=bar\nBAZ_2=qux"- got, err := parseLifecycleEnvTextInternal(&raw)+ got, err := ParseEnvText(&raw)require.NoError(t, err)assert.Equal(t, map[string]string{"FOO": "bar", "BAZ_2": "qux"}, got)}@@ -76,21 +94,21 @@ func TestParseLifecycleEnvText_RejectsInvalidKeys(t *testing.T) {}for name, raw := range cases {t.Run(name, func(t *testing.T) {- _, err := parseLifecycleEnvTextInternal(&raw)+ _, err := ParseEnvText(&raw)require.Error(t, err)})}}func TestParseLifecycleExtraMountsText_Empty(t *testing.T) {- got, err := parseLifecycleExtraMountsTextInternal(nil)+ got, err := ParseExtraMountsText(nil)require.NoError(t, err)assert.Nil(t, got)}func TestParseLifecycleExtraMountsText_Valid(t *testing.T) {raw := "/host/age.key:/age.key:ro\n/host/data:/data"- got, err := parseLifecycleExtraMountsTextInternal(&raw)+ got, err := ParseExtraMountsText(&raw)require.NoError(t, err)require.Len(t, got, 2)assert.Equal(t, "/host/age.key", got[0].Source)@@ -108,7 +126,7 @@ func TestParseLifecycleExtraMountsText_RejectsRelativePaths(t *testing.T) {}for _, raw := range cases {t.Run(raw, func(t *testing.T) {- _, err := parseLifecycleExtraMountsTextInternal(&raw)+ _, err := ParseExtraMountsText(&raw)require.Error(t, err)})}@@ -116,7 +134,7 @@ func TestParseLifecycleExtraMountsText_RejectsRelativePaths(t *testing.T) {func TestParseLifecycleExtraMountsText_RejectsInvalidMode(t *testing.T) {raw := "/host/data:/data:wat"- _, err := parseLifecycleExtraMountsTextInternal(&raw)+ _, err := ParseExtraMountsText(&raw)require.Error(t, err)}diff --git a/backend/api/middleware/activity_batch.go b/backend/internal/middleware/activity_batch.gosimilarity index 87%rename from backend/api/middleware/activity_batch.gorename to backend/internal/middleware/activity_batch.goindex 03243c179..16ea88d20 100644--- a/backend/api/middleware/activity_batch.go+++ b/backend/internal/middleware/activity_batch.go@@ -8,8 +8,8 @@ import (pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils")-// NewActivityBatchID lifts the client-supplied activity batch ID header into-// the request context so activities spawned by one logical bulk action can be+// NewActivityBatchID lifts the client-supplied activity batch ID header into the+// request context so activities spawned by one logical bulk action can be// grouped without threading the ID through every handler. Invalid values are// ignored by utils.WithActivityBatchID.func NewActivityBatchID() func(ctx huma.Context, next func(huma.Context)) {diff --git a/backend/internal/middleware/context.go b/backend/internal/middleware/context.gonew file mode 100644index 000000000..528b2fb5e--- /dev/null+++ b/backend/internal/middleware/context.go@@ -0,0 +1,51 @@+package middleware++import (+ "context"++ "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+)++// ContextKey is a type for context keys used by Huma handlers.+type ContextKey string++const (+ // ContextKeyUserID is the context key for the authenticated user's ID.+ ContextKeyUserID ContextKey = "userID"+ // ContextKeyCurrentSessionID is the context key for the authenticated session ID.+ ContextKeyCurrentSessionID ContextKey = "currentSessionID"+ // ContextKeyUserPermissions is the context key for the caller's resolved+ // PermissionSet, attached by the auth bridge.+ ContextKeyUserPermissions ContextKey = "userPermissions"+ // ContextKeyRemoteAddr is the context key for the request remote address.+ ContextKeyRemoteAddr ContextKey = "remoteAddr"+ // ContextKeyCurrentUser is the Echo context key for the authenticated user.+ ContextKeyCurrentUser ContextKey = "currentUser"+ // ContextKeyAuthMethod is the Echo context key for the authentication method.+ ContextKeyAuthMethod ContextKey = "authMethod"+)++// GetUserIDFromContext retrieves the user ID from the context.+func GetUserIDFromContext(ctx context.Context) (string, bool) {+ userID, ok := ctx.Value(ContextKeyUserID).(string)+ return userID, ok+}++// GetCurrentSessionIDFromContext retrieves the current session ID from the context.+func GetCurrentSessionIDFromContext(ctx context.Context) (string, bool) {+ sessionID, ok := ctx.Value(ContextKeyCurrentSessionID).(string)+ return sessionID, ok+}++// PermissionsFromContext retrieves the caller's resolved PermissionSet.+// Returns nil, false on unauthenticated paths.+func PermissionsFromContext(ctx context.Context) (*authz.PermissionSet, bool) {+ ps, ok := ctx.Value(ContextKeyUserPermissions).(*authz.PermissionSet)+ return ps, ok+}++// GetRemoteAddrFromContext retrieves the request remote address from context.+func GetRemoteAddrFromContext(ctx context.Context) string {+ remoteAddr, _ := ctx.Value(ContextKeyRemoteAddr).(string)+ return remoteAddr+}diff --git a/backend/api/middleware/role.go b/backend/internal/middleware/role.gosimilarity index 78%rename from backend/api/middleware/role.gorename to backend/internal/middleware/role.goindex 8e2826e0a..db0eeac10 100644--- a/backend/api/middleware/role.go+++ b/backend/internal/middleware/role.go@@ -6,7 +6,9 @@ import ("net/http""github.com/danielgtaylor/huma/v2"+ "github.com/labstack/echo/v5"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/backend/v2/pkg/authz")@@ -30,8 +32,30 @@ func RegisterWithPermission[I, O any](api huma.API, op huma.Operation, perm strihuma.Register(api, op, handler)}-// RequirePermission returns a per-operation Huma middleware that rejects-// callers lacking `perm`. For env-scoped permissions, the env ID is extracted+// RequireEchoPermission rejects Echo requests lacking perm for the environment+// in the request path, or globally for organization-level permissions. It must+// run after auth.AuthMiddleware has attached the caller's permission set.+func RequireEchoPermission(perm string) echo.MiddlewareFunc {+ return func(next echo.HandlerFunc) echo.HandlerFunc {+ return func(c *echo.Context) error {+ ps, _ := c.Get(string(ContextKeyUserPermissions)).(*authz.PermissionSet)+ envID := ""+ if authz.IsEnvScoped(perm) {+ envID = authz.EnvIDFromPath(c.Request().URL.Path)+ }+ if !ps.Allows(perm, envID) {+ return c.JSON(http.StatusForbidden, models.APIError{+ Code: "FORBIDDEN",+ Message: "permission denied: " + perm,+ })+ }+ return next(c)+ }+ }+}++// RequirePermission returns per-operation Huma middleware that rejects callers+// lacking `perm`. For env-scoped permissions, the env ID is extracted// from the request path (/environments/{id}/...). For org-level permissions,// the env ID segment, if any, is ignored.//diff --git a/backend/api/middleware/role_test.go b/backend/internal/middleware/role_test.gosimilarity index 100%rename from backend/api/middleware/role_test.gorename to backend/internal/middleware/role_test.godiff --git a/backend/internal/models/mfa_methods.go b/backend/internal/models/mfa_methods.gonew file mode 100644index 000000000..c057fd679--- /dev/null+++ b/backend/internal/models/mfa_methods.go@@ -0,0 +1,8 @@+package models++const (+ // PasskeyMFAMethod and RecoveryCodeMFAMethod name the second factor a login+ // completed with, recorded on the session for auditing.+ PasskeyMFAMethod = "passkey"+ RecoveryCodeMFAMethod = "recovery_code"+)diff --git a/backend/internal/models/system_user.go b/backend/internal/models/system_user.gonew file mode 100644index 000000000..c0e7bc55e--- /dev/null+++ b/backend/internal/models/system_user.go@@ -0,0 +1,8 @@+package models++// SystemUser is the actor recorded for work Arcane performs on its own behalf —+// scheduled jobs, startup reconciliation, GitOps syncs — rather than in response+// to a signed-in user.+var SystemUser = User{+ Username: "System",+}diff --git a/backend/internal/models/user.go b/backend/internal/models/user.goindex 149b70d14..e2bd31e7f 100644--- a/backend/internal/models/user.go+++ b/backend/internal/models/user.go@@ -39,7 +39,7 @@ func (User) TableName() string {// CurrentUserContextKey is the context key holding the authenticated user// model, set via context.WithValue(ctx, models.CurrentUserContextKey{}, user).-// It lives here (rather than in api/middleware) so that services, which cannot+// It lives here (rather than in transport middleware) so that services, which cannot// import the middleware package, can read the requesting user for per-user// preferences.type CurrentUserContextKey struct{}diff --git a/backend/api/handlers/networks.go b/backend/internal/network/handler.gosimilarity index 88%rename from backend/api/handlers/networks.gorename to backend/internal/network/handler.goindex 23fefae27..7a2f89489 100644--- a/backend/api/handlers/networks.go+++ b/backend/internal/network/handler.go@@ -1,4 +1,4 @@-package handlers+package networkimport ("bytes"@@ -10,12 +10,14 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper""github.com/getarcaneapp/arcane/types/v2/base"networktypes "github.com/getarcaneapp/arcane/types/v2/network"@@ -24,9 +26,9 @@ import ()type NetworkHandler struct {- networkService *services.NetworkService- dockerService *services.DockerClientService- activityService *services.ActivityService+ networkService *NetworkService+ dockerService *docker.DockerClientService+ activityService *activity.ActivityServiceappCtx context.Context}@@ -98,80 +100,80 @@ type PruneNetworksOutput struct {}// RegisterNetworks registers network endpoints.-func RegisterNetworks(api huma.API, networkSvc *services.NetworkService, dockerSvc *services.DockerClientService, activitySvc *services.ActivityService, appCtx ActivityAppContext) {+func RegisterNetworks(api huma.API, networkSvc *NetworkService, dockerSvc *docker.DockerClientService, activitySvc *activity.ActivityService, appCtx handlerutil.ActivityAppContext) {h := &NetworkHandler{networkService: networkSvc,dockerService: dockerSvc,activityService: activitySvc,- appCtx: appCtx.contextInternal(),+ appCtx: appCtx.Context(),}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-networks",Method: http.MethodGet,Path: "/environments/{id}/networks",Summary: "List networks",Tags: []string{"Networks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNetworksList, h.ListNetworks)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "network-counts",Method: http.MethodGet,Path: "/environments/{id}/networks/counts",Summary: "Network counts",Tags: []string{"Networks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNetworksList, h.GetNetworkCounts)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-network",Method: http.MethodPost,Path: "/environments/{id}/networks",Summary: "Create network",Tags: []string{"Networks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNetworksCreate, h.CreateNetwork)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-network-topology",Method: http.MethodGet,Path: "/environments/{id}/networks/topology",Summary: "Get network topology",Tags: []string{"Networks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNetworksRead, h.GetNetworkTopology)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-network",Method: http.MethodGet,Path: "/environments/{id}/networks/{networkId}",Summary: "Get network",Tags: []string{"Networks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNetworksRead, h.GetNetwork)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-network",Method: http.MethodDelete,Path: "/environments/{id}/networks/{networkId}",Summary: "Delete network",Tags: []string{"Networks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNetworksDelete, h.DeleteNetwork)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "prune-networks",Method: http.MethodPost,Path: "/environments/{id}/networks/prune",Summary: "Prune networks",Tags: []string{"Networks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNetworksPrune, h.PruneNetworks)}func (h *NetworkHandler) ListNetworks(ctx context.Context, input *ListNetworksInput) (*ListNetworksOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if input.InUse != "" {params.Filters["inUse"] = input.InUse}@@ -186,7 +188,7 @@ func (h *NetworkHandler) ListNetworks(ctx context.Context, input *ListNetworksInSuccess: true,Data: networks,Counts: counts,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -210,7 +212,7 @@ func (h *NetworkHandler) GetNetworkCounts(ctx context.Context, input *GetNetwork}func (h *NetworkHandler) CreateNetwork(ctx context.Context, input *CreateNetworkInput) (*CreateNetworkOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -360,7 +362,7 @@ func (h *NetworkHandler) GetNetworkTopology(ctx context.Context, input *GetNetwo}func (h *NetworkHandler) DeleteNetwork(ctx context.Context, input *DeleteNetworkInput) (*DeleteNetworkOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}diff --git a/backend/internal/services/network_service.go b/backend/internal/network/service.gosimilarity index 96%rename from backend/internal/services/network_service.gorename to backend/internal/network/service.goindex d3d0369ca..9b2f4ea45 100644--- a/backend/internal/services/network_service.go+++ b/backend/internal/network/service.go@@ -1,4 +1,4 @@-package services+package networkimport ("context"@@ -9,6 +9,8 @@ import ("emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"@@ -23,11 +25,11 @@ import (type NetworkService struct {db *database.DB- dockerService *DockerClientService- eventService *EventService+ dockerService *docker.DockerClientService+ eventService *event.EventService}-func NewNetworkService(db *database.DB, dockerService *DockerClientService, eventService *EventService) *NetworkService {+func NewNetworkService(db *database.DB, dockerService *docker.DockerClientService, eventService *event.EventService) *NetworkService {return &NetworkService{db: db,dockerService: dockerService,@@ -258,7 +260,7 @@ func (s *NetworkService) PruneNetworks(ctx context.Context) (*network.PruneRepor"action": "prune","networksDeleted": len(pruneReport.NetworksDeleted),}- if logErr := s.eventService.LogNetworkEvent(ctx, models.EventTypeNetworkDelete, "", "bulk_prune", systemUser.ID, systemUser.Username, "0", metadata); logErr != nil {+ if logErr := s.eventService.LogNetworkEvent(ctx, models.EventTypeNetworkDelete, "", "bulk_prune", models.SystemUser.ID, models.SystemUser.Username, "0", metadata); logErr != nil {slog.WarnContext(ctx, "could not log network prune action", "error", logErr)}@@ -289,7 +291,7 @@ func (s *NetworkService) ListNetworksPaginated(ctx context.Context, params paginconfig := s.buildNetworkPaginationConfig()result := pagination.SearchOrderAndPaginate(items, params, config)counts := s.calculateNetworkUsageCounts(items)- paginationResp := pagination.BuildResponseFromFilterResult(result, params)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)return result.Items, paginationResp, counts, nil}diff --git a/backend/api/handlers/notifications.go b/backend/internal/notification/handler.gosimilarity index 86%rename from backend/api/handlers/notifications.gorename to backend/internal/notification/handler.goindex 0f5ba5187..a5e8ad2a4 100644--- a/backend/api/handlers/notifications.go+++ b/backend/internal/notification/handler.go@@ -1,4 +1,4 @@-package handlers+package notificationimport ("context"@@ -8,17 +8,17 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/notification")type NotificationHandler struct {- notificationService *services.NotificationService+ notificationService *NotificationServiceconfig *config.Config}@@ -76,19 +76,10 @@ type DispatchNotificationOutput struct {Body base.ApiResponse[notification.DispatchResponse]}-var supportedNotificationTestTypes = map[string]struct{}{- "simple": {},- "image-update": {},- "batch-image-update": {},- "vulnerability-found": {},- "prune-report": {},- "auto-heal": {},-}-func normalizeNotificationTestType(testType string) string {normalized := strings.TrimSpace(testType)if normalized == "" {- return "simple"+ return notificationTestTypeSimple}return normalized}@@ -99,55 +90,55 @@ func isSupportedNotificationTestType(testType string) bool {}// RegisterNotifications registers notification endpoints.-func RegisterNotifications(api huma.API, notificationSvc *services.NotificationService, cfg *config.Config) {+func RegisterNotifications(api huma.API, notificationSvc *NotificationService, cfg *config.Config) {h := &NotificationHandler{notificationService: notificationSvc,config: cfg,}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-all-notification-settings",Method: http.MethodGet,Path: "/environments/{id}/notifications/settings",Summary: "Get all notification settings",Tags: []string{"Notifications"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNotificationsManage, h.GetAllNotificationSettings)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-notification-settings",Method: http.MethodGet,Path: "/environments/{id}/notifications/settings/{provider}",Summary: "Get notification settings by provider",Tags: []string{"Notifications"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNotificationsManage, h.GetNotificationSettings)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-or-update-notification-settings",Method: http.MethodPost,Path: "/environments/{id}/notifications/settings",Summary: "Create or update notification settings",Tags: []string{"Notifications"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNotificationsManage, h.CreateOrUpdateNotificationSettings)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-notification-settings",Method: http.MethodDelete,Path: "/environments/{id}/notifications/settings/{provider}",Summary: "Delete notification settings",Tags: []string{"Notifications"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNotificationsManage, h.DeleteNotificationSettings)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "test-notification",Method: http.MethodPost,Path: "/environments/{id}/notifications/test/{provider}",Summary: "Test notification",Tags: []string{"Notifications"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermNotificationsManage, h.TestNotification)// Environment tokens are authenticated by ApiKeyAuth and revalidated by the@@ -184,7 +175,7 @@ func (h *NotificationHandler) GetAllNotificationSettings(ctx context.Context, inID: setting.ID,Provider: notification.Provider(setting.Provider),Enabled: setting.Enabled,- Config: base.JsonObject(services.RedactNotificationConfigCredentials(setting.Provider, setting.Config)),+ Config: base.JsonObject(RedactNotificationConfigCredentials(setting.Provider, setting.Config)),}}@@ -206,7 +197,7 @@ func (h *NotificationHandler) GetNotificationSettings(ctx context.Context, inputID: settings.ID,Provider: notification.Provider(settings.Provider),Enabled: settings.Enabled,- Config: base.JsonObject(services.RedactNotificationConfigCredentials(settings.Provider, settings.Config)),+ Config: base.JsonObject(RedactNotificationConfigCredentials(settings.Provider, settings.Config)),}return &GetNotificationSettingsOutput{Body: response}, nil@@ -239,7 +230,7 @@ func (h *NotificationHandler) CreateOrUpdateNotificationSettings(ctx context.ConID: settings.ID,Provider: notification.Provider(settings.Provider),Enabled: settings.Enabled,- Config: base.JsonObject(services.RedactNotificationConfigCredentials(settings.Provider, settings.Config)),+ Config: base.JsonObject(RedactNotificationConfigCredentials(settings.Provider, settings.Config)),}return &CreateOrUpdateNotificationSettingsOutput{Body: response}, nil@@ -298,10 +289,10 @@ func (h *NotificationHandler) DispatchNotification(ctx context.Context, input *D}dispatchResponse, err := h.notificationService.DispatchNotification(ctx, input.APIKey, input.Body)if err != nil {- if errors.Is(err, services.ErrUnsupportedDispatchKind) {+ if errors.Is(err, ErrUnsupportedDispatchKind) {return nil, huma.Error400BadRequest("unsupported dispatch kind")}- if errors.Is(err, services.ErrUnauthorizedNotificationDispatch) {+ if errors.Is(err, ErrUnauthorizedNotificationDispatch) {return nil, huma.Error401Unauthorized("unauthorized")}return nil, huma.Error500InternalServerError("dispatch failed")diff --git a/backend/internal/notification/module.go b/backend/internal/notification/module.gonew file mode 100644index 000000000..c053c2d08--- /dev/null+++ b/backend/internal/notification/module.go@@ -0,0 +1,32 @@+// Package notification owns notification settings, dispatch, and HTTP routes.+package notification++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/config"+)++type Module struct {+ service *NotificationService+ config *config.Config+}++func New(service *NotificationService, cfg *config.Config) *Module {+ return &Module{service: service, config: cfg}+}++func (m *Module) Service() *NotificationService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterNotifications(api, nil, nil)+ return+ }+ RegisterNotifications(api, m.service, m.config)+}diff --git a/backend/internal/services/notification_service.go b/backend/internal/notification/notification.gosimilarity index 99%rename from backend/internal/services/notification_service.gorename to backend/internal/notification/notification.goindex 5c0e900f8..2e828b719 100644--- a/backend/internal/services/notification_service.go+++ b/backend/internal/notification/notification.go@@ -1,4 +1,4 @@-package services+package notificationimport ("bytes"@@ -20,6 +20,8 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/notifications""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/validation"@@ -59,8 +61,8 @@ const ErrUnsupportedDispatchKind = errors.Sentinel("unsupported notification distype NotificationService struct {db *database.DBconfig *config.Config- environmentSvc *EnvironmentService- eventSvc *EventService+ environmentSvc *environment.EnvironmentService+ eventSvc *event.EventServicehttpClient *http.Client}@@ -82,7 +84,7 @@ func (s *NotificationService) ResolveNotificationTarget(ctx context.Context, envreturn s.resolveNotificationTargetInternal(ctx, environmentID)}-func NewNotificationService(db *database.DB, cfg *config.Config, environmentSvc *EnvironmentService, eventSvc *EventService) *NotificationService {+func NewNotificationService(db *database.DB, cfg *config.Config, environmentSvc *environment.EnvironmentService, eventSvc *event.EventService) *NotificationService {return &NotificationService{db: db,config: cfg,@@ -131,7 +133,7 @@ func (s *NotificationService) resolveNotificationTargetForAccessTokenInternal(ctenv, err := s.environmentSvc.ResolveEnvironmentByAccessToken(ctx, accessToken)if err != nil {- if errors.Is(err, ErrEnvironmentAccessTokenRequired) || errors.Is(err, ErrInvalidEnvironmentAccessToken) {+ if errors.Is(err, environment.ErrEnvironmentAccessTokenRequired) || errors.Is(err, environment.ErrInvalidEnvironmentAccessToken) {return NotificationTarget{}, errors.WithStackIf(ErrUnauthorizedNotificationDispatch)}return NotificationTarget{}, err@@ -478,7 +480,7 @@ func (s *NotificationService) logNotification(ctx context.Context, environmentIDresourceType := "notification"providerName := string(provider)- if _, err := s.eventSvc.CreateEvent(ctx, CreateEventRequest{+ if _, err := s.eventSvc.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeNotificationSend,Severity: severity,Title: title,diff --git a/backend/internal/services/notification_service_test.go b/backend/internal/notification/service_test.gosimilarity index 97%rename from backend/internal/services/notification_service_test.gorename to backend/internal/notification/service_test.goindex 2ca2ae0a2..79f8cd5f9 100644--- a/backend/internal/services/notification_service_test.go+++ b/backend/internal/notification/service_test.go@@ -1,4 +1,4 @@-package services+package notificationimport ("bytes"@@ -21,6 +21,8 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/notifications""github.com/getarcaneapp/arcane/types/v2/imageupdate"@@ -50,17 +52,17 @@ func setupNotificationTestDB(t *testing.T) *database.DB {return &database.DB{DB: db}}-func setupNotificationTestServiceInternal(t *testing.T) (*database.DB, *EnvironmentService, *NotificationService) {+func setupNotificationTestServiceInternal(t *testing.T) (*database.DB, *environment.EnvironmentService, *NotificationService) {t.Helper()db := setupNotificationTestDB(t)- envSvc := NewEnvironmentService(db, nil, nil, nil, nil, nil)+ envSvc := environment.NewEnvironmentService(db, nil, nil, nil, nil, nil)cfg := &config.Config{AppUrl: "http://localhost:3552",}- return db, envSvc, NewNotificationService(db, cfg, envSvc, NewEventService(db, cfg, nil))+ return db, envSvc, NewNotificationService(db, cfg, envSvc, event.NewEventService(db, cfg, nil))}func newNotificationTestUpdateInfoInternal() *imageupdate.Response {@@ -211,7 +213,7 @@ func TestNotificationService_DispatchNotification_LogsManagerDispatchForAgent(tfunc TestNotificationService_SendImageUpdateNotification_AgentModeDispatchesToManager(t *testing.T) {ctx := context.Background()db := setupNotificationTestDB(t)- envSvc := NewEnvironmentService(db, nil, nil, nil, nil, nil)+ envSvc := environment.NewEnvironmentService(db, nil, nil, nil, nil, nil)var calls atomic.Int32var dispatched notificationdto.DispatchRequest@@ -260,7 +262,7 @@ func TestNotificationService_SendImageUpdateNotification_AgentModeDispatchesToMafunc TestNotificationService_SendBatchImageUpdateNotification_AgentModeUsesManagerDeliveredCountInternal(t *testing.T) {ctx := context.Background()db := setupNotificationTestDB(t)- envSvc := NewEnvironmentService(db, nil, nil, nil, nil, nil)+ envSvc := environment.NewEnvironmentService(db, nil, nil, nil, nil, nil)var dispatched notificationdto.DispatchRequestserver := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {@@ -308,7 +310,7 @@ func TestNotificationService_SendBatchImageUpdateNotification_AgentModeUsesManagfunc TestNotificationService_SendImageUpdateNotification_AgentModeRequiresUpdateInfo(t *testing.T) {ctx := context.Background()db := setupNotificationTestDB(t)- envSvc := NewEnvironmentService(db, nil, nil, nil, nil, nil)+ envSvc := environment.NewEnvironmentService(db, nil, nil, nil, nil, nil)svc := NewNotificationService(db, &config.Config{AppUrl: "http://localhost:3552",@@ -323,7 +325,7 @@ func TestNotificationService_SendImageUpdateNotification_AgentModeRequiresUpdatefunc TestNotificationService_SendBatchImageUpdateNotification_AgentModeSkipsNoOpDispatchInternal(t *testing.T) {ctx := context.Background()db := setupNotificationTestDB(t)- envSvc := NewEnvironmentService(db, nil, nil, nil, nil, nil)+ envSvc := environment.NewEnvironmentService(db, nil, nil, nil, nil, nil)var calls atomic.Int32server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {@@ -728,7 +730,7 @@ func TestNotificationService_CreateOrUpdateSettingsKeepsConfigWhenDisabledInternfunc TestNotificationService_NotifyEnabledProvidersInternal_SkipsFiltersAndAggregatesInternal(t *testing.T) {ctx := context.Background()db := setupNotificationTestDB(t)- svc := NewNotificationService(db, &config.Config{}, nil, NewEventService(db, nil, nil))+ svc := NewNotificationService(db, &config.Config{}, nil, event.NewEventService(db, nil, nil))rows := []models.NotificationSettings{{Provider: models.NotificationProviderDiscord, Enabled: false, Config: models.JSON{}},diff --git a/backend/api/handlers/oidc.go b/backend/internal/oidc/handler.gosimilarity index 88%rename from backend/api/handlers/oidc.gorename to backend/internal/oidc/handler.goindex 5bfeffba6..49c80f28a 100644--- a/backend/api/handlers/oidc.go+++ b/backend/internal/oidc/handler.go@@ -1,4 +1,4 @@-package handlers+package oidcimport ("context"@@ -7,16 +7,21 @@ import ("strings""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth"+"emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/passkey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/cookie"- httputils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"- "github.com/getarcaneapp/arcane/types/v2/auth"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"+ authtypes "github.com/getarcaneapp/arcane/types/v2/auth""github.com/getarcaneapp/arcane/types/v2/base"roletypes "github.com/getarcaneapp/arcane/types/v2/role")@@ -24,11 +29,11 @@ import (// OidcHandler handles OIDC authentication endpoints, plus OIDC group → role// mapping management (since mappings only make sense in the OIDC context).type OidcHandler struct {- authService *services.AuthService- passkeyService *services.PasskeyService- oidcService *services.OidcService- roleService *services.RoleService- userService *services.UserService+ authService *auth.AuthService+ passkeyService *passkey.PasskeyService+ oidcService *OidcService+ roleService *role.RoleService+ userService *user.UserServiceconfig *config.Config}@@ -47,30 +52,30 @@ type OidcHeaders struct {type GetOidcStatusInput struct{}type GetOidcStatusOutput struct {- Body auth.OidcStatusInfo+ Body authtypes.OidcStatusInfo}type GetOidcAuthUrlInput struct {OidcHeaders- Body auth.OidcAuthUrlRequest+ Body authtypes.OidcAuthUrlRequest}type GetOidcAuthUrlOutput struct {SetCookie string `header:"Set-Cookie" doc:"OIDC state cookie"`- Body auth.OidcAuthUrlResponse+ Body authtypes.OidcAuthUrlResponse}type HandleOidcCallbackInput struct {OidcHeadersOidcStateCookie string `cookie:"oidc_state" doc:"OIDC state cookie from auth URL request"`- Body auth.OidcCallbackRequest+ Body authtypes.OidcCallbackRequest}type HandleOidcCallbackOutput struct {SetCookie []string `header:"Set-Cookie" doc:"Session and clear state cookies"`- Body auth.OidcCallbackResponse+ Body authtypes.OidcCallbackResponse}type GetOidcConfigInput struct {@@ -78,23 +83,23 @@ type GetOidcConfigInput struct {}type GetOidcConfigOutput struct {- Body auth.OidcConfigResponse+ Body authtypes.OidcConfigResponse}type InitiateDeviceAuthInput struct{}type InitiateDeviceAuthOutput struct {- Body auth.OidcDeviceAuthResponse+ Body authtypes.OidcDeviceAuthResponse}type ExchangeDeviceTokenInput struct {UserAgent string `header:"User-Agent"`- Body auth.OidcDeviceTokenRequest+ Body authtypes.OidcDeviceTokenRequest}type ExchangeDeviceTokenOutput struct {SetCookie []string `header:"Set-Cookie" doc:"Session token cookie"`- Body auth.AuthenticationResponse+ Body authtypes.AuthenticationResponse}// --- OIDC role mapping I/O ---@@ -139,7 +144,7 @@ type DeleteOidcRoleMappingOutput struct {// RegisterOidc registers all OIDC authentication endpoints (plus the OIDC// group → role mapping CRUD) using Huma.-func RegisterOidc(api huma.API, authService *services.AuthService, passkeyService *services.PasskeyService, oidcService *services.OidcService, roleService *services.RoleService, userService *services.UserService, cfg *config.Config) {+func RegisterOidc(api huma.API, authService *auth.AuthService, passkeyService *passkey.PasskeyService, oidcService *OidcService, roleService *role.RoleService, userService *user.UserService, cfg *config.Config) {h := &OidcHandler{authService: authService, passkeyService: passkeyService, oidcService: oidcService, roleService: roleService, userService: userService, config: cfg}huma.Register(api, huma.Operation{@@ -211,8 +216,8 @@ func RegisterOidc(api huma.API, authService *services.AuthService, passkeyServicSummary: "List OIDC group → role mappings",Description: "Returns every mapping. On each OIDC login the user's group claim is matched against ClaimValue and matching rows become source='oidc' role assignments.",Tags: []string{"OIDC"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.ListOidcRoleMappings)huma.Register(api, huma.Operation{@@ -221,8 +226,8 @@ func RegisterOidc(api huma.API, authService *services.AuthService, passkeyServicPath: "/oidc/role-mappings",Summary: "Create an OIDC role mapping",Tags: []string{"OIDC"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.CreateOidcRoleMapping)huma.Register(api, huma.Operation{@@ -231,8 +236,8 @@ func RegisterOidc(api huma.API, authService *services.AuthService, passkeyServicPath: "/oidc/role-mappings/{id}",Summary: "Update an OIDC role mapping",Tags: []string{"OIDC"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.UpdateOidcRoleMapping)huma.Register(api, huma.Operation{@@ -241,8 +246,8 @@ func RegisterOidc(api huma.API, authService *services.AuthService, passkeyServicPath: "/oidc/role-mappings/{id}",Summary: "Delete an OIDC role mapping",Tags: []string{"OIDC"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.DeleteOidcRoleMapping)}@@ -273,10 +278,10 @@ func (h *OidcHandler) GetOidcConfig(ctx context.Context, input *GetOidcConfigInpif h.config != nil {appUrl = h.config.AppUrl}- origin := httputils.GetClientBaseURL(input.Origin, input.XForwardedHost, input.XForwardedProto, input.Host, appUrl)+ origin := httpx.GetClientBaseURL(input.Origin, input.XForwardedHost, input.XForwardedProto, input.Host, appUrl)return &GetOidcConfigOutput{- Body: auth.OidcConfigResponse{+ Body: authtypes.OidcConfigResponse{ClientID: oidcConfig.ClientID,RedirectUri: h.oidcService.GetOidcRedirectURL(origin),IssuerUrl: oidcConfig.IssuerURL,@@ -303,7 +308,7 @@ func (h *OidcHandler) GetOidcAuthUrl(ctx context.Context, input *GetOidcAuthUrlIif h.config != nil {appUrl = h.config.AppUrl}- origin := httputils.GetClientBaseURL(input.Origin, input.XForwardedHost, input.XForwardedProto, input.Host, appUrl)+ origin := httpx.GetClientBaseURL(input.Origin, input.XForwardedHost, input.XForwardedProto, input.Host, appUrl)mobileRedirectURI := input.Body.MobileRedirectUriif mobileRedirectURI != "" {@@ -325,7 +330,7 @@ func (h *OidcHandler) GetOidcAuthUrl(ctx context.Context, input *GetOidcAuthUrlIreturn &GetOidcAuthUrlOutput{SetCookie: stateCookie,- Body: auth.OidcAuthUrlResponse{+ Body: authtypes.OidcAuthUrlResponse{AuthUrl: authUrl,},}, nil@@ -342,7 +347,7 @@ func (h *OidcHandler) HandleOidcCallback(ctx context.Context, input *HandleOidcCif h.config != nil {appUrl = h.config.AppUrl}- origin := httputils.GetClientBaseURL(input.Origin, input.XForwardedHost, input.XForwardedProto, input.Host, appUrl)+ origin := httpx.GetClientBaseURL(input.Origin, input.XForwardedHost, input.XForwardedProto, input.Host, appUrl)mobileRedirectURI := input.Body.MobileRedirectUriif mobileRedirectURI != "" {@@ -361,7 +366,7 @@ func (h *OidcHandler) HandleOidcCallback(ctx context.Context, input *HandleOidcC// Reconcile the provider identity first, then require passkey MFA before// creating a session for accounts that enabled it.- meta := sessionMetaFromContextInternal(ctx, input.UserAgent)+ meta := handlerutil.SessionMetaFromContext(ctx, input.UserAgent)userModel, isNewUser, err := h.authService.PrepareOidcLogin(ctx, *userInfo, tokenResp)if err != nil {return nil, huma.Error500InternalServerError("Authentication failed")@@ -379,9 +384,9 @@ func (h *OidcHandler) HandleOidcCallback(ctx context.Context, input *HandleOidcC}return &HandleOidcCallbackOutput{SetCookie: setCookies,- Body: auth.OidcCallbackResponse{+ Body: authtypes.OidcCallbackResponse{Success: true,- Status: auth.AuthenticationStatusMFARequired,+ Status: authtypes.AuthenticationStatusMFARequired,MFA: challenge,},}, nil@@ -407,9 +412,9 @@ func (h *OidcHandler) HandleOidcCallback(ctx context.Context, input *HandleOidcCreturn &HandleOidcCallbackOutput{SetCookie: setCookies,- Body: auth.OidcCallbackResponse{+ Body: authtypes.OidcCallbackResponse{Success: true,- Status: auth.AuthenticationStatusAuthenticated,+ Status: authtypes.AuthenticationStatusAuthenticated,Token: tokenPair.AccessToken,RefreshToken: tokenPair.RefreshToken,ExpiresAt: &expiresAt,@@ -463,7 +468,7 @@ func (h *OidcHandler) ExchangeDeviceToken(ctx context.Context, input *ExchangeDe}}- meta := sessionMetaFromContextInternal(ctx, input.UserAgent)+ meta := handlerutil.SessionMetaFromContext(ctx, input.UserAgent)userModel, isNewUser, err := h.authService.PrepareOidcLogin(ctx, *userInfo, tokenResp)if err != nil {return nil, huma.Error500InternalServerError("Authentication failed")@@ -496,9 +501,9 @@ func (h *OidcHandler) ExchangeDeviceToken(ctx context.Context, input *ExchangeDereturn &ExchangeDeviceTokenOutput{SetCookie: []string{tokenCookie},- Body: auth.AuthenticationResponse{+ Body: authtypes.AuthenticationResponse{Success: true,- Status: auth.AuthenticationStatusAuthenticated,+ Status: authtypes.AuthenticationStatusAuthenticated,Token: tokenPair.AccessToken,RefreshToken: tokenPair.RefreshToken,ExpiresAt: &expiresAt,diff --git a/backend/internal/services/oidc_service.go b/backend/internal/oidc/service.gosimilarity index 96%rename from backend/internal/services/oidc_service.gorename to backend/internal/oidc/service.goindex f9392c5ce..6517d98a4 100644--- a/backend/internal/services/oidc_service.go+++ b/backend/internal/oidc/service.go@@ -1,4 +1,4 @@-package services+package oidcimport ("context"@@ -14,6 +14,8 @@ import ("sync""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth"+"emperror.dev/errors""github.com/coreos/go-oidc/v3/oidc"@@ -23,14 +25,15 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/jwtclaims"- "github.com/getarcaneapp/arcane/types/v2/auth"+ authtypes "github.com/getarcaneapp/arcane/types/v2/auth")type OidcService struct {- authService *AuthService- settingsService *SettingsService+ authService *auth.AuthService+ settingsService *settings.SettingsServiceconfig *config.ConfighttpClient *http.ClientinsecureHttpClient *http.Client@@ -51,7 +54,7 @@ type OidcState struct {CreatedAt time.Time `json:"created_at"`}-func NewOidcService(authService *AuthService, settingsService *SettingsService, cfg *config.Config, httpClient *http.Client) *OidcService {+func NewOidcService(authService *auth.AuthService, settingsService *settings.SettingsService, cfg *config.Config, httpClient *http.Client) *OidcService {if httpClient == nil {httpClient = http.DefaultClient}@@ -443,7 +446,7 @@ func (s *OidcService) fetchUserInfoClaimsInternal(ctx context.Context, cfg *modereturn claims, nil}-func (s *OidcService) HandleCallback(ctx context.Context, code, state, storedState, origin, mobileRedirectURI string) (*auth.OidcUserInfo, *auth.OidcTokenResponse, error) {+func (s *OidcService) HandleCallback(ctx context.Context, code, state, storedState, origin, mobileRedirectURI string) (*authtypes.OidcUserInfo, *authtypes.OidcTokenResponse, error) {slog.Debug("HandleCallback: processing callback", "code_present", code != "", "state_present", state != "")stateData, err := s.validateStateInternal(state, storedState)@@ -554,7 +557,7 @@ func (s *OidcService) verifyIDTokenInternal(ctx context.Context, provider *oidc.return idToken, rawIDToken, nil}-func (s *OidcService) buildUserInfoInternal(ctx context.Context, provider *oidc.Provider, cfg *models.OidcConfig, token *oauth2.Token, idToken *oidc.IDToken, rawIDToken string) (*auth.OidcUserInfo, *auth.OidcTokenResponse, error) {+func (s *OidcService) buildUserInfoInternal(ctx context.Context, provider *oidc.Provider, cfg *models.OidcConfig, token *oauth2.Token, idToken *oidc.IDToken, rawIDToken string) (*authtypes.OidcUserInfo, *authtypes.OidcTokenResponse, error) {claims, err := s.fetchClaimsInternal(ctx, cfg, provider, token, idToken)if err != nil {slog.Error("HandleCallback: failed to fetch claims", "error", err)@@ -567,7 +570,7 @@ func (s *OidcService) buildUserInfoInternal(ctx context.Context, provider *oidc.return nil, nil, errors.New("missing required 'sub' claim in user info")}- userInfoDto := auth.OidcUserInfo{+ userInfoDto := authtypes.OidcUserInfo{Subject: subject,Name: jwtclaims.GetStringClaim(claims, "name"),Email: jwtclaims.GetStringClaim(claims, "email"),@@ -586,7 +589,7 @@ func (s *OidcService) buildUserInfoInternal(ctx context.Context, provider *oidc.tokenType = "Bearer"}- tokenResp := &auth.OidcTokenResponse{+ tokenResp := &authtypes.OidcTokenResponse{AccessToken: token.AccessToken,TokenType: tokenType,RefreshToken: token.RefreshToken,@@ -618,7 +621,7 @@ func (s *OidcService) decodeStateInternal(encodedState string) (*OidcState, erro}// InitiateDeviceAuth initiates the OIDC device authorization flow.-func (s *OidcService) InitiateDeviceAuth(ctx context.Context) (*auth.OidcDeviceAuthResponse, error) {+func (s *OidcService) InitiateDeviceAuth(ctx context.Context) (*authtypes.OidcDeviceAuthResponse, error) {cfg, err := s.getEffectiveConfigInternal(ctx)if err != nil {slog.Error("InitiateDeviceAuth: failed to get OIDC config", "error", err)@@ -666,7 +669,7 @@ func (s *OidcService) InitiateDeviceAuth(ctx context.Context) (*auth.OidcDeviceAreturn nil, errors.New("invalid expires_in in response")}- response := &auth.OidcDeviceAuthResponse{+ response := &authtypes.OidcDeviceAuthResponse{DeviceCode: deviceCode,UserCode: userCode,VerificationUri: verificationUri,@@ -749,7 +752,7 @@ func (s *OidcService) makeDeviceAuthRequestInternal(ctx context.Context, endpoin}// ExchangeDeviceToken exchanges a device code for tokens.-func (s *OidcService) ExchangeDeviceToken(ctx context.Context, deviceCode string) (*auth.OidcUserInfo, *auth.OidcTokenResponse, error) {+func (s *OidcService) ExchangeDeviceToken(ctx context.Context, deviceCode string) (*authtypes.OidcUserInfo, *authtypes.OidcTokenResponse, error) {cfg, err := s.getEffectiveConfigInternal(ctx)if err != nil {slog.Error("ExchangeDeviceToken: failed to get OIDC config", "error", err)diff --git a/backend/internal/services/oidc_service_test.go b/backend/internal/oidc/service_test.gosimilarity index 62%rename from backend/internal/services/oidc_service_test.gorename to backend/internal/oidc/service_test.goindex 4fb21ced6..a245e2a50 100644--- a/backend/internal/services/oidc_service_test.go+++ b/backend/internal/oidc/service_test.go@@ -1,11 +1,19 @@-package services+package oidcimport ("context""testing"+ "time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/libtnb/sqlite""github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest"+ "gorm.io/gorm")func TestValidateMobileRedirectURI(t *testing.T) {@@ -68,7 +76,7 @@ func TestGetMobileRedirectAllowlistUsesSettings(t *testing.T) {settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err,- "NewSettingsService: %v", err)+ "settings.NewSettingsService: %v", err){err := settingsService.UpdateSetting(ctx, "oidcMobileRedirectUris", "arcane-mobile://db-callback")@@ -96,3 +104,32 @@ func TestGetMobileRedirectAllowlistUsesSettings(t *testing.T) {}}++// Test fixtures shared by this package's tests.++func setupSettingsTestDB(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.SettingVariable{}))+ return &database.DB{DB: db}+}++func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ executor, err := actors.NewExecutor(t.Context(), runtime, "settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, executor.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, executor, effects)+}diff --git a/backend/api/handlers/passkey.go b/backend/internal/passkey/handler.gosimilarity index 85%rename from backend/api/handlers/passkey.gorename to backend/internal/passkey/handler.goindex d9dafb1ab..830a1649f 100644--- a/backend/api/handlers/passkey.go+++ b/backend/internal/passkey/handler.go@@ -1,4 +1,4 @@-package handlers+package passkeyimport ("context"@@ -7,20 +7,23 @@ import ("strings""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/auth"+"emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/cookie"- "github.com/getarcaneapp/arcane/types/v2/auth"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+ authtypes "github.com/getarcaneapp/arcane/types/v2/auth""github.com/getarcaneapp/arcane/types/v2/base")type PasskeyHandler struct {- passkeyService *services.PasskeyService- authService *services.AuthService- userService *services.UserService+ passkeyService *PasskeyService+ authService *auth.AuthService+ userService *user.UserService}type passkeyCredentialBody struct {@@ -78,7 +81,7 @@ type PasskeyLoginFinishInput struct {type PasskeyLoginFinishOutput struct {SetCookie []string `header:"Set-Cookie" doc:"Session cookie"`- Body base.ApiResponse[auth.AuthenticationResponse]+ Body base.ApiResponse[authtypes.AuthenticationResponse]}type MobilePasskeyFinishInput struct {@@ -86,7 +89,7 @@ type MobilePasskeyFinishInput struct {}type MobilePasskeyFinishOutput struct {- Body base.ApiResponse[auth.MobilePasskeyCompletion]+ Body base.ApiResponse[authtypes.MobilePasskeyCompletion]}type MobilePasskeyExchangeInput struct {@@ -96,7 +99,7 @@ type MobilePasskeyExchangeInput struct {type MobilePasskeyExchangeOutput struct {SetCookie []string `header:"Set-Cookie" doc:"Session cookie"`- Body base.ApiResponse[auth.AuthenticationResponse]+ Body base.ApiResponse[authtypes.AuthenticationResponse]}type PasskeyMFAStartInput struct {@@ -104,7 +107,7 @@ type PasskeyMFAStartInput struct {}type PasskeyMFAStartOutput struct {- Body base.ApiResponse[auth.MFAChallenge]+ Body base.ApiResponse[authtypes.MFAChallenge]}type PasskeyMFAFinishInput struct {@@ -113,7 +116,7 @@ type PasskeyMFAFinishInput struct {type PasskeyMFAFinishOutput struct {SetCookie []string `header:"Set-Cookie" doc:"Session cookie"`- Body base.ApiResponse[auth.AuthenticationResponse]+ Body base.ApiResponse[authtypes.AuthenticationResponse]}type PasskeyRecoveryInput struct {@@ -122,7 +125,7 @@ type PasskeyRecoveryInput struct {type PasskeyRecoveryOutput struct {SetCookie []string `header:"Set-Cookie" doc:"Session cookie"`- Body base.ApiResponse[auth.AuthenticationResponse]+ Body base.ApiResponse[authtypes.AuthenticationResponse]}type passkeyBeginResponse struct {@@ -133,11 +136,11 @@ type passkeyBeginResponse struct {}type ListMyPasskeysOutput struct {- Body base.ApiResponse[[]services.PasskeySummary]+ Body base.ApiResponse[[]PasskeySummary]}type GetPasskeyCapabilitiesOutput struct {- Body base.ApiResponse[services.PasskeyCapabilities]+ Body base.ApiResponse[PasskeyCapabilities]}type BeginPasskeyRegistrationInput struct {@@ -154,7 +157,7 @@ type FinishPasskeyRegistrationInput struct {}type FinishPasskeyRegistrationOutput struct {- Body base.ApiResponse[services.PasskeySummary]+ Body base.ApiResponse[PasskeySummary]}type RenamePasskeyInput struct {@@ -164,7 +167,7 @@ type RenamePasskeyInput struct {}type RenamePasskeyOutput struct {- Body base.ApiResponse[services.PasskeySummary]+ Body base.ApiResponse[PasskeySummary]}type DeletePasskeyInput struct {@@ -187,7 +190,7 @@ type FinishStepUpInput struct {}type FinishStepUpOutput struct {- Body base.ApiResponse[services.StepUpGrant]+ Body base.ApiResponse[StepUpGrant]}type PasswordStepUpInput struct {@@ -195,11 +198,11 @@ type PasswordStepUpInput struct {}type PasswordStepUpOutput struct {- Body base.ApiResponse[services.StepUpGrant]+ Body base.ApiResponse[StepUpGrant]}type GetMFAStatusOutput struct {- Body base.ApiResponse[services.MFAStatus]+ Body base.ApiResponse[MFAStatus]}type MFASettingsInput struct {@@ -214,7 +217,7 @@ type MFARecoveryCodesOutput struct {Body base.ApiResponse[RecoveryCodesResponse]}-func RegisterPasskeys(api huma.API, passkeyService *services.PasskeyService, authService *services.AuthService, userService *services.UserService) {+func RegisterPasskeys(api huma.API, passkeyService *PasskeyService, authService *auth.AuthService, userService *user.UserService) {h := &PasskeyHandler{passkeyService: passkeyService, authService: authService, userService: userService}public := []map[string][]string{}interactive := []map[string][]string{{"BearerAuth": {}}}@@ -408,7 +411,7 @@ func (h *PasskeyHandler) FinishPasskeyLogin(ctx context.Context, input *PasskeyLif err != nil {return nil, passkeyHTTPErrorInternal(err)}- meta := auth.SessionMeta{UserAgent: input.UserAgent, IPAddress: humamw.GetRemoteAddrFromContext(ctx)}+ meta := authtypes.SessionMeta{UserAgent: input.UserAgent, IPAddress: middleware.GetRemoteAddrFromContext(ctx)}tokenPair, err := h.authService.CompleteLogin(ctx, userModel, meta, models.UserSessionSourcePasskey, "")if err != nil {return nil, huma.Error500InternalServerError("authentication failed")@@ -419,7 +422,7 @@ func (h *PasskeyHandler) FinishPasskeyLogin(ctx context.Context, input *PasskeyL}return &PasskeyLoginFinishOutput{SetCookie: tokenCookieInternal(ctx, tokenPair),- Body: base.ApiResponse[auth.AuthenticationResponse]{Success: true, Data: *response},+ Body: base.ApiResponse[authtypes.AuthenticationResponse]{Success: true, Data: *response},}, nil}@@ -432,7 +435,7 @@ func (h *PasskeyHandler) FinishMobilePasskeyLogin(ctx context.Context, input *Moif err != nil {return nil, passkeyHTTPErrorInternal(err)}- return &MobilePasskeyFinishOutput{Body: base.ApiResponse[auth.MobilePasskeyCompletion]{Success: true, Data: *completion}}, nil+ return &MobilePasskeyFinishOutput{Body: base.ApiResponse[authtypes.MobilePasskeyCompletion]{Success: true, Data: *completion}}, nil}func (h *PasskeyHandler) ExchangeMobilePasskeyLogin(ctx context.Context, input *MobilePasskeyExchangeInput) (*MobilePasskeyExchangeOutput, error) {@@ -440,7 +443,7 @@ func (h *PasskeyHandler) ExchangeMobilePasskeyLogin(ctx context.Context, input *if err != nil {return nil, passkeyHTTPErrorInternal(err)}- meta := auth.SessionMeta{UserAgent: input.UserAgent, IPAddress: humamw.GetRemoteAddrFromContext(ctx)}+ meta := authtypes.SessionMeta{UserAgent: input.UserAgent, IPAddress: middleware.GetRemoteAddrFromContext(ctx)}tokenPair, err := h.authService.CompleteLogin(ctx, userModel, meta, models.UserSessionSourcePasskey, "")if err != nil {return nil, huma.Error500InternalServerError("authentication failed")@@ -451,7 +454,7 @@ func (h *PasskeyHandler) ExchangeMobilePasskeyLogin(ctx context.Context, input *}return &MobilePasskeyExchangeOutput{SetCookie: tokenCookieInternal(ctx, tokenPair),- Body: base.ApiResponse[auth.AuthenticationResponse]{Success: true, Data: *response},+ Body: base.ApiResponse[authtypes.AuthenticationResponse]{Success: true, Data: *response},}, nil}@@ -460,7 +463,7 @@ func (h *PasskeyHandler) BeginMFA(ctx context.Context, input *PasskeyMFAStartInpif err != nil {return nil, passkeyHTTPErrorInternal(err)}- return &PasskeyMFAStartOutput{Body: base.ApiResponse[auth.MFAChallenge]{Success: true, Data: *challenge}}, nil+ return &PasskeyMFAStartOutput{Body: base.ApiResponse[authtypes.MFAChallenge]{Success: true, Data: *challenge}}, nil}func (h *PasskeyHandler) FinishMFA(ctx context.Context, input *PasskeyMFAFinishInput) (*PasskeyMFAFinishOutput, error) {@@ -482,7 +485,7 @@ func (h *PasskeyHandler) FinishMFA(ctx context.Context, input *PasskeyMFAFinishI}return &PasskeyMFAFinishOutput{SetCookie: tokenCookieInternal(ctx, tokenPair),- Body: base.ApiResponse[auth.AuthenticationResponse]{Success: true, Data: *response},+ Body: base.ApiResponse[authtypes.AuthenticationResponse]{Success: true, Data: *response},}, nil}@@ -501,7 +504,7 @@ func (h *PasskeyHandler) UseRecoveryCode(ctx context.Context, input *PasskeyReco}return &PasskeyRecoveryOutput{SetCookie: tokenCookieInternal(ctx, tokenPair),- Body: base.ApiResponse[auth.AuthenticationResponse]{Success: true, Data: *response},+ Body: base.ApiResponse[authtypes.AuthenticationResponse]{Success: true, Data: *response},}, nil}@@ -515,7 +518,7 @@ func (h *PasskeyHandler) ListMyPasskeys(ctx context.Context, _ *struct{}) (*Listif err != nil {return nil, passkeyHTTPErrorInternal(err)}- return &ListMyPasskeysOutput{Body: base.ApiResponse[[]services.PasskeySummary]{Success: true, Data: rows}}, nil+ return &ListMyPasskeysOutput{Body: base.ApiResponse[[]PasskeySummary]{Success: true, Data: rows}}, nil}func (h *PasskeyHandler) GetCapabilities(ctx context.Context, _ *struct{}) (*GetPasskeyCapabilitiesOutput, error) {@@ -531,7 +534,7 @@ func (h *PasskeyHandler) GetCapabilities(ctx context.Context, _ *struct{}) (*Getif err != nil {return nil, passkeyHTTPErrorInternal(err)}- return &GetPasskeyCapabilitiesOutput{Body: base.ApiResponse[services.PasskeyCapabilities]{Success: true, Data: *capabilities}}, nil+ return &GetPasskeyCapabilitiesOutput{Body: base.ApiResponse[PasskeyCapabilities]{Success: true, Data: *capabilities}}, nil}func (h *PasskeyHandler) BeginRegistration(ctx context.Context, input *BeginPasskeyRegistrationInput) (*BeginPasskeyRegistrationOutput, error) {@@ -559,7 +562,7 @@ func (h *PasskeyHandler) FinishRegistration(ctx context.Context, input *FinishPaif err != nil {return nil, passkeyHTTPErrorInternal(err)}- return &FinishPasskeyRegistrationOutput{Body: base.ApiResponse[services.PasskeySummary]{Success: true, Data: *row}}, nil+ return &FinishPasskeyRegistrationOutput{Body: base.ApiResponse[PasskeySummary]{Success: true, Data: *row}}, nil}func (h *PasskeyHandler) RenamePasskey(ctx context.Context, input *RenamePasskeyInput) (*RenamePasskeyOutput, error) {@@ -571,7 +574,7 @@ func (h *PasskeyHandler) RenamePasskey(ctx context.Context, input *RenamePasskeyif err != nil {return nil, passkeyHTTPErrorInternal(err)}- return &RenamePasskeyOutput{Body: base.ApiResponse[services.PasskeySummary]{Success: true, Data: *row}}, nil+ return &RenamePasskeyOutput{Body: base.ApiResponse[PasskeySummary]{Success: true, Data: *row}}, nil}func (h *PasskeyHandler) DeletePasskey(ctx context.Context, input *DeletePasskeyInput) (*DeletePasskeyOutput, error) {@@ -594,7 +597,7 @@ func (h *PasskeyHandler) BeginStepUp(ctx context.Context, _ *BeginStepUpInput) (if err != nil {return nil, err}- challenge, err := h.passkeyService.BeginStepUp(ctx, userModel.ID, sessionID, auth.SessionMeta{IPAddress: humamw.GetRemoteAddrFromContext(ctx)})+ challenge, err := h.passkeyService.BeginStepUp(ctx, userModel.ID, sessionID, authtypes.SessionMeta{IPAddress: middleware.GetRemoteAddrFromContext(ctx)})if err != nil {return nil, passkeyHTTPErrorInternal(err)}@@ -614,7 +617,7 @@ func (h *PasskeyHandler) FinishStepUp(ctx context.Context, input *FinishStepUpInif err != nil {return nil, passkeyHTTPErrorInternal(err)}- return &FinishStepUpOutput{Body: base.ApiResponse[services.StepUpGrant]{Success: true, Data: *grant}}, nil+ return &FinishStepUpOutput{Body: base.ApiResponse[StepUpGrant]{Success: true, Data: *grant}}, nil}func (h *PasskeyHandler) PasswordStepUp(ctx context.Context, input *PasswordStepUpInput) (*PasswordStepUpOutput, error) {@@ -632,7 +635,7 @@ func (h *PasskeyHandler) PasswordStepUp(ctx context.Context, input *PasswordStepif err != nil {return nil, passkeyHTTPErrorInternal(err)}- return &PasswordStepUpOutput{Body: base.ApiResponse[services.StepUpGrant]{Success: true, Data: *grant}}, nil+ return &PasswordStepUpOutput{Body: base.ApiResponse[StepUpGrant]{Success: true, Data: *grant}}, nil}func (h *PasskeyHandler) GetMFAStatus(ctx context.Context, _ *struct{}) (*GetMFAStatusOutput, error) {@@ -644,7 +647,7 @@ func (h *PasskeyHandler) GetMFAStatus(ctx context.Context, _ *struct{}) (*GetMFAif err != nil {return nil, passkeyHTTPErrorInternal(err)}- return &GetMFAStatusOutput{Body: base.ApiResponse[services.MFAStatus]{Success: true, Data: *status}}, nil+ return &GetMFAStatusOutput{Body: base.ApiResponse[MFAStatus]{Success: true, Data: *status}}, nil}func (h *PasskeyHandler) EnableMFA(ctx context.Context, input *MFASettingsInput) (*MFARecoveryCodesOutput, error) {@@ -682,7 +685,7 @@ func (h *PasskeyHandler) RegenerateRecoveryCodes(ctx context.Context, input *MFAreturn &MFARecoveryCodesOutput{Body: base.ApiResponse[RecoveryCodesResponse]{Success: true, Data: RecoveryCodesResponse{Codes: codes}}}, nil}-func (h *PasskeyHandler) authenticationResponseInternal(ctx context.Context, userModel *models.User, tokenPair *services.TokenPair) (*auth.AuthenticationResponse, error) {+func (h *PasskeyHandler) authenticationResponseInternal(ctx context.Context, userModel *models.User, tokenPair *auth.TokenPair) (*authtypes.AuthenticationResponse, error) {if userModel == nil || tokenPair == nil {return nil, huma.Error500InternalServerError("authentication failed")}@@ -691,9 +694,9 @@ func (h *PasskeyHandler) authenticationResponseInternal(ctx context.Context, usereturn nil, huma.Error500InternalServerError("failed to map user")}expiresAt := tokenPair.ExpiresAt- return &auth.AuthenticationResponse{+ return &authtypes.AuthenticationResponse{Success: true,- Status: auth.AuthenticationStatusAuthenticated,+ Status: authtypes.AuthenticationStatusAuthenticated,Token: tokenPair.AccessToken,RefreshToken: tokenPair.RefreshToken,ExpiresAt: &expiresAt,@@ -702,18 +705,18 @@ func (h *PasskeyHandler) authenticationResponseInternal(ctx context.Context, use}func requireInteractiveSessionInternal(ctx context.Context) (*models.User, string, error) {- userModel, err := requireUserInternal(ctx)+ userModel, err := handlerutil.RequireUser(ctx)if err != nil {return nil, "", err}- sessionID, ok := humamw.GetCurrentSessionIDFromContext(ctx)+ sessionID, ok := middleware.GetCurrentSessionIDFromContext(ctx)if !ok || strings.TrimSpace(sessionID) == "" {return nil, "", huma.Error403Forbidden("passkey management requires an interactive session")}return userModel, sessionID, nil}-func passkeyBeginResponseFromChallengeInternal(challenge *services.PasskeyChallenge) passkeyBeginResponse {+func passkeyBeginResponseFromChallengeInternal(challenge *PasskeyChallenge) passkeyBeginResponse {return passkeyBeginResponse{CeremonyID: challenge.CeremonyID,TransactionID: challenge.TransactionID,@@ -729,36 +732,36 @@ func marshalCredentialInternal(credential map[string]any) ([]byte, error) {return stdjson.Marshal(credential)}-func tokenCookieInternal(ctx context.Context, tokenPair *services.TokenPair) []string {+func tokenCookieInternal(ctx context.Context, tokenPair *auth.TokenPair) []string {maxAge := max(int(time.Until(tokenPair.ExpiresAt).Seconds()), 0) + 60return []string{cookie.BuildTokenCookieStringFor(maxAge, tokenPair.AccessToken, cookie.SecureCookieFromContext(ctx))}}func passkeyHTTPErrorInternal(err error) error {switch {- case errors.Is(err, services.ErrPasskeyNotFound):+ case errors.Is(err, ErrPasskeyNotFound):return huma.Error404NotFound("passkey not found")- case errors.Is(err, services.ErrPasskeyExists):+ case errors.Is(err, ErrPasskeyExists):return huma.Error409Conflict("passkey is already registered")- case errors.Is(err, services.ErrPasskeyMFAEnabled):+ case errors.Is(err, ErrPasskeyMFAEnabled):return huma.Error409Conflict("disable passkey MFA before deleting the last passkey")- case errors.Is(err, services.ErrPasskeyLastCredential):+ case errors.Is(err, ErrPasskeyLastCredential):return huma.Error409Conflict("another usable authentication method is required")- case errors.Is(err, services.ErrPasskeyStepUpRequired):+ case errors.Is(err, ErrPasskeyStepUpRequired):return huma.Error401Unauthorized("fresh step-up authentication is required")- case errors.Is(err, services.ErrPasskeyNoCredential):+ case errors.Is(err, ErrPasskeyNoCredential):return huma.Error409Conflict("no passkeys are registered")- case errors.Is(err, services.ErrPasskeyServiceUnavailable):+ case errors.Is(err, ErrPasskeyServiceUnavailable):return huma.Error500InternalServerError("passkey service is unavailable")- case errors.Is(err, services.ErrPasskeyRecoveryCode):+ case errors.Is(err, ErrPasskeyRecoveryCode):return huma.Error401Unauthorized("invalid recovery code")- case errors.Is(err, services.ErrPasskeyName):+ case errors.Is(err, ErrPasskeyName):return huma.Error400BadRequest("invalid passkey name")- case errors.Is(err, services.ErrPasskeyMFAAlreadyEnabled):+ case errors.Is(err, ErrPasskeyMFAAlreadyEnabled):return huma.Error409Conflict("passkey MFA is already enabled")- case errors.Is(err, services.ErrPasskeyMFANotEnabled):+ case errors.Is(err, ErrPasskeyMFANotEnabled):return huma.Error409Conflict("passkey MFA is not enabled")- case errors.Is(err, services.ErrPasskeyCeremony), errors.Is(err, services.ErrPasskeyTransaction), errors.Is(err, services.ErrPasskeyResponse):+ case errors.Is(err, ErrPasskeyCeremony), errors.Is(err, ErrPasskeyTransaction), errors.Is(err, ErrPasskeyResponse):return huma.Error400BadRequest("invalid or expired passkey authentication attempt")default:return huma.Error500InternalServerError("passkey authentication failed")diff --git a/backend/internal/services/passkey_service.go b/backend/internal/passkey/service.gosimilarity index 99%rename from backend/internal/services/passkey_service.gorename to backend/internal/passkey/service.goindex ef4de0541..e7fb532bb 100644--- a/backend/internal/services/passkey_service.go+++ b/backend/internal/passkey/service.go@@ -1,4 +1,4 @@-package services+package passkeyimport ("context"@@ -30,9 +30,6 @@ import ()const (- PasskeyMFAMethod = "passkey"- RecoveryCodeMFAMethod = "recovery_code"-passkeyCeremonyPurposeLogin = "login"passkeyCeremonyPurposeMFA = "mfa"passkeyCeremonyPurposeRegistration = "registration"@@ -142,7 +139,7 @@ type passkeyService struct {// PasskeyService owns WebAuthn ceremonies, passkey persistence, MFA// transactions, recovery codes, and step-up grants. It intentionally does not-// issue JWTs; AuthService remains the single token/session issuer.+// issue JWTs; auth.AuthService remains the single token/session issuer.type PasskeyService = passkeyServicefunc NewPasskeyService(db *database.DB, cfg *config.Config) *PasskeyService {@@ -385,7 +382,7 @@ func (s *passkeyService) beginMFAForTransactionInternal(ctx context.Context, trareturn &auth.MFAChallenge{TransactionID: transaction.ID,- Method: PasskeyMFAMethod,+ Method: models.PasskeyMFAMethod,Options: assertion.Response,ExpiresAt: ceremony.ExpiresAt,}, nil@@ -635,7 +632,7 @@ func (s *passkeyService) FinishMFA(ctx context.Context, transactionID string, panow := time.Now()return &AuthenticationCompletion{User: user,- Meta: transactionSessionMetaInternal(transaction, PasskeyMFAMethod, &now),+ Meta: transactionSessionMetaInternal(transaction, models.PasskeyMFAMethod, &now),Source: transaction.Source,}, nil}@@ -705,7 +702,7 @@ func (s *passkeyService) FinishRecoveryCode(ctx context.Context, transactionID,now := time.Now()return &AuthenticationCompletion{User: user,- Meta: transactionSessionMetaInternal(&transaction, RecoveryCodeMFAMethod, &now),+ Meta: transactionSessionMetaInternal(&transaction, models.RecoveryCodeMFAMethod, &now),Source: transaction.Source,}, nil}diff --git a/backend/internal/services/passkey_service_test.go b/backend/internal/passkey/service_test.gosimilarity index 97%rename from backend/internal/services/passkey_service_test.gorename to backend/internal/passkey/service_test.goindex 2ecf97985..39c961faa 100644--- a/backend/internal/services/passkey_service_test.go+++ b/backend/internal/passkey/service_test.go@@ -1,4 +1,4 @@-package services+package passkeyimport ("context"@@ -13,6 +13,7 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session""github.com/getarcaneapp/arcane/types/v2/auth")@@ -101,7 +102,7 @@ func TestPasskeyService_FirstEnrollmentUsesActiveSession(t *testing.T) {service := newPasskeyServiceForTest(t, db)ctx := context.Background()user := createPasskeyTestUser(t, db, "first-enrollment-user")- sessionService := NewSessionService(db)+ sessionService := session.NewSessionService(db)session, _, err := sessionService.CreateSession(ctx, user.ID, time.Now().Add(time.Hour), auth.SessionMeta{})require.NoError(t, err)@@ -142,7 +143,7 @@ func TestPasskeyService_PasswordStepUpGrantIsBoundToActiveSessionAndReusableUntiservice := newPasskeyServiceForTest(t, db)ctx := context.Background()user := createPasskeyTestUser(t, db, "step-up-user")- sessionService := NewSessionService(db)+ sessionService := session.NewSessionService(db)session, _, err := sessionService.CreateSession(ctx, user.ID, time.Now().Add(time.Hour), auth.SessionMeta{})require.NoError(t, err)@@ -191,7 +192,7 @@ func TestPasskeyService_RecoveryCodeConsumptionIsAtomicAndSingleUse(t *testing.Trequire.NoError(t, err)require.Equal(t, user.ID, completion.User.ID)require.Equal(t, models.UserSessionSourceLocal, completion.Source)- require.Equal(t, RecoveryCodeMFAMethod, completion.Meta.MFAMethod)+ require.Equal(t, models.RecoveryCodeMFAMethod, completion.Meta.MFAMethod)require.Equal(t, "test-agent", completion.Meta.UserAgent)require.Equal(t, "127.0.0.1", completion.Meta.IPAddress)require.NotNil(t, completion.Meta.MFAVerifiedAt)@@ -215,7 +216,7 @@ func TestPasskeyService_EnableAndDisableMFAManagesCodesAndSessions(t *testing.T)ctx := context.Background()user := createPasskeyTestUser(t, db, "mfa-user")createPasskeyTestCredential(t, db, service, user.ID, "mfa-passkey")- sessionService := NewSessionService(db)+ sessionService := session.NewSessionService(db)currentSession, _, err := sessionService.CreateSession(ctx, user.ID, time.Now().Add(time.Hour), auth.SessionMeta{})require.NoError(t, err)otherSession, _, err := sessionService.CreateSession(ctx, user.ID, time.Now().Add(time.Hour), auth.SessionMeta{})@@ -260,7 +261,7 @@ func TestPasskeyService_ResetMFARevokesSessionsAndPreservesPasskeys(t *testing.Tuser.PasskeyMFAEnabled = truerequire.NoError(t, db.Save(user).Error)passkey := createPasskeyTestCredential(t, db, service, user.ID, "reset-passkey")- sessionService := NewSessionService(db)+ sessionService := session.NewSessionService(db)firstSession, _, err := sessionService.CreateSession(ctx, user.ID, time.Now().Add(time.Hour), auth.SessionMeta{})require.NoError(t, err)secondSession, _, err := sessionService.CreateSession(ctx, user.ID, time.Now().Add(time.Hour), auth.SessionMeta{})diff --git a/backend/internal/playwright/handler.go b/backend/internal/playwright/handler.gonew file mode 100644index 000000000..dfa274587--- /dev/null+++ b/backend/internal/playwright/handler.go@@ -0,0 +1,80 @@+//go:build playwright++package playwright++import (+ "net/http"++ "github.com/getarcaneapp/arcane/backend/v2/internal/federated"+ "github.com/labstack/echo/v5"+)++type playwrightHandlerInternal struct {+ service *PlaywrightService+ federated *federated.FederatedCredentialService+}++type createTestAPIKeysRequestInternal struct {+ Count int `json:"count"`+}++type createTestFederatedCredentialRequestInternal struct {+ IssuerURL string `json:"issuerUrl"`+ Audiences []string `json:"audiences"`+ Subject string `json:"subject"`+ RoleID string `json:"roleId"`+ TokenTTLSeconds int `json:"tokenTtlSeconds"`+}++// SetupRoutes registers endpoints used by the Playwright end-to-end suite.+func SetupRoutes(api *echo.Group, service *PlaywrightService, federatedService *federated.FederatedCredentialService) {+ group := api.Group("/playwright")+ handler := &playwrightHandlerInternal{service: service, federated: federatedService}++ group.POST("/create-test-api-keys", handler.createTestAPIKeysInternal)+ group.POST("/delete-test-api-keys", handler.deleteTestAPIKeysInternal)+ group.POST("/create-test-federated-credential", handler.createTestFederatedCredentialInternal)+}++func (h *playwrightHandlerInternal) createTestAPIKeysInternal(c *echo.Context) error {+ var req createTestAPIKeysRequestInternal+ if err := c.Bind(&req); err != nil || req.Count <= 0 {+ req.Count = 2+ }++ apiKeys, err := h.service.CreateTestApiKeys(c.Request().Context(), req.Count)+ if err != nil {+ return c.JSON(http.StatusInternalServerError, map[string]any{"error": err.Error()})+ }++ return c.JSON(http.StatusCreated, map[string]any{"apiKeys": apiKeys})+}++func (h *playwrightHandlerInternal) deleteTestAPIKeysInternal(c *echo.Context) error {+ if err := h.service.DeleteAllTestApiKeys(c.Request().Context()); err != nil {+ return c.JSON(http.StatusInternalServerError, map[string]any{"error": err.Error()})+ }++ return c.NoContent(http.StatusNoContent)+}++func (h *playwrightHandlerInternal) createTestFederatedCredentialInternal(c *echo.Context) error {+ var req createTestFederatedCredentialRequestInternal+ if err := c.Bind(&req); err != nil {+ return c.JSON(http.StatusBadRequest, map[string]any{"error": "invalid request body"})+ }++ credentialID, err := h.federated.CreatePlaywrightCredential(+ c.Request().Context(),+ req.IssuerURL,+ req.Audiences,+ req.Subject,+ req.RoleID,+ req.TokenTTLSeconds,+ )+ if err != nil {+ return c.JSON(http.StatusInternalServerError, map[string]any{"error": err.Error()})+ }++ return c.JSON(http.StatusCreated, map[string]any{"credential": map[string]string{"id": credentialID}})+}diff --git a/backend/internal/playwright/service.go b/backend/internal/playwright/service.gonew file mode 100644index 000000000..767b4562f--- /dev/null+++ b/backend/internal/playwright/service.go@@ -0,0 +1,96 @@+//go:build playwright++package playwright++import (+ "context"+ "fmt"+ "log/slog"++ "emperror.dev/errors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/apikey"+ "github.com/getarcaneapp/arcane/backend/v2/internal/user"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ apikeytypes "github.com/getarcaneapp/arcane/types/v2/apikey"+)++type PlaywrightService struct {+ apiKeyService *apikey.ApiKeyService+ userService *user.UserService+}++func NewPlaywrightService(apiKeyService *apikey.ApiKeyService, userService *user.UserService) *PlaywrightService {+ return &PlaywrightService{+ apiKeyService: apiKeyService,+ userService: userService,+ }+}++func (ps *PlaywrightService) CreateTestApiKeys(ctx context.Context, count int) ([]*apikeytypes.ApiKeyCreatedDto, error) {+ slog.Info("Playwright: Creating test API keys", "count", count)++ // Get the arcane user to associate the API keys with+ user, err := ps.userService.GetUserByUsername(ctx, "arcane")+ if err != nil {+ return nil, errors.WrapIf(err, "failed to get arcane user")+ }++ // Grant every recognized permission globally so the test key behaves like+ // the legacy "admin-everywhere" credential the e2e suite expects. There is+ // no request context here, so grant validation runs against a sudo set.+ allPerms := authz.AllPermissions()+ grants := make([]apikeytypes.PermissionGrant, len(allPerms))+ for i, p := range allPerms {+ grants[i] = apikeytypes.PermissionGrant{Permission: p}+ }++ var createdKeys []*apikeytypes.ApiKeyCreatedDto+ for i := range count {+ description := fmt.Sprintf("Test API key %d for Playwright tests", i+1)+ req := apikeytypes.CreateApiKey{+ Name: fmt.Sprintf("test-api-key-%d", i+1),+ Description: &description,+ Permissions: grants,+ }++ apiKey, err := ps.apiKeyService.CreateApiKey(ctx, user.ID, authz.SudoPermissionSet(), req)+ if err != nil {+ return nil, errors.WrapIff(err, "failed to create test API key %d", i+1)+ }++ createdKeys = append(createdKeys, apiKey)+ }++ slog.Info("Playwright: Test API keys created successfully", "count", len(createdKeys))+ return createdKeys, nil+}++func (ps *PlaywrightService) DeleteAllTestApiKeys(ctx context.Context) error {+ slog.Info("Playwright: Deleting all test API keys")++ // Get all API keys with test prefix+ params := pagination.QueryParams{+ SearchQuery: pagination.SearchQuery{+ Search: "test-api-key",+ },+ Params: pagination.Params{+ Start: 0,+ Limit: 1000,+ },+ }++ apiKeys, _, err := ps.apiKeyService.ListApiKeys(ctx, params)+ if err != nil {+ return errors.WrapIf(err, "failed to list API keys")+ }++ for _, apiKey := range apiKeys {+ if err := ps.apiKeyService.DeleteApiKey(ctx, apiKey.ID); err != nil {+ slog.Warn("Failed to delete test API key", "id", apiKey.ID, "error", err)+ }+ }++ slog.Info("Playwright: Test API keys deleted", "count", len(apiKeys))+ return nil+}diff --git a/backend/api/handlers/ports.go b/backend/internal/port/handler.gosimilarity index 73%rename from backend/api/handlers/ports.gorename to backend/internal/port/handler.goindex 4f4f2291e..43440609d 100644--- a/backend/api/handlers/ports.go+++ b/backend/internal/port/handler.go@@ -1,19 +1,19 @@-package handlers+package portimport ("context""net/http""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"porttypes "github.com/getarcaneapp/arcane/types/v2/port")type PortHandler struct {- portService *services.PortService+ portService *PortService}type ListPortsInput struct {@@ -29,21 +29,21 @@ type ListPortsOutput struct {Body base.Paginated[porttypes.PortMapping]}-func RegisterPorts(api huma.API, portSvc *services.PortService) {+func RegisterPorts(api huma.API, portSvc *PortService) {h := &PortHandler{portService: portSvc}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-ports",Method: http.MethodGet,Path: "/environments/{id}/ports",Summary: "List port mappings",Tags: []string{"Ports"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersList, h.ListPorts)}func (h *PortHandler) ListPorts(ctx context.Context, input *ListPortsInput) (*ListPortsOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)items, paginationResp, err := h.portService.ListPortsPaginated(ctx, params)if err != nil {@@ -54,7 +54,7 @@ func (h *PortHandler) ListPorts(ctx context.Context, input *ListPortsInput) (*LiBody: base.Paginated[porttypes.PortMapping]{Success: true,Data: items,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}diff --git a/backend/internal/services/port_service.go b/backend/internal/port/service.gosimilarity index 95%rename from backend/internal/services/port_service.gorename to backend/internal/port/service.goindex 821dfe62f..97fb8e058 100644--- a/backend/internal/services/port_service.go+++ b/backend/internal/port/service.go@@ -1,4 +1,4 @@-package services+package portimport ("context"@@ -7,6 +7,7 @@ import ("strings""emperror.dev/errors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"containertypes "github.com/getarcaneapp/arcane/types/v2/container"porttypes "github.com/getarcaneapp/arcane/types/v2/port"@@ -14,10 +15,10 @@ import ()type PortService struct {- dockerService *DockerClientService+ dockerService *docker.DockerClientService}-func NewPortService(dockerService *DockerClientService) *PortService {+func NewPortService(dockerService *docker.DockerClientService) *PortService {return &PortService{dockerService: dockerService}}@@ -51,7 +52,7 @@ func (s *PortService) ListPortsPaginated(ctx context.Context, params pagination.}result := pagination.SearchOrderAndPaginate(items, params, s.buildPortPaginationConfig())- return result.Items, pagination.BuildResponseFromFilterResult(result, params), nil+ return result.Items, pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params), nil}func (s *PortService) buildPortPaginationConfig() pagination.Config[porttypes.PortMapping] {diff --git a/backend/internal/services/port_service_test.go b/backend/internal/port/service_test.gosimilarity index 89%rename from backend/internal/services/port_service_test.gorename to backend/internal/port/service_test.goindex f31aac8d9..0d82860a4 100644--- a/backend/internal/services/port_service_test.go+++ b/backend/internal/port/service_test.go@@ -1,4 +1,4 @@-package services+package portimport ("context"@@ -9,13 +9,15 @@ import ("strings""testing"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"dockercontainer "github.com/moby/moby/api/types/container"+ "github.com/moby/moby/client""github.com/stretchr/testify/assert""github.com/stretchr/testify/require")-func newPortServiceTestDockerService(t *testing.T, containers []dockercontainer.Summary) *DockerClientService {+func newPortServiceTestDockerService(t *testing.T, containers []dockercontainer.Summary) *docker.DockerClientService {t.Helper()server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {@@ -32,7 +34,7 @@ func newPortServiceTestDockerService(t *testing.T, containers []dockercontainer.}))t.Cleanup(server.Close)- return &DockerClientService{client: newTestDockerClient(t, server)}+ return docker.NewDockerClientService(t.Context(), nil, nil, nil).WithClient(newTestDockerClientInternal(t, server))}func TestPortService_ListPortsPaginated_FlattensPublishedAndExposedPorts(t *testing.T) {@@ -224,3 +226,24 @@ func TestPrimaryContainerName_TrimsDockerPrefix(t *testing.T) {assert.Equal(t, "web", primaryContainerNameInternal([]string{"/web"}, "container-id"))assert.Equal(t, "abcdef123456", primaryContainerNameInternal(nil, "abcdef1234567890"))}++// Test fixtures shared by this package's tests.++// newTestDockerClientInternal builds a Docker client pointed at a fake Docker HTTP server.+func newTestDockerClientInternal(t *testing.T, server *httptest.Server) *client.Client {+ t.Helper()++ httpClient := server.Client()+ cli, err := client.New(+ client.WithHost(server.URL),+ client.WithAPIVersion("1.41"),+ client.WithHTTPClient(httpClient),+ )+ require.NoError(t, err)++ t.Cleanup(func() {+ _ = cli.Close()+ })++ return cli+}diff --git a/backend/api/handlers/projects.go b/backend/internal/project/handler.gosimilarity index 92%rename from backend/api/handlers/projects.gorename to backend/internal/project/handler.goindex 3bf15f8b4..a38c61017 100644--- a/backend/api/handlers/projects.go+++ b/backend/internal/project/handler.go@@ -1,4 +1,4 @@-package handlers+package projectimport ("context"@@ -12,15 +12,16 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity""github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/volumes""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper""github.com/getarcaneapp/arcane/types/v2/base"@@ -30,8 +31,8 @@ import (// ProjectHandler provides Huma-based project management endpoints.type ProjectHandler struct {- projectService *services.ProjectService- activityService *services.ActivityService+ projectService *ProjectService+ activityService *activity.ActivityServiceappCtx context.Context}@@ -201,228 +202,228 @@ type BuildProjectInput struct {}// RegisterProjects registers project management routes using Huma.-// Note: WebSocket and streaming endpoints remain as Gin handlers.-func RegisterProjects(api huma.API, projectService *services.ProjectService, activityService *services.ActivityService, appCtx ActivityAppContext) {+// WebSocket and streaming endpoints live in api/ws.+func RegisterProjects(api huma.API, projectService *ProjectService, activityService *activity.ActivityService, appCtx handlerutil.ActivityAppContext) {h := &ProjectHandler{projectService: projectService,activityService: activityService,- appCtx: appCtx.contextInternal(),+ appCtx: appCtx.Context(),}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-projects",Method: http.MethodGet,Path: "/environments/{id}/projects",Summary: "List projects",Description: "Get a paginated list of Docker Compose projects",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsList, h.ListProjects)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-project-status-counts",Method: http.MethodGet,Path: "/environments/{id}/projects/counts",Summary: "Get project status counts",Description: "Get counts of running, stopped, and total projects",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsList, h.GetProjectStatusCounts)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "deploy-project",Method: http.MethodPost,Path: "/environments/{id}/projects/{projectId}/up",Summary: "Deploy a project",Description: "Deploy a Docker Compose project (docker-compose up)",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsDeploy, h.DeployProject)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "down-project",Method: http.MethodPost,Path: "/environments/{id}/projects/{projectId}/down",Summary: "Bring down a project",Description: "Bring down a Docker Compose project (docker-compose down)",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsDown, h.DownProject)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-project",Method: http.MethodPost,Path: "/environments/{id}/projects",Summary: "Create a project",Description: "Create a new Docker Compose project",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsCreate, h.CreateProject)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-project",Method: http.MethodGet,Path: "/environments/{id}/projects/{projectId}",Summary: "Get a project",Description: "Get a Docker Compose project by ID",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsRead, h.GetProject)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-project-compose",Method: http.MethodGet,Path: "/environments/{id}/projects/{projectId}/compose",Summary: "Get project compose details",Description: "Get compose content, includes, and service configs for a project",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsRead, h.GetProjectCompose)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-project-files",Method: http.MethodGet,Path: "/environments/{id}/projects/{projectId}/files",Summary: "Get project files",Description: "Get directory files for a project",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsRead, h.GetProjectFiles)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-project-runtime",Method: http.MethodGet,Path: "/environments/{id}/projects/{projectId}/runtime",Summary: "Get project runtime",Description: "Get runtime service state for a project",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsRead, h.GetProjectRuntime)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-project-updates",Method: http.MethodGet,Path: "/environments/{id}/projects/{projectId}/updates",Summary: "Get project updates",Description: "Get image update summary for a project",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsRead, h.GetProjectUpdates)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-project-file",Method: http.MethodGet,Path: "/environments/{id}/projects/{projectId}/file",Summary: "Get a project file",Description: "Get the contents of a single project-related file by relative path",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsRead, h.GetProjectFile)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "redeploy-project",Method: http.MethodPost,Path: "/environments/{id}/projects/{projectId}/redeploy",Summary: "Redeploy a project",Description: "Redeploy a Docker Compose project (down + up)",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsDeploy, h.RedeployProject)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "destroy-project",Method: http.MethodDelete,Path: "/environments/{id}/projects/{projectId}/destroy",Summary: "Destroy a project",Description: "Destroy a Docker Compose project and optionally remove files/volumes",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsDelete, h.DestroyProject)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-project",Method: http.MethodPut,Path: "/environments/{id}/projects/{projectId}",Summary: "Update a project",Description: "Update a Docker Compose project configuration",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsUpdate, h.UpdateProject)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-project-include",Method: http.MethodPut,Path: "/environments/{id}/projects/{projectId}/includes",Summary: "Update project include file",Description: "Update an include file within a Docker Compose project",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsUpdate, h.UpdateProjectInclude)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "restart-project",Method: http.MethodPost,Path: "/environments/{id}/projects/{projectId}/restart",Summary: "Restart a project",Description: "Restart all containers in a Docker Compose project",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsRestart, h.RestartProject)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-project-services",Method: http.MethodPost,Path: "/environments/{id}/projects/{projectId}/update-services",Summary: "Update project services",Description: "Pull latest images and recreate the given services (all services when none are specified)",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsUpdate, h.UpdateProjectServices)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "archive-project",Method: http.MethodPost,Path: "/environments/{id}/projects/{projectId}/archive",Summary: "Archive a project",Description: "Archive a stopped Docker Compose project",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsArchive, h.ArchiveProject)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "unarchive-project",Method: http.MethodPost,Path: "/environments/{id}/projects/{projectId}/unarchive",Summary: "Unarchive a project",Description: "Unarchive a Docker Compose project",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsArchive, h.UnarchiveProject)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "pull-project-images",Method: http.MethodPost,Path: "/environments/{id}/projects/{projectId}/pull",Summary: "Pull project images",Description: "Pull all images for a Docker Compose project with streaming progress output",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsDeploy, h.PullProjectImages)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "build-project-images",Method: http.MethodPost,Path: "/environments/{id}/projects/{projectId}/build",Summary: "Build project images",Description: "Build Docker Compose services with build directives using BuildKit",Tags: []string{"Projects"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermProjectsDeploy, h.BuildProjectImages)}// ListProjects returns a paginated list of projects.func (h *ProjectHandler) ListProjects(ctx context.Context, input *ListProjectsInput) (*ListProjectsOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if input.Status != "" {params.Filters["status"] = input.Status}@@ -449,7 +450,7 @@ func (h *ProjectHandler) ListProjects(ctx context.Context, input *ListProjectsInBody: base.Paginated[project.Details]{Success: true,Data: projects,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -552,7 +553,7 @@ func (h *ProjectHandler) DeployProject(ctx context.Context, input *DeployProjectreturn nil, huma.Error400BadRequest("Project ID is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -572,7 +573,7 @@ func (h *ProjectHandler) DeployProject(ctx context.Context, input *DeployProject// DownProject brings down a Docker Compose project.func (h *ProjectHandler) DownProject(ctx context.Context, input *DownProjectInput) (*DownProjectOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -633,7 +634,7 @@ func projectFileHTTPError(err error) error {// CreateProject creates a new Docker Compose project.func (h *ProjectHandler) CreateProject(ctx context.Context, input *CreateProjectInput) (*CreateProjectOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -790,7 +791,7 @@ func (h *ProjectHandler) RedeployProject(ctx context.Context, input *RedeployProreturn nil, huma.Error400BadRequest("Project ID is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -810,7 +811,7 @@ func (h *ProjectHandler) RedeployProject(ctx context.Context, input *RedeployPro// DestroyProject destroys a Docker Compose project.func (h *ProjectHandler) DestroyProject(ctx context.Context, input *DestroyProjectInput) (*DestroyProjectOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -860,7 +861,7 @@ func (h *ProjectHandler) UpdateProject(ctx context.Context, input *UpdateProjectreturn nil, huma.Error400BadRequest("Project ID is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -919,7 +920,7 @@ func (h *ProjectHandler) UpdateProjectInclude(ctx context.Context, input *Updatereturn nil, huma.Error400BadRequest("Project ID is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -1081,7 +1082,7 @@ func (h *ProjectHandler) runProjectActivityActionInternal(ctx context.Context, ereturn base.MessageResponse{}, huma.Error400BadRequest("Project ID is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return base.MessageResponse{}, err}@@ -1115,7 +1116,7 @@ func (h *ProjectHandler) ArchiveProject(ctx context.Context, input *ArchiveProjereturn nil, huma.Error400BadRequest("Project ID is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -1140,7 +1141,7 @@ func (h *ProjectHandler) UnarchiveProject(ctx context.Context, input *UnarchivePreturn nil, huma.Error400BadRequest("Project ID is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -1163,7 +1164,7 @@ func (h *ProjectHandler) PullProjectImages(ctx context.Context, input *PullProjereturn nil, huma.Error400BadRequest("Project ID is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -1187,12 +1188,12 @@ func (h *ProjectHandler) BuildProjectImages(ctx context.Context, input *BuildProreturn nil, huma.Error400BadRequest("Project ID is required")}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}- options := services.ProjectBuildOptions{}+ options := ProjectBuildOptions{}if input.Body != nil {options.Services = input.Body.Servicesoptions.Provider = input.Body.Providerdiff --git a/backend/internal/project/module.go b/backend/internal/project/module.gonew file mode 100644index 000000000..9546b5f98--- /dev/null+++ b/backend/internal/project/module.go@@ -0,0 +1,33 @@+// Package project owns compose project persistence, lifecycle, discovery, and routes.+package project++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+)++type Module struct {+ service *ProjectService+ activity *activity.ActivityService+}++func New(service *ProjectService, activityService *activity.ActivityService) *Module {+ return &Module{service: service, activity: activityService}+}++func (m *Module) Service() *ProjectService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API, appCtx handlerutil.ActivityAppContext) {+ if m == nil {+ RegisterProjects(api, nil, nil, appCtx)+ return+ }+ RegisterProjects(api, m.service, m.activity, appCtx)+}diff --git a/backend/internal/project/project.go b/backend/internal/project/project.gonew file mode 100644index 000000000..5a5caf786--- /dev/null+++ b/backend/internal/project/project.go@@ -0,0 +1,420 @@+package project++import (+ "context"+ "io"+ "log/slog"+ "os"+ "path/filepath"+ "strings"+ "sync"+ "time"++ "github.com/getarcaneapp/arcane/backend/v2/internal/lifecycle"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/types/v2/containerregistry"+ dockerregistry "github.com/moby/moby/api/types/registry"+ "github.com/moby/moby/client"+ "github.com/samber/hot"+ "github.com/samber/mo"+ buildtypes "go.getarcane.app/builds/types"+ "gorm.io/gorm"+)++type buildServiceInternal interface {+ BuildImage(ctx context.Context, environmentID string, req buildtypes.BuildRequest, progressWriter io.Writer, serviceName string, user *models.User) (*buildtypes.BuildResult, error)+ BuildSettings() buildtypes.BuildSettings+}++type ProjectService struct {+ db *database.DB+ settingsService *settings.SettingsService+ eventService *event.EventService+ imageService *image.ImageService+ dockerService *docker.DockerClientService+ buildService buildServiceInternal+ lifecycleService *lifecycle.LifecycleService+ kvService *kv.KVService+ containerRegistryService *registry.ContainerRegistryService+ config *config.Config+ registryCredentialsProvider registryCredentialsProviderInternal++ // syncMu serializes SyncProjectsFromFileSystem: its discovery walk and its+ // cleanup pass must not interleave with another run's.+ syncMu sync.Mutex++ composeNames composeNameCacheInternal+ parsedCompose *parsedComposeCacheInternal+ // metaCache holds per-project icon/URL metadata, keyed by project ID. Deriving+ // it costs a full compose load (interpolation plus .env reads) and, for GitOps+ // projects, a gitops_syncs lookup — per project, on every list request. Mirrors+ // ContainerService.iconMetaCache.+ metaCache *hot.HotCache[string, projects.ArcaneComposeMetadata]+}++// EnsureGitOpsProjectLinked persists the bidirectional GitOps/project binding+// and refreshes the compose-name cache as one domain operation.+func (s *ProjectService) EnsureGitOpsProjectLinked(ctx context.Context, sync *models.GitOpsSync, project *models.Project) error {+ if sync == nil || project == nil {+ return nil+ }+ if project.GitOpsManagedBy != nil && *project.GitOpsManagedBy != "" && *project.GitOpsManagedBy != sync.ID {+ return errors.Errorf("project %s is already managed by a different GitOps sync", project.ID)+ }++ cacheBinding := func() {+ s.composeNames.put(projects.NormalizeProjectName(project.Name), project.ID)+ }+ if sync.ProjectID != nil && *sync.ProjectID == project.ID && project.GitOpsManagedBy != nil && *project.GitOpsManagedBy == sync.ID {+ cacheBinding()+ return nil+ }++ updatesSync := map[string]any{}+ updatesProject := map[string]any{}+ if sync.ProjectID == nil || *sync.ProjectID != project.ID {+ updatesSync["project_id"] = project.ID+ }+ if project.GitOpsManagedBy == nil || *project.GitOpsManagedBy != sync.ID {+ updatesProject["gitops_managed_by"] = sync.ID+ }+ if len(updatesSync) == 0 && len(updatesProject) == 0 {+ cacheBinding()+ return nil+ }++ if err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {+ if len(updatesSync) > 0 {+ if err := tx.Model(&models.GitOpsSync{}).Where("id = ?", sync.ID).Updates(updatesSync).Error; err != nil {+ return errors.WrapIff(err, "failed to relink GitOps sync %s", sync.ID)+ }+ }+ if len(updatesProject) > 0 {+ if err := tx.Model(&models.Project{}).Where("id = ?", project.ID).Updates(updatesProject).Error; err != nil {+ return errors.WrapIff(err, "failed to relink project %s to GitOps sync %s", project.ID, sync.ID)+ }+ }+ return nil+ }); err != nil {+ return err+ }++ sync.ProjectID = &project.ID+ project.GitOpsManagedBy = &sync.ID+ cacheBinding()+ return nil+}++// ValidateComposeDirectory loads a staged compose tree with the same settings,+// Docker path mapping, and validation rules used by managed projects.+func (s *ProjectService) ValidateComposeDirectory(ctx context.Context, projectName, projectPath, composeFileName string) (int, error) {+ projectsDirectory, err := s.GetProjectsDirectory(ctx)+ if err != nil {+ return 0, err+ }+ var dockerClient *client.Client+ if s.dockerService != nil {+ dockerClient, _ = s.dockerService.GetClient(ctx)+ }+ pathMapper := projects.NewPathMapperForConfiguredDirectory(+ ctx,+ s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"),+ "/app/data/projects",+ dockerClient,+ )+ composeProject, err := projects.LoadComposeProject(+ ctx,+ filepath.Join(projectPath, composeFileName),+ projects.NormalizeProjectName(projectName),+ projectsDirectory,+ s.settingsService.GetBoolSetting(ctx, "autoInjectEnv", false),+ pathMapper,+ nil,+ nil,+ true,+ )+ if err != nil {+ return 0, err+ }+ return len(composeProject.Services), nil+}++// CreateGitOpsManagedProject persists a promoted GitOps project, links both+// records, updates the compose-name cache, and records the creation event.+func (s *ProjectService) CreateGitOpsManagedProject(ctx context.Context, sync *models.GitOpsSync, project *models.Project, actor models.User, logEventOptions ...bool) error {+ if sync == nil || project == nil {+ return errors.New("GitOps sync and project are required")+ }+ if err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {+ if err := tx.Create(project).Error; err != nil {+ return errors.WrapIf(err, "failed to create project")+ }+ if err := tx.Model(&models.GitOpsSync{}).Where("id = ?", sync.ID).Update("project_id", project.ID).Error; err != nil {+ return errors.WrapIf(err, "failed to update sync with project ID")+ }+ if err := tx.Model(&models.Project{}).Where("id = ?", project.ID).Update("gitops_managed_by", sync.ID).Error; err != nil {+ return errors.WrapIf(err, "failed to mark project as GitOps-managed")+ }+ return nil+ }); err != nil {+ return err+ }++ sync.ProjectID = &project.ID+ project.GitOpsManagedBy = &sync.ID+ s.composeNames.put(projects.NormalizeProjectName(project.Name), project.ID)+ logEvent := true+ if len(logEventOptions) > 0 {+ logEvent = logEventOptions[0]+ }+ if logEvent && s.eventService != nil {+ metadata := models.JSON{"action": "create", "projectID": project.ID, "projectName": project.Name, "path": project.Path}+ if err := s.eventService.LogProjectEvent(ctx, models.EventTypeProjectCreate, project.ID, project.Name, actor.ID, actor.Username, "0", metadata); err != nil {+ slog.ErrorContext(ctx, "could not log project creation", "error", err)+ }+ }+ return nil+}++// projectMetadataEnvInternal carries the inputs ParseArcaneComposeMetadata needs+// beyond the project itself. Resolving them costs a settings clone and a stat+// syscall each, so list paths resolve once and reuse across every project.+type projectMetadataEnvInternal struct {+ projectsDirectory string+ autoInjectEnv bool+}++// projectMetadataTTL bounds how stale cached compose metadata can be. It matches+// containerIconMetadataTTL so icons resolved from either service agree.+const projectMetadataTTL = 5 * time.Second++type registryCredentialsProviderInternal func(context.Context) ([]containerregistry.Credential, error)++func NewProjectService(db *database.DB, settingsService *settings.SettingsService, eventService *event.EventService, imageService *image.ImageService, dockerService *docker.DockerClientService, buildService buildServiceInternal, lifecycleService *lifecycle.LifecycleService, containerRegistryService *registry.ContainerRegistryService, cfg *config.Config) *ProjectService {+ return &ProjectService{+ db: db,+ settingsService: settingsService,+ eventService: eventService,+ imageService: imageService,+ dockerService: dockerService,+ buildService: buildService,+ lifecycleService: lifecycleService,+ containerRegistryService: containerRegistryService,+ config: cfg,+ parsedCompose: newParsedComposeCacheInternal(),+ metaCache: hot.NewHotCache[string, projects.ArcaneComposeMetadata](hot.LRU, 1024).+ WithTTL(projectMetadataTTL).+ WithJanitor().+ Build(),+ }+}++func (s *ProjectService) WithRegistryCredentialsProvider(provider func(context.Context) ([]containerregistry.Credential, error)) *ProjectService {+ if s == nil {+ return nil+ }+ s.registryCredentialsProvider = provider+ return s+}++func (s *ProjectService) WithKVService(kvService *kv.KVService) *ProjectService {+ if s == nil {+ return nil+ }+ s.kvService = kvService+ return s+}++func (s *ProjectService) ResolveRegistryCredentials(ctx context.Context) ([]containerregistry.Credential, error) {+ if s == nil || s.registryCredentialsProvider == nil {+ return nil, nil+ }++ credentials, err := s.registryCredentialsProvider(ctx)+ if err != nil {+ return nil, errors.WrapIf(err, "get enabled registry credentials")+ }++ return credentials, nil+}++func (s *ProjectService) composeRegistryAuthConfigsInternal(ctx context.Context) map[string]dockerregistry.AuthConfig {+ if s == nil || s.containerRegistryService == nil {+ return nil+ }++ authConfigs, err := s.containerRegistryService.GetAllRegistryAuthConfigs(ctx)+ if err != nil {+ slog.WarnContext(ctx, "failed to load registry auth for compose pulls", "error", err)+ return nil+ }++ return authConfigs+}++func (s *ProjectService) GetProjectsDirectory(ctx context.Context) (string, error) {+ projectsDirSetting := s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects")+ projectsDir, err := projects.GetProjectsDirectory(ctx, strings.TrimSpace(projectsDirSetting))+ if err != nil {+ return "", err+ }++ return filepath.Clean(projectsDir), nil+}++func getProjectsDirectoryOrDefaultInternal(ctx context.Context, cfg *models.Settings) string {+ projectsDirectory, err := projects.GetProjectsDirectory(ctx, strings.TrimSpace(cfg.ProjectsDirectory.Value))+ if err != nil {+ slog.WarnContext(ctx, "unable to determine projects directory; using default", "error", err)+ return "/app/data/projects"+ }+ return projectsDirectory+}++func (s *ProjectService) getMutableProjectInternal(ctx context.Context, projectID string) (*models.Project, error) {+ proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return nil, err+ }+ if err := ensureProjectMutableInternal(proj); err != nil {+ return nil, err+ }+ return proj, nil+}++func (s *ProjectService) logProjectEventInternal(ctx context.Context, eventType models.EventType, projectID, projectName string, user models.User, metadata models.JSON, action string) {+ if s.eventService == nil {+ return+ }+ if logErr := s.eventService.LogProjectEvent(ctx, eventType, projectID, projectName, user.ID, user.Username, "0", metadata); logErr != nil {+ slog.ErrorContext(ctx, action, "error", logErr)+ }+}++func (s *ProjectService) GetProjectRelativePath(ctx context.Context, projectPath string) string {+ projectsDir, err := s.GetProjectsDirectory(ctx)+ if err != nil {+ return ""+ }++ return getProjectRelativePathInternal(projectsDir, projectPath)+}++func getProjectRelativePathInternal(projectsDir, projectPath string) string {+ if strings.TrimSpace(projectsDir) == "" {+ return ""+ }++ relativePath, err := filepath.Rel(projectsDir, filepath.Clean(projectPath))+ if err != nil {+ return ""+ }+ if relativePath == "." {+ return ""+ }+ if relativePath == ".." || strings.HasPrefix(relativePath, ".."+string(os.PathSeparator)) {+ return ""+ }++ return filepath.ToSlash(relativePath)+}++func (s *ProjectService) GetProjectFromDatabaseByID(ctx context.Context, id string) (*models.Project, error) {+ var projectModel models.Project+ if err := s.db.WithContext(ctx).Where("id = ?", id).First(&projectModel).Error; err != nil {+ if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {+ return nil, errors.New("request canceled or timed out")+ }+ if errors.Is(err, gorm.ErrRecordNotFound) {+ return nil, errors.New("project not found")+ }+ return nil, errors.WrapIf(err, "failed to get project")+ }+ return &projectModel, nil+}++func (s *ProjectService) GetProjectByComposeName(ctx context.Context, name string) (*models.Project, error) {+ if name == "" {+ return nil, errors.New("project name is empty")+ }+ normalized := projects.NormalizeProjectName(name)++ var proj models.Project+ err := s.db.WithContext(ctx).Where("name = ? OR name = ?", name, normalized).First(&proj).Error+ if err == nil {+ s.composeNames.put(normalized, proj.ID)+ return &proj, nil+ }+ if !errors.Is(err, gorm.ErrRecordNotFound) {+ return nil, errors.WrapIf(err, "failed to get project by name")+ }++ if cachedProject, found, cacheErr := s.lookupProjectByCachedComposeNameInternal(ctx, normalized); cacheErr != nil {+ return nil, cacheErr+ } else if found {+ return cachedProject, nil+ }++ if err := s.rebuildComposeNameCacheInternal(ctx); err != nil {+ return nil, errors.WrapIf(err, "failed to list projects by compose name")+ }++ if cachedProject, found, cacheErr := s.lookupProjectByCachedComposeNameInternal(ctx, normalized); cacheErr != nil {+ return nil, cacheErr+ } else if found {+ return cachedProject, nil+ }++ return nil, errors.Errorf("project not found: %s", name)+}++// EnsureProjectPathUnderRoot validates that the project's path is a safe subdirectory of the configured projects root.+// If not, it normalizes the path to `<projectsRoot>/<dirName or sanitized project name>`. When persist=true, it saves+// the updated project path to the database.+func (s *ProjectService) EnsureProjectPathUnderRoot(ctx context.Context, proj *models.Project, persist bool) error {+ projectsDirectory, err := projects.GetProjectsDirectory(ctx, s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"))+ if err != nil {+ return errors.WrapIf(err, "failed to get projects directory")+ }++ rootAbs, _ := filepath.Abs(projectsDirectory)+ rootAbs = filepath.Clean(rootAbs)++ projPathAbs := proj.Path+ if abs, aerr := filepath.Abs(proj.Path); aerr == nil {+ projPathAbs = filepath.Clean(abs)+ }++ if projects.IsSafeSubdirectory(rootAbs, projPathAbs) {+ return nil+ }++ // Attempt to repair using known directory name or sanitized project name+ dirName := mo.PointerToOption(proj.DirName).OrEmpty()+ if strings.TrimSpace(dirName) == "" {+ dirName = projects.SanitizeProjectName(proj.Name)+ }+ candidate := filepath.Join(projectsDirectory, dirName)++ slog.WarnContext(ctx, "Normalizing project path to projects root", "projectID", proj.ID, "oldPath", proj.Path, "newPath", candidate, "root", projectsDirectory)+ proj.Path = filepath.Clean(candidate)++ if persist {+ if saveErr := s.db.WithContext(ctx).Save(proj).Error; saveErr != nil {+ slog.WarnContext(ctx, "failed to persist normalized project path", "error", saveErr)+ }+ }+ return nil+}diff --git a/backend/internal/project/project_compose_cache.go b/backend/internal/project/project_compose_cache.gonew file mode 100644index 000000000..98ca2b95e--- /dev/null+++ b/backend/internal/project/project_compose_cache.go@@ -0,0 +1,332 @@+package project++import (+ "context"+ "log/slog"+ "os"+ "path/filepath"+ "strings"+ "sync"+ "time"++ "emperror.dev/errors"++ composetypes "github.com/compose-spec/compose-go/v2/types"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/moby/moby/client"+ "github.com/samber/hot"+ "github.com/samber/mo"+ "gorm.io/gorm"+)++// composeNameCacheInternal maps normalized compose project names to project+// IDs so a name lookup skips the projects table scan.+type composeNameCacheInternal struct {+ mu sync.RWMutex+ byName map[string]string+}++// parsedComposeCacheInternal caches parsed compose projects by project ID. An+// entry is dropped once its compose file or any of its includes changes on disk.+type parsedComposeCacheInternal struct {+ entries *hot.HotCache[string, composeCacheEntry]+}++type composeCacheEntry struct {+ composePath string+ composeMtime time.Time+ includeMtimes map[string]time.Time+ project *composetypes.Project+}++func newParsedComposeCacheInternal() *parsedComposeCacheInternal {+ return &parsedComposeCacheInternal{entries: hot.NewHotCache[string, composeCacheEntry](hot.LRU, 2048).Build()}+}++func (c *composeNameCacheInternal) projectID(normalizedName string) mo.Option[string] {+ if normalizedName == "" {+ return mo.None[string]()+ }++ c.mu.RLock()+ defer c.mu.RUnlock()++ if c.byName == nil {+ return mo.None[string]()+ }++ projectID, ok := c.byName[normalizedName]+ return mo.TupleToOption(projectID, ok)+}++func (c *composeNameCacheInternal) put(normalizedName, projectID string) {+ if normalizedName == "" || projectID == "" {+ return+ }++ c.mu.Lock()+ defer c.mu.Unlock()++ if c.byName == nil {+ c.byName = make(map[string]string)+ }+ c.byName[normalizedName] = projectID+}++func (c *composeNameCacheInternal) invalidate(normalizedName string) {+ if normalizedName == "" {+ return+ }++ c.mu.Lock()+ defer c.mu.Unlock()++ delete(c.byName, normalizedName)+}++func (c *composeNameCacheInternal) replace(byName map[string]string) {+ c.mu.Lock()+ c.byName = byName+ c.mu.Unlock()+}++func (c *parsedComposeCacheInternal) invalidate(projectID string) {+ if c == nil || c.entries == nil || strings.TrimSpace(projectID) == "" {+ return+ }+ c.entries.Delete(projectID)+}++func (s *ProjectService) lookupProjectByCachedComposeNameInternal(ctx context.Context, normalizedName string) (*models.Project, bool, error) {+ projectID, ok := s.composeNames.projectID(normalizedName).Get()+ if !ok {+ return nil, false, nil+ }++ var projectModel models.Project+ if err := s.db.WithContext(ctx).Where("id = ?", projectID).First(&projectModel).Error; err != nil {+ if errors.Is(err, gorm.ErrRecordNotFound) {+ s.composeNames.invalidate(normalizedName)+ return nil, false, nil+ }+ if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {+ return nil, false, errors.WrapIf(err, "request canceled or timed out")+ }+ return nil, false, errors.WrapIf(err, "failed to get project by cached compose name")+ }+ if projects.NormalizeProjectName(projectModel.Name) != normalizedName {+ s.composeNames.invalidate(normalizedName)+ return nil, false, nil+ }++ return &projectModel, true, nil+}++func (s *ProjectService) rebuildComposeNameCacheInternal(ctx context.Context) error {+ var projectModels []models.Project+ if err := s.db.WithContext(ctx).Select("id", "name").Find(&projectModels).Error; err != nil {+ return err+ }++ byName := make(map[string]string, len(projectModels))+ for i := range projectModels {+ normalizedName := projects.NormalizeProjectName(projectModels[i].Name)+ if normalizedName == "" {+ continue+ }+ if _, exists := byName[normalizedName]; !exists {+ byName[normalizedName] = projectModels[i].ID+ }+ }++ s.composeNames.replace(byName)++ return nil+}++func (s *ProjectService) ResolveProjectComposeFile(ctx context.Context, proj *models.Project) (string, error) {+ if proj == nil {+ return "", errors.New("project is nil")+ }++ if proj.GitOpsManagedBy != nil && strings.TrimSpace(*proj.GitOpsManagedBy) != "" {+ var syncRecord models.GitOpsSync+ if err := s.db.WithContext(ctx).+ Select("compose_path").+ Where("id = ?", *proj.GitOpsManagedBy).+ First(&syncRecord).Error; err == nil {+ composeFileName := strings.TrimSpace(filepath.Base(syncRecord.ComposePath))+ if composeFileName != "" && composeFileName != "." {+ candidate := filepath.Join(proj.Path, composeFileName)+ if info, statErr := os.Stat(candidate); statErr == nil {+ if !info.IsDir() {+ return candidate, nil+ }+ } else if !os.IsNotExist(statErr) {+ return "", errors.WrapIff(statErr, "failed to inspect GitOps compose file %s", candidate)+ }+ }+ } else if !errors.Is(err, gorm.ErrRecordNotFound) {+ return "", errors.WrapIff(err, "failed to resolve GitOps compose path for project %s", proj.ID)+ }+ }++ composeFile, err := projects.DetectComposeFile(proj.Path)+ if err != nil {+ return "", common.Classify(common.ErrProjectComposeFileNotFound, errors.WrapIf(err, "Project compose file not found"))+ }++ return composeFile, nil+}++func (s *ProjectService) loadComposeProjectForProjectInternal(ctx context.Context, proj *models.Project, cfg *models.Settings) (*composetypes.Project, string, error) {+ composeFileFullPath, err := s.ResolveProjectComposeFile(ctx, proj)+ if err != nil {+ return nil, "", err+ }++ if cfg == nil {+ cfg = s.settingsService.GetSettingsOrDefaults(ctx)+ }+ projectsDirectory := getProjectsDirectoryOrDefaultInternal(ctx, cfg)++ var dockerClient *client.Client+ if s.dockerService != nil {+ dockerClient, _ = s.dockerService.GetClient(ctx)+ }+ pathMapper := projects.NewPathMapperForConfiguredDirectory(+ ctx,+ s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"),+ "/app/data/projects",+ dockerClient,+ )++ composeProject, loadErr := projects.LoadComposeProject(ctx, composeFileFullPath, projects.NormalizeProjectName(proj.Name), projectsDirectory, utils.BoolOrDefault(cfg.AutoInjectEnv.Value, false), pathMapper, nil, nil, false)+ if loadErr != nil {+ return nil, "", loadErr+ }++ return composeProject, composeFileFullPath, nil+}++func (s *ProjectService) getCachedComposeProjectInternal(ctx context.Context, proj *models.Project, cfg *models.Settings) (*composetypes.Project, error) {+ if proj == nil {+ return nil, errors.New("project is nil")+ }+ if s.parsedCompose == nil {+ s.parsedCompose = newParsedComposeCacheInternal()+ }++ if cached, ok := s.parsedCompose.entries.Peek(proj.ID); ok {+ if validComposeCacheEntryInternal(cached) {+ return cached.project, nil+ }+ s.parsedCompose.entries.Delete(proj.ID)+ }++ entry, found, err := s.parsedCompose.entries.GetWithLoaders(proj.ID, func(_ []string) (map[string]composeCacheEntry, error) {+ composeProject, composePath, err := s.loadComposeProjectForProjectInternal(ctx, proj, cfg)+ if err != nil {+ return nil, err+ }++ entry := composeCacheEntry{+ composePath: composePath,+ includeMtimes: make(map[string]time.Time),+ project: composeProject,+ }+ if info, statErr := os.Stat(composePath); statErr == nil {+ entry.composeMtime = info.ModTime()+ } else {+ return nil, errors.WrapIf(statErr, "stat compose file")+ }+ if composeProject != nil {+ for _, composeFile := range composeProject.ComposeFiles {+ if composeFile == "" || composeFile == composePath {+ continue+ }+ info, statErr := os.Stat(composeFile)+ if statErr != nil {+ return nil, errors.WrapIff(statErr, "stat compose include %s", composeFile)+ }+ entry.includeMtimes[composeFile] = info.ModTime()+ }+ }++ return map[string]composeCacheEntry{proj.ID: entry}, nil+ })+ if err != nil {+ return nil, err+ }+ if !found {+ return nil, errors.New("compose cache loader returned no project")+ }++ return entry.project, nil+}++func validComposeCacheEntryInternal(entry composeCacheEntry) bool {+ if entry.project == nil || entry.composePath == "" {+ return false+ }++ info, err := os.Stat(entry.composePath)+ if err != nil || !info.ModTime().Equal(entry.composeMtime) {+ return false+ }+ for includePath, cachedMtime := range entry.includeMtimes {+ info, err := os.Stat(includePath)+ if err != nil || !info.ModTime().Equal(cachedMtime) {+ return false+ }+ }+ return true+}++func (s *ProjectService) refreshComposeProjectNameInternal(ctx context.Context, proj *models.Project) {+ if proj == nil {+ return+ }++ dirName := proj.Name+ if proj.DirName != nil && *proj.DirName != "" {+ dirName = *proj.DirName+ }++ meta, err := s.loadComposeMetadataForSyncInternal(ctx, proj.Path, dirName)+ if err != nil {+ slog.WarnContext(ctx, "failed to refresh compose project name", "projectID", proj.ID, "path", proj.Path, "error", err)+ return+ }++ updates := map[string]any{}+ shouldUpdateName := meta.explicitProjectName || projects.NormalizeProjectName(proj.Name) != proj.Name+ if shouldUpdateName && meta.resolvedProjectName != "" && proj.Name != meta.resolvedProjectName {+ updates["name"] = meta.resolvedProjectName+ }+ if mo.PointerToOption(proj.ComposeProjectName) != mo.PointerToOption(meta.composeProjectName) {+ updates["compose_project_name"] = meta.composeProjectName+ }+ if len(updates) == 0 {+ return+ }++ updates["updated_at"] = time.Now()+ if err := s.db.WithContext(ctx).+ Model(&models.Project{}).+ Where("id = ?", proj.ID).+ Updates(updates).Error; err != nil {+ slog.WarnContext(ctx, "failed to persist refreshed compose project name", "projectID", proj.ID, "error", err)+ return+ }++ if name, ok := updates["name"].(string); ok {+ proj.Name = name+ }+ if _, ok := updates["compose_project_name"]; ok {+ proj.ComposeProjectName = meta.composeProjectName+ }+}diff --git a/backend/internal/project/project_details.go b/backend/internal/project/project_details.gonew file mode 100644index 000000000..cafde37bc--- /dev/null+++ b/backend/internal/project/project_details.go@@ -0,0 +1,789 @@+package project++import (+ "bufio"+ "context"+ "io"+ "log/slog"+ "os"+ "path/filepath"+ "strings"+ "sync"+ "time"++ "emperror.dev/errors"++ composetypes "github.com/compose-spec/compose-go/v2/types"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/iconcatalog"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper"+ imagetypes "github.com/getarcaneapp/arcane/types/v2/image"+ "github.com/getarcaneapp/arcane/types/v2/project"+ "github.com/samber/mo"+ "go.getarcane.app/sys/cgroup"+ "go.getarcane.app/updater/labels"+)++type ProjectServiceInfo struct {+ Name string `json:"name"`+ Image string `json:"image"`+ Status string `json:"status"`+ ContainerID string `json:"container_id"`+ ContainerName string `json:"container_name"`+ Ports []string `json:"ports"`+ Health *string `json:"health,omitempty"`+ IconLightURL string `json:"icon_light_url,omitempty"`+ IconDarkURL string `json:"icon_dark_url,omitempty"`+ ServiceConfig *composetypes.ServiceConfig `json:"service_config,omitempty"`+ Labels map[string]string `json:"labels,omitempty"`+ RedeployDisabled bool `json:"redeploy_disabled,omitempty"`+}++func getServiceCounts(services []ProjectServiceInfo) (total int, running int) {+ total = len(services)+ for _, service := range services {+ st := strings.ToLower(strings.TrimSpace(service.Status))+ if st == "running" || st == "up" {+ running+++ }+ }+ return total, running+}++func (s *ProjectService) updateProjectStatusandCountsInternal(ctx context.Context, projectID string, status models.ProjectStatus) error {+ services, err := s.GetProjectServices(ctx, projectID)+ if err != nil {+ slog.Error("GetProjectServices failed during status update", "projectID", projectID, "error", err)+ return s.updateProjectStatusInternal(ctx, projectID, status)+ }++ serviceCount, runningCount := getServiceCounts(services)++ if err := s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", projectID).Updates(map[string]any{+ "status": status,+ "service_count": serviceCount,+ "running_count": runningCount,+ "updated_at": time.Now(),+ }).Error; err != nil {+ return errors.WrapIf(err, "failed to update project status and counts")+ }++ return nil+}++func (s *ProjectService) updateProjectStatusInternal(ctx context.Context, id string, status models.ProjectStatus) error {+ now := time.Now()+ res := s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", id).Updates(map[string]any{+ "status": status,+ "updated_at": now,+ })++ if res.Error != nil {+ return errors.WrapIf(res.Error, "failed to update project status")+ }++ return nil+}++func (s *ProjectService) GetProjectServices(ctx context.Context, projectID string) ([]ProjectServiceInfo, error) {+ projectFromDb, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return nil, err+ }++ composeProject, composeFileFullPath, derr := s.loadComposeProjectForProjectInternal(ctx, projectFromDb, nil)+ if derr != nil {+ return []ProjectServiceInfo{}, errors.WrapIff(derr, "failed to load compose project in %s", projectFromDb.Path)+ }++ projectsDirectory, projectsDirErr := s.GetProjectsDirectory(ctx)+ if projectsDirErr != nil {+ slog.WarnContext(ctx, "failed to resolve projects directory for Arcane compose metadata", "path", composeFileFullPath, "error", projectsDirErr)+ }+ autoInjectEnv := s.settingsService.GetBoolSetting(ctx, "autoInjectEnv", false)++ meta, metaErr := projects.ParseArcaneComposeMetadata(ctx, composeFileFullPath, projectsDirectory, autoInjectEnv)+ if metaErr != nil {+ slog.WarnContext(ctx, "failed to parse Arcane compose metadata", "path", composeFileFullPath, "error", metaErr)+ }++ containers, err := projects.ComposePs(ctx, composeProject, nil, true)+ if err != nil {+ slog.Error("compose ps error", "projectName", composeProject.Name, "error", err)+ return nil, errors.WrapIf(err, "failed to get compose services status")+ }+ currentContainerID, currentContainerErr := cgroup.CurrentContainerID()++ have := map[string]bool{}+ var services []ProjectServiceInfo++ // Create a map for quick lookup of service config+ serviceConfigs := make(map[string]composetypes.ServiceConfig)+ for _, svc := range composeProject.Services {+ serviceConfigs[svc.Name] = svc+ }++ for _, c := range containers {+ var health *string+ if c.Health != "" {+ health = new(string(c.Health))+ }++ var svcConfig *composetypes.ServiceConfig+ if cfg, ok := serviceConfigs[c.Service]; ok {+ svcConfig = &cfg+ }++ resolvedIcon := iconcatalog.Resolve(IconCatalogForContext(ctx), iconcatalog.FirstNonEmpty(+ projects.FindArcaneIconSet(c.Labels),+ meta.ServiceIconSets[c.Service],+ meta.ProjectIcon,+ ))+ services = append(services, ProjectServiceInfo{+ Name: c.Service,+ Image: c.Image,+ Status: string(c.State),+ ContainerID: c.ID,+ ContainerName: c.Name,+ Ports: projects.FormatPorts(c.Publishers),+ Health: health,+ IconLightURL: resolvedIcon.IconLightURL,+ IconDarkURL: resolvedIcon.IconDarkURL,+ ServiceConfig: svcConfig,+ Labels: c.Labels,+ RedeployDisabled: labels.ShouldDisableArcaneServerRedeploy(c.Labels, c.ID, currentContainerID, currentContainerErr),+ })+ have[c.Service] = true+ }++ for _, svc := range composeProject.Services {+ if !have[svc.Name] {+ resolvedIcon := iconcatalog.Resolve(IconCatalogForContext(ctx), iconcatalog.FirstNonEmpty(+ meta.ServiceIconSets[svc.Name],+ meta.ProjectIcon,+ ))+ services = append(services, ProjectServiceInfo{+ Name: svc.Name,+ Image: svc.Image,+ Status: "stopped",+ Ports: []string{},+ IconLightURL: resolvedIcon.IconLightURL,+ IconDarkURL: resolvedIcon.IconDarkURL,+ ServiceConfig: new(svc),+ })+ }+ }++ return services, nil+}++func (s *ProjectService) GetProjectContent(ctx context.Context, projectID string) (composeContent, envContent, overrideContent string, err error) {+ proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return "", "", "", err+ }++ composePath, composeErr := s.ResolveProjectComposeFile(ctx, proj)+ if composeErr != nil {+ composePath = ""+ }++ composeContent, envContent, err = projects.ReadProjectFiles(proj.Path, composePath)+ if err != nil {+ return "", "", "", err+ }++ return composeContent, envContent, projects.ReadComposeOverrideContent(proj.Path), nil+}++func (s *ProjectService) GetProjectDetails(ctx context.Context, projectID string, opts project.DetailsOptions) (project.Details, error) {+ proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return project.Details{}, err+ }+ projectsDir, _ := s.GetProjectsDirectory(ctx)++ var resp project.Details+ if err := mapper.MapStruct(proj, &resp); err != nil {+ return project.Details{}, errors.WrapIf(err, "failed to map project")+ }++ resp.CreatedAt = proj.CreatedAt.Format(time.RFC3339)+ resp.UpdatedAt = proj.UpdatedAt.Format(time.RFC3339)+ resp.IsArchived = proj.IsArchived+ resp.ArchivedAt = proj.ArchivedAt+ resp.HasBuildDirective = false+ resp.DirName = mo.PointerToOption(proj.DirName).OrEmpty()+ resp.RelativePath = getProjectRelativePathInternal(projectsDir, proj.Path)+ resp.GitOpsManagedBy = proj.GitOpsManagedBy+ meta := s.ProjectMetadata(ctx, *proj, nil)+ applyResolvedProjectIconInternal(&resp, iconcatalog.Resolve(IconCatalogForContext(ctx), meta.ProjectIcon))+ resp.URLs = meta.ProjectURLS++ // Default counts/status from DB (will be overridden if runtime check succeeds)+ resp.ServiceCount = proj.ServiceCount+ resp.RunningCount = proj.RunningCount+ resp.Status = string(proj.Status)++ if opts.IncludeComposeContent {+ composeContent, _, overrideContent, _ := s.GetProjectContent(ctx, projectID)+ resp.ComposeContent = composeContent+ resp.OverrideContent = overrideContent+ if overridePath := projects.DetectComposeOverrideFile(proj.Path); overridePath != "" {+ resp.OverrideFileName = filepath.Base(overridePath)+ }+ }+ if opts.IncludeEnvState {+ envState, err := projects.ReadProjectEnvState(proj.Path)+ if err != nil {+ return project.Details{}, errors.WrapIf(err, "failed to read project env state")+ }+ effectiveEnvContent, err := resolveStoredEffectiveEnvContentInternal(envState)+ if err != nil {+ return project.Details{}, err+ }+ resp.EnvContent = effectiveEnvContent+ }++ // Enrich with details+ composeFile, composeFileErr := s.ResolveProjectComposeFile(ctx, proj)+ if composeFileErr == nil {+ resp.ComposeFileName = filepath.Base(composeFile)+ if opts.IncludeIncludeFiles {+ s.enrichWithIncludeFiles(ctx, composeFile, &resp)+ }+ if opts.IncludeServiceConfigs {+ s.enrichWithComposeServiceConfigs(ctx, proj, composeFile, &resp)+ }+ }+ if opts.IncludeDirectoryFiles {+ s.enrichWithDirectoryFiles(ctx, proj.Path, &resp)+ }+ if opts.IncludeProjectFiles {+ s.enrichWithProjectFiles(ctx, proj.Path, resp.ComposeFileName, &resp)+ }+ s.enrichWithGitOpsInfo(ctx, proj, &resp)++ // Refresh runtime status/counts even when callers do not request the full+ // runtime service array. DB values are only a fallback when Docker lookup+ // or compose loading fails.+ services, serr := s.GetProjectServices(ctx, projectID)+ if serr == nil && services != nil {+ resp.ServiceCount = len(services)+ _, runningCount := getServiceCounts(services)+ resp.RunningCount = runningCount+ resp.Status = string(calculateProjectStatus(services))++ if opts.IncludeRuntimeServices {+ resp.RuntimeServices = buildProjectRuntimeServicesInternal(services)+ for _, svc := range services {+ if svc.RedeployDisabled {+ resp.RedeployDisabled = true+ break+ }+ }+ }+ }++ if opts.IncludeUpdateInfo {+ s.enrichProjectUpdateInfoInternal(ctx, &resp)+ }++ return resp, nil+}++func buildProjectRuntimeServicesInternal(services []ProjectServiceInfo) []project.RuntimeService {+ runtimeServices := make([]project.RuntimeService, len(services))+ for i, svc := range services {+ runtimeServices[i] = project.RuntimeService{+ Name: svc.Name,+ Image: svc.Image,+ Status: svc.Status,+ ContainerID: svc.ContainerID,+ ContainerName: svc.ContainerName,+ Ports: svc.Ports,+ Health: svc.Health,+ IconLightURL: svc.IconLightURL,+ IconDarkURL: svc.IconDarkURL,+ ServiceConfig: svc.ServiceConfig,+ RedeployDisabled: svc.RedeployDisabled,+ }+ }+ return runtimeServices+}++func (s *ProjectService) GetProjectFileContent(ctx context.Context, projectID, relativePath string) (project.IncludeFile, error) {+ proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return project.IncludeFile{}, err+ }+ if strings.TrimSpace(relativePath) == "" {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: relative path is required"))+ }+ normalizedRelativePath, err := projects.NormalizeProjectRelativePath(relativePath)+ if err != nil {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.WrapIf(err, "Forbidden project file path"))+ }++ composeFile, detectErr := s.ResolveProjectComposeFile(ctx, proj)+ if detectErr == nil {+ cfg := s.settingsService.GetSettingsOrDefaults(ctx)+ projectsDirectory, _ := projects.GetProjectsDirectory(ctx, strings.TrimSpace(cfg.ProjectsDirectory.Value))+ envLoader := projects.NewEnvLoader(projectsDirectory, filepath.Dir(composeFile), utils.BoolOrDefault(cfg.AutoInjectEnv.Value, false))+ envMap, _, _ := envLoader.LoadEnvironment(ctx)++ includes, parseErr := projects.ParseIncludes(composeFile, envMap, false)+ if parseErr == nil {+ for _, inc := range includes {+ if inc.RelativePath != normalizedRelativePath {+ continue+ }+ if !projects.IsSafeSubdirectory(proj.Path, inc.Path) {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.New("Forbidden project file path: include resolves outside project directory"))+ }+ return readProjectIncludeFileContentInternal(proj.Path, inc)+ }+ }+ }++ absFilePath, err := projects.ValidateIncludePathForWrite(proj.Path, normalizedRelativePath)+ if err != nil {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.WrapIf(err, "Forbidden project file path"))+ }++ info, err := os.Lstat(absFilePath)+ if err != nil {+ if os.IsNotExist(err) {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileNotFound, errors.New("Project file not found"))+ }+ return project.IncludeFile{}, errors.WrapIf(err, "failed to stat file")+ }+ if info.IsDir() {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: path is a directory"))+ }+ if info.Mode()&os.ModeSymlink != 0 {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.New("Forbidden project file path: symlinks are not allowed"))+ }++ content, err := os.ReadFile(absFilePath)+ if err != nil {+ if os.IsNotExist(err) {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileNotFound, errors.New("Project file not found"))+ }+ return project.IncludeFile{}, errors.WrapIf(err, "failed to read file")+ }+ if projects.IsBinaryProjectFileContent(content) {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: binary files are not supported"))+ }++ return project.IncludeFile{+ Path: absFilePath,+ RelativePath: normalizedRelativePath,+ Content: string(content),+ }, nil+}++func readProjectIncludeFileContentInternal(projectPath string, inc projects.IncludeFile) (project.IncludeFile, error) {+ validatedPath, err := projects.ValidateIncludePathForWrite(projectPath, inc.Path)+ if err != nil {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.WrapIf(err, "Forbidden project file path"))+ }++ resolvedPath, err := utils.ResolveWithinRoot(projectPath, validatedPath)+ if err != nil {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.WrapIf(err, "Forbidden project file path"))+ }++ info, err := os.Stat(resolvedPath)+ if os.IsNotExist(err) {+ return project.IncludeFile{+ Path: validatedPath,+ RelativePath: inc.RelativePath,+ Content: "# This file will be created when you save changes\nservices:\n",+ }, nil+ }+ if err != nil {+ return project.IncludeFile{}, errors.WrapIf(err, "failed to stat include file")+ }+ if info.IsDir() {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: path is a directory"))+ }++ content, err := os.ReadFile(resolvedPath)+ if err != nil {+ if os.IsNotExist(err) {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileNotFound, errors.New("Project file not found"))+ }+ return project.IncludeFile{}, errors.WrapIf(err, "failed to read include file")+ }+ if projects.IsBinaryProjectFileContent(content) {+ return project.IncludeFile{}, common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: binary files are not supported"))+ }++ return project.IncludeFile{+ Path: resolvedPath,+ RelativePath: inc.RelativePath,+ Content: string(content),+ }, nil+}++func (s *ProjectService) enrichWithIncludeFiles(ctx context.Context, composeFile string, resp *project.Details) {+ if strings.TrimSpace(composeFile) == "" {+ return+ }++ // Load environment variables so that include paths with ${VAR} references are expanded+ cfg := s.settingsService.GetSettingsOrDefaults(ctx)+ projectsDirectory, _ := projects.GetProjectsDirectory(ctx, strings.TrimSpace(cfg.ProjectsDirectory.Value))+ envLoader := projects.NewEnvLoader(projectsDirectory, filepath.Dir(composeFile), utils.BoolOrDefault(cfg.AutoInjectEnv.Value, false))+ envMap, _, _ := envLoader.LoadEnvironment(ctx)++ includes, parseErr := projects.ParseIncludes(composeFile, envMap, false)+ if parseErr == nil {+ var includeFiles []project.IncludeFile+ for _, inc := range includes {+ includeFiles = append(includeFiles, project.IncludeFile{+ Path: inc.Path,+ RelativePath: inc.RelativePath,+ })+ }+ resp.IncludeFiles = includeFiles+ } else {+ slog.WarnContext(ctx, "Failed to parse includes", "error", parseErr, "path", composeFile)+ }+}++func (s *ProjectService) enrichProjectUpdateInfoInternal(ctx context.Context, resp *project.Details) {+ if resp == nil {+ return+ }++ imageRefs := projects.ImageRefsFromComposeConfigs(resp.Services)+ if len(imageRefs) == 0 {+ imageRefs = projects.ImageRefsFromRuntimeServices(resp.RuntimeServices)+ }++ var updateInfoByRef map[string]*imagetypes.UpdateInfo+ if len(imageRefs) > 0 && s.imageService != nil {+ lookupResult, err := s.imageService.GetUpdateInfoByImageRefs(ctx, imageRefs)+ if err != nil {+ slog.WarnContext(ctx, "failed to fetch project update info", "projectID", resp.ID, "projectName", resp.Name, "error", err)+ } else {+ updateInfoByRef = lookupResult+ }+ }++ resp.UpdateInfo = buildProjectUpdateInfoSummaryInternal(imageRefs, updateInfoByRef)+}++func (s *ProjectService) enrichProjectsWithUpdateInfoInternal(+ ctx context.Context,+ projectsList []models.Project,+ details []project.Details,+) {+ if len(projectsList) == 0 || len(details) == 0 {+ return+ }++ imageRefsByProjectID := make(map[string][]string, len(projectsList))+ allImageRefs := make([]string, 0)+ cfg := s.settingsService.GetSettingsOrDefaults(ctx)++ const maxConcurrentComposeReads = 8+ type imageRefsResult struct {+ projectID string+ refs []string+ }++ sem := make(chan struct{}, maxConcurrentComposeReads)+ resultsCh := make(chan imageRefsResult, len(projectsList))++ var wg sync.WaitGroup+ for _, proj := range projectsList {+ if refs := projects.ParseImageRefsJSON(proj.ImageRefsJSON); len(refs) > 0 {+ imageRefsByProjectID[proj.ID] = refs+ allImageRefs = append(allImageRefs, refs...)+ continue+ }++ wg.Add(1)+ go func(proj models.Project) {+ defer wg.Done()++ select {+ case sem <- struct{}{}:+ case <-ctx.Done():+ return+ }+ defer func() { <-sem }()++ refs, _, err := s.getProjectImageRefsFromComposeInternal(ctx, proj, cfg)+ if err != nil {+ slog.WarnContext(ctx, "failed to resolve project image refs for update summary", "projectID", proj.ID, "projectName", proj.Name, "error", err)+ return+ }++ resultsCh <- imageRefsResult{projectID: proj.ID, refs: refs}+ }(proj)+ }++ wg.Wait()+ close(resultsCh)++ for result := range resultsCh {+ imageRefsByProjectID[result.projectID] = result.refs+ allImageRefs = append(allImageRefs, result.refs...)+ }++ var updateInfoByRef map[string]*imagetypes.UpdateInfo+ if len(allImageRefs) > 0 && s.imageService != nil {+ lookupResult, err := s.imageService.GetUpdateInfoByImageRefs(ctx, allImageRefs)+ if err != nil {+ slog.WarnContext(ctx, "failed to fetch project list update info", "error", err)+ } else {+ updateInfoByRef = lookupResult+ }+ }++ for i := range details {+ details[i].UpdateInfo = buildProjectUpdateInfoSummaryInternal(imageRefsByProjectID[details[i].ID], updateInfoByRef)+ }+}++func (s *ProjectService) getProjectImageRefsFromComposeInternal(ctx context.Context, proj models.Project, cfg *models.Settings) ([]string, []string, error) {+ composeProject, err := s.getCachedComposeProjectInternal(ctx, &proj, cfg)+ if err != nil {+ return nil, nil, errors.WrapIf(err, "load compose project")+ }++ return projects.ImageRefsFromComposeServices(composeProject.Services), projects.BuildImageRefsFromComposeProject(composeProject), nil+}++func buildProjectUpdateInfoSummaryInternal(+ imageRefs []string,+ updateInfoByRef map[string]*imagetypes.UpdateInfo,+) *project.UpdateInfo {+ imageCount := len(imageRefs)+ summary := &project.UpdateInfo{+ Status: "unknown",+ HasUpdate: false,+ ImageCount: imageCount,+ ImageRefs: append([]string(nil), imageRefs...),+ }++ if imageCount == 0 {+ return summary+ }++ var latestCheckTime *time.Time++ for _, imageRef := range imageRefs {+ info := updateInfoByRef[imageRef]+ if info == nil {+ continue+ }++ summary.CheckedImageCount+++ if info.HasUpdate {+ summary.HasUpdate = true+ summary.ImagesWithUpdates+++ summary.UpdatedImageRefs = append(summary.UpdatedImageRefs, imageRef)+ }+ if strings.TrimSpace(info.Error) != "" {+ summary.ErrorCount+++ if summary.ErrorMessage == nil {+ summary.ErrorMessage = new(strings.TrimSpace(info.Error))+ }+ }+ if !info.CheckTime.IsZero() && (latestCheckTime == nil || info.CheckTime.After(*latestCheckTime)) {+ latestCheckTime = new(info.CheckTime)+ }+ }++ summary.LastCheckedAt = latestCheckTime++ switch {+ case summary.ImagesWithUpdates > 0:+ summary.Status = "has_update"+ case summary.ErrorCount > 0:+ summary.Status = "error"+ case summary.CheckedImageCount == imageCount:+ summary.Status = "up_to_date"+ default:+ summary.Status = "unknown"+ }++ return summary+}++func (s *ProjectService) enrichWithDirectoryFiles(ctx context.Context, projectPath string, resp *project.Details) {+ if projectPath == "" {+ return+ }++ // Build set of already-shown files to skip+ shownFiles := map[string]bool{+ ".env": true,+ "compose.yaml": true,+ "compose.yml": true,+ "docker-compose.yaml": true,+ "docker-compose.yml": true,+ "podman-compose.yaml": true,+ "podman-compose.yml": true,+ }+ for _, inc := range resp.IncludeFiles {+ shownFiles[inc.RelativePath] = true+ }++ dirFiles, err := projects.ReadProjectDirectoryFiles(projectPath, shownFiles, s.config.ProjectScanMaxDepth, s.config.ProjectScanSkipDirs)+ if err != nil {+ slog.WarnContext(ctx, "Failed to scan project directory files", "error", err, "path", projectPath)+ }++ resp.DirectoryFiles = dirFiles+}++func (s *ProjectService) enrichWithProjectFiles(ctx context.Context, projectPath, composeFileName string, resp *project.Details) {+ if projectPath == "" {+ return+ }++ files, revision, truncated, err := projects.ReadProjectFileTree(projectPath, s.config.ProjectFileTreeMaxDepth, s.config.ProjectScanSkipDirs, composeFileName, projects.DefaultProjectFileTreeMaxEntries)+ if err != nil {+ slog.WarnContext(ctx, "Failed to scan project file tree", "error", err, "path", projectPath)+ return+ }++ resp.ProjectFiles = files+ resp.FileTreeRevision = revision+ resp.FileTreeTruncated = truncated+}++func (s *ProjectService) enrichWithGitOpsInfo(ctx context.Context, proj *models.Project, resp *project.Details) {+ if proj.GitOpsManagedBy != nil {+ var syncRecord models.GitOpsSync+ if err := s.db.WithContext(ctx).Preload("Repository").Where("id = ?", *proj.GitOpsManagedBy).First(&syncRecord).Error; err == nil {+ resp.LastSyncCommit = syncRecord.LastSyncCommit+ if syncRecord.Repository != nil {+ resp.GitRepositoryURL = syncRecord.Repository.URL+ }+ }+ }+}++func (s *ProjectService) enrichWithComposeServiceConfigs(ctx context.Context, proj *models.Project, composeFile string, resp *project.Details) {+ composeProj, loadErr := s.getCachedComposeProjectInternal(ctx, proj, nil)+ if loadErr != nil {+ slog.WarnContext(ctx, "failed to load compose service configs", "path", composeFile, "error", loadErr)+ return+ }++ if composeProj == nil {+ return+ }++ // Convert map to slice+ svcList := make([]composetypes.ServiceConfig, 0, len(composeProj.Services))+ hasBuildDirective := false+ for _, svc := range composeProj.Services {+ svcList = append(svcList, svc)+ if svc.Build != nil {+ hasBuildDirective = true+ }+ }+ resp.Services = svcList+ resp.HasBuildDirective = resp.HasBuildDirective || hasBuildDirective+}++func (s *ProjectService) StreamProjectLogs(ctx context.Context, projectID string, logsChan chan<- string, follow bool, tail, since string, timestamps bool) error {+ proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return err+ }++ pr, pw := io.Pipe()+ defer func() { _ = pw.Close() }()++ done := make(chan error, 2)++ // Reader goroutine: forward lines to channel+ go func() {+ // Closing the read half unblocks any pending pw.Write in ComposeLogs.+ // Without it, an abandoned tail (ctx cancel, or a >1MiB line tripping+ // bufio.ErrTooLong) wedges the writer goroutine forever and this+ // function never collects its second done value.+ defer func() { _ = pr.CloseWithError(io.ErrClosedPipe) }()++ sc := bufio.NewScanner(pr)+ sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)+ for sc.Scan() {+ select {+ case <-ctx.Done():+ done <- ctx.Err()+ return+ case logsChan <- sc.Text():+ }+ }+ done <- sc.Err()+ }()++ // Writer goroutine: compose logs -> pipe+ go func() {+ err := projects.ComposeLogs(ctx, projects.NormalizeProjectName(proj.Name), pw, follow, tail, since, timestamps)+ _ = pw.Close()+ done <- err+ }()++ // Wait for both goroutines to finish to avoid sending on a closed channel+ err1 := <-done+ err2 := <-done++ for _, e := range []error{err1, err2} {+ if e != nil && !errors.Is(e, io.EOF) && !errors.Is(e, context.Canceled) {+ return e+ }+ }+ return nil+}++func (s *ProjectService) CountServicesFromCompose(ctx context.Context, p models.Project) (int, error) {+ proj, _, err := s.loadComposeProjectForProjectInternal(ctx, &p, nil)+ if err != nil {+ return 0, err+ }++ return len(proj.Services), nil+}++func calculateProjectStatus(services []ProjectServiceInfo) models.ProjectStatus {+ if len(services) == 0 {+ return models.ProjectStatusUnknown+ }++ runningCount := 0+ stoppedCount := 0++ for _, svc := range services {+ state := strings.ToLower(strings.TrimSpace(svc.Status))+ switch state {+ case "running", "up":+ runningCount+++ case "exited", "stopped", "dead":+ stoppedCount+++ }+ }++ if runningCount == len(services) {+ return models.ProjectStatusRunning+ }+ if runningCount > 0 {+ return models.ProjectStatusPartiallyRunning+ }+ if stoppedCount > 0 {+ return models.ProjectStatusStopped+ }+ return models.ProjectStatusUnknown+}diff --git a/backend/internal/project/project_env.go b/backend/internal/project/project_env.gonew file mode 100644index 000000000..a7efa6a92--- /dev/null+++ b/backend/internal/project/project_env.go@@ -0,0 +1,243 @@+package project++import (+ "log/slog"+ "strings"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+)++type gitSyncEnvUpdateInternal struct {+ state projects.ProjectEnvState+ gitEnvContent *string+ overrideContent string+ effectiveContent *string+}++func (s *ProjectService) resolveEffectiveEnvContentForUpdateInternal(projectPath string, envContent *string) (*string, error) {+ if envContent != nil {+ return envContent, nil+ }++ state, err := projects.ReadProjectEnvState(projectPath)+ if err != nil {+ return nil, errors.WrapIf(err, "read project env state")+ }++ effectiveContent, err := resolveStoredEffectiveEnvContentInternal(state)+ if err != nil {+ return nil, err+ }+ if effectiveContent == "" && !state.HasEffective && !state.HasGitSource && !state.HasOverride {+ return nil, nil+ }++ return &effectiveContent, nil+}++func resolveStoredEffectiveEnvContentInternal(state projects.ProjectEnvState) (string, error) {+ if state.HasEffective {+ return state.EffectiveContent, nil+ }+ if state.HasGitSource || state.HasOverride {+ effectiveContent, err := projects.BuildEffectiveEnvContent(state.GitContent, state.OverrideContent)+ if err != nil {+ return "", errors.WrapIf(err, "build effective env content")+ }+ return effectiveContent, nil+ }+ return state.DirectContent, nil+}++func persistEffectiveEnvContentInternal(projectPath, projectsDirectory, envContent string) error {+ state, err := projects.ReadProjectEnvState(projectPath)+ if err != nil {+ return errors.WrapIf(err, "read project env state")+ }++ if state.HasGitSource && state.HasEffective && envContent == state.EffectiveContent {+ storedEffectiveContent, buildErr := projects.BuildEffectiveEnvContent(state.GitContent, state.OverrideContent)+ if buildErr == nil && envContent == storedEffectiveContent {+ return nil+ }+ }++ if !state.HasGitSource {+ if state.HasOverride {+ if err := projects.RemoveProjectFile(projectsDirectory, projectPath, projects.OverrideEnvFileName); err != nil {+ return err+ }+ }+ return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, state.EffectiveUnreadable, envContent)+ }++ overrideContent, err := projects.BuildOverrideEnvContent(state.GitContent, envContent)+ if err != nil {+ return errors.WrapIf(err, "build override env content")+ }++ effectiveContent, err := projects.BuildEffectiveEnvContent(state.GitContent, overrideContent)+ if err != nil {+ return errors.WrapIf(err, "build effective env content")+ }++ if err := projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, state.EffectiveUnreadable, effectiveContent); err != nil {+ return err+ }++ return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.OverrideEnvFileName, state.OverrideUnreadable, overrideContent)+}++func (s *ProjectService) ensureEffectiveEnvFileInternal(projectPath, projectsDirectory string) error {+ state, err := projects.ReadProjectEnvState(projectPath)+ if err != nil {+ return errors.WrapIf(err, "read project env state")+ }++ if !state.HasGitSource {+ if state.HasOverride {+ if err := projects.RemoveProjectFile(projectsDirectory, projectPath, projects.OverrideEnvFileName); err != nil {+ return err+ }+ effectiveContent, err := resolveStoredEffectiveEnvContentInternal(state)+ if err != nil {+ return err+ }+ return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, state.EffectiveUnreadable, effectiveContent)+ }+ return projects.EnsureEnvFile(projectsDirectory, projectPath)+ }++ effectiveContent, err := projects.BuildEffectiveEnvContent(state.GitContent, state.OverrideContent)+ if err != nil {+ return errors.WrapIf(err, "build effective env content")+ }++ return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, state.EffectiveUnreadable, effectiveContent)+}++func (s *ProjectService) prepareGitSyncEnvUpdateInternal(projectPath string, gitEnvContent *string) (gitSyncEnvUpdateInternal, error) {+ state, err := projects.ReadProjectEnvState(projectPath)+ if err != nil {+ return gitSyncEnvUpdateInternal{}, errors.WrapIf(err, "read project env state")+ }++ update := gitSyncEnvUpdateInternal{+ state: state,+ gitEnvContent: gitEnvContent,+ }++ if gitEnvContent == nil {+ effectiveContent, err := resolveStoredEffectiveEnvContentInternal(state)+ if err != nil {+ return gitSyncEnvUpdateInternal{}, err+ }+ if effectiveContent == "" && !state.HasEffective && !state.HasGitSource && !state.HasOverride {+ return update, nil+ }+ update.effectiveContent = &effectiveContent+ return update, nil+ }++ overrideContent, err := s.resolveOverrideContentForGitSyncInternal(state, *gitEnvContent)+ if err != nil {+ return gitSyncEnvUpdateInternal{}, err+ }+ update.overrideContent = overrideContent++ effectiveContent, err := projects.BuildEffectiveEnvContent(*gitEnvContent, overrideContent)+ if err != nil {+ return gitSyncEnvUpdateInternal{}, errors.WrapIf(err, "build effective env content")+ }+ update.effectiveContent = &effectiveContent++ return update, nil+}++func (s *ProjectService) resolveOverrideContentForGitSyncInternal(state projects.ProjectEnvState, gitEnvContent string) (string, error) {+ switch {+ case state.HasGitSource:+ overrideContent, err := projects.BuildOverrideEnvContent(state.GitContent, state.OverrideContent)+ if err != nil {+ return "", errors.WrapIf(err, "build override env content")+ }+ return overrideContent, nil+ case state.HasOverride:+ effectiveContent, err := resolveStoredEffectiveEnvContentInternal(state)+ if err != nil {+ return "", err+ }+ overrideContent, err := projects.BuildOverrideEnvContent(gitEnvContent, effectiveContent)+ if err != nil {+ return "", errors.WrapIf(err, "build override env content")+ }+ return overrideContent, nil+ case strings.TrimSpace(state.DirectContent) != "":+ overrideContent, err := projects.BuildAdditiveOverrideEnvContent(gitEnvContent, state.DirectContent)+ if err != nil {+ return "", errors.WrapIf(err, "build override env content")+ }+ return overrideContent, nil+ default:+ return "", nil+ }+}++func persistGitSyncEnvFilesInternal(projectPath, projectsDirectory string, update gitSyncEnvUpdateInternal) error {+ if update.gitEnvContent == nil {+ if update.state.HasGitSource {+ if err := projects.RemoveProjectFile(projectsDirectory, projectPath, projects.GitSourceEnvFileName); err != nil {+ return err+ }+ }+ if update.state.HasOverride {+ if err := projects.RemoveProjectFile(projectsDirectory, projectPath, projects.OverrideEnvFileName); err != nil {+ return err+ }+ }+ if update.effectiveContent != nil || update.state.HasEffective || update.state.HasGitSource || update.state.HasOverride {+ effectiveContent := ""+ if update.effectiveContent != nil {+ effectiveContent = *update.effectiveContent+ }+ return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, update.state.EffectiveUnreadable, effectiveContent)+ }+ if update.state.EffectiveUnreadable {+ slog.Warn("skipping permission-locked .env file; leaving it untouched", "projectPath", projectPath)+ return nil+ }+ return projects.EnsureEnvFile(projectsDirectory, projectPath)+ }++ if update.effectiveContent == nil {+ return errors.New("missing effective env content for git sync update")+ }++ if err := projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, update.state.EffectiveUnreadable, *update.effectiveContent); err != nil {+ return err+ }+ if err := projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.GitSourceEnvFileName, update.state.GitSourceUnreadable, *update.gitEnvContent); err != nil {+ return err+ }+ return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.OverrideEnvFileName, update.state.OverrideUnreadable, update.overrideContent)+}++// ApplyGitSyncEnvToDirectory applies the same managed three-file environment+// merge used by single-file project syncs and returns the effective content+// before and after the update.+func (s *ProjectService) ApplyGitSyncEnvToDirectory(projectPath, projectsDirectory string, gitEnvContent *string) (before, after string, err error) {+ update, err := s.prepareGitSyncEnvUpdateInternal(projectPath, gitEnvContent)+ if err != nil {+ return "", "", errors.WrapIf(err, "failed to resolve git env state")+ }+ before = update.state.DirectContent+ if update.effectiveContent != nil {+ after = *update.effectiveContent+ }+ if err := persistGitSyncEnvFilesInternal(projectPath, projectsDirectory, update); err != nil {+ return "", "", errors.WrapIf(err, "failed to sync git env files")+ }+ return before, after, nil+}diff --git a/backend/internal/project/project_lifecycle.go b/backend/internal/project/project_lifecycle.gonew file mode 100644index 000000000..56c7433fe--- /dev/null+++ b/backend/internal/project/project_lifecycle.go@@ -0,0 +1,997 @@+package project++import (+ "context"+ "fmt"+ "io"+ "log/slog"+ "os"+ "path/filepath"+ "strings"+ "time"++ "emperror.dev/errors"++ composetypes "github.com/compose-spec/compose-go/v2/types"+ "github.com/docker/compose/v5/pkg/api"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/types/v2"+ "github.com/getarcaneapp/arcane/types/v2/containerregistry"+ "github.com/getarcaneapp/arcane/types/v2/project"+ "github.com/moby/moby/client"+ buildtypes "go.getarcane.app/builds/types"+ "go.getarcane.app/sys/cgroup"+ "go.getarcane.app/updater/labels"+ "gorm.io/gorm"+)++type ProjectBuildOptions struct {+ Services []string+ Provider string+ Push *bool+ Load *bool+}++var (+ composeStopProjectServicesInternal = projects.ComposeStop+ composeUpProjectServicesInternal = projects.ComposeUp+)++func (s *ProjectService) reconcilePulledImageRefsInternal(ctx context.Context, imageRefs []string) {+ if s.imageService == nil {+ return+ }++ for _, imageRef := range imageRefs {+ if err := s.imageService.ReconcilePulledImageUpdate(ctx, imageRef); err != nil {+ slog.WarnContext(ctx, "failed to reconcile pulled image update state", "image", imageRef, "error", err)+ }+ }+}++func (s *ProjectService) composePullSelectedServicesInternal(+ ctx context.Context,+ compProj *composetypes.Project,+ servicesToUpdate []string,+ user models.User,+ credentials []containerregistry.Credential,+) error {+ if compProj == nil {+ return nil+ }++ imageRefsToPull := projects.SelectedImageRefs(compProj, servicesToUpdate)+ if len(imageRefsToPull) == 0 {+ return nil+ }++ progressWriter, _ := ctx.Value(dockerutil.ProgressWriterKey{}).(io.Writer)+ for _, imageRef := range imageRefsToPull {+ if err := s.pullAndReconcileImageInternal(ctx, imageRef, progressWriter, user, credentials); err != nil {+ return err+ }+ }++ return nil+}++func (s *ProjectService) pullAndReconcileImageInternal(+ ctx context.Context,+ imageRef string,+ progressWriter io.Writer,+ user models.User,+ credentials []containerregistry.Credential,+) error {+ if s == nil || s.imageService == nil {+ return errors.New("image service not available")+ }++ settings := s.settingsService.GetSettingsConfig()++ pullCtx, pullCancel := context.WithTimeout(ctx, timeouts.GetDuration(settings.DockerImagePullTimeout.AsInt(), timeouts.DefaultDockerImagePull))+ defer pullCancel()++ if err := s.imageService.PullImage(pullCtx, imageRef, progressWriter, user, credentials); err != nil {+ if errors.Is(pullCtx.Err(), context.DeadlineExceeded) {+ return errors.Errorf("image pull timed out for %s (increase DOCKER_IMAGE_PULL_TIMEOUT or setting)", imageRef)+ }+ return errors.WrapIff(err, "failed to pull image %s", imageRef)+ }++ s.reconcilePulledImageRefsInternal(ctx, []string{imageRef})+ return nil+}++func (s *ProjectService) UpdateProjectServices(ctx context.Context, projectID string, servicesToUpdate []string, user models.User) error {+ projectFromDb, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return err+ }+ previousStatus := projectFromDb.Status++ // 1. Load project+ compProj, _, err := s.loadComposeProjectForProjectInternal(ctx, projectFromDb, nil)+ if err != nil {+ return errors.WrapIf(err, "failed to load compose project")+ }++ // 2. Set status to deploying/restarting+ if err := s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusDeploying); err != nil {+ return err+ }++ credentials, err := s.ResolveRegistryCredentials(ctx)+ if err != nil {+ if statusErr := s.updateProjectStatusInternal(ctx, projectID, previousStatus); statusErr != nil {+ slog.ErrorContext(ctx, "UpdateProjectServices: failed to restore project status after credential lookup failure", "projectID", projectID, "error", statusErr)+ }+ return errors.WrapIf(err, "resolve registry credentials")+ }++ // 3. Pull images for specific services+ if err := s.composePullSelectedServicesInternal(ctx, compProj, servicesToUpdate, user, credentials); err != nil {+ if statusErr := s.updateProjectStatusInternal(ctx, projectID, previousStatus); statusErr != nil {+ slog.ErrorContext(ctx, "UpdateProjectServices: failed to restore project status after compose pull failure", "projectID", projectID, "error", statusErr)+ }+ return errors.WrapIf(err, "pull updated service images")+ }++ // 4. Stop specific services+ if err := composeStopProjectServicesInternal(ctx, compProj, servicesToUpdate); err != nil {+ slog.WarnContext(ctx, "compose stop failed, continuing", "error", err)+ }++ // 5. Up specific services+ if err := composeUpProjectServicesInternal(ctx, compProj, servicesToUpdate, false, true, false, s.composeRegistryAuthConfigsInternal(ctx)); err != nil {+ s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)+ return errors.WrapIf(err, "failed to up services")+ }++ // 6. Finalize status+ if err := s.updateProjectStatusandCountsInternal(ctx, projectID, models.ProjectStatusRunning); err != nil {+ return err+ }++ metadata := models.JSON{+ "action": "update_services",+ "projectID": projectID,+ "projectName": projectFromDb.Name,+ "services": append([]string(nil), servicesToUpdate...),+ }+ s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, projectID, projectFromDb.Name, user, metadata, "could not log project service update action")++ return nil+}++func ensureProjectMutableInternal(proj *models.Project) error {+ if proj != nil && proj.IsArchived {+ return common.Classify(common.ErrProjectArchived, errors.New("project is archived and must be unarchived before this action"))+ }+ return nil+}++func isProjectArchiveBlockedInternal(proj *models.Project) bool {+ if proj == nil {+ return false+ }+ if proj.RunningCount > 0 {+ return true+ }+ switch proj.Status {+ case models.ProjectStatusRunning, models.ProjectStatusPartiallyRunning, models.ProjectStatusDeploying, models.ProjectStatusRestarting:+ return true+ case models.ProjectStatusStopped, models.ProjectStatusUnknown, models.ProjectStatusStopping:+ return false+ default:+ return false+ }+}++func (s *ProjectService) ArchiveProject(ctx context.Context, projectID string, user models.User) error {+ proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return err+ }+ if proj.IsArchived {+ return nil+ }+ if isProjectArchiveBlockedInternal(proj) {+ return common.Classify(common.ErrProjectMustBeStopped, errors.New("project must be stopped before archiving"))+ }++ now := time.Now()+ if err := s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", projectID).Updates(map[string]any{+ "is_archived": true,+ "archived_at": now,+ }).Error; err != nil {+ return errors.WrapIf(err, "failed to archive project")+ }++ metadata := models.JSON{"action": "archived", "projectID": projectID, "projectName": proj.Name}+ s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, projectID, proj.Name, user, metadata, "could not log project archive action")++ return nil+}++func (s *ProjectService) UnarchiveProject(ctx context.Context, projectID string, user models.User) error {+ proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return err+ }+ if !proj.IsArchived {+ return nil+ }++ if err := s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", projectID).Updates(map[string]any{+ "is_archived": false,+ "archived_at": gorm.Expr("NULL"),+ }).Error; err != nil {+ return errors.WrapIf(err, "failed to unarchive project")+ }++ metadata := models.JSON{"action": "unarchived", "projectID": projectID, "projectName": proj.Name}+ s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, projectID, proj.Name, user, metadata, "could not log project unarchive action")++ return nil+}++// resolveRemoveOrphansInternal decides whether compose up should remove orphan containers.+// GitOps-managed projects always remove orphans so the running stack matches the+// tracked compose file. Non-GitOps callers may opt in per request via DeployOptions.+func resolveRemoveOrphansInternal(gitOpsManaged bool, options *project.DeployOptions) bool {+ return gitOpsManaged || (options != nil && options.RemoveOrphans)+}++func (s *ProjectService) DeployProject(ctx context.Context, projectID string, user models.User, options *project.DeployOptions) error {+ projectFromDb, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return errors.WrapIf(err, "failed to get project")+ }+ if err := ensureProjectMutableInternal(projectFromDb); err != nil {+ return err+ }++ resolvedPullPolicy := ""+ forceRecreate := false+ recreateVolumes := false+ if options != nil {+ resolvedPullPolicy = projects.NormalizeDeployPullPolicy(options.PullPolicy)+ forceRecreate = options.ForceRecreate+ recreateVolumes = options.RecreateVolumes+ }+ if resolvedPullPolicy == "" {+ resolvedPullPolicy = projects.NormalizeDeployPullPolicy(s.settingsService.GetStringSetting(ctx, "defaultDeployPullPolicy", "missing"))+ }+ if resolvedPullPolicy == "" {+ resolvedPullPolicy = "missing"+ }++ if err := s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusDeploying); err != nil {+ return errors.WrapIf(err, "failed to update project status to deploying")+ }++ // Run any configured pre-deploy lifecycle hook before loading the compose+ // project so hooks can produce files that compose then consumes (e.g.+ // `sops -d secrets.enc.env > .env.runtime` for an `env_file: .env.runtime`+ // service). A failed hook aborts the deploy.+ if s.lifecycleService != nil {+ if lerr := s.lifecycleService.RunPreDeploy(ctx, projectFromDb, user); lerr != nil {+ s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)+ return errors.WrapIf(lerr, "pre-deploy lifecycle hook failed")+ }+ }++ projectModel, _, derr := s.loadComposeProjectForProjectInternal(ctx, projectFromDb, nil)+ if derr != nil {+ s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)+ return errors.WrapIff(derr, "failed to load compose project in %s", projectFromDb.Path)+ }++ credentials, cerr := s.ResolveRegistryCredentials(ctx)+ if cerr != nil {+ s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)+ return errors.WrapIf(cerr, "resolve registry credentials")+ }++ progressWriter, _ := ctx.Value(dockerutil.ProgressWriterKey{}).(io.Writer)+ if perr := s.prepareProjectImagesForDeploy(ctx, projectID, projectModel, progressWriter, credentials, &user, resolvedPullPolicy); perr != nil {+ s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)+ return errors.WrapIf(perr, "failed to prepare project images for deploy")+ }++ gitOpsManaged := projectFromDb.GitOpsManagedBy != nil && *projectFromDb.GitOpsManagedBy != ""+ removeOrphans := resolveRemoveOrphansInternal(gitOpsManaged, options)++ slog.Info("starting compose up with health check support", "projectID", projectID, "projectName", projectModel.Name, "services", len(projectModel.Services), "removeOrphans", removeOrphans)+ // Health/progress streaming (if any) is handled inside projects.ComposeUp via ctx.+ if err := projects.ComposeUp(ctx, projectModel, nil, removeOrphans, forceRecreate, recreateVolumes, s.composeRegistryAuthConfigsInternal(ctx)); err != nil {+ slog.Error("compose up failed", "projectName", projectModel.Name, "projectID", projectID, "error", err)+ if containers, psErr := s.GetProjectServices(ctx, projectID); psErr == nil {+ slog.Info("containers after failed deploy", "projectID", projectID, "containers", containers)+ }+ s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)++ // Provide more helpful error messages+ errMsg := err.Error()+ if strings.Contains(errMsg, "timeout") || strings.Contains(errMsg, "context deadline exceeded") {+ return errors.WrapIf(err, "deployment timed out - check if services with 'condition: service_healthy' have healthchecks defined")+ }+ return errors.WrapIf(err, "failed to deploy project")+ }+ slog.Info("compose up completed successfully", "projectID", projectID, "projectName", projectModel.Name)++ metadata := models.JSON{"action": "deploy", "projectID": projectID, "projectName": projectModel.Name}+ s.logProjectEventInternal(ctx, models.EventTypeProjectDeploy, projectID, projectModel.Name, user, metadata, "could not log project deployment action")++ err = s.updateProjectStatusandCountsInternal(ctx, projectID, models.ProjectStatusRunning)+ if err != nil {+ slog.Error("failed to update project status and counts after deploy", "projectID", projectID, "error", err)+ }+ return err+}++func (s *ProjectService) DownProject(ctx context.Context, projectID string, user models.User) error {+ projectFromDb, err := s.getMutableProjectInternal(ctx, projectID)+ if err != nil {+ return err+ }++ proj, _, lerr := s.loadComposeProjectForProjectInternal(ctx, projectFromDb, nil)+ if lerr != nil {+ _ = s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusRunning)+ return errors.WrapIf(lerr, "failed to load compose project")+ }++ if err := s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusStopped); err != nil {+ return errors.WrapIf(err, "failed to update project status to stopping")+ }++ if err := projects.ComposeDown(ctx, proj, false); err != nil {+ _ = s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusRunning)+ return errors.WrapIf(err, "failed to bring down project")+ }++ metadata := models.JSON{+ "action": "down",+ "projectID": projectID,+ "projectName": projectFromDb.Name,+ }+ s.logProjectEventInternal(ctx, models.EventTypeProjectStop, projectID, projectFromDb.Name, user, metadata, "could not log project down action")++ return s.updateProjectStatusandCountsInternal(ctx, projectID, models.ProjectStatusStopped)+}++// CreateProject creates a project's directory, files, and DB row. When+// allowNameSuffix is true a directory-name collision is resolved by appending+// "-N" (the interactive default). When false a collision returns+// projects.ErrProjectDirExists (wrapped) so GitOps creates fail loudly instead of+// minting runaway "-N" duplicate projects on a broken binding.+func (s *ProjectService) CreateProject(ctx context.Context, name, composeContent string, envContent *string, projectFiles []project.ProjectFileDraft, user models.User, allowNameSuffixOptions ...bool) (*models.Project, error) {+ allowNameSuffix := true+ if len(allowNameSuffixOptions) > 0 {+ allowNameSuffix = allowNameSuffixOptions[0]+ }+ // A top-level `name:` in the compose file is authoritative over the+ // submitted project name.+ if yamlName := projects.ComposeContentProjectName(composeContent); yamlName != "" {+ name = yamlName+ }+ sanitized := projects.SanitizeProjectName(name)++ projectsDirectory, err := projects.GetProjectsDirectory(ctx, s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"))+ if err != nil {+ return nil, errors.WrapIf(err, "failed to get projects directory")+ }++ basePath := filepath.Join(projectsDirectory, sanitized)+ var projectPath, folderName string+ if allowNameSuffix {+ projectPath, folderName, err = projects.CreateUniqueDir(projectsDirectory, basePath, name, utils.DirPerm)+ } else {+ projectPath, folderName, err = projects.CreateExactDir(projectsDirectory, basePath, name, utils.DirPerm)+ }+ if err != nil {+ return nil, errors.WrapIf(err, "failed to create project directory")+ }++ proj := &models.Project{+ Name: name,+ DirName: &folderName,+ Path: projectPath,+ Status: models.ProjectStatusStopped,+ ServiceCount: 0,+ RunningCount: 0,+ }++ if err := projects.ApplyProjectFileDrafts(projectPath, projectFiles, projects.ProjectFileApplyOptions{+ MaxDepth: s.config.ProjectFileTreeMaxDepth,+ SkipDirectories: s.config.ProjectScanSkipDirs,+ ComposeFileName: projects.DefaultComposeFileName,+ }); err != nil {+ _ = os.RemoveAll(projectPath)+ return nil, wrapProjectFileErrorInternal(err)+ }++ // GitOps-originated creates (allowNameSuffix=false) tolerate not-yet-supplied+ // ${VAR} references the same way single-file git sync updates do; interactive+ // creates (allowNameSuffix=true) stay strict.+ if err := validateComposeContentForUpdate(ctx, projectsDirectory, projectPath, name, composeContent, envContent, nil, "", !allowNameSuffix); err != nil {+ _ = os.RemoveAll(projectPath)+ return nil, errors.WrapIf(err, "invalid compose file")+ }++ if err := projects.WriteProjectFiles(projectsDirectory, projectPath, composeContent, envContent); err != nil {+ // Best-effort cleanup to restore pre-transaction behavior.+ _ = os.RemoveAll(projectPath)+ return nil, errors.WrapIf(err, "failed to save project files")+ }++ if err := s.db.WithContext(ctx).Create(proj).Error; err != nil {+ _ = os.RemoveAll(projectPath)+ return nil, errors.WrapIf(err, "failed to create project")+ }+ s.refreshComposeProjectNameInternal(ctx, proj)+ s.refreshProjectImageRefsInternal(ctx, proj)++ metadata := models.JSON{"action": "create", "projectID": proj.ID, "projectName": proj.Name, "path": projectPath}+ s.logProjectEventInternal(ctx, models.EventTypeProjectCreate, proj.ID, proj.Name, user, metadata, "could not log project creation")++ return proj, nil+}++func (s *ProjectService) DestroyProject(ctx context.Context, projectID string, removeFiles bool, removeVolumes bool, user models.User) error {+ slog.DebugContext(ctx, "DestroyProject service called",+ "projectID", projectID,+ "removeFiles", removeFiles,+ "removeVolumes", removeVolumes,+ "userID", user.ID,+ "username", user.Username)++ proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)+ if err != nil {+ return err+ }++ slog.DebugContext(ctx, "Found project to destroy",+ "projectName", proj.Name,+ "projectPath", proj.Path)++ if err := s.DownProject(ctx, projectID, models.SystemUser); err != nil {+ slog.WarnContext(ctx, "failed to bring down project", "error", err)+ }++ if removeVolumes {+ if compProj, _, lerr := s.loadComposeProjectForProjectInternal(ctx, proj, nil); lerr == nil {+ if derr := projects.ComposeDown(ctx, compProj, true); derr != nil {+ slog.WarnContext(ctx, "failed to remove volumes", "error", derr)+ }+ } else {+ slog.WarnContext(ctx, "failed to load compose project for volume removal", "error", lerr)+ }+ }++ if removeFiles {+ slog.DebugContext(ctx, "Removing project files", "path", proj.Path)+ if err := os.RemoveAll(proj.Path); err != nil {+ slog.ErrorContext(ctx, "Failed to remove project files", "path", proj.Path, "error", err)+ return errors.WrapIf(err, "failed to remove project files")+ }+ slog.InfoContext(ctx, "Project files removed successfully", "path", proj.Path)+ }++ if err := s.db.WithContext(ctx).Delete(proj).Error; err != nil {+ return errors.WrapIf(err, "failed to delete project from database")+ }++ if !removeFiles {+ if projectsDir, dirErr := s.GetProjectsDirectory(ctx); dirErr != nil {+ slog.WarnContext(ctx, "Failed to resolve projects directory for quarantine", "error", dirErr)+ } else if projects.IsSafeSubdirectory(projectsDir, proj.Path) && filepath.Clean(projectsDir) != filepath.Clean(proj.Path) {+ trashPath := filepath.Join(filepath.Dir(proj.Path), fmt.Sprintf("%s%s-%d", projects.ArcaneTrashPrefix, filepath.Base(proj.Path), time.Now().Unix()))+ if err := os.Rename(proj.Path, trashPath); err != nil {+ slog.WarnContext(ctx, "Failed to quarantine project files", "path", proj.Path, "trashPath", trashPath, "error", err)+ } else {+ slog.InfoContext(ctx, "Project files quarantined successfully", "path", proj.Path, "trashPath", trashPath)+ }+ }+ }+ s.parsedCompose.invalidate(projectID)++ metadata := models.JSON{"action": "destroy", "projectID": projectID, "projectName": proj.Name, "removeFiles": removeFiles, "removeVolumes": removeVolumes}+ s.logProjectEventInternal(ctx, models.EventTypeProjectDelete, projectID, proj.Name, user, metadata, "could not log project destroy action")++ return nil+}++func (s *ProjectService) RedeployProject(ctx context.Context, projectID string, user models.User, options *project.DeployOptions) error {+ proj, err := s.getMutableProjectInternal(ctx, projectID)+ if err != nil {+ return err+ }++ disabled := projectRedeployDisabledInternal(ctx, *proj)+ if disabled {+ return errors.New("arcane cannot redeploy itself; use the system upgrade flow (Settings -> Updates) instead")+ }++ progressWriter, _ := ctx.Value(dockerutil.ProgressWriterKey{}).(io.Writer)+ if progressWriter == nil {+ progressWriter = io.Discard+ }++ credentials, cerr := s.ResolveRegistryCredentials(ctx)+ if cerr != nil {+ slog.WarnContext(ctx, "failed to resolve registry credentials for redeploy pull", "error", cerr)+ }+ if err := s.PullProjectImages(ctx, projectID, progressWriter, user, credentials); err != nil {+ slog.WarnContext(ctx, "failed to pull project images", "error", err)+ }++ metadata := models.JSON{"action": "redeploy", "projectID": projectID, "projectName": proj.Name}+ s.logProjectEventInternal(ctx, models.EventTypeProjectDeploy, projectID, proj.Name, user, metadata, "could not log project redeploy action")++ return s.DeployProject(ctx, projectID, user, options)+}++func projectRedeployDisabledInternal(ctx context.Context, proj models.Project) bool {+ containers, err := projects.ListGlobalComposeContainers(ctx)+ if err != nil {+ slog.WarnContext(ctx, "could not list compose containers to check self-redeploy guard; skipping guard", "error", err)+ return false+ }++ containersByProject := groupComposeContainersByProjectInternal(containers)++ currentContainerID, currentContainerErr := cgroup.CurrentContainerID()+ for _, containerSummary := range lookupProjectContainers(proj, containersByProject) {+ if labels.ShouldDisableArcaneServerRedeploy(containerSummary.Labels, containerSummary.ID, currentContainerID, currentContainerErr) {+ return true+ }+ }++ return false+}++func (s *ProjectService) PullProjectImages(ctx context.Context, projectID string, progressWriter io.Writer, user models.User, credentials []containerregistry.Credential) error {+ proj, err := s.getMutableProjectInternal(ctx, projectID)+ if err != nil {+ return err+ }++ compProj, _, lerr := s.loadComposeProjectForProjectInternal(ctx, proj, nil)+ if lerr != nil {+ return errors.WrapIf(lerr, "failed to load compose project")+ }++ images := map[string]struct{}{}+ for _, svc := range compProj.Services {+ if svc.Build == nil {+ if img := strings.TrimSpace(svc.Image); img != "" {+ images[img] = struct{}{}+ }+ }+ // pre_start hook images and type:image volume sources are pulled from a+ // registry even for build services — they cannot be built.+ for _, img := range api.GetDependentImages(svc, compProj.Name) {+ images[img] = struct{}{}+ }+ for _, vol := range svc.Volumes {+ if vol.Type == composetypes.VolumeTypeImage && strings.TrimSpace(vol.Source) != "" {+ images[strings.TrimSpace(vol.Source)] = struct{}{}+ }+ }+ }++ for img := range images {+ if err := s.pullAndReconcileImageInternal(ctx, img, progressWriter, user, credentials); err != nil {+ return err+ }+ }+ return nil+}++func (s *ProjectService) BuildProjectServices(ctx context.Context, projectID string, options ProjectBuildOptions, progressWriter io.Writer, user *models.User) error {+ projectFromDb, err := s.getMutableProjectInternal(ctx, projectID)+ if err != nil {+ return err+ }++ projectModel, _, derr := s.loadComposeProjectForProjectInternal(ctx, projectFromDb, nil)+ if derr != nil {+ return errors.WrapIff(derr, "failed to load compose project in %s", projectFromDb.Path)+ }++ return s.buildProjectServicesInternal(ctx, projectID, projectModel, options, progressWriter, user)+}++// EnsureProjectImagesPresent checks all compose service images for the project and+// pulls based on service pull policy:+// - always/refresh: always pull+// - missing/if_not_present/default: pull only if local image is missing+// - never: never pull (fails early if image is missing locally)+func (s *ProjectService) EnsureProjectImagesPresent(ctx context.Context, projectID string, progressWriter io.Writer, user models.User, credentials []containerregistry.Credential) error {+ proj, err := s.getMutableProjectInternal(ctx, projectID)+ if err != nil {+ return err+ }++ compProj, _, lerr := s.loadComposeProjectForProjectInternal(ctx, proj, nil)+ if lerr != nil {+ return errors.WrapIf(lerr, "failed to load compose project")+ }++ pullPlan := projects.BuildImagePullPlan(compProj)++ return s.ensureImagesPresent(ctx, pullPlan, progressWriter, credentials, user)+}++func (s *ProjectService) ensureImagesPresent(ctx context.Context, pullPlan map[string]projects.ImagePullMode, progressWriter io.Writer, credentials []containerregistry.Credential, user models.User) error {+ for img, mode := range pullPlan {+ exists, ierr := s.imageService.ImageExistsLocally(ctx, img)+ if ierr != nil && mode != projects.ImagePullModeAlways {+ slog.WarnContext(ctx, "failed to check local image existence", "image", img, "error", ierr)+ // Non-fatal: attempt to pull to be safe+ }++ if mode == projects.ImagePullModeNever {+ if ierr != nil {+ slog.WarnContext(ctx, "pull_policy is 'never' but image presence check failed; continuing without pull", "image", img, "error", ierr)+ continue+ }+ if !exists {+ return errors.Errorf("image %s is not available locally and pull_policy is 'never'", img)+ }+ slog.DebugContext(ctx, "pull_policy is 'never'; using local image without pull", "image", img)+ continue+ }++ if mode == projects.ImagePullModeIfMissing && exists {+ slog.DebugContext(ctx, "image already present locally; skipping pull", "image", img)+ continue+ }++ if err := s.pullAndReconcileImageInternal(ctx, img, progressWriter, user, credentials); err != nil {+ return err+ }+ }+ return nil+}++func (s *ProjectService) prepareProjectImagesForDeploy(+ ctx context.Context,+ projectID string,+ project *composetypes.Project,+ progressWriter io.Writer,+ credentials []containerregistry.Credential,+ user *models.User,+ pullPolicyOverride string,+) error {+ if project == nil {+ return nil+ }++ for name, svc := range project.Services {+ svc, imageName, updated := projects.PrepareDeployServiceConfig(projectID, project.Name, name, svc)+ if updated {+ project.Services[name] = svc+ }++ if imageName == "" {+ continue+ }++ decision := projects.DecideDeployImageAction(svc, pullPolicyOverride)+ if updated {+ decision = projects.DeployImageDecision{Build: true}+ }+ if err := s.ensureDeployServiceImageReady(ctx, projectID, project, name, svc, imageName, decision, progressWriter, credentials, user); err != nil {+ return err+ }++ // pre_start hook images and type:image volume sources cannot be built:+ // they follow the service's pull decision minus the build/fallback paths.+ dependentDecision := projects.DeployImageDecision{PullIfMissing: true}+ switch {+ case decision.RequireLocalOnly:+ dependentDecision = projects.DeployImageDecision{RequireLocalOnly: true}+ case decision.PullAlways:+ dependentDecision = projects.DeployImageDecision{PullAlways: true}+ }+ dependentImages := api.GetDependentImages(svc, project.Name)+ for _, vol := range svc.Volumes {+ if vol.Type == composetypes.VolumeTypeImage && strings.TrimSpace(vol.Source) != "" {+ dependentImages = append(dependentImages, strings.TrimSpace(vol.Source))+ }+ }+ for _, img := range dependentImages {+ if err := s.ensureDeployServiceImageReady(ctx, projectID, project, name, svc, img, dependentDecision, progressWriter, credentials, user); err != nil {+ return err+ }+ }+ }++ return nil+}++func (s *ProjectService) ensureDeployServiceImageReady(+ ctx context.Context,+ projectID string,+ project *composetypes.Project,+ serviceName string,+ svc composetypes.ServiceConfig,+ imageName string,+ decision projects.DeployImageDecision,+ progressWriter io.Writer,+ credentials []containerregistry.Credential,+ user *models.User,+) error {+ if decision.Build {+ return s.buildServiceImageForDeploy(ctx, projectID, project, serviceName, svc, progressWriter, user)+ }++ exists, err := s.imageService.ImageExistsLocally(ctx, imageName)+ if err != nil {+ slog.WarnContext(ctx, "failed to check local image existence", "image", imageName, "error", err)+ }++ if decision.RequireLocalOnly {+ if !exists {+ return errors.Errorf("image %s is not available locally and pull_policy is set to never", imageName)+ }+ return nil+ }++ if !projects.ShouldPullDeployImage(decision, exists) {+ return nil+ }++ err = s.pullAndReconcileImageInternal(ctx, imageName, progressWriter, models.SystemUser, credentials)+ if err == nil {+ return nil+ }+ if svc.Build != nil && decision.FallbackBuildOnPullFail {+ slog.WarnContext(ctx, "image pull failed, falling back to build", "service", serviceName, "image", imageName, "error", err)+ return s.buildServiceImageForDeploy(ctx, projectID, project, serviceName, svc, progressWriter, user)+ }+ return errors.WrapIff(err, "failed to pull image %s", imageName)+}++func (s *ProjectService) buildServiceImageForDeploy(+ ctx context.Context,+ projectID string,+ project *composetypes.Project,+ serviceName string,+ svc composetypes.ServiceConfig,+ progressWriter io.Writer,+ user *models.User,+) error {+ if s.buildService == nil {+ return errors.Errorf("build service not available for service %s", serviceName)+ }++ buildReq, updatedSvc, updated, err := s.prepareServiceBuildRequest(ctx, projectID, project, serviceName, svc, ProjectBuildOptions{})+ if err != nil {+ return err+ }+ if updated {+ project.Services[serviceName] = updatedSvc+ }++ if _, err := s.buildService.BuildImage(ctx, types.LocalDockerEnvironmentID, buildReq, progressWriter, serviceName, user); err != nil {+ return err+ }++ return nil+}++func (s *ProjectService) resolveEffectiveBuildProvider(override string) string {+ provider := strings.ToLower(strings.TrimSpace(override))+ if provider != "" {+ return provider+ }++ if s.buildService != nil {+ provider = strings.ToLower(strings.TrimSpace(s.buildService.BuildSettings().BuildProvider))+ }++ if provider == "" {+ provider = "local"+ }++ return provider+}++func (s *ProjectService) prepareServiceBuildRequest(+ ctx context.Context,+ projectID string,+ project *composetypes.Project,+ serviceName string,+ svc composetypes.ServiceConfig,+ options ProjectBuildOptions,+) (buildtypes.BuildRequest, composetypes.ServiceConfig, bool, error) {+ _ = ctx+ imageName, updatedSvc, updated := projects.EnsureServiceImage(projectID, project.Name, serviceName, svc)+ effectiveProvider := s.resolveEffectiveBuildProvider(options.Provider)++ if updated && effectiveProvider == "depot" {+ return buildtypes.BuildRequest{}, updatedSvc, updated, errors.Errorf("service %s must define an image when using depot build provider", serviceName)+ }+ if updated && options.Push != nil && *options.Push {+ return buildtypes.BuildRequest{}, updatedSvc, updated, errors.Errorf("service %s must define an image when push is enabled", serviceName)+ }++ // The build context (and any absolute Dockerfile path) is read locally by+ // Arcane — both the docker provider (`archive.TarWithOptions`) and the+ // buildkit provider (`SolveOpt.LocalDirs`) stream the directory contents+ // to the daemon from the Arcane process's own filesystem. It must+ // therefore stay as a container path; translating it to the host path+ // (which is what bind mount sources need) makes `os.Stat` fail because+ // the host path doesn't exist inside the Arcane container. See #2314.+ // For that reason the build context dir is deliberately left untranslated.+ contextDir, err := projects.ResolveBuildContext(project.WorkingDir, updatedSvc, serviceName)+ if err != nil {+ return buildtypes.BuildRequest{}, updatedSvc, updated, err+ }++ dockerfileInline := updatedSvc.Build.DockerfileInline+ if strings.TrimSpace(updatedSvc.Build.Dockerfile) != "" && strings.TrimSpace(dockerfileInline) != "" {+ return buildtypes.BuildRequest{}, updatedSvc, updated, errors.Errorf("service %s cannot define both dockerfile and dockerfile_inline", serviceName)+ }++ dockerfilePath := ""+ if strings.TrimSpace(dockerfileInline) == "" {+ dockerfilePath = projects.ResolveDockerfilePath(updatedSvc)+ }++ buildReq := buildtypes.BuildRequest{+ ContextDir: contextDir,+ Dockerfile: dockerfilePath,+ DockerfileInline: dockerfileInline,+ Tags: projects.MergeBuildTags(imageName, updatedSvc.Build.Tags),+ Target: strings.TrimSpace(updatedSvc.Build.Target),+ BuildArgs: projects.BuildArgsFromCompose(updatedSvc.Build.Args),+ Labels: projects.LabelsFromCompose(updatedSvc.Build.Labels),+ CacheFrom: append([]string(nil), updatedSvc.Build.CacheFrom...),+ CacheTo: append([]string(nil), updatedSvc.Build.CacheTo...),+ NoCache: updatedSvc.Build.NoCache,+ Pull: updatedSvc.Build.Pull,+ Network: strings.TrimSpace(updatedSvc.Build.Network),+ Isolation: strings.TrimSpace(updatedSvc.Build.Isolation),+ ShmSize: int64(updatedSvc.Build.ShmSize),+ Ulimits: projects.UlimitsFromCompose(updatedSvc.Build.Ulimits),+ Entitlements: append(+ []string(nil),+ updatedSvc.Build.Entitlements...,+ ),+ Privileged: updatedSvc.Build.Privileged,+ ExtraHosts: updatedSvc.Build.ExtraHosts.AsList(":"),+ Platforms: projects.BuildPlatformsFromCompose(updatedSvc),+ Provider: effectiveProvider,+ }+ if options.Push != nil {+ buildReq.Push = *options.Push+ }+ if options.Load != nil {+ buildReq.Load = *options.Load+ }++ return buildReq, updatedSvc, updated, nil+}++func (s *ProjectService) restoreProjectStatusAfterFailedDeployInternal(ctx context.Context, projectID string) {+ services, err := s.GetProjectServices(ctx, projectID)+ if err == nil {+ serviceCount, runningCount := getServiceCounts(services)+ status := calculateProjectStatus(services)+ updateErr := s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", projectID).Updates(map[string]any{+ "status": status,+ "service_count": serviceCount,+ "running_count": runningCount,+ "updated_at": time.Now(),+ }).Error+ if updateErr == nil {+ return+ }+ slog.WarnContext(ctx, "failed to restore project status after deploy failure", "projectID", projectID, "error", updateErr)+ } else {+ slog.WarnContext(ctx, "failed to inspect project services after deploy failure", "projectID", projectID, "error", err)+ }++ if updateErr := s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusStopped); updateErr != nil {+ slog.WarnContext(ctx, "failed to set stopped status after deploy failure", "projectID", projectID, "error", updateErr)+ }+}++func (s *ProjectService) buildProjectServicesInternal(ctx context.Context, projectID string, project *composetypes.Project, options ProjectBuildOptions, progressWriter io.Writer, user *models.User) error {+ if s.buildService == nil {+ return nil+ }+ if project == nil {+ return nil+ }++ selected := projects.NormalizeBuildSelections(options.Services)++ buildCount := 0+ for name, svc := range project.Services {+ if svc.Build == nil {+ continue+ }+ if !projects.ServiceSelected(selected, name) {+ continue+ }++ buildReq, updatedSvc, updated, err := s.prepareServiceBuildRequest(ctx, projectID, project, name, svc, options)+ if err != nil {+ return err+ }+ if updated {+ project.Services[name] = updatedSvc+ }++ buildCount+++ if _, err := s.buildService.BuildImage(ctx, types.LocalDockerEnvironmentID, buildReq, progressWriter, name, user); err != nil {+ return err+ }+ }++ if buildCount == 0 && len(selected) > 0 {+ return errors.Errorf("no build-enabled services matched: %s", strings.Join(options.Services, ", "))+ }++ return nil+}++func (s *ProjectService) RestartProject(ctx context.Context, projectID string, services []string, user models.User) error {+ proj, err := s.getMutableProjectInternal(ctx, projectID)+ if err != nil {+ return err+ }++ if err := s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusRestarting); err != nil {+ return errors.WrapIf(err, "failed to update project status to restarting")+ }++ // Get configured projects directory from settings+ cfg := s.settingsService.GetSettingsOrDefaults(ctx)+ projectsDirectory := getProjectsDirectoryOrDefaultInternal(ctx, cfg)++ var dockerClient *client.Client+ if s.dockerService != nil {+ dockerClient, _ = s.dockerService.GetClient(ctx)+ }+ pathMapper := projects.NewPathMapperForConfiguredDirectory(+ ctx,+ s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"),+ "/app/data/projects",+ dockerClient,+ )++ compProj, _, lerr := projects.LoadComposeProjectFromDir(ctx, proj.Path, projects.NormalizeProjectName(proj.Name), projectsDirectory, utils.BoolOrDefault(cfg.AutoInjectEnv.Value, false), pathMapper)+ if lerr != nil {+ _ = s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusRunning)+ return errors.WrapIf(lerr, "failed to load compose project")+ }++ if err := projects.ComposeRestart(ctx, compProj, services); err != nil {+ _ = s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusRunning)+ return errors.WrapIf(err, "failed to restart project")+ }++ metadata := models.JSON{+ "action": "restart",+ "projectID": projectID,+ "projectName": proj.Name,+ }+ if len(services) > 0 {+ metadata["services"] = append([]string(nil), services...)+ }+ s.logProjectEventInternal(ctx, models.EventTypeProjectStart, projectID, proj.Name, user, metadata, "could not log project restart action")++ return s.updateProjectStatusandCountsInternal(ctx, projectID, models.ProjectStatusRunning)+}diff --git a/backend/internal/project/project_listing.go b/backend/internal/project/project_listing.gonew file mode 100644index 000000000..ab6950876--- /dev/null+++ b/backend/internal/project/project_listing.go@@ -0,0 +1,932 @@+package project++import (+ "context"+ "log/slog"+ "maps"+ "os"+ "strings"+ "time"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/iconcatalog"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper"+ imagetypes "github.com/getarcaneapp/arcane/types/v2/image"+ "github.com/getarcaneapp/arcane/types/v2/project"+ "github.com/moby/moby/api/types/container"+ "github.com/samber/mo"+ "go.getarcane.app/sys/cgroup"+ "go.getarcane.app/updater/labels"+ "gorm.io/gorm"+)++func groupComposeContainersByProjectInternal(containers []container.Summary) map[string][]container.Summary {+ containersByProject := make(map[string][]container.Summary)+ for _, c := range containers {+ projectName := dockerutil.ComposeProjectLabel(c.Labels)+ if projectName != "" {+ containersByProject[projectName] = append(containersByProject[projectName], c)+ }+ }+ return containersByProject+}++// lookupProjectContainers returns containers matched to a project, trying the+// normalized directory name first and falling back to the effective compose+// project name (from COMPOSE_PROJECT_NAME) when it differs.+func lookupProjectContainers(p models.Project, containersByProject map[string][]container.Summary) []container.Summary {+ normName := projects.NormalizeProjectName(p.Name)+ if c := containersByProject[normName]; len(c) > 0 {+ return c+ }+ if p.ComposeProjectName != nil && *p.ComposeProjectName != normName {+ return containersByProject[*p.ComposeProjectName]+ }+ return nil+}++func (s *ProjectService) ListAllProjects(ctx context.Context) ([]models.Project, error) {+ var items []models.Project+ if err := s.db.WithContext(ctx).Find(&items).Error; err != nil {+ return nil, errors.WrapIf(err, "list projects")+ }+ return items, nil+}++func (s *ProjectService) countProjectFolders(ctx context.Context) (int, error) {+ followProjectSymlinks := s.settingsService.GetBoolSetting(ctx, "followProjectSymlinks", false)+ projectsDir, err := s.GetProjectsDirectory(ctx)+ if err != nil {+ return 0, errors.WrapIf(err, "could not determine projects directory")+ }++ info, statErr := os.Stat(projectsDir)+ if os.IsNotExist(statErr) {+ // Directory missing, treat as zero+ return 0, nil+ }+ if statErr != nil {+ return 0, errors.WrapIff(statErr, "unable to access projects directory %s", projectsDir)+ }+ if !info.IsDir() {+ return 0, nil+ }++ discoveredProjects, discoveryErr := projects.DiscoverProjectDirectories(projectsDir, followProjectSymlinks, s.config.ProjectScanMaxDepth)+ if discoveryErr != nil {+ return 0, errors.WrapIff(discoveryErr, "failed to discover project directories in %s", projectsDir)+ }++ return len(discoveredProjects), nil+}++func incrementStatusCounts(status models.ProjectStatus, running, stopped *int) {+ switch status {+ case models.ProjectStatusRunning, models.ProjectStatusPartiallyRunning, models.ProjectStatusDeploying, models.ProjectStatusRestarting:+ *running+++ case models.ProjectStatusStopped, models.ProjectStatusStopping:+ *stopped+++ case models.ProjectStatusUnknown:+ // Don't count unknown+ }+}++func (s *ProjectService) GetProjectStatusCounts(ctx context.Context) (folderCount, runningProjects, stoppedProjects, totalProjects, archivedProjects int, err error) {+ folderCount, _ = s.countProjectFolders(ctx)++ var projectsList []models.Project+ if err := s.db.WithContext(ctx).Find(&projectsList).Error; err != nil {+ return folderCount, 0, 0, 0, 0, errors.WrapIf(err, "failed to list projects")+ }++ totalProjects = len(projectsList)+ runningProjects = 0+ stoppedProjects = 0+ activeProjects := make([]models.Project, 0, len(projectsList))+ for _, p := range projectsList {+ if p.IsArchived {+ archivedProjects+++ continue+ }+ activeProjects = append(activeProjects, p)+ }++ // 1. Fetch all compose containers+ containers, err := projects.ListGlobalComposeContainers(ctx)+ if err != nil {+ slog.ErrorContext(ctx, "Failed to list global compose containers for counts", "error", err)+ // Fallback to DB status+ for _, p := range activeProjects {+ incrementStatusCounts(p.Status, &runningProjects, &stoppedProjects)+ }+ return folderCount, runningProjects, stoppedProjects, totalProjects, archivedProjects, nil+ }++ // 2. Group by project+ containersByProject := groupComposeContainersByProjectInternal(containers)++ // 3. Calculate status for each project+ for _, p := range activeProjects {+ projectContainers := lookupProjectContainers(p, containersByProject)++ // Convert to ProjectServiceInfo (minimal needed for calculateProjectStatus)+ var services []ProjectServiceInfo+ for _, c := range projectContainers {+ services = append(services, ProjectServiceInfo{+ Status: string(c.State),+ })+ }++ var status models.ProjectStatus+ if len(services) == 0 {+ status = models.ProjectStatusStopped+ } else {+ status = calculateProjectStatus(services)+ }++ incrementStatusCounts(status, &runningProjects, &stoppedProjects)+ }++ return folderCount, runningProjects, stoppedProjects, totalProjects, archivedProjects, nil+}++func (s *ProjectService) ListProjects(ctx context.Context, params pagination.QueryParams) ([]project.Details, pagination.Response, error) {+ query := s.db.WithContext(ctx).Model(&models.Project{})+ statusFilter := ""+ updatesFilter := ""+ archivedFilter := ""+ if params.Filters != nil {+ statusFilter = strings.TrimSpace(params.Filters["status"])+ updatesFilter = strings.TrimSpace(params.Filters["updates"])+ archivedFilter = strings.TrimSpace(params.Filters["archived"])+ }+ query = applyProjectArchivedDBFilterInternal(query, archivedFilter)+ if statusFilter != "" || updatesFilter != "" {+ return s.listProjectsWithDerivedFiltersInternal(ctx, params, query)+ }++ if term := strings.TrimSpace(params.Search); term != "" {+ searchPattern := "%" + term + "%"+ query = query.Where(+ "name LIKE ? OR path LIKE ? OR status LIKE ? OR COALESCE(dir_name, '') LIKE ?",+ searchPattern, searchPattern, searchPattern, searchPattern,+ )+ }++ query = pagination.ApplyFilter(query, "status", params.Filters["status"])++ var projectsArray []models.Project+ paginationResp, err := pagination.PaginateAndSortDB(params, query, &projectsArray)+ if err != nil {+ return nil, pagination.Response{}, errors.WrapIf(err, "failed to paginate projects")+ }++ slog.DebugContext(ctx, "Retrieved projects from database",+ "count", len(projectsArray))++ // Fetch live status concurrently for all projects+ result := s.fetchProjectStatusConcurrently(ctx, projectsArray)+ s.enrichProjectsWithUpdateInfoInternal(ctx, projectsArray, result)++ slog.DebugContext(ctx, "Completed ListProjects request",+ "result_count", len(result))++ return result, paginationResp, nil+}++func applyProjectArchivedDBFilterInternal(query *gorm.DB, filterValue string) *gorm.DB {+ switch strings.ToLower(strings.TrimSpace(filterValue)) {+ case "true":+ return query.Where("is_archived = ?", true)+ case "all":+ return query+ default:+ return query.Where("is_archived = ?", false)+ }+}++func (s *ProjectService) listProjectsWithDerivedFiltersInternal(+ ctx context.Context,+ params pagination.QueryParams,+ query *gorm.DB,+) ([]project.Details, pagination.Response, error) {+ limit := params.Limit+ switch {+ case limit == -1:+ // Public API contract: exact -1 means "all" (used by the table page-size selector).+ case limit <= 0:+ limit = 20+ case limit > 100:+ limit = 100+ }+ params.Limit = limit++ result, err := s.filterProjectsWithDerivedFiltersInternal(ctx, params, query)+ if err != nil {+ return nil, pagination.Response{}, err+ }+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)++ return result.Items, paginationResp, nil+}++func (s *ProjectService) filterProjectsWithDerivedFiltersInternal(+ ctx context.Context,+ params pagination.QueryParams,+ query *gorm.DB,+) (pagination.FilterResult[project.Details], error) {+ var projectsArray []models.Project+ if term := strings.TrimSpace(params.Search); term != "" {+ searchPattern := "%" + term + "%"+ query = query.Where(+ "name LIKE ? OR path LIKE ? OR status LIKE ? OR COALESCE(dir_name, '') LIKE ?",+ searchPattern, searchPattern, searchPattern, searchPattern,+ )+ }+ if err := query.Find(&projectsArray).Error; err != nil {+ return pagination.FilterResult[project.Details]{}, errors.WrapIf(err, "failed to list projects")+ }++ items := s.fetchProjectStatusConcurrently(ctx, projectsArray)+ s.enrichProjectsWithUpdateInfoInternal(ctx, projectsArray, items)+ items = s.appendDiscoveredComposeProjectUpdatesInternal(ctx, params, projectsArray, items)++ return pagination.SearchOrderAndPaginate(items, params, s.buildProjectDerivedPaginationConfigInternal()), nil+}++func (s *ProjectService) appendDiscoveredComposeProjectUpdatesInternal(+ ctx context.Context,+ params pagination.QueryParams,+ projectsArray []models.Project,+ items []project.Details,+) []project.Details {+ if !shouldIncludeDiscoveredComposeProjectUpdatesInternal(params) {+ return items+ }++ composeContainers, err := projects.ListGlobalComposeContainers(ctx)+ if err != nil {+ slog.WarnContext(ctx, "failed to list compose containers for project update rows", "error", err)+ return items+ }++ knownProjectNames := s.buildKnownComposeProjectNameSetInternal(ctx, projectsArray)+ discovered := buildDiscoveredComposeProjectUpdateRowsInternal(ctx, composeContainers, knownProjectNames, s.imageService, IconCatalogForContext(ctx))+ if len(discovered) == 0 {+ return items+ }++ return append(items, discovered...)+}++func shouldIncludeDiscoveredComposeProjectUpdatesInternal(params pagination.QueryParams) bool {+ if params.Filters == nil {+ return false+ }++ return strings.EqualFold(strings.TrimSpace(params.Filters["updates"]), "has_update")+}++func (s *ProjectService) buildKnownComposeProjectNameSetInternal(ctx context.Context, projectsArray []models.Project) map[string]struct{} {+ known := make(map[string]struct{}, len(projectsArray)*2)+ for _, proj := range projectsArray {+ addKnownComposeProjectNameInternal(known, proj.Name)+ if proj.ComposeProjectName != nil {+ addKnownComposeProjectNameInternal(known, *proj.ComposeProjectName)+ }+ }++ if s.db == nil {+ return known+ }++ var allProjects []models.Project+ if err := s.db.WithContext(ctx).Select("name", "compose_project_name").Find(&allProjects).Error; err != nil {+ slog.WarnContext(ctx, "failed to load known project names for compose update discovery", "error", err)+ return known+ }++ for _, proj := range allProjects {+ addKnownComposeProjectNameInternal(known, proj.Name)+ if proj.ComposeProjectName != nil {+ addKnownComposeProjectNameInternal(known, *proj.ComposeProjectName)+ }+ }++ return known+}++func addKnownComposeProjectNameInternal(known map[string]struct{}, name string) {+ name = strings.TrimSpace(name)+ if name == "" {+ return+ }++ known[name] = struct{}{}+ if normalized := projects.NormalizeProjectName(name); normalized != "" {+ known[normalized] = struct{}{}+ }+}++func buildDiscoveredComposeProjectUpdateRowsInternal(+ ctx context.Context,+ composeContainers []container.Summary,+ knownProjectNames map[string]struct{},+ imageService *image.ImageService,+ iconCatalog string,+) []project.Details {+ containersByProject := make(map[string][]container.Summary)+ for _, c := range composeContainers {+ projectName := dockerutil.ComposeProjectLabel(c.Labels)+ if projectName == "" {+ continue+ }+ if _, exists := knownProjectNames[projectName]; exists {+ continue+ }+ if normalized := projects.NormalizeProjectName(projectName); normalized != "" {+ if _, exists := knownProjectNames[normalized]; exists {+ continue+ }+ }++ containersByProject[projectName] = append(containersByProject[projectName], c)+ }++ if len(containersByProject) == 0 {+ return nil+ }++ updateInfoByRef := getRuntimeContainerUpdateInfoByRefInternal(ctx, composeContainers, imageService)+ rows := make([]project.Details, 0, len(containersByProject))+ for projectName, projectContainers := range containersByProject {+ runtimeServices := buildDiscoveredRuntimeServicesInternal(projectContainers, iconCatalog)+ imageRefs := projects.ImageRefsFromRuntimeServices(runtimeServices)+ updateInfo := buildProjectUpdateInfoSummaryInternal(imageRefs, updateInfoByRef)+ if updateInfo == nil || !updateInfo.HasUpdate {+ continue+ }++ runningCount := 0+ for _, runtimeService := range runtimeServices {+ if runtimeService.Status == "running" {+ runningCount+++ }+ }++ lastCheckedAt := ""+ if updateInfo.LastCheckedAt != nil {+ lastCheckedAt = updateInfo.LastCheckedAt.Format(time.RFC3339)+ }++ rows = append(rows, project.Details{+ ID: "compose:" + projectName,+ Name: projectName,+ Path: "",+ Status: resolveDiscoveredProjectStatusInternal(len(runtimeServices), runningCount),+ ServiceCount: len(runtimeServices),+ RunningCount: runningCount,+ IsDiscovered: true,+ CreatedAt: lastCheckedAt,+ UpdatedAt: lastCheckedAt,+ RuntimeServices: runtimeServices,+ UpdateInfo: updateInfo,+ })+ }++ return rows+}++func getRuntimeContainerUpdateInfoByRefInternal(+ ctx context.Context,+ composeContainers []container.Summary,+ imageService *image.ImageService,+) map[string]*imagetypes.UpdateInfo {+ if imageService == nil || len(composeContainers) == 0 {+ return nil+ }++ imageRefs := make([]string, 0, len(composeContainers))+ imageIDsByRef := make(map[string][]string, len(composeContainers))+ seenRefs := make(map[string]struct{}, len(composeContainers))+ for _, c := range composeContainers {+ imageRef := strings.TrimSpace(c.Image)+ if imageRef == "" {+ continue+ }+ if _, exists := seenRefs[imageRef]; !exists {+ seenRefs[imageRef] = struct{}{}+ imageRefs = append(imageRefs, imageRef)+ }+ if imageID := strings.TrimSpace(c.ImageID); imageID != "" {+ imageIDsByRef[imageRef] = append(imageIDsByRef[imageRef], imageID)+ }+ }++ updateInfoByRef := make(map[string]*imagetypes.UpdateInfo, len(imageRefs))+ if len(imageRefs) > 0 {+ if refResults, err := imageService.GetUpdateInfoByImageRefs(ctx, imageRefs); err == nil {+ maps.Copy(updateInfoByRef, refResults)+ } else {+ slog.WarnContext(ctx, "failed to fetch compose project update info by image ref", "error", err)+ }+ }++ missingImageIDs := make([]string, 0)+ for _, imageRef := range imageRefs {+ if updateInfoByRef[imageRef] != nil {+ continue+ }+ missingImageIDs = append(missingImageIDs, imageIDsByRef[imageRef]...)+ }++ if len(missingImageIDs) == 0 {+ return updateInfoByRef+ }++ updateInfoByID, err := imageService.GetUpdateInfoByImageIDs(ctx, missingImageIDs)+ if err != nil {+ slog.WarnContext(ctx, "failed to fetch compose project update info by image id", "error", err)+ return updateInfoByRef+ }++ for imageRef, imageIDs := range imageIDsByRef {+ if updateInfoByRef[imageRef] != nil {+ continue+ }+ for _, imageID := range imageIDs {+ if info := updateInfoByID[imageID]; info != nil {+ updateInfoByRef[imageRef] = info+ break+ }+ }+ }++ return updateInfoByRef+}++func buildDiscoveredRuntimeServicesInternal(containers []container.Summary, iconCatalog string) []project.RuntimeService {+ runtimeServices := make([]project.RuntimeService, 0, len(containers))+ seenServices := make(map[string]struct{}, len(containers))+ for _, c := range containers {+ imageRef := strings.TrimSpace(c.Image)+ if imageRef == "" {+ continue+ }++ serviceName := dockerutil.ComposeServiceLabel(c.Labels)+ if serviceName == "" {+ serviceName = c.ID+ }+ key := serviceName + "\x00" + imageRef+ if _, exists := seenServices[key]; exists {+ continue+ }+ seenServices[key] = struct{}{}++ containerName := dockerutil.ContainerNameFromNames(c.Names)++ resolvedIcon := iconcatalog.Resolve(iconCatalog, projects.FindArcaneIconSet(c.Labels))+ runtimeServices = append(runtimeServices, project.RuntimeService{+ Name: serviceName,+ Image: imageRef,+ Status: string(c.State),+ ContainerID: c.ID,+ ContainerName: containerName,+ Ports: projects.FormatDockerPorts(c.Ports),+ IconLightURL: resolvedIcon.IconLightURL,+ IconDarkURL: resolvedIcon.IconDarkURL,+ })+ }++ return runtimeServices+}++func resolveDiscoveredProjectStatusInternal(serviceCount int, runningCount int) string {+ switch {+ case serviceCount == 0:+ return string(models.ProjectStatusUnknown)+ case runningCount >= serviceCount:+ return string(models.ProjectStatusRunning)+ case runningCount > 0:+ return string(models.ProjectStatusPartiallyRunning)+ default:+ return string(models.ProjectStatusStopped)+ }+}++func (s *ProjectService) buildProjectDerivedPaginationConfigInternal() pagination.Config[project.Details] {+ return pagination.Config[project.Details]{+ SearchAccessors: []pagination.SearchAccessor[project.Details]{+ func(p project.Details) (string, error) { return p.Name, nil },+ func(p project.Details) (string, error) { return p.Path, nil },+ func(p project.Details) (string, error) { return p.RelativePath, nil },+ func(p project.Details) (string, error) { return p.Status, nil },+ func(p project.Details) (string, error) { return p.DirName, nil },+ },+ SortBindings: []pagination.SortBinding[project.Details]{+ {+ Key: "name",+ Fn: func(a, b project.Details) int {+ return strings.Compare(a.Name, b.Name)+ },+ },+ {+ Key: "status",+ Fn: func(a, b project.Details) int {+ return strings.Compare(a.Status, b.Status)+ },+ },+ {+ Key: "serviceCount",+ Fn: func(a, b project.Details) int {+ if a.ServiceCount < b.ServiceCount {+ return -1+ }+ if a.ServiceCount > b.ServiceCount {+ return 1+ }+ return 0+ },+ },+ {+ Key: "path",+ Fn: func(a, b project.Details) int {+ return strings.Compare(a.RelativePath, b.RelativePath)+ },+ },+ {+ Key: "createdAt",+ Fn: func(a, b project.Details) int {+ at, aerr := time.Parse(time.RFC3339, a.CreatedAt)+ bt, berr := time.Parse(time.RFC3339, b.CreatedAt)+ if aerr != nil || berr != nil {+ return strings.Compare(a.CreatedAt, b.CreatedAt)+ }+ if at.Before(bt) {+ return -1+ }+ if at.After(bt) {+ return 1+ }+ return 0+ },+ },+ },+ FilterAccessors: []pagination.FilterAccessor[project.Details]{+ buildProjectStatusFilterAccessorInternal(),+ buildProjectUpdatesFilterAccessorInternal(),+ buildProjectArchivedFilterAccessorInternal(),+ },+ }+}++func buildProjectStatusFilterAccessorInternal() pagination.FilterAccessor[project.Details] {+ return pagination.FilterAccessor[project.Details]{+ Key: "status",+ Fn: func(p project.Details, filterValue string) bool {+ return strings.EqualFold(strings.TrimSpace(p.Status), strings.TrimSpace(filterValue))+ },+ }+}++func buildProjectUpdatesFilterAccessorInternal() pagination.FilterAccessor[project.Details] {+ return pagination.FilterAccessor[project.Details]{+ Key: "updates",+ Fn: func(p project.Details, filterValue string) bool {+ return strings.EqualFold(strings.TrimSpace(getProjectUpdateStatusInternal(p.UpdateInfo)), strings.TrimSpace(filterValue))+ },+ }+}++func buildProjectArchivedFilterAccessorInternal() pagination.FilterAccessor[project.Details] {+ return pagination.FilterAccessor[project.Details]{+ Key: "archived",+ Fn: func(p project.Details, filterValue string) bool {+ switch strings.ToLower(strings.TrimSpace(filterValue)) {+ case "true":+ return p.IsArchived+ case "all":+ return true+ default:+ return !p.IsArchived+ }+ },+ }+}++func getProjectUpdateStatusInternal(updateInfo *project.UpdateInfo) string {+ if updateInfo == nil || strings.TrimSpace(updateInfo.Status) == "" {+ return "unknown"+ }++ return updateInfo.Status+}++// CountProjectsWithPendingUpdates counts non-archived projects with at+// least one image update pending, plus compose projects running on the daemon+// that Arcane does not track. It deliberately avoids the project-list pipeline:+// that path builds full project DTOs (live status, icons, URLs, GitOps lookups)+// and then throws all of them away for a single number, costing several full+// container lists and a compose parse per project on every dashboard load.+//+// allContainers is the caller's already-fetched container list; pass nil to have+// it fetched here.+func (s *ProjectService) CountProjectsWithPendingUpdates(ctx context.Context, allContainers []container.Summary) (int, error) {+ if s.db == nil {+ return 0, nil+ }++ var projectsArray []models.Project+ if err := s.db.WithContext(ctx).Where("is_archived = ?", false).Find(&projectsArray).Error; err != nil {+ return 0, errors.WrapIf(err, "failed to list projects for update count")+ }++ // enrichProjectsWithUpdateInfoInternal keys off Details.ID, so the summaries+ // only need identity — no status, icons or URLs are read here.+ details := make([]project.Details, len(projectsArray))+ for i, proj := range projectsArray {+ details[i].ID = proj.ID+ }+ s.enrichProjectsWithUpdateInfoInternal(ctx, projectsArray, details)++ count := 0+ for i := range details {+ if details[i].UpdateInfo != nil && details[i].UpdateInfo.HasUpdate {+ count+++ }+ }++ return count + s.countDiscoveredComposeProjectUpdatesInternal(ctx, projectsArray, allContainers), nil+}++// countDiscoveredComposeProjectUpdatesInternal counts compose projects running on+// the daemon that Arcane does not track but that have a pending image update, so+// the dashboard badge matches the projects table. Errors are logged and counted+// as zero: a missing container list should degrade the badge, not fail the load.+func (s *ProjectService) countDiscoveredComposeProjectUpdatesInternal(ctx context.Context, projectsArray []models.Project, allContainers []container.Summary) int {+ if allContainers == nil {+ var err error+ allContainers, err = projects.ListGlobalComposeContainers(ctx)+ if err != nil {+ slog.WarnContext(ctx, "failed to list compose containers for project update count", "error", err)+ return 0+ }+ }++ composeContainers := make([]container.Summary, 0, len(allContainers))+ for _, c := range allContainers {+ if dockerutil.ComposeProjectLabel(c.Labels) != "" {+ composeContainers = append(composeContainers, c)+ }+ }+ if len(composeContainers) == 0 {+ return 0+ }++ knownProjectNames := s.buildKnownComposeProjectNameSetInternal(ctx, projectsArray)+ // Only rows with a pending update are returned, so the length is the count.+ return len(buildDiscoveredComposeProjectUpdateRowsInternal(ctx, composeContainers, knownProjectNames, s.imageService, IconCatalogForContext(ctx)))+}++// fetchProjectStatusConcurrently fetches live Docker status for multiple projects in parallel+// Optimized to use a single Docker API call instead of N calls + N file reads+func (s *ProjectService) fetchProjectStatusConcurrently(ctx context.Context, projectsList []models.Project) []project.Details {+ projectsDir, err := s.GetProjectsDirectory(ctx)+ if err != nil {+ slog.WarnContext(ctx, "failed to resolve projects directory for relative project paths", "error", err)+ }++ // Resolved once for the whole list: ProjectMetadata would+ // otherwise re-stat the projects directory and re-clone settings per project.+ metaEnv := &projectMetadataEnvInternal{+ projectsDirectory: projectsDir,+ autoInjectEnv: s.settingsService.GetBoolSetting(ctx, "autoInjectEnv", false),+ }++ // 1. Fetch all compose containers in one go+ containers, err := projects.ListGlobalComposeContainers(ctx)+ if err != nil {+ slog.ErrorContext(ctx, "Failed to list global compose containers", "error", err)+ // Fallback: return basic info with unknown status+ results := make([]project.Details, len(projectsList))+ for i, p := range projectsList {+ _ = mapper.MapStruct(p, &results[i])+ results[i].CreatedAt = p.CreatedAt.Format(time.RFC3339)+ results[i].UpdatedAt = p.UpdatedAt.Format(time.RFC3339)+ results[i].DirName = mo.PointerToOption(p.DirName).OrEmpty()+ results[i].RelativePath = getProjectRelativePathInternal(projectsDir, p.Path)+ results[i].GitOpsManagedBy = p.GitOpsManagedBy+ meta := s.ProjectMetadata(ctx, p, metaEnv)+ applyResolvedProjectIconInternal(&results[i], iconcatalog.Resolve(IconCatalogForContext(ctx), meta.ProjectIcon))+ results[i].URLs = meta.ProjectURLS+ results[i].Status = string(models.ProjectStatusUnknown)+ }+ return results+ }++ // 2. Group containers by project name+ containersByProject := groupComposeContainersByProjectInternal(containers)++ // 3. Map to DTOs+ results := make([]project.Details, len(projectsList))+ currentContainerID, currentContainerErr := cgroup.CurrentContainerID()+ for i, p := range projectsList {+ results[i] = s.mapProjectToDto(ctx, projectsDir, p, containersByProject, currentContainerID, currentContainerErr, metaEnv)+ }++ return results+}++func (s *ProjectService) mapProjectToDto(ctx context.Context, projectsDir string, p models.Project, containersByProject map[string][]container.Summary, currentContainerID string, currentContainerErr error, metaEnv *projectMetadataEnvInternal) project.Details {+ var resp project.Details+ _ = mapper.MapStruct(p, &resp)++ resp.CreatedAt = p.CreatedAt.Format(time.RFC3339)+ resp.UpdatedAt = p.UpdatedAt.Format(time.RFC3339)+ resp.IsArchived = p.IsArchived+ resp.ArchivedAt = p.ArchivedAt+ resp.DirName = mo.PointerToOption(p.DirName).OrEmpty()+ resp.RelativePath = getProjectRelativePathInternal(projectsDir, p.Path)+ resp.GitOpsManagedBy = p.GitOpsManagedBy+ meta := s.ProjectMetadata(ctx, p, metaEnv)+ applyResolvedProjectIconInternal(&resp, iconcatalog.Resolve(IconCatalogForContext(ctx), meta.ProjectIcon))+ resp.URLs = meta.ProjectURLS++ projectContainers := lookupProjectContainers(p, containersByProject)++ services := make([]ProjectServiceInfo, 0, len(projectContainers))++ for _, c := range projectContainers {+ svcName := dockerutil.ComposeServiceLabel(c.Labels)+ state := c.State // "running", "exited", etc.++ // Parse health from Status string if possible+ var health *string+ statusLower := strings.ToLower(c.Status)+ switch {+ case strings.Contains(statusLower, "(healthy)"):+ health = new("healthy")+ case strings.Contains(statusLower, "(unhealthy)"):+ health = new("unhealthy")+ case strings.Contains(statusLower, "(starting)"):+ health = new("starting")+ }++ containerName := dockerutil.ContainerNameFromNames(c.Names)++ redeployDisabled := labels.ShouldDisableArcaneServerRedeploy(c.Labels, c.ID, currentContainerID, currentContainerErr)+ if redeployDisabled {+ resp.RedeployDisabled = true+ }++ resolvedIcon := iconcatalog.Resolve(IconCatalogForContext(ctx), iconcatalog.FirstNonEmpty(+ projects.FindArcaneIconSet(c.Labels),+ meta.ServiceIconSets[svcName],+ meta.ProjectIcon,+ ))+ services = append(services, ProjectServiceInfo{+ Name: svcName,+ Image: c.Image,+ Status: string(state),+ ContainerID: c.ID,+ ContainerName: containerName,+ Ports: projects.FormatDockerPorts(c.Ports),+ Health: health,+ IconLightURL: resolvedIcon.IconLightURL,+ IconDarkURL: resolvedIcon.IconDarkURL,+ Labels: c.Labels,+ RedeployDisabled: redeployDisabled,+ })+ }+ _, runningCount := getServiceCounts(services)++ // Convert to RuntimeServices+ runtimeServices := make([]project.RuntimeService, len(services))+ for k, s := range services {+ runtimeServices[k] = project.RuntimeService{+ Name: s.Name,+ Image: s.Image,+ Status: s.Status,+ ContainerID: s.ContainerID,+ ContainerName: s.ContainerName,+ Ports: s.Ports,+ Health: s.Health,+ IconLightURL: s.IconLightURL,+ IconDarkURL: s.IconDarkURL,+ ServiceConfig: s.ServiceConfig,+ RedeployDisabled: s.RedeployDisabled,+ }+ }+ resp.RuntimeServices = runtimeServices++ // Use DB service count as the source of truth for "Total Services"+ // since we are not parsing the YAML here.+ resp.ServiceCount = p.ServiceCount+ resp.RunningCount = runningCount+ if resp.ServiceCount == 0 && len(services) > 0 {+ resp.ServiceCount = len(services)+ // Persist the inferred count so later list loads do not need compose parsing.+ go func(ctx context.Context, pid string, count int) {+ s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", pid).Update("service_count", count)+ }(context.WithoutCancel(ctx), p.ID, resp.ServiceCount)+ }++ // For missing service count (e.g. newly discovered projects), skip the+ // expensive CountServicesFromCompose call which loads and parses the entire+ // compose project. The count will be populated the next time the project+ // detail endpoint is called or during the periodic filesystem sync.++ // Calculate Status using actual container count from Docker rather than the+ // (potentially stale) DB ServiceCount. The DB value can become outdated when+ // a service is removed from the compose file but compose parsing fails during+ // filesystem sync, leaving the old count in the database. This mirrors the+ // logic in calculateProjectStatus and GetProjectDetails, which both use the+ // live container/service list as the source of truth.+ actualServiceCount := len(services)+ if actualServiceCount == 0 {+ resp.Status = string(models.ProjectStatusStopped)+ } else {+ switch {+ case runningCount >= actualServiceCount:+ resp.Status = string(models.ProjectStatusRunning)+ case runningCount > 0:+ resp.Status = string(models.ProjectStatusPartiallyRunning)+ default:+ resp.Status = string(models.ProjectStatusStopped)+ }+ }++ return resp+}++// ProjectMetadata resolves a project's icon sets and service URLs.+// Results are cached for projectMetadataTTL because deriving them is expensive+// (compose load with interpolation and .env reads, plus a gitops_syncs query for+// GitOps-managed projects) and every project row on the list page needs it.+//+// env may be nil, in which case the projects directory and autoInjectEnv setting+// are resolved here; callers iterating over many projects should resolve them+// once and pass them in.+func (s *ProjectService) ProjectMetadata(ctx context.Context, p models.Project, env *projectMetadataEnvInternal) projects.ArcaneComposeMetadata {+ if s.metaCache != nil && p.ID != "" {+ if meta, ok, _ := s.metaCache.Get(p.ID); ok {+ return meta+ }+ }++ empty := projects.ArcaneComposeMetadata{ServiceIconSets: map[string]projects.IconSet{}}++ composeFile, err := s.ResolveProjectComposeFile(ctx, &p)+ if err != nil {+ return empty+ }++ if env == nil {+ projectsDirectory, projectsDirErr := s.GetProjectsDirectory(ctx)+ if projectsDirErr != nil {+ slog.WarnContext(ctx, "failed to resolve projects directory for Arcane compose metadata", "path", composeFile, "error", projectsDirErr)+ }+ env = &projectMetadataEnvInternal{+ projectsDirectory: projectsDirectory,+ autoInjectEnv: s.settingsService.GetBoolSetting(ctx, "autoInjectEnv", false),+ }+ }++ meta, err := projects.ParseArcaneComposeMetadata(ctx, composeFile, env.projectsDirectory, env.autoInjectEnv)+ if err != nil {+ slog.WarnContext(ctx, "failed to parse Arcane compose metadata", "path", composeFile, "error", err)+ return empty+ }++ if s.metaCache != nil && p.ID != "" {+ s.metaCache.Set(p.ID, meta)+ }++ return meta+}++// IconCatalogForContext resolves the icon catalog of the requesting+// user. On agent-proxied calls the caller is a synthetic user whose preference+// is populated from the X-Arcane-Icon-Catalog header the manager forwards.+// Background jobs have no user attached and fall back to the default catalog.+func IconCatalogForContext(ctx context.Context) string {+ if u, ok := models.CurrentUserFromContext(ctx); ok && u != nil && u.Preferences.IconCatalog != nil && *u.Preferences.IconCatalog != "" {+ return *u.Preferences.IconCatalog+ }+ return iconcatalog.DefaultCatalog+}++func applyResolvedProjectIconInternal(resp *project.Details, icon iconcatalog.ResolvedIconSet) {+ if resp == nil {+ return+ }+ resp.IconLightURL = icon.IconLightURL+ resp.IconDarkURL = icon.IconDarkURL+}diff --git a/backend/internal/project/project_rename_journal.go b/backend/internal/project/project_rename_journal.gonew file mode 100644index 000000000..a2b5a2d75--- /dev/null+++ b/backend/internal/project/project_rename_journal.go@@ -0,0 +1,545 @@+package project++import (+ "context"+ "encoding/json/v2"+ stderrors "errors"+ "log/slog"+ "os"+ "path/filepath"+ "strings"+ "time"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/volumes"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/moby/moby/client"+ "gorm.io/gorm"+)++type activeProjectRenameSyncStateInternal struct {+ skipDiscoveredPaths map[string]struct{}+ protectSeenPaths map[string]struct{}+}++const (+ projectRenameJournalKeyPrefixInternal = "project_rename_journal:"+ projectRenameRollbackCleanupKeyPrefixInternal = "project_rename_rollback_cleanup:"++ projectRenameJournalPhaseStartedInternal = "started"+ projectRenameJournalPhaseTargetsCopiedInternal = "targets_copied"+ projectRenameJournalPhaseOldVolumesRemovedInternal = "old_volumes_removed"+ projectRenameJournalPhaseProjectStateCommittedInternal = "project_state_committed"+ projectRenameJournalPhaseSourceCleanupPendingInternal = "source_cleanup_pending"+ projectRenameJournalPhaseProjectStateRolledBackInternal = "project_state_rolled_back"+)++type projectRenameJournalInternal struct {+ ProjectID string `json:"projectId"`+ OldName string `json:"oldName"`+ NewName string `json:"newName"`+ OldPath string `json:"oldPath"`+ NewPath string `json:"newPath"`+ OldDirName *string `json:"oldDirName,omitempty"`+ NewDirName string `json:"newDirName"`+ Phase string `json:"phase"`+ Volumes []volumes.JournalVolume `json:"volumes,omitempty"`+ UpdatedAt time.Time `json:"updatedAt"`+}++type projectRenameRollbackCleanupInternal struct {+ ProjectID string `json:"projectId"`+ OldName string `json:"oldName"`+ OldPath string `json:"oldPath"`+ NewName string `json:"newName"`+ NewPath string `json:"newPath"`+ Volumes []volumes.JournalVolume `json:"volumes,omitempty"`+ UpdatedAt time.Time `json:"updatedAt"`+}++func (s *ProjectService) activeProjectRenameSyncStateInternal(ctx context.Context) activeProjectRenameSyncStateInternal {+ state := activeProjectRenameSyncStateInternal{+ skipDiscoveredPaths: make(map[string]struct{}),+ protectSeenPaths: make(map[string]struct{}),+ }+ if s == nil || s.kvService == nil {+ return state+ }++ entries, err := s.kvService.ListByPrefix(ctx, projectRenameJournalKeyPrefixInternal)+ if err != nil {+ slog.WarnContext(ctx, "failed to list project rename journals during filesystem sync", "error", err)+ return state+ }++ for _, entry := range entries {+ var journal projectRenameJournalInternal+ if err := json.Unmarshal([]byte(entry.Value), &journal); err != nil {+ slog.WarnContext(ctx, "failed to decode project rename journal during filesystem sync", "key", entry.Key, "error", err)+ continue+ }+ if !projectRenameJournalFilesystemSyncPendingInternal(journal.Phase) {+ continue+ }+ if oldPath := strings.TrimSpace(journal.OldPath); oldPath != "" {+ state.protectSeenPaths[filepath.Clean(oldPath)] = struct{}{}+ }+ if newPath := strings.TrimSpace(journal.NewPath); newPath != "" {+ state.skipDiscoveredPaths[filepath.Clean(newPath)] = struct{}{}+ }+ }++ return state+}++func (s activeProjectRenameSyncStateInternal) skipDiscoveredPathInternal(path string) bool {+ _, ok := s.skipDiscoveredPaths[filepath.Clean(path)]+ return ok+}++func (s activeProjectRenameSyncStateInternal) markProtectedPathsSeenInternal(seen map[string]struct{}) {+ for seenPath := range s.protectSeenPaths {+ seen[seenPath] = struct{}{}+ }+}++func (s *ProjectService) startProjectRenameJournalInternal(ctx context.Context, journal *projectRenameJournalInternal) (bool, error) {+ if journal == nil {+ return false, nil+ }+ if err := s.writeProjectRenameJournalInternal(ctx, journal, projectRenameJournalPhaseStartedInternal); err != nil {+ return false, err+ }+ return true, nil+}++func (s *ProjectService) applyProjectVolumeMigrationForUpdateInternal(ctx context.Context, volumeMigration volumes.Migration, renameJournal *projectRenameJournalInternal, applied *bool) error {+ if volumeMigration == nil {+ return nil+ }+ if err := volumeMigration.Apply(ctx); err != nil {+ return errors.WrapIf(err, "failed to rename project volumes")+ }+ *applied = true+ return s.writeProjectRenameJournalInternal(ctx, renameJournal, projectRenameJournalPhaseTargetsCopiedInternal)+}++func (s *ProjectService) finalizeProjectRenameAfterCommitInternal(ctx context.Context, projectID string, volumeMigration volumes.Migration, renameJournal *projectRenameJournalInternal, journalActive *bool) {+ if renameJournal != nil {+ if err := s.writeProjectRenameJournalInternal(ctx, renameJournal, projectRenameJournalPhaseProjectStateCommittedInternal); err != nil {+ slog.WarnContext(ctx, "failed to mark project rename journal committed", "projectID", projectID, "error", err)+ }+ }++ if committer, ok := volumeMigration.(volumes.Committer); ok {+ if err := committer.Commit(ctx); err != nil {+ slog.WarnContext(ctx, "failed to clean up project source volumes after committed rename", "projectID", projectID, "error", err)+ var cleanupErr *volumes.SourceCleanupError+ if errors.As(err, &cleanupErr) {+ if writeErr := s.writeProjectRenameJournalInternal(ctx, renameJournal, projectRenameJournalPhaseSourceCleanupPendingInternal); writeErr != nil {+ slog.WarnContext(ctx, "failed to mark project rename source cleanup pending", "projectID", projectID, "error", writeErr)+ }+ }+ return+ } else if err := s.writeProjectRenameJournalInternal(ctx, renameJournal, projectRenameJournalPhaseOldVolumesRemovedInternal); err != nil {+ slog.WarnContext(ctx, "failed to mark old project rename volumes removed", "projectID", projectID, "error", err)+ }+ }++ s.completeProjectRenameJournalForUpdateInternal(ctx, renameJournal, projectID, journalActive)+}++func (s *ProjectService) completeProjectRenameJournalForUpdateInternal(ctx context.Context, renameJournal *projectRenameJournalInternal, projectID string, journalActive *bool) {+ if renameJournal == nil {+ return+ }+ if clearErr := s.clearProjectRenameJournalInternal(ctx, projectID); clearErr != nil {+ slog.WarnContext(ctx, "failed to clear project rename journal", "projectID", projectID, "error", clearErr)+ return+ }+ *journalActive = false+}++func withProjectRenameRollback(ctx context.Context, proj *models.Project, projectStateCommitted *bool, run func() error) error {+ originalPath := proj.Path+ originalDirName := proj.DirName++ if err := run(); err != nil {+ if projectStateCommitted != nil && *projectStateCommitted {+ return err+ }+ if proj.Path != originalPath {+ if renameErr := os.Rename(proj.Path, originalPath); renameErr != nil {+ slog.WarnContext(ctx, "failed to rollback project directory rename", "from", proj.Path, "to", originalPath, "error", renameErr)+ return err+ }+ proj.Path = originalPath+ proj.DirName = originalDirName+ }+ return err+ }++ return nil+}++func projectRenameJournalKeyInternal(projectID string) string {+ return projectRenameJournalKeyPrefixInternal + strings.TrimSpace(projectID)+}++func projectRenameRollbackCleanupKeyInternal(projectID string) string {+ return projectRenameRollbackCleanupKeyPrefixInternal + strings.TrimSpace(projectID)+}++func (s *ProjectService) prepareProjectRenameJournalInternal(proj *models.Project, name *string, projectsDirectory string, migration volumes.Migration) *projectRenameJournalInternal {+ if s == nil || s.kvService == nil || proj == nil || name == nil {+ return nil+ }++ newName := strings.TrimSpace(*name)+ if newName == "" || proj.Name == newName {+ return nil+ }++ newDirName := strings.TrimSpace(projects.SanitizeProjectName(newName))+ if newDirName == "" || strings.Trim(newDirName, "_") == "" {+ return nil+ }++ journal := &projectRenameJournalInternal{+ ProjectID: proj.ID,+ OldName: proj.Name,+ NewName: newName,+ OldPath: filepath.Clean(proj.Path),+ NewPath: filepath.Clean(filepath.Join(projectsDirectory, newDirName)),+ OldDirName: cloneStringPtrInternal(proj.DirName),+ NewDirName: newDirName,+ Phase: projectRenameJournalPhaseStartedInternal,+ }++ if source, ok := migration.(volumes.JournalSource); ok {+ journal.Volumes = source.JournalVolumes()+ }++ return journal+}++func (s *ProjectService) writeProjectRenameJournalInternal(ctx context.Context, journal *projectRenameJournalInternal, phase string) error {+ if s == nil || s.kvService == nil || journal == nil {+ return nil+ }+ journal.Phase = phase+ journal.UpdatedAt = time.Now().UTC()++ payload, err := json.Marshal(journal)+ if err != nil {+ return errors.WrapIf(err, "marshal project rename journal")+ }++ if err := s.kvService.Set(ctx, projectRenameJournalKeyInternal(journal.ProjectID), string(payload)); err != nil {+ return errors.WrapIf(err, "write project rename journal")+ }+ return nil+}++func (s *ProjectService) clearProjectRenameJournalInternal(ctx context.Context, projectID string) error {+ if s == nil || s.kvService == nil || strings.TrimSpace(projectID) == "" {+ return nil+ }+ return s.kvService.Delete(ctx, projectRenameJournalKeyInternal(projectID))+}++func (s *ProjectService) writeProjectRenameRollbackCleanupInternal(ctx context.Context, journal *projectRenameJournalInternal) error {+ if s == nil || s.kvService == nil || journal == nil || strings.TrimSpace(journal.ProjectID) == "" || len(journal.Volumes) == 0 {+ return nil+ }++ cleanup := projectRenameRollbackCleanupInternal{+ ProjectID: journal.ProjectID,+ OldName: journal.OldName,+ OldPath: filepath.Clean(journal.OldPath),+ NewName: journal.NewName,+ NewPath: filepath.Clean(journal.NewPath),+ Volumes: journal.Volumes,+ UpdatedAt: time.Now().UTC(),+ }+ payload, err := json.Marshal(cleanup)+ if err != nil {+ return errors.WrapIf(err, "marshal project rename rollback cleanup")+ }+ if err := s.kvService.Set(ctx, projectRenameRollbackCleanupKeyInternal(journal.ProjectID), string(payload)); err != nil {+ return errors.WrapIf(err, "write project rename rollback cleanup")+ }+ return nil+}++func (s *ProjectService) clearProjectRenameRollbackCleanupInternal(ctx context.Context, projectID string) error {+ if s == nil || s.kvService == nil || strings.TrimSpace(projectID) == "" {+ return nil+ }+ return s.kvService.Delete(ctx, projectRenameRollbackCleanupKeyInternal(projectID))+}++func projectRenameJournalTargetsCopiedInternal(phase string) bool {+ switch phase {+ case projectRenameJournalPhaseTargetsCopiedInternal,+ projectRenameJournalPhaseOldVolumesRemovedInternal,+ projectRenameJournalPhaseProjectStateCommittedInternal,+ projectRenameJournalPhaseSourceCleanupPendingInternal:+ return true+ default:+ return false+ }+}++func projectRenameJournalFilesystemSyncPendingInternal(phase string) bool {+ switch phase {+ case projectRenameJournalPhaseStartedInternal,+ projectRenameJournalPhaseTargetsCopiedInternal:+ return true+ default:+ return false+ }+}++func (s *ProjectService) RecoverProjectRenameJournals(ctx context.Context) error {+ if s == nil || s.kvService == nil {+ return nil+ }++ entries, err := s.kvService.ListByPrefix(ctx, projectRenameJournalKeyPrefixInternal)+ if err != nil {+ return err+ }++ var recoverErr error+ for _, entry := range entries {+ var journal projectRenameJournalInternal+ if err := json.Unmarshal([]byte(entry.Value), &journal); err != nil {+ recoverErr = stderrors.Join(recoverErr, errors.WrapIff(err, "decode project rename journal %s", entry.Key))+ continue+ }+ if err := s.recoverProjectRenameJournalInternal(ctx, &journal); err != nil {+ recoverErr = stderrors.Join(recoverErr, errors.WrapIff(err, "recover project rename journal %s", entry.Key))+ continue+ }+ }+ return stderrors.Join(recoverErr, s.recoverProjectRenameRollbackCleanupsInternal(ctx))+}++func (s *ProjectService) recoverProjectRenameJournalForProjectInternal(ctx context.Context, projectID string) error {+ if s == nil || s.kvService == nil || strings.TrimSpace(projectID) == "" {+ return nil+ }++ raw, ok, err := s.kvService.Get(ctx, projectRenameJournalKeyInternal(projectID))+ if err != nil || !ok {+ return err+ }++ var journal projectRenameJournalInternal+ if err := json.Unmarshal([]byte(raw), &journal); err != nil {+ return errors.WrapIf(err, "decode project rename journal")+ }+ return s.recoverProjectRenameJournalInternal(ctx, &journal)+}++func (s *ProjectService) recoverProjectRenameJournalInternal(ctx context.Context, journal *projectRenameJournalInternal) error {+ if s == nil || journal == nil || strings.TrimSpace(journal.ProjectID) == "" {+ return nil+ }++ var proj models.Project+ dbErr := s.db.WithContext(ctx).First(&proj, "id = ?", journal.ProjectID).Error+ if dbErr != nil && !errors.Is(dbErr, gorm.ErrRecordNotFound) {+ return errors.WrapIf(dbErr, "load project for rename recovery")+ }++ projectCommitted := dbErr == nil && (proj.Name == journal.NewName || filepath.Clean(proj.Path) == filepath.Clean(journal.NewPath))+ if projectCommitted {+ if journal.Phase == projectRenameJournalPhaseSourceCleanupPendingInternal {+ if err := s.cleanupProjectRenameJournalSourcesInternal(ctx, journal); err != nil {+ return err+ }+ return s.clearProjectRenameJournalInternal(ctx, journal.ProjectID)+ }+ if err := s.cleanupProjectRenameJournalSourcesInternal(ctx, journal); err != nil {+ var cleanupErr *volumes.SourceCleanupError+ if errors.As(err, &cleanupErr) {+ if writeErr := s.writeProjectRenameJournalInternal(ctx, journal, projectRenameJournalPhaseSourceCleanupPendingInternal); writeErr != nil {+ return stderrors.Join(err, writeErr)+ }+ }+ return err+ }+ return s.clearProjectRenameJournalInternal(ctx, journal.ProjectID)+ }++ if err := s.rollbackProjectRenameJournalInternal(ctx, journal); err != nil {+ return err+ }+ if err := s.writeProjectRenameJournalInternal(ctx, journal, projectRenameJournalPhaseProjectStateRolledBackInternal); err != nil {+ return err+ }+ return s.clearProjectRenameJournalInternal(ctx, journal.ProjectID)+}++func (s *ProjectService) cleanupProjectRenameJournalSourcesInternal(ctx context.Context, journal *projectRenameJournalInternal) error {+ dockerClient, err := s.projectRenameRecoveryDockerInternal(ctx, len(journal.Volumes) > 0)+ if err != nil {+ return err+ }++ if err := volumes.EnsureTargetsReadyForCleanup(ctx, dockerClient, journal.Volumes); err != nil {+ var missingWithSource *volumes.TargetMissingWithSourceError+ if errors.As(err, &missingWithSource) {+ slog.WarnContext(ctx, "rolling back project rename because target volume is missing and source volume remains", "projectID", journal.ProjectID, "sourceVolume", missingWithSource.SourceVolume, "targetVolume", missingWithSource.TargetVolume)+ return s.rollbackProjectRenameJournalInternal(ctx, journal)+ }+ var externallyRemoved *volumes.VolumesExternallyRemovedError+ if errors.As(err, &externallyRemoved) {+ slog.WarnContext(ctx, "project rename cleanup found source and target volumes externally removed", "projectID", journal.ProjectID, "volumeCount", len(externallyRemoved.Volumes), "error", externallyRemoved)+ } else {+ return err+ }+ }++ return volumes.RemoveSourceVolumes(ctx, dockerClient, journal.Volumes)+}++func (s *ProjectService) rollbackProjectRenameJournalInternal(ctx context.Context, journal *projectRenameJournalInternal) error {+ pathsMissing, directoryErr := projects.RollbackRenamedProjectDirectory(journal.OldPath, journal.NewPath)++ volumeErr := s.rollbackProjectRenameJournalVolumesInternal(ctx, journal)++ if err := s.db.WithContext(ctx).Model(&models.Project{}).+ Where("id = ?", journal.ProjectID).+ Updates(map[string]any{+ "name": journal.OldName,+ "path": journal.OldPath,+ "dir_name": journal.OldDirName,+ }).Error; err != nil {+ return stderrors.Join(directoryErr, volumeErr, errors.WrapIf(err, "restore project database state"))+ }++ if directoryErr != nil {+ slog.WarnContext(ctx, "keeping project rename journal after restoring database state because directory rollback failed", "projectID", journal.ProjectID, "pathsMissing", pathsMissing, "error", directoryErr)+ }++ if volumeErr != nil {+ if volumes.OnlyPreservedTargetErrors(volumeErr) {+ slog.WarnContext(ctx, "clearing project rename journal after preserving target volume data", "projectID", journal.ProjectID, "pathsMissing", pathsMissing, "error", volumeErr)+ } else {+ if cleanupErr := s.writeProjectRenameRollbackCleanupInternal(ctx, journal); cleanupErr != nil {+ return stderrors.Join(directoryErr, volumeErr, cleanupErr)+ }+ slog.WarnContext(ctx, "queued project rename target volume cleanup after restoring database state despite volume rollback failure", "projectID", journal.ProjectID, "pathsMissing", pathsMissing, "error", volumeErr)+ }+ }++ return directoryErr+}++func (s *ProjectService) recoverProjectRenameRollbackCleanupsInternal(ctx context.Context) error {+ if s == nil || s.kvService == nil {+ return nil+ }++ entries, err := s.kvService.ListByPrefix(ctx, projectRenameRollbackCleanupKeyPrefixInternal)+ if err != nil {+ return err+ }++ var recoverErr error+ for _, entry := range entries {+ var cleanup projectRenameRollbackCleanupInternal+ if err := json.Unmarshal([]byte(entry.Value), &cleanup); err != nil {+ recoverErr = stderrors.Join(recoverErr, errors.WrapIff(err, "decode project rename rollback cleanup %s", entry.Key))+ continue+ }+ if err := s.recoverProjectRenameRollbackCleanupInternal(ctx, &cleanup); err != nil {+ recoverErr = stderrors.Join(recoverErr, errors.WrapIff(err, "recover project rename rollback cleanup %s", entry.Key))+ continue+ }+ }+ return recoverErr+}++func (s *ProjectService) recoverProjectRenameRollbackCleanupInternal(ctx context.Context, cleanup *projectRenameRollbackCleanupInternal) error {+ if s == nil || cleanup == nil || strings.TrimSpace(cleanup.ProjectID) == "" {+ return nil+ }+ if len(cleanup.Volumes) == 0 {+ return s.clearProjectRenameRollbackCleanupInternal(ctx, cleanup.ProjectID)+ }++ var proj models.Project+ dbErr := s.db.WithContext(ctx).First(&proj, "id = ?", cleanup.ProjectID).Error+ if dbErr != nil {+ if errors.Is(dbErr, gorm.ErrRecordNotFound) {+ slog.WarnContext(ctx, "clearing project rename rollback cleanup because project no longer exists", "projectID", cleanup.ProjectID)+ return s.clearProjectRenameRollbackCleanupInternal(ctx, cleanup.ProjectID)+ }+ return errors.WrapIf(dbErr, "load project for rename rollback cleanup")+ }++ if proj.Name != cleanup.OldName || filepath.Clean(proj.Path) != filepath.Clean(cleanup.OldPath) {+ slog.WarnContext(ctx, "clearing project rename rollback cleanup because project state changed", "projectID", cleanup.ProjectID, "projectName", proj.Name, "projectPath", proj.Path)+ return s.clearProjectRenameRollbackCleanupInternal(ctx, cleanup.ProjectID)+ }++ dockerClient, err := s.projectRenameRecoveryDockerInternal(ctx, len(cleanup.Volumes) > 0)+ if err != nil {+ return err+ }++ if err := volumes.CleanupRollbackTargetVolumes(ctx, dockerClient, cleanup.Volumes); err != nil {+ if volumes.OnlyPreservedTargetErrors(err) {+ slog.WarnContext(ctx, "clearing project rename rollback cleanup after preserving target volume data", "projectID", cleanup.ProjectID, "error", err)+ return s.clearProjectRenameRollbackCleanupInternal(ctx, cleanup.ProjectID)+ }+ return err+ }++ dockerutil.InvalidateVolumeUsageCache(dockerClient)+ return s.clearProjectRenameRollbackCleanupInternal(ctx, cleanup.ProjectID)+}++func (s *ProjectService) rollbackProjectRenameJournalVolumesInternal(ctx context.Context, journal *projectRenameJournalInternal) error {+ if !projectRenameJournalTargetsCopiedInternal(journal.Phase) {+ return nil+ }++ dockerClient, err := s.projectRenameRecoveryDockerInternal(ctx, len(journal.Volumes) > 0)+ if err != nil {+ return err+ }++ return volumes.RollbackVolumes(ctx, dockerClient, journal.Volumes)+}++func (s *ProjectService) projectRenameRecoveryDockerInternal(ctx context.Context, dockerRequired bool) (*client.Client, error) {+ if !dockerRequired {+ return nil, nil+ }+ if s.dockerService == nil {+ return nil, errors.New("docker service unavailable")+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to connect to Docker")+ }++ return dockerClient, nil+}++func cloneStringPtrInternal(value *string) *string {+ if value == nil {+ return nil+ }+ cloned := *value+ return &cloned+}diff --git a/backend/internal/project/project_sync.go b/backend/internal/project/project_sync.gonew file mode 100644index 000000000..f036c40df--- /dev/null+++ b/backend/internal/project/project_sync.go@@ -0,0 +1,510 @@+package project++import (+ "context"+ "log/slog"+ "os"+ "path/filepath"+ "strings"+ "time"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/moby/moby/client"+ "github.com/samber/mo"+ "gorm.io/gorm"+)++// projectCleanupDecision records a project the reconcile pass intends to delete,+// alongside the reason logged when the deletion is carried out.+type projectCleanupDecision struct {+ project models.Project+ reason string+}++type composeSyncMetadataInternal struct {+ serviceCount int+ resolvedProjectName string+ composeProjectName *string+ explicitProjectName bool+}++func (s *ProjectService) refreshProjectImageRefsInternal(ctx context.Context, proj *models.Project) {+ if proj == nil || proj.ID == "" {+ return+ }++ s.parsedCompose.invalidate(proj.ID)+ refs, buildRefs, err := s.getProjectImageRefsFromComposeInternal(ctx, *proj, nil)+ if err != nil {+ if dbErr := s.db.WithContext(ctx).+ Model(&models.Project{}).+ Where("id = ?", proj.ID).+ Updates(map[string]any{+ "image_refs_json": "",+ "build_image_refs_json": nil,+ }).Error; dbErr != nil {+ slog.WarnContext(ctx, "failed to clear stale project image refs", "projectID", proj.ID, "error", dbErr)+ }+ proj.ImageRefsJSON = ""+ proj.BuildImageRefsJSON = nil+ slog.WarnContext(ctx, "failed to refresh project image refs", "projectID", proj.ID, "projectName", proj.Name, "error", err)+ return+ }+ imageRefsJSON := projects.MarshalImageRefsJSON(refs)+ buildImageRefsJSON := projects.MarshalImageRefsJSON(buildRefs)+ if buildImageRefsJSON == "" {+ buildImageRefsJSON = "[]"+ }+ if err := s.db.WithContext(ctx).+ Model(&models.Project{}).+ Where("id = ?", proj.ID).+ Updates(map[string]any{+ "image_refs_json": imageRefsJSON,+ "build_image_refs_json": buildImageRefsJSON,+ }).Error; err != nil {+ slog.WarnContext(ctx, "failed to persist project image refs", "projectID", proj.ID, "error", err)+ return+ }+ proj.ImageRefsJSON = imageRefsJSON+ proj.BuildImageRefsJSON = new(buildImageRefsJSON)+}++func (s *ProjectService) HandleProjectFilesChanged(ctx context.Context, paths []string) {+ if len(paths) == 0 || s.db == nil {+ return+ }++ affected, err := s.resolveProjectsByChangedPathsInternal(ctx, paths)+ if err != nil {+ slog.WarnContext(ctx, "failed to resolve changed project files", "error", err)+ return+ }+ for i := range affected {+ s.parsedCompose.invalidate(affected[i].ID)+ s.refreshProjectImageRefsInternal(ctx, &affected[i])+ }+}++func (s *ProjectService) BackfillProjectImageRefs(ctx context.Context) (int, error) {+ if s.db == nil {+ return 0, nil+ }++ var projectsList []models.Project+ if err := s.db.WithContext(ctx).+ Where("build_image_refs_json IS NULL").+ Find(&projectsList).Error; err != nil {+ return 0, errors.WrapIf(err, "list projects for image ref backfill")+ }+ for i := range projectsList {+ if err := ctx.Err(); err != nil {+ return i, err+ }+ s.refreshProjectImageRefsInternal(ctx, &projectsList[i])+ }+ return len(projectsList), nil+}++func (s *ProjectService) resolveProjectsByChangedPathsInternal(ctx context.Context, paths []string) ([]models.Project, error) {+ var projectsList []models.Project+ if err := s.db.WithContext(ctx).Find(&projectsList).Error; err != nil {+ return nil, errors.WrapIf(err, "list projects for changed paths")+ }++ seen := make(map[string]struct{})+ affected := make([]models.Project, 0)+ for _, changedPath := range paths {+ cleanChangedPath := filepath.Clean(changedPath)+ for _, proj := range projectsList {+ projectPath := filepath.Clean(proj.Path)+ if cleanChangedPath != projectPath && !strings.HasPrefix(cleanChangedPath, projectPath+string(os.PathSeparator)) {+ continue+ }+ if _, ok := seen[proj.ID]; ok {+ continue+ }+ seen[proj.ID] = struct{}{}+ affected = append(affected, proj)+ }+ }+ return affected, nil+}++func (s *ProjectService) SyncProjectsFromFileSystem(ctx context.Context) error {+ // Serialized because the walk and the cleanup are two halves of one+ // decision: overlapping syncs let an older walk's cleanup delete a project a+ // newer walk had just upserted, because the older walk's `seen` set predates+ // it. Filesystem-watcher debounces fire these back to back.+ s.syncMu.Lock()+ defer s.syncMu.Unlock()++ followProjectSymlinks := s.settingsService.GetBoolSetting(ctx, "followProjectSymlinks", false)+ projectsDir, err := s.GetProjectsDirectory(ctx)+ if err != nil {+ slog.WarnContext(ctx, "unable to prepare projects directory", "error", err)+ return nil+ }++ discoveredProjects, discoveryErr := projects.DiscoverProjectDirectories(projectsDir, followProjectSymlinks, s.config.ProjectScanMaxDepth)+ if discoveryErr != nil {+ if os.IsNotExist(discoveryErr) {+ return nil+ }+ return errors.WrapIff(discoveryErr, "Failed to discover projects in %q", projectsDir)+ }++ renameSyncState := s.activeProjectRenameSyncStateInternal(ctx)+ seen := map[string]struct{}{}+ for _, discoveredProject := range discoveredProjects {+ if renameSyncState.skipDiscoveredPathInternal(discoveredProject.Path) {+ continue+ }+ if uerr := s.upsertProjectForDir(ctx, discoveredProject.DirName, discoveredProject.Path); uerr != nil {+ slog.WarnContext(ctx, "failed to sync project from folder", "dir", discoveredProject.Path, "error", uerr)+ continue+ }+ seen[discoveredProject.Path] = struct{}{}+ }+ renameSyncState.markProtectedPathsSeenInternal(seen)++ if cerr := s.cleanupDBProjectsInternal(ctx, seen, followProjectSymlinks, projectsDir, s.config.ProjectScanMaxDepth); cerr != nil {+ slog.WarnContext(ctx, "error during DB cleanup of projects", "error", cerr)+ }++ return nil+}++func (s *ProjectService) upsertProjectForDir(ctx context.Context, dirName, dirPath string) error {+ var existing models.Project+ err := s.db.WithContext(ctx).+ Where("path = ?", dirPath).+ First(&existing).Error++ composeMetadata, serviceCountErr := s.loadComposeMetadataForSyncInternal(ctx, dirPath, dirName)++ if errors.Is(err, gorm.ErrRecordNotFound) {+ // Create a minimal project entry+ reason := "Project discovered from filesystem, status pending Docker service query"+ proj := &models.Project{+ Name: composeMetadata.resolvedProjectName,+ DirName: new(dirName),+ Path: dirPath,+ Status: models.ProjectStatusUnknown,+ StatusReason: new(reason),+ ServiceCount: composeMetadata.serviceCount,+ RunningCount: 0,+ ComposeProjectName: composeMetadata.composeProjectName,+ }+ slog.InfoContext(ctx, "Discovered new project with unknown status",+ "project", dirName,+ "path", dirPath,+ "reason", reason)+ if serviceCountErr != nil {+ slog.WarnContext(ctx, "failed to read compose service count during project discovery", "project", dirName, "path", dirPath, "error", serviceCountErr)+ }+ if cerr := s.db.WithContext(ctx).Create(proj).Error; cerr != nil {+ return errors.WrapIff(cerr, "create project for %q failed", dirPath)+ }+ s.warnDuplicateComposeNameForPathInternal(ctx, composeMetadata.resolvedProjectName, dirPath, proj.ID)+ return nil+ }+ if err != nil {+ return errors.WrapIff(err, "query existing project for %q failed", dirPath)+ }++ updates := map[string]any{}+ if existing.Path != dirPath {+ updates["path"] = dirPath+ }+ if existing.DirName == nil || *existing.DirName != dirName {+ updates["dir_name"] = dirName+ }+ if serviceCountErr == nil && existing.ServiceCount != composeMetadata.serviceCount {+ updates["service_count"] = composeMetadata.serviceCount+ } else if serviceCountErr != nil {+ slog.WarnContext(ctx, "failed to refresh compose service count during project sync", "projectID", existing.ID, "path", dirPath, "error", serviceCountErr)+ }+ if serviceCountErr == nil && mo.PointerToOption(existing.ComposeProjectName) != mo.PointerToOption(composeMetadata.composeProjectName) {+ updates["compose_project_name"] = composeMetadata.composeProjectName+ }+ if serviceCountErr == nil {+ if composeMetadata.explicitProjectName {+ if existing.Name != composeMetadata.resolvedProjectName {+ updates["name"] = composeMetadata.resolvedProjectName+ }+ } else if normalizedExistingName := projects.NormalizeProjectName(existing.Name); normalizedExistingName != existing.Name {+ updates["name"] = normalizedExistingName+ }+ }+ if len(updates) == 0 {+ return nil+ }++ updates["updated_at"] = time.Now()+ if uerr := s.db.WithContext(ctx).+ Model(&models.Project{}).+ Where("id = ?", existing.ID).+ Updates(updates).Error; uerr != nil {+ return errors.WrapIff(uerr, "update project %s failed", existing.ID)+ }+ if serviceCountErr == nil {+ s.warnDuplicateComposeNameForPathInternal(ctx, composeMetadata.resolvedProjectName, dirPath, existing.ID)+ }+ return nil+}++func (s *ProjectService) warnDuplicateComposeNameForPathInternal(ctx context.Context, composeProjectName, dirPath, projectID string) {+ if strings.TrimSpace(composeProjectName) == "" {+ return+ }++ var count int64+ if err := s.db.WithContext(ctx).+ Model(&models.Project{}).+ Where("name = ? AND path <> ? AND id <> ?", composeProjectName, dirPath, projectID).+ Count(&count).Error; err != nil {+ slog.WarnContext(ctx, "failed to check duplicate compose project names during project sync", "composeProjectName", composeProjectName, "path", dirPath, "error", err)+ return+ }+ if count > 0 {+ slog.WarnContext(ctx, "multiple project directories resolve to the same compose project name", "composeProjectName", composeProjectName, "path", dirPath, "duplicates", count)+ }+}++func (s *ProjectService) cleanupDBProjectsInternal(ctx context.Context, seen map[string]struct{}, followProjectSymlinks bool, projectsDir string, maxDepth int) error {+ var all []models.Project+ if err := s.db.WithContext(ctx).Find(&all).Error; err != nil {+ return errors.WrapIf(err, "list projects for cleanup failed")+ }++ // Decide deletions without performing them. Collecting decisions up front lets+ // the mass-wipe guard veto an entire suspicious pass (e.g. the projects volume+ // is unmounted, so every path is missing at once) before any rows are removed.+ candidates := 0+ pendingDeletions := make([]projectCleanupDecision, 0)+ tempDeletions := make([]projectCleanupDecision, 0)+ for _, p := range all {+ if skipProjectCleanupInternal(p, seen) {+ continue+ }+ if isInternalScratchProjectInternal(p) {+ tempDeletions = append(tempDeletions, projectCleanupDecision{project: p, reason: "removed internal Arcane scratch record (project-update/gitops temp dir)"})+ continue+ }+ // Projects inside filesystem snapshot/trash directories (e.g. BTRFS+ // #snapshot) are point-in-time copies mistakenly registered by earlier+ // discovery passes. The decision is name-based, not missing-path-based,+ // so it bypasses the mass-wipe guard like the scratch records above.+ if rel := getProjectRelativePathInternal(projectsDir, p.Path); rel != "" && projects.PathContainsSnapshotDirectory(rel) {+ tempDeletions = append(tempDeletions, projectCleanupDecision{project: p, reason: "removed project inside a filesystem snapshot/trash directory"})+ continue+ }+ candidates+++ if decision, remove := s.evaluateProjectCleanupInternal(ctx, p, followProjectSymlinks, projectsDir, maxDepth).Get(); remove {+ pendingDeletions = append(pendingDeletions, decision)+ }+ }++ for _, decision := range tempDeletions {+ s.deleteProjectDuringCleanupInternal(ctx, decision.project, decision.reason)+ }++ if cleanupWouldMassWipeInternal(ctx, candidates, len(pendingDeletions), projectsDir) {+ return nil+ }++ for _, decision := range pendingDeletions {+ s.deleteProjectDuringCleanupInternal(ctx, decision.project, decision.reason)+ }+ return nil+}++// cleanupWouldMassWipeInternal reports whether the pending deletions look like an+// accidental mass wipe rather than legitimate removals. It engages when a single+// pass would prune more than one project AND more than half of the cleanup+// candidates — so the table cannot be near-emptied at once (e.g. when the projects+// directory is unmounted or mis-mapped and every path goes missing), no matter how+// few projects the deployment has. A single removal is always allowed: it is+// indistinguishable from a legitimate "deleted my only project" and is not a mass+// wipe. When the guard engages it logs a WARN pointing the operator at the likely+// volume/mount misconfiguration and the caller skips every deletion in the pass.+func cleanupWouldMassWipeInternal(ctx context.Context, candidates, deleteCount int, projectsDir string) bool {+ if deleteCount <= 1 || deleteCount*2 <= candidates {+ return false+ }++ slog.WarnContext(ctx,+ "skipping project cleanup: this reconcile would delete most projects in a single pass, which usually means the projects directory is empty, unmounted, or mis-mapped; preserving DB records — check the projects volume is mounted and mapped correctly",+ "wouldDelete", deleteCount,+ "cleanupCandidates", candidates,+ "projectsDir", projectsDir,+ )+ return true+}++func skipProjectCleanupInternal(p models.Project, seen map[string]struct{}) bool {+ // Skip paths seen in this pass.+ if _, ok := seen[p.Path]; ok {+ return true+ }++ // Skip projects whose lifecycle is owned by the gitops system. Their compose+ // files may not exist on disk yet (e.g. during a sync or after an SSH/clone+ // failure) and should never be deleted here.+ return p.GitOpsManagedBy != nil && strings.TrimSpace(*p.GitOpsManagedBy) != ""+}++// isInternalScratchProjectInternal reports whether a project row was imported from+// one of Arcane's own scratch directories (project-update preview/backup, or GitOps+// sync-stage/backup). Such rows are never real user projects — a crash or restart+// mid-operation can leak the scratch dir, which the filesystem discovery then imports.+// They are force-removed during cleanup regardless of whether the dir still exists.+func isInternalScratchProjectInternal(p models.Project) bool {+ if projects.IsInternalScratchDirName(p.Name) || projects.IsInternalScratchDirName(filepath.Base(p.Path)) {+ return true+ }+ return p.DirName != nil && projects.IsInternalScratchDirName(*p.DirName)+}++// evaluateProjectCleanupInternal decides whether a project that was not seen in+// the current filesystem pass should be pruned. It performs only read-only checks+// (warning in place for the "keep" cases); the actual deletion is deferred to the+// caller so the mass-wipe guard can veto an entire suspicious pass.+func (s *ProjectService) evaluateProjectCleanupInternal(ctx context.Context, p models.Project, followProjectSymlinks bool, projectsDir string, maxDepth int) mo.Option[projectCleanupDecision] {+ if s.projectExceedsScanDepthInternal(p, projectsDir, maxDepth) {+ return mo.Some(projectCleanupDecision{project: p, reason: "removed project: directory is beyond the configured scan depth"})+ }++ validDir, err := projects.IsProjectDirectoryPath(p.Path, followProjectSymlinks)+ if err != nil {+ return evaluateProjectPathErrorInternal(ctx, p, err)+ }+ if !validDir {+ return mo.Some(projectCleanupDecision{project: p, reason: "removed project: path is no longer a valid project directory"})+ }++ return s.evaluateProjectComposeFileInternal(ctx, p)+}++func (s *ProjectService) projectExceedsScanDepthInternal(p models.Project, projectsDir string, maxDepth int) bool {+ // Remove projects that still exist on disk but now fall outside the configured+ // scan depth (e.g. after PROJECT_SCAN_MAX_DEPTH was lowered). They are no+ // longer discovered, so they must not linger in the list. Projects at the+ // projects root or outside it (relativePath == "") are left to the on-disk+ // validation below.+ if maxDepth <= 0 {+ return false+ }++ rel := getProjectRelativePathInternal(projectsDir, p.Path)+ return rel != "" && strings.Count(rel, "/")+1 > maxDepth+}++func evaluateProjectPathErrorInternal(ctx context.Context, p models.Project, err error) mo.Option[projectCleanupDecision] {+ if os.IsNotExist(err) {+ return mo.Some(projectCleanupDecision{project: p, reason: "removed project: directory no longer exists"})+ }++ slog.WarnContext(ctx, "stat error during cleanup; keeping DB record", "path", p.Path, "error", err)+ return mo.None[projectCleanupDecision]()+}++func (s *ProjectService) evaluateProjectComposeFileInternal(ctx context.Context, p models.Project) mo.Option[projectCleanupDecision] {+ _, err := s.ResolveProjectComposeFile(ctx, &p)+ if err == nil {+ return mo.None[projectCleanupDecision]()+ }++ // The project directory still exists here (it passed the directory-validity+ // check above). Only prune the DB record when the directory genuinely has no+ // compose file. Any other resolution failure — an ambiguous match ("multiple+ // custom compose files"), an unreadable directory, a transient parse error —+ // means the project still has compose content on disk and may be deployable.+ // Deleting it would silently destroy a live project whose files are intact, so+ // keep the record and warn instead.+ if !errors.Is(err, common.ErrComposeFileNotFound) {+ slog.WarnContext(ctx, "project directory present but compose file unresolved during cleanup; keeping DB record",+ "projectID", p.ID, "path", p.Path, "error", err)+ return mo.None[projectCleanupDecision]()+ }++ return mo.Some(projectCleanupDecision{project: p, reason: "removed orphaned project: directory present but contains no compose file"})+}++// deleteProjectDuringCleanupInternal removes a project record discovered to be+// stale during the filesystem reconcile. Every removal is logged (WARN on+// success, ERROR on failure) so this destructive operation always leaves an+// audit trail — previously successful deletions were silent.+func (s *ProjectService) deleteProjectDuringCleanupInternal(ctx context.Context, p models.Project, reason string, attrs ...any) {+ logAttrs := make([]any, 0, 6+len(attrs))+ logAttrs = append(logAttrs, "projectID", p.ID, "name", p.Name, "path", p.Path)+ logAttrs = append(logAttrs, attrs...)++ if derr := s.db.WithContext(ctx).Delete(&models.Project{}, "id = ?", p.ID).Error; derr != nil {+ slog.ErrorContext(ctx, "failed to delete project during filesystem cleanup",+ append(logAttrs, "reason", reason, "error", derr)...)+ return+ }++ slog.WarnContext(ctx, reason, logAttrs...)+}++// loadComposeMetadataForSyncInternal loads the compose file once and returns+// the service count plus compose-go's effective project name.+func (s *ProjectService) loadComposeMetadataForSyncInternal(ctx context.Context, dirPath, dirName string) (composeSyncMetadataInternal, error) {+ normName := projects.NormalizeProjectName(dirName)+ meta := composeSyncMetadataInternal{+ resolvedProjectName: normName,+ }++ cfg := s.settingsService.GetSettingsOrDefaults(ctx)+ projectsDirectory, pErr := projects.GetProjectsDirectory(ctx, strings.TrimSpace(cfg.ProjectsDirectory.Value))+ if pErr != nil {+ return meta, pErr+ }++ var dockerClient *client.Client+ if s.dockerService != nil {+ dockerClient, _ = s.dockerService.GetClient(ctx)+ }+ pathMapper := projects.NewPathMapperForConfiguredDirectory(+ ctx,+ s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"),+ "/app/data/projects",+ dockerClient,+ )++ autoInjectEnv := utils.BoolOrDefault(cfg.AutoInjectEnv.Value, false)++ // First, try loading without forcing a project name so compose-go can+ // resolve COMPOSE_PROJECT_NAME from the .env file. If this fails (e.g.+ // no .env and directory name is not a valid compose project name), fall+ // back to the normalized directory name.+ proj, _, err := projects.LoadComposeProjectFromDir(ctx, dirPath, "", projectsDirectory, autoInjectEnv, pathMapper)+ if err != nil {+ proj, _, err = projects.LoadComposeProjectFromDir(ctx, dirPath, normName, projectsDirectory, autoInjectEnv, pathMapper)+ if err != nil {+ return meta, err+ }+ } else if proj.Name != "" && proj.Name != normName {+ meta.explicitProjectName = true+ }++ meta.serviceCount = len(proj.Services)+ if proj.Name != "" {+ meta.resolvedProjectName = proj.Name+ }++ // If compose-go resolved a different name (from COMPOSE_PROJECT_NAME),+ // store it so we can match containers correctly.+ if proj.Name != "" && proj.Name != normName {+ meta.composeProjectName = new(proj.Name)+ }++ return meta, nil+}diff --git a/backend/internal/project/project_update.go b/backend/internal/project/project_update.gonew file mode 100644index 000000000..4c60ead4c--- /dev/null+++ b/backend/internal/project/project_update.go@@ -0,0 +1,766 @@+package project++import (+ "context"+ stderrors "errors"+ "log/slog"+ "os"+ "path/filepath"+ "strings"++ "emperror.dev/emperror"+ "emperror.dev/errors"++ "github.com/compose-spec/compose-go/v2/loader"+ composetypes "github.com/compose-spec/compose-go/v2/types"+ "github.com/docker/compose/v5/pkg/api"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/volumes"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/types/v2/project"+ "gorm.io/gorm"+)++func (s *ProjectService) UpdateProject(ctx context.Context, projectID string, name *string, composeContent, envContent, overrideContent *string, fileTreeRevision *string, fileChanges []project.ProjectFileChange, user models.User) (*models.Project, error) {+ proj, projectsDirectory, err := s.getProjectForUpdate(ctx, projectID)+ if err != nil {+ return nil, err+ }++ name = resolveAuthoritativeProjectNameInternal(&proj, name, composeContent)+ renameRequested := isProjectRenameRequestedInternal(&proj, name)+ if err := s.recoverProjectRenameJournalForProjectInternal(ctx, projectID); err != nil {+ if renameRequested {+ return nil, err+ }+ slog.WarnContext(ctx, "project rename journal recovery failed before non-rename update; continuing", "projectID", projectID, "error", err)+ } else {+ proj, projectsDirectory, err = s.getProjectForUpdate(ctx, projectID)+ if err != nil {+ return nil, err+ }+ name = resolveAuthoritativeProjectNameInternal(&proj, name, composeContent)+ }++ if err := ensureProjectMutableInternal(&proj); err != nil {+ return nil, err+ }+ if len(fileChanges) > 0 && isGitOpsManagedProjectInternal(&proj) {+ return nil, common.Classify(common.ErrProjectFileForbidden, errors.New("Forbidden project file path: GitOps-managed project files cannot be edited"))+ }++ if err := s.ensureProjectStoppedForRenameInternal(ctx, &proj, name); err != nil {+ return nil, err+ }++ volumeMigration, err := s.prepareProjectRenameVolumeMigrationForUpdateInternal(ctx, &proj, name, projectsDirectory, composeContent, envContent, overrideContent, fileTreeRevision, fileChanges)+ if err != nil {+ return nil, err+ }++ renameJournal := s.prepareProjectRenameJournalInternal(&proj, name, projectsDirectory, volumeMigration)++ backup, cleanupBackup, err := s.prepareProjectUpdateBackupInternal(ctx, projectsDirectory, proj.Path, composeContent, envContent, overrideContent, fileChanges)+ if err != nil {+ return nil, err+ }+ defer cleanupBackup()++ journalActive, err := s.startProjectRenameJournalInternal(ctx, renameJournal)+ if err != nil {+ return nil, err+ }++ projectStateCommitted := false+ if err := withProjectRenameRollback(ctx, &proj, &projectStateCommitted, func() error {+ return s.applyProjectUpdateWithRenameJournalInternal(ctx, &proj, name, projectsDirectory, composeContent, envContent, overrideContent, fileTreeRevision, fileChanges, volumeMigration, renameJournal, &journalActive, &projectStateCommitted)+ }); err != nil {+ err = s.handleProjectUpdateFailureInternal(ctx, projectID, projectsDirectory, &proj, backup, &journalActive, projectStateCommitted, err)+ return nil, err+ }++ s.refreshProjectAfterContentUpdateInternal(ctx, &proj, composeContent, overrideContent, fileChanges)+ s.logProjectUpdateEventInternal(ctx, &proj, composeContent, envContent, overrideContent, fileChanges, user)++ slog.InfoContext(ctx, "project updated", "projectID", proj.ID, "name", proj.Name)+ return &proj, nil+}++// resolveAuthoritativeProjectNameInternal enforces that a top-level `name:` in+// the compose file is authoritative over the submitted project name. For+// name-only renames, it checks the compose file on disk so the lock can't be+// bypassed via the API.+func resolveAuthoritativeProjectNameInternal(proj *models.Project, name *string, composeContent *string) *string {+ if composeContent != nil {+ if yamlName := projects.ComposeContentProjectName(*composeContent); yamlName != "" {+ return &yamlName+ }+ return name+ }+ if name != nil {+ if onDiskCompose, _, readErr := projects.ReadProjectFiles(proj.Path, ""); readErr == nil {+ if yamlName := projects.ComposeContentProjectName(onDiskCompose); yamlName != "" {+ return &yamlName+ }+ }+ }+ return name+}++func (s *ProjectService) prepareProjectUpdateBackupInternal(ctx context.Context, projectsDirectory, projectPath string, composeContent, envContent, overrideContent *string, fileChanges []project.ProjectFileChange) (*projects.ProjectUpdateBackup, func(), error) {+ if composeContent == nil && envContent == nil && overrideContent == nil && len(fileChanges) == 0 {+ return nil, func() {}, nil+ }++ scope := projects.BuildUpdateBackupScope(projectPath, composeContent, envContent, overrideContent, fileChanges)+ if scope.IsEmpty() {+ return nil, func() {}, nil+ }++ backup, err := backupProjectDirectoryInternal(ctx, projectsDirectory, projectPath, scope)+ if err != nil {+ return nil, nil, err+ }++ return backup, func() { _ = os.RemoveAll(backup.BackupDir) }, nil+}++func (s *ProjectService) applyProjectUpdateWithRenameJournalInternal(ctx context.Context, proj *models.Project, name *string, projectsDirectory string, composeContent, envContent, overrideContent *string, fileTreeRevision *string, fileChanges []project.ProjectFileChange, volumeMigration volumes.Migration, renameJournal *projectRenameJournalInternal, journalActive *bool, projectStateCommitted *bool) (err error) {+ volumeMigrationApplied := false+ defer func() {+ stateCommitted := projectStateCommitted != nil && *projectStateCommitted+ if err != nil && volumeMigrationApplied && !stateCommitted {+ if rollbackErr := volumeMigration.Rollback(ctx); rollbackErr != nil {+ err = stderrors.Join(err, errors.WrapIf(rollbackErr, "failed to rollback project volume rename"))+ }+ }+ }()++ if err = s.applyProjectRenameIfNeeded(proj, name, projectsDirectory); err != nil {+ return err+ }+ if err = s.persistProjectFileChanges(ctx, proj, fileTreeRevision, fileChanges); err != nil {+ return err+ }+ if err = s.persistUpdatedProjectFiles(ctx, proj, projectsDirectory, composeContent, envContent, overrideContent); err != nil {+ return err+ }+ if err = s.applyProjectVolumeMigrationForUpdateInternal(ctx, volumeMigration, renameJournal, &volumeMigrationApplied); err != nil {+ return err+ }+ if err = s.saveProjectUpdateInternal(ctx, proj); err != nil {+ return err+ }+ if projectStateCommitted != nil {+ *projectStateCommitted = true+ }+ s.finalizeProjectRenameAfterCommitInternal(ctx, proj.ID, volumeMigration, renameJournal, journalActive)+ return nil+}++func (s *ProjectService) saveProjectUpdateInternal(ctx context.Context, proj *models.Project) error {+ tx := s.db.WithContext(ctx).Begin()+ if tx.Error != nil {+ return errors.WrapIf(tx.Error, "failed to start project update transaction")+ }++ txCommitted := false+ defer func() {+ if !txCommitted {+ _ = tx.Rollback().Error+ }+ }()++ if err := tx.Save(proj).Error; err != nil {+ return errors.WrapIf(err, "failed to update project")+ }+ if err := tx.Commit().Error; err != nil {+ return errors.WrapIf(err, "failed to commit project update")+ }+ txCommitted = true+ return nil+}++func (s *ProjectService) handleProjectUpdateFailureInternal(ctx context.Context, projectID, projectsDirectory string, proj *models.Project, backup *projects.ProjectUpdateBackup, journalActive *bool, projectStateCommitted bool, err error) error {+ if projectStateCommitted {+ return err+ }++ if backup != nil {+ if restoreErr := restoreProjectDirectoryBackupInternal(ctx, projectsDirectory, proj.Path, backup); restoreErr != nil {+ err = stderrors.Join(err, errors.WrapIf(restoreErr, "failed to restore project files after update failure"))+ }+ }+ if *journalActive {+ if recoverErr := s.recoverProjectRenameJournalForProjectInternal(ctx, projectID); recoverErr != nil {+ err = stderrors.Join(err, errors.WrapIf(recoverErr, "project rename recovery failed"))+ } else {+ *journalActive = false+ }+ }+ return err+}++func (s *ProjectService) logProjectUpdateEventInternal(ctx context.Context, proj *models.Project, composeContent, envContent, overrideContent *string, fileChanges []project.ProjectFileChange, user models.User) {+ metadata := models.JSON{+ "action": "update",+ "projectID": proj.ID,+ "projectName": proj.Name,+ }+ if composeContent != nil {+ metadata["composeUpdated"] = true+ }+ if envContent != nil {+ metadata["envUpdated"] = true+ }+ if overrideContent != nil {+ metadata["overrideUpdated"] = true+ }+ if len(fileChanges) > 0 {+ metadata["projectFilesUpdated"] = true+ metadata["projectFileChangeCount"] = len(fileChanges)+ }+ s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, proj.ID, proj.Name, user, metadata, "could not log project update action")+}++func (s *ProjectService) refreshProjectAfterContentUpdateInternal(ctx context.Context, proj *models.Project, composeContent, overrideContent *string, fileChanges []project.ProjectFileChange) {+ if composeContent == nil && overrideContent == nil && len(fileChanges) == 0 {+ return+ }++ s.refreshComposeProjectNameInternal(ctx, proj)+ s.refreshProjectImageRefsInternal(ctx, proj)+ if err := s.updateProjectStatusandCountsInternal(ctx, proj.ID, proj.Status); err != nil {+ slog.WarnContext(ctx, "failed to update service counts after compose edit", "projectID", proj.ID, "error", err)+ }+}++func (s *ProjectService) ApplyGitSyncProjectFiles(ctx context.Context, projectID string, composeContent string, gitEnvContent *string, gitOverrideContent *string, gitOverrideFileName string, user models.User) (*models.Project, error) {+ proj, projectsDirectory, err := s.getProjectForUpdate(ctx, projectID)+ if err != nil {+ return nil, err+ }+ if err := ensureProjectMutableInternal(&proj); err != nil {+ return nil, err+ }++ envUpdate, err := s.prepareGitSyncEnvUpdateInternal(proj.Path, gitEnvContent)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to resolve git env state")+ }++ if err := validateComposeContentForUpdate(ctx, projectsDirectory, proj.Path, proj.Name, composeContent, envUpdate.effectiveContent, gitOverrideContent, gitOverrideFileName, true); err != nil {+ return nil, errors.WrapIf(err, "invalid compose file")+ }++ if err := projects.WriteComposeFile(projectsDirectory, proj.Path, composeContent); err != nil {+ return nil, errors.WrapIf(err, "failed to save compose file")+ }+ if err := persistGitSyncEnvFilesInternal(proj.Path, projectsDirectory, envUpdate); err != nil {+ return nil, errors.WrapIf(err, "failed to sync git env files")+ }+ if err := projects.WriteComposeOverrideFile(projectsDirectory, proj.Path, gitOverrideContent, gitOverrideFileName); err != nil {+ return nil, errors.WrapIf(err, "failed to sync git override file")+ }+ if err := s.db.WithContext(ctx).Save(&proj).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to update project")+ }+ s.refreshComposeProjectNameInternal(ctx, &proj)+ s.refreshProjectImageRefsInternal(ctx, &proj)++ // Recalculate service counts and status after compose file sync+ if err := s.updateProjectStatusandCountsInternal(ctx, proj.ID, proj.Status); err != nil {+ slog.WarnContext(ctx, "failed to update service counts after git sync", "projectID", proj.ID, "error", err)+ }++ metadata := models.JSON{+ "action": "git_sync_update",+ "projectID": proj.ID,+ "projectName": proj.Name,+ "composeUpdated": true,+ "envUpdated": gitEnvContent != nil,+ "overrideUpdated": gitOverrideContent != nil,+ }+ if gitEnvContent == nil {+ metadata["envSourceRemoved"] = true+ }+ s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, proj.ID, proj.Name, user, metadata, "could not log git sync project update action")++ return &proj, nil+}++func (s *ProjectService) getProjectForUpdate(ctx context.Context, projectID string) (models.Project, string, error) {+ var proj models.Project+ if err := s.db.WithContext(ctx).First(&proj, "id = ?", projectID).Error; err != nil {+ if errors.Is(err, gorm.ErrRecordNotFound) {+ return models.Project{}, "", errors.New("project not found")+ }+ return models.Project{}, "", errors.WrapIf(err, "failed to get project")+ }++ projectsDirectory, err := projects.GetProjectsDirectory(ctx, s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"))+ if err != nil {+ return models.Project{}, "", errors.WrapIf(err, "failed to get projects directory")+ }++ if err := s.EnsureProjectPathUnderRoot(ctx, &proj, false); err != nil {+ return models.Project{}, "", err+ }++ return proj, projectsDirectory, nil+}++func (s *ProjectService) prepareProjectRenameVolumeMigrationForUpdateInternal(ctx context.Context, proj *models.Project, name *string, projectsDirectory string, composeContent, envContent, overrideContent *string, fileTreeRevision *string, fileChanges []project.ProjectFileChange) (volumes.Migration, error) {+ if !isProjectRenameRequestedInternal(proj, name) {+ return nil, nil+ }+ if err := requireProjectFileRevisionInternal(fileTreeRevision, fileChanges); err != nil {+ return nil, err+ }++ if composeContent == nil && envContent == nil && overrideContent == nil && len(fileChanges) == 0 {+ return s.prepareProjectRenameVolumeMigrationInternal(ctx, proj, name)+ }++ previewPath, err := os.MkdirTemp(projectsDirectory, ".project-update-preview-*")+ if err != nil {+ return nil, errors.WrapIf(err, "failed to create project update preview")+ }+ defer func() {+ if removeErr := os.RemoveAll(previewPath); removeErr != nil {+ slog.WarnContext(ctx, "failed to remove project update preview", "path", previewPath, "error", removeErr)+ }+ }()++ if err := projects.CopyDirectoryContents(proj.Path, previewPath, nil); err != nil {+ return nil, errors.WrapIf(err, "failed to prepare project update preview")+ }++ previewProject := *proj+ previewProject.Path = previewPath+ if err := s.applyProjectFileChangesInternal(ctx, &previewProject, "", fileChanges); err != nil {+ return nil, errors.WrapIf(err, "failed to prepare project update preview")+ }+ if err := s.persistUpdatedProjectFiles(ctx, &previewProject, projectsDirectory, composeContent, envContent, overrideContent); err != nil {+ return nil, errors.WrapIf(err, "failed to prepare project update preview")+ }++ return s.prepareProjectRenameVolumeMigrationInternal(ctx, &previewProject, name)+}++func (s *ProjectService) prepareProjectRenameVolumeMigrationInternal(ctx context.Context, proj *models.Project, name *string) (volumes.Migration, error) {+ oldComposeName, newComposeName, ok := projectRenameVolumeMigrationComposeNamesInternal(s, proj, name)+ if !ok {+ return nil, nil+ }++ composeProject, _, err := s.loadComposeProjectForProjectInternal(ctx, proj, nil)+ if err != nil {+ if errors.Is(err, common.ErrProjectComposeFileNotFound) {+ return nil, nil+ }+ return nil, errors.WrapIf(err, "failed to load compose project for volume rename")+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to connect to Docker for volume rename")+ }++ return volumes.PlanMigration(ctx, dockerClient, composeProject, oldComposeName, newComposeName)+}++func projectRenameVolumeMigrationComposeNamesInternal(s *ProjectService, proj *models.Project, name *string) (string, string, bool) {+ if s == nil || s.dockerService == nil || proj == nil || name == nil {+ return "", "", false+ }++ newProjectName := strings.TrimSpace(*name)+ if newProjectName == "" || proj.Name == newProjectName || proj.Status != models.ProjectStatusStopped {+ return "", "", false+ }++ oldComposeName := projects.NormalizeProjectName(proj.Name)+ newComposeName := projects.NormalizeProjectName(newProjectName)+ if oldComposeName == "" || newComposeName == "" || oldComposeName == newComposeName {+ return "", "", false+ }++ return oldComposeName, newComposeName, true+}++func isProjectRenameRequestedInternal(proj *models.Project, name *string) bool {+ if proj == nil || name == nil {+ return false+ }+ newName := strings.TrimSpace(*name)+ return newName != "" && proj.Name != newName+}++func isGitOpsManagedProjectInternal(proj *models.Project) bool {+ return proj != nil && proj.GitOpsManagedBy != nil && strings.TrimSpace(*proj.GitOpsManagedBy) != ""+}++func requireProjectFileRevisionInternal(fileTreeRevision *string, fileChanges []project.ProjectFileChange) error {+ if len(fileChanges) == 0 {+ return nil+ }+ if fileTreeRevision == nil || strings.TrimSpace(*fileTreeRevision) == "" {+ return common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: file tree revision is required"))+ }+ return nil+}++func (s *ProjectService) persistProjectFileChanges(ctx context.Context, proj *models.Project, fileTreeRevision *string, fileChanges []project.ProjectFileChange) error {+ if err := requireProjectFileRevisionInternal(fileTreeRevision, fileChanges); err != nil {+ return err+ }+ expectedRevision := ""+ if fileTreeRevision != nil {+ expectedRevision = strings.TrimSpace(*fileTreeRevision)+ }+ return s.applyProjectFileChangesInternal(ctx, proj, expectedRevision, fileChanges)+}++func (s *ProjectService) applyProjectFileChangesInternal(ctx context.Context, proj *models.Project, expectedRevision string, fileChanges []project.ProjectFileChange) error {+ if len(fileChanges) == 0 {+ return nil+ }+ opts := s.projectFileApplyOptionsInternal(ctx, proj, expectedRevision)+ if err := projects.ApplyProjectFileChanges(proj.Path, fileChanges, opts); err != nil {+ return wrapProjectFileErrorInternal(err)+ }+ return nil+}++func (s *ProjectService) projectFileApplyOptionsInternal(ctx context.Context, proj *models.Project, expectedRevision string) projects.ProjectFileApplyOptions {+ composeFileName := projects.DefaultComposeFileName+ if composeFile, err := s.ResolveProjectComposeFile(ctx, proj); err == nil {+ composeFileName = filepath.Base(composeFile)+ }++ return projects.ProjectFileApplyOptions{+ ExpectedRevision: strings.TrimSpace(expectedRevision),+ MaxDepth: s.config.ProjectFileTreeMaxDepth,+ MaxEntries: projects.DefaultProjectFileTreeMaxEntries,+ SkipDirectories: s.config.ProjectScanSkipDirs,+ ComposeFileName: composeFileName,+ }+}++// wrapProjectFileErrorInternal maps pkg/projects sentinel errors onto the+// common.ProjectFile* error structs the handlers translate into HTTP statuses.+func wrapProjectFileErrorInternal(err error) error {+ switch {+ case errors.Is(err, projects.ErrProjectFileRevisionConflict):+ return common.Classify(common.ErrProjectFileConflict, errors.WithStackIf(err))+ case errors.Is(err, projects.ErrProjectFileOutsideProjectDirectory),+ errors.Is(err, projects.ErrProjectFileProtectedPath),+ errors.Is(err, projects.ErrProjectFileSymlinkPath):+ return common.Classify(common.ErrProjectFileForbidden, errors.WrapIf(err, "Forbidden project file path"))++ default:+ return common.Classify(common.ErrProjectFileBadRequest, errors.++ // projectUpdateBackupScopeInternal derives the exact set of paths an update+ // can mutate so the backup never copies out-of-scope data directories.+ // Changes that fail normalization are skipped: the apply step rejects them+ // before mutating anything, so there is nothing to roll back for them.+ WrapIf(err, "Invalid project file request"))+ }+}++func backupProjectDirectoryInternal(ctx context.Context, projectsDirectory, projectPath string, scope projects.ProjectUpdateBackupScope) (*projects.ProjectUpdateBackup, error) {+ projectAbs, err := filepath.Abs(projectPath)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to resolve project path")+ }+ projectAbs = filepath.Clean(projectAbs)++ rootAbs, err := filepath.Abs(projectsDirectory)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to resolve projects directory")+ }+ rootAbs = filepath.Clean(rootAbs)+ if !projects.IsSafeSubdirectory(rootAbs, projectAbs) || projectAbs == rootAbs {+ return nil, errors.New("project path is outside projects directory")+ }++ backupPath, err := os.MkdirTemp(projectsDirectory, ".project-update-backup-*")+ if err != nil {+ return nil, errors.WrapIf(err, "failed to create project backup directory")+ }+ // Tolerate files Arcane cannot read (e.g. foreign-owned secrets): skip them+ // in the backup so an unrelated unreadable file can't block the whole save.+ // The skipped paths are recorded so the rollback restore can preserve them.+ backup, err := projects.BackupProjectUpdateScope(projectAbs, backupPath, scope)+ if err != nil {+ _ = os.RemoveAll(backupPath)+ return nil, errors.WrapIf(err, "failed to backup project files")+ }+ if len(backup.Skipped) > 0 {+ slog.WarnContext(ctx, "skipped unreadable files while backing up project; they will be left untouched on rollback", "projectPath", projectAbs, "skipped", backup.Skipped)+ }+ return backup, nil+}++func restoreProjectDirectoryBackupInternal(ctx context.Context, projectsDirectory, projectPath string, backup *projects.ProjectUpdateBackup) error {+ projectAbs, err := filepath.Abs(projectPath)+ if err != nil {+ return errors.WrapIf(err, "failed to resolve project path")+ }+ projectAbs = filepath.Clean(projectAbs)++ rootAbs, err := filepath.Abs(projectsDirectory)+ if err != nil {+ return errors.WrapIf(err, "failed to resolve projects directory")+ }+ rootAbs = filepath.Clean(rootAbs)+ if !projects.IsSafeSubdirectory(rootAbs, projectAbs) || projectAbs == rootAbs {+ return errors.New("project path is outside projects directory")+ }++ slog.DebugContext(ctx, "restoring project directory backup", "path", projectAbs, "backup", backup.BackupDir)+ if err := os.MkdirAll(projectAbs, utils.DirPerm); err != nil {+ return errors.WrapIf(err, "failed to recreate project directory")+ }+ // Restore only the paths the update could have mutated, in place: files+ // that were skipped during backup (unreadable, e.g. foreign-owned secrets)+ // are preserved, and out-of-scope files are never touched.+ if err := projects.RestoreProjectUpdateBackup(projectAbs, backup); err != nil {+ return errors.WrapIf(err, "failed to restore project backup")+ }+ return nil+}++func (s *ProjectService) persistUpdatedProjectFiles(ctx context.Context, proj *models.Project, projectsDirectory string, composeContent, envContent, overrideContent *string) error {+ switch {+ case composeContent != nil:+ effectiveEnvContent, err := s.resolveEffectiveEnvContentForUpdateInternal(proj.Path, envContent)+ if err != nil {+ return errors.WrapIf(err, "invalid compose file")+ }+ valOverride, valOverrideName := projects.ResolveEffectiveOverrideForValidation(proj.Path, overrideContent)+ if err := validateComposeContentForUpdate(ctx, projectsDirectory, proj.Path, proj.Name, *composeContent, effectiveEnvContent, valOverride, valOverrideName, false); err != nil {+ return errors.WrapIf(err, "invalid compose file")+ }+ if err := projects.WriteComposeFile(projectsDirectory, proj.Path, *composeContent); err != nil {+ return errors.WrapIf(err, "failed to save project files")+ }+ if envContent != nil {+ if err := persistEffectiveEnvContentInternal(proj.Path, projectsDirectory, *envContent); err != nil {+ return errors.WrapIf(err, "failed to save project files")+ }+ } else if err := s.ensureEffectiveEnvFileInternal(proj.Path, projectsDirectory); err != nil {+ return errors.WrapIf(err, "failed to save project files")+ }+ if err := projects.ApplyOverrideFileChange(projectsDirectory, proj.Path, overrideContent); err != nil {+ return errors.WrapIf(err, "failed to save project files")+ }+ case overrideContent != nil:+ if err := s.persistOverrideOnlyUpdateInternal(ctx, proj, projectsDirectory, envContent, overrideContent); err != nil {+ return err+ }+ case envContent != nil:+ if err := persistEffectiveEnvContentInternal(proj.Path, projectsDirectory, *envContent); err != nil {+ return err+ }+ }++ return nil+}++// persistOverrideOnlyUpdateInternal handles a save that changes the override (and+// optionally the env) without touching the base compose file. It validates the+// on-disk base merged with the requested override so a base that is only valid+// *with* its override still validates, and a delete that would break the base+// fails before touching disk.+func (s *ProjectService) persistOverrideOnlyUpdateInternal(ctx context.Context, proj *models.Project, projectsDirectory string, envContent, overrideContent *string) error {+ baseContent, _, err := projects.ReadProjectFiles(proj.Path, "")+ if err != nil {+ return errors.WrapIf(err, "failed to read project files")+ }+ effectiveEnvContent, err := s.resolveEffectiveEnvContentForUpdateInternal(proj.Path, envContent)+ if err != nil {+ return errors.WrapIf(err, "invalid compose file")+ }+ valOverride, valOverrideName := projects.ResolveEffectiveOverrideForValidation(proj.Path, overrideContent)+ if err := validateComposeContentForUpdate(ctx, projectsDirectory, proj.Path, proj.Name, baseContent, effectiveEnvContent, valOverride, valOverrideName, false); err != nil {+ return errors.WrapIf(err, "invalid compose file")+ }+ if envContent != nil {+ if err := persistEffectiveEnvContentInternal(proj.Path, projectsDirectory, *envContent); err != nil {+ return errors.WrapIf(err, "failed to save project files")+ }+ }+ if err := projects.ApplyOverrideFileChange(projectsDirectory, proj.Path, overrideContent); err != nil {+ return errors.WrapIf(err, "failed to save project files")+ }+ return nil+}++func validateComposeContentForUpdate(ctx context.Context, projectsDirectory, projectPath, projectName, composeContent string, effectiveEnvContent *string, overrideContent *string, overrideFileName string, lenient bool) (err error) {+ defer func() {+ if panicErr := emperror.Recover(recover()); panicErr != nil {+ err = errors.WrapIf(panicErr, "compose file contains invalid syntax")+ }+ }()++ fullEnvMap, envErr := projects.BuildValidationEnvironment(projectsDirectory, projectPath, effectiveEnvContent)+ if envErr != nil {+ return envErr+ }++ if err := projects.ValidateIncludePathsForContent(projectPath, composeContent, fullEnvMap); err != nil {+ return err+ }++ validationProjectName := projects.NormalizeProjectName(projectName)+ configFiles := []composetypes.ConfigFile{+ {Filename: filepath.Join(projectPath, "compose.yaml"), Content: []byte(composeContent)},+ }+ // When an override is supplied, validate the *merged* config as `docker+ // compose` would deploy it. Overrides can add services and introduce their+ // own include: paths, so they get the same traversal check as the base and+ // are layered on top (listed after the base so the override wins).+ if overrideContent != nil {+ if err := projects.ValidateIncludePathsForContent(projectPath, *overrideContent, fullEnvMap); err != nil {+ return err+ }+ overrideName := strings.TrimSpace(overrideFileName)+ if overrideName == "" {+ overrideName = projects.DefaultComposeOverrideFileName+ }+ configFiles = append(configFiles, composetypes.ConfigFile{+ Filename: filepath.Join(projectPath, overrideName),+ Content: []byte(*overrideContent),+ })+ }++ cfg := composetypes.ConfigDetails{+ Version: api.ComposeVersion,+ WorkingDir: projectPath,+ ConfigFiles: configFiles,+ Environment: composetypes.Mapping(fullEnvMap),+ }++ missingIncludeLoader := projects.NewMissingIncludeStubLoader(projectPath)+ defer missingIncludeLoader.Cleanup()++ err = projects.WithTransientValidationEnvFile(projectPath, effectiveEnvContent, func() error {+ _, loadErr := loader.LoadWithContext(ctx, cfg, func(opts *loader.Options) {+ opts.ResourceLoaders = append([]loader.ResourceLoader{missingIncludeLoader}, opts.ResourceLoaders...)+ if validationProjectName != "" {+ opts.SetProjectName(validationProjectName, false)+ }+ if lenient {+ projects.ApplyLenientLoaderOptions(ctx, opts, cfg.ConfigFiles[0].Filename)+ }+ })+ return loadErr+ })++ return err+}++func (s *ProjectService) ensureProjectStoppedForRenameInternal(ctx context.Context, proj *models.Project, name *string) error {+ if !isProjectRenameRequestedInternal(proj, name) {+ return nil+ }+ if proj.Status != models.ProjectStatusStopped && proj.Status != models.ProjectStatusUnknown {+ return errors.Errorf("project must be stopped before renaming (current status: %s)", proj.Status)+ }++ services, err := s.GetProjectServices(ctx, proj.ID)+ if err != nil {+ slog.WarnContext(ctx, "failed to resolve project status before rename", "projectID", proj.ID, "error", err)+ return errors.WrapIff(err, "project must be stopped before renaming (current status: %s): failed to verify live status", proj.Status)+ }++ status := calculateProjectStatus(services)+ if status != models.ProjectStatusStopped {+ return errors.Errorf("project must be stopped before renaming (current status: %s)", status)+ }++ serviceCount, runningCount := getServiceCounts(services)+ proj.Status = models.ProjectStatusStopped+ proj.StatusReason = nil+ proj.ServiceCount = serviceCount+ proj.RunningCount = runningCount+ return nil+}++func (s *ProjectService) applyProjectRenameIfNeeded(proj *models.Project, name *string, projectsDirectory string) error {+ if name == nil {+ return nil+ }++ newName := strings.TrimSpace(*name)+ if newName == "" || proj.Name == newName {+ return nil+ }++ if proj.Status != models.ProjectStatusStopped {+ return errors.Errorf("project must be stopped before renaming (current status: %s)", proj.Status)+ }++ newDirName := projects.SanitizeProjectName(newName)+ if newDirName == "" || strings.Trim(newDirName, "_") == "" {+ return errors.New("invalid project name: results in empty directory name")+ }++ currentPath := filepath.Clean(proj.Path)+ targetPath := filepath.Clean(filepath.Join(projectsDirectory, newDirName))+ if currentPath != targetPath {+ if _, statErr := os.Stat(targetPath); statErr == nil {+ return errors.Errorf("project directory already exists: %s", targetPath)+ } else if !os.IsNotExist(statErr) {+ return errors.WrapIf(statErr, "failed to check project directory rename target")+ }++ if err := os.Rename(currentPath, targetPath); err != nil {+ return errors.WrapIf(err, "failed to rename project directory")+ }++ proj.Path = targetPath+ }++ proj.DirName = &newDirName+ proj.Name = newName+ return nil+}++func (s *ProjectService) UpdateProjectIncludeFile(ctx context.Context, projectID, relativePath, content string, user models.User) error {+ proj, err := s.getMutableProjectInternal(ctx, projectID)+ if err != nil {+ return err+ }++ // Normalize and persist project path to ensure include writes occur under projects root+ if err := s.EnsureProjectPathUnderRoot(ctx, proj, true); err != nil {+ return err+ }++ if err := projects.WriteIncludeFile(proj.Path, relativePath, content); err != nil {+ return errors.WrapIf(err, "failed to update include file")+ }+ s.refreshProjectImageRefsInternal(ctx, proj)++ // Recalculate service counts since include files can define services+ if err := s.updateProjectStatusandCountsInternal(ctx, proj.ID, proj.Status); err != nil {+ slog.WarnContext(ctx, "failed to update service counts after include file edit", "projectID", proj.ID, "error", err)+ }++ metadata := models.JSON{+ "action": "update_include",+ "projectID": proj.ID,+ "projectName": proj.Name,+ "relativePath": relativePath,+ }+ s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, proj.ID, proj.Name, user, metadata, "could not log project include update action")++ slog.InfoContext(ctx, "project include file updated", "projectID", proj.ID, "file", relativePath)+ return nil+}diff --git a/backend/internal/services/project_service_test.go b/backend/internal/project/service_test.gosimilarity index 95%rename from backend/internal/services/project_service_test.gorename to backend/internal/project/service_test.goindex d2f51b441..985e2fd42 100644--- a/backend/internal/services/project_service_test.go+++ b/backend/internal/project/service_test.go@@ -1,4 +1,4 @@-package services+package projectimport ("context"@@ -20,8 +20,15 @@ import (composetypes "github.com/compose-spec/compose-go/v2/types"composeapi "github.com/docker/compose/v5/pkg/api"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/volumes""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/backend/v2/pkg/projects"@@ -30,16 +37,19 @@ import (imagetypes "github.com/getarcaneapp/arcane/types/v2/image"projecttypes "github.com/getarcaneapp/arcane/types/v2/project""github.com/libtnb/sqlite"+ dockerauthconfig "github.com/moby/moby/api/pkg/authconfig""github.com/moby/moby/api/types/container"dockertypesimage "github.com/moby/moby/api/types/image"dockerregistry "github.com/moby/moby/api/types/registry""github.com/moby/moby/api/types/volume"+ "github.com/moby/moby/client""github.com/opencontainers/go-digest""github.com/samber/mo""github.com/stretchr/testify/assert""github.com/stretchr/testify/require"buildtypes "go.getarcane.app/builds/types""go.getarcane.app/updater/labels"+ "go.uber.org/fx/fxtest""gorm.io/gorm""github.com/getarcaneapp/arcane/backend/v2/internal/database"@@ -50,14 +60,18 @@ type testBuildBuilder struct {err error}-func (b testBuildBuilder) BuildImage(_ context.Context, _ buildtypes.BuildRequest, _ io.Writer, _ string) (*buildtypes.BuildResult, error) {+func (b testBuildBuilder) BuildImage(_ context.Context, _ string, _ buildtypes.BuildRequest, _ io.Writer, _ string, _ *models.User) (*buildtypes.BuildResult, error) {if b.err != nil {return nil, b.err}return &buildtypes.BuildResult{Provider: "local"}, nil}-var _ buildtypes.Builder = testBuildBuilder{}+func (testBuildBuilder) BuildSettings() buildtypes.BuildSettings {+ return buildtypes.BuildSettings{}+}++var _ buildServiceInternal = testBuildBuilder{}func setupProjectTestDB(t *testing.T) *database.DB {t.Helper()@@ -67,6 +81,91 @@ func setupProjectTestDB(t *testing.T) *database.DB {return &database.DB{DB: db}}+func newSettingsServiceForTestInternal(t *testing.T, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ executor, err := actors.NewExecutor(t.Context(), runtime, "project-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "project-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, executor.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, executor, effects)+}++func newTestDockerClientInternal(t *testing.T, server *httptest.Server) *client.Client {+ t.Helper()++ cli, err := client.New(+ client.WithHost(server.URL),+ client.WithAPIVersion("1.41"),+ client.WithHTTPClient(server.Client()),+ )+ require.NoError(t, err)+ t.Cleanup(func() { _ = cli.Close() })+ return cli+}++func decodeRegistryAuthInternal(t *testing.T, encoded string) dockerregistry.AuthConfig {+ t.Helper()+ cfg, err := dockerauthconfig.Decode(encoded)+ require.NoError(t, err)+ return *cfg+}++func newImagePullServerWithObserverInternal(t *testing.T, inspectByRef map[string]dockertypesimage.InspectResponse, onPull func(fullRef string, authHeader string)) *httptest.Server {+ t.Helper()++ server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ switch {+ case strings.HasSuffix(r.URL.Path, "/images/create"):+ fullRef := strings.TrimSpace(r.URL.Query().Get("fromImage"))+ tag := strings.TrimSpace(r.URL.Query().Get("tag"))+ if fullRef != "" && tag != "" {+ lastSlash := strings.LastIndex(fullRef, "/")+ lastColon := strings.LastIndex(fullRef, ":")+ if lastColon <= lastSlash {+ fullRef += ":" + tag+ }+ }+ if onPull != nil {+ onPull(fullRef, strings.TrimSpace(r.Header.Get("X-Registry-Auth")))+ }+ w.Header().Set("Content-Type", "application/json")+ _ = json.NewEncoder(w).Encode(map[string]string{"status": "Pulled", "id": fullRef})+ case strings.Contains(r.URL.Path, "/images/") && strings.HasSuffix(r.URL.Path, "/json"):+ path := r.URL.Path+ imagePathIndex := strings.Index(path, "/images/")+ if !assert.NotEqual(t, -1, imagePathIndex) {+ return+ }+ encodedRef := strings.TrimSuffix(path[imagePathIndex+len("/images/"):], "/json")+ imageRef, err := url.PathUnescape(encodedRef)+ if !assert.NoError(t, err) {+ return+ }+ inspect, ok := inspectByRef[imageRef]+ if !ok {+ http.Error(w, "not found", http.StatusNotFound)+ return+ }+ w.Header().Set("Content-Type", "application/json")+ _ = json.NewEncoder(w).Encode(inspect)+ default:+ http.NotFound(w, r)+ }+ }))+ t.Cleanup(server.Close)+ return server+}+func TestProjectService_RefreshProjectImageRefs_PersistsBuildMetadata(t *testing.T) {ctx := context.Background()db := setupProjectTestDB(t)@@ -173,14 +272,10 @@ func setupProjectDestroyTestServiceInternal(t *testing.T) (*ProjectService, *datprojectsDir := t.TempDir()require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- eventService := NewEventService(db, config.Load(), nil)+ eventService := event.NewEventService(db, config.Load(), nil)return NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load()), db, projectsDir}-func newProjectImagePullServer(t *testing.T, inspectByRef map[string]dockertypesimage.InspectResponse) *httptest.Server {- return newProjectImagePullServerWithObserverInternal(t, inspectByRef, nil)-}-func TestProjectService_DestroyProject_RemovesFilesWhenRequested(t *testing.T) {ctx := context.Background()svc, db, projectsDir := setupProjectDestroyTestServiceInternal(t)@@ -237,61 +332,6 @@ func TestProjectService_DestroyProject_PreservesFilesWhenRequested(t *testing.T)assert.FileExists(t, filepath.Join(projectsDir, trashName, "project-data.txt"))}-func newProjectImagePullServerWithObserverInternal(t *testing.T, inspectByRef map[string]dockertypesimage.InspectResponse, onPull func(fullRef string, authHeader string)) *httptest.Server {- t.Helper()-- server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {- switch {- case strings.HasSuffix(r.URL.Path, "/images/create"):- fullRef := strings.TrimSpace(r.URL.Query().Get("fromImage"))- tag := strings.TrimSpace(r.URL.Query().Get("tag"))- if fullRef != "" && tag != "" {- lastSlash := strings.LastIndex(fullRef, "/")- lastColon := strings.LastIndex(fullRef, ":")- if lastColon <= lastSlash {- fullRef += ":" + tag- }- }- if onPull != nil {- onPull(fullRef, strings.TrimSpace(r.Header.Get("X-Registry-Auth")))- }-- w.Header().Set("Content-Type", "application/json")- _, _ = io.WriteString(w, fmt.Sprintf(`{"status":"Pulled","id":%q}`+"\n", fullRef))- return- case strings.Contains(r.URL.Path, "/images/") && strings.HasSuffix(r.URL.Path, "/json"):- path := r.URL.Path- imagePathIndex := strings.Index(path, "/images/")- if !assert.NotEqual(t, -1, imagePathIndex) {- return- }- encodedRef := strings.TrimSuffix(path[imagePathIndex+len("/images/"):], "/json")- imageRef, err := url.PathUnescape(encodedRef)- if !assert.NoError(t, err) {- return- }-- inspect, ok := inspectByRef[imageRef]- if !ok {- http.Error(w, "not found", http.StatusNotFound)- return- }-- w.Header().Set("Content-Type", "application/json")- if !assert.NoError(t, json.NewEncoder(w).Encode(inspect)) {- return- }- return- default:- http.NotFound(w, r)- }- }))-- t.Cleanup(server.Close)-- return server-}-func TestProjectService_GetProjectFromDatabaseByID(t *testing.T) {db := setupProjectTestDB(t)ctx := context.Background()@@ -322,8 +362,6 @@ func TestProjectService_GetProjectFromDatabaseByID(t *testing.T) {}func TestProjectService_GetServiceCounts(t *testing.T) {- svc := &ProjectService{}-tests := []struct {name stringservices []ProjectServiceInfo@@ -358,7 +396,7 @@ func TestProjectService_GetServiceCounts(t *testing.T) {for _, tt := range tests {t.Run(tt.name, func(t *testing.T) {- total, running := svc.getServiceCounts(tt.services)+ total, running := getServiceCounts(tt.services)assert.Equal(t, tt.wantTotal, total)assert.Equal(t, tt.wantRunning, running)})@@ -366,8 +404,6 @@ func TestProjectService_GetServiceCounts(t *testing.T) {}func TestProjectService_CalculateProjectStatus(t *testing.T) {- svc := &ProjectService{}-tests := []struct {name stringservices []ProjectServiceInfo@@ -406,7 +442,7 @@ func TestProjectService_CalculateProjectStatus(t *testing.T) {for _, tt := range tests {t.Run(tt.name, func(t *testing.T) {- got := svc.calculateProjectStatus(tt.services)+ got := calculateProjectStatus(tt.services)assert.Equal(t, tt.want, got)})}@@ -439,58 +475,22 @@ func TestProjectService_UpdateProjectStatusInternal(t *testing.T) {}func TestProjectService_IncrementStatusCounts(t *testing.T) {- svc := &ProjectService{}running := 0stopped := 0- svc.incrementStatusCounts(models.ProjectStatusRunning, &running, &stopped)+ incrementStatusCounts(models.ProjectStatusRunning, &running, &stopped)assert.Equal(t, 1, running)assert.Equal(t, 0, stopped)- svc.incrementStatusCounts(models.ProjectStatusStopped, &running, &stopped)+ incrementStatusCounts(models.ProjectStatusStopped, &running, &stopped)assert.Equal(t, 1, running)assert.Equal(t, 1, stopped)- svc.incrementStatusCounts(models.ProjectStatusUnknown, &running, &stopped)+ incrementStatusCounts(models.ProjectStatusUnknown, &running, &stopped)assert.Equal(t, 1, running)assert.Equal(t, 1, stopped)}-func TestProjectService_FormatDockerPorts(t *testing.T) {- tests := []struct {- name string- input []container.PortSummary- expected []string- }{- {- name: "public port",- input: []container.PortSummary{- {PublicPort: 8080, PrivatePort: 80, Type: "tcp"},- },- expected: []string{"8080:80/tcp"},- },- {- name: "private only",- input: []container.PortSummary{- {PrivatePort: 80, Type: "tcp"},- },- expected: []string{"80/tcp"},- },- {- name: "empty",- input: []container.PortSummary{},- expected: nil,- },- }-- for _, tt := range tests {- t.Run(tt.name, func(t *testing.T) {- got := formatDockerPorts(tt.input)- assert.Equal(t, tt.expected, got)- })- }-}-func TestProjectService_GetProjectByComposeName(t *testing.T) {ctx := context.Background()@@ -557,7 +557,7 @@ func TestProjectService_GetProjectByComposeName(t *testing.T) {require.NoError(t, err)assert.Equal(t, original.ID, found.ID)- cachedProjectID, cached := svc.getCachedComposeProjectIDInternal("myproject").Get()+ cachedProjectID, cached := svc.composeNames.projectID("myproject").Get()require.True(t, cached)assert.Equal(t, original.ID, cachedProjectID)@@ -574,7 +574,7 @@ func TestProjectService_GetProjectByComposeName(t *testing.T) {require.NoError(t, err)assert.Equal(t, replacement.ID, found.ID)- cachedProjectID, cached = svc.getCachedComposeProjectIDInternal("myproject").Get()+ cachedProjectID, cached = svc.composeNames.projectID("myproject").Get()require.True(t, cached)assert.Equal(t, replacement.ID, cachedProjectID)})@@ -594,7 +594,7 @@ func TestProjectService_GetProjectByComposeName(t *testing.T) {require.NoError(t, err)assert.Equal(t, original.ID, found.ID)- cachedProjectID, cached := svc.getCachedComposeProjectIDInternal("myapp").Get()+ cachedProjectID, cached := svc.composeNames.projectID("myapp").Get()require.True(t, cached)assert.Equal(t, original.ID, cachedProjectID)@@ -607,7 +607,7 @@ func TestProjectService_GetProjectByComposeName(t *testing.T) {require.Error(t, err)assert.Contains(t, err.Error(), "project not found")- _, cached = svc.getCachedComposeProjectIDInternal("myapp").Get()+ _, cached = svc.composeNames.projectID("myapp").Get()assert.False(t, cached)})}@@ -629,18 +629,18 @@ func TestProjectService_PullProjectImages_UpdatesCurrentImageRecordAfterPull(t *imageDigest := digest.FromString("team-app-digest").String()buildRepository := "arcane.local/demo-builder/service"- server := newProjectImagePullServer(t, map[string]dockertypesimage.InspectResponse{+ server := newImagePullServerWithObserverInternal(t, map[string]dockertypesimage.InspectResponse{imageRef: {ID: imageID,RepoTags: []string{imageRef},RepoDigests: []string{repository + "@" + imageDigest},},- })+ }, nil)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- eventService := NewEventService(db, nil, nil)- imageUpdateService := NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)- imageService := NewImageService(db, dockerService, nil, imageUpdateService, nil, eventService)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}+ eventService := event.NewEventService(db, nil, nil)+ imageUpdateService := imageupdate.NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)+ imageService := image.NewImageService(db, dockerService, nil, imageUpdateService, nil, eventService)svc := NewProjectService(db, settingsService, nil, imageService, dockerService, nil, nil, nil, config.Load())projectPath := createComposeProjectDir(t, projectsDir, "compose-pull")@@ -686,7 +686,7 @@ func TestProjectService_PullProjectImages_UpdatesCurrentImageRecordAfterPull(t *CheckTime: now.Add(2 * time.Minute),}).Error)- require.NoError(t, svc.PullProjectImages(ctx, projectRecord.ID, io.Discard, systemUser, nil))+ require.NoError(t, svc.PullProjectImages(ctx, projectRecord.ID, io.Discard, models.SystemUser, nil))// sha256:old-* records represent update records for OTHER containers still running// the old image. Pulling the image for one container must not mark them as up-to-date@@ -725,18 +725,18 @@ func TestProjectService_EnsureImagesPresent_UpdatesCurrentImageRecordAfterPull(timageID := "sha256:team-api"imageDigest := digest.FromString("team-api-digest").String()- server := newProjectImagePullServer(t, map[string]dockertypesimage.InspectResponse{+ server := newImagePullServerWithObserverInternal(t, map[string]dockertypesimage.InspectResponse{imageRef: {ID: imageID,RepoTags: []string{imageRef},RepoDigests: []string{repository + "@" + imageDigest},},- })+ }, nil)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- eventService := NewEventService(db, nil, nil)- imageUpdateService := NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)- imageService := NewImageService(db, dockerService, nil, imageUpdateService, nil, eventService)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}+ eventService := event.NewEventService(db, nil, nil)+ imageUpdateService := imageupdate.NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)+ imageService := image.NewImageService(db, dockerService, nil, imageUpdateService, nil, eventService)svc := NewProjectService(db, settingsService, nil, imageService, dockerService, nil, nil, nil, config.Load())require.NoError(t, db.Create(&models.ImageUpdateRecord{@@ -751,7 +751,7 @@ func TestProjectService_EnsureImagesPresent_UpdatesCurrentImageRecordAfterPull(trequire.NoError(t, svc.ensureImagesPresent(ctx, map[string]projects.ImagePullMode{imageRef: projects.ImagePullModeAlways,- }, io.Discard, nil, systemUser))+ }, io.Discard, nil, models.SystemUser))// sha256:old-api may still be in use by another container — pulling for one container// must not clear it (fixes #2453).@@ -777,18 +777,18 @@ func TestProjectService_PullImageForService_UpdatesCurrentImageRecordAfterPull(timageID := "sha256:team-worker"imageDigest := digest.FromString("team-worker-digest").String()- server := newProjectImagePullServer(t, map[string]dockertypesimage.InspectResponse{+ server := newImagePullServerWithObserverInternal(t, map[string]dockertypesimage.InspectResponse{imageRef: {ID: imageID,RepoTags: []string{imageRef},RepoDigests: []string{repository + "@" + imageDigest},},- })+ }, nil)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- eventService := NewEventService(db, nil, nil)- imageUpdateService := NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)- imageService := NewImageService(db, dockerService, nil, imageUpdateService, nil, eventService)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}+ eventService := event.NewEventService(db, nil, nil)+ imageUpdateService := imageupdate.NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)+ imageService := image.NewImageService(db, dockerService, nil, imageUpdateService, nil, eventService)svc := NewProjectService(db, settingsService, nil, imageService, dockerService, nil, nil, nil, config.Load())require.NoError(t, db.Create(&models.ImageUpdateRecord{@@ -801,7 +801,7 @@ func TestProjectService_PullImageForService_UpdatesCurrentImageRecordAfterPull(tCheckTime: time.Now().UTC().Add(-time.Hour),}).Error)- require.NoError(t, svc.pullAndReconcileImageInternal(ctx, imageRef, io.Discard, systemUser, nil))+ require.NoError(t, svc.pullAndReconcileImageInternal(ctx, imageRef, io.Discard, models.SystemUser, nil))// sha256:old-worker may still be in use by another container — must not be cleared (fixes #2453).var oldRecord models.ImageUpdateRecord@@ -832,7 +832,7 @@ func TestProjectService_ComposePullSelectedServicesInternal_ReconcilesOnlyOnSuccpullsByRef := map[string]int{}authHeadersByRef := map[string]string{}- server := newProjectImagePullServerWithObserverInternal(t, map[string]dockertypesimage.InspectResponse{+ server := newImagePullServerWithObserverInternal(t, map[string]dockertypesimage.InspectResponse{privateImageRef: {ID: privateImageID,RepoTags: []string{privateImageRef},@@ -848,10 +848,10 @@ func TestProjectService_ComposePullSelectedServicesInternal_ReconcilesOnlyOnSuccauthHeadersByRef[fullRef] = authHeader})- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- eventService := NewEventService(db, nil, nil)- imageUpdateService := NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)- imageService := NewImageService(db, dockerService, nil, imageUpdateService, nil, eventService)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}+ eventService := event.NewEventService(db, nil, nil)+ imageUpdateService := imageupdate.NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)+ imageService := image.NewImageService(db, dockerService, nil, imageUpdateService, nil, eventService)svc := NewProjectService(db, settingsService, nil, imageService, dockerService, nil, nil, nil, config.Load())projectDef := &composetypes.Project{@@ -903,12 +903,12 @@ func TestProjectService_ComposePullSelectedServicesInternal_ReconcilesOnlyOnSuccEnabled: true,}}- require.NoError(t, svc.composePullSelectedServicesInternal(ctx, projectDef, []string{"app", "app-copy", "sidecar", "builder"}, systemUser, credentials))+ require.NoError(t, svc.composePullSelectedServicesInternal(ctx, projectDef, []string{"app", "app-copy", "sidecar", "builder"}, models.SystemUser, credentials))assert.Equal(t, 1, pullsByRef[privateImageRef], "duplicate service refs should only be pulled once")assert.Equal(t, 1, pullsByRef[publicImageRef], "selected public image should still be pulled")assert.Len(t, pullsByRef, 2, "build-backed services should not trigger image pulls")- privateAuth := decodeRegistryAuth(t, authHeadersByRef[privateImageRef])+ privateAuth := decodeRegistryAuthInternal(t, authHeadersByRef[privateImageRef])assert.Equal(t, "arcane-user", privateAuth.Username)assert.Equal(t, "arcane-token", privateAuth.Password)assert.Equal(t, "registry.example.com", privateAuth.ServerAddress)@@ -953,9 +953,9 @@ func TestProjectService_ComposePullSelectedServicesInternal_LeavesRecordsWhenPul}))t.Cleanup(failingServer.Close)- dockerService := &DockerClientService{client: newTestDockerClient(t, failingServer)}- imageUpdateService := NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)- imageService := NewImageService(db, dockerService, nil, imageUpdateService, nil, NewEventService(db, nil, nil))+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, failingServer)}+ imageUpdateService := imageupdate.NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)+ imageService := image.NewImageService(db, dockerService, nil, imageUpdateService, nil, event.NewEventService(db, nil, nil))svc := NewProjectService(db, settingsService, nil, imageService, dockerService, nil, nil, nil, config.Load())projectDef := &composetypes.Project{@@ -978,7 +978,7 @@ func TestProjectService_ComposePullSelectedServicesInternal_LeavesRecordsWhenPulCheckTime: time.Now().UTC().Add(-time.Hour),}).Error)- err = svc.composePullSelectedServicesInternal(ctx, projectDef, []string{"app"}, systemUser, nil)+ err = svc.composePullSelectedServicesInternal(ctx, projectDef, []string{"app"}, models.SystemUser, nil)require.Error(t, err)require.ErrorContains(t, err, "failed to pull image")@@ -1011,9 +1011,9 @@ func TestProjectService_UpdateProjectServicesHardFailsWhenPullFailsInternal(t *t}))t.Cleanup(failingServer.Close)- dockerService := &DockerClientService{client: newTestDockerClient(t, failingServer)}- imageUpdateService := NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)- imageService := NewImageService(db, dockerService, nil, imageUpdateService, nil, NewEventService(db, nil, nil))+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, failingServer)}+ imageUpdateService := imageupdate.NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)+ imageService := image.NewImageService(db, dockerService, nil, imageUpdateService, nil, event.NewEventService(db, nil, nil))projectPath := createComposeProjectDir(t, projectsDir, "compose-update-pull-fail")require.NoError(t, os.WriteFile(filepath.Join(projectPath, "compose.yaml"), []byte("services:\n app:\n image: "+imageRef+"\n"), 0o644))@@ -1047,7 +1047,7 @@ func TestProjectService_UpdateProjectServicesHardFailsWhenPullFailsInternal(t *t}svc := NewProjectService(db, settingsService, nil, imageService, dockerService, nil, nil, nil, config.Load())- err = svc.UpdateProjectServices(ctx, projectRecord.ID, []string{"app"}, systemUser)+ err = svc.UpdateProjectServices(ctx, projectRecord.ID, []string{"app"}, models.SystemUser)require.Error(t, err)require.ErrorContains(t, err, "pull updated service images")assert.False(t, upCalled, "compose up must not run after a pull failure")@@ -1076,17 +1076,17 @@ func TestProjectService_UpdateProjectServicesForcesRecreateInternal(t *testing.TimageID := "sha256:project-update-force"imageDigest := digest.FromString("project-update-force-digest").String()- server := newProjectImagePullServer(t, map[string]dockertypesimage.InspectResponse{+ server := newImagePullServerWithObserverInternal(t, map[string]dockertypesimage.InspectResponse{imageRef: {ID: imageID,RepoTags: []string{imageRef},RepoDigests: []string{repository + "@" + imageDigest},},- })+ }, nil)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- imageUpdateService := NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)- imageService := NewImageService(db, dockerService, nil, imageUpdateService, nil, NewEventService(db, nil, nil))+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}+ imageUpdateService := imageupdate.NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)+ imageService := image.NewImageService(db, dockerService, nil, imageUpdateService, nil, event.NewEventService(db, nil, nil))projectPath := createComposeProjectDir(t, projectsDir, "compose-update-force")require.NoError(t, os.WriteFile(filepath.Join(projectPath, "compose.yaml"), []byte("services:\n app:\n image: "+imageRef+"\n"), 0o644))@@ -1120,7 +1120,7 @@ func TestProjectService_UpdateProjectServicesForcesRecreateInternal(t *testing.T}svc := NewProjectService(db, settingsService, nil, imageService, dockerService, nil, nil, nil, config.Load())- err = svc.UpdateProjectServices(ctx, projectRecord.ID, []string{"app"}, systemUser)+ err = svc.UpdateProjectServices(ctx, projectRecord.ID, []string{"app"}, models.SystemUser)require.Error(t, err)assert.True(t, upCalled)assert.True(t, forceRecreate, "service updates must force recreate after pulling the updated image")@@ -1161,7 +1161,7 @@ func TestProjectService_UpdateProject_RenameFailsWhenVolumeMigrationPreparationFrequire.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)dockerServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {switch {case strings.HasSuffix(r.URL.Path, "/_ping"):@@ -1194,7 +1194,7 @@ func TestProjectService_UpdateProject_RenameFailsWhenVolumeMigrationPreparationFt.Cleanup(dockerServer.Close)t.Setenv("DOCKER_HOST", dockerHostFromProjectRuntimeServerURLInternal(t, dockerServer.URL))- dockerService := &DockerClientService{client: newTestDockerClient(t, dockerServer)}+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, dockerServer)}svc := NewProjectService(db, settingsService, eventService, nil, dockerService, nil, nil, nil, config.Load())originalDirName := "Foo"@@ -1236,7 +1236,7 @@ func TestProjectService_ApplyProjectUpdateWithRenameJournal_AppliesVolumeMigratisettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())migration := &fakeProjectVolumeRenameMigrationInternal{}@@ -1311,7 +1311,7 @@ func TestProjectService_PrepareProjectRenameVolumeMigrationForUpdate_UsesCompose}))t.Cleanup(dockerServer.Close)- dockerService := &DockerClientService{client: newTestDockerClient(t, dockerServer)}+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, dockerServer)}svc := NewProjectService(db, settingsService, nil, nil, dockerService, nil, nil, nil, config.Load())projectPath := filepath.Join(projectsDir, "nginx")@@ -1402,7 +1402,7 @@ func TestProjectService_ApplyProjectUpdateWithRenameJournal_RollsBackVolumeMigrasettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())migration := &fakeProjectVolumeRenameMigrationInternal{}@@ -1428,7 +1428,7 @@ func TestProjectService_ApplyProjectUpdateWithRenameJournal_RollsBackVolumeMigraprojectForUpdate := *projectjournalActive := falseprojectStateCommitted := false- err = svc.withProjectRenameRollback(ctx, &projectForUpdate, &projectStateCommitted, func() error {+ err = withProjectRenameRollback(ctx, &projectForUpdate, &projectStateCommitted, func() error {return svc.applyProjectUpdateWithRenameJournalInternal(ctx, &projectForUpdate, new("bar"), projectsDir, nil, nil, nil, nil, nil, migration, nil, &journalActive, &projectStateCommitted)})require.Error(t, err)@@ -1451,7 +1451,7 @@ func TestProjectService_ApplyProjectUpdateWithRenameJournal_SucceedsCommittedRensettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())migration := &fakeProjectVolumeRenameMigrationInternal{@@ -1473,7 +1473,7 @@ func TestProjectService_ApplyProjectUpdateWithRenameJournal_SucceedsCommittedRenprojectForUpdate := *projectjournalActive := falseprojectStateCommitted := false- err = svc.withProjectRenameRollback(ctx, &projectForUpdate, &projectStateCommitted, func() error {+ err = withProjectRenameRollback(ctx, &projectForUpdate, &projectStateCommitted, func() error {return svc.applyProjectUpdateWithRenameJournalInternal(ctx, &projectForUpdate, new("bar"), projectsDir, nil, nil, nil, nil, nil, migration, nil, &journalActive, &projectStateCommitted)})require.NoError(t, err)@@ -1502,8 +1502,8 @@ func TestProjectService_UpdateProject_ClearsJournalForNonRenameWhenRecoveryDockerequire.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- eventService := NewEventService(db, nil, nil)- kvService := NewKVService(db)+ eventService := event.NewEventService(db, nil, nil)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)oldDir := "nginx"@@ -1568,8 +1568,8 @@ func TestProjectService_UpdateProject_AllowsRenameAfterJournalRecoveryDockerUnavrequire.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- eventService := NewEventService(db, nil, nil)- kvService := NewKVService(db)+ eventService := event.NewEventService(db, nil, nil)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)oldDir := "nginx"@@ -1630,7 +1630,7 @@ func TestProjectService_UpdateProject_RenamesDirectoryWhenNameChanges(t *testingsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())configureProjectRuntimeDockerInternal(t, nil)@@ -1679,7 +1679,7 @@ func TestProjectService_UpdateProject_RenameFailsWhenTargetDirectoryExists(t *tesettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())configureProjectRuntimeDockerInternal(t, nil)@@ -1726,7 +1726,7 @@ func TestProjectService_UpdateProject_RenameFailsWhenProjectRunning(t *testing.TsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())originalDirName := "Foo"@@ -1771,7 +1771,7 @@ func TestProjectService_UpdateProject_RenameRejectsStaleStoppedWhenRuntimeIsRunnrequire.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())configureProjectRuntimeDockerInternal(t, []container.Summary{{@@ -1829,7 +1829,7 @@ func TestProjectService_UpdateProject_RenameResolvesUnknownStoppedStatusBeforeVorequire.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())server := newProjectRuntimeDockerServerInternal(t, nil)@@ -1887,7 +1887,7 @@ func TestProjectService_UpdateProject_RenameRejectsUnknownWhenRuntimeIsRunning(trequire.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())server := newProjectRuntimeDockerServerInternal(t, []container.Summary{@@ -1950,7 +1950,7 @@ func TestProjectService_UpdateProject_ValidatesComposeUsingExistingProjectName(tsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "demo"@@ -1993,7 +1993,7 @@ func TestProjectService_UpdateProject_AllowsMissingEnvFileDuringComposeValidatiosettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "env-required"@@ -2038,7 +2038,7 @@ func TestProjectService_UpdateProject_AllowsMissingLocalIncludeDuringComposeValisettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "include-new"@@ -2103,7 +2103,7 @@ func TestProjectService_UpdateProject_RejectsMissingExternalIncludeDuringComposesettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "include-external"@@ -2148,7 +2148,7 @@ func TestProjectService_CreateProject_RejectsExternalInclude(t *testing.T) {settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())require.NoError(t, os.WriteFile(filepath.Join(projectsDir, "metadata.yaml"), []byte("services: {}\n"), 0o644))@@ -2184,7 +2184,7 @@ func TestProjectService_CreateProject_RejectsArrayPathInclude(t *testing.T) {settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())require.NoError(t, os.WriteFile(filepath.Join(projectsDir, "metadata.yaml"), []byte("services: {}\n"), 0o644))@@ -2222,7 +2222,7 @@ func TestProjectService_CreateProject_WritesStagedProjectFiles(t *testing.T) {settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())includeContent := "services: {}\n"@@ -2276,7 +2276,7 @@ func TestProjectService_GetProjectDetails_UsesFileTreeMaxDepthForProjectFiles(tsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)cfg := config.Load()svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, cfg)@@ -2319,7 +2319,7 @@ func TestProjectService_UpdateProject_AppliesStagedProjectFileChanges(t *testingsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())project, err := svc.CreateProject(ctx, "editable-files", "services:\n app:\n image: nginx:alpine\n", nil, nil, models.User{@@ -2362,7 +2362,7 @@ func TestProjectService_UpdateProject_RejectsStaleProjectFileRevision(t *testingsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())project, err := svc.CreateProject(ctx, "stale-files", "services:\n app:\n image: nginx:alpine\n", nil, nil, models.User{@@ -2399,7 +2399,7 @@ func TestProjectService_UpdateProject_RejectsStaleDeepProjectFileRevision(t *tessettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())project, err := svc.CreateProject(ctx, "stale-deep-files", "services:\n app:\n image: nginx:alpine\n", nil, nil, models.User{@@ -2438,7 +2438,7 @@ func TestProjectService_GetProjectFileContent_RejectsExternalInclude(t *testing.settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "include-read"@@ -2479,7 +2479,7 @@ func TestProjectService_GetProjectFileContent_RejectsSymlinkInclude(t *testing.TsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "include-symlink"@@ -2523,7 +2523,7 @@ func TestProjectService_GetProjectFileContent_RejectsIntermediateSymlinkInclude(settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "include-intermediate-symlink"@@ -2567,7 +2567,7 @@ func TestProjectService_GetProjectFileContent_RejectsIntermediateSymlinkProjectFsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "project-file-intermediate-symlink"@@ -2604,7 +2604,7 @@ func TestProjectService_UpdateProject_UsesExistingEnvFileDuringComposeValidationsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "env-existing"@@ -2654,7 +2654,7 @@ func newProjectServiceForOverrideTestInternal(t *testing.T, dirName, baseComposesettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())projectPath := filepath.Join(projectsDir, dirName)@@ -2765,7 +2765,7 @@ func TestProjectService_UpdateProject_UsesProvidedEnvContentDuringComposeValidatsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "env-updated"@@ -2812,7 +2812,7 @@ func TestProjectService_UpdateProject_ReturnsEnvParseErrorDuringComposeValidatiosettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "env-invalid"@@ -2857,7 +2857,7 @@ func TestProjectService_UpdateProject_UsesGlobalEnvDuringComposeValidation(t *tesettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "global-env-update"@@ -2906,7 +2906,7 @@ func TestProjectService_UpdateProject_DoesNotResolveHostEnvThroughGlobalEnvDurinsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "host-env-guard"@@ -2950,7 +2950,7 @@ func TestProjectService_UpdateProject_DerivesProjectOverrideEnvWhenGitSourceExissettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "override-edit"@@ -2997,7 +2997,7 @@ func TestProjectService_UpdateProject_UnchangedGitEnvLeavesFilesUntouched(t *tessettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "unchanged-git-env"@@ -3050,8 +3050,7 @@ func TestProjectService_PersistEffectiveEnvContent_RemovesStaleOverrideWithoutGirequire.NoError(t, os.WriteFile(filepath.Join(projectPath, ".env"), []byte(effectiveContent), 0o600))require.NoError(t, os.WriteFile(filepath.Join(projectPath, "project.env"), []byte("TOKEN=stale\n"), 0o600))- svc := &ProjectService{}- require.NoError(t, svc.persistEffectiveEnvContentInternal(projectPath, projectsDir, effectiveContent))+ require.NoError(t, persistEffectiveEnvContentInternal(projectPath, projectsDir, effectiveContent))effectiveBytes, readErr := os.ReadFile(filepath.Join(projectPath, ".env"))require.NoError(t, readErr)@@ -3071,8 +3070,7 @@ func TestProjectService_PersistEffectiveEnvContent_RemovesStaleGitOverride(t *terequire.NoError(t, os.WriteFile(filepath.Join(projectPath, ".env.git"), []byte(effectiveContent), 0o600))require.NoError(t, os.WriteFile(filepath.Join(projectPath, "project.env"), []byte("TOKEN=stale\n"), 0o600))- svc := &ProjectService{}- require.NoError(t, svc.persistEffectiveEnvContentInternal(projectPath, projectsDir, effectiveContent))+ require.NoError(t, persistEffectiveEnvContentInternal(projectPath, projectsDir, effectiveContent))effectiveBytes, readErr := os.ReadFile(filepath.Join(projectPath, ".env"))require.NoError(t, readErr)@@ -3096,7 +3094,7 @@ func TestProjectService_UpdateProject_DeletingGitBackedKeyFallsBackToGit(t *testsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "override-delete"@@ -3146,7 +3144,7 @@ func TestProjectService_ApplyGitSyncProjectFiles_MigratesDirectEnvIntoProjectOvesettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "git-sync-migrate"@@ -3197,7 +3195,7 @@ func TestProjectService_ApplyGitSyncProjectFiles_PreservesGitEnvSyntax(t *testinsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "git-sync-env-syntax"@@ -3258,7 +3256,7 @@ func TestProjectService_ApplyGitSyncProjectFiles_NormalizesStaleCopiedGitOverridsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "git-sync-normalize"@@ -3306,7 +3304,7 @@ func TestProjectService_ApplyGitSyncProjectFiles_RemovesLegacyDeletedGitMasks(tsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "git-sync-delete-mask"@@ -3355,7 +3353,7 @@ func TestProjectService_ApplyGitSyncProjectFiles_RemovesGitEnvSource(t *testing.settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "git-sync-remove"@@ -3399,7 +3397,7 @@ func TestProjectService_ApplyGitSyncProjectFiles_WritesAndRemovesComposeOverridesettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "git-sync-override"@@ -3450,7 +3448,7 @@ func TestProjectService_ApplyGitSyncProjectFiles_UsesGlobalEnvDuringComposeValidsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "git-sync-global-env"@@ -3501,7 +3499,7 @@ func TestProjectService_ApplyGitSyncProjectFiles_TolerantOfUndefinedComposeVar(tsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "git-sync-undefined-var"@@ -3549,7 +3547,7 @@ func TestProjectService_PersistGitSyncEnvFiles_UsesPreparedState(t *testing.T) {settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "git-sync-prepared-state"@@ -3566,7 +3564,7 @@ func TestProjectService_PersistGitSyncEnvFiles_UsesPreparedState(t *testing.T) {require.NoError(t, os.WriteFile(filepath.Join(projectPath, "project.env"), []byte("TOKEN=unexpected\n"), 0o600))- require.NoError(t, svc.persistGitSyncEnvFilesInternal(projectPath, projectsDir, update))+ require.NoError(t, persistGitSyncEnvFilesInternal(projectPath, projectsDir, update))overrideBytes, readErr := os.ReadFile(filepath.Join(projectPath, "project.env"))require.NoError(t, readErr)@@ -3587,7 +3585,7 @@ func TestProjectService_GetProjectDetails_ReturnsEffectiveEnvContent(t *testing.settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "details-override"@@ -3714,7 +3712,7 @@ func TestProjectService_GetProjectDetails_IncludesUpdateInfo(t *testing.T) {require.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- imageService := &ImageService{db: db}+ imageService := image.NewImageService(db, nil, nil, nil, nil, nil)svc := NewProjectService(db, settingsService, nil, imageService, nil, nil, nil, nil, config.Load())projectPath := createComposeProjectDir(t, projectsDir, "updates-demo")@@ -3889,7 +3887,7 @@ func TestProjectService_ListProjects_FiltersByUpdateStatus(t *testing.T) {require.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- imageService := &ImageService{db: db}+ imageService := image.NewImageService(db, nil, nil, nil, nil, nil)svc := NewProjectService(db, settingsService, nil, imageService, nil, nil, nil, nil, config.Load())updatedPath := createComposeProjectDir(t, projectsDir, "updated-demo")@@ -3995,7 +3993,7 @@ func TestProjectService_ListProjects_FiltersByUpdateStatus(t *testing.T) {func TestBuildDiscoveredComposeProjectUpdateRowsInternal(t *testing.T) {db := setupProjectTestDB(t)ctx := context.Background()- imageService := &ImageService{db: db}+ imageService := image.NewImageService(db, nil, nil, nil, nil, nil)now := time.Now().UTC()require.NoError(t, db.Create(&models.ImageUpdateRecord{@@ -4064,7 +4062,7 @@ func TestBuildDiscoveredComposeProjectUpdateRowsInternal(t *testing.T) {func TestBuildDiscoveredComposeProjectUpdateRowsInternal_FallsBackToImageID(t *testing.T) {db := setupProjectTestDB(t)ctx := context.Background()- imageService := &ImageService{db: db}+ imageService := image.NewImageService(db, nil, nil, nil, nil, nil)require.NoError(t, db.Create(&models.ImageUpdateRecord{ID: "sha256:media-image",@@ -4580,7 +4578,7 @@ func TestProjectService_DeployProject_StopsOnBuildPreparationError(t *testing.T)}require.NoError(t, db.Create(proj).Error)- buildSvc := &BuildService{builder: testBuildBuilder{err: errors.New("boom build")}}+ buildSvc := testBuildBuilder{err: errors.New("boom build")}svc := NewProjectService(db, settingsService, nil, nil, nil, buildSvc, nil, nil, config.Load())err = svc.DeployProject(ctx, "p1", models.User{BaseModel: models.BaseModel{ID: "u1"}, Username: "tester"}, nil)@@ -4622,7 +4620,7 @@ func TestProjectService_DeployProject_BuildsGeneratedImageWithoutPull(t *testing}require.NoError(t, db.Create(proj).Error)- buildSvc := &BuildService{builder: testBuildBuilder{err: errors.New("boom build")}}+ buildSvc := testBuildBuilder{err: errors.New("boom build")}svc := NewProjectService(db, settingsService, nil, nil, nil, buildSvc, nil, nil, config.Load())err = svc.DeployProject(ctx, proj.ID, models.User{BaseModel: models.BaseModel{ID: "u1"}, Username: "tester"}, nil)@@ -5442,7 +5440,7 @@ func TestProjectService_UpdateProject_WritesThroughSymlinkedProjectPath(t *testirequire.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsRoot))require.NoError(t, settingsService.SetStringSetting(ctx, "followProjectSymlinks", "true"))- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())project := &models.Project{@@ -5497,7 +5495,7 @@ func TestProjectService_UpdateProject_WritesThroughExternalEnvSymlink(t *testingoriginalLinkTarget, err := os.Readlink(envPath)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "demo"project := &models.Project{@@ -5556,7 +5554,7 @@ func TestProjectService_UpdateProject_RestoresExternalEnvSymlinkTargetWhenProjecoriginalLinkTarget, err := os.Readlink(envPath)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "demo"project := &models.Project{@@ -5710,7 +5708,7 @@ func TestProjectService_UpdateProject_RenameRollsBackWhenFileRevisionIsStale(t *require.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())configureProjectRuntimeDockerInternal(t, nil)@@ -5772,7 +5770,7 @@ func TestProjectService_RecoverProjectRenameJournals_RollsBackUncommittedDirecto}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -5826,7 +5824,7 @@ func TestProjectService_RecoverProjectRenameJournals_StartedPhaseSkipsVolumeRoll}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -5889,7 +5887,7 @@ func TestProjectService_RecoverProjectRenameJournals_RelocatesTargetWhenBothPath}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -5952,7 +5950,7 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsStartedJournalWhenDir}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -6017,8 +6015,8 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsPreservedTargetJourna}))t.Cleanup(server.Close)- kvService := NewKVService(db)- svc := NewProjectService(db, nil, nil, nil, &DockerClientService{client: newTestDockerClient(t, server)}, nil, nil, nil, config.Load()).WithKVService(kvService)+ kvService := kv.NewKVService(db)+ svc := NewProjectService(db, nil, nil, nil, &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,OldName: "nginx",@@ -6079,7 +6077,7 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsCommittedJournal(t *t}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -6119,7 +6117,7 @@ func TestProjectService_FinalizeProjectRenameAfterCommit_ClearsJournalAfterSourc}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)journal := &projectRenameJournalInternal{ProjectID: project.ID,@@ -6160,7 +6158,7 @@ func TestProjectService_FinalizeProjectRenameAfterCommit_KeepsJournalWhenSourceC}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)journal := &projectRenameJournalInternal{ProjectID: project.ID,@@ -6244,8 +6242,8 @@ func TestProjectService_RecoverProjectRenameJournals_KeepsJournalWhenDirectoryRo}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -6338,8 +6336,8 @@ func TestProjectService_RecoverProjectRenameJournals_CompletesCommittedVolumeJou}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -6411,8 +6409,8 @@ func TestProjectService_RecoverProjectRenameJournals_RollsBackCommittedJournalWh}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -6521,8 +6519,8 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsJournalAfterDBRestore}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -6599,7 +6597,7 @@ func TestProjectService_RecoverProjectRenameJournals_KeepsRollbackCleanupWhenDoc}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)cleanup := projectRenameRollbackCleanupInternal{ProjectID: project.ID,@@ -6661,8 +6659,8 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsCommittedJournalWhenS}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -6747,8 +6745,8 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsCommittedJournalAndCl}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -6830,8 +6828,8 @@ func TestProjectService_RecoverProjectRenameJournals_MarksSourceCleanupPendingWh}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -6915,8 +6913,8 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsSourceCleanupPendingJ}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -7014,8 +7012,8 @@ func TestProjectService_RecoverProjectRenameJournals_RollsBackSourceCleanupPendi}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -7104,8 +7102,8 @@ func TestProjectService_RecoverProjectRenameJournals_KeepsSourceCleanupPendingJo}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -7171,7 +7169,7 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsStartedJournalWhenDir}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)+ kvService := kv.NewKVService(db)svc := NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -7240,8 +7238,8 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsMissingPathJournalWhe}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -7314,8 +7312,8 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsJournalWhenRollbackSo}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -7393,8 +7391,8 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsJournalWhenRollbackTa}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -7478,8 +7476,8 @@ func TestProjectService_RecoverProjectRenameJournals_ClearsJournalWhenTargetPres}require.NoError(t, db.Create(project).Error)- kvService := NewKVService(db)- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}+ kvService := kv.NewKVService(db)+ dockerService := &docker.DockerClientService{Client: newTestDockerClientInternal(t, server)}svc := NewProjectService(db, nil, nil, nil, dockerService, nil, nil, nil, config.Load()).WithKVService(kvService)journal := projectRenameJournalInternal{ProjectID: project.ID,@@ -7521,7 +7519,7 @@ func TestProjectUpdateBackup_ScopedBackupAndRestore(t *testing.T) {require.NoError(t, os.WriteFile(filepath.Join(projectPath, "data", "keep.bin"), []byte("keep"), 0o644))compose := "services: {}\n"- scope := projectUpdateBackupScopeInternal(projectPath, &compose, nil, nil, []projecttypes.ProjectFileChange{+ scope := projects.BuildUpdateBackupScope(projectPath, &compose, nil, nil, []projecttypes.ProjectFileChange{{Operation: projecttypes.FileOpCreateFile, RelativePath: "notes.txt", Content: ptr("notes\n")},})@@ -7560,7 +7558,7 @@ func TestProjectService_UpdateProject_FileChangeFailureRollsBackScoped(t *testinrequire.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "projectsDirectory", projectsDir))- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())configureProjectRuntimeDockerInternal(t, nil)@@ -7607,7 +7605,7 @@ func TestProjectUpdateBackup_RenamedDirDemotedToCopyWhenBatchTouchesIt(t *testin// Batch: rename a -> b, then delete b — the inverse rename cannot roll// this back, so the rename must be demoted to a full copy of a.- scope := projectUpdateBackupScopeInternal(projectPath, nil, nil, nil, []projecttypes.ProjectFileChange{+ scope := projects.BuildUpdateBackupScope(projectPath, nil, nil, nil, []projecttypes.ProjectFileChange{{Operation: projecttypes.FileOpRename, RelativePath: "a", NewName: "b"},{Operation: projecttypes.FileOpDelete, RelativePath: "b", Recursive: true},})diff --git a/backend/internal/services/project_service_unix_test.go b/backend/internal/project/service_unix_test.gosimilarity index 95%rename from backend/internal/services/project_service_unix_test.gorename to backend/internal/project/service_unix_test.goindex a53d883f0..0eba5c0ed 100644--- a/backend/internal/services/project_service_unix_test.go+++ b/backend/internal/project/service_unix_test.go@@ -1,6 +1,6 @@//go:build unix-package services+package projectimport ("context"@@ -9,6 +9,7 @@ import ("testing""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/stretchr/testify/assert""github.com/stretchr/testify/require"@@ -32,7 +33,7 @@ func TestProjectService_ApplyGitSyncProjectFiles_TolerantOfPermissionLockedEnv(tsettingsService, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- eventService := NewEventService(db, nil, nil)+ eventService := event.NewEventService(db, nil, nil)svc := NewProjectService(db, settingsService, eventService, nil, nil, nil, nil, nil, config.Load())dirName := "git-sync-locked-env"diff --git a/backend/internal/services/ecr_token_service.go b/backend/internal/registry/ecr_token.gosimilarity index 99%rename from backend/internal/services/ecr_token_service.gorename to backend/internal/registry/ecr_token.goindex a8cda20b7..a2c905119 100644--- a/backend/internal/services/ecr_token_service.go+++ b/backend/internal/registry/ecr_token.go@@ -1,4 +1,4 @@-package services+package registryimport ("context"diff --git a/backend/api/handlers/container_registries.go b/backend/internal/registry/handler.gosimilarity index 86%rename from backend/api/handlers/container_registries.gorename to backend/internal/registry/handler.goindex 0ee3fc818..3abbcd233 100644--- a/backend/api/handlers/container_registries.go+++ b/backend/internal/registry/handler.go@@ -1,4 +1,4 @@-package handlers+package registryimport ("context"@@ -7,10 +7,10 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/containerregistry"@@ -19,8 +19,8 @@ import (// ContainerRegistryHandler handles container registry management endpoints.type ContainerRegistryHandler struct {- registryService *services.ContainerRegistryService- environmentService *services.EnvironmentService+ registryService *ContainerRegistryService+ syncRemoteRegistries func(context.Context) error}// ============================================================================@@ -96,13 +96,12 @@ type SyncContainerRegistriesOutput struct {// Registration// ============================================================================-// RegisterContainerRegistries registers all container registry endpoints.-func RegisterContainerRegistries(api huma.API, registryService *services.ContainerRegistryService, environmentService *services.EnvironmentService) {- h := &ContainerRegistryHandler{- registryService: registryService,- environmentService: environmentService,- }+// NewHandler builds the container registry HTTP handler.+func NewHandler(registryService *ContainerRegistryService, syncRemoteRegistries func(context.Context) error) *ContainerRegistryHandler {+ return &ContainerRegistryHandler{registryService: registryService, syncRemoteRegistries: syncRemoteRegistries}+}+func RegisterContainerRegistries(api huma.API, h *ContainerRegistryHandler) {huma.Register(api, huma.Operation{OperationID: "listContainerRegistries",Method: "GET",@@ -110,8 +109,8 @@ func RegisterContainerRegistries(api huma.API, registryService *services.ContainSummary: "List container registries",Description: "Get a paginated list of container registries",Tags: []string{"Container Registries"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermRegistriesList),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermRegistriesList),}, h.ListRegistries)huma.Register(api, huma.Operation{@@ -121,8 +120,8 @@ func RegisterContainerRegistries(api huma.API, registryService *services.ContainSummary: "Create a container registry",Description: "Create a new container registry",Tags: []string{"Container Registries"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermRegistriesCreate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermRegistriesCreate),}, h.CreateRegistry)huma.Register(api, huma.Operation{@@ -132,8 +131,8 @@ func RegisterContainerRegistries(api huma.API, registryService *services.ContainSummary: "Sync container registries",Description: "Sync container registries from a remote source",Tags: []string{"Container Registries"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermRegistriesUpdate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermRegistriesUpdate),}, h.SyncRegistries)huma.Register(api, huma.Operation{@@ -143,8 +142,8 @@ func RegisterContainerRegistries(api huma.API, registryService *services.ContainSummary: "Get container registry pull usage",Description: "Get configured registry pull usage and rate limit visibility",Tags: []string{"Container Registries"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermRegistriesRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermRegistriesRead),}, h.GetPullUsage)huma.Register(api, huma.Operation{@@ -154,8 +153,8 @@ func RegisterContainerRegistries(api huma.API, registryService *services.ContainSummary: "Get a container registry",Description: "Get a container registry by ID",Tags: []string{"Container Registries"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermRegistriesRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermRegistriesRead),}, h.GetRegistry)huma.Register(api, huma.Operation{@@ -165,8 +164,8 @@ func RegisterContainerRegistries(api huma.API, registryService *services.ContainSummary: "Update a container registry",Description: "Update an existing container registry",Tags: []string{"Container Registries"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermRegistriesUpdate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermRegistriesUpdate),}, h.UpdateRegistry)huma.Register(api, huma.Operation{@@ -176,8 +175,8 @@ func RegisterContainerRegistries(api huma.API, registryService *services.ContainSummary: "Delete a container registry",Description: "Delete a container registry by ID",Tags: []string{"Container Registries"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermRegistriesDelete),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermRegistriesDelete),}, h.DeleteRegistry)huma.Register(api, huma.Operation{@@ -187,8 +186,8 @@ func RegisterContainerRegistries(api huma.API, registryService *services.ContainSummary: "Test a container registry",Description: "Test connectivity and authentication to a container registry",Tags: []string{"Container Registries"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermRegistriesTest),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermRegistriesTest),}, h.TestRegistry)}@@ -198,7 +197,7 @@ func RegisterContainerRegistries(api huma.API, registryService *services.Contain// ListRegistries returns a paginated list of container registries.func (h *ContainerRegistryHandler) ListRegistries(ctx context.Context, input *ListContainerRegistriesInput) (*ListContainerRegistriesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)registries, paginationResp, err := h.registryService.GetRegistriesPaginated(ctx, params)if err != nil {@@ -209,7 +208,7 @@ func (h *ContainerRegistryHandler) ListRegistries(ctx context.Context, input *LiBody: base.Paginated[containerregistry.ContainerRegistry]{Success: true,Data: registries,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -384,14 +383,14 @@ func (h *ContainerRegistryHandler) SyncRegistries(ctx context.Context, input *Sy// ============================================================================func (h *ContainerRegistryHandler) triggerRemoteRegistrySync(ctx context.Context, reason string) {- if h.environmentService == nil {+ if h.syncRemoteRegistries == nil {return}detachedCtx := context.WithoutCancel(ctx)go func(syncCtx context.Context, syncReason string) {- if err := h.environmentService.SyncRegistriesToRemoteEnvironments(syncCtx); err != nil {+ if err := h.syncRemoteRegistries(syncCtx); err != nil {slog.WarnContext(syncCtx, "Failed to fan out registry sync to remote environments", "reason", syncReason, "error", err.Error())}}(detachedCtx, reason)diff --git a/backend/internal/registry/module.go b/backend/internal/registry/module.gonew file mode 100644index 000000000..25b1e0dc0--- /dev/null+++ b/backend/internal/registry/module.go@@ -0,0 +1,54 @@+// Package registry owns registry credentials, digest inspection, pull usage,+// synchronization, and the registry HTTP surface.+package registry++import (+ "context"++ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv"+)++type Dependencies struct {+ DB *database.DB+ Docker *docker.DockerClientService+ KV *kv.KVService+ SyncRemoteRegistries func(context.Context) error+}++type Module struct {+ service *ContainerRegistryService+ handler *ContainerRegistryHandler+}++func New(deps Dependencies) *Module {+ service := NewContainerRegistryService(deps.DB, func(ctx context.Context) (RegistryDaemonClient, error) {+ return deps.Docker.GetClient(ctx)+ }, deps.KV)+ return &Module{service: service, handler: NewHandler(service, deps.SyncRemoteRegistries)}+}++func (m *Module) Handler() *ContainerRegistryHandler {+ if m == nil {+ return nil+ }+ return m.handler+}++func (m *Module) Service() *ContainerRegistryService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterContainerRegistries(api, NewHandler(nil, nil))+ return+ }+ RegisterContainerRegistries(api, m.handler)+}diff --git a/backend/internal/services/container_registry_service.go b/backend/internal/registry/service.gosimilarity index 94%rename from backend/internal/services/container_registry_service.gorename to backend/internal/registry/service.goindex f473064da..23f2392f3 100644--- a/backend/internal/services/container_registry_service.go+++ b/backend/internal/registry/service.go@@ -1,4 +1,4 @@-package services+package registryimport ("context"@@ -19,6 +19,7 @@ import (cerrdefs "github.com/containerd/errdefs""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv""github.com/getarcaneapp/arcane/backend/v2/internal/models"utilsregistry "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/registryauth""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"@@ -36,8 +37,8 @@ import (const (registryCacheTTL = 30 * time.Minute- registryTypeGeneric string = "generic"- registryTypeECR string = "ecr"+ RegistryTypeGeneric string = "generic"+ RegistryTypeECR string = "ecr"registryPullCountKeyPrefix = "container_registry:pulls:"registryRateLimitKeyPrefix = "container_registry:rate_limits:"dockerHubRateLimitRepository = "ratelimitpreview/test"@@ -51,14 +52,6 @@ type RegistryDaemonClient interface {type registryDaemonGetter func(context.Context) (RegistryDaemonClient, error)-type registryDigestResult struct {- Digest string- AuthMethod string- AuthUsername string- AuthRegistry string- UsedCredential bool-}-type resolvedRegistryCredential struct {Username stringToken string@@ -82,16 +75,20 @@ type ContainerRegistryService struct {cache *hot.HotCache[string, string]ecrRefreshGroup singleflight.GroupdistributionHTTPClient *http.Client- kvService *KVService+ kvService *kv.KVService}// NewContainerRegistryService creates a registry service. kvService may be nil// in tests that do not need pull tracking or rate-limit caching.-func NewContainerRegistryService(db *database.DB, dockerClient registryDaemonGetter, kvService *KVService) *ContainerRegistryService {+func NewContainerRegistryService(db *database.DB, dockerClient registryDaemonGetter, kvService *kv.KVService, distributionHTTPClients ...*http.Client) *ContainerRegistryService {+ distributionHTTPClient := registry.NewHTTPClient()+ if len(distributionHTTPClients) > 0 && distributionHTTPClients[0] != nil {+ distributionHTTPClient = distributionHTTPClients[0]+ }service := &ContainerRegistryService{db: db,dockerClient: dockerClient,- distributionHTTPClient: registry.NewHTTPClient(),+ distributionHTTPClient: distributionHTTPClient,kvService: kvService,}backgroundLoader := func(imageRefs []string) (map[string]string, error) {@@ -101,10 +98,10 @@ func NewContainerRegistryService(db *database.DB, dockerClient registryDaemonGet// Per-ref timeout: a single deadline shared across the whole// sequential batch left later refs with whatever the earlier ones// had not already spent, so one slow registry starved the tail.- result, err := func() (*registryDigestResult, error) {+ result, err := func() (*containerregistry.DigestResult, error) {ctx, cancel := context.WithTimeout(context.Background(), timeouts.DefaultRegistry)defer cancel()- return service.inspectImageDigestInternal(ctx, imageRef, nil)+ return service.InspectImageDigest(ctx, imageRef, nil)}()if err != nil {slog.Debug("registry revalidation failed for image", "imageRef", imageRef, "error", err)@@ -171,7 +168,7 @@ func (s *ContainerRegistryService) GetRegistryByID(ctx context.Context, id strin}func (s *ContainerRegistryService) CreateRegistry(ctx context.Context, req models.CreateContainerRegistryRequest) (*models.ContainerRegistry, error) {- registryType, err := normalizeRegistryTypeInternal(req.RegistryType)+ registryType, err := NormalizeRegistryType(req.RegistryType)if err != nil {return nil, err}@@ -191,7 +188,7 @@ func (s *ContainerRegistryService) CreateRegistry(ctx context.Context, req modelUpdatedAt: time.Now(),}- if registryType == registryTypeECR {+ if registryType == RegistryTypeECR {if strings.TrimSpace(req.AWSAccessKeyID) == "" {return nil, common.Classify(common.ErrValidation, errors.WithDetails(errors.New("AWS Access Key ID is required"), "field", "awsAccessKeyId"))}@@ -242,7 +239,7 @@ func (s *ContainerRegistryService) UpdateRegistry(ctx context.Context, id stringstoredCredentials := map[string]bool{"token": registryRecord.Token != ""}updatedCredentials := map[string]bool{"token": req.Token != nil && *req.Token != ""}- if registryRecord.RegistryType == registryTypeECR {+ if registryRecord.RegistryType == RegistryTypeECR {storedCredentials = map[string]bool{"awsAccessKeyId": registryRecord.AWSAccessKeyID != "","awsSecretAccessKey": registryRecord.AWSSecretAccessKey != "",@@ -278,7 +275,7 @@ func (s *ContainerRegistryService) UpdateRegistry(ctx context.Context, id stringregistryRecord.RepositoryNames = repositoryNames}- if registryRecord.RegistryType == registryTypeECR {+ if registryRecord.RegistryType == RegistryTypeECR {if err := s.updateECRRegistryFieldsInternal(registryRecord, req); err != nil {return nil, err}@@ -300,7 +297,7 @@ func (s *ContainerRegistryService) applyRegistryTypeUpdateInternal(registry *modreturn nil}- nextType, err := normalizeRegistryTypeInternal(*registryType)+ nextType, err := NormalizeRegistryType(*registryType)if err != nil {return err}@@ -393,7 +390,7 @@ func (s *ContainerRegistryService) GetEnabledRegistries(ctx context.Context) ([]// GetRegistryAuthForImage returns X-Registry-Auth for the image's registry host.//-// The registry-auth methods tolerate a nil receiver: callers such as BuildService may+// The registry-auth methods tolerate a nil receiver: callers such as build.BuildService may// hold no registry service, and are wired in as a buildtypes.RegistryAuthProvider where a// typed-nil pointer would otherwise satisfy the interface's nil checks and panic on use.func (s *ContainerRegistryService) GetRegistryAuthForImage(ctx context.Context, imageRef string) (string, error) {@@ -617,7 +614,7 @@ func (s *ContainerRegistryService) getObservedPullsInternal(ctx context.Context,}func (s *ContainerRegistryService) dockerHubCredentialForRegistryInternal(reg models.ContainerRegistry) (*registry.Credentials, string, string, error) {- if reg.RegistryType != registryTypeGeneric {+ if reg.RegistryType != RegistryTypeGeneric {return nil, "anonymous", "", nil}@@ -734,7 +731,7 @@ func registryRateLimitKeyInternal(registryID string) string {}func registryProviderInternal(registryHost, registryType string) string {- if registryType == registryTypeECR {+ if registryType == RegistryTypeECR {return "ecr"}if registryHost == "docker.io" {@@ -744,7 +741,7 @@ func registryProviderInternal(registryHost, registryType string) string {}func registryDisplayNameInternal(registryHost, registryType string) string {- if registryType == registryTypeECR {+ if registryType == RegistryTypeECR {return "Amazon ECR"}switch {@@ -821,7 +818,7 @@ func (s *ContainerRegistryService) ImageDigest(ctx context.Context, imageRef strloadCtx, cancel := context.WithTimeout(ctx, timeouts.DefaultRegistry)defer cancel()- result, loadErr := s.inspectImageDigestInternal(loadCtx, normalizedRef, nil)+ result, loadErr := s.InspectImageDigest(loadCtx, normalizedRef, nil)if loadErr != nil {return nil, loadErr}@@ -836,13 +833,13 @@ func (s *ContainerRegistryService) ImageDigest(ctx context.Context, imageRef strreturn digestValue, nil}-func (s *ContainerRegistryService) inspectImageDigestInternal(ctx context.Context, imageRef string, externalCreds []containerregistry.Credential) (*registryDigestResult, error) {+func (s *ContainerRegistryService) InspectImageDigest(ctx context.Context, imageRef string, externalCreds []containerregistry.Credential) (*containerregistry.DigestResult, error) {parts, err := refs.NormalizeReference(imageRef)if err != nil {return nil, err}- var lastResult *registryDigestResult+ var lastResult *containerregistry.DigestResultvar lastErr errorfallbackWarningLogged := false@@ -851,7 +848,7 @@ func (s *ContainerRegistryService) inspectImageDigestInternal(ctx context.Contexbo.MaxInterval = 10 * time.Secondbo.RandomizationFactor = 0.3- _, retryErr := backoff.Retry(ctx, func() (*registryDigestResult, error) {+ _, retryErr := backoff.Retry(ctx, func() (*containerregistry.DigestResult, error) {result, err := s.inspectImageDigestViaDaemonInternal(ctx, parts.NormalizedRef, parts.RegistryHost, externalCreds)if err == nil {lastResult = result@@ -911,7 +908,7 @@ func (s *ContainerRegistryService) inspectImageDigestInternal(ctx context.Contexreturn lastResult, nil}-func (s *ContainerRegistryService) inspectImageDigestViaDaemonInternal(ctx context.Context, normalizedRef, registryHost string, externalCreds []containerregistry.Credential) (*registryDigestResult, error) {+func (s *ContainerRegistryService) inspectImageDigestViaDaemonInternal(ctx context.Context, normalizedRef, registryHost string, externalCreds []containerregistry.Credential) (*containerregistry.DigestResult, error) {dockerClient, err := s.getDockerClientInternal(ctx)if err != nil {return nil, err@@ -930,27 +927,27 @@ func (s *ContainerRegistryService) inspectImageDigestViaDaemonInternal(ctx conteif normalizeErr != nil {return nil, errors.WrapIff(normalizeErr, "distribution inspect returned invalid digest for %s", normalizedRef)}- return ®istryDigestResult{+ return &containerregistry.DigestResult{Digest: digestValue,AuthMethod: "anonymous",AuthRegistry: registryHost,}, nil}if !isUnauthorizedRegistryErrorInternal(err) {- return ®istryDigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}, errors.+ return &containerregistry.DigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}, errors.WrapIff(err, "distribution inspect failed for %s", normalizedRef)}credentials, credErr := s.getMatchingRegistryCredentialsInternal(ctx, registryHost, externalCreds)if credErr != nil {- return ®istryDigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}, errors.+ return &containerregistry.DigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}, errors.WrapIf(stderrors.Join(err, credErr), "distribution inspect: anonymous access unauthorized; credential lookup failed")}return s.inspectImageDigestWithCredentialsInternal(ctx, dockerClient, normalizedRef, registryHost, credentials, err)}-func (s *ContainerRegistryService) inspectImageDigestWithCredentialsInternal(ctx context.Context, dockerClient RegistryDaemonClient, normalizedRef, registryHost string, credentials []resolvedRegistryCredential, previousErr error) (*registryDigestResult, error) {+func (s *ContainerRegistryService) inspectImageDigestWithCredentialsInternal(ctx context.Context, dockerClient RegistryDaemonClient, normalizedRef, registryHost string, credentials []resolvedRegistryCredential, previousErr error) (*containerregistry.DigestResult, error) {lastErr := previousErrvar lastCred resolvedRegistryCredentialfor _, credential := range credentials {@@ -968,7 +965,7 @@ func (s *ContainerRegistryService) inspectImageDigestWithCredentialsInternal(ctxif normalizeErr != nil {return nil, errors.WrapIff(normalizeErr, "distribution inspect returned invalid digest for %s", normalizedRef)}- return ®istryDigestResult{+ return &containerregistry.DigestResult{Digest: digestValue,AuthMethod: "credential",AuthUsername: credential.Username,@@ -978,7 +975,7 @@ func (s *ContainerRegistryService) inspectImageDigestWithCredentialsInternal(ctx}lastErr = errif !isUnauthorizedRegistryErrorInternal(err) {- return ®istryDigestResult{+ return &containerregistry.DigestResult{AuthMethod: "credential",AuthUsername: credential.Username,AuthRegistry: registryHost,@@ -987,7 +984,7 @@ func (s *ContainerRegistryService) inspectImageDigestWithCredentialsInternal(ctx}}- partial := ®istryDigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}+ partial := &containerregistry.DigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}if lastCred.Username != "" {partial.AuthMethod = "credential"partial.AuthUsername = lastCred.Username@@ -999,23 +996,23 @@ func (s *ContainerRegistryService) inspectImageDigestWithCredentialsInternal(ctxreturn partial, errors.WrapIff(lastErr, "distribution inspect failed for %s", normalizedRef)}-func (s *ContainerRegistryService) inspectImageDigestViaRegistryInternal(ctx context.Context, registryHost, repository, tag string, externalCreds []containerregistry.Credential) (*registryDigestResult, error) {+func (s *ContainerRegistryService) inspectImageDigestViaRegistryInternal(ctx context.Context, registryHost, repository, tag string, externalCreds []containerregistry.Credential) (*containerregistry.DigestResult, error) {digestValue, err := s.fetchDigestFromRegistryInternal(ctx, registryHost, repository, tag, nil)if err == nil {- return ®istryDigestResult{+ return &containerregistry.DigestResult{Digest: digestValue,AuthMethod: "anonymous",AuthRegistry: registryHost,}, nil}if !isUnauthorizedRegistryErrorInternal(err) {- return ®istryDigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}, errors.+ return &containerregistry.DigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}, errors.WrapIff(err, "registry manifest inspect failed for %s/%s:%s", registryHost, repository, tag)}credentials, credErr := s.getMatchingRegistryCredentialsInternal(ctx, registryHost, externalCreds)if credErr != nil {- return ®istryDigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}, errors.+ return &containerregistry.DigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}, errors.WrapIf(stderrors.Join(err, credErr), "registry manifest inspect: anonymous access unauthorized; credential lookup failed")}@@ -1026,7 +1023,7 @@ func (s *ContainerRegistryService) inspectImageDigestViaRegistryInternal(ctx condigestValue, err = s.fetchDigestFromRegistryInternal(ctx, registryHost, repository, tag, &credential)if err == nil {- return ®istryDigestResult{+ return &containerregistry.DigestResult{Digest: digestValue,AuthMethod: "credential",AuthUsername: credential.Username,@@ -1038,7 +1035,7 @@ func (s *ContainerRegistryService) inspectImageDigestViaRegistryInternal(ctx conlastErr = err}- partial := ®istryDigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}+ partial := &containerregistry.DigestResult{AuthMethod: "anonymous", AuthRegistry: registryHost}if lastCred.Username != "" {partial.AuthMethod = "credential"partial.AuthUsername = lastCred.Username@@ -1197,7 +1194,7 @@ func (s *ContainerRegistryService) updateExistingRegistryInternal(ctx context.Co}func (s *ContainerRegistryService) checkRegistryNeedsUpdateInternal(item containerregistry.Sync, existing *models.ContainerRegistry) (bool, error) {- newType, err := normalizeRegistryTypeInternal(item.RegistryType)+ newType, err := NormalizeRegistryType(item.RegistryType)if err != nil {return false, err}@@ -1217,7 +1214,7 @@ func (s *ContainerRegistryService) checkRegistryNeedsUpdateInternal(item contain// Clear stale credentials when registry type changes during syncif newType != existing.RegistryType {- if newType == registryTypeECR {+ if newType == RegistryTypeECR {existing.Username = ""existing.Token = ""} else {@@ -1232,7 +1229,7 @@ func (s *ContainerRegistryService) checkRegistryNeedsUpdateInternal(item containneedsUpdate = utils.ApplyChanged(&existing.RegistryType, mo.Some(newType)) || needsUpdate- if newType == registryTypeGeneric {+ if newType == RegistryTypeGeneric {needsUpdate = utils.ApplyChanged(&existing.Username, mo.Some(item.Username)) || needsUpdateencryptedToken, err := crypto.Encrypt(item.Token)@@ -1269,7 +1266,7 @@ func (s *ContainerRegistryService) checkRegistryNeedsUpdateInternal(item contain}func (s *ContainerRegistryService) createNewRegistryInternal(ctx context.Context, item containerregistry.Sync) error {- registryType, err := normalizeRegistryTypeInternal(item.RegistryType)+ registryType, err := NormalizeRegistryType(item.RegistryType)if err != nil {return err}@@ -1294,7 +1291,7 @@ func (s *ContainerRegistryService) createNewRegistryInternal(ctx context.ContextUpdatedAt: time.Now(),}- if registryType == registryTypeGeneric {+ if registryType == RegistryTypeGeneric {newRegistry.Username = item.UsernameencryptedToken, err := crypto.Encrypt(item.Token)@@ -1317,14 +1314,14 @@ func (s *ContainerRegistryService) createNewRegistryInternal(ctx context.Contextreturn nil}-func normalizeRegistryTypeInternal(value string) (string, error) {+func NormalizeRegistryType(value string) (string, error) {registryType := strings.ToLower(strings.TrimSpace(value))if registryType == "" {- return registryTypeGeneric, nil+ return RegistryTypeGeneric, nil}switch registryType {- case registryTypeGeneric, registryTypeECR:+ case RegistryTypeGeneric, RegistryTypeECR:return registryType, nildefault:return "", common.Classify(common.ErrValidation, errors.WithDetails(errors.New("Registry type must be one of: generic, ecr"), "field", "registryType"))@@ -1421,10 +1418,10 @@ func isRateLimitErrorInternal(err error) bool {if err == nil {return false}- return isRateLimitErrorStringInternal(err.Error())+ return IsRateLimitErrorString(err.Error())}-func isRateLimitErrorStringInternal(msg string) bool {+func IsRateLimitErrorString(msg string) bool {msgLower := strings.ToLower(msg)indicators := []string{"toomanyrequests",diff --git a/backend/internal/services/container_registry_service_test.go b/backend/internal/registry/service_test.gosimilarity index 87%rename from backend/internal/services/container_registry_service_test.gorename to backend/internal/registry/service_test.goindex c07282b8b..0205aed2b 100644--- a/backend/internal/services/container_registry_service_test.go+++ b/backend/internal/registry/service_test.go@@ -1,4 +1,4 @@-package services+package registryimport ("context"@@ -13,8 +13,11 @@ import ("emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/types/v2/containerregistry"+ "github.com/libtnb/sqlite"+ dockerauthconfig "github.com/moby/moby/api/pkg/authconfig"dockerregistry "github.com/moby/moby/api/types/registry""github.com/moby/moby/client""github.com/opencontainers/go-digest"@@ -22,6 +25,7 @@ import ("github.com/stretchr/testify/assert""github.com/stretchr/testify/require""go.getarcane.app/sys/crypto"+ "gorm.io/gorm")type fakeRegistryDaemonClient struct {@@ -29,6 +33,45 @@ type fakeRegistryDaemonClient struct {distributionInspectFn func(ctx context.Context, imageRef string, options client.DistributionInspectOptions) (client.DistributionInspectResult, error)}+func setupContainerRegistryTestDBInternal(t *testing.T) *database.DB {+ t.Helper()++ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.ContainerRegistry{}, &models.KVEntry{}, &models.Project{}))++ crypto.InitEncryption(&crypto.Config{+ Environment: "test",+ EncryptionKey: "test-encryption-key-for-testing-32bytes-min",+ })++ return &database.DB{DB: db}+}++func createTestPullRegistryInternal(t *testing.T, db *database.DB, url, username, token string) {+ t.Helper()++ encryptedToken, err := crypto.Encrypt(token)+ require.NoError(t, err)++ record := &models.ContainerRegistry{+ URL: url,+ Username: username,+ Token: encryptedToken,+ Enabled: true,+ RegistryType: RegistryTypeGeneric,+ }+ require.NoError(t, db.WithContext(context.Background()).Create(record).Error)+}++func decodeRegistryAuthInternal(t *testing.T, encoded string) dockerregistry.AuthConfig {+ t.Helper()++ cfg, err := dockerauthconfig.Decode(encoded)+ require.NoError(t, err)+ return *cfg+}+type roundTripFunc func(*http.Request) (*http.Response, error)func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {@@ -49,24 +92,6 @@ func (f *fakeRegistryDaemonClient) DistributionInspect(ctx context.Context, imagreturn f.distributionInspectFn(ctx, imageRef, options)}-func newTestDockerClient(t *testing.T, server *httptest.Server) *client.Client {- t.Helper()-- httpClient := server.Client()- cli, err := client.New(- client.WithHost(server.URL),- client.WithAPIVersion("1.41"),- client.WithHTTPClient(httpClient),- )- require.NoError(t, err)-- t.Cleanup(func() {- _ = cli.Close()- })-- return cli-}-func newDockerHubRateLimitTestClient(t *testing.T, handler http.HandlerFunc) *http.Client {t.Helper()@@ -98,9 +123,9 @@ func TestNewContainerRegistryService_InitializesDistributionHTTPClient(t *testin}func TestContainerRegistryService_GetAllRegistryAuthConfigs_NormalizesHosts(t *testing.T) {- _, db := setupImageServiceAuthTest(t)- createTestPullRegistry(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")- createTestPullRegistry(t, db, "https://GHCR.IO/", "gh-user", "gh-token")+ db := setupContainerRegistryTestDBInternal(t)+ createTestPullRegistryInternal(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")+ createTestPullRegistryInternal(t, db, "https://GHCR.IO/", "gh-user", "gh-token")svc := NewContainerRegistryService(db, nil, nil)authConfigs, err := svc.GetAllRegistryAuthConfigs(context.Background())@@ -124,10 +149,10 @@ func TestContainerRegistryService_GetAllRegistryAuthConfigs_NormalizesHosts(t *t}func TestContainerRegistryService_GetAllRegistryAuthConfigs_SkipsInvalidEntries(t *testing.T) {- _, db := setupImageServiceAuthTest(t)- createTestPullRegistry(t, db, "https://docker.io", " ", "docker-token")- createTestPullRegistry(t, db, "https://ghcr.io", "gh-user", " ")- createTestPullRegistry(t, db, "https://registry.example.com", "example-user", "example-token")+ db := setupContainerRegistryTestDBInternal(t)+ createTestPullRegistryInternal(t, db, "https://docker.io", " ", "docker-token")+ createTestPullRegistryInternal(t, db, "https://ghcr.io", "gh-user", " ")+ createTestPullRegistryInternal(t, db, "https://registry.example.com", "example-user", "example-token")svc := NewContainerRegistryService(db, nil, nil)authConfigs, err := svc.GetAllRegistryAuthConfigs(context.Background())@@ -145,30 +170,30 @@ func TestContainerRegistryService_GetAllRegistryAuthConfigs_SkipsInvalidEntries(}func TestContainerRegistryService_GetRegistryAuthForHost_UsesDatabaseCredentials(t *testing.T) {- _, db := setupImageServiceAuthTest(t)- createTestPullRegistry(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")+ db := setupContainerRegistryTestDBInternal(t)+ createTestPullRegistryInternal(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")svc := NewContainerRegistryService(db, nil, nil)auth, err := svc.GetRegistryAuthForHost(context.Background(), "registry-1.docker.io")require.NoError(t, err)require.NotEmpty(t, auth)- cfg := decodeRegistryAuth(t, auth)+ cfg := decodeRegistryAuthInternal(t, auth)assert.Equal(t, "docker-user", cfg.Username)assert.Equal(t, "docker-token", cfg.Password)assert.Equal(t, "https://index.docker.io/v1/", cfg.ServerAddress)}func TestContainerRegistryService_GetRegistryAuthForImage_UsesHostLookup(t *testing.T) {- _, db := setupImageServiceAuthTest(t)- createTestPullRegistry(t, db, "https://ghcr.io", "gh-user", "gh-token")+ db := setupContainerRegistryTestDBInternal(t)+ createTestPullRegistryInternal(t, db, "https://ghcr.io", "gh-user", "gh-token")svc := NewContainerRegistryService(db, nil, nil)auth, err := svc.GetRegistryAuthForImage(context.Background(), "ghcr.io/getarcaneapp/arcane:latest")require.NoError(t, err)require.NotEmpty(t, auth)- cfg := decodeRegistryAuth(t, auth)+ cfg := decodeRegistryAuthInternal(t, auth)assert.Equal(t, "gh-user", cfg.Username)assert.Equal(t, "gh-token", cfg.Password)assert.Equal(t, "ghcr.io", cfg.ServerAddress)@@ -192,16 +217,16 @@ func TestContainerRegistryService_NilService_RegistryAuthMethodsReturnEmpty(t *t}func TestContainerRegistryService_GetRegistryPullUsage_AnonymousDockerHubLimit(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)require.NoError(t, db.WithContext(context.Background()).Create(&models.ContainerRegistry{URL: "docker.io",Enabled: true,- RegistryType: registryTypeGeneric,+ RegistryType: RegistryTypeGeneric,}).Error)var tokenCalls int- svc := NewContainerRegistryService(db, nil, NewKVService(db))+ svc := NewContainerRegistryService(db, nil, kv.NewKVService(db))svc.distributionHTTPClient = newDockerHubRateLimitTestClient(t, func(w http.ResponseWriter, r *http.Request) {switch {case r.URL.Path == "/token":@@ -249,12 +274,12 @@ func TestContainerRegistryService_GetRegistryPullUsage_AnonymousDockerHubLimit(t}func TestContainerRegistryService_GetRegistryPullUsage_UsesDockerHubCredential(t *testing.T) {- _, db := setupImageServiceAuthTest(t)- createTestPullRegistry(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")- createTestPullRegistry(t, db, "https://ghcr.io", "gh-user", "gh-token")+ db := setupContainerRegistryTestDBInternal(t)+ createTestPullRegistryInternal(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")+ createTestPullRegistryInternal(t, db, "https://ghcr.io", "gh-user", "gh-token")var tokenAuth string- svc := NewContainerRegistryService(db, nil, NewKVService(db))+ svc := NewContainerRegistryService(db, nil, kv.NewKVService(db))svc.distributionHTTPClient = newDockerHubRateLimitTestClient(t, func(w http.ResponseWriter, r *http.Request) {switch {case r.URL.Path == "/token":@@ -292,10 +317,10 @@ func TestContainerRegistryService_GetRegistryPullUsage_UsesDockerHubCredential(t}func TestContainerRegistryService_GetRegistryPullUsage_CredentialErrorIsNonFatal(t *testing.T) {- _, db := setupImageServiceAuthTest(t)- createTestPullRegistry(t, db, "https://docker.io", "docker-user", "docker-token")+ db := setupContainerRegistryTestDBInternal(t)+ createTestPullRegistryInternal(t, db, "https://docker.io", "docker-user", "docker-token")- svc := NewContainerRegistryService(db, nil, NewKVService(db))+ svc := NewContainerRegistryService(db, nil, kv.NewKVService(db))svc.distributionHTTPClient = newDockerHubRateLimitTestClient(t, func(w http.ResponseWriter, r *http.Request) {switch {case r.URL.Path == "/token":@@ -321,14 +346,14 @@ func TestContainerRegistryService_GetRegistryPullUsage_CredentialErrorIsNonFatal}func TestContainerRegistryService_RecordImagePull_IncrementsObservedRegistryCount(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)require.NoError(t, db.WithContext(context.Background()).Create(&models.ContainerRegistry{URL: "ghcr.io",Enabled: true,- RegistryType: registryTypeGeneric,+ RegistryType: RegistryTypeGeneric,}).Error)- svc := NewContainerRegistryService(db, nil, NewKVService(db))+ svc := NewContainerRegistryService(db, nil, kv.NewKVService(db))require.NoError(t, svc.RecordImagePull(context.Background(), "ghcr.io/example/app:latest"))require.NoError(t, svc.RecordImagePull(context.Background(), "ghcr.io/example/worker:latest"))@@ -341,7 +366,7 @@ func TestContainerRegistryService_RecordImagePull_IncrementsObservedRegistryCoun}func TestContainerRegistryService_CreateRegistry_RejectsUnsupportedRegistryType(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)_, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -355,7 +380,7 @@ func TestContainerRegistryService_CreateRegistry_RejectsUnsupportedRegistryType(}func TestContainerRegistryService_CreateRegistry_RejectsEmptyUsernameForGeneric(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)_, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -370,7 +395,7 @@ func TestContainerRegistryService_CreateRegistry_RejectsEmptyUsernameForGeneric(}func TestContainerRegistryService_CreateRegistry_RejectsEmptyTokenForGeneric(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)_, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -385,7 +410,7 @@ func TestContainerRegistryService_CreateRegistry_RejectsEmptyTokenForGeneric(t *}func TestContainerRegistryService_CreateRegistry_AcceptsValidGenericCredentials(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)reg, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -399,7 +424,7 @@ func TestContainerRegistryService_CreateRegistry_AcceptsValidGenericCredentials(}func TestContainerRegistryService_UpdateRegistry_RejectsBlankingUsername(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)reg, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -419,7 +444,7 @@ func TestContainerRegistryService_UpdateRegistry_RejectsBlankingUsername(t *test}func TestContainerRegistryService_UpdateRegistry_KeepsExistingTokenWhenNotProvided(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)reg, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -449,7 +474,7 @@ func TestContainerRegistryService_UpdateRegistry_RejectsTargetChangeWhenStoredTofor _, tt := range tests {t.Run(tt.name, func(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)registry, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -478,7 +503,7 @@ func TestContainerRegistryService_UpdateRegistry_RejectsTargetChangeWhenStoredTo}func TestContainerRegistryService_UpdateRegistry_AllowsPathChangeOnSameHostWithoutToken(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)registry, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -498,7 +523,7 @@ func TestContainerRegistryService_UpdateRegistry_AllowsPathChangeOnSameHostWitho}func TestContainerRegistryService_UpdateRegistry_AllowsTargetChangeWhenTokenIsResupplied(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)registry, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -521,12 +546,12 @@ func TestContainerRegistryService_UpdateRegistry_AllowsTargetChangeWhenTokenIsRe}func TestContainerRegistryService_UpdateRegistryRejectsECRTargetChangeWithStoredCredentialsInternal(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)registry, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{URL: "123456789012.dkr.ecr.us-east-1.amazonaws.com",- RegistryType: registryTypeECR,+ RegistryType: RegistryTypeECR,AWSAccessKeyID: "old-access-key",AWSSecretAccessKey: "old-secret-key",AWSRegion: "us-east-1",@@ -563,7 +588,7 @@ func TestContainerRegistryService_UpdateRegistryRejectsECRTargetChangeWithStored}func TestContainerRegistryService_UpdateRegistry_RejectsChangingRegistryType(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)reg, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -583,7 +608,7 @@ func TestContainerRegistryService_UpdateRegistry_RejectsChangingRegistryType(t *}func TestContainerRegistryService_UpdateRegistry_AllowsSameRegistryType(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)reg, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -603,8 +628,8 @@ func TestContainerRegistryService_UpdateRegistry_AllowsSameRegistryType(t *testi}func TestContainerRegistryService_SyncRegistries_ClearsGenericTokenWhenManagerSendsEmptyValue(t *testing.T) {- _, db := setupImageServiceAuthTest(t)- createTestPullRegistry(t, db, "https://registry.example.com", "registry-user", "old-token")+ db := setupContainerRegistryTestDBInternal(t)+ createTestPullRegistryInternal(t, db, "https://registry.example.com", "registry-user", "old-token")var existing models.ContainerRegistryrequire.NoError(t, db.WithContext(context.Background()).First(&existing).Error)@@ -617,7 +642,7 @@ func TestContainerRegistryService_SyncRegistries_ClearsGenericTokenWhenManagerSeUsername: existing.Username,Token: "",Enabled: true,- RegistryType: registryTypeGeneric,+ RegistryType: RegistryTypeGeneric,CreatedAt: existing.CreatedAt,UpdatedAt: existing.UpdatedAt,},@@ -699,7 +724,7 @@ func TestContainerRegistryService_InspectImageDigest_AnonymousSuccess(t *testing}, nil}, nil)- result, err := svc.inspectImageDigestInternal(context.Background(), "registry.example.com:5443/team/app:1.2.3", nil)+ result, err := svc.InspectImageDigest(context.Background(), "registry.example.com:5443/team/app:1.2.3", nil)require.NoError(t, err)assert.Equal(t, wantDigest, result.Digest)assert.Equal(t, "anonymous", result.AuthMethod)@@ -746,8 +771,8 @@ func TestContainerRegistryService_GetImageDigest_HonorsCallerCancellation(t *tes}func TestContainerRegistryService_InspectImageDigest_UsesStoredDockerHubCredentialsOnFirstAttempt(t *testing.T) {- _, db := setupImageServiceAuthTest(t)- createTestPullRegistry(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")+ db := setupContainerRegistryTestDBInternal(t)+ createTestPullRegistryInternal(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")wantDigest := digest.FromString("stored-credentials").String()var calls int@@ -757,7 +782,7 @@ func TestContainerRegistryService_InspectImageDigest_UsesStoredDockerHubCredenticalls++assert.Equal(t, "docker.io/library/nginx:latest", imageRef)- authCfg := decodeRegistryAuth(t, options.EncodedRegistryAuth)+ authCfg := decodeRegistryAuthInternal(t, options.EncodedRegistryAuth)assert.Equal(t, "docker-user", authCfg.Username)assert.Equal(t, "docker-token", authCfg.Password)assert.Equal(t, "https://index.docker.io/v1/", authCfg.ServerAddress)@@ -773,7 +798,7 @@ func TestContainerRegistryService_InspectImageDigest_UsesStoredDockerHubCredenti}, nil}, nil)- result, err := svc.inspectImageDigestInternal(context.Background(), "registry-1.docker.io/library/nginx:latest", nil)+ result, err := svc.InspectImageDigest(context.Background(), "registry-1.docker.io/library/nginx:latest", nil)require.NoError(t, err)assert.Equal(t, 1, calls)assert.Equal(t, wantDigest, result.Digest)@@ -783,8 +808,8 @@ func TestContainerRegistryService_InspectImageDigest_UsesStoredDockerHubCredenti}func TestContainerRegistryService_InspectImageDigest_UsesStoredDockerHubCredentialsForRegistryImage(t *testing.T) {- _, db := setupImageServiceAuthTest(t)- createTestPullRegistry(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")+ db := setupContainerRegistryTestDBInternal(t)+ createTestPullRegistryInternal(t, db, "https://index.docker.io/v1/", "docker-user", "docker-token")wantDigest := digest.FromString("stored-credentials-rate-limit").String()var calls int@@ -794,7 +819,7 @@ func TestContainerRegistryService_InspectImageDigest_UsesStoredDockerHubCredenticalls++assert.Equal(t, "docker.io/library/registry:3", imageRef)- authCfg := decodeRegistryAuth(t, options.EncodedRegistryAuth)+ authCfg := decodeRegistryAuthInternal(t, options.EncodedRegistryAuth)assert.Equal(t, "docker-user", authCfg.Username)assert.Equal(t, "docker-token", authCfg.Password)assert.Equal(t, "https://index.docker.io/v1/", authCfg.ServerAddress)@@ -810,7 +835,7 @@ func TestContainerRegistryService_InspectImageDigest_UsesStoredDockerHubCredenti}, nil}, nil)- result, err := svc.inspectImageDigestInternal(context.Background(), "docker.io/library/registry:3", nil)+ result, err := svc.InspectImageDigest(context.Background(), "docker.io/library/registry:3", nil)require.NoError(t, err)assert.Equal(t, 1, calls)assert.Equal(t, wantDigest, result.Digest)@@ -848,7 +873,7 @@ func TestContainerRegistryService_InspectImageDigest_FallsBackWhenDistributionNo}, nil)svc.distributionHTTPClient = server.Client()- result, err := svc.inspectImageDigestInternal(context.Background(), serverURL.Host+"/team/app:1.2.3", nil)+ result, err := svc.InspectImageDigest(context.Background(), serverURL.Host+"/team/app:1.2.3", nil)require.NoError(t, err)assert.Equal(t, 1, calls)assert.Equal(t, wantDigest, result.Digest)@@ -885,7 +910,7 @@ func TestContainerRegistryService_InspectImageDigest_FallsBackWhenDistributionFo}, nil)svc.distributionHTTPClient = server.Client()- result, err := svc.inspectImageDigestInternal(context.Background(), serverURL.Host+"/team/app:1.2.3", nil)+ result, err := svc.InspectImageDigest(context.Background(), serverURL.Host+"/team/app:1.2.3", nil)require.NoError(t, err)assert.Equal(t, 1, calls)assert.Equal(t, wantDigest, result.Digest)@@ -894,7 +919,7 @@ func TestContainerRegistryService_InspectImageDigest_FallsBackWhenDistributionFo}func TestContainerRegistryService_InspectImageDigest_RetriesStoredCredentialsAfterRegistryAuth403(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)wantDigest := digest.FromString("stored-credential-fallback").String()var authHeaders []string@@ -951,7 +976,7 @@ func TestContainerRegistryService_InspectImageDigest_RetriesStoredCredentialsAftserverURL, err := url.Parse(server.URL)require.NoError(t, err)- createTestPullRegistry(t, db, server.URL, "stored-user", "stored-token")+ createTestPullRegistryInternal(t, db, server.URL, "stored-user", "stored-token")svc := NewContainerRegistryService(db, func(context.Context) (RegistryDaemonClient, error) {return &fakeRegistryDaemonClient{@@ -962,7 +987,7 @@ func TestContainerRegistryService_InspectImageDigest_RetriesStoredCredentialsAft}, nil)svc.distributionHTTPClient = server.Client()- result, err := svc.inspectImageDigestInternal(context.Background(), serverURL.Host+"/team/app:1.2.3", nil)+ result, err := svc.InspectImageDigest(context.Background(), serverURL.Host+"/team/app:1.2.3", nil)require.NoError(t, err)assert.Equal(t, wantDigest, result.Digest)assert.Equal(t, "credential", result.AuthMethod)@@ -985,7 +1010,7 @@ func TestContainerRegistryService_InspectImageDigest_DoesNotFallbackOnTLSFailure}, nil}, nil)- result, err := svc.inspectImageDigestInternal(context.Background(), "registry.example.com/team/app:1.2.3", nil)+ result, err := svc.InspectImageDigest(context.Background(), "registry.example.com/team/app:1.2.3", nil)require.Error(t, err)require.NotNil(t, result)assert.Contains(t, strings.ToLower(err.Error()), "x509")@@ -1011,7 +1036,7 @@ func TestContainerRegistryService_InspectImageDigest_PreservesDaemonAndFallbackE}),}- result, err := svc.inspectImageDigestInternal(context.Background(), "registry.example.com/team/app:1.2.3", nil)+ result, err := svc.InspectImageDigest(context.Background(), "registry.example.com/team/app:1.2.3", nil)require.Error(t, err)require.NotNil(t, result)require.ErrorIs(t, err, daemonErr)@@ -1019,7 +1044,7 @@ func TestContainerRegistryService_InspectImageDigest_PreservesDaemonAndFallbackE}func TestContainerRegistryService_InspectImageDigest_PreservesAnonymousUnauthorizedWhenCredentialLookupFails(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)sqlDB, err := db.DB.DB()require.NoError(t, err)require.NoError(t, sqlDB.Close())@@ -1055,7 +1080,7 @@ func TestContainerRegistryService_InspectImageDigest_PreservesAnonymousUnauthori}, nil)svc.distributionHTTPClient = server.Client()- result, err := svc.inspectImageDigestInternal(context.Background(), serverURL.Host+"/team/app:1.2.3", nil)+ result, err := svc.InspectImageDigest(context.Background(), serverURL.Host+"/team/app:1.2.3", nil)require.Error(t, err)require.NotNil(t, result)assert.Equal(t, "anonymous", result.AuthMethod)@@ -1088,7 +1113,7 @@ func fetchRegistry(t *testing.T, db *database.DB, id string) models.ContainerReg}func TestContainerRegistryService_CreateRegistry_NormalizesAndPersistsRepositoryNames(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)// Entries are trimmed, empties dropped and duplicates removed while@@ -1099,7 +1124,7 @@ func TestContainerRegistryService_CreateRegistry_NormalizesAndPersistsRepository}func TestContainerRegistryService_CreateRegistry_RejectsInvalidRepositoryName(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)_, err := svc.CreateRegistry(context.Background(), models.CreateContainerRegistryRequest{@@ -1111,7 +1136,7 @@ func TestContainerRegistryService_CreateRegistry_RejectsInvalidRepositoryName(t}func TestContainerRegistryService_UpdateRegistry_RepositoryNamesPointerSemantics(t *testing.T) {- _, db := setupImageServiceAuthTest(t)+ db := setupContainerRegistryTestDBInternal(t)svc := NewContainerRegistryService(db, nil, nil)registry := createRegistryWithRepositoryNames(t, svc, "team", "team/platform")diff --git a/backend/api/handlers/roles.go b/backend/internal/role/handler.gosimilarity index 91%rename from backend/api/handlers/roles.gorename to backend/internal/role/handler.goindex 73c0c8f8a..5ee59f5dd 100644--- a/backend/api/handlers/roles.go+++ b/backend/internal/role/handler.go@@ -1,4 +1,4 @@-package handlers+package roleimport ("context"@@ -6,17 +6,17 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"roletypes "github.com/getarcaneapp/arcane/types/v2/role")type RoleHandler struct {- roleService *services.RoleService+ roleService *RoleService}// ---------- I/O wrappers ----------@@ -89,7 +89,7 @@ type SetUserRoleAssignmentsOutput struct {// ---------- Registration -----------func RegisterRoles(api huma.API, roleService *services.RoleService) {+func RegisterRoles(api huma.API, roleService *RoleService) {h := &RoleHandler{roleService: roleService}huma.Register(api, huma.Operation{@@ -99,8 +99,8 @@ func RegisterRoles(api huma.API, roleService *services.RoleService) {Summary: "List roles",Description: "Get a paginated list of roles (built-in + custom)",Tags: []string{"Roles"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermRolesList),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermRolesList),}, h.ListRoles)huma.Register(api, huma.Operation{@@ -109,8 +109,8 @@ func RegisterRoles(api huma.API, roleService *services.RoleService) {Path: "/roles/{id}",Summary: "Get a role",Tags: []string{"Roles"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermRolesRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermRolesRead),}, h.GetRole)huma.Register(api, huma.Operation{@@ -120,8 +120,8 @@ func RegisterRoles(api huma.API, roleService *services.RoleService) {Summary: "Create a custom role",Description: "Built-in roles cannot be created via this endpoint; only custom roles are accepted. Reserved for global admins.",Tags: []string{"Roles"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.CreateRole)huma.Register(api, huma.Operation{@@ -131,8 +131,8 @@ func RegisterRoles(api huma.API, roleService *services.RoleService) {Summary: "Update a custom role",Description: "Built-in roles are read-only and return 403 on update. Reserved for global admins.",Tags: []string{"Roles"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.UpdateRole)huma.Register(api, huma.Operation{@@ -142,8 +142,8 @@ func RegisterRoles(api huma.API, roleService *services.RoleService) {Summary: "Delete a custom role",Description: "Built-in roles are protected; deleting cascades all user assignments. Reserved for global admins.",Tags: []string{"Roles"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.DeleteRole)huma.Register(api, huma.Operation{@@ -153,7 +153,7 @@ func RegisterRoles(api huma.API, roleService *services.RoleService) {Summary: "Get the permission manifest",Description: "Returns every permission the server recognizes, grouped by resource. Used by permission-picking UIs.",Tags: []string{"Roles"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.GetPermissionsManifest)huma.Register(api, huma.Operation{@@ -163,8 +163,8 @@ func RegisterRoles(api huma.API, roleService *services.RoleService) {Summary: "List a user's role assignments",Description: "Reserved for global admins.",Tags: []string{"Roles"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.ListUserRoleAssignments)huma.Register(api, huma.Operation{@@ -174,15 +174,15 @@ func RegisterRoles(api huma.API, roleService *services.RoleService) {Summary: "Replace a user's manual role assignments",Description: "Replaces every source='manual' assignment for the user. source='oidc' assignments are not touched. Reserved for global admins; enforces the last-admin guard.",Tags: []string{"Roles"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireGlobalAdmin(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireGlobalAdmin(api),}, h.SetUserRoleAssignments)}// ---------- Handler implementations ----------func (h *RoleHandler) ListRoles(ctx context.Context, input *ListRolesInput) (*ListRolesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)roles, paginationResp, err := h.roleService.ListRoles(ctx, params)if err != nil {return nil, huma.Error500InternalServerError("failed to list roles: " + err.Error())@@ -195,7 +195,7 @@ func (h *RoleHandler) ListRoles(ctx context.Context, input *ListRolesInput) (*LiBody: base.Paginated[roletypes.Role]{Success: true,Data: dtos,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -214,7 +214,7 @@ func (h *RoleHandler) GetRole(ctx context.Context, input *GetRoleInput) (*GetRol}func (h *RoleHandler) CreateRole(ctx context.Context, input *CreateRoleInput) (*CreateRoleOutput, error) {- callerPS, _ := humamw.PermissionsFromContext(ctx)+ callerPS, _ := middleware.PermissionsFromContext(ctx)if err := h.roleService.ValidatePermissionsAgainstCaller(callerPS, input.Body.Permissions); err != nil {switch {case errors.Is(err, common.ErrUnknownPermission):@@ -240,7 +240,7 @@ func (h *RoleHandler) CreateRole(ctx context.Context, input *CreateRoleInput) (*}func (h *RoleHandler) UpdateRole(ctx context.Context, input *UpdateRoleInput) (*UpdateRoleOutput, error) {- callerPS, _ := humamw.PermissionsFromContext(ctx)+ callerPS, _ := middleware.PermissionsFromContext(ctx)if err := h.roleService.ValidatePermissionsAgainstCaller(callerPS, input.Body.Permissions); err != nil {switch {case errors.Is(err, common.ErrUnknownPermission):@@ -313,7 +313,7 @@ func (h *RoleHandler) SetUserRoleAssignments(ctx context.Context, input *SetUser}if err := h.roleService.SetUserAssignments(ctx, input.UserID, desired); err != nil {switch {- case errors.Is(err, services.ErrUserNotFound):+ case errors.Is(err, common.ErrUserNotFound):return nil, huma.Error404NotFound(err.Error())case errors.Is(err, common.ErrInvalidRoleAssignment):return nil, huma.Error400BadRequest(err.Error())diff --git a/backend/internal/role/module.go b/backend/internal/role/module.gonew file mode 100644index 000000000..a2a4549fe--- /dev/null+++ b/backend/internal/role/module.go@@ -0,0 +1,39 @@+// Package role owns RBAC roles, assignments, permission resolution, and role HTTP routes.+package role++import (+ "github.com/danielgtaylor/huma/v2"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+)++// Dependencies are the collaborators the role domain needs.+type Dependencies struct {+ DB *database.DB+}++// Module owns role persistence and its HTTP surface.+type Module struct {+ service *RoleService+}++// New builds the role domain from its dependencies.+func New(deps Dependencies) *Module {+ return &Module{service: NewRoleService(deps.DB)}+}++// Service exposes role operations to authentication and authorization collaborators.+func (m *Module) Service() *RoleService {+ if m == nil {+ return nil+ }+ return m.service+}++// RegisterRoutes mounts role endpoints for runtime and schema discovery.+func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterRoles(api, nil)+ return+ }+ RegisterRoles(api, m.service)+}diff --git a/backend/internal/services/role_service.go b/backend/internal/role/service.gosimilarity index 97%rename from backend/internal/services/role_service.gorename to backend/internal/role/service.goindex 6f988b13d..fb2c66a09 100644--- a/backend/internal/services/role_service.go+++ b/backend/internal/role/service.go@@ -1,4 +1,4 @@-package services+package roleimport ("context"@@ -255,7 +255,7 @@ func (s *RoleService) backfillPermsForKeyInternal(ctx context.Context, tx *gorm.}return nil, errors.WrapIf(err, "failed to load api key owner")}- ps, err := s.resolveUserPermissionsInternal(ctx, tx, owner.ID)+ ps, err := s.ResolveUserPermissionsInDB(ctx, tx, owner.ID)if err != nil {return nil, err}@@ -339,7 +339,7 @@ func (s *RoleService) CreateRole(ctx context.Context, name string, description *// lockAssignedUserRowsInternal locks the user rows of everyone currently// assigned to roleID, in deterministic id order. Role-definition writes call-// this so they serialize with UserService mutation transactions, whose in-tx+// this so they serialize with user-domain mutation transactions, whose in-transaction// privilege checks lock the same rows: without it a role edit could change a// holder's effective admin status between that check and the mutation commit.// Returns the affected user ids for post-commit cache invalidation.@@ -488,13 +488,13 @@ func (s *RoleService) replaceUserAssignmentsForSourceInternal(ctx context.Contex}err := dbutil.WithTx(ctx, s.db.DB, func(tx *gorm.DB) error {// Lock the target's user row so assignment swaps serialize with- // UserService update/delete transactions, whose in-tx privilege+ // user-domain update/delete transactions, whose in-transaction privilege// checks lock the same row.if err := tx.Clauses(clause.Locking{Strength: "UPDATE"}).Where("id = ?", userID).First(&models.User{}).Error; err != nil {if errors.Is(err, gorm.ErrRecordNotFound) {- return ErrUserNotFound+ return common.ErrUserNotFound}return errors.WrapIf(err, "failed to lock user for assignment update")}@@ -660,7 +660,7 @@ func (s *RoleService) ResolvePermissions(ctx context.Context, user *models.User)if ps, ok, _ := s.userCache.Get(user.ID); ok {return ps, nil}- ps, err := s.resolveUserPermissionsInternal(ctx, s.db.WithContext(ctx), user.ID)+ ps, err := s.ResolveUserPermissionsInDB(ctx, s.db.WithContext(ctx), user.ID)if err != nil {return nil, err}@@ -668,7 +668,8 @@ func (s *RoleService) ResolvePermissions(ctx context.Context, user *models.User)return ps, nil}-func (s *RoleService) resolveUserPermissionsInternal(_ context.Context, tx *gorm.DB, userID string) (*authz.PermissionSet, error) {+// ResolveUserPermissionsInDB resolves permissions using the supplied transaction or database handle.+func (s *RoleService) ResolveUserPermissionsInDB(_ context.Context, tx *gorm.DB, userID string) (*authz.PermissionSet, error) {// Scan into raw string for the permissions JSON column to avoid GORM's// schema-introspection on anonymous local structs (which can't see the// type tags needed to wire models.StringSlice's Scanner).@@ -739,7 +740,7 @@ func (s *RoleService) ResolveApiKeyPermissions(ctx context.Context, apiKeyID str// creator's capabilities happens in the handler layer.func (s *RoleService) SetApiKeyPermissions(ctx context.Context, apiKeyID string, grants []models.ApiKeyPermission) error {err := dbutil.WithTx(ctx, s.db.DB, func(tx *gorm.DB) error {- return s.setApiKeyPermissionsInternal(ctx, tx, apiKeyID, grants)+ return s.SetApiKeyPermissionsInDB(ctx, tx, apiKeyID, grants)})if err != nil {return err@@ -748,7 +749,8 @@ func (s *RoleService) SetApiKeyPermissions(ctx context.Context, apiKeyID string,return nil}-func (s *RoleService) setApiKeyPermissionsInternal(ctx context.Context, tx *gorm.DB, apiKeyID string, grants []models.ApiKeyPermission) error {+// SetApiKeyPermissionsInDB replaces permission rows using the supplied transaction or database handle.+func (s *RoleService) SetApiKeyPermissionsInDB(ctx context.Context, tx *gorm.DB, apiKeyID string, grants []models.ApiKeyPermission) error {for i := range grants {grants[i].ApiKeyID = apiKeyID}diff --git a/backend/internal/services/role_service_test.go b/backend/internal/role/service_test.gosimilarity index 84%rename from backend/internal/services/role_service_test.gorename to backend/internal/role/service_test.goindex 84385bcae..c881c21a3 100644--- a/backend/internal/services/role_service_test.go+++ b/backend/internal/role/service_test.go@@ -1,15 +1,19 @@-package services+package roleimport ("context""slices""testing"+ "time""emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/libtnb/sqlite""github.com/stretchr/testify/require"+ "gorm.io/gorm")func TestBackfillPermsForKeyDeduplicatesGlobalAndEnvironmentPermissions(t *testing.T) {@@ -257,3 +261,56 @@ func TestEffectiveGlobalAdminCountIgnoresEnvScopedAndServiceAccounts(t *testing.require.NoError(t, err)require.Equal(t, 1, count)}++func setupAuthServiceTestDB(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(+ &models.SettingVariable{},+ &models.User{},+ &models.UserSession{},+ &models.Environment{},+ &models.Role{},+ &models.UserRoleAssignment{},+ &models.ApiKey{},+ &models.ApiKeyPermission{},+ &models.OidcRoleMapping{},+ ))+ return &database.DB{DB: db}+}++func setupUserAndRoleServices(t *testing.T) (*database.DB, *RoleService) {+ t.Helper()+ db := setupAuthServiceTestDB(t)+ roleService := NewRoleService(db)+ require.NoError(t, roleService.EnsureBuiltInRoles(context.Background()))+ return db, roleService+}++func createTestUser(t *testing.T, db *database.DB, id, username string) *models.User {+ t.Helper()+ created := &models.User{BaseModel: models.BaseModel{ID: id}, Username: username}+ require.NoError(t, db.WithContext(context.Background()).Create(created).Error)+ return created+}++func grantGlobalAdmin(t *testing.T, roleService *RoleService, userID string) {+ t.Helper()+ require.NoError(t, roleService.SetUserAssignments(context.Background(), userID, []models.UserRoleAssignment{+ {RoleID: authz.BuiltInRoleAdmin},+ }))+}++func createTestEnvironment(t *testing.T, db *database.DB, id, apiURL string, accessToken *string) {+ t.Helper()+ now := time.Now()+ require.NoError(t, db.WithContext(context.Background()).Create(&models.Environment{+ BaseModel: models.BaseModel{ID: id, CreatedAt: now, UpdatedAt: &now},+ Name: "env-" + id,+ ApiUrl: apiURL,+ Status: string(models.EnvironmentStatusOnline),+ Enabled: true,+ AccessToken: accessToken,+ }).Error)+}diff --git a/backend/api/handlers/customize.go b/backend/internal/search/customize_handler.gosimilarity index 81%rename from backend/api/handlers/customize.gorename to backend/internal/search/customize_handler.goindex 8ce90a6a7..2851ddaa8 100644--- a/backend/api/handlers/customize.go+++ b/backend/internal/search/customize_handler.go@@ -1,4 +1,4 @@-package handlers+package searchimport ("context"@@ -6,26 +6,26 @@ import ("strings""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/category"- "github.com/getarcaneapp/arcane/types/v2/search"+ searchtypes "github.com/getarcaneapp/arcane/types/v2/search")// CustomizeHandler handles customization search endpoints.type CustomizeHandler struct {- customizeSearchService *services.CustomizeSearchService+ customizeSearchService *CustomizeSearchService}// --- Input/Output Types ---type SearchCustomizeInput struct {- Body search.Request+ Body searchtypes.Request}type SearchCustomizeOutput struct {- Body search.Response+ Body searchtypes.Response}type GetCustomizeCategoriesInput struct{}@@ -35,7 +35,7 @@ type GetCustomizeCategoriesOutput struct {}// RegisterCustomize registers customization endpoints using Huma.-func RegisterCustomize(api huma.API, customizeSearchService *services.CustomizeSearchService) {+func RegisterCustomize(api huma.API, customizeSearchService *CustomizeSearchService) {h := &CustomizeHandler{customizeSearchService: customizeSearchService,}@@ -47,7 +47,7 @@ func RegisterCustomize(api huma.API, customizeSearchService *services.CustomizeSSummary: "Search customization options",Description: "Search customization categories and options by query",Tags: []string{"Customize"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.Search)huma.Register(api, huma.Operation{@@ -57,7 +57,7 @@ func RegisterCustomize(api huma.API, customizeSearchService *services.CustomizeSSummary: "Get customization categories",Description: "Get all available customization categories with metadata",Tags: []string{"Customize"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.GetCategories)}@@ -80,7 +80,7 @@ func (h *CustomizeHandler) Search(ctx context.Context, input *SearchCustomizeInpreturn nil, huma.Error400BadRequest("Query parameter is required")}- ps, _ := humamw.PermissionsFromContext(ctx)+ ps, _ := middleware.PermissionsFromContext(ctx)results := h.customizeSearchService.Search(input.Body.Query)results.Results = filterCustomizeCategoriesInternal(ps, results.Results)results.Count = len(results.Results)@@ -92,7 +92,7 @@ func (h *CustomizeHandler) Search(ctx context.Context, input *SearchCustomizeInp// GetCategories returns all available customization categories.func (h *CustomizeHandler) GetCategories(ctx context.Context, input *GetCustomizeCategoriesInput) (*GetCustomizeCategoriesOutput, error) {- ps, _ := humamw.PermissionsFromContext(ctx)+ ps, _ := middleware.PermissionsFromContext(ctx)categories := filterCustomizeCategoriesInternal(ps, h.customizeSearchService.GetCustomizeCategories())return &GetCustomizeCategoriesOutput{diff --git a/backend/internal/services/customize_search_service.go b/backend/internal/search/customize_service.gosimilarity index 96%rename from backend/internal/services/customize_search_service.gorename to backend/internal/search/customize_service.goindex ccd108f9d..710b51b88 100644--- a/backend/internal/services/customize_search_service.go+++ b/backend/internal/search/customize_service.go@@ -1,4 +1,4 @@-package services+package searchimport ("reflect"@@ -11,7 +11,7 @@ import ("github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/types/v2/category""github.com/getarcaneapp/arcane/types/v2/meta"- "github.com/getarcaneapp/arcane/types/v2/search"+ searchtypes "github.com/getarcaneapp/arcane/types/v2/search")type CustomizeSearchService struct {@@ -113,10 +113,10 @@ func (s *CustomizeSearchService) buildCategoriesFromModel() []category.Category}// Search performs a relevance-scored search across all customization categories and items-func (s *CustomizeSearchService) Search(query string) search.Response {+func (s *CustomizeSearchService) Search(query string) searchtypes.Response {query = strings.ToLower(strings.TrimSpace(query))if query == "" {- return search.Response{+ return searchtypes.Response{Results: []category.Category{},Query: query,Count: 0,@@ -158,7 +158,7 @@ func (s *CustomizeSearchService) Search(query string) search.Response {return results[i].RelevanceScore > results[j].RelevanceScore})- return search.Response{+ return searchtypes.Response{Results: results,Query: query,Count: len(results),diff --git a/backend/internal/search/module.go b/backend/internal/search/module.gonew file mode 100644index 000000000..10f0c6471--- /dev/null+++ b/backend/internal/search/module.go@@ -0,0 +1,44 @@+// Package search owns the metadata-backed settings and customization search indexes+// and the HTTP surface for customization search.+package search++import "github.com/danielgtaylor/huma/v2"++// Module owns both search indexes and mounts the customization search routes.+type Module struct {+ settings *SettingsSearchService+ customize *CustomizeSearchService+}++// New builds the settings and customization search indexes.+func New() *Module {+ return &Module{+ settings: NewSettingsSearchService(),+ customize: NewCustomizeSearchService(),+ }+}++// SettingsService exposes the settings search index to the settings domain.+func (m *Module) SettingsService() *SettingsSearchService {+ if m == nil {+ return nil+ }+ return m.settings+}++// CustomizeService exposes the customization search index to direct collaborators.+func (m *Module) CustomizeService() *CustomizeSearchService {+ if m == nil {+ return nil+ }+ return m.customize+}++// RegisterRoutes mounts customization search endpoints for runtime and schema discovery.+func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterCustomize(api, nil)+ return+ }+ RegisterCustomize(api, m.customize)+}diff --git a/backend/internal/services/settings_search_service.go b/backend/internal/search/settings_service.gosimilarity index 96%rename from backend/internal/services/settings_search_service.gorename to backend/internal/search/settings_service.goindex 1dfa2471f..faac25d45 100644--- a/backend/internal/services/settings_search_service.go+++ b/backend/internal/search/settings_service.go@@ -1,4 +1,4 @@-package services+package searchimport ("reflect"@@ -11,7 +11,7 @@ import ("github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/types/v2/category""github.com/getarcaneapp/arcane/types/v2/meta"- "github.com/getarcaneapp/arcane/types/v2/search"+ searchtypes "github.com/getarcaneapp/arcane/types/v2/search")type SettingsSearchService struct {@@ -122,10 +122,10 @@ func (s *SettingsSearchService) buildCategoriesFromModel() []category.Category {}// Search performs a relevance-scored search across settings categories and individual settings-func (s *SettingsSearchService) Search(query string) search.Response {+func (s *SettingsSearchService) Search(query string) searchtypes.Response {query = strings.ToLower(strings.TrimSpace(query))if query == "" {- return search.Response{+ return searchtypes.Response{Results: []category.Category{},Query: query,Count: 0,@@ -168,7 +168,7 @@ func (s *SettingsSearchService) Search(query string) search.Response {// Sort by relevance (highest first)s.sortByRelevance(results)- return search.Response{+ return searchtypes.Response{Results: results,Query: query,Count: len(results),diff --git a/backend/internal/services/build_workspace_service_test.go b/backend/internal/services/build_workspace_service_test.godeleted file mode 100644index 491874812..000000000--- a/backend/internal/services/build_workspace_service_test.go+++ /dev/null@@ -1,26 +0,0 @@-package services--import (- "context"- "os"- "path/filepath"- "testing"-- "github.com/stretchr/testify/require"-)--func TestBuildWorkspaceServiceDeleteFileRejectsCurrentDirectoryInternal(t *testing.T) {- ctx := context.Background()- db := setupSettingsTestDB(t)- settingsService, err := newSettingsServiceForTestInternal(t, ctx, db)- require.NoError(t, err)-- root := t.TempDir()- require.NoError(t, settingsService.UpdateSetting(ctx, "buildsDirectory", root))- sentinel := filepath.Join(root, "keep.txt")- require.NoError(t, os.WriteFile(sentinel, []byte("keep"), 0o600))-- err = NewBuildWorkspaceService(settingsService).DeleteFile(ctx, ".")- require.ErrorContains(t, err, "cannot delete root directory")- require.FileExists(t, sentinel)-}diff --git a/backend/internal/services/environment_service.go b/backend/internal/services/environment_service.godeleted file mode 100644index 96b8ba456..000000000--- a/backend/internal/services/environment_service.go+++ /dev/null@@ -1,2362 +0,0 @@-package services--import (- "github.com/samber/mo"-- "context"- "encoding/json/v2"- "fmt"- "log/slog"- "maps"- "net/http"- "strings"- "sync"- "time"-- "emperror.dev/errors"-- "github.com/getarcaneapp/arcane/backend/v2/internal/actors"- "github.com/getarcaneapp/arcane/backend/v2/internal/common"- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"- "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"- "github.com/getarcaneapp/arcane/backend/v2/pkg/remenv"- pkgutils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"- httputils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"- "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper"- "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/validation"- "github.com/getarcaneapp/arcane/types/v2/containerregistry"- "github.com/getarcaneapp/arcane/types/v2/environment"- "github.com/getarcaneapp/arcane/types/v2/gitops"- schedulertypes "github.com/getarcaneapp/arcane/types/v2/scheduler"- "github.com/google/uuid"- "github.com/moby/moby/client"- "github.com/samber/hot"- "go.getarcane.app/sys/crypto"- "gorm.io/gorm"-)--type EnvironmentService struct {- db *database.DB- httpClient *http.Client- dockerService *DockerClientService- eventService *EventService- settingsService *SettingsService- apiKeyService *ApiKeyService- remoteClient *remenv.Client- tokenCacheMu sync.RWMutex- tokenCache *hot.HotCache[string, string]- tokenByEnvID map[string]string- remoteEnvMu sync.RWMutex- remoteEnvs map[string]models.Environment-- // scheduler and lifecycleCtx are injected post-construction via SetScheduler- // (manager-only). Each enabled environment gets its own health-check job; this- // replaces the single global environment-health job.- scheduler schedulertypes.DynamicScheduler- lifecycleCtx context.Context- admissionGate *actors.Gate[actors.AdmissionKey]-- // variableSyncer is injected post-construction via SetVariableSyncer- // (manager-only) to avoid a wire cycle with VariableService.- variableSyncer VariableSyncer-- // runtimeWatchers receive a coalesced wake-up whenever an environment's- // liveness changes. See environment_runtime_notify.go.- runtimeWatchMu sync.Mutex- runtimeWatchers map[int]chan struct{}- runtimeWatcherSeq int-}--// VariableSyncer pushes the effective global-variable set to one environment.-// Implemented by VariableService.-type VariableSyncer interface {- SyncEnvironment(ctx context.Context, envID string) error-}--const (- defaultEnvironmentHealthInterval = "0 */2 * * * *"- environmentHealthCheckTimeout = 90 * time.Second-)--const edgeTokenCacheTTL = time.Minute--const (- ErrEnvironmentAccessTokenRequired = errors.Sentinel("environment access token required")- ErrInvalidEnvironmentAccessToken = errors.Sentinel("invalid environment access token")-)--func NewEnvironmentService(db *database.DB, httpClient *http.Client, dockerService *DockerClientService, eventService *EventService, settingsService *SettingsService, apiKeyService *ApiKeyService) *EnvironmentService {- if httpClient == nil {- httpClient = http.DefaultClient- }- return &EnvironmentService{- db: db,- httpClient: httpClient,- dockerService: dockerService,- eventService: eventService,- settingsService: settingsService,- apiKeyService: apiKeyService,- remoteClient: remenv.NewClient(httpClient, remenv.TunnelTransportFuncs{- EnsureAvailableFunc: ensureRemoteEnvironmentTunnelAvailableInternal,- DoFunc: doRemoteEnvironmentTunnelRequestInternal,- }),- tokenCache: hot.NewHotCache[string, string](hot.LRU, 1024).- WithTTL(edgeTokenCacheTTL).- WithJanitor().- Build(),- tokenByEnvID: make(map[string]string),- remoteEnvs: make(map[string]models.Environment),- }-}--const environmentHealthJobPrefix = "environment-health:"-const environmentHealthAdmissionScopeInternal = "environment-health"--func environmentHealthJobNameInternal(envID string) string { return environmentHealthJobPrefix + envID }--// SetScheduler injects the job scheduler and app lifecycle context. Called during-// bootstrap on the manager only (agent mode leaves scheduler nil, so all health-job-// registration becomes a no-op).-func (s *EnvironmentService) SetScheduler(ctx context.Context, scheduler schedulertypes.DynamicScheduler, admissionGate *actors.Gate[actors.AdmissionKey]) error { //nolint:contextcheck // health-check jobs must capture the app lifecycle context, not request contexts- if scheduler == nil || admissionGate == nil {- return errors.New("environment scheduler dependencies unavailable")- }- if ctx == nil {- ctx = context.Background()- }- s.lifecycleCtx = ctx- s.scheduler = scheduler- s.admissionGate = admissionGate- return nil-}--// SetVariableSyncer injects the global-variable syncer. Called during-// bootstrap on the manager only; agents leave it nil.-func (s *EnvironmentService) SetVariableSyncer(syncer VariableSyncer) {- s.variableSyncer = syncer-}--func (s *EnvironmentService) schedulerCtxInternal(ctx context.Context) context.Context {- if s.lifecycleCtx != nil {- return s.lifecycleCtx- }- if ctx != nil {- return context.WithoutCancel(ctx)- }- return context.Background()-}--// buildHealthJobInternal builds the dynamic job for one environment. The schedule is-// the global environmentHealthInterval (environment health has no per-entity-// interval); the run body re-reads nothing it can't and self-cancels cleanly when-// the environment is gone.-func (s *EnvironmentService) buildHealthJobInternal(envID string) *schedulertypes.GenericJob {- return &schedulertypes.GenericJob{- JobName: environmentHealthJobNameInternal(envID),- ScheduleFn: func(ctx context.Context) string {- sched := s.settingsService.GetStringSetting(ctx, "environmentHealthInterval", defaultEnvironmentHealthInterval)- if sched == "" {- return defaultEnvironmentHealthInterval- }- return sched- },- RunFn: func(ctx context.Context) {- s.runHealthCheckInternal(ctx, envID)- },- }-}--func (s *EnvironmentService) registerHealthJobInternal(ctx context.Context, envID string) {- if s.scheduler == nil {- return- }- schedulerCtx := s.schedulerCtxInternal(ctx)- if err := s.scheduler.AddJob(schedulerCtx, s.buildHealthJobInternal(envID)); err != nil {- slog.ErrorContext(schedulerCtx, "Failed to register environment health job", "environment_id", envID, "error", err)- }-}--func (s *EnvironmentService) removeHealthJobInternal(ctx context.Context, envID string) {- if s.scheduler == nil {- return- }- s.scheduler.RemoveJob(s.schedulerCtxInternal(ctx), environmentHealthJobNameInternal(envID))-}--func (s *EnvironmentService) listEnabledEnvironmentIDsInternal(ctx context.Context) ([]string, error) {- var ids []string- if err := s.db.WithContext(ctx).- Table("environments").- Where("enabled = ?", true).- Pluck("id", &ids).Error; err != nil {- return nil, errors.WrapIf(err, "failed to list enabled environments")- }- return ids, nil-}--func (s *EnvironmentService) registerAllEnabledHealthJobsInternal(ctx context.Context) int {- ids, err := s.listEnabledEnvironmentIDsInternal(ctx)- if err != nil {- slog.ErrorContext(ctx, "failed to list environments for health jobs", "error", err)- return 0- }- for _, id := range ids {- s.registerHealthJobInternal(ctx, id)- }- return len(ids)-}--// RegisterHealthJobsOnStartup registers a health-check job for every enabled-// environment. Replaces the old global environment-health job.-func (s *EnvironmentService) RegisterHealthJobsOnStartup(ctx context.Context) {- if s.scheduler == nil {- return- }- n := s.registerAllEnabledHealthJobsInternal(ctx)- slog.InfoContext(ctx, "Registered environment health jobs on startup", "count", n)-}--// RescheduleHealthJobs re-registers all enabled environments' health jobs, picking-// up a changed global interval. Wired from the Jobs UI via JobService.-func (s *EnvironmentService) RescheduleHealthJobs(ctx context.Context) {- if s.scheduler == nil {- return- }- s.registerAllEnabledHealthJobsInternal(ctx)-}--// RunHealthChecksNow runs every enabled environment's health check synchronously.-// Backs the "run now" button for the environment-health job in the Jobs UI.-func (s *EnvironmentService) RunHealthChecksNow(ctx context.Context) error {- ids, err := s.listEnabledEnvironmentIDsInternal(ctx)- if err != nil {- return err- }- for _, id := range ids {- s.runHealthCheckInternal(ctx, id)- }- return nil-}--// runHealthCheckInternal tests one environment's connection (updating its DB status)-// and, for online remotes, syncs registries and repositories to it.-func (s *EnvironmentService) runHealthCheckInternal(ctx context.Context, envID string) {- lease, admitted, err := s.admissionGate.TryAcquire(ctx, actors.AdmissionKey{Scope: environmentHealthAdmissionScopeInternal, ID: envID})- if err != nil {- slog.ErrorContext(ctx, "environment health check admission failed", "environment_id", envID, "error", err)- return- }- if !admitted {- slog.WarnContext(ctx, "environment health check skipped; previous run still in progress", "environment_id", envID)- return- }- defer lease.Release()-- status, err := s.TestConnection(ctx, envID, nil)- switch {- case err != nil:- slog.WarnContext(ctx, "environment health check failed", "environment_id", envID, "status", status, "error", err)- return- case status != "online":- return- }-- // Local environment (ID "0") has no registries/repositories to push.- if envID == "0" {- return- }-- syncCtx, cancel := context.WithTimeout(ctx, environmentHealthCheckTimeout)- defer cancel()- if err := s.SyncRegistriesToEnvironment(syncCtx, envID); err != nil {- slog.WarnContext(syncCtx, "failed to sync registries during health check", "environment_id", envID, "error", err)- }- if err := s.SyncRepositoriesToEnvironment(syncCtx, envID); err != nil {- slog.WarnContext(syncCtx, "failed to sync git repositories during health check", "environment_id", envID, "error", err)- }- if s.variableSyncer != nil {- if err := s.variableSyncer.SyncEnvironment(syncCtx, envID); err != nil {- slog.WarnContext(syncCtx, "failed to sync global variables during health check", "environment_id", envID, "error", err)- }- }-}--func (s *EnvironmentService) ResolveEdgeEnvironmentByToken(ctx context.Context, token string) (string, error) {- token = strings.TrimSpace(token)- if token == "" {- return "", errors.New("agent token required")- }-- if envID, ok := s.getCachedEnvironmentIDForTokenInternal(token).Get(); ok {- return envID, nil- }-- var env models.Environment- if err := s.db.WithContext(ctx).- Select("id", "access_token").- Where("is_edge = ?", true).- Where("access_token = ?", token).- First(&env).Error; err != nil {- if errors.Is(err, gorm.ErrRecordNotFound) {- s.logEdgeTokenResolveMissInternal(ctx, token)- return "", errors.New("invalid agent token")- }- return "", errors.WrapIf(err, "failed to resolve edge environment by token")- }-- s.cacheEnvironmentTokenInternal(env.ID, token)- return env.ID, nil-}--// logEdgeTokenResolveMissInternal emits a debug log diagnosing why an agent-// token failed to resolve to an edge environment. Counts existing edge-// environments (by access_token presence) so operators can distinguish-// "no edge envs configured" from "token does not match any configured env".-// Token contents are never logged in full — only length and a short-// fingerprint that cannot be reversed.-func (s *EnvironmentService) logEdgeTokenResolveMissInternal(ctx context.Context, token string) {- if s == nil || s.db == nil {- return- }- if !slog.Default().Enabled(ctx, slog.LevelDebug) {- return- }-- var totalEdgeEnvs int64- var edgeEnvsWithToken int64- totalEdgeEnvsErr := s.db.WithContext(ctx).Model(&models.Environment{}).Where("is_edge = ?", true).Count(&totalEdgeEnvs).Error- edgeEnvsWithTokenErr := s.db.WithContext(ctx).Model(&models.Environment{}).- Where("is_edge = ?", true).- Where("access_token IS NOT NULL AND access_token != ?", "").- Count(&edgeEnvsWithToken).Error-- args := []any{- "token_length", len(token),- "token_fingerprint", remenv.RedactedTokenFingerprint(token),- }- if totalEdgeEnvsErr == nil {- args = append(args, "edge_envs_total", totalEdgeEnvs)- }- if edgeEnvsWithTokenErr == nil {- args = append(args, "edge_envs_with_access_token", edgeEnvsWithToken)- }-- slog.DebugContext(ctx, "Edge agent token did not match any environment", args...)-}--func (s *EnvironmentService) getCachedEnvironmentIDForTokenInternal(token string) mo.Option[string] {- if s == nil || s.tokenCache == nil || token == "" {- return mo.None[string]()- }-- staleEnvironmentID, wasCached := s.tokenCache.Peek(token)- environmentID, ok, _ := s.tokenCache.Get(token)- if ok {- return mo.Some(environmentID)- }- if wasCached {- s.tokenCacheMu.Lock()- if currentToken, indexed := s.tokenByEnvID[staleEnvironmentID]; indexed && currentToken == token {- delete(s.tokenByEnvID, staleEnvironmentID)- }- s.tokenCacheMu.Unlock()- }- return mo.None[string]()-}--func (s *EnvironmentService) cacheEnvironmentTokenInternal(envID string, token string) {- if s == nil || s.tokenCache == nil || envID == "" || token == "" {- return- }-- s.tokenCacheMu.Lock()- defer s.tokenCacheMu.Unlock()-- if previousToken, ok := s.tokenByEnvID[envID]; ok && previousToken != token {- s.tokenCache.Delete(previousToken)- }-- s.tokenByEnvID[envID] = token- s.tokenCache.Set(token, envID)-}--func (s *EnvironmentService) invalidateEnvironmentTokenInternal(envID string) {- if s == nil || envID == "" {- return- }-- s.tokenCacheMu.Lock()- defer s.tokenCacheMu.Unlock()-- if token, ok := s.tokenByEnvID[envID]; ok {- delete(s.tokenByEnvID, envID)- if s.tokenCache != nil {- s.tokenCache.Delete(token)- }- }-}--func (s *EnvironmentService) syncEnvironmentTokenCacheInternal(envID string, token string) {- if s == nil || envID == "" {- return- }-- s.invalidateEnvironmentTokenInternal(envID)-- resolvedToken := strings.TrimSpace(token)-- if resolvedToken != "" {- s.cacheEnvironmentTokenInternal(envID, resolvedToken)- }-}--// GetActiveRemoteEnvironmentSnapshot returns the latest in-process snapshot for-// an enabled, visible, non-local remote environment.-func (s *EnvironmentService) GetActiveRemoteEnvironmentSnapshot(environmentID string) mo.Option[models.Environment] {- if s == nil || environmentID == "" {- return mo.None[models.Environment]()- }-- s.remoteEnvMu.RLock()- envRecord, ok := s.remoteEnvs[environmentID]- s.remoteEnvMu.RUnlock()- if !ok || !isActiveRemoteEnvironmentInternal(envRecord) {- return mo.None[models.Environment]()- }- return mo.Some(envRecord)-}--func isActiveRemoteEnvironmentInternal(environment models.Environment) bool {- return environment.ID != "" && environment.ID != "0" && environment.Enabled && !environment.Hidden-}--func (s *EnvironmentService) syncRemoteEnvironmentSnapshotsInternal(environments []models.Environment) {- if s == nil {- return- }-- next := make(map[string]models.Environment, len(environments))- for _, envRecord := range environments {- if isActiveRemoteEnvironmentInternal(envRecord) {- next[envRecord.ID] = envRecord- }- }-- s.remoteEnvMu.Lock()- s.remoteEnvs = next- s.remoteEnvMu.Unlock()-}--func (s *EnvironmentService) cacheRemoteEnvironmentSnapshotInternal(environment models.Environment) {- if s == nil {- return- }-- if !isActiveRemoteEnvironmentInternal(environment) {- s.removeRemoteEnvironmentSnapshotInternal(environment.ID)- return- }-- s.remoteEnvMu.Lock()- s.remoteEnvs[environment.ID] = environment- s.remoteEnvMu.Unlock()-}--func (s *EnvironmentService) removeRemoteEnvironmentSnapshotInternal(environmentID string) {- if s == nil || environmentID == "" {- return- }-- s.remoteEnvMu.Lock()- delete(s.remoteEnvs, environmentID)- s.remoteEnvMu.Unlock()-}--func (s *EnvironmentService) updateRemoteEnvironmentSnapshotInternal(environmentID string, update func(*models.Environment)) {- if s == nil || environmentID == "" || update == nil {- return- }-- s.remoteEnvMu.Lock()- defer s.remoteEnvMu.Unlock()-- envRecord, ok := s.remoteEnvs[environmentID]- if !ok {- return- }- update(&envRecord)- if isActiveRemoteEnvironmentInternal(envRecord) {- s.remoteEnvs[environmentID] = envRecord- } else {- delete(s.remoteEnvs, environmentID)- }-}--// LocalEnvironmentID is the reserved ID of the environment Arcane manages directly —-// the manager itself, as opposed to a remote agent.-const LocalEnvironmentID = "0"--// localEnvironmentFallbackNameInternal labels the local environment when its stored-// name is unreadable or blank. Its real name is user-editable, so this is only ever a-// fallback — never assume it, resolve the name instead.-const localEnvironmentFallbackNameInternal = "Local"--// EnvironmentDisplayName returns the label to show for an environment: its stored name-// when set, otherwise a readable fallback — the local environment gets a friendly-// label, any other environment falls back to its ID.-func EnvironmentDisplayName(environmentID, storedName string) string {- if name := strings.TrimSpace(storedName); name != "" {- return name- }- id := strings.TrimSpace(environmentID)- if id == "" || id == LocalEnvironmentID {- return localEnvironmentFallbackNameInternal- }- return id-}--// ResolveEnvironmentName looks up an environment and returns the label to show for it.-// Use this instead of hardcoding a name for a known ID: names are user-editable, so-// even the local environment's is not fixed.-func (s *EnvironmentService) ResolveEnvironmentName(ctx context.Context, environmentID string) string {- if strings.TrimSpace(environmentID) == "" {- environmentID = LocalEnvironmentID- }- env, err := s.GetEnvironmentByID(ctx, environmentID)- if err != nil || env == nil {- slog.WarnContext(ctx, "failed to resolve environment name", "environmentID", environmentID, "error", err)- return EnvironmentDisplayName(environmentID, "")- }- return EnvironmentDisplayName(env.ID, env.Name)-}--func (s *EnvironmentService) EnsureLocalEnvironment(ctx context.Context, appUrl string) error {- var existingEnv models.Environment- err := s.db.WithContext(ctx).Where("id = ?", LocalEnvironmentID).First(&existingEnv).Error-- if err == nil {- // Local environment already exists, ensure ApiUrl matches current appUrl- if existingEnv.ApiUrl != appUrl {- if err := s.db.WithContext(ctx).Model(&existingEnv).Update("api_url", appUrl).Error; err != nil {- return errors.WrapIf(err, "failed to update local environment api url")- }- slog.InfoContext(ctx, "updated local environment api url", "id", LocalEnvironmentID, "url", appUrl)- }- return nil- }-- if !errors.Is(err, gorm.ErrRecordNotFound) {- return errors.WrapIf(err, "failed to check for local environment")- }-- // Create the local environment- now := time.Now()- localEnv := &models.Environment{- BaseModel: models.BaseModel{- ID: LocalEnvironmentID,- CreatedAt: now,- UpdatedAt: new(now),- },- Name: "Local Docker",- ApiUrl: appUrl,- Status: string(models.EnvironmentStatusOnline),- Enabled: true,- }-- if err := s.db.WithContext(ctx).Create(localEnv).Error; err != nil {- return errors.WrapIf(err, "failed to create local environment")- }-- slog.InfoContext(ctx, "created local environment record", "id", LocalEnvironmentID)- return nil-}--func (s *EnvironmentService) CreateEnvironment(ctx context.Context, environment *models.Environment, userID, username *string) (*models.Environment, error) {- environment.ID = uuid.New().String()-- // Only set status to offline if not already set (e.g., API key flow sets it to pending)- if environment.Status == "" {- environment.Status = string(models.EnvironmentStatusOffline)- }-- now := time.Now()- environment.CreatedAt = now- environment.UpdatedAt = new(now)-- if err := s.db.WithContext(ctx).Create(environment).Error; err != nil {- return nil, errors.WrapIf(err, "failed to create environment")- }-- // Create event in background- go s.createEnvironmentEvent(context.WithoutCancel(ctx), environment.ID, environment.Name, models.EventTypeEnvironmentCreate, "Environment Created", fmt.Sprintf("Environment '%s' was created", environment.Name), models.EventSeveritySuccess, userID, username)-- if environment.Enabled {- s.registerHealthJobInternal(ctx, environment.ID)- }- s.cacheRemoteEnvironmentSnapshotInternal(*environment)-- return environment, nil-}--func (s *EnvironmentService) ListSwarmNodeAgentEnvironments(ctx context.Context, parentEnvironmentID string) ([]models.Environment, error) {- var envs []models.Environment- if err := s.db.WithContext(ctx).- Model(&models.Environment{}).- Where("parent_environment_id = ?", parentEnvironmentID).- Find(&envs).Error; err != nil {- return nil, errors.WrapIf(err, "failed to list swarm node agent environments")- }-- return envs, nil-}--// ListSwarmNodeCandidateEnvironments returns enabled visible environments that-// can provide swarm-node coverage for a manager environment.-func (s *EnvironmentService) ListSwarmNodeCandidateEnvironments(ctx context.Context) ([]models.Environment, error) {- var envs []models.Environment- if err := s.db.WithContext(ctx).- Model(&models.Environment{}).- Where("hidden = ?", false).- Where("enabled = ?", true).- Where("id <> ?", "0").- Order("name ASC").- Find(&envs).Error; err != nil {- return nil, errors.WrapIf(err, "failed to list swarm node candidate environments")- }-- return envs, nil-}--// BindSwarmNodeEnvironment binds an existing visible environment to a swarm-// node without modifying its connection details or agent token.-func (s *EnvironmentService) BindSwarmNodeEnvironment(- ctx context.Context,- parentEnvironmentID, nodeID, environmentID string,- rebind bool,-) (*models.Environment, error) {- var envRecord models.Environment- err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {- if err := tx.Where("id = ?", environmentID).First(&envRecord).Error; err != nil {- return errors.WrapIf(err, "failed to load environment for swarm node binding")- }- if envRecord.Hidden {- return errors.New("dedicated agent environments cannot be attached as visible environments")- }- if !envRecord.Enabled {- return errors.New("disabled environments cannot be attached to swarm nodes")- }-- boundElsewhere := envRecord.ParentEnvironmentID != nil && envRecord.SwarmNodeID != nil &&- (strings.TrimSpace(*envRecord.ParentEnvironmentID) != parentEnvironmentID || strings.TrimSpace(*envRecord.SwarmNodeID) != nodeID)- if boundElsewhere && !rebind {- return errors.New("environment is already bound to another swarm node; explicit rebinding is required")- }-- var existingVisibleBindings int64- if err := tx.Model(&models.Environment{}).- Where("hidden = ? AND parent_environment_id = ? AND swarm_node_id = ? AND id <> ?", false, parentEnvironmentID, nodeID, environmentID).- Count(&existingVisibleBindings).Error; err != nil {- return errors.WrapIf(err, "failed to inspect existing swarm node binding")- }- if existingVisibleBindings > 0 && !rebind {- return errors.New("swarm node already has a visible environment binding; explicit rebinding is required")- }- if rebind {- if err := tx.Model(&models.Environment{}).- Where("hidden = ? AND parent_environment_id = ? AND swarm_node_id = ? AND id <> ?", false, parentEnvironmentID, nodeID, environmentID).- Updates(map[string]any{"parent_environment_id": nil, "swarm_node_id": nil, "updated_at": new(time.Now())}).Error; err != nil {- return errors.WrapIf(err, "failed to clear previous swarm node binding")- }- }-- if err := tx.Model(&models.Environment{}).Where("id = ?", environmentID).Updates(map[string]any{- "parent_environment_id": parentEnvironmentID,- "swarm_node_id": nodeID,- "updated_at": new(time.Now()),- }).Error; err != nil {- return errors.WrapIf(err, "failed to bind environment to swarm node")- }-- return tx.Where("id = ?", environmentID).First(&envRecord).Error- })- if err != nil {- return nil, err- }-- s.cacheRemoteEnvironmentSnapshotInternal(envRecord)- return &envRecord, nil-}--// DetachSwarmNodeEnvironment clears a visible environment binding from a node.-func (s *EnvironmentService) DetachSwarmNodeEnvironment(ctx context.Context, parentEnvironmentID, nodeID string) error {- now := time.Now()- if err := s.db.WithContext(ctx).Model(&models.Environment{}).- Where("hidden = ? AND parent_environment_id = ? AND swarm_node_id = ?", false, parentEnvironmentID, nodeID).- Updates(map[string]any{"parent_environment_id": nil, "swarm_node_id": nil, "updated_at": &now}).Error; err != nil {- return errors.WrapIf(err, "failed to detach swarm node environment")- }-- return nil-}--// DeleteSwarmNodeAgentDeployment removes a dedicated hidden agent registration-// while leaving visible remote environments untouched.-func (s *EnvironmentService) DeleteSwarmNodeAgentDeployment(ctx context.Context, parentEnvironmentID, nodeID string, userID, username *string) error {- var envRecord models.Environment- if err := s.db.WithContext(ctx).- Where("hidden = ? AND parent_environment_id = ? AND swarm_node_id = ?", true, parentEnvironmentID, nodeID).- First(&envRecord).Error; err != nil {- if errors.Is(err, gorm.ErrRecordNotFound) {- return nil- }- return errors.WrapIf(err, "failed to load swarm node agent deployment")- }-- return s.DeleteEnvironment(ctx, envRecord.ID, userID, username)-}--func buildSwarmNodeAgentNameInternal(hostname, nodeID string) string {- trimmedHostname := strings.TrimSpace(hostname)- if trimmedHostname != "" {- return trimmedHostname- }- if len(nodeID) > 12 {- nodeID = nodeID[:12]- }- return "Swarm Node " + nodeID-}--func buildSwarmNodeAgentURLInternal(nodeID string) string {- shortNodeID := nodeID- if len(shortNodeID) > 12 {- shortNodeID = shortNodeID[:12]- }- return "edge://swarm-node-" + shortNodeID-}--func (s *EnvironmentService) applySwarmNodeAgentApiKeyInternal(- ctx context.Context,- env *models.Environment,- userID, username string,- rotate bool,-) (string, error) {- if env == nil {- return "", errors.New("environment is required")- }-- if !rotate && env.AccessToken != nil && strings.TrimSpace(*env.AccessToken) != "" {- return strings.TrimSpace(*env.AccessToken), nil- }-- if s.apiKeyService == nil {- return "", errors.New("api key service not configured")- }-- oldApiKeyID := env.ApiKeyID-- apiKeyDto, err := s.apiKeyService.CreateEnvironmentApiKey(ctx, env.ID)- if err != nil {- return "", errors.WrapIf(err, "failed to create environment API key")- }-- if err := s.RegenerateEnvironmentApiKey(ctx, env.ID, apiKeyDto.ID, apiKeyDto.Key, userID, username, env.Name); err != nil {- // The new key was never linked; remove it so a failed rotation does- // not leave an orphaned valid credential behind.- if delErr := s.apiKeyService.DeleteApiKey(ctx, apiKeyDto.ID); delErr != nil && !errors.Is(delErr, ErrApiKeyNotFound) {- slog.ErrorContext(ctx, "Failed to clean up unlinked environment API key", "environmentID", env.ID, "error", delErr.Error())- }- return "", err- }-- // Delete the previous key only after the environment points at the new- // one — while still referenced it is protected and the delete would be- // rejected. Failure is non-fatal for the rotation itself, but the old key- // remains a valid credential until deleted, so log it as an error; the key- // stays visible and deletable on the API Keys page.- if oldApiKeyID != nil && *oldApiKeyID != apiKeyDto.ID {- if err := s.apiKeyService.DeleteApiKey(ctx, *oldApiKeyID); err != nil && !errors.Is(err, ErrApiKeyNotFound) {- slog.ErrorContext(ctx, "Failed to delete previous environment API key; the old key remains valid until deleted manually", "environmentID", env.ID, "error", err.Error())- }- }-- return apiKeyDto.Key, nil-}--func (s *EnvironmentService) EnsureSwarmNodeAgentEnvironment(- ctx context.Context,- parentEnvironmentID, nodeID, hostname, userID, username string,- rotate bool,-) (*models.Environment, string, error) {- if strings.TrimSpace(parentEnvironmentID) == "" {- return nil, "", errors.New("parent environment ID is required")- }- if strings.TrimSpace(nodeID) == "" {- return nil, "", errors.New("swarm node ID is required")- }-- var env models.Environment- // Prefer an existing visible binding. Legacy hidden registrations remain- // reusable, but all newly provisioned node agents are normal Remote- // Environments so one token and one agent can serve both use cases.- err := s.db.WithContext(ctx).- Where("parent_environment_id = ?", parentEnvironmentID).- Where("swarm_node_id = ?", nodeID).- Order("hidden ASC").- First(&env).Error- if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {- return nil, "", errors.WrapIf(err, "failed to load swarm node agent environment")- }-- if errors.Is(err, gorm.ErrRecordNotFound) {- createdEnv := &models.Environment{- Name: buildSwarmNodeAgentNameInternal(hostname, nodeID),- ApiUrl: buildSwarmNodeAgentURLInternal(nodeID),- Status: string(models.EnvironmentStatusPending),- Enabled: true,- IsEdge: true,- Hidden: false,- ParentEnvironmentID: new(parentEnvironmentID),- SwarmNodeID: new(nodeID),- }-- if _, createErr := s.CreateEnvironment(ctx, createdEnv, new(userID), new(username)); createErr != nil {- return nil, "", errors.WrapIf(createErr, "failed to create swarm node agent environment")- }- env = *createdEnv- }-- apiKey, err := s.applySwarmNodeAgentApiKeyInternal(ctx, &env, userID, username, rotate)- if err != nil {- return nil, "", err- }-- refreshedEnv, err := s.GetEnvironmentByID(ctx, env.ID)- if err != nil {- return nil, "", errors.WrapIf(err, "failed to refresh swarm node agent environment")- }-- return refreshedEnv, apiKey, nil-}--func (s *EnvironmentService) UpdateSwarmNodeIdentity(ctx context.Context, envID, swarmNodeID string) error {- updates := map[string]any{- "swarm_node_id": swarmNodeID,- "updated_at": new(time.Now()),- }-- if err := s.db.WithContext(ctx).Model(&models.Environment{}).Where("id = ?", envID).Updates(updates).Error; err != nil {- return errors.WrapIf(err, "failed to update swarm node identity")- }-- return nil-}--func (s *EnvironmentService) GetEnvironmentByID(ctx context.Context, id string) (*models.Environment, error) {- var envRecord models.Environment- if err := s.db.WithContext(ctx).Where("id = ?", id).First(&envRecord).Error; err != nil {- if errors.Is(err, gorm.ErrRecordNotFound) {- return nil, errors.New("environment not found")- }- return nil, errors.WrapIf(err, "failed to get environment")- }- return &envRecord, nil-}--func (s *EnvironmentService) ListEnvironmentsPaginated(ctx context.Context, params pagination.QueryParams, accessibleEnvIDs []string) ([]environment.Environment, pagination.Response, error) {- if strings.TrimSpace(params.Filters["type"]) != "" {- return s.listEnvironmentsPaginatedWithRuntimeFiltersInternal(ctx, params, accessibleEnvIDs)- }-- var envs []models.Environment- q := s.db.WithContext(ctx).Model(&models.Environment{}).Where("hidden = ?", false)- // accessibleEnvIDs == nil means "no restriction". A non-nil slice limits the- // result to those environment IDs; an empty slice therefore matches nothing.- switch {- case accessibleEnvIDs == nil:- // no restriction- case len(accessibleEnvIDs) == 0:- q = q.Where("1 = 0")- default:- q = q.Where("id IN ?", accessibleEnvIDs)- }-- if term := strings.TrimSpace(params.Search); term != "" {- searchPattern := "%" + term + "%"- q = q.Where(- "name LIKE ? OR api_url LIKE ?",- searchPattern, searchPattern,- )- }-- q = pagination.ApplyFilter(q, "status", params.Filters["status"])- q = pagination.ApplyBooleanFilter(q, "enabled", params.Filters["enabled"])-- paginationResp, err := pagination.PaginateAndSortDB(params, q, &envs)- if err != nil {- return nil, pagination.Response{}, errors.WrapIf(err, "failed to paginate environments")- }-- out, mapErr := mapper.MapSlice[models.Environment, environment.Environment](envs)- if mapErr != nil {- return nil, pagination.Response{}, errors.WrapIf(mapErr, "failed to map environments")- }-- return out, paginationResp, nil-}--// filterEnvironmentsByIDInternal returns only the environments whose ID is in-// allowedIDs. A non-nil but empty allowedIDs yields an empty slice. Used to-// restrict the runtime-filtered list path to a caller's accessible environments.-func filterEnvironmentsByIDInternal(items []environment.Environment, allowedIDs []string) []environment.Environment {- allowed := make(map[string]struct{}, len(allowedIDs))- for _, id := range allowedIDs {- allowed[id] = struct{}{}- }- out := make([]environment.Environment, 0, len(items))- for _, item := range items {- if _, ok := allowed[item.ID]; ok {- out = append(out, item)- }- }- return out-}--func (s *EnvironmentService) listEnvironmentsPaginatedWithRuntimeFiltersInternal(ctx context.Context, params pagination.QueryParams, accessibleEnvIDs []string) ([]environment.Environment, pagination.Response, error) {- var envs []models.Environment- if err := s.db.WithContext(ctx).- Model(&models.Environment{}).- Where("hidden = ?", false).- Find(&envs).Error; err != nil {- return nil, pagination.Response{}, errors.WrapIf(err, "failed to list environments")- }-- items, mapErr := mapper.MapSlice[models.Environment, environment.Environment](envs)- if mapErr != nil {- return nil, pagination.Response{}, errors.WrapIf(mapErr, "failed to map environments")- }-- // nil = no restriction; non-nil restricts to the caller's accessible envs.- if accessibleEnvIDs != nil {- items = filterEnvironmentsByIDInternal(items, accessibleEnvIDs)- }-- for i := range items {- ApplyEnvironmentRuntimeState(&items[i])- }-- config := pagination.Config[environment.Environment]{- SearchAccessors: []pagination.SearchAccessor[environment.Environment]{- func(env environment.Environment) (string, error) { return env.Name, nil },- func(env environment.Environment) (string, error) { return env.ApiUrl, nil },- },- SortBindings: []pagination.SortBinding[environment.Environment]{- {- Key: "id",- Fn: func(a, b environment.Environment) int {- return strings.Compare(strings.ToLower(a.ID), strings.ToLower(b.ID))- },- },- {- Key: "name",- Fn: func(a, b environment.Environment) int {- return strings.Compare(strings.ToLower(a.Name), strings.ToLower(b.Name))- },- },- {- Key: "status",- Fn: func(a, b environment.Environment) int {- return strings.Compare(strings.ToLower(a.Status), strings.ToLower(b.Status))- },- },- {- Key: "enabled",- Fn: func(a, b environment.Environment) int {- if a.Enabled == b.Enabled {- return 0- }- if a.Enabled {- return 1- }- return -1- },- },- {- Key: "apiUrl",- Fn: func(a, b environment.Environment) int {- return strings.Compare(strings.ToLower(a.ApiUrl), strings.ToLower(b.ApiUrl))- },- },- },- FilterAccessors: []pagination.FilterAccessor[environment.Environment]{- {- Key: "status",- Fn: func(item environment.Environment, filterValue string) bool {- return strings.EqualFold(item.Status, strings.TrimSpace(filterValue))- },- },- {- Key: "enabled",- Fn: func(item environment.Environment, filterValue string) bool {- switch strings.ToLower(strings.TrimSpace(filterValue)) {- case "true", "1":- return item.Enabled- case "false", "0":- return !item.Enabled- default:- return true- }- },- },- {- Key: "type",- Fn: environmentTypeMatchesInternal,- },- },- }-- result := pagination.SearchOrderAndPaginate(items, params, config)- paginationResp := pagination.BuildResponseFromFilterResult(result, params)-- return result.Items, paginationResp, nil-}--func environmentTypeMatchesInternal(env environment.Environment, filterValue string) bool {- return environmentTypeKeyInternal(env) == strings.ToLower(strings.TrimSpace(filterValue))-}--func environmentTypeKeyInternal(env environment.Environment) string {- if !env.IsEdge {- return "http"- }- transport := ""- if env.Connected != nil && *env.Connected && env.EdgeTransport != nil {- transport = *env.EdgeTransport- } else if env.LastEdgeTransport != nil {- // Disconnected or poll-only agents classify by the transport they- // last used rather than collapsing into the generic edge bucket.- transport = *env.LastEdgeTransport- }- switch strings.ToLower(strings.TrimSpace(transport)) {- case edge.EdgeTransportWebSocket:- return "websocket"- case edge.EdgeTransportGRPC:- return "grpc"- default:- return "edge"- }-}--func (s *EnvironmentService) ListVisibleEnvironments(ctx context.Context) ([]environment.Environment, error) {- var envs []models.Environment- if err := s.db.WithContext(ctx).- Model(&models.Environment{}).- Where("hidden = ?", false).- Order("created_at asc, id asc").- Find(&envs).Error; err != nil {- return nil, errors.WrapIf(err, "failed to list visible environments")- }-- out, mapErr := mapper.MapSlice[models.Environment, environment.Environment](envs)- if mapErr != nil {- return nil, errors.WrapIf(mapErr, "failed to map environments")- }-- for i := range out {- ApplyEnvironmentRuntimeState(&out[i])- }-- return out, nil-}--// ListRemoteEnvironmentIDs returns the IDs of enabled remote environments; it-// satisfies agg.RemoteEnvironmentLister for aggregated stream handlers.-func (s *EnvironmentService) ListRemoteEnvironmentIDs(ctx context.Context) ([]string, error) {- envs, err := s.ListRemoteEnvironments(ctx)- if err != nil {- return nil, err- }- ids := make([]string, 0, len(envs))- for _, env := range envs {- ids = append(ids, env.ID)- }- return ids, nil-}--// ListRemoteEnvironments returns all non-local, enabled environments for syncing purposes.-func (s *EnvironmentService) ListRemoteEnvironments(ctx context.Context) ([]models.Environment, error) {- var envs []models.Environment- err := s.db.WithContext(ctx).- Model(&models.Environment{}).- Where("id != ?", "0").- Where("enabled = ?", true).- Where("hidden = ?", false).- Find(&envs).Error- if err != nil {- return nil, errors.WrapIf(err, "failed to list remote environments")- }- s.syncRemoteEnvironmentSnapshotsInternal(envs)- return envs, nil-}--// SyncRegistriesToRemoteEnvironments syncs container registries to all eligible remote environments.-// Eligibility requires a non-local, enabled environment with a configured access token.-func (s *EnvironmentService) SyncRegistriesToRemoteEnvironments(ctx context.Context) error {- envs, err := s.ListRemoteEnvironments(ctx)- if err != nil {- return errors.WrapIf(err, "failed to list remote environments for registry sync")- }-- if len(envs) == 0 {- return nil- }-- var failedCount int- for _, env := range envs {- if env.AccessToken == nil || *env.AccessToken == "" {- slog.DebugContext(ctx, "Skipping registry sync for environment without access token",- "environmentID", env.ID,- "environmentName", env.Name)- continue- }-- if err := s.SyncRegistriesToEnvironment(ctx, env.ID); err != nil {- failedCount++- slog.WarnContext(ctx, "Failed to sync registries to remote environment",- "environmentID", env.ID,- "environmentName", env.Name,- "error", err.Error())- }- }-- if failedCount > 0 {- return errors.Errorf("failed to sync registries to %d remote environment(s)", failedCount)- }-- return nil-}--func (s *EnvironmentService) UpdateEnvironment(ctx context.Context, id string, updates map[string]any, userID, username *string) (*models.Environment, error) {- current, err := s.GetEnvironmentByID(ctx, id)- if err != nil {- return nil, err- }-- var nextAPIURL *string- if rawAPIURL, ok := updates["api_url"]; ok {- if apiURL, isString := rawAPIURL.(string); isString {- nextAPIURL = new(apiURL)- }- }- _, accessTokenUpdated := updates["access_token"]- if err := validation.ValidateCredentialTargetChange(- "environment API URL",- current.ApiUrl,- nextAPIURL,- func(value string) string {- normalized, normalizeErr := normalizeEnvironmentBaseURLInternal(value)- if normalizeErr != nil {- return strings.TrimSpace(value)- }- return normalized- },- map[string]bool{"accessToken": current.AccessToken != nil && *current.AccessToken != ""},- map[string]bool{"accessToken": accessTokenUpdated},- ); err != nil {- return nil, err- }-- updates["updated_at"] = new(time.Now())-- if err := s.db.WithContext(ctx).Model(&models.Environment{}).Where("id = ?", id).Updates(updates).Error; err != nil {- return nil, errors.WrapIf(err, "failed to update environment")- }-- updated, err := s.GetEnvironmentByID(ctx, id)- if err != nil {- return nil, err- }- s.cacheRemoteEnvironmentSnapshotInternal(*updated)-- if rawAccessToken, ok := updates["access_token"]; ok {- accessToken, _ := rawAccessToken.(string)- s.syncEnvironmentTokenCacheInternal(id, accessToken)- }-- // Reconcile the per-environment health job when the enabled flag is toggled.- if rawEnabled, ok := updates["enabled"]; ok {- if enabled, isBool := rawEnabled.(bool); isBool {- if enabled {- s.registerHealthJobInternal(ctx, id)- } else {- s.removeHealthJobInternal(ctx, id)- }- }- }-- // Create event in background (skip for local environment)- if id != "0" {- go s.createEnvironmentEvent(context.WithoutCancel(ctx), id, updated.Name, models.EventTypeEnvironmentUpdate, "Environment Updated", fmt.Sprintf("Environment '%s' was updated", updated.Name), models.EventSeverityInfo, userID, username)- }-- return updated, nil-}--func (s *EnvironmentService) DeleteEnvironment(ctx context.Context, id string, userID, username *string) error {- // Get environment details before deletion- env, err := s.GetEnvironmentByID(ctx, id)- if err != nil {- return err- }-- // Stop the per-environment health job before the row is removed.- s.removeHealthJobInternal(ctx, id)-- var syncIDs []string- if err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {- if err := tx.Model(&models.GitOpsSync{}).- Where("environment_id = ?", id).- Pluck("id", &syncIDs).Error; err != nil {- return errors.WrapIf(err, "failed to list environment gitops syncs")- }-- if len(syncIDs) > 0 {- if err := tx.Model(&models.Project{}).- Where("gitops_managed_by IN ?", syncIDs).- Update("gitops_managed_by", nil).Error; err != nil {- return errors.WrapIf(err, "failed to clear environment gitops project references")- }- if err := tx.Where("environment_id = ?", id).Delete(&models.GitOpsSync{}).Error; err != nil {- return errors.WrapIf(err, "failed to delete environment gitops syncs")- }- }-- if err := tx.Delete(&models.Environment{}, "id = ?", id).Error; err != nil {- return errors.WrapIf(err, "failed to delete environment")- }-- return nil- }); err != nil {- if env.Enabled {- s.registerHealthJobInternal(ctx, env.ID)- }- return err- }-- if s.scheduler != nil {- schedulerCtx := s.schedulerCtxInternal(ctx)- for _, syncID := range syncIDs {- s.scheduler.RemoveJob(schedulerCtx, gitOpsSyncJobNameInternal(syncID))- }- }-- s.invalidateEnvironmentTokenInternal(id)- s.removeRemoteEnvironmentSnapshotInternal(id)-- // Create event in background- go s.createEnvironmentEvent(context.WithoutCancel(ctx), id, env.Name, models.EventTypeEnvironmentDelete, "Environment Deleted", fmt.Sprintf("Environment '%s' was deleted", env.Name), models.EventSeverityWarning, userID, username)-- return nil-}--func (s *EnvironmentService) TestConnection(ctx context.Context, id string, customApiUrl *string) (string, error) {- envRecord, err := s.GetEnvironmentByID(ctx, id)- if err != nil {- return "error", err- }- connectionFailure := func(status string, failure error) (string, error) {- if customApiUrl == nil {- return status, failure- }- slog.WarnContext(ctx, "Environment custom URL connection test failed", "environment_id", id, "error", failure)- return "error", common.ErrEnvironmentConnectionTestFailed- }-- // Special handling for local Docker environment (ID "0")- if id == "0" && customApiUrl == nil {- return s.testLocalDockerConnection(ctx, id)- }-- // For edge environments, check if there's an active tunnel and route through it- if envRecord.IsEdge && customApiUrl == nil {- return s.testEdgeConnection(ctx, id)- }-- apiUrl := envRecord.ApiUrl- if customApiUrl != nil {- apiUrl = *customApiUrl- }-- healthURL, err := buildEnvironmentEndpointURLInternal(apiUrl, "/api/health")- if err != nil {- if customApiUrl == nil {- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))- }- return connectionFailure("offline", errors.WrapIf(err, "invalid environment API URL"))- }-- reqCtx, cancel := context.WithTimeout(ctx, 10*time.Second)- defer cancel()- req, err := http.NewRequestWithContext(reqCtx, http.MethodGet, healthURL, nil)- if err != nil {- if customApiUrl == nil {- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))- }- return connectionFailure("offline", errors.WrapIf(err, "failed to create request"))- }- resp, err := s.httpClient.Do(req)- if err != nil {- if customApiUrl == nil {- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))- }- return connectionFailure("offline", errors.WrapIf(err, "connection failed"))- }- defer func() { _ = resp.Body.Close() }()-- if resp.StatusCode == http.StatusOK {- if customApiUrl == nil {- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOnline))- }- return "online", nil- }-- if customApiUrl == nil {- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusError))- }- return connectionFailure("error", errors.Errorf("unexpected status code: %d", resp.StatusCode))-}--// testEdgeConnection tests connection to an edge agent via its tunnel-func (s *EnvironmentService) testEdgeConnection(ctx context.Context, id string) (string, error) {- if !edge.HasActiveTunnel(id) {- if _, ok := edge.RequestTunnelAndWait(ctx, id, edge.DefaultTunnelDemandTTL, edge.DefaultTunnelAcquireTimeout()).Get(); !ok {- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))- return "offline", errors.New("edge agent is not connected")- }- }-- reqCtx, cancel := context.WithTimeout(ctx, 10*time.Second)- defer cancel()-- statusCode, _, err := edge.DoRequest(reqCtx, id, http.MethodGet, "/api/health", nil)- if err != nil {- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))- return "offline", errors.WrapIf(err, "health check via tunnel failed")- }-- if statusCode == http.StatusOK {- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOnline))- return "online", nil- }-- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusError))- return "error", errors.Errorf("unexpected status code: %d", statusCode)-}--func (s *EnvironmentService) testLocalDockerConnection(ctx context.Context, id string) (string, error) {- // Test local Docker socket by pinging Docker- reqCtx, cancel := context.WithTimeout(ctx, 5*time.Second)- defer cancel()-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))- return "offline", errors.WrapIf(err, "failed to connect to Docker")- }-- _, err = dockerClient.Ping(reqCtx, client.PingOptions{})- if err != nil {- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOffline))- return "offline", errors.WrapIf(err, "docker ping failed")- }-- _ = s.updateEnvironmentStatusInternal(ctx, id, string(models.EnvironmentStatusOnline))- return "online", nil-}--func (s *EnvironmentService) updateEnvironmentStatusInternal(ctx context.Context, id, status string) error {- var currentEnv models.Environment- if err := s.db.WithContext(ctx).Select("status", "is_edge").Where("id = ?", id).First(¤tEnv).Error; err != nil {- return errors.WrapIf(err, "failed to check environment status")- }-- if currentEnv.Status == string(models.EnvironmentStatusPending) {- // Edge envs must complete pairing via the agent's outbound tunnel — manager- // can't dial them directly, so a manager-side reachability check means nothing.- // Direct envs are reachable from the manager, so a successful health check IS- // the pairing signal. Don't promote on offline/error ticks though, or a transient- // blip during initial setup would flip the env out of pending.- if currentEnv.IsEdge || status != string(models.EnvironmentStatusOnline) {- slog.DebugContext(ctx, "skipping status update for pending environment", "environment_id", id)- return nil- }- slog.InfoContext(ctx, "promoted pending direct environment to online via reachability check", "environment_id", id)- }-- now := time.Now()- updates := map[string]any{- "status": status,- "last_seen": &now,- "updated_at": &now,- }- if err := s.db.WithContext(ctx).Model(&models.Environment{}).Where("id = ?", id).Updates(updates).Error; err != nil {- return errors.WrapIf(err, "failed to update environment status")- }- // Direct environments have no tunnel callback, so this health-check write is- // the only moment their liveness changes.- s.NotifyRuntimeStateChanged()- return nil-}--func (s *EnvironmentService) UpdateEnvironmentHeartbeat(ctx context.Context, id string) error {- now := time.Now()-- // Use Exec with raw SQL for better performance- // Only update if last_seen is NULL or older than 30 seconds to reduce write frequency- result := s.db.WithContext(ctx).Exec(`- UPDATE environments- SET last_seen = ?, status = ?, updated_at = ?- WHERE id = ?- AND (last_seen IS NULL OR last_seen < ?)- `, new(now), string(models.EnvironmentStatusOnline), new(now), id, now.Add(-30*time.Second))-- if result.Error != nil {- return errors.WrapIf(result.Error, "failed to update environment heartbeat")- }-- // The 30s throttle above doubles as the notify throttle: a no-op heartbeat- // changed nothing worth waking a stream for.- if result.RowsAffected > 0 {- s.NotifyRuntimeStateChanged()- }-- return nil-}--// UpdateEnvironmentConnectionState updates runtime connectivity status without creating-// a generic "environment updated" event. This is used for edge tunnel connect/disconnect.-func (s *EnvironmentService) UpdateEnvironmentConnectionState(ctx context.Context, id string, connected bool) error {- now := time.Now()-- updates := map[string]any{- "updated_at": &now,- }- if connected {- updates["status"] = string(models.EnvironmentStatusOnline)- updates["last_seen"] = &now- // Remember the tunnel transport so the UI can keep showing it after- // the tunnel drops or while the agent is poll-only.- if state, ok := edge.GetTunnelRuntimeState(id).Get(); ok && state.Transport != "" {- updates["last_edge_transport"] = state.Transport- }- } else {- updates["status"] = string(models.EnvironmentStatusOffline)- }-- if err := s.db.WithContext(ctx).Model(&models.Environment{}).Where("id = ?", id).Updates(updates).Error; err != nil {- return errors.WrapIf(err, "failed to update environment connection state")- }-- s.NotifyRuntimeStateChanged()-- return nil-}--// ApplyEnvironmentRuntimeState normalizes edge environment runtime status using-// in-memory tunnel and poll registries without mutating persisted state.-func ApplyEnvironmentRuntimeState(env *environment.Environment) {- if env == nil || !env.IsEdge {- return- }-- connected := false- env.Connected = &connected- env.ConnectedAt = nil- env.LastHeartbeat = nil- env.LastPollAt = nil- env.EdgeTransport = nil- env.EdgeSecurityMode = nil- env.EdgeSessionID = nil- env.EdgeAgentInstance = nil- env.EdgeCapabilities = nil-- if pollState, ok := edge.GetPollRuntimeRegistry().Get(env.ID, time.Now()).Get(); ok {- env.LastPollAt = pollState.LastPollAt- }-- if runtimeState, ok := edge.GetTunnelRuntimeState(env.ID).Get(); ok {- connected = true- env.Connected = &connected- env.Status = string(models.EnvironmentStatusOnline)- env.ConnectedAt = runtimeState.ConnectedAt- env.LastHeartbeat = runtimeState.LastHeartbeat- if runtimeState.SecurityMode != "" {- env.EdgeSecurityMode = &runtimeState.SecurityMode- }- if runtimeState.SessionID != "" {- env.EdgeSessionID = &runtimeState.SessionID- }- if runtimeState.AgentInstance != "" {- env.EdgeAgentInstance = &runtimeState.AgentInstance- }- if len(runtimeState.Capabilities) > 0 {- env.EdgeCapabilities = append([]string(nil), runtimeState.Capabilities...)- }- if transport, ok := edge.GetActiveTunnelTransport(env.ID).Get(); ok {- env.EdgeTransport = &transport- } else if runtimeState.Transport != "" {- env.EdgeTransport = &runtimeState.Transport- }- return- }-- if env.LastPollAt != nil {- env.Status = string(models.EnvironmentStatusStandby)- return- }-- if env.Status != string(models.EnvironmentStatusPending) {- env.Status = string(models.EnvironmentStatusOffline)- }-}--// Environment liveness lives in process-local registries (the edge tunnel-// registry and the poll registry), so nothing in the database changes when an-// agent connects or drops. Open status streams therefore need a nudge rather-// than a payload: they re-derive every environment they cover on wake, which is-// why a bare signal is enough and why a dropped signal is harmless — the-// watcher has one pending already and will read current state when it runs.--// SubscribeRuntimeChanges returns a channel that receives a coalesced wake-up-// whenever environment liveness may have changed, plus a function to release-// it. The channel is never closed; callers select on it alongside their own-// context.-func (s *EnvironmentService) SubscribeRuntimeChanges() (<-chan struct{}, func()) {- s.runtimeWatchMu.Lock()- defer s.runtimeWatchMu.Unlock()-- if s.runtimeWatchers == nil {- s.runtimeWatchers = make(map[int]chan struct{})- }-- s.runtimeWatcherSeq++- id := s.runtimeWatcherSeq- // Capacity 1: a second signal arriving before the watcher wakes is- // redundant, because the watcher reads live state rather than a queue.- ch := make(chan struct{}, 1)- s.runtimeWatchers[id] = ch-- return ch, func() {- s.runtimeWatchMu.Lock()- defer s.runtimeWatchMu.Unlock()- delete(s.runtimeWatchers, id)- }-}--// NotifyRuntimeStateChanged wakes every runtime watcher. It never blocks, so it-// is safe to call from connection callbacks on the tunnel hot path.-func (s *EnvironmentService) NotifyRuntimeStateChanged() {- if s == nil {- return- }-- s.runtimeWatchMu.Lock()- defer s.runtimeWatchMu.Unlock()-- for _, ch := range s.runtimeWatchers {- select {- case ch <- struct{}{}:- default:- }- }-}--// ReconcileEdgeStatusesOnStartup resets edge environments to offline when the manager starts.-// Live edge tunnels are process-local runtime state, so persisted "online" flags can be stale-// after a restart until agents reconnect. Pending environments are left untouched.-func (s *EnvironmentService) ReconcileEdgeStatusesOnStartup(ctx context.Context) error {- result := s.db.WithContext(ctx).Model(&models.Environment{}).- Where("is_edge = ?", true).- Where("status <> ?", string(models.EnvironmentStatusPending)).- Where("status <> ?", string(models.EnvironmentStatusOffline)).- Updates(map[string]any{- "status": string(models.EnvironmentStatusOffline),- "updated_at": new(time.Now()),- })- if result.Error != nil {- return errors.WrapIf(result.Error, "failed to reconcile edge environment statuses")- }-- if result.RowsAffected > 0 {- slog.InfoContext(ctx, "Reconciled stale edge environment statuses on startup", "count", result.RowsAffected)- }-- return nil-}--func (s *EnvironmentService) createEnvironmentEvent(ctx context.Context, envID, envName string, eventType models.EventType, title, description string, severity models.EventSeverity, userID, username *string) {- if s == nil || s.eventService == nil {- return- }-- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{- Type: eventType,- Severity: severity,- Title: title,- Description: description,- ResourceType: new("environment"),- ResourceID: new(envID),- ResourceName: new(envName),- UserID: userID,- Username: username,- EnvironmentID: new(envID),- })-}--func (s *EnvironmentService) RegenerateEnvironmentApiKey(ctx context.Context, envID string, newApiKeyID string, apiKey string, userID, username string, envName string) error {- // Trim once at the boundary so the value persisted, the value cached,- // and the value returned by callers (which already TrimSpace before- // returning) all stay byte-identical. Any divergence here would surface- // as a 401 "invalid agent token" because lookup is direct equality.- apiKey = strings.TrimSpace(apiKey)-- updates := map[string]any{- "api_key_id": newApiKeyID,- "access_token": apiKey,- "status": string(models.EnvironmentStatusPending),- "last_seen": nil, // Clear last seen time- }-- result := s.db.WithContext(ctx).Model(&models.Environment{}).Where("id = ?", envID).Updates(updates)- if result.Error != nil {- return errors.WrapIf(result.Error, "failed to update environment with new API key")- }- if result.RowsAffected == 0 {- // A zero-row update would otherwise report a successful rotation while- // the new key was never linked to anything.- return errors.New("environment not found")- }-- s.syncEnvironmentTokenCacheInternal(envID, apiKey)- now := time.Now()- s.updateRemoteEnvironmentSnapshotInternal(envID, func(environment *models.Environment) {- environment.ApiKeyID = &newApiKeyID- environment.AccessToken = &apiKey- environment.Status = string(models.EnvironmentStatusPending)- environment.LastSeen = nil- environment.UpdatedAt = &now- })-- // Create event log in background- go s.createEnvironmentEvent(context.WithoutCancel(ctx), envID, envName, models.EventTypeEnvironmentApiKeyRegenerated, "API Key Regenerated", "Environment API key was regenerated and status set to pending", models.EventSeverityInfo, new(userID), new(username))-- return nil-}--func (s *EnvironmentService) GetDB() *database.DB {- return s.db-}--func (s *EnvironmentService) ResolveEnvironmentByAccessToken(ctx context.Context, token string) (*models.Environment, error) {- token = strings.TrimSpace(token)- if token == "" {- return nil, ErrEnvironmentAccessTokenRequired- }-- var env models.Environment- if err := s.db.WithContext(ctx).- Where("access_token = ?", token).- First(&env).Error; err != nil {- if errors.Is(err, gorm.ErrRecordNotFound) {- return nil, ErrInvalidEnvironmentAccessToken- }- return nil, errors.WrapIf(err, "failed to resolve environment by access token")- }-- return &env, nil-}--func (s *EnvironmentService) GetEnabledRegistryCredentials(ctx context.Context) ([]containerregistry.Credential, error) {- var registries []models.ContainerRegistry- if err := s.db.WithContext(ctx).Where("enabled = ?", true).Find(®istries).Error; err != nil {- return nil, errors.WrapIf(err, "failed to get enabled container registries")- }-- var creds []containerregistry.Credential- for _, reg := range registries {- if !reg.Enabled || reg.Username == "" || reg.Token == "" {- continue- }-- decryptedToken, err := crypto.Decrypt(reg.Token)- if err != nil {- slog.WarnContext(ctx, "Failed to decrypt registry token", "registryURL", reg.URL, "error", err.Error())- continue- }-- creds = append(creds, containerregistry.Credential{- URL: reg.URL,- Username: reg.Username,- Token: decryptedToken,- Enabled: reg.Enabled,- })- }-- return creds, nil-}--// DeploymentSnippets contains deployment configuration snippets for an environment.-type DeploymentSnippets struct {- DockerRun string- DockerCompose string- MTLS *DeploymentSnippetMTLS-}--type DeploymentSnippetFile struct {- Name string- Content string- ContainerPath string- Permissions string-}--type DeploymentSnippetMTLS struct {- DockerRun string- DockerCompose string- Files []DeploymentSnippetFile- HostDirHint string-}--const (- deploymentSnippetsDataPath = "/app/data"- deploymentSnippetsMTLSPath = "/app/data/edge-mtls-agent"-)--// GenerateDeploymentSnippets generates Docker deployment snippets for an environment.-func (s *EnvironmentService) GenerateDeploymentSnippets(ctx context.Context, envID string, envAddress string, apiKey string) (*DeploymentSnippets, error) {- managerURL := strings.TrimRight(envAddress, "/")-- dockerRun := strings.Join([]string{- "docker run -d \\",- " --name arcane-agent \\",- " --restart unless-stopped \\",- " -e AGENT_MODE=true \\",- " -e EDGE_TRANSPORT=poll \\",- fmt.Sprintf(" -e AGENT_TOKEN=%s \\", apiKey),- fmt.Sprintf(" -e MANAGER_API_URL=%s \\", managerURL),- " -p 3553:3553 \\",- " -v /var/run/docker.sock:/var/run/docker.sock \\",- fmt.Sprintf(" -v arcane-data:%s \\", deploymentSnippetsDataPath),- " ghcr.io/getarcaneapp/agent:latest",- }, "\n")-- dockerCompose := strings.Join([]string{- "services:",- " arcane-agent:",- " image: ghcr.io/getarcaneapp/agent:latest",- " container_name: arcane-agent",- " restart: unless-stopped",- " environment:",- " - AGENT_MODE=true",- " - EDGE_TRANSPORT=poll",- " - AGENT_TOKEN=" + apiKey,- " - MANAGER_API_URL=" + managerURL,- " ports:",- " - \"3553:3553\"",- " volumes:",- " - /var/run/docker.sock:/var/run/docker.sock",- " - arcane-data:" + deploymentSnippetsDataPath,- "",- "volumes:",- " arcane-data:",- }, "\n")-- return &DeploymentSnippets{- DockerRun: dockerRun,- DockerCompose: dockerCompose,- }, nil-}--// GenerateEdgeDeploymentSnippets generates Docker deployment snippets for an edge agent.-// Edge agents connect outbound to the manager and don't require exposed ports.-func (s *EnvironmentService) GenerateEdgeDeploymentSnippets(ctx context.Context, envID string, managerURL string, apiKey string, edgeCfg *edge.Config) (*DeploymentSnippets, error) {- managerURL = strings.TrimRight(managerURL, "/")-- dockerRun := strings.Join([]string{- "docker run -d \\",- " --name arcane-edge-agent \\",- " --restart unless-stopped \\",- " -e EDGE_AGENT=true \\",- " -e EDGE_TRANSPORT=poll \\",- fmt.Sprintf(" -e AGENT_TOKEN=%s \\", apiKey),- fmt.Sprintf(" -e MANAGER_API_URL=%s \\", managerURL),- " -v /var/run/docker.sock:/var/run/docker.sock \\",- fmt.Sprintf(" -v arcane-data:%s \\", deploymentSnippetsDataPath),- " ghcr.io/getarcaneapp/agent:latest",- }, "\n")-- dockerCompose := strings.Join([]string{- "# Edge agent - connects outbound, no exposed ports required",- "services:",- " arcane-edge-agent:",- " image: ghcr.io/getarcaneapp/agent:latest",- " container_name: arcane-edge-agent",- " restart: unless-stopped",- " environment:",- " - EDGE_AGENT=true",- " - EDGE_TRANSPORT=poll",- " - AGENT_TOKEN=" + apiKey,- " - MANAGER_API_URL=" + managerURL,- " volumes:",- " - /var/run/docker.sock:/var/run/docker.sock",- " - arcane-data:" + deploymentSnippetsDataPath,- "",- "volumes:",- " arcane-data:",- }, "\n")-- snippets := &DeploymentSnippets{- DockerRun: dockerRun,- DockerCompose: dockerCompose,- }-- envName := ""- if s != nil && s.db != nil {- if env, getErr := s.GetEnvironmentByID(ctx, envID); getErr == nil && env != nil {- envName = env.Name- }- }- if edgeCfg != nil && strings.TrimSpace(edgeCfg.AppURL) == "" {- edgeCfg.AppURL = managerURL- }-- generatedAssets, err := edge.GenerateManagerClientMTLSAssetsWithContext(ctx, edgeCfg, envID, envName)- if err != nil {- slog.WarnContext(ctx, "Failed to generate edge mTLS assets; returning basic snippets only", "environment_id", envID, "error", err)- return snippets, nil- }- if generatedAssets == nil {- return snippets, nil- }- s.logGeneratedMTLSEventsInternal(ctx, envID, envName, generatedAssets)-- snippets.MTLS = buildMTLSDeploymentSnippetInternal(managerURL, apiKey, generatedAssets)- return snippets, nil-}--func (s *EnvironmentService) logGeneratedMTLSEventsInternal(ctx context.Context, envID string, envName string, assets *edge.GeneratedMTLSAssets) {- if s == nil || s.eventService == nil || assets == nil {- return- }- if assets.CAGenerated {- if _, err := s.eventService.CreateEvent(ctx, CreateEventRequest{- Type: models.EventTypeEnvironmentMTLSCAGenerated,- Severity: models.EventSeverityInfo,- Title: "Edge mTLS CA generated",- Description: "Arcane generated a new edge mTLS certificate authority",- Metadata: models.JSON{"kind": "ca"},- }); err != nil {- slog.WarnContext(ctx, "Failed to create edge mTLS CA generation event", "error", err)- }- }- if assets.CertIssued {- envIDCopy := envID- if _, err := s.eventService.CreateEvent(ctx, CreateEventRequest{- Type: models.EventTypeEnvironmentMTLSCertIssued,- Severity: models.EventSeverityInfo,- Title: "Edge mTLS certificate issued",- Description: fmt.Sprintf("Arcane issued an edge mTLS client certificate for environment '%s'", envName),- ResourceType: new("environment"),- ResourceID: &envIDCopy,- ResourceName: new(envName),- EnvironmentID: &envIDCopy,- Metadata: models.JSON{"kind": "client"},- }); err != nil {- slog.WarnContext(ctx, "Failed to create edge mTLS certificate issuance event", "environment_id", envID, "error", err)- }- }-}--func buildMTLSDeploymentSnippetInternal(managerURL string, apiKey string, generatedAssets *edge.GeneratedMTLSAssets) *DeploymentSnippetMTLS {- if generatedAssets == nil {- return nil- }-- mtlsDockerRun := strings.Join([]string{- "docker run -d \\",- " --name arcane-edge-agent \\",- " --restart unless-stopped \\",- " -e EDGE_AGENT=true \\",- " -e EDGE_TRANSPORT=poll \\",- " -e EDGE_MTLS_MODE=required \\",- fmt.Sprintf(" -e EDGE_MTLS_ASSETS_DIR=%s \\", deploymentSnippetsMTLSPath),- fmt.Sprintf(" -e AGENT_TOKEN=%s \\", apiKey),- fmt.Sprintf(" -e MANAGER_API_URL=%s \\", managerURL),- " -v /var/run/docker.sock:/var/run/docker.sock \\",- fmt.Sprintf(" -v arcane-data:%s \\", deploymentSnippetsDataPath),- " ghcr.io/getarcaneapp/agent:latest",- }, "\n")-- mtlsDockerCompose := strings.Join([]string{- "# Edge agent with automatic mTLS enrollment",- "services:",- " arcane-edge-agent:",- " image: ghcr.io/getarcaneapp/agent:latest",- " container_name: arcane-edge-agent",- " restart: unless-stopped",- " environment:",- " - EDGE_AGENT=true",- " - EDGE_TRANSPORT=poll",- " - EDGE_MTLS_MODE=required",- " - EDGE_MTLS_ASSETS_DIR=" + deploymentSnippetsMTLSPath,- " - AGENT_TOKEN=" + apiKey,- " - MANAGER_API_URL=" + managerURL,- " volumes:",- " - /var/run/docker.sock:/var/run/docker.sock",- " - arcane-data:" + deploymentSnippetsDataPath,- "",- "volumes:",- " arcane-data:",- }, "\n")-- files := make([]DeploymentSnippetFile, 0, len(generatedAssets.Files))- for _, file := range generatedAssets.Files {- files = append(files, DeploymentSnippetFile{- Name: file.Name,- Content: file.Content,- ContainerPath: file.ContainerPath,- Permissions: file.Permissions,- })- }-- return &DeploymentSnippetMTLS{- DockerRun: mtlsDockerRun,- DockerCompose: mtlsDockerCompose,- Files: files,- HostDirHint: strings.TrimSpace(generatedAssets.HostDirHint),- }-}--type remoteEnvironmentTargetInternal struct {- ID string- Name string- IsEdge bool- AccessToken *string- TargetURL string-}--func (s *EnvironmentService) resolveRemoteEnvironmentTargetInternal(ctx context.Context, envID string) (*remoteEnvironmentTargetInternal, error) {- envRecord, err := s.GetEnvironmentByID(ctx, envID)- if err != nil {- return nil, errors.WrapIf(err, "failed to get environment")- }-- return s.remoteEnvironmentTargetFromModelInternal(*envRecord)-}--func (s *EnvironmentService) remoteEnvironmentTargetFromModelInternal(environment models.Environment) (*remoteEnvironmentTargetInternal, error) {- if environment.ID == "0" {- return nil, errors.New("cannot proxy request to local environment")- }-- targetURL := strings.TrimRight(environment.ApiUrl, "/")- if !environment.IsEdge {- validatedTargetURL, err := normalizeEnvironmentBaseURLInternal(environment.ApiUrl)- if err != nil {- return nil, errors.WrapIf(err, "invalid environment API URL")- }- targetURL = validatedTargetURL- }-- return &remoteEnvironmentTargetInternal{- ID: environment.ID,- Name: environment.Name,- IsEdge: environment.IsEdge,- AccessToken: environment.AccessToken,- TargetURL: targetURL,- }, nil-}--func normalizeEnvironmentBaseURLInternal(apiURL string) (string, error) {- parsed, err := httputils.ValidateOutboundHTTPURL(apiURL)- if err != nil {- return "", err- }-- parsed.RawQuery = ""- parsed.Fragment = ""- parsed.Path = strings.TrimRight(parsed.Path, "/")- return parsed.String(), nil-}--func buildEnvironmentEndpointURLInternal(apiURL, endpointPath string) (string, error) {- baseURL, err := normalizeEnvironmentBaseURLInternal(apiURL)- if err != nil {- return "", err- }-- return strings.TrimRight(baseURL, "/") + endpointPath, nil-}--func (s *EnvironmentService) getProxyRequestContextInternal(ctx context.Context) (context.Context, context.CancelFunc) {- if s != nil && s.settingsService != nil {- settings := s.settingsService.GetSettingsConfig()- return context.WithTimeout(ctx, timeouts.GetDuration(settings.ProxyRequestTimeout.AsInt(), timeouts.DefaultProxyRequest))- }-- return context.WithTimeout(ctx, timeouts.DefaultProxyRequest)-}--func (s *EnvironmentService) buildRemoteRequestInternal(- target *remoteEnvironmentTargetInternal,- method string,- path string,- body []byte,- headers map[string]string,-) (remenv.Request, error) {- if target == nil {- return remenv.Request{}, errors.New("remote environment target is required")- }-- requestHeaders := make(map[string]string, len(headers)+2)- maps.Copy(requestHeaders, headers)- if len(body) > 0 && method != http.MethodGet && requestHeaders["Content-Type"] == "" {- requestHeaders["Content-Type"] = "application/json"- }- remenv.ApplyAgentTokenHeaderMap(requestHeaders, target.AccessToken)-- return remenv.Request{- EnvironmentID: target.ID,- IsEdge: target.IsEdge,- Method: method,- URL: target.TargetURL + path,- Path: path,- Headers: requestHeaders,- Body: body,- }, nil-}--func (s *EnvironmentService) ExecuteRemoteRequest(ctx context.Context, envID string, method string, path string, body []byte) (*remenv.Response, error) {- target, err := s.resolveRemoteEnvironmentTargetInternal(ctx, envID)- if err != nil {- return nil, err- }-- return s.executeRemoteRequestForTargetInternal(ctx, target, method, path, body)-}--func (s *EnvironmentService) executeRemoteRequestForTargetInternal(- ctx context.Context,- target *remoteEnvironmentTargetInternal,- method string,- path string,- body []byte,-) (*remenv.Response, error) {- // Forward the activity batch ID so bulk actions proxied to a remote- // environment group the same way they do locally.- var headers map[string]string- if batchID := pkgutils.ActivityBatchIDFromContext(ctx); batchID != "" {- headers = map[string]string{pkgutils.HeaderActivityBatchID: batchID}- }- request, err := s.buildRemoteRequestInternal(target, method, path, body, headers)- if err != nil {- return nil, err- }-- resp, err := s.remoteClient.Do(ctx, request)- if err != nil {- return nil, errors.WrapIff(err, "failed to send request to environment %s", target.Name)- }-- return resp, nil-}--func (s *EnvironmentService) ProxyJSONRequest(ctx context.Context, envID string, method string, path string, body []byte, out any) error {- proxyCtx, cancel := s.getProxyRequestContextInternal(ctx)- defer cancel()-- target, err := s.resolveRemoteEnvironmentTargetInternal(proxyCtx, envID)- if err != nil {- return err- }-- return s.proxyJSONRequestForTargetInternal(proxyCtx, target, method, path, body, out)-}--// ProxyJSONRequestForEnvironment sends a JSON request using an already-loaded-// environment row, avoiding an extra environment lookup on hot stream paths.-func (s *EnvironmentService) ProxyJSONRequestForEnvironment(ctx context.Context, environment models.Environment, method string, path string, body []byte, out any) error {- proxyCtx, cancel := s.getProxyRequestContextInternal(ctx)- defer cancel()-- target, err := s.remoteEnvironmentTargetFromModelInternal(environment)- if err != nil {- return err- }-- return s.proxyJSONRequestForTargetInternal(proxyCtx, target, method, path, body, out)-}--func (s *EnvironmentService) proxyJSONRequestForTargetInternal(- ctx context.Context,- target *remoteEnvironmentTargetInternal,- method string,- path string,- body []byte,- out any,-) error {- resp, err := s.executeRemoteRequestForTargetInternal(ctx, target, method, path, body)- if err != nil {- return err- }- if err := resp.RequireSuccess(); err != nil {- return err- }- if err := resp.DecodeJSON(out); err != nil {- return err- }-- return nil-}--func ensureRemoteEnvironmentTunnelAvailableInternal(ctx context.Context, envID string) error {- if edge.HasActiveTunnel(envID) {- return nil- }-- if _, ok := edge.RequestTunnelAndWait(ctx, envID, edge.DefaultTunnelDemandTTL, edge.DefaultTunnelAcquireTimeout()).Get(); ok {- return nil- }-- return errors.New("edge agent is not connected")-}--func doRemoteEnvironmentTunnelRequestInternal(- ctx context.Context,- envID string,- method string,- path string,- headers map[string]string,- body []byte,-) (*remenv.Response, error) {- tunnel, ok := edge.GetRegistry().Get(envID).Get()- if !ok {- return nil, errors.Errorf("no active tunnel for environment %s", envID)- }- if tunnel.Conn.IsClosed() {- return nil, errors.Errorf("tunnel for environment %s is closed", envID)- }-- statusCode, respHeaders, respBody, err := edge.ProxyRequest(ctx, tunnel, method, path, "", headers, body)- if err != nil {- return nil, errors.WrapIf(err, "tunnel request failed")- }-- return &remenv.Response{- StatusCode: statusCode,- Body: respBody,- Headers: respHeaders,- }, nil-}--// SyncRegistriesToEnvironment syncs all registries from this manager to a remote environment-func (s *EnvironmentService) SyncRegistriesToEnvironment(ctx context.Context, environmentID string) error {- target, err := s.resolveRemoteEnvironmentTargetInternal(ctx, environmentID)- if err != nil {- return err- }-- slog.InfoContext(ctx, "Starting registry sync to environment", "environmentID", environmentID, "environmentName", target.Name, "apiUrl", target.TargetURL)-- // Get all registries from this manager- var registries []models.ContainerRegistry- if err := s.db.WithContext(ctx).Find(®istries).Error; err != nil {- return errors.WrapIf(err, "failed to get registries")- }-- slog.InfoContext(ctx, "Found registries to sync", "count", len(registries))-- // Prepare sync items with decrypted credentials- syncItems := make([]containerregistry.Sync, 0, len(registries))- for _, reg := range registries {- registryType, typeErr := normalizeRegistryTypeInternal(reg.RegistryType)- if typeErr != nil {- return errors.WrapIff(typeErr, "normalize registry type for sync %s", reg.ID)- }-- syncItem := containerregistry.Sync{- ID: reg.ID,- URL: reg.URL,- Description: reg.Description,- Insecure: reg.Insecure,- Enabled: reg.Enabled,- RegistryType: registryType,- RepositoryNames: reg.RepositoryNames,- CreatedAt: reg.CreatedAt,- UpdatedAt: reg.UpdatedAt,- }-- if registryType == registryTypeECR {- decryptedSecret, err := crypto.Decrypt(reg.AWSSecretAccessKey)- if err != nil {- slog.WarnContext(ctx, "Failed to decrypt ECR secret for sync", "registryID", reg.ID, "registryURL", reg.URL, "error", err.Error())- continue- }-- syncItem.AWSAccessKeyID = reg.AWSAccessKeyID- syncItem.AWSSecretAccessKey = decryptedSecret- syncItem.AWSRegion = reg.AWSRegion- } else {- decryptedToken, err := crypto.Decrypt(reg.Token)- if err != nil {- slog.WarnContext(ctx, "Failed to decrypt registry token for sync", "registryID", reg.ID, "registryURL", reg.URL, "error", err.Error())- continue- }-- syncItem.Username = reg.Username- syncItem.Token = decryptedToken- }-- syncItems = append(syncItems, syncItem)- }-- // Prepare the sync request- syncReq := containerregistry.SyncRequest{- Registries: syncItems,- }-- // Marshal the request- reqBody, err := json.Marshal(syncReq)- if err != nil {- return errors.WrapIf(err, "failed to marshal sync request")- }-- reqCtx, cancel := context.WithTimeout(ctx, 30*time.Second)- defer cancel()-- slog.InfoContext(ctx, "Sending sync request to agent", "url", target.TargetURL+"/api/container-registries/sync", "registryCount", len(syncItems), "isEdge", target.IsEdge)-- var result struct {- Success bool `json:"success"`- Data struct {- Message string `json:"message"`- } `json:"data"`- }- if err := s.proxyJSONRequestForTargetInternal(reqCtx, target, http.MethodPost, "/api/container-registries/sync", reqBody, &result); err != nil {- return errors.WrapIf(err, "failed to send sync request")- }-- if !result.Success {- return errors.Errorf("sync failed: %s", result.Data.Message)- }-- slog.InfoContext(ctx, "Successfully synced registries to environment", "environmentID", environmentID, "environmentName", target.Name)-- return nil-}--// SyncRepositoriesToEnvironment syncs all git repositories from this manager to a remote environment-func (s *EnvironmentService) SyncRepositoriesToEnvironment(ctx context.Context, environmentID string) error {- target, err := s.resolveRemoteEnvironmentTargetInternal(ctx, environmentID)- if err != nil {- return err- }-- slog.InfoContext(ctx, "Starting git repository sync to environment", "environmentID", environmentID, "environmentName", target.Name, "apiUrl", target.TargetURL)-- // Get all git repositories from this manager- var repositories []models.GitRepository- if err := s.db.WithContext(ctx).Find(&repositories).Error; err != nil {- return errors.WrapIf(err, "failed to get git repositories")- }-- slog.InfoContext(ctx, "Found git repositories to sync", "count", len(repositories))-- // Prepare sync items with decrypted credentials- syncItems := make([]gitops.RepositorySync, 0, len(repositories))- for _, repo := range repositories {- item := gitops.RepositorySync{- ID: repo.ID,- Name: repo.Name,- URL: repo.URL,- AuthType: repo.AuthType,- Username: repo.Username,- Description: repo.Description,- Enabled: repo.Enabled,- CreatedAt: repo.CreatedAt,- }- if repo.UpdatedAt != nil {- item.UpdatedAt = *repo.UpdatedAt- }-- // Decrypt token if present- if repo.Token != "" {- decryptedToken, err := crypto.Decrypt(repo.Token)- if err != nil {- slog.WarnContext(ctx, "Failed to decrypt repository token for sync", "repositoryID", repo.ID, "repositoryName", repo.Name, "error", err.Error())- continue- }- item.Token = decryptedToken- }-- // Decrypt SSH key if present- if repo.SSHKey != "" {- decryptedSSHKey, err := crypto.Decrypt(repo.SSHKey)- if err != nil {- slog.WarnContext(ctx, "Failed to decrypt repository SSH key for sync", "repositoryID", repo.ID, "repositoryName", repo.Name, "error", err.Error())- continue- }- item.SSHKey = decryptedSSHKey- }-- syncItems = append(syncItems, item)- }-- // Prepare the sync request- syncReq := gitops.RepositorySyncRequest{- Repositories: syncItems,- }-- // Marshal the request- reqBody, err := json.Marshal(syncReq)- if err != nil {- return errors.WrapIf(err, "failed to marshal sync request")- }-- reqCtx, cancel := context.WithTimeout(ctx, 30*time.Second)- defer cancel()-- slog.InfoContext(ctx, "Sending git repository sync request to agent", "url", target.TargetURL+"/api/git-repositories/sync", "repositoryCount", len(syncItems), "isEdge", target.IsEdge)-- var result struct {- Success bool `json:"success"`- Data struct {- Message string `json:"message"`- } `json:"data"`- }- if err := s.proxyJSONRequestForTargetInternal(reqCtx, target, http.MethodPost, "/api/git-repositories/sync", reqBody, &result); err != nil {- return errors.WrapIf(err, "failed to send sync request")- }-- if !result.Success {- return errors.Errorf("sync failed: %s", result.Data.Message)- }-- slog.InfoContext(ctx, "Successfully synced git repositories to environment", "environmentID", environmentID, "environmentName", target.Name)-- return nil-}--// ProxyRequest sends a request to a remote environment's API.-func (s *EnvironmentService) ProxyRequest(ctx context.Context, envID string, method string, path string, body []byte) ([]byte, int, error) {- proxyCtx, cancel := s.getProxyRequestContextInternal(ctx)- defer cancel()-- resp, err := s.ExecuteRemoteRequest(proxyCtx, envID, method, path, body)- if err != nil {- return nil, 0, err- }-- return resp.Body, resp.StatusCode, nil-}diff --git a/backend/internal/services/playwright_service.go b/backend/internal/services/playwright_service.godeleted file mode 100644index 6e9bd9a0e..000000000--- a/backend/internal/services/playwright_service.go+++ /dev/null@@ -1,155 +0,0 @@-//go:build playwright--package services--import (- "context"- "fmt"- "log/slog"- "strings"-- "emperror.dev/errors"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/pkg/authz"- "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"- "github.com/getarcaneapp/arcane/types/v2/apikey"- "github.com/google/uuid"- "gorm.io/gorm"-)--type PlaywrightService struct {- apiKeyService *ApiKeyService- userService *UserService- federatedCredentialService *FederatedCredentialService-}--func NewPlaywrightService(apiKeyService *ApiKeyService, userService *UserService, federatedCredentialService *FederatedCredentialService) *PlaywrightService {- return &PlaywrightService{- apiKeyService: apiKeyService,- userService: userService,- federatedCredentialService: federatedCredentialService,- }-}--func (ps *PlaywrightService) CreateTestApiKeys(ctx context.Context, count int) ([]*apikey.ApiKeyCreatedDto, error) {- slog.Info("Playwright: Creating test API keys", "count", count)-- // Get the arcane user to associate the API keys with- user, err := ps.userService.GetUserByUsername(ctx, "arcane")- if err != nil {- return nil, errors.WrapIf(err, "failed to get arcane user")- }-- // Grant every recognized permission globally so the test key behaves like- // the legacy "admin-everywhere" credential the e2e suite expects. There is- // no request context here, so grant validation runs against a sudo set.- allPerms := authz.AllPermissions()- grants := make([]apikey.PermissionGrant, len(allPerms))- for i, p := range allPerms {- grants[i] = apikey.PermissionGrant{Permission: p}- }-- var createdKeys []*apikey.ApiKeyCreatedDto- for i := 0; i < count; i++ {- description := fmt.Sprintf("Test API key %d for Playwright tests", i+1)- req := apikey.CreateApiKey{- Name: fmt.Sprintf("test-api-key-%d", i+1),- Description: &description,- Permissions: grants,- }-- apiKey, err := ps.apiKeyService.CreateApiKey(ctx, user.ID, authz.SudoPermissionSet(), req)- if err != nil {- return nil, errors.WrapIff(err, "failed to create test API key %d", i+1)- }-- createdKeys = append(createdKeys, apiKey)- }-- slog.Info("Playwright: Test API keys created successfully", "count", len(createdKeys))- return createdKeys, nil-}--func (ps *PlaywrightService) DeleteAllTestApiKeys(ctx context.Context) error {- slog.Info("Playwright: Deleting all test API keys")-- // Get all API keys with test prefix- params := pagination.QueryParams{- SearchQuery: pagination.SearchQuery{- Search: "test-api-key",- },- Params: pagination.Params{- Start: 0,- Limit: 1000,- },- }-- apiKeys, _, err := ps.apiKeyService.ListApiKeys(ctx, params)- if err != nil {- return errors.WrapIf(err, "failed to list API keys")- }-- for _, apiKey := range apiKeys {- if err := ps.apiKeyService.DeleteApiKey(ctx, apiKey.ID); err != nil {- slog.Warn("Failed to delete test API key", "id", apiKey.ID, "error", err)- }- }-- slog.Info("Playwright: Test API keys deleted", "count", len(apiKeys))- return nil-}--func (ps *PlaywrightService) CreateTestFederatedCredential(ctx context.Context, issuerURL string, audiences []string, subject string, roleID string, tokenTTLSeconds int) (string, error) {- if ps.federatedCredentialService == nil || ps.federatedCredentialService.db == nil {- return "", errors.Errorf("federated credential service is not available")- }- if strings.TrimSpace(issuerURL) == "" || strings.TrimSpace(subject) == "" || strings.TrimSpace(roleID) == "" || len(audiences) == 0 {- return "", errors.Errorf("issuerUrl, subject, roleId, and audiences are required")- }- if tokenTTLSeconds <= 0 {- tokenTTLSeconds = 600- }-- var credentialID string- err := ps.federatedCredentialService.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {- serviceUser := models.User{- Username: "svc_federated_e2e_" + strings.ReplaceAll(uuid.NewString(), "-", ""),- IsServiceAccount: true,- }- if err := tx.Create(&serviceUser).Error; err != nil {- return errors.WrapIf(err, "failed to create federated e2e user")- }-- credential := models.FederatedCredential{- Name: "Playwright Federated Credential",- Enabled: true,- IssuerURL: strings.TrimRight(strings.TrimSpace(issuerURL), "/"),- Audiences: models.StringSlice(audiences),- SubjectClaim: "sub",- SubjectMatch: strings.TrimSpace(subject),- MatchType: models.FederatedCredentialMatchExact,- RoleID: strings.TrimSpace(roleID),- IdentityUserID: serviceUser.ID,- TokenTTLSeconds: tokenTTLSeconds,- }- if err := tx.Create(&credential).Error; err != nil {- return errors.WrapIf(err, "failed to create federated e2e credential")- }-- assignment := models.UserRoleAssignment{- UserID: serviceUser.ID,- RoleID: credential.RoleID,- Source: models.RoleAssignmentSourceManual,- }- if err := tx.Create(&assignment).Error; err != nil {- return errors.WrapIf(err, "failed to create federated e2e role assignment")- }-- credentialID = credential.ID- return nil- })- if err != nil {- return "", err- }- return credentialID, nil-}diff --git a/backend/internal/services/project_service.go b/backend/internal/services/project_service.godeleted file mode 100644index a93e8d2f6..000000000--- a/backend/internal/services/project_service.go+++ /dev/null@@ -1,5517 +0,0 @@-package services--import (- "bufio"- "context"- "encoding/json/v2"- stderrors "errors"- "fmt"- "io"- "log/slog"- "maps"- "os"- "path"- "path/filepath"- "slices"- "strings"- "sync"- "time"-- "emperror.dev/emperror"- "emperror.dev/errors"-- "github.com/compose-spec/compose-go/v2/dotenv"- "github.com/compose-spec/compose-go/v2/loader"- composetypes "github.com/compose-spec/compose-go/v2/types"- "github.com/docker/compose/v5/pkg/api"- "github.com/getarcaneapp/arcane/backend/v2/internal/common"- "github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/volumes"- "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"- "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"- "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"- "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/iconcatalog"- "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper"- "github.com/getarcaneapp/arcane/types/v2"- "github.com/getarcaneapp/arcane/types/v2/containerregistry"- imagetypes "github.com/getarcaneapp/arcane/types/v2/image"- "github.com/getarcaneapp/arcane/types/v2/project"- "github.com/moby/moby/api/types/container"- dockerregistry "github.com/moby/moby/api/types/registry"- "github.com/moby/moby/client"- "github.com/samber/hot"- "github.com/samber/mo"- buildtypes "go.getarcane.app/builds/types"- "go.getarcane.app/sys/cgroup"- "go.getarcane.app/updater/labels"- "gorm.io/gorm"-)--type ProjectService struct {- db *database.DB- settingsService *SettingsService- eventService *EventService- imageService *ImageService- dockerService *DockerClientService- buildService *BuildService- lifecycleService *LifecycleService- kvService *KVService- containerRegistryService *ContainerRegistryService- config *config.Config- registryCredentialsProvider registryCredentialsProviderInternal-- // syncMu serializes SyncProjectsFromFileSystem: its discovery walk and its- // cleanup pass must not interleave with another run's.- syncMu sync.Mutex-- composeNameCacheMu sync.RWMutex- composeNameToProjID map[string]string- composeCache *hot.HotCache[string, composeCacheEntry]- // metaCache holds per-project icon/URL metadata, keyed by project ID. Deriving- // it costs a full compose load (interpolation plus .env reads) and, for GitOps- // projects, a gitops_syncs lookup — per project, on every list request. Mirrors- // ContainerService.iconMetaCache.- metaCache *hot.HotCache[string, projects.ArcaneComposeMetadata]-}--// projectMetadataEnvInternal carries the inputs ParseArcaneComposeMetadata needs-// beyond the project itself. Resolving them costs a settings clone and a stat-// syscall each, so list paths resolve once and reuse across every project.-type projectMetadataEnvInternal struct {- projectsDirectory string- autoInjectEnv bool-}--// projectMetadataTTL bounds how stale cached compose metadata can be. It matches-// containerIconMetadataTTL so icons resolved from either service agree.-const projectMetadataTTL = 5 * time.Second--type registryCredentialsProviderInternal func(context.Context) ([]containerregistry.Credential, error)--type composeCacheEntry struct {- composePath string- composeMtime time.Time- includeMtimes map[string]time.Time- project *composetypes.Project-}--func NewProjectService(db *database.DB, settingsService *SettingsService, eventService *EventService, imageService *ImageService, dockerService *DockerClientService, buildService *BuildService, lifecycleService *LifecycleService, containerRegistryService *ContainerRegistryService, cfg *config.Config) *ProjectService {- return &ProjectService{- db: db,- settingsService: settingsService,- eventService: eventService,- imageService: imageService,- dockerService: dockerService,- buildService: buildService,- lifecycleService: lifecycleService,- containerRegistryService: containerRegistryService,- config: cfg,- composeCache: hot.NewHotCache[string, composeCacheEntry](hot.LRU, 2048).Build(),- metaCache: hot.NewHotCache[string, projects.ArcaneComposeMetadata](hot.LRU, 1024).- WithTTL(projectMetadataTTL).- WithJanitor().- Build(),- }-}--func (s *ProjectService) WithRegistryCredentialsProvider(provider func(context.Context) ([]containerregistry.Credential, error)) *ProjectService {- if s == nil {- return nil- }- s.registryCredentialsProvider = provider- return s-}--func (s *ProjectService) WithKVService(kvService *KVService) *ProjectService {- if s == nil {- return nil- }- s.kvService = kvService- return s-}--func (s *ProjectService) resolveRegistryCredentialsInternal(ctx context.Context) ([]containerregistry.Credential, error) {- if s == nil || s.registryCredentialsProvider == nil {- return nil, nil- }-- credentials, err := s.registryCredentialsProvider(ctx)- if err != nil {- return nil, errors.WrapIf(err, "get enabled registry credentials")- }-- return credentials, nil-}--func (s *ProjectService) composeRegistryAuthConfigsInternal(ctx context.Context) map[string]dockerregistry.AuthConfig {- if s == nil || s.containerRegistryService == nil {- return nil- }-- authConfigs, err := s.containerRegistryService.GetAllRegistryAuthConfigs(ctx)- if err != nil {- slog.WarnContext(ctx, "failed to load registry auth for compose pulls", "error", err)- return nil- }-- return authConfigs-}--func (s *ProjectService) getProjectsDirectoryInternal(ctx context.Context) (string, error) {- projectsDirSetting := s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects")- projectsDir, err := projects.GetProjectsDirectory(ctx, strings.TrimSpace(projectsDirSetting))- if err != nil {- return "", err- }-- return filepath.Clean(projectsDir), nil-}--func (s *ProjectService) getProjectsDirectoryOrDefaultInternal(ctx context.Context, cfg *models.Settings) string {- projectsDirectory, err := projects.GetProjectsDirectory(ctx, strings.TrimSpace(cfg.ProjectsDirectory.Value))- if err != nil {- slog.WarnContext(ctx, "unable to determine projects directory; using default", "error", err)- return "/app/data/projects"- }- return projectsDirectory-}--func (s *ProjectService) getMutableProjectInternal(ctx context.Context, projectID string) (*models.Project, error) {- proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return nil, err- }- if err := ensureProjectMutableInternal(proj); err != nil {- return nil, err- }- return proj, nil-}--func (s *ProjectService) logProjectEventInternal(ctx context.Context, eventType models.EventType, projectID, projectName string, user models.User, metadata models.JSON, action string) {- if s.eventService == nil {- return- }- if logErr := s.eventService.LogProjectEvent(ctx, eventType, projectID, projectName, user.ID, user.Username, "0", metadata); logErr != nil {- slog.ErrorContext(ctx, action, "error", logErr)- }-}--func groupComposeContainersByProjectInternal(containers []container.Summary) map[string][]container.Summary {- containersByProject := make(map[string][]container.Summary)- for _, c := range containers {- projectName := dockerutil.ComposeProjectLabel(c.Labels)- if projectName != "" {- containersByProject[projectName] = append(containersByProject[projectName], c)- }- }- return containersByProject-}--func (s *ProjectService) GetProjectRelativePath(ctx context.Context, projectPath string) string {- projectsDir, err := s.getProjectsDirectoryInternal(ctx)- if err != nil {- return ""- }-- return s.getProjectRelativePathInternal(projectsDir, projectPath)-}--func (s *ProjectService) getProjectRelativePathInternal(projectsDir, projectPath string) string {- if strings.TrimSpace(projectsDir) == "" {- return ""- }-- relativePath, err := filepath.Rel(projectsDir, filepath.Clean(projectPath))- if err != nil {- return ""- }- if relativePath == "." {- return ""- }- if relativePath == ".." || strings.HasPrefix(relativePath, ".."+string(os.PathSeparator)) {- return ""- }-- return filepath.ToSlash(relativePath)-}--// Helpers--type ProjectServiceInfo struct {- Name string `json:"name"`- Image string `json:"image"`- Status string `json:"status"`- ContainerID string `json:"container_id"`- ContainerName string `json:"container_name"`- Ports []string `json:"ports"`- Health *string `json:"health,omitempty"`- IconLightURL string `json:"icon_light_url,omitempty"`- IconDarkURL string `json:"icon_dark_url,omitempty"`- ServiceConfig *composetypes.ServiceConfig `json:"service_config,omitempty"`- Labels map[string]string `json:"labels,omitempty"`- RedeployDisabled bool `json:"redeploy_disabled,omitempty"`-}--type ProjectBuildOptions struct {- Services []string- Provider string- Push *bool- Load *bool-}--var (- composeStopProjectServicesInternal = projects.ComposeStop- composeUpProjectServicesInternal = projects.ComposeUp-)--// lookupProjectContainers returns containers matched to a project, trying the-// normalized directory name first and falling back to the effective compose-// project name (from COMPOSE_PROJECT_NAME) when it differs.-func lookupProjectContainers(p models.Project, containersByProject map[string][]container.Summary) []container.Summary {- normName := projects.NormalizeProjectName(p.Name)- if c := containersByProject[normName]; len(c) > 0 {- return c- }- if p.ComposeProjectName != nil && *p.ComposeProjectName != normName {- return containersByProject[*p.ComposeProjectName]- }- return nil-}--func (s *ProjectService) getCachedComposeProjectIDInternal(normalizedName string) mo.Option[string] {- if normalizedName == "" {- return mo.None[string]()- }-- s.composeNameCacheMu.RLock()- defer s.composeNameCacheMu.RUnlock()-- if s.composeNameToProjID == nil {- return mo.None[string]()- }-- projectID, ok := s.composeNameToProjID[normalizedName]- return mo.TupleToOption(projectID, ok)-}--func (s *ProjectService) cacheComposeProjectIDInternal(normalizedName, projectID string) {- if normalizedName == "" || projectID == "" {- return- }-- s.composeNameCacheMu.Lock()- defer s.composeNameCacheMu.Unlock()-- if s.composeNameToProjID == nil {- s.composeNameToProjID = make(map[string]string)- }- s.composeNameToProjID[normalizedName] = projectID-}--func (s *ProjectService) invalidateCachedComposeProjectIDInternal(normalizedName string) {- if normalizedName == "" {- return- }-- s.composeNameCacheMu.Lock()- defer s.composeNameCacheMu.Unlock()-- delete(s.composeNameToProjID, normalizedName)-}--func (s *ProjectService) lookupProjectByCachedComposeNameInternal(ctx context.Context, normalizedName string) (*models.Project, bool, error) {- projectID, ok := s.getCachedComposeProjectIDInternal(normalizedName).Get()- if !ok {- return nil, false, nil- }-- var projectModel models.Project- if err := s.db.WithContext(ctx).Where("id = ?", projectID).First(&projectModel).Error; err != nil {- if errors.Is(err, gorm.ErrRecordNotFound) {- s.invalidateCachedComposeProjectIDInternal(normalizedName)- return nil, false, nil- }- if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {- return nil, false, errors.WrapIf(err, "request canceled or timed out")- }- return nil, false, errors.WrapIf(err, "failed to get project by cached compose name")- }- if projects.NormalizeProjectName(projectModel.Name) != normalizedName {- s.invalidateCachedComposeProjectIDInternal(normalizedName)- return nil, false, nil- }-- return &projectModel, true, nil-}--func (s *ProjectService) rebuildComposeNameCacheInternal(ctx context.Context) error {- var projectModels []models.Project- if err := s.db.WithContext(ctx).Select("id", "name").Find(&projectModels).Error; err != nil {- return err- }-- cache := make(map[string]string, len(projectModels))- for i := range projectModels {- normalizedName := projects.NormalizeProjectName(projectModels[i].Name)- if normalizedName == "" {- continue- }- if _, exists := cache[normalizedName]; !exists {- cache[normalizedName] = projectModels[i].ID- }- }-- s.composeNameCacheMu.Lock()- s.composeNameToProjID = cache- s.composeNameCacheMu.Unlock()-- return nil-}--func (s *ProjectService) GetProjectFromDatabaseByID(ctx context.Context, id string) (*models.Project, error) {- var projectModel models.Project- if err := s.db.WithContext(ctx).Where("id = ?", id).First(&projectModel).Error; err != nil {- if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {- return nil, errors.New("request canceled or timed out")- }- if errors.Is(err, gorm.ErrRecordNotFound) {- return nil, errors.New("project not found")- }- return nil, errors.WrapIf(err, "failed to get project")- }- return &projectModel, nil-}--func (s *ProjectService) GetProjectByComposeName(ctx context.Context, name string) (*models.Project, error) {- if name == "" {- return nil, errors.New("project name is empty")- }- normalized := projects.NormalizeProjectName(name)-- var proj models.Project- err := s.db.WithContext(ctx).Where("name = ? OR name = ?", name, normalized).First(&proj).Error- if err == nil {- s.cacheComposeProjectIDInternal(normalized, proj.ID)- return &proj, nil- }- if !errors.Is(err, gorm.ErrRecordNotFound) {- return nil, errors.WrapIf(err, "failed to get project by name")- }-- if cachedProject, found, cacheErr := s.lookupProjectByCachedComposeNameInternal(ctx, normalized); cacheErr != nil {- return nil, cacheErr- } else if found {- return cachedProject, nil- }-- if err := s.rebuildComposeNameCacheInternal(ctx); err != nil {- return nil, errors.WrapIf(err, "failed to list projects by compose name")- }-- if cachedProject, found, cacheErr := s.lookupProjectByCachedComposeNameInternal(ctx, normalized); cacheErr != nil {- return nil, cacheErr- } else if found {- return cachedProject, nil- }-- return nil, errors.Errorf("project not found: %s", name)-}--func (s *ProjectService) resolveProjectComposeFileInternal(ctx context.Context, proj *models.Project) (string, error) {- if proj == nil {- return "", errors.New("project is nil")- }-- if proj.GitOpsManagedBy != nil && strings.TrimSpace(*proj.GitOpsManagedBy) != "" {- var syncRecord models.GitOpsSync- if err := s.db.WithContext(ctx).- Select("compose_path").- Where("id = ?", *proj.GitOpsManagedBy).- First(&syncRecord).Error; err == nil {- composeFileName := strings.TrimSpace(filepath.Base(syncRecord.ComposePath))- if composeFileName != "" && composeFileName != "." {- candidate := filepath.Join(proj.Path, composeFileName)- if info, statErr := os.Stat(candidate); statErr == nil {- if !info.IsDir() {- return candidate, nil- }- } else if !os.IsNotExist(statErr) {- return "", errors.WrapIff(statErr, "failed to inspect GitOps compose file %s", candidate)- }- }- } else if !errors.Is(err, gorm.ErrRecordNotFound) {- return "", errors.WrapIff(err, "failed to resolve GitOps compose path for project %s", proj.ID)- }- }-- composeFile, err := projects.DetectComposeFile(proj.Path)- if err != nil {- return "", common.Classify(common.ErrProjectComposeFileNotFound, errors.WrapIf(err, "Project compose file not found"))- }-- return composeFile, nil-}--func (s *ProjectService) loadComposeProjectForProjectInternal(ctx context.Context, proj *models.Project, cfg *models.Settings) (*composetypes.Project, string, error) {- composeFileFullPath, err := s.resolveProjectComposeFileInternal(ctx, proj)- if err != nil {- return nil, "", err- }-- if cfg == nil {- cfg = s.settingsService.GetSettingsOrDefaults(ctx)- }- projectsDirectory := s.getProjectsDirectoryOrDefaultInternal(ctx, cfg)-- var dockerClient *client.Client- if s.dockerService != nil {- dockerClient, _ = s.dockerService.GetClient(ctx)- }- pathMapper := projects.NewPathMapperForConfiguredDirectory(- ctx,- s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"),- "/app/data/projects",- dockerClient,- )-- composeProject, loadErr := projects.LoadComposeProject(ctx, composeFileFullPath, projects.NormalizeProjectName(proj.Name), projectsDirectory, utils.BoolOrDefault(cfg.AutoInjectEnv.Value, false), pathMapper, nil, nil, false)- if loadErr != nil {- return nil, "", loadErr- }-- return composeProject, composeFileFullPath, nil-}--func (s *ProjectService) getCachedComposeProjectInternal(ctx context.Context, proj *models.Project, cfg *models.Settings) (*composetypes.Project, error) {- if proj == nil {- return nil, errors.New("project is nil")- }- if s.composeCache == nil {- s.composeCache = hot.NewHotCache[string, composeCacheEntry](hot.LRU, 2048).Build()- }-- if cached, ok := s.composeCache.Peek(proj.ID); ok {- if validComposeCacheEntryInternal(cached) {- return cached.project, nil- }- s.composeCache.Delete(proj.ID)- }-- entry, found, err := s.composeCache.GetWithLoaders(proj.ID, func(_ []string) (map[string]composeCacheEntry, error) {- composeProject, composePath, err := s.loadComposeProjectForProjectInternal(ctx, proj, cfg)- if err != nil {- return nil, err- }-- entry := composeCacheEntry{- composePath: composePath,- includeMtimes: make(map[string]time.Time),- project: composeProject,- }- if info, statErr := os.Stat(composePath); statErr == nil {- entry.composeMtime = info.ModTime()- } else {- return nil, errors.WrapIf(statErr, "stat compose file")- }- if composeProject != nil {- for _, composeFile := range composeProject.ComposeFiles {- if composeFile == "" || composeFile == composePath {- continue- }- info, statErr := os.Stat(composeFile)- if statErr != nil {- return nil, errors.WrapIff(statErr, "stat compose include %s", composeFile)- }- entry.includeMtimes[composeFile] = info.ModTime()- }- }-- return map[string]composeCacheEntry{proj.ID: entry}, nil- })- if err != nil {- return nil, err- }- if !found {- return nil, errors.New("compose cache loader returned no project")- }-- return entry.project, nil-}--func validComposeCacheEntryInternal(entry composeCacheEntry) bool {- if entry.project == nil || entry.composePath == "" {- return false- }-- info, err := os.Stat(entry.composePath)- if err != nil || !info.ModTime().Equal(entry.composeMtime) {- return false- }- for includePath, cachedMtime := range entry.includeMtimes {- info, err := os.Stat(includePath)- if err != nil || !info.ModTime().Equal(cachedMtime) {- return false- }- }- return true-}--func (s *ProjectService) invalidateComposeCacheInternal(projectID string) {- if s.composeCache == nil || strings.TrimSpace(projectID) == "" {- return- }- s.composeCache.Delete(projectID)-}--func (s *ProjectService) refreshProjectImageRefsInternal(ctx context.Context, proj *models.Project) {- if proj == nil || proj.ID == "" {- return- }-- s.invalidateComposeCacheInternal(proj.ID)- refs, buildRefs, err := s.getProjectImageRefsFromComposeInternal(ctx, *proj, nil)- if err != nil {- if dbErr := s.db.WithContext(ctx).- Model(&models.Project{}).- Where("id = ?", proj.ID).- Updates(map[string]any{- "image_refs_json": "",- "build_image_refs_json": nil,- }).Error; dbErr != nil {- slog.WarnContext(ctx, "failed to clear stale project image refs", "projectID", proj.ID, "error", dbErr)- }- proj.ImageRefsJSON = ""- proj.BuildImageRefsJSON = nil- slog.WarnContext(ctx, "failed to refresh project image refs", "projectID", proj.ID, "projectName", proj.Name, "error", err)- return- }- imageRefsJSON := projects.MarshalImageRefsJSON(refs)- buildImageRefsJSON := projects.MarshalImageRefsJSON(buildRefs)- if buildImageRefsJSON == "" {- buildImageRefsJSON = "[]"- }- if err := s.db.WithContext(ctx).- Model(&models.Project{}).- Where("id = ?", proj.ID).- Updates(map[string]any{- "image_refs_json": imageRefsJSON,- "build_image_refs_json": buildImageRefsJSON,- }).Error; err != nil {- slog.WarnContext(ctx, "failed to persist project image refs", "projectID", proj.ID, "error", err)- return- }- proj.ImageRefsJSON = imageRefsJSON- proj.BuildImageRefsJSON = new(buildImageRefsJSON)-}--func (s *ProjectService) HandleProjectFilesChanged(ctx context.Context, paths []string) {- if len(paths) == 0 || s.db == nil {- return- }-- affected, err := s.resolveProjectsByChangedPathsInternal(ctx, paths)- if err != nil {- slog.WarnContext(ctx, "failed to resolve changed project files", "error", err)- return- }- for i := range affected {- s.invalidateComposeCacheInternal(affected[i].ID)- s.refreshProjectImageRefsInternal(ctx, &affected[i])- }-}--func (s *ProjectService) BackfillProjectImageRefs(ctx context.Context) (int, error) {- if s.db == nil {- return 0, nil- }-- var projectsList []models.Project- if err := s.db.WithContext(ctx).- Where("build_image_refs_json IS NULL").- Find(&projectsList).Error; err != nil {- return 0, errors.WrapIf(err, "list projects for image ref backfill")- }- for i := range projectsList {- if err := ctx.Err(); err != nil {- return i, err- }- s.refreshProjectImageRefsInternal(ctx, &projectsList[i])- }- return len(projectsList), nil-}--func (s *ProjectService) resolveProjectsByChangedPathsInternal(ctx context.Context, paths []string) ([]models.Project, error) {- var projectsList []models.Project- if err := s.db.WithContext(ctx).Find(&projectsList).Error; err != nil {- return nil, errors.WrapIf(err, "list projects for changed paths")- }-- seen := make(map[string]struct{})- affected := make([]models.Project, 0)- for _, changedPath := range paths {- cleanChangedPath := filepath.Clean(changedPath)- for _, proj := range projectsList {- projectPath := filepath.Clean(proj.Path)- if cleanChangedPath != projectPath && !strings.HasPrefix(cleanChangedPath, projectPath+string(os.PathSeparator)) {- continue- }- if _, ok := seen[proj.ID]; ok {- continue- }- seen[proj.ID] = struct{}{}- affected = append(affected, proj)- }- }- return affected, nil-}--func buildSelectedProjectImageRefsInternal(compProj *composetypes.Project, servicesToUpdate []string) []string {- if compProj == nil {- return nil- }-- selected := normalizeBuildSelections(servicesToUpdate)- refs := make([]string, 0, len(compProj.Services))- seen := make(map[string]struct{}, len(compProj.Services))-- for name, svc := range compProj.Services {- if !serviceSelected(selected, name) || svc.Build != nil {- continue- }-- imageRef := strings.TrimSpace(svc.Image)- if imageRef == "" {- continue- }- if _, exists := seen[imageRef]; exists {- continue- }-- seen[imageRef] = struct{}{}- refs = append(refs, imageRef)- }-- return refs-}--func (s *ProjectService) reconcilePulledImageRefsInternal(ctx context.Context, imageRefs []string) {- if s.imageService == nil {- return- }-- for _, imageRef := range imageRefs {- if err := s.imageService.ReconcilePulledImageUpdate(ctx, imageRef); err != nil {- slog.WarnContext(ctx, "failed to reconcile pulled image update state", "image", imageRef, "error", err)- }- }-}--func (s *ProjectService) composePullSelectedServicesInternal(- ctx context.Context,- compProj *composetypes.Project,- servicesToUpdate []string,- user models.User,- credentials []containerregistry.Credential,-) error {- if compProj == nil {- return nil- }-- imageRefsToPull := buildSelectedProjectImageRefsInternal(compProj, servicesToUpdate)- if len(imageRefsToPull) == 0 {- return nil- }-- progressWriter, _ := ctx.Value(dockerutil.ProgressWriterKey{}).(io.Writer)- for _, imageRef := range imageRefsToPull {- if err := s.pullAndReconcileImageInternal(ctx, imageRef, progressWriter, user, credentials); err != nil {- return err- }- }-- return nil-}--func (s *ProjectService) pullAndReconcileImageInternal(- ctx context.Context,- imageRef string,- progressWriter io.Writer,- user models.User,- credentials []containerregistry.Credential,-) error {- if s == nil || s.imageService == nil {- return errors.New("image service not available")- }-- settings := s.settingsService.GetSettingsConfig()-- pullCtx, pullCancel := context.WithTimeout(ctx, timeouts.GetDuration(settings.DockerImagePullTimeout.AsInt(), timeouts.DefaultDockerImagePull))- defer pullCancel()-- if err := s.imageService.PullImage(pullCtx, imageRef, progressWriter, user, credentials); err != nil {- if errors.Is(pullCtx.Err(), context.DeadlineExceeded) {- return errors.Errorf("image pull timed out for %s (increase DOCKER_IMAGE_PULL_TIMEOUT or setting)", imageRef)- }- return errors.WrapIff(err, "failed to pull image %s", imageRef)- }-- s.reconcilePulledImageRefsInternal(ctx, []string{imageRef})- return nil-}--func (s *ProjectService) UpdateProjectServices(ctx context.Context, projectID string, servicesToUpdate []string, user models.User) error {- projectFromDb, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return err- }- previousStatus := projectFromDb.Status-- // 1. Load project- compProj, _, err := s.loadComposeProjectForProjectInternal(ctx, projectFromDb, nil)- if err != nil {- return errors.WrapIf(err, "failed to load compose project")- }-- // 2. Set status to deploying/restarting- if err := s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusDeploying); err != nil {- return err- }-- credentials, err := s.resolveRegistryCredentialsInternal(ctx)- if err != nil {- if statusErr := s.updateProjectStatusInternal(ctx, projectID, previousStatus); statusErr != nil {- slog.ErrorContext(ctx, "UpdateProjectServices: failed to restore project status after credential lookup failure", "projectID", projectID, "error", statusErr)- }- return errors.WrapIf(err, "resolve registry credentials")- }-- // 3. Pull images for specific services- if err := s.composePullSelectedServicesInternal(ctx, compProj, servicesToUpdate, user, credentials); err != nil {- if statusErr := s.updateProjectStatusInternal(ctx, projectID, previousStatus); statusErr != nil {- slog.ErrorContext(ctx, "UpdateProjectServices: failed to restore project status after compose pull failure", "projectID", projectID, "error", statusErr)- }- return errors.WrapIf(err, "pull updated service images")- }-- // 4. Stop specific services- if err := composeStopProjectServicesInternal(ctx, compProj, servicesToUpdate); err != nil {- slog.WarnContext(ctx, "compose stop failed, continuing", "error", err)- }-- // 5. Up specific services- if err := composeUpProjectServicesInternal(ctx, compProj, servicesToUpdate, false, true, false, s.composeRegistryAuthConfigsInternal(ctx)); err != nil {- s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)- return errors.WrapIf(err, "failed to up services")- }-- // 6. Finalize status- if err := s.updateProjectStatusandCountsInternal(ctx, projectID, models.ProjectStatusRunning); err != nil {- return err- }-- metadata := models.JSON{- "action": "update_services",- "projectID": projectID,- "projectName": projectFromDb.Name,- "services": append([]string(nil), servicesToUpdate...),- }- s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, projectID, projectFromDb.Name, user, metadata, "could not log project service update action")-- return nil-}--func (s *ProjectService) getServiceCounts(services []ProjectServiceInfo) (total int, running int) {- total = len(services)- for _, service := range services {- st := strings.ToLower(strings.TrimSpace(service.Status))- if st == "running" || st == "up" {- running++- }- }- return total, running-}--func (s *ProjectService) updateProjectStatusandCountsInternal(ctx context.Context, projectID string, status models.ProjectStatus) error {- services, err := s.GetProjectServices(ctx, projectID)- if err != nil {- slog.Error("GetProjectServices failed during status update", "projectID", projectID, "error", err)- return s.updateProjectStatusInternal(ctx, projectID, status)- }-- serviceCount, runningCount := s.getServiceCounts(services)-- if err := s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", projectID).Updates(map[string]any{- "status": status,- "service_count": serviceCount,- "running_count": runningCount,- "updated_at": time.Now(),- }).Error; err != nil {- return errors.WrapIf(err, "failed to update project status and counts")- }-- return nil-}--func (s *ProjectService) updateProjectStatusInternal(ctx context.Context, id string, status models.ProjectStatus) error {- now := time.Now()- res := s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", id).Updates(map[string]any{- "status": status,- "updated_at": now,- })-- if res.Error != nil {- return errors.WrapIf(res.Error, "failed to update project status")- }-- return nil-}--func (s *ProjectService) GetProjectServices(ctx context.Context, projectID string) ([]ProjectServiceInfo, error) {- projectFromDb, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return nil, err- }-- composeProject, composeFileFullPath, derr := s.loadComposeProjectForProjectInternal(ctx, projectFromDb, nil)- if derr != nil {- return []ProjectServiceInfo{}, errors.WrapIff(derr, "failed to load compose project in %s", projectFromDb.Path)- }-- projectsDirectory, projectsDirErr := s.getProjectsDirectoryInternal(ctx)- if projectsDirErr != nil {- slog.WarnContext(ctx, "failed to resolve projects directory for Arcane compose metadata", "path", composeFileFullPath, "error", projectsDirErr)- }- autoInjectEnv := s.settingsService.GetBoolSetting(ctx, "autoInjectEnv", false)-- meta, metaErr := projects.ParseArcaneComposeMetadata(ctx, composeFileFullPath, projectsDirectory, autoInjectEnv)- if metaErr != nil {- slog.WarnContext(ctx, "failed to parse Arcane compose metadata", "path", composeFileFullPath, "error", metaErr)- }-- containers, err := projects.ComposePs(ctx, composeProject, nil, true)- if err != nil {- slog.Error("compose ps error", "projectName", composeProject.Name, "error", err)- return nil, errors.WrapIf(err, "failed to get compose services status")- }- currentContainerID, currentContainerErr := cgroup.CurrentContainerID()-- have := map[string]bool{}- var services []ProjectServiceInfo-- // Create a map for quick lookup of service config- serviceConfigs := make(map[string]composetypes.ServiceConfig)- for _, svc := range composeProject.Services {- serviceConfigs[svc.Name] = svc- }-- for _, c := range containers {- var health *string- if c.Health != "" {- health = new(string(c.Health))- }-- var svcConfig *composetypes.ServiceConfig- if cfg, ok := serviceConfigs[c.Service]; ok {- svcConfig = &cfg- }-- resolvedIcon := s.resolveIconSetInternal(ctx, iconcatalog.FirstNonEmpty(- projects.FindArcaneIconSet(c.Labels),- meta.ServiceIconSets[c.Service],- meta.ProjectIcon,- ))- services = append(services, ProjectServiceInfo{- Name: c.Service,- Image: c.Image,- Status: string(c.State),- ContainerID: c.ID,- ContainerName: c.Name,- Ports: formatPorts(c.Publishers),- Health: health,- IconLightURL: resolvedIcon.IconLightURL,- IconDarkURL: resolvedIcon.IconDarkURL,- ServiceConfig: svcConfig,- Labels: c.Labels,- RedeployDisabled: labels.ShouldDisableArcaneServerRedeploy(c.Labels, c.ID, currentContainerID, currentContainerErr),- })- have[c.Service] = true- }-- for _, svc := range composeProject.Services {- if !have[svc.Name] {- resolvedIcon := s.resolveIconSetInternal(ctx, iconcatalog.FirstNonEmpty(- meta.ServiceIconSets[svc.Name],- meta.ProjectIcon,- ))- services = append(services, ProjectServiceInfo{- Name: svc.Name,- Image: svc.Image,- Status: "stopped",- Ports: []string{},- IconLightURL: resolvedIcon.IconLightURL,- IconDarkURL: resolvedIcon.IconDarkURL,- ServiceConfig: new(svc),- })- }- }-- return services, nil-}--func (s *ProjectService) GetProjectContent(ctx context.Context, projectID string) (composeContent, envContent, overrideContent string, err error) {- proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return "", "", "", err- }-- composePath, composeErr := s.resolveProjectComposeFileInternal(ctx, proj)- if composeErr != nil {- composePath = ""- }-- composeContent, envContent, err = projects.ReadProjectFiles(proj.Path, composePath)- if err != nil {- return "", "", "", err- }-- return composeContent, envContent, projects.ReadComposeOverrideContent(proj.Path), nil-}--func (s *ProjectService) GetProjectDetails(ctx context.Context, projectID string, opts project.DetailsOptions) (project.Details, error) {- proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return project.Details{}, err- }- projectsDir, _ := s.getProjectsDirectoryInternal(ctx)-- var resp project.Details- if err := mapper.MapStruct(proj, &resp); err != nil {- return project.Details{}, errors.WrapIf(err, "failed to map project")- }-- resp.CreatedAt = proj.CreatedAt.Format(time.RFC3339)- resp.UpdatedAt = proj.UpdatedAt.Format(time.RFC3339)- resp.IsArchived = proj.IsArchived- resp.ArchivedAt = proj.ArchivedAt- resp.HasBuildDirective = false- resp.DirName = mo.PointerToOption(proj.DirName).OrEmpty()- resp.RelativePath = s.getProjectRelativePathInternal(projectsDir, proj.Path)- resp.GitOpsManagedBy = proj.GitOpsManagedBy- meta := s.getProjectMetadataForProject(ctx, *proj, nil)- applyResolvedProjectIconInternal(&resp, s.resolveIconSetInternal(ctx, meta.ProjectIcon))- resp.URLs = meta.ProjectURLS-- // Default counts/status from DB (will be overridden if runtime check succeeds)- resp.ServiceCount = proj.ServiceCount- resp.RunningCount = proj.RunningCount- resp.Status = string(proj.Status)-- if opts.IncludeComposeContent {- composeContent, _, overrideContent, _ := s.GetProjectContent(ctx, projectID)- resp.ComposeContent = composeContent- resp.OverrideContent = overrideContent- if overridePath := projects.DetectComposeOverrideFile(proj.Path); overridePath != "" {- resp.OverrideFileName = filepath.Base(overridePath)- }- }- if opts.IncludeEnvState {- envState, err := projects.ReadProjectEnvState(proj.Path)- if err != nil {- return project.Details{}, errors.WrapIf(err, "failed to read project env state")- }- effectiveEnvContent, err := s.resolveStoredEffectiveEnvContentInternal(envState)- if err != nil {- return project.Details{}, err- }- resp.EnvContent = effectiveEnvContent- }-- // Enrich with details- composeFile, composeFileErr := s.resolveProjectComposeFileInternal(ctx, proj)- if composeFileErr == nil {- resp.ComposeFileName = filepath.Base(composeFile)- if opts.IncludeIncludeFiles {- s.enrichWithIncludeFiles(ctx, composeFile, &resp)- }- if opts.IncludeServiceConfigs {- s.enrichWithComposeServiceConfigs(ctx, proj, composeFile, &resp)- }- }- if opts.IncludeDirectoryFiles {- s.enrichWithDirectoryFiles(ctx, proj.Path, &resp)- }- if opts.IncludeProjectFiles {- s.enrichWithProjectFiles(ctx, proj.Path, resp.ComposeFileName, &resp)- }- s.enrichWithGitOpsInfo(ctx, proj, &resp)-- // Refresh runtime status/counts even when callers do not request the full- // runtime service array. DB values are only a fallback when Docker lookup- // or compose loading fails.- services, serr := s.GetProjectServices(ctx, projectID)- if serr == nil && services != nil {- resp.ServiceCount = len(services)- _, runningCount := s.getServiceCounts(services)- resp.RunningCount = runningCount- resp.Status = string(s.calculateProjectStatus(services))-- if opts.IncludeRuntimeServices {- resp.RuntimeServices = buildProjectRuntimeServicesInternal(services)- for _, svc := range services {- if svc.RedeployDisabled {- resp.RedeployDisabled = true- break- }- }- }- }-- if opts.IncludeUpdateInfo {- s.enrichProjectUpdateInfoInternal(ctx, &resp)- }-- return resp, nil-}--func buildProjectRuntimeServicesInternal(services []ProjectServiceInfo) []project.RuntimeService {- runtimeServices := make([]project.RuntimeService, len(services))- for i, svc := range services {- runtimeServices[i] = project.RuntimeService{- Name: svc.Name,- Image: svc.Image,- Status: svc.Status,- ContainerID: svc.ContainerID,- ContainerName: svc.ContainerName,- Ports: svc.Ports,- Health: svc.Health,- IconLightURL: svc.IconLightURL,- IconDarkURL: svc.IconDarkURL,- ServiceConfig: svc.ServiceConfig,- RedeployDisabled: svc.RedeployDisabled,- }- }- return runtimeServices-}--func (s *ProjectService) GetProjectFileContent(ctx context.Context, projectID, relativePath string) (project.IncludeFile, error) {- proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return project.IncludeFile{}, err- }- if strings.TrimSpace(relativePath) == "" {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: relative path is required"))- }- normalizedRelativePath, err := projects.NormalizeProjectRelativePath(relativePath)- if err != nil {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.WrapIf(err, "Forbidden project file path"))- }-- composeFile, detectErr := s.resolveProjectComposeFileInternal(ctx, proj)- if detectErr == nil {- cfg := s.settingsService.GetSettingsOrDefaults(ctx)- projectsDirectory, _ := projects.GetProjectsDirectory(ctx, strings.TrimSpace(cfg.ProjectsDirectory.Value))- envLoader := projects.NewEnvLoader(projectsDirectory, filepath.Dir(composeFile), utils.BoolOrDefault(cfg.AutoInjectEnv.Value, false))- envMap, _, _ := envLoader.LoadEnvironment(ctx)-- includes, parseErr := projects.ParseIncludes(composeFile, envMap, false)- if parseErr == nil {- for _, inc := range includes {- if inc.RelativePath != normalizedRelativePath {- continue- }- if !projects.IsSafeSubdirectory(proj.Path, inc.Path) {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.New("Forbidden project file path: include resolves outside project directory"))- }- return readProjectIncludeFileContentInternal(proj.Path, inc)- }- }- }-- absFilePath, err := projects.ValidateIncludePathForWrite(proj.Path, normalizedRelativePath)- if err != nil {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.WrapIf(err, "Forbidden project file path"))- }-- info, err := os.Lstat(absFilePath)- if err != nil {- if os.IsNotExist(err) {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileNotFound, errors.New("Project file not found"))- }- return project.IncludeFile{}, errors.WrapIf(err, "failed to stat file")- }- if info.IsDir() {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: path is a directory"))- }- if info.Mode()&os.ModeSymlink != 0 {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.New("Forbidden project file path: symlinks are not allowed"))- }-- content, err := os.ReadFile(absFilePath)- if err != nil {- if os.IsNotExist(err) {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileNotFound, errors.New("Project file not found"))- }- return project.IncludeFile{}, errors.WrapIf(err, "failed to read file")- }- if projects.IsBinaryProjectFileContent(content) {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: binary files are not supported"))- }-- return project.IncludeFile{- Path: absFilePath,- RelativePath: normalizedRelativePath,- Content: string(content),- }, nil-}--func readProjectIncludeFileContentInternal(projectPath string, inc projects.IncludeFile) (project.IncludeFile, error) {- validatedPath, err := projects.ValidateIncludePathForWrite(projectPath, inc.Path)- if err != nil {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.WrapIf(err, "Forbidden project file path"))- }-- resolvedPath, err := utils.ResolveWithinRoot(projectPath, validatedPath)- if err != nil {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileForbidden, errors.WrapIf(err, "Forbidden project file path"))- }-- info, err := os.Stat(resolvedPath)- if os.IsNotExist(err) {- return project.IncludeFile{- Path: validatedPath,- RelativePath: inc.RelativePath,- Content: "# This file will be created when you save changes\nservices:\n",- }, nil- }- if err != nil {- return project.IncludeFile{}, errors.WrapIf(err, "failed to stat include file")- }- if info.IsDir() {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: path is a directory"))- }-- content, err := os.ReadFile(resolvedPath)- if err != nil {- if os.IsNotExist(err) {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileNotFound, errors.New("Project file not found"))- }- return project.IncludeFile{}, errors.WrapIf(err, "failed to read include file")- }- if projects.IsBinaryProjectFileContent(content) {- return project.IncludeFile{}, common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: binary files are not supported"))- }-- return project.IncludeFile{- Path: resolvedPath,- RelativePath: inc.RelativePath,- Content: string(content),- }, nil-}--func (s *ProjectService) enrichWithIncludeFiles(ctx context.Context, composeFile string, resp *project.Details) {- if strings.TrimSpace(composeFile) == "" {- return- }-- // Load environment variables so that include paths with ${VAR} references are expanded- cfg := s.settingsService.GetSettingsOrDefaults(ctx)- projectsDirectory, _ := projects.GetProjectsDirectory(ctx, strings.TrimSpace(cfg.ProjectsDirectory.Value))- envLoader := projects.NewEnvLoader(projectsDirectory, filepath.Dir(composeFile), utils.BoolOrDefault(cfg.AutoInjectEnv.Value, false))- envMap, _, _ := envLoader.LoadEnvironment(ctx)-- includes, parseErr := projects.ParseIncludes(composeFile, envMap, false)- if parseErr == nil {- var includeFiles []project.IncludeFile- for _, inc := range includes {- includeFiles = append(includeFiles, project.IncludeFile{- Path: inc.Path,- RelativePath: inc.RelativePath,- })- }- resp.IncludeFiles = includeFiles- } else {- slog.WarnContext(ctx, "Failed to parse includes", "error", parseErr, "path", composeFile)- }-}--func (s *ProjectService) enrichProjectUpdateInfoInternal(ctx context.Context, resp *project.Details) {- if resp == nil {- return- }-- imageRefs := projects.ImageRefsFromComposeConfigs(resp.Services)- if len(imageRefs) == 0 {- imageRefs = projects.ImageRefsFromRuntimeServices(resp.RuntimeServices)- }-- var updateInfoByRef map[string]*imagetypes.UpdateInfo- if len(imageRefs) > 0 && s.imageService != nil {- lookupResult, err := s.imageService.GetUpdateInfoByImageRefs(ctx, imageRefs)- if err != nil {- slog.WarnContext(ctx, "failed to fetch project update info", "projectID", resp.ID, "projectName", resp.Name, "error", err)- } else {- updateInfoByRef = lookupResult- }- }-- resp.UpdateInfo = buildProjectUpdateInfoSummaryInternal(imageRefs, updateInfoByRef)-}--func (s *ProjectService) enrichProjectsWithUpdateInfoInternal(- ctx context.Context,- projectsList []models.Project,- details []project.Details,-) {- if len(projectsList) == 0 || len(details) == 0 {- return- }-- imageRefsByProjectID := make(map[string][]string, len(projectsList))- allImageRefs := make([]string, 0)- cfg := s.settingsService.GetSettingsOrDefaults(ctx)-- const maxConcurrentComposeReads = 8- type imageRefsResult struct {- projectID string- refs []string- }-- sem := make(chan struct{}, maxConcurrentComposeReads)- resultsCh := make(chan imageRefsResult, len(projectsList))-- var wg sync.WaitGroup- for _, proj := range projectsList {- if refs := projects.ParseImageRefsJSON(proj.ImageRefsJSON); len(refs) > 0 {- imageRefsByProjectID[proj.ID] = refs- allImageRefs = append(allImageRefs, refs...)- continue- }-- wg.Add(1)- go func(proj models.Project) {- defer wg.Done()-- select {- case sem <- struct{}{}:- case <-ctx.Done():- return- }- defer func() { <-sem }()-- refs, _, err := s.getProjectImageRefsFromComposeInternal(ctx, proj, cfg)- if err != nil {- slog.WarnContext(ctx, "failed to resolve project image refs for update summary", "projectID", proj.ID, "projectName", proj.Name, "error", err)- return- }-- resultsCh <- imageRefsResult{projectID: proj.ID, refs: refs}- }(proj)- }-- wg.Wait()- close(resultsCh)-- for result := range resultsCh {- imageRefsByProjectID[result.projectID] = result.refs- allImageRefs = append(allImageRefs, result.refs...)- }-- var updateInfoByRef map[string]*imagetypes.UpdateInfo- if len(allImageRefs) > 0 && s.imageService != nil {- lookupResult, err := s.imageService.GetUpdateInfoByImageRefs(ctx, allImageRefs)- if err != nil {- slog.WarnContext(ctx, "failed to fetch project list update info", "error", err)- } else {- updateInfoByRef = lookupResult- }- }-- for i := range details {- details[i].UpdateInfo = buildProjectUpdateInfoSummaryInternal(imageRefsByProjectID[details[i].ID], updateInfoByRef)- }-}--func (s *ProjectService) getProjectImageRefsFromComposeInternal(ctx context.Context, proj models.Project, cfg *models.Settings) ([]string, []string, error) {- composeProject, err := s.getCachedComposeProjectInternal(ctx, &proj, cfg)- if err != nil {- return nil, nil, errors.WrapIf(err, "load compose project")- }-- return projects.ImageRefsFromComposeServices(composeProject.Services), projects.BuildImageRefsFromComposeProject(composeProject), nil-}--func buildProjectUpdateInfoSummaryInternal(- imageRefs []string,- updateInfoByRef map[string]*imagetypes.UpdateInfo,-) *project.UpdateInfo {- imageCount := len(imageRefs)- summary := &project.UpdateInfo{- Status: "unknown",- HasUpdate: false,- ImageCount: imageCount,- ImageRefs: append([]string(nil), imageRefs...),- }-- if imageCount == 0 {- return summary- }-- var latestCheckTime *time.Time-- for _, imageRef := range imageRefs {- info := updateInfoByRef[imageRef]- if info == nil {- continue- }-- summary.CheckedImageCount++- if info.HasUpdate {- summary.HasUpdate = true- summary.ImagesWithUpdates++- summary.UpdatedImageRefs = append(summary.UpdatedImageRefs, imageRef)- }- if strings.TrimSpace(info.Error) != "" {- summary.ErrorCount++- if summary.ErrorMessage == nil {- summary.ErrorMessage = new(strings.TrimSpace(info.Error))- }- }- if !info.CheckTime.IsZero() && (latestCheckTime == nil || info.CheckTime.After(*latestCheckTime)) {- latestCheckTime = new(info.CheckTime)- }- }-- summary.LastCheckedAt = latestCheckTime-- switch {- case summary.ImagesWithUpdates > 0:- summary.Status = "has_update"- case summary.ErrorCount > 0:- summary.Status = "error"- case summary.CheckedImageCount == imageCount:- summary.Status = "up_to_date"- default:- summary.Status = "unknown"- }-- return summary-}--func (s *ProjectService) enrichWithDirectoryFiles(ctx context.Context, projectPath string, resp *project.Details) {- if projectPath == "" {- return- }-- // Build set of already-shown files to skip- shownFiles := map[string]bool{- ".env": true,- "compose.yaml": true,- "compose.yml": true,- "docker-compose.yaml": true,- "docker-compose.yml": true,- "podman-compose.yaml": true,- "podman-compose.yml": true,- }- for _, inc := range resp.IncludeFiles {- shownFiles[inc.RelativePath] = true- }-- dirFiles, err := projects.ReadProjectDirectoryFiles(projectPath, shownFiles, s.config.ProjectScanMaxDepth, s.config.ProjectScanSkipDirs)- if err != nil {- slog.WarnContext(ctx, "Failed to scan project directory files", "error", err, "path", projectPath)- }-- resp.DirectoryFiles = dirFiles-}--func (s *ProjectService) enrichWithProjectFiles(ctx context.Context, projectPath, composeFileName string, resp *project.Details) {- if projectPath == "" {- return- }-- files, revision, truncated, err := projects.ReadProjectFileTree(projectPath, s.config.ProjectFileTreeMaxDepth, s.config.ProjectScanSkipDirs, composeFileName, projects.DefaultProjectFileTreeMaxEntries)- if err != nil {- slog.WarnContext(ctx, "Failed to scan project file tree", "error", err, "path", projectPath)- return- }-- resp.ProjectFiles = files- resp.FileTreeRevision = revision- resp.FileTreeTruncated = truncated-}--func (s *ProjectService) enrichWithGitOpsInfo(ctx context.Context, proj *models.Project, resp *project.Details) {- if proj.GitOpsManagedBy != nil {- var syncRecord models.GitOpsSync- if err := s.db.WithContext(ctx).Preload("Repository").Where("id = ?", *proj.GitOpsManagedBy).First(&syncRecord).Error; err == nil {- resp.LastSyncCommit = syncRecord.LastSyncCommit- if syncRecord.Repository != nil {- resp.GitRepositoryURL = syncRecord.Repository.URL- }- }- }-}--func (s *ProjectService) enrichWithComposeServiceConfigs(ctx context.Context, proj *models.Project, composeFile string, resp *project.Details) {- composeProj, loadErr := s.getCachedComposeProjectInternal(ctx, proj, nil)- if loadErr != nil {- slog.WarnContext(ctx, "failed to load compose service configs", "path", composeFile, "error", loadErr)- return- }-- if composeProj == nil {- return- }-- // Convert map to slice- svcList := make([]composetypes.ServiceConfig, 0, len(composeProj.Services))- hasBuildDirective := false- for _, svc := range composeProj.Services {- svcList = append(svcList, svc)- if svc.Build != nil {- hasBuildDirective = true- }- }- resp.Services = svcList- resp.HasBuildDirective = resp.HasBuildDirective || hasBuildDirective-}--func (s *ProjectService) SyncProjectsFromFileSystem(ctx context.Context) error {- // Serialized because the walk and the cleanup are two halves of one- // decision: overlapping syncs let an older walk's cleanup delete a project a- // newer walk had just upserted, because the older walk's `seen` set predates- // it. Filesystem-watcher debounces fire these back to back.- s.syncMu.Lock()- defer s.syncMu.Unlock()-- followProjectSymlinks := s.settingsService.GetBoolSetting(ctx, "followProjectSymlinks", false)- projectsDir, err := s.getProjectsDirectoryInternal(ctx)- if err != nil {- slog.WarnContext(ctx, "unable to prepare projects directory", "error", err)- return nil- }-- discoveredProjects, discoveryErr := projects.DiscoverProjectDirectories(projectsDir, followProjectSymlinks, s.config.ProjectScanMaxDepth)- if discoveryErr != nil {- if os.IsNotExist(discoveryErr) {- return nil- }- return errors.WrapIff(discoveryErr, "Failed to discover projects in %q", projectsDir)- }-- renameSyncState := s.activeProjectRenameSyncStateInternal(ctx)- seen := map[string]struct{}{}- for _, discoveredProject := range discoveredProjects {- if renameSyncState.skipDiscoveredPathInternal(discoveredProject.Path) {- continue- }- if uerr := s.upsertProjectForDir(ctx, discoveredProject.DirName, discoveredProject.Path); uerr != nil {- slog.WarnContext(ctx, "failed to sync project from folder", "dir", discoveredProject.Path, "error", uerr)- continue- }- seen[discoveredProject.Path] = struct{}{}- }- renameSyncState.markProtectedPathsSeenInternal(seen)-- if cerr := s.cleanupDBProjectsInternal(ctx, seen, followProjectSymlinks, projectsDir, s.config.ProjectScanMaxDepth); cerr != nil {- slog.WarnContext(ctx, "error during DB cleanup of projects", "error", cerr)- }-- return nil-}--func (s *ProjectService) upsertProjectForDir(ctx context.Context, dirName, dirPath string) error {- var existing models.Project- err := s.db.WithContext(ctx).- Where("path = ?", dirPath).- First(&existing).Error-- composeMetadata, serviceCountErr := s.loadComposeMetadataForSyncInternal(ctx, dirPath, dirName)-- if errors.Is(err, gorm.ErrRecordNotFound) {- // Create a minimal project entry- reason := "Project discovered from filesystem, status pending Docker service query"- proj := &models.Project{- Name: composeMetadata.resolvedProjectName,- DirName: new(dirName),- Path: dirPath,- Status: models.ProjectStatusUnknown,- StatusReason: new(reason),- ServiceCount: composeMetadata.serviceCount,- RunningCount: 0,- ComposeProjectName: composeMetadata.composeProjectName,- }- slog.InfoContext(ctx, "Discovered new project with unknown status",- "project", dirName,- "path", dirPath,- "reason", reason)- if serviceCountErr != nil {- slog.WarnContext(ctx, "failed to read compose service count during project discovery", "project", dirName, "path", dirPath, "error", serviceCountErr)- }- if cerr := s.db.WithContext(ctx).Create(proj).Error; cerr != nil {- return errors.WrapIff(cerr, "create project for %q failed", dirPath)- }- s.warnDuplicateComposeNameForPathInternal(ctx, composeMetadata.resolvedProjectName, dirPath, proj.ID)- return nil- }- if err != nil {- return errors.WrapIff(err, "query existing project for %q failed", dirPath)- }-- updates := map[string]any{}- if existing.Path != dirPath {- updates["path"] = dirPath- }- if existing.DirName == nil || *existing.DirName != dirName {- updates["dir_name"] = dirName- }- if serviceCountErr == nil && existing.ServiceCount != composeMetadata.serviceCount {- updates["service_count"] = composeMetadata.serviceCount- } else if serviceCountErr != nil {- slog.WarnContext(ctx, "failed to refresh compose service count during project sync", "projectID", existing.ID, "path", dirPath, "error", serviceCountErr)- }- if serviceCountErr == nil && mo.PointerToOption(existing.ComposeProjectName) != mo.PointerToOption(composeMetadata.composeProjectName) {- updates["compose_project_name"] = composeMetadata.composeProjectName- }- if serviceCountErr == nil {- if composeMetadata.explicitProjectName {- if existing.Name != composeMetadata.resolvedProjectName {- updates["name"] = composeMetadata.resolvedProjectName- }- } else if normalizedExistingName := projects.NormalizeProjectName(existing.Name); normalizedExistingName != existing.Name {- updates["name"] = normalizedExistingName- }- }- if len(updates) == 0 {- return nil- }-- updates["updated_at"] = time.Now()- if uerr := s.db.WithContext(ctx).- Model(&models.Project{}).- Where("id = ?", existing.ID).- Updates(updates).Error; uerr != nil {- return errors.WrapIff(uerr, "update project %s failed", existing.ID)- }- if serviceCountErr == nil {- s.warnDuplicateComposeNameForPathInternal(ctx, composeMetadata.resolvedProjectName, dirPath, existing.ID)- }- return nil-}--func (s *ProjectService) warnDuplicateComposeNameForPathInternal(ctx context.Context, composeProjectName, dirPath, projectID string) {- if strings.TrimSpace(composeProjectName) == "" {- return- }-- var count int64- if err := s.db.WithContext(ctx).- Model(&models.Project{}).- Where("name = ? AND path <> ? AND id <> ?", composeProjectName, dirPath, projectID).- Count(&count).Error; err != nil {- slog.WarnContext(ctx, "failed to check duplicate compose project names during project sync", "composeProjectName", composeProjectName, "path", dirPath, "error", err)- return- }- if count > 0 {- slog.WarnContext(ctx, "multiple project directories resolve to the same compose project name", "composeProjectName", composeProjectName, "path", dirPath, "duplicates", count)- }-}--// projectCleanupDecision records a project the reconcile pass intends to delete,-// alongside the reason logged when the deletion is carried out.-type projectCleanupDecision struct {- project models.Project- reason string-}--type activeProjectRenameSyncStateInternal struct {- skipDiscoveredPaths map[string]struct{}- protectSeenPaths map[string]struct{}-}--func (s *ProjectService) activeProjectRenameSyncStateInternal(ctx context.Context) activeProjectRenameSyncStateInternal {- state := activeProjectRenameSyncStateInternal{- skipDiscoveredPaths: make(map[string]struct{}),- protectSeenPaths: make(map[string]struct{}),- }- if s == nil || s.kvService == nil {- return state- }-- entries, err := s.kvService.ListByPrefix(ctx, projectRenameJournalKeyPrefixInternal)- if err != nil {- slog.WarnContext(ctx, "failed to list project rename journals during filesystem sync", "error", err)- return state- }-- for _, entry := range entries {- var journal projectRenameJournalInternal- if err := json.Unmarshal([]byte(entry.Value), &journal); err != nil {- slog.WarnContext(ctx, "failed to decode project rename journal during filesystem sync", "key", entry.Key, "error", err)- continue- }- if !projectRenameJournalFilesystemSyncPendingInternal(journal.Phase) {- continue- }- if oldPath := strings.TrimSpace(journal.OldPath); oldPath != "" {- state.protectSeenPaths[filepath.Clean(oldPath)] = struct{}{}- }- if newPath := strings.TrimSpace(journal.NewPath); newPath != "" {- state.skipDiscoveredPaths[filepath.Clean(newPath)] = struct{}{}- }- }-- return state-}--func (s activeProjectRenameSyncStateInternal) skipDiscoveredPathInternal(path string) bool {- _, ok := s.skipDiscoveredPaths[filepath.Clean(path)]- return ok-}--func (s activeProjectRenameSyncStateInternal) markProtectedPathsSeenInternal(seen map[string]struct{}) {- for seenPath := range s.protectSeenPaths {- seen[seenPath] = struct{}{}- }-}--func (s *ProjectService) cleanupDBProjectsInternal(ctx context.Context, seen map[string]struct{}, followProjectSymlinks bool, projectsDir string, maxDepth int) error {- var all []models.Project- if err := s.db.WithContext(ctx).Find(&all).Error; err != nil {- return errors.WrapIf(err, "list projects for cleanup failed")- }-- // Decide deletions without performing them. Collecting decisions up front lets- // the mass-wipe guard veto an entire suspicious pass (e.g. the projects volume- // is unmounted, so every path is missing at once) before any rows are removed.- candidates := 0- pendingDeletions := make([]projectCleanupDecision, 0)- tempDeletions := make([]projectCleanupDecision, 0)- for _, p := range all {- if skipProjectCleanupInternal(p, seen) {- continue- }- if isInternalScratchProjectInternal(p) {- tempDeletions = append(tempDeletions, projectCleanupDecision{project: p, reason: "removed internal Arcane scratch record (project-update/gitops temp dir)"})- continue- }- // Projects inside filesystem snapshot/trash directories (e.g. BTRFS- // #snapshot) are point-in-time copies mistakenly registered by earlier- // discovery passes. The decision is name-based, not missing-path-based,- // so it bypasses the mass-wipe guard like the scratch records above.- if rel := s.getProjectRelativePathInternal(projectsDir, p.Path); rel != "" && projects.PathContainsSnapshotDirectory(rel) {- tempDeletions = append(tempDeletions, projectCleanupDecision{project: p, reason: "removed project inside a filesystem snapshot/trash directory"})- continue- }- candidates++- if decision, remove := s.evaluateProjectCleanupInternal(ctx, p, followProjectSymlinks, projectsDir, maxDepth).Get(); remove {- pendingDeletions = append(pendingDeletions, decision)- }- }-- for _, decision := range tempDeletions {- s.deleteProjectDuringCleanupInternal(ctx, decision.project, decision.reason)- }-- if s.cleanupWouldMassWipeInternal(ctx, candidates, len(pendingDeletions), projectsDir) {- return nil- }-- for _, decision := range pendingDeletions {- s.deleteProjectDuringCleanupInternal(ctx, decision.project, decision.reason)- }- return nil-}--// cleanupWouldMassWipeInternal reports whether the pending deletions look like an-// accidental mass wipe rather than legitimate removals. It engages when a single-// pass would prune more than one project AND more than half of the cleanup-// candidates — so the table cannot be near-emptied at once (e.g. when the projects-// directory is unmounted or mis-mapped and every path goes missing), no matter how-// few projects the deployment has. A single removal is always allowed: it is-// indistinguishable from a legitimate "deleted my only project" and is not a mass-// wipe. When the guard engages it logs a WARN pointing the operator at the likely-// volume/mount misconfiguration and the caller skips every deletion in the pass.-func (s *ProjectService) cleanupWouldMassWipeInternal(ctx context.Context, candidates, deleteCount int, projectsDir string) bool {- if deleteCount <= 1 || deleteCount*2 <= candidates {- return false- }-- slog.WarnContext(ctx,- "skipping project cleanup: this reconcile would delete most projects in a single pass, which usually means the projects directory is empty, unmounted, or mis-mapped; preserving DB records — check the projects volume is mounted and mapped correctly",- "wouldDelete", deleteCount,- "cleanupCandidates", candidates,- "projectsDir", projectsDir,- )- return true-}--func skipProjectCleanupInternal(p models.Project, seen map[string]struct{}) bool {- // Skip paths seen in this pass.- if _, ok := seen[p.Path]; ok {- return true- }-- // Skip projects whose lifecycle is owned by the gitops system. Their compose- // files may not exist on disk yet (e.g. during a sync or after an SSH/clone- // failure) and should never be deleted here.- return p.GitOpsManagedBy != nil && strings.TrimSpace(*p.GitOpsManagedBy) != ""-}--// isInternalScratchProjectInternal reports whether a project row was imported from-// one of Arcane's own scratch directories (project-update preview/backup, or GitOps-// sync-stage/backup). Such rows are never real user projects — a crash or restart-// mid-operation can leak the scratch dir, which the filesystem discovery then imports.-// They are force-removed during cleanup regardless of whether the dir still exists.-func isInternalScratchProjectInternal(p models.Project) bool {- if projects.IsInternalScratchDirName(p.Name) || projects.IsInternalScratchDirName(filepath.Base(p.Path)) {- return true- }- return p.DirName != nil && projects.IsInternalScratchDirName(*p.DirName)-}--// evaluateProjectCleanupInternal decides whether a project that was not seen in-// the current filesystem pass should be pruned. It performs only read-only checks-// (warning in place for the "keep" cases); the actual deletion is deferred to the-// caller so the mass-wipe guard can veto an entire suspicious pass.-func (s *ProjectService) evaluateProjectCleanupInternal(ctx context.Context, p models.Project, followProjectSymlinks bool, projectsDir string, maxDepth int) mo.Option[projectCleanupDecision] {- if s.projectExceedsScanDepthInternal(p, projectsDir, maxDepth) {- return mo.Some(projectCleanupDecision{project: p, reason: "removed project: directory is beyond the configured scan depth"})- }-- validDir, err := projects.IsProjectDirectoryPath(p.Path, followProjectSymlinks)- if err != nil {- return s.evaluateProjectPathErrorInternal(ctx, p, err)- }- if !validDir {- return mo.Some(projectCleanupDecision{project: p, reason: "removed project: path is no longer a valid project directory"})- }-- return s.evaluateProjectComposeFileInternal(ctx, p)-}--func (s *ProjectService) projectExceedsScanDepthInternal(p models.Project, projectsDir string, maxDepth int) bool {- // Remove projects that still exist on disk but now fall outside the configured- // scan depth (e.g. after PROJECT_SCAN_MAX_DEPTH was lowered). They are no- // longer discovered, so they must not linger in the list. Projects at the- // projects root or outside it (relativePath == "") are left to the on-disk- // validation below.- if maxDepth <= 0 {- return false- }-- rel := s.getProjectRelativePathInternal(projectsDir, p.Path)- return rel != "" && strings.Count(rel, "/")+1 > maxDepth-}--func (s *ProjectService) evaluateProjectPathErrorInternal(ctx context.Context, p models.Project, err error) mo.Option[projectCleanupDecision] {- if os.IsNotExist(err) {- return mo.Some(projectCleanupDecision{project: p, reason: "removed project: directory no longer exists"})- }-- slog.WarnContext(ctx, "stat error during cleanup; keeping DB record", "path", p.Path, "error", err)- return mo.None[projectCleanupDecision]()-}--func (s *ProjectService) evaluateProjectComposeFileInternal(ctx context.Context, p models.Project) mo.Option[projectCleanupDecision] {- _, err := s.resolveProjectComposeFileInternal(ctx, &p)- if err == nil {- return mo.None[projectCleanupDecision]()- }-- // The project directory still exists here (it passed the directory-validity- // check above). Only prune the DB record when the directory genuinely has no- // compose file. Any other resolution failure — an ambiguous match ("multiple- // custom compose files"), an unreadable directory, a transient parse error —- // means the project still has compose content on disk and may be deployable.- // Deleting it would silently destroy a live project whose files are intact, so- // keep the record and warn instead.- if !errors.Is(err, common.ErrComposeFileNotFound) {- slog.WarnContext(ctx, "project directory present but compose file unresolved during cleanup; keeping DB record",- "projectID", p.ID, "path", p.Path, "error", err)- return mo.None[projectCleanupDecision]()- }-- return mo.Some(projectCleanupDecision{project: p, reason: "removed orphaned project: directory present but contains no compose file"})-}--// deleteProjectDuringCleanupInternal removes a project record discovered to be-// stale during the filesystem reconcile. Every removal is logged (WARN on-// success, ERROR on failure) so this destructive operation always leaves an-// audit trail — previously successful deletions were silent.-func (s *ProjectService) deleteProjectDuringCleanupInternal(ctx context.Context, p models.Project, reason string, attrs ...any) {- logAttrs := make([]any, 0, 6+len(attrs))- logAttrs = append(logAttrs, "projectID", p.ID, "name", p.Name, "path", p.Path)- logAttrs = append(logAttrs, attrs...)-- if derr := s.db.WithContext(ctx).Delete(&models.Project{}, "id = ?", p.ID).Error; derr != nil {- slog.ErrorContext(ctx, "failed to delete project during filesystem cleanup",- append(logAttrs, "reason", reason, "error", derr)...)- return- }-- slog.WarnContext(ctx, reason, logAttrs...)-}--func (s *ProjectService) ListAllProjects(ctx context.Context) ([]models.Project, error) {- var items []models.Project- if err := s.db.WithContext(ctx).Find(&items).Error; err != nil {- return nil, errors.WrapIf(err, "list projects")- }- return items, nil-}--func formatPorts(publishers []api.PortPublisher) []string {- var ports []string- for _, pub := range publishers {- if pub.PublishedPort > 0 {- ports = append(ports, fmt.Sprintf("%d:%d/%s", pub.PublishedPort, pub.TargetPort, pub.Protocol))- } else {- ports = append(ports, fmt.Sprintf("%d/%s", pub.TargetPort, pub.Protocol))- }- }- return ports-}--func formatDockerPorts(ports []container.PortSummary) []string {- var res []string- for _, p := range ports {- if p.PublicPort == 0 {- res = append(res, fmt.Sprintf("%d/%s", p.PrivatePort, p.Type))- } else {- res = append(res, fmt.Sprintf("%d:%d/%s", p.PublicPort, p.PrivatePort, p.Type))- }- }- return res-}--func (s *ProjectService) countProjectFolders(ctx context.Context) (int, error) {- followProjectSymlinks := s.settingsService.GetBoolSetting(ctx, "followProjectSymlinks", false)- projectsDir, err := s.getProjectsDirectoryInternal(ctx)- if err != nil {- return 0, errors.WrapIf(err, "could not determine projects directory")- }-- info, statErr := os.Stat(projectsDir)- if os.IsNotExist(statErr) {- // Directory missing, treat as zero- return 0, nil- }- if statErr != nil {- return 0, errors.WrapIff(statErr, "unable to access projects directory %s", projectsDir)- }- if !info.IsDir() {- return 0, nil- }-- discoveredProjects, discoveryErr := projects.DiscoverProjectDirectories(projectsDir, followProjectSymlinks, s.config.ProjectScanMaxDepth)- if discoveryErr != nil {- return 0, errors.WrapIff(discoveryErr, "failed to discover project directories in %s", projectsDir)- }-- return len(discoveredProjects), nil-}--func (s *ProjectService) incrementStatusCounts(status models.ProjectStatus, running, stopped *int) {- switch status {- case models.ProjectStatusRunning, models.ProjectStatusPartiallyRunning, models.ProjectStatusDeploying, models.ProjectStatusRestarting:- *running++- case models.ProjectStatusStopped, models.ProjectStatusStopping:- *stopped++- case models.ProjectStatusUnknown:- // Don't count unknown- }-}--func (s *ProjectService) GetProjectStatusCounts(ctx context.Context) (folderCount, runningProjects, stoppedProjects, totalProjects, archivedProjects int, err error) {- folderCount, _ = s.countProjectFolders(ctx)-- var projectsList []models.Project- if err := s.db.WithContext(ctx).Find(&projectsList).Error; err != nil {- return folderCount, 0, 0, 0, 0, errors.WrapIf(err, "failed to list projects")- }-- totalProjects = len(projectsList)- runningProjects = 0- stoppedProjects = 0- activeProjects := make([]models.Project, 0, len(projectsList))- for _, p := range projectsList {- if p.IsArchived {- archivedProjects++- continue- }- activeProjects = append(activeProjects, p)- }-- // 1. Fetch all compose containers- containers, err := projects.ListGlobalComposeContainers(ctx)- if err != nil {- slog.ErrorContext(ctx, "Failed to list global compose containers for counts", "error", err)- // Fallback to DB status- for _, p := range activeProjects {- s.incrementStatusCounts(p.Status, &runningProjects, &stoppedProjects)- }- return folderCount, runningProjects, stoppedProjects, totalProjects, archivedProjects, nil- }-- // 2. Group by project- containersByProject := groupComposeContainersByProjectInternal(containers)-- // 3. Calculate status for each project- for _, p := range activeProjects {- projectContainers := lookupProjectContainers(p, containersByProject)-- // Convert to ProjectServiceInfo (minimal needed for calculateProjectStatus)- var services []ProjectServiceInfo- for _, c := range projectContainers {- services = append(services, ProjectServiceInfo{- Status: string(c.State),- })- }-- var status models.ProjectStatus- if len(services) == 0 {- status = models.ProjectStatusStopped- } else {- status = s.calculateProjectStatus(services)- }-- s.incrementStatusCounts(status, &runningProjects, &stoppedProjects)- }-- return folderCount, runningProjects, stoppedProjects, totalProjects, archivedProjects, nil-}--// End Helpers--// Project Actions--func ensureProjectMutableInternal(proj *models.Project) error {- if proj != nil && proj.IsArchived {- return common.Classify(common.ErrProjectArchived, errors.New("project is archived and must be unarchived before this action"))- }- return nil-}--func isProjectArchiveBlockedInternal(proj *models.Project) bool {- if proj == nil {- return false- }- if proj.RunningCount > 0 {- return true- }- switch proj.Status {- case models.ProjectStatusRunning, models.ProjectStatusPartiallyRunning, models.ProjectStatusDeploying, models.ProjectStatusRestarting:- return true- case models.ProjectStatusStopped, models.ProjectStatusUnknown, models.ProjectStatusStopping:- return false- default:- return false- }-}--func (s *ProjectService) ArchiveProject(ctx context.Context, projectID string, user models.User) error {- proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return err- }- if proj.IsArchived {- return nil- }- if isProjectArchiveBlockedInternal(proj) {- return common.Classify(common.ErrProjectMustBeStopped, errors.New("project must be stopped before archiving"))- }-- now := time.Now()- if err := s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", projectID).Updates(map[string]any{- "is_archived": true,- "archived_at": now,- }).Error; err != nil {- return errors.WrapIf(err, "failed to archive project")- }-- metadata := models.JSON{"action": "archived", "projectID": projectID, "projectName": proj.Name}- s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, projectID, proj.Name, user, metadata, "could not log project archive action")-- return nil-}--func (s *ProjectService) UnarchiveProject(ctx context.Context, projectID string, user models.User) error {- proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return err- }- if !proj.IsArchived {- return nil- }-- if err := s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", projectID).Updates(map[string]any{- "is_archived": false,- "archived_at": gorm.Expr("NULL"),- }).Error; err != nil {- return errors.WrapIf(err, "failed to unarchive project")- }-- metadata := models.JSON{"action": "unarchived", "projectID": projectID, "projectName": proj.Name}- s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, projectID, proj.Name, user, metadata, "could not log project unarchive action")-- return nil-}--// resolveRemoveOrphansInternal decides whether compose up should remove orphan containers.-// GitOps-managed projects always remove orphans so the running stack matches the-// tracked compose file. Non-GitOps callers may opt in per request via DeployOptions.-func resolveRemoveOrphansInternal(gitOpsManaged bool, options *project.DeployOptions) bool {- return gitOpsManaged || (options != nil && options.RemoveOrphans)-}--func (s *ProjectService) DeployProject(ctx context.Context, projectID string, user models.User, options *project.DeployOptions) error {- projectFromDb, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return errors.WrapIf(err, "failed to get project")- }- if err := ensureProjectMutableInternal(projectFromDb); err != nil {- return err- }-- resolvedPullPolicy := ""- forceRecreate := false- recreateVolumes := false- if options != nil {- resolvedPullPolicy = projects.NormalizeDeployPullPolicy(options.PullPolicy)- forceRecreate = options.ForceRecreate- recreateVolumes = options.RecreateVolumes- }- if resolvedPullPolicy == "" {- resolvedPullPolicy = projects.NormalizeDeployPullPolicy(s.settingsService.GetStringSetting(ctx, "defaultDeployPullPolicy", "missing"))- }- if resolvedPullPolicy == "" {- resolvedPullPolicy = "missing"- }-- if err := s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusDeploying); err != nil {- return errors.WrapIf(err, "failed to update project status to deploying")- }-- // Run any configured pre-deploy lifecycle hook before loading the compose- // project so hooks can produce files that compose then consumes (e.g.- // `sops -d secrets.enc.env > .env.runtime` for an `env_file: .env.runtime`- // service). A failed hook aborts the deploy.- if s.lifecycleService != nil {- if lerr := s.lifecycleService.RunPreDeploy(ctx, projectFromDb, user); lerr != nil {- s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)- return errors.WrapIf(lerr, "pre-deploy lifecycle hook failed")- }- }-- projectModel, _, derr := s.loadComposeProjectForProjectInternal(ctx, projectFromDb, nil)- if derr != nil {- s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)- return errors.WrapIff(derr, "failed to load compose project in %s", projectFromDb.Path)- }-- credentials, cerr := s.resolveRegistryCredentialsInternal(ctx)- if cerr != nil {- s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)- return errors.WrapIf(cerr, "resolve registry credentials")- }-- progressWriter, _ := ctx.Value(dockerutil.ProgressWriterKey{}).(io.Writer)- if perr := s.prepareProjectImagesForDeploy(ctx, projectID, projectModel, progressWriter, credentials, &user, resolvedPullPolicy); perr != nil {- s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)- return errors.WrapIf(perr, "failed to prepare project images for deploy")- }-- gitOpsManaged := projectFromDb.GitOpsManagedBy != nil && *projectFromDb.GitOpsManagedBy != ""- removeOrphans := resolveRemoveOrphansInternal(gitOpsManaged, options)-- slog.Info("starting compose up with health check support", "projectID", projectID, "projectName", projectModel.Name, "services", len(projectModel.Services), "removeOrphans", removeOrphans)- // Health/progress streaming (if any) is handled inside projects.ComposeUp via ctx.- if err := projects.ComposeUp(ctx, projectModel, nil, removeOrphans, forceRecreate, recreateVolumes, s.composeRegistryAuthConfigsInternal(ctx)); err != nil {- slog.Error("compose up failed", "projectName", projectModel.Name, "projectID", projectID, "error", err)- if containers, psErr := s.GetProjectServices(ctx, projectID); psErr == nil {- slog.Info("containers after failed deploy", "projectID", projectID, "containers", containers)- }- s.restoreProjectStatusAfterFailedDeployInternal(ctx, projectID)-- // Provide more helpful error messages- errMsg := err.Error()- if strings.Contains(errMsg, "timeout") || strings.Contains(errMsg, "context deadline exceeded") {- return errors.WrapIf(err, "deployment timed out - check if services with 'condition: service_healthy' have healthchecks defined")- }- return errors.WrapIf(err, "failed to deploy project")- }- slog.Info("compose up completed successfully", "projectID", projectID, "projectName", projectModel.Name)-- metadata := models.JSON{"action": "deploy", "projectID": projectID, "projectName": projectModel.Name}- s.logProjectEventInternal(ctx, models.EventTypeProjectDeploy, projectID, projectModel.Name, user, metadata, "could not log project deployment action")-- err = s.updateProjectStatusandCountsInternal(ctx, projectID, models.ProjectStatusRunning)- if err != nil {- slog.Error("failed to update project status and counts after deploy", "projectID", projectID, "error", err)- }- return err-}--func (s *ProjectService) DownProject(ctx context.Context, projectID string, user models.User) error {- projectFromDb, err := s.getMutableProjectInternal(ctx, projectID)- if err != nil {- return err- }-- proj, _, lerr := s.loadComposeProjectForProjectInternal(ctx, projectFromDb, nil)- if lerr != nil {- _ = s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusRunning)- return errors.WrapIf(lerr, "failed to load compose project")- }-- if err := s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusStopped); err != nil {- return errors.WrapIf(err, "failed to update project status to stopping")- }-- if err := projects.ComposeDown(ctx, proj, false); err != nil {- _ = s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusRunning)- return errors.WrapIf(err, "failed to bring down project")- }-- metadata := models.JSON{- "action": "down",- "projectID": projectID,- "projectName": projectFromDb.Name,- }- s.logProjectEventInternal(ctx, models.EventTypeProjectStop, projectID, projectFromDb.Name, user, metadata, "could not log project down action")-- return s.updateProjectStatusandCountsInternal(ctx, projectID, models.ProjectStatusStopped)-}--func (s *ProjectService) CreateProject(ctx context.Context, name, composeContent string, envContent *string, projectFiles []project.ProjectFileDraft, user models.User) (*models.Project, error) {- return s.createProjectInternal(ctx, name, composeContent, envContent, projectFiles, user, true)-}--// createProjectInternal creates a project's directory, files, and DB row. When-// allowNameSuffix is true a directory-name collision is resolved by appending-// "-N" (the interactive default). When false a collision returns-// projects.ErrProjectDirExists (wrapped) so GitOps creates fail loudly instead of-// minting runaway "-N" duplicate projects on a broken binding.-func (s *ProjectService) createProjectInternal(ctx context.Context, name, composeContent string, envContent *string, projectFiles []project.ProjectFileDraft, user models.User, allowNameSuffix bool) (*models.Project, error) {- // A top-level `name:` in the compose file is authoritative over the- // submitted project name.- if yamlName := projects.ComposeContentProjectName(composeContent); yamlName != "" {- name = yamlName- }- sanitized := projects.SanitizeProjectName(name)-- projectsDirectory, err := projects.GetProjectsDirectory(ctx, s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"))- if err != nil {- return nil, errors.WrapIf(err, "failed to get projects directory")- }-- basePath := filepath.Join(projectsDirectory, sanitized)- var projectPath, folderName string- if allowNameSuffix {- projectPath, folderName, err = projects.CreateUniqueDir(projectsDirectory, basePath, name, common.DirPerm)- } else {- projectPath, folderName, err = projects.CreateExactDir(projectsDirectory, basePath, name, common.DirPerm)- }- if err != nil {- return nil, errors.WrapIf(err, "failed to create project directory")- }-- proj := &models.Project{- Name: name,- DirName: &folderName,- Path: projectPath,- Status: models.ProjectStatusStopped,- ServiceCount: 0,- RunningCount: 0,- }-- if err := projects.ApplyProjectFileDrafts(projectPath, projectFiles, projects.ProjectFileApplyOptions{- MaxDepth: s.config.ProjectFileTreeMaxDepth,- SkipDirectories: s.config.ProjectScanSkipDirs,- ComposeFileName: projects.DefaultComposeFileName,- }); err != nil {- _ = os.RemoveAll(projectPath)- return nil, wrapProjectFileErrorInternal(err)- }-- // GitOps-originated creates (allowNameSuffix=false) tolerate not-yet-supplied- // ${VAR} references the same way single-file git sync updates do; interactive- // creates (allowNameSuffix=true) stay strict.- if err := s.validateComposeContentForUpdate(ctx, projectsDirectory, projectPath, name, composeContent, envContent, nil, "", !allowNameSuffix); err != nil {- _ = os.RemoveAll(projectPath)- return nil, errors.WrapIf(err, "invalid compose file")- }-- if err := projects.WriteProjectFiles(projectsDirectory, projectPath, composeContent, envContent); err != nil {- // Best-effort cleanup to restore pre-transaction behavior.- _ = os.RemoveAll(projectPath)- return nil, errors.WrapIf(err, "failed to save project files")- }-- if err := s.db.WithContext(ctx).Create(proj).Error; err != nil {- _ = os.RemoveAll(projectPath)- return nil, errors.WrapIf(err, "failed to create project")- }- s.refreshComposeProjectNameInternal(ctx, proj)- s.refreshProjectImageRefsInternal(ctx, proj)-- metadata := models.JSON{"action": "create", "projectID": proj.ID, "projectName": proj.Name, "path": projectPath}- s.logProjectEventInternal(ctx, models.EventTypeProjectCreate, proj.ID, proj.Name, user, metadata, "could not log project creation")-- return proj, nil-}--func (s *ProjectService) DestroyProject(ctx context.Context, projectID string, removeFiles bool, removeVolumes bool, user models.User) error {- slog.DebugContext(ctx, "DestroyProject service called",- "projectID", projectID,- "removeFiles", removeFiles,- "removeVolumes", removeVolumes,- "userID", user.ID,- "username", user.Username)-- proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return err- }-- slog.DebugContext(ctx, "Found project to destroy",- "projectName", proj.Name,- "projectPath", proj.Path)-- if err := s.DownProject(ctx, projectID, systemUser); err != nil {- slog.WarnContext(ctx, "failed to bring down project", "error", err)- }-- if removeVolumes {- if compProj, _, lerr := s.loadComposeProjectForProjectInternal(ctx, proj, nil); lerr == nil {- if derr := projects.ComposeDown(ctx, compProj, true); derr != nil {- slog.WarnContext(ctx, "failed to remove volumes", "error", derr)- }- } else {- slog.WarnContext(ctx, "failed to load compose project for volume removal", "error", lerr)- }- }-- if removeFiles {- slog.DebugContext(ctx, "Removing project files", "path", proj.Path)- if err := os.RemoveAll(proj.Path); err != nil {- slog.ErrorContext(ctx, "Failed to remove project files", "path", proj.Path, "error", err)- return errors.WrapIf(err, "failed to remove project files")- }- slog.InfoContext(ctx, "Project files removed successfully", "path", proj.Path)- }-- if err := s.db.WithContext(ctx).Delete(proj).Error; err != nil {- return errors.WrapIf(err, "failed to delete project from database")- }-- if !removeFiles {- if projectsDir, dirErr := s.getProjectsDirectoryInternal(ctx); dirErr != nil {- slog.WarnContext(ctx, "Failed to resolve projects directory for quarantine", "error", dirErr)- } else if projects.IsSafeSubdirectory(projectsDir, proj.Path) && filepath.Clean(projectsDir) != filepath.Clean(proj.Path) {- trashPath := filepath.Join(filepath.Dir(proj.Path), fmt.Sprintf("%s%s-%d", projects.ArcaneTrashPrefix, filepath.Base(proj.Path), time.Now().Unix()))- if err := os.Rename(proj.Path, trashPath); err != nil {- slog.WarnContext(ctx, "Failed to quarantine project files", "path", proj.Path, "trashPath", trashPath, "error", err)- } else {- slog.InfoContext(ctx, "Project files quarantined successfully", "path", proj.Path, "trashPath", trashPath)- }- }- }- s.invalidateComposeCacheInternal(projectID)-- metadata := models.JSON{"action": "destroy", "projectID": projectID, "projectName": proj.Name, "removeFiles": removeFiles, "removeVolumes": removeVolumes}- s.logProjectEventInternal(ctx, models.EventTypeProjectDelete, projectID, proj.Name, user, metadata, "could not log project destroy action")-- return nil-}--func (s *ProjectService) RedeployProject(ctx context.Context, projectID string, user models.User, options *project.DeployOptions) error {- proj, err := s.getMutableProjectInternal(ctx, projectID)- if err != nil {- return err- }-- disabled := s.projectRedeployDisabledInternal(ctx, *proj)- if disabled {- return errors.New("arcane cannot redeploy itself; use the system upgrade flow (Settings -> Updates) instead")- }-- progressWriter, _ := ctx.Value(dockerutil.ProgressWriterKey{}).(io.Writer)- if progressWriter == nil {- progressWriter = io.Discard- }-- credentials, cerr := s.resolveRegistryCredentialsInternal(ctx)- if cerr != nil {- slog.WarnContext(ctx, "failed to resolve registry credentials for redeploy pull", "error", cerr)- }- if err := s.PullProjectImages(ctx, projectID, progressWriter, user, credentials); err != nil {- slog.WarnContext(ctx, "failed to pull project images", "error", err)- }-- metadata := models.JSON{"action": "redeploy", "projectID": projectID, "projectName": proj.Name}- s.logProjectEventInternal(ctx, models.EventTypeProjectDeploy, projectID, proj.Name, user, metadata, "could not log project redeploy action")-- return s.DeployProject(ctx, projectID, user, options)-}--func (s *ProjectService) projectRedeployDisabledInternal(ctx context.Context, proj models.Project) bool {- containers, err := projects.ListGlobalComposeContainers(ctx)- if err != nil {- slog.WarnContext(ctx, "could not list compose containers to check self-redeploy guard; skipping guard", "error", err)- return false- }-- containersByProject := groupComposeContainersByProjectInternal(containers)-- currentContainerID, currentContainerErr := cgroup.CurrentContainerID()- for _, containerSummary := range lookupProjectContainers(proj, containersByProject) {- if labels.ShouldDisableArcaneServerRedeploy(containerSummary.Labels, containerSummary.ID, currentContainerID, currentContainerErr) {- return true- }- }-- return false-}--func (s *ProjectService) PullProjectImages(ctx context.Context, projectID string, progressWriter io.Writer, user models.User, credentials []containerregistry.Credential) error {- proj, err := s.getMutableProjectInternal(ctx, projectID)- if err != nil {- return err- }-- compProj, _, lerr := s.loadComposeProjectForProjectInternal(ctx, proj, nil)- if lerr != nil {- return errors.WrapIf(lerr, "failed to load compose project")- }-- images := map[string]struct{}{}- for _, svc := range compProj.Services {- if svc.Build == nil {- if img := strings.TrimSpace(svc.Image); img != "" {- images[img] = struct{}{}- }- }- // pre_start hook images and type:image volume sources are pulled from a- // registry even for build services — they cannot be built.- for _, img := range api.GetDependentImages(svc, compProj.Name) {- images[img] = struct{}{}- }- for _, vol := range svc.Volumes {- if vol.Type == composetypes.VolumeTypeImage && strings.TrimSpace(vol.Source) != "" {- images[strings.TrimSpace(vol.Source)] = struct{}{}- }- }- }-- for img := range images {- if err := s.pullAndReconcileImageInternal(ctx, img, progressWriter, user, credentials); err != nil {- return err- }- }- return nil-}--func (s *ProjectService) BuildProjectServices(ctx context.Context, projectID string, options ProjectBuildOptions, progressWriter io.Writer, user *models.User) error {- projectFromDb, err := s.getMutableProjectInternal(ctx, projectID)- if err != nil {- return err- }-- projectModel, _, derr := s.loadComposeProjectForProjectInternal(ctx, projectFromDb, nil)- if derr != nil {- return errors.WrapIff(derr, "failed to load compose project in %s", projectFromDb.Path)- }-- return s.buildProjectServicesInternal(ctx, projectID, projectModel, options, progressWriter, user)-}--// EnsureProjectImagesPresent checks all compose service images for the project and-// pulls based on service pull policy:-// - always/refresh: always pull-// - missing/if_not_present/default: pull only if local image is missing-// - never: never pull (fails early if image is missing locally)-func (s *ProjectService) EnsureProjectImagesPresent(ctx context.Context, projectID string, progressWriter io.Writer, user models.User, credentials []containerregistry.Credential) error {- proj, err := s.getMutableProjectInternal(ctx, projectID)- if err != nil {- return err- }-- compProj, _, lerr := s.loadComposeProjectForProjectInternal(ctx, proj, nil)- if lerr != nil {- return errors.WrapIf(lerr, "failed to load compose project")- }-- pullPlan := projects.BuildImagePullPlan(compProj)-- return s.ensureImagesPresent(ctx, pullPlan, progressWriter, credentials, user)-}--func (s *ProjectService) ensureImagesPresent(ctx context.Context, pullPlan map[string]projects.ImagePullMode, progressWriter io.Writer, credentials []containerregistry.Credential, user models.User) error {- for img, mode := range pullPlan {- exists, ierr := s.imageService.ImageExistsLocally(ctx, img)- if ierr != nil && mode != projects.ImagePullModeAlways {- slog.WarnContext(ctx, "failed to check local image existence", "image", img, "error", ierr)- // Non-fatal: attempt to pull to be safe- }-- if mode == projects.ImagePullModeNever {- if ierr != nil {- slog.WarnContext(ctx, "pull_policy is 'never' but image presence check failed; continuing without pull", "image", img, "error", ierr)- continue- }- if !exists {- return errors.Errorf("image %s is not available locally and pull_policy is 'never'", img)- }- slog.DebugContext(ctx, "pull_policy is 'never'; using local image without pull", "image", img)- continue- }-- if mode == projects.ImagePullModeIfMissing && exists {- slog.DebugContext(ctx, "image already present locally; skipping pull", "image", img)- continue- }-- if err := s.pullAndReconcileImageInternal(ctx, img, progressWriter, user, credentials); err != nil {- return err- }- }- return nil-}--func (s *ProjectService) prepareProjectImagesForDeploy(- ctx context.Context,- projectID string,- project *composetypes.Project,- progressWriter io.Writer,- credentials []containerregistry.Credential,- user *models.User,- pullPolicyOverride string,-) error {- if project == nil {- return nil- }-- for name, svc := range project.Services {- svc, imageName, updated := prepareDeployServiceConfig(projectID, project.Name, name, svc)- if updated {- project.Services[name] = svc- }-- if imageName == "" {- continue- }-- decision := projects.DecideDeployImageAction(svc, pullPolicyOverride)- if updated {- decision = projects.DeployImageDecision{Build: true}- }- if err := s.ensureDeployServiceImageReady(ctx, projectID, project, name, svc, imageName, decision, progressWriter, credentials, user); err != nil {- return err- }-- // pre_start hook images and type:image volume sources cannot be built:- // they follow the service's pull decision minus the build/fallback paths.- dependentDecision := projects.DeployImageDecision{PullIfMissing: true}- switch {- case decision.RequireLocalOnly:- dependentDecision = projects.DeployImageDecision{RequireLocalOnly: true}- case decision.PullAlways:- dependentDecision = projects.DeployImageDecision{PullAlways: true}- }- dependentImages := api.GetDependentImages(svc, project.Name)- for _, vol := range svc.Volumes {- if vol.Type == composetypes.VolumeTypeImage && strings.TrimSpace(vol.Source) != "" {- dependentImages = append(dependentImages, strings.TrimSpace(vol.Source))- }- }- for _, img := range dependentImages {- if err := s.ensureDeployServiceImageReady(ctx, projectID, project, name, svc, img, dependentDecision, progressWriter, credentials, user); err != nil {- return err- }- }- }-- return nil-}--func prepareDeployServiceConfig(projectID, projectName, serviceName string, svc composetypes.ServiceConfig) (composetypes.ServiceConfig, string, bool) {- if svc.Build == nil {- return svc, strings.TrimSpace(svc.Image), false- }-- resolvedImage, updatedSvc, updated := ensureServiceImage(projectID, projectName, serviceName, svc)- return updatedSvc, resolvedImage, updated-}--func shouldPullDeployImage(decision projects.DeployImageDecision, exists bool) bool {- return decision.PullAlways || (decision.PullIfMissing && !exists)-}--func (s *ProjectService) ensureDeployServiceImageReady(- ctx context.Context,- projectID string,- project *composetypes.Project,- serviceName string,- svc composetypes.ServiceConfig,- imageName string,- decision projects.DeployImageDecision,- progressWriter io.Writer,- credentials []containerregistry.Credential,- user *models.User,-) error {- if decision.Build {- return s.buildServiceImageForDeploy(ctx, projectID, project, serviceName, svc, progressWriter, user)- }-- exists, err := s.imageService.ImageExistsLocally(ctx, imageName)- if err != nil {- slog.WarnContext(ctx, "failed to check local image existence", "image", imageName, "error", err)- }-- if decision.RequireLocalOnly {- if !exists {- return errors.Errorf("image %s is not available locally and pull_policy is set to never", imageName)- }- return nil- }-- if !shouldPullDeployImage(decision, exists) {- return nil- }-- err = s.pullAndReconcileImageInternal(ctx, imageName, progressWriter, systemUser, credentials)- if err == nil {- return nil- }- if svc.Build != nil && decision.FallbackBuildOnPullFail {- slog.WarnContext(ctx, "image pull failed, falling back to build", "service", serviceName, "image", imageName, "error", err)- return s.buildServiceImageForDeploy(ctx, projectID, project, serviceName, svc, progressWriter, user)- }- return errors.WrapIff(err, "failed to pull image %s", imageName)-}--func (s *ProjectService) buildServiceImageForDeploy(- ctx context.Context,- projectID string,- project *composetypes.Project,- serviceName string,- svc composetypes.ServiceConfig,- progressWriter io.Writer,- user *models.User,-) error {- if s.buildService == nil {- return errors.Errorf("build service not available for service %s", serviceName)- }-- buildReq, updatedSvc, updated, err := s.prepareServiceBuildRequest(ctx, projectID, project, serviceName, svc, ProjectBuildOptions{})- if err != nil {- return err- }- if updated {- project.Services[serviceName] = updatedSvc- }-- if _, err := s.buildService.BuildImage(ctx, types.LocalDockerEnvironmentID, buildReq, progressWriter, serviceName, user); err != nil {- return err- }-- return nil-}--func normalizeBuildSelections(services []string) map[string]struct{} {- selected := map[string]struct{}{}- for _, name := range services {- name = strings.TrimSpace(name)- if name == "" {- continue- }- selected[name] = struct{}{}- }- return selected-}--func serviceSelected(selected map[string]struct{}, name string) bool {- if len(selected) == 0 {- return true- }- _, ok := selected[name]- return ok-}--func ensureServiceImage(projectID, projectName, serviceName string, svc composetypes.ServiceConfig) (string, composetypes.ServiceConfig, bool) {- imageName := strings.TrimSpace(svc.Image)- if imageName == "" {- imageName = projects.BuildLocalImageTag(projectID, projectName, serviceName)- svc.Image = imageName- return imageName, svc, true- }- return imageName, svc, false-}--func (s *ProjectService) resolveEffectiveBuildProvider(override string) string {- provider := strings.ToLower(strings.TrimSpace(override))- if provider != "" {- return provider- }-- if s.buildService != nil {- provider = strings.ToLower(strings.TrimSpace(s.buildService.BuildSettings().BuildProvider))- }-- if provider == "" {- provider = "local"- }-- return provider-}--func (s *ProjectService) prepareServiceBuildRequest(- ctx context.Context,- projectID string,- project *composetypes.Project,- serviceName string,- svc composetypes.ServiceConfig,- options ProjectBuildOptions,-) (buildtypes.BuildRequest, composetypes.ServiceConfig, bool, error) {- _ = ctx- imageName, updatedSvc, updated := ensureServiceImage(projectID, project.Name, serviceName, svc)- effectiveProvider := s.resolveEffectiveBuildProvider(options.Provider)-- if updated && effectiveProvider == "depot" {- return buildtypes.BuildRequest{}, updatedSvc, updated, errors.Errorf("service %s must define an image when using depot build provider", serviceName)- }- if updated && options.Push != nil && *options.Push {- return buildtypes.BuildRequest{}, updatedSvc, updated, errors.Errorf("service %s must define an image when push is enabled", serviceName)- }-- // The build context (and any absolute Dockerfile path) is read locally by- // Arcane — both the docker provider (`archive.TarWithOptions`) and the- // buildkit provider (`SolveOpt.LocalDirs`) stream the directory contents- // to the daemon from the Arcane process's own filesystem. It must- // therefore stay as a container path; translating it to the host path- // (which is what bind mount sources need) makes `os.Stat` fail because- // the host path doesn't exist inside the Arcane container. See #2314.- // For that reason the build context dir is deliberately left untranslated.- contextDir, err := projects.ResolveBuildContext(project.WorkingDir, updatedSvc, serviceName)- if err != nil {- return buildtypes.BuildRequest{}, updatedSvc, updated, err- }-- dockerfileInline := updatedSvc.Build.DockerfileInline- if strings.TrimSpace(updatedSvc.Build.Dockerfile) != "" && strings.TrimSpace(dockerfileInline) != "" {- return buildtypes.BuildRequest{}, updatedSvc, updated, errors.Errorf("service %s cannot define both dockerfile and dockerfile_inline", serviceName)- }-- dockerfilePath := ""- if strings.TrimSpace(dockerfileInline) == "" {- dockerfilePath = projects.ResolveDockerfilePath(updatedSvc)- }-- buildReq := buildtypes.BuildRequest{- ContextDir: contextDir,- Dockerfile: dockerfilePath,- DockerfileInline: dockerfileInline,- Tags: projects.MergeBuildTags(imageName, updatedSvc.Build.Tags),- Target: strings.TrimSpace(updatedSvc.Build.Target),- BuildArgs: projects.BuildArgsFromCompose(updatedSvc.Build.Args),- Labels: projects.LabelsFromCompose(updatedSvc.Build.Labels),- CacheFrom: append([]string(nil), updatedSvc.Build.CacheFrom...),- CacheTo: append([]string(nil), updatedSvc.Build.CacheTo...),- NoCache: updatedSvc.Build.NoCache,- Pull: updatedSvc.Build.Pull,- Network: strings.TrimSpace(updatedSvc.Build.Network),- Isolation: strings.TrimSpace(updatedSvc.Build.Isolation),- ShmSize: int64(updatedSvc.Build.ShmSize),- Ulimits: projects.UlimitsFromCompose(updatedSvc.Build.Ulimits),- Entitlements: append(- []string(nil),- updatedSvc.Build.Entitlements...,- ),- Privileged: updatedSvc.Build.Privileged,- ExtraHosts: updatedSvc.Build.ExtraHosts.AsList(":"),- Platforms: projects.BuildPlatformsFromCompose(updatedSvc),- Provider: effectiveProvider,- }- if options.Push != nil {- buildReq.Push = *options.Push- }- if options.Load != nil {- buildReq.Load = *options.Load- }-- return buildReq, updatedSvc, updated, nil-}--func (s *ProjectService) restoreProjectStatusAfterFailedDeployInternal(ctx context.Context, projectID string) {- services, err := s.GetProjectServices(ctx, projectID)- if err == nil {- serviceCount, runningCount := s.getServiceCounts(services)- status := s.calculateProjectStatus(services)- updateErr := s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", projectID).Updates(map[string]any{- "status": status,- "service_count": serviceCount,- "running_count": runningCount,- "updated_at": time.Now(),- }).Error- if updateErr == nil {- return- }- slog.WarnContext(ctx, "failed to restore project status after deploy failure", "projectID", projectID, "error", updateErr)- } else {- slog.WarnContext(ctx, "failed to inspect project services after deploy failure", "projectID", projectID, "error", err)- }-- if updateErr := s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusStopped); updateErr != nil {- slog.WarnContext(ctx, "failed to set stopped status after deploy failure", "projectID", projectID, "error", updateErr)- }-}--func (s *ProjectService) buildProjectServicesInternal(ctx context.Context, projectID string, project *composetypes.Project, options ProjectBuildOptions, progressWriter io.Writer, user *models.User) error {- if s.buildService == nil {- return nil- }- if project == nil {- return nil- }-- selected := normalizeBuildSelections(options.Services)-- buildCount := 0- for name, svc := range project.Services {- if svc.Build == nil {- continue- }- if !serviceSelected(selected, name) {- continue- }-- buildReq, updatedSvc, updated, err := s.prepareServiceBuildRequest(ctx, projectID, project, name, svc, options)- if err != nil {- return err- }- if updated {- project.Services[name] = updatedSvc- }-- buildCount++- if _, err := s.buildService.BuildImage(ctx, types.LocalDockerEnvironmentID, buildReq, progressWriter, name, user); err != nil {- return err- }- }-- if buildCount == 0 && len(selected) > 0 {- return errors.Errorf("no build-enabled services matched: %s", strings.Join(options.Services, ", "))- }-- return nil-}--func (s *ProjectService) RestartProject(ctx context.Context, projectID string, services []string, user models.User) error {- proj, err := s.getMutableProjectInternal(ctx, projectID)- if err != nil {- return err- }-- if err := s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusRestarting); err != nil {- return errors.WrapIf(err, "failed to update project status to restarting")- }-- // Get configured projects directory from settings- cfg := s.settingsService.GetSettingsOrDefaults(ctx)- projectsDirectory := s.getProjectsDirectoryOrDefaultInternal(ctx, cfg)-- var dockerClient *client.Client- if s.dockerService != nil {- dockerClient, _ = s.dockerService.GetClient(ctx)- }- pathMapper := projects.NewPathMapperForConfiguredDirectory(- ctx,- s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"),- "/app/data/projects",- dockerClient,- )-- compProj, _, lerr := projects.LoadComposeProjectFromDir(ctx, proj.Path, projects.NormalizeProjectName(proj.Name), projectsDirectory, utils.BoolOrDefault(cfg.AutoInjectEnv.Value, false), pathMapper)- if lerr != nil {- _ = s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusRunning)- return errors.WrapIf(lerr, "failed to load compose project")- }-- if err := projects.ComposeRestart(ctx, compProj, services); err != nil {- _ = s.updateProjectStatusInternal(ctx, projectID, models.ProjectStatusRunning)- return errors.WrapIf(err, "failed to restart project")- }-- metadata := models.JSON{- "action": "restart",- "projectID": projectID,- "projectName": proj.Name,- }- if len(services) > 0 {- metadata["services"] = append([]string(nil), services...)- }- s.logProjectEventInternal(ctx, models.EventTypeProjectStart, projectID, proj.Name, user, metadata, "could not log project restart action")-- return s.updateProjectStatusandCountsInternal(ctx, projectID, models.ProjectStatusRunning)-}--func (s *ProjectService) UpdateProject(ctx context.Context, projectID string, name *string, composeContent, envContent, overrideContent *string, fileTreeRevision *string, fileChanges []project.ProjectFileChange, user models.User) (*models.Project, error) {- proj, projectsDirectory, err := s.getProjectForUpdate(ctx, projectID)- if err != nil {- return nil, err- }-- name = resolveAuthoritativeProjectNameInternal(&proj, name, composeContent)- renameRequested := isProjectRenameRequestedInternal(&proj, name)- if err := s.recoverProjectRenameJournalForProjectInternal(ctx, projectID); err != nil {- if renameRequested {- return nil, err- }- slog.WarnContext(ctx, "project rename journal recovery failed before non-rename update; continuing", "projectID", projectID, "error", err)- } else {- proj, projectsDirectory, err = s.getProjectForUpdate(ctx, projectID)- if err != nil {- return nil, err- }- name = resolveAuthoritativeProjectNameInternal(&proj, name, composeContent)- }-- if err := ensureProjectMutableInternal(&proj); err != nil {- return nil, err- }- if len(fileChanges) > 0 && isGitOpsManagedProjectInternal(&proj) {- return nil, common.Classify(common.ErrProjectFileForbidden, errors.New("Forbidden project file path: GitOps-managed project files cannot be edited"))- }-- if err := s.ensureProjectStoppedForRenameInternal(ctx, &proj, name); err != nil {- return nil, err- }-- volumeMigration, err := s.prepareProjectRenameVolumeMigrationForUpdateInternal(ctx, &proj, name, projectsDirectory, composeContent, envContent, overrideContent, fileTreeRevision, fileChanges)- if err != nil {- return nil, err- }-- renameJournal := s.prepareProjectRenameJournalInternal(&proj, name, projectsDirectory, volumeMigration)-- backup, cleanupBackup, err := s.prepareProjectUpdateBackupInternal(ctx, projectsDirectory, proj.Path, composeContent, envContent, overrideContent, fileChanges)- if err != nil {- return nil, err- }- defer cleanupBackup()-- journalActive, err := s.startProjectRenameJournalInternal(ctx, renameJournal)- if err != nil {- return nil, err- }-- projectStateCommitted := false- if err := s.withProjectRenameRollback(ctx, &proj, &projectStateCommitted, func() error {- return s.applyProjectUpdateWithRenameJournalInternal(ctx, &proj, name, projectsDirectory, composeContent, envContent, overrideContent, fileTreeRevision, fileChanges, volumeMigration, renameJournal, &journalActive, &projectStateCommitted)- }); err != nil {- err = s.handleProjectUpdateFailureInternal(ctx, projectID, projectsDirectory, &proj, backup, &journalActive, projectStateCommitted, err)- return nil, err- }-- s.refreshProjectAfterContentUpdateInternal(ctx, &proj, composeContent, overrideContent, fileChanges)- s.logProjectUpdateEventInternal(ctx, &proj, composeContent, envContent, overrideContent, fileChanges, user)-- slog.InfoContext(ctx, "project updated", "projectID", proj.ID, "name", proj.Name)- return &proj, nil-}--// resolveAuthoritativeProjectNameInternal enforces that a top-level `name:` in-// the compose file is authoritative over the submitted project name. For-// name-only renames, it checks the compose file on disk so the lock can't be-// bypassed via the API.-func resolveAuthoritativeProjectNameInternal(proj *models.Project, name *string, composeContent *string) *string {- if composeContent != nil {- if yamlName := projects.ComposeContentProjectName(*composeContent); yamlName != "" {- return &yamlName- }- return name- }- if name != nil {- if onDiskCompose, _, readErr := projects.ReadProjectFiles(proj.Path, ""); readErr == nil {- if yamlName := projects.ComposeContentProjectName(onDiskCompose); yamlName != "" {- return &yamlName- }- }- }- return name-}--func (s *ProjectService) prepareProjectUpdateBackupInternal(ctx context.Context, projectsDirectory, projectPath string, composeContent, envContent, overrideContent *string, fileChanges []project.ProjectFileChange) (*projects.ProjectUpdateBackup, func(), error) {- if composeContent == nil && envContent == nil && overrideContent == nil && len(fileChanges) == 0 {- return nil, func() {}, nil- }-- scope := projectUpdateBackupScopeInternal(projectPath, composeContent, envContent, overrideContent, fileChanges)- if scope.IsEmpty() {- return nil, func() {}, nil- }-- backup, err := s.backupProjectDirectoryInternal(ctx, projectsDirectory, projectPath, scope)- if err != nil {- return nil, nil, err- }-- return backup, func() { _ = os.RemoveAll(backup.BackupDir) }, nil-}--func (s *ProjectService) startProjectRenameJournalInternal(ctx context.Context, journal *projectRenameJournalInternal) (bool, error) {- if journal == nil {- return false, nil- }- if err := s.writeProjectRenameJournalInternal(ctx, journal, projectRenameJournalPhaseStartedInternal); err != nil {- return false, err- }- return true, nil-}--func (s *ProjectService) applyProjectUpdateWithRenameJournalInternal(ctx context.Context, proj *models.Project, name *string, projectsDirectory string, composeContent, envContent, overrideContent *string, fileTreeRevision *string, fileChanges []project.ProjectFileChange, volumeMigration volumes.Migration, renameJournal *projectRenameJournalInternal, journalActive *bool, projectStateCommitted *bool) (err error) {- volumeMigrationApplied := false- defer func() {- stateCommitted := projectStateCommitted != nil && *projectStateCommitted- if err != nil && volumeMigrationApplied && !stateCommitted {- if rollbackErr := volumeMigration.Rollback(ctx); rollbackErr != nil {- err = stderrors.Join(err, errors.WrapIf(rollbackErr, "failed to rollback project volume rename"))- }- }- }()-- if err = s.applyProjectRenameIfNeeded(proj, name, projectsDirectory); err != nil {- return err- }- if err = s.persistProjectFileChanges(ctx, proj, fileTreeRevision, fileChanges); err != nil {- return err- }- if err = s.persistUpdatedProjectFiles(ctx, proj, projectsDirectory, composeContent, envContent, overrideContent); err != nil {- return err- }- if err = s.applyProjectVolumeMigrationForUpdateInternal(ctx, volumeMigration, renameJournal, &volumeMigrationApplied); err != nil {- return err- }- if err = s.saveProjectUpdateInternal(ctx, proj); err != nil {- return err- }- if projectStateCommitted != nil {- *projectStateCommitted = true- }- s.finalizeProjectRenameAfterCommitInternal(ctx, proj.ID, volumeMigration, renameJournal, journalActive)- return nil-}--func (s *ProjectService) applyProjectVolumeMigrationForUpdateInternal(ctx context.Context, volumeMigration volumes.Migration, renameJournal *projectRenameJournalInternal, applied *bool) error {- if volumeMigration == nil {- return nil- }- if err := volumeMigration.Apply(ctx); err != nil {- return errors.WrapIf(err, "failed to rename project volumes")- }- *applied = true- return s.writeProjectRenameJournalInternal(ctx, renameJournal, projectRenameJournalPhaseTargetsCopiedInternal)-}--func (s *ProjectService) saveProjectUpdateInternal(ctx context.Context, proj *models.Project) error {- tx := s.db.WithContext(ctx).Begin()- if tx.Error != nil {- return errors.WrapIf(tx.Error, "failed to start project update transaction")- }-- txCommitted := false- defer func() {- if !txCommitted {- _ = tx.Rollback().Error- }- }()-- if err := tx.Save(proj).Error; err != nil {- return errors.WrapIf(err, "failed to update project")- }- if err := tx.Commit().Error; err != nil {- return errors.WrapIf(err, "failed to commit project update")- }- txCommitted = true- return nil-}--func (s *ProjectService) finalizeProjectRenameAfterCommitInternal(ctx context.Context, projectID string, volumeMigration volumes.Migration, renameJournal *projectRenameJournalInternal, journalActive *bool) {- if renameJournal != nil {- if err := s.writeProjectRenameJournalInternal(ctx, renameJournal, projectRenameJournalPhaseProjectStateCommittedInternal); err != nil {- slog.WarnContext(ctx, "failed to mark project rename journal committed", "projectID", projectID, "error", err)- }- }-- if committer, ok := volumeMigration.(volumes.Committer); ok {- if err := committer.Commit(ctx); err != nil {- slog.WarnContext(ctx, "failed to clean up project source volumes after committed rename", "projectID", projectID, "error", err)- var cleanupErr *volumes.SourceCleanupError- if errors.As(err, &cleanupErr) {- if writeErr := s.writeProjectRenameJournalInternal(ctx, renameJournal, projectRenameJournalPhaseSourceCleanupPendingInternal); writeErr != nil {- slog.WarnContext(ctx, "failed to mark project rename source cleanup pending", "projectID", projectID, "error", writeErr)- }- }- return- } else if err := s.writeProjectRenameJournalInternal(ctx, renameJournal, projectRenameJournalPhaseOldVolumesRemovedInternal); err != nil {- slog.WarnContext(ctx, "failed to mark old project rename volumes removed", "projectID", projectID, "error", err)- }- }-- s.completeProjectRenameJournalForUpdateInternal(ctx, renameJournal, projectID, journalActive)-}--func (s *ProjectService) completeProjectRenameJournalForUpdateInternal(ctx context.Context, renameJournal *projectRenameJournalInternal, projectID string, journalActive *bool) {- if renameJournal == nil {- return- }- if clearErr := s.clearProjectRenameJournalInternal(ctx, projectID); clearErr != nil {- slog.WarnContext(ctx, "failed to clear project rename journal", "projectID", projectID, "error", clearErr)- return- }- *journalActive = false-}--func (s *ProjectService) handleProjectUpdateFailureInternal(ctx context.Context, projectID, projectsDirectory string, proj *models.Project, backup *projects.ProjectUpdateBackup, journalActive *bool, projectStateCommitted bool, err error) error {- if projectStateCommitted {- return err- }-- if backup != nil {- if restoreErr := s.restoreProjectDirectoryBackupInternal(ctx, projectsDirectory, proj.Path, backup); restoreErr != nil {- err = stderrors.Join(err, errors.WrapIf(restoreErr, "failed to restore project files after update failure"))- }- }- if *journalActive {- if recoverErr := s.recoverProjectRenameJournalForProjectInternal(ctx, projectID); recoverErr != nil {- err = stderrors.Join(err, errors.WrapIf(recoverErr, "project rename recovery failed"))- } else {- *journalActive = false- }- }- return err-}--func (s *ProjectService) logProjectUpdateEventInternal(ctx context.Context, proj *models.Project, composeContent, envContent, overrideContent *string, fileChanges []project.ProjectFileChange, user models.User) {- metadata := models.JSON{- "action": "update",- "projectID": proj.ID,- "projectName": proj.Name,- }- if composeContent != nil {- metadata["composeUpdated"] = true- }- if envContent != nil {- metadata["envUpdated"] = true- }- if overrideContent != nil {- metadata["overrideUpdated"] = true- }- if len(fileChanges) > 0 {- metadata["projectFilesUpdated"] = true- metadata["projectFileChangeCount"] = len(fileChanges)- }- s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, proj.ID, proj.Name, user, metadata, "could not log project update action")-}--func (s *ProjectService) refreshProjectAfterContentUpdateInternal(ctx context.Context, proj *models.Project, composeContent, overrideContent *string, fileChanges []project.ProjectFileChange) {- if composeContent == nil && overrideContent == nil && len(fileChanges) == 0 {- return- }-- s.refreshComposeProjectNameInternal(ctx, proj)- s.refreshProjectImageRefsInternal(ctx, proj)- if err := s.updateProjectStatusandCountsInternal(ctx, proj.ID, proj.Status); err != nil {- slog.WarnContext(ctx, "failed to update service counts after compose edit", "projectID", proj.ID, "error", err)- }-}--func (s *ProjectService) ApplyGitSyncProjectFiles(ctx context.Context, projectID string, composeContent string, gitEnvContent *string, gitOverrideContent *string, gitOverrideFileName string, user models.User) (*models.Project, error) {- proj, projectsDirectory, err := s.getProjectForUpdate(ctx, projectID)- if err != nil {- return nil, err- }- if err := ensureProjectMutableInternal(&proj); err != nil {- return nil, err- }-- envUpdate, err := s.prepareGitSyncEnvUpdateInternal(proj.Path, gitEnvContent)- if err != nil {- return nil, errors.WrapIf(err, "failed to resolve git env state")- }-- if err := s.validateComposeContentForUpdate(ctx, projectsDirectory, proj.Path, proj.Name, composeContent, envUpdate.effectiveContent, gitOverrideContent, gitOverrideFileName, true); err != nil {- return nil, errors.WrapIf(err, "invalid compose file")- }-- if err := projects.WriteComposeFile(projectsDirectory, proj.Path, composeContent); err != nil {- return nil, errors.WrapIf(err, "failed to save compose file")- }- if err := s.persistGitSyncEnvFilesInternal(proj.Path, projectsDirectory, envUpdate); err != nil {- return nil, errors.WrapIf(err, "failed to sync git env files")- }- if err := projects.WriteComposeOverrideFile(projectsDirectory, proj.Path, gitOverrideContent, gitOverrideFileName); err != nil {- return nil, errors.WrapIf(err, "failed to sync git override file")- }- if err := s.db.WithContext(ctx).Save(&proj).Error; err != nil {- return nil, errors.WrapIf(err, "failed to update project")- }- s.refreshComposeProjectNameInternal(ctx, &proj)- s.refreshProjectImageRefsInternal(ctx, &proj)-- // Recalculate service counts and status after compose file sync- if err := s.updateProjectStatusandCountsInternal(ctx, proj.ID, proj.Status); err != nil {- slog.WarnContext(ctx, "failed to update service counts after git sync", "projectID", proj.ID, "error", err)- }-- metadata := models.JSON{- "action": "git_sync_update",- "projectID": proj.ID,- "projectName": proj.Name,- "composeUpdated": true,- "envUpdated": gitEnvContent != nil,- "overrideUpdated": gitOverrideContent != nil,- }- if gitEnvContent == nil {- metadata["envSourceRemoved"] = true- }- s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, proj.ID, proj.Name, user, metadata, "could not log git sync project update action")-- return &proj, nil-}--func (s *ProjectService) getProjectForUpdate(ctx context.Context, projectID string) (models.Project, string, error) {- var proj models.Project- if err := s.db.WithContext(ctx).First(&proj, "id = ?", projectID).Error; err != nil {- if errors.Is(err, gorm.ErrRecordNotFound) {- return models.Project{}, "", errors.New("project not found")- }- return models.Project{}, "", errors.WrapIf(err, "failed to get project")- }-- projectsDirectory, err := projects.GetProjectsDirectory(ctx, s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"))- if err != nil {- return models.Project{}, "", errors.WrapIf(err, "failed to get projects directory")- }-- if err := s.ensureProjectPathUnderRoot(ctx, &proj, false); err != nil {- return models.Project{}, "", err- }-- return proj, projectsDirectory, nil-}--func (s *ProjectService) prepareProjectRenameVolumeMigrationForUpdateInternal(ctx context.Context, proj *models.Project, name *string, projectsDirectory string, composeContent, envContent, overrideContent *string, fileTreeRevision *string, fileChanges []project.ProjectFileChange) (volumes.Migration, error) {- if !isProjectRenameRequestedInternal(proj, name) {- return nil, nil- }- if err := requireProjectFileRevisionInternal(fileTreeRevision, fileChanges); err != nil {- return nil, err- }-- if composeContent == nil && envContent == nil && overrideContent == nil && len(fileChanges) == 0 {- return s.prepareProjectRenameVolumeMigrationInternal(ctx, proj, name)- }-- previewPath, err := os.MkdirTemp(projectsDirectory, ".project-update-preview-*")- if err != nil {- return nil, errors.WrapIf(err, "failed to create project update preview")- }- defer func() {- if removeErr := os.RemoveAll(previewPath); removeErr != nil {- slog.WarnContext(ctx, "failed to remove project update preview", "path", previewPath, "error", removeErr)- }- }()-- if err := projects.CopyDirectoryContents(proj.Path, previewPath, nil); err != nil {- return nil, errors.WrapIf(err, "failed to prepare project update preview")- }-- previewProject := *proj- previewProject.Path = previewPath- if err := s.applyProjectFileChangesInternal(ctx, &previewProject, "", fileChanges); err != nil {- return nil, errors.WrapIf(err, "failed to prepare project update preview")- }- if err := s.persistUpdatedProjectFiles(ctx, &previewProject, projectsDirectory, composeContent, envContent, overrideContent); err != nil {- return nil, errors.WrapIf(err, "failed to prepare project update preview")- }-- return s.prepareProjectRenameVolumeMigrationInternal(ctx, &previewProject, name)-}--func (s *ProjectService) prepareProjectRenameVolumeMigrationInternal(ctx context.Context, proj *models.Project, name *string) (volumes.Migration, error) {- oldComposeName, newComposeName, ok := projectRenameVolumeMigrationComposeNamesInternal(s, proj, name)- if !ok {- return nil, nil- }-- composeProject, _, err := s.loadComposeProjectForProjectInternal(ctx, proj, nil)- if err != nil {- if errors.Is(err, common.ErrProjectComposeFileNotFound) {- return nil, nil- }- return nil, errors.WrapIf(err, "failed to load compose project for volume rename")- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, errors.WrapIf(err, "failed to connect to Docker for volume rename")- }-- return volumes.PlanMigration(ctx, dockerClient, composeProject, oldComposeName, newComposeName)-}--func projectRenameVolumeMigrationComposeNamesInternal(s *ProjectService, proj *models.Project, name *string) (string, string, bool) {- if s == nil || s.dockerService == nil || proj == nil || name == nil {- return "", "", false- }-- newProjectName := strings.TrimSpace(*name)- if newProjectName == "" || proj.Name == newProjectName || proj.Status != models.ProjectStatusStopped {- return "", "", false- }-- oldComposeName := projects.NormalizeProjectName(proj.Name)- newComposeName := projects.NormalizeProjectName(newProjectName)- if oldComposeName == "" || newComposeName == "" || oldComposeName == newComposeName {- return "", "", false- }-- return oldComposeName, newComposeName, true-}--func isProjectRenameRequestedInternal(proj *models.Project, name *string) bool {- if proj == nil || name == nil {- return false- }- newName := strings.TrimSpace(*name)- return newName != "" && proj.Name != newName-}--func isGitOpsManagedProjectInternal(proj *models.Project) bool {- return proj != nil && proj.GitOpsManagedBy != nil && strings.TrimSpace(*proj.GitOpsManagedBy) != ""-}--func requireProjectFileRevisionInternal(fileTreeRevision *string, fileChanges []project.ProjectFileChange) error {- if len(fileChanges) == 0 {- return nil- }- if fileTreeRevision == nil || strings.TrimSpace(*fileTreeRevision) == "" {- return common.Classify(common.ErrProjectFileBadRequest, errors.New("Invalid project file request: file tree revision is required"))- }- return nil-}--func (s *ProjectService) persistProjectFileChanges(ctx context.Context, proj *models.Project, fileTreeRevision *string, fileChanges []project.ProjectFileChange) error {- if err := requireProjectFileRevisionInternal(fileTreeRevision, fileChanges); err != nil {- return err- }- expectedRevision := ""- if fileTreeRevision != nil {- expectedRevision = strings.TrimSpace(*fileTreeRevision)- }- return s.applyProjectFileChangesInternal(ctx, proj, expectedRevision, fileChanges)-}--func (s *ProjectService) applyProjectFileChangesInternal(ctx context.Context, proj *models.Project, expectedRevision string, fileChanges []project.ProjectFileChange) error {- if len(fileChanges) == 0 {- return nil- }- opts := s.projectFileApplyOptionsInternal(ctx, proj, expectedRevision)- if err := projects.ApplyProjectFileChanges(proj.Path, fileChanges, opts); err != nil {- return wrapProjectFileErrorInternal(err)- }- return nil-}--func (s *ProjectService) projectFileApplyOptionsInternal(ctx context.Context, proj *models.Project, expectedRevision string) projects.ProjectFileApplyOptions {- composeFileName := projects.DefaultComposeFileName- if composeFile, err := s.resolveProjectComposeFileInternal(ctx, proj); err == nil {- composeFileName = filepath.Base(composeFile)- }-- return projects.ProjectFileApplyOptions{- ExpectedRevision: strings.TrimSpace(expectedRevision),- MaxDepth: s.config.ProjectFileTreeMaxDepth,- MaxEntries: projects.DefaultProjectFileTreeMaxEntries,- SkipDirectories: s.config.ProjectScanSkipDirs,- ComposeFileName: composeFileName,- }-}--// wrapProjectFileErrorInternal maps pkg/projects sentinel errors onto the-// common.ProjectFile* error structs the handlers translate into HTTP statuses.-func wrapProjectFileErrorInternal(err error) error {- switch {- case errors.Is(err, projects.ErrProjectFileRevisionConflict):- return common.Classify(common.ErrProjectFileConflict, errors.WithStackIf(err))- case errors.Is(err, projects.ErrProjectFileOutsideProjectDirectory),- errors.Is(err, projects.ErrProjectFileProtectedPath),- errors.Is(err, projects.ErrProjectFileSymlinkPath):- return common.Classify(common.ErrProjectFileForbidden, errors.WrapIf(err, "Forbidden project file path"))-- default:- return common.Classify(common.ErrProjectFileBadRequest, errors.-- // projectUpdateBackupScopeInternal derives the exact set of paths an update- // can mutate so the backup never copies out-of-scope data directories.- // Changes that fail normalization are skipped: the apply step rejects them- // before mutating anything, so there is nothing to roll back for them.- WrapIf(err, "Invalid project file request"))- }-}--func projectUpdateBackupScopeInternal(projectPath string, composeContent, envContent, overrideContent *string, fileChanges []project.ProjectFileChange) projects.ProjectUpdateBackupScope {- scope := projects.ProjectUpdateBackupScope{- TopLevelFiles: composeContent != nil || envContent != nil || overrideContent != nil,- }-- for _, change := range fileChanges {- rel, err := projects.NormalizeProjectRelativePath(change.RelativePath)- if err != nil {- continue- }-- var dest string- switch change.Operation {- case project.FileOpRename:- newName, nameErr := projects.ValidateProjectFileName(change.NewName)- if nameErr != nil {- continue- }- dest = path.Join(path.Dir(rel), newName)- case project.FileOpMove:- parent := strings.TrimSpace(change.NewParentPath)- if parent != "" {- normalizedParent, parentErr := projects.NormalizeProjectRelativePath(parent)- if parentErr != nil {- continue- }- parent = normalizedParent- }- dest = path.Join(parent, path.Base(rel))- default:- scope.Paths = append(scope.Paths, rel)- continue- }-- // Rename/move of an existing directory rolls back via an inverse- // rename — never a copy of a potentially huge tree.- if info, statErr := os.Lstat(filepath.Join(projectPath, filepath.FromSlash(rel))); statErr == nil && info.IsDir() {- scope.RenamedDirs = append(scope.RenamedDirs, [2]string{rel, dest})- } else {- scope.Paths = append(scope.Paths, rel, dest)- }- }-- demoteInterferingRenamedDirsInternal(&scope)- return scope-}--// demoteInterferingRenamedDirsInternal downgrades a directory rename to a full-// copy when another change in the same batch touches its source or destination-// subtree (e.g. rename a -> b then delete b, or update_file b/x). The inverse-// rename alone cannot roll those back: the destination may be mutated or gone-// by the time the batch fails, so the original directory must be backed up.-func demoteInterferingRenamedDirsInternal(scope *projects.ProjectUpdateBackupScope) {- overlaps := func(a, b string) bool {- return a == b || strings.HasPrefix(a, b+"/") || strings.HasPrefix(b, a+"/")- }- touchesPair := func(p string, pair [2]string) bool {- return overlaps(p, pair[0]) || overlaps(p, pair[1])- }-- for i := 0; i < len(scope.RenamedDirs); i++ {- pair := scope.RenamedDirs[i]- conflict := slices.ContainsFunc(scope.Paths, func(p string) bool { return touchesPair(p, pair) })- if !conflict {- for j, other := range scope.RenamedDirs {- if j != i && (touchesPair(other[0], pair) || touchesPair(other[1], pair)) {- conflict = true- break- }- }- }- if conflict {- scope.Paths = append(scope.Paths, pair[0], pair[1])- scope.RenamedDirs = slices.Delete(scope.RenamedDirs, i, i+1)- i--- }- }-}--func (s *ProjectService) backupProjectDirectoryInternal(ctx context.Context, projectsDirectory, projectPath string, scope projects.ProjectUpdateBackupScope) (*projects.ProjectUpdateBackup, error) {- projectAbs, err := filepath.Abs(projectPath)- if err != nil {- return nil, errors.WrapIf(err, "failed to resolve project path")- }- projectAbs = filepath.Clean(projectAbs)-- rootAbs, err := filepath.Abs(projectsDirectory)- if err != nil {- return nil, errors.WrapIf(err, "failed to resolve projects directory")- }- rootAbs = filepath.Clean(rootAbs)- if !projects.IsSafeSubdirectory(rootAbs, projectAbs) || projectAbs == rootAbs {- return nil, errors.New("project path is outside projects directory")- }-- backupPath, err := os.MkdirTemp(projectsDirectory, ".project-update-backup-*")- if err != nil {- return nil, errors.WrapIf(err, "failed to create project backup directory")- }- // Tolerate files Arcane cannot read (e.g. foreign-owned secrets): skip them- // in the backup so an unrelated unreadable file can't block the whole save.- // The skipped paths are recorded so the rollback restore can preserve them.- backup, err := projects.BackupProjectUpdateScope(projectAbs, backupPath, scope)- if err != nil {- _ = os.RemoveAll(backupPath)- return nil, errors.WrapIf(err, "failed to backup project files")- }- if len(backup.Skipped) > 0 {- slog.WarnContext(ctx, "skipped unreadable files while backing up project; they will be left untouched on rollback", "projectPath", projectAbs, "skipped", backup.Skipped)- }- return backup, nil-}--func (s *ProjectService) restoreProjectDirectoryBackupInternal(ctx context.Context, projectsDirectory, projectPath string, backup *projects.ProjectUpdateBackup) error {- projectAbs, err := filepath.Abs(projectPath)- if err != nil {- return errors.WrapIf(err, "failed to resolve project path")- }- projectAbs = filepath.Clean(projectAbs)-- rootAbs, err := filepath.Abs(projectsDirectory)- if err != nil {- return errors.WrapIf(err, "failed to resolve projects directory")- }- rootAbs = filepath.Clean(rootAbs)- if !projects.IsSafeSubdirectory(rootAbs, projectAbs) || projectAbs == rootAbs {- return errors.New("project path is outside projects directory")- }-- slog.DebugContext(ctx, "restoring project directory backup", "path", projectAbs, "backup", backup.BackupDir)- if err := os.MkdirAll(projectAbs, common.DirPerm); err != nil {- return errors.WrapIf(err, "failed to recreate project directory")- }- // Restore only the paths the update could have mutated, in place: files- // that were skipped during backup (unreadable, e.g. foreign-owned secrets)- // are preserved, and out-of-scope files are never touched.- if err := projects.RestoreProjectUpdateBackup(projectAbs, backup); err != nil {- return errors.WrapIf(err, "failed to restore project backup")- }- return nil-}--func (s *ProjectService) withProjectRenameRollback(ctx context.Context, proj *models.Project, projectStateCommitted *bool, run func() error) error {- originalPath := proj.Path- originalDirName := proj.DirName-- if err := run(); err != nil {- if projectStateCommitted != nil && *projectStateCommitted {- return err- }- if proj.Path != originalPath {- if renameErr := os.Rename(proj.Path, originalPath); renameErr != nil {- slog.WarnContext(ctx, "failed to rollback project directory rename", "from", proj.Path, "to", originalPath, "error", renameErr)- return err- }- proj.Path = originalPath- proj.DirName = originalDirName- }- return err- }-- return nil-}--// existingOrDefaultOverrideNameInternal returns the base name of the project's-// on-disk override file, or the default override filename when the project has-// none yet.-func existingOrDefaultOverrideNameInternal(projectPath string) string {- if overridePath := projects.DetectComposeOverrideFile(projectPath); overridePath != "" {- return filepath.Base(overridePath)- }- return projects.DefaultComposeOverrideFileName-}--// resolveEffectiveOverrideForValidationInternal computes the override content and-// filename to merge into the base during validation for a requested override-// change: nil validates against the on-disk override (untouched); a blank request-// validates the post-delete state (no override), so a base that is only valid-// *with* its override fails deletion up front; a non-empty request validates the-// provided content under the existing or default override filename.-func resolveEffectiveOverrideForValidationInternal(projectPath string, overrideContent *string) (*string, string) {- switch {- case overrideContent == nil:- overridePath := projects.DetectComposeOverrideFile(projectPath)- if overridePath == "" {- return nil, ""- }- onDisk := projects.ReadComposeOverrideContent(projectPath)- return &onDisk, filepath.Base(overridePath)- case strings.TrimSpace(*overrideContent) == "":- return nil, ""- default:- return overrideContent, existingOrDefaultOverrideNameInternal(projectPath)- }-}--// applyOverrideFileChangeInternal writes or deletes the on-disk override per the-// requested change: nil leaves it untouched; a blank request removes every-// override candidate; a non-empty request writes it under the existing override-// filename, or the default name when the project has none yet.-func applyOverrideFileChangeInternal(projectsDirectory, projectPath string, overrideContent *string) error {- if overrideContent == nil {- return nil- }- if strings.TrimSpace(*overrideContent) == "" {- return projects.WriteComposeOverrideFile(projectsDirectory, projectPath, nil, "")- }- return projects.WriteComposeOverrideFile(projectsDirectory, projectPath, overrideContent, existingOrDefaultOverrideNameInternal(projectPath))-}--func (s *ProjectService) persistUpdatedProjectFiles(ctx context.Context, proj *models.Project, projectsDirectory string, composeContent, envContent, overrideContent *string) error {- switch {- case composeContent != nil:- effectiveEnvContent, err := s.resolveEffectiveEnvContentForUpdateInternal(proj.Path, envContent)- if err != nil {- return errors.WrapIf(err, "invalid compose file")- }- valOverride, valOverrideName := resolveEffectiveOverrideForValidationInternal(proj.Path, overrideContent)- if err := s.validateComposeContentForUpdate(ctx, projectsDirectory, proj.Path, proj.Name, *composeContent, effectiveEnvContent, valOverride, valOverrideName, false); err != nil {- return errors.WrapIf(err, "invalid compose file")- }- if err := projects.WriteComposeFile(projectsDirectory, proj.Path, *composeContent); err != nil {- return errors.WrapIf(err, "failed to save project files")- }- if envContent != nil {- if err := s.persistEffectiveEnvContentInternal(proj.Path, projectsDirectory, *envContent); err != nil {- return errors.WrapIf(err, "failed to save project files")- }- } else if err := s.ensureEffectiveEnvFileInternal(proj.Path, projectsDirectory); err != nil {- return errors.WrapIf(err, "failed to save project files")- }- if err := applyOverrideFileChangeInternal(projectsDirectory, proj.Path, overrideContent); err != nil {- return errors.WrapIf(err, "failed to save project files")- }- case overrideContent != nil:- if err := s.persistOverrideOnlyUpdateInternal(ctx, proj, projectsDirectory, envContent, overrideContent); err != nil {- return err- }- case envContent != nil:- if err := s.persistEffectiveEnvContentInternal(proj.Path, projectsDirectory, *envContent); err != nil {- return err- }- }-- return nil-}--// persistOverrideOnlyUpdateInternal handles a save that changes the override (and-// optionally the env) without touching the base compose file. It validates the-// on-disk base merged with the requested override so a base that is only valid-// *with* its override still validates, and a delete that would break the base-// fails before touching disk.-func (s *ProjectService) persistOverrideOnlyUpdateInternal(ctx context.Context, proj *models.Project, projectsDirectory string, envContent, overrideContent *string) error {- baseContent, _, err := projects.ReadProjectFiles(proj.Path, "")- if err != nil {- return errors.WrapIf(err, "failed to read project files")- }- effectiveEnvContent, err := s.resolveEffectiveEnvContentForUpdateInternal(proj.Path, envContent)- if err != nil {- return errors.WrapIf(err, "invalid compose file")- }- valOverride, valOverrideName := resolveEffectiveOverrideForValidationInternal(proj.Path, overrideContent)- if err := s.validateComposeContentForUpdate(ctx, projectsDirectory, proj.Path, proj.Name, baseContent, effectiveEnvContent, valOverride, valOverrideName, false); err != nil {- return errors.WrapIf(err, "invalid compose file")- }- if envContent != nil {- if err := s.persistEffectiveEnvContentInternal(proj.Path, projectsDirectory, *envContent); err != nil {- return errors.WrapIf(err, "failed to save project files")- }- }- if err := applyOverrideFileChangeInternal(projectsDirectory, proj.Path, overrideContent); err != nil {- return errors.WrapIf(err, "failed to save project files")- }- return nil-}--func (s *ProjectService) validateComposeContentForUpdate(ctx context.Context, projectsDirectory, projectPath, projectName, composeContent string, effectiveEnvContent *string, overrideContent *string, overrideFileName string, lenient bool) (err error) {- defer func() {- if panicErr := emperror.Recover(recover()); panicErr != nil {- err = errors.WrapIf(panicErr, "compose file contains invalid syntax")- }- }()-- fullEnvMap, envErr := buildComposeValidationEnvironment(projectsDirectory, projectPath, effectiveEnvContent)- if envErr != nil {- return envErr- }-- if err := validateComposeIncludePathsForProjectInternal(projectPath, composeContent, fullEnvMap); err != nil {- return err- }-- validationProjectName := projects.NormalizeProjectName(projectName)- configFiles := []composetypes.ConfigFile{- {Filename: filepath.Join(projectPath, "compose.yaml"), Content: []byte(composeContent)},- }- // When an override is supplied, validate the *merged* config as `docker- // compose` would deploy it. Overrides can add services and introduce their- // own include: paths, so they get the same traversal check as the base and- // are layered on top (listed after the base so the override wins).- if overrideContent != nil {- if err := validateComposeIncludePathsForProjectInternal(projectPath, *overrideContent, fullEnvMap); err != nil {- return err- }- overrideName := strings.TrimSpace(overrideFileName)- if overrideName == "" {- overrideName = projects.DefaultComposeOverrideFileName- }- configFiles = append(configFiles, composetypes.ConfigFile{- Filename: filepath.Join(projectPath, overrideName),- Content: []byte(*overrideContent),- })- }-- cfg := composetypes.ConfigDetails{- Version: api.ComposeVersion,- WorkingDir: projectPath,- ConfigFiles: configFiles,- Environment: composetypes.Mapping(fullEnvMap),- }-- missingIncludeLoader := projects.NewMissingIncludeStubLoader(projectPath)- defer missingIncludeLoader.Cleanup()-- err = projects.WithTransientValidationEnvFile(projectPath, effectiveEnvContent, func() error {- _, loadErr := loader.LoadWithContext(ctx, cfg, func(opts *loader.Options) {- opts.ResourceLoaders = append([]loader.ResourceLoader{missingIncludeLoader}, opts.ResourceLoaders...)- if validationProjectName != "" {- opts.SetProjectName(validationProjectName, false)- }- if lenient {- projects.ApplyLenientLoaderOptions(ctx, opts, cfg.ConfigFiles[0].Filename)- }- })- return loadErr- })-- return err-}--func validateComposeIncludePathsForProjectInternal(projectPath, composeContent string, envMap projects.EnvMap) error {- includes, err := projects.ParseIncludesFromContent(filepath.Join(projectPath, "compose.yaml"), []byte(composeContent), envMap, false)- if err != nil {- return err- }- for _, inc := range includes {- if _, err := projects.ValidateIncludePathForWrite(projectPath, inc.Path); err != nil {- return errors.WrapIff(err, "include path %q is outside project directory", inc.RelativePath)- }- }- return nil-}--func buildComposeValidationEnvironment(projectsDirectory, projectPath string, effectiveEnvContent *string) (projects.EnvMap, error) {- // Validation should match project-visible env sources without inheriting the- // Arcane process environment, which may contain unrelated secrets.- fullEnvMap := make(projects.EnvMap)- if absWorkdir, absErr := filepath.Abs(projectPath); absErr == nil {- fullEnvMap["PWD"] = absWorkdir- } else {- fullEnvMap["PWD"] = projectPath- }-- globalEnvPath := filepath.Join(projectsDirectory, projects.GlobalEnvFileName)- globalEnv, err := parseComposeValidationEnvFile(globalEnvPath, fullEnvMap)- if err != nil {- return nil, errors.WrapIf(err, "parse global env file")- }- maps.Copy(fullEnvMap, globalEnv)-- if effectiveEnvContent != nil {- projectEnv, err := parseComposeValidationEnvContent(*effectiveEnvContent, fullEnvMap)- if err != nil {- return nil, errors.WrapIf(err, "parse provided env content")- }- maps.Copy(fullEnvMap, projectEnv)- return fullEnvMap, nil- }-- projectEnvPath := filepath.Join(projectPath, ".env")- projectEnv, err := parseComposeValidationEnvFile(projectEnvPath, fullEnvMap)- if err != nil {- return nil, errors.WrapIf(err, "parse project env file")- }- maps.Copy(fullEnvMap, projectEnv)-- return fullEnvMap, nil-}--func parseComposeValidationEnvFile(path string, contextEnv projects.EnvMap) (projects.EnvMap, error) {- if _, err := os.Stat(path); err != nil {- if os.IsNotExist(err) {- return nil, nil- }- return nil, errors.WrapIf(err, "stat file")- }-- content, err := os.ReadFile(path)- if err != nil {- return nil, errors.WrapIf(err, "read file")- }-- return parseComposeValidationEnvContent(string(content), contextEnv)-}--func parseComposeValidationEnvContent(content string, contextEnv projects.EnvMap) (projects.EnvMap, error) {- lookupFn := func(key string) (string, bool) {- value, ok := contextEnv[key]- return value, ok- }-- envMap, err := dotenv.ParseWithLookup(strings.NewReader(content), lookupFn)- if err != nil {- return nil, errors.WrapIf(err, "parse env")- }-- return envMap, nil-}--func (s *ProjectService) resolveEffectiveEnvContentForUpdateInternal(projectPath string, envContent *string) (*string, error) {- if envContent != nil {- return envContent, nil- }-- state, err := projects.ReadProjectEnvState(projectPath)- if err != nil {- return nil, errors.WrapIf(err, "read project env state")- }-- effectiveContent, err := s.resolveStoredEffectiveEnvContentInternal(state)- if err != nil {- return nil, err- }- if effectiveContent == "" && !state.HasEffective && !state.HasGitSource && !state.HasOverride {- return nil, nil- }-- return &effectiveContent, nil-}--func (s *ProjectService) resolveStoredEffectiveEnvContentInternal(state projects.ProjectEnvState) (string, error) {- if state.HasEffective {- return state.EffectiveContent, nil- }- if state.HasGitSource || state.HasOverride {- effectiveContent, err := projects.BuildEffectiveEnvContent(state.GitContent, state.OverrideContent)- if err != nil {- return "", errors.WrapIf(err, "build effective env content")- }- return effectiveContent, nil- }- return state.DirectContent, nil-}--func (s *ProjectService) persistEffectiveEnvContentInternal(projectPath, projectsDirectory, envContent string) error {- state, err := projects.ReadProjectEnvState(projectPath)- if err != nil {- return errors.WrapIf(err, "read project env state")- }-- if state.HasGitSource && state.HasEffective && envContent == state.EffectiveContent {- storedEffectiveContent, buildErr := projects.BuildEffectiveEnvContent(state.GitContent, state.OverrideContent)- if buildErr == nil && envContent == storedEffectiveContent {- return nil- }- }-- if !state.HasGitSource {- if state.HasOverride {- if err := projects.RemoveProjectFile(projectsDirectory, projectPath, projects.OverrideEnvFileName); err != nil {- return err- }- }- return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, state.EffectiveUnreadable, envContent)- }-- overrideContent, err := projects.BuildOverrideEnvContent(state.GitContent, envContent)- if err != nil {- return errors.WrapIf(err, "build override env content")- }-- effectiveContent, err := projects.BuildEffectiveEnvContent(state.GitContent, overrideContent)- if err != nil {- return errors.WrapIf(err, "build effective env content")- }-- if err := projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, state.EffectiveUnreadable, effectiveContent); err != nil {- return err- }-- return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.OverrideEnvFileName, state.OverrideUnreadable, overrideContent)-}--func (s *ProjectService) ensureEffectiveEnvFileInternal(projectPath, projectsDirectory string) error {- state, err := projects.ReadProjectEnvState(projectPath)- if err != nil {- return errors.WrapIf(err, "read project env state")- }-- if !state.HasGitSource {- if state.HasOverride {- if err := projects.RemoveProjectFile(projectsDirectory, projectPath, projects.OverrideEnvFileName); err != nil {- return err- }- effectiveContent, err := s.resolveStoredEffectiveEnvContentInternal(state)- if err != nil {- return err- }- return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, state.EffectiveUnreadable, effectiveContent)- }- return projects.EnsureEnvFile(projectsDirectory, projectPath)- }-- effectiveContent, err := projects.BuildEffectiveEnvContent(state.GitContent, state.OverrideContent)- if err != nil {- return errors.WrapIf(err, "build effective env content")- }-- return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, state.EffectiveUnreadable, effectiveContent)-}--type gitSyncEnvUpdateInternal struct {- state projects.ProjectEnvState- gitEnvContent *string- overrideContent string- effectiveContent *string-}--func (s *ProjectService) prepareGitSyncEnvUpdateInternal(projectPath string, gitEnvContent *string) (gitSyncEnvUpdateInternal, error) {- state, err := projects.ReadProjectEnvState(projectPath)- if err != nil {- return gitSyncEnvUpdateInternal{}, errors.WrapIf(err, "read project env state")- }-- update := gitSyncEnvUpdateInternal{- state: state,- gitEnvContent: gitEnvContent,- }-- if gitEnvContent == nil {- effectiveContent, err := s.resolveStoredEffectiveEnvContentInternal(state)- if err != nil {- return gitSyncEnvUpdateInternal{}, err- }- if effectiveContent == "" && !state.HasEffective && !state.HasGitSource && !state.HasOverride {- return update, nil- }- update.effectiveContent = &effectiveContent- return update, nil- }-- overrideContent, err := s.resolveOverrideContentForGitSyncInternal(state, *gitEnvContent)- if err != nil {- return gitSyncEnvUpdateInternal{}, err- }- update.overrideContent = overrideContent-- effectiveContent, err := projects.BuildEffectiveEnvContent(*gitEnvContent, overrideContent)- if err != nil {- return gitSyncEnvUpdateInternal{}, errors.WrapIf(err, "build effective env content")- }- update.effectiveContent = &effectiveContent-- return update, nil-}--func (s *ProjectService) resolveOverrideContentForGitSyncInternal(state projects.ProjectEnvState, gitEnvContent string) (string, error) {- switch {- case state.HasGitSource:- overrideContent, err := projects.BuildOverrideEnvContent(state.GitContent, state.OverrideContent)- if err != nil {- return "", errors.WrapIf(err, "build override env content")- }- return overrideContent, nil- case state.HasOverride:- effectiveContent, err := s.resolveStoredEffectiveEnvContentInternal(state)- if err != nil {- return "", err- }- overrideContent, err := projects.BuildOverrideEnvContent(gitEnvContent, effectiveContent)- if err != nil {- return "", errors.WrapIf(err, "build override env content")- }- return overrideContent, nil- case strings.TrimSpace(state.DirectContent) != "":- overrideContent, err := projects.BuildAdditiveOverrideEnvContent(gitEnvContent, state.DirectContent)- if err != nil {- return "", errors.WrapIf(err, "build override env content")- }- return overrideContent, nil- default:- return "", nil- }-}--func (s *ProjectService) persistGitSyncEnvFilesInternal(projectPath, projectsDirectory string, update gitSyncEnvUpdateInternal) error {- if update.gitEnvContent == nil {- if update.state.HasGitSource {- if err := projects.RemoveProjectFile(projectsDirectory, projectPath, projects.GitSourceEnvFileName); err != nil {- return err- }- }- if update.state.HasOverride {- if err := projects.RemoveProjectFile(projectsDirectory, projectPath, projects.OverrideEnvFileName); err != nil {- return err- }- }- if update.effectiveContent != nil || update.state.HasEffective || update.state.HasGitSource || update.state.HasOverride {- effectiveContent := ""- if update.effectiveContent != nil {- effectiveContent = *update.effectiveContent- }- return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, update.state.EffectiveUnreadable, effectiveContent)- }- if update.state.EffectiveUnreadable {- slog.Warn("skipping permission-locked .env file; leaving it untouched", "projectPath", projectPath)- return nil- }- return projects.EnsureEnvFile(projectsDirectory, projectPath)- }-- if update.effectiveContent == nil {- return errors.New("missing effective env content for git sync update")- }-- if err := projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.EffectiveEnvFileName, update.state.EffectiveUnreadable, *update.effectiveContent); err != nil {- return err- }- if err := projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.GitSourceEnvFileName, update.state.GitSourceUnreadable, *update.gitEnvContent); err != nil {- return err- }- return projects.WriteManagedEnvFile(projectsDirectory, projectPath, projects.OverrideEnvFileName, update.state.OverrideUnreadable, update.overrideContent)-}--func (s *ProjectService) ensureProjectStoppedForRenameInternal(ctx context.Context, proj *models.Project, name *string) error {- if !isProjectRenameRequestedInternal(proj, name) {- return nil- }- if proj.Status != models.ProjectStatusStopped && proj.Status != models.ProjectStatusUnknown {- return errors.Errorf("project must be stopped before renaming (current status: %s)", proj.Status)- }-- services, err := s.GetProjectServices(ctx, proj.ID)- if err != nil {- slog.WarnContext(ctx, "failed to resolve project status before rename", "projectID", proj.ID, "error", err)- return errors.WrapIff(err, "project must be stopped before renaming (current status: %s): failed to verify live status", proj.Status)- }-- status := s.calculateProjectStatus(services)- if status != models.ProjectStatusStopped {- return errors.Errorf("project must be stopped before renaming (current status: %s)", status)- }-- serviceCount, runningCount := s.getServiceCounts(services)- proj.Status = models.ProjectStatusStopped- proj.StatusReason = nil- proj.ServiceCount = serviceCount- proj.RunningCount = runningCount- return nil-}--func (s *ProjectService) applyProjectRenameIfNeeded(proj *models.Project, name *string, projectsDirectory string) error {- if name == nil {- return nil- }-- newName := strings.TrimSpace(*name)- if newName == "" || proj.Name == newName {- return nil- }-- if proj.Status != models.ProjectStatusStopped {- return errors.Errorf("project must be stopped before renaming (current status: %s)", proj.Status)- }-- newDirName := projects.SanitizeProjectName(newName)- if newDirName == "" || strings.Trim(newDirName, "_") == "" {- return errors.New("invalid project name: results in empty directory name")- }-- currentPath := filepath.Clean(proj.Path)- targetPath := filepath.Clean(filepath.Join(projectsDirectory, newDirName))- if currentPath != targetPath {- if _, statErr := os.Stat(targetPath); statErr == nil {- return errors.Errorf("project directory already exists: %s", targetPath)- } else if !os.IsNotExist(statErr) {- return errors.WrapIf(statErr, "failed to check project directory rename target")- }-- if err := os.Rename(currentPath, targetPath); err != nil {- return errors.WrapIf(err, "failed to rename project directory")- }-- proj.Path = targetPath- }-- proj.DirName = &newDirName- proj.Name = newName- return nil-}--func (s *ProjectService) UpdateProjectIncludeFile(ctx context.Context, projectID, relativePath, content string, user models.User) error {- proj, err := s.getMutableProjectInternal(ctx, projectID)- if err != nil {- return err- }-- // Normalize and persist project path to ensure include writes occur under projects root- if err := s.ensureProjectPathUnderRoot(ctx, proj, true); err != nil {- return err- }-- if err := projects.WriteIncludeFile(proj.Path, relativePath, content); err != nil {- return errors.WrapIf(err, "failed to update include file")- }- s.refreshProjectImageRefsInternal(ctx, proj)-- // Recalculate service counts since include files can define services- if err := s.updateProjectStatusandCountsInternal(ctx, proj.ID, proj.Status); err != nil {- slog.WarnContext(ctx, "failed to update service counts after include file edit", "projectID", proj.ID, "error", err)- }-- metadata := models.JSON{- "action": "update_include",- "projectID": proj.ID,- "projectName": proj.Name,- "relativePath": relativePath,- }- s.logProjectEventInternal(ctx, models.EventTypeProjectUpdate, proj.ID, proj.Name, user, metadata, "could not log project include update action")-- slog.InfoContext(ctx, "project include file updated", "projectID", proj.ID, "file", relativePath)- return nil-}--// ensureProjectPathUnderRoot validates that the project's path is a safe subdirectory of the configured projects root.-// If not, it normalizes the path to `<projectsRoot>/<dirName or sanitized project name>`. When persist=true, it saves-// the updated project path to the database.-func (s *ProjectService) ensureProjectPathUnderRoot(ctx context.Context, proj *models.Project, persist bool) error {- projectsDirectory, err := projects.GetProjectsDirectory(ctx, s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"))- if err != nil {- return errors.WrapIf(err, "failed to get projects directory")- }-- rootAbs, _ := filepath.Abs(projectsDirectory)- rootAbs = filepath.Clean(rootAbs)-- projPathAbs := proj.Path- if abs, aerr := filepath.Abs(proj.Path); aerr == nil {- projPathAbs = filepath.Clean(abs)- }-- if projects.IsSafeSubdirectory(rootAbs, projPathAbs) {- return nil- }-- // Attempt to repair using known directory name or sanitized project name- dirName := mo.PointerToOption(proj.DirName).OrEmpty()- if strings.TrimSpace(dirName) == "" {- dirName = projects.SanitizeProjectName(proj.Name)- }- candidate := filepath.Join(projectsDirectory, dirName)-- slog.WarnContext(ctx, "Normalizing project path to projects root", "projectID", proj.ID, "oldPath", proj.Path, "newPath", candidate, "root", projectsDirectory)- proj.Path = filepath.Clean(candidate)-- if persist {- if saveErr := s.db.WithContext(ctx).Save(proj).Error; saveErr != nil {- slog.WarnContext(ctx, "failed to persist normalized project path", "error", saveErr)- }- }- return nil-}--func (s *ProjectService) StreamProjectLogs(ctx context.Context, projectID string, logsChan chan<- string, follow bool, tail, since string, timestamps bool) error {- proj, err := s.GetProjectFromDatabaseByID(ctx, projectID)- if err != nil {- return err- }-- pr, pw := io.Pipe()- defer func() { _ = pw.Close() }()-- done := make(chan error, 2)-- // Reader goroutine: forward lines to channel- go func() {- // Closing the read half unblocks any pending pw.Write in ComposeLogs.- // Without it, an abandoned tail (ctx cancel, or a >1MiB line tripping- // bufio.ErrTooLong) wedges the writer goroutine forever and this- // function never collects its second done value.- defer func() { _ = pr.CloseWithError(io.ErrClosedPipe) }()-- sc := bufio.NewScanner(pr)- sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)- for sc.Scan() {- select {- case <-ctx.Done():- done <- ctx.Err()- return- case logsChan <- sc.Text():- }- }- done <- sc.Err()- }()-- // Writer goroutine: compose logs -> pipe- go func() {- err := projects.ComposeLogs(ctx, projects.NormalizeProjectName(proj.Name), pw, follow, tail, since, timestamps)- _ = pw.Close()- done <- err- }()-- // Wait for both goroutines to finish to avoid sending on a closed channel- err1 := <-done- err2 := <-done-- for _, e := range []error{err1, err2} {- if e != nil && !errors.Is(e, io.EOF) && !errors.Is(e, context.Canceled) {- return e- }- }- return nil-}--// End Project Actions--// Table Functions--func (s *ProjectService) ListProjects(ctx context.Context, params pagination.QueryParams) ([]project.Details, pagination.Response, error) {- query := s.db.WithContext(ctx).Model(&models.Project{})- statusFilter := ""- updatesFilter := ""- archivedFilter := ""- if params.Filters != nil {- statusFilter = strings.TrimSpace(params.Filters["status"])- updatesFilter = strings.TrimSpace(params.Filters["updates"])- archivedFilter = strings.TrimSpace(params.Filters["archived"])- }- query = applyProjectArchivedDBFilterInternal(query, archivedFilter)- if statusFilter != "" || updatesFilter != "" {- return s.listProjectsWithDerivedFiltersInternal(ctx, params, query)- }-- if term := strings.TrimSpace(params.Search); term != "" {- searchPattern := "%" + term + "%"- query = query.Where(- "name LIKE ? OR path LIKE ? OR status LIKE ? OR COALESCE(dir_name, '') LIKE ?",- searchPattern, searchPattern, searchPattern, searchPattern,- )- }-- query = pagination.ApplyFilter(query, "status", params.Filters["status"])-- var projectsArray []models.Project- paginationResp, err := pagination.PaginateAndSortDB(params, query, &projectsArray)- if err != nil {- return nil, pagination.Response{}, errors.WrapIf(err, "failed to paginate projects")- }-- slog.DebugContext(ctx, "Retrieved projects from database",- "count", len(projectsArray))-- // Fetch live status concurrently for all projects- result := s.fetchProjectStatusConcurrently(ctx, projectsArray)- s.enrichProjectsWithUpdateInfoInternal(ctx, projectsArray, result)-- slog.DebugContext(ctx, "Completed ListProjects request",- "result_count", len(result))-- return result, paginationResp, nil-}--func applyProjectArchivedDBFilterInternal(query *gorm.DB, filterValue string) *gorm.DB {- switch strings.ToLower(strings.TrimSpace(filterValue)) {- case "true":- return query.Where("is_archived = ?", true)- case "all":- return query- default:- return query.Where("is_archived = ?", false)- }-}--func (s *ProjectService) listProjectsWithDerivedFiltersInternal(- ctx context.Context,- params pagination.QueryParams,- query *gorm.DB,-) ([]project.Details, pagination.Response, error) {- limit := params.Limit- switch {- case limit == -1:- // Public API contract: exact -1 means "all" (used by the table page-size selector).- case limit <= 0:- limit = 20- case limit > 100:- limit = 100- }- params.Limit = limit-- result, err := s.filterProjectsWithDerivedFiltersInternal(ctx, params, query)- if err != nil {- return nil, pagination.Response{}, err- }- paginationResp := pagination.BuildResponseFromFilterResult(result, params)-- return result.Items, paginationResp, nil-}--func (s *ProjectService) filterProjectsWithDerivedFiltersInternal(- ctx context.Context,- params pagination.QueryParams,- query *gorm.DB,-) (pagination.FilterResult[project.Details], error) {- var projectsArray []models.Project- if term := strings.TrimSpace(params.Search); term != "" {- searchPattern := "%" + term + "%"- query = query.Where(- "name LIKE ? OR path LIKE ? OR status LIKE ? OR COALESCE(dir_name, '') LIKE ?",- searchPattern, searchPattern, searchPattern, searchPattern,- )- }- if err := query.Find(&projectsArray).Error; err != nil {- return pagination.FilterResult[project.Details]{}, errors.WrapIf(err, "failed to list projects")- }-- items := s.fetchProjectStatusConcurrently(ctx, projectsArray)- s.enrichProjectsWithUpdateInfoInternal(ctx, projectsArray, items)- items = s.appendDiscoveredComposeProjectUpdatesInternal(ctx, params, projectsArray, items)-- return pagination.SearchOrderAndPaginate(items, params, s.buildProjectDerivedPaginationConfigInternal()), nil-}--func (s *ProjectService) appendDiscoveredComposeProjectUpdatesInternal(- ctx context.Context,- params pagination.QueryParams,- projectsArray []models.Project,- items []project.Details,-) []project.Details {- if !shouldIncludeDiscoveredComposeProjectUpdatesInternal(params) {- return items- }-- composeContainers, err := projects.ListGlobalComposeContainers(ctx)- if err != nil {- slog.WarnContext(ctx, "failed to list compose containers for project update rows", "error", err)- return items- }-- knownProjectNames := s.buildKnownComposeProjectNameSetInternal(ctx, projectsArray)- discovered := buildDiscoveredComposeProjectUpdateRowsInternal(ctx, composeContainers, knownProjectNames, s.imageService, iconCatalogForContextInternal(ctx))- if len(discovered) == 0 {- return items- }-- return append(items, discovered...)-}--func shouldIncludeDiscoveredComposeProjectUpdatesInternal(params pagination.QueryParams) bool {- if params.Filters == nil {- return false- }-- return strings.EqualFold(strings.TrimSpace(params.Filters["updates"]), "has_update")-}--func (s *ProjectService) buildKnownComposeProjectNameSetInternal(ctx context.Context, projectsArray []models.Project) map[string]struct{} {- known := make(map[string]struct{}, len(projectsArray)*2)- for _, proj := range projectsArray {- addKnownComposeProjectNameInternal(known, proj.Name)- if proj.ComposeProjectName != nil {- addKnownComposeProjectNameInternal(known, *proj.ComposeProjectName)- }- }-- if s.db == nil {- return known- }-- var allProjects []models.Project- if err := s.db.WithContext(ctx).Select("name", "compose_project_name").Find(&allProjects).Error; err != nil {- slog.WarnContext(ctx, "failed to load known project names for compose update discovery", "error", err)- return known- }-- for _, proj := range allProjects {- addKnownComposeProjectNameInternal(known, proj.Name)- if proj.ComposeProjectName != nil {- addKnownComposeProjectNameInternal(known, *proj.ComposeProjectName)- }- }-- return known-}--func addKnownComposeProjectNameInternal(known map[string]struct{}, name string) {- name = strings.TrimSpace(name)- if name == "" {- return- }-- known[name] = struct{}{}- if normalized := projects.NormalizeProjectName(name); normalized != "" {- known[normalized] = struct{}{}- }-}--func buildDiscoveredComposeProjectUpdateRowsInternal(- ctx context.Context,- composeContainers []container.Summary,- knownProjectNames map[string]struct{},- imageService *ImageService,- iconCatalog string,-) []project.Details {- containersByProject := make(map[string][]container.Summary)- for _, c := range composeContainers {- projectName := dockerutil.ComposeProjectLabel(c.Labels)- if projectName == "" {- continue- }- if _, exists := knownProjectNames[projectName]; exists {- continue- }- if normalized := projects.NormalizeProjectName(projectName); normalized != "" {- if _, exists := knownProjectNames[normalized]; exists {- continue- }- }-- containersByProject[projectName] = append(containersByProject[projectName], c)- }-- if len(containersByProject) == 0 {- return nil- }-- updateInfoByRef := getRuntimeContainerUpdateInfoByRefInternal(ctx, composeContainers, imageService)- rows := make([]project.Details, 0, len(containersByProject))- for projectName, projectContainers := range containersByProject {- runtimeServices := buildDiscoveredRuntimeServicesInternal(projectContainers, iconCatalog)- imageRefs := projects.ImageRefsFromRuntimeServices(runtimeServices)- updateInfo := buildProjectUpdateInfoSummaryInternal(imageRefs, updateInfoByRef)- if updateInfo == nil || !updateInfo.HasUpdate {- continue- }-- runningCount := 0- for _, runtimeService := range runtimeServices {- if runtimeService.Status == "running" {- runningCount++- }- }-- lastCheckedAt := ""- if updateInfo.LastCheckedAt != nil {- lastCheckedAt = updateInfo.LastCheckedAt.Format(time.RFC3339)- }-- rows = append(rows, project.Details{- ID: "compose:" + projectName,- Name: projectName,- Path: "",- Status: resolveDiscoveredProjectStatusInternal(len(runtimeServices), runningCount),- ServiceCount: len(runtimeServices),- RunningCount: runningCount,- IsDiscovered: true,- CreatedAt: lastCheckedAt,- UpdatedAt: lastCheckedAt,- RuntimeServices: runtimeServices,- UpdateInfo: updateInfo,- })- }-- return rows-}--func getRuntimeContainerUpdateInfoByRefInternal(- ctx context.Context,- composeContainers []container.Summary,- imageService *ImageService,-) map[string]*imagetypes.UpdateInfo {- if imageService == nil || len(composeContainers) == 0 {- return nil- }-- imageRefs := make([]string, 0, len(composeContainers))- imageIDsByRef := make(map[string][]string, len(composeContainers))- seenRefs := make(map[string]struct{}, len(composeContainers))- for _, c := range composeContainers {- imageRef := strings.TrimSpace(c.Image)- if imageRef == "" {- continue- }- if _, exists := seenRefs[imageRef]; !exists {- seenRefs[imageRef] = struct{}{}- imageRefs = append(imageRefs, imageRef)- }- if imageID := strings.TrimSpace(c.ImageID); imageID != "" {- imageIDsByRef[imageRef] = append(imageIDsByRef[imageRef], imageID)- }- }-- updateInfoByRef := make(map[string]*imagetypes.UpdateInfo, len(imageRefs))- if len(imageRefs) > 0 {- if refResults, err := imageService.GetUpdateInfoByImageRefs(ctx, imageRefs); err == nil {- maps.Copy(updateInfoByRef, refResults)- } else {- slog.WarnContext(ctx, "failed to fetch compose project update info by image ref", "error", err)- }- }-- missingImageIDs := make([]string, 0)- for _, imageRef := range imageRefs {- if updateInfoByRef[imageRef] != nil {- continue- }- missingImageIDs = append(missingImageIDs, imageIDsByRef[imageRef]...)- }-- if len(missingImageIDs) == 0 {- return updateInfoByRef- }-- updateInfoByID, err := imageService.GetUpdateInfoByImageIDs(ctx, missingImageIDs)- if err != nil {- slog.WarnContext(ctx, "failed to fetch compose project update info by image id", "error", err)- return updateInfoByRef- }-- for imageRef, imageIDs := range imageIDsByRef {- if updateInfoByRef[imageRef] != nil {- continue- }- for _, imageID := range imageIDs {- if info := updateInfoByID[imageID]; info != nil {- updateInfoByRef[imageRef] = info- break- }- }- }-- return updateInfoByRef-}--func buildDiscoveredRuntimeServicesInternal(containers []container.Summary, iconCatalog string) []project.RuntimeService {- runtimeServices := make([]project.RuntimeService, 0, len(containers))- seenServices := make(map[string]struct{}, len(containers))- for _, c := range containers {- imageRef := strings.TrimSpace(c.Image)- if imageRef == "" {- continue- }-- serviceName := dockerutil.ComposeServiceLabel(c.Labels)- if serviceName == "" {- serviceName = c.ID- }- key := serviceName + "\x00" + imageRef- if _, exists := seenServices[key]; exists {- continue- }- seenServices[key] = struct{}{}-- containerName := dockerutil.ContainerNameFromNames(c.Names)-- resolvedIcon := iconcatalog.Resolve(iconCatalog, projects.FindArcaneIconSet(c.Labels))- runtimeServices = append(runtimeServices, project.RuntimeService{- Name: serviceName,- Image: imageRef,- Status: string(c.State),- ContainerID: c.ID,- ContainerName: containerName,- Ports: formatDockerPorts(c.Ports),- IconLightURL: resolvedIcon.IconLightURL,- IconDarkURL: resolvedIcon.IconDarkURL,- })- }-- return runtimeServices-}--func resolveDiscoveredProjectStatusInternal(serviceCount int, runningCount int) string {- switch {- case serviceCount == 0:- return string(models.ProjectStatusUnknown)- case runningCount >= serviceCount:- return string(models.ProjectStatusRunning)- case runningCount > 0:- return string(models.ProjectStatusPartiallyRunning)- default:- return string(models.ProjectStatusStopped)- }-}--func (s *ProjectService) buildProjectDerivedPaginationConfigInternal() pagination.Config[project.Details] {- return pagination.Config[project.Details]{- SearchAccessors: []pagination.SearchAccessor[project.Details]{- func(p project.Details) (string, error) { return p.Name, nil },- func(p project.Details) (string, error) { return p.Path, nil },- func(p project.Details) (string, error) { return p.RelativePath, nil },- func(p project.Details) (string, error) { return p.Status, nil },- func(p project.Details) (string, error) { return p.DirName, nil },- },- SortBindings: []pagination.SortBinding[project.Details]{- {- Key: "name",- Fn: func(a, b project.Details) int {- return strings.Compare(a.Name, b.Name)- },- },- {- Key: "status",- Fn: func(a, b project.Details) int {- return strings.Compare(a.Status, b.Status)- },- },- {- Key: "serviceCount",- Fn: func(a, b project.Details) int {- if a.ServiceCount < b.ServiceCount {- return -1- }- if a.ServiceCount > b.ServiceCount {- return 1- }- return 0- },- },- {- Key: "path",- Fn: func(a, b project.Details) int {- return strings.Compare(a.RelativePath, b.RelativePath)- },- },- {- Key: "createdAt",- Fn: func(a, b project.Details) int {- at, aerr := time.Parse(time.RFC3339, a.CreatedAt)- bt, berr := time.Parse(time.RFC3339, b.CreatedAt)- if aerr != nil || berr != nil {- return strings.Compare(a.CreatedAt, b.CreatedAt)- }- if at.Before(bt) {- return -1- }- if at.After(bt) {- return 1- }- return 0- },- },- },- FilterAccessors: []pagination.FilterAccessor[project.Details]{- s.buildProjectStatusFilterAccessorInternal(),- s.buildProjectUpdatesFilterAccessorInternal(),- s.buildProjectArchivedFilterAccessorInternal(),- },- }-}--func (s *ProjectService) buildProjectStatusFilterAccessorInternal() pagination.FilterAccessor[project.Details] {- return pagination.FilterAccessor[project.Details]{- Key: "status",- Fn: func(p project.Details, filterValue string) bool {- return strings.EqualFold(strings.TrimSpace(p.Status), strings.TrimSpace(filterValue))- },- }-}--func (s *ProjectService) buildProjectUpdatesFilterAccessorInternal() pagination.FilterAccessor[project.Details] {- return pagination.FilterAccessor[project.Details]{- Key: "updates",- Fn: func(p project.Details, filterValue string) bool {- return strings.EqualFold(strings.TrimSpace(getProjectUpdateStatusInternal(p.UpdateInfo)), strings.TrimSpace(filterValue))- },- }-}--func (s *ProjectService) buildProjectArchivedFilterAccessorInternal() pagination.FilterAccessor[project.Details] {- return pagination.FilterAccessor[project.Details]{- Key: "archived",- Fn: func(p project.Details, filterValue string) bool {- switch strings.ToLower(strings.TrimSpace(filterValue)) {- case "true":- return p.IsArchived- case "all":- return true- default:- return !p.IsArchived- }- },- }-}--func getProjectUpdateStatusInternal(updateInfo *project.UpdateInfo) string {- if updateInfo == nil || strings.TrimSpace(updateInfo.Status) == "" {- return "unknown"- }-- return updateInfo.Status-}--// countProjectsWithPendingUpdatesInternal counts non-archived projects with at-// least one image update pending, plus compose projects running on the daemon-// that Arcane does not track. It deliberately avoids the project-list pipeline:-// that path builds full project DTOs (live status, icons, URLs, GitOps lookups)-// and then throws all of them away for a single number, costing several full-// container lists and a compose parse per project on every dashboard load.-//-// allContainers is the caller's already-fetched container list; pass nil to have-// it fetched here.-func (s *ProjectService) countProjectsWithPendingUpdatesInternal(ctx context.Context, allContainers []container.Summary) (int, error) {- if s.db == nil {- return 0, nil- }-- var projectsArray []models.Project- if err := s.db.WithContext(ctx).Where("is_archived = ?", false).Find(&projectsArray).Error; err != nil {- return 0, errors.WrapIf(err, "failed to list projects for update count")- }-- // enrichProjectsWithUpdateInfoInternal keys off Details.ID, so the summaries- // only need identity — no status, icons or URLs are read here.- details := make([]project.Details, len(projectsArray))- for i, proj := range projectsArray {- details[i].ID = proj.ID- }- s.enrichProjectsWithUpdateInfoInternal(ctx, projectsArray, details)-- count := 0- for i := range details {- if details[i].UpdateInfo != nil && details[i].UpdateInfo.HasUpdate {- count++- }- }-- return count + s.countDiscoveredComposeProjectUpdatesInternal(ctx, projectsArray, allContainers), nil-}--// countDiscoveredComposeProjectUpdatesInternal counts compose projects running on-// the daemon that Arcane does not track but that have a pending image update, so-// the dashboard badge matches the projects table. Errors are logged and counted-// as zero: a missing container list should degrade the badge, not fail the load.-func (s *ProjectService) countDiscoveredComposeProjectUpdatesInternal(ctx context.Context, projectsArray []models.Project, allContainers []container.Summary) int {- if allContainers == nil {- var err error- allContainers, err = projects.ListGlobalComposeContainers(ctx)- if err != nil {- slog.WarnContext(ctx, "failed to list compose containers for project update count", "error", err)- return 0- }- }-- composeContainers := make([]container.Summary, 0, len(allContainers))- for _, c := range allContainers {- if dockerutil.ComposeProjectLabel(c.Labels) != "" {- composeContainers = append(composeContainers, c)- }- }- if len(composeContainers) == 0 {- return 0- }-- knownProjectNames := s.buildKnownComposeProjectNameSetInternal(ctx, projectsArray)- // Only rows with a pending update are returned, so the length is the count.- return len(buildDiscoveredComposeProjectUpdateRowsInternal(ctx, composeContainers, knownProjectNames, s.imageService, iconCatalogForContextInternal(ctx)))-}--// fetchProjectStatusConcurrently fetches live Docker status for multiple projects in parallel-// Optimized to use a single Docker API call instead of N calls + N file reads-func (s *ProjectService) fetchProjectStatusConcurrently(ctx context.Context, projectsList []models.Project) []project.Details {- projectsDir, err := s.getProjectsDirectoryInternal(ctx)- if err != nil {- slog.WarnContext(ctx, "failed to resolve projects directory for relative project paths", "error", err)- }-- // Resolved once for the whole list: getProjectMetadataForProject would- // otherwise re-stat the projects directory and re-clone settings per project.- metaEnv := &projectMetadataEnvInternal{- projectsDirectory: projectsDir,- autoInjectEnv: s.settingsService.GetBoolSetting(ctx, "autoInjectEnv", false),- }-- // 1. Fetch all compose containers in one go- containers, err := projects.ListGlobalComposeContainers(ctx)- if err != nil {- slog.ErrorContext(ctx, "Failed to list global compose containers", "error", err)- // Fallback: return basic info with unknown status- results := make([]project.Details, len(projectsList))- for i, p := range projectsList {- _ = mapper.MapStruct(p, &results[i])- results[i].CreatedAt = p.CreatedAt.Format(time.RFC3339)- results[i].UpdatedAt = p.UpdatedAt.Format(time.RFC3339)- results[i].DirName = mo.PointerToOption(p.DirName).OrEmpty()- results[i].RelativePath = s.getProjectRelativePathInternal(projectsDir, p.Path)- results[i].GitOpsManagedBy = p.GitOpsManagedBy- meta := s.getProjectMetadataForProject(ctx, p, metaEnv)- applyResolvedProjectIconInternal(&results[i], s.resolveIconSetInternal(ctx, meta.ProjectIcon))- results[i].URLs = meta.ProjectURLS- results[i].Status = string(models.ProjectStatusUnknown)- }- return results- }-- // 2. Group containers by project name- containersByProject := groupComposeContainersByProjectInternal(containers)-- // 3. Map to DTOs- results := make([]project.Details, len(projectsList))- currentContainerID, currentContainerErr := cgroup.CurrentContainerID()- for i, p := range projectsList {- results[i] = s.mapProjectToDto(ctx, projectsDir, p, containersByProject, currentContainerID, currentContainerErr, metaEnv)- }-- return results-}--func (s *ProjectService) mapProjectToDto(ctx context.Context, projectsDir string, p models.Project, containersByProject map[string][]container.Summary, currentContainerID string, currentContainerErr error, metaEnv *projectMetadataEnvInternal) project.Details {- var resp project.Details- _ = mapper.MapStruct(p, &resp)-- resp.CreatedAt = p.CreatedAt.Format(time.RFC3339)- resp.UpdatedAt = p.UpdatedAt.Format(time.RFC3339)- resp.IsArchived = p.IsArchived- resp.ArchivedAt = p.ArchivedAt- resp.DirName = mo.PointerToOption(p.DirName).OrEmpty()- resp.RelativePath = s.getProjectRelativePathInternal(projectsDir, p.Path)- resp.GitOpsManagedBy = p.GitOpsManagedBy- meta := s.getProjectMetadataForProject(ctx, p, metaEnv)- applyResolvedProjectIconInternal(&resp, s.resolveIconSetInternal(ctx, meta.ProjectIcon))- resp.URLs = meta.ProjectURLS-- projectContainers := lookupProjectContainers(p, containersByProject)-- services := make([]ProjectServiceInfo, 0, len(projectContainers))-- for _, c := range projectContainers {- svcName := dockerutil.ComposeServiceLabel(c.Labels)- state := c.State // "running", "exited", etc.-- // Parse health from Status string if possible- var health *string- statusLower := strings.ToLower(c.Status)- switch {- case strings.Contains(statusLower, "(healthy)"):- health = new("healthy")- case strings.Contains(statusLower, "(unhealthy)"):- health = new("unhealthy")- case strings.Contains(statusLower, "(starting)"):- health = new("starting")- }-- containerName := dockerutil.ContainerNameFromNames(c.Names)-- redeployDisabled := labels.ShouldDisableArcaneServerRedeploy(c.Labels, c.ID, currentContainerID, currentContainerErr)- if redeployDisabled {- resp.RedeployDisabled = true- }-- resolvedIcon := s.resolveIconSetInternal(ctx, iconcatalog.FirstNonEmpty(- projects.FindArcaneIconSet(c.Labels),- meta.ServiceIconSets[svcName],- meta.ProjectIcon,- ))- services = append(services, ProjectServiceInfo{- Name: svcName,- Image: c.Image,- Status: string(state),- ContainerID: c.ID,- ContainerName: containerName,- Ports: formatDockerPorts(c.Ports),- Health: health,- IconLightURL: resolvedIcon.IconLightURL,- IconDarkURL: resolvedIcon.IconDarkURL,- Labels: c.Labels,- RedeployDisabled: redeployDisabled,- })- }- _, runningCount := s.getServiceCounts(services)-- // Convert to RuntimeServices- runtimeServices := make([]project.RuntimeService, len(services))- for k, s := range services {- runtimeServices[k] = project.RuntimeService{- Name: s.Name,- Image: s.Image,- Status: s.Status,- ContainerID: s.ContainerID,- ContainerName: s.ContainerName,- Ports: s.Ports,- Health: s.Health,- IconLightURL: s.IconLightURL,- IconDarkURL: s.IconDarkURL,- ServiceConfig: s.ServiceConfig,- RedeployDisabled: s.RedeployDisabled,- }- }- resp.RuntimeServices = runtimeServices-- // Use DB service count as the source of truth for "Total Services"- // since we are not parsing the YAML here.- resp.ServiceCount = p.ServiceCount- resp.RunningCount = runningCount- if resp.ServiceCount == 0 && len(services) > 0 {- resp.ServiceCount = len(services)- // Persist the inferred count so later list loads do not need compose parsing.- go func(ctx context.Context, pid string, count int) {- s.db.WithContext(ctx).Model(&models.Project{}).Where("id = ?", pid).Update("service_count", count)- }(context.WithoutCancel(ctx), p.ID, resp.ServiceCount)- }-- // For missing service count (e.g. newly discovered projects), skip the- // expensive countServicesFromCompose call which loads and parses the entire- // compose project. The count will be populated the next time the project- // detail endpoint is called or during the periodic filesystem sync.-- // Calculate Status using actual container count from Docker rather than the- // (potentially stale) DB ServiceCount. The DB value can become outdated when- // a service is removed from the compose file but compose parsing fails during- // filesystem sync, leaving the old count in the database. This mirrors the- // logic in calculateProjectStatus and GetProjectDetails, which both use the- // live container/service list as the source of truth.- actualServiceCount := len(services)- if actualServiceCount == 0 {- resp.Status = string(models.ProjectStatusStopped)- } else {- switch {- case runningCount >= actualServiceCount:- resp.Status = string(models.ProjectStatusRunning)- case runningCount > 0:- resp.Status = string(models.ProjectStatusPartiallyRunning)- default:- resp.Status = string(models.ProjectStatusStopped)- }- }-- return resp-}--// getProjectMetadataForProject resolves a project's icon sets and service URLs.-// Results are cached for projectMetadataTTL because deriving them is expensive-// (compose load with interpolation and .env reads, plus a gitops_syncs query for-// GitOps-managed projects) and every project row on the list page needs it.-//-// env may be nil, in which case the projects directory and autoInjectEnv setting-// are resolved here; callers iterating over many projects should resolve them-// once and pass them in.-func (s *ProjectService) getProjectMetadataForProject(ctx context.Context, p models.Project, env *projectMetadataEnvInternal) projects.ArcaneComposeMetadata {- if s.metaCache != nil && p.ID != "" {- if meta, ok, _ := s.metaCache.Get(p.ID); ok {- return meta- }- }-- empty := projects.ArcaneComposeMetadata{ServiceIconSets: map[string]projects.IconSet{}}-- composeFile, err := s.resolveProjectComposeFileInternal(ctx, &p)- if err != nil {- return empty- }-- if env == nil {- projectsDirectory, projectsDirErr := s.getProjectsDirectoryInternal(ctx)- if projectsDirErr != nil {- slog.WarnContext(ctx, "failed to resolve projects directory for Arcane compose metadata", "path", composeFile, "error", projectsDirErr)- }- env = &projectMetadataEnvInternal{- projectsDirectory: projectsDirectory,- autoInjectEnv: s.settingsService.GetBoolSetting(ctx, "autoInjectEnv", false),- }- }-- meta, err := projects.ParseArcaneComposeMetadata(ctx, composeFile, env.projectsDirectory, env.autoInjectEnv)- if err != nil {- slog.WarnContext(ctx, "failed to parse Arcane compose metadata", "path", composeFile, "error", err)- return empty- }-- if s.metaCache != nil && p.ID != "" {- s.metaCache.Set(p.ID, meta)- }-- return meta-}--// iconCatalogForContextInternal resolves the icon catalog of the requesting-// user. On agent-proxied calls the caller is a synthetic user whose preference-// is populated from the X-Arcane-Icon-Catalog header the manager forwards.-// Background jobs have no user attached and fall back to the default catalog.-func iconCatalogForContextInternal(ctx context.Context) string {- if u, ok := models.CurrentUserFromContext(ctx); ok && u != nil && u.Preferences.IconCatalog != nil && *u.Preferences.IconCatalog != "" {- return *u.Preferences.IconCatalog- }- return iconcatalog.DefaultCatalog-}--func (s *ProjectService) resolveIconSetInternal(ctx context.Context, iconSet iconcatalog.IconSet) iconcatalog.ResolvedIconSet {- return iconcatalog.Resolve(iconCatalogForContextInternal(ctx), iconSet)-}--func applyResolvedProjectIconInternal(resp *project.Details, icon iconcatalog.ResolvedIconSet) {- if resp == nil {- return- }- resp.IconLightURL = icon.IconLightURL- resp.IconDarkURL = icon.IconDarkURL-}--// End Table Functions--func (s *ProjectService) countServicesFromCompose(ctx context.Context, p models.Project) (int, error) {- proj, _, err := s.loadComposeProjectForProjectInternal(ctx, &p, nil)- if err != nil {- return 0, err- }-- return len(proj.Services), nil-}--type composeSyncMetadataInternal struct {- serviceCount int- resolvedProjectName string- composeProjectName *string- explicitProjectName bool-}--// loadComposeMetadataForSyncInternal loads the compose file once and returns-// the service count plus compose-go's effective project name.-func (s *ProjectService) loadComposeMetadataForSyncInternal(ctx context.Context, dirPath, dirName string) (composeSyncMetadataInternal, error) {- normName := projects.NormalizeProjectName(dirName)- meta := composeSyncMetadataInternal{- resolvedProjectName: normName,- }-- cfg := s.settingsService.GetSettingsOrDefaults(ctx)- projectsDirectory, pErr := projects.GetProjectsDirectory(ctx, strings.TrimSpace(cfg.ProjectsDirectory.Value))- if pErr != nil {- return meta, pErr- }-- var dockerClient *client.Client- if s.dockerService != nil {- dockerClient, _ = s.dockerService.GetClient(ctx)- }- pathMapper := projects.NewPathMapperForConfiguredDirectory(- ctx,- s.settingsService.GetStringSetting(ctx, "projectsDirectory", "/app/data/projects"),- "/app/data/projects",- dockerClient,- )-- autoInjectEnv := utils.BoolOrDefault(cfg.AutoInjectEnv.Value, false)-- // First, try loading without forcing a project name so compose-go can- // resolve COMPOSE_PROJECT_NAME from the .env file. If this fails (e.g.- // no .env and directory name is not a valid compose project name), fall- // back to the normalized directory name.- proj, _, err := projects.LoadComposeProjectFromDir(ctx, dirPath, "", projectsDirectory, autoInjectEnv, pathMapper)- if err != nil {- proj, _, err = projects.LoadComposeProjectFromDir(ctx, dirPath, normName, projectsDirectory, autoInjectEnv, pathMapper)- if err != nil {- return meta, err- }- } else if proj.Name != "" && proj.Name != normName {- meta.explicitProjectName = true- }-- meta.serviceCount = len(proj.Services)- if proj.Name != "" {- meta.resolvedProjectName = proj.Name- }-- // If compose-go resolved a different name (from COMPOSE_PROJECT_NAME),- // store it so we can match containers correctly.- if proj.Name != "" && proj.Name != normName {- meta.composeProjectName = new(proj.Name)- }-- return meta, nil-}--func (s *ProjectService) refreshComposeProjectNameInternal(ctx context.Context, proj *models.Project) {- if proj == nil {- return- }-- dirName := proj.Name- if proj.DirName != nil && *proj.DirName != "" {- dirName = *proj.DirName- }-- meta, err := s.loadComposeMetadataForSyncInternal(ctx, proj.Path, dirName)- if err != nil {- slog.WarnContext(ctx, "failed to refresh compose project name", "projectID", proj.ID, "path", proj.Path, "error", err)- return- }-- updates := map[string]any{}- shouldUpdateName := meta.explicitProjectName || projects.NormalizeProjectName(proj.Name) != proj.Name- if shouldUpdateName && meta.resolvedProjectName != "" && proj.Name != meta.resolvedProjectName {- updates["name"] = meta.resolvedProjectName- }- if mo.PointerToOption(proj.ComposeProjectName) != mo.PointerToOption(meta.composeProjectName) {- updates["compose_project_name"] = meta.composeProjectName- }- if len(updates) == 0 {- return- }-- updates["updated_at"] = time.Now()- if err := s.db.WithContext(ctx).- Model(&models.Project{}).- Where("id = ?", proj.ID).- Updates(updates).Error; err != nil {- slog.WarnContext(ctx, "failed to persist refreshed compose project name", "projectID", proj.ID, "error", err)- return- }-- if name, ok := updates["name"].(string); ok {- proj.Name = name- }- if _, ok := updates["compose_project_name"]; ok {- proj.ComposeProjectName = meta.composeProjectName- }-}--func (s *ProjectService) calculateProjectStatus(services []ProjectServiceInfo) models.ProjectStatus {- if len(services) == 0 {- return models.ProjectStatusUnknown- }-- runningCount := 0- stoppedCount := 0-- for _, svc := range services {- state := strings.ToLower(strings.TrimSpace(svc.Status))- switch state {- case "running", "up":- runningCount++- case "exited", "stopped", "dead":- stoppedCount++- }- }-- if runningCount == len(services) {- return models.ProjectStatusRunning- }- if runningCount > 0 {- return models.ProjectStatusPartiallyRunning- }- if stoppedCount > 0 {- return models.ProjectStatusStopped- }- return models.ProjectStatusUnknown-}--const (- projectRenameJournalKeyPrefixInternal = "project_rename_journal:"- projectRenameRollbackCleanupKeyPrefixInternal = "project_rename_rollback_cleanup:"-- projectRenameJournalPhaseStartedInternal = "started"- projectRenameJournalPhaseTargetsCopiedInternal = "targets_copied"- projectRenameJournalPhaseOldVolumesRemovedInternal = "old_volumes_removed"- projectRenameJournalPhaseProjectStateCommittedInternal = "project_state_committed"- projectRenameJournalPhaseSourceCleanupPendingInternal = "source_cleanup_pending"- projectRenameJournalPhaseProjectStateRolledBackInternal = "project_state_rolled_back"-)--type projectRenameJournalInternal struct {- ProjectID string `json:"projectId"`- OldName string `json:"oldName"`- NewName string `json:"newName"`- OldPath string `json:"oldPath"`- NewPath string `json:"newPath"`- OldDirName *string `json:"oldDirName,omitempty"`- NewDirName string `json:"newDirName"`- Phase string `json:"phase"`- Volumes []volumes.JournalVolume `json:"volumes,omitempty"`- UpdatedAt time.Time `json:"updatedAt"`-}--type projectRenameRollbackCleanupInternal struct {- ProjectID string `json:"projectId"`- OldName string `json:"oldName"`- OldPath string `json:"oldPath"`- NewName string `json:"newName"`- NewPath string `json:"newPath"`- Volumes []volumes.JournalVolume `json:"volumes,omitempty"`- UpdatedAt time.Time `json:"updatedAt"`-}--func projectRenameJournalKeyInternal(projectID string) string {- return projectRenameJournalKeyPrefixInternal + strings.TrimSpace(projectID)-}--func projectRenameRollbackCleanupKeyInternal(projectID string) string {- return projectRenameRollbackCleanupKeyPrefixInternal + strings.TrimSpace(projectID)-}--func (s *ProjectService) prepareProjectRenameJournalInternal(proj *models.Project, name *string, projectsDirectory string, migration volumes.Migration) *projectRenameJournalInternal {- if s == nil || s.kvService == nil || proj == nil || name == nil {- return nil- }-- newName := strings.TrimSpace(*name)- if newName == "" || proj.Name == newName {- return nil- }-- newDirName := strings.TrimSpace(projects.SanitizeProjectName(newName))- if newDirName == "" || strings.Trim(newDirName, "_") == "" {- return nil- }-- journal := &projectRenameJournalInternal{- ProjectID: proj.ID,- OldName: proj.Name,- NewName: newName,- OldPath: filepath.Clean(proj.Path),- NewPath: filepath.Clean(filepath.Join(projectsDirectory, newDirName)),- OldDirName: cloneStringPtrInternal(proj.DirName),- NewDirName: newDirName,- Phase: projectRenameJournalPhaseStartedInternal,- }-- if source, ok := migration.(volumes.JournalSource); ok {- journal.Volumes = source.JournalVolumes()- }-- return journal-}--func (s *ProjectService) writeProjectRenameJournalInternal(ctx context.Context, journal *projectRenameJournalInternal, phase string) error {- if s == nil || s.kvService == nil || journal == nil {- return nil- }- journal.Phase = phase- journal.UpdatedAt = time.Now().UTC()-- payload, err := json.Marshal(journal)- if err != nil {- return errors.WrapIf(err, "marshal project rename journal")- }-- if err := s.kvService.Set(ctx, projectRenameJournalKeyInternal(journal.ProjectID), string(payload)); err != nil {- return errors.WrapIf(err, "write project rename journal")- }- return nil-}--func (s *ProjectService) clearProjectRenameJournalInternal(ctx context.Context, projectID string) error {- if s == nil || s.kvService == nil || strings.TrimSpace(projectID) == "" {- return nil- }- return s.kvService.Delete(ctx, projectRenameJournalKeyInternal(projectID))-}--func (s *ProjectService) writeProjectRenameRollbackCleanupInternal(ctx context.Context, journal *projectRenameJournalInternal) error {- if s == nil || s.kvService == nil || journal == nil || strings.TrimSpace(journal.ProjectID) == "" || len(journal.Volumes) == 0 {- return nil- }-- cleanup := projectRenameRollbackCleanupInternal{- ProjectID: journal.ProjectID,- OldName: journal.OldName,- OldPath: filepath.Clean(journal.OldPath),- NewName: journal.NewName,- NewPath: filepath.Clean(journal.NewPath),- Volumes: journal.Volumes,- UpdatedAt: time.Now().UTC(),- }- payload, err := json.Marshal(cleanup)- if err != nil {- return errors.WrapIf(err, "marshal project rename rollback cleanup")- }- if err := s.kvService.Set(ctx, projectRenameRollbackCleanupKeyInternal(journal.ProjectID), string(payload)); err != nil {- return errors.WrapIf(err, "write project rename rollback cleanup")- }- return nil-}--func (s *ProjectService) clearProjectRenameRollbackCleanupInternal(ctx context.Context, projectID string) error {- if s == nil || s.kvService == nil || strings.TrimSpace(projectID) == "" {- return nil- }- return s.kvService.Delete(ctx, projectRenameRollbackCleanupKeyInternal(projectID))-}--func projectRenameJournalTargetsCopiedInternal(phase string) bool {- switch phase {- case projectRenameJournalPhaseTargetsCopiedInternal,- projectRenameJournalPhaseOldVolumesRemovedInternal,- projectRenameJournalPhaseProjectStateCommittedInternal,- projectRenameJournalPhaseSourceCleanupPendingInternal:- return true- default:- return false- }-}--func projectRenameJournalFilesystemSyncPendingInternal(phase string) bool {- switch phase {- case projectRenameJournalPhaseStartedInternal,- projectRenameJournalPhaseTargetsCopiedInternal:- return true- default:- return false- }-}--func (s *ProjectService) RecoverProjectRenameJournals(ctx context.Context) error {- if s == nil || s.kvService == nil {- return nil- }-- entries, err := s.kvService.ListByPrefix(ctx, projectRenameJournalKeyPrefixInternal)- if err != nil {- return err- }-- var recoverErr error- for _, entry := range entries {- var journal projectRenameJournalInternal- if err := json.Unmarshal([]byte(entry.Value), &journal); err != nil {- recoverErr = stderrors.Join(recoverErr, errors.WrapIff(err, "decode project rename journal %s", entry.Key))- continue- }- if err := s.recoverProjectRenameJournalInternal(ctx, &journal); err != nil {- recoverErr = stderrors.Join(recoverErr, errors.WrapIff(err, "recover project rename journal %s", entry.Key))- continue- }- }- return stderrors.Join(recoverErr, s.recoverProjectRenameRollbackCleanupsInternal(ctx))-}--func (s *ProjectService) recoverProjectRenameJournalForProjectInternal(ctx context.Context, projectID string) error {- if s == nil || s.kvService == nil || strings.TrimSpace(projectID) == "" {- return nil- }-- raw, ok, err := s.kvService.Get(ctx, projectRenameJournalKeyInternal(projectID))- if err != nil || !ok {- return err- }-- var journal projectRenameJournalInternal- if err := json.Unmarshal([]byte(raw), &journal); err != nil {- return errors.WrapIf(err, "decode project rename journal")- }- return s.recoverProjectRenameJournalInternal(ctx, &journal)-}--func (s *ProjectService) recoverProjectRenameJournalInternal(ctx context.Context, journal *projectRenameJournalInternal) error {- if s == nil || journal == nil || strings.TrimSpace(journal.ProjectID) == "" {- return nil- }-- var proj models.Project- dbErr := s.db.WithContext(ctx).First(&proj, "id = ?", journal.ProjectID).Error- if dbErr != nil && !errors.Is(dbErr, gorm.ErrRecordNotFound) {- return errors.WrapIf(dbErr, "load project for rename recovery")- }-- projectCommitted := dbErr == nil && (proj.Name == journal.NewName || filepath.Clean(proj.Path) == filepath.Clean(journal.NewPath))- if projectCommitted {- if journal.Phase == projectRenameJournalPhaseSourceCleanupPendingInternal {- if err := s.cleanupProjectRenameJournalSourcesInternal(ctx, journal); err != nil {- return err- }- return s.clearProjectRenameJournalInternal(ctx, journal.ProjectID)- }- if err := s.cleanupProjectRenameJournalSourcesInternal(ctx, journal); err != nil {- var cleanupErr *volumes.SourceCleanupError- if errors.As(err, &cleanupErr) {- if writeErr := s.writeProjectRenameJournalInternal(ctx, journal, projectRenameJournalPhaseSourceCleanupPendingInternal); writeErr != nil {- return stderrors.Join(err, writeErr)- }- }- return err- }- return s.clearProjectRenameJournalInternal(ctx, journal.ProjectID)- }-- if err := s.rollbackProjectRenameJournalInternal(ctx, journal); err != nil {- return err- }- if err := s.writeProjectRenameJournalInternal(ctx, journal, projectRenameJournalPhaseProjectStateRolledBackInternal); err != nil {- return err- }- return s.clearProjectRenameJournalInternal(ctx, journal.ProjectID)-}--func (s *ProjectService) cleanupProjectRenameJournalSourcesInternal(ctx context.Context, journal *projectRenameJournalInternal) error {- dockerClient, err := s.projectRenameRecoveryDockerInternal(ctx, len(journal.Volumes) > 0)- if err != nil {- return err- }-- if err := volumes.EnsureTargetsReadyForCleanup(ctx, dockerClient, journal.Volumes); err != nil {- var missingWithSource *volumes.TargetMissingWithSourceError- if errors.As(err, &missingWithSource) {- slog.WarnContext(ctx, "rolling back project rename because target volume is missing and source volume remains", "projectID", journal.ProjectID, "sourceVolume", missingWithSource.SourceVolume, "targetVolume", missingWithSource.TargetVolume)- return s.rollbackProjectRenameJournalInternal(ctx, journal)- }- var externallyRemoved *volumes.VolumesExternallyRemovedError- if errors.As(err, &externallyRemoved) {- slog.WarnContext(ctx, "project rename cleanup found source and target volumes externally removed", "projectID", journal.ProjectID, "volumeCount", len(externallyRemoved.Volumes), "error", externallyRemoved)- } else {- return err- }- }-- return volumes.RemoveSourceVolumes(ctx, dockerClient, journal.Volumes)-}--func (s *ProjectService) rollbackProjectRenameJournalInternal(ctx context.Context, journal *projectRenameJournalInternal) error {- pathsMissing, directoryErr := projects.RollbackRenamedProjectDirectory(journal.OldPath, journal.NewPath)-- volumeErr := s.rollbackProjectRenameJournalVolumesInternal(ctx, journal)-- if err := s.db.WithContext(ctx).Model(&models.Project{}).- Where("id = ?", journal.ProjectID).- Updates(map[string]any{- "name": journal.OldName,- "path": journal.OldPath,- "dir_name": journal.OldDirName,- }).Error; err != nil {- return stderrors.Join(directoryErr, volumeErr, errors.WrapIf(err, "restore project database state"))- }-- if directoryErr != nil {- slog.WarnContext(ctx, "keeping project rename journal after restoring database state because directory rollback failed", "projectID", journal.ProjectID, "pathsMissing", pathsMissing, "error", directoryErr)- }-- if volumeErr != nil {- if volumes.OnlyPreservedTargetErrors(volumeErr) {- slog.WarnContext(ctx, "clearing project rename journal after preserving target volume data", "projectID", journal.ProjectID, "pathsMissing", pathsMissing, "error", volumeErr)- } else {- if cleanupErr := s.writeProjectRenameRollbackCleanupInternal(ctx, journal); cleanupErr != nil {- return stderrors.Join(directoryErr, volumeErr, cleanupErr)- }- slog.WarnContext(ctx, "queued project rename target volume cleanup after restoring database state despite volume rollback failure", "projectID", journal.ProjectID, "pathsMissing", pathsMissing, "error", volumeErr)- }- }-- return directoryErr-}--func (s *ProjectService) recoverProjectRenameRollbackCleanupsInternal(ctx context.Context) error {- if s == nil || s.kvService == nil {- return nil- }-- entries, err := s.kvService.ListByPrefix(ctx, projectRenameRollbackCleanupKeyPrefixInternal)- if err != nil {- return err- }-- var recoverErr error- for _, entry := range entries {- var cleanup projectRenameRollbackCleanupInternal- if err := json.Unmarshal([]byte(entry.Value), &cleanup); err != nil {- recoverErr = stderrors.Join(recoverErr, errors.WrapIff(err, "decode project rename rollback cleanup %s", entry.Key))- continue- }- if err := s.recoverProjectRenameRollbackCleanupInternal(ctx, &cleanup); err != nil {- recoverErr = stderrors.Join(recoverErr, errors.WrapIff(err, "recover project rename rollback cleanup %s", entry.Key))- continue- }- }- return recoverErr-}--func (s *ProjectService) recoverProjectRenameRollbackCleanupInternal(ctx context.Context, cleanup *projectRenameRollbackCleanupInternal) error {- if s == nil || cleanup == nil || strings.TrimSpace(cleanup.ProjectID) == "" {- return nil- }- if len(cleanup.Volumes) == 0 {- return s.clearProjectRenameRollbackCleanupInternal(ctx, cleanup.ProjectID)- }-- var proj models.Project- dbErr := s.db.WithContext(ctx).First(&proj, "id = ?", cleanup.ProjectID).Error- if dbErr != nil {- if errors.Is(dbErr, gorm.ErrRecordNotFound) {- slog.WarnContext(ctx, "clearing project rename rollback cleanup because project no longer exists", "projectID", cleanup.ProjectID)- return s.clearProjectRenameRollbackCleanupInternal(ctx, cleanup.ProjectID)- }- return errors.WrapIf(dbErr, "load project for rename rollback cleanup")- }-- if proj.Name != cleanup.OldName || filepath.Clean(proj.Path) != filepath.Clean(cleanup.OldPath) {- slog.WarnContext(ctx, "clearing project rename rollback cleanup because project state changed", "projectID", cleanup.ProjectID, "projectName", proj.Name, "projectPath", proj.Path)- return s.clearProjectRenameRollbackCleanupInternal(ctx, cleanup.ProjectID)- }-- dockerClient, err := s.projectRenameRecoveryDockerInternal(ctx, len(cleanup.Volumes) > 0)- if err != nil {- return err- }-- if err := volumes.CleanupRollbackTargetVolumes(ctx, dockerClient, cleanup.Volumes); err != nil {- if volumes.OnlyPreservedTargetErrors(err) {- slog.WarnContext(ctx, "clearing project rename rollback cleanup after preserving target volume data", "projectID", cleanup.ProjectID, "error", err)- return s.clearProjectRenameRollbackCleanupInternal(ctx, cleanup.ProjectID)- }- return err- }-- dockerutil.InvalidateVolumeUsageCache(dockerClient)- return s.clearProjectRenameRollbackCleanupInternal(ctx, cleanup.ProjectID)-}--func (s *ProjectService) rollbackProjectRenameJournalVolumesInternal(ctx context.Context, journal *projectRenameJournalInternal) error {- if !projectRenameJournalTargetsCopiedInternal(journal.Phase) {- return nil- }-- dockerClient, err := s.projectRenameRecoveryDockerInternal(ctx, len(journal.Volumes) > 0)- if err != nil {- return err- }-- return volumes.RollbackVolumes(ctx, dockerClient, journal.Volumes)-}--func (s *ProjectService) projectRenameRecoveryDockerInternal(ctx context.Context, dockerRequired bool) (*client.Client, error) {- if !dockerRequired {- return nil, nil- }- if s.dockerService == nil {- return nil, errors.New("docker service unavailable")- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, errors.WrapIf(err, "failed to connect to Docker")- }-- return dockerClient, nil-}--func cloneStringPtrInternal(value *string) *string {- if value == nil {- return nil- }- cloned := *value- return &cloned-}diff --git a/backend/internal/services/volume_service.go b/backend/internal/services/volume_service.godeleted file mode 100644index e2ba9fd09..000000000--- a/backend/internal/services/volume_service.go+++ /dev/null@@ -1,2440 +0,0 @@-package services--import (- "archive/tar"- "bytes"- "compress/gzip"- "context"- "fmt"- "io"- "log/slog"- "net/http"- "os"- "path"- "strconv"- "strings"- "sync"- "time"-- "emperror.dev/errors"-- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- docker "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/volumehelper"- "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"- "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"- volumetypes "github.com/getarcaneapp/arcane/types/v2/volume"- "github.com/google/uuid"- "github.com/moby/moby/api/pkg/stdcopy"- "github.com/moby/moby/api/types/container"- "github.com/moby/moby/api/types/mount"- "github.com/moby/moby/api/types/volume"- "github.com/moby/moby/client"- "github.com/samber/mo"- "golang.org/x/sync/singleflight"-)--type VolumeService struct {- db *database.DB- dockerService *DockerClientService- eventService *EventService- settingsService *SettingsService- containerService *ContainerService- imageService *ImageService- backupVolumeName string- helperMu sync.Mutex- helperByVolume map[string]*volumeHelper- // helperGroup deduplicates concurrent read-only helper creation per volume.- // Without it two simultaneous browse requests each create a helper and the- // second overwrites the first in helperByVolume, orphaning a `sleep infinity`- // container that pins the volume until restart.- helperGroup singleflight.Group-}--// volumeHelper tracks a reused read-only browse helper container and the last-// time it serviced a request, so idle helpers can be reaped. inUse counts the-// requests currently holding the helper: the reaper must not remove a helper-// mid-download just because it was acquired longer ago than the idle timeout.-type volumeHelper struct {- id string- lastUsedAt time.Time- inUse int-}--const volumeHelperImage = volumehelper.DefaultToolsImage--type backupStorageMode string--const (- // backupStorageModeArcaneMount means backup helpers mirror an existing Arcane- // container mount at /backups. This intentionally covers any mount the Arcane- // container already has at /backups, not exclusively bind mounts.- backupStorageModeArcaneMount backupStorageMode = "arcane_mount"- // backupStorageModeNamedVolumeFallback means no suitable Arcane container- // mount was found, so Arcane's dedicated named backup volume is used.- backupStorageModeNamedVolumeFallback backupStorageMode = "named_volume_fallback"-)--const backupMountMissingWarning = "No volume is mounted at /backups in the Arcane container. Backups will only live inside Docker unless you mount a host path."-const trivyCacheVolumePruneFilterValue = libarcane.InternalResourceLabel + "=true"--type backupStorageMountInternal struct {- mode backupStorageMode- mount mount.Mount- requiresEnsure bool-}--func NewVolumeService(db *database.DB, dockerService *DockerClientService, eventService *EventService, settingsService *SettingsService, containerService *ContainerService, imageService *ImageService, backupVolumeName string) *VolumeService {- slog.Debug("volume service: new")- if strings.TrimSpace(backupVolumeName) == "" {- backupVolumeName = "arcane-backups"- }- return &VolumeService{- db: db,- dockerService: dockerService,- eventService: eventService,- settingsService: settingsService,- containerService: containerService,- imageService: imageService,- backupVolumeName: backupVolumeName,- helperByVolume: make(map[string]*volumeHelper),- }-}--func (s *VolumeService) GetVolumeByName(ctx context.Context, name string) (*volumetypes.Volume, error) {- slog.DebugContext(ctx, "volume service: get volume", "volume", name)- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, errors.WrapIf(err, "failed to connect to Docker")- }-- volResult, err := dockerClient.VolumeInspect(ctx, name, client.VolumeInspectOptions{})- if err != nil {- return nil, errors.WrapIf(err, "volume not found")- }- vol := volResult.Volume-- settings := s.settingsService.GetSettingsConfig()- usageCtx, usageCancel := context.WithTimeout(ctx, timeouts.GetDuration(settings.DockerAPITimeout.AsInt(), timeouts.DefaultDockerAPI))- defer usageCancel()- if usageVolumes, ok := docker.GetVolumeUsageDataStaleWhileRevalidate(usageCtx, dockerClient).Get(); ok {- for _, uv := range usageVolumes {- if uv.Name == vol.Name && uv.UsageData != nil {- vol.UsageData = uv.UsageData- slog.DebugContext(ctx, "attached volume usage data", "volume", vol.Name, "size_bytes", uv.UsageData.Size, "ref_count", uv.UsageData.RefCount)- break- }- }- }-- v := volumetypes.NewSummary(vol)-- containerIDs, err := docker.GetContainersUsingVolume(ctx, dockerClient, name)- if err != nil {- slog.WarnContext(ctx, "failed to get containers using volume", "volume", name, "error", err.Error())- } else {- v.Containers = containerIDs- if len(containerIDs) > 0 {- v.InUse = true- }- }-- return &v, nil-}--func (s *VolumeService) CreateVolume(ctx context.Context, options client.VolumeCreateOptions, user models.User) (*volumetypes.Volume, error) {- slog.DebugContext(ctx, "volume service: create volume", "volume", options.Name, "driver", options.Driver, "user", user.ID)- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- s.eventService.LogErrorEvent(ctx, models.EventTypeVolumeError, "volume", "", options.Name, user.ID, user.Username, "0", err, models.JSON{"action": "create", "driver": options.Driver})- return nil, errors.WrapIf(err, "failed to connect to Docker")- }-- created, err := dockerClient.VolumeCreate(ctx, options)- if err != nil {- s.eventService.LogErrorEvent(ctx, models.EventTypeVolumeError, "volume", "", options.Name, user.ID, user.Username, "0", err, models.JSON{"action": "create", "driver": options.Driver})- return nil, errors.WrapIf(err, "failed to create volume")- }-- vol, err := dockerClient.VolumeInspect(ctx, created.Volume.Name, client.VolumeInspectOptions{})- if err != nil {- s.eventService.LogErrorEvent(ctx, models.EventTypeVolumeError, "volume", created.Volume.Name, created.Volume.Name, user.ID, user.Username, "0", err, models.JSON{"action": "create", "driver": options.Driver, "step": "inspect"})- return nil, errors.WrapIf(err, "failed to inspect created volume")- }-- metadata := models.JSON{- "action": "create",- "driver": vol.Volume.Driver,- "name": vol.Volume.Name,- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeCreate, vol.Volume.Name, vol.Volume.Name, user.ID, user.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume creation action", "volume", vol.Volume.Name, "error", logErr.Error())- }-- docker.InvalidateVolumeUsageCache(dockerClient)-- return new(volumetypes.NewSummary(vol.Volume)), nil-}--func (s *VolumeService) DeleteVolume(ctx context.Context, name string, force bool, user models.User) error {- slog.DebugContext(ctx, "volume service: delete volume", "volume", name, "force", force, "user", user.ID)- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- s.eventService.LogErrorEvent(ctx, models.EventTypeVolumeError, "volume", name, name, user.ID, user.Username, "0", err, models.JSON{"action": "delete", "force": force})- return errors.WrapIf(err, "failed to connect to Docker")- }-- // Stop any read-only browse helper first; a helper mounting the volume would- // otherwise block a non-forced VolumeRemove with "volume is in use".- if stopErr := s.StopHelper(ctx, name); stopErr != nil {- slog.WarnContext(ctx, "could not stop volume browse helper before delete", "volume", name, "error", stopErr.Error())- }-- if _, err := dockerClient.VolumeRemove(ctx, name, client.VolumeRemoveOptions{- Force: force,- }); err != nil {- s.eventService.LogErrorEvent(ctx, models.EventTypeVolumeError, "volume", name, name, user.ID, user.Username, "0", err, models.JSON{"action": "delete", "force": force})- return errors.WrapIf(err, "failed to remove volume")- }-- metadata := models.JSON{- "action": "delete",- "name": name,- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeDelete, name, name, user.ID, user.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume deletion action", "volume", name, "error", logErr.Error())- }-- s.removeHelperEntry(name)- docker.InvalidateVolumeUsageCache(dockerClient)- return nil-}--func (s *VolumeService) PruneVolumes(ctx context.Context) (*volumetypes.PruneReport, error) {- slog.DebugContext(ctx, "volume service: prune volumes")- return s.PruneVolumesWithOptions(ctx, false)-}--func (s *VolumeService) PruneVolumesWithOptions(ctx context.Context, all bool) (*volumetypes.PruneReport, error) {- slog.DebugContext(ctx, "volume service: prune volumes with options", "all", all)- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, errors.WrapIf(err, "failed to connect to Docker")- }-- // Stop all read-only browse helpers first; a helper mounting a volume marks it- // "in use" and would prevent VolumePrune from reclaiming an otherwise-unused- // volume. Helpers are re-created on demand on the next browse request.- s.CleanupHelperContainers(ctx)-- preserveTrivyCache := s.preserveTrivyCacheOnVolumePruneInternal()-- // Docker's VolumesPrune behavior (API v1.42+):- // - Without 'all' flag: Only removes anonymous (unnamed) volumes that are not in use- // - With 'all=true' flag: Removes ALL unused volumes (both named and anonymous)- // Note: Volumes are considered "in use" if referenced by any container (running or stopped)- volumePruneOptions := buildVolumePruneOptionsInternal(all, preserveTrivyCache)- volumePruneResult, err := dockerClient.VolumePrune(ctx, volumePruneOptions)- if err != nil {- return nil, errors.WrapIf(err, "failed to prune volumes")- }-- metadata := buildVolumePruneMetadataInternal(all, len(volumePruneResult.Report.VolumesDeleted), volumePruneResult.Report.SpaceReclaimed, preserveTrivyCache)- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeDelete, "", "bulk_prune", systemUser.ID, systemUser.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume prune action", "error", logErr.Error())- }-- for _, volumeName := range volumePruneResult.Report.VolumesDeleted {- s.removeHelperEntry(volumeName)- }-- docker.InvalidateVolumeUsageCache(dockerClient)-- return &volumetypes.PruneReport{- VolumesDeleted: volumePruneResult.Report.VolumesDeleted,- SpaceReclaimed: volumePruneResult.Report.SpaceReclaimed,- }, nil-}--func (s *VolumeService) preserveTrivyCacheOnVolumePruneInternal() bool {- if s.settingsService == nil {- return true- }-- return s.settingsService.GetSettingsConfig().TrivyPreserveCacheOnVolumePrune.IsTrue()-}--func buildVolumePruneOptionsInternal(all, preserveTrivyCache bool) client.VolumePruneOptions {- options := client.VolumePruneOptions{- All: all,- }- if !preserveTrivyCache {- return options- }-- filters := make(client.Filters)- filters = filters.Add("label!", trivyCacheVolumePruneFilterValue)- options.Filters = filters-- return options-}--func buildVolumePruneMetadataInternal(all bool, volumesDeleted int, spaceReclaimed uint64, preserveTrivyCache bool) models.JSON {- return models.JSON{- "action": "prune",- "all": all,- "volumesDeleted": volumesDeleted,- "spaceReclaimed": spaceReclaimed,- "preserveTrivyCache": preserveTrivyCache,- "trivyCacheFilterLabel": trivyCacheVolumePruneFilterValue,- }-}--// --- Volume Browsing & Backup -----// isBrowsableVolumeInternal returns an error if the volume uses driver options-// that prevent it from being mounted inside a helper container, such as-// type=none or o=bind (host bind-mounts that require a device path on the host).-func (s *VolumeService) isBrowsableVolumeInternal(ctx context.Context, volumeName string) error {- vol, err := s.GetVolumeByName(ctx, volumeName)- if err != nil {- return errors.WrapIf(err, "failed to inspect volume")- }- if vol.Options["type"] == "none" || strings.Contains(vol.Options["o"], "bind") {- return errors.Errorf("volume %q uses a custom mount configuration and cannot be browsed", volumeName)- }- return nil-}--func (s *VolumeService) ListDirectory(ctx context.Context, volumeName, dirPath string) ([]volumetypes.FileEntry, error) {- slog.DebugContext(ctx, "volume service: list directory", "volume", volumeName, "path", dirPath)-- if err := s.isBrowsableVolumeInternal(ctx, volumeName); err != nil {- return nil, err- }-- sanitizedPath, err := utils.SanitizeBrowsePath(dirPath)- if err != nil {- return nil, errors.WrapIf(err, "invalid path")- }-- containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, true)- if err != nil {- return nil, err- }- defer cleanup()-- targetPath := path.Join("/volume", sanitizedPath)- cmd := []string{- "sh",- "-c",- `find "$1" -mindepth 1 -maxdepth 1 | while IFS= read -r f; do out=$(stat -c "%s %Y %f %A" -- "$f" 2>/dev/null) || continue; printf "%s\0%s\0" "$f" "$out"; done`,- "sh",- targetPath,- }- stdout, _, err := s.execInContainerInternal(ctx, containerID, cmd)- if err != nil {- return nil, errors.WrapIf(err, "failed to list directory")- }-- lines := strings.Split(stdout, "\x00")- entries := make([]volumetypes.FileEntry, 0)- for i := 0; i+1 < len(lines); i += 2 {- fullPath := lines[i]- meta := strings.Fields(strings.TrimSpace(lines[i+1]))- if fullPath == "" || len(meta) < 4 {- continue- }- name := path.Base(fullPath)- size, _ := strconv.ParseInt(meta[0], 10, 64)- modTimeSec, _ := strconv.ParseInt(meta[1], 10, 64)- mode := meta[3]-- isDir := strings.HasPrefix(mode, "d")- isSymlink := strings.HasPrefix(mode, "l")-- relPath := strings.TrimPrefix(fullPath, "/volume")- if relPath == "" {- relPath = "/"- }-- entry := volumetypes.FileEntry{- Name: name,- Path: relPath,- IsDirectory: isDir,- Size: size,- ModTime: time.Unix(modTimeSec, 0),- Mode: mode,- IsSymlink: isSymlink,- }-- if isSymlink {- // Use readlink without -f to get the raw symlink target (not resolved)- // This prevents exposing paths outside the volume- target, _, _ := s.execInContainerInternal(ctx, containerID, []string{"readlink", fullPath})- target = strings.TrimSpace(target)- if target != "" {- // If target is relative, it's safe to show- // If target is absolute and within /volume, strip the /volume prefix- // If target points outside /volume, indicate it's external- switch {- case strings.HasPrefix(target, "/volume/"):- entry.LinkTarget = strings.TrimPrefix(target, "/volume")- case strings.HasPrefix(target, "/volume"):- entry.LinkTarget = "/"- case !strings.HasPrefix(target, "/"):- // Relative path - safe to show as-is- entry.LinkTarget = target- default:- // Absolute path outside /volume - indicate it's external- entry.LinkTarget = "(external)"- }- }- }-- entries = append(entries, entry)- }-- return entries, nil-}--func (s *VolumeService) GetFileContent(ctx context.Context, volumeName, filePath string, maxBytes int64) ([]byte, string, error) {- slog.DebugContext(ctx, "volume service: get file content", "volume", volumeName, "path", filePath, "max_bytes", maxBytes)-- if err := s.isBrowsableVolumeInternal(ctx, volumeName); err != nil {- return nil, "", err- }-- sanitizedPath, err := utils.SanitizeBrowsePath(filePath)- if err != nil {- return nil, "", errors.WrapIf(err, "invalid path")- }-- containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, true)- if err != nil {- return nil, "", err- }- defer cleanup()-- targetPath := path.Join("/volume", sanitizedPath)- cmd := []string{"head", "-c", strconv.FormatInt(maxBytes, 10), targetPath}- stdout, _, err := s.execInContainerInternal(ctx, containerID, cmd)- if err != nil {- return nil, "", errors.WrapIf(err, "failed to read file")- }-- content := []byte(stdout)- mimeType := http.DetectContentType(content)-- return content, mimeType, nil-}--func (s *VolumeService) DownloadFile(ctx context.Context, volumeName, filePath string) (io.ReadCloser, int64, error) {- slog.DebugContext(ctx, "volume service: download file", "volume", volumeName, "path", filePath)-- if err := s.isBrowsableVolumeInternal(ctx, volumeName); err != nil {- return nil, 0, err- }-- sanitizedPath, err := utils.SanitizeBrowsePath(filePath)- if err != nil {- return nil, 0, errors.WrapIf(err, "invalid path")- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, 0, err- }-- containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, true)- if err != nil {- return nil, 0, err- }-- targetPath := path.Join("/volume", sanitizedPath)- return s.downloadFileFromContainerInternal(ctx, dockerClient, containerID, targetPath, cleanup)-}--func getVolumeHelperImageInternal(ctx context.Context, dockerService *DockerClientService, imageService *ImageService, dockerClient *client.Client) (string, error) {- slog.DebugContext(ctx, "volume service: resolve helper image")- var err error- if dockerClient == nil {- if dockerService == nil {- return "", errors.New("docker service unavailable")- }- dockerClient, err = dockerService.GetClient(ctx)- if err != nil {- return "", errors.WrapIf(err, "failed to get docker client")- }- }-- if _, err := dockerClient.ImageInspect(ctx, volumeHelperImage); err == nil {- slog.InfoContext(ctx, "volume service: helper image strategy selected", "strategy", "tools-local", "image", volumeHelperImage)- return volumeHelperImage, nil- }-- var pullErr error- if imageService != nil {- pullImageErr := imageService.PullImage(ctx, volumeHelperImage, io.Discard, systemUser, nil)- if pullImageErr == nil {- slog.InfoContext(ctx, "volume service: helper image strategy selected", "strategy", "tools-pulled", "image", volumeHelperImage)- return volumeHelperImage, nil- }- pullErr = pullImageErr- slog.WarnContext(ctx, "volume service: failed to pull tools helper image, attempting arcane fallback", "error", pullImageErr.Error())- } else {- pullErr = errors.New("image service unavailable")- slog.WarnContext(ctx, "volume service: image service unavailable, attempting arcane fallback")- }-- if fallback, ok := volumehelper.ResolveArcaneRuntimeImage(ctx, dockerClient).Get(); ok {- slog.InfoContext(ctx, "volume service: helper image strategy selected", "strategy", "arcane-fallback", "source", fallback.Source, "image", fallback.Image)- return fallback.Image, nil- }-- return "", errors.WrapIf(pullErr, "failed to resolve helper image: tools image unavailable and arcane fallback not found")-}--func resolveBackupStorageMountFromMountsInternal(mounts []container.MountPoint, target string, readOnly bool) mo.Option[backupStorageMountInternal] {- mirroredMount := docker.MountForDestination(mounts, "/backups", target)- if mirroredMount == nil {- return mo.None[backupStorageMountInternal]()- }- // MountForDestination only returns non-nil for bind and named volume mounts.-- if !readOnly && mirroredMount.ReadOnly {- slog.Warn("volume service: requested writable backup mount but source is read-only; writes may fail")- }- mirroredMount.ReadOnly = readOnly-- return mo.Some(backupStorageMountInternal{- mode: backupStorageModeArcaneMount,- mount: *mirroredMount,- })-}--func (s *VolumeService) resolveBackupStorageMountInternal(ctx context.Context, dockerClient *client.Client, target string, readOnly bool) backupStorageMountInternal {- if dockerClient != nil {- containerID := s.getArcaneContainerIDInternal(ctx, dockerClient)- if containerID != "" {- inspect, err := libarcane.ContainerInspectWithCompatibility(ctx, dockerClient, containerID, client.ContainerInspectOptions{})- if err != nil {- slog.WarnContext(ctx, "volume service: failed to inspect arcane container for backup mount resolution, falling back to named volume", "container_id", containerID, "error", err.Error())- } else if resolved, ok := resolveBackupStorageMountFromMountsInternal(inspect.Container.Mounts, target, readOnly).Get(); ok {- return resolved- }- }- }-- return backupStorageMountInternal{- mode: backupStorageModeNamedVolumeFallback,- mount: mount.Mount{- Type: mount.TypeVolume,- Source: s.backupVolumeName,- Target: target,- ReadOnly: readOnly,- },- requiresEnsure: true,- }-}--func (s *VolumeService) resolveUsableBackupStorageMountInternal(ctx context.Context, dockerClient *client.Client, target string, readOnly bool) (backupStorageMountInternal, error) {- backupStorage := s.resolveBackupStorageMountInternal(ctx, dockerClient, target, readOnly)- if backupStorage.requiresEnsure {- if err := s.ensureBackupVolumeInternal(ctx); err != nil {- return backupStorageMountInternal{}, err- }- }- return backupStorage, nil-}--func backupMountWarningForStorageInternal(storage backupStorageMountInternal) string {- if storage.mode == backupStorageModeArcaneMount {- return ""- }- return backupMountMissingWarning-}--func backupMountWarningFromArcaneMountsInternal(mounts []container.MountPoint) string {- backupStorage, ok := resolveBackupStorageMountFromMountsInternal(mounts, "/backups", true).Get()- if ok {- return backupMountWarningForStorageInternal(backupStorage)- }-- // Backward compatibility: historically either /backups or /restores mount- // suppressed the warning. Preserve that user-visible behavior.- for _, m := range mounts {- if m.Destination == "/restores" {- return ""- }- }-- return backupMountMissingWarning-}--func (s *VolumeService) BackupMountWarning(ctx context.Context) string {- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return ""- }-- containerID := s.getArcaneContainerIDInternal(ctx, dockerClient)- if containerID == "" {- // Cannot determine Arcane mount status (e.g. running outside Docker); suppress warning.- return ""- }-- inspect, err := libarcane.ContainerInspectWithCompatibility(ctx, dockerClient, containerID, client.ContainerInspectOptions{})- if err != nil {- return ""- }-- return backupMountWarningFromArcaneMountsInternal(inspect.Container.Mounts)-}--func (s *VolumeService) getArcaneContainerIDInternal(ctx context.Context, dockerClient *client.Client) string {- hostname, _ := os.Hostname()- if hostname != "" {- if inspect, err := libarcane.ContainerInspectWithCompatibility(ctx, dockerClient, hostname, client.ContainerInspectOptions{}); err == nil {- return inspect.Container.ID- }- }-- filter := make(client.Filters)- filter = filter.Add("label", "com.getarcaneapp.arcane=true")- containers, err := dockerClient.ContainerList(ctx, client.ContainerListOptions{Filters: filter, All: true})- if err != nil || len(containers.Items) == 0 {- return ""- }-- for _, c := range containers.Items {- if c.State == container.StateRunning {- return c.ID- }- }-- return containers.Items[0].ID-}--func (s *VolumeService) createBackupTempContainerWithMountInternal(ctx context.Context, dockerClient *client.Client, helperImage string, backupMount mount.Mount) (string, func(), error) {- var err error- if dockerClient == nil {- dockerClient, err = s.dockerService.GetClient(ctx)- if err != nil {- return "", nil, err- }- }-- if strings.TrimSpace(helperImage) == "" {- helperImage, err = getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)- if err != nil {- return "", nil, err- }- }-- config := &container.Config{- Image: helperImage,- Cmd: []string{"sleep", "infinity"},- NetworkDisabled: true,- Labels: volumehelper.Labels(),- }-- hostConfig := volumehelper.HostConfig(helperImage, nil, []mount.Mount{backupMount})-- resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{- Config: config,- HostConfig: hostConfig,- })- if err != nil {- return "", nil, errors.WrapIf(err, "failed to create backup temp container")- }-- if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {- _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())- return "", nil, errors.WrapIf(err, "failed to start backup temp container")- }-- cleanup := func() {- _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())- }-- return resp.ID, cleanup, nil-}--func (s *VolumeService) createBackupTempContainerInternal(ctx context.Context, dockerClient *client.Client, target string, readOnly bool) (string, func(), error) {- slog.DebugContext(ctx, "volume service: create backup temp container", "target", target, "read_only", readOnly)- var err error- if dockerClient == nil {- dockerClient, err = s.dockerService.GetClient(ctx)- if err != nil {- return "", nil, err- }- }-- backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, target, readOnly)- if err != nil {- return "", nil, err- }-- return s.createBackupTempContainerWithMountInternal(ctx, dockerClient, "", backupStorage.mount)-}--type cleanupReadCloser struct {- io.Reader- io.Closer-- cleanup func()-}--func isLegacyVolumeHelperContainerInternal(c container.Summary) bool {- if !libarcane.IsInternalContainer(c.Labels) {- return false- }-- command := strings.ToLower(c.Command)- if !strings.Contains(command, "sleep") || !strings.Contains(command, "infinity") {- return false- }-- for _, m := range c.Mounts {- if m.Destination == "/volume" {- return true- }- }-- return false-}--func isVolumeHelperContainerInternal(c container.Summary) bool {- if isLegacyVolumeHelperContainerInternal(c) {- return true- }- if !libarcane.IsInternalContainer(c.Labels) {- return false- }-- return strings.EqualFold(c.Labels[volumehelper.ContainerLabel], "true")-}--func (c *cleanupReadCloser) Close() error {- err := c.Closer.Close()- c.cleanup()- return err-}--func (s *VolumeService) createTempContainerInternal(ctx context.Context, volumeName string, readOnly bool) (string, func(), error) {- slog.DebugContext(ctx, "volume service: create temp container", "volume", volumeName, "read_only", readOnly)- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return "", nil, err- }-- if !readOnly {- return s.startHelperContainerInternal(ctx, dockerClient, volumeName, false)- }-- // Read-only helpers are shared and reaped when idle; the caller's cleanup- // releases its in-use hold instead of removing the container. The resolve →- // acquire gap is racy against the reaper, so retry a resolve whose helper- // was reaped before this caller could take its hold.- for range 3 {- containerID, err := s.resolveReadOnlyHelperInternal(ctx, dockerClient, volumeName)- if err != nil {- return "", nil, err- }- if release, ok := s.acquireHelperInternal(volumeName, containerID); ok {- return containerID, release, nil- }- }- return "", nil, errors.New("failed to acquire volume helper container")-}--// resolveReadOnlyHelperInternal returns the shared read-only helper container ID-// for volumeName, creating it if needed. singleflight collapses concurrent-// misses: without it both requests create a helper and the loser is orphaned,-// holding the volume mount with no cleanup path until Arcane restarts. The-// creation itself can pull an image, so it must not run under helperMu.-func (s *VolumeService) resolveReadOnlyHelperInternal(ctx context.Context, dockerClient *client.Client, volumeName string) (string, error) {- resultCh := s.helperGroup.DoChan(volumeName, func() (any, error) {- if containerID, ok := s.getReusableReadOnlyContainerInternal(ctx, dockerClient, volumeName).Get(); ok {- return containerID, nil- }-- // Detached from the caller's context so a client that walks away- // mid-create doesn't abort the helper the other waiters are blocked on,- // but still bounded: WithoutCancel alone would let a hung daemon or- // image pull wedge this singleflight key forever, queueing every later- // browse of the volume behind it with no recovery until restart.- createCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), timeouts.DefaultDockerImagePull)- defer cancel()-- containerID, _, createErr := s.startHelperContainerInternal(createCtx, dockerClient, volumeName, true)- if createErr != nil {- return nil, createErr- }-- s.helperMu.Lock()- s.helperByVolume[volumeName] = &volumeHelper{id: containerID, lastUsedAt: time.Now()}- s.helperMu.Unlock()- return containerID, nil- })-- // DoChan instead of Do so a caller whose request is canceled unblocks- // immediately; the shared create keeps running for the remaining waiters- // (and lands in helperByVolume for the next request even if none remain).- var shared any- select {- case <-ctx.Done():- return "", errors.WrapIf(ctx.Err(), "canceled waiting for volume helper container")- case result := <-resultCh:- if result.Err != nil {- return "", result.Err- }- shared = result.Val- }-- containerID, ok := shared.(string)- if !ok || containerID == "" {- return "", errors.New("failed to resolve volume helper container")- }- return containerID, nil-}--// acquireHelperInternal takes an in-use hold on volumeName's helper, reporting-// false when the helper has been removed (e.g. reaped) since it was resolved.-// The returned release drops the hold and refreshes the idle clock, so the idle-// timeout is measured from the end of the last request, not its start — a-// download running longer than the timeout used to have its helper reaped out-// from under it, truncating the stream.-func (s *VolumeService) acquireHelperInternal(volumeName, containerID string) (func(), bool) {- s.helperMu.Lock()- defer s.helperMu.Unlock()-- helper := s.helperByVolume[volumeName]- if helper == nil || helper.id != containerID {- return nil, false- }-- helper.inUse++- helper.lastUsedAt = time.Now()-- return func() {- s.helperMu.Lock()- defer s.helperMu.Unlock()- if current := s.helperByVolume[volumeName]; current == helper {- helper.inUse--- helper.lastUsedAt = time.Now()- }- }, true-}--// startHelperContainerInternal creates and starts a volume helper container and-// returns a cleanup that removes it. Tracking of reusable read-only helpers is-// the caller's responsibility.-func (s *VolumeService) startHelperContainerInternal(ctx context.Context, dockerClient *client.Client, volumeName string, readOnly bool) (string, func(), error) {- helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)- if err != nil {- return "", nil, err- }-- config := &container.Config{- Image: helperImage,- Cmd: []string{"sleep", "infinity"},- NetworkDisabled: true,- Labels: volumehelper.Labels(),- }-- hostConfig := volumehelper.HostConfig(helperImage, []string{- fmt.Sprintf("%s:/volume%s", volumeName, func() string {- if readOnly {- return ":ro"- }- return ""- }()),- }, nil)-- resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{- Config: config,- HostConfig: hostConfig,- })- if err != nil {- return "", nil, errors.WrapIf(err, "failed to create temp container")- }-- if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {- _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())- return "", nil, errors.WrapIf(err, "failed to start temp container")- }-- cleanup := func() {- _, _ = dockerClient.ContainerRemove(context.WithoutCancel(ctx), resp.ID, volumehelper.RemoveOptions())- }-- return resp.ID, cleanup, nil-}--func (s *VolumeService) getReusableReadOnlyContainerInternal(ctx context.Context, dockerClient *client.Client, volumeName string) mo.Option[string] {- s.helperMu.Lock()- helper := s.helperByVolume[volumeName]- s.helperMu.Unlock()- if helper == nil || helper.id == "" {- return mo.None[string]()- }-- inspect, err := libarcane.ContainerInspectWithCompatibility(ctx, dockerClient, helper.id, client.ContainerInspectOptions{})- if err != nil || inspect.Container.State == nil || !inspect.Container.State.Running {- s.helperMu.Lock()- delete(s.helperByVolume, volumeName)- s.helperMu.Unlock()- return mo.None[string]()- }-- s.touchHelperInternal(volumeName)-- return mo.Some(helper.id)-}--// touchHelperInternal records that the helper for volumeName just serviced a-// request, resetting its idle clock. No-op if the entry is gone.-func (s *VolumeService) touchHelperInternal(volumeName string) {- s.helperMu.Lock()- defer s.helperMu.Unlock()- if helper := s.helperByVolume[volumeName]; helper != nil {- helper.lastUsedAt = time.Now()- }-}--func (s *VolumeService) CleanupHelperContainers(ctx context.Context) {- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- slog.WarnContext(ctx, "failed to get docker client for helper cleanup", "error", err)- return- }-- s.helperMu.Lock()- helperIDs := make([]string, 0, len(s.helperByVolume))- for _, helper := range s.helperByVolume {- if helper != nil && helper.id != "" {- helperIDs = append(helperIDs, helper.id)- }- }- s.helperByVolume = make(map[string]*volumeHelper)- s.helperMu.Unlock()-- for _, containerID := range helperIDs {- if _, err := dockerClient.ContainerRemove(ctx, containerID, volumehelper.RemoveOptions()); err != nil {- slog.WarnContext(ctx, "failed to remove helper container", "container_id", containerID, "error", err.Error())- }- }-}--// ReapIdleHelpers removes reused read-only browse helper containers that have-// not serviced a request within idleTimeout. It is map-driven (orphaned helpers-// not tracked in helperByVolume are left to the startup orphan sweep). Entries-// are removed from the map before the container is removed, so a concurrent-// request simply gets a cache miss and re-creates a fresh helper.-func (s *VolumeService) ReapIdleHelpers(ctx context.Context, idleTimeout time.Duration) (int, error) {- if idleTimeout <= 0 {- return 0, nil- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return 0, errors.WrapIf(err, "failed to get docker client for idle helper reap")- }-- staleIDs := s.collectStaleHelperIDsInternal(time.Now(), idleTimeout)-- removed := 0- for _, containerID := range staleIDs {- if _, err := dockerClient.ContainerRemove(ctx, containerID, volumehelper.RemoveOptions()); err != nil {- slog.WarnContext(ctx, "failed to remove idle helper container", "container_id", containerID, "error", err.Error())- continue- }- removed++- }-- return removed, nil-}--// collectStaleHelperIDsInternal removes idle (and any nil) entries from the helper-// map and returns the container IDs that should be removed. Pure map/mutex-// bookkeeping with no Docker calls, so it can be unit-tested directly. Entries are-// dropped before their containers are removed so a concurrent request gets a clean-// cache miss and re-creates a fresh helper.-func (s *VolumeService) collectStaleHelperIDsInternal(now time.Time, idleTimeout time.Duration) []string {- staleIDs := make([]string, 0)- s.helperMu.Lock()- defer s.helperMu.Unlock()- for volumeName, helper := range s.helperByVolume {- if helper == nil {- delete(s.helperByVolume, volumeName)- continue- }- // A helper with active holds is serving a request right now (e.g. a- // download streaming for longer than the idle timeout); it becomes- // reapable once released, since release refreshes lastUsedAt.- if helper.inUse > 0 {- continue- }- if now.Sub(helper.lastUsedAt) >= idleTimeout {- staleIDs = append(staleIDs, helper.id)- delete(s.helperByVolume, volumeName)- }- }- return staleIDs-}--// StopHelper removes the reused read-only browse helper for a single volume, if-// one exists. It is idempotent: stopping a volume with no active helper returns-// nil.-func (s *VolumeService) StopHelper(ctx context.Context, volumeName string) error {- if strings.TrimSpace(volumeName) == "" {- return nil- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return errors.WrapIf(err, "failed to get docker client for helper stop")- }-- containerID := s.takeHelperIDInternal(volumeName)- if containerID == "" {- return nil- }-- if _, err := dockerClient.ContainerRemove(ctx, containerID, volumehelper.RemoveOptions()); err != nil {- return errors.WrapIf(err, "failed to remove helper container")- }-- return nil-}--// takeHelperIDInternal removes the helper entry for volumeName and returns its-// container ID, or "" if there was none. Pure map/mutex bookkeeping.-func (s *VolumeService) takeHelperIDInternal(volumeName string) string {- s.helperMu.Lock()- defer s.helperMu.Unlock()- helper := s.helperByVolume[volumeName]- delete(s.helperByVolume, volumeName)- if helper == nil {- return ""- }- return helper.id-}--func (s *VolumeService) CleanupOrphanedVolumeHelpers(ctx context.Context) (int, error) {- slog.DebugContext(ctx, "volume service: cleanup orphaned volume helper containers")-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return 0, errors.WrapIf(err, "failed to get docker client for orphan helper cleanup")- }-- containers, err := dockerClient.ContainerList(ctx, client.ContainerListOptions{All: true})- if err != nil {- return 0, errors.WrapIf(err, "failed to list containers for orphan helper cleanup")- }-- removedCount := 0- for _, c := range containers.Items {- if !isVolumeHelperContainerInternal(c) {- continue- }-- if _, err := dockerClient.ContainerRemove(ctx, c.ID, volumehelper.RemoveOptions()); err != nil {- slog.WarnContext(ctx,- "volume service: failed to remove orphaned volume helper container",- "container_id", c.ID,- "container_names", c.Names,- "error", err.Error(),- )- continue- }-- removedCount++- }-- return removedCount, nil-}--func (s *VolumeService) removeHelperEntry(volumeName string) {- if strings.TrimSpace(volumeName) == "" {- return- }- s.helperMu.Lock()- delete(s.helperByVolume, volumeName)- s.helperMu.Unlock()-}--func (s *VolumeService) execInContainerInternal(ctx context.Context, containerID string, cmd []string) (string, string, error) {- slog.DebugContext(ctx, "volume service: exec in container", "container_id", containerID, "cmd", cmd)- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return "", "", err- }-- execConfig := client.ExecCreateOptions{- AttachStdout: true,- AttachStderr: true,- Cmd: cmd,- }-- execResp, err := dockerClient.ExecCreate(ctx, containerID, execConfig)- if err != nil {- return "", "", err- }-- resp, err := dockerClient.ExecAttach(ctx, execResp.ID, client.ExecAttachOptions{})- if err != nil {- return "", "", err- }- defer resp.Close()-- var stdout, stderr bytes.Buffer- _, err = stdcopy.StdCopy(&stdout, &stderr, resp.Reader)- if err != nil {- return "", "", err- }-- inspect, err := dockerClient.ExecInspect(ctx, execResp.ID, client.ExecInspectOptions{})- if err != nil {- return stdout.String(), stderr.String(), errors.WrapIf(err, "failed to inspect exec result")- }-- if inspect.ExitCode != 0 {- execErr := strings.TrimSpace(stderr.String())- if execErr == "" {- execErr = strings.TrimSpace(stdout.String())- }- if execErr != "" {- return stdout.String(), stderr.String(), errors.Errorf("command exited with code %d: %s", inspect.ExitCode, execErr)- }- return stdout.String(), stderr.String(), errors.Errorf("command exited with code %d", inspect.ExitCode)- }-- return stdout.String(), stderr.String(), nil-}--func (s *VolumeService) DeleteFile(ctx context.Context, volumeName, filePath string, user *models.User) error {- slog.DebugContext(ctx, "volume service: delete file", "volume", volumeName, "path", filePath)-- sanitizedPath, err := utils.SanitizeBrowsePath(filePath)- if err != nil {- return errors.WrapIf(err, "invalid path")- }- // Prevent deleting root- if sanitizedPath == "/" {- return errors.New("cannot delete root directory")- }-- containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, false)- if err != nil {- return err- }- defer cleanup()-- targetPath := path.Join("/volume", sanitizedPath)- _, stderr, err := s.execInContainerInternal(ctx, containerID, []string{"rm", "-rf", targetPath})- if err != nil {- return err- }- if stderr != "" {- return errors.Errorf("delete failed: %s", stderr)- }-- actingUser := user- if actingUser == nil {- actingUser = &systemUser- }- metadata := models.JSON{- "action": "file_delete",- "path": filePath,- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeFileDelete, volumeName, volumeName, actingUser.ID, actingUser.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume file delete event", "volume", volumeName, "error", logErr.Error())- }- return nil-}--func (s *VolumeService) CreateDirectory(ctx context.Context, volumeName, dirPath string, user *models.User) error {- slog.DebugContext(ctx, "volume service: create directory", "volume", volumeName, "path", dirPath)-- sanitizedPath, err := utils.SanitizeBrowsePath(dirPath)- if err != nil {- return errors.WrapIf(err, "invalid path")- }-- containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, false)- if err != nil {- return err- }- defer cleanup()-- targetPath := path.Join("/volume", sanitizedPath)- _, stderr, err := s.execInContainerInternal(ctx, containerID, []string{"mkdir", "-p", targetPath})- if err != nil {- return err- }- if stderr != "" {- return errors.Errorf("mkdir failed: %s", stderr)- }-- actingUser := user- if actingUser == nil {- actingUser = &systemUser- }- metadata := models.JSON{- "action": "file_create",- "path": dirPath,- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeFileCreate, volumeName, volumeName, actingUser.ID, actingUser.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume file create event", "volume", volumeName, "error", logErr.Error())- }- return nil-}--func (s *VolumeService) UploadFile(ctx context.Context, volumeName, destPath string, content io.Reader, filename string, user *models.User) error {- slog.DebugContext(ctx, "volume service: upload file", "volume", volumeName, "dest_path", destPath, "filename", filename)-- sanitizedPath, err := utils.SanitizeBrowsePath(destPath)- if err != nil {- return errors.WrapIf(err, "invalid path")- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return err- }-- containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, false)- if err != nil {- return err- }- defer cleanup()-- var buf bytes.Buffer- tw := tar.NewWriter(&buf)-- contentBytes, err := io.ReadAll(content)- if err != nil {- return err- }-- hdr := &tar.Header{- Name: filename,- Mode: 0o644,- Size: int64(len(contentBytes)),- }- if err := tw.WriteHeader(hdr); err != nil {- return err- }- if _, err := tw.Write(contentBytes); err != nil {- return err- }- if err := tw.Close(); err != nil {- return err- }-- targetDir := path.Join("/volume", sanitizedPath)- _, err = dockerClient.CopyToContainer(ctx, containerID, client.CopyToContainerOptions{- DestinationPath: targetDir,- Content: &buf,- })- if err != nil {- return errors.WrapIf(err, "failed to upload")- }-- actingUser := user- if actingUser == nil {- actingUser = &systemUser- }- metadata := models.JSON{- "action": "file_upload",- "path": destPath,- "filename": filename,- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeFileUpload, volumeName, volumeName, actingUser.ID, actingUser.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume file upload event", "volume", volumeName, "error", logErr.Error())- }-- return nil-}--func (s *VolumeService) ensureBackupVolumeInternal(ctx context.Context) error {- slog.DebugContext(ctx, "volume service: ensure backup volume", "backup_volume", s.backupVolumeName)- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return err- }-- _, err = dockerClient.VolumeInspect(ctx, s.backupVolumeName, client.VolumeInspectOptions{})- if err != nil {- _, err = dockerClient.VolumeCreate(ctx, client.VolumeCreateOptions{- Name: s.backupVolumeName,- })- if err != nil {- return errors.WrapIf(err, "failed to create backup volume")- }- }- return nil-}--func (s *VolumeService) CreateBackup(ctx context.Context, volumeName string, user models.User) (*models.VolumeBackup, error) {- slog.DebugContext(ctx, "volume service: create backup", "volume", volumeName, "user", user.ID)- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, err- }-- backupID := fmt.Sprintf("%s-%d-%s", volumeName, time.Now().UnixNano(), uuid.NewString()[:8])- filename, err := s.backupArchiveFilenameInternal(backupID)- if err != nil {- return nil, err- }-- helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)- if err != nil {- return nil, err- }-- backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, "/backups", false)- if err != nil {- return nil, err- }-- config := &container.Config{- Image: helperImage,- Cmd: []string{"sh", "-c", fmt.Sprintf("tar -czf /backups/%s -C /volume .", filename)},- Labels: volumehelper.Labels(),- }-- hostConfig := volumehelper.HostConfig(helperImage, []string{- volumeName + ":/volume:ro",- }, []mount.Mount{backupStorage.mount})-- resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{- Config: config,- HostConfig: hostConfig,- })- if err != nil {- return nil, errors.WrapIf(err, "failed to create backup container")- }-- if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {- _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())- return nil, errors.WrapIf(err, "failed to start backup container")- }-- waitResult := dockerClient.ContainerWait(ctx, resp.ID, client.ContainerWaitOptions{Condition: container.WaitConditionNotRunning})- select {- case err := <-waitResult.Error:- if err != nil {- return nil, err- }- case status := <-waitResult.Result:- if status.StatusCode != 0 {- return nil, errors.Errorf("backup container exited with status %d", status.StatusCode)- }- }-- sizeCheckMount := backupStorage.mount- sizeCheckMount.Target = "/volume"- sizeCheckMount.ReadOnly = true-- tempContainerID, cleanup, err := s.createBackupTempContainerWithMountInternal(ctx, dockerClient, "", sizeCheckMount)- if err != nil {- return nil, err- }- defer cleanup()-- sizeStr, _, err := s.execInContainerInternal(ctx, tempContainerID, []string{"stat", "-c", "%s", path.Join("/volume", filename)})- if err != nil {- return nil, err- }- size, err := strconv.ParseInt(strings.TrimSpace(sizeStr), 10, 64)- if err != nil {- return nil, err- }-- backup := &models.VolumeBackup{- VolumeName: volumeName,- Size: size,- CreatedAt: time.Now(),- }- backup.ID = backupID-- if err := s.db.WithContext(ctx).Create(backup).Error; err != nil {- return nil, err- }-- metadata := models.JSON{- "action": "backup_create",- "backup_id": backup.ID,- "filename": filename,- "size": size,- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupCreate, volumeName, volumeName, user.ID, user.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume backup create event", "volume", volumeName, "error", logErr.Error())- }-- return backup, nil-}--func (s *VolumeService) ListBackupsPaginated(ctx context.Context, volumeName string, params pagination.QueryParams) ([]models.VolumeBackup, pagination.Response, error) {- slog.DebugContext(ctx, "volume service: list backups paginated", "volume", volumeName, "search", params.Search, "sort", params.Sort, "order", params.Order, "start", params.Start, "limit", params.Limit)- var backups []models.VolumeBackup- query := s.db.WithContext(ctx).Model(&models.VolumeBackup{}).Where("volume_name = ?", volumeName)-- if params.Search != "" {- query = query.Where("id LIKE ?", "%"+params.Search+"%")- }-- var totalItems int64- if err := query.Count(&totalItems).Error; err != nil {- return nil, pagination.Response{}, err- }-- sortCol := "created_at"- sortOrder := "DESC"- if params.Sort != "" {- switch params.Sort {- case "createdAt", "created_at":- sortCol = "created_at"- case "id":- sortCol = "id"- case "size":- sortCol = "size"- default:- sortCol = "created_at"- }-- if params.Order == pagination.SortDesc {- sortOrder = "DESC"- } else {- sortOrder = "ASC"- }- }- query = query.Order(fmt.Sprintf("%s %s", sortCol, sortOrder))-- if params.Limit > 0 {- query = query.Offset(params.Start).Limit(params.Limit)- }-- if err := query.Find(&backups).Error; err != nil {- return nil, pagination.Response{}, err- }-- paginationResp := s.buildPaginationResponseFromCountsInternal(totalItems, totalItems, params)- return backups, paginationResp, nil-}--func (s *VolumeService) buildPaginationResponseFromCountsInternal(totalCount int64, totalAvailable int64, params pagination.QueryParams) pagination.Response {- slog.Debug("volume service: build pagination response", "total_count", totalCount, "total_available", totalAvailable, "start", params.Start, "limit", params.Limit)- totalPages := int64(0)- if params.Limit > 0 {- totalPages = (totalCount + int64(params.Limit) - 1) / int64(params.Limit)- }-- page := 1- if params.Limit > 0 {- page = (params.Start / params.Limit) + 1- }-- return pagination.Response{- TotalPages: totalPages,- TotalItems: totalCount,- CurrentPage: page,- ItemsPerPage: params.Limit,- GrandTotalItems: totalAvailable,- }-}--func (s *VolumeService) ListBackups(ctx context.Context, volumeName string) ([]models.VolumeBackup, error) {- slog.DebugContext(ctx, "volume service: list backups", "volume", volumeName)- var backups []models.VolumeBackup- err := s.db.WithContext(ctx).Where("volume_name = ?", volumeName).Order("created_at DESC").Find(&backups).Error- return backups, err-}--func (s *VolumeService) DeleteBackup(ctx context.Context, backupID string, user *models.User) error {- slog.DebugContext(ctx, "volume service: delete backup", "backup_id", backupID)- var backup models.VolumeBackup- if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err != nil {- return err- }-- // Delete from DB first - if this fails, no changes are made.- // If file deletion fails afterward, we just have an orphan file (easier to clean up)- // rather than an orphan DB record pointing to a non-existent file.- volumeName := backup.VolumeName // Save before deletion- if err := s.db.WithContext(ctx).Delete(&backup).Error; err != nil {- return err- }-- // Now delete the actual file - best effort since DB record is already gone- containerID, cleanup, err := s.createBackupTempContainerInternal(ctx, nil, "/volume", false)- if err != nil {- slog.WarnContext(ctx, "failed to create container for backup file cleanup", "backup_id", backupID, "error", err.Error())- } else {- defer cleanup()- filename, filenameErr := s.backupArchiveFilenameInternal(backupID)- if filenameErr != nil {- slog.WarnContext(ctx, "failed to sanitize backup id for file cleanup", "backup_id", backupID, "error", filenameErr.Error())- } else if _, _, err = s.execInContainerInternal(ctx, containerID, []string{"rm", "-f", path.Join("/volume", filename)}); err != nil {- slog.WarnContext(ctx, "failed to delete backup file (orphan file may remain)", "backup_id", backupID, "error", err.Error())- }- }-- actingUser := user- if actingUser == nil {- actingUser = &systemUser- }- metadata := models.JSON{- "action": "backup_delete",- "backup_id": backupID,- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupDelete, volumeName, volumeName, actingUser.ID, actingUser.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume backup delete event", "volume", volumeName, "error", logErr.Error())- }-- return nil-}--func (s *VolumeService) RestoreBackup(ctx context.Context, volumeName, backupID string, user models.User) error {- slog.DebugContext(ctx, "volume service: restore backup", "volume", volumeName, "backup_id", backupID, "user", user.ID)- var backup models.VolumeBackup- if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err != nil {- return err- }-- // Validate backup belongs to volume- if backup.VolumeName != volumeName {- return errors.Errorf("backup does not belong to volume %s", volumeName)- }-- // Check if volume is in use by running containers- inUse, containerIDs, err := s.GetVolumeUsage(ctx, volumeName)- if err != nil {- slog.WarnContext(ctx, "could not check volume usage", "volume", volumeName, "error", err.Error())- } else if inUse {- return errors.Errorf("volume is in use by %d container(s): restoring while containers are running may cause data corruption. Stop the containers first or use selective file restore", len(containerIDs))- }-- preBackup, err := s.CreateBackup(ctx, volumeName, user)- if err != nil {- return errors.WrapIf(err, "failed to create pre-restore backup")- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return err- }-- filename, err := s.backupArchiveFilenameInternal(backupID)- if err != nil {- return err- }-- helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)- if err != nil {- return err- }-- backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, "/backups", true)- if err != nil {- return err- }-- config := &container.Config{- Image: helperImage,- Cmd: []string{- "sh",- "-c",- fmt.Sprintf("set -e; tmp=$(mktemp -d /volume/.restore_tmp.XXXXXX); tar -tzf /backups/%s >/dev/null; tar -xzf /backups/%s -C \"$tmp\"; find /volume -mindepth 1 -maxdepth 1 -exec rm -rf -- {} +; find \"$tmp\" -mindepth 1 -maxdepth 1 -exec mv -- {} /volume/ \\;; rmdir \"$tmp\"", filename, filename),- },- Labels: volumehelper.Labels(),- }-- hostConfig := volumehelper.HostConfig(helperImage, []string{- volumeName + ":/volume",- }, []mount.Mount{backupStorage.mount})-- resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{- Config: config,- HostConfig: hostConfig,- })- if err != nil {- return errors.WrapIf(err, "failed to create restore container")- }-- if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {- _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())- return errors.WrapIf(err, "failed to start restore container")- }-- waitResult := dockerClient.ContainerWait(ctx, resp.ID, client.ContainerWaitOptions{Condition: container.WaitConditionNotRunning})- var waitBody container.WaitResponse- select {- case err := <-waitResult.Error:- if err != nil {- return err- }- case waitBody = <-waitResult.Result:- }-- if waitBody.StatusCode != 0 {- return errors.Errorf("restore container exited with code %d (volume may be partially wiped)", waitBody.StatusCode)- }-- metadata := models.JSON{- "action": "backup_restore",- "backup_id": backupID,- "pre_restore_backupId": preBackup.ID,- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupRestore, volumeName, volumeName, user.ID, user.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume backup restore event", "volume", volumeName, "error", logErr.Error())- }-- return nil-}--func (s *VolumeService) sanitizeBackupPathInternal(input string) (string, error) {- trimmed := strings.TrimSpace(input)- if trimmed == "" {- return "", errors.New("invalid path: empty")- }- cleaned := path.Clean(trimmed)- if cleaned == "." || cleaned == "/" {- return "", errors.Errorf("invalid path: %s", input)- }- if path.IsAbs(cleaned) {- cleaned = strings.TrimPrefix(cleaned, "/")- }- if cleaned == "" || cleaned == "." || cleaned == "/" || strings.HasPrefix(cleaned, "..") || strings.Contains(cleaned, "/../") {- return "", errors.Errorf("invalid path: %s", input)- }- return cleaned, nil-}--func (s *VolumeService) sanitizeBackupIDInternal(backupID string) (string, error) {- cleaned, err := s.sanitizeBackupPathInternal(backupID)- if err != nil {- return "", errors.WrapIf(err, "invalid backup id")- }- if strings.Contains(cleaned, "/") {- return "", errors.New("invalid backup id: path separators not allowed")- }- return cleaned, nil-}--func (s *VolumeService) backupArchiveFilenameInternal(backupID string) (string, error) {- sanitizedBackupID, err := s.sanitizeBackupIDInternal(backupID)- if err != nil {- return "", err- }-- return sanitizedBackupID + ".tar.gz", nil-}--func (s *VolumeService) BackupHasPath(ctx context.Context, backupID string, filePath string) (bool, error) {- slog.DebugContext(ctx, "volume service: backup has path", "backup_id", backupID, "path", filePath)- cleaned, err := s.sanitizeBackupPathInternal(filePath)- if err != nil {- return false, err- }- filename, err := s.backupArchiveFilenameInternal(backupID)- if err != nil {- return false, err- }-- var backup models.VolumeBackup- if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err != nil {- return false, err- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return false, err- }-- helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)- if err != nil {- return false, err- }-- backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, "/volume", true)- if err != nil {- return false, err- }-- containerID, cleanup, err := s.createBackupTempContainerWithMountInternal(ctx, dockerClient, helperImage, backupStorage.mount)- if err != nil {- return false, err- }- defer cleanup()-- archivePath := path.Join("/volume", filename)- cmd := []string{"tar", "-tzf", archivePath}- stdout, stderr, err := s.execInContainerInternal(ctx, containerID, cmd)- if err != nil {- return false, err- }- if strings.TrimSpace(stderr) != "" {- return false, errors.Errorf("failed to list backup contents: %s", strings.TrimSpace(stderr))- }-- for line := range strings.SplitSeq(stdout, "\n") {- entry := strings.TrimSpace(line)- if entry == "" {- continue- }- entry = strings.TrimPrefix(entry, "./")- if entry == cleaned || strings.TrimSuffix(entry, "/") == cleaned {- return true, nil- }- }-- return false, nil-}--func (s *VolumeService) ListBackupFiles(ctx context.Context, backupID string) ([]string, error) {- slog.DebugContext(ctx, "volume service: list backup files", "backup_id", backupID)- filename, err := s.backupArchiveFilenameInternal(backupID)- if err != nil {- return nil, err- }-- var backup models.VolumeBackup- if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err != nil {- return nil, err- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, err- }-- helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)- if err != nil {- return nil, err- }-- backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, "/volume", true)- if err != nil {- return nil, err- }-- containerID, cleanup, err := s.createBackupTempContainerWithMountInternal(ctx, dockerClient, helperImage, backupStorage.mount)- if err != nil {- return nil, err- }- defer cleanup()-- archivePath := path.Join("/volume", filename)- cmd := []string{"tar", "-tzf", archivePath}- stdout, _, err := s.execInContainerInternal(ctx, containerID, cmd)- if err != nil {- return nil, err- }-- lines := strings.Split(strings.TrimSpace(stdout), "\n")- files := make([]string, 0, len(lines))- seen := make(map[string]struct{})- for _, line := range lines {- clean := strings.TrimSpace(line)- if clean == "" {- continue- }- clean = strings.TrimPrefix(clean, "./")- if strings.HasSuffix(clean, "/") {- continue- }- if _, ok := seen[clean]; ok {- continue- }- seen[clean] = struct{}{}- files = append(files, clean)- }-- return files, nil-}--func (s *VolumeService) RestoreBackupFiles(ctx context.Context, volumeName, backupID string, paths []string, user models.User) error {- slog.DebugContext(ctx, "volume service: restore backup files", "volume", volumeName, "backup_id", backupID, "paths_count", len(paths), "user", user.ID)- if len(paths) == 0 {- return errors.New("no paths provided")- }- filename, err := s.backupArchiveFilenameInternal(backupID)- if err != nil {- return err- }-- var backup models.VolumeBackup- if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err != nil {- return err- }- if backup.VolumeName != volumeName {- return errors.New("backup does not belong to volume")- }-- // Create pre-restore backup for safety (consistent with RestoreBackup behavior)- preBackup, err := s.CreateBackup(ctx, volumeName, user)- if err != nil {- return errors.WrapIf(err, "failed to create pre-restore backup")- }- slog.DebugContext(ctx, "created pre-restore backup", "volume", volumeName, "pre_backup_id", preBackup.ID)-- cleanedPaths := make([]string, 0, len(paths))- for _, p := range paths {- cleaned, err := s.sanitizeBackupPathInternal(p)- if err != nil {- return err- }- cleanedPaths = append(cleanedPaths, cleaned)- }- if len(cleanedPaths) == 0 {- return errors.New("no valid paths provided")- }-- tarPaths := make([]string, 0, len(cleanedPaths))- for _, p := range cleanedPaths {- tarPaths = append(tarPaths, "./"+p)- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return err- }-- helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)- if err != nil {- return err- }-- backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, "/backups", true)- if err != nil {- return err- }-- config := &container.Config{- Image: helperImage,- Cmd: []string{"sleep", "infinity"},- NetworkDisabled: true,- Labels: volumehelper.Labels(),- }-- hostConfig := volumehelper.HostConfig(helperImage, []string{- volumeName + ":/volume",- }, []mount.Mount{backupStorage.mount})-- resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{- Config: config,- HostConfig: hostConfig,- })- if err != nil {- return errors.WrapIf(err, "failed to create restore container")- }-- if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {- _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())- return errors.WrapIf(err, "failed to start restore container")- }-- cleanup := func() {- _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())- }- defer cleanup()-- cmd := append([]string{"tar", "-xzf", path.Join("/backups", filename), "-C", "/volume", "--"}, tarPaths...)- _, stderr, err := s.execInContainerInternal(ctx, resp.ID, cmd)- if err != nil {- return errors.WrapIf(err, "failed to restore files")- }- if strings.TrimSpace(stderr) != "" {- slog.DebugContext(ctx, "volume service: restore files stderr", "backup_id", backupID, "stderr", strings.TrimSpace(stderr))- }-- metadata := models.JSON{- "action": "backup_restore_files",- "backup_id": backupID,- "pre_restore_backupId": preBackup.ID,- "paths_count": len(cleanedPaths),- }- if len(cleanedPaths) > 0 {- limit := min(len(cleanedPaths), 5)- metadata["paths_sample"] = cleanedPaths[:limit]- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupRestoreFiles, volumeName, volumeName, user.ID, user.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume backup restore files event", "volume", volumeName, "error", logErr.Error())- }-- return nil-}--func (s *VolumeService) DownloadBackup(ctx context.Context, backupID string, user *models.User) (io.ReadCloser, int64, error) {- slog.DebugContext(ctx, "volume service: download backup", "backup_id", backupID)- filename, err := s.backupArchiveFilenameInternal(backupID)- if err != nil {- return nil, 0, err- }- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, 0, err- }-- containerID, cleanup, err := s.createBackupTempContainerInternal(ctx, dockerClient, "/volume", true)- if err != nil {- return nil, 0, err- }-- reader, size, err := s.downloadFileFromContainerInternal(ctx, dockerClient, containerID, path.Join("/volume", filename), cleanup)- if err != nil {- return nil, 0, err- }-- actingUser := user- if actingUser == nil {- actingUser = &systemUser- }- volumeName := ""- var backup models.VolumeBackup- if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err == nil {- volumeName = backup.VolumeName- }- if volumeName != "" {- metadata := models.JSON{- "action": "backup_download",- "backup_id": backupID,- "size": size,- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupDownload, volumeName, volumeName, actingUser.ID, actingUser.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume backup download event", "volume", volumeName, "error", logErr.Error())- }- }-- return reader, size, nil-}--func (s *VolumeService) UploadAndRestore(ctx context.Context, volumeName string, archive io.Reader, filename string, user models.User) error {- slog.DebugContext(ctx, "volume service: upload and restore", "volume", volumeName, "filename", filename, "user", user.ID)-- tmpFile, err := os.CreateTemp("", "arcane-restore-*.tar.gz")- if err != nil {- return errors.WrapIf(err, "failed to buffer upload")- }- defer func() {- _ = tmpFile.Close()- _ = os.Remove(tmpFile.Name())- }()- if _, err := io.Copy(tmpFile, archive); err != nil {- return errors.WrapIf(err, "failed to buffer upload")- }- if _, err := tmpFile.Seek(0, io.SeekStart); err != nil {- return errors.WrapIf(err, "failed to read buffered upload")- }- gzr, err := gzip.NewReader(tmpFile)- if err != nil {- return errors.WrapIf(err, "invalid archive")- }- if _, err := tar.NewReader(gzr).Next(); err != nil {- _ = gzr.Close()- return errors.WrapIf(err, "invalid archive")- }- _ = gzr.Close()-- preBackup, err := s.CreateBackup(ctx, volumeName, user)- if err != nil {- return errors.WrapIf(err, "failed to create pre-restore backup")- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return err- }-- containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, false)- if err != nil {- return err- }- defer cleanup()-- tmpDir := fmt.Sprintf("/volume/.restore_tmp_%d", time.Now().UnixNano())- _, stderr, err := s.execInContainerInternal(ctx, containerID, []string{"mkdir", "-p", tmpDir})- if err != nil {- return errors.WrapIf(err, "failed to create temp restore dir")- }- if strings.TrimSpace(stderr) != "" {- slog.DebugContext(ctx, "volume service: restore temp dir stderr", "volume", volumeName, "stderr", strings.TrimSpace(stderr))- }-- if _, err := tmpFile.Seek(0, io.SeekStart); err != nil {- return errors.WrapIf(err, "failed to read buffered upload")- }- _, err = dockerClient.CopyToContainer(ctx, containerID, client.CopyToContainerOptions{- DestinationPath: tmpDir,- Content: tmpFile,- })- if err != nil {- return errors.WrapIf(err, "failed to restore from uploaded archive")- }-- _, stderr, err = s.execInContainerInternal(ctx, containerID, []string{"sh", "-c", fmt.Sprintf("test -n \"$(find %s -mindepth 1 -maxdepth 1 -print -quit)\"", tmpDir)})- if err != nil {- return errors.WrapIf(err, "uploaded archive appears empty or invalid")- }- if strings.TrimSpace(stderr) != "" {- slog.DebugContext(ctx, "volume service: restore validate stderr", "volume", volumeName, "stderr", strings.TrimSpace(stderr))- }-- _, stderr, err = s.execInContainerInternal(ctx, containerID, []string{"sh", "-c", "rm -rf /volume/* /volume/.[!.]* /volume/..?* 2>/dev/null || true"})- if err != nil {- return errors.WrapIf(err, "failed to clear volume before restore")- }- if strings.TrimSpace(stderr) != "" {- slog.DebugContext(ctx, "volume service: restore clear stderr", "volume", volumeName, "stderr", strings.TrimSpace(stderr))- }-- moveCmd := fmt.Sprintf("find %s -mindepth 1 -maxdepth 1 -exec mv -- {} /volume/ \\; && rmdir %s", tmpDir, tmpDir)- _, stderr, err = s.execInContainerInternal(ctx, containerID, []string{"sh", "-c", moveCmd})- if err != nil {- return errors.WrapIf(err, "failed to move restored files into place")- }- if strings.TrimSpace(stderr) != "" {- slog.DebugContext(ctx, "volume service: restore move stderr", "volume", volumeName, "stderr", strings.TrimSpace(stderr))- }-- metadata := models.JSON{- "action": "backup_upload_restore",- "filename": filename,- "pre_restore_backupId": preBackup.ID,- }- if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupRestore, volumeName, volumeName, user.ID, user.Username, "0", metadata); logErr != nil {- slog.WarnContext(ctx, "could not log volume backup upload restore event", "volume", volumeName, "error", logErr.Error())- }-- return nil-}--func (s *VolumeService) GetVolumeUsage(ctx context.Context, name string) (bool, []string, error) {- slog.DebugContext(ctx, "volume service: get volume usage", "volume", name)- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return false, nil, errors.WrapIf(err, "failed to connect to Docker")- }-- vol, err := dockerClient.VolumeInspect(ctx, name, client.VolumeInspectOptions{})- if err != nil {- return false, nil, errors.WrapIf(err, "volume not found")- }-- containerIDs, err := docker.GetContainersUsingVolume(ctx, dockerClient, vol.Volume.Name)- if err != nil {- return false, nil, errors.WrapIf(err, "failed to get containers using volume")- }-- inUse := len(containerIDs) > 0- return inUse, containerIDs, nil-}--// VolumeSizeData holds size information for a volume.-type VolumeSizeData struct {- Size int64- RefCount int64-}--// GetVolumeSizes returns disk usage data for all volumes.-// This is a slow operation as it calls Docker's DiskUsage API.-func (s *VolumeService) GetVolumeSizes(ctx context.Context) (map[string]VolumeSizeData, error) {- slog.DebugContext(ctx, "volume service: get volume sizes")- settings := s.settingsService.GetSettingsConfig()- apiCtx, cancel := context.WithTimeout(ctx, timeouts.GetDuration(settings.DockerAPITimeout.AsInt(), timeouts.DefaultDockerAPI))- defer cancel()-- dockerClient, err := s.dockerService.GetClient(apiCtx)- if err != nil {- return nil, errors.WrapIf(err, "failed to connect to Docker")- }-- usageVolumes, err := docker.GetVolumeUsageData(apiCtx, dockerClient)- if err != nil {- return nil, errors.WrapIf(err, "failed to get volume usage data")- }-- result := make(map[string]VolumeSizeData, len(usageVolumes))- for _, v := range usageVolumes {- if v.UsageData != nil {- result[v.Name] = VolumeSizeData{- Size: v.UsageData.Size,- RefCount: v.UsageData.RefCount,- }- }- }-- return result, nil-}--func (s *VolumeService) enrichVolumesWithUsageDataInternal(volumes []volume.Volume, usageVolumes []volume.Volume) []volume.Volume {- usageByName := make(map[string]*volume.UsageData, len(usageVolumes))- for _, uv := range usageVolumes {- if uv.Name == "" || uv.UsageData == nil {- continue- }- // Keep first-seen value to preserve previous nested-loop behavior.- if _, exists := usageByName[uv.Name]; !exists {- usageByName[uv.Name] = uv.UsageData- }- }-- result := make([]volume.Volume, 0, len(volumes))- for _, v := range volumes {- if usageData, exists := usageByName[v.Name]; exists {- v.UsageData = usageData- }-- result = append(result, v)- }- return result-}--func (s *VolumeService) buildVolumeContainerMapInternal(ctx context.Context, dockerClient *client.Client) (map[string][]string, error) {- containers, err := dockerClient.ContainerList(ctx, client.ContainerListOptions{All: true})- if err != nil {- return nil, errors.WrapIf(err, "failed to list containers")- }-- volumeContainerMap := make(map[string][]string)- for _, c := range containers.Items {- for _, m := range c.Mounts {- if m.Type == mount.TypeVolume && m.Name != "" {- volumeContainerMap[m.Name] = append(volumeContainerMap[m.Name], c.ID)- }- }- }-- return volumeContainerMap, nil-}--func (s *VolumeService) buildVolumePaginationConfigInternal() pagination.Config[volumetypes.Volume] {- return pagination.Config[volumetypes.Volume]{- SearchAccessors: []pagination.SearchAccessor[volumetypes.Volume]{- func(v volumetypes.Volume) (string, error) { return v.Name, nil },- func(v volumetypes.Volume) (string, error) { return v.Driver, nil },- func(v volumetypes.Volume) (string, error) { return v.Mountpoint, nil },- func(v volumetypes.Volume) (string, error) { return v.Scope, nil },- },- SortBindings: s.buildVolumeSortBindingsInternal(),- FilterAccessors: s.buildVolumeFilterAccessorsInternal(),- }-}--func (s *VolumeService) buildVolumeSortBindingsInternal() []pagination.SortBinding[volumetypes.Volume] {- createdSortFn := s.compareVolumeCreatedInternal-- return []pagination.SortBinding[volumetypes.Volume]{- {- Key: "name",- Fn: func(a, b volumetypes.Volume) int { return strings.Compare(a.Name, b.Name) },- },- {- Key: "driver",- Fn: func(a, b volumetypes.Volume) int { return strings.Compare(a.Driver, b.Driver) },- },- {- Key: "mountpoint",- Fn: func(a, b volumetypes.Volume) int { return strings.Compare(a.Mountpoint, b.Mountpoint) },- },- {- Key: "scope",- Fn: func(a, b volumetypes.Volume) int { return strings.Compare(a.Scope, b.Scope) },- },- {- Key: "created",- Fn: createdSortFn,- },- {- Key: "createdAt",- Fn: createdSortFn,- },- {- Key: "inUse",- Fn: func(a, b volumetypes.Volume) int {- if a.InUse == b.InUse {- return 0- }- if a.InUse {- return -1- }- return 1- },- },- {- Key: "size",- Fn: s.compareVolumeSizesInternal,- },- }-}--func (s *VolumeService) compareVolumeSizesInternal(a, b volumetypes.Volume) int {- aSize := a.Size- bSize := b.Size-- if aSize == 0 && a.UsageData != nil {- aSize = a.UsageData.Size- }- if bSize == 0 && b.UsageData != nil {- bSize = b.UsageData.Size- }-- if aSize == bSize {- return strings.Compare(a.Name, b.Name)- }- if aSize < bSize {- return -1- }- return 1-}--func (s *VolumeService) compareVolumeCreatedInternal(a, b volumetypes.Volume) int {- aTime, aOk := s.parseVolumeCreatedAtInternal(a.CreatedAt).Get()- bTime, bOk := s.parseVolumeCreatedAtInternal(b.CreatedAt).Get()- if aOk && bOk {- if aTime.Before(bTime) {- return -1- }- if aTime.After(bTime) {- return 1- }- return 0- }- return strings.Compare(a.CreatedAt, b.CreatedAt)-}--func (s *VolumeService) parseVolumeCreatedAtInternal(createdAt string) mo.Option[time.Time] {- if createdAt == "" {- return mo.None[time.Time]()- }- if parsed, err := time.Parse(time.RFC3339Nano, createdAt); err == nil {- return mo.Some(parsed)- }- if parsed, err := time.Parse(time.RFC3339, createdAt); err == nil {- return mo.Some(parsed)- }- return mo.None[time.Time]()-}--func (s *VolumeService) buildVolumeFilterAccessorsInternal() []pagination.FilterAccessor[volumetypes.Volume] {- return []pagination.FilterAccessor[volumetypes.Volume]{- {- Key: "inUse",- Fn: func(v volumetypes.Volume, filterValue string) bool {- if filterValue == "true" {- return v.InUse- }- if filterValue == "false" {- return !v.InUse- }- return true- },- },- }-}--func (s *VolumeService) calculateVolumeUsageCountsInternal(items []volumetypes.Volume) volumetypes.UsageCounts {- counts := volumetypes.UsageCounts{- Total: len(items),- }- for _, v := range items {- if v.InUse {- counts.Inuse++- } else {- counts.Unused++- }- }- return counts-}--// countVolumeUsageFromSnapshotInternal derives volume usage counts from an-// already-fetched volume and container listing. It mirrors the semantics of-// ListVolumesPaginated — internal volumes are excluded and a volume counts as-// in use when any container mounts it — so dashboard tile numbers agree with-// the volumes page.-func (s *VolumeService) countVolumeUsageFromSnapshotInternal(volumes []volume.Volume, containers []container.Summary) volumetypes.UsageCounts {- inUse := make(map[string]struct{})- for _, c := range containers {- for _, m := range c.Mounts {- if m.Type == mount.TypeVolume && m.Name != "" {- inUse[m.Name] = struct{}{}- }- }- }-- counts := volumetypes.UsageCounts{}- for _, v := range volumes {- if s.isInternalVolumeInternal(volumetypes.NewSummary(v)) {- continue- }- counts.Total++- if _, ok := inUse[v.Name]; ok {- counts.Inuse++- } else {- counts.Unused++- }- }- return counts-}--func (s *VolumeService) isInternalVolumeInternal(v volumetypes.Volume) bool {- if strings.EqualFold(strings.TrimSpace(v.Name), strings.TrimSpace(s.backupVolumeName)) {- return true- }-- return libarcane.IsInternalContainer(v.Labels)-}--func (s *VolumeService) ListVolumesPaginated(ctx context.Context, params pagination.QueryParams, includeInternal bool) ([]volumetypes.Volume, pagination.Response, volumetypes.UsageCounts, error) {- startedAt := time.Now()- slog.DebugContext(ctx, "volume service: list volumes paginated", "search", params.Search, "sort", params.Sort, "order", params.Order, "start", params.Start, "limit", params.Limit, "include_internal", includeInternal)- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, pagination.Response{}, volumetypes.UsageCounts{}, errors.WrapIf(err, "failed to connect to Docker")- }-- // Run volume list and container list in parallel for better performance- type volumeListResult struct {- volumes []volume.Volume- err error- }- type containerMapResult struct {- containerMap map[string][]string- err error- }-- volChan := make(chan volumeListResult, 1)- containerChan := make(chan containerMapResult, 1)-- settings := s.settingsService.GetSettingsConfig()- apiCtx, cancel := context.WithTimeout(ctx, timeouts.GetDuration(settings.DockerAPITimeout.AsInt(), timeouts.DefaultDockerAPI))- defer cancel()-- go func(ctx context.Context) {- volListBody, err := dockerClient.VolumeList(ctx, client.VolumeListOptions{})- volChan <- volumeListResult{volumes: volListBody.Items, err: err}- }(apiCtx)-- go func(ctx context.Context) {- containerMap, err := s.buildVolumeContainerMapInternal(ctx, dockerClient)- containerChan <- containerMapResult{containerMap: containerMap, err: err}- }(apiCtx)-- // Wait for both results- volResult := <-volChan- if volResult.err != nil {- return nil, pagination.Response{}, volumetypes.UsageCounts{}, errors.WrapIf(volResult.err, "failed to list Docker volumes")- }-- containerResult := <-containerChan- volumeContainerMap := containerResult.containerMap- if containerResult.err != nil {- slog.WarnContext(ctx, "failed to build volume-container map", "error", containerResult.err.Error())- volumeContainerMap = make(map[string][]string)- }-- effectiveParams := params- usageCacheSnapshot := "not_requested"-- // Size sorting consumes the current cache snapshot and refreshes it in the- // background so this list request never waits for Docker's DiskUsage call.- var usageVolumes []volume.Volume- if params.Sort == "size" {- if uv, found := docker.GetVolumeUsageDataStaleWhileRevalidate(apiCtx, dockerClient).Get(); found && (len(uv) > 0 || len(volResult.volumes) == 0) {- usageVolumes = uv- usageCacheSnapshot = "available"- } else {- usageCacheSnapshot = "missing"- effectiveParams.Sort = "name"- effectiveParams.Order = pagination.SortAsc- }- }-- volumes := s.enrichVolumesWithUsageDataInternal(volResult.volumes, usageVolumes)-- items := make([]volumetypes.Volume, 0, len(volumes))- for _, v := range volumes {- volDto := volumetypes.NewSummary(v)- if !includeInternal && s.isInternalVolumeInternal(volDto) {- continue- }- if containerIDs, ok := volumeContainerMap[v.Name]; ok {- volDto.Containers = containerIDs- if len(containerIDs) > 0 {- volDto.InUse = true- }- }- items = append(items, volDto)- }-- config := s.buildVolumePaginationConfigInternal()- result := pagination.SearchOrderAndPaginate(items, effectiveParams, config)- counts := s.calculateVolumeUsageCountsInternal(items)- paginationResp := pagination.BuildResponseFromFilterResult(result, effectiveParams)- slog.DebugContext(ctx, "volume service: listed volumes",- "docker_host", dockerClient.DaemonHost(),- "requested_sort", params.Sort,- "requested_order", params.Order,- "effective_sort", effectiveParams.Sort,- "effective_order", effectiveParams.Order,- "usage_cache_snapshot", usageCacheSnapshot,- "docker_volumes", len(volResult.volumes),- "usage_volumes", len(usageVolumes),- "included_volumes", len(items),- "matched_volumes", result.TotalCount,- "returned_volumes", len(result.Items),- "container_volume_count", len(volumeContainerMap),- "filter_count", len(params.Filters),- "current_page", paginationResp.CurrentPage,- "total_pages", paginationResp.TotalPages,- "duration", time.Since(startedAt),- )-- return result.Items, paginationResp, counts, nil-}--func (s *VolumeService) downloadFileFromContainerInternal(- ctx context.Context,- dockerClient *client.Client,- containerID string,- containerPath string,- cleanup func(),-) (io.ReadCloser, int64, error) {- copyResult, err := dockerClient.CopyFromContainer(ctx, containerID, client.CopyFromContainerOptions{- SourcePath: containerPath,- })- if err != nil {- cleanup()- return nil, 0, errors.WrapIf(err, "failed to download")- }- reader := copyResult.Content-- tr := tar.NewReader(reader)- hdr, err := tr.Next()- if err != nil {- _ = reader.Close()- cleanup()- return nil, 0, errors.WrapIf(err, "failed to read tar stream")- }- if hdr.FileInfo().IsDir() {- _ = reader.Close()- cleanup()- return nil, 0, errors.New("path is a directory")- }-- return &cleanupReadCloser{- Reader: tr,- Closer: reader,- cleanup: cleanup,- }, hdr.Size, nil-}diff --git a/backend/internal/services/vulnerability_service.go b/backend/internal/services/vulnerability_service.godeleted file mode 100644index ee8967a66..000000000--- a/backend/internal/services/vulnerability_service.go+++ /dev/null@@ -1,4023 +0,0 @@-package services--import (- "archive/tar"- "bytes"- "context"- "encoding/json/jsontext"- "encoding/json/v2"- "fmt"- "io"- "log/slog"- "math"- "net"- "os"- "path/filepath"- "sort"- "strconv"- "strings"- "sync"- "time"-- "emperror.dev/errors"-- cerrdefs "github.com/containerd/errdefs"- "github.com/moby/moby/api/pkg/stdcopy"- containertypes "github.com/moby/moby/api/types/container"- imagetypes "github.com/moby/moby/api/types/image"- mounttypes "github.com/moby/moby/api/types/mount"- "github.com/moby/moby/client"- "github.com/samber/mo"- "gorm.io/gorm"- "gorm.io/gorm/clause"-- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"- activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/vuln"- "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"- "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"- "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"- "github.com/getarcaneapp/arcane/types/v2/vulnerability"- "go.getarcane.app/sys/cgroup"- "go.getarcane.app/updater/labels"-)--const (- // DefaultArcaneToolsImage is the shared Arcane toolbox image used for- // helper commands and Trivy-based vulnerability scans.- DefaultArcaneToolsImage = "ghcr.io/getarcaneapp/tools:latest"- // DefaultTrivyImage preserves the existing setting name, but now points at- // the shared Arcane tools image that includes the Trivy binary.- DefaultTrivyImage = DefaultArcaneToolsImage- DefaultTrivyNetworkMode = "bridge"- trivyCacheVolumeName = "arcane-trivy-cache"- trivyCacheMountTarget = "/root/.cache"- trivyCacheSlotRootDir = trivyCacheMountTarget + "/arcane-scan-slots"- scanStaleTimeout = 30 * time.Minute- defaultTrivyCPULimitCores = 1.0- defaultTrivyMemoryLimitMB int64 = 0- trivyNanoCPUsPerCore = int64(1_000_000_000)- trivyBytesPerMB = int64(1024 * 1024)- trivyErrorExcerptSize = int64(32 * 1024)- defaultScanSaveRetryAttempts = 3- defaultScanSaveRetryDelay = 200 * time.Millisecond- defaultScanSaveRetryMaxDelay = 2 * time.Second- trivyOutputPathPrefix = "/tmp/arcane-trivy-result-"- trivyRegistryConfigDir = "/tmp/arcane-registry-auth"- trivyRegistryConfigCopyDest = "/tmp"- trivyRegistryConfigTarName = "arcane-registry-auth/config.json"-)--type trivyRuntimeOptions struct {- ContainerEnv []string- Mounts []mounttypes.Mount- NetworkMode string-}--type trivyRuntimeSecurityInternal struct {- securityOpts []string- privileged bool-}--// VulnerabilityService handles vulnerability scanning of container images-type VulnerabilityService struct {- db *database.DB- dockerService *DockerClientService- eventService *EventService- settingsService *SettingsService- notificationService *NotificationService- activityService *ActivityService- registryService *ContainerRegistryService- // scanLocks provides per-image locking to allow concurrent scans of different- // images while preventing duplicate scans of the same image.- scanLocks utils.KeyedMutex- // trivyScanSlots limits concurrent Trivy scans (manual + scheduled) and- // provides deterministic slot IDs for shared-cache sharding.- trivyScanSlots chan int- trivyScanMu sync.Mutex- // scanPhases tracks live scan phases in-memory for in-progress scans.- scanPhases sync.Map // map[string]vulnerability.ScanPhase-}--func (s *VulnerabilityService) setScanPhaseInternal(imageID string, phase vulnerability.ScanPhase) {- if imageID == "" || phase == "" {- return- }- s.scanPhases.Store(imageID, phase)-}--func (s *VulnerabilityService) clearScanPhaseInternal(imageID string) {- if imageID == "" {- return- }- s.scanPhases.Delete(imageID)-}--func (s *VulnerabilityService) getScanPhaseInternal(imageID string) vulnerability.ScanPhase {- if imageID == "" {- return ""- }-- phase, ok := s.scanPhases.Load(imageID)- if !ok {- return ""- }-- if typed, ok := phase.(vulnerability.ScanPhase); ok {- return typed- }-- if str, ok := phase.(string); ok {- return vulnerability.ScanPhase(str)- }-- return ""-}--func (s *VulnerabilityService) resolveScanPhaseInternal(imageID string, status vulnerability.ScanStatus) vulnerability.ScanPhase {- if status != vulnerability.ScanStatusPending && status != vulnerability.ScanStatusScanning {- return ""- }-- if phase := s.getScanPhaseInternal(imageID); phase != "" {- return phase- }-- if status == vulnerability.ScanStatusPending {- return vulnerability.ScanPhaseCreatingContainer- }-- return vulnerability.ScanPhaseScanningImage-}--func (s *VulnerabilityService) applyScanPhaseToSummaryInternal(summary *vulnerability.ScanSummary) {- if summary == nil {- return- }-- summary.ScanPhase = s.resolveScanPhaseInternal(summary.ImageID, summary.Status)-}--func (s *VulnerabilityService) applyScanPhaseToResultInternal(result *vulnerability.ScanResult) {- if result == nil {- return- }-- result.ScanPhase = s.resolveScanPhaseInternal(result.ImageID, result.Status)-}--func (s *VulnerabilityService) acquireTrivyScanSlotInternal(ctx context.Context) (int, func(), error) {- limit := s.getTrivyConcurrentScanContainersInternal()- slotCh := s.getTrivyScanSlotChannelInternal(limit)- if slotCh == nil {- return 0, func() {}, nil- }-- select {- case slotID := <-slotCh:- return slotID, func() { slotCh <- slotID }, nil- case <-ctx.Done():- return 0, nil, ctx.Err()- }-}--func (s *VulnerabilityService) getTrivyConcurrentScanContainersInternal() int {- if s.settingsService == nil {- return 1- }-- cfg := s.settingsService.GetSettingsConfig()- if cfg == nil {- return 1- }-- limit := cfg.TrivyConcurrentScanContainers.AsInt()- if limit < 1 {- return 1- }-- return limit-}--func (s *VulnerabilityService) getTrivyScanSlotChannelInternal(limit int) chan int {- if limit < 1 {- limit = 1- }-- createSlots := func(size int) chan int {- slots := make(chan int, size)- for i := range size {- slots <- i- }- return slots- }-- s.trivyScanMu.Lock()- defer s.trivyScanMu.Unlock()-- if s.trivyScanSlots == nil {- s.trivyScanSlots = createSlots(limit)- return s.trivyScanSlots- }-- if cap(s.trivyScanSlots) == limit {- return s.trivyScanSlots- }-- // Apply capacity changes only when idle to avoid splitting tokens across channels.- if len(s.trivyScanSlots) == cap(s.trivyScanSlots) {- s.trivyScanSlots = createSlots(limit)- }-- return s.trivyScanSlots-}--// NewVulnerabilityService creates a new VulnerabilityService instance-func NewVulnerabilityService(db *database.DB, dockerService *DockerClientService, eventService *EventService, settingsService *SettingsService, notificationService *NotificationService, activityService *ActivityService, registryService *ContainerRegistryService) *VulnerabilityService {- return &VulnerabilityService{- db: db,- dockerService: dockerService,- eventService: eventService,- settingsService: settingsService,- notificationService: notificationService,- activityService: activityService,- registryService: registryService,- trivyScanSlots: nil,- }-}--func (s *VulnerabilityService) buildTrivyRegistryConfigInternal(ctx context.Context) []byte {- if s.registryService == nil {- return nil- }-- authConfigs, err := s.registryService.GetAllRegistryAuthConfigs(ctx)- if err != nil {- slog.WarnContext(ctx, "failed to load registry credentials for trivy scan; continuing anonymously", "error", err)- return nil- }-- configJSON, err := vuln.BuildDockerConfigJSON(authConfigs)- if err != nil {- slog.WarnContext(ctx, "failed to build trivy registry config; continuing anonymously", "error", err)- return nil- }-- return configJSON-}--// ScanImage scans an image for vulnerabilities using Trivy-func (s *VulnerabilityService) ScanImage(ctx context.Context, envID string, imageID string, user models.User) (*vulnerability.ScanResult, error) {- scanCtx := utils.ActivityRuntimeContext(ctx, nil)-- // Get Docker client to inspect the image- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, errors.WrapIf(err, "failed to connect to Docker")- }-- // Inspect the image to get details- imageInspect, err := dockerClient.ImageInspect(scanCtx, imageID)- if err != nil {- return nil, errors.WrapIf(err, "failed to inspect image")- }-- // Determine image name to scan- imageName := imageID- if len(imageInspect.RepoTags) > 0 {- imageName = imageInspect.RepoTags[0]- } else if len(imageInspect.RepoDigests) > 0 {- imageName = imageInspect.RepoDigests[0]- }-- slog.InfoContext(scanCtx,- "vulnerability scan queued",- "environmentId", envID,- "imageId", imageID,- "imageName", imageName,- )- activityID := s.startVulnerabilityScanActivityInternal(scanCtx, envID, imageID, imageName, &user)- // Make the scan cancelable: cancelling the activity cancels scanCtx, which- // unblocks the Trivy container wait and triggers its (cancel-safe) teardown.- scanCtx = s.activityService.Track(scanCtx, activityID)- s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseCreatingContainer)-- // Create pending scan record- pendingResult := &vulnerability.ScanResult{- ImageID: imageID,- ImageName: imageName,- ScanTime: time.Now(),- Status: vulnerability.ScanStatusScanning,- ActivityID: mo.EmptyableToOption(strings.TrimSpace(activityID)).ToPointer(),- }- s.applyScanPhaseToResultInternal(pendingResult)- if saveErr := s.saveScanResultWithRetryInternal(scanCtx, pendingResult); saveErr != nil {- slog.WarnContext(scanCtx,- "failed to save pending scan result",- "error", saveErr,- "imageId", imageID,- "imageName", imageName,- )- } else {- slog.DebugContext(scanCtx,- "saved pending vulnerability scan record",- "imageId", imageID,- "imageName", imageName,- "scanTime", pendingResult.ScanTime,- )- }-- go func(bgCtx context.Context, scanEnvID, imgID, imgName string, scanUser models.User, scanActivityID string) {- s.scanImageInBackgroundInternal(bgCtx, scanEnvID, imgID, imgName, scanUser, scanActivityID)- }(scanCtx, envID, imageID, imageName, user, activityID)-- return pendingResult, nil-}--func (s *VulnerabilityService) markStaleScanIfNeeded(ctx context.Context, record *models.VulnerabilityScanRecord) {- if record == nil {- return- }- if record.Status != models.ScanStatusScanning && record.Status != models.ScanStatusPending {- return- }- if time.Since(record.ScanTime) <= scanStaleTimeout {- return- }-- errMsg := "Scan timed out or was interrupted. Please retry."- record.Status = models.ScanStatusFailed- record.Error = &errMsg- record.Duration = time.Since(record.ScanTime).Milliseconds()- s.clearScanPhaseInternal(record.ID)-- if s.db == nil {- return- }-- if err := s.db.WithContext(ctx).- Model(&models.VulnerabilityScanRecord{}).- Where("id = ?", record.ID).- Updates(map[string]any{- "status": record.Status,- "error": errMsg,- "duration": record.Duration,- }).Error; err != nil {- slog.WarnContext(ctx, "failed to mark stale scan as failed", "error", err, "scan_id", record.ID)- return- }-}--func (s *VulnerabilityService) scanImageInBackgroundInternal(ctx context.Context, envID string, imageID, imageName string, user models.User, activityID string) {- defer s.clearScanPhaseInternal(imageID)- s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseCreatingContainer)- s.updateVulnerabilityScanActivityInternal(ctx, activityID, vulnerability.ScanPhaseCreatingContainer, 10, "Preparing vulnerability scanner")-- slog.DebugContext(ctx,- "starting background vulnerability scan",- "environmentId", envID,- "imageId", imageID,- "imageName", imageName,- )-- trivyImage, err := s.ensureTrivyImageInternal(ctx)- if err != nil {- s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseStoringResults)- s.updateVulnerabilityScanActivityInternal(ctx, activityID, vulnerability.ScanPhaseStoringResults, 90, "Unable to prepare vulnerability scanner")- result := &vulnerability.ScanResult{- ImageID: imageID,- ImageName: imageName,- ScanTime: time.Now(),- Status: vulnerability.ScanStatusFailed,- ActivityID: mo.EmptyableToOption(strings.TrimSpace(activityID)).ToPointer(),- Error: "Trivy scanner is not available: " + err.Error(),- }- if saveErr := s.saveScanResultWithRetryInternal(ctx, result); saveErr != nil {- slog.WarnContext(ctx, "failed to save scan result", "error", saveErr)- }- slog.WarnContext(ctx,- "background vulnerability scan failed before execution",- "environmentId", envID,- "imageId", imageID,- "imageName", imageName,- "error", result.Error,- )- s.completeVulnerabilityScanActivityInternal(ctx, activityID, false, result.Error)- return- }-- s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseScanningImage)- s.updateVulnerabilityScanActivityInternal(ctx, activityID, vulnerability.ScanPhaseScanningImage, 45, "Scanning image for vulnerabilities")- startTime := time.Now()- result, err := s.runTrivyScan(ctx, trivyImage, imageName, imageID)- duration := time.Since(startTime).Milliseconds()-- if err != nil {- s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseStoringResults)- s.updateVulnerabilityScanActivityInternal(ctx, activityID, vulnerability.ScanPhaseStoringResults, 90, "Storing failed scan result")- failedResult := &vulnerability.ScanResult{- ImageID: imageID,- ImageName: imageName,- ScanTime: time.Now(),- Status: vulnerability.ScanStatusFailed,- ActivityID: mo.EmptyableToOption(strings.TrimSpace(activityID)).ToPointer(),- Error: err.Error(),- Duration: duration,- }- if saveErr := s.saveScanResultWithRetryInternal(ctx, failedResult); saveErr != nil {- slog.WarnContext(ctx, "failed to save failed scan result", "error", saveErr)- }- slog.WarnContext(ctx,- "background vulnerability scan failed",- "environmentId", envID,- "imageId", imageID,- "imageName", imageName,- "durationMs", duration,- "error", err,- )- s.logScanEvent(ctx, envID, imageID, imageName, user, false, err.Error())- s.completeVulnerabilityScanActivityInternal(ctx, activityID, false, err.Error())- return- }-- result.Duration = duration- result.ActivityID = mo.EmptyableToOption(strings.TrimSpace(activityID)).ToPointer()- s.ensureSummary(result)- s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseStoringResults)- s.updateVulnerabilityScanActivityInternal(ctx, activityID, vulnerability.ScanPhaseStoringResults, 90, "Storing vulnerability results")- if saveErr := s.saveScanResultWithRetryInternal(ctx, result); saveErr != nil {- slog.WarnContext(ctx, "failed to save scan result", "error", saveErr)- }-- slog.InfoContext(ctx,- "background vulnerability scan completed",- "environmentId", envID,- "imageId", imageID,- "imageName", imageName,- "durationMs", duration,- "vulnerabilities", result.Summary.Total,- )-- s.logScanEvent(ctx, envID, imageID, imageName, user, true, "")- s.completeVulnerabilityScanActivityInternal(ctx, activityID, true, "")-}--func (s *VulnerabilityService) startVulnerabilityScanActivityInternal(ctx context.Context, envID, imageID, imageName string, user *models.User) string {- if s.activityService == nil {- return ""- }-- activity, err := s.activityService.StartActivity(ctx, StartActivityRequest{- EnvironmentID: envID,- Type: models.ActivityTypeVulnerabilityScan,- ResourceType: new("image"),- ResourceID: &imageID,- ResourceName: &imageName,- StartedBy: user,- Progress: new(0),- Step: string(vulnerability.ScanPhaseCreatingContainer),- LatestMessage: "Vulnerability scan queued",- })- if err != nil {- slog.WarnContext(ctx, "failed to create vulnerability scan activity", "error", err, "imageId", imageID)- return ""- }- return activity.ID-}--func (s *VulnerabilityService) updateVulnerabilityScanActivityInternal(ctx context.Context, activityID string, phase vulnerability.ScanPhase, progress int, message string) {- if s.activityService == nil || activityID == "" {- return- }-- if _, err := s.activityService.AppendMessage(ctx, activityID, AppendActivityMessageRequest{- Level: models.ActivityMessageLevelInfo,- Message: message,- Progress: &progress,- Step: string(phase),- }); err != nil {- slog.DebugContext(ctx, "failed to append vulnerability scan activity message", "activityId", activityID, "error", err)- }-}--func (s *VulnerabilityService) completeVulnerabilityScanActivityInternal(ctx context.Context, activityID string, success bool, errMessage string) {- if s.activityService == nil || activityID == "" {- return- }-- status := models.ActivityStatusSuccess- message := "Vulnerability scan completed"- var errorPtr *string- if !success {- if activitylib.CancelledByContext(ctx) {- status = models.ActivityStatusCancelled- message = "Vulnerability scan cancelled"- } else {- status = models.ActivityStatusFailed- message = "Vulnerability scan failed"- if strings.TrimSpace(errMessage) != "" {- errorPtr = &errMessage- message = errMessage- }- }- }-- if _, err := s.activityService.CompleteActivity(utils.ActivityRuntimeContext(ctx, nil), activityID, status, message, errorPtr); err != nil {- slog.DebugContext(ctx, "failed to complete vulnerability scan activity", "activityId", activityID, "error", err)- }-}--// GetScanResult retrieves the most recent scan result for an image-func (s *VulnerabilityService) GetScanResult(ctx context.Context, imageID string) (*vulnerability.ScanResult, error) {- if s.db == nil {- return nil, nil- }-- var record models.VulnerabilityScanRecord- err := s.db.WithContext(ctx).Where("id = ?", imageID).First(&record).Error- if err != nil {- if errors.Is(err, gorm.ErrRecordNotFound) {- return nil, nil- }- return nil, errors.WrapIf(err, "failed to get scan result")- }-- s.markStaleScanIfNeeded(ctx, &record)-- return s.convertRecordToResult(&record)-}--// GetScanSummary retrieves just the summary for an image (for list views)-func (s *VulnerabilityService) GetScanSummary(ctx context.Context, imageID string) (*vulnerability.ScanSummary, error) {- if s.db == nil {- return nil, nil- }-- var record models.VulnerabilityScanRecord- err := s.db.WithContext(ctx).- Select("id", "scan_time", "status", "critical_count", "high_count", "medium_count", "low_count", "unknown_count", "total_count", "error").- Where("id = ?", imageID).- First(&record).Error- if err != nil {- if errors.Is(err, gorm.ErrRecordNotFound) {- return nil, nil- }- return nil, errors.WrapIf(err, "failed to get scan summary")- }-- s.markStaleScanIfNeeded(ctx, &record)-- summary := &vulnerability.ScanSummary{- ImageID: record.ID,- ScanTime: record.ScanTime,- Status: vulnerability.ScanStatus(record.Status),- Summary: &vulnerability.SeveritySummary{- Critical: record.CriticalCount,- High: record.HighCount,- Medium: record.MediumCount,- Low: record.LowCount,- Unknown: record.UnknownCount,- Total: record.TotalCount,- },- Error: mo.PointerToOption(record.Error).OrEmpty(),- }- s.applyScanPhaseToSummaryInternal(summary)-- return summary, nil-}--// ListVulnerabilities returns a paginated, filtered list of vulnerabilities for an image.-func (s *VulnerabilityService) ListVulnerabilities(ctx context.Context, imageID string, params pagination.QueryParams) ([]vulnerability.Vulnerability, pagination.Response, error) {- result, err := s.GetScanResult(ctx, imageID)- if err != nil {- return nil, pagination.Response{}, errors.WrapIf(err, "failed to get scan result")- }- if result == nil {- limit := params.Limit- if limit <= 0 {- limit = 20- }- return []vulnerability.Vulnerability{}, pagination.Response{- TotalPages: 1,- TotalItems: 0,- CurrentPage: 1,- ItemsPerPage: limit,- GrandTotalItems: 0,- }, nil- }-- config := pagination.Config[vulnerability.Vulnerability]{- SearchAccessors: []pagination.SearchAccessor[vulnerability.Vulnerability]{- func(item vulnerability.Vulnerability) (string, error) { return item.VulnerabilityID, nil },- func(item vulnerability.Vulnerability) (string, error) { return item.PkgName, nil },- func(item vulnerability.Vulnerability) (string, error) { return item.InstalledVersion, nil },- func(item vulnerability.Vulnerability) (string, error) { return item.FixedVersion, nil },- func(item vulnerability.Vulnerability) (string, error) { return item.Title, nil },- },- SortBindings: []pagination.SortBinding[vulnerability.Vulnerability]{- {- Key: "vulnerabilityId",- Fn: func(a, b vulnerability.Vulnerability) int {- return strings.Compare(a.VulnerabilityID, b.VulnerabilityID)- },- },- {- Key: "pkgName",- Fn: func(a, b vulnerability.Vulnerability) int { return strings.Compare(a.PkgName, b.PkgName) },- },- {- Key: "severity",- Fn: func(a, b vulnerability.Vulnerability) int {- return severityRankInternal(a.Severity) - severityRankInternal(b.Severity)- },- },- {- Key: "installedVersion",- Fn: func(a, b vulnerability.Vulnerability) int {- return strings.Compare(a.InstalledVersion, b.InstalledVersion)- },- },- {- Key: "fixedVersion",- Fn: func(a, b vulnerability.Vulnerability) int { return strings.Compare(a.FixedVersion, b.FixedVersion) },- },- },- FilterAccessors: []pagination.FilterAccessor[vulnerability.Vulnerability]{- {- Key: "severity",- Fn: func(item vulnerability.Vulnerability, value string) bool {- return strings.EqualFold(string(item.Severity), value)- },- },- },- }-- filtered := pagination.SearchOrderAndPaginate(result.Vulnerabilities, params, config)- response := pagination.BuildResponseFromFilterResult(filtered, params)-- return filtered.Items, response, nil-}--// GetEnvironmentSummary returns aggregated vulnerability counts across all images.-func (s *VulnerabilityService) GetEnvironmentSummary(ctx context.Context) (*vulnerability.EnvironmentVulnerabilitySummary, error) {- if s.db != nil && s.dockerService != nil {- if deleted, err := s.CleanupOrphanedScanRecords(ctx); err != nil {- slog.WarnContext(ctx, "failed to cleanup orphaned vulnerability scan records", "error", err)- } else if deleted > 0 {- slog.DebugContext(ctx, "cleaned up orphaned vulnerability scan records", "deleted", deleted)- }- }-- summary := &vulnerability.EnvironmentVulnerabilitySummary{- Summary: &vulnerability.SeveritySummary{},- }-- if s.dockerService != nil {- _, _, _, total, err := s.dockerService.GetAllImages(ctx)- if err != nil {- slog.WarnContext(ctx, "failed to list images for vulnerability summary", "error", err)- } else {- summary.TotalImages = total- }- }-- if s.db == nil {- return summary, nil- }-- type aggregate struct {- Critical int `gorm:"column:critical"`- High int `gorm:"column:high"`- Medium int `gorm:"column:medium"`- Low int `gorm:"column:low"`- Unknown int `gorm:"column:unknown"`- Total int `gorm:"column:total"`- Scanned int `gorm:"column:scanned"`- }-- var agg aggregate- err := s.db.WithContext(ctx).- Model(&models.VulnerabilityScanRecord{}).- Select(- "COALESCE(SUM(critical_count), 0) AS critical",- "COALESCE(SUM(high_count), 0) AS high",- "COALESCE(SUM(medium_count), 0) AS medium",- "COALESCE(SUM(low_count), 0) AS low",- "COALESCE(SUM(unknown_count), 0) AS unknown",- "COALESCE(SUM(total_count), 0) AS total",- "COUNT(*) AS scanned",- ).- Where("status = ?", models.ScanStatusCompleted).- Scan(&agg).Error- if err != nil {- return nil, errors.WrapIf(err, "failed to aggregate vulnerability summary")- }-- summary.ScannedImages = agg.Scanned- summary.Summary = &vulnerability.SeveritySummary{- Critical: agg.Critical,- High: agg.High,- Medium: agg.Medium,- Low: agg.Low,- Unknown: agg.Unknown,- Total: agg.Total,- }-- return summary, nil-}--// getActionableCountExcludingIgnoredInternal counts critical and high severity vulnerabilities-// across all completed scans, excluding any that have been ignored. This is used by the dashboard-// to show an accurate action item count that reflects ignore decisions.-func (s *VulnerabilityService) getActionableCountExcludingIgnoredInternal(ctx context.Context) (int, error) {- if s.db == nil {- return 0, nil- }-- var records []models.VulnerabilityScanRecord- err := s.db.WithContext(ctx).- Select("id", "vulnerabilities").- Where("status = ?", models.ScanStatusCompleted).- Where("critical_count + high_count > 0").- Find(&records).Error- if err != nil {- return 0, errors.WrapIf(err, "failed to list vulnerability scans")- }-- if len(records) == 0 {- return 0, nil- }-- // Load all ignore records (not scoped to a single environment, since an- // Arcane instance serves one environment and the dashboard does not carry- // an environment ID through the service layer today).- var ignores []models.VulnerabilityIgnore- if err := s.db.WithContext(ctx).Find(&ignores).Error; err != nil {- return 0, errors.WrapIf(err, "failed to get ignore records")- }-- ignoredKeys := make(map[string]struct{}, len(ignores))- for _, ig := range ignores {- ignoredKeys[vulnerabilityIgnoreKeyInternal(ig.ImageID, ig.VulnerabilityID, ig.PkgName, ig.InstalledVersion)] = struct{}{}- }-- count := 0- for _, record := range records {- if len(record.Vulnerabilities) == 0 || record.Vulnerabilities[0] == "" {- continue- }-- var vulns []vulnerability.Vulnerability- if err := json.Unmarshal([]byte(record.Vulnerabilities[0]), &vulns); err != nil {- slog.WarnContext(ctx, "failed to unmarshal vulnerabilities", "error", err, "image_id", record.ID)- continue- }-- for _, vulnEntry := range vulns {- if vulnEntry.Severity != vulnerability.SeverityCritical && vulnEntry.Severity != vulnerability.SeverityHigh {- continue- }- if len(ignoredKeys) > 0 {- key := vulnerabilityIgnoreKeyInternal(record.ID, vulnEntry.VulnerabilityID, vulnEntry.PkgName, vulnEntry.InstalledVersion)- if _, isIgnored := ignoredKeys[key]; isIgnored {- continue- }- }- count++- }- }-- return count, nil-}--// ListAllVulnerabilities returns a paginated list of vulnerabilities across all scanned images.-func (s *VulnerabilityService) ListAllVulnerabilities(ctx context.Context, envID string, params pagination.QueryParams) ([]vulnerability.VulnerabilityWithImage, pagination.Response, error) {- if params.Limit == 0 {- params.Limit = 20- }-- if s.db == nil {- return []vulnerability.VulnerabilityWithImage{}, pagination.Response{- TotalPages: 1,- TotalItems: 0,- CurrentPage: 1,- ItemsPerPage: params.Limit,- GrandTotalItems: 0,- }, nil- }-- recordsQuery := s.db.WithContext(ctx).- Select("id", "image_name", "vulnerabilities", "total_count").- Where("status = ?", models.ScanStatusCompleted).- Where("total_count > 0")- recordsQuery, hasPossibleMatches := applyListAllVulnerabilityRecordPrefiltersInternal(recordsQuery, params.Filters).Get()- if !hasPossibleMatches {- filtered := pagination.FilterResult[vulnerability.VulnerabilityWithImage]{- Items: []vulnerability.VulnerabilityWithImage{},- TotalCount: 0,- TotalAvailable: 0,- }- response := pagination.BuildResponseFromFilterResult(filtered, params)- return filtered.Items, response, nil- }-- var records []models.VulnerabilityScanRecord- if err := recordsQuery.Find(&records).Error; err != nil {- return nil, pagination.Response{}, errors.WrapIf(err, "failed to list vulnerability scans")- }-- ignoredKeys, err := s.getIgnoredVulnerabilityKeySetByEnvironmentInternal(ctx, envID)- if err != nil {- slog.WarnContext(ctx, "failed to filter ignored vulnerabilities", "error", err)- }-- items := make([]vulnerability.VulnerabilityWithImage, 0, estimatedVulnerabilityRecordCountInternal(records))- for _, record := range records {- if len(record.Vulnerabilities) == 0 || record.Vulnerabilities[0] == "" {- continue- }-- var vulns []vulnerability.Vulnerability- if err := json.Unmarshal([]byte(record.Vulnerabilities[0]), &vulns); err != nil {- slog.WarnContext(ctx, "failed to unmarshal vulnerabilities", "error", err, "image_id", record.ID)- continue- }-- for _, vulnEntry := range vulns {- if len(ignoredKeys) > 0 {- key := vulnerabilityIgnoreKeyInternal(record.ID, vulnEntry.VulnerabilityID, vulnEntry.PkgName, vulnEntry.InstalledVersion)- if _, isIgnored := ignoredKeys[key]; isIgnored {- continue- }- }-- items = append(items, vulnerability.VulnerabilityWithImage{- Vulnerability: vulnEntry,- ImageID: record.ID,- ImageName: record.ImageName,- })- }- }-- config := pagination.Config[vulnerability.VulnerabilityWithImage]{- SearchAccessors: []pagination.SearchAccessor[vulnerability.VulnerabilityWithImage]{- func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.VulnerabilityID, nil },- func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.PkgName, nil },- func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.InstalledVersion, nil },- func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.FixedVersion, nil },- func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.Title, nil },- func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.ImageName, nil },- func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.ImageID, nil },- },- SortBindings: []pagination.SortBinding[vulnerability.VulnerabilityWithImage]{- {- Key: "vulnerabilityId",- Fn: func(a, b vulnerability.VulnerabilityWithImage) int {- return strings.Compare(a.VulnerabilityID, b.VulnerabilityID)- },- },- {- Key: "pkgName",- Fn: func(a, b vulnerability.VulnerabilityWithImage) int { return strings.Compare(a.PkgName, b.PkgName) },- },- {- Key: "severity",- Fn: func(a, b vulnerability.VulnerabilityWithImage) int {- return severityRankInternal(a.Severity) - severityRankInternal(b.Severity)- },- },- {- Key: "installedVersion",- Fn: func(a, b vulnerability.VulnerabilityWithImage) int {- return strings.Compare(a.InstalledVersion, b.InstalledVersion)- },- },- {- Key: "fixedVersion",- Fn: func(a, b vulnerability.VulnerabilityWithImage) int {- return strings.Compare(a.FixedVersion, b.FixedVersion)- },- },- {- Key: "imageName",- Fn: func(a, b vulnerability.VulnerabilityWithImage) int { return strings.Compare(a.ImageName, b.ImageName) },- },- },- FilterAccessors: []pagination.FilterAccessor[vulnerability.VulnerabilityWithImage]{- {- Key: "severity",- Fn: func(item vulnerability.VulnerabilityWithImage, value string) bool {- return strings.EqualFold(string(item.Severity), value)- },- },- {- Key: "imageName",- Fn: func(item vulnerability.VulnerabilityWithImage, value string) bool {- itemLower := strings.ToLower(item.ImageName)- for part := range strings.SplitSeq(value, ",") {- if part = strings.TrimSpace(part); part != "" && strings.Contains(itemLower, strings.ToLower(part)) {- return true- }- }- return false- },- },- },- }-- filtered := pagination.SearchOrderAndPaginate(items, params, config)- response := pagination.BuildResponseFromFilterResult(filtered, params)-- return filtered.Items, response, nil-}--// ListAllVulnerabilityImageOptions returns unique image names (or image IDs when name is empty)-// for vulnerability filtering, optionally constrained by severity.-func (s *VulnerabilityService) ListAllVulnerabilityImageOptions(ctx context.Context, severityFilter string) ([]string, error) {- if s.db == nil {- return []string{}, nil- }-- query := s.db.WithContext(ctx).- Model(&models.VulnerabilityScanRecord{}).- Select("id", "image_name").- Where("status = ?", models.ScanStatusCompleted).- Where("total_count > 0")-- if strings.TrimSpace(severityFilter) != "" {- var hasPossibleMatches bool- query, hasPossibleMatches = applyListAllVulnerabilityRecordPrefiltersInternal(query, map[string]string{- "severity": severityFilter,- }).Get()- if !hasPossibleMatches {- return []string{}, nil- }- }-- var records []models.VulnerabilityScanRecord- if err := query.Find(&records).Error; err != nil {- return nil, errors.WrapIf(err, "failed to list vulnerability image options")- }-- seen := make(map[string]struct{}, len(records))- options := make([]string, 0, len(records))- for _, record := range records {- label := strings.TrimSpace(record.ImageName)- if label == "" {- label = strings.TrimSpace(record.ID)- }- if label == "" {- continue- }- if _, exists := seen[label]; exists {- continue- }-- seen[label] = struct{}{}- options = append(options, label)- }-- sort.Strings(options)- return options, nil-}--func applyListAllVulnerabilityRecordPrefiltersInternal(query *gorm.DB, filters map[string]string) mo.Option[*gorm.DB] {- if query == nil || len(filters) == 0 {- return mo.Some(query)- }-- if severityFilter, ok := filters["severity"]; ok {- severityColumns := severityCountColumnsForFilterInternal(severityFilter)- if len(severityColumns) == 0 {- return mo.None[*gorm.DB]()- }-- parts := make([]string, 0, len(severityColumns))- args := make([]any, 0, len(severityColumns))- for _, col := range severityColumns {- parts = append(parts, col+" > ?")- args = append(args, 0)- }- query = query.Where("("+strings.Join(parts, " OR ")+")", args...)- }-- if imageNameFilter, ok := filters["imageName"]; ok {- imageNameParts := splitNonEmptyCSVInternal(imageNameFilter)- if len(imageNameParts) == 0 {- return mo.None[*gorm.DB]()- }-- clauses := make([]string, 0, len(imageNameParts))- args := make([]any, 0, len(imageNameParts))- for _, part := range imageNameParts {- clauses = append(clauses, "LOWER(image_name) LIKE ?")- args = append(args, "%"+strings.ToLower(part)+"%")- }- query = query.Where("("+strings.Join(clauses, " OR ")+")", args...)- }-- return mo.Some(query)-}--func severityCountColumnsForFilterInternal(value string) []string {- parts := splitNonEmptyCSVInternal(value)- if len(parts) == 0 {- return nil- }-- columns := make([]string, 0, len(parts))- seen := make(map[string]struct{}, len(parts))- for _, part := range parts {- var col string- switch {- case strings.EqualFold(part, string(vulnerability.SeverityCritical)):- col = "critical_count"- case strings.EqualFold(part, string(vulnerability.SeverityHigh)):- col = "high_count"- case strings.EqualFold(part, string(vulnerability.SeverityMedium)):- col = "medium_count"- case strings.EqualFold(part, string(vulnerability.SeverityLow)):- col = "low_count"- case strings.EqualFold(part, string(vulnerability.SeverityUnknown)):- col = "unknown_count"- default:- continue- }-- if _, exists := seen[col]; exists {- continue- }-- seen[col] = struct{}{}- columns = append(columns, col)- }-- return columns-}--func splitNonEmptyCSVInternal(value string) []string {- parts := make([]string, 0)- for part := range strings.SplitSeq(value, ",") {- part = strings.TrimSpace(part)- if part == "" {- continue- }- parts = append(parts, part)- }- return parts-}--func estimatedVulnerabilityRecordCountInternal(records []models.VulnerabilityScanRecord) int {- if len(records) == 0 {- return 0- }-- total := 0- for i := range records {- if records[i].TotalCount <= 0 {- continue- }-- if total > math.MaxInt-records[i].TotalCount {- return math.MaxInt- }- total += records[i].TotalCount- }-- return total-}--func vulnerabilityIgnoreKeyInternal(imageID, vulnerabilityID, pkgName, installedVersion string) string {- return imageID + ":" + vulnerabilityID + ":" + pkgName + ":" + installedVersion-}--func (s *VulnerabilityService) getIgnoredVulnerabilityKeySetByEnvironmentInternal(ctx context.Context, envID string) (map[string]struct{}, error) {- if s.db == nil {- return nil, nil- }-- var ignores []models.VulnerabilityIgnore- if err := s.db.WithContext(ctx).Where("environment_id = ?", envID).Find(&ignores).Error; err != nil {- return nil, errors.WrapIf(err, "failed to get ignore records")- }-- if len(ignores) == 0 {- return nil, nil- }-- ignoredKeys := make(map[string]struct{}, len(ignores))- for _, ignore := range ignores {- key := vulnerabilityIgnoreKeyInternal(ignore.ImageID, ignore.VulnerabilityID, ignore.PkgName, ignore.InstalledVersion)- ignoredKeys[key] = struct{}{}- }-- return ignoredKeys, nil-}--func severityRankInternal(severity vulnerability.Severity) int {- switch severity {- case vulnerability.SeverityCritical:- return 4- case vulnerability.SeverityHigh:- return 3- case vulnerability.SeverityMedium:- return 2- case vulnerability.SeverityLow:- return 1- case vulnerability.SeverityUnknown:- return 0- default:- return 0- }-}--// GetScanSummariesByImageIDs retrieves scan summaries for multiple images-func (s *VulnerabilityService) GetScanSummariesByImageIDs(ctx context.Context, imageIDs []string) (map[string]*vulnerability.ScanSummary, error) {- if s.db == nil || len(imageIDs) == 0 {- return make(map[string]*vulnerability.ScanSummary), nil- }-- var records []models.VulnerabilityScanRecord- err := s.db.WithContext(ctx).- Select("id", "scan_time", "status", "critical_count", "high_count", "medium_count", "low_count", "unknown_count", "total_count", "error").- Where("id IN ?", imageIDs).- Find(&records).Error- if err != nil {- return nil, errors.WrapIf(err, "failed to get scan summaries")- }-- result := make(map[string]*vulnerability.ScanSummary, len(records))- for _, record := range records {- s.markStaleScanIfNeeded(ctx, &record)- var errStr string- if record.Error != nil {- errStr = *record.Error- }-- result[record.ID] = &vulnerability.ScanSummary{- ImageID: record.ID,- ScanTime: record.ScanTime,- Status: vulnerability.ScanStatus(record.Status),- Summary: &vulnerability.SeveritySummary{- Critical: record.CriticalCount,- High: record.HighCount,- Medium: record.MediumCount,- Low: record.LowCount,- Unknown: record.UnknownCount,- Total: record.TotalCount,- },- Error: errStr,- }- s.applyScanPhaseToSummaryInternal(result[record.ID])- }-- return result, nil-}--type scheduledScanNotificationSummary struct {- totalFixable int- imagesWithFixable int- severityCounts map[string]int- sampleCVEs []string- seenCVEs map[string]struct{}-}--func newScheduledScanNotificationSummary() *scheduledScanNotificationSummary {- return &scheduledScanNotificationSummary{- severityCounts: map[string]int{- "CRITICAL": 0,- "HIGH": 0,- "MEDIUM": 0,- "LOW": 0,- "UNKNOWN": 0,- },- sampleCVEs: make([]string, 0, 5),- seenCVEs: make(map[string]struct{}),- }-}--func scheduledScanImageTargetInternal(img imagetypes.Summary) (imageID, imageName string, shouldScan bool) {- imageID = img.ID- imageName = imageID- if len(img.RepoTags) > 0 {- imageName = img.RepoTags[0]- } else if len(img.RepoDigests) > 0 {- imageName = img.RepoDigests[0]- }-- // Skip intermediate / dangling images with no usable name.- if imageName == "<none>:<none>" || imageName == imageID {- return imageID, imageName, false- }-- return imageID, imageName, true-}--func (s *VulnerabilityService) runScheduledImageScanInternal(- ctx context.Context,- containerID, imageName, imageID, scannerVersion string,-) (*vulnerability.ScanResult, int64, error) {- startTime := time.Now()- _, releaseSlot, slotErr := s.acquireTrivyScanSlotInternal(ctx)- if slotErr != nil {- return nil, 0, slotErr- }-- var result *vulnerability.ScanResult- var scanErr error- func() {- defer releaseSlot()- result, scanErr = s.execTrivyScanInContainer(ctx, containerID, imageName, imageID, scannerVersion)- }()-- return result, time.Since(startTime).Milliseconds(), scanErr-}--func (s *VulnerabilityService) saveScheduledFailedScanInternal(- ctx context.Context,- envID, imageID, imageName string,- user models.User,- duration int64,- scanErr error,-) {- failedResult := &vulnerability.ScanResult{- ImageID: imageID,- ImageName: imageName,- ScanTime: time.Now(),- Status: vulnerability.ScanStatusFailed,- Error: scanErr.Error(),- Duration: duration,- }- if saveErr := s.saveScanResultWithRetryInternal(ctx, failedResult); saveErr != nil {- slog.WarnContext(ctx, "failed to save failed scan result", "error", saveErr)- }- s.logScheduledScanEvent(ctx, envID, imageID, imageName, user, false, scanErr.Error())-}--func (s *VulnerabilityService) saveScheduledSuccessfulScanInternal(- ctx context.Context,- envID, imageID, imageName string,- user models.User,- result *vulnerability.ScanResult,- duration int64,-) {- result.Duration = duration- s.ensureSummary(result)- if saveErr := s.saveScanResultWithRetryInternal(ctx, result); saveErr != nil {- slog.WarnContext(ctx, "failed to save scan result", "error", saveErr)- }- s.logScheduledScanEvent(ctx, envID, imageID, imageName, user, true, "")-}--func (s *VulnerabilityService) collectScheduledFixableSummaryInternal(- ctx context.Context,- result *vulnerability.ScanResult,- summary *scheduledScanNotificationSummary,-) {- fixableVulns := s.fixedVulnerabilitiesForNotifications(ctx, result)- if len(fixableVulns) == 0 {- return- }-- summary.imagesWithFixable++- for i := range fixableVulns {- v := fixableVulns[i]- summary.totalFixable++-- sev := strings.ToUpper(strings.TrimSpace(string(v.Severity)))- if _, ok := summary.severityCounts[sev]; !ok {- sev = "UNKNOWN"- }- summary.severityCounts[sev]++-- cveID := strings.TrimSpace(v.VulnerabilityID)- if cveID == "" {- continue- }- if _, alreadyAdded := summary.seenCVEs[cveID]; alreadyAdded {- continue- }- summary.seenCVEs[cveID] = struct{}{}- if len(summary.sampleCVEs) < 5 {- summary.sampleCVEs = append(summary.sampleCVEs, cveID)- }- }-}--func (s *VulnerabilityService) sendScheduledVulnerabilitySummaryInternal(- ctx context.Context,- scanned int,- summary *scheduledScanNotificationSummary,-) {- if summary.totalFixable == 0 || s.notificationService == nil {- return- }-- summaryDate := time.Now().UTC().Format("2006-01-02")- payload := VulnerabilityNotificationPayload{- CVEID: "Daily Summary - " + summaryDate,- Severity: fmt.Sprintf("Critical:%d High:%d Medium:%d Low:%d Unknown:%d", summary.severityCounts["CRITICAL"], summary.severityCounts["HIGH"], summary.severityCounts["MEDIUM"], summary.severityCounts["LOW"], summary.severityCounts["UNKNOWN"]),- ImageName: fmt.Sprintf("%d image(s) scanned, %d with fixable vulnerabilities", scanned, summary.imagesWithFixable),- FixedVersion: fmt.Sprintf("%d fixable vulnerability record(s)", summary.totalFixable),- }- if len(summary.sampleCVEs) > 0 {- payload.PkgName = strings.Join(summary.sampleCVEs, ", ")- }-- if err := s.notificationService.SendVulnerabilityNotification(ctx, payload); err != nil {- slog.WarnContext(ctx, "failed to send daily vulnerability summary notification", "error", err)- }-}--// ScanAllImages scans all Docker images for vulnerabilities. It is intended-// for use by the scheduled vulnerability scan job. A single long-running Trivy-// container is created and reused for every image via docker exec, which avoids-// the overhead of creating/destroying a container per scan. The caller-supplied-// user is recorded in the event log.-func (s *VulnerabilityService) ScanAllImages(ctx context.Context, envID string, user models.User) (scanned, failed int, err error) {- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return 0, 0, errors.WrapIf(err, "failed to connect to Docker")- }-- imageList, err := dockerClient.ImageList(ctx, client.ImageListOptions{})- if err != nil {- return 0, 0, errors.WrapIf(err, "failed to list images")- }- images := imageList.Items-- containerID, cleanup, err := s.setupTrivyBatchContainer(ctx)- if err != nil {- return 0, 0, err- }- defer cleanup()-- scannerVersion := s.GetTrivyVersion(ctx)- notificationSummary := newScheduledScanNotificationSummary()-- for _, img := range images {- if ctx.Err() != nil {- return scanned, failed, ctx.Err()- }-- switch s.scanSingleImage(ctx, containerID, img, scannerVersion, envID, user, notificationSummary) {- case scanStatusSuccess:- scanned++- case scanStatusFailed:- failed++- case scanStatusSkipped:- // no-op- }- }-- s.sendScheduledVulnerabilitySummaryInternal(ctx, scanned, notificationSummary)-- return scanned, failed, nil-}--type scanResultStatus int--const (- scanStatusSkipped scanResultStatus = iota- scanStatusSuccess- scanStatusFailed-)--func (s *VulnerabilityService) setupTrivyBatchContainer(ctx context.Context) (string, func(), error) {- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return "", nil, errors.WrapIf(err, "failed to connect to Docker")- }-- trivyImage, err := s.ensureTrivyImageInternal(ctx)- if err != nil {- return "", nil, errors.WrapIf(err, "trivy scanner not available")- }-- serverURL, _ := s.getTrivyServerConfig()- var cacheVolume string- if serverURL == "" {- cacheVolume, err = s.ensureTrivyCacheVolumeInternal(ctx)- if err != nil {- return "", nil, errors.WrapIf(err, "failed to ensure trivy cache volume")- }- }-- containerID, err := s.createBatchTrivyContainer(ctx, trivyImage, cacheVolume)- if err != nil {- return "", nil, errors.WrapIf(err, "failed to create batch trivy container")- }-- cleanup := func() {- removeDockerContainerInternal(ctx, dockerClient, containerID, "failed to remove batch trivy container")- }-- return containerID, cleanup, nil-}--func (s *VulnerabilityService) scanSingleImage(- ctx context.Context,- containerID string,- img imagetypes.Summary,- scannerVersion, envID string,- user models.User,- notificationSummary *scheduledScanNotificationSummary,-) scanResultStatus {- imageID, imageName, shouldScan := scheduledScanImageTargetInternal(img)- if !shouldScan {- return scanStatusSkipped- }-- slog.InfoContext(ctx, "scheduled vulnerability scan: scanning image", "image", imageName, "imageId", imageID)-- result, duration, scanErr := s.runScheduledImageScanInternal(ctx, containerID, imageName, imageID, scannerVersion)- if scanErr != nil {- s.saveScheduledFailedScanInternal(ctx, envID, imageID, imageName, user, duration, scanErr)- return scanStatusFailed- }-- s.saveScheduledSuccessfulScanInternal(ctx, envID, imageID, imageName, user, result, duration)- s.collectScheduledFixableSummaryInternal(ctx, result, notificationSummary)- return scanStatusSuccess-}--// createBatchTrivyContainer creates a long-running Trivy container that stays-// alive so we can exec individual scan commands into it. This avoids the-// overhead of creating/starting/removing a container for every image.-func (s *VulnerabilityService) createBatchTrivyContainer(ctx context.Context, trivyImage, cacheVolume string) (string, error) {- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return "", errors.WrapIf(err, "failed to connect to Docker")- }-- runtimeOptions, err := s.resolveTrivyRuntimeOptionsInternal(ctx, dockerClient)- if err != nil {- return "", err- }-- registryConfigJSON := s.buildTrivyRegistryConfigInternal(ctx)- env := append([]string(nil), runtimeOptions.ContainerEnv...)- if len(registryConfigJSON) > 0 {- env = append(env, "DOCKER_CONFIG="+trivyRegistryConfigDir)- // ECR tokens are refreshed when the batch container is created; a multi-hour- // batch could outlive the token and fall back to anonymous pulls.- }-- config := &containertypes.Config{- Image: trivyImage,- Entrypoint: []string{"sh", "-c", "trap 'exit 0' TERM; while :; do sleep 1; done"},- Env: env,- Labels: map[string]string{- libarcane.InternalResourceLabel: "true",- },- }-- runtimeSecurity := s.getTrivyRuntimeSecurityInternal().OrEmpty()- securityOpts, privileged := runtimeSecurity.securityOpts, runtimeSecurity.privileged- hostConfig := buildTrivyBatchHostConfigInternal(cacheVolume, runtimeOptions, securityOpts, privileged)-- resources, cpuSet, applyLimits := s.getTrivyContainerResourceOptionsInternal(ctx, dockerClient)- applyTrivyContainerResourcesInternal(hostConfig, resources, cpuSet, applyLimits)-- containerID, err := s.createAndStartTrivyContainerInternal(ctx, dockerClient, "batch", config, hostConfig, registryConfigJSON)- if err != nil {- return "", err- }-- slog.InfoContext(ctx, "batch trivy container started", "containerId", containerID)-- return containerID, nil-}--// execTrivyScanInContainer runs a trivy scan for a single image inside an-// already-running container via docker exec.-func (s *VulnerabilityService) execTrivyScanInContainer(ctx context.Context, containerID, imageName, imageID, scannerVersion string) (*vulnerability.ScanResult, error) {- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, errors.WrapIf(err, "failed to connect to Docker")- }-- stdoutFile, err := createTrivyLogTempFileInternal("trivy-exec-stdout-*")- if err != nil {- return nil, errors.WrapIf(err, "failed to create trivy exec stdout temp file")- }-- stderrFile, err := createTrivyLogTempFileInternal("trivy-exec-stderr-*")- if err != nil {- cleanupTrivyLogTempFilesInternal(ctx, stdoutFile)- return nil, errors.WrapIf(err, "failed to create trivy exec stderr temp file")- }- defer cleanupTrivyLogTempFilesInternal(ctx, stdoutFile, stderrFile)-- trivyTimeout := s.getTrivyScanTimeoutArg()- outputPath := newTrivyOutputPathInternal()- defer cleanupTrivyOutputFileInContainerInternal(ctx, dockerClient, containerID, outputPath)-- serverURL, serverToken := s.getTrivyServerConfig()- execCmd := make([]string, 0, 16)- execCmd = append(execCmd, "trivy", "image", "--format", "json", "--quiet", "--output", outputPath, "--timeout", trivyTimeout)- execCmd = append(execCmd, vuln.ScanSourceArgs(serverURL)...)- execCmd = append(execCmd, vuln.IgnoreUnfixedArgs(s.getTrivyIgnoreUnfixed())...)- execCmd = append(execCmd, imageName)-- execCfg := client.ExecCreateOptions{- Cmd: execCmd,- Env: vuln.ServerTokenEnv(serverToken),- AttachStdout: true,- AttachStderr: true,- }-- execResp, err := dockerClient.ExecCreate(ctx, containerID, execCfg)- if err != nil {- return nil, errors.WrapIf(err, "failed to create exec")- }-- attachResp, err := dockerClient.ExecAttach(ctx, execResp.ID, client.ExecAttachOptions{})- if err != nil {- return nil, errors.WrapIf(err, "failed to attach to exec")- }- defer attachResp.Close()-- logDone := startDockerLogCopyInternal(attachResp.Reader, stdoutFile, stderrFile)- if err := waitForDockerLogCopyInternal(logDone); err != nil {- return nil, errors.WrapIf(err, "failed to read exec output")- }-- // Check exit code- execInspect, err := dockerClient.ExecInspect(ctx, execResp.ID, client.ExecInspectOptions{})- if err != nil {- return nil, errors.WrapIf(err, "failed to inspect exec")- }-- if execInspect.ExitCode != 0 {- errMsg := readTempFileExcerptInternal(stderrFile)- if errMsg == "" {- errMsg = readTempFileExcerptInternal(stdoutFile)- }- if errMsg == "" {- errMsg = fmt.Sprintf("exit status %d", execInspect.ExitCode)- }- slog.WarnContext(ctx,- "trivy scan diagnostics",- "image", imageName,- "imageId", imageID,- "reason", "exec non-zero exit",- "exitCode", int64(execInspect.ExitCode),- "stdoutBytes", tempFileSizeInternal(stdoutFile),- "stderrBytes", tempFileSizeInternal(stderrFile),- )- return nil, formatTrivyExitErrorInternal(int64(execInspect.ExitCode), errMsg, imageName)- }-- trivyOutput, err := readTrivyOutputFromContainerFileInternal(ctx, dockerClient, containerID, outputPath)- if err != nil {- slog.WarnContext(ctx,- "failed to read trivy output file from batch container; falling back to stdout parsing",- "containerId", containerID,- "outputPath", outputPath,- "error", err,- )- trivyOutput = nil- }-- var trivyReport *vulnerability.TrivyReport- if len(trivyOutput) > 0 {- trivyReport, err = decodeTrivyReportFromBytesInternal(trivyOutput)- } else {- trivyReport, err = decodeTrivyReportFromFileInternal(stdoutFile)- }- if err != nil {- if errors.Is(err, io.EOF) {- errMsg := readTempFileExcerptInternal(stderrFile)- if errMsg == "" {- errMsg = "trivy scan produced no output"- }- slog.WarnContext(ctx,- "trivy scan diagnostics",- "image", imageName,- "imageId", imageID,- "reason", "exec empty output",- "exitCode", int64(execInspect.ExitCode),- "stdoutBytes", tempFileSizeInternal(stdoutFile),- "stderrBytes", tempFileSizeInternal(stderrFile),- )- return nil, errors.Errorf("trivy scan failed: %s", errMsg)- }-- slog.WarnContext(ctx,- "trivy scan diagnostics",- "image", imageName,- "imageId", imageID,- "reason", "exec parse error",- "exitCode", int64(execInspect.ExitCode),- "stdoutBytes", tempFileSizeInternal(stdoutFile),- "stderrBytes", tempFileSizeInternal(stderrFile),- )- return nil, errors.WrapIf(err, "failed to parse trivy output")- }-- if trivyReport == nil {- errMsg := readTempFileExcerptInternal(stderrFile)- if errMsg == "" {- errMsg = "trivy scan produced no output"- }- slog.WarnContext(ctx,- "trivy scan diagnostics",- "image", imageName,- "imageId", imageID,- "reason", "exec nil report",- "exitCode", int64(execInspect.ExitCode),- "stdoutBytes", tempFileSizeInternal(stdoutFile),- "stderrBytes", tempFileSizeInternal(stderrFile),- )- return nil, errors.Errorf("trivy scan failed: %s", errMsg)- }-- result := vulnerability.ConvertTrivyReportToScanResult(trivyReport, imageID, time.Now(), 0)- result.ScannerVersion = scannerVersion-- return result, nil-}--// DeleteScanResult deletes the scan result for an image-func (s *VulnerabilityService) DeleteScanResult(ctx context.Context, imageID string) error {- if s.db == nil {- return nil- }-- return s.db.WithContext(ctx).Where("id = ?", imageID).Delete(&models.VulnerabilityScanRecord{}).Error-}--// DeleteScanResultsByImageIDs deletes scan results for multiple images in one query.-func (s *VulnerabilityService) DeleteScanResultsByImageIDs(ctx context.Context, imageIDs []string) error {- if s.db == nil || len(imageIDs) == 0 {- return nil- }-- return s.db.WithContext(ctx).Where("id IN ?", imageIDs).Delete(&models.VulnerabilityScanRecord{}).Error-}--// CleanupOrphanedScanRecords removes vulnerability scan records for images that-// no longer exist in Docker. This keeps "images scanned" counts in sync (e.g.-// avoids "5/3" when images were deleted after being scanned). Safe to call-// even when no images exist; returns the number of records deleted.-func (s *VulnerabilityService) CleanupOrphanedScanRecords(ctx context.Context) (deleted int64, err error) {- if s.db == nil {- return 0, nil- }-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return 0, errors.WrapIf(err, "failed to connect to Docker")- }-- imageList, err := dockerClient.ImageList(ctx, client.ImageListOptions{})- if err != nil {- return 0, errors.WrapIf(err, "failed to list images")- }- images := imageList.Items-- currentIDs := make(map[string]struct{}, len(images))- for _, img := range images {- currentIDs[img.ID] = struct{}{}- }-- // Build list for NOT IN. If no images, we delete all scan records.- var ids []string- if len(currentIDs) > 0 {- ids = make([]string, 0, len(currentIDs))- for id := range currentIDs {- ids = append(ids, id)- }- }-- var result *gorm.DB- if len(ids) == 0 {- result = s.db.WithContext(ctx).Where("1 = 1").Delete(&models.VulnerabilityScanRecord{})- } else {- result = s.db.WithContext(ctx).Where("id NOT IN ?", ids).Delete(&models.VulnerabilityScanRecord{})- }- if result.Error != nil {- return 0, result.Error- }-- return result.RowsAffected, nil-}--// GetTrivyVersion returns the Trivy version from the configured scanner image.-func (s *VulnerabilityService) GetTrivyVersion(ctx context.Context) string {- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return ""- }-- trivyImage, err := s.ensureTrivyImageInternal(ctx)- if err != nil {- return ""- }-- config := &containertypes.Config{- Image: trivyImage,- Entrypoint: []string{"trivy"},- Cmd: []string{"--version"},- Labels: map[string]string{- libarcane.InternalResourceLabel: "true",- },- }-- hostConfig := &containertypes.HostConfig{- AutoRemove: true,- }-- resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{- Config: config,- HostConfig: hostConfig,- })- if err != nil {- return ""- }-- if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {- _, _ = dockerClient.ContainerRemove(ctx, resp.ID, client.ContainerRemoveOptions{Force: true})- return ""- }-- logs, err := dockerClient.ContainerLogs(ctx, resp.ID, client.ContainerLogsOptions{- ShowStdout: true,- ShowStderr: true,- Follow: true,- })- if err != nil {- _, _ = dockerClient.ContainerRemove(ctx, resp.ID, client.ContainerRemoveOptions{Force: true})- return ""- }- var closeLogsOnce sync.Once- closeLogs := func() {- closeLogsOnce.Do(func() {- if closeErr := logs.Close(); closeErr != nil && !isExpectedDockerStreamEndErrorInternal(closeErr) {- slog.WarnContext(ctx, "failed to close trivy version logs stream", "error", closeErr)- }- })- }- defer closeLogs()-- stdoutFile, err := createTrivyLogTempFileInternal("trivy-version-stdout-*")- if err != nil {- return ""- }-- stderrFile, err := createTrivyLogTempFileInternal("trivy-version-stderr-*")- if err != nil {- cleanupTrivyLogTempFilesInternal(ctx, stdoutFile)- return ""- }- defer cleanupTrivyLogTempFilesInternal(ctx, stdoutFile, stderrFile)-- logDone := startDockerLogCopyInternal(logs, stdoutFile, stderrFile)-- waitResult := dockerClient.ContainerWait(ctx, resp.ID, client.ContainerWaitOptions{Condition: containertypes.WaitConditionNotRunning})- var waitResp containertypes.WaitResponse- select {- case err := <-waitResult.Error:- if err != nil {- return ""- }- case waitResp = <-waitResult.Result:- }-- closeLogs()-- if err := waitForDockerLogCopyInternal(logDone); err != nil {- return ""- }-- if waitResp.StatusCode != 0 {- return ""- }-- return vuln.ParseVersion(readTempFileExcerptInternal(stdoutFile))-}--func (s *VulnerabilityService) getTrivyImageRef() string {- return DefaultTrivyImage-}--func (s *VulnerabilityService) getConfiguredTrivyNetworkModeInternal() string {- if s.settingsService == nil {- return ""- }-- cfg := s.settingsService.GetSettingsConfig()- if cfg == nil {- return ""- }-- return strings.TrimSpace(cfg.TrivyNetwork.Value)-}--func (s *VulnerabilityService) getEffectiveDockerHostInternal() string {- if s.dockerService != nil && s.dockerService.config != nil {- if host := strings.TrimSpace(s.dockerService.config.DockerHost); host != "" {- return host- }- }-- if s.settingsService != nil {- if cfg := s.settingsService.GetSettingsConfig(); cfg != nil {- if host := strings.TrimSpace(cfg.DockerHost.Value); host != "" {- return host- }- }- }-- return vuln.DefaultDockerHostURI-}--func resolveTrivyUnixSocketSourceInternal(- ctx context.Context,- socketPath string,- discoverHostPath func(context.Context, string) (string, error),- isRunningInDocker func() bool,-) (string, error) {- socketPath = strings.TrimSpace(socketPath)- if socketPath == "" {- return "", errors.New("unix docker host is missing a socket path")- }-- if discoverHostPath != nil {- hostPath, err := discoverHostPath(ctx, socketPath)- if err == nil && strings.TrimSpace(hostPath) != "" {- return strings.TrimSpace(hostPath), nil- }- if err != nil && (isRunningInDocker == nil || isRunningInDocker()) {- return "", errors.WrapIff(err, "failed to resolve socket path %q", socketPath)- }- }-- if isRunningInDocker != nil && isRunningInDocker() {- return "", errors.Errorf("failed to resolve socket path %q to a daemon-visible host path", socketPath)- }-- return socketPath, nil-}--func findArcaneContainerByLabelInternal(ctx context.Context, dockerClient *client.Client) string {- if dockerClient == nil {- return ""- }-- filters := make(client.Filters)- filters = filters.Add("label", labels.LabelArcane+"=true")-- containerList, err := dockerClient.ContainerList(ctx, client.ContainerListOptions{- All: true,- Filters: filters,- })- if err != nil {- return ""- }-- var fallbackID string- for _, containerSummary := range containerList.Items {- if value, ok := containerSummary.Labels["com.getarcaneapp.arcane.upgrader"]; ok &&- strings.EqualFold(strings.TrimSpace(value), "true") {- continue- }- if containerSummary.State == containertypes.StateRunning {- return containerSummary.ID- }- if fallbackID == "" {- fallbackID = containerSummary.ID- }- }-- return fallbackID-}--func (s *VulnerabilityService) inspectCurrentArcaneContainerInternal(- ctx context.Context,- dockerClient *client.Client,-) (*containertypes.InspectResponse, error) {- if dockerClient == nil {- return nil, errors.New("docker client is not available")- }-- if containerID, err := cgroup.CurrentContainerID(); err == nil {- inspect, inspectErr := libarcane.ContainerInspectWithCompatibility(- ctx,- dockerClient,- containerID,- client.ContainerInspectOptions{},- )- if inspectErr == nil {- return &inspect.Container, nil- }- }-- containerID := findArcaneContainerByLabelInternal(ctx, dockerClient)- if containerID == "" {- return nil, errors.New("could not detect Arcane container")- }-- inspect, err := libarcane.ContainerInspectWithCompatibility(- ctx,- dockerClient,- containerID,- client.ContainerInspectOptions{},- )- if err != nil {- return nil, errors.WrapIf(err, "inspect Arcane container")- }-- return &inspect.Container, nil-}--func selectTrivyAutoNetworkModeInternal(inspect *containertypes.InspectResponse) string {- if inspect == nil {- return DefaultTrivyNetworkMode- }-- if inspect.HostConfig != nil {- networkMode := strings.TrimSpace(string(inspect.HostConfig.NetworkMode))- if networkMode != "" && networkMode != "default" && networkMode != DefaultTrivyNetworkMode &&- !containertypes.NetworkMode(networkMode).IsContainer() {- return networkMode- }- }-- if inspect.NetworkSettings != nil && len(inspect.NetworkSettings.Networks) > 0 {- networkNames := make([]string, 0, len(inspect.NetworkSettings.Networks))- for networkName := range inspect.NetworkSettings.Networks {- networkName = strings.TrimSpace(networkName)- if networkName != "" {- networkNames = append(networkNames, networkName)- }- }- sort.Strings(networkNames)-- for _, networkName := range networkNames {- if !dockerutils.IsDefaultNetwork(networkName) {- return networkName- }- }-- for _, networkName := range networkNames {- if containertypes.NetworkMode(networkName).IsContainer() {- continue- }- if networkName == "host" || networkName == "none" || networkName == DefaultTrivyNetworkMode {- return networkName- }- }- }-- if inspect.HostConfig != nil {- networkMode := strings.TrimSpace(string(inspect.HostConfig.NetworkMode))- if networkMode != "" && networkMode != "default" && !containertypes.NetworkMode(networkMode).IsContainer() {- return networkMode- }- }-- return DefaultTrivyNetworkMode-}--func (s *VulnerabilityService) resolveTrivyNetworkModeInternal(ctx context.Context, dockerClient *client.Client) string {- if configuredNetwork := s.getConfiguredTrivyNetworkModeInternal(); configuredNetwork != "" {- return configuredNetwork- }-- inspect, err := s.inspectCurrentArcaneContainerInternal(ctx, dockerClient)- if err == nil {- return selectTrivyAutoNetworkModeInternal(inspect)- }-- slog.DebugContext(ctx, "failed to auto-detect Trivy network; falling back to bridge", "error", err)- return DefaultTrivyNetworkMode-}--func (s *VulnerabilityService) resolveTrivyRuntimeOptionsInternal(ctx context.Context, dockerClient *client.Client) (trivyRuntimeOptions, error) {- options := trivyRuntimeOptions{- NetworkMode: s.resolveTrivyNetworkModeInternal(ctx, dockerClient),- }-- dockerHost := s.getEffectiveDockerHostInternal()- scheme, socketPath, err := vuln.ParseDockerHost(dockerHost)- if err != nil {- return trivyRuntimeOptions{}, errors.WrapIf(err, "failed to resolve trivy docker host")- }-- options.ContainerEnv = vuln.BuildDockerHostEnv(dockerHost)- if scheme != "unix" {- return options, nil- }-- socketSource, err := resolveTrivyUnixSocketSourceInternal(- ctx,- socketPath,- func(ctx context.Context, containerPath string) (string, error) {- return projects.GetHostPathForContainerPath(ctx, dockerClient, containerPath)- },- func() bool {- _, err := cgroup.CurrentContainerID()- return err == nil- },- )- if err != nil {- return trivyRuntimeOptions{}, errors.WrapIf(err, "failed to resolve trivy docker host")- }-- options.Mounts = append(options.Mounts, mounttypes.Mount{- Type: mounttypes.TypeBind,- Source: socketSource,- Target: socketPath,- })-- return options, nil-}--func (s *VulnerabilityService) getTrivyRuntimeSecurityInternal() mo.Option[trivyRuntimeSecurityInternal] {- if s.settingsService == nil {- return mo.None[trivyRuntimeSecurityInternal]()- }-- cfg := s.settingsService.GetSettingsConfig()- if cfg == nil {- return mo.None[trivyRuntimeSecurityInternal]()- }-- return mo.Some(trivyRuntimeSecurityInternal{- securityOpts: vuln.ParseSecurityOpts(cfg.TrivySecurityOpts.Value),- privileged: cfg.TrivyPrivileged.IsTrue(),- })-}--func (s *VulnerabilityService) getTrivyScanTimeoutArg() string {- timeoutSeconds := 0- if s.settingsService != nil {- timeoutSeconds = s.settingsService.GetSettingsConfig().TrivyScanTimeout.AsInt()- }-- return timeouts.GetDuration(timeoutSeconds, timeouts.DefaultTrivyScan).String()-}--func newTrivyOutputPathInternal() string {- return trivyOutputPathPrefix + strconv.FormatInt(time.Now().UnixNano(), 10) + ".json"-}--func (s *VulnerabilityService) getTrivySingleScanCacheDirForSlotInternal(slotID int) string {- if slotID < 0 {- slotID = 0- }-- // Use a fixed cache directory per concurrency slot on the shared cache- // volume. This avoids Trivy fs-cache lock contention while bounding cache- // growth to the configured slot count.- return fmt.Sprintf("%s/slot-%d", trivyCacheSlotRootDir, slotID)-}--func (s *VulnerabilityService) ensureTrivyImageInternal(ctx context.Context) (string, error) {- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return "", errors.WrapIf(err, "failed to connect to Docker")- }-- trivyImage := s.getTrivyImageRef()- if _, err := dockerClient.ImageInspect(ctx, trivyImage); err == nil {- return trivyImage, nil- }-- pullTimeoutSeconds := 0- if s.settingsService != nil && s.settingsService.GetSettingsConfig() != nil {- pullTimeoutSeconds = s.settingsService.GetSettingsConfig().DockerImagePullTimeout.AsInt()- }-- pullCtx, pullCancel := context.WithTimeout(ctx, timeouts.GetDuration(pullTimeoutSeconds, timeouts.DefaultDockerImagePull))- defer pullCancel()-- pullReader, err := dockerClient.ImagePull(pullCtx, trivyImage, client.ImagePullOptions{})- if err != nil {- if errors.Is(pullCtx.Err(), context.DeadlineExceeded) {- return "", errors.Errorf("scanner image pull timed out for %s (increase DOCKER_IMAGE_PULL_TIMEOUT or setting)", trivyImage)- }- return "", errors.WrapIff(err, "pull scanner image %s", trivyImage)- }- if err := dockerutils.RenderJSONMessageStream(pullReader, io.Discard); err != nil {- _ = pullReader.Close()- return "", errors.WrapIf(err, "failed to complete scanner image pull")- }- _ = pullReader.Close()-- return trivyImage, nil-}--func (s *VulnerabilityService) ensureTrivyCacheVolumeInternal(ctx context.Context) (string, error) {- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return "", err- }-- if _, err := dockerClient.VolumeInspect(ctx, trivyCacheVolumeName, client.VolumeInspectOptions{}); err == nil {- return trivyCacheVolumeName, nil- }-- _, err = dockerClient.VolumeCreate(ctx, client.VolumeCreateOptions{- Name: trivyCacheVolumeName,- Labels: map[string]string{- libarcane.InternalResourceLabel: "true",- },- })- if err != nil {- return "", errors.WrapIf(err, "failed to create trivy cache volume")- }-- return trivyCacheVolumeName, nil-}--// runTrivyScan executes Trivy scan on an image-func (s *VulnerabilityService) getTrivyConfigFiles() (configContent, ignoreContent string) {- if s.settingsService == nil {- return "", ""- }-- cfg := s.settingsService.GetSettingsConfig()- if cfg == nil {- return "", ""- }-- return cfg.TrivyConfig.Value, cfg.TrivyIgnore.Value-}--// getTrivyServerConfig returns the remote Trivy server URL and optional token when-// client/server mode is enabled, or empty strings for standalone (local DB) scanning.-func (s *VulnerabilityService) getTrivyServerConfig() (serverURL, token string) {- if s.settingsService == nil {- return "", ""- }-- cfg := s.settingsService.GetSettingsConfig()- if cfg == nil || !cfg.TrivyServerEnabled.IsTrue() {- return "", ""- }-- return strings.TrimSpace(cfg.TrivyServerUrl.Value), strings.TrimSpace(cfg.TrivyServerToken.Value)-}--// getTrivyIgnoreUnfixed reports whether scans should only return vulnerabilities-// that have a known fix (--ignore-unfixed).-func (s *VulnerabilityService) getTrivyIgnoreUnfixed() bool {- if s.settingsService == nil {- return false- }-- cfg := s.settingsService.GetSettingsConfig()- return cfg != nil && cfg.TrivyIgnoreUnfixed.IsTrue()-}--func (s *VulnerabilityService) createTrivyConfigTempFile(ctx context.Context, content string) mo.Option[string] {- configFile, err := os.CreateTemp("", "trivy-config-*.yaml")- if err != nil {- slog.WarnContext(ctx, "failed to create trivy config temp file", "error", err)- return mo.None[string]()- }- if _, err := configFile.WriteString(content); err != nil {- slog.WarnContext(ctx, "failed to write trivy config", "error", err)- _ = configFile.Close()- _ = os.Remove(configFile.Name())- return mo.None[string]()- }- _ = configFile.Close()- return mo.Some(configFile.Name())-}--func (s *VulnerabilityService) createTrivyIgnoreTempFile(ctx context.Context, content string) mo.Option[string] {- ignoreFile, err := os.CreateTemp("", "trivy-ignore-*")- if err != nil {- slog.WarnContext(ctx, "failed to create trivy ignore temp file", "error", err)- return mo.None[string]()- }- if _, err := ignoreFile.WriteString(content); err != nil {- slog.WarnContext(ctx, "failed to write trivy ignore", "error", err)- _ = ignoreFile.Close()- _ = os.Remove(ignoreFile.Name())- return mo.None[string]()- }- _ = ignoreFile.Close()- return mo.Some(ignoreFile.Name())-}--func cleanupTempFiles(ctx context.Context, tempFiles []string) {- for _, f := range tempFiles {- if err := os.Remove(f); err != nil {- slog.WarnContext(ctx, "failed to remove trivy temp file", "path", f, "error", err)- }- }-}--// trivyScanCacheBackendArgsForArchInternal returns extra Trivy flags that switch-// the scan cache to Trivy's documented in-memory backend on 32-bit architectures.-func (s *VulnerabilityService) buildTrivyCommandArgs(- ctx context.Context,- imageName string,- configContent string,- ignoreContent string,- cacheDir string,- outputPath string,-) ([]string, []string) {- serverURL, _ := s.getTrivyServerConfig()- cmdArgs := []string{"image", "--format", "json", "--quiet", "--timeout", s.getTrivyScanTimeoutArg()}- cmdArgs = append(cmdArgs, vuln.ScanSourceArgs(serverURL)...)- cmdArgs = append(cmdArgs, vuln.IgnoreUnfixedArgs(s.getTrivyIgnoreUnfixed())...)- var tempFiles []string-- if strings.TrimSpace(configContent) != "" {- if tempFile, ok := s.createTrivyConfigTempFile(ctx, configContent).Get(); ok {- tempFiles = append(tempFiles, tempFile)- cmdArgs = append(cmdArgs, "--config", "/tmp/trivy-config.yaml")- }- }-- if strings.TrimSpace(ignoreContent) != "" {- if tempFile, ok := s.createTrivyIgnoreTempFile(ctx, ignoreContent).Get(); ok {- tempFiles = append(tempFiles, tempFile)- cmdArgs = append(cmdArgs, "--ignorefile", "/tmp/trivy-ignore")- }- }-- if strings.TrimSpace(cacheDir) != "" {- cmdArgs = append(cmdArgs, "--cache-dir", cacheDir)- }-- if strings.TrimSpace(outputPath) != "" {- cmdArgs = append(cmdArgs, "--output", outputPath)- }-- cmdArgs = append(cmdArgs, imageName)-- return cmdArgs, tempFiles-}--func buildTrivyBatchHostConfigInternal(- cacheVolume string,- runtimeOptions trivyRuntimeOptions,- securityOpts []string,- privileged bool,-) *containertypes.HostConfig {- mounts := append([]mounttypes.Mount{}, runtimeOptions.Mounts...)- if strings.TrimSpace(cacheVolume) != "" {- mounts = append(mounts, mounttypes.Mount{- Type: mounttypes.TypeVolume,- Source: cacheVolume,- Target: trivyCacheMountTarget,- })- }-- hostConfig := &containertypes.HostConfig{- NetworkMode: containertypes.NetworkMode(selectDefaultTrivyNetworkModeInternal(runtimeOptions.NetworkMode)),- Mounts: mounts,- }-- applyTrivyRuntimeSecurityInternal(hostConfig, securityOpts, privileged)- return hostConfig-}--func buildTrivyHostConfig(- cacheVolume string,- tempFiles []string,- resources containertypes.Resources,- cpuSet string,- applyLimits bool,- runtimeOptions trivyRuntimeOptions,- securityOpts []string,- privileged bool,-) *containertypes.HostConfig {- mounts := append([]mounttypes.Mount{}, runtimeOptions.Mounts...)- if strings.TrimSpace(cacheVolume) != "" {- mounts = append(mounts, mounttypes.Mount{- Type: mounttypes.TypeVolume,- Source: cacheVolume,- Target: trivyCacheMountTarget,- })- }-- hostConfig := &containertypes.HostConfig{- NetworkMode: containertypes.NetworkMode(selectDefaultTrivyNetworkModeInternal(runtimeOptions.NetworkMode)),- // Keep single-scan containers until explicit cleanup so we can reliably- // wait for completion and collect logs across Docker variants.- AutoRemove: false,- Mounts: mounts,- }-- applyTrivyRuntimeSecurityInternal(hostConfig, securityOpts, privileged)- applyTrivyContainerResourcesInternal(hostConfig, resources, cpuSet, applyLimits)-- addTrivyTempFileMounts(hostConfig, tempFiles)- return hostConfig-}--func selectDefaultTrivyNetworkModeInternal(networkMode string) string {- networkMode = strings.TrimSpace(networkMode)- if networkMode == "" {- return DefaultTrivyNetworkMode- }- return networkMode-}--func applyTrivyRuntimeSecurityInternal(hostConfig *containertypes.HostConfig, securityOpts []string, privileged bool) {- if hostConfig == nil {- return- }-- if len(securityOpts) > 0 {- hostConfig.SecurityOpt = append([]string(nil), securityOpts...)- }- hostConfig.Privileged = privileged-}--func applyTrivyContainerResourcesInternal(hostConfig *containertypes.HostConfig, resources containertypes.Resources, cpuSet string, applyLimits bool) {- if hostConfig == nil || !applyLimits {- return- }-- if cpuSet != "" {- resources.NanoCPUs = 0- resources.CpusetCpus = cpuSet- }-- hostConfig.Resources = resources-}--func addTrivyTempFileMounts(hostConfig *containertypes.HostConfig, tempFiles []string) {- for _, tempFile := range tempFiles {- switch {- case strings.Contains(tempFile, "trivy-config"):- hostConfig.Mounts = append(hostConfig.Mounts, mounttypes.Mount{- Type: mounttypes.TypeBind,- Source: tempFile,- Target: "/tmp/trivy-config.yaml",- ReadOnly: true,- })- case strings.Contains(tempFile, "trivy-ignore"):- hostConfig.Mounts = append(hostConfig.Mounts, mounttypes.Mount{- Type: mounttypes.TypeBind,- Source: tempFile,- Target: "/tmp/trivy-ignore",- ReadOnly: true,- })- }- }-}--func createTrivyLogTempFileInternal(prefix string) (*os.File, error) {- // Try the default system temp dir first.- file, primaryErr := os.CreateTemp("", prefix)- if primaryErr == nil {- return file, nil- }- // /tmp is not writable (e.g. hardened container). Fall back to a- // per-user cache directory so Trivy scans can still run.- cacheDir, err := os.UserCacheDir()- if err != nil {- return nil, errors.WrapIff(err, "failed to create trivy temp file (primary: %s) and user cache dir unavailable", primaryErr.Error())- }- fallbackDir := filepath.Join(cacheDir, "arcane", "trivy-tmp")- if err := os.MkdirAll(fallbackDir, 0o700); err != nil {- return nil, errors.WrapIff(err, "failed to create trivy fallback temp dir %s (primary: %s)", fallbackDir, primaryErr.Error())- }- fallbackFile, err := os.CreateTemp(fallbackDir, prefix)- if err != nil {- return nil, errors.WrapIff(err, "failed to create trivy temp file in fallback dir %s (primary: %s)", fallbackDir, primaryErr.Error())- }- return fallbackFile, nil-}--func cleanupTrivyLogTempFilesInternal(ctx context.Context, files ...*os.File) {- for _, file := range files {- if file == nil {- continue- }-- path := file.Name()- if err := file.Close(); err != nil {- slog.WarnContext(ctx, "failed to close trivy temp file", "path", path, "error", err)- }-- if path == "" {- continue- }-- if err := os.Remove(path); err != nil {- slog.WarnContext(ctx, "failed to remove trivy temp file", "path", path, "error", err)- }- }-}--func startDockerLogCopyInternal(stream io.Reader, stdoutWriter, stderrWriter io.Writer) <-chan error {- logDone := make(chan error, 1)- go func() {- _, err := stdcopy.StdCopy(stdoutWriter, stderrWriter, stream)- logDone <- err- }()- return logDone-}--func waitForDockerLogCopyInternal(logDone <-chan error) error {- err := <-logDone- if err == nil || isExpectedDockerStreamEndErrorInternal(err) {- return nil- }- return err-}--func trivyMountSummariesInternal(hostConfig *containertypes.HostConfig) []string {- if hostConfig == nil || len(hostConfig.Mounts) == 0 {- return nil- }-- summaries := make([]string, 0, len(hostConfig.Mounts))- for _, mount := range hostConfig.Mounts {- source := mount.Source- if source == "" {- source = string(mount.Type)- }- summaries = append(summaries, fmt.Sprintf("%s:%s", source, mount.Target))- }-- return summaries-}--func logTrivyContainerStartupRequestInternal(ctx context.Context, scope string, config *containertypes.Config, hostConfig *containertypes.HostConfig) {- if config == nil {- return- }-- resources := containertypes.Resources{}- autoRemove := false- var mounts []string- networkMode := ""- var securityOpts []string- privileged := false- if hostConfig != nil {- resources = hostConfig.Resources- autoRemove = hostConfig.AutoRemove- mounts = trivyMountSummariesInternal(hostConfig)- networkMode = string(hostConfig.NetworkMode)- securityOpts = append([]string(nil), hostConfig.SecurityOpt...)- privileged = hostConfig.Privileged- }-- slog.DebugContext(ctx,- "preparing trivy container startup",- "scope", scope,- "image", config.Image,- "cmd", config.Cmd,- "entrypoint", config.Entrypoint,- "networkMode", networkMode,- "securityOpts", securityOpts,- "privileged", privileged,- "autoRemove", autoRemove,- "mounts", mounts,- "nanoCPUs", resources.NanoCPUs,- "cpuset", resources.CpusetCpus,- "memory", resources.Memory,- "memorySwap", resources.MemorySwap,- )-}--func truncateTrivyLogOutputInternal(content string, maxBytes int64) string {- content = strings.TrimSpace(content)- if content == "" || maxBytes <= 0 {- return content- }-- raw := []byte(content)- if int64(len(raw)) <= maxBytes {- return content- }-- return strings.TrimSpace(string(raw[:maxBytes])) + " ...<truncated>"-}--func readTrivyStartupLogsInternal(ctx context.Context, dockerClient *client.Client, containerID string) (string, string) {- if dockerClient == nil || containerID == "" {- return "", ""- }-- logsCtx, logsCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))- defer logsCancel()-- logs, err := dockerClient.ContainerLogs(logsCtx, containerID, client.ContainerLogsOptions{- ShowStdout: true,- ShowStderr: true,- Tail: "200",- })- if err != nil {- slog.DebugContext(ctx, "failed to read trivy startup logs", "containerId", containerID, "error", err)- return "", ""- }- defer func() {- if closeErr := logs.Close(); closeErr != nil {- slog.WarnContext(ctx, "failed to close trivy startup logs stream", "containerId", containerID, "error", closeErr)- }- }()-- var stdoutBuf bytes.Buffer- var stderrBuf bytes.Buffer- if _, err := stdcopy.StdCopy(&stdoutBuf, &stderrBuf, logs); err != nil && !isExpectedDockerStreamEndErrorInternal(err) {- slog.DebugContext(ctx, "failed to decode trivy startup logs", "containerId", containerID, "error", err)- }-- stdoutLog := truncateTrivyLogOutputInternal(stdoutBuf.String(), trivyErrorExcerptSize)- stderrLog := truncateTrivyLogOutputInternal(stderrBuf.String(), trivyErrorExcerptSize)- return stdoutLog, stderrLog-}--func removeDockerContainerInternal(ctx context.Context, dockerClient *client.Client, containerID, warningMessage string) {- if dockerClient == nil || containerID == "" {- return- }-- cleanupCtx := context.WithoutCancel(ctx)- cleanupCtx, cleanupCancel := context.WithTimeout(cleanupCtx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))- defer cleanupCancel()-- if _, err := dockerClient.ContainerRemove(cleanupCtx, containerID, client.ContainerRemoveOptions{Force: true}); err != nil && !cerrdefs.IsNotFound(err) {- slog.WarnContext(cleanupCtx, warningMessage, "containerId", containerID, "error", err)- }-}--func copyRegistryConfigToContainerInternal(ctx context.Context, dockerClient *client.Client, containerID string, configJSON []byte) error {- if dockerClient == nil || containerID == "" || len(configJSON) == 0 {- return nil- }-- var buf bytes.Buffer- tw := tar.NewWriter(&buf)-- if err := tw.WriteHeader(&tar.Header{Name: "arcane-registry-auth/", Mode: 0o755, Typeflag: tar.TypeDir}); err != nil {- return err- }- if err := tw.WriteHeader(&tar.Header{Name: trivyRegistryConfigTarName, Mode: 0o644, Size: int64(len(configJSON))}); err != nil {- return err- }- if _, err := tw.Write(configJSON); err != nil {- return err- }- if err := tw.Close(); err != nil {- return err- }-- copyCtx, copyCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))- defer copyCancel()-- _, err := dockerClient.CopyToContainer(copyCtx, containerID, client.CopyToContainerOptions{- DestinationPath: trivyRegistryConfigCopyDest,- Content: &buf,- })- return err-}--func (s *VulnerabilityService) createAndStartTrivyContainerInternal(- ctx context.Context,- dockerClient *client.Client,- scope string,- config *containertypes.Config,- hostConfig *containertypes.HostConfig,- registryConfigJSON []byte,-) (string, error) {- logTrivyContainerStartupRequestInternal(ctx, scope, config, hostConfig)-- createCtx, createCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))- defer createCancel()-- resp, err := dockerClient.ContainerCreate(createCtx, client.ContainerCreateOptions{- Config: config,- HostConfig: hostConfig,- })- if err != nil {- slog.WarnContext(ctx, "failed to create trivy container", "scope", scope, "error", err)- return "", errors.WrapIf(err, "failed to create container")- }-- if len(registryConfigJSON) > 0 {- if err := copyRegistryConfigToContainerInternal(ctx, dockerClient, resp.ID, registryConfigJSON); err != nil {- // DOCKER_CONFIG may still point at the target directory. When the- // config file is absent, Trivy falls back to anonymous registry access.- slog.WarnContext(ctx,- "failed to inject registry credentials into trivy container; continuing anonymously",- "scope", scope,- "containerId", resp.ID,- "error", err,- )- }- }-- startCtx, startCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))- defer startCancel()-- if _, err := dockerClient.ContainerStart(startCtx, resp.ID, client.ContainerStartOptions{}); err != nil {- stdoutLog, stderrLog := readTrivyStartupLogsInternal(ctx, dockerClient, resp.ID)- slog.WarnContext(ctx,- "failed to start trivy container",- "scope", scope,- "containerId", resp.ID,- "error", err,- "stdout", stdoutLog,- "stderr", stderrLog,- )-- removeDockerContainerInternal(ctx, dockerClient, resp.ID, "failed to cleanup trivy container after start failure")- return "", errors.WrapIf(err, "failed to start container")- }-- return resp.ID, nil-}--func parseTrivyCPULimitInternal(raw string, fallback float64) float64 {- raw = strings.TrimSpace(raw)- if raw == "" {- return fallback- }-- limit, err := strconv.ParseFloat(raw, 64)- if err != nil {- return fallback- }-- if limit < 0 {- return 0- }-- return limit-}--func parseTrivyMemoryLimitMBInternal(raw string, fallback int64) int64 {- raw = strings.TrimSpace(raw)- if raw == "" {- return fallback- }-- limit, err := strconv.ParseInt(raw, 10, 64)- if err != nil {- return fallback- }-- if limit < 0 {- return 0- }-- return limit-}--func trivyContainerWaitTimeoutInternal(trivyTimeoutArg string) time.Duration {- trivyTimeoutArg = strings.TrimSpace(trivyTimeoutArg)- if trivyTimeoutArg == "" {- return timeouts.DefaultTrivyScan + timeouts.DefaultDockerAPI- }-- timeout, err := time.ParseDuration(trivyTimeoutArg)- if err != nil || timeout <= 0 {- return timeouts.DefaultTrivyScan + timeouts.DefaultDockerAPI- }-- return timeout + timeouts.DefaultDockerAPI-}--func isSynologyDockerHostInternal(operatingSystem string) bool {- // Docker info may return a multi-line OS string on some Synology models.- // e.g. DS925+ (Kernel 5.x) returns "Synology NAS\n (containerized)".- // Check each line independently so a match on any line is sufficient.- for line := range strings.SplitSeq(operatingSystem, "\n") {- if strings.Contains(strings.ToLower(strings.TrimSpace(line)), "synology") {- return true- }- }- return false-}--func buildTrivyCPUSetInternal(nanoCPUs int64, hostCPUs int) string {- if nanoCPUs <= 0 {- return ""- }-- if hostCPUs <= 0 {- hostCPUs = 1- }-- requestedCPUs := min(max(int(math.Floor(float64(nanoCPUs)/float64(trivyNanoCPUsPerCore))), 1), hostCPUs)-- if requestedCPUs == 1 {- return "0"- }-- return fmt.Sprintf("0-%d", requestedCPUs-1)-}--func (s *VulnerabilityService) getTrivyContainerResourcesInternal() mo.Option[containertypes.Resources] {- var cfg *models.Settings- if s.settingsService != nil {- cfg = s.settingsService.GetSettingsConfig()- }-- return buildTrivyContainerResourcesInternal(cfg)-}--func (s *VulnerabilityService) getTrivyContainerResourceOptionsInternal(ctx context.Context, dockerClient *client.Client) (containertypes.Resources, string, bool) {- resources, applyLimits := s.getTrivyContainerResourcesInternal().Get()- if !applyLimits || dockerClient == nil {- return resources, "", applyLimits- }-- infoCtx, infoCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))- defer infoCancel()-- slog.DebugContext(ctx, "inspecting docker host for trivy resource strategy")-- info, err := dockerClient.Info(infoCtx, client.InfoOptions{})- if err != nil {- if errors.Is(infoCtx.Err(), context.DeadlineExceeded) || errors.Is(err, context.DeadlineExceeded) {- slog.WarnContext(ctx,- "timed out inspecting docker info for trivy resource strategy; falling back to default limits",- "timeout", timeouts.DefaultDockerAPI.String(),- "error", err,- )- } else {- slog.DebugContext(ctx, "failed to inspect docker info for trivy resource strategy", "error", err)- }- return resources, "", applyLimits- }-- slog.DebugContext(ctx,- "resolved docker host info for trivy resource strategy",- "operatingSystem", info.Info.OperatingSystem,- "ncpu", info.Info.NCPU,- )-- if !isSynologyDockerHostInternal(info.Info.OperatingSystem) {- return resources, "", applyLimits- }-- cpuSet := buildTrivyCPUSetInternal(resources.NanoCPUs, info.Info.NCPU)- resources.NanoCPUs = 0-- if cpuSet != "" {- slog.InfoContext(ctx,- "detected Synology Docker host; applying Trivy CPU limit via cpuset",- "operatingSystem", info.Info.OperatingSystem,- "cpuset", cpuSet,- )- } else {- slog.InfoContext(ctx,- "detected Synology Docker host; skipping Trivy NanoCPUs limit",- "operatingSystem", info.Info.OperatingSystem,- )- }-- return resources, cpuSet, applyLimits-}--func isExpectedDockerStreamEndErrorInternal(err error) bool {- if err == nil {- return false- }-- if errors.Is(err, io.EOF) || errors.Is(err, net.ErrClosed) || errors.Is(err, context.Canceled) {- return true- }-- errMsg := strings.ToLower(err.Error())- return strings.Contains(errMsg, "use of closed network connection") ||- strings.Contains(errMsg, "context canceled") ||- strings.Contains(errMsg, "broken pipe") ||- strings.Contains(errMsg, "connection reset by peer")-}--func tempFileSizeInternal(file *os.File) int64 {- if file == nil {- return 0- }-- stat, err := file.Stat()- if err != nil {- return 0- }-- return stat.Size()-}--func readTempFileExcerptInternal(file *os.File) string {- if file == nil {- return ""- }-- if _, err := file.Seek(0, io.SeekStart); err != nil {- return ""- }-- content, err := io.ReadAll(io.LimitReader(file, trivyErrorExcerptSize))- if err != nil {- return ""- }-- return strings.TrimSpace(string(content))-}--func readTrivyOutputFromContainerFileInternal(ctx context.Context, dockerClient *client.Client, containerID, outputPath string) ([]byte, error) {- if dockerClient == nil {- return nil, errors.New("docker client is nil")- }- if containerID == "" {- return nil, errors.New("container id is empty")- }- if strings.TrimSpace(outputPath) == "" {- return nil, errors.New("trivy output path is empty")- }-- copyCtx, copyCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))- defer copyCancel()-- copyResult, err := dockerClient.CopyFromContainer(copyCtx, containerID, client.CopyFromContainerOptions{SourcePath: outputPath})- if err != nil {- return nil, errors.WrapIf(err, "copy trivy output file")- }- archiveReader := copyResult.Content- defer func() {- if closeErr := archiveReader.Close(); closeErr != nil && !isExpectedDockerStreamEndErrorInternal(closeErr) {- slog.WarnContext(ctx,- "failed to close trivy output archive stream",- "containerId", containerID,- "outputPath", outputPath,- "error", closeErr,- )- }- }()-- rawOutput, err := extractFileFromContainerArchiveInternal(archiveReader)- if err != nil {- return nil, errors.WrapIf(err, "extract trivy output file")- }-- if len(bytes.TrimSpace(rawOutput)) == 0 {- return nil, io.EOF- }-- return rawOutput, nil-}--func cleanupTrivyOutputFileInContainerInternal(ctx context.Context, dockerClient *client.Client, containerID, outputPath string) {- if dockerClient == nil || containerID == "" || strings.TrimSpace(outputPath) == "" {- return- }-- execCtx, execCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))- defer execCancel()-- execResp, err := dockerClient.ExecCreate(execCtx, containerID, client.ExecCreateOptions{- Cmd: []string{"rm", "-f", outputPath},- })- if err != nil {- slog.DebugContext(ctx,- "failed to create cleanup exec for trivy output file",- "containerId", containerID,- "outputPath", outputPath,- "error", err,- )- return- }-- if _, err := dockerClient.ExecStart(execCtx, execResp.ID, client.ExecStartOptions{}); err != nil {- slog.DebugContext(ctx,- "failed to start cleanup exec for trivy output file",- "containerId", containerID,- "outputPath", outputPath,- "error", err,- )- return- }-}--func extractFileFromContainerArchiveInternal(archiveReader io.Reader) ([]byte, error) {- if archiveReader == nil {- return nil, errors.New("container archive reader is nil")- }-- tarReader := tar.NewReader(archiveReader)- for {- header, err := tarReader.Next()- if errors.Is(err, io.EOF) {- break- }- if err != nil {- return nil, errors.WrapIf(err, "read tar header")- }-- if header == nil || header.Typeflag == tar.TypeDir {- continue- }-- if header.Typeflag != tar.TypeReg {- continue- }-- data, err := io.ReadAll(tarReader)- if err != nil {- return nil, errors.WrapIf(err, "read tar file content")- }-- return data, nil- }-- return nil, errors.New("container archive did not include a file")-}--func decodeTrivyReportFromFileInternal(file *os.File) (*vulnerability.TrivyReport, error) {- if file == nil {- return nil, errors.New("trivy output file is nil")- }-- if _, err := file.Seek(0, io.SeekStart); err != nil {- return nil, errors.WrapIf(err, "failed to seek trivy output file")- }-- rawOutput, err := io.ReadAll(file)- if err != nil {- return nil, errors.WrapIf(err, "failed to read trivy output file")- }-- return decodeTrivyReportFromBytesInternal(rawOutput)-}--func decodeTrivyReportFromBytesInternal(rawOutput []byte) (*vulnerability.TrivyReport, error) {- trimmedOutput := bytes.TrimSpace(rawOutput)- if len(trimmedOutput) == 0 {- return nil, io.EOF- }-- report, strictErr := decodeSingleTrivyReportInternal(trimmedOutput)- if strictErr == nil {- return report, nil- }-- // Trivy output can occasionally include non-JSON log lines or additional JSON- // objects around the report depending on Docker variant / stream behavior.- // Extract top-level JSON objects and return the first valid Trivy report.- jsonCandidates := extractTopLevelJSONObjectCandidatesInternal(trimmedOutput)- if len(jsonCandidates) == 0 {- return nil, strictErr- }-- lastErr := strictErr- for _, candidate := range jsonCandidates {- report, err := decodeSingleTrivyReportInternal(candidate)- if err == nil {- return report, nil- }- lastErr = err- }-- return nil, lastErr-}--func decodeSingleTrivyReportInternal(rawOutput []byte) (*vulnerability.TrivyReport, error) {- decoder := jsontext.NewDecoder(bytes.NewReader(rawOutput))-- var report vulnerability.TrivyReport- if err := json.UnmarshalDecode(decoder, &report); err != nil {- return nil, err- }-- if !isLikelyTrivyReportInternal(&report) {- return nil, errors.New("decoded JSON object is not a trivy report")- }-- if hasTrailingNonWhitespaceInternal(rawOutput, decoder.InputOffset()) {- return nil, errors.New("trivy output contains trailing data")- }-- return &report, nil-}--func hasTrailingNonWhitespaceInternal(rawOutput []byte, offset int64) bool {- if offset < 0 {- return len(bytes.TrimSpace(rawOutput)) > 0- }-- if offset >= int64(len(rawOutput)) {- return false- }-- return len(bytes.TrimSpace(rawOutput[offset:])) > 0-}--func isLikelyTrivyReportInternal(report *vulnerability.TrivyReport) bool {- if report == nil {- return false- }-- return report.SchemaVersion > 0 ||- strings.TrimSpace(report.ArtifactName) != "" ||- strings.TrimSpace(report.ArtifactType) != "" ||- len(report.Results) > 0-}--func extractTopLevelJSONObjectCandidatesInternal(rawOutput []byte) [][]byte {- if len(rawOutput) == 0 {- return nil- }-- candidates := make([][]byte, 0, 4)- start := -1- depth := 0- inString := false- escaped := false-- for i, b := range rawOutput {- if start == -1 {- if b == '{' {- start = i- depth = 1- inString = false- escaped = false- }- continue- }-- if inString {- if escaped {- escaped = false- continue- }-- switch b {- case '\\':- escaped = true- case '"':- inString = false- }- continue- }-- switch b {- case '"':- inString = true- case '{':- depth++- case '}':- depth--- if depth == 0 {- candidate := bytes.TrimSpace(rawOutput[start : i+1])- if len(candidate) > 0 {- candidates = append(candidates, candidate)- }- start = -1- }- }- }-- return candidates-}--func formatTrivyExitErrorInternal(exitCode int64, errMsg, imageName string) error {- errMsg = strings.TrimSpace(errMsg)-- if exitCode == 137 {- if errMsg == "" || errMsg == fmt.Sprintf("exit status %d", exitCode) {- return errors.Errorf("trivy scan failed: process killed with exit status 137 (likely out of memory while scanning %s)", imageName)- }- return errors.Errorf("trivy scan failed: %s (process killed with exit status 137; likely out of memory)", errMsg)- }-- if strings.Contains(strings.ToLower(errMsg), "deadline exceeded") {- return errors.Errorf("trivy scan timed out for %s (increase TRIVY_SCAN_TIMEOUT or trivyScanTimeout setting)", imageName)- }-- return errors.Errorf("trivy scan failed: %s", errMsg)-}--func buildTrivyContainerResourcesInternal(cfg *models.Settings) mo.Option[containertypes.Resources] {- limitsEnabled := true- cpuLimitCores := defaultTrivyCPULimitCores- memoryLimitMB := defaultTrivyMemoryLimitMB-- if cfg != nil {- limitsEnabled = cfg.TrivyResourceLimitsEnabled.IsTrue()- cpuLimitCores = parseTrivyCPULimitInternal(cfg.TrivyCpuLimit.Value, defaultTrivyCPULimitCores)- memoryLimitMB = parseTrivyMemoryLimitMBInternal(cfg.TrivyMemoryLimitMb.Value, defaultTrivyMemoryLimitMB)- }-- if !limitsEnabled {- return mo.None[containertypes.Resources]()- }-- resources := containertypes.Resources{}- if cpuLimitCores > 0 {- resources.NanoCPUs = int64(cpuLimitCores * float64(trivyNanoCPUsPerCore))- }-- if memoryLimitMB > 0 {- memoryBytes := memoryLimitMB * trivyBytesPerMB- resources.Memory = memoryBytes- resources.MemorySwap = memoryBytes- }-- return mo.Some(resources)-}--func (s *VulnerabilityService) runTrivyContainer(- ctx context.Context,- dockerClient *client.Client,- imageID string,- config *containertypes.Config,- hostConfig *containertypes.HostConfig,- registryConfigJSON []byte,-) (*os.File, *os.File, int64, int64, string, error) {- stdoutFile, err := createTrivyLogTempFileInternal("trivy-stdout-*")- if err != nil {- return nil, nil, 0, 0, "", errors.WrapIf(err, "failed to create trivy stdout temp file")- }-- stderrFile, err := createTrivyLogTempFileInternal("trivy-stderr-*")- if err != nil {- cleanupTrivyLogTempFilesInternal(ctx, stdoutFile)- return nil, nil, 0, 0, "", errors.WrapIf(err, "failed to create trivy stderr temp file")- }-- success := false- containerID := ""- defer func() {- if !success {- removeDockerContainerInternal(ctx, dockerClient, containerID, "failed to cleanup trivy scan container")- cleanupTrivyLogTempFilesInternal(ctx, stdoutFile, stderrFile)- }- }()-- containerID, err = s.createAndStartTrivyContainerInternal(ctx, dockerClient, "single-scan", config, hostConfig, registryConfigJSON)- if err != nil {- return nil, nil, 0, 0, "", errors.WrapIf(err, "failed to prepare trivy scan container")- }-- slog.DebugContext(ctx, "started trivy scan container", "containerId", containerID)- s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseScanningImage)-- logsCtx, logsCancel := context.WithCancel(ctx)- logs, err := dockerClient.ContainerLogs(logsCtx, containerID, client.ContainerLogsOptions{- ShowStdout: true,- ShowStderr: true,- Follow: true,- })- if err != nil {- logsCancel()- return nil, nil, 0, 0, "", errors.WrapIf(err, "failed to stream trivy logs")- }-- var closeLogsOnce sync.Once- closeLogs := func() {- closeLogsOnce.Do(func() {- logsCancel()- if closeErr := logs.Close(); closeErr != nil && !isExpectedDockerStreamEndErrorInternal(closeErr) {- slog.WarnContext(ctx, "failed to close trivy scan logs stream", "error", closeErr)- }- })- }- defer closeLogs()-- logDone := startDockerLogCopyInternal(logs, stdoutFile, stderrFile)-- startTime := time.Now()- waitTimeout := trivyContainerWaitTimeoutInternal(s.getTrivyScanTimeoutArg())- waitCtx, waitCancel := context.WithTimeout(ctx, waitTimeout)- defer waitCancel()-- statusCode, err := s.waitForTrivyContainer(waitCtx, dockerClient, containerID)- if err != nil {- closeLogs()- if logErr := waitForDockerLogCopyInternal(logDone); logErr != nil {- slog.DebugContext(ctx, "trivy log stream ended with error while handling wait failure", "error", logErr)- }- return nil, nil, 0, 0, "", err- }-- // Wait for the log copy goroutine to complete naturally. On some Docker- // variants (e.g. Synology with Docker API 1.43), the daemon may not have- // fully flushed container stdout before ContainerWait returns. Cancelling- // the stream immediately after ContainerWait can truncate the JSON output- // and cause "unexpected EOF" or empty-output decode errors.- //- // Strategy: wait up to logDrainDeadline for the stream to close on its own,- // then force-close it (for Docker variants that keep the stream open after- // container exit, e.g. older Docker builds that never send EOF).- const logDrainDeadline = 30 * time.Second- logDrainTimer := time.NewTimer(logDrainDeadline)- defer logDrainTimer.Stop()-- select {- case logErr := <-logDone:- closeLogs() // clean up context / connection- if logErr != nil && !isExpectedDockerStreamEndErrorInternal(logErr) {- return nil, nil, 0, 0, "", errors.WrapIf(logErr, "failed to read trivy logs")- }- case <-logDrainTimer.C:- // Stream did not close on its own; force-close so the goroutine exits.- slog.DebugContext(ctx, "trivy log stream did not close after container exit; force-closing", "containerId", containerID)- closeLogs()- if logErr := waitForDockerLogCopyInternal(logDone); logErr != nil {- return nil, nil, 0, 0, "", errors.WrapIf(logErr, "failed to read trivy logs")- }- }-- duration := time.Since(startTime).Milliseconds()- success = true- return stdoutFile, stderrFile, duration, statusCode, containerID, nil-}--func (s *VulnerabilityService) waitForTrivyContainer(- ctx context.Context,- dockerClient *client.Client,- containerID string,-) (int64, error) {- waitResult := dockerClient.ContainerWait(ctx, containerID, client.ContainerWaitOptions{Condition: containertypes.WaitConditionNotRunning})-- statusCode, err := awaitContainerWaitResponseInternal(ctx, waitResult.Result, waitResult.Error)- if err != nil {- if errors.Is(err, context.DeadlineExceeded) {- return 0, errors.WrapIf(err, "trivy container wait timed out")- }- if errors.Is(err, context.Canceled) {- return 0, errors.WrapIf(err, "scan cancelled")- }- return 0, errors.WrapIf(err, "trivy container wait failed")- }-- return statusCode, nil-}--func awaitContainerWaitResponseInternal(- ctx context.Context,- statusCh <-chan containertypes.WaitResponse,- errCh <-chan error,-) (int64, error) {- for statusCh != nil || errCh != nil {- select {- case waitResp, ok := <-statusCh:- if !ok {- statusCh = nil- continue- }- return waitResp.StatusCode, nil- case err, ok := <-errCh:- if !ok {- errCh = nil- continue- }- if err != nil {- return 0, err- }- case <-ctx.Done():- return 0, ctx.Err()- }- }-- return 0, errors.New("trivy container wait ended without status")-}--func (s *VulnerabilityService) runTrivyScan(ctx context.Context, trivyImage string, imageName string, imageID string) (*vulnerability.ScanResult, error) {- slog.DebugContext(ctx,- "running trivy scan",- "imageId", imageID,- "imageName", imageName,- "trivyImage", trivyImage,- )-- slotID, releaseSlot, err := s.acquireTrivyScanSlotInternal(ctx)- if err != nil {- return nil, err- }- defer releaseSlot()-- // Use per-image locking to allow concurrent scans of different images- defer s.scanLocks.Lock(imageID)()-- dockerClient, err := s.dockerService.GetClient(ctx)- if err != nil {- return nil, errors.WrapIf(err, "failed to connect to Docker")- }-- serverURL, serverToken := s.getTrivyServerConfig()- var cacheVolume string- if serverURL == "" {- cacheVolume, err = s.ensureTrivyCacheVolumeInternal(ctx)- if err != nil {- return nil, err- }- }-- // Get Trivy config files if they exist- configContent, ignoreContent := s.getTrivyConfigFiles()-- var cacheDir string- if serverURL == "" {- cacheDir = s.getTrivySingleScanCacheDirForSlotInternal(slotID)- }- if cacheDir != "" {- slog.DebugContext(ctx,- "using slot-based trivy cache dir for scan",- "slotId", slotID,- "cacheDir", cacheDir,- "concurrentScanContainers", s.getTrivyConcurrentScanContainersInternal(),- )- }-- outputPath := newTrivyOutputPathInternal()-- cmdArgs, tempFiles := s.buildTrivyCommandArgs(ctx, imageName, configContent, ignoreContent, cacheDir, outputPath)- defer cleanupTempFiles(ctx, tempFiles)-- runtimeOptions, err := s.resolveTrivyRuntimeOptionsInternal(ctx, dockerClient)- if err != nil {- return nil, err- }-- registryConfigJSON := s.buildTrivyRegistryConfigInternal(ctx)- env := append([]string(nil), runtimeOptions.ContainerEnv...)- if len(registryConfigJSON) > 0 {- env = append(env, "DOCKER_CONFIG="+trivyRegistryConfigDir)- }- if serverToken != "" {- env = append(env, vuln.ServerTokenEnv(serverToken)...)- }-- config := vuln.BuildContainerConfig(trivyImage, cmdArgs, env)- resources, cpuSet, applyLimits := s.getTrivyContainerResourceOptionsInternal(ctx, dockerClient)- runtimeSecurity := s.getTrivyRuntimeSecurityInternal().OrEmpty()- securityOpts, privileged := runtimeSecurity.securityOpts, runtimeSecurity.privileged- hostConfig := buildTrivyHostConfig(- cacheVolume,- tempFiles,- resources,- cpuSet,- applyLimits,- runtimeOptions,- securityOpts,- privileged,- )- s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseCreatingContainer)-- stdoutFile, stderrFile, duration, statusCode, containerID, err := s.runTrivyContainer(ctx, dockerClient, imageID, config, hostConfig, registryConfigJSON)- if err != nil {- return nil, err- }- defer cleanupTrivyLogTempFilesInternal(ctx, stdoutFile, stderrFile)- defer removeDockerContainerInternal(ctx, dockerClient, containerID, "failed to cleanup trivy scan container")-- if statusCode != 0 {- errMsg := readTempFileExcerptInternal(stderrFile)- if errMsg == "" {- errMsg = readTempFileExcerptInternal(stdoutFile)- }- if errMsg == "" {- errMsg = fmt.Sprintf("exit status %d", statusCode)- }- slog.WarnContext(ctx,- "trivy scan diagnostics",- "image", imageName,- "imageId", imageID,- "reason", "container non-zero exit",- "exitCode", statusCode,- "stdoutBytes", tempFileSizeInternal(stdoutFile),- "stderrBytes", tempFileSizeInternal(stderrFile),- )- return nil, formatTrivyExitErrorInternal(statusCode, errMsg, imageName)- }-- trivyOutput, outputErr := readTrivyOutputFromContainerFileInternal(ctx, dockerClient, containerID, outputPath)- if outputErr != nil {- slog.WarnContext(ctx,- "failed to read trivy output file from single-scan container; falling back to stdout parsing",- "containerId", containerID,- "outputPath", outputPath,- "error", outputErr,- )- }-- var trivyReport *vulnerability.TrivyReport- if len(trivyOutput) > 0 {- trivyReport, err = decodeTrivyReportFromBytesInternal(trivyOutput)- } else {- trivyReport, err = decodeTrivyReportFromFileInternal(stdoutFile)- }- if err != nil {- if errors.Is(err, io.EOF) {- errMsg := readTempFileExcerptInternal(stderrFile)- if errMsg == "" {- errMsg = "trivy scan produced no output"- }- slog.WarnContext(ctx,- "trivy scan diagnostics",- "image", imageName,- "imageId", imageID,- "reason", "container empty output",- "exitCode", statusCode,- "stdoutBytes", tempFileSizeInternal(stdoutFile),- "stderrBytes", tempFileSizeInternal(stderrFile),- )- return nil, errors.Errorf("trivy scan failed: %s", errMsg)- }-- slog.WarnContext(ctx,- "trivy scan diagnostics",- "image", imageName,- "imageId", imageID,- "reason", "container parse error",- "exitCode", statusCode,- "stdoutBytes", tempFileSizeInternal(stdoutFile),- "stderrBytes", tempFileSizeInternal(stderrFile),- )- return nil, errors.WrapIf(err, "failed to parse trivy output")- }-- if trivyReport == nil {- errMsg := readTempFileExcerptInternal(stderrFile)- if errMsg == "" {- errMsg = "trivy scan produced no output"- }- slog.WarnContext(ctx,- "trivy scan diagnostics",- "image", imageName,- "imageId", imageID,- "reason", "container nil report",- "exitCode", statusCode,- "stdoutBytes", tempFileSizeInternal(stdoutFile),- "stderrBytes", tempFileSizeInternal(stderrFile),- )- return nil, errors.Errorf("trivy scan failed: %s", errMsg)- }-- // Convert Trivy report to our ScanResult format- result := vulnerability.ConvertTrivyReportToScanResult(trivyReport, imageID, time.Now(), duration)- result.ScannerVersion = s.GetTrivyVersion(ctx)- totalVulnerabilities := 0- if result.Summary != nil {- totalVulnerabilities = result.Summary.Total- }-- slog.DebugContext(ctx,- "trivy scan finished",- "imageId", imageID,- "imageName", imageName,- "durationMs", duration,- "status", result.Status,- "vulnerabilities", totalVulnerabilities,- )-- return result, nil-}--func (s *VulnerabilityService) saveScanResultWithRetryInternal(- ctx context.Context,- result *vulnerability.ScanResult,-) error {- const (- maxAttempts = defaultScanSaveRetryAttempts- retryDelay = defaultScanSaveRetryDelay- )-- var lastErr error- for attempt := 1; attempt <= maxAttempts; attempt++ {- lastErr = s.saveScanResult(ctx, result)- if lastErr == nil {- if attempt > 1 {- slog.DebugContext(ctx,- "save scan result succeeded after retry",- "attempt", attempt,- "maxAttempts", maxAttempts,- "imageId", result.ImageID,- "status", result.Status,- )- }- return nil- }-- if attempt < maxAttempts {- if !isRetryableDBWriteErrorInternal(lastErr) {- break- }-- delay := min(retryDelay*time.Duration(1<<(attempt-1)), defaultScanSaveRetryMaxDelay)-- timer := time.NewTimer(delay)- select {- case <-ctx.Done():- timer.Stop()- s.saveScanResultStatusFallbackInternal(ctx, result, ctx.Err())- return ctx.Err()- case <-timer.C:- }- }- }-- // Ensure scan status does not stay stuck as scanning when full result- // persistence fails. Best-effort minimal status write.- s.saveScanResultStatusFallbackInternal(ctx, result, lastErr)-- return lastErr-}--func isRetryableDBWriteErrorInternal(err error) bool {- if err == nil {- return false- }-- errMsg := strings.ToLower(strings.TrimSpace(err.Error()))- return strings.Contains(errMsg, "database is locked") ||- strings.Contains(errMsg, "database table is locked") ||- strings.Contains(errMsg, "database is busy") ||- strings.Contains(errMsg, "resource busy") ||- strings.Contains(errMsg, "timeout")-}--func (s *VulnerabilityService) saveScanResultStatusFallbackInternal(ctx context.Context, result *vulnerability.ScanResult, originalErr error) {- if s.db == nil || result == nil || result.ImageID == "" {- return- }-- var summary *vulnerability.SeveritySummary- if result.Summary != nil {- summary = result.Summary- } else {- summary = &vulnerability.SeveritySummary{}- }-- updates := map[string]any{- "image_name": result.ImageName,- "status": string(result.Status),- "scan_time": result.ScanTime,- "duration": result.Duration,- "critical_count": summary.Critical,- "high_count": summary.High,- "medium_count": summary.Medium,- "low_count": summary.Low,- "unknown_count": summary.Unknown,- "total_count": summary.Total,- "scanner_version": result.ScannerVersion,- }-- errMsg := strings.TrimSpace(result.Error)- if errMsg == "" && originalErr != nil {- errMsg = fmt.Sprintf("failed to persist full scan result: %v", originalErr)- }- if errMsg == "" {- updates["error"] = nil- } else {- updates["error"] = errMsg- }-- if err := s.db.WithContext(ctx).- Model(&models.VulnerabilityScanRecord{}).- Where("id = ?", result.ImageID).- Updates(updates).Error; err != nil {- slog.WarnContext(ctx,- "failed fallback scan status persistence",- "imageId", result.ImageID,- "status", result.Status,- "error", err,- "originalError", originalErr,- )- return- }-- slog.WarnContext(ctx,- "persisted fallback scan status after full save failure",- "imageId", result.ImageID,- "status", result.Status,- "originalError", originalErr,- )-}--// saveScanResult saves the scan result to the database-func (s *VulnerabilityService) saveScanResult(ctx context.Context, result *vulnerability.ScanResult) error {- if s.db == nil {- return nil- }-- s.ensureSummary(result)-- // Convert vulnerabilities to JSON- var vulnJSON models.StringSlice- if len(result.Vulnerabilities) > 0 {- vulnBytes, err := json.Marshal(result.Vulnerabilities)- if err != nil {- return errors.WrapIf(err, "failed to marshal vulnerabilities")- }- vulnJSON = models.StringSlice{string(vulnBytes)}- }-- var errPtr *string- if result.Error != "" {- errPtr = &result.Error- }-- var summary *vulnerability.SeveritySummary- if result.Summary != nil {- summary = result.Summary- } else {- summary = &vulnerability.SeveritySummary{}- }-- record := models.VulnerabilityScanRecord{- ID: result.ImageID,- ImageName: result.ImageName,- Status: string(result.Status),- ScanTime: result.ScanTime,- Duration: result.Duration,- CriticalCount: summary.Critical,- HighCount: summary.High,- MediumCount: summary.Medium,- LowCount: summary.Low,- UnknownCount: summary.Unknown,- TotalCount: summary.Total,- Vulnerabilities: vulnJSON,- Error: errPtr,- ScannerVersion: result.ScannerVersion,- }-- // Upsert the record- return s.db.WithContext(ctx).Clauses(clause.OnConflict{- Columns: []clause.Column{{Name: "id"}},- UpdateAll: true,- }).Create(&record).Error-}--func (s *VulnerabilityService) fixedVulnerabilitiesForNotifications(- ctx context.Context,- result *vulnerability.ScanResult,-) []vulnerability.Vulnerability {- if result == nil || result.Status != vulnerability.ScanStatusCompleted || len(result.Vulnerabilities) == 0 {- return nil- }-- vulnerabilities := result.Vulnerabilities- filtered, err := s.filterIgnoredVulnerabilitiesForImage(ctx, result.ImageID, vulnerabilities)- if err != nil {- slog.WarnContext(ctx, "failed to filter ignored vulnerabilities for notifications", "error", err)- } else {- vulnerabilities = filtered- }-- fixable := make([]vulnerability.Vulnerability, 0, len(vulnerabilities))- for i := range vulnerabilities {- v := vulnerabilities[i]- if strings.TrimSpace(v.FixedVersion) == "" {- continue- }- fixable = append(fixable, v)- }-- return fixable-}--func (s *VulnerabilityService) filterIgnoredVulnerabilitiesForImage(- ctx context.Context,- imageID string,- vulns []vulnerability.Vulnerability,-) ([]vulnerability.Vulnerability, error) {- if s.db == nil || len(vulns) == 0 || imageID == "" {- return vulns, nil- }-- var ignores []models.VulnerabilityIgnore- if err := s.db.WithContext(ctx).Where("image_id = ?", imageID).Find(&ignores).Error; err != nil {- return nil, err- }- if len(ignores) == 0 {- return vulns, nil- }-- ignoredKeys := make(map[string]struct{}, len(ignores))- for _, ignore := range ignores {- key := vulnerabilityIgnoreKeyInternal(ignore.ImageID, ignore.VulnerabilityID, ignore.PkgName, ignore.InstalledVersion)- ignoredKeys[key] = struct{}{}- }-- filtered := make([]vulnerability.Vulnerability, 0, len(vulns))- for _, vulnEntry := range vulns {- key := vulnerabilityIgnoreKeyInternal(imageID, vulnEntry.VulnerabilityID, vulnEntry.PkgName, vulnEntry.InstalledVersion)- if _, isIgnored := ignoredKeys[key]; isIgnored {- continue- }- filtered = append(filtered, vulnEntry)- }-- return filtered, nil-}--// convertRecordToResult converts a database record to a ScanResult-func (s *VulnerabilityService) convertRecordToResult(record *models.VulnerabilityScanRecord) (*vulnerability.ScanResult, error) {- result := &vulnerability.ScanResult{- ImageID: record.ID,- ImageName: record.ImageName,- ScanTime: record.ScanTime,- Status: vulnerability.ScanStatus(record.Status),- Duration: record.Duration,- Summary: &vulnerability.SeveritySummary{- Critical: record.CriticalCount,- High: record.HighCount,- Medium: record.MediumCount,- Low: record.LowCount,- Unknown: record.UnknownCount,- Total: record.TotalCount,- },- ScannerVersion: record.ScannerVersion,- }-- if record.Error != nil {- result.Error = *record.Error- }-- // Parse vulnerabilities from JSON- if len(record.Vulnerabilities) > 0 && record.Vulnerabilities[0] != "" {- var vulns []vulnerability.Vulnerability- if err := json.Unmarshal([]byte(record.Vulnerabilities[0]), &vulns); err != nil {- slog.Warn("failed to unmarshal vulnerabilities", "error", err)- } else {- result.Vulnerabilities = vulns- }- }-- s.ensureSummary(result)- s.applyScanPhaseToResultInternal(result)-- return result, nil-}--func (s *VulnerabilityService) ensureSummary(result *vulnerability.ScanResult) {- if result == nil {- return- }-- if len(result.Vulnerabilities) == 0 {- if result.Summary == nil {- result.Summary = &vulnerability.SeveritySummary{}- }- return- }-- if result.Summary != nil && result.Summary.Total > 0 {- return- }-- result.Summary = buildSeveritySummaryFromVulnerabilitiesInternal(result.Vulnerabilities)-}--func buildSeveritySummaryFromVulnerabilitiesInternal(vulns []vulnerability.Vulnerability) *vulnerability.SeveritySummary {- summary := &vulnerability.SeveritySummary{}- for _, vulnEntry := range vulns {- switch vulnEntry.Severity {- case vulnerability.SeverityCritical:- summary.Critical++- case vulnerability.SeverityHigh:- summary.High++- case vulnerability.SeverityMedium:- summary.Medium++- case vulnerability.SeverityLow:- summary.Low++- case vulnerability.SeverityUnknown:- summary.Unknown++- default:- summary.Unknown++- }- summary.Total++- }- return summary-}--// logScanEvent logs a vulnerability scan event-func (s *VulnerabilityService) logScanEvent(ctx context.Context, envID string, imageID, imageName string, user models.User, success bool, errMsg string) {- metadata := models.JSON{- "action": "vulnerability_scan",- "imageId": imageID,- "imageName": imageName,- "success": success,- }- if errMsg != "" {- metadata["error"] = errMsg- }-- environmentID := envID- if environmentID == "" {- environmentID = "0"- }-- eventType := models.EventTypeImageScan- if err := s.eventService.LogImageEvent(ctx, eventType, imageID, imageName, user.ID, user.Username, environmentID, metadata); err != nil {- slog.WarnContext(ctx, "failed to log vulnerability scan event", "error", err)- }-}--// logScheduledScanEvent logs a vulnerability scan event using the dedicated scheduled scan event type.-func (s *VulnerabilityService) logScheduledScanEvent(ctx context.Context, envID string, imageID, imageName string, user models.User, success bool, errMsg string) {- metadata := models.JSON{- "action": "scheduled_vulnerability_scan",- "imageId": imageID,- "imageName": imageName,- "success": success,- }- if errMsg != "" {- metadata["error"] = errMsg- }-- environmentID := envID- if environmentID == "" {- environmentID = "0"- }-- if err := s.eventService.LogImageEvent(ctx, models.EventTypeImageVulnerabilityScan, imageID, imageName, user.ID, user.Username, environmentID, metadata); err != nil {- slog.WarnContext(ctx, "failed to log scheduled vulnerability scan event", "error", err)- }-}--// IgnoreVulnerability creates a new ignore record for a vulnerability-func (s *VulnerabilityService) IgnoreVulnerability(ctx context.Context, envID string, payload *vulnerability.IgnorePayload) (*models.VulnerabilityIgnore, error) {- if s.db == nil {- return nil, errors.New("database not available")- }-- // Check if already ignored (composite key check)- var existing models.VulnerabilityIgnore- err := s.db.WithContext(ctx).Where(- "environment_id = ? AND image_id = ? AND vulnerability_id = ? AND pkg_name = ? AND installed_version = ?",- envID, payload.ImageID, payload.VulnerabilityID, payload.PkgName, payload.InstalledVersion,- ).First(&existing).Error-- if err == nil {- return nil, errors.New("vulnerability is already ignored")- }-- if !errors.Is(err, gorm.ErrRecordNotFound) {- return nil, errors.WrapIf(err, "failed to check existing ignore")- }-- ignore := &models.VulnerabilityIgnore{- EnvironmentID: envID,- ImageID: payload.ImageID,- VulnerabilityID: payload.VulnerabilityID,- PkgName: payload.PkgName,- InstalledVersion: payload.InstalledVersion,- Reason: payload.Reason,- CreatedBy: payload.CreatedBy,- }-- if err := s.db.WithContext(ctx).Create(ignore).Error; err != nil {- return nil, errors.WrapIf(err, "failed to create ignore record")- }-- slog.InfoContext(ctx, "vulnerability ignored",- "vulnerability_id", payload.VulnerabilityID,- "image_id", payload.ImageID,- "pkg_name", payload.PkgName,- )-- return ignore, nil-}--// UnignoreVulnerability removes an ignore record-func (s *VulnerabilityService) UnignoreVulnerability(ctx context.Context, envID string, ignoreID string) error {- if s.db == nil {- return errors.New("database not available")- }-- result := s.db.WithContext(ctx).Where("id = ? AND environment_id = ?", ignoreID, envID).Delete(&models.VulnerabilityIgnore{})- if result.Error != nil {- return errors.WrapIf(result.Error, "failed to delete ignore record")- }-- if result.RowsAffected == 0 {- return errors.New("ignore record not found")- }-- slog.InfoContext(ctx, "vulnerability unignored", "ignore_id", ignoreID)- return nil-}--func applyIgnoredVulnerabilitiesSort(query *gorm.DB, sort string) *gorm.DB {- if sort == "" {- return query.Order("created_at DESC")- }-- for part := range strings.SplitSeq(sort, ",") {- part = strings.TrimSpace(part)- if part == "" {- continue- }-- desc := false- switch {- case strings.HasPrefix(part, "-"):- desc = true- part = strings.TrimPrefix(part, "-")- case strings.HasPrefix(part, "+"):- part = strings.TrimPrefix(part, "+")- }-- query = applyIgnoredVulnerabilitiesSortPart(query, part, desc)- }-- return query-}--func applyIgnoredVulnerabilitiesSortPart(query *gorm.DB, part string, desc bool) *gorm.DB {- direction := "ASC"- if desc {- direction = "DESC"- }-- switch part {- case "createdAt":- return query.Order("created_at " + direction)- case "vulnerabilityId":- return query.Order("vulnerability_id " + direction)- default:- return query.Order("created_at DESC")- }-}--func mapIgnoredVulnerabilities(ignores []models.VulnerabilityIgnore) []vulnerability.IgnoredVulnerability {- result := make([]vulnerability.IgnoredVulnerability, len(ignores))- for i, ignore := range ignores {- result[i] = vulnerability.IgnoredVulnerability{- ID: ignore.ID,- EnvironmentID: ignore.EnvironmentID,- ImageID: ignore.ImageID,- VulnerabilityID: ignore.VulnerabilityID,- PkgName: ignore.PkgName,- InstalledVersion: ignore.InstalledVersion,- Reason: ignore.Reason,- CreatedBy: ignore.CreatedBy,- CreatedAt: ignore.CreatedAt,- }- }- return result-}--func buildIgnoredVulnerabilitiesResponse(total int64, params pagination.QueryParams, ignores []models.VulnerabilityIgnore) pagination.Response {- filtered := pagination.FilterResult[models.VulnerabilityIgnore]{- Items: ignores,- TotalCount: total,- TotalAvailable: total,- }- return pagination.BuildResponseFromFilterResult(filtered, params)-}--// ListIgnoredVulnerabilities returns a list of ignored vulnerabilities for an environment-func (s *VulnerabilityService) ListIgnoredVulnerabilities(ctx context.Context, envID string, params pagination.QueryParams) ([]vulnerability.IgnoredVulnerability, pagination.Response, error) {- if params.Limit == 0 {- params.Limit = 20- }-- if s.db == nil {- return []vulnerability.IgnoredVulnerability{}, pagination.Response{- TotalPages: 1,- TotalItems: 0,- CurrentPage: 1,- ItemsPerPage: params.Limit,- GrandTotalItems: 0,- }, nil- }-- var ignores []models.VulnerabilityIgnore- query := s.db.WithContext(ctx).Where("environment_id = ?", envID)- query = applyIgnoredVulnerabilitiesSort(query, params.Sort)-- // Count total- var total int64- if err := query.Model(&models.VulnerabilityIgnore{}).Count(&total).Error; err != nil {- return nil, pagination.Response{}, errors.WrapIf(err, "failed to count ignored vulnerabilities")- }-- // Apply pagination- offset := params.Start- if err := query.Offset(offset).Limit(params.Limit).Find(&ignores).Error; err != nil {- return nil, pagination.Response{}, errors.WrapIf(err, "failed to list ignored vulnerabilities")- }-- result := mapIgnoredVulnerabilities(ignores)- response := buildIgnoredVulnerabilitiesResponse(total, params, ignores)-- return result, response, nil-}--// GetIgnoreRecordsForImage retrieves all ignore records for a specific image-func (s *VulnerabilityService) GetIgnoreRecordsForImage(ctx context.Context, envID string, imageID string) ([]models.VulnerabilityIgnore, error) {- if s.db == nil {- return nil, nil- }-- var ignores []models.VulnerabilityIgnore- if err := s.db.WithContext(ctx).Where("environment_id = ? AND image_id = ?", envID, imageID).Find(&ignores).Error; err != nil {- return nil, errors.WrapIf(err, "failed to get ignore records")- }-- return ignores, nil-}diff --git a/backend/internal/services/vulnerability_service_test.go b/backend/internal/services/vulnerability_service_test.godeleted file mode 100644index 11f692a64..000000000--- a/backend/internal/services/vulnerability_service_test.go+++ /dev/null@@ -1,942 +0,0 @@-package services--import (- "context"- "encoding/json"- "errors"- "fmt"- "io"- "net"- "os"- "strings"- "testing"- "time"-- "github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/vuln"- "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"- "github.com/getarcaneapp/arcane/types/v2/vulnerability"- "github.com/libtnb/sqlite"- containertypes "github.com/moby/moby/api/types/container"- mounttypes "github.com/moby/moby/api/types/mount"- networktypes "github.com/moby/moby/api/types/network"- "github.com/stretchr/testify/require"- "gorm.io/gorm"-)--func TestIsExpectedDockerStreamEndErrorInternal(t *testing.T) {- tests := []struct {- name string- err error- want bool- }{- {name: "nil", err: nil, want: false},- {name: "eof", err: io.EOF, want: true},- {name: "wrapped eof", err: fmt.Errorf("wrapped: %w", io.EOF), want: true},- {name: "net err closed", err: net.ErrClosed, want: true},- {name: "closed network connection", err: errors.New("read tcp 127.0.0.1:1234->127.0.0.1:5678: use of closed network connection"), want: true},- {name: "broken pipe", err: errors.New("write: broken pipe"), want: true},- {name: "connection reset by peer", err: errors.New("read: connection reset by peer"), want: true},- {name: "unexpected", err: errors.New("some other error"), want: false},- }-- for _, tt := range tests {- t.Run(tt.name, func(t *testing.T) {- require.Equal(t, tt.want, isExpectedDockerStreamEndErrorInternal(tt.err))- })- }-}--func TestDecodeTrivyReportFromFileInternal_LargePayload(t *testing.T) {- tmpFile, err := os.CreateTemp("", "trivy-report-*.json")- require.NoError(t, err)- defer func() {- require.NoError(t, os.Remove(tmpFile.Name()))- }()- defer func() {- require.NoError(t, tmpFile.Close())- }()-- const vulnCount = 4000- vulns := make([]vulnerability.TrivyVulnerability, 0, vulnCount)- for i := range vulnCount {- vulns = append(vulns, vulnerability.TrivyVulnerability{- VulnerabilityID: fmt.Sprintf("CVE-2026-%04d", i),- PkgName: fmt.Sprintf("pkg-%d", i),- InstalledVersion: "1.0.0",- FixedVersion: "1.0.1",- Severity: "HIGH",- })- }-- report := vulnerability.TrivyReport{- SchemaVersion: 2,- ArtifactName: "example/image:latest",- ArtifactType: "container_image",- Results: []vulnerability.TrivyResults{- {- Target: "alpine:3.20",- Class: "os-pkgs",- Type: "alpine",- Vulnerabilities: vulns,- },- },- }-- encoder := json.NewEncoder(tmpFile)- require.NoError(t, encoder.Encode(report))-- decoded, err := decodeTrivyReportFromFileInternal(tmpFile)- require.NoError(t, err)- require.NotNil(t, decoded)- require.Equal(t, report.ArtifactName, decoded.ArtifactName)- require.Len(t, decoded.Results, 1)- require.Len(t, decoded.Results[0].Vulnerabilities, vulnCount)-}--func TestDecodeTrivyReportFromFileInternal_RecoversFromPrefixedAndTrailingNoise(t *testing.T) {- tmpFile, err := os.CreateTemp("", "trivy-report-noisy-*.json")- require.NoError(t, err)- defer func() {- require.NoError(t, os.Remove(tmpFile.Name()))- }()- defer func() {- require.NoError(t, tmpFile.Close())- }()-- report := vulnerability.TrivyReport{- SchemaVersion: 2,- ArtifactName: "ghcr.io/getarcaneapp/example:latest",- ArtifactType: "container_image",- Results: []vulnerability.TrivyResults{- {Target: "alpine:3.20", Class: "os-pkgs", Type: "alpine"},- },- }-- reportBytes, err := json.Marshal(report)- require.NoError(t, err)-- _, err = tmpFile.WriteString("WARN scanner warming cache\n")- require.NoError(t, err)- _, err = tmpFile.Write(reportBytes)- require.NoError(t, err)- _, err = tmpFile.WriteString("\nINFO scan completed\n")- require.NoError(t, err)-- decoded, err := decodeTrivyReportFromFileInternal(tmpFile)- require.NoError(t, err)- require.NotNil(t, decoded)- require.Equal(t, report.ArtifactName, decoded.ArtifactName)- require.Equal(t, report.SchemaVersion, decoded.SchemaVersion)-}--func TestDecodeTrivyReportFromFileInternal_SelectsReportFromMultipleJSONObjects(t *testing.T) {- tmpFile, err := os.CreateTemp("", "trivy-report-multi-json-*.json")- require.NoError(t, err)- defer func() {- require.NoError(t, os.Remove(tmpFile.Name()))- }()- defer func() {- require.NoError(t, tmpFile.Close())- }()-- report := vulnerability.TrivyReport{- SchemaVersion: 2,- ArtifactName: "ghcr.io/getarcaneapp/example:stable",- ArtifactType: "container_image",- Results: []vulnerability.TrivyResults{- {Target: "debian:12", Class: "os-pkgs", Type: "debian"},- },- }-- reportBytes, err := json.Marshal(report)- require.NoError(t, err)-- _, err = tmpFile.WriteString(`{"action":"vulnerability_scan","success":false}` + "\n")- require.NoError(t, err)- _, err = tmpFile.Write(reportBytes)- require.NoError(t, err)-- decoded, err := decodeTrivyReportFromFileInternal(tmpFile)- require.NoError(t, err)- require.NotNil(t, decoded)- require.Equal(t, report.ArtifactName, decoded.ArtifactName)- require.Equal(t, report.SchemaVersion, decoded.SchemaVersion)-}--func TestDecodeTrivyReportFromFileInternal_HandlesConcatenatedJSONObjects(t *testing.T) {- tmpFile, err := os.CreateTemp("", "trivy-report-concatenated-*.json")- require.NoError(t, err)- defer func() {- require.NoError(t, os.Remove(tmpFile.Name()))- }()- defer func() {- require.NoError(t, tmpFile.Close())- }()-- report := vulnerability.TrivyReport{- SchemaVersion: 2,- ArtifactName: "ghcr.io/getarcaneapp/example:concat",- ArtifactType: "container_image",- Results: []vulnerability.TrivyResults{- {Target: "ubuntu:24.04", Class: "os-pkgs", Type: "ubuntu"},- },- }-- reportBytes, err := json.Marshal(report)- require.NoError(t, err)-- _, err = tmpFile.Write(reportBytes)- require.NoError(t, err)- _, err = tmpFile.WriteString(`{"message":"done"}`)- require.NoError(t, err)-- decoded, err := decodeTrivyReportFromFileInternal(tmpFile)- require.NoError(t, err)- require.NotNil(t, decoded)- require.Equal(t, report.ArtifactName, decoded.ArtifactName)- require.Equal(t, report.SchemaVersion, decoded.SchemaVersion)-}--func TestCleanupTrivyLogTempFilesInternal_RemovesFiles(t *testing.T) {- stdoutFile, err := os.CreateTemp("", "trivy-stdout-test-*")- require.NoError(t, err)- stderrFile, err := os.CreateTemp("", "trivy-stderr-test-*")- require.NoError(t, err)-- stdoutPath := stdoutFile.Name()- stderrPath := stderrFile.Name()-- _, err = stdoutFile.WriteString("stdout")- require.NoError(t, err)- _, err = stderrFile.WriteString("stderr")- require.NoError(t, err)-- cleanupTrivyLogTempFilesInternal(context.Background(), stdoutFile, stderrFile)-- _, err = os.Stat(stdoutPath)- require.Error(t, err)- require.True(t, os.IsNotExist(err))-- _, err = os.Stat(stderrPath)- require.Error(t, err)- require.True(t, os.IsNotExist(err))-}--func TestBuildTrivyContainerResourcesInternal_Defaults(t *testing.T) {- resources, applyLimits := buildTrivyContainerResourcesInternal(nil).Get()-- require.True(t, applyLimits)- require.Equal(t, int64(1_000_000_000), resources.NanoCPUs)- require.Equal(t, int64(0), resources.Memory)- require.Equal(t, int64(0), resources.MemorySwap)-}--func TestBuildTrivyContainerResourcesInternal_LimitsDisabled(t *testing.T) {- cfg := &models.Settings{- TrivyResourceLimitsEnabled: models.SettingVariable{Value: "false"},- TrivyCpuLimit: models.SettingVariable{Value: "2"},- TrivyMemoryLimitMb: models.SettingVariable{Value: "2048"},- }-- resources, applyLimits := buildTrivyContainerResourcesInternal(cfg).Get()-- require.False(t, applyLimits)- require.Equal(t, int64(0), resources.NanoCPUs)- require.Equal(t, int64(0), resources.Memory)- require.Equal(t, int64(0), resources.MemorySwap)-}--func TestBuildTrivyContainerResourcesInternal_CustomLimits(t *testing.T) {- cfg := &models.Settings{- TrivyResourceLimitsEnabled: models.SettingVariable{Value: "true"},- TrivyCpuLimit: models.SettingVariable{Value: "2.5"},- TrivyMemoryLimitMb: models.SettingVariable{Value: "2048"},- }-- resources, applyLimits := buildTrivyContainerResourcesInternal(cfg).Get()-- require.True(t, applyLimits)- require.Equal(t, int64(2_500_000_000), resources.NanoCPUs)- require.Equal(t, int64(2_147_483_648), resources.Memory)- require.Equal(t, int64(2_147_483_648), resources.MemorySwap)-}--func TestBuildTrivyCommandArgs_WithCacheDir(t *testing.T) {- svc := &VulnerabilityService{}- outputPath := "/tmp/trivy-result-test.json"-- cmdArgs, tempFiles := svc.buildTrivyCommandArgs(context.Background(), "alpine:3.20", "", "", "/tmp/trivy-cache", outputPath)-- require.Empty(t, tempFiles)- require.Contains(t, cmdArgs, "--cache-dir")- require.Contains(t, cmdArgs, "/tmp/trivy-cache")- require.Contains(t, cmdArgs, "--output")- require.Contains(t, cmdArgs, outputPath)- require.Equal(t, "alpine:3.20", cmdArgs[len(cmdArgs)-1])-}--func TestBuildTrivyCommandArgs_WithoutCacheDir(t *testing.T) {- svc := &VulnerabilityService{}- outputPath := "/tmp/trivy-result-test.json"-- cmdArgs, tempFiles := svc.buildTrivyCommandArgs(context.Background(), "alpine:3.20", "", "", "", outputPath)-- require.Empty(t, tempFiles)- require.NotContains(t, cmdArgs, "--cache-dir")- require.Contains(t, cmdArgs, "--output")- require.Contains(t, cmdArgs, outputPath)- require.Equal(t, "alpine:3.20", cmdArgs[len(cmdArgs)-1])-}--func TestBuildTrivyCommandArgs_IncludesDefaultRepositories(t *testing.T) {- svc := &VulnerabilityService{}- cmdArgs, _ := svc.buildTrivyCommandArgs(context.Background(), "alpine:3.20", "", "", "", "/tmp/out.json")-- require.Contains(t, cmdArgs, "--db-repository")- require.Contains(t, cmdArgs, vuln.DefaultDBRepository)- require.Contains(t, cmdArgs, "--java-db-repository")- require.Contains(t, cmdArgs, vuln.DefaultJavaDBRepository)- require.Contains(t, cmdArgs, "--checks-bundle-repository")- require.Contains(t, cmdArgs, vuln.DefaultChecksBundleRepository)-}--func TestBuildTrivyHostConfig_ExcludesResourcesWhenLimitsDisabled(t *testing.T) {- resources := containertypes.Resources{- NanoCPUs: int64(2_000_000_000),- Memory: int64(1_073_741_824),- MemorySwap: int64(1_073_741_824),- }-- hostConfig := buildTrivyHostConfig(- "cache-volume",- nil,- resources,- "",- false,- trivyRuntimeOptions{NetworkMode: "bridge"},- nil,- false,- )- require.Equal(t, containertypes.NetworkMode("bridge"), hostConfig.NetworkMode)- require.Equal(t, int64(0), hostConfig.NanoCPUs)- require.Equal(t, int64(0), hostConfig.Memory)- require.Equal(t, int64(0), hostConfig.MemorySwap)- require.Empty(t, hostConfig.SecurityOpt)- require.False(t, hostConfig.Privileged)-}--func TestBuildTrivyHostConfig_IncludesResourcesWhenLimitsEnabled(t *testing.T) {- resources := containertypes.Resources{- NanoCPUs: int64(2_000_000_000),- Memory: int64(1_073_741_824),- MemorySwap: int64(1_073_741_824),- }-- hostConfig := buildTrivyHostConfig(- "cache-volume",- nil,- resources,- "",- true,- trivyRuntimeOptions{NetworkMode: "arcane-external"},- nil,- false,- )- require.Equal(t, containertypes.NetworkMode("arcane-external"), hostConfig.NetworkMode)- require.Equal(t, resources.NanoCPUs, hostConfig.NanoCPUs)- require.Equal(t, resources.Memory, hostConfig.Memory)- require.Equal(t, resources.MemorySwap, hostConfig.MemorySwap)-}--func TestBuildTrivyHostConfig_UsesCPUSetAndClearsNanoCPUs(t *testing.T) {- resources := containertypes.Resources{- NanoCPUs: int64(2_000_000_000),- Memory: int64(1_073_741_824),- MemorySwap: int64(1_073_741_824),- }-- hostConfig := buildTrivyHostConfig(- "cache-volume",- nil,- resources,- "0-1",- true,- trivyRuntimeOptions{NetworkMode: "bridge"},- nil,- false,- )- require.Equal(t, containertypes.NetworkMode("bridge"), hostConfig.NetworkMode)- require.Equal(t, int64(0), hostConfig.NanoCPUs)- require.Equal(t, "0-1", hostConfig.CpusetCpus)- require.Equal(t, resources.Memory, hostConfig.Memory)- require.Equal(t, resources.MemorySwap, hostConfig.MemorySwap)-}--func TestBuildTrivyHostConfig_AppliesRuntimeSecurity(t *testing.T) {- hostConfig := buildTrivyHostConfig(- "cache-volume",- nil,- containertypes.Resources{},- "",- false,- trivyRuntimeOptions{NetworkMode: "bridge"},- []string{"label=disable", "label=type:container_runtime_t"},- true,- )-- require.Equal(t, []string{"label=disable", "label=type:container_runtime_t"}, hostConfig.SecurityOpt)- require.True(t, hostConfig.Privileged)-}--func TestBuildTrivyHostConfig_SkipsCacheMountWhenVolumeEmpty(t *testing.T) {- hostConfig := buildTrivyHostConfig(- "",- nil,- containertypes.Resources{},- "",- false,- trivyRuntimeOptions{NetworkMode: "bridge"},- nil,- false,- )-- require.Empty(t, hostConfig.Mounts)-- socketMount := mounttypes.Mount{- Type: mounttypes.TypeBind,- Source: "/run/user/1000/podman/podman.sock",- Target: "/run/user/1000/podman/podman.sock",- }- hostConfig = buildTrivyHostConfig(- "",- nil,- containertypes.Resources{},- "",- false,- trivyRuntimeOptions{- NetworkMode: "bridge",- Mounts: []mounttypes.Mount{socketMount},- },- nil,- false,- )-- require.Len(t, hostConfig.Mounts, 1)- require.Equal(t, socketMount.Source, hostConfig.Mounts[0].Source)-}--func TestBuildTrivyBatchHostConfig_AppliesRuntimeSecurity(t *testing.T) {- hostConfig := buildTrivyBatchHostConfigInternal(- "cache-volume",- trivyRuntimeOptions{NetworkMode: "arcane-external"},- []string{"label=disable"},- true,- )-- require.Equal(t, containertypes.NetworkMode("arcane-external"), hostConfig.NetworkMode)- require.Equal(t, []string{"label=disable"}, hostConfig.SecurityOpt)- require.True(t, hostConfig.Privileged)- require.Len(t, hostConfig.Mounts, 1)- require.Equal(t, "cache-volume", hostConfig.Mounts[0].Source)-}--func TestBuildTrivyBatchHostConfig_SkipsCacheMountWhenVolumeEmpty(t *testing.T) {- hostConfig := buildTrivyBatchHostConfigInternal(- "",- trivyRuntimeOptions{NetworkMode: "arcane-external"},- nil,- false,- )-- require.Empty(t, hostConfig.Mounts)-}--func TestBuildTrivyHostConfig_IncludesInheritedSocketMount(t *testing.T) {- hostConfig := buildTrivyHostConfig(- "cache-volume",- nil,- containertypes.Resources{},- "",- false,- trivyRuntimeOptions{- NetworkMode: "bridge",- Mounts: []mounttypes.Mount{- {- Type: mounttypes.TypeBind,- Source: "/run/user/1000/podman/podman.sock",- Target: "/run/user/1000/podman/podman.sock",- },- },- },- nil,- false,- )-- require.Len(t, hostConfig.Mounts, 2)- require.Equal(t, "/run/user/1000/podman/podman.sock", hostConfig.Mounts[0].Source)- require.Equal(t, "/run/user/1000/podman/podman.sock", hostConfig.Mounts[0].Target)- require.Equal(t, "cache-volume", hostConfig.Mounts[1].Source)-}--func TestApplyTrivyRuntimeSecurityInternal(t *testing.T) {- hostConfig := &containertypes.HostConfig{}-- applyTrivyRuntimeSecurityInternal(hostConfig, []string{"label=disable"}, true)-- require.Equal(t, []string{"label=disable"}, hostConfig.SecurityOpt)- require.True(t, hostConfig.Privileged)-}--func TestSelectDefaultTrivyNetworkModeInternal(t *testing.T) {- require.Equal(t, DefaultTrivyNetworkMode, selectDefaultTrivyNetworkModeInternal(""))- require.Equal(t, DefaultTrivyNetworkMode, selectDefaultTrivyNetworkModeInternal(" \t\n "))- require.Equal(t, "arcane-external", selectDefaultTrivyNetworkModeInternal("arcane-external"))-}--func TestResolveTrivyUnixSocketSourceInternal(t *testing.T) {- t.Run("uses resolved host path", func(t *testing.T) {- source, err := resolveTrivyUnixSocketSourceInternal(- context.Background(),- "/run/user/1000/podman/podman.sock",- func(context.Context, string) (string, error) {- return "/host/podman/podman.sock", nil- },- func() bool { return true },- )-- require.NoError(t, err)- require.Equal(t, "/host/podman/podman.sock", source)- })-- t.Run("fails when socket cannot be resolved in docker", func(t *testing.T) {- _, err := resolveTrivyUnixSocketSourceInternal(- context.Background(),- "/run/user/1000/podman/podman.sock",- func(context.Context, string) (string, error) {- return "", nil- },- func() bool { return true },- )-- require.Error(t, err)- require.Contains(t, err.Error(), "daemon-visible host path")- })-- t.Run("falls back to original path outside docker", func(t *testing.T) {- source, err := resolveTrivyUnixSocketSourceInternal(- context.Background(),- "/var/run/docker.sock",- func(context.Context, string) (string, error) {- return "", errors.New("not running in docker")- },- func() bool { return false },- )-- require.NoError(t, err)- require.Equal(t, "/var/run/docker.sock", source)- })-}--func TestSelectTrivyAutoNetworkModeInternal(t *testing.T) {- t.Run("prefers explicit host network mode", func(t *testing.T) {- mode := selectTrivyAutoNetworkModeInternal(&containertypes.InspectResponse{- HostConfig: &containertypes.HostConfig{NetworkMode: "host"},- })-- require.Equal(t, "host", mode)- })-- t.Run("prefers attached custom network over bridge", func(t *testing.T) {- mode := selectTrivyAutoNetworkModeInternal(&containertypes.InspectResponse{- HostConfig: &containertypes.HostConfig{NetworkMode: "bridge"},- NetworkSettings: &containertypes.NetworkSettings{- Networks: map[string]*networktypes.EndpointSettings{- "bridge": {},- "arcane-internal": {},- },- },- })-- require.Equal(t, "arcane-internal", mode)- })-- t.Run("falls back to bridge when no custom network is attached", func(t *testing.T) {- mode := selectTrivyAutoNetworkModeInternal(&containertypes.InspectResponse{- HostConfig: &containertypes.HostConfig{NetworkMode: "bridge"},- NetworkSettings: &containertypes.NetworkSettings{- Networks: map[string]*networktypes.EndpointSettings{- "bridge": {},- },- },- })-- require.Equal(t, "bridge", mode)- })-}--func TestIsSynologyDockerHostInternal(t *testing.T) {- // DS220+ (Kernel 4.x): single-line- require.True(t, isSynologyDockerHostInternal("Synology NAS"))- // DS925+ (Kernel 5.x): OS field contains a newline before "(containerized)"- require.True(t, isSynologyDockerHostInternal("Synology NAS\n (containerized)"))- // Variation: Windows-style line ending- require.True(t, isSynologyDockerHostInternal("Synology NAS\r\n (containerized)"))- // Case-insensitive- require.True(t, isSynologyDockerHostInternal("synology nas"))- // Padded with spaces- require.True(t, isSynologyDockerHostInternal(" Synology NAS "))- // Empty / non-Synology- require.False(t, isSynologyDockerHostInternal(""))- require.False(t, isSynologyDockerHostInternal("Ubuntu 24.04.1 LTS"))- require.False(t, isSynologyDockerHostInternal("Ubuntu 24.04.1 LTS\n (containerized)"))-}--func TestBuildTrivyCPUSetInternal(t *testing.T) {- require.Empty(t, buildTrivyCPUSetInternal(0, 4))- require.Equal(t, "0", buildTrivyCPUSetInternal(1_000_000_000, 4))- require.Equal(t, "0-1", buildTrivyCPUSetInternal(2_500_000_000, 4))- require.Equal(t, "0-1", buildTrivyCPUSetInternal(3_000_000_000, 2))- require.Equal(t, "0", buildTrivyCPUSetInternal(1_000_000_000, 0))-}--func TestGetTrivySingleScanCacheDirForSlotInternal(t *testing.T) {- svc := &VulnerabilityService{}-- require.Equal(t, "/root/.cache/arcane-scan-slots/slot-0", svc.getTrivySingleScanCacheDirForSlotInternal(0))- require.Equal(t, "/root/.cache/arcane-scan-slots/slot-2", svc.getTrivySingleScanCacheDirForSlotInternal(2))- // Negative slot IDs should normalize to slot-0- require.Equal(t, "/root/.cache/arcane-scan-slots/slot-0", svc.getTrivySingleScanCacheDirForSlotInternal(-1))-}--func TestGetTrivyScanSlotChannelInternal_PrefillsSlots(t *testing.T) {- svc := &VulnerabilityService{}-- slotCh := svc.getTrivyScanSlotChannelInternal(3)- require.NotNil(t, slotCh)- require.Equal(t, 3, cap(slotCh))- require.Len(t, slotCh, 3)-- seen := map[int]bool{}- for range 3 {- slotID := <-slotCh- seen[slotID] = true- }-- require.True(t, seen[0])- require.True(t, seen[1])- require.True(t, seen[2])- require.Empty(t, slotCh)-}--func TestTrivyMountSummariesInternal(t *testing.T) {- require.Nil(t, trivyMountSummariesInternal(nil))-- hostConfig := &containertypes.HostConfig{- Mounts: []mounttypes.Mount{- {- Type: mounttypes.TypeBind,- Source: "/var/run/docker.sock",- Target: "/var/run/docker.sock",- },- {- Type: mounttypes.TypeVolume,- Source: "arcane-trivy-cache",- Target: "/root/.cache",- },- {- Type: mounttypes.TypeTmpfs,- Target: "/tmp",- },- },- }-- require.Equal(t,- []string{- "/var/run/docker.sock:/var/run/docker.sock",- "arcane-trivy-cache:/root/.cache",- "tmpfs:/tmp",- },- trivyMountSummariesInternal(hostConfig),- )-}--func TestTruncateTrivyLogOutputInternal(t *testing.T) {- require.Empty(t, truncateTrivyLogOutputInternal("", 32))- require.Equal(t, "keep me", truncateTrivyLogOutputInternal(" keep me ", 32))-- longOutput := strings.Repeat("x", 80)- truncated := truncateTrivyLogOutputInternal(longOutput, 16)- require.Equal(t, strings.Repeat("x", 16)+" ...<truncated>", truncated)-}--func TestTrivyContainerWaitTimeoutInternal(t *testing.T) {- require.Equal(t, timeouts.DefaultTrivyScan+timeouts.DefaultDockerAPI, trivyContainerWaitTimeoutInternal(""))- require.Equal(t, 5*time.Minute+timeouts.DefaultDockerAPI, trivyContainerWaitTimeoutInternal("5m"))- require.Equal(t, timeouts.DefaultTrivyScan+timeouts.DefaultDockerAPI, trivyContainerWaitTimeoutInternal("not-a-duration"))- require.Equal(t, timeouts.DefaultTrivyScan+timeouts.DefaultDockerAPI, trivyContainerWaitTimeoutInternal("0s"))-}--func TestAwaitContainerWaitResponseInternal_Status(t *testing.T) {- statusCh := make(chan containertypes.WaitResponse, 1)- errCh := make(chan error)- statusCh <- containertypes.WaitResponse{StatusCode: 12}-- status, err := awaitContainerWaitResponseInternal(context.Background(), statusCh, errCh)- require.NoError(t, err)- require.Equal(t, int64(12), status)-}--func TestAwaitContainerWaitResponseInternal_Error(t *testing.T) {- statusCh := make(chan containertypes.WaitResponse)- errCh := make(chan error, 1)- errCh <- errors.New("boom")-- status, err := awaitContainerWaitResponseInternal(context.Background(), statusCh, errCh)- require.Error(t, err)- require.Contains(t, err.Error(), "boom")- require.Equal(t, int64(0), status)-}--func TestAwaitContainerWaitResponseInternal_ClosedErrorChannelStillReadsStatus(t *testing.T) {- statusCh := make(chan containertypes.WaitResponse, 1)- errCh := make(chan error)-- close(errCh)- statusCh <- containertypes.WaitResponse{StatusCode: 7}-- status, err := awaitContainerWaitResponseInternal(context.Background(), statusCh, errCh)- require.NoError(t, err)- require.Equal(t, int64(7), status)-}--func TestAwaitContainerWaitResponseInternal_ContextDone(t *testing.T) {- statusCh := make(chan containertypes.WaitResponse)- errCh := make(chan error)- ctx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond)- defer cancel()-- status, err := awaitContainerWaitResponseInternal(ctx, statusCh, errCh)- require.Error(t, err)- require.ErrorIs(t, err, context.DeadlineExceeded)- require.Equal(t, int64(0), status)-}--func TestAwaitContainerWaitResponseInternal_NoStatus(t *testing.T) {- statusCh := make(chan containertypes.WaitResponse)- errCh := make(chan error)- close(statusCh)- close(errCh)-- status, err := awaitContainerWaitResponseInternal(context.Background(), statusCh, errCh)- require.Error(t, err)- require.Contains(t, err.Error(), "ended without status")- require.Equal(t, int64(0), status)-}--func setupVulnerabilityScanTestDB(t *testing.T) *database.DB {- t.Helper()- dsn := fmt.Sprintf("file:vulnerability-scan-test-%d?mode=memory&cache=shared", time.Now().UnixNano())- db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})- require.NoError(t, err)- require.NoError(t, db.AutoMigrate(&models.VulnerabilityScanRecord{}, &models.VulnerabilityIgnore{}))- return &database.DB{DB: db}-}--func TestVulnerabilityService_DeleteScanResultsByImageIDs(t *testing.T) {- ctx := context.Background()- db := setupVulnerabilityScanTestDB(t)- svc := &VulnerabilityService{db: db}-- records := []models.VulnerabilityScanRecord{- {ID: "sha256:img1", ImageName: "nginx:latest", Status: models.ScanStatusCompleted, ScanTime: time.Now()},- {ID: "sha256:img2", ImageName: "redis:7", Status: models.ScanStatusCompleted, ScanTime: time.Now()},- {ID: "sha256:img3", ImageName: "postgres:16", Status: models.ScanStatusCompleted, ScanTime: time.Now()},- }- for i := range records {- require.NoError(t, db.Create(&records[i]).Error)- }-- require.NoError(t, svc.DeleteScanResultsByImageIDs(ctx, []string{"sha256:img1", "sha256:img3"}))-- var remaining []models.VulnerabilityScanRecord- require.NoError(t, db.Find(&remaining).Error)- require.Len(t, remaining, 1)- require.Equal(t, "sha256:img2", remaining[0].ID)-}--func TestVulnerabilityService_DeleteScanResultsByImageIDs_EmptyInput(t *testing.T) {- ctx := context.Background()- db := setupVulnerabilityScanTestDB(t)- svc := &VulnerabilityService{db: db}-- require.NoError(t, svc.DeleteScanResultsByImageIDs(ctx, nil))- require.NoError(t, svc.DeleteScanResultsByImageIDs(ctx, []string{}))-}--func TestSeverityCountColumnsForFilterInternal(t *testing.T) {- tests := []struct {- name string- value string- want []string- wantNil bool- }{- {- name: "single severity",- value: "critical",- want: []string{"critical_count"},- },- {- name: "multiple severities and duplicates",- value: "high,critical,high,UNKNOWN",- want: []string{"high_count", "critical_count", "unknown_count"},- },- {- name: "invalid severities only",- value: "foo,bar",- want: []string{},- },- {- name: "empty value",- value: " , , ",- wantNil: true,- },- }-- for _, tt := range tests {- t.Run(tt.name, func(t *testing.T) {- got := severityCountColumnsForFilterInternal(tt.value)- if tt.wantNil {- require.Nil(t, got)- return- }- require.Equal(t, tt.want, got)- })- }-}--func TestApplyListAllVulnerabilityRecordPrefiltersInternal_NoPossibleMatches(t *testing.T) {- db := setupVulnerabilityScanTestDB(t)- base := db.WithContext(context.Background()).Model(&models.VulnerabilityScanRecord{})-- _, ok := applyListAllVulnerabilityRecordPrefiltersInternal(base, map[string]string{"severity": "not-a-severity"}).Get()- require.False(t, ok)-- _, ok = applyListAllVulnerabilityRecordPrefiltersInternal(base, map[string]string{"imageName": " , "}).Get()- require.False(t, ok)-- _, ok = applyListAllVulnerabilityRecordPrefiltersInternal(base, map[string]string{- "severity": "high,bad",- "imageName": "nginx",- }).Get()- require.True(t, ok)-}--func TestEstimatedVulnerabilityRecordCountInternal(t *testing.T) {- records := []models.VulnerabilityScanRecord{- {TotalCount: 5},- {TotalCount: 0},- {TotalCount: -3},- {TotalCount: 12},- }-- require.Equal(t, 17, estimatedVulnerabilityRecordCountInternal(records))-}--func TestVulnerabilityService_ListAllVulnerabilities_FiltersIgnoredInline(t *testing.T) {- ctx := context.Background()- db := setupVulnerabilityScanTestDB(t)- svc := &VulnerabilityService{db: db}-- vulns := []vulnerability.Vulnerability{- {- VulnerabilityID: "CVE-2026-0001",- PkgName: "openssl",- InstalledVersion: "1.0.0",- Severity: vulnerability.SeverityHigh,- },- }- rawVulns, err := json.Marshal(vulns)- require.NoError(t, err)-- scan := models.VulnerabilityScanRecord{- ID: "sha256:list-all-1",- ImageName: "nginx:latest",- Status: models.ScanStatusCompleted,- ScanTime: time.Now(),- HighCount: 1,- TotalCount: 1,- Vulnerabilities: models.StringSlice{string(rawVulns)},- }- require.NoError(t, db.Create(&scan).Error)-- ignore := models.VulnerabilityIgnore{- EnvironmentID: "env-1",- ImageID: scan.ID,- VulnerabilityID: "CVE-2026-0001",- PkgName: "openssl",- InstalledVersion: "1.0.0",- Reason: new("accepted risk"),- CreatedBy: "tester",- }- require.NoError(t, db.Create(&ignore).Error)-- items, page, err := svc.ListAllVulnerabilities(ctx, "env-1", pagination.QueryParams{- Params: pagination.Params{Start: 0, Limit: 20},- })- require.NoError(t, err)- require.Empty(t, items)- require.Equal(t, int64(0), page.TotalItems)- require.Equal(t, int64(0), page.GrandTotalItems)-}--func TestVulnerabilityService_ListAllVulnerabilityImageOptions_SeverityFilter(t *testing.T) {- ctx := context.Background()- db := setupVulnerabilityScanTestDB(t)- svc := &VulnerabilityService{db: db}-- now := time.Now()- records := []models.VulnerabilityScanRecord{- {ID: "sha256:critical-a", ImageName: "zeta:latest", Status: models.ScanStatusCompleted, ScanTime: now, CriticalCount: 2, TotalCount: 2},- {ID: "sha256:high-a", ImageName: "alpha:latest", Status: models.ScanStatusCompleted, ScanTime: now, HighCount: 4, TotalCount: 4},- {ID: "sha256:critical-b", ImageName: "", Status: models.ScanStatusCompleted, ScanTime: now, CriticalCount: 1, TotalCount: 1},- {ID: "sha256:critical-c", ImageName: "zeta:latest", Status: models.ScanStatusCompleted, ScanTime: now, CriticalCount: 1, TotalCount: 1},- {ID: "sha256:failed", ImageName: "failed:latest", Status: models.ScanStatusFailed, ScanTime: now, CriticalCount: 9, TotalCount: 9},- {ID: "sha256:zero", ImageName: "zero:latest", Status: models.ScanStatusCompleted, ScanTime: now, CriticalCount: 1, TotalCount: 0},- }-- for i := range records {- require.NoError(t, db.Create(&records[i]).Error)- }-- items, err := svc.ListAllVulnerabilityImageOptions(ctx, "critical")- require.NoError(t, err)- require.Equal(t, []string{"sha256:critical-b", "zeta:latest"}, items)-}--func TestVulnerabilityService_ListAllVulnerabilityImageOptions_InvalidSeverityAndNoFilter(t *testing.T) {- ctx := context.Background()- db := setupVulnerabilityScanTestDB(t)- svc := &VulnerabilityService{db: db}-- now := time.Now()- records := []models.VulnerabilityScanRecord{- {ID: "sha256:one", ImageName: "beta:latest", Status: models.ScanStatusCompleted, ScanTime: now, HighCount: 1, TotalCount: 1},- {ID: "sha256:two", ImageName: "", Status: models.ScanStatusCompleted, ScanTime: now, MediumCount: 2, TotalCount: 2},- {ID: "sha256:three", ImageName: "alpha:latest", Status: models.ScanStatusCompleted, ScanTime: now, LowCount: 1, TotalCount: 1},- }-- for i := range records {- require.NoError(t, db.Create(&records[i]).Error)- }-- invalid, err := svc.ListAllVulnerabilityImageOptions(ctx, "not-a-severity")- require.NoError(t, err)- require.Empty(t, invalid)-- all, err := svc.ListAllVulnerabilityImageOptions(ctx, "")- require.NoError(t, err)- require.Equal(t, []string{"alpha:latest", "beta:latest", "sha256:two"}, all)-}diff --git a/backend/internal/services/session_service.go b/backend/internal/session/service.gosimilarity index 85%rename from backend/internal/services/session_service.gorename to backend/internal/session/service.goindex c09c524a3..8fe4d4abe 100644--- a/backend/internal/services/session_service.go+++ b/backend/internal/session/service.go@@ -1,4 +1,4 @@-package services+package sessionimport ("context"@@ -10,6 +10,7 @@ import ("emperror.dev/errors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/dbutil"@@ -77,13 +78,13 @@ func (s *SessionService) CreateFederatedSession(ctx context.Context, userID strifunc (s *SessionService) GetSessionByID(ctx context.Context, sessionID string) (*models.UserSession, error) {if strings.TrimSpace(sessionID) == "" {- return nil, ErrInvalidToken+ return nil, common.ErrInvalidToken}var session models.UserSessionif err := s.db.WithContext(ctx).Where("id = ?", sessionID).First(&session).Error; err != nil {if errors.Is(err, gorm.ErrRecordNotFound) {- return nil, ErrInvalidToken+ return nil, common.ErrInvalidToken}return nil, errors.WrapIf(err, "failed to get user session")}@@ -92,7 +93,7 @@ func (s *SessionService) GetSessionByID(ctx context.Context, sessionID string) (func (s *SessionService) RotateRefreshToken(ctx context.Context, sessionID string, refreshJTI string, meta auth.SessionMeta) (*models.UserSession, string, error) {if strings.TrimSpace(sessionID) == "" || strings.TrimSpace(refreshJTI) == "" {- return nil, "", ErrInvalidToken+ return nil, "", common.ErrInvalidToken}newRefreshJTI := uuid.NewString()@@ -105,15 +106,15 @@ func (s *SessionService) RotateRefreshToken(ctx context.Context, sessionID strinvar session models.UserSessionif err := tx.Where("id = ?", sessionID).First(&session).Error; err != nil {if errors.Is(err, gorm.ErrRecordNotFound) {- return ErrInvalidToken+ return common.ErrInvalidToken}return errors.WrapIf(err, "failed to get user session for rotation")}- if err := validateSessionActiveInternal(&session); err != nil {+ if err := ValidateActive(&session); err != nil {return err}if subtle.ConstantTimeCompare([]byte(session.RefreshTokenHash), []byte(hashRefreshJTIInternal(refreshJTI))) != 1 {- return ErrInvalidToken+ return common.ErrInvalidToken}updates := map[string]any{@@ -130,7 +131,7 @@ func (s *SessionService) RotateRefreshToken(ctx context.Context, sessionID strinreturn errors.WrapIf(result.Error, "failed to rotate refresh token")}if result.RowsAffected != 1 {- return ErrInvalidToken+ return common.ErrInvalidToken}rotated = sessionreturn nil@@ -182,12 +183,13 @@ func hashRefreshJTIInternal(jti string) string {// RevokeAllUserSessionsExcept revokes every active session for userID, leaving// exceptSessionID active. Pass "" to revoke all sessions.func (s *SessionService) RevokeAllUserSessionsExcept(ctx context.Context, userID, exceptSessionID string) error {- return revokeAllUserSessionsExceptInternal(ctx, s.db.DB, userID, exceptSessionID)+ return RevokeAllUserSessionsExceptInDB(ctx, s.db.DB, userID, exceptSessionID)}-func revokeAllUserSessionsExceptInternal(ctx context.Context, db *gorm.DB, userID, exceptSessionID string) error {+// RevokeAllUserSessionsExceptInDB revokes sessions using the supplied transaction or database handle.+func RevokeAllUserSessionsExceptInDB(ctx context.Context, db *gorm.DB, userID, exceptSessionID string) error {if strings.TrimSpace(userID) == "" {- return ErrInvalidToken+ return common.ErrInvalidToken}now := time.Now()@@ -201,3 +203,17 @@ func revokeAllUserSessionsExceptInternal(ctx context.Context, db *gorm.DB, userI}return nil}++// ValidateActive verifies that a persisted user session can still authorize requests.+func ValidateActive(userSession *models.UserSession) error {+ if userSession == nil {+ return common.ErrInvalidToken+ }+ if userSession.RevokedAt != nil {+ return common.Classify(common.ErrSessionRevoked, errors.New("Session has been revoked"))+ }+ if time.Now().After(userSession.ExpiresAt) {+ return common.ErrExpiredToken+ }+ return nil+}diff --git a/backend/internal/services/session_service_test.go b/backend/internal/session/service_test.gosimilarity index 75%rename from backend/internal/services/session_service_test.gorename to backend/internal/session/service_test.goindex 9166eaea7..08a01eba4 100644--- a/backend/internal/services/session_service_test.go+++ b/backend/internal/session/service_test.go@@ -1,20 +1,23 @@-package services+package sessionimport ("context""testing""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/types/v2/auth"+ "github.com/libtnb/sqlite""github.com/stretchr/testify/require"+ "gorm.io/gorm")func TestSessionService_RotateRefreshTokenRequiresCurrentHash(t *testing.T) {ctx := context.Background()db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)- require.NoError(t, userSvc.db.Create(&models.User{+ require.NoError(t, db.Create(&models.User{BaseModel: models.BaseModel{ID: "u-session"},Username: "session-user",}).Error)@@ -30,14 +33,13 @@ func TestSessionService_RotateRefreshTokenRequiresCurrentHash(t *testing.T) {require.Equal(t, hashRefreshJTIInternal(newJTI), rotated.RefreshTokenHash)_, _, err = sessionSvc.RotateRefreshToken(ctx, session.ID, refreshJTI, auth.SessionMeta{})- require.ErrorIs(t, err, ErrInvalidToken)+ require.ErrorIs(t, err, common.ErrInvalidToken)}func TestSessionService_DeleteExpiredSessions(t *testing.T) {ctx := context.Background()db := setupAuthServiceTestDB(t)- userSvc := NewUserService(db)- require.NoError(t, userSvc.db.Create(&models.User{+ require.NoError(t, db.Create(&models.User{BaseModel: models.BaseModel{ID: "u-cleanup"},Username: "cleanup-user",}).Error)@@ -70,3 +72,21 @@ func TestSessionService_DeleteExpiredSessions(t *testing.T) {Count(&deletedCount).Error)require.Zero(t, deletedCount)}++func setupAuthServiceTestDB(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(+ &models.SettingVariable{},+ &models.User{},+ &models.UserSession{},+ &models.Environment{},+ &models.Role{},+ &models.UserRoleAssignment{},+ &models.ApiKey{},+ &models.ApiKeyPermission{},+ &models.OidcRoleMapping{},+ ))+ return &database.DB{DB: db}+}diff --git a/backend/api/handlers/settings.go b/backend/internal/settings/handler.gosimilarity index 78%rename from backend/api/handlers/settings.gorename to backend/internal/settings/handler.goindex 5d630f30f..421f9cc3a 100644--- a/backend/api/handlers/settings.go+++ b/backend/internal/settings/handler.go@@ -1,4 +1,4 @@-package handlers+package settingsimport ("context"@@ -8,27 +8,30 @@ import ("strconv""strings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/search"+"emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge""github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/category"- "github.com/getarcaneapp/arcane/types/v2/search"- "github.com/getarcaneapp/arcane/types/v2/settings"+ searchtypes "github.com/getarcaneapp/arcane/types/v2/search"+ settingstypes "github.com/getarcaneapp/arcane/types/v2/settings")// SettingsHandler provides Huma-based settings management endpoints.type SettingsHandler struct {- settingsService *services.SettingsService- settingsSearchService *services.SettingsSearchService- environmentService *services.EnvironmentService+ settingsService *SettingsService+ settingsSearchService *search.SettingsSearchService+ proxyRemoteJSON handlerutil.RemoteJSONProxycfg *config.Config}@@ -39,7 +42,7 @@ type GetSettingsInput struct {}type GetSettingsOutput struct {- Body []settings.PublicSetting+ Body []settingstypes.PublicSetting}type GetPublicSettingsInput struct {@@ -47,24 +50,24 @@ type GetPublicSettingsInput struct {}type GetPublicSettingsOutput struct {- Body []settings.PublicSetting+ Body []settingstypes.PublicSetting}type UpdateSettingsInput struct {- EnvironmentID string `path:"id" doc:"Environment ID"`- Body settings.Update `doc:"Settings update data"`+ EnvironmentID string `path:"id" doc:"Environment ID"`+ Body settingstypes.Update `doc:"Settings update data"`}type UpdateSettingsOutput struct {- Body base.ApiResponse[[]settings.SettingDto]+ Body base.ApiResponse[[]settingstypes.SettingDto]}type SearchSettingsInput struct {- Body search.Request `doc:"Search query"`+ Body searchtypes.Request `doc:"Search query"`}type SearchSettingsOutput struct {- Body search.Response+ Body searchtypes.Response}type GetCategoriesOutput struct {@@ -112,11 +115,11 @@ func validateAbsoluteDirectoryPathInternal(path string) error {}// RegisterSettings registers settings management routes using Huma.-func RegisterSettings(api huma.API, settingsService *services.SettingsService, settingsSearchService *services.SettingsSearchService, environmentService *services.EnvironmentService, cfg *config.Config) {+func RegisterSettings(api huma.API, settingsService *SettingsService, settingsSearchService *search.SettingsSearchService, proxyRemoteJSON handlerutil.RemoteJSONProxy, cfg *config.Config) {h := &SettingsHandler{settingsService: settingsService,settingsSearchService: settingsSearchService,- environmentService: environmentService,+ proxyRemoteJSON: proxyRemoteJSON,cfg: cfg,}@@ -131,24 +134,24 @@ func RegisterSettings(api huma.API, settingsService *services.SettingsService, sSecurity: []map[string][]string{},}, h.GetPublicSettings)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-settings",Method: http.MethodGet,Path: "/environments/{id}/settings",Summary: "Get settings",Description: "Get all settings for an environment",Tags: []string{"Settings"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermSettingsRead, h.GetSettings)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-settings",Method: http.MethodPut,Path: "/environments/{id}/settings",Summary: "Update settings",Description: "Update settings for an environment",Tags: []string{"Settings"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermSettingsWrite, h.UpdateSettings)// Top-level settings endpoints (not environment-scoped)@@ -159,7 +162,7 @@ func RegisterSettings(api huma.API, settingsService *services.SettingsService, sSummary: "Search settings",Description: "Search settings categories and individual settings by query",Tags: []string{"Settings"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.Search)huma.Register(api, huma.Operation{@@ -169,7 +172,7 @@ func RegisterSettings(api huma.API, settingsService *services.SettingsService, sSummary: "Get settings categories",Description: "Get all available settings categories with metadata",Tags: []string{"Settings"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, h.GetCategories)}@@ -201,7 +204,7 @@ func canAccessSettingsCategoryAtAnyScopeInternal(ps *authz.PermissionSet, categoreturn false}-func (h *SettingsHandler) appendRuntimeSettingsInternal(settingsDto []settings.PublicSetting, includeAuthenticatedOnly bool) []settings.PublicSetting {+func (h *SettingsHandler) appendRuntimeSettingsInternal(settingsDto []settingstypes.PublicSetting, includeAuthenticatedOnly bool) []settingstypes.PublicSetting {if !includeAuthenticatedOnly {return settingsDto}@@ -210,7 +213,7 @@ func (h *SettingsHandler) appendRuntimeSettingsInternal(settingsDto []settings.Pif h.cfg != nil {uiConfigDisabled = h.cfg.UIConfigurationDisabled}- settingsDto = append(settingsDto, settings.PublicSetting{+ settingsDto = append(settingsDto, settingstypes.PublicSetting{Key: "uiConfigDisabled",Value: strconv.FormatBool(uiConfigDisabled),Type: "boolean",@@ -220,22 +223,31 @@ func (h *SettingsHandler) appendRuntimeSettingsInternal(settingsDto []settings.Pif h.cfg != nil && strings.TrimSpace(h.cfg.BackupVolumeName) != "" {backupVolumeName = h.cfg.BackupVolumeName}- settingsDto = append(settingsDto, settings.PublicSetting{+ settingsDto = append(settingsDto, settingstypes.PublicSetting{Key: "backupVolumeName",Value: backupVolumeName,Type: "string",})- settingsDto = append(settingsDto, settings.PublicSetting{+ var edgeCfg *edge.Config+ if h.cfg != nil {+ edgeCfg = &edge.Config{+ EdgeMTLSMode: h.cfg.EdgeMTLSMode,+ EdgeMTLSCAFile: h.cfg.EdgeMTLSCAFile,+ EdgeMTLSAssetsDir: h.cfg.EdgeMTLSAssetsDir,+ }+ }+ _, edgeMTLSCAErr := edge.AvailableManagerMTLSCAPath(edgeCfg)+ settingsDto = append(settingsDto, settingstypes.PublicSetting{Key: "edgeMTLSManagerCAAvailable",- Value: strconv.FormatBool(hasGeneratedEdgeMTLSCAInternal(h.cfg)),+ Value: strconv.FormatBool(edgeMTLSCAErr == nil),Type: "boolean",})if h.settingsService != nil {cfg := h.settingsService.GetSettingsConfig()depotConfigured := strings.TrimSpace(cfg.DepotProjectId.Value) != "" && strings.TrimSpace(cfg.DepotToken.Value) != ""- settingsDto = append(settingsDto, settings.PublicSetting{+ settingsDto = append(settingsDto, settingstypes.PublicSetting{Key: "depotConfigured",Value: strconv.FormatBool(depotConfigured),Type: "boolean",@@ -248,7 +260,7 @@ func (h *SettingsHandler) appendRuntimeSettingsInternal(settingsDto []settings.P// GetPublicSettings returns public settings for an environment.func (h *SettingsHandler) GetPublicSettings(ctx context.Context, input *GetPublicSettingsInput) (*GetPublicSettingsOutput, error) {if input.EnvironmentID != "0" {- settingsDto, err := proxyRemoteJSONInternal[[]settings.PublicSetting](ctx, h.environmentService, input.EnvironmentID, http.MethodGet, "/api/environments/0/settings/public", nil)+ settingsDto, err := handlerutil.ProxyRemoteJSON[[]settingstypes.PublicSetting](ctx, h.proxyRemoteJSON, input.EnvironmentID, http.MethodGet, "/api/environments/0/settings/public", nil)if err != nil {return nil, err}@@ -257,7 +269,7 @@ func (h *SettingsHandler) GetPublicSettings(ctx context.Context, input *GetPublisettingsList := h.settingsService.ListSettings(models.SettingVisibilityPublic)- var settingsDto []settings.PublicSetting+ var settingsDto []settingstypes.PublicSettingif err := mapper.MapStructList(settingsList, &settingsDto); err != nil {return nil, huma.Error500InternalServerError("Failed to map settings")}@@ -267,7 +279,7 @@ func (h *SettingsHandler) GetPublicSettings(ctx context.Context, input *GetPubli// GetSettings returns all settings for an environment.func (h *SettingsHandler) GetSettings(ctx context.Context, input *GetSettingsInput) (*GetSettingsOutput, error) {- ps, _ := humamw.PermissionsFromContext(ctx)+ ps, _ := middleware.PermissionsFromContext(ctx)isAdmin := ps.IsGlobalAdmin()visibility := models.SettingVisibilityNonAdminif isAdmin {@@ -275,7 +287,7 @@ func (h *SettingsHandler) GetSettings(ctx context.Context, input *GetSettingsInp}if input.EnvironmentID != "0" {- settingsDto, err := proxyRemoteJSONInternal[[]settings.PublicSetting](ctx, h.environmentService, input.EnvironmentID, http.MethodGet, "/api/environments/0/settings", nil)+ settingsDto, err := handlerutil.ProxyRemoteJSON[[]settingstypes.PublicSetting](ctx, h.proxyRemoteJSON, input.EnvironmentID, http.MethodGet, "/api/environments/0/settings", nil)if err != nil {return nil, err}@@ -284,7 +296,7 @@ func (h *SettingsHandler) GetSettings(ctx context.Context, input *GetSettingsInpfor _, setting := range h.settingsService.ListSettings(visibility) {allowedKeys[setting.Key] = struct{}{}}- filtered := make([]settings.PublicSetting, 0, len(*settingsDto))+ filtered := make([]settingstypes.PublicSetting, 0, len(*settingsDto))for _, setting := range *settingsDto {if _, ok := allowedKeys[setting.Key]; ok {filtered = append(filtered, setting)@@ -297,7 +309,7 @@ func (h *SettingsHandler) GetSettings(ctx context.Context, input *GetSettingsInpsettingsList := h.settingsService.ListSettings(visibility)- var settingsDto []settings.PublicSetting+ var settingsDto []settingstypes.PublicSettingif err := mapper.MapStructList(settingsList, &settingsDto); err != nil {return nil, huma.Error500InternalServerError("Failed to map settings")}@@ -318,7 +330,7 @@ func (h *SettingsHandler) UpdateSettings(ctx context.Context, input *UpdateSettireturn h.updateSettingsForLocalEnvironment(ctx, input.Body)}-func (h *SettingsHandler) validateSettingsUpdateInput(input settings.Update) error {+func (h *SettingsHandler) validateSettingsUpdateInput(input settingstypes.Update) error {// Validate projects directory if provided and changed from current value.// Skip validation when the value matches the current (possibly env-overridden) setting// so that saving unrelated settings doesn't fail due to env-provided directory formats.@@ -356,7 +368,7 @@ func (h *SettingsHandler) updateSettingsForRemoteEnvironment(ctx context.Contextreturn nil, huma.Error403Forbidden("Authentication settings can only be updated from the main environment")}- apiResp, err := proxyRemoteJSONInternal[base.ApiResponse[[]settings.SettingDto]](ctx, h.environmentService, input.EnvironmentID, http.MethodPut, "/api/environments/0/settings", input.Body)+ apiResp, err := handlerutil.ProxyRemoteJSON[base.ApiResponse[[]settingstypes.SettingDto]](ctx, h.proxyRemoteJSON, input.EnvironmentID, http.MethodPut, "/api/environments/0/settings", input.Body)if err != nil {return nil, err}@@ -364,7 +376,7 @@ func (h *SettingsHandler) updateSettingsForRemoteEnvironment(ctx context.Contextreturn &UpdateSettingsOutput{Body: *apiResp}, nil}-func (h *SettingsHandler) updateSettingsForLocalEnvironment(ctx context.Context, input settings.Update) (*UpdateSettingsOutput, error) {+func (h *SettingsHandler) updateSettingsForLocalEnvironment(ctx context.Context, input settingstypes.Update) (*UpdateSettingsOutput, error) {if input.ProjectsDirectory != nil && *input.ProjectsDirectory != "" {currentDir := h.settingsService.GetSettingsConfig().ProjectsDirectory.Valueif *input.ProjectsDirectory != currentDir {@@ -391,10 +403,10 @@ func (h *SettingsHandler) updateSettingsForLocalEnvironment(ctx context.Context,return nil, huma.NewError(apiErr.HTTPStatus(), apiErr.Message)}- settingDtos := make([]settings.SettingDto, 0, len(updatedSettings))+ settingDtos := make([]settingstypes.SettingDto, 0, len(updatedSettings))for _, setting := range updatedSettings {- settingDtos = append(settingDtos, settings.SettingDto{- PublicSetting: settings.PublicSetting{+ settingDtos = append(settingDtos, settingstypes.SettingDto{+ PublicSetting: settingstypes.PublicSetting{Key: setting.Key,Type: "string",Value: setting.Value,@@ -403,14 +415,14 @@ func (h *SettingsHandler) updateSettingsForLocalEnvironment(ctx context.Context,}return &UpdateSettingsOutput{- Body: base.ApiResponse[[]settings.SettingDto]{+ Body: base.ApiResponse[[]settingstypes.SettingDto]{Success: true,Data: settingDtos,},}, nil}-func hasAuthSettingsUpdateInternal(req settings.Update) bool {+func hasAuthSettingsUpdateInternal(req settingstypes.Update) bool {return req.AuthLocalEnabled != nil || req.OidcEnabled != nil ||req.AuthSessionTimeout != nil || req.AuthPasswordPolicy != nil ||req.OidcClientId != nil ||@@ -428,7 +440,7 @@ func (h *SettingsHandler) Search(ctx context.Context, input *SearchSettingsInputreturn nil, huma.Error400BadRequest("Query parameter is required")}- ps, _ := humamw.PermissionsFromContext(ctx)+ ps, _ := middleware.PermissionsFromContext(ctx)results := h.settingsSearchService.Search(input.Body.Query)results.Results = filterSettingsCategoriesInternal(ps, results.Results)results.Count = len(results.Results)@@ -437,7 +449,7 @@ func (h *SettingsHandler) Search(ctx context.Context, input *SearchSettingsInput// GetCategories returns all available settings categories.func (h *SettingsHandler) GetCategories(ctx context.Context, input *struct{}) (*GetCategoriesOutput, error) {- ps, _ := humamw.PermissionsFromContext(ctx)+ ps, _ := middleware.PermissionsFromContext(ctx)categories := filterSettingsCategoriesInternal(ps, h.settingsSearchService.GetSettingsCategories())return &GetCategoriesOutput{Body: categories}, nil}diff --git a/backend/internal/settings/module.go b/backend/internal/settings/module.gonew file mode 100644index 000000000..46cef6cdf--- /dev/null+++ b/backend/internal/settings/module.go@@ -0,0 +1,44 @@+// Package settings owns persisted application settings and their HTTP surface.+package settings++import (+ "github.com/danielgtaylor/huma/v2"+ "github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/search"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+)++// Dependencies are the handler-side collaborators for the settings domain.+type Dependencies struct {+ Search *search.SettingsSearchService+ ProxyRemoteJSON handlerutil.RemoteJSONProxy+ Config *config.Config+}++// Module joins the settings service with its route dependencies.+type Module struct {+ service *SettingsService+ deps Dependencies+}++// New builds the settings domain around its initialized actor-backed service.+func New(service *SettingsService, deps Dependencies) *Module {+ return &Module{service: service, deps: deps}+}++// Service exposes settings operations to collaborating domains.+func (m *Module) Service() *SettingsService {+ if m == nil {+ return nil+ }+ return m.service+}++// RegisterRoutes mounts settings endpoints for runtime and schema discovery.+func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterSettings(api, nil, nil, nil, nil)+ return+ }+ RegisterSettings(api, m.service, m.deps.Search, m.deps.ProxyRemoteJSON, m.deps.Config)+}diff --git a/backend/internal/services/settings_service.go b/backend/internal/settings/service.gosimilarity index 97%rename from backend/internal/services/settings_service.gorename to backend/internal/settings/service.goindex 2f55ca864..55e8941f0 100644--- a/backend/internal/services/settings_service.go+++ b/backend/internal/settings/service.go@@ -1,4 +1,4 @@-package services+package settingsimport ("context"@@ -28,8 +28,16 @@ import ("github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""github.com/getarcaneapp/arcane/backend/v2/pkg/projects""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/validation"- "github.com/getarcaneapp/arcane/types/v2/settings"+ settingstypes "github.com/getarcaneapp/arcane/types/v2/settings"+)++const (+ // DefaultArcaneToolsImage is the shared Arcane toolbox image used for helper commands.+ DefaultArcaneToolsImage = "ghcr.io/getarcaneapp/tools:latest"+ // DefaultTrivyImage is the default vulnerability scanner image setting.+ DefaultTrivyImage = DefaultArcaneToolsImage)type SettingsService struct {@@ -415,9 +423,9 @@ func resolveSettingsEnvOverridesInternal() []settingsEnvOverride {return overrides}-// isEnvOverrideActiveInternal returns true when the given setting key has an envOverride tag+// IsEnvOverrideActive reports whether an environment override currently controls the setting.// and its corresponding environment variable is currently set to a non-empty value.-func (s *SettingsService) isEnvOverrideActiveInternal(key string) bool {+func (s *SettingsService) IsEnvOverrideActive(key string) bool {for _, override := range s.envOverrides {if override.key == key {return true@@ -494,7 +502,7 @@ func (s *SettingsService) updateSettingValueNoRefreshInternal(ctx context.Contex// UpdateSettings publishes the refreshed snapshot before returning. Each// subscriber's effects remain ordered and may finish after this method returns.-func (s *SettingsService) UpdateSettings(ctx context.Context, updates settings.Update) ([]models.SettingVariable, error) {+func (s *SettingsService) UpdateSettings(ctx context.Context, updates settingstypes.Update) ([]models.SettingVariable, error) {result, err := actors.Execute(ctx, s.writes, "update settings", func(writeCtx context.Context) (settingsUpdateResultInternal, error) {return s.updateSettingsInternal(writeCtx, updates)}, func(result settingsUpdateResultInternal, err error) {@@ -523,12 +531,12 @@ func (s *SettingsService) publishSettingsChangesInternal(updates []libarcane.Set}}-func (s *SettingsService) updateSettingsInternal(ctx context.Context, updates settings.Update) (settingsUpdateResultInternal, error) {+func (s *SettingsService) updateSettingsInternal(ctx context.Context, updates settingstypes.Update) (settingsUpdateResultInternal, error) {defaultCfg := s.getDefaultSettings()cfg := s.GetSettingsConfig().Clone()- trivyServerTokenUpdated := updates.TrivyServerToken != nil && *updates.TrivyServerToken != "" && !s.isEnvOverrideActiveInternal("trivyServerToken")+ trivyServerTokenUpdated := updates.TrivyServerToken != nil && *updates.TrivyServerToken != "" && !s.IsEnvOverrideActive("trivyServerToken")normalizeTargetURL := func(value string) string {- normalized, err := normalizeEnvironmentBaseURLInternal(value)+ normalized, err := httpx.NormalizeBaseURL(value)if err != nil {return strings.TrimSpace(value)}@@ -586,8 +594,8 @@ func (s *SettingsService) updateSettingsInternal(ctx context.Context, updates se}, nil}-func (s *SettingsService) prepareUpdateValues(updates settings.Update, cfg, defaultCfg *models.Settings) ([]models.SettingVariable, error) {- rt := reflect.TypeFor[settings.Update]()+func (s *SettingsService) prepareUpdateValues(updates settingstypes.Update, cfg, defaultCfg *models.Settings) ([]models.SettingVariable, error) {+ rt := reflect.TypeFor[settingstypes.Update]()rv := reflect.ValueOf(updates)valuesToUpdate := make([]models.SettingVariable, 0)diff --git a/backend/internal/services/settings_service_test.go b/backend/internal/settings/service_test.gosimilarity index 94%rename from backend/internal/services/settings_service_test.gorename to backend/internal/settings/service_test.goindex 094b92265..bbc3a79f4 100644--- a/backend/internal/services/settings_service_test.go+++ b/backend/internal/settings/service_test.go@@ -1,4 +1,4 @@-package services+package settingsimport ("context"@@ -19,7 +19,7 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"- "github.com/getarcaneapp/arcane/types/v2/settings"+ settingstypes "github.com/getarcaneapp/arcane/types/v2/settings")func setupSettingsTestDB(t *testing.T) *database.DB {@@ -150,7 +150,7 @@ func TestSettingsService_ImageEventWatcherSettingPersists(t *testing.T) {require.NoError(t, svc.EnsureDefaultSettings(ctx))require.False(t, svc.GetBoolSetting(ctx, "imageEventWatcherEnabled", true))- _, err = svc.UpdateSettings(ctx, settings.Update{ImageEventWatcherEnabled: new("true")})+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{ImageEventWatcherEnabled: new("true")})require.NoError(t, err)require.True(t, svc.GetBoolSetting(ctx, "imageEventWatcherEnabled", false))@@ -213,7 +213,7 @@ func TestSettingsService_AvatarMaxUploadSizeDefaultAndUpdate(t *testing.T) {require.Equal(t, "2", current.AvatarMaxUploadSizeMb.Value)updatedValue := "8"- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{AvatarMaxUploadSizeMb: &updatedValue,})require.NoError(t, err)@@ -231,7 +231,7 @@ func TestSettingsServiceUpdateSettingsRejectsOIDCIssuerChangeWithStoredSecretIntrequire.NoError(t, svc.UpdateSetting(ctx, "oidcIssuerUrl", "https://issuer.example.com"))require.NoError(t, svc.UpdateSetting(ctx, "oidcClientSecret", "old-client-secret"))- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{OidcIssuerUrl: new("https://attacker.example.com"),OidcClientSecret: new(""),})@@ -251,7 +251,7 @@ func TestSettingsServiceUpdateSettingsAllowsOIDCIssuerChangeWithReplacementSecrerequire.NoError(t, svc.UpdateSetting(ctx, "oidcIssuerUrl", "https://issuer.example.com"))require.NoError(t, svc.UpdateSetting(ctx, "oidcClientSecret", "old-client-secret"))- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{OidcIssuerUrl: new("https://replacement.example.com"),OidcClientSecret: new("new-client-secret"),})@@ -271,7 +271,7 @@ func TestSettingsServiceUpdateSettingsRejectsTrivyServerChangeWithStoredTokenIntrequire.NoError(t, svc.UpdateSetting(ctx, "trivyServerUrl", "https://trivy.example.com"))require.NoError(t, svc.UpdateSetting(ctx, "trivyServerToken", "old-trivy-token"))- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyServerUrl: new("https://attacker.example.com"),})require.ErrorIs(t, err, common.ErrValidation)@@ -290,7 +290,7 @@ func TestSettingsServiceUpdateSettingsAllowsTrivyServerChangeWithReplacementTokerequire.NoError(t, svc.UpdateSetting(ctx, "trivyServerUrl", "https://trivy.example.com"))require.NoError(t, svc.UpdateSetting(ctx, "trivyServerToken", "old-trivy-token"))- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyServerUrl: new("https://replacement.example.com"),TrivyServerToken: new("new-trivy-token"),})@@ -310,7 +310,7 @@ func TestSettingsServiceUpdateSettingsAllowsClearingTrivyServerWithStoredTokenInrequire.NoError(t, svc.UpdateSetting(ctx, "trivyServerUrl", "https://trivy.example.com"))require.NoError(t, svc.UpdateSetting(ctx, "trivyServerToken", "old-trivy-token"))- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyServerUrl: new(""),})require.NoError(t, err)@@ -328,7 +328,7 @@ func TestSettingsServiceUpdateSettingsAllowsTrivyServerChangeWithoutStoredTokenIrequire.NoError(t, err)require.NoError(t, svc.UpdateSetting(ctx, "trivyServerUrl", "https://trivy.example.com"))- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyServerUrl: new("https://replacement.example.com"),})require.NoError(t, err)@@ -472,17 +472,17 @@ func TestSettingsService_isEnvOverrideActiveInternal(t *testing.T) {svc, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- require.False(t, svc.isEnvOverrideActiveInternal("oidcEnabled"))+ require.False(t, svc.IsEnvOverrideActive("oidcEnabled"))t.Setenv("OIDC_ENABLED", "false")svcWithOverride, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- require.True(t, svcWithOverride.isEnvOverrideActiveInternal("oidcEnabled"))+ require.True(t, svcWithOverride.IsEnvOverrideActive("oidcEnabled"))t.Setenv("AUTH_SESSION_TIMEOUT", "120")svcWithNonOverrideEnv, err := newSettingsServiceForTestInternal(t, ctx, db)require.NoError(t, err)- require.False(t, svcWithNonOverrideEnv.isEnvOverrideActiveInternal("authSessionTimeout"))+ require.False(t, svcWithNonOverrideEnv.IsEnvOverrideActive("authSessionTimeout"))}func TestSettingsService_GetSetHelpers(t *testing.T) {@@ -561,7 +561,7 @@ func TestSettingsService_UpdateSettings_PruneModesDoNotTriggerScheduledPruneCallcallbackCalls++})- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{PruneImageMode: new("all"),PruneContainerUntil: new("24h"),})@@ -581,7 +581,7 @@ func TestSettingsService_UpdateSettings_ScheduledPruneScheduleTriggersCallback(tcallbackCalls++})- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{ScheduledPruneEnabled: new("true"),})require.NoError(t, err)@@ -600,12 +600,12 @@ func TestSettingsServiceActorSerializesConcurrentUpdatesInternal(t *testing.T) {results := make(chan error, 2)go func() {<-start- _, updateErr := svc.UpdateSettings(ctx, settings.Update{ProjectsDirectory: new("/data/projects-a")})+ _, updateErr := svc.UpdateSettings(ctx, settingstypes.Update{ProjectsDirectory: new("/data/projects-a")})results <- updateErr}()go func() {<-start- _, updateErr := svc.UpdateSettings(ctx, settings.Update{TemplatesDirectory: new("/data/templates-b")})+ _, updateErr := svc.UpdateSettings(ctx, settingstypes.Update{TemplatesDirectory: new("/data/templates-b")})results <- updateErr}()close(start)@@ -633,7 +633,7 @@ func TestSettingsServiceActorPublishesSnapshotBeforeAsynchronousNotificationInteupdateResult := make(chan error, 1)go func() {- _, updateErr := svc.UpdateSettings(ctx, settings.Update{BaseServerURL: new("https://actor.example")})+ _, updateErr := svc.UpdateSettings(ctx, settingstypes.Update{BaseServerURL: new("https://actor.example")})updateResult <- updateErr}()@@ -647,7 +647,7 @@ func TestSettingsServiceActorPublishesSnapshotBeforeAsynchronousNotificationIntesecondUpdate := make(chan error, 1)go func() {- _, updateErr := svc.UpdateSettings(ctx, settings.Update{ProjectsDirectory: new("/data/unblocked")})+ _, updateErr := svc.UpdateSettings(ctx, settingstypes.Update{ProjectsDirectory: new("/data/unblocked")})secondUpdate <- updateErr}()select {@@ -673,7 +673,7 @@ func TestSettingsServiceActorNotifiesSubscriberOnceForMultipleMatchingKeysInternnotified <- updates})- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{PollingEnabled: new("false"),PollingInterval: new("0 */5 * * * *"),})@@ -693,7 +693,7 @@ func TestSettingsServiceActorDoesNotPublishSensitiveValuesInternal(t *testing.T)svc.SubscribeSettingsChanges([]string{"oidcClientSecret"}, func([]libarcane.SettingUpdate) {calls.Add(1)})- _, err = svc.UpdateSettings(ctx, settings.Update{OidcClientSecret: new("should-not-leave-settings-service")})+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{OidcClientSecret: new("should-not-leave-settings-service")})require.NoError(t, err)waitForSettingsNotificationsInternal(t, svc)require.Zero(t, calls.Load())@@ -835,7 +835,7 @@ func TestSettingsService_UpdateSettings_RefreshesCache(t *testing.T) {require.NoError(t, svc.EnsureDefaultSettings(ctx))newDir := "custom/projects2"- req := settings.Update{+ req := settingstypes.Update{ProjectsDirectory: &newDir,}@@ -863,7 +863,7 @@ func TestSettingsService_UpdateSettings_ReturnsEnvOverriddenValues(t *testing.T)require.NoError(t, err)require.NoError(t, svc.EnsureDefaultSettings(ctx))- settingsList, err := svc.UpdateSettings(ctx, settings.Update{+ settingsList, err := svc.UpdateSettings(ctx, settingstypes.Update{ProjectsDirectory: new("custom/projects2"),})require.NoError(t, err)@@ -890,7 +890,7 @@ func TestSettingsService_UpdateSettings_TimeoutCallbackIncludesTrivyScanTimeout(callbackPayload = timeoutSettings})- _, err = svc.UpdateSettings(ctx, settings.Update{TrivyScanTimeout: new("1200")})+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyScanTimeout: new("1200")})require.NoError(t, err)waitForSettingsNotificationsInternal(t, svc)@@ -910,7 +910,7 @@ func TestSettingsService_UpdateSettings_TimeoutCallbackIncludesTrivyResourceLimicallbackPayload = timeoutSettings})- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyResourceLimitsEnabled: new("false"),TrivyCpuLimit: new("2.5"),TrivyMemoryLimitMb: new("3072"),@@ -936,7 +936,7 @@ func TestSettingsService_UpdateSettings_TimeoutCallbackIncludesTrivyConcurrentSccallbackPayload = timeoutSettings})- _, err = svc.UpdateSettings(ctx, settings.Update{TrivyConcurrentScanContainers: new("4")})+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyConcurrentScanContainers: new("4")})require.NoError(t, err)waitForSettingsNotificationsInternal(t, svc)@@ -956,7 +956,7 @@ func TestSettingsService_UpdateSettings_TrivyNetworkTriggersVulnerabilityCallbaccallbackCalled = true})- _, err = svc.UpdateSettings(ctx, settings.Update{TrivyNetwork: new("arcane-external")})+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyNetwork: new("arcane-external")})require.NoError(t, err)waitForSettingsNotificationsInternal(t, svc)require.True(t, callbackCalled)@@ -974,7 +974,7 @@ func TestSettingsService_UpdateSettings_TrivyNetworkDoesNotTriggerTimeoutCallbaccallbackPayload = timeoutSettings})- _, err = svc.UpdateSettings(ctx, settings.Update{TrivyNetwork: new("arcane-external")})+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyNetwork: new("arcane-external")})require.NoError(t, err)waitForSettingsNotificationsInternal(t, svc)require.Nil(t, callbackPayload)@@ -992,7 +992,7 @@ func TestSettingsService_UpdateSettings_TrivyRuntimeSecurityTriggersVulnerabilitcallbackCalled = true})- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivySecurityOpts: new("label=disable"),TrivyPrivileged: new("true"),})@@ -1013,7 +1013,7 @@ func TestSettingsService_UpdateSettings_TrivyRuntimeSecurityDoesNotTriggerTimeoucallbackPayload = timeoutSettings})- _, err = svc.UpdateSettings(ctx, settings.Update{+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivySecurityOpts: new("label=disable"),TrivyPrivileged: new("true"),})@@ -1029,7 +1029,7 @@ func TestSettingsService_UpdateSettings_TrivyPreserveCacheOnVolumePrunePersists(require.NoError(t, err)require.NoError(t, svc.EnsureDefaultSettings(ctx))- _, err = svc.UpdateSettings(ctx, settings.Update{TrivyPreserveCacheOnVolumePrune: new("false")})+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyPreserveCacheOnVolumePrune: new("false")})require.NoError(t, err)current, err := svc.GetSettings(ctx)@@ -1054,7 +1054,7 @@ func TestSettingsService_UpdateSettings_TrivyPreserveCacheOnVolumePruneDoesNotTrcallbackPayload = timeoutSettings})- _, err = svc.UpdateSettings(ctx, settings.Update{TrivyPreserveCacheOnVolumePrune: new("false")})+ _, err = svc.UpdateSettings(ctx, settingstypes.Update{TrivyPreserveCacheOnVolumePrune: new("false")})require.NoError(t, err)waitForSettingsNotificationsInternal(t, svc)require.Nil(t, callbackPayload)diff --git a/backend/api/handlers/swarm.go b/backend/internal/swarm/handler.gosimilarity index 79%rename from backend/api/handlers/swarm.gorename to backend/internal/swarm/handler.goindex 2497cccf2..a34176bdf 100644--- a/backend/api/handlers/swarm.go+++ b/backend/internal/swarm/handler.go@@ -1,4 +1,4 @@-package handlers+package swarmimport ("context"@@ -10,22 +10,24 @@ import ("emperror.dev/errors""github.com/containerd/errdefs""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"swarmtypes "github.com/getarcaneapp/arcane/types/v2/swarm")type SwarmHandler struct {- swarmService *services.SwarmService- environmentService *services.EnvironmentService- eventService *services.EventService+ swarmService *SwarmService+ environmentService *environment.EnvironmentService+ eventService *event.EventServicecfg *config.Config}@@ -143,7 +145,7 @@ type GetSwarmNodeAgentDeploymentInput struct {}type SwarmNodeAgentDeployment struct {- DeploymentSnippet+ environment.DeploymentSnippetEnvironmentID string `json:"environmentId"`Agent swarmtypes.NodeAgentStatus `json:"agent"`@@ -193,7 +195,7 @@ type DeleteSwarmNodeAgentDeploymentOutput struct {type GetSwarmNodeIdentityInput struct{}type GetSwarmNodeIdentityOutput struct {- Body base.ApiResponse[services.SwarmNodeIdentity]+ Body base.ApiResponse[SwarmNodeIdentity]}type UpdateSwarmNodeInput struct {@@ -541,7 +543,7 @@ type DeleteSwarmSecretOutput struct {// environmentSvc provides environment and agent-deployment helpers used by node endpoints.// eventSvc records audit events for mutating operations when available.// cfg provides application configuration needed by deployment-snippet endpoints.-func RegisterSwarm(api huma.API, swarmSvc *services.SwarmService, environmentSvc *services.EnvironmentService, eventSvc *services.EventService, cfg *config.Config) {+func RegisterSwarm(api huma.API, swarmSvc *SwarmService, environmentSvc *environment.EnvironmentService, eventSvc *event.EventService, cfg *config.Config) {h := &SwarmHandler{swarmService: swarmSvc,environmentService: environmentSvc,@@ -549,63 +551,63 @@ func RegisterSwarm(api huma.API, swarmSvc *services.SwarmService, environmentSvccfg: cfg,}- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-services", Method: http.MethodGet, Path: "/environments/{id}/swarm/services", Summary: "List swarm services", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.ListServices)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-service", Method: http.MethodGet, Path: "/environments/{id}/swarm/services/{serviceId}", Summary: "Get swarm service", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.GetService)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "create-swarm-service", Method: http.MethodPost, Path: "/environments/{id}/swarm/services", Summary: "Create swarm service", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmServices, h.CreateService)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "update-swarm-service", Method: http.MethodPut, Path: "/environments/{id}/swarm/services/{serviceId}", Summary: "Update swarm service", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmServices, h.UpdateService)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-service", Method: http.MethodDelete, Path: "/environments/{id}/swarm/services/{serviceId}", Summary: "Delete swarm service", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmServices, h.DeleteService)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-service-tasks", Method: http.MethodGet, Path: "/environments/{id}/swarm/services/{serviceId}/tasks", Summary: "List tasks for a swarm service", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.ListServiceTasks)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "rollback-swarm-service", Method: http.MethodPost, Path: "/environments/{id}/swarm/services/{serviceId}/rollback", Summary: "Rollback a swarm service", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmServices, h.RollbackService)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "scale-swarm-service", Method: http.MethodPost, Path: "/environments/{id}/swarm/services/{serviceId}/scale", Summary: "Scale a swarm service", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmServices, h.ScaleService)-- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-nodes", Method: http.MethodGet, Path: "/environments/{id}/swarm/nodes", Summary: "List swarm nodes", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.ListNodes)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-node", Method: http.MethodGet, Path: "/environments/{id}/swarm/nodes/{nodeId}", Summary: "Get swarm node", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.GetNode)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-node-agent-deployment", Method: http.MethodPost, Path: "/environments/{id}/swarm/nodes/{nodeId}/agent/deployment", Summary: "Get swarm node agent deployment snippets", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmNodes, h.GetNodeAgentDeployment)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "reconcile-swarm-node-agents", Method: http.MethodPost, Path: "/environments/{id}/swarm/nodes/agents/reconcile", Summary: "Reconcile swarm node agent bindings", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmNodes, h.ReconcileNodeAgents)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "put-swarm-node-agent-binding", Method: http.MethodPut, Path: "/environments/{id}/swarm/nodes/{nodeId}/agent/binding", Summary: "Attach a visible environment to a swarm node", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmNodes, h.PutNodeAgentBinding)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-node-agent-binding", Method: http.MethodDelete, Path: "/environments/{id}/swarm/nodes/{nodeId}/agent/binding", Summary: "Detach a visible environment from a swarm node", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmNodes, h.DeleteNodeAgentBinding)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-node-agent-deployment", Method: http.MethodDelete, Path: "/environments/{id}/swarm/nodes/{nodeId}/agent/deployment", Summary: "Remove a dedicated swarm node agent registration", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmNodes, h.DeleteNodeAgentDeployment)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "update-swarm-node", Method: http.MethodPatch, Path: "/environments/{id}/swarm/nodes/{nodeId}", Summary: "Update swarm node", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmNodes, h.UpdateNode)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-node", Method: http.MethodDelete, Path: "/environments/{id}/swarm/nodes/{nodeId}", Summary: "Delete swarm node", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmNodes, h.DeleteNode)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "promote-swarm-node", Method: http.MethodPost, Path: "/environments/{id}/swarm/nodes/{nodeId}/promote", Summary: "Promote swarm node", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmNodes, h.PromoteNode)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "demote-swarm-node", Method: http.MethodPost, Path: "/environments/{id}/swarm/nodes/{nodeId}/demote", Summary: "Demote swarm node", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmNodes, h.DemoteNode)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-node-tasks", Method: http.MethodGet, Path: "/environments/{id}/swarm/nodes/{nodeId}/tasks", Summary: "List tasks for a swarm node", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.ListNodeTasks)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-node-identity", Method: http.MethodGet, Path: "/swarm/node-identity", Summary: "Get local swarm node identity", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.GetNodeIdentity)-- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-tasks", Method: http.MethodGet, Path: "/environments/{id}/swarm/tasks", Summary: "List swarm tasks", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.ListTasks)-- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-stacks", Method: http.MethodGet, Path: "/environments/{id}/swarm/stacks", Summary: "List swarm stacks", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.ListStacks)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "deploy-swarm-stack", Method: http.MethodPost, Path: "/environments/{id}/swarm/stacks", Summary: "Deploy swarm stack", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmStacks, h.DeployStack)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-stack", Method: http.MethodGet, Path: "/environments/{id}/swarm/stacks/{name}", Summary: "Get swarm stack", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.GetStack)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-stack-source", Method: http.MethodGet, Path: "/environments/{id}/swarm/stacks/{name}/source", Summary: "Get swarm stack source", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmStacks, h.GetStackSource)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "update-swarm-stack-source", Method: http.MethodPut, Path: "/environments/{id}/swarm/stacks/{name}/source", Summary: "Update swarm stack source", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmStacks, h.UpdateStackSource)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-stack", Method: http.MethodDelete, Path: "/environments/{id}/swarm/stacks/{name}", Summary: "Delete swarm stack", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmStacks, h.DeleteStack)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-stack-services", Method: http.MethodGet, Path: "/environments/{id}/swarm/stacks/{name}/services", Summary: "List swarm stack services", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.ListStackServices)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-stack-tasks", Method: http.MethodGet, Path: "/environments/{id}/swarm/stacks/{name}/tasks", Summary: "List swarm stack tasks", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.ListStackTasks)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "render-swarm-stack-config", Method: http.MethodPost, Path: "/environments/{id}/swarm/stacks/config/render", Summary: "Render/validate swarm stack config", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.RenderStackConfig)-- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-status", Method: http.MethodGet, Path: "/environments/{id}/swarm/status", Summary: "Get swarm status", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.GetSwarmStatus)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-info", Method: http.MethodGet, Path: "/environments/{id}/swarm/info", Summary: "Get swarm info", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.GetSwarmInfo)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "init-swarm", Method: http.MethodPost, Path: "/environments/{id}/swarm/init", Summary: "Initialize swarm", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmInit, h.InitSwarm)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "join-swarm", Method: http.MethodPost, Path: "/environments/{id}/swarm/join", Summary: "Join swarm", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmJoin, h.JoinSwarm)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-join-candidates", Method: http.MethodGet, Path: "/environments/{id}/swarm/join-candidates", Summary: "List environments available for Easy Join", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmJoin, h.GetJoinCandidates)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "join-swarm-environments", Method: http.MethodPost, Path: "/environments/{id}/swarm/join-environments", Summary: "Join environments to a swarm", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmJoin, h.JoinEnvironments)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "leave-swarm", Method: http.MethodPost, Path: "/environments/{id}/swarm/leave", Summary: "Leave swarm", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmLeave, h.LeaveSwarm)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "unlock-swarm", Method: http.MethodPost, Path: "/environments/{id}/swarm/unlock", Summary: "Unlock swarm", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmUnlock, h.UnlockSwarm)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-unlock-key", Method: http.MethodGet, Path: "/environments/{id}/swarm/unlock-key", Summary: "Get swarm unlock key", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmUnlock, h.GetUnlockKey)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-join-tokens", Method: http.MethodGet, Path: "/environments/{id}/swarm/join-tokens", Summary: "Get swarm join tokens", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmUnlock, h.GetJoinTokens)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "rotate-swarm-join-tokens", Method: http.MethodPost, Path: "/environments/{id}/swarm/join-tokens/rotate", Summary: "Rotate swarm join tokens", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmUnlock, h.RotateJoinTokens)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "update-swarm-spec", Method: http.MethodPut, Path: "/environments/{id}/swarm/spec", Summary: "Update swarm spec", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmSpec, h.UpdateSwarmSpec)-- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-configs", Method: http.MethodGet, Path: "/environments/{id}/swarm/configs", Summary: "List swarm configs", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.ListConfigs)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-config", Method: http.MethodGet, Path: "/environments/{id}/swarm/configs/{configId}", Summary: "Get swarm config", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.GetConfig)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "create-swarm-config", Method: http.MethodPost, Path: "/environments/{id}/swarm/configs", Summary: "Create swarm config", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmConfigs, h.CreateConfig)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-config", Method: http.MethodDelete, Path: "/environments/{id}/swarm/configs/{configId}", Summary: "Delete swarm config", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmConfigs, h.DeleteConfig)-- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-secrets", Method: http.MethodGet, Path: "/environments/{id}/swarm/secrets", Summary: "List swarm secrets", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.ListSecrets)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-secret", Method: http.MethodGet, Path: "/environments/{id}/swarm/secrets/{secretId}", Summary: "Get swarm secret", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmRead, h.GetSecret)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "create-swarm-secret", Method: http.MethodPost, Path: "/environments/{id}/swarm/secrets", Summary: "Create swarm secret", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmSecrets, h.CreateSecret)- humamw.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-secret", Method: http.MethodDelete, Path: "/environments/{id}/swarm/secrets/{secretId}", Summary: "Delete swarm secret", Tags: []string{"Swarm"}, Security: defaultOperationSecurityInternal()}, authz.PermSwarmSecrets, h.DeleteSecret)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-services", Method: http.MethodGet, Path: "/environments/{id}/swarm/services", Summary: "List swarm services", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.ListServices)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-service", Method: http.MethodGet, Path: "/environments/{id}/swarm/services/{serviceId}", Summary: "Get swarm service", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.GetService)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-swarm-service", Method: http.MethodPost, Path: "/environments/{id}/swarm/services", Summary: "Create swarm service", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmServices, h.CreateService)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-swarm-service", Method: http.MethodPut, Path: "/environments/{id}/swarm/services/{serviceId}", Summary: "Update swarm service", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmServices, h.UpdateService)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-service", Method: http.MethodDelete, Path: "/environments/{id}/swarm/services/{serviceId}", Summary: "Delete swarm service", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmServices, h.DeleteService)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-service-tasks", Method: http.MethodGet, Path: "/environments/{id}/swarm/services/{serviceId}/tasks", Summary: "List tasks for a swarm service", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.ListServiceTasks)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "rollback-swarm-service", Method: http.MethodPost, Path: "/environments/{id}/swarm/services/{serviceId}/rollback", Summary: "Rollback a swarm service", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmServices, h.RollbackService)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "scale-swarm-service", Method: http.MethodPost, Path: "/environments/{id}/swarm/services/{serviceId}/scale", Summary: "Scale a swarm service", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmServices, h.ScaleService)++ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-nodes", Method: http.MethodGet, Path: "/environments/{id}/swarm/nodes", Summary: "List swarm nodes", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.ListNodes)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-node", Method: http.MethodGet, Path: "/environments/{id}/swarm/nodes/{nodeId}", Summary: "Get swarm node", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.GetNode)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-node-agent-deployment", Method: http.MethodPost, Path: "/environments/{id}/swarm/nodes/{nodeId}/agent/deployment", Summary: "Get swarm node agent deployment snippets", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmNodes, h.GetNodeAgentDeployment)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "reconcile-swarm-node-agents", Method: http.MethodPost, Path: "/environments/{id}/swarm/nodes/agents/reconcile", Summary: "Reconcile swarm node agent bindings", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmNodes, h.ReconcileNodeAgents)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "put-swarm-node-agent-binding", Method: http.MethodPut, Path: "/environments/{id}/swarm/nodes/{nodeId}/agent/binding", Summary: "Attach a visible environment to a swarm node", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmNodes, h.PutNodeAgentBinding)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-node-agent-binding", Method: http.MethodDelete, Path: "/environments/{id}/swarm/nodes/{nodeId}/agent/binding", Summary: "Detach a visible environment from a swarm node", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmNodes, h.DeleteNodeAgentBinding)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-node-agent-deployment", Method: http.MethodDelete, Path: "/environments/{id}/swarm/nodes/{nodeId}/agent/deployment", Summary: "Remove a dedicated swarm node agent registration", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmNodes, h.DeleteNodeAgentDeployment)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-swarm-node", Method: http.MethodPatch, Path: "/environments/{id}/swarm/nodes/{nodeId}", Summary: "Update swarm node", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmNodes, h.UpdateNode)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-node", Method: http.MethodDelete, Path: "/environments/{id}/swarm/nodes/{nodeId}", Summary: "Delete swarm node", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmNodes, h.DeleteNode)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "promote-swarm-node", Method: http.MethodPost, Path: "/environments/{id}/swarm/nodes/{nodeId}/promote", Summary: "Promote swarm node", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmNodes, h.PromoteNode)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "demote-swarm-node", Method: http.MethodPost, Path: "/environments/{id}/swarm/nodes/{nodeId}/demote", Summary: "Demote swarm node", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmNodes, h.DemoteNode)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-node-tasks", Method: http.MethodGet, Path: "/environments/{id}/swarm/nodes/{nodeId}/tasks", Summary: "List tasks for a swarm node", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.ListNodeTasks)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-node-identity", Method: http.MethodGet, Path: "/swarm/node-identity", Summary: "Get local swarm node identity", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.GetNodeIdentity)++ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-tasks", Method: http.MethodGet, Path: "/environments/{id}/swarm/tasks", Summary: "List swarm tasks", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.ListTasks)++ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-stacks", Method: http.MethodGet, Path: "/environments/{id}/swarm/stacks", Summary: "List swarm stacks", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.ListStacks)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "deploy-swarm-stack", Method: http.MethodPost, Path: "/environments/{id}/swarm/stacks", Summary: "Deploy swarm stack", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmStacks, h.DeployStack)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-stack", Method: http.MethodGet, Path: "/environments/{id}/swarm/stacks/{name}", Summary: "Get swarm stack", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.GetStack)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-stack-source", Method: http.MethodGet, Path: "/environments/{id}/swarm/stacks/{name}/source", Summary: "Get swarm stack source", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmStacks, h.GetStackSource)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-swarm-stack-source", Method: http.MethodPut, Path: "/environments/{id}/swarm/stacks/{name}/source", Summary: "Update swarm stack source", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmStacks, h.UpdateStackSource)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-stack", Method: http.MethodDelete, Path: "/environments/{id}/swarm/stacks/{name}", Summary: "Delete swarm stack", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmStacks, h.DeleteStack)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-stack-services", Method: http.MethodGet, Path: "/environments/{id}/swarm/stacks/{name}/services", Summary: "List swarm stack services", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.ListStackServices)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-stack-tasks", Method: http.MethodGet, Path: "/environments/{id}/swarm/stacks/{name}/tasks", Summary: "List swarm stack tasks", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.ListStackTasks)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "render-swarm-stack-config", Method: http.MethodPost, Path: "/environments/{id}/swarm/stacks/config/render", Summary: "Render/validate swarm stack config", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.RenderStackConfig)++ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-status", Method: http.MethodGet, Path: "/environments/{id}/swarm/status", Summary: "Get swarm status", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.GetSwarmStatus)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-info", Method: http.MethodGet, Path: "/environments/{id}/swarm/info", Summary: "Get swarm info", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.GetSwarmInfo)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "init-swarm", Method: http.MethodPost, Path: "/environments/{id}/swarm/init", Summary: "Initialize swarm", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmInit, h.InitSwarm)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "join-swarm", Method: http.MethodPost, Path: "/environments/{id}/swarm/join", Summary: "Join swarm", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmJoin, h.JoinSwarm)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-join-candidates", Method: http.MethodGet, Path: "/environments/{id}/swarm/join-candidates", Summary: "List environments available for Easy Join", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmJoin, h.GetJoinCandidates)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "join-swarm-environments", Method: http.MethodPost, Path: "/environments/{id}/swarm/join-environments", Summary: "Join environments to a swarm", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmJoin, h.JoinEnvironments)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "leave-swarm", Method: http.MethodPost, Path: "/environments/{id}/swarm/leave", Summary: "Leave swarm", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmLeave, h.LeaveSwarm)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "unlock-swarm", Method: http.MethodPost, Path: "/environments/{id}/swarm/unlock", Summary: "Unlock swarm", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmUnlock, h.UnlockSwarm)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-unlock-key", Method: http.MethodGet, Path: "/environments/{id}/swarm/unlock-key", Summary: "Get swarm unlock key", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmUnlock, h.GetUnlockKey)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-join-tokens", Method: http.MethodGet, Path: "/environments/{id}/swarm/join-tokens", Summary: "Get swarm join tokens", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmUnlock, h.GetJoinTokens)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "rotate-swarm-join-tokens", Method: http.MethodPost, Path: "/environments/{id}/swarm/join-tokens/rotate", Summary: "Rotate swarm join tokens", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmUnlock, h.RotateJoinTokens)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-swarm-spec", Method: http.MethodPut, Path: "/environments/{id}/swarm/spec", Summary: "Update swarm spec", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmSpec, h.UpdateSwarmSpec)++ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-configs", Method: http.MethodGet, Path: "/environments/{id}/swarm/configs", Summary: "List swarm configs", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.ListConfigs)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-config", Method: http.MethodGet, Path: "/environments/{id}/swarm/configs/{configId}", Summary: "Get swarm config", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.GetConfig)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-swarm-config", Method: http.MethodPost, Path: "/environments/{id}/swarm/configs", Summary: "Create swarm config", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmConfigs, h.CreateConfig)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-config", Method: http.MethodDelete, Path: "/environments/{id}/swarm/configs/{configId}", Summary: "Delete swarm config", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmConfigs, h.DeleteConfig)++ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-swarm-secrets", Method: http.MethodGet, Path: "/environments/{id}/swarm/secrets", Summary: "List swarm secrets", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.ListSecrets)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-swarm-secret", Method: http.MethodGet, Path: "/environments/{id}/swarm/secrets/{secretId}", Summary: "Get swarm secret", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmRead, h.GetSecret)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-swarm-secret", Method: http.MethodPost, Path: "/environments/{id}/swarm/secrets", Summary: "Create swarm secret", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmSecrets, h.CreateSecret)+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-swarm-secret", Method: http.MethodDelete, Path: "/environments/{id}/swarm/secrets/{secretId}", Summary: "Delete swarm secret", Tags: []string{"Swarm"}, Security: handlerutil.DefaultOperationSecurity()}, authz.PermSwarmSecrets, h.DeleteSecret)}// ListServices lists swarm services for an environment and returns a paginated response.@@ -621,7 +623,7 @@ func RegisterSwarm(api huma.API, swarmSvc *services.SwarmService, environmentSvc// Returns an HTTP-shaped error if the swarm service is unavailable or if the// underlying swarm lookup fails.func (h *SwarmHandler) ListServices(ctx context.Context, input *ListSwarmServicesInput) (*ListSwarmServicesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)items, paginationResp, err := h.swarmService.ListServicesPaginated(ctx, params)if err != nil {return nil, mapSwarmServiceError(err, errors.WithMessage(err, "Failed to list swarm services").Error())@@ -630,7 +632,7 @@ func (h *SwarmHandler) ListServices(ctx context.Context, input *ListSwarmServiceitems = []swarmtypes.ServiceSummary{}}- return &ListSwarmServicesOutput{Body: base.Paginated[swarmtypes.ServiceSummary]{Success: true, Data: items, Pagination: toPaginationResponseInternal(paginationResp)}}, nil+ return &ListSwarmServicesOutput{Body: base.Paginated[swarmtypes.ServiceSummary]{Success: true, Data: items, Pagination: handlerutil.PaginationResponse(paginationResp)}}, nil}// GetService returns detailed information for a single swarm service.@@ -736,7 +738,7 @@ func (h *SwarmHandler) DeleteService(ctx context.Context, input *DeleteSwarmServ// Returns a paginated list of task summaries for the service.// Returns a mapped HTTP error when the swarm task lookup fails.func (h *SwarmHandler) ListServiceTasks(ctx context.Context, input *ListSwarmServiceTasksInput) (*ListSwarmServiceTasksOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)items, paginationResp, err := h.swarmService.ListServiceTasksPaginated(ctx, input.ServiceID, params)if err != nil {return nil, mapSwarmServiceError(err, errors.WithMessage(err, "Failed to list swarm tasks").Error())@@ -745,7 +747,7 @@ func (h *SwarmHandler) ListServiceTasks(ctx context.Context, input *ListSwarmSeritems = []swarmtypes.TaskSummary{}}- return &ListSwarmServiceTasksOutput{Body: base.Paginated[swarmtypes.TaskSummary]{Success: true, Data: items, Pagination: toPaginationResponseInternal(paginationResp)}}, nil+ return &ListSwarmServiceTasksOutput{Body: base.Paginated[swarmtypes.TaskSummary]{Success: true, Data: items, Pagination: handlerutil.PaginationResponse(paginationResp)}}, nil}// RollbackService requests a server-side rollback for a swarm service.@@ -803,7 +805,7 @@ func (h *SwarmHandler) ScaleService(ctx context.Context, input *ScaleSwarmServic// Returns a paginated list of node summaries.// Returns a mapped HTTP error when node enumeration fails.func (h *SwarmHandler) ListNodes(ctx context.Context, input *ListSwarmNodesInput) (*ListSwarmNodesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)items, paginationResp, err := h.swarmService.ListNodesPaginated(ctx, input.EnvironmentID, params)if err != nil {return nil, mapSwarmServiceError(err, errors.WithMessage(err, "Failed to list swarm nodes").Error())@@ -812,7 +814,7 @@ func (h *SwarmHandler) ListNodes(ctx context.Context, input *ListSwarmNodesInputitems = []swarmtypes.NodeSummary{}}- return &ListSwarmNodesOutput{Body: base.Paginated[swarmtypes.NodeSummary]{Success: true, Data: items, Pagination: toPaginationResponseInternal(paginationResp)}}, nil+ return &ListSwarmNodesOutput{Body: base.Paginated[swarmtypes.NodeSummary]{Success: true, Data: items, Pagination: handlerutil.PaginationResponse(paginationResp)}}, nil}// GetNode returns detailed information for a single swarm node.@@ -863,7 +865,7 @@ func (h *SwarmHandler) GetNodeAgentDeployment(ctx context.Context, input *GetSwareturn nil, mapSwarmServiceError(err, errors.WithMessage(err, "Swarm node not found").Error())}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -881,7 +883,7 @@ func (h *SwarmHandler) GetNodeAgentDeployment(ctx context.Context, input *GetSwareturn nil, huma.Error500InternalServerError(err.Error())}- var snippets *services.DeploymentSnippets+ var snippets *environment.DeploymentSnippetsif env.IsEdge {snippets, err = h.environmentService.GenerateEdgeDeploymentSnippets(ctx, env.ID, h.cfg.GetAppURL(), apiKey, &edge.Config{EdgeMTLSMode: h.cfg.EdgeMTLSMode,@@ -905,7 +907,7 @@ func (h *SwarmHandler) GetNodeAgentDeployment(ctx context.Context, input *GetSwaBody: base.ApiResponse[SwarmNodeAgentDeployment]{Success: true,Data: SwarmNodeAgentDeployment{- DeploymentSnippet: DeploymentSnippet{+ DeploymentSnippet: environment.DeploymentSnippet{DockerRun: snippets.DockerRun,DockerCompose: snippets.DockerCompose,},@@ -931,7 +933,7 @@ func (h *SwarmHandler) PutNodeAgentBinding(ctx context.Context, input *PutSwarmNreturn nil, huma.Error400BadRequest(err.Error())}if input.Body.ReplaceDeployment {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -957,7 +959,7 @@ func (h *SwarmHandler) DeleteNodeAgentBinding(ctx context.Context, input *Delete// DeleteNodeAgentDeployment removes a dedicated hidden node-agent registration.func (h *SwarmHandler) DeleteNodeAgentDeployment(ctx context.Context, input *DeleteSwarmNodeAgentDeploymentInput) (*DeleteSwarmNodeAgentDeploymentOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -985,7 +987,7 @@ func (h *SwarmHandler) GetNodeIdentity(ctx context.Context, _ *GetSwarmNodeIdent}return &GetSwarmNodeIdentityOutput{- Body: base.ApiResponse[services.SwarmNodeIdentity]{+ Body: base.ApiResponse[SwarmNodeIdentity]{Success: true,Data: *identity,},@@ -1087,7 +1089,7 @@ func (h *SwarmHandler) DemoteNode(ctx context.Context, input *DemoteSwarmNodeInp// Returns a paginated list of node task summaries.// Returns a mapped HTTP error when the underlying lookup fails.func (h *SwarmHandler) ListNodeTasks(ctx context.Context, input *ListSwarmNodeTasksInput) (*ListSwarmNodeTasksOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)items, paginationResp, err := h.swarmService.ListNodeTasksPaginated(ctx, input.NodeID, params)if err != nil {return nil, mapSwarmServiceError(err, errors.WithMessage(err, "Failed to list swarm tasks").Error())@@ -1096,7 +1098,7 @@ func (h *SwarmHandler) ListNodeTasks(ctx context.Context, input *ListSwarmNodeTaitems = []swarmtypes.TaskSummary{}}- return &ListSwarmNodeTasksOutput{Body: base.Paginated[swarmtypes.TaskSummary]{Success: true, Data: items, Pagination: toPaginationResponseInternal(paginationResp)}}, nil+ return &ListSwarmNodeTasksOutput{Body: base.Paginated[swarmtypes.TaskSummary]{Success: true, Data: items, Pagination: handlerutil.PaginationResponse(paginationResp)}}, nil}// ListTasks lists swarm tasks across the current environment.@@ -1110,7 +1112,7 @@ func (h *SwarmHandler) ListNodeTasks(ctx context.Context, input *ListSwarmNodeTa// Returns a paginated task listing for the environment.// Returns a mapped HTTP error when task enumeration fails.func (h *SwarmHandler) ListTasks(ctx context.Context, input *ListSwarmTasksInput) (*ListSwarmTasksOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)items, paginationResp, err := h.swarmService.ListTasksPaginated(ctx, params)if err != nil {return nil, mapSwarmServiceError(err, errors.WithMessage(err, "Failed to list swarm tasks").Error())@@ -1119,7 +1121,7 @@ func (h *SwarmHandler) ListTasks(ctx context.Context, input *ListSwarmTasksInputitems = []swarmtypes.TaskSummary{}}- return &ListSwarmTasksOutput{Body: base.Paginated[swarmtypes.TaskSummary]{Success: true, Data: items, Pagination: toPaginationResponseInternal(paginationResp)}}, nil+ return &ListSwarmTasksOutput{Body: base.Paginated[swarmtypes.TaskSummary]{Success: true, Data: items, Pagination: handlerutil.PaginationResponse(paginationResp)}}, nil}// ListStacks lists swarm stacks for the current environment.@@ -1133,7 +1135,7 @@ func (h *SwarmHandler) ListTasks(ctx context.Context, input *ListSwarmTasksInput// Returns a paginated list of stack summaries.// Returns a mapped HTTP error when stack enumeration fails.func (h *SwarmHandler) ListStacks(ctx context.Context, input *ListSwarmStacksInput) (*ListSwarmStacksOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)items, paginationResp, err := h.swarmService.ListStacksPaginated(ctx, input.EnvironmentID, params)if err != nil {return nil, mapSwarmServiceError(err, errors.WithMessage(err, "Failed to list swarm stacks").Error())@@ -1142,7 +1144,7 @@ func (h *SwarmHandler) ListStacks(ctx context.Context, input *ListSwarmStacksInpitems = []swarmtypes.StackSummary{}}- return &ListSwarmStacksOutput{Body: base.Paginated[swarmtypes.StackSummary]{Success: true, Data: items, Pagination: toPaginationResponseInternal(paginationResp)}}, nil+ return &ListSwarmStacksOutput{Body: base.Paginated[swarmtypes.StackSummary]{Success: true, Data: items, Pagination: handlerutil.PaginationResponse(paginationResp)}}, nil}// DeployStack deploys or updates a swarm stack.@@ -1267,7 +1269,7 @@ func (h *SwarmHandler) DeleteStack(ctx context.Context, input *DeleteSwarmStackI// Returns `404 Not Found` when the stack does not exist or another mapped HTTP// error when the lookup fails.func (h *SwarmHandler) ListStackServices(ctx context.Context, input *ListSwarmStackServicesInput) (*ListSwarmStackServicesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)items, paginationResp, err := h.swarmService.ListStackServicesPaginated(ctx, input.Name, params)if err != nil {if errdefs.IsNotFound(err) {@@ -1279,7 +1281,7 @@ func (h *SwarmHandler) ListStackServices(ctx context.Context, input *ListSwarmStitems = []swarmtypes.ServiceSummary{}}- return &ListSwarmStackServicesOutput{Body: base.Paginated[swarmtypes.ServiceSummary]{Success: true, Data: items, Pagination: toPaginationResponseInternal(paginationResp)}}, nil+ return &ListSwarmStackServicesOutput{Body: base.Paginated[swarmtypes.ServiceSummary]{Success: true, Data: items, Pagination: handlerutil.PaginationResponse(paginationResp)}}, nil}// ListStackTasks lists tasks belonging to a swarm stack.@@ -1294,7 +1296,7 @@ func (h *SwarmHandler) ListStackServices(ctx context.Context, input *ListSwarmSt// Returns `404 Not Found` when the stack does not exist or another mapped HTTP// error when the lookup fails.func (h *SwarmHandler) ListStackTasks(ctx context.Context, input *ListSwarmStackTasksInput) (*ListSwarmStackTasksOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)items, paginationResp, err := h.swarmService.ListStackTasksPaginated(ctx, input.Name, params)if err != nil {if errdefs.IsNotFound(err) {@@ -1306,7 +1308,7 @@ func (h *SwarmHandler) ListStackTasks(ctx context.Context, input *ListSwarmStackitems = []swarmtypes.TaskSummary{}}- return &ListSwarmStackTasksOutput{Body: base.Paginated[swarmtypes.TaskSummary]{Success: true, Data: items, Pagination: toPaginationResponseInternal(paginationResp)}}, nil+ return &ListSwarmStackTasksOutput{Body: base.Paginated[swarmtypes.TaskSummary]{Success: true, Data: items, Pagination: handlerutil.PaginationResponse(paginationResp)}}, nil}// RenderStackConfig renders and validates a swarm stack configuration without deploying it.@@ -1430,7 +1432,7 @@ func (h *SwarmHandler) GetJoinCandidates(ctx context.Context, input *GetSwarmJoi}}- permissions, _ := humamw.PermissionsFromContext(ctx)+ permissions, _ := middleware.PermissionsFromContext(ctx)environments, err := h.environmentService.ListSwarmNodeCandidateEnvironments(ctx)if err != nil {return nil, huma.Error500InternalServerError(err.Error())@@ -1464,7 +1466,7 @@ func (h *SwarmHandler) JoinEnvironments(ctx context.Context, input *JoinSwarmEnvreturn nil, err}- permissions, _ := humamw.PermissionsFromContext(ctx)+ permissions, _ := middleware.PermissionsFromContext(ctx)for _, target := range input.Body.Targets {if target.EnvironmentID == input.EnvironmentID {return nil, huma.Error400BadRequest("selected swarm manager cannot also be a join target")@@ -1497,7 +1499,7 @@ func (h *SwarmHandler) JoinEnvironments(ctx context.Context, input *JoinSwarmEnv}func requireEasyJoinManagerPermissionsInternal(ctx context.Context, environmentID string) error {- permissions, _ := humamw.PermissionsFromContext(ctx)+ permissions, _ := middleware.PermissionsFromContext(ctx)if permissions == nil || !permissions.Allows(authz.PermSwarmJoin, environmentID) {return huma.Error403Forbidden("swarm:join permission is required on the manager environment")}@@ -1855,7 +1857,7 @@ func (h *SwarmHandler) auditSwarmMutation(ctx context.Context, environmentID, acmeta := models.JSON{"action": action}maps.Copy(meta, metadata)- _, err := h.eventService.CreateEvent(ctx, services.CreateEventRequest{+ _, err := h.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventType("swarm." + action),Severity: models.EventSeverityInfo,Title: "Swarm operation: " + action,diff --git a/backend/internal/swarm/module.go b/backend/internal/swarm/module.gonew file mode 100644index 000000000..e6d14456b--- /dev/null+++ b/backend/internal/swarm/module.go@@ -0,0 +1,40 @@+// Package swarm owns Docker Swarm management and its HTTP routes.+package swarm++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+)++type Dependencies struct {+ Environment *environment.EnvironmentService+ Event *event.EventService+ Config *config.Config+}++type Module struct {+ service *SwarmService+ deps Dependencies+}++func New(service *SwarmService, deps Dependencies) *Module {+ return &Module{service: service, deps: deps}+}++func (m *Module) Service() *SwarmService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterSwarm(api, nil, nil, nil, nil)+ return+ }+ RegisterSwarm(api, m.service, m.deps.Environment, m.deps.Event, m.deps.Config)+}diff --git a/backend/internal/services/swarm_service_test.go b/backend/internal/swarm/service_test.gosimilarity index 80%rename from backend/internal/services/swarm_service_test.gorename to backend/internal/swarm/service_test.goindex 35cff7558..e3ddb4c4e 100644--- a/backend/internal/services/swarm_service_test.go+++ b/backend/internal/swarm/service_test.go@@ -1,4 +1,4 @@-package services+package swarmimport ("context"@@ -10,14 +10,76 @@ import ("time"cerrdefs "github.com/containerd/errdefs"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"swarmtypes "github.com/getarcaneapp/arcane/types/v2/swarm"+ "github.com/libtnb/sqlite""github.com/moby/moby/api/types/swarm""github.com/moby/moby/api/types/system"+ "github.com/moby/moby/client""github.com/stretchr/testify/assert""github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest"+ "gorm.io/gorm")+func setupSwarmServiceTestDBInternal(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.SettingVariable{}, &models.Environment{}))+ return &database.DB{DB: db}+}++func newSettingsServiceForSwarmTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ writes, err := actors.NewExecutor(t.Context(), runtime, "swarm-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "swarm-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, writes.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, writes, effects)+}++func createSwarmTestEnvironmentInternal(t *testing.T, db *database.DB, id, apiURL, status string, isEdge bool, accessToken *string) {+ t.Helper()+ now := time.Now()+ require.NoError(t, db.Create(&models.Environment{+ BaseModel: models.BaseModel{ID: id, CreatedAt: now, UpdatedAt: &now},+ Name: "env-" + id,+ ApiUrl: apiURL,+ Status: status,+ Enabled: true,+ IsEdge: isEdge,+ AccessToken: accessToken,+ }).Error)+}++func newSwarmTestDockerClientInternal(t *testing.T, server *httptest.Server) *client.Client {+ t.Helper()+ cli, err := client.New(+ client.WithHost(server.URL),+ client.WithAPIVersion("1.41"),+ client.WithHTTPClient(server.Client()),+ )+ require.NoError(t, err)+ t.Cleanup(func() { _ = cli.Close() })+ return cli+}+func TestDecodeSwarmSpecInternal_AllowsEmptyObject(t *testing.T) {spec, err := decodeSwarmSpecInternal(json.RawMessage(`{}`))require.NoError(t, err)@@ -38,10 +100,10 @@ func TestDefaultSwarmListenAddrInternal(t *testing.T) {func TestSwarmService_FetchSwarmNodeIdentityViaEdgeInternal_UsesEnvironmentAccessToken(t *testing.T) {ctx := context.Background()- db := setupEnvironmentServiceTestDB(t)- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)+ db := setupSwarmServiceTestDBInternal(t)+ settingsSvc, err := newSettingsServiceForSwarmTestInternal(t, ctx, db)require.NoError(t, err)- envSvc := NewEnvironmentService(db, nil, nil, nil, settingsSvc, nil)+ envSvc := environment.NewEnvironmentService(db, nil, nil, nil, settingsSvc, nil)accessToken := "token-123"server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {@@ -63,7 +125,7 @@ func TestSwarmService_FetchSwarmNodeIdentityViaEdgeInternal_UsesEnvironmentAcces}))defer server.Close()- createTestEnvironmentWithState(+ createSwarmTestEnvironmentInternal(t,db,"env-1",@@ -87,11 +149,11 @@ func TestSwarmService_FetchSwarmNodeIdentityViaEdgeInternal_UsesEnvironmentAccesfunc TestSwarmService_UpdateAndGetStackSource_UsesStoredFilesWithoutSwarmManager(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSwarmServiceTestDBInternal(t)rootDir := t.TempDir()t.Setenv("SWARM_STACK_SOURCES_DIRECTORY", rootDir)- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)+ settingsSvc, err := newSettingsServiceForSwarmTestInternal(t, ctx, db)require.NoError(t, err)svc := NewSwarmService(nil, settingsSvc, nil, nil, nil)@@ -145,11 +207,11 @@ func TestSwarmService_UpdateAndGetStackSource_UsesStoredFilesWithoutSwarmManagerfunc TestSwarmService_UpdateAndGetStackSource_RoundTripsOverride(t *testing.T) {ctx := context.Background()- db := setupSettingsTestDB(t)+ db := setupSwarmServiceTestDBInternal(t)rootDir := t.TempDir()t.Setenv("SWARM_STACK_SOURCES_DIRECTORY", rootDir)- settingsSvc, err := newSettingsServiceForTestInternal(t, ctx, db)+ settingsSvc, err := newSettingsServiceForSwarmTestInternal(t, ctx, db)require.NoError(t, err)svc := NewSwarmService(nil, settingsSvc, nil, nil, nil)@@ -277,7 +339,7 @@ func TestSwarmService_ScaleService_HandlesServiceModesInternal(t *testing.T) {}))t.Cleanup(server.Close)- svc := NewSwarmService(&DockerClientService{client: newTestDockerClient(t, server)}, nil, nil, nil, nil)+ svc := NewSwarmService(&docker.DockerClientService{Client: newSwarmTestDockerClientInternal(t, server)}, nil, nil, nil, nil)resp, err := svc.ScaleService(ctx, "service-1", replicas)if tt.wantErr {diff --git a/backend/internal/services/swarm_service.go b/backend/internal/swarm/swarm.gosimilarity index 97%rename from backend/internal/services/swarm_service.gorename to backend/internal/swarm/swarm.goindex 862987a85..3fc2242a2 100644--- a/backend/internal/services/swarm_service.go+++ b/backend/internal/swarm/swarm.go@@ -1,4 +1,4 @@-package services+package swarmimport ("bytes"@@ -17,10 +17,16 @@ import ("sync""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+"emperror.dev/errors"cerrdefs "github.com/containerd/errdefs""github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/edge"libswarm "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/swarm"@@ -45,20 +51,20 @@ const (// SwarmService provides Docker Swarm related operations.type SwarmService struct {- dockerService *DockerClientService- settingsService *SettingsService- kvService *KVService- registryService *ContainerRegistryService- environmentService *EnvironmentService+ dockerService *docker.DockerClientService+ settingsService *settings.SettingsService+ kvService *kv.KVService+ registryService *registry.ContainerRegistryService+ environmentService *environment.EnvironmentServiceidentityCache *hot.HotCache[string, SwarmNodeIdentity]}func NewSwarmService(- dockerService *DockerClientService,- settingsService *SettingsService,- kvService *KVService,- registryService *ContainerRegistryService,- environmentService *EnvironmentService,+ dockerService *docker.DockerClientService,+ settingsService *settings.SettingsService,+ kvService *kv.KVService,+ registryService *registry.ContainerRegistryService,+ environmentService *environment.EnvironmentService,) *SwarmService {return &SwarmService{dockerService: dockerService,@@ -183,7 +189,7 @@ func (s *SwarmService) ListServicesPaginated(ctx context.Context, params paginatconfig := s.buildServicePaginationConfigInternal()result := pagination.SearchOrderAndPaginate(items, params, config)- paginationResp := buildPaginationResponseInternal(result, params)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)return result.Items, paginationResp, nil}@@ -495,7 +501,7 @@ func (s *SwarmService) ListNodesPaginated(ctx context.Context, environmentID strs.enrichNodeAgentStatusesInternal(ctx, environmentID, remote.Data)result := pagination.SearchOrderAndPaginate(remote.Data, params, s.buildNodePaginationConfigInternal())- return result.Items, buildPaginationResponseInternal(result, params), nil+ return result.Items, pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params), nil}if err := s.ensureSwarmManagerInternal(ctx); err != nil {@@ -522,7 +528,7 @@ func (s *SwarmService) ListNodesPaginated(ctx context.Context, environmentID strconfig := s.buildNodePaginationConfigInternal()result := pagination.SearchOrderAndPaginate(items, params, config)- paginationResp := buildPaginationResponseInternal(result, params)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)return result.Items, paginationResp, nil}@@ -740,12 +746,12 @@ func (s *SwarmService) resolveNodeAgentCoverageInternal(ctx context.Context, envboundEnvsByNodeID: boundEnvsByNodeID,candidatesByNodeID: candidatesByNodeID,}- if environmentID == LocalEnvironmentID {+ if environmentID == environment.LocalEnvironmentID {identity, identityErr := s.GetLocalNodeIdentity(ctx)if identityErr == nil {coverage.localIdentity = identity}- coverage.localEnvironmentName = s.environmentService.ResolveEnvironmentName(ctx, LocalEnvironmentID)+ coverage.localEnvironmentName = s.environmentService.ResolveEnvironmentName(ctx, environment.LocalEnvironmentID)}return coverage, nil}@@ -755,11 +761,11 @@ func (s *SwarmService) applyNodeAgentCoverageInternal(environmentID string, itemreturn}nodeID := strings.TrimSpace(item.ID)- if environmentID == LocalEnvironmentID && coverage.localIdentity != nil && coverage.localIdentity.SwarmActive && strings.TrimSpace(coverage.localIdentity.SwarmNodeID) == nodeID {+ if environmentID == environment.LocalEnvironmentID && coverage.localIdentity != nil && coverage.localIdentity.SwarmActive && strings.TrimSpace(coverage.localIdentity.SwarmNodeID) == nodeID {connected := truebindingKind := swarmtypes.NodeAgentBindingKindLocal- localID, localType := LocalEnvironmentID, "local"- localName := EnvironmentDisplayName(LocalEnvironmentID, coverage.localEnvironmentName)+ localID, localType := environment.LocalEnvironmentID, "local"+ localName := environment.DisplayName(environment.LocalEnvironmentID, coverage.localEnvironmentName)item.Agent = swarmtypes.NodeAgentStatus{State: swarmtypes.NodeAgentStateConnected,BindingKind: &bindingKind,@@ -1177,7 +1183,7 @@ func (s *SwarmService) ListTasksPaginated(ctx context.Context, params paginationconfig := s.buildTaskPaginationConfigInternal()result := pagination.SearchOrderAndPaginate(items, params, config)- paginationResp := buildPaginationResponseInternal(result, params)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)return result.Items, paginationResp, nil}@@ -1245,7 +1251,7 @@ func (s *SwarmService) ListStacksPaginated(ctx context.Context, environmentID stconfig := s.buildStackPaginationConfigInternal()result := pagination.SearchOrderAndPaginate(items, params, config)- paginationResp := buildPaginationResponseInternal(result, params)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)return result.Items, paginationResp, nil}@@ -2007,7 +2013,7 @@ func (s *SwarmService) ListStackServicesPaginated(ctx context.Context, stackNameconfig := s.buildServicePaginationConfigInternal()result := pagination.SearchOrderAndPaginate(summaries, params, config)- paginationResp := buildPaginationResponseInternal(result, params)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)return result.Items, paginationResp, nil}@@ -2288,7 +2294,7 @@ func (s *SwarmService) listTasksPaginatedWithFiltersInternal(ctx context.Contextconfig := s.buildTaskPaginationConfigInternal()result := pagination.SearchOrderAndPaginate(items, params, config)- paginationResp := buildPaginationResponseInternal(result, params)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)return result.Items, paginationResp, nil}@@ -2510,7 +2516,7 @@ func (s *SwarmService) upsertStackSourceInternal(ctx context.Context, environmenreturn err}- if err := os.MkdirAll(stackSourceDir, common.DirPerm); err != nil {+ if err := os.MkdirAll(stackSourceDir, utils.DirPerm); err != nil {return errors.WrapIf(err, "failed to create swarm stack source directory")}@@ -2803,26 +2809,6 @@ func (s *SwarmService) buildStackPaginationConfigInternal() pagination.Config[sw}}-func buildPaginationResponseInternal[T any](result pagination.FilterResult[T], params pagination.QueryParams) pagination.Response {- totalPages := int64(0)- if params.Limit > 0 {- totalPages = (result.TotalCount + int64(params.Limit) - 1) / int64(params.Limit)- }-- page := 1- if params.Limit > 0 {- page = (params.Start / params.Limit) + 1- }-- return pagination.Response{- TotalPages: totalPages,- TotalItems: result.TotalCount,- CurrentPage: page,- ItemsPerPage: params.Limit,- GrandTotalItems: result.TotalAvailable,- }-}-func toServiceNetworkIPAMConfigInternal(cfg networktypes.IPAMConfig) swarmtypes.ServiceNetworkIPAMConfig {out := swarmtypes.ServiceNetworkIPAMConfig{}if cfg.Subnet.IsValid() {diff --git a/backend/api/handlers/system.go b/backend/internal/system/handler.gosimilarity index 89%rename from backend/api/handlers/system.gorename to backend/internal/system/handler.goindex 644f79e45..ba7eec43c 100644--- a/backend/api/handlers/system.go+++ b/backend/internal/system/handler.go@@ -1,4 +1,4 @@-package handlers+package systemimport ("context"@@ -12,13 +12,16 @@ import (dockersystem "github.com/moby/moby/api/types/system""github.com/moby/moby/client"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"containertypes "github.com/getarcaneapp/arcane/types/v2/container""github.com/getarcaneapp/arcane/types/v2/dockerinfo"@@ -31,11 +34,11 @@ import (// SystemHandler handles system management endpoints.type SystemHandler struct {- dockerService *services.DockerClientService- systemService *services.SystemService- upgradeService *services.SystemUpgradeService- environmentService *services.EnvironmentService- activityService *services.ActivityService+ dockerService *docker.DockerClientService+ systemService *SystemService+ upgradeService *SystemUpgradeService+ environmentService *environment.EnvironmentService+ activityService *activity.ActivityServicecfg *config.ConfigappCtx context.Context}@@ -148,8 +151,8 @@ type UpdateAllStatusOutput struct {}// RegisterSystem registers system management endpoints using Huma.-// Note: WebSocket endpoints (stats) remain in the Gin handler.-func RegisterSystem(api huma.API, dockerService *services.DockerClientService, systemService *services.SystemService, upgradeService *services.SystemUpgradeService, environmentService *services.EnvironmentService, cfg *config.Config, activityService *services.ActivityService, appCtx ActivityAppContext) {+// WebSocket statistics endpoints live in api/ws.+func RegisterSystem(api huma.API, dockerService *docker.DockerClientService, systemService *SystemService, upgradeService *SystemUpgradeService, environmentService *environment.EnvironmentService, cfg *config.Config, activityService *activity.ActivityService, appCtx handlerutil.ActivityAppContext) {h := &SystemHandler{dockerService: dockerService,systemService: systemService,@@ -157,10 +160,10 @@ func RegisterSystem(api huma.API, dockerService *services.DockerClientService, senvironmentService: environmentService,activityService: activityService,cfg: cfg,- appCtx: appCtx.contextInternal(),+ appCtx: appCtx.Context(),}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "system-health",Method: http.MethodHead,Path: "/environments/{id}/system/health",@@ -168,80 +171,80 @@ func RegisterSystem(api huma.API, dockerService *services.DockerClientService, sDescription: "Check if the Docker daemon is responsive",Tags: []string{"System"},DefaultStatus: http.StatusOK,- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermSystemRead, h.Health)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-docker-info",Method: http.MethodGet,Path: "/environments/{id}/system/docker/info",Summary: "Get Docker info",Description: "Get Docker daemon version and system information",Tags: []string{"System"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermSystemRead, h.GetDockerInfo)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "prune-all",Method: http.MethodPost,Path: "/environments/{id}/system/prune",Summary: "Prune Docker resources",Description: "Remove unused Docker resources (containers, images, volumes, networks)",Tags: []string{"System"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermSystemPrune, h.PruneAll)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "start-all-containers",Method: http.MethodPost,Path: "/environments/{id}/system/containers/start-all",Summary: "Start all containers",Description: "Start all Docker containers",Tags: []string{"System"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersStart, h.StartAllContainers)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "start-all-stopped-containers",Method: http.MethodPost,Path: "/environments/{id}/system/containers/start-stopped",Summary: "Start all stopped containers",Description: "Start all stopped Docker containers",Tags: []string{"System"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersStart, h.StartAllStoppedContainers)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "stop-all-containers",Method: http.MethodPost,Path: "/environments/{id}/system/containers/stop-all",Summary: "Stop all containers",Description: "Stop all running Docker containers",Tags: []string{"System"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersStop, h.StopAllContainers)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "convert-docker-run",Method: http.MethodPost,Path: "/environments/{id}/system/convert",Summary: "Convert docker run command",Description: "Convert a docker run command to docker-compose format",Tags: []string{"System"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermContainersCreate, h.ConvertDockerRun)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "check-upgrade",Method: http.MethodGet,Path: "/environments/{id}/system/upgrade/check",Summary: "Check for system upgrade",Description: "Check if a system upgrade is available",Tags: []string{"System"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermSystemRead, h.CheckUpgradeAvailable)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "trigger-upgrade",Method: http.MethodPost,Path: "/environments/{id}/system/upgrade",@@ -249,10 +252,10 @@ func RegisterSystem(api huma.API, dockerService *services.DockerClientService, sDescription: "Trigger a system upgrade",DefaultStatus: http.StatusAccepted,Tags: []string{"System"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermSystemUpgrade, h.TriggerUpgrade)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "trigger-update-all",Method: http.MethodPost,Path: "/environments/{id}/system/upgrade/all",@@ -260,17 +263,17 @@ func RegisterSystem(api huma.API, dockerService *services.DockerClientService, sDescription: "Upgrade every Arcane environment, starting with the manager",DefaultStatus: http.StatusAccepted,Tags: []string{"System"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermSystemUpgrade, h.TriggerUpdateAll)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-all-status",Method: http.MethodGet,Path: "/environments/{id}/system/upgrade/all/status",Summary: "Get update-all status",Description: "Get the status of the latest update-all-environments job",Tags: []string{"System"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermSystemRead, h.GetUpdateAllStatus)}@@ -505,7 +508,7 @@ func (h *SystemHandler) CheckUpgradeAvailable(ctx context.Context, input *CheckU// TriggerUpgrade triggers a system upgrade.func (h *SystemHandler) TriggerUpgrade(ctx context.Context, input *TriggerUpgradeInput) (*TriggerUpgradeOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -550,7 +553,7 @@ func (h *SystemHandler) TriggerUpdateAll(ctx context.Context, input *TriggerUpdareturn nil, err}- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}diff --git a/backend/internal/system/module.go b/backend/internal/system/module.gonew file mode 100644index 000000000..8e67975b1--- /dev/null+++ b/backend/internal/system/module.go@@ -0,0 +1,76 @@+// Package system owns Docker system-wide operations — prune, disk usage, host+// info — and the HTTP surface that exposes them.+package system++import (+ "github.com/danielgtaylor/huma/v2"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/container"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/network"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"++ "github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/volume"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+)++// Dependencies are the collaborators the system domain needs.+type Dependencies struct {+ DB *database.DB+ Config *config.Config+ Docker *docker.DockerClientService+ Container *container.ContainerService+ Image *image.ImageService+ Volume *volume.VolumeService+ Network *network.NetworkService+ Settings *settings.SettingsService+ Activity *activity.ActivityService+ SystemUpgrade *SystemUpgradeService+ Environment *environment.EnvironmentService+}++// Module wires the system domain and mounts its routes.+type Module struct {+ service *SystemService+ deps Dependencies+}++// New builds the system domain from its dependencies.+func New(deps Dependencies) *Module {+ return &Module{+ service: NewSystemService(+ deps.DB,+ deps.Docker,+ deps.Container,+ deps.Image,+ deps.Volume,+ deps.Network,+ deps.Settings,+ deps.Activity,+ ),+ deps: deps,+ }+}++// Service exposes the system service to collaborators that need it directly,+// such as the scheduled prune job.+func (m *Module) Service() *SystemService {+ if m == nil {+ return nil+ }+ return m.service+}++// RegisterRoutes mounts the system endpoints. A nil module still registers, so+// OpenAPI spec generation can discover the routes without a service graph.+func (m *Module) RegisterRoutes(api huma.API, appCtx handlerutil.ActivityAppContext) {+ if m == nil {+ RegisterSystem(api, nil, nil, nil, nil, nil, nil, appCtx)+ return+ }+ RegisterSystem(api, m.deps.Docker, m.service, m.deps.SystemUpgrade, m.deps.Environment, m.deps.Config, m.deps.Activity, appCtx)+}diff --git a/backend/internal/services/system_service.go b/backend/internal/system/system.gosimilarity index 92%rename from backend/internal/services/system_service.gorename to backend/internal/system/system.goindex 6dc511527..4b50b6bd2 100644--- a/backend/internal/services/system_service.go+++ b/backend/internal/system/system.go@@ -1,4 +1,4 @@-package services+package systemimport ("context"@@ -7,15 +7,24 @@ import ("strings""sync"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/network"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"++ "github.com/getarcaneapp/arcane/backend/v2/internal/container"+"emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/volume"activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"containertypes "github.com/getarcaneapp/arcane/types/v2/container""github.com/getarcaneapp/arcane/types/v2/system"- "github.com/moby/moby/api/types/container"+ mobycontainer "github.com/moby/moby/api/types/container""github.com/moby/moby/client""github.com/samber/mo""go.getarcane.app/updater/labels"@@ -24,26 +33,26 @@ import (type SystemService struct {db *database.DB- dockerService *DockerClientService- containerService *ContainerService- imageService *ImageService- volumeService *VolumeService- networkService *NetworkService- settingsService *SettingsService- activityService *ActivityService+ dockerService *docker.DockerClientService+ containerService *container.ContainerService+ imageService *image.ImageService+ volumeService *volume.VolumeService+ networkService *network.NetworkService+ settingsService *settings.SettingsService+ activityService *activity.ActivityServicepruneMu sync.MutexrunningPrunes map[string]string}func NewSystemService(db *database.DB,- dockerService *DockerClientService,- containerService *ContainerService,- imageService *ImageService,- volumeService *VolumeService,- networkService *NetworkService,- settingsService *SettingsService,- activityService *ActivityService,+ dockerService *docker.DockerClientService,+ containerService *container.ContainerService,+ imageService *image.ImageService,+ volumeService *volume.VolumeService,+ networkService *network.NetworkService,+ settingsService *settings.SettingsService,+ activityService *activity.ActivityService,) *SystemService {return &SystemService{db: db,@@ -58,10 +67,6 @@ func NewSystemService(}}-var systemUser = models.User{- Username: "System",-}-func (s *SystemService) PruneAll(ctx context.Context, environmentID string, req system.PruneAllRequest) (*system.PruneAllResult, bool, error) {slog.InfoContext(ctx, "Starting selective prune operation","containers", req.Containers,@@ -253,7 +258,7 @@ func (s *SystemService) startSystemPruneActivityInternal(ctx context.Context, enif s.activityService == nil {return ""}- activity, err := s.activityService.StartActivity(ctx, StartActivityRequest{+ activity, err := s.activityService.StartActivity(ctx, activity.StartActivityRequest{EnvironmentID: environmentID,Type: models.ActivityTypeSystemPrune,ResourceType: new("system"),@@ -279,7 +284,7 @@ func (s *SystemService) appendSystemPruneActivityMessageInternal(ctx context.Conif s.activityService == nil || activityID == "" {return}- if _, err := s.activityService.AppendMessage(ctx, activityID, AppendActivityMessageRequest{+ if _, err := s.activityService.AppendMessage(ctx, activityID, activity.AppendActivityMessageRequest{Level: models.ActivityMessageLevelInfo,Message: message,Progress: &progress,@@ -312,7 +317,7 @@ func (s *SystemService) completeSystemPruneActivityInternal(ctx context.Context,}}-func (s *SystemService) performBatchContainerAction(ctx context.Context, containers []container.Summary, actionName string, shouldProcess func(container.Summary) bool, action func(context.Context, string) error) *containertypes.ActionResult {+func (s *SystemService) performBatchContainerAction(ctx context.Context, containers []mobycontainer.Summary, actionName string, shouldProcess func(mobycontainer.Summary) bool, action func(context.Context, string) error) *containertypes.ActionResult {result := &containertypes.ActionResult{Success: true}var mu sync.Mutex@@ -359,7 +364,7 @@ func (s *SystemService) StartAllContainers(ctx context.Context, environmentID stStartMessage: "Starting all containers",FailureMessage: "Starting all containers failed",SuccessMessage: "Started all containers",- ShouldStart: func(c container.Summary) bool { return c.State != "running" },+ ShouldStart: func(c mobycontainer.Summary) bool { return c.State != "running" },})}@@ -369,7 +374,7 @@ func (s *SystemService) StartAllStoppedContainers(ctx context.Context, environmeStartMessage: "Starting stopped containers",FailureMessage: "Starting stopped containers failed",SuccessMessage: "Started stopped containers",- ShouldStart: func(c container.Summary) bool { return c.State == "exited" },+ ShouldStart: func(c mobycontainer.Summary) bool { return c.State == "exited" },})}@@ -378,7 +383,7 @@ type startMatchingContainersOptionsInternal struct {StartMessage stringFailureMessage stringSuccessMessage string- ShouldStart func(container.Summary) bool+ ShouldStart func(mobycontainer.Summary) bool}func (s *SystemService) startMatchingContainersInternal(ctx context.Context, environmentID string, opts startMatchingContainersOptionsInternal) (*containertypes.ActionResult, error) {@@ -396,7 +401,7 @@ func (s *SystemService) startMatchingContainersInternal(ctx context.Context, env}result := s.performBatchContainerAction(ctx, containers, "start", opts.ShouldStart, func(ctx context.Context, id string) error {- return s.containerService.StartContainer(ctx, id, systemUser)+ return s.containerService.StartContainer(ctx, id, models.SystemUser)})result.ActivityID = mo.EmptyableToOption(strings.TrimSpace(activityID)).ToPointer()s.completeSystemContainerActivityInternal(ctx, activityID, opts.SuccessMessage, result)@@ -418,12 +423,12 @@ func (s *SystemService) StopAllContainers(ctx context.Context, environmentID str}result := s.performBatchContainerAction(ctx, containers, "stop",- func(c container.Summary) bool {+ func(c mobycontainer.Summary) bool {// Skip Arcane containerreturn !labels.IsArcaneContainer(c.Labels)},func(ctx context.Context, id string) error {- return s.containerService.StopContainer(ctx, id, systemUser)+ return s.containerService.StopContainer(ctx, id, models.SystemUser)})result.ActivityID = mo.EmptyableToOption(strings.TrimSpace(activityID)).ToPointer()s.completeSystemContainerActivityInternal(ctx, activityID, "Stopped all containers", result)@@ -434,7 +439,7 @@ func (s *SystemService) startSystemContainerActivityInternal(ctx context.Contextif s.activityService == nil {return ""}- activity, err := s.activityService.StartActivity(ctx, StartActivityRequest{+ activity, err := s.activityService.StartActivity(ctx, activity.StartActivityRequest{EnvironmentID: environmentID,Type: activityType,ResourceType: new("system"),diff --git a/backend/internal/services/system_upgrade_service.go b/backend/internal/system/upgrade_service.gosimilarity index 96%rename from backend/internal/services/system_upgrade_service.gorename to backend/internal/system/upgrade_service.goindex ee34fb22b..06df9d792 100644--- a/backend/internal/services/system_upgrade_service.go+++ b/backend/internal/system/upgrade_service.go@@ -1,4 +1,4 @@-package services+package systemimport ("context"@@ -18,14 +18,19 @@ import (cerrdefs "github.com/containerd/errdefs""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/version"dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/vuln""github.com/getarcaneapp/arcane/backend/v2/pkg/projects""github.com/getarcaneapp/arcane/backend/v2/pkg/remenv"- "github.com/getarcaneapp/arcane/types/v2/version"+ versiontypes "github.com/getarcaneapp/arcane/types/v2/version""github.com/moby/moby/api/types/container""github.com/moby/moby/api/types/mount""github.com/moby/moby/client"@@ -40,10 +45,10 @@ type SystemUpgradeService struct {upgrading atomic.BoolupdatingAll atomic.Booldb *database.DB- dockerService *DockerClientService- versionService *VersionService- eventService *EventService- settingsService *SettingsService+ dockerService *docker.DockerClientService+ versionService *version.VersionService+ eventService *event.EventService+ settingsService *settings.SettingsService}type upgraderRuntimeOptionsInternal struct {@@ -54,10 +59,10 @@ type upgraderRuntimeOptionsInternal struct {func NewSystemUpgradeService(db *database.DB,- dockerService *DockerClientService,- versionService *VersionService,- eventService *EventService,- settingsService *SettingsService,+ dockerService *docker.DockerClientService,+ versionService *version.VersionService,+ eventService *event.EventService,+ settingsService *settings.SettingsService,) *SystemUpgradeService {return &SystemUpgradeService{db: db,@@ -338,11 +343,11 @@ func resolveSystemUpgraderRuntimeOptionsInternal(}if scheme != "unix" {- options.NetworkMode = container.NetworkMode(selectTrivyAutoNetworkModeInternal(currentContainer))+ options.NetworkMode = container.NetworkMode(vuln.SelectAutoNetworkMode(currentContainer))return options, nil}- socketSource, err := resolveTrivyUnixSocketSourceInternal(+ socketSource, err := vuln.ResolveUnixSocketSource(ctx,socketPath,discoverHostPath,@@ -456,7 +461,7 @@ const (// self-upgrade as its final step (job left pending_restart, finalized at next// boot). Every environment pulls the latest image, whether or not it reports an// update available.-func (s *SystemUpgradeService) StartUpdateAll(ctx context.Context, user models.User, env *EnvironmentService) (*models.EnvironmentUpdateJob, error) {+func (s *SystemUpgradeService) StartUpdateAll(ctx context.Context, user models.User, env *environment.EnvironmentService) (*models.EnvironmentUpdateJob, error) {// Guard the check-then-create against concurrent callers (e.g. a double-click).// A dedicated flag rather than s.upgrading, because the manager branch below// calls TriggerUpgradeViaCLI which acquires s.upgrading itself. The persisted@@ -478,8 +483,8 @@ func (s *SystemUpgradeService) StartUpdateAll(ctx context.Context, user models.Uinfo := s.versionService.GetAppVersionInfo(ctx)managerResult := models.EnvironmentUpdateResult{- EnvironmentID: LocalEnvironmentID,- EnvironmentName: env.ResolveEnvironmentName(ctx, LocalEnvironmentID),+ EnvironmentID: environment.LocalEnvironmentID,+ EnvironmentName: env.ResolveEnvironmentName(ctx, environment.LocalEnvironmentID),FromVersion: info.CurrentVersion,}@@ -589,7 +594,7 @@ func resolveResumeActionInternal(job *models.EnvironmentUpdateJob, currentVersio// persisting progress after each one so the status endpoint can report live// progress, then triggers the manager's own self-upgrade as the final step// (leaving the job pending_restart for the next boot to finalize).-func (s *SystemUpgradeService) runAgentsPhaseInternal(ctx context.Context, jobID string, env *EnvironmentService, user models.User) {+func (s *SystemUpgradeService) runAgentsPhaseInternal(ctx context.Context, jobID string, env *environment.EnvironmentService, user models.User) {job, err := s.getUpdateAllJobByIDInternal(ctx, jobID)if err != nil || job == nil {slog.WarnContext(ctx, "update-all: failed to reload job for agents phase", "jobId", jobID, "error", err)@@ -626,7 +631,7 @@ func (s *SystemUpgradeService) runAgentsPhaseInternal(ctx context.Context, jobID// that if the manager dies the instant the upgrader starts, the next boot sees// pending_restart and finalizes it.for i := range job.Results {- if job.Results[i].EnvironmentID == LocalEnvironmentID {+ if job.Results[i].EnvironmentID == environment.LocalEnvironmentID {job.Results[i].Status = models.EnvironmentUpdateResultStatusUpdatingbreak}@@ -822,7 +827,7 @@ func (s *SystemUpgradeService) waitForUpgraderExitInternal(ctx context.Context,// seedRemoteResultsInternal builds a pending result row for every remote environment// so the dialog can render the whole fleet immediately. Best effort: on error it// returns nil and the agents phase appends rows as it processes them.-func (s *SystemUpgradeService) seedRemoteResultsInternal(ctx context.Context, env *EnvironmentService) models.EnvironmentUpdateResults {+func (s *SystemUpgradeService) seedRemoteResultsInternal(ctx context.Context, env *environment.EnvironmentService) models.EnvironmentUpdateResults {envs, err := env.ListRemoteEnvironments(ctx)if err != nil {slog.WarnContext(ctx, "update-all: failed to pre-list remote environments for seeding", "error", err)@@ -859,9 +864,9 @@ func upsertPendingResultInternal(job *models.EnvironmentUpdateJob, envID, envNam// upgradeAgentInternal triggers and confirms a single remote environment's// self-upgrade, recording the outcome on result. The upgrade always runs — the// agent pulls the latest image even when it reports no update available.-func (s *SystemUpgradeService) upgradeAgentInternal(ctx context.Context, env *EnvironmentService, envID string, result *models.EnvironmentUpdateResult) {+func (s *SystemUpgradeService) upgradeAgentInternal(ctx context.Context, env *environment.EnvironmentService, envID string, result *models.EnvironmentUpdateResult) {versionCtx, cancel := context.WithTimeout(ctx, updateAllAgentRequestTimeoutInternal)- var info version.Info+ var info versiontypes.Infoerr := env.ProxyJSONRequest(versionCtx, envID, http.MethodGet, "/api/app-version", nil, &info)cancel()if err != nil {@@ -908,7 +913,7 @@ func (s *SystemUpgradeService) upgradeAgentInternal(ctx context.Context, env *En// environment already runs the newest image. Only a positive answer is actionable:// anything unresolved or contradictory reports false, which keeps the full// trigger-and-confirm flow rather than claiming there was nothing to do.-func agentAlreadyOnTargetInternal(info *version.Info) bool {+func agentAlreadyOnTargetInternal(info *versiontypes.Info) bool {if info == nil || info.UpdateAvailable {return false}@@ -957,7 +962,7 @@ func updateAllAgentFailureStatusInternal(err error) models.EnvironmentUpdateResu// force-update of an already-latest agent — including one whose version check// could not determine the latest release — confirms on a same-image recreation// instead of timing out to triggered.-func (s *SystemUpgradeService) confirmAgentUpgradedInternal(ctx context.Context, env *EnvironmentService, envID string, baseline version.Info) bool {+func (s *SystemUpgradeService) confirmAgentUpgradedInternal(ctx context.Context, env *environment.EnvironmentService, envID string, baseline versiontypes.Info) bool {target := updateAllTargetVersionInternal(&baseline)deadline := time.Now().Add(updateAllConfirmTimeoutInternal)ticker := time.NewTicker(updateAllConfirmPollIntervalInternal)@@ -969,7 +974,7 @@ func (s *SystemUpgradeService) confirmAgentUpgradedInternal(ctx context.Context,return falsecase <-ticker.C:reqCtx, cancel := context.WithTimeout(ctx, updateAllAgentRequestTimeoutInternal)- var info version.Info+ var info versiontypes.Infoerr := env.ProxyJSONRequest(reqCtx, envID, http.MethodGet, "/api/app-version", nil, &info)cancel()if err == nil {@@ -1057,7 +1062,7 @@ func (s *SystemUpgradeService) markUpdateAllFailedInternal(ctx context.Context,// Surface the failure in the events audit log; the success path logs via// logUpdateAllEventInternal. LogUserEvent hardcodes an info-severity "completed"// title, so create the event directly with error severity and the reason.- if _, err := s.eventService.CreateEvent(ctx, CreateEventRequest{+ if _, err := s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeSystemUpgrade,Severity: models.EventSeverityError,Title: "Update all environments failed",@@ -1082,7 +1087,7 @@ func (s *SystemUpgradeService) markUpdateAllFailedInternal(ctx context.Context,// in, or failed otherwise.func (s *SystemUpgradeService) recordManagerResultInternal(job *models.EnvironmentUpdateJob, status models.EnvironmentUpdateResultStatus, currentVersion string) {for i := range job.Results {- if job.Results[i].EnvironmentID != LocalEnvironmentID {+ if job.Results[i].EnvironmentID != environment.LocalEnvironmentID {continue}job.Results[i].Status = status@@ -1139,7 +1144,7 @@ func (s *SystemUpgradeService) logUpdateAllEventInternal(ctx context.Context, jo// The job ran to completion but some environments failed to update — record a// warning-severity event so those failures still show in the audit log.- if _, err := s.eventService.CreateEvent(ctx, CreateEventRequest{+ if _, err := s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeSystemUpgrade,Severity: models.EventSeverityWarning,Title: "Update all environments completed with errors",@@ -1159,7 +1164,7 @@ func (s *SystemUpgradeService) logUpdateAllEventInternal(ctx context.Context, jo// force-update with an unknown latest is wherever the pull lands — recording the// current version keeps the resume check able to recognize a same-image recreation// as success instead of finalizing it as failed.-func updateAllTargetVersionInternal(info *version.Info) string {+func updateAllTargetVersionInternal(info *versiontypes.Info) string {if info == nil {return ""}diff --git a/backend/internal/services/system_upgrade_service_test.go b/backend/internal/system/upgrade_service_test.gosimilarity index 99%rename from backend/internal/services/system_upgrade_service_test.gorename to backend/internal/system/upgrade_service_test.goindex 9b2b8432d..eda6b3538 100644--- a/backend/internal/services/system_upgrade_service_test.go+++ b/backend/internal/system/upgrade_service_test.go@@ -1,4 +1,4 @@-package services+package systemimport ("context"diff --git a/backend/internal/services/system_upgrade_update_all_test.go b/backend/internal/system/upgrade_update_all_test.gosimilarity index 91%rename from backend/internal/services/system_upgrade_update_all_test.gorename to backend/internal/system/upgrade_update_all_test.goindex 5be06144b..a0e8e8a66 100644--- a/backend/internal/services/system_upgrade_update_all_test.go+++ b/backend/internal/system/upgrade_update_all_test.go@@ -1,13 +1,16 @@-package services+package systemimport ("context""testing""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/version"+"github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/types/v2/version"+ versiontypes "github.com/getarcaneapp/arcane/types/v2/version""github.com/libtnb/sqlite""github.com/stretchr/testify/require""gorm.io/gorm"@@ -92,9 +95,9 @@ func TestUpdateAllResolveResumeAction(t *testing.T) {// must still record a target — the current identifiers — so the resume check can// recognize a same-image recreation as success instead of finalizing it as failed.func TestUpdateAllTargetVersionFallsBackToCurrent(t *testing.T) {- require.Equal(t, "v2.0.0", updateAllTargetVersionInternal(&version.Info{NewestVersion: "v2.0.0", CurrentVersion: "v1.2.3"}))- require.Equal(t, "v1.2.3", updateAllTargetVersionInternal(&version.Info{CurrentVersion: "v1.2.3", CurrentDigest: "sha256:a"}))- require.Equal(t, "sha256:a", updateAllTargetVersionInternal(&version.Info{CurrentDigest: "sha256:a"}))+ require.Equal(t, "v2.0.0", updateAllTargetVersionInternal(&versiontypes.Info{NewestVersion: "v2.0.0", CurrentVersion: "v1.2.3"}))+ require.Equal(t, "v1.2.3", updateAllTargetVersionInternal(&versiontypes.Info{CurrentVersion: "v1.2.3", CurrentDigest: "sha256:a"}))+ require.Equal(t, "sha256:a", updateAllTargetVersionInternal(&versiontypes.Info{CurrentDigest: "sha256:a"}))}func TestUpsertPendingResult(t *testing.T) {@@ -143,7 +146,7 @@ func TestUpdateAllFailedJobMarksUpdatingResultsFailed(t *testing.T) {db := &database.DB{DB: gormDB}require.NoError(t, db.AutoMigrate(&models.EnvironmentUpdateJob{}, &models.Event{}))- svc := NewSystemUpgradeService(db, nil, nil, NewEventService(db, nil, nil), nil)+ svc := NewSystemUpgradeService(db, nil, nil, event.NewEventService(db, nil, nil), nil)job := &models.EnvironmentUpdateJob{Status: models.EnvironmentUpdateJobStatusRunning,UserID: "user-1",@@ -192,7 +195,7 @@ func TestUpdateAllFinalizesUpToDateManagerWithoutRestart(t *testing.T) {db := &database.DB{DB: gormDB}require.NoError(t, db.AutoMigrate(&models.EnvironmentUpdateJob{}, &models.Event{}))- svc := NewSystemUpgradeService(db, nil, nil, NewEventService(db, nil, nil), nil)+ svc := NewSystemUpgradeService(db, nil, nil, event.NewEventService(db, nil, nil), nil)job := &models.EnvironmentUpdateJob{Status: models.EnvironmentUpdateJobStatusPendingRestart,UserID: "user-1",@@ -229,22 +232,22 @@ func TestUpdateAllFinalizesUpToDateManagerWithoutRestart(t *testing.T) {func TestAgentAlreadyOnTarget(t *testing.T) {tests := []struct {name string- info version.Info+ info versiontypes.Infowant bool}{{name: "on newest version",- info: version.Info{CurrentVersion: "1.2.3", NewestVersion: "v1.2.3"},+ info: versiontypes.Info{CurrentVersion: "1.2.3", NewestVersion: "v1.2.3"},want: true,},{name: "update available",- info: version.Info{CurrentVersion: "1.2.3", NewestVersion: "v1.3.0", UpdateAvailable: true},+ info: versiontypes.Info{CurrentVersion: "1.2.3", NewestVersion: "v1.3.0", UpdateAvailable: true},want: false,},{name: "on newest digest",- info: version.Info{CurrentDigest: "sha256:abc", NewestDigest: "sha256:abc"},+ info: versiontypes.Info{CurrentDigest: "sha256:abc", NewestDigest: "sha256:abc"},want: true,},{@@ -252,7 +255,7 @@ func TestAgentAlreadyOnTarget(t *testing.T) {// update, but the differing digest means the pull really will replace the// image, so this must not be treated as already current.name: "same version, rebuilt digest",- info: version.Info{+ info: versiontypes.Info{CurrentVersion: "1.2.3",NewestVersion: "v1.2.3",CurrentDigest: "sha256:old",@@ -264,7 +267,7 @@ func TestAgentAlreadyOnTarget(t *testing.T) {// Only one digest resolved, so a rebuild cannot be ruled out: the matching// version tag must not be enough on its own.name: "same version, remote digest unresolved",- info: version.Info{+ info: versiontypes.Info{CurrentVersion: "1.2.3",NewestVersion: "v1.2.3",CurrentDigest: "sha256:running",@@ -273,7 +276,7 @@ func TestAgentAlreadyOnTarget(t *testing.T) {},{name: "inconclusive check resolves nothing",- info: version.Info{CurrentVersion: "1.2.3"},+ info: versiontypes.Info{CurrentVersion: "1.2.3"},want: false,},}@@ -299,8 +302,8 @@ func TestResumeUpdateAllFinalizesManagerWithoutRerunningAgents(t *testing.T) {// disabled=true keeps GetAppVersionInfo offline; nil docker => empty current digest.// The reported version differs from ManagerVersionAtStart, so the manager upgrade// is judged successful.- versionSvc := NewVersionService(nil, true, "v9.9.9-new", "", nil, nil, nil)- svc := NewSystemUpgradeService(db, nil, versionSvc, NewEventService(db, nil, nil), nil)+ versionSvc := version.NewVersionService(nil, true, "v9.9.9-new", "", nil, nil, nil)+ svc := NewSystemUpgradeService(db, nil, versionSvc, event.NewEventService(db, nil, nil), nil)job := &models.EnvironmentUpdateJob{Status: models.EnvironmentUpdateJobStatusPendingRestart,diff --git a/backend/api/handlers/templates.go b/backend/internal/template/handler.gosimilarity index 83%rename from backend/api/handlers/templates.gorename to backend/internal/template/handler.goindex faa1667c5..f3904d67f 100644--- a/backend/api/handlers/templates.go+++ b/backend/internal/template/handler.go@@ -1,4 +1,4 @@-package handlers+package templateimport ("context"@@ -7,19 +7,19 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper""github.com/getarcaneapp/arcane/types/v2/base"- "github.com/getarcaneapp/arcane/types/v2/template"+ templatetypes "github.com/getarcaneapp/arcane/types/v2/template")// TemplateHandler handles template management endpoints.type TemplateHandler struct {- templateService *services.TemplateService+ templateService *TemplateService}// ============================================================================@@ -36,13 +36,13 @@ type ListTemplatesInput struct {}type ListTemplatesOutput struct {- Body base.Paginated[template.Template]+ Body base.Paginated[templatetypes.Template]}type GetAllTemplatesInput struct{}type GetAllTemplatesOutput struct {- Body base.ApiResponse[[]template.Template]+ Body base.ApiResponse[[]templatetypes.Template]}type GetTemplateInput struct {@@ -50,7 +50,7 @@ type GetTemplateInput struct {}type GetTemplateOutput struct {- Body base.ApiResponse[template.Template]+ Body base.ApiResponse[templatetypes.Template]}type GetTemplateContentInput struct {@@ -58,24 +58,24 @@ type GetTemplateContentInput struct {}type GetTemplateContentOutput struct {- Body base.ApiResponse[template.TemplateContent]+ Body base.ApiResponse[templatetypes.TemplateContent]}type CreateTemplateInput struct {- Body template.CreateRequest+ Body templatetypes.CreateRequest}type CreateTemplateOutput struct {- Body base.ApiResponse[template.Template]+ Body base.ApiResponse[templatetypes.Template]}type UpdateTemplateInput struct {ID string `path:"id" doc:"Template ID"`- Body template.UpdateRequest+ Body templatetypes.UpdateRequest}type UpdateTemplateOutput struct {- Body base.ApiResponse[template.Template]+ Body base.ApiResponse[templatetypes.Template]}type DeleteTemplateInput struct {@@ -91,17 +91,17 @@ type DownloadTemplateInput struct {}type DownloadTemplateOutput struct {- Body base.ApiResponse[template.Template]+ Body base.ApiResponse[templatetypes.Template]}type GetDefaultTemplatesInput struct{}type GetDefaultTemplatesOutput struct {- Body base.ApiResponse[template.DefaultTemplatesResponse]+ Body base.ApiResponse[templatetypes.DefaultTemplatesResponse]}type SaveDefaultTemplatesInput struct {- Body template.SaveDefaultTemplatesRequest+ Body templatetypes.SaveDefaultTemplatesRequest}type SaveDefaultTemplatesOutput struct {@@ -111,20 +111,20 @@ type SaveDefaultTemplatesOutput struct {type GetTemplateRegistriesInput struct{}type GetTemplateRegistriesOutput struct {- Body base.ApiResponse[[]template.TemplateRegistry]+ Body base.ApiResponse[[]templatetypes.TemplateRegistry]}type CreateTemplateRegistryInput struct {- Body template.CreateRegistryRequest+ Body templatetypes.CreateRegistryRequest}type CreateTemplateRegistryOutput struct {- Body base.ApiResponse[template.TemplateRegistry]+ Body base.ApiResponse[templatetypes.TemplateRegistry]}type UpdateTemplateRegistryInput struct {ID string `path:"id" doc:"Registry ID"`- Body template.UpdateRegistryRequest+ Body templatetypes.UpdateRegistryRequest}type UpdateTemplateRegistryOutput struct {@@ -144,7 +144,7 @@ type FetchTemplateRegistryInput struct {}type FetchTemplateRegistryOutput struct {- Body base.ApiResponse[template.RemoteRegistry]+ Body base.ApiResponse[templatetypes.RemoteRegistry]}// ============================================================================@@ -152,7 +152,7 @@ type FetchTemplateRegistryOutput struct {// ============================================================================// RegisterTemplates registers all template management endpoints.-func RegisterTemplates(api huma.API, templateService *services.TemplateService) {+func RegisterTemplates(api huma.API, templateService *TemplateService) {h := &TemplateHandler{templateService: templateService}// Template registry endpoint.@@ -163,8 +163,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Fetch remote registry",Description: "Fetch templates from a remote registry URL",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesRead),}, h.FetchRegistry)huma.Register(api, huma.Operation{@@ -174,7 +174,7 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "List templates (paginated)",Description: "Get a paginated list of compose templates",Tags: []string{"Templates"},- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesList),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesList),}, h.ListTemplates)huma.Register(api, huma.Operation{@@ -184,7 +184,7 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "List all templates",Description: "Get all compose templates without pagination",Tags: []string{"Templates"},- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesList),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesList),}, h.GetAllTemplates)huma.Register(api, huma.Operation{@@ -194,7 +194,7 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Get a template",Description: "Get a compose template by ID",Tags: []string{"Templates"},- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesRead),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesRead),}, h.GetTemplate)huma.Register(api, huma.Operation{@@ -204,7 +204,7 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Get template content",Description: "Get the compose content for a template with parsed data",Tags: []string{"Templates"},- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesRead),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesRead),}, h.GetTemplateContent)// Protected endpoints@@ -215,8 +215,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Create a template",Description: "Create a new compose template",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesCreate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesCreate),}, h.CreateTemplate)huma.Register(api, huma.Operation{@@ -226,8 +226,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Update a template",Description: "Update an existing compose template",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesUpdate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesUpdate),}, h.UpdateTemplate)huma.Register(api, huma.Operation{@@ -237,8 +237,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Delete a template",Description: "Delete a compose template",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesDelete),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesDelete),}, h.DeleteTemplate)huma.Register(api, huma.Operation{@@ -248,8 +248,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Download a template",Description: "Download a remote template to local storage",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesRead),}, h.DownloadTemplate)huma.Register(api, huma.Operation{@@ -259,8 +259,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Get default templates",Description: "Get the default compose and env templates",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesRead),}, h.GetDefaultTemplates)huma.Register(api, huma.Operation{@@ -270,8 +270,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Save default templates",Description: "Save the default compose and env templates",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesUpdate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesUpdate),}, h.SaveDefaultTemplates)huma.Register(api, huma.Operation{@@ -281,8 +281,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "List template registries",Description: "Get all template registries",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesList),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesList),}, h.GetRegistries)huma.Register(api, huma.Operation{@@ -292,8 +292,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Create a template registry",Description: "Create a new template registry",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesCreate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesCreate),}, h.CreateRegistry)huma.Register(api, huma.Operation{@@ -303,8 +303,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Update a template registry",Description: "Update an existing template registry",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesUpdate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesUpdate),}, h.UpdateRegistry)huma.Register(api, huma.Operation{@@ -314,8 +314,8 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)Summary: "Delete a template registry",Description: "Delete a template registry",Tags: []string{"Templates"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermTemplatesDelete),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermTemplatesDelete),}, h.DeleteRegistry)}@@ -325,7 +325,7 @@ func RegisterTemplates(api huma.API, templateService *services.TemplateService)// ListTemplates returns a paginated list of templates.func (h *TemplateHandler) ListTemplates(ctx context.Context, input *ListTemplatesInput) (*ListTemplatesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if params.Limit == 0 {params.Limit = 20}@@ -339,10 +339,10 @@ func (h *TemplateHandler) ListTemplates(ctx context.Context, input *ListTemplate}return &ListTemplatesOutput{- Body: base.Paginated[template.Template]{+ Body: base.Paginated[templatetypes.Template]{Success: true,Data: templates,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -354,13 +354,13 @@ func (h *TemplateHandler) GetAllTemplates(ctx context.Context, _ *GetAllTemplatereturn nil, huma.Error500InternalServerError(errors.WithMessage(err, "Failed to get templates").Error())}- out, mapErr := mapper.MapSlice[models.ComposeTemplate, template.Template](templates)+ out, mapErr := mapper.MapSlice[models.ComposeTemplate, templatetypes.Template](templates)if mapErr != nil {return nil, huma.Error500InternalServerError(errors.WithMessage(mapErr, "Failed to map templates").Error())}return &GetAllTemplatesOutput{- Body: base.ApiResponse[[]template.Template]{+ Body: base.ApiResponse[[]templatetypes.Template]{Success: true,Data: out,},@@ -389,13 +389,13 @@ func (h *TemplateHandler) GetTemplate(ctx context.Context, input *GetTemplateInpreturn nil, huma.Error500InternalServerError(errors.WithMessage(err, "Failed to get template").Error())}- var out template.Template+ var out templatetypes.Templateif mapErr := mapper.MapStruct(tmpl, &out); mapErr != nil {return nil, huma.Error500InternalServerError(errors.WithMessage(mapErr, "Failed to map templates").Error())}return &GetTemplateOutput{- Body: base.ApiResponse[template.Template]{+ Body: base.ApiResponse[templatetypes.Template]{Success: true,Data: out,},@@ -422,14 +422,14 @@ func (h *TemplateHandler) GetTemplateContent(ctx context.Context, input *GetTemp}return &GetTemplateContentOutput{- Body: base.ApiResponse[template.TemplateContent]{+ Body: base.ApiResponse[templatetypes.TemplateContent]{Success: true,Data: *contentData,},}, nil}-// CreateTemplate creates a new template.+// CreateTemplate creates a new templatetypes.func (h *TemplateHandler) CreateTemplate(ctx context.Context, input *CreateTemplateInput) (*CreateTemplateOutput, error) {tmpl := &models.ComposeTemplate{Name: input.Body.Name,@@ -446,20 +446,20 @@ func (h *TemplateHandler) CreateTemplate(ctx context.Context, input *CreateTemplreturn nil, huma.Error500InternalServerError(errors.WithMessage(err, "Failed to create template").Error())}- var out template.Template+ var out templatetypes.Templateif mapErr := mapper.MapStruct(tmpl, &out); mapErr != nil {return nil, huma.Error500InternalServerError(errors.WithMessage(mapErr, "Failed to map templates").Error())}return &CreateTemplateOutput{- Body: base.ApiResponse[template.Template]{+ Body: base.ApiResponse[templatetypes.Template]{Success: true,Data: out,},}, nil}-// UpdateTemplate updates a template.+// UpdateTemplate updates a templatetypes.func (h *TemplateHandler) UpdateTemplate(ctx context.Context, input *UpdateTemplateInput) (*UpdateTemplateOutput, error) {if input.ID == "" {return nil, huma.Error400BadRequest("Template ID is required")@@ -493,20 +493,20 @@ func (h *TemplateHandler) UpdateTemplate(ctx context.Context, input *UpdateTemplreturn nil, huma.Error500InternalServerError(errors.WithMessage(err, "Failed to get template").Error())}- var out template.Template+ var out templatetypes.Templateif mapErr := mapper.MapStruct(updated, &out); mapErr != nil {return nil, huma.Error500InternalServerError(errors.WithMessage(mapErr, "Failed to map templates").Error())}return &UpdateTemplateOutput{- Body: base.ApiResponse[template.Template]{+ Body: base.ApiResponse[templatetypes.Template]{Success: true,Data: out,},}, nil}-// DeleteTemplate deletes a template.+// DeleteTemplate deletes a templatetypes.func (h *TemplateHandler) DeleteTemplate(ctx context.Context, input *DeleteTemplateInput) (*DeleteTemplateOutput, error) {if input.ID == "" {return nil, huma.Error400BadRequest("Template ID is required")@@ -561,13 +561,13 @@ func (h *TemplateHandler) DownloadTemplate(ctx context.Context, input *DownloadTreturn nil, huma.Error500InternalServerError(errors.WithMessage(err, "Failed to download template").Error())}- var out template.Template+ var out templatetypes.Templateif mapErr := mapper.MapStruct(localTemplate, &out); mapErr != nil {return nil, huma.Error500InternalServerError(errors.WithMessage(mapErr, "Failed to map templates").Error())}return &DownloadTemplateOutput{- Body: base.ApiResponse[template.Template]{+ Body: base.ApiResponse[templatetypes.Template]{Success: true,Data: out,},@@ -582,9 +582,9 @@ func (h *TemplateHandler) GetDefaultTemplates(ctx context.Context, _ *GetDefaultenvTemplate := h.templateService.GetEnvTemplate()return &GetDefaultTemplatesOutput{- Body: base.ApiResponse[template.DefaultTemplatesResponse]{+ Body: base.ApiResponse[templatetypes.DefaultTemplatesResponse]{Success: true,- Data: template.DefaultTemplatesResponse{+ Data: templatetypes.DefaultTemplatesResponse{ComposeTemplate: composeTemplate,SwarmStackTemplate: swarmStackTemplate,SwarmStackEnvTemplate: swarmStackEnvTemplate,@@ -621,7 +621,7 @@ func (h *TemplateHandler) GetRegistries(ctx context.Context, _ *GetTemplateRegisreturn nil, huma.Error500InternalServerError("Failed to fetch registry")}- out, mapErr := mapper.MapSlice[models.TemplateRegistry, template.TemplateRegistry](registries)+ out, mapErr := mapper.MapSlice[models.TemplateRegistry, templatetypes.TemplateRegistry](registries)if mapErr != nil {return nil, huma.Error500InternalServerError("Failed to fetch registry")}@@ -637,7 +637,7 @@ func (h *TemplateHandler) GetRegistries(ctx context.Context, _ *GetTemplateRegis}return &GetTemplateRegistriesOutput{- Body: base.ApiResponse[[]template.TemplateRegistry]{+ Body: base.ApiResponse[[]templatetypes.TemplateRegistry]{Success: true,Data: out,},@@ -656,13 +656,13 @@ func (h *TemplateHandler) CreateRegistry(ctx context.Context, input *CreateTemplreturn nil, huma.Error500InternalServerError(errors.WithMessage(err, "Failed to create registry").Error())}- var out template.TemplateRegistry+ var out templatetypes.TemplateRegistryif mapErr := mapper.MapStruct(registry, &out); mapErr != nil {return nil, huma.Error500InternalServerError(errors.WithMessage(mapErr, "Failed to map registry").Error())}return &CreateTemplateRegistryOutput{- Body: base.ApiResponse[template.TemplateRegistry]{+ Body: base.ApiResponse[templatetypes.TemplateRegistry]{Success: true,Data: out,},@@ -732,13 +732,13 @@ func (h *TemplateHandler) FetchRegistry(ctx context.Context, input *FetchTemplatreturn nil, huma.Error502BadGateway("Failed to fetch registry")}- var registry template.RemoteRegistry+ var registry templatetypes.RemoteRegistryif err := json.Unmarshal(body, ®istry); err != nil {return nil, huma.Error502BadGateway("Invalid JSON response")}return &FetchTemplateRegistryOutput{- Body: base.ApiResponse[template.RemoteRegistry]{+ Body: base.ApiResponse[templatetypes.RemoteRegistry]{Success: true,Data: registry,},diff --git a/backend/internal/template/module.go b/backend/internal/template/module.gonew file mode 100644index 000000000..7adb13eb1--- /dev/null+++ b/backend/internal/template/module.go@@ -0,0 +1,42 @@+// Package template owns compose templates and the template HTTP surface.+package template++import (+ "context"+ "net/http"++ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+)++type Dependencies struct {+ Context context.Context+ DB *database.DB+ HTTPClient *http.Client+ Settings *settings.SettingsService+}++type Module struct {+ service *TemplateService+}++func New(deps Dependencies) *Module {+ return &Module{service: NewTemplateService(deps.Context, deps.DB, deps.HTTPClient, deps.Settings)}+}++func (m *Module) Service() *TemplateService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterTemplates(api, nil)+ return+ }+ RegisterTemplates(api, m.service)+}diff --git a/backend/internal/services/template_service.go b/backend/internal/template/service.gosimilarity index 98%rename from backend/internal/services/template_service.gorename to backend/internal/template/service.goindex d4cb861e9..f4e018e70 100644--- a/backend/internal/services/template_service.go+++ b/backend/internal/template/service.go@@ -1,4 +1,4 @@-package services+package templateimport ("context"@@ -21,6 +21,7 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/backend/v2/pkg/projects""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"@@ -45,7 +46,7 @@ type TemplateService struct {httpClient *http.ClientsafeHTTPClient *http.ClientlookupIP httputils.LookupIPFunc- settingsService *SettingsService+ settingsService *settings.SettingsServiceremoteCache *hot.HotCache[struct{}, []models.ComposeTemplate]@@ -70,7 +71,7 @@ const remoteIDPrefix = "remote"var errNoRemoteTemplates = errors.New("remote template registries returned no templates")-func NewTemplateService(ctx context.Context, db *database.DB, httpClient *http.Client, settingsService *SettingsService) *TemplateService {+func NewTemplateService(ctx context.Context, db *database.DB, httpClient *http.Client, settingsService *settings.SettingsService) *TemplateService {if httpClient == nil {httpClient = http.DefaultClient}@@ -232,7 +233,7 @@ func (s *TemplateService) GetAllTemplatesPaginated(ctx context.Context, params p}result := pagination.SearchOrderAndPaginate(items, params, config)- paginationResp := pagination.BuildResponseFromFilterResult(result, params)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, params)return result.Items, paginationResp, nil}@@ -1108,7 +1109,7 @@ func (s *TemplateService) ParseComposeServices(ctx context.Context, composeConte// Create a dummy .env file to prevent env file errorsenvPath := filepath.Join(tmpDir, ".env")- if err := projects.WriteFileWithPerm(envPath, "", common.FilePerm); err != nil {+ if err := projects.WriteFileWithPerm(envPath, "", utils.FilePerm); err != nil {slog.WarnContext(ctx, "Failed to create dummy env file", "error", err)}@@ -1150,7 +1151,7 @@ func (s *TemplateService) resolveTemplateIconURL(ctx context.Context, composeCondefer func() { _ = os.RemoveAll(tmpDir) }()envPath := filepath.Join(tmpDir, ".env")- if err := projects.WriteFileWithPerm(envPath, envContent, common.FilePerm); err != nil {+ if err := projects.WriteFileWithPerm(envPath, envContent, utils.FilePerm); err != nil {slog.WarnContext(ctx, "failed to create temp env file for template icon parsing", "error", err)}diff --git a/backend/internal/services/template_service_test.go b/backend/internal/template/service_test.gosimilarity index 93%rename from backend/internal/services/template_service_test.gorename to backend/internal/template/service_test.goindex 32757bb91..23301701a 100644--- a/backend/internal/services/template_service_test.go+++ b/backend/internal/template/service_test.go@@ -1,4 +1,4 @@-package services+package templateimport ("context"@@ -17,11 +17,14 @@ import ("github.com/libtnb/sqlite""github.com/samber/mo""github.com/stretchr/testify/require"+ "go.uber.org/fx/fxtest""gorm.io/gorm"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"httputils "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/httpx"tmpl "github.com/getarcaneapp/arcane/types/v2/template"@@ -478,7 +481,7 @@ func TestGetTemplate_ForceRefreshesRemoteCacheOnMiss(t *testing.T) {require.Contains(t, err.Error(), "not found")}-func minimalSettingsServiceForTest(t *testing.T) *SettingsService {+func minimalSettingsServiceForTest(t *testing.T) *settings.SettingsService {t.Helper()db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})require.NoError(t, err)@@ -487,3 +490,22 @@ func minimalSettingsServiceForTest(t *testing.T) *SettingsService {require.NoError(t, err)return svc}++func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ executor, err := actors.NewExecutor(t.Context(), runtime, "template-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "template-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, executor.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, executor, effects)+}diff --git a/backend/api/handlers/updater.go b/backend/internal/updater/handler.gosimilarity index 86%rename from backend/api/handlers/updater.gorename to backend/internal/updater/handler.goindex 097288783..14d16d9e1 100644--- a/backend/api/handlers/updater.go+++ b/backend/internal/updater/handler.go@@ -1,4 +1,4 @@-package handlers+package updaterimport ("context"@@ -6,18 +6,18 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/updater")// UpdaterHandler provides Huma-based updater management endpoints.type UpdaterHandler struct {- updaterService *services.UpdaterService+ updaterService *UpdaterServiceappCtx context.Context}@@ -59,50 +59,50 @@ type GetUpdaterHistoryOutput struct {}// RegisterUpdater registers updater management routes using Huma.-func RegisterUpdater(api huma.API, updaterService *services.UpdaterService, appCtx ActivityAppContext) {+func RegisterUpdater(api huma.API, updaterService *UpdaterService, appCtx handlerutil.ActivityAppContext) {h := &UpdaterHandler{updaterService: updaterService,- appCtx: appCtx.contextInternal(),+ appCtx: appCtx.Context(),}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "run-updater",Method: http.MethodPost,Path: "/environments/{id}/updater/run",Summary: "Run updater",Description: "Apply pending container updates",Tags: []string{"Updater"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesCheck, h.RunUpdater)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-updater-status",Method: http.MethodGet,Path: "/environments/{id}/updater/status",Summary: "Get updater status",Description: "Get the current status of the updater",Tags: []string{"Updater"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesRead, h.GetUpdaterStatus)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-updater-history",Method: http.MethodGet,Path: "/environments/{id}/updater/history",Summary: "Get updater history",Description: "Get the history of update operations",Tags: []string{"Updater"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesRead, h.GetUpdaterHistory)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-container",Method: http.MethodPost,Path: "/environments/{id}/containers/{containerId}/update",Summary: "Update a single container",Description: "Pull the latest image and apply the appropriate update strategy for a specific container",Tags: []string{"Updater", "Containers"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermImageUpdatesCheck, h.UpdateContainer)}diff --git a/backend/internal/updater/module.go b/backend/internal/updater/module.gonew file mode 100644index 000000000..250495b37--- /dev/null+++ b/backend/internal/updater/module.go@@ -0,0 +1,79 @@+// Package updater owns automatic container and project updates: deciding what+// is out of date, applying the update, and the HTTP surface that drives it.+package updater++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+)++// Dependencies are the collaborators the updater domain needs.+type Dependencies struct {+ DB *database.DB+ Settings *settings.SettingsService+ Docker *docker.DockerClientService+ Project *project.ProjectService+ ImageUpdate *imageupdate.ImageUpdateService+ Registry *registry.ContainerRegistryService+ Event *event.EventService+ Image *image.ImageService+ Notification *notification.NotificationService+ SelfUpgrade selfUpgradeServiceInternal+ Activity *activity.ActivityService+}++// Module wires the updater domain and mounts its routes.+type Module struct {+ service *UpdaterService+}++// New builds the updater domain from its dependencies.+func New(deps Dependencies) (*Module, error) {+ service, err := NewUpdaterService(+ deps.DB,+ deps.Settings,+ deps.Docker,+ deps.Project,+ deps.ImageUpdate,+ deps.Registry,+ deps.Event,+ deps.Image,+ deps.Notification,+ deps.SelfUpgrade,+ deps.Activity,+ )+ if err != nil {+ return nil, err+ }+ return &Module{service: service}, nil+}++// Service exposes the updater service to collaborators that trigger updates+// outside the HTTP surface, such as the auto-update job and webhooks.+func (m *Module) Service() *UpdaterService {+ if m == nil {+ return nil+ }+ return m.service+}++// RegisterRoutes mounts the updater endpoints. A nil module still registers, so+// OpenAPI spec generation can discover the routes without a service graph.+func (m *Module) RegisterRoutes(api huma.API, appCtx handlerutil.ActivityAppContext) {+ if m == nil {+ RegisterUpdater(api, nil, appCtx)+ return+ }+ RegisterUpdater(api, m.service, appCtx)+}diff --git a/backend/internal/services/updater_service.go b/backend/internal/updater/service.gosimilarity index 96%rename from backend/internal/services/updater_service.gorename to backend/internal/updater/service.goindex 2278a0ac5..625523fb6 100644--- a/backend/internal/services/updater_service.go+++ b/backend/internal/updater/service.go@@ -1,4 +1,4 @@-package services+package updaterimport ("context"@@ -15,9 +15,18 @@ import ("emperror.dev/errors""github.com/compose-spec/compose-go/v2/loader"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity"@@ -47,16 +56,16 @@ type UpdaterService struct {type updaterDependenciesInternal struct {DB *database.DB- Docker *DockerClientService- Settings *SettingsService- Projects *ProjectService- ImagePuller *ImageService- ImageUpdates *ImageUpdateService- RegistryDigestResolver *ContainerRegistryService- Events *EventService- Notifications *NotificationService+ Docker *docker.DockerClientService+ Settings *settings.SettingsService+ Projects *project.ProjectService+ ImagePuller *image.ImageService+ ImageUpdates *imageupdate.ImageUpdateService+ RegistryDigestResolver *registry.ContainerRegistryService+ Events *event.EventService+ Notifications *notification.NotificationServiceSelfUpgrade selfUpgradeServiceInternal- Activity *ActivityService+ Activity *activity.ActivityServiceSystemUser models.UserLogger *slog.Logger}@@ -70,16 +79,16 @@ type selfUpgradeServiceInternal interface {// NewUpdaterService constructs the Arcane updater facade.func NewUpdaterService(db *database.DB,- settings *SettingsService,- docker *DockerClientService,- projects *ProjectService,- imageUpdates *ImageUpdateService,- registries *ContainerRegistryService,- events *EventService,- imageSvc *ImageService,- notifications *NotificationService,+ settings *settings.SettingsService,+ docker *docker.DockerClientService,+ projects *project.ProjectService,+ imageUpdates *imageupdate.ImageUpdateService,+ registries *registry.ContainerRegistryService,+ events *event.EventService,+ imageSvc *image.ImageService,+ notifications *notification.NotificationService,upgrade selfUpgradeServiceInternal,- activityService *ActivityService,+ activityService *activity.ActivityService,) (*UpdaterService, error) {service := &UpdaterService{deps: updaterDependenciesInternal{@@ -94,7 +103,7 @@ func NewUpdaterService(Notifications: notifications,SelfUpgrade: upgrade,Activity: activityService,- SystemUser: systemUser,+ SystemUser: models.SystemUser,},}engine, err := updater.New(service.configInternal())@@ -474,7 +483,7 @@ func (s *UpdaterService) recordAutoUpdateEventInternal(ctx context.Context, severeturn}phase, _ := metadata["phase"].(string)- _, err := s.deps.Events.CreateEvent(ctx, CreateEventRequest{+ _, err := s.deps.Events.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeSystemAutoUpdate,Severity: severity,Title: autoUpdateEventTitleInternal(phase, metadata),@@ -554,7 +563,7 @@ func (s *UpdaterService) DockerClient(ctx context.Context) (*client.Client, erro}// PullImage pulls an image through Arcane's image service. The pull is-// bounded by the dockerImagePullTimeout setting — ImageService.PullImage does+// bounded by the dockerImagePullTimeout setting — image.ImageService.PullImage does// not bound itself, and an unbounded engine pull would hold the auto-update// run (and its activity slot) indefinitely.func (s *UpdaterService) PullImage(ctx context.Context, imageRef string, progress io.Writer) error {@@ -573,7 +582,7 @@ func (s *UpdaterService) PullImage(ctx context.Context, imageRef string, progresdefer cancelPull()if s.deps.Projects != nil {- resolved, err := s.deps.Projects.resolveRegistryCredentialsInternal(pullCtx)+ resolved, err := s.deps.Projects.ResolveRegistryCredentials(pullCtx)if err != nil {return errors.WrapIf(err, "resolve registry credentials")}@@ -598,7 +607,7 @@ func (s *UpdaterService) PendingImageUpdates(ctx context.Context) ([]updater.Ima// consumes and clears these records, otherwise the notification for an// update applied here is silently lost (#3132).if s.deps.ImageUpdates != nil {- s.deps.ImageUpdates.sendBatchImageUpdateNotificationsInternal(ctx)+ s.deps.ImageUpdates.SendBatchUpdateNotifications(ctx)}s.appendAutoUpdateActivityMessageInternal(ctx,diff --git a/backend/internal/services/updater_service_test.go b/backend/internal/updater/service_test.gosimilarity index 81%rename from backend/internal/services/updater_service_test.gorename to backend/internal/updater/service_test.goindex 0501f2c25..2ba487c23 100644--- a/backend/internal/services/updater_service_test.go+++ b/backend/internal/updater/service_test.go@@ -1,4 +1,4 @@-package services+package updaterimport ("bytes"@@ -15,10 +15,23 @@ import ("time""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"arcaneupdater "github.com/getarcaneapp/arcane/types/v2/updater"+ "github.com/libtnb/sqlite"+ dockerauthconfig "github.com/moby/moby/api/pkg/authconfig""github.com/moby/moby/api/types/container"dockertypesimage "github.com/moby/moby/api/types/image"+ dockerregistry "github.com/moby/moby/api/types/registry""github.com/moby/moby/client""github.com/stretchr/testify/assert""github.com/stretchr/testify/require"@@ -26,6 +39,7 @@ import ("go.getarcane.app/updater""go.getarcane.app/updater/labels""go.getarcane.app/updater/refs"+ "gorm.io/gorm")type fakeDockerClientProviderInternal struct {@@ -194,7 +208,7 @@ func (m *mockSystemUpgradeServiceInternal) TriggerUpgradeViaCLI(_ context.Contexfunc TestUpdaterService_ApplyPendingNoRecordsInternal(t *testing.T) {ctx := context.Background()- db := setupProjectTestDB(t)+ db := setupProjectTestDBInternal(t)svc, svcErr := NewUpdaterService(db, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil)require.NoError(t, svcErr)@@ -261,8 +275,8 @@ func TestUpdaterService_TriggerSelfUpdateViaCLIInternal(t *testing.T) {require.NoError(t, err)assert.True(t, mockUpgrade.triggerCalled)require.NotNil(t, mockUpgrade.capturedUser)- assert.Equal(t, systemUser.ID, mockUpgrade.capturedUser.ID)- assert.Equal(t, systemUser.Username, mockUpgrade.capturedUser.Username)+ assert.Equal(t, models.SystemUser.ID, mockUpgrade.capturedUser.ID)+ assert.Equal(t, models.SystemUser.Username, mockUpgrade.capturedUser.Username)require.NotNil(t, mockUpgrade.capturedTarget)assert.Equal(t, "container-1", mockUpgrade.capturedTarget.ContainerID)assert.Equal(t, "arcane", mockUpgrade.capturedTarget.ContainerName)@@ -359,9 +373,9 @@ func TestUpdaterService_DockerClientAdapterInternal(t *testing.T) {})t.Run("delegates to configured docker service", func(t *testing.T) {- server := newProjectImagePullServer(t, nil)- wantClient := newTestDockerClient(t, server)- dockerSvc := &DockerClientService{client: wantClient}+ server := newImagePullServerWithObserverInternal(t, nil, nil)+ wantClient := newTestDockerClientInternal(t, server)+ dockerSvc := docker.NewDockerClientService(t.Context(), nil, nil, nil).WithClient(wantClient)svc, svcErr := NewUpdaterService(nil, nil, dockerSvc, nil, nil, nil, nil, nil, nil, nil, nil)require.NoError(t, svcErr)@@ -386,10 +400,10 @@ func TestUpdaterService_PullImageAdapterInternal(t *testing.T) {})t.Run("delegates to Arcane image puller", func(t *testing.T) {- db := setupProjectTestDB(t)- server := newProjectImagePullServer(t, nil)- dockerSvc := &DockerClientService{client: newTestDockerClient(t, server)}- imageSvc := NewImageService(db, dockerSvc, nil, nil, nil, NewEventService(db, nil, nil))+ db := setupProjectTestDBInternal(t)+ server := newImagePullServerWithObserverInternal(t, nil, nil)+ dockerSvc := docker.NewDockerClientService(t.Context(), nil, nil, nil).WithClient(newTestDockerClientInternal(t, server))+ imageSvc := image.NewImageService(db, dockerSvc, nil, nil, nil, event.NewEventService(db, nil, nil))svc, svcErr := NewUpdaterService(db, nil, dockerSvc, nil, nil, nil, nil, imageSvc, nil, nil, nil)require.NoError(t, svcErr)var progress bytes.Buffer@@ -401,26 +415,26 @@ func TestUpdaterService_PullImageAdapterInternal(t *testing.T) {})t.Run("passes database registry credentials to Arcane image puller", func(t *testing.T) {- db := setupProjectTestDB(t)+ db := setupProjectTestDBInternal(t)require.NoError(t, db.AutoMigrate(&models.ContainerRegistry{}, &models.KVEntry{}))crypto.InitEncryption(&crypto.Config{Environment: string(config.AppEnvironmentTest),EncryptionKey: "test-encryption-key-for-testing-32bytes-min",})- createTestPullRegistry(t, db, "https://registry.example.com", "db-user", "db-token")+ createTestPullRegistryInternal(t, db, "https://registry.example.com", "db-user", "db-token")imageRef := "registry.example.com/team/app:latest"var gotAuth string- server := newProjectImagePullServerWithObserverInternal(t, nil, func(fullRef string, authHeader string) {+ server := newImagePullServerWithObserverInternal(t, nil, func(fullRef string, authHeader string) {if fullRef == imageRef {gotAuth = authHeader}})- dockerSvc := &DockerClientService{client: newTestDockerClient(t, server)}- registrySvc := NewContainerRegistryService(db, nil, NewKVService(db))- imageSvc := NewImageService(db, dockerSvc, registrySvc, nil, nil, NewEventService(db, nil, nil))- envSvc := NewEnvironmentService(db, nil, nil, nil, nil, nil)- projectSvc := NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, nil).+ dockerSvc := docker.NewDockerClientService(t.Context(), nil, nil, nil).WithClient(newTestDockerClientInternal(t, server))+ registrySvc := registry.NewContainerRegistryService(db, nil, kv.NewKVService(db))+ imageSvc := image.NewImageService(db, dockerSvc, registrySvc, nil, nil, event.NewEventService(db, nil, nil))+ envSvc := environment.NewEnvironmentService(db, nil, nil, nil, nil, nil)+ projectSvc := project.NewProjectService(db, nil, nil, nil, nil, nil, nil, nil, nil).WithRegistryCredentialsProvider(envSvc.GetEnabledRegistryCredentials)svc, svcErr := NewUpdaterService(db, nil, dockerSvc, projectSvc, nil, nil, nil, imageSvc, nil, nil, nil)require.NoError(t, svcErr)@@ -430,7 +444,7 @@ func TestUpdaterService_PullImageAdapterInternal(t *testing.T) {require.NoError(t, err)require.NotEmpty(t, gotAuth)- authConfig := decodeRegistryAuth(t, gotAuth)+ authConfig := decodeRegistryAuthInternal(t, gotAuth)assert.Equal(t, "db-user", authConfig.Username)assert.Equal(t, "db-token", authConfig.Password)assert.Contains(t, progress.String(), "Pulled")@@ -439,7 +453,7 @@ func TestUpdaterService_PullImageAdapterInternal(t *testing.T) {func TestUpdaterService_PendingImageUpdatesAdapterInternal(t *testing.T) {ctx := context.Background()- db := setupProjectTestDB(t)+ db := setupProjectTestDBInternal(t)latest := "1.2.4"currentDigest := "sha256:1111111111111111111111111111111111111111111111111111111111111111"latestDigest := "sha256:2222222222222222222222222222222222222222222222222222222222222222"@@ -491,7 +505,7 @@ func TestUpdaterService_PendingImageUpdatesAdapterInternal(t *testing.T) {// time was silently lost. PendingImageUpdates must flush it before the engine runs.func TestUpdaterService_PendingImageUpdatesFlushesPendingNotificationsInternal(t *testing.T) {ctx := context.Background()- db := setupProjectTestDB(t)+ db := setupProjectTestDBInternal(t)require.NoError(t, db.AutoMigrate(&models.ImageUpdateRecord{}, &models.NotificationSettings{}))var calls atomic.Int32@@ -510,8 +524,8 @@ func TestUpdaterService_PendingImageUpdatesFlushesPendingNotificationsInternal(t},}).Error)- notif := NewNotificationService(db, nil, nil, nil)- imageUpdates := NewImageUpdateService(db, nil, nil, nil, nil, notif, nil)+ notif := notification.NewNotificationService(db, nil, nil, nil)+ imageUpdates := imageupdate.NewImageUpdateService(db, nil, nil, nil, nil, notif, nil)svc, svcErr := NewUpdaterService(db, nil, nil, nil, imageUpdates, nil, nil, nil, notif, nil, nil)require.NoError(t, svcErr)@@ -538,11 +552,11 @@ func TestUpdaterService_PendingImageUpdatesFlushesPendingNotificationsInternal(t// delivered" semantics).func TestUpdaterService_PendingImageUpdatesNoProvidersLeavesUnnotifiedInternal(t *testing.T) {ctx := context.Background()- db := setupProjectTestDB(t)+ db := setupProjectTestDBInternal(t)require.NoError(t, db.AutoMigrate(&models.ImageUpdateRecord{}, &models.NotificationSettings{}))- notif := NewNotificationService(db, nil, nil, nil)- imageUpdates := NewImageUpdateService(db, nil, nil, nil, nil, notif, nil)+ notif := notification.NewNotificationService(db, nil, nil, nil)+ imageUpdates := imageupdate.NewImageUpdateService(db, nil, nil, nil, nil, notif, nil)svc, svcErr := NewUpdaterService(db, nil, nil, nil, imageUpdates, nil, nil, nil, notif, nil, nil)require.NoError(t, svcErr)@@ -565,7 +579,7 @@ func TestUpdaterService_PendingImageUpdatesNoProvidersLeavesUnnotifiedInternal(tfunc TestUpdaterService_RecordUpdateRunAdapterInternal(t *testing.T) {ctx := context.Background()- db := setupProjectTestDB(t)+ db := setupProjectTestDBInternal(t)require.NoError(t, db.AutoMigrate(&models.AutoUpdateRecord{}))svc, svcErr := NewUpdaterService(db, nil, nil, nil, nil, nil, nil, nil, nil, nil, nil)require.NoError(t, svcErr)@@ -679,7 +693,7 @@ func TestUpdaterService_ApplyPending_ProjectFailureDoesNotBlockOtherProjectsInte}server := newUpdaterApplyPendingDockerServerInternal(t, containers, verificationByService, inspectByID, imageInspectByRef)- dockerProvider := fakeDockerClientProviderInternal{client: newTestDockerClient(t, server)}+ dockerProvider := fakeDockerClientProviderInternal{client: newTestDockerClientInternal(t, server)}puller := &fakeImagePullerInternal{}recorder := &fakeRunRecorderInternal{}projectUpdater := &fakeProjectUpdaterInternal{@@ -705,7 +719,7 @@ func TestUpdaterService_ApplyPending_ProjectFailureDoesNotBlockOtherProjectsInteHasUpdate: true,UpdateType: updater.UpdateTypeTag,CurrentVersion: "1.0.0",- LatestVersion: ptr("1.0.1"),+ LatestVersion: new("1.0.1"),},updater.ImageUpdateRecord{ID: oldImageIDUpdated,@@ -714,7 +728,7 @@ func TestUpdaterService_ApplyPending_ProjectFailureDoesNotBlockOtherProjectsInteHasUpdate: true,UpdateType: updater.UpdateTypeTag,CurrentVersion: "2.0.0",- LatestVersion: ptr("2.0.1"),+ LatestVersion: new("2.0.1"),},),RunRecorder: recorder,@@ -814,7 +828,7 @@ func TestUpdaterService_ApplyPending_RoutesLegacyArcaneServerThroughSelfUpgradeI}server := newUpdaterApplyPendingDockerServerInternal(t, containers, verificationByService, inspectByID, imageInspectByRef)- dockerProvider := fakeDockerClientProviderInternal{client: newTestDockerClient(t, server)}+ dockerProvider := fakeDockerClientProviderInternal{client: newTestDockerClientInternal(t, server)}puller := &fakeImagePullerInternal{}projectUpdater := &fakeProjectUpdaterInternal{projects: map[string]updater.ComposeProject{@@ -835,7 +849,7 @@ func TestUpdaterService_ApplyPending_RoutesLegacyArcaneServerThroughSelfUpgradeIHasUpdate: true,UpdateType: updater.UpdateTypeTag,CurrentVersion: "1.0.0",- LatestVersion: ptr("1.0.1"),+ LatestVersion: new("1.0.1"),}),Settings: fakeSettingsProviderInternal{},ProjectUpdater: projectUpdater,@@ -901,3 +915,115 @@ func TestPullableImageRefInternal(t *testing.T) {})}}++// Test fixtures shared by this package's tests.++// createTestPullRegistryInternal inserts an enabled generic registry with an encrypted token.+func createTestPullRegistryInternal(t *testing.T, db *database.DB, url, username, token string) {+ t.Helper()++ encryptedToken, err := crypto.Encrypt(token)+ require.NoError(t, err)++ reg := &models.ContainerRegistry{+ URL: url,+ Username: username,+ Token: encryptedToken,+ Enabled: true,+ RegistryType: registry.RegistryTypeGeneric,+ }+ require.NoError(t, db.WithContext(context.Background()).Create(reg).Error)+}++// decodeRegistryAuthInternal decodes a base64 X-Registry-Auth header.+func decodeRegistryAuthInternal(t *testing.T, encoded string) dockerregistry.AuthConfig {+ t.Helper()++ cfg, err := dockerauthconfig.Decode(encoded)+ require.NoError(t, err)+ return *cfg+}++// newTestDockerClientInternal builds a Docker client pointed at a fake Docker HTTP server.+func newTestDockerClientInternal(t *testing.T, server *httptest.Server) *client.Client {+ t.Helper()++ httpClient := server.Client()+ cli, err := client.New(+ client.WithHost(server.URL),+ client.WithAPIVersion("1.41"),+ client.WithHTTPClient(httpClient),+ )+ require.NoError(t, err)++ t.Cleanup(func() {+ _ = cli.Close()+ })++ return cli+}++// newImagePullServerWithObserverInternal is NewImagePullServer with a pull callback.+func newImagePullServerWithObserverInternal(t *testing.T, inspectByRef map[string]dockertypesimage.InspectResponse, onPull func(fullRef string, authHeader string)) *httptest.Server {+ t.Helper()++ server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ switch {+ case strings.HasSuffix(r.URL.Path, "/images/create"):+ fullRef := strings.TrimSpace(r.URL.Query().Get("fromImage"))+ tag := strings.TrimSpace(r.URL.Query().Get("tag"))+ if fullRef != "" && tag != "" {+ lastSlash := strings.LastIndex(fullRef, "/")+ lastColon := strings.LastIndex(fullRef, ":")+ if lastColon <= lastSlash {+ fullRef += ":" + tag+ }+ }+ if onPull != nil {+ onPull(fullRef, strings.TrimSpace(r.Header.Get("X-Registry-Auth")))+ }++ w.Header().Set("Content-Type", "application/json")+ _ = json.NewEncoder(w).Encode(map[string]string{"status": "Pulled", "id": fullRef})+ return+ case strings.Contains(r.URL.Path, "/images/") && strings.HasSuffix(r.URL.Path, "/json"):+ path := r.URL.Path+ imagePathIndex := strings.Index(path, "/images/")+ if !assert.NotEqual(t, -1, imagePathIndex) {+ return+ }+ encodedRef := strings.TrimSuffix(path[imagePathIndex+len("/images/"):], "/json")+ imageRef, err := url.PathUnescape(encodedRef)+ if !assert.NoError(t, err) {+ return+ }++ inspect, ok := inspectByRef[imageRef]+ if !ok {+ http.Error(w, "not found", http.StatusNotFound)+ return+ }++ w.Header().Set("Content-Type", "application/json")+ if !assert.NoError(t, json.NewEncoder(w).Encode(inspect)) {+ return+ }+ return+ default:+ http.NotFound(w, r)+ }+ }))++ t.Cleanup(server.Close)++ return server+}++// setupProjectTestDBInternal builds an in-memory DB migrated for project-related tests.+func setupProjectTestDBInternal(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.Project{}, &models.SettingVariable{}, &models.ImageUpdateRecord{}, &models.Event{}))+ return &database.DB{DB: db}+}diff --git a/backend/api/handlers/users.go b/backend/internal/user/handler.gosimilarity index 76%rename from backend/api/handlers/users.gorename to backend/internal/user/handler.goindex c8a86737f..5540c90ac 100644--- a/backend/api/handlers/users.go+++ b/backend/internal/user/handler.go@@ -1,4 +1,4 @@-package handlers+package userimport ("context"@@ -8,20 +8,22 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/validation""github.com/getarcaneapp/arcane/types/v2/base"- "github.com/getarcaneapp/arcane/types/v2/user"+ usertypes "github.com/getarcaneapp/arcane/types/v2/user")// UserHandler handles user management endpoints.type UserHandler struct {- userService *services.UserService- authService *services.AuthService- settingsService *services.SettingsService+ userService *UserService+ invalidateUserTokenCache func(string)+ settingsService *settings.SettingsService}// ============================================================================@@ -37,15 +39,15 @@ type ListUsersInput struct {}type ListUsersOutput struct {- Body base.Paginated[user.User]+ Body base.Paginated[usertypes.User]}type CreateUserInput struct {- Body user.CreateUser+ Body usertypes.CreateUser}type CreateUserOutput struct {- Body base.ApiResponse[user.User]+ Body base.ApiResponse[usertypes.User]}type GetUserInput struct {@@ -53,16 +55,16 @@ type GetUserInput struct {}type GetUserOutput struct {- Body base.ApiResponse[user.User]+ Body base.ApiResponse[usertypes.User]}type UpdateUserInput struct {UserID string `path:"userId" doc:"User ID"`- Body user.UpdateUser+ Body usertypes.UpdateUser}type UpdateUserOutput struct {- Body base.ApiResponse[user.User]+ Body base.ApiResponse[usertypes.User]}type DeleteUserInput struct {@@ -89,8 +91,8 @@ type GetUserAvatarOutput struct {// ============================================================================// RegisterUsers registers all user management endpoints.-func RegisterUsers(api huma.API, userService *services.UserService, authService *services.AuthService, settingsService *services.SettingsService) {- h := &UserHandler{userService: userService, authService: authService, settingsService: settingsService}+func RegisterUsers(api huma.API, userService *UserService, invalidateUserTokenCache func(string), settingsService *settings.SettingsService) {+ h := &UserHandler{userService: userService, invalidateUserTokenCache: invalidateUserTokenCache, settingsService: settingsService}huma.Register(api, huma.Operation{OperationID: "listUsers",@@ -99,8 +101,8 @@ func RegisterUsers(api huma.API, userService *services.UserService, authServiceSummary: "List users",Description: "Get a paginated list of all users",Tags: []string{"Users"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermUsersList),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermUsersList),}, h.ListUsers)huma.Register(api, huma.Operation{@@ -110,8 +112,8 @@ func RegisterUsers(api huma.API, userService *services.UserService, authServiceSummary: "Create a user",Description: "Create a new user account",Tags: []string{"Users"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermUsersCreate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermUsersCreate),}, h.CreateUser)huma.Register(api, huma.Operation{@@ -121,8 +123,8 @@ func RegisterUsers(api huma.API, userService *services.UserService, authServiceSummary: "Get a user",Description: "Get a user by ID",Tags: []string{"Users"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermUsersRead),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermUsersRead),}, h.GetUser)huma.Register(api, huma.Operation{@@ -132,8 +134,8 @@ func RegisterUsers(api huma.API, userService *services.UserService, authServiceSummary: "Update a user",Description: "Update an existing user's information",Tags: []string{"Users"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermUsersUpdate),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermUsersUpdate),}, h.UpdateUser)huma.Register(api, huma.Operation{@@ -143,8 +145,8 @@ func RegisterUsers(api huma.API, userService *services.UserService, authServiceSummary: "Delete a user",Description: "Delete a user by ID",Tags: []string{"Users"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequirePermission(api, authz.PermUsersDelete),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequirePermission(api, authz.PermUsersDelete),}, h.DeleteUser)// Unauthenticated by design: profile pictures are publicly visible@@ -166,7 +168,7 @@ func RegisterUsers(api huma.API, userService *services.UserService, authService// ListUsers returns a paginated list of users.func (h *UserHandler) ListUsers(ctx context.Context, input *ListUsersInput) (*ListUsersOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)users, paginationResp, err := h.userService.ListUsersPaginated(ctx, params)if err != nil {@@ -174,17 +176,17 @@ func (h *UserHandler) ListUsers(ctx context.Context, input *ListUsersInput) (*Li}return &ListUsersOutput{- Body: base.Paginated[user.User]{+ Body: base.Paginated[usertypes.User]{Success: true,Data: users,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}-// CreateUser creates a new user.+// CreateUser creates a new usertypes.func (h *UserHandler) CreateUser(ctx context.Context, input *CreateUserInput) (*CreateUserOutput, error) {- normalizedEmail, err := normalizeOptionalEmailInternal(input.Body.Email)+ normalizedEmail, err := NormalizeOptionalEmail(input.Body.Email)if err != nil {return nil, huma.Error400BadRequest(err.Error())}@@ -205,7 +207,7 @@ func (h *UserHandler) CreateUser(ctx context.Context, input *CreateUserInput) (*DisplayName: input.Body.DisplayName,Email: input.Body.Email,Locale: input.Body.Locale,- TimeFormat: user.TimeFormatAuto,+ TimeFormat: usertypes.TimeFormatAuto,BaseModel: models.BaseModel{CreatedAt: time.Now(),},@@ -225,7 +227,7 @@ func (h *UserHandler) CreateUser(ctx context.Context, input *CreateUserInput) (*}return &CreateUserOutput{- Body: base.ApiResponse[user.User]{+ Body: base.ApiResponse[usertypes.User]{Success: true,Data: out,},@@ -245,21 +247,21 @@ func (h *UserHandler) GetUser(ctx context.Context, input *GetUserInput) (*GetUse}return &GetUserOutput{- Body: base.ApiResponse[user.User]{+ Body: base.ApiResponse[usertypes.User]{Success: true,Data: out,},}, nil}-// UpdateUser updates a user.+// UpdateUser updates a usertypes.func (h *UserHandler) UpdateUser(ctx context.Context, input *UpdateUserInput) (*UpdateUserOutput, error) {userModel, err := h.userService.GetUserByID(ctx, input.UserID)if err != nil {return nil, huma.Error404NotFound("User not found")}- normalizedEmail, err := normalizeOptionalEmailInternal(input.Body.Email)+ normalizedEmail, err := NormalizeOptionalEmail(input.Body.Email)if err != nil {return nil, huma.Error400BadRequest(err.Error())}@@ -301,20 +303,20 @@ func (h *UserHandler) UpdateUser(ctx context.Context, input *UpdateUserInput) (*updatedUser, err := h.userService.UpdateUser(ctx, userModel, callerPerms)if err != nil {- if errors.Is(err, services.ErrInsufficientPrivilege) {- return nil, huma.Error403Forbidden(services.ErrInsufficientPrivilege.Error())+ if errors.Is(err, ErrInsufficientPrivilege) {+ return nil, huma.Error403Forbidden(ErrInsufficientPrivilege.Error())}- if errors.Is(err, services.ErrCannotRemoveLastAdmin) {- return nil, huma.Error409Conflict(services.ErrCannotRemoveLastAdmin.Error())+ if errors.Is(err, ErrCannotRemoveLastAdmin) {+ return nil, huma.Error409Conflict(ErrCannotRemoveLastAdmin.Error())}- if errors.Is(err, services.ErrUserNotFound) {+ if errors.Is(err, common.ErrUserNotFound) {return nil, huma.Error404NotFound("User not found")}return nil, huma.Error500InternalServerError("Failed to update user")}- if h.authService != nil {- h.authService.InvalidateUserTokenCache(updatedUser.ID)+ if h.invalidateUserTokenCache != nil {+ h.invalidateUserTokenCache(updatedUser.ID)}out, err := h.userService.ToUserResponseDto(ctx, *updatedUser)@@ -323,19 +325,19 @@ func (h *UserHandler) UpdateUser(ctx context.Context, input *UpdateUserInput) (*}return &UpdateUserOutput{- Body: base.ApiResponse[user.User]{+ Body: base.ApiResponse[usertypes.User]{Success: true,Data: out,},}, nil}-// DeleteUser deletes a user.+// DeleteUser deletes a usertypes.func (h *UserHandler) DeleteUser(ctx context.Context, input *DeleteUserInput) (*DeleteUserOutput, error) {// Privilege ordering: a non-admin caller may not delete a global admin// target. The service enforces the same check; this pre-check produces a// clean 403 without entering the delete path.- callerPerms, _ := humamw.PermissionsFromContext(ctx)+ callerPerms, _ := middleware.PermissionsFromContext(ctx)caller, _ := models.CurrentUserFromContext(ctx)if callerPerms != nil && !callerPerms.IsGlobalAdmin() && caller != nil && caller.ID != input.UserID {targetPerms, err := h.userService.ResolveUserPermissions(ctx, input.UserID)@@ -343,25 +345,25 @@ func (h *UserHandler) DeleteUser(ctx context.Context, input *DeleteUserInput) (*return nil, huma.Error500InternalServerError("Failed to resolve target permissions")}if targetPerms != nil && targetPerms.IsGlobalAdmin() {- return nil, huma.Error403Forbidden(services.ErrInsufficientPrivilege.Error())+ return nil, huma.Error403Forbidden(ErrInsufficientPrivilege.Error())}}if err := h.userService.DeleteUser(ctx, input.UserID, callerPerms); err != nil {- if errors.Is(err, services.ErrInsufficientPrivilege) {- return nil, huma.Error403Forbidden(services.ErrInsufficientPrivilege.Error())+ if errors.Is(err, ErrInsufficientPrivilege) {+ return nil, huma.Error403Forbidden(ErrInsufficientPrivilege.Error())}- if errors.Is(err, services.ErrCannotRemoveLastAdmin) {- return nil, huma.Error409Conflict(services.ErrCannotRemoveLastAdmin.Error())+ if errors.Is(err, ErrCannotRemoveLastAdmin) {+ return nil, huma.Error409Conflict(ErrCannotRemoveLastAdmin.Error())}- if errors.Is(err, services.ErrUserNotFound) {+ if errors.Is(err, common.ErrUserNotFound) {return nil, huma.Error404NotFound("User not found")}return nil, huma.Error500InternalServerError("Failed to delete user")}- if h.authService != nil {- h.authService.InvalidateUserTokenCache(input.UserID)+ if h.invalidateUserTokenCache != nil {+ h.invalidateUserTokenCache(input.UserID)}return &DeleteUserOutput{@@ -374,11 +376,11 @@ func (h *UserHandler) DeleteUser(ctx context.Context, input *DeleteUserInput) (*}, nil}-// GetUserAvatar returns the custom profile picture for a user.+// GetUserAvatar returns the custom profile picture for a usertypes.func (h *UserHandler) GetUserAvatar(ctx context.Context, input *GetUserAvatarInput) (*GetUserAvatarOutput, error) {data, mimeType, err := h.userService.GetAvatar(ctx, input.UserID)if err != nil {- if errors.Is(err, services.ErrUserNotFound) {+ if errors.Is(err, common.ErrUserNotFound) {return nil, huma.Error404NotFound("User not found")}return nil, huma.Error500InternalServerError("failed to retrieve avatar")@@ -396,7 +398,7 @@ func (h *UserHandler) GetUserAvatar(ctx context.Context, input *GetUserAvatarInp}, nil}-func normalizeOptionalEmailInternal(email *string) (*string, error) {+func NormalizeOptionalEmail(email *string) (*string, error) {if email == nil {return nil, nil}@@ -417,18 +419,18 @@ func normalizeOptionalEmailInternal(email *string) (*string, error) {// modify a global admin target, nor set another user's password. The service// re-enforces the target-admin check.func (h *UserHandler) checkUpdateUserPrivilegesInternal(ctx context.Context, input *UpdateUserInput, targetID string) (*authz.PermissionSet, error) {- callerPerms, _ := humamw.PermissionsFromContext(ctx)+ callerPerms, _ := middleware.PermissionsFromContext(ctx)caller, _ := models.CurrentUserFromContext(ctx)if callerPerms != nil && !callerPerms.IsGlobalAdmin() {if input.Body.Password != nil && *input.Body.Password != "" && caller != nil && caller.ID != targetID {- return nil, huma.Error403Forbidden(services.ErrInsufficientPrivilege.Error())+ return nil, huma.Error403Forbidden(ErrInsufficientPrivilege.Error())}targetPerms, err := h.userService.ResolveUserPermissions(ctx, targetID)if err != nil {return nil, huma.Error500InternalServerError("Failed to resolve target permissions")}if targetPerms != nil && targetPerms.IsGlobalAdmin() {- return nil, huma.Error403Forbidden(services.ErrInsufficientPrivilege.Error())+ return nil, huma.Error403Forbidden(ErrInsufficientPrivilege.Error())}}return callerPerms, nildiff --git a/backend/internal/user/module.go b/backend/internal/user/module.gonew file mode 100644index 000000000..fd7d45389--- /dev/null+++ b/backend/internal/user/module.go@@ -0,0 +1,38 @@+// Package user owns user persistence, password hashing, authorization guards,+// and the user HTTP surface.+package user++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+)++type Dependencies struct {+ Service *UserService+ InvalidateUserTokenCache func(string)+ Settings *settings.SettingsService+}++type Module struct {+ deps Dependencies+}++func New(deps Dependencies) *Module {+ return &Module{deps: deps}+}++func (m *Module) Service() *UserService {+ if m == nil {+ return nil+ }+ return m.deps.Service+}++func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterUsers(api, nil, nil, nil)+ return+ }+ RegisterUsers(api, m.deps.Service, m.deps.InvalidateUserTokenCache, m.deps.Settings)+}diff --git a/backend/internal/services/user_service.go b/backend/internal/user/service.gosimilarity index 94%rename from backend/internal/services/user_service.gorename to backend/internal/user/service.goindex 5f4b9b95a..c1c58101b 100644--- a/backend/internal/services/user_service.go+++ b/backend/internal/user/service.go@@ -1,4 +1,4 @@-package services+package userimport ("context"@@ -15,8 +15,11 @@ import ("gorm.io/gorm""gorm.io/gorm/clause"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/backend/v2/pkg/utils/dbutil"@@ -43,7 +46,7 @@ func DefaultArgon2Params() *Argon2Params {type UserService struct {db *database.DB- roleService *RoleService+ roleService *role.RoleServiceargon2Params *Argon2Params}@@ -63,10 +66,10 @@ func NewUserService(db *database.DB) *UserService {}}-// WithRoleService wires the RoleService dependency. Separated from the-// constructor so the bootstrap can construct UserService first (RoleService+// WithRoleService wires the role.RoleService dependency. Separated from the+// constructor so the bootstrap can construct UserService first (role.RoleService// itself has no UserService dependency).-func (s *UserService) WithRoleService(roleService *RoleService) *UserService {+func (s *UserService) WithRoleService(roleService *role.RoleService) *UserService {s.roleService = roleServicereturn s}@@ -149,23 +152,23 @@ func (s *UserService) CreateUser(ctx context.Context, user *models.User) (*model}func (s *UserService) GetUserByUsername(ctx context.Context, username string) (*models.User, error) {- return dbutil.FirstWhere[models.User](ctx, s.db.DB, ErrUserNotFound, "username = ?", username)+ return dbutil.FirstWhere[models.User](ctx, s.db.DB, common.ErrUserNotFound, "username = ?", username)}func (s *UserService) GetUserByID(ctx context.Context, id string) (*models.User, error) {- return dbutil.FirstWhere[models.User](ctx, s.db.DB, ErrUserNotFound, "id = ?", id)+ return dbutil.FirstWhere[models.User](ctx, s.db.DB, common.ErrUserNotFound, "id = ?", id)}func (s *UserService) GetUserByOidcSubjectId(ctx context.Context, subjectId string) (*models.User, error) {- return dbutil.FirstWhere[models.User](ctx, s.db.DB, ErrUserNotFound, "oidc_subject_id = ?", subjectId)+ return dbutil.FirstWhere[models.User](ctx, s.db.DB, common.ErrUserNotFound, "oidc_subject_id = ?", subjectId)}func (s *UserService) GetUserByEmail(ctx context.Context, email string) (*models.User, error) {- return dbutil.FirstWhere[models.User](ctx, s.db.DB, ErrUserNotFound, "email = ?", email)+ return dbutil.FirstWhere[models.User](ctx, s.db.DB, common.ErrUserNotFound, "email = ?", email)}// ResolveUserPermissions returns the effective PermissionSet for the given-// user ID, or nil when no RoleService is wired (RBAC disabled paths).+// user ID, or nil when no role.RoleService is wired (RBAC disabled paths).func (s *UserService) ResolveUserPermissions(ctx context.Context, userID string) (*authz.PermissionSet, error) {if s.roleService == nil {return nil, nil@@ -215,7 +218,7 @@ func (s *UserService) checkTargetPrivilegeInternal(ctx context.Context, tx *gormactorFound = actorFound || lockedUsers[i].ID == actorID}if !targetFound {- return ErrUserNotFound+ return common.ErrUserNotFound}if !actorFound {return ErrInsufficientPrivilege@@ -225,7 +228,7 @@ func (s *UserService) checkTargetPrivilegeInternal(ctx context.Context, tx *gormreturn nil}if revalidateActor {- currentActorPerms, err := s.roleService.resolveUserPermissionsInternal(ctx, tx, actorID)+ currentActorPerms, err := s.roleService.ResolveUserPermissionsInDB(ctx, tx, actorID)if err != nil {return errors.WrapIf(err, "failed to resolve actor permissions")}@@ -233,7 +236,7 @@ func (s *UserService) checkTargetPrivilegeInternal(ctx context.Context, tx *gormreturn nil}}- targetPerms, err := s.roleService.resolveUserPermissionsInternal(ctx, tx, targetID)+ targetPerms, err := s.roleService.ResolveUserPermissionsInDB(ctx, tx, targetID)if err != nil {return errors.WrapIf(err, "failed to resolve target permissions")}@@ -288,7 +291,7 @@ func (s *UserService) SetPasswordAndRevokeSessionsExcept(ctx context.Context, usreturn nil, errors.New("user is nil")}if strings.TrimSpace(user.ID) == "" {- return nil, ErrUserNotFound+ return nil, common.ErrUserNotFound}hashedPassword, err := s.HashPassword(password)@@ -303,7 +306,7 @@ func (s *UserService) SetPasswordAndRevokeSessionsExcept(ctx context.Context, usWhere("id = ?", user.ID).First(&updatedUser).Error; err != nil {if errors.Is(err, gorm.ErrRecordNotFound) {- return ErrUserNotFound+ return common.ErrUserNotFound}return errors.WrapIf(err, "failed to load user for password reset")}@@ -313,7 +316,7 @@ func (s *UserService) SetPasswordAndRevokeSessionsExcept(ctx context.Context, usif err := tx.Save(&updatedUser).Error; err != nil {return errors.WrapIf(err, "failed to update user password")}- return revokeAllUserSessionsExceptInternal(ctx, tx, updatedUser.ID, exceptSessionID)+ return session.RevokeAllUserSessionsExceptInDB(ctx, tx, updatedUser.ID, exceptSessionID)})if err != nil {return nil, err@@ -339,7 +342,7 @@ func (s *UserService) AttachOidcSubjectTransactional(ctx context.Context, userIDWhere("id = ?", userID).First(&u).Error; err != nil {if errors.Is(err, gorm.ErrRecordNotFound) {- return ErrUserNotFound+ return common.ErrUserNotFound}return errors.WrapIf(err, "failed to load user for OIDC merge")}@@ -492,7 +495,7 @@ func (s *UserService) grantDefaultAdminRoleInternal(ctx context.Context, adminUsfunc (s *UserService) DeleteUser(ctx context.Context, id string, actorPerms *authz.PermissionSet) error {// Last-admin guard: if this user's effective permissions make them the only// global admin, refuse the delete. Checked outside the transaction because- // RoleService uses its own session and the guard spans rows.+ // role.RoleService uses its own session and the guard spans rows.if s.roleService != nil {ps, err := s.roleService.ResolvePermissions(ctx, &models.User{BaseModel: models.BaseModel{ID: id}})if err != nil {diff --git a/backend/internal/services/user_service_test.go b/backend/internal/user/service_test.gosimilarity index 92%rename from backend/internal/services/user_service_test.gorename to backend/internal/user/service_test.goindex 9706f484b..597e67b99 100644--- a/backend/internal/services/user_service_test.go+++ b/backend/internal/user/service_test.go@@ -1,23 +1,46 @@-package services+package userimport ("context""testing""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/role""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/types/v2/user"+ "github.com/libtnb/sqlite""github.com/stretchr/testify/require"+ "gorm.io/gorm")+func setupAuthServiceTestDBInternal(t *testing.T) *database.DB {+ t.Helper()+ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(+ &models.SettingVariable{},+ &models.User{},+ &models.UserSession{},+ &models.Environment{},+ &models.Role{},+ &models.UserRoleAssignment{},+ &models.ApiKey{},+ &models.ApiKeyPermission{},+ &models.OidcRoleMapping{},+ ))+ return &database.DB{DB: db}+}+// setupUserAndRoleServices wires both services together the way bootstrap// does, so the legacy-admin-guard tests exercise the real RBAC path.-func setupUserAndRoleServices(t *testing.T) (*UserService, *RoleService) {+func setupUserAndRoleServices(t *testing.T) (*UserService, *role.RoleService) {t.Helper()- db := setupAuthServiceTestDB(t)- role := NewRoleService(db)+ db := setupAuthServiceTestDBInternal(t)+ role := role.NewRoleService(db)require.NoError(t, role.EnsureBuiltInRoles(context.Background()))userRecord := NewUserService(db).WithRoleService(role)return userRecord, role@@ -34,7 +57,7 @@ func createTestUser(t *testing.T, svc *UserService, id, username string) *models}// grantGlobalAdmin assigns the built-in Admin role globally to the user.-func grantGlobalAdmin(t *testing.T, role *RoleService, userID string) {+func grantGlobalAdmin(t *testing.T, role *role.RoleService, userID string) {t.Helper()require.NoError(t, role.SetUserAssignments(context.Background(), userID, []models.UserRoleAssignment{{RoleID: authz.BuiltInRoleAdmin, EnvironmentID: nil},@@ -92,7 +115,7 @@ func TestDeleteUserAllowsDeletingNonAdmin(t *testing.T) {require.NoError(t, err)_, err = userSvc.GetUserByID(ctx, nonAdmin.ID)- require.ErrorIs(t, err, ErrUserNotFound)+ require.ErrorIs(t, err, common.ErrUserNotFound)}func TestDeleteUserAllowsDeletingAdminWhenAnotherAdminExists(t *testing.T) {@@ -108,7 +131,7 @@ func TestDeleteUserAllowsDeletingAdminWhenAnotherAdminExists(t *testing.T) {require.NoError(t, err)_, err = userSvc.GetUserByID(ctx, adminToDelete.ID)- require.ErrorIs(t, err, ErrUserNotFound)+ require.ErrorIs(t, err, common.ErrUserNotFound)}func TestListUsersPaginatedSetsCanDeleteFromGlobalAdminCount(t *testing.T) {diff --git a/backend/api/handlers/variables.go b/backend/internal/variable/handler.gosimilarity index 85%rename from backend/api/handlers/variables.gorename to backend/internal/variable/handler.goindex 586a514c1..73611bc5a 100644--- a/backend/api/handlers/variables.go+++ b/backend/internal/variable/handler.go@@ -1,4 +1,4 @@-package handlers+package variableimport ("context"@@ -6,10 +6,11 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/common"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/env")@@ -17,8 +18,8 @@ import (// VariableHandler handles manager-level global variables and the separate// agent-only materialization channel used to push effective values.type VariableHandler struct {- variableService *services.VariableService- environmentService *services.EnvironmentService+ variableService *VariableService+ environmentService *environment.EnvironmentService}// ============================================================================@@ -89,74 +90,74 @@ type UpdateGlobalVariablesOutput struct {// Route Registration// ============================================================================-func RegisterVariables(api huma.API, variableService *services.VariableService, environmentService *services.EnvironmentService) {+func RegisterVariables(api huma.API, variableService *VariableService, environmentService *environment.EnvironmentService) {h := &VariableHandler{variableService: variableService, environmentService: environmentService}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "listVariables",Method: "GET",Path: "/variables",Summary: "List global variables",Description: "List all global variables with their environment scope (secret values are redacted)",Tags: []string{"Variables"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVariablesRead, h.ListVariables)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "createVariable",Method: "POST",Path: "/variables",Summary: "Create a global variable",Description: "Create a global variable scoped to all or specific environments",Tags: []string{"Variables"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVariablesCreate, h.CreateVariable)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "updateVariable",Method: "PUT",Path: "/variables/{id}",Summary: "Update a global variable",Description: "Update a global variable's key, value, secret flag, or environment scope",Tags: []string{"Variables"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVariablesUpdate, h.UpdateVariable)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "deleteVariable",Method: "DELETE",Path: "/variables/{id}",Summary: "Delete a global variable",Description: "Delete a global variable and re-sync affected environments",Tags: []string{"Variables"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVariablesDelete, h.DeleteVariable)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "syncVariables",Method: "POST",Path: "/variables/sync",Summary: "Sync global variables",Description: "Push the effective global variable set to every environment now",Tags: []string{"Variables"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVariablesSync, h.SyncVariables)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "getVariableSyncStatus",Method: "GET",Path: "/variables/sync-status",Summary: "Get variable sync status",Description: "Get the last global-variable sync result per environment",Tags: []string{"Variables"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVariablesRead, h.GetSyncStatus)}// RegisterMaterializedVariables registers the plaintext synchronization// channel on agents. It must never be registered on a manager API because its// response intentionally contains decrypted values for .env.global.-func RegisterMaterializedVariables(api huma.API, variableService *services.VariableService, environmentService *services.EnvironmentService) {+func RegisterMaterializedVariables(api huma.API, variableService *VariableService, environmentService *environment.EnvironmentService) {h := &VariableHandler{variableService: variableService, environmentService: environmentService}huma.Register(api, huma.Operation{@@ -166,8 +167,8 @@ func RegisterMaterializedVariables(api huma.API, variableService *services.VariaSummary: "Get materialized variables",Description: "Get the materialized variable set for an environment. Managed via /variables on the manager.",Tags: []string{"Variables"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireSudo(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireSudo(api),}, h.GetMaterializedVariables)huma.Register(api, huma.Operation{@@ -177,8 +178,8 @@ func RegisterMaterializedVariables(api huma.API, variableService *services.VariaSummary: "Update materialized variables",Description: "Replace the materialized variable set for an environment. Managed via /variables on the manager.",Tags: []string{"Variables"},- Security: defaultOperationSecurityInternal(),- Middlewares: humamw.RequireSudo(api),+ Security: handlerutil.DefaultOperationSecurity(),+ Middlewares: middleware.RequireSudo(api),}, h.UpdateMaterializedVariables)}@@ -271,7 +272,7 @@ func (h *VariableHandler) GetSyncStatus(_ context.Context, _ *GetGlobalVariableS// content (local file for environment "0", proxied to the agent otherwise).func (h *VariableHandler) GetMaterializedVariables(ctx context.Context, input *GetGlobalVariablesInput) (*GetGlobalVariablesOutput, error) {if input.EnvironmentID != "0" {- response, err := proxyRemoteJSONInternal[base.ApiResponse[[]env.Variable]](ctx, h.environmentService, input.EnvironmentID, http.MethodGet, "/api/environments/0/templates/variables", nil)+ response, err := handlerutil.ProxyRemoteJSON[base.ApiResponse[[]env.Variable]](ctx, h.environmentService.ProxyJSONRequest, input.EnvironmentID, http.MethodGet, "/api/environments/0/templates/variables", nil)if err != nil {return nil, err}@@ -295,7 +296,7 @@ func (h *VariableHandler) GetMaterializedVariables(ctx context.Context, input *G// .env.global content (local file for environment "0", proxied otherwise).func (h *VariableHandler) UpdateMaterializedVariables(ctx context.Context, input *UpdateGlobalVariablesInput) (*UpdateGlobalVariablesOutput, error) {if input.EnvironmentID != "0" {- response, err := proxyRemoteJSONInternal[base.ApiResponse[base.MessageResponse]](ctx, h.environmentService, input.EnvironmentID, http.MethodPut, "/api/environments/0/templates/variables", input.Body)+ response, err := handlerutil.ProxyRemoteJSON[base.ApiResponse[base.MessageResponse]](ctx, h.environmentService.ProxyJSONRequest, input.EnvironmentID, http.MethodPut, "/api/environments/0/templates/variables", input.Body)if err != nil {return nil, err}diff --git a/backend/internal/variable/module.go b/backend/internal/variable/module.gonew file mode 100644index 000000000..80763dd8f--- /dev/null+++ b/backend/internal/variable/module.go@@ -0,0 +1,41 @@+// Package variable owns global variables, environment materialization, and routes.+package variable++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+)++type Module struct {+ service *VariableService+ environment *environment.EnvironmentService+}++func New(service *VariableService, environmentService *environment.EnvironmentService) *Module {+ return &Module{service: service, environment: environmentService}+}++func (m *Module) Service() *VariableService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API, cfg *config.Config) {+ if cfg != nil && cfg.AgentMode {+ if m == nil {+ RegisterMaterializedVariables(api, nil, nil)+ return+ }+ RegisterMaterializedVariables(api, m.service, m.environment)+ return+ }+ if m == nil {+ RegisterVariables(api, nil, nil)+ return+ }+ RegisterVariables(api, m.service, m.environment)+}diff --git a/backend/internal/services/variable_service_test.go b/backend/internal/variable/service_test.gosimilarity index 83%rename from backend/internal/services/variable_service_test.gorename to backend/internal/variable/service_test.goindex c2dd1986a..7f50ae67f 100644--- a/backend/internal/services/variable_service_test.go+++ b/backend/internal/variable/service_test.go@@ -1,4 +1,4 @@-package services+package variableimport ("context"@@ -12,20 +12,45 @@ import ("strings""sync""testing"+ "time""emperror.dev/errors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/remenv"envtypes "github.com/getarcaneapp/arcane/types/v2/env""github.com/libtnb/sqlite""github.com/stretchr/testify/assert""github.com/stretchr/testify/require""go.getarcane.app/sys/crypto"+ "go.uber.org/fx/fxtest""gorm.io/gorm")+func newSettingsServiceForVariableTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {+ t.Helper()+ lifecycle := fxtest.NewLifecycle(t)+ runtime, err := actors.NewRuntime(t.Context(), lifecycle)+ require.NoError(t, err)+ writes, err := actors.NewExecutor(t.Context(), runtime, "variable-settings-test", t.Name(), 3)+ require.NoError(t, err)+ effects, err := actors.NewExecutor(t.Context(), runtime, "variable-settings-effects-test", t.Name(), 3)+ require.NoError(t, err)+ t.Cleanup(func() {+ stopCtx, cancel := context.WithTimeout(context.Background(), time.Second)+ defer cancel()+ require.NoError(t, writes.Stop(stopCtx))+ require.NoError(t, effects.Stop(stopCtx))+ require.NoError(t, lifecycle.Stop(stopCtx))+ })+ return settings.NewSettingsService(ctx, db, writes, effects)+}+func setupVariableServiceTest(t *testing.T) (*VariableService, *database.DB, string) {t.Helper()@@ -51,11 +76,11 @@ func setupVariableServiceTest(t *testing.T) (*VariableService, *database.DB, strprojectsDir := t.TempDir()dbWrap := &database.DB{DB: db}- settingsSvc, err := newSettingsServiceForTestInternal(t, context.Background(), dbWrap)+ settingsSvc, err := newSettingsServiceForVariableTestInternal(t, context.Background(), dbWrap)require.NoError(t, err)require.NoError(t, settingsSvc.UpdateSetting(context.Background(), "projectsDirectory", projectsDir))- service := NewVariableService(dbWrap, nil, settingsSvc, NewKVService(dbWrap))+ service := NewVariableService(dbWrap, nil, settingsSvc, kv.NewKVService(dbWrap))return service, dbWrap, projectsDir}@@ -263,7 +288,9 @@ func TestSyncEnvironment_DirectMaterializesSecretsThroughAgentRoute(t *testing.TEnabled: true,AccessToken: &token,}).Error)- service.environmentService = NewEnvironmentService(db, nil, nil, nil, service.settingsService, nil)+ environmentService := environment.NewEnvironmentService(db, nil, nil, nil, service.settingsService, nil)+ service.proxyEnvironmentJSON = environmentService.ProxyJSONRequest+ service.listEnabledEnvironmentIDs = environmentService.ListEnabledEnvironmentIDscreateMaterializedSecretVariableInternal(t, service, "API_TOKEN", "super-secret")require.NoError(t, service.SyncEnvironment(context.Background(), "env-direct"))@@ -286,31 +313,26 @@ func TestSyncEnvironment_EdgeMaterializesSecretsThroughAgentRoute(t *testing.T)AccessToken: &token,}).Error)- environmentService := NewEnvironmentService(db, nil, nil, nil, service.settingsService, nil)var requestMu sync.Mutexrequests := make([]capturedVariableSyncRequestInternal, 0, 2)- environmentService.remoteClient = remenv.NewClient(nil, remenv.TunnelTransportFuncs{- EnsureAvailableFunc: func(_ context.Context, environmentID string) error {- require.Equal(t, "env-edge", environmentID)- return nil- },- DoFunc: func(_ context.Context, environmentID, method, path string, headers map[string]string, body []byte) (*remenv.Response, error) {- require.Equal(t, "env-edge", environmentID)- requestMu.Lock()- requests = append(requests, capturedVariableSyncRequestInternal{- Method: method,- Path: path,- Headers: headers,- Body: append([]byte(nil), body...),- })- requestMu.Unlock()- if method == http.MethodGet {- return &remenv.Response{StatusCode: http.StatusOK, Body: []byte(`{"success":true,"data":[]}`)}, nil- }- return &remenv.Response{StatusCode: http.StatusOK, Body: []byte(`{"success":true,"data":{"message":"updated"}}`)}, nil- },- })- service.environmentService = environmentService+ service.proxyEnvironmentJSON = func(_ context.Context, environmentID, method, path string, body []byte, response any) error {+ require.Equal(t, "env-edge", environmentID)+ requestMu.Lock()+ requests = append(requests, capturedVariableSyncRequestInternal{+ Method: method,+ Path: path,+ Headers: map[string]string{+ remenv.HeaderAPIKey: token,+ remenv.HeaderAgentToken: token,+ },+ Body: append([]byte(nil), body...),+ })+ requestMu.Unlock()+ if method == http.MethodGet {+ return json.Unmarshal([]byte(`{"success":true,"data":[]}`), response)+ }+ return json.Unmarshal([]byte(`{"success":true,"data":{"message":"updated"}}`), response)+ }createMaterializedSecretVariableInternal(t, service, "API_TOKEN", "super-secret")require.NoError(t, service.SyncEnvironment(context.Background(), "env-edge"))diff --git a/backend/internal/services/variable_service.go b/backend/internal/variable/variable.gosimilarity index 92%rename from backend/internal/services/variable_service.gorename to backend/internal/variable/variable.goindex c8403aaa1..10d9ef609 100644--- a/backend/internal/services/variable_service.go+++ b/backend/internal/variable/variable.go@@ -1,4 +1,4 @@-package services+package variableimport ("bufio"@@ -19,8 +19,12 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/getarcaneapp/arcane/types/v2/env""go.getarcane.app/sys/crypto""gorm.io/gorm"@@ -39,26 +43,31 @@ const (// VariableService manages manager-level global variables (DB source of truth)// and materializes each environment's effective set into that environment's-// .env.global file: locally via TemplateService, remotely via the existing+// .env.global file: locally via template.TemplateService, remotely via the existing// agent variables endpoint.type VariableService struct {- db *database.DB- environmentService *EnvironmentService- settingsService *SettingsService- kvService *KVService+ db *database.DB+ proxyEnvironmentJSON func(context.Context, string, string, string, []byte, any) error+ listEnabledEnvironmentIDs func(context.Context) ([]string, error)+ settingsService *settings.SettingsService+ kvService *kv.KVServicestatusMu sync.RWMutexsyncStatus map[string]env.EnvironmentSyncStatus}-func NewVariableService(db *database.DB, environmentService *EnvironmentService, settingsService *SettingsService, kvService *KVService) *VariableService {- return &VariableService{- db: db,- environmentService: environmentService,- settingsService: settingsService,- kvService: kvService,- syncStatus: make(map[string]env.EnvironmentSyncStatus),+func NewVariableService(db *database.DB, environmentService *environment.EnvironmentService, settingsService *settings.SettingsService, kvService *kv.KVService) *VariableService {+ service := &VariableService{+ db: db,+ settingsService: settingsService,+ kvService: kvService,+ syncStatus: make(map[string]env.EnvironmentSyncStatus),}+ if environmentService != nil {+ service.proxyEnvironmentJSON = environmentService.ProxyJSONRequest+ service.listEnabledEnvironmentIDs = environmentService.ListEnabledEnvironmentIDs+ }+ return service}//@@ -307,7 +316,7 @@ func (s *VariableService) SyncEnvironment(ctx context.Context, envID string) err}func (s *VariableService) syncEnvironmentInternal(ctx context.Context, envID string) error {- if envID != localEnvironmentID {+ if envID != environment.LocalEnvironmentID {if err := s.importRemoteLegacyVarsOnceInternal(ctx, envID); err != nil {return errors.WrapIff(err, "failed to import existing variables from environment %s", envID)}@@ -318,7 +327,7 @@ func (s *VariableService) syncEnvironmentInternal(ctx context.Context, envID strreturn err}- if envID == localEnvironmentID {+ if envID == environment.LocalEnvironmentID {return s.WriteLocalEnvFile(ctx, vars)}@@ -330,19 +339,19 @@ func (s *VariableService) syncEnvironmentInternal(ctx context.Context, envID strvar out struct {Success bool `json:"success"`}- return s.environmentService.ProxyJSONRequest(ctx, envID, http.MethodPut, agentVariablesPath, body, &out)+ return s.proxyEnvironmentJSON(ctx, envID, http.MethodPut, agentVariablesPath, body, &out)}// syncTargetsInternal returns the local environment plus every enabled// environment, deduplicated.func (s *VariableService) syncTargetsInternal(ctx context.Context) []string {- envIDs, err := s.environmentService.listEnabledEnvironmentIDsInternal(ctx)+ envIDs, err := s.listEnabledEnvironmentIDs(ctx)if err != nil {slog.ErrorContext(ctx, "Failed to list environments for global variable sync", "error", err)envIDs = nil}- if !slices.Contains(envIDs, localEnvironmentID) {- envIDs = append([]string{localEnvironmentID}, envIDs...)+ if !slices.Contains(envIDs, environment.LocalEnvironmentID) {+ envIDs = append([]string{environment.LocalEnvironmentID}, envIDs...)}return envIDs}@@ -360,11 +369,11 @@ func (s *VariableService) SyncAll(ctx context.Context) []env.EnvironmentSyncStat// so mutation requests never block on slow or unreachable agents. Remotes are// reported as "pending" until the background push records their result.func (s *VariableService) SyncAllBackground(ctx context.Context) []env.EnvironmentSyncStatus {- if err := s.SyncEnvironment(ctx, localEnvironmentID); err != nil {+ if err := s.SyncEnvironment(ctx, environment.LocalEnvironmentID); err != nil {slog.WarnContext(ctx, "Failed to sync global variables locally", "error", err)}- remoteIDs := slices.DeleteFunc(s.syncTargetsInternal(ctx), func(id string) bool { return id == localEnvironmentID })+ remoteIDs := slices.DeleteFunc(s.syncTargetsInternal(ctx), func(id string) bool { return id == environment.LocalEnvironmentID })for _, envID := range remoteIDs {s.recordSyncPendingInternal(envID)}@@ -434,7 +443,7 @@ func (s *VariableService) recordSyncStatusInternal(envID string, err error) {}//-// Local .env.global file primitives (moved from TemplateService). These back+// Local .env.global file primitives (moved from template.TemplateService). These back// the local materialization and the agent-side variables endpoint.//@@ -514,7 +523,7 @@ func (s *VariableService) WriteLocalEnvFile(ctx context.Context, vars []env.Vari}projectsDirectory := filepath.Dir(envPath)- if err := os.MkdirAll(projectsDirectory, common.DirPerm); err != nil {+ if err := os.MkdirAll(projectsDirectory, utils.DirPerm); err != nil {return errors.WrapIf(err, "failed to create projects directory")}@@ -548,7 +557,7 @@ func (s *VariableService) WriteLocalEnvFile(ctx context.Context, vars []env.Vari_, _ = fmt.Fprintf(&builder, "%s=%s\n", key, value)}- if err := projects.WriteFileWithPerm(envPath, builder.String(), common.FilePerm); err != nil {+ if err := projects.WriteFileWithPerm(envPath, builder.String(), utils.FilePerm); err != nil {return errors.WrapIf(err, "failed to write global variables file")}@@ -567,7 +576,7 @@ func (s *VariableService) WriteLocalEnvFile(ctx context.Context, vars []env.Vari// as all-environment readable variables before the first materialization// overwrites the file. Runs once, guarded by a KV flag.func (s *VariableService) ImportLegacyLocalEnvFile(ctx context.Context) error {- flag := globalVariablesImportedKVPrefix + localEnvironmentID+ flag := globalVariablesImportedKVPrefix + environment.LocalEnvironmentIDimported, err := s.kvService.GetBool(ctx, flag, false)if err != nil {return err@@ -605,7 +614,7 @@ func (s *VariableService) importRemoteLegacyVarsOnceInternal(ctx context.ContextSuccess bool `json:"success"`Data []env.Variable `json:"data"`}- if err := s.environmentService.ProxyJSONRequest(ctx, envID, http.MethodGet, agentVariablesPath, nil, &out); err != nil {+ if err := s.proxyEnvironmentJSON(ctx, envID, http.MethodGet, agentVariablesPath, nil, &out); err != nil {return err}diff --git a/backend/api/handlers/version.go b/backend/internal/version/handler.gosimilarity index 92%rename from backend/api/handlers/version.gorename to backend/internal/version/handler.goindex 83e950c05..557183b12 100644--- a/backend/api/handlers/version.go+++ b/backend/internal/version/handler.go@@ -1,17 +1,16 @@-package handlers+package versionimport ("context""strings""github.com/danielgtaylor/huma/v2"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/types/v2/version")// VersionHandler handles version information endpoints.type VersionHandler struct {- versionService *services.VersionService+ versionService *VersionService}// ============================================================================@@ -37,7 +36,7 @@ type GetAppVersionOutput struct {// ============================================================================// RegisterVersion registers version endpoints.-func RegisterVersion(api huma.API, versionService *services.VersionService) {+func RegisterVersion(api huma.API, versionService *VersionService) {h := &VersionHandler{versionService: versionService}huma.Register(api, huma.Operation{diff --git a/backend/internal/services/version_service.go b/backend/internal/version/service.gosimilarity index 87%rename from backend/internal/services/version_service.gorename to backend/internal/version/service.goindex b08d0fe32..32979f89f 100644--- a/backend/internal/services/version_service.go+++ b/backend/internal/version/service.go@@ -1,4 +1,4 @@-package services+package versionimport ("context"@@ -19,11 +19,13 @@ import ("github.com/getarcaneapp/arcane/backend/v2/buildables""github.com/getarcaneapp/arcane/backend/v2/internal/config"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/registry""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""github.com/getarcaneapp/arcane/types/v2/version""github.com/samber/hot""go.getarcane.app/sys/cgroup"- "go.getarcane.app/updater/labels")const (@@ -44,12 +46,12 @@ type VersionService struct {disabled boolversion stringrevision string- containerRegistryService *ContainerRegistryService- dockerService *DockerClientService- imageUpdateService *ImageUpdateService+ containerRegistryService *registry.ContainerRegistryService+ dockerService *docker.DockerClientService+ imageUpdateService *imageupdate.ImageUpdateService}-func NewVersionService(httpClient *http.Client, disabled bool, version string, revision string, containerRegistryService *ContainerRegistryService, dockerService *DockerClientService, imageUpdateService *ImageUpdateService) *VersionService {+func NewVersionService(httpClient *http.Client, disabled bool, version string, revision string, containerRegistryService *registry.ContainerRegistryService, dockerService *docker.DockerClientService, imageUpdateService *imageupdate.ImageUpdateService) *VersionService {if httpClient == nil {httpClient = http.DefaultClient}@@ -302,7 +304,7 @@ func (s *VersionService) GetAppVersionInfo(ctx context.Context) *version.Info {func (s *VersionService) storedOrDigestBasedUpdateInternal(ctx context.Context, currentImageID, currentTag, currentDigest, currentImageRef string) (bool, string) {if s.imageUpdateService != nil && strings.TrimSpace(currentImageID) != "" {- record, found, err := s.imageUpdateService.getStoredUpdateByImageIDInternal(ctx, currentImageID)+ record, found, err := s.imageUpdateService.StoredUpdateByImageID(ctx, currentImageID)if err != nil {slog.WarnContext(ctx, "Failed to read stored Arcane image update state", "imageID", currentImageID, "error", err)} else if found {@@ -396,43 +398,7 @@ func (s *VersionService) detectContainerID(ctx context.Context, dockerClient *clslog.Debug("detectContainerID: getCurrentContainerID failed, trying label fallback", "error", err)// Fallback: locate the Arcane container by label (works even when cgroup/hostname detection fails)- return s.findArcaneContainerByLabel(ctx, dockerClient)-}--// findArcaneContainerByLabel searches for the Arcane container using labels-func (s *VersionService) findArcaneContainerByLabel(ctx context.Context, dockerClient *client.Client) string {- f := make(client.Filters)- f = f.Add("label", labels.LabelArcane+"=true")- list, err := dockerClient.ContainerList(ctx, client.ContainerListOptions{All: true, Filters: f})- if err != nil {- slog.Debug("findArcaneContainerByLabel: failed to list containers", "error", err)- return ""- }- slog.Debug("findArcaneContainerByLabel: found containers with arcane label", "count", len(list.Items))-- var fallbackID string- for _, c := range list.Items {- slog.Debug("findArcaneContainerByLabel: checking container", "id", c.ID[:12], "state", c.State, "labels", c.Labels)- // Skip the upgrader helper container- if v, ok := c.Labels["com.getarcaneapp.arcane.upgrader"]; ok && strings.EqualFold(strings.TrimSpace(v), "true") {- slog.Debug("findArcaneContainerByLabel: skipping upgrader container", "id", c.ID[:12])- continue- }- // Prefer running containers- if c.State == containertypes.StateRunning {- slog.Debug("findArcaneContainerByLabel: found running container", "id", c.ID[:12])- return c.ID- }- if fallbackID == "" {- fallbackID = c.ID- }- }- if fallbackID != "" {- slog.Debug("findArcaneContainerByLabel: using fallback container", "id", fallbackID[:12])- } else {- slog.Debug("findArcaneContainerByLabel: no container found")- }- return fallbackID+ return libarcane.FindArcaneContainerIDByLabel(ctx, dockerClient)}// extractImageDetails extracts digest and imageRef from a container's imagediff --git a/backend/internal/services/version_service_test.go b/backend/internal/version/service_test.gosimilarity index 72%rename from backend/internal/services/version_service_test.gorename to backend/internal/version/service_test.goindex 703eb2a32..ed2d5c15e 100644--- a/backend/internal/services/version_service_test.go+++ b/backend/internal/version/service_test.go@@ -1,8 +1,9 @@-package services+package versionimport ("context""encoding/json"+ "fmt""io""net/http""net/http/httptest"@@ -11,13 +12,19 @@ import ("testing""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate""github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/libtnb/sqlite""github.com/moby/moby/api/types/container"dockertypesimage "github.com/moby/moby/api/types/image"+ "github.com/moby/moby/client""github.com/opencontainers/go-digest""github.com/stretchr/testify/assert""github.com/stretchr/testify/require""go.getarcane.app/updater/labels"+ "gorm.io/gorm")func TestVersionService_GetAppVersionInfoDoesNotUseStoredDigestUpdateForSemverBuildInternal(t *testing.T) {@@ -102,8 +109,8 @@ func TestVersionService_GetAppVersionInfoDoesNotUseStoredDigestUpdateForSemverBuRequest: req,}, nil})}- dockerService := &DockerClientService{client: newTestDockerClient(t, server)}- imageUpdateService := NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)+ dockerService := docker.NewDockerClientService(t.Context(), nil, nil, nil).WithClient(newTestDockerClientInternal(t, server))+ imageUpdateService := imageupdate.NewImageUpdateService(db, nil, nil, dockerService, nil, nil, nil)svc := NewVersionService(httpClient, false, "1.2.3", "revision", nil, dockerService, imageUpdateService)info := svc.GetAppVersionInfo(ctx)@@ -135,3 +142,40 @@ func TestVersionService_GetAppVersionInfoPreservesDevVersionInternal(t *testing.assert.Equal(t, "dev", info.DisplayVersion)assert.False(t, info.IsSemverVersion)}++// Test fixtures shared by this package's tests.++// setupImageUpdateTestDB creates an in-memory SQLite database for testing+func setupImageUpdateTestDB(t *testing.T) *database.DB {+ t.Helper()+ dsn := fmt.Sprintf("file:image-update-test-%d?mode=memory&cache=shared", time.Now().UnixNano())+ db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.ImageUpdateRecord{}, &models.Event{}, &models.Project{}))+ return &database.DB{DB: db}+}++type roundTripFunc func(*http.Request) (*http.Response, error)++func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {+ return f(req)+}++// newTestDockerClientInternal builds a Docker client pointed at a fake Docker HTTP server.+func newTestDockerClientInternal(t *testing.T, server *httptest.Server) *client.Client {+ t.Helper()++ httpClient := server.Client()+ cli, err := client.New(+ client.WithHost(server.URL),+ client.WithAPIVersion("1.41"),+ client.WithHTTPClient(httpClient),+ )+ require.NoError(t, err)++ t.Cleanup(func() {+ _ = cli.Close()+ })++ return cli+}diff --git a/backend/internal/volume/backup.go b/backend/internal/volume/backup.gonew file mode 100644index 000000000..00985bb71--- /dev/null+++ b/backend/internal/volume/backup.go@@ -0,0 +1,934 @@+package volume++import (+ "archive/tar"+ "compress/gzip"+ "context"+ "fmt"+ "io"+ "log/slog"+ "os"+ "path"+ "strconv"+ "strings"+ "time"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ docker "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/volumehelper"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ "github.com/google/uuid"+ "github.com/moby/moby/api/types/container"+ "github.com/moby/moby/api/types/mount"+ "github.com/moby/moby/client"+ "github.com/samber/mo"+)++type backupStorageMode string++const (+ // backupStorageModeArcaneMount means backup helpers mirror an existing Arcane+ // container mount at /backups. This intentionally covers any mount the Arcane+ // container already has at /backups, not exclusively bind mounts.+ backupStorageModeArcaneMount backupStorageMode = "arcane_mount"+ // backupStorageModeNamedVolumeFallback means no suitable Arcane container+ // mount was found, so Arcane's dedicated named backup volume is used.+ backupStorageModeNamedVolumeFallback backupStorageMode = "named_volume_fallback"+)++const backupMountMissingWarning = "No volume is mounted at /backups in the Arcane container. Backups will only live inside Docker unless you mount a host path."++type backupStorageMountInternal struct {+ mode backupStorageMode+ mount mount.Mount+ requiresEnsure bool+}++func resolveBackupStorageMountFromMountsInternal(mounts []container.MountPoint, target string, readOnly bool) mo.Option[backupStorageMountInternal] {+ mirroredMount := docker.MountForDestination(mounts, "/backups", target)+ if mirroredMount == nil {+ return mo.None[backupStorageMountInternal]()+ }+ // MountForDestination only returns non-nil for bind and named volume mounts.++ if !readOnly && mirroredMount.ReadOnly {+ slog.Warn("volume service: requested writable backup mount but source is read-only; writes may fail")+ }+ mirroredMount.ReadOnly = readOnly++ return mo.Some(backupStorageMountInternal{+ mode: backupStorageModeArcaneMount,+ mount: *mirroredMount,+ })+}++func (s *VolumeService) resolveBackupStorageMountInternal(ctx context.Context, dockerClient *client.Client, target string, readOnly bool) backupStorageMountInternal {+ if dockerClient != nil {+ inspect, err := libarcane.InspectCurrentArcaneContainer(ctx, dockerClient)+ if err != nil {+ slog.WarnContext(ctx, "volume service: failed to inspect arcane container for backup mount resolution, falling back to named volume", "error", err.Error())+ } else if resolved, ok := resolveBackupStorageMountFromMountsInternal(inspect.Mounts, target, readOnly).Get(); ok {+ return resolved+ }+ }++ return backupStorageMountInternal{+ mode: backupStorageModeNamedVolumeFallback,+ mount: mount.Mount{+ Type: mount.TypeVolume,+ Source: s.backupVolumeName,+ Target: target,+ ReadOnly: readOnly,+ },+ requiresEnsure: true,+ }+}++func (s *VolumeService) resolveUsableBackupStorageMountInternal(ctx context.Context, dockerClient *client.Client, target string, readOnly bool) (backupStorageMountInternal, error) {+ backupStorage := s.resolveBackupStorageMountInternal(ctx, dockerClient, target, readOnly)+ if backupStorage.requiresEnsure {+ if err := s.ensureBackupVolumeInternal(ctx); err != nil {+ return backupStorageMountInternal{}, err+ }+ }+ return backupStorage, nil+}++func backupMountWarningForStorageInternal(storage backupStorageMountInternal) string {+ if storage.mode == backupStorageModeArcaneMount {+ return ""+ }+ return backupMountMissingWarning+}++func backupMountWarningFromArcaneMountsInternal(mounts []container.MountPoint) string {+ backupStorage, ok := resolveBackupStorageMountFromMountsInternal(mounts, "/backups", true).Get()+ if ok {+ return backupMountWarningForStorageInternal(backupStorage)+ }++ // Backward compatibility: historically either /backups or /restores mount+ // suppressed the warning. Preserve that user-visible behavior.+ for _, m := range mounts {+ if m.Destination == "/restores" {+ return ""+ }+ }++ return backupMountMissingWarning+}++func (s *VolumeService) BackupMountWarning(ctx context.Context) string {+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return ""+ }++ // Cannot determine Arcane mount status (e.g. running outside Docker); suppress warning.+ inspect, err := libarcane.InspectCurrentArcaneContainer(ctx, dockerClient)+ if err != nil {+ return ""+ }++ return backupMountWarningFromArcaneMountsInternal(inspect.Mounts)+}++func (s *VolumeService) createBackupTempContainerWithMountInternal(ctx context.Context, dockerClient *client.Client, helperImage string, backupMount mount.Mount) (string, func(), error) {+ var err error+ if dockerClient == nil {+ dockerClient, err = s.dockerService.GetClient(ctx)+ if err != nil {+ return "", nil, err+ }+ }++ if strings.TrimSpace(helperImage) == "" {+ helperImage, err = getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)+ if err != nil {+ return "", nil, err+ }+ }++ config := &container.Config{+ Image: helperImage,+ Cmd: []string{"sleep", "infinity"},+ NetworkDisabled: true,+ Labels: volumehelper.Labels(),+ }++ hostConfig := volumehelper.HostConfig(helperImage, nil, []mount.Mount{backupMount})++ resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{+ Config: config,+ HostConfig: hostConfig,+ })+ if err != nil {+ return "", nil, errors.WrapIf(err, "failed to create backup temp container")+ }++ if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {+ _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())+ return "", nil, errors.WrapIf(err, "failed to start backup temp container")+ }++ cleanup := func() {+ _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())+ }++ return resp.ID, cleanup, nil+}++func (s *VolumeService) createBackupTempContainerInternal(ctx context.Context, dockerClient *client.Client, target string, readOnly bool) (string, func(), error) {+ slog.DebugContext(ctx, "volume service: create backup temp container", "target", target, "read_only", readOnly)+ var err error+ if dockerClient == nil {+ dockerClient, err = s.dockerService.GetClient(ctx)+ if err != nil {+ return "", nil, err+ }+ }++ backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, target, readOnly)+ if err != nil {+ return "", nil, err+ }++ return s.createBackupTempContainerWithMountInternal(ctx, dockerClient, "", backupStorage.mount)+}++func (s *VolumeService) ensureBackupVolumeInternal(ctx context.Context) error {+ slog.DebugContext(ctx, "volume service: ensure backup volume", "backup_volume", s.backupVolumeName)+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return err+ }++ _, err = dockerClient.VolumeInspect(ctx, s.backupVolumeName, client.VolumeInspectOptions{})+ if err != nil {+ _, err = dockerClient.VolumeCreate(ctx, client.VolumeCreateOptions{+ Name: s.backupVolumeName,+ })+ if err != nil {+ return errors.WrapIf(err, "failed to create backup volume")+ }+ }+ return nil+}++func (s *VolumeService) CreateBackup(ctx context.Context, volumeName string, user models.User) (*models.VolumeBackup, error) {+ slog.DebugContext(ctx, "volume service: create backup", "volume", volumeName, "user", user.ID)+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, err+ }++ backupID := fmt.Sprintf("%s-%d-%s", volumeName, time.Now().UnixNano(), uuid.NewString()[:8])+ filename, err := s.backupArchiveFilenameInternal(backupID)+ if err != nil {+ return nil, err+ }++ helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)+ if err != nil {+ return nil, err+ }++ backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, "/backups", false)+ if err != nil {+ return nil, err+ }++ config := &container.Config{+ Image: helperImage,+ Cmd: []string{"sh", "-c", fmt.Sprintf("tar -czf /backups/%s -C /volume .", filename)},+ Labels: volumehelper.Labels(),+ }++ hostConfig := volumehelper.HostConfig(helperImage, []string{+ volumeName + ":/volume:ro",+ }, []mount.Mount{backupStorage.mount})++ resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{+ Config: config,+ HostConfig: hostConfig,+ })+ if err != nil {+ return nil, errors.WrapIf(err, "failed to create backup container")+ }++ if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {+ _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())+ return nil, errors.WrapIf(err, "failed to start backup container")+ }++ waitResult := dockerClient.ContainerWait(ctx, resp.ID, client.ContainerWaitOptions{Condition: container.WaitConditionNotRunning})+ select {+ case err := <-waitResult.Error:+ if err != nil {+ return nil, err+ }+ case status := <-waitResult.Result:+ if status.StatusCode != 0 {+ return nil, errors.Errorf("backup container exited with status %d", status.StatusCode)+ }+ }++ sizeCheckMount := backupStorage.mount+ sizeCheckMount.Target = "/volume"+ sizeCheckMount.ReadOnly = true++ tempContainerID, cleanup, err := s.createBackupTempContainerWithMountInternal(ctx, dockerClient, "", sizeCheckMount)+ if err != nil {+ return nil, err+ }+ defer cleanup()++ sizeStr, _, err := s.execInContainerInternal(ctx, tempContainerID, []string{"stat", "-c", "%s", path.Join("/volume", filename)})+ if err != nil {+ return nil, err+ }+ size, err := strconv.ParseInt(strings.TrimSpace(sizeStr), 10, 64)+ if err != nil {+ return nil, err+ }++ backup := &models.VolumeBackup{+ VolumeName: volumeName,+ Size: size,+ CreatedAt: time.Now(),+ }+ backup.ID = backupID++ if err := s.db.WithContext(ctx).Create(backup).Error; err != nil {+ return nil, err+ }++ metadata := models.JSON{+ "action": "backup_create",+ "backup_id": backup.ID,+ "filename": filename,+ "size": size,+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupCreate, volumeName, volumeName, user.ID, user.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume backup create event", "volume", volumeName, "error", logErr.Error())+ }++ return backup, nil+}++func (s *VolumeService) ListBackupsPaginated(ctx context.Context, volumeName string, params pagination.QueryParams) ([]models.VolumeBackup, pagination.Response, error) {+ slog.DebugContext(ctx, "volume service: list backups paginated", "volume", volumeName, "search", params.Search, "sort", params.Sort, "order", params.Order, "start", params.Start, "limit", params.Limit)+ var backups []models.VolumeBackup+ query := s.db.WithContext(ctx).Model(&models.VolumeBackup{}).Where("volume_name = ?", volumeName)++ if params.Search != "" {+ query = query.Where("id LIKE ?", "%"+params.Search+"%")+ }++ var totalItems int64+ if err := query.Count(&totalItems).Error; err != nil {+ return nil, pagination.Response{}, err+ }++ sortCol := "created_at"+ sortOrder := "DESC"+ if params.Sort != "" {+ switch params.Sort {+ case "createdAt", "created_at":+ sortCol = "created_at"+ case "id":+ sortCol = "id"+ case "size":+ sortCol = "size"+ default:+ sortCol = "created_at"+ }++ if params.Order == pagination.SortDesc {+ sortOrder = "DESC"+ } else {+ sortOrder = "ASC"+ }+ }+ query = query.Order(fmt.Sprintf("%s %s", sortCol, sortOrder))++ if params.Limit > 0 {+ query = query.Offset(params.Start).Limit(params.Limit)+ }++ if err := query.Find(&backups).Error; err != nil {+ return nil, pagination.Response{}, err+ }++ paginationResp := pagination.BuildResponse(totalItems, totalItems, params)+ return backups, paginationResp, nil+}++func (s *VolumeService) ListBackups(ctx context.Context, volumeName string) ([]models.VolumeBackup, error) {+ slog.DebugContext(ctx, "volume service: list backups", "volume", volumeName)+ var backups []models.VolumeBackup+ err := s.db.WithContext(ctx).Where("volume_name = ?", volumeName).Order("created_at DESC").Find(&backups).Error+ return backups, err+}++func (s *VolumeService) DeleteBackup(ctx context.Context, backupID string, user *models.User) error {+ slog.DebugContext(ctx, "volume service: delete backup", "backup_id", backupID)+ var backup models.VolumeBackup+ if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err != nil {+ return err+ }++ // Delete from DB first - if this fails, no changes are made.+ // If file deletion fails afterward, we just have an orphan file (easier to clean up)+ // rather than an orphan DB record pointing to a non-existent file.+ volumeName := backup.VolumeName // Save before deletion+ if err := s.db.WithContext(ctx).Delete(&backup).Error; err != nil {+ return err+ }++ // Now delete the actual file - best effort since DB record is already gone+ containerID, cleanup, err := s.createBackupTempContainerInternal(ctx, nil, "/volume", false)+ if err != nil {+ slog.WarnContext(ctx, "failed to create container for backup file cleanup", "backup_id", backupID, "error", err.Error())+ } else {+ defer cleanup()+ filename, filenameErr := s.backupArchiveFilenameInternal(backupID)+ if filenameErr != nil {+ slog.WarnContext(ctx, "failed to sanitize backup id for file cleanup", "backup_id", backupID, "error", filenameErr.Error())+ } else if _, _, err = s.execInContainerInternal(ctx, containerID, []string{"rm", "-f", path.Join("/volume", filename)}); err != nil {+ slog.WarnContext(ctx, "failed to delete backup file (orphan file may remain)", "backup_id", backupID, "error", err.Error())+ }+ }++ actingUser := user+ if actingUser == nil {+ actingUser = &models.SystemUser+ }+ metadata := models.JSON{+ "action": "backup_delete",+ "backup_id": backupID,+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupDelete, volumeName, volumeName, actingUser.ID, actingUser.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume backup delete event", "volume", volumeName, "error", logErr.Error())+ }++ return nil+}++func (s *VolumeService) RestoreBackup(ctx context.Context, volumeName, backupID string, user models.User) error {+ slog.DebugContext(ctx, "volume service: restore backup", "volume", volumeName, "backup_id", backupID, "user", user.ID)+ var backup models.VolumeBackup+ if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err != nil {+ return err+ }++ // Validate backup belongs to volume+ if backup.VolumeName != volumeName {+ return errors.Errorf("backup does not belong to volume %s", volumeName)+ }++ // Check if volume is in use by running containers+ inUse, containerIDs, err := s.GetVolumeUsage(ctx, volumeName)+ if err != nil {+ slog.WarnContext(ctx, "could not check volume usage", "volume", volumeName, "error", err.Error())+ } else if inUse {+ return errors.Errorf("volume is in use by %d container(s): restoring while containers are running may cause data corruption. Stop the containers first or use selective file restore", len(containerIDs))+ }++ preBackup, err := s.CreateBackup(ctx, volumeName, user)+ if err != nil {+ return errors.WrapIf(err, "failed to create pre-restore backup")+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return err+ }++ filename, err := s.backupArchiveFilenameInternal(backupID)+ if err != nil {+ return err+ }++ helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)+ if err != nil {+ return err+ }++ backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, "/backups", true)+ if err != nil {+ return err+ }++ config := &container.Config{+ Image: helperImage,+ Cmd: []string{+ "sh",+ "-c",+ fmt.Sprintf("set -e; tmp=$(mktemp -d /volume/.restore_tmp.XXXXXX); tar -tzf /backups/%s >/dev/null; tar -xzf /backups/%s -C \"$tmp\"; find /volume -mindepth 1 -maxdepth 1 -exec rm -rf -- {} +; find \"$tmp\" -mindepth 1 -maxdepth 1 -exec mv -- {} /volume/ \\;; rmdir \"$tmp\"", filename, filename),+ },+ Labels: volumehelper.Labels(),+ }++ hostConfig := volumehelper.HostConfig(helperImage, []string{+ volumeName + ":/volume",+ }, []mount.Mount{backupStorage.mount})++ resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{+ Config: config,+ HostConfig: hostConfig,+ })+ if err != nil {+ return errors.WrapIf(err, "failed to create restore container")+ }++ if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {+ _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())+ return errors.WrapIf(err, "failed to start restore container")+ }++ waitResult := dockerClient.ContainerWait(ctx, resp.ID, client.ContainerWaitOptions{Condition: container.WaitConditionNotRunning})+ var waitBody container.WaitResponse+ select {+ case err := <-waitResult.Error:+ if err != nil {+ return err+ }+ case waitBody = <-waitResult.Result:+ }++ if waitBody.StatusCode != 0 {+ return errors.Errorf("restore container exited with code %d (volume may be partially wiped)", waitBody.StatusCode)+ }++ metadata := models.JSON{+ "action": "backup_restore",+ "backup_id": backupID,+ "pre_restore_backupId": preBackup.ID,+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupRestore, volumeName, volumeName, user.ID, user.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume backup restore event", "volume", volumeName, "error", logErr.Error())+ }++ return nil+}++func (s *VolumeService) sanitizeBackupPathInternal(input string) (string, error) {+ trimmed := strings.TrimSpace(input)+ if trimmed == "" {+ return "", errors.New("invalid path: empty")+ }+ cleaned := path.Clean(trimmed)+ if cleaned == "." || cleaned == "/" {+ return "", errors.Errorf("invalid path: %s", input)+ }+ if path.IsAbs(cleaned) {+ cleaned = strings.TrimPrefix(cleaned, "/")+ }+ if cleaned == "" || cleaned == "." || cleaned == "/" || strings.HasPrefix(cleaned, "..") || strings.Contains(cleaned, "/../") {+ return "", errors.Errorf("invalid path: %s", input)+ }+ return cleaned, nil+}++func (s *VolumeService) sanitizeBackupIDInternal(backupID string) (string, error) {+ cleaned, err := s.sanitizeBackupPathInternal(backupID)+ if err != nil {+ return "", errors.WrapIf(err, "invalid backup id")+ }+ if strings.Contains(cleaned, "/") {+ return "", errors.New("invalid backup id: path separators not allowed")+ }+ return cleaned, nil+}++func (s *VolumeService) backupArchiveFilenameInternal(backupID string) (string, error) {+ sanitizedBackupID, err := s.sanitizeBackupIDInternal(backupID)+ if err != nil {+ return "", err+ }++ return sanitizedBackupID + ".tar.gz", nil+}++func (s *VolumeService) BackupHasPath(ctx context.Context, backupID string, filePath string) (bool, error) {+ slog.DebugContext(ctx, "volume service: backup has path", "backup_id", backupID, "path", filePath)+ cleaned, err := s.sanitizeBackupPathInternal(filePath)+ if err != nil {+ return false, err+ }+ filename, err := s.backupArchiveFilenameInternal(backupID)+ if err != nil {+ return false, err+ }++ var backup models.VolumeBackup+ if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err != nil {+ return false, err+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return false, err+ }++ helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)+ if err != nil {+ return false, err+ }++ backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, "/volume", true)+ if err != nil {+ return false, err+ }++ containerID, cleanup, err := s.createBackupTempContainerWithMountInternal(ctx, dockerClient, helperImage, backupStorage.mount)+ if err != nil {+ return false, err+ }+ defer cleanup()++ archivePath := path.Join("/volume", filename)+ cmd := []string{"tar", "-tzf", archivePath}+ stdout, stderr, err := s.execInContainerInternal(ctx, containerID, cmd)+ if err != nil {+ return false, err+ }+ if strings.TrimSpace(stderr) != "" {+ return false, errors.Errorf("failed to list backup contents: %s", strings.TrimSpace(stderr))+ }++ for line := range strings.SplitSeq(stdout, "\n") {+ entry := strings.TrimSpace(line)+ if entry == "" {+ continue+ }+ entry = strings.TrimPrefix(entry, "./")+ if entry == cleaned || strings.TrimSuffix(entry, "/") == cleaned {+ return true, nil+ }+ }++ return false, nil+}++func (s *VolumeService) ListBackupFiles(ctx context.Context, backupID string) ([]string, error) {+ slog.DebugContext(ctx, "volume service: list backup files", "backup_id", backupID)+ filename, err := s.backupArchiveFilenameInternal(backupID)+ if err != nil {+ return nil, err+ }++ var backup models.VolumeBackup+ if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err != nil {+ return nil, err+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, err+ }++ helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)+ if err != nil {+ return nil, err+ }++ backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, "/volume", true)+ if err != nil {+ return nil, err+ }++ containerID, cleanup, err := s.createBackupTempContainerWithMountInternal(ctx, dockerClient, helperImage, backupStorage.mount)+ if err != nil {+ return nil, err+ }+ defer cleanup()++ archivePath := path.Join("/volume", filename)+ cmd := []string{"tar", "-tzf", archivePath}+ stdout, _, err := s.execInContainerInternal(ctx, containerID, cmd)+ if err != nil {+ return nil, err+ }++ lines := strings.Split(strings.TrimSpace(stdout), "\n")+ files := make([]string, 0, len(lines))+ seen := make(map[string]struct{})+ for _, line := range lines {+ clean := strings.TrimSpace(line)+ if clean == "" {+ continue+ }+ clean = strings.TrimPrefix(clean, "./")+ if strings.HasSuffix(clean, "/") {+ continue+ }+ if _, ok := seen[clean]; ok {+ continue+ }+ seen[clean] = struct{}{}+ files = append(files, clean)+ }++ return files, nil+}++func (s *VolumeService) RestoreBackupFiles(ctx context.Context, volumeName, backupID string, paths []string, user models.User) error {+ slog.DebugContext(ctx, "volume service: restore backup files", "volume", volumeName, "backup_id", backupID, "paths_count", len(paths), "user", user.ID)+ if len(paths) == 0 {+ return errors.New("no paths provided")+ }+ filename, err := s.backupArchiveFilenameInternal(backupID)+ if err != nil {+ return err+ }++ var backup models.VolumeBackup+ if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err != nil {+ return err+ }+ if backup.VolumeName != volumeName {+ return errors.New("backup does not belong to volume")+ }++ // Create pre-restore backup for safety (consistent with RestoreBackup behavior)+ preBackup, err := s.CreateBackup(ctx, volumeName, user)+ if err != nil {+ return errors.WrapIf(err, "failed to create pre-restore backup")+ }+ slog.DebugContext(ctx, "created pre-restore backup", "volume", volumeName, "pre_backup_id", preBackup.ID)++ cleanedPaths := make([]string, 0, len(paths))+ for _, p := range paths {+ cleaned, err := s.sanitizeBackupPathInternal(p)+ if err != nil {+ return err+ }+ cleanedPaths = append(cleanedPaths, cleaned)+ }+ if len(cleanedPaths) == 0 {+ return errors.New("no valid paths provided")+ }++ tarPaths := make([]string, 0, len(cleanedPaths))+ for _, p := range cleanedPaths {+ tarPaths = append(tarPaths, "./"+p)+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return err+ }++ helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)+ if err != nil {+ return err+ }++ backupStorage, err := s.resolveUsableBackupStorageMountInternal(ctx, dockerClient, "/backups", true)+ if err != nil {+ return err+ }++ config := &container.Config{+ Image: helperImage,+ Cmd: []string{"sleep", "infinity"},+ NetworkDisabled: true,+ Labels: volumehelper.Labels(),+ }++ hostConfig := volumehelper.HostConfig(helperImage, []string{+ volumeName + ":/volume",+ }, []mount.Mount{backupStorage.mount})++ resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{+ Config: config,+ HostConfig: hostConfig,+ })+ if err != nil {+ return errors.WrapIf(err, "failed to create restore container")+ }++ if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {+ _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())+ return errors.WrapIf(err, "failed to start restore container")+ }++ cleanup := func() {+ _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())+ }+ defer cleanup()++ cmd := append([]string{"tar", "-xzf", path.Join("/backups", filename), "-C", "/volume", "--"}, tarPaths...)+ _, stderr, err := s.execInContainerInternal(ctx, resp.ID, cmd)+ if err != nil {+ return errors.WrapIf(err, "failed to restore files")+ }+ if strings.TrimSpace(stderr) != "" {+ slog.DebugContext(ctx, "volume service: restore files stderr", "backup_id", backupID, "stderr", strings.TrimSpace(stderr))+ }++ metadata := models.JSON{+ "action": "backup_restore_files",+ "backup_id": backupID,+ "pre_restore_backupId": preBackup.ID,+ "paths_count": len(cleanedPaths),+ }+ if len(cleanedPaths) > 0 {+ limit := min(len(cleanedPaths), 5)+ metadata["paths_sample"] = cleanedPaths[:limit]+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupRestoreFiles, volumeName, volumeName, user.ID, user.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume backup restore files event", "volume", volumeName, "error", logErr.Error())+ }++ return nil+}++func (s *VolumeService) DownloadBackup(ctx context.Context, backupID string, user *models.User) (io.ReadCloser, int64, error) {+ slog.DebugContext(ctx, "volume service: download backup", "backup_id", backupID)+ filename, err := s.backupArchiveFilenameInternal(backupID)+ if err != nil {+ return nil, 0, err+ }+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, 0, err+ }++ containerID, cleanup, err := s.createBackupTempContainerInternal(ctx, dockerClient, "/volume", true)+ if err != nil {+ return nil, 0, err+ }++ reader, size, err := volumehelper.DownloadFileFromContainer(ctx, dockerClient, containerID, path.Join("/volume", filename), cleanup)+ if err != nil {+ return nil, 0, err+ }++ actingUser := user+ if actingUser == nil {+ actingUser = &models.SystemUser+ }+ volumeName := ""+ var backup models.VolumeBackup+ if err := s.db.WithContext(ctx).Where("id = ?", backupID).First(&backup).Error; err == nil {+ volumeName = backup.VolumeName+ }+ if volumeName != "" {+ metadata := models.JSON{+ "action": "backup_download",+ "backup_id": backupID,+ "size": size,+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupDownload, volumeName, volumeName, actingUser.ID, actingUser.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume backup download event", "volume", volumeName, "error", logErr.Error())+ }+ }++ return reader, size, nil+}++func (s *VolumeService) UploadAndRestore(ctx context.Context, volumeName string, archive io.Reader, filename string, user models.User) error {+ slog.DebugContext(ctx, "volume service: upload and restore", "volume", volumeName, "filename", filename, "user", user.ID)++ tmpFile, err := os.CreateTemp("", "arcane-restore-*.tar.gz")+ if err != nil {+ return errors.WrapIf(err, "failed to buffer upload")+ }+ defer func() {+ _ = tmpFile.Close()+ _ = os.Remove(tmpFile.Name())+ }()+ if _, err := io.Copy(tmpFile, archive); err != nil {+ return errors.WrapIf(err, "failed to buffer upload")+ }+ if _, err := tmpFile.Seek(0, io.SeekStart); err != nil {+ return errors.WrapIf(err, "failed to read buffered upload")+ }+ gzr, err := gzip.NewReader(tmpFile)+ if err != nil {+ return errors.WrapIf(err, "invalid archive")+ }+ if _, err := tar.NewReader(gzr).Next(); err != nil {+ _ = gzr.Close()+ return errors.WrapIf(err, "invalid archive")+ }+ _ = gzr.Close()++ preBackup, err := s.CreateBackup(ctx, volumeName, user)+ if err != nil {+ return errors.WrapIf(err, "failed to create pre-restore backup")+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return err+ }++ containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, false)+ if err != nil {+ return err+ }+ defer cleanup()++ tmpDir := fmt.Sprintf("/volume/.restore_tmp_%d", time.Now().UnixNano())+ _, stderr, err := s.execInContainerInternal(ctx, containerID, []string{"mkdir", "-p", tmpDir})+ if err != nil {+ return errors.WrapIf(err, "failed to create temp restore dir")+ }+ if strings.TrimSpace(stderr) != "" {+ slog.DebugContext(ctx, "volume service: restore temp dir stderr", "volume", volumeName, "stderr", strings.TrimSpace(stderr))+ }++ if _, err := tmpFile.Seek(0, io.SeekStart); err != nil {+ return errors.WrapIf(err, "failed to read buffered upload")+ }+ _, err = dockerClient.CopyToContainer(ctx, containerID, client.CopyToContainerOptions{+ DestinationPath: tmpDir,+ Content: tmpFile,+ })+ if err != nil {+ return errors.WrapIf(err, "failed to restore from uploaded archive")+ }++ _, stderr, err = s.execInContainerInternal(ctx, containerID, []string{"sh", "-c", fmt.Sprintf("test -n \"$(find %s -mindepth 1 -maxdepth 1 -print -quit)\"", tmpDir)})+ if err != nil {+ return errors.WrapIf(err, "uploaded archive appears empty or invalid")+ }+ if strings.TrimSpace(stderr) != "" {+ slog.DebugContext(ctx, "volume service: restore validate stderr", "volume", volumeName, "stderr", strings.TrimSpace(stderr))+ }++ _, stderr, err = s.execInContainerInternal(ctx, containerID, []string{"sh", "-c", "rm -rf /volume/* /volume/.[!.]* /volume/..?* 2>/dev/null || true"})+ if err != nil {+ return errors.WrapIf(err, "failed to clear volume before restore")+ }+ if strings.TrimSpace(stderr) != "" {+ slog.DebugContext(ctx, "volume service: restore clear stderr", "volume", volumeName, "stderr", strings.TrimSpace(stderr))+ }++ moveCmd := fmt.Sprintf("find %s -mindepth 1 -maxdepth 1 -exec mv -- {} /volume/ \\; && rmdir %s", tmpDir, tmpDir)+ _, stderr, err = s.execInContainerInternal(ctx, containerID, []string{"sh", "-c", moveCmd})+ if err != nil {+ return errors.WrapIf(err, "failed to move restored files into place")+ }+ if strings.TrimSpace(stderr) != "" {+ slog.DebugContext(ctx, "volume service: restore move stderr", "volume", volumeName, "stderr", strings.TrimSpace(stderr))+ }++ metadata := models.JSON{+ "action": "backup_upload_restore",+ "filename": filename,+ "pre_restore_backupId": preBackup.ID,+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeBackupRestore, volumeName, volumeName, user.ID, user.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume backup upload restore event", "volume", volumeName, "error", logErr.Error())+ }++ return nil+}diff --git a/backend/internal/volume/files.go b/backend/internal/volume/files.gonew file mode 100644index 000000000..d77511b02--- /dev/null+++ b/backend/internal/volume/files.go@@ -0,0 +1,330 @@+package volume++import (+ "archive/tar"+ "bytes"+ "context"+ "io"+ "log/slog"+ "net/http"+ "path"+ "strconv"+ "strings"+ "time"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/volumehelper"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ volumetypes "github.com/getarcaneapp/arcane/types/v2/volume"+ "github.com/moby/moby/client"+)++// isBrowsableVolumeInternal returns an error if the volume uses driver options+// that prevent it from being mounted inside a helper container, such as+// type=none or o=bind (host bind-mounts that require a device path on the host).+func (s *VolumeService) isBrowsableVolumeInternal(ctx context.Context, volumeName string) error {+ vol, err := s.GetVolumeByName(ctx, volumeName)+ if err != nil {+ return errors.WrapIf(err, "failed to inspect volume")+ }+ if vol.Options["type"] == "none" || strings.Contains(vol.Options["o"], "bind") {+ return errors.Errorf("volume %q uses a custom mount configuration and cannot be browsed", volumeName)+ }+ return nil+}++func (s *VolumeService) ListDirectory(ctx context.Context, volumeName, dirPath string) ([]volumetypes.FileEntry, error) {+ slog.DebugContext(ctx, "volume service: list directory", "volume", volumeName, "path", dirPath)++ if err := s.isBrowsableVolumeInternal(ctx, volumeName); err != nil {+ return nil, err+ }++ sanitizedPath, err := utils.SanitizeBrowsePath(dirPath)+ if err != nil {+ return nil, errors.WrapIf(err, "invalid path")+ }++ containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, true)+ if err != nil {+ return nil, err+ }+ defer cleanup()++ targetPath := path.Join("/volume", sanitizedPath)+ cmd := []string{+ "sh",+ "-c",+ `find "$1" -mindepth 1 -maxdepth 1 | while IFS= read -r f; do out=$(stat -c "%s %Y %f %A" -- "$f" 2>/dev/null) || continue; printf "%s\0%s\0" "$f" "$out"; done`,+ "sh",+ targetPath,+ }+ stdout, _, err := s.execInContainerInternal(ctx, containerID, cmd)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to list directory")+ }++ lines := strings.Split(stdout, "\x00")+ entries := make([]volumetypes.FileEntry, 0)+ for i := 0; i+1 < len(lines); i += 2 {+ fullPath := lines[i]+ meta := strings.Fields(strings.TrimSpace(lines[i+1]))+ if fullPath == "" || len(meta) < 4 {+ continue+ }+ name := path.Base(fullPath)+ size, _ := strconv.ParseInt(meta[0], 10, 64)+ modTimeSec, _ := strconv.ParseInt(meta[1], 10, 64)+ mode := meta[3]++ isDir := strings.HasPrefix(mode, "d")+ isSymlink := strings.HasPrefix(mode, "l")++ relPath := strings.TrimPrefix(fullPath, "/volume")+ if relPath == "" {+ relPath = "/"+ }++ entry := volumetypes.FileEntry{+ Name: name,+ Path: relPath,+ IsDirectory: isDir,+ Size: size,+ ModTime: time.Unix(modTimeSec, 0),+ Mode: mode,+ IsSymlink: isSymlink,+ }++ if isSymlink {+ // Use readlink without -f to get the raw symlink target (not resolved)+ // This prevents exposing paths outside the volume+ target, _, _ := s.execInContainerInternal(ctx, containerID, []string{"readlink", fullPath})+ target = strings.TrimSpace(target)+ if target != "" {+ // If target is relative, it's safe to show+ // If target is absolute and within /volume, strip the /volume prefix+ // If target points outside /volume, indicate it's external+ switch {+ case strings.HasPrefix(target, "/volume/"):+ entry.LinkTarget = strings.TrimPrefix(target, "/volume")+ case strings.HasPrefix(target, "/volume"):+ entry.LinkTarget = "/"+ case !strings.HasPrefix(target, "/"):+ // Relative path - safe to show as-is+ entry.LinkTarget = target+ default:+ // Absolute path outside /volume - indicate it's external+ entry.LinkTarget = "(external)"+ }+ }+ }++ entries = append(entries, entry)+ }++ return entries, nil+}++func (s *VolumeService) GetFileContent(ctx context.Context, volumeName, filePath string, maxBytes int64) ([]byte, string, error) {+ slog.DebugContext(ctx, "volume service: get file content", "volume", volumeName, "path", filePath, "max_bytes", maxBytes)++ if err := s.isBrowsableVolumeInternal(ctx, volumeName); err != nil {+ return nil, "", err+ }++ sanitizedPath, err := utils.SanitizeBrowsePath(filePath)+ if err != nil {+ return nil, "", errors.WrapIf(err, "invalid path")+ }++ containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, true)+ if err != nil {+ return nil, "", err+ }+ defer cleanup()++ targetPath := path.Join("/volume", sanitizedPath)+ cmd := []string{"head", "-c", strconv.FormatInt(maxBytes, 10), targetPath}+ stdout, _, err := s.execInContainerInternal(ctx, containerID, cmd)+ if err != nil {+ return nil, "", errors.WrapIf(err, "failed to read file")+ }++ content := []byte(stdout)+ mimeType := http.DetectContentType(content)++ return content, mimeType, nil+}++func (s *VolumeService) DownloadFile(ctx context.Context, volumeName, filePath string) (io.ReadCloser, int64, error) {+ slog.DebugContext(ctx, "volume service: download file", "volume", volumeName, "path", filePath)++ if err := s.isBrowsableVolumeInternal(ctx, volumeName); err != nil {+ return nil, 0, err+ }++ sanitizedPath, err := utils.SanitizeBrowsePath(filePath)+ if err != nil {+ return nil, 0, errors.WrapIf(err, "invalid path")+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, 0, err+ }++ containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, true)+ if err != nil {+ return nil, 0, err+ }++ targetPath := path.Join("/volume", sanitizedPath)+ return volumehelper.DownloadFileFromContainer(ctx, dockerClient, containerID, targetPath, cleanup)+}++func (s *VolumeService) DeleteFile(ctx context.Context, volumeName, filePath string, user *models.User) error {+ slog.DebugContext(ctx, "volume service: delete file", "volume", volumeName, "path", filePath)++ sanitizedPath, err := utils.SanitizeBrowsePath(filePath)+ if err != nil {+ return errors.WrapIf(err, "invalid path")+ }+ // Prevent deleting root+ if sanitizedPath == "/" {+ return errors.New("cannot delete root directory")+ }++ containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, false)+ if err != nil {+ return err+ }+ defer cleanup()++ targetPath := path.Join("/volume", sanitizedPath)+ _, stderr, err := s.execInContainerInternal(ctx, containerID, []string{"rm", "-rf", targetPath})+ if err != nil {+ return err+ }+ if stderr != "" {+ return errors.Errorf("delete failed: %s", stderr)+ }++ actingUser := user+ if actingUser == nil {+ actingUser = &models.SystemUser+ }+ metadata := models.JSON{+ "action": "file_delete",+ "path": filePath,+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeFileDelete, volumeName, volumeName, actingUser.ID, actingUser.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume file delete event", "volume", volumeName, "error", logErr.Error())+ }+ return nil+}++func (s *VolumeService) CreateDirectory(ctx context.Context, volumeName, dirPath string, user *models.User) error {+ slog.DebugContext(ctx, "volume service: create directory", "volume", volumeName, "path", dirPath)++ sanitizedPath, err := utils.SanitizeBrowsePath(dirPath)+ if err != nil {+ return errors.WrapIf(err, "invalid path")+ }++ containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, false)+ if err != nil {+ return err+ }+ defer cleanup()++ targetPath := path.Join("/volume", sanitizedPath)+ _, stderr, err := s.execInContainerInternal(ctx, containerID, []string{"mkdir", "-p", targetPath})+ if err != nil {+ return err+ }+ if stderr != "" {+ return errors.Errorf("mkdir failed: %s", stderr)+ }++ actingUser := user+ if actingUser == nil {+ actingUser = &models.SystemUser+ }+ metadata := models.JSON{+ "action": "file_create",+ "path": dirPath,+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeFileCreate, volumeName, volumeName, actingUser.ID, actingUser.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume file create event", "volume", volumeName, "error", logErr.Error())+ }+ return nil+}++func (s *VolumeService) UploadFile(ctx context.Context, volumeName, destPath string, content io.Reader, filename string, user *models.User) error {+ slog.DebugContext(ctx, "volume service: upload file", "volume", volumeName, "dest_path", destPath, "filename", filename)++ sanitizedPath, err := utils.SanitizeBrowsePath(destPath)+ if err != nil {+ return errors.WrapIf(err, "invalid path")+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return err+ }++ containerID, cleanup, err := s.createTempContainerInternal(ctx, volumeName, false)+ if err != nil {+ return err+ }+ defer cleanup()++ var buf bytes.Buffer+ tw := tar.NewWriter(&buf)++ contentBytes, err := io.ReadAll(content)+ if err != nil {+ return err+ }++ hdr := &tar.Header{+ Name: filename,+ Mode: 0o644,+ Size: int64(len(contentBytes)),+ }+ if err := tw.WriteHeader(hdr); err != nil {+ return err+ }+ if _, err := tw.Write(contentBytes); err != nil {+ return err+ }+ if err := tw.Close(); err != nil {+ return err+ }++ targetDir := path.Join("/volume", sanitizedPath)+ _, err = dockerClient.CopyToContainer(ctx, containerID, client.CopyToContainerOptions{+ DestinationPath: targetDir,+ Content: &buf,+ })+ if err != nil {+ return errors.WrapIf(err, "failed to upload")+ }++ actingUser := user+ if actingUser == nil {+ actingUser = &models.SystemUser+ }+ metadata := models.JSON{+ "action": "file_upload",+ "path": destPath,+ "filename": filename,+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeFileUpload, volumeName, volumeName, actingUser.ID, actingUser.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume file upload event", "volume", volumeName, "error", logErr.Error())+ }++ return nil+}diff --git a/backend/api/handlers/volumes.go b/backend/internal/volume/handler.gosimilarity index 91%rename from backend/api/handlers/volumes.gorename to backend/internal/volume/handler.goindex 0c439f902..0c4ad56ed 100644--- a/backend/api/handlers/volumes.go+++ b/backend/internal/volume/handler.go@@ -1,4 +1,4 @@-package handlers+package volumeimport ("context"@@ -9,15 +9,18 @@ import ("strconv""strings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+"emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity""github.com/getarcaneapp/arcane/backend/v2/pkg/pagination""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base"volumetypes "github.com/getarcaneapp/arcane/types/v2/volume""github.com/moby/moby/client"@@ -26,9 +29,9 @@ import (// VolumeHandler provides Huma-based volume management endpoints.type VolumeHandler struct {- volumeService *services.VolumeService- dockerService *services.DockerClientService- activityService *services.ActivityService+ volumeService *VolumeService+ dockerService *docker.DockerClientService+ activityService *activity.ActivityServiceappCtx context.Context}@@ -294,130 +297,130 @@ type UploadAndRestoreOutput struct {}// RegisterVolumes registers volume management routes using Huma.-func RegisterVolumes(api huma.API, dockerService *services.DockerClientService, volumeService *services.VolumeService, activityService *services.ActivityService, appCtx ActivityAppContext) {+func RegisterVolumes(api huma.API, dockerService *docker.DockerClientService, volumeService *VolumeService, activityService *activity.ActivityService, appCtx handlerutil.ActivityAppContext) {h := &VolumeHandler{volumeService: volumeService,dockerService: dockerService,activityService: activityService,- appCtx: appCtx.contextInternal(),+ appCtx: appCtx.Context(),}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-volume-usage-counts",Method: http.MethodGet,Path: "/environments/{id}/volumes/counts",Summary: "Get volume usage counts",Description: "Get counts of volumes in use, unused, and total",Tags: []string{"Volumes"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesList, h.GetVolumeUsageCounts)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-volumes",Method: http.MethodGet,Path: "/environments/{id}/volumes",Summary: "List volumes",Description: "Get a paginated list of Docker volumes",Tags: []string{"Volumes"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesList, h.ListVolumes)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-volume",Method: http.MethodGet,Path: "/environments/{id}/volumes/{volumeName}",Summary: "Get volume by name",Description: "Get a Docker volume by its name",Tags: []string{"Volumes"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesRead, h.GetVolume)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-volume",Method: http.MethodPost,Path: "/environments/{id}/volumes",Summary: "Create a volume",Description: "Create a new Docker volume",Tags: []string{"Volumes"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesCreate, h.CreateVolume)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "remove-volume",Method: http.MethodDelete,Path: "/environments/{id}/volumes/{volumeName}",Summary: "Remove a volume",Description: "Remove a Docker volume by name",Tags: []string{"Volumes"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesDelete, h.RemoveVolume)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "prune-volumes",Method: http.MethodPost,Path: "/environments/{id}/volumes/prune",Summary: "Prune unused volumes",Description: "Remove all unused Docker volumes",Tags: []string{"Volumes"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesPrune, h.PruneVolumes)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-volume-usage",Method: http.MethodGet,Path: "/environments/{id}/volumes/{volumeName}/usage",Summary: "Get volume usage",Description: "Get containers using a specific volume",Tags: []string{"Volumes"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesRead, h.GetVolumeUsage)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-volume-sizes",Method: http.MethodGet,Path: "/environments/{id}/volumes/sizes",Summary: "Get volume sizes",Description: "Get disk usage sizes for all volumes (slow operation)",Tags: []string{"Volumes"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesList, h.GetVolumeSizes)// --- Volume Browsing Endpoints ---- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "browse-volume-directory",Method: http.MethodGet,Path: "/environments/{id}/volumes/{volumeName}/browse",Summary: "List volume directory",Tags: []string{"Volume Browser"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBrowse, h.BrowseDirectory)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-volume-file-content",Method: http.MethodGet,Path: "/environments/{id}/volumes/{volumeName}/browse/content",Summary: "Get file content preview",Tags: []string{"Volume Browser"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBrowse, h.GetFileContent)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "download-volume-file",Method: http.MethodGet,Path: "/environments/{id}/volumes/{volumeName}/browse/download",Summary: "Download file from volume",Tags: []string{"Volume Browser"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBrowse, h.DownloadFile)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "upload-volume-file",Method: http.MethodPost,Path: "/environments/{id}/volumes/{volumeName}/browse/upload",Summary: "Upload file to volume",Tags: []string{"Volume Browser"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),RequestBody: &huma.RequestBody{Content: map[string]*huma.MediaType{"multipart/form-data": {@@ -437,105 +440,105 @@ func RegisterVolumes(api huma.API, dockerService *services.DockerClientService,},}, authz.PermVolumesUpload, h.UploadFile)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-volume-directory",Method: http.MethodPost,Path: "/environments/{id}/volumes/{volumeName}/browse/mkdir",Summary: "Create directory in volume",Tags: []string{"Volume Browser"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesUpload, h.CreateDirectory)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-volume-file",Method: http.MethodDelete,Path: "/environments/{id}/volumes/{volumeName}/browse",Summary: "Delete file or directory in volume",Tags: []string{"Volume Browser"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesDelete, h.DeleteFile)// --- Volume Backup Endpoints ---- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-volume-backups",Method: http.MethodGet,Path: "/environments/{id}/volumes/{volumeName}/backups",Summary: "List volume backups",Tags: []string{"Volume Backup"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBrowse, h.ListBackups)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-volume-backup",Method: http.MethodPost,Path: "/environments/{id}/volumes/{volumeName}/backups",Summary: "Create volume backup",Tags: []string{"Volume Backup"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBackup, h.CreateBackup)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "restore-volume-backup",Method: http.MethodPost,Path: "/environments/{id}/volumes/{volumeName}/backups/{backupId}/restore",Summary: "Restore volume backup",Tags: []string{"Volume Backup"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBackup, h.RestoreBackup)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "restore-volume-backup-files",Method: http.MethodPost,Path: "/environments/{id}/volumes/{volumeName}/backups/{backupId}/restore-files",Summary: "Restore specific files from a volume backup",Tags: []string{"Volume Backup"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBackup, h.RestoreBackupFiles)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-volume-backup",Method: http.MethodDelete,Path: "/environments/{id}/volumes/backups/{backupId}",Summary: "Delete volume backup",Tags: []string{"Volume Backup"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBackup, h.DeleteBackup)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "download-volume-backup",Method: http.MethodGet,Path: "/environments/{id}/volumes/backups/{backupId}/download",Summary: "Download volume backup",Tags: []string{"Volume Backup"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBrowse, h.DownloadBackup)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "backup-has-path",Method: http.MethodGet,Path: "/environments/{id}/volumes/backups/{backupId}/has-path",Summary: "Check if backup contains path",Tags: []string{"Volume Backup"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBrowse, h.BackupHasPath)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-backup-files",Method: http.MethodGet,Path: "/environments/{id}/volumes/backups/{backupId}/files",Summary: "List files in a volume backup",Tags: []string{"Volume Backup"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVolumesBrowse, h.ListBackupFiles)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "upload-volume-backup",Method: http.MethodPost,Path: "/environments/{id}/volumes/{volumeName}/backups/upload",Summary: "Upload and restore volume backup",Tags: []string{"Volume Backup"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),RequestBody: &huma.RequestBody{Content: map[string]*huma.MediaType{"multipart/form-data": {@@ -558,7 +561,7 @@ func RegisterVolumes(api huma.API, dockerService *services.DockerClientService,// ListVolumes returns a paginated list of volumes.func (h *VolumeHandler) ListVolumes(ctx context.Context, input *ListVolumesInput) (*ListVolumesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if input.InUse != "" {params.Filters["inUse"] = input.InUse}@@ -585,7 +588,7 @@ func (h *VolumeHandler) ListVolumes(ctx context.Context, input *ListVolumesInputUnused: counts.Unused,Total: counts.Total,},- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -607,7 +610,7 @@ func (h *VolumeHandler) GetVolume(ctx context.Context, input *GetVolumeInput) (*// CreateVolume creates a new Docker volume.func (h *VolumeHandler) CreateVolume(ctx context.Context, input *CreateVolumeInput) (*CreateVolumeOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -655,7 +658,7 @@ func (h *VolumeHandler) CreateVolume(ctx context.Context, input *CreateVolumeInp// RemoveVolume removes a Docker volume.func (h *VolumeHandler) RemoveVolume(ctx context.Context, input *RemoveVolumeInput) (*RemoveVolumeOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -840,7 +843,7 @@ func (h *VolumeHandler) DownloadFile(ctx context.Context, input *DownloadFileInp}func (h *VolumeHandler) UploadFile(ctx context.Context, input *UploadFileInput) (*base.ApiResponse[base.MessageResponse], error) {- file, fileHeader, err := openUploadedFileInternal(input.RawBody)+ file, fileHeader, err := handlerutil.OpenUploadedFile(input.RawBody)if err != nil {return nil, err}@@ -968,7 +971,7 @@ func (h *VolumeHandler) ListBackups(ctx context.Context, input *ListBackupsInputBody: VolumeBackupPaginatedResponse{Success: true,Data: backups,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),Warnings: func() []string {if warning == "" {return nil@@ -980,7 +983,7 @@ func (h *VolumeHandler) ListBackups(ctx context.Context, input *ListBackupsInput}func (h *VolumeHandler) CreateBackup(ctx context.Context, input *CreateBackupInput) (*CreateBackupOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -1016,7 +1019,7 @@ func (h *VolumeHandler) CreateBackup(ctx context.Context, input *CreateBackupInp}func (h *VolumeHandler) RestoreBackup(ctx context.Context, input *RestoreBackupInput) (*RestoreBackupOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -1051,7 +1054,7 @@ func (h *VolumeHandler) RestoreBackup(ctx context.Context, input *RestoreBackupI}func (h *VolumeHandler) RestoreBackupFiles(ctx context.Context, input *RestoreBackupFilesInput) (*RestoreBackupFilesOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -1176,12 +1179,12 @@ func (h *VolumeHandler) DownloadBackup(ctx context.Context, input *DownloadBacku}func (h *VolumeHandler) UploadAndRestore(ctx context.Context, input *UploadAndRestoreInput) (*UploadAndRestoreOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}- file, fileHeader, err := openUploadedFileInternal(input.RawBody)+ file, fileHeader, err := handlerutil.OpenUploadedFile(input.RawBody)if err != nil {return nil, err}diff --git a/backend/internal/volume/helper_container.go b/backend/internal/volume/helper_container.gonew file mode 100644index 000000000..7c3d96440--- /dev/null+++ b/backend/internal/volume/helper_container.go@@ -0,0 +1,453 @@+package volume++import (+ "bytes"+ "context"+ "fmt"+ "io"+ "log/slog"+ "strings"+ "time"++ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/volumehelper"+ "github.com/moby/moby/api/types/container"+ "github.com/moby/moby/client"+ "github.com/samber/mo"+)++// volumeHelper tracks a reused read-only browse helper container and the last+// time it serviced a request, so idle helpers can be reaped. inUse counts the+// requests currently holding the helper: the reaper must not remove a helper+// mid-download just because it was acquired longer ago than the idle timeout.+type volumeHelper struct {+ id string+ lastUsedAt time.Time+ inUse int+}++const volumeHelperImage = volumehelper.DefaultToolsImage++func getVolumeHelperImageInternal(ctx context.Context, dockerService *docker.DockerClientService, imageService *image.ImageService, dockerClient *client.Client) (string, error) {+ slog.DebugContext(ctx, "volume service: resolve helper image")+ var err error+ if dockerClient == nil {+ if dockerService == nil {+ return "", errors.New("docker service unavailable")+ }+ dockerClient, err = dockerService.GetClient(ctx)+ if err != nil {+ return "", errors.WrapIf(err, "failed to get docker client")+ }+ }++ if _, err := dockerClient.ImageInspect(ctx, volumeHelperImage); err == nil {+ slog.InfoContext(ctx, "volume service: helper image strategy selected", "strategy", "tools-local", "image", volumeHelperImage)+ return volumeHelperImage, nil+ }++ var pullErr error+ if imageService != nil {+ pullImageErr := imageService.PullImage(ctx, volumeHelperImage, io.Discard, models.SystemUser, nil)+ if pullImageErr == nil {+ slog.InfoContext(ctx, "volume service: helper image strategy selected", "strategy", "tools-pulled", "image", volumeHelperImage)+ return volumeHelperImage, nil+ }+ pullErr = pullImageErr+ slog.WarnContext(ctx, "volume service: failed to pull tools helper image, attempting arcane fallback", "error", pullImageErr.Error())+ } else {+ pullErr = errors.New("image service unavailable")+ slog.WarnContext(ctx, "volume service: image service unavailable, attempting arcane fallback")+ }++ if fallback, ok := volumehelper.ResolveArcaneRuntimeImage(ctx, dockerClient).Get(); ok {+ slog.InfoContext(ctx, "volume service: helper image strategy selected", "strategy", "arcane-fallback", "source", fallback.Source, "image", fallback.Image)+ return fallback.Image, nil+ }++ return "", errors.WrapIf(pullErr, "failed to resolve helper image: tools image unavailable and arcane fallback not found")+}++func (s *VolumeService) createTempContainerInternal(ctx context.Context, volumeName string, readOnly bool) (string, func(), error) {+ slog.DebugContext(ctx, "volume service: create temp container", "volume", volumeName, "read_only", readOnly)+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return "", nil, err+ }++ if !readOnly {+ return s.startHelperContainerInternal(ctx, dockerClient, volumeName, false)+ }++ // Read-only helpers are shared and reaped when idle; the caller's cleanup+ // releases its in-use hold instead of removing the container. The resolve →+ // acquire gap is racy against the reaper, so retry a resolve whose helper+ // was reaped before this caller could take its hold.+ for range 3 {+ containerID, err := s.resolveReadOnlyHelperInternal(ctx, dockerClient, volumeName)+ if err != nil {+ return "", nil, err+ }+ if release, ok := s.acquireHelperInternal(volumeName, containerID); ok {+ return containerID, release, nil+ }+ }+ return "", nil, errors.New("failed to acquire volume helper container")+}++// resolveReadOnlyHelperInternal returns the shared read-only helper container ID+// for volumeName, creating it if needed. singleflight collapses concurrent+// misses: without it both requests create a helper and the loser is orphaned,+// holding the volume mount with no cleanup path until Arcane restarts. The+// creation itself can pull an image, so it must not run under helperMu.+func (s *VolumeService) resolveReadOnlyHelperInternal(ctx context.Context, dockerClient *client.Client, volumeName string) (string, error) {+ resultCh := s.helperGroup.DoChan(volumeName, func() (any, error) {+ if containerID, ok := s.getReusableReadOnlyContainerInternal(ctx, dockerClient, volumeName).Get(); ok {+ return containerID, nil+ }++ // Detached from the caller's context so a client that walks away+ // mid-create doesn't abort the helper the other waiters are blocked on,+ // but still bounded: WithoutCancel alone would let a hung daemon or+ // image pull wedge this singleflight key forever, queueing every later+ // browse of the volume behind it with no recovery until restart.+ createCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), timeouts.DefaultDockerImagePull)+ defer cancel()++ containerID, _, createErr := s.startHelperContainerInternal(createCtx, dockerClient, volumeName, true)+ if createErr != nil {+ return nil, createErr+ }++ s.helperMu.Lock()+ s.helperByVolume[volumeName] = &volumeHelper{id: containerID, lastUsedAt: time.Now()}+ s.helperMu.Unlock()+ return containerID, nil+ })++ // DoChan instead of Do so a caller whose request is canceled unblocks+ // immediately; the shared create keeps running for the remaining waiters+ // (and lands in helperByVolume for the next request even if none remain).+ var shared any+ select {+ case <-ctx.Done():+ return "", errors.WrapIf(ctx.Err(), "canceled waiting for volume helper container")+ case result := <-resultCh:+ if result.Err != nil {+ return "", result.Err+ }+ shared = result.Val+ }++ containerID, ok := shared.(string)+ if !ok || containerID == "" {+ return "", errors.New("failed to resolve volume helper container")+ }+ return containerID, nil+}++// acquireHelperInternal takes an in-use hold on volumeName's helper, reporting+// false when the helper has been removed (e.g. reaped) since it was resolved.+// The returned release drops the hold and refreshes the idle clock, so the idle+// timeout is measured from the end of the last request, not its start — a+// download running longer than the timeout used to have its helper reaped out+// from under it, truncating the stream.+func (s *VolumeService) acquireHelperInternal(volumeName, containerID string) (func(), bool) {+ s.helperMu.Lock()+ defer s.helperMu.Unlock()++ helper := s.helperByVolume[volumeName]+ if helper == nil || helper.id != containerID {+ return nil, false+ }++ helper.inUse+++ helper.lastUsedAt = time.Now()++ return func() {+ s.helperMu.Lock()+ defer s.helperMu.Unlock()+ if current := s.helperByVolume[volumeName]; current == helper {+ helper.inUse--+ helper.lastUsedAt = time.Now()+ }+ }, true+}++// startHelperContainerInternal creates and starts a volume helper container and+// returns a cleanup that removes it. Tracking of reusable read-only helpers is+// the caller's responsibility.+func (s *VolumeService) startHelperContainerInternal(ctx context.Context, dockerClient *client.Client, volumeName string, readOnly bool) (string, func(), error) {+ helperImage, err := getVolumeHelperImageInternal(ctx, s.dockerService, s.imageService, dockerClient)+ if err != nil {+ return "", nil, err+ }++ config := &container.Config{+ Image: helperImage,+ Cmd: []string{"sleep", "infinity"},+ NetworkDisabled: true,+ Labels: volumehelper.Labels(),+ }++ hostConfig := volumehelper.HostConfig(helperImage, []string{+ fmt.Sprintf("%s:/volume%s", volumeName, func() string {+ if readOnly {+ return ":ro"+ }+ return ""+ }()),+ }, nil)++ resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{+ Config: config,+ HostConfig: hostConfig,+ })+ if err != nil {+ return "", nil, errors.WrapIf(err, "failed to create temp container")+ }++ if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {+ _, _ = dockerClient.ContainerRemove(ctx, resp.ID, volumehelper.RemoveOptions())+ return "", nil, errors.WrapIf(err, "failed to start temp container")+ }++ cleanup := func() {+ _, _ = dockerClient.ContainerRemove(context.WithoutCancel(ctx), resp.ID, volumehelper.RemoveOptions())+ }++ return resp.ID, cleanup, nil+}++func (s *VolumeService) getReusableReadOnlyContainerInternal(ctx context.Context, dockerClient *client.Client, volumeName string) mo.Option[string] {+ s.helperMu.Lock()+ helper := s.helperByVolume[volumeName]+ s.helperMu.Unlock()+ if helper == nil || helper.id == "" {+ return mo.None[string]()+ }++ inspect, err := libarcane.ContainerInspectWithCompatibility(ctx, dockerClient, helper.id, client.ContainerInspectOptions{})+ if err != nil || inspect.Container.State == nil || !inspect.Container.State.Running {+ s.helperMu.Lock()+ delete(s.helperByVolume, volumeName)+ s.helperMu.Unlock()+ return mo.None[string]()+ }++ s.touchHelperInternal(volumeName)++ return mo.Some(helper.id)+}++// touchHelperInternal records that the helper for volumeName just serviced a+// request, resetting its idle clock. No-op if the entry is gone.+func (s *VolumeService) touchHelperInternal(volumeName string) {+ s.helperMu.Lock()+ defer s.helperMu.Unlock()+ if helper := s.helperByVolume[volumeName]; helper != nil {+ helper.lastUsedAt = time.Now()+ }+}++func (s *VolumeService) CleanupHelperContainers(ctx context.Context) {+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ slog.WarnContext(ctx, "failed to get docker client for helper cleanup", "error", err)+ return+ }++ s.helperMu.Lock()+ helperIDs := make([]string, 0, len(s.helperByVolume))+ for _, helper := range s.helperByVolume {+ if helper != nil && helper.id != "" {+ helperIDs = append(helperIDs, helper.id)+ }+ }+ s.helperByVolume = make(map[string]*volumeHelper)+ s.helperMu.Unlock()++ for _, containerID := range helperIDs {+ if _, err := dockerClient.ContainerRemove(ctx, containerID, volumehelper.RemoveOptions()); err != nil {+ slog.WarnContext(ctx, "failed to remove helper container", "container_id", containerID, "error", err.Error())+ }+ }+}++// ReapIdleHelpers removes reused read-only browse helper containers that have+// not serviced a request within idleTimeout. It is map-driven (orphaned helpers+// not tracked in helperByVolume are left to the startup orphan sweep). Entries+// are removed from the map before the container is removed, so a concurrent+// request simply gets a cache miss and re-creates a fresh helper.+func (s *VolumeService) ReapIdleHelpers(ctx context.Context, idleTimeout time.Duration) (int, error) {+ if idleTimeout <= 0 {+ return 0, nil+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return 0, errors.WrapIf(err, "failed to get docker client for idle helper reap")+ }++ staleIDs := s.collectStaleHelperIDsInternal(time.Now(), idleTimeout)++ removed := 0+ for _, containerID := range staleIDs {+ if _, err := dockerClient.ContainerRemove(ctx, containerID, volumehelper.RemoveOptions()); err != nil {+ slog.WarnContext(ctx, "failed to remove idle helper container", "container_id", containerID, "error", err.Error())+ continue+ }+ removed+++ }++ return removed, nil+}++// collectStaleHelperIDsInternal removes idle (and any nil) entries from the helper+// map and returns the container IDs that should be removed. Pure map/mutex+// bookkeeping with no Docker calls, so it can be unit-tested directly. Entries are+// dropped before their containers are removed so a concurrent request gets a clean+// cache miss and re-creates a fresh helper.+func (s *VolumeService) collectStaleHelperIDsInternal(now time.Time, idleTimeout time.Duration) []string {+ staleIDs := make([]string, 0)+ s.helperMu.Lock()+ defer s.helperMu.Unlock()+ for volumeName, helper := range s.helperByVolume {+ if helper == nil {+ delete(s.helperByVolume, volumeName)+ continue+ }+ // A helper with active holds is serving a request right now (e.g. a+ // download streaming for longer than the idle timeout); it becomes+ // reapable once released, since release refreshes lastUsedAt.+ if helper.inUse > 0 {+ continue+ }+ if now.Sub(helper.lastUsedAt) >= idleTimeout {+ staleIDs = append(staleIDs, helper.id)+ delete(s.helperByVolume, volumeName)+ }+ }+ return staleIDs+}++// StopHelper removes the reused read-only browse helper for a single volume, if+// one exists. It is idempotent: stopping a volume with no active helper returns+// nil.+func (s *VolumeService) StopHelper(ctx context.Context, volumeName string) error {+ if strings.TrimSpace(volumeName) == "" {+ return nil+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return errors.WrapIf(err, "failed to get docker client for helper stop")+ }++ containerID := s.takeHelperIDInternal(volumeName)+ if containerID == "" {+ return nil+ }++ if _, err := dockerClient.ContainerRemove(ctx, containerID, volumehelper.RemoveOptions()); err != nil {+ return errors.WrapIf(err, "failed to remove helper container")+ }++ return nil+}++// takeHelperIDInternal removes the helper entry for volumeName and returns its+// container ID, or "" if there was none. Pure map/mutex bookkeeping.+func (s *VolumeService) takeHelperIDInternal(volumeName string) string {+ s.helperMu.Lock()+ defer s.helperMu.Unlock()+ helper := s.helperByVolume[volumeName]+ delete(s.helperByVolume, volumeName)+ if helper == nil {+ return ""+ }+ return helper.id+}++func (s *VolumeService) CleanupOrphanedVolumeHelpers(ctx context.Context) (int, error) {+ slog.DebugContext(ctx, "volume service: cleanup orphaned volume helper containers")++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return 0, errors.WrapIf(err, "failed to get docker client for orphan helper cleanup")+ }++ containers, err := dockerClient.ContainerList(ctx, client.ContainerListOptions{All: true})+ if err != nil {+ return 0, errors.WrapIf(err, "failed to list containers for orphan helper cleanup")+ }++ removedCount := 0+ for _, c := range containers.Items {+ if !volumehelper.IsHelperContainer(c) {+ continue+ }++ if _, err := dockerClient.ContainerRemove(ctx, c.ID, volumehelper.RemoveOptions()); err != nil {+ slog.WarnContext(ctx,+ "volume service: failed to remove orphaned volume helper container",+ "container_id", c.ID,+ "container_names", c.Names,+ "error", err.Error(),+ )+ continue+ }++ removedCount+++ }++ return removedCount, nil+}++func (s *VolumeService) removeHelperEntry(volumeName string) {+ if strings.TrimSpace(volumeName) == "" {+ return+ }+ s.helperMu.Lock()+ delete(s.helperByVolume, volumeName)+ s.helperMu.Unlock()+}++func (s *VolumeService) execInContainerInternal(ctx context.Context, containerID string, cmd []string) (string, string, error) {+ slog.DebugContext(ctx, "volume service: exec in container", "container_id", containerID, "cmd", cmd)+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return "", "", err+ }++ var stdout, stderr bytes.Buffer+ exitCode, err := dockerutil.ExecInContainer(ctx, dockerClient, containerID, client.ExecCreateOptions{+ AttachStdout: true,+ AttachStderr: true,+ Cmd: cmd,+ }, &stdout, &stderr)+ if err != nil {+ return stdout.String(), stderr.String(), err+ }++ if exitCode != 0 {+ execErr := strings.TrimSpace(stderr.String())+ if execErr == "" {+ execErr = strings.TrimSpace(stdout.String())+ }+ if execErr != "" {+ return stdout.String(), stderr.String(), errors.Errorf("command exited with code %d: %s", exitCode, execErr)+ }+ return stdout.String(), stderr.String(), errors.Errorf("command exited with code %d", exitCode)+ }++ return stdout.String(), stderr.String(), nil+}diff --git a/backend/internal/volume/listing.go b/backend/internal/volume/listing.gonew file mode 100644index 000000000..5c6be0a35--- /dev/null+++ b/backend/internal/volume/listing.go@@ -0,0 +1,405 @@+package volume++import (+ "context"+ "log/slog"+ "strings"+ "time"++ "emperror.dev/errors"++ docker "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ volumetypes "github.com/getarcaneapp/arcane/types/v2/volume"+ "github.com/moby/moby/api/types/container"+ "github.com/moby/moby/api/types/mount"+ "github.com/moby/moby/api/types/volume"+ "github.com/moby/moby/client"+ "github.com/samber/mo"+)++func (s *VolumeService) GetVolumeUsage(ctx context.Context, name string) (bool, []string, error) {+ slog.DebugContext(ctx, "volume service: get volume usage", "volume", name)+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return false, nil, errors.WrapIf(err, "failed to connect to Docker")+ }++ vol, err := dockerClient.VolumeInspect(ctx, name, client.VolumeInspectOptions{})+ if err != nil {+ return false, nil, errors.WrapIf(err, "volume not found")+ }++ containerIDs, err := docker.GetContainersUsingVolume(ctx, dockerClient, vol.Volume.Name)+ if err != nil {+ return false, nil, errors.WrapIf(err, "failed to get containers using volume")+ }++ inUse := len(containerIDs) > 0+ return inUse, containerIDs, nil+}++// VolumeSizeData holds size information for a volume.+type VolumeSizeData struct {+ Size int64+ RefCount int64+}++// GetVolumeSizes returns disk usage data for all volumes.+// This is a slow operation as it calls Docker's DiskUsage API.+func (s *VolumeService) GetVolumeSizes(ctx context.Context) (map[string]VolumeSizeData, error) {+ slog.DebugContext(ctx, "volume service: get volume sizes")+ settings := s.settingsService.GetSettingsConfig()+ apiCtx, cancel := context.WithTimeout(ctx, timeouts.GetDuration(settings.DockerAPITimeout.AsInt(), timeouts.DefaultDockerAPI))+ defer cancel()++ dockerClient, err := s.dockerService.GetClient(apiCtx)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to connect to Docker")+ }++ usageVolumes, err := docker.GetVolumeUsageData(apiCtx, dockerClient)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to get volume usage data")+ }++ result := make(map[string]VolumeSizeData, len(usageVolumes))+ for _, v := range usageVolumes {+ if v.UsageData != nil {+ result[v.Name] = VolumeSizeData{+ Size: v.UsageData.Size,+ RefCount: v.UsageData.RefCount,+ }+ }+ }++ return result, nil+}++func enrichVolumesWithUsageDataInternal(volumes []volume.Volume, usageVolumes []volume.Volume) []volume.Volume {+ usageByName := make(map[string]*volume.UsageData, len(usageVolumes))+ for _, uv := range usageVolumes {+ if uv.Name == "" || uv.UsageData == nil {+ continue+ }+ // Keep first-seen value to preserve previous nested-loop behavior.+ if _, exists := usageByName[uv.Name]; !exists {+ usageByName[uv.Name] = uv.UsageData+ }+ }++ result := make([]volume.Volume, 0, len(volumes))+ for _, v := range volumes {+ if usageData, exists := usageByName[v.Name]; exists {+ v.UsageData = usageData+ }++ result = append(result, v)+ }+ return result+}++func buildVolumeContainerMapInternal(ctx context.Context, dockerClient *client.Client) (map[string][]string, error) {+ containers, err := dockerClient.ContainerList(ctx, client.ContainerListOptions{All: true})+ if err != nil {+ return nil, errors.WrapIf(err, "failed to list containers")+ }++ volumeContainerMap := make(map[string][]string)+ for _, c := range containers.Items {+ for _, m := range c.Mounts {+ if m.Type == mount.TypeVolume && m.Name != "" {+ volumeContainerMap[m.Name] = append(volumeContainerMap[m.Name], c.ID)+ }+ }+ }++ return volumeContainerMap, nil+}++func (s *VolumeService) buildVolumePaginationConfigInternal() pagination.Config[volumetypes.Volume] {+ return pagination.Config[volumetypes.Volume]{+ SearchAccessors: []pagination.SearchAccessor[volumetypes.Volume]{+ func(v volumetypes.Volume) (string, error) { return v.Name, nil },+ func(v volumetypes.Volume) (string, error) { return v.Driver, nil },+ func(v volumetypes.Volume) (string, error) { return v.Mountpoint, nil },+ func(v volumetypes.Volume) (string, error) { return v.Scope, nil },+ },+ SortBindings: s.buildVolumeSortBindingsInternal(),+ FilterAccessors: buildVolumeFilterAccessorsInternal(),+ }+}++func (s *VolumeService) buildVolumeSortBindingsInternal() []pagination.SortBinding[volumetypes.Volume] {+ createdSortFn := s.compareVolumeCreatedInternal++ return []pagination.SortBinding[volumetypes.Volume]{+ {+ Key: "name",+ Fn: func(a, b volumetypes.Volume) int { return strings.Compare(a.Name, b.Name) },+ },+ {+ Key: "driver",+ Fn: func(a, b volumetypes.Volume) int { return strings.Compare(a.Driver, b.Driver) },+ },+ {+ Key: "mountpoint",+ Fn: func(a, b volumetypes.Volume) int { return strings.Compare(a.Mountpoint, b.Mountpoint) },+ },+ {+ Key: "scope",+ Fn: func(a, b volumetypes.Volume) int { return strings.Compare(a.Scope, b.Scope) },+ },+ {+ Key: "created",+ Fn: createdSortFn,+ },+ {+ Key: "createdAt",+ Fn: createdSortFn,+ },+ {+ Key: "inUse",+ Fn: func(a, b volumetypes.Volume) int {+ if a.InUse == b.InUse {+ return 0+ }+ if a.InUse {+ return -1+ }+ return 1+ },+ },+ {+ Key: "size",+ Fn: compareVolumeSizesInternal,+ },+ }+}++func compareVolumeSizesInternal(a, b volumetypes.Volume) int {+ aSize := a.Size+ bSize := b.Size++ if aSize == 0 && a.UsageData != nil {+ aSize = a.UsageData.Size+ }+ if bSize == 0 && b.UsageData != nil {+ bSize = b.UsageData.Size+ }++ if aSize == bSize {+ return strings.Compare(a.Name, b.Name)+ }+ if aSize < bSize {+ return -1+ }+ return 1+}++func (s *VolumeService) compareVolumeCreatedInternal(a, b volumetypes.Volume) int {+ aTime, aOk := parseVolumeCreatedAtInternal(a.CreatedAt).Get()+ bTime, bOk := parseVolumeCreatedAtInternal(b.CreatedAt).Get()+ if aOk && bOk {+ if aTime.Before(bTime) {+ return -1+ }+ if aTime.After(bTime) {+ return 1+ }+ return 0+ }+ return strings.Compare(a.CreatedAt, b.CreatedAt)+}++func parseVolumeCreatedAtInternal(createdAt string) mo.Option[time.Time] {+ if createdAt == "" {+ return mo.None[time.Time]()+ }+ if parsed, err := time.Parse(time.RFC3339Nano, createdAt); err == nil {+ return mo.Some(parsed)+ }+ if parsed, err := time.Parse(time.RFC3339, createdAt); err == nil {+ return mo.Some(parsed)+ }+ return mo.None[time.Time]()+}++func buildVolumeFilterAccessorsInternal() []pagination.FilterAccessor[volumetypes.Volume] {+ return []pagination.FilterAccessor[volumetypes.Volume]{+ {+ Key: "inUse",+ Fn: func(v volumetypes.Volume, filterValue string) bool {+ if filterValue == "true" {+ return v.InUse+ }+ if filterValue == "false" {+ return !v.InUse+ }+ return true+ },+ },+ }+}++func calculateVolumeUsageCountsInternal(items []volumetypes.Volume) volumetypes.UsageCounts {+ counts := volumetypes.UsageCounts{+ Total: len(items),+ }+ for _, v := range items {+ if v.InUse {+ counts.Inuse+++ } else {+ counts.Unused+++ }+ }+ return counts+}++// CountUsageFromSnapshot derives volume usage counts from an+// already-fetched volume and container listing. It mirrors the semantics of+// ListVolumesPaginated — internal volumes are excluded and a volume counts as+// in use when any container mounts it — so dashboard tile numbers agree with+// the volumes page.+func (s *VolumeService) CountUsageFromSnapshot(volumes []volume.Volume, containers []container.Summary) volumetypes.UsageCounts {+ inUse := make(map[string]struct{})+ for _, c := range containers {+ for _, m := range c.Mounts {+ if m.Type == mount.TypeVolume && m.Name != "" {+ inUse[m.Name] = struct{}{}+ }+ }+ }++ counts := volumetypes.UsageCounts{}+ for _, v := range volumes {+ if s.isInternalVolumeInternal(volumetypes.NewSummary(v)) {+ continue+ }+ counts.Total+++ if _, ok := inUse[v.Name]; ok {+ counts.Inuse+++ } else {+ counts.Unused+++ }+ }+ return counts+}++func (s *VolumeService) isInternalVolumeInternal(v volumetypes.Volume) bool {+ if strings.EqualFold(strings.TrimSpace(v.Name), strings.TrimSpace(s.backupVolumeName)) {+ return true+ }++ return libarcane.IsInternalContainer(v.Labels)+}++func (s *VolumeService) ListVolumesPaginated(ctx context.Context, params pagination.QueryParams, includeInternal bool) ([]volumetypes.Volume, pagination.Response, volumetypes.UsageCounts, error) {+ startedAt := time.Now()+ slog.DebugContext(ctx, "volume service: list volumes paginated", "search", params.Search, "sort", params.Sort, "order", params.Order, "start", params.Start, "limit", params.Limit, "include_internal", includeInternal)+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, pagination.Response{}, volumetypes.UsageCounts{}, errors.WrapIf(err, "failed to connect to Docker")+ }++ // Run volume list and container list in parallel for better performance+ type volumeListResult struct {+ volumes []volume.Volume+ err error+ }+ type containerMapResult struct {+ containerMap map[string][]string+ err error+ }++ volChan := make(chan volumeListResult, 1)+ containerChan := make(chan containerMapResult, 1)++ settings := s.settingsService.GetSettingsConfig()+ apiCtx, cancel := context.WithTimeout(ctx, timeouts.GetDuration(settings.DockerAPITimeout.AsInt(), timeouts.DefaultDockerAPI))+ defer cancel()++ go func(ctx context.Context) {+ volListBody, err := dockerClient.VolumeList(ctx, client.VolumeListOptions{})+ volChan <- volumeListResult{volumes: volListBody.Items, err: err}+ }(apiCtx)++ go func(ctx context.Context) {+ containerMap, err := buildVolumeContainerMapInternal(ctx, dockerClient)+ containerChan <- containerMapResult{containerMap: containerMap, err: err}+ }(apiCtx)++ // Wait for both results+ volResult := <-volChan+ if volResult.err != nil {+ return nil, pagination.Response{}, volumetypes.UsageCounts{}, errors.WrapIf(volResult.err, "failed to list Docker volumes")+ }++ containerResult := <-containerChan+ volumeContainerMap := containerResult.containerMap+ if containerResult.err != nil {+ slog.WarnContext(ctx, "failed to build volume-container map", "error", containerResult.err.Error())+ volumeContainerMap = make(map[string][]string)+ }++ effectiveParams := params+ usageCacheSnapshot := "not_requested"++ // Size sorting consumes the current cache snapshot and refreshes it in the+ // background so this list request never waits for Docker's DiskUsage call.+ var usageVolumes []volume.Volume+ if params.Sort == "size" {+ if uv, found := docker.GetVolumeUsageDataStaleWhileRevalidate(apiCtx, dockerClient).Get(); found && (len(uv) > 0 || len(volResult.volumes) == 0) {+ usageVolumes = uv+ usageCacheSnapshot = "available"+ } else {+ usageCacheSnapshot = "missing"+ effectiveParams.Sort = "name"+ effectiveParams.Order = pagination.SortAsc+ }+ }++ volumes := enrichVolumesWithUsageDataInternal(volResult.volumes, usageVolumes)++ items := make([]volumetypes.Volume, 0, len(volumes))+ for _, v := range volumes {+ volDto := volumetypes.NewSummary(v)+ if !includeInternal && s.isInternalVolumeInternal(volDto) {+ continue+ }+ if containerIDs, ok := volumeContainerMap[v.Name]; ok {+ volDto.Containers = containerIDs+ if len(containerIDs) > 0 {+ volDto.InUse = true+ }+ }+ items = append(items, volDto)+ }++ config := s.buildVolumePaginationConfigInternal()+ result := pagination.SearchOrderAndPaginate(items, effectiveParams, config)+ counts := calculateVolumeUsageCountsInternal(items)+ paginationResp := pagination.BuildResponse(result.TotalCount, result.TotalAvailable, effectiveParams)+ slog.DebugContext(ctx, "volume service: listed volumes",+ "docker_host", dockerClient.DaemonHost(),+ "requested_sort", params.Sort,+ "requested_order", params.Order,+ "effective_sort", effectiveParams.Sort,+ "effective_order", effectiveParams.Order,+ "usage_cache_snapshot", usageCacheSnapshot,+ "docker_volumes", len(volResult.volumes),+ "usage_volumes", len(usageVolumes),+ "included_volumes", len(items),+ "matched_volumes", result.TotalCount,+ "returned_volumes", len(result.Items),+ "container_volume_count", len(volumeContainerMap),+ "filter_count", len(params.Filters),+ "current_page", paginationResp.CurrentPage,+ "total_pages", paginationResp.TotalPages,+ "duration", time.Since(startedAt),+ )++ return result.Items, paginationResp, counts, nil+}diff --git a/backend/internal/volume/module.go b/backend/internal/volume/module.gonew file mode 100644index 000000000..ae2a26e82--- /dev/null+++ b/backend/internal/volume/module.go@@ -0,0 +1,59 @@+// Package volume owns Docker volumes: CRUD and pruning, the helper-container+// file browser, backup and restore, and the HTTP surface for all of it.+package volume++import (+ "github.com/danielgtaylor/huma/v2"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"++ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+)++// Dependencies are the collaborators the volume domain needs.+type Dependencies struct {+ DB *database.DB+ Docker *docker.DockerClientService+ Event *event.EventService+ Settings *settings.SettingsService+ Image *image.ImageService+ Activity *activity.ActivityService+ BackupVolumeName string+}++// Module wires the volume domain and mounts its routes.+type Module struct {+ service *VolumeService+ deps Dependencies+}++// New builds the volume domain from its dependencies.+func New(deps Dependencies) *Module {+ return &Module{+ service: NewVolumeService(deps.DB, deps.Docker, deps.Event, deps.Settings, deps.Image, deps.BackupVolumeName),+ deps: deps,+ }+}++// Service exposes the volume service to collaborators that use it directly,+// such as the helper-container reaper job.+func (m *Module) Service() *VolumeService {+ if m == nil {+ return nil+ }+ return m.service+}++// RegisterRoutes mounts the volume endpoints. A nil module still registers, so+// OpenAPI spec generation can discover the routes without a service graph.+func (m *Module) RegisterRoutes(api huma.API, appCtx handlerutil.ActivityAppContext) {+ if m == nil {+ RegisterVolumes(api, nil, nil, nil, appCtx)+ return+ }+ RegisterVolumes(api, m.deps.Docker, m.service, m.deps.Activity, appCtx)+}diff --git a/backend/internal/volume/service.go b/backend/internal/volume/service.gonew file mode 100644index 000000000..05b17f2d5--- /dev/null+++ b/backend/internal/volume/service.go@@ -0,0 +1,247 @@+package volume++import (+ "context"+ "log/slog"+ "strings"+ "sync"++ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/image"++ "emperror.dev/errors"++ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"+ volumetypes "github.com/getarcaneapp/arcane/types/v2/volume"+ "github.com/moby/moby/client"+ "golang.org/x/sync/singleflight"+)++type VolumeService struct {+ db *database.DB+ dockerService *docker.DockerClientService+ eventService *event.EventService+ settingsService *settings.SettingsService+ imageService *image.ImageService+ backupVolumeName string+ helperMu sync.Mutex+ helperByVolume map[string]*volumeHelper+ // helperGroup deduplicates concurrent read-only helper creation per volume.+ // Without it two simultaneous browse requests each create a helper and the+ // second overwrites the first in helperByVolume, orphaning a `sleep infinity`+ // container that pins the volume until restart.+ helperGroup singleflight.Group+}++const trivyCacheVolumePruneFilterValue = libarcane.InternalResourceLabel + "=true"++func NewVolumeService(db *database.DB, dockerService *docker.DockerClientService, eventService *event.EventService, settingsService *settings.SettingsService, imageService *image.ImageService, backupVolumeName string) *VolumeService {+ slog.Debug("volume service: new")+ if strings.TrimSpace(backupVolumeName) == "" {+ backupVolumeName = "arcane-backups"+ }+ return &VolumeService{+ db: db,+ dockerService: dockerService,+ eventService: eventService,+ settingsService: settingsService,+ imageService: imageService,+ backupVolumeName: backupVolumeName,+ helperByVolume: make(map[string]*volumeHelper),+ }+}++func (s *VolumeService) GetVolumeByName(ctx context.Context, name string) (*volumetypes.Volume, error) {+ slog.DebugContext(ctx, "volume service: get volume", "volume", name)+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to connect to Docker")+ }++ volResult, err := dockerClient.VolumeInspect(ctx, name, client.VolumeInspectOptions{})+ if err != nil {+ return nil, errors.WrapIf(err, "volume not found")+ }+ vol := volResult.Volume++ settings := s.settingsService.GetSettingsConfig()+ usageCtx, usageCancel := context.WithTimeout(ctx, timeouts.GetDuration(settings.DockerAPITimeout.AsInt(), timeouts.DefaultDockerAPI))+ defer usageCancel()+ if usageVolumes, ok := dockerutil.GetVolumeUsageDataStaleWhileRevalidate(usageCtx, dockerClient).Get(); ok {+ for _, uv := range usageVolumes {+ if uv.Name == vol.Name && uv.UsageData != nil {+ vol.UsageData = uv.UsageData+ slog.DebugContext(ctx, "attached volume usage data", "volume", vol.Name, "size_bytes", uv.UsageData.Size, "ref_count", uv.UsageData.RefCount)+ break+ }+ }+ }++ v := volumetypes.NewSummary(vol)++ containerIDs, err := dockerutil.GetContainersUsingVolume(ctx, dockerClient, name)+ if err != nil {+ slog.WarnContext(ctx, "failed to get containers using volume", "volume", name, "error", err.Error())+ } else {+ v.Containers = containerIDs+ if len(containerIDs) > 0 {+ v.InUse = true+ }+ }++ return &v, nil+}++func (s *VolumeService) CreateVolume(ctx context.Context, options client.VolumeCreateOptions, user models.User) (*volumetypes.Volume, error) {+ slog.DebugContext(ctx, "volume service: create volume", "volume", options.Name, "driver", options.Driver, "user", user.ID)+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ s.eventService.LogErrorEvent(ctx, models.EventTypeVolumeError, "volume", "", options.Name, user.ID, user.Username, "0", err, models.JSON{"action": "create", "driver": options.Driver})+ return nil, errors.WrapIf(err, "failed to connect to Docker")+ }++ created, err := dockerClient.VolumeCreate(ctx, options)+ if err != nil {+ s.eventService.LogErrorEvent(ctx, models.EventTypeVolumeError, "volume", "", options.Name, user.ID, user.Username, "0", err, models.JSON{"action": "create", "driver": options.Driver})+ return nil, errors.WrapIf(err, "failed to create volume")+ }++ vol, err := dockerClient.VolumeInspect(ctx, created.Volume.Name, client.VolumeInspectOptions{})+ if err != nil {+ s.eventService.LogErrorEvent(ctx, models.EventTypeVolumeError, "volume", created.Volume.Name, created.Volume.Name, user.ID, user.Username, "0", err, models.JSON{"action": "create", "driver": options.Driver, "step": "inspect"})+ return nil, errors.WrapIf(err, "failed to inspect created volume")+ }++ metadata := models.JSON{+ "action": "create",+ "driver": vol.Volume.Driver,+ "name": vol.Volume.Name,+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeCreate, vol.Volume.Name, vol.Volume.Name, user.ID, user.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume creation action", "volume", vol.Volume.Name, "error", logErr.Error())+ }++ dockerutil.InvalidateVolumeUsageCache(dockerClient)++ return new(volumetypes.NewSummary(vol.Volume)), nil+}++func (s *VolumeService) DeleteVolume(ctx context.Context, name string, force bool, user models.User) error {+ slog.DebugContext(ctx, "volume service: delete volume", "volume", name, "force", force, "user", user.ID)+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ s.eventService.LogErrorEvent(ctx, models.EventTypeVolumeError, "volume", name, name, user.ID, user.Username, "0", err, models.JSON{"action": "delete", "force": force})+ return errors.WrapIf(err, "failed to connect to Docker")+ }++ // Stop any read-only browse helper first; a helper mounting the volume would+ // otherwise block a non-forced VolumeRemove with "volume is in use".+ if stopErr := s.StopHelper(ctx, name); stopErr != nil {+ slog.WarnContext(ctx, "could not stop volume browse helper before delete", "volume", name, "error", stopErr.Error())+ }++ if _, err := dockerClient.VolumeRemove(ctx, name, client.VolumeRemoveOptions{+ Force: force,+ }); err != nil {+ s.eventService.LogErrorEvent(ctx, models.EventTypeVolumeError, "volume", name, name, user.ID, user.Username, "0", err, models.JSON{"action": "delete", "force": force})+ return errors.WrapIf(err, "failed to remove volume")+ }++ metadata := models.JSON{+ "action": "delete",+ "name": name,+ }+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeDelete, name, name, user.ID, user.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume deletion action", "volume", name, "error", logErr.Error())+ }++ s.removeHelperEntry(name)+ dockerutil.InvalidateVolumeUsageCache(dockerClient)+ return nil+}++func (s *VolumeService) PruneVolumes(ctx context.Context) (*volumetypes.PruneReport, error) {+ slog.DebugContext(ctx, "volume service: prune volumes")+ return s.PruneVolumesWithOptions(ctx, false)+}++func (s *VolumeService) PruneVolumesWithOptions(ctx context.Context, all bool) (*volumetypes.PruneReport, error) {+ slog.DebugContext(ctx, "volume service: prune volumes with options", "all", all)+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to connect to Docker")+ }++ // Stop all read-only browse helpers first; a helper mounting a volume marks it+ // "in use" and would prevent VolumePrune from reclaiming an otherwise-unused+ // volume. Helpers are re-created on demand on the next browse request.+ s.CleanupHelperContainers(ctx)++ preserveTrivyCache := s.preserveTrivyCacheOnVolumePruneInternal()++ // Docker's VolumesPrune behavior (API v1.42+):+ // - Without 'all' flag: Only removes anonymous (unnamed) volumes that are not in use+ // - With 'all=true' flag: Removes ALL unused volumes (both named and anonymous)+ // Note: Volumes are considered "in use" if referenced by any container (running or stopped)+ volumePruneOptions := buildVolumePruneOptionsInternal(all, preserveTrivyCache)+ volumePruneResult, err := dockerClient.VolumePrune(ctx, volumePruneOptions)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to prune volumes")+ }++ metadata := buildVolumePruneMetadataInternal(all, len(volumePruneResult.Report.VolumesDeleted), volumePruneResult.Report.SpaceReclaimed, preserveTrivyCache)+ if logErr := s.eventService.LogVolumeEvent(ctx, models.EventTypeVolumeDelete, "", "bulk_prune", models.SystemUser.ID, models.SystemUser.Username, "0", metadata); logErr != nil {+ slog.WarnContext(ctx, "could not log volume prune action", "error", logErr.Error())+ }++ for _, volumeName := range volumePruneResult.Report.VolumesDeleted {+ s.removeHelperEntry(volumeName)+ }++ dockerutil.InvalidateVolumeUsageCache(dockerClient)++ return &volumetypes.PruneReport{+ VolumesDeleted: volumePruneResult.Report.VolumesDeleted,+ SpaceReclaimed: volumePruneResult.Report.SpaceReclaimed,+ }, nil+}++func (s *VolumeService) preserveTrivyCacheOnVolumePruneInternal() bool {+ if s.settingsService == nil {+ return true+ }++ return s.settingsService.GetSettingsConfig().TrivyPreserveCacheOnVolumePrune.IsTrue()+}++func buildVolumePruneOptionsInternal(all, preserveTrivyCache bool) client.VolumePruneOptions {+ options := client.VolumePruneOptions{+ All: all,+ }+ if !preserveTrivyCache {+ return options+ }++ filters := make(client.Filters)+ filters = filters.Add("label!", trivyCacheVolumePruneFilterValue)+ options.Filters = filters++ return options+}++func buildVolumePruneMetadataInternal(all bool, volumesDeleted int, spaceReclaimed uint64, preserveTrivyCache bool) models.JSON {+ return models.JSON{+ "action": "prune",+ "all": all,+ "volumesDeleted": volumesDeleted,+ "spaceReclaimed": spaceReclaimed,+ "preserveTrivyCache": preserveTrivyCache,+ "trivyCacheFilterLabel": trivyCacheVolumePruneFilterValue,+ }+}diff --git a/backend/internal/services/volume_service_test.go b/backend/internal/volume/service_test.gosimilarity index 83%rename from backend/internal/services/volume_service_test.gorename to backend/internal/volume/service_test.goindex a63caac58..7dc6a23c3 100644--- a/backend/internal/services/volume_service_test.go+++ b/backend/internal/volume/service_test.go@@ -1,12 +1,10 @@-package services+package volumeimport ("context""testing""time"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"- "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/volumehelper"volumetypes "github.com/getarcaneapp/arcane/types/v2/volume""github.com/moby/moby/api/types/container""github.com/moby/moby/api/types/mount"@@ -14,99 +12,7 @@ import ("github.com/stretchr/testify/require")-func TestIsLegacyVolumeHelperContainerInternal(t *testing.T) {- tests := []struct {- name string- summary container.Summary- want bool- }{- {- name: "legacy helper signature matches",- summary: container.Summary{- Labels: map[string]string{- libarcane.InternalResourceLabel: "true",- },- Command: "sleep infinity",- Mounts: []container.MountPoint{- {Destination: "/volume"},- },- },- want: true,- },- {- name: "internal trivy-like helper is not treated as legacy volume helper",- summary: container.Summary{- Labels: map[string]string{- libarcane.InternalResourceLabel: "true",- },- Command: "trivy image --quiet alpine:latest",- Mounts: []container.MountPoint{- {Destination: "/var/run/docker.sock"},- },- },- want: false,- },- }-- for _, tt := range tests {- t.Run(tt.name, func(t *testing.T) {- require.Equal(t, tt.want, isLegacyVolumeHelperContainerInternal(tt.summary))- })- }-}--func TestIsVolumeHelperContainerInternal_UsesExplicitHelperLabel(t *testing.T) {- tests := []struct {- name string- summary container.Summary- want bool- }{- {- name: "new helper label matches",- summary: container.Summary{- Labels: map[string]string{- libarcane.InternalResourceLabel: "true",- volumehelper.ContainerLabel: "true",- },- },- want: true,- },- {- name: "generic internal volume mount does not match",- summary: container.Summary{- Labels: map[string]string{- libarcane.InternalResourceLabel: "true",- },- Mounts: []container.MountPoint{- {Destination: "/volume"},- },- },- want: false,- },- {- name: "legacy helper still matches",- summary: container.Summary{- Labels: map[string]string{- libarcane.InternalResourceLabel: "true",- },- Command: "sleep infinity",- Mounts: []container.MountPoint{- {Destination: "/volume"},- },- },- want: true,- },- }-- for _, tt := range tests {- t.Run(tt.name, func(t *testing.T) {- require.Equal(t, tt.want, isVolumeHelperContainerInternal(tt.summary))- })- }-}-func TestEnrichVolumesWithUsageDataInternal(t *testing.T) {- svc := &VolumeService{}tests := []struct {name string@@ -153,7 +59,7 @@ func TestEnrichVolumesWithUsageDataInternal(t *testing.T) {for _, tt := range tests {t.Run(tt.name, func(t *testing.T) {- got := svc.enrichVolumesWithUsageDataInternal(tt.volumes, tt.usageVolumes)+ got := enrichVolumesWithUsageDataInternal(tt.volumes, tt.usageVolumes)require.Len(t, got, tt.wantLen)tt.assertions(t, got)})diff --git a/backend/api/handlers/vulnerabilities.go b/backend/internal/vulnerability/handler.gosimilarity index 88%rename from backend/api/handlers/vulnerabilities.gorename to backend/internal/vulnerability/handler.goindex e3ed72bd6..0dd850abb 100644--- a/backend/api/handlers/vulnerabilities.go+++ b/backend/internal/vulnerability/handler.go@@ -1,4 +1,4 @@-package handlers+package vulnerabilityimport ("context"@@ -6,17 +6,17 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/vulnerability")// VulnerabilityHandler provides Huma-based vulnerability scanning endpoints.type VulnerabilityHandler struct {- vulnerabilityService *services.VulnerabilityService+ vulnerabilityService *VulnerabilityServiceappCtx context.Context}@@ -124,136 +124,136 @@ type GetScannerStatusOutput struct {}// RegisterVulnerability registers vulnerability scanning routes using Huma.-func RegisterVulnerability(api huma.API, vulnerabilityService *services.VulnerabilityService, appCtx ActivityAppContext) {+func RegisterVulnerability(api huma.API, vulnerabilityService *VulnerabilityService, appCtx handlerutil.ActivityAppContext) {h := &VulnerabilityHandler{vulnerabilityService: vulnerabilityService,- appCtx: appCtx.contextInternal(),+ appCtx: appCtx.Context(),}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "scan-image-vulnerabilities",Method: http.MethodPost,Path: "/environments/{id}/images/{imageId}/vulnerabilities/scan",Summary: "Scan image for vulnerabilities",Description: "Initiates a vulnerability scan for the specified image using Trivy",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsScan, h.ScanImage)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-image-vulnerabilities",Method: http.MethodGet,Path: "/environments/{id}/images/{imageId}/vulnerabilities",Summary: "Get vulnerability scan result",Description: "Retrieves the most recent vulnerability scan result for an image",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsRead, h.GetScanResult)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-image-vulnerability-summary",Method: http.MethodGet,Path: "/environments/{id}/images/{imageId}/vulnerabilities/summary",Summary: "Get vulnerability scan summary",Description: "Retrieves just the summary of vulnerabilities for an image (for list views)",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsRead, h.GetScanSummary)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-image-vulnerability-summaries",Method: http.MethodPost,Path: "/environments/{id}/images/vulnerabilities/summaries",Summary: "Get vulnerability scan summaries",Description: "Retrieves scan summaries for a list of images (batch)",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsRead, h.GetScanSummaries)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-image-vulnerabilities",Method: http.MethodGet,Path: "/environments/{id}/images/{imageId}/vulnerabilities/list",Summary: "List image vulnerabilities",Description: "Retrieves paginated vulnerabilities for an image",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsRead, h.ListImageVulnerabilities)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-vulnerability-scanner-status",Method: http.MethodGet,Path: "/environments/{id}/vulnerabilities/scanner-status",Summary: "Get vulnerability scanner status",Description: "Check if the vulnerability scanner (Trivy) is available and get its version",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsRead, h.GetScannerStatus)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "get-environment-vulnerability-summary",Method: http.MethodGet,Path: "/environments/{id}/vulnerabilities/summary",Summary: "Get environment vulnerability summary",Description: "Retrieves aggregated vulnerability counts across all images in the environment",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsRead, h.GetEnvironmentSummary)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-environment-vulnerabilities",Method: http.MethodGet,Path: "/environments/{id}/vulnerabilities/all",Summary: "List environment vulnerabilities",Description: "Retrieves paginated vulnerabilities across all scanned images in the environment",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsRead, h.ListAllVulnerabilities)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-environment-vulnerability-image-options",Method: http.MethodGet,Path: "/environments/{id}/vulnerabilities/image-options",Summary: "List vulnerability image options",Description: "Retrieves available image filter options for environment vulnerabilities",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsRead, h.ListAllVulnerabilityImageOptions)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "ignore-vulnerability",Method: http.MethodPost,Path: "/environments/{id}/vulnerabilities/ignore",Summary: "Ignore a vulnerability",Description: "Creates an ignore record for a specific vulnerability",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsManage, h.IgnoreVulnerability)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "unignore-vulnerability",Method: http.MethodDelete,Path: "/environments/{id}/vulnerabilities/ignore/{ignoreId}",Summary: "Unignore a vulnerability",Description: "Removes an ignore record for a vulnerability",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsManage, h.UnignoreVulnerability)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-ignored-vulnerabilities",Method: http.MethodGet,Path: "/environments/{id}/vulnerabilities/ignored",Summary: "List ignored vulnerabilities",Description: "Retrieves a list of all ignored vulnerabilities for the environment",Tags: []string{"Vulnerabilities"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermVulnsRead, h.ListIgnoredVulnerabilities)}// ScanImage initiates a vulnerability scan for an image.func (h *VulnerabilityHandler) ScanImage(ctx context.Context, input *ScanImageInput) (*ScanImageOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -341,7 +341,7 @@ func (h *VulnerabilityHandler) GetScanSummary(ctx context.Context, input *GetSca// ListImageVulnerabilities returns a paginated list of vulnerabilities for an image.func (h *VulnerabilityHandler) ListImageVulnerabilities(ctx context.Context, input *ListImageVulnerabilitiesInput) (*ListImageVulnerabilitiesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if params.Limit == 0 {params.Limit = 20}@@ -362,7 +362,7 @@ func (h *VulnerabilityHandler) ListImageVulnerabilities(ctx context.Context, inpBody: base.Paginated[vulnerability.Vulnerability]{Success: true,Data: items,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -388,7 +388,7 @@ func (h *VulnerabilityHandler) GetEnvironmentSummary(ctx context.Context, input// ListAllVulnerabilities returns a paginated list of vulnerabilities across all images.func (h *VulnerabilityHandler) ListAllVulnerabilities(ctx context.Context, input *ListAllVulnerabilitiesInput) (*ListAllVulnerabilitiesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if params.Limit == 0 {params.Limit = 20}@@ -412,7 +412,7 @@ func (h *VulnerabilityHandler) ListAllVulnerabilities(ctx context.Context, inputBody: base.Paginated[vulnerability.VulnerabilityWithImage]{Success: true,Data: items,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}@@ -463,7 +463,7 @@ type IgnoreVulnerabilityOutput struct {// IgnoreVulnerability creates an ignore record for a vulnerability.func (h *VulnerabilityHandler) IgnoreVulnerability(ctx context.Context, input *IgnoreVulnerabilityInput) (*IgnoreVulnerabilityOutput, error) {- user, err := requireUserInternal(ctx)+ user, err := handlerutil.RequireUser(ctx)if err != nil {return nil, err}@@ -537,7 +537,7 @@ type ListIgnoredVulnerabilitiesOutput struct {// ListIgnoredVulnerabilities returns a list of ignored vulnerabilities.func (h *VulnerabilityHandler) ListIgnoredVulnerabilities(ctx context.Context, input *ListIgnoredVulnerabilitiesInput) (*ListIgnoredVulnerabilitiesOutput, error) {- params := buildPaginationParamsInternal(input.Start, input.Limit, input.Sort, input.Order, input.Search)+ params := handlerutil.PaginationParams(input.Start, input.Limit, input.Sort, input.Order, input.Search)if params.Limit == 0 {params.Limit = 20}@@ -555,7 +555,7 @@ func (h *VulnerabilityHandler) ListIgnoredVulnerabilities(ctx context.Context, iBody: base.Paginated[vulnerability.IgnoredVulnerability]{Success: true,Data: items,- Pagination: toPaginationResponseInternal(paginationResp),+ Pagination: handlerutil.PaginationResponse(paginationResp),},}, nil}diff --git a/backend/internal/vulnerability/module.go b/backend/internal/vulnerability/module.gonew file mode 100644index 000000000..f1bf3cde2--- /dev/null+++ b/backend/internal/vulnerability/module.go@@ -0,0 +1,31 @@+// Package vulnerability owns vulnerability scanning, persistence, and HTTP routes.+package vulnerability++import (+ "github.com/danielgtaylor/huma/v2"++ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil"+)++type Module struct {+ service *VulnerabilityService+}++func New(service *VulnerabilityService) *Module {+ return &Module{service: service}+}++func (m *Module) Service() *VulnerabilityService {+ if m == nil {+ return nil+ }+ return m.service+}++func (m *Module) RegisterRoutes(api huma.API, appCtx handlerutil.ActivityAppContext) {+ if m == nil {+ RegisterVulnerability(api, nil, appCtx)+ return+ }+ RegisterVulnerability(api, m.service, appCtx)+}diff --git a/backend/internal/vulnerability/service_test.go b/backend/internal/vulnerability/service_test.gonew file mode 100644index 000000000..de0e6aafb--- /dev/null+++ b/backend/internal/vulnerability/service_test.go@@ -0,0 +1,330 @@+package vulnerability++import (+ "context"+ "encoding/json"+ "fmt"+ "testing"+ "time"++ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/vuln"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ "github.com/getarcaneapp/arcane/types/v2/vulnerability"+ "github.com/libtnb/sqlite"+ "github.com/stretchr/testify/require"+ "gorm.io/gorm"+)++func TestBuildTrivyContainerResourcesInternal_Defaults(t *testing.T) {+ resources, applyLimits := buildTrivyContainerResourcesInternal(nil).Get()++ require.True(t, applyLimits)+ require.Equal(t, int64(1_000_000_000), resources.NanoCPUs)+ require.Equal(t, int64(0), resources.Memory)+ require.Equal(t, int64(0), resources.MemorySwap)+}++func TestBuildTrivyContainerResourcesInternal_LimitsDisabled(t *testing.T) {+ cfg := &models.Settings{+ TrivyResourceLimitsEnabled: models.SettingVariable{Value: "false"},+ TrivyCpuLimit: models.SettingVariable{Value: "2"},+ TrivyMemoryLimitMb: models.SettingVariable{Value: "2048"},+ }++ resources, applyLimits := buildTrivyContainerResourcesInternal(cfg).Get()++ require.False(t, applyLimits)+ require.Equal(t, int64(0), resources.NanoCPUs)+ require.Equal(t, int64(0), resources.Memory)+ require.Equal(t, int64(0), resources.MemorySwap)+}++func TestBuildTrivyContainerResourcesInternal_CustomLimits(t *testing.T) {+ cfg := &models.Settings{+ TrivyResourceLimitsEnabled: models.SettingVariable{Value: "true"},+ TrivyCpuLimit: models.SettingVariable{Value: "2.5"},+ TrivyMemoryLimitMb: models.SettingVariable{Value: "2048"},+ }++ resources, applyLimits := buildTrivyContainerResourcesInternal(cfg).Get()++ require.True(t, applyLimits)+ require.Equal(t, int64(2_500_000_000), resources.NanoCPUs)+ require.Equal(t, int64(2_147_483_648), resources.Memory)+ require.Equal(t, int64(2_147_483_648), resources.MemorySwap)+}++func TestBuildTrivyCommandArgs_WithCacheDir(t *testing.T) {+ svc := &VulnerabilityService{}+ outputPath := "/tmp/trivy-result-test.json"++ cmdArgs, tempFiles := svc.buildTrivyCommandArgs(context.Background(), "alpine:3.20", "", "", "/tmp/trivy-cache", outputPath)++ require.Empty(t, tempFiles)+ require.Contains(t, cmdArgs, "--cache-dir")+ require.Contains(t, cmdArgs, "/tmp/trivy-cache")+ require.Contains(t, cmdArgs, "--output")+ require.Contains(t, cmdArgs, outputPath)+ require.Equal(t, "alpine:3.20", cmdArgs[len(cmdArgs)-1])+}++func TestBuildTrivyCommandArgs_WithoutCacheDir(t *testing.T) {+ svc := &VulnerabilityService{}+ outputPath := "/tmp/trivy-result-test.json"++ cmdArgs, tempFiles := svc.buildTrivyCommandArgs(context.Background(), "alpine:3.20", "", "", "", outputPath)++ require.Empty(t, tempFiles)+ require.NotContains(t, cmdArgs, "--cache-dir")+ require.Contains(t, cmdArgs, "--output")+ require.Contains(t, cmdArgs, outputPath)+ require.Equal(t, "alpine:3.20", cmdArgs[len(cmdArgs)-1])+}++func TestBuildTrivyCommandArgs_IncludesDefaultRepositories(t *testing.T) {+ svc := &VulnerabilityService{}+ cmdArgs, _ := svc.buildTrivyCommandArgs(context.Background(), "alpine:3.20", "", "", "", "/tmp/out.json")++ require.Contains(t, cmdArgs, "--db-repository")+ require.Contains(t, cmdArgs, vuln.DefaultDBRepository)+ require.Contains(t, cmdArgs, "--java-db-repository")+ require.Contains(t, cmdArgs, vuln.DefaultJavaDBRepository)+ require.Contains(t, cmdArgs, "--checks-bundle-repository")+ require.Contains(t, cmdArgs, vuln.DefaultChecksBundleRepository)+}++func TestGetTrivySingleScanCacheDirForSlotInternal(t *testing.T) {+ svc := &VulnerabilityService{}++ require.Equal(t, "/root/.cache/arcane-scan-slots/slot-0", svc.getTrivySingleScanCacheDirForSlotInternal(0))+ require.Equal(t, "/root/.cache/arcane-scan-slots/slot-2", svc.getTrivySingleScanCacheDirForSlotInternal(2))+ // Negative slot IDs should normalize to slot-0+ require.Equal(t, "/root/.cache/arcane-scan-slots/slot-0", svc.getTrivySingleScanCacheDirForSlotInternal(-1))+}++func TestGetTrivyScanSlotChannelInternal_PrefillsSlots(t *testing.T) {+ svc := &VulnerabilityService{}++ slotCh := svc.getTrivyScanSlotChannelInternal(3)+ require.NotNil(t, slotCh)+ require.Equal(t, 3, cap(slotCh))+ require.Len(t, slotCh, 3)++ seen := map[int]bool{}+ for range 3 {+ slotID := <-slotCh+ seen[slotID] = true+ }++ require.True(t, seen[0])+ require.True(t, seen[1])+ require.True(t, seen[2])+ require.Empty(t, slotCh)+}++func setupVulnerabilityScanTestDB(t *testing.T) *database.DB {+ t.Helper()+ dsn := fmt.Sprintf("file:vulnerability-scan-test-%d?mode=memory&cache=shared", time.Now().UnixNano())+ db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})+ require.NoError(t, err)+ require.NoError(t, db.AutoMigrate(&models.VulnerabilityScanRecord{}, &models.VulnerabilityIgnore{}))+ return &database.DB{DB: db}+}++func TestVulnerabilityService_DeleteScanResultsByImageIDs(t *testing.T) {+ ctx := context.Background()+ db := setupVulnerabilityScanTestDB(t)+ svc := &VulnerabilityService{db: db}++ records := []models.VulnerabilityScanRecord{+ {ID: "sha256:img1", ImageName: "nginx:latest", Status: models.ScanStatusCompleted, ScanTime: time.Now()},+ {ID: "sha256:img2", ImageName: "redis:7", Status: models.ScanStatusCompleted, ScanTime: time.Now()},+ {ID: "sha256:img3", ImageName: "postgres:16", Status: models.ScanStatusCompleted, ScanTime: time.Now()},+ }+ for i := range records {+ require.NoError(t, db.Create(&records[i]).Error)+ }++ require.NoError(t, svc.DeleteScanResultsByImageIDs(ctx, []string{"sha256:img1", "sha256:img3"}))++ var remaining []models.VulnerabilityScanRecord+ require.NoError(t, db.Find(&remaining).Error)+ require.Len(t, remaining, 1)+ require.Equal(t, "sha256:img2", remaining[0].ID)+}++func TestVulnerabilityService_DeleteScanResultsByImageIDs_EmptyInput(t *testing.T) {+ ctx := context.Background()+ db := setupVulnerabilityScanTestDB(t)+ svc := &VulnerabilityService{db: db}++ require.NoError(t, svc.DeleteScanResultsByImageIDs(ctx, nil))+ require.NoError(t, svc.DeleteScanResultsByImageIDs(ctx, []string{}))+}++func TestSeverityCountColumnsForFilterInternal(t *testing.T) {+ tests := []struct {+ name string+ value string+ want []string+ wantNil bool+ }{+ {+ name: "single severity",+ value: "critical",+ want: []string{"critical_count"},+ },+ {+ name: "multiple severities and duplicates",+ value: "high,critical,high,UNKNOWN",+ want: []string{"high_count", "critical_count", "unknown_count"},+ },+ {+ name: "invalid severities only",+ value: "foo,bar",+ want: []string{},+ },+ {+ name: "empty value",+ value: " , , ",+ wantNil: true,+ },+ }++ for _, tt := range tests {+ t.Run(tt.name, func(t *testing.T) {+ got := severityCountColumnsForFilterInternal(tt.value)+ if tt.wantNil {+ require.Nil(t, got)+ return+ }+ require.Equal(t, tt.want, got)+ })+ }+}++func TestApplyListAllVulnerabilityRecordPrefiltersInternal_NoPossibleMatches(t *testing.T) {+ db := setupVulnerabilityScanTestDB(t)+ base := db.WithContext(context.Background()).Model(&models.VulnerabilityScanRecord{})++ _, ok := applyListAllVulnerabilityRecordPrefiltersInternal(base, map[string]string{"severity": "not-a-severity"}).Get()+ require.False(t, ok)++ _, ok = applyListAllVulnerabilityRecordPrefiltersInternal(base, map[string]string{"imageName": " , "}).Get()+ require.False(t, ok)++ _, ok = applyListAllVulnerabilityRecordPrefiltersInternal(base, map[string]string{+ "severity": "high,bad",+ "imageName": "nginx",+ }).Get()+ require.True(t, ok)+}++func TestEstimatedVulnerabilityRecordCountInternal(t *testing.T) {+ records := []models.VulnerabilityScanRecord{+ {TotalCount: 5},+ {TotalCount: 0},+ {TotalCount: -3},+ {TotalCount: 12},+ }++ require.Equal(t, 17, estimatedVulnerabilityRecordCountInternal(records))+}++func TestVulnerabilityService_ListAllVulnerabilities_FiltersIgnoredInline(t *testing.T) {+ ctx := context.Background()+ db := setupVulnerabilityScanTestDB(t)+ svc := &VulnerabilityService{db: db}++ vulns := []vulnerability.Vulnerability{+ {+ VulnerabilityID: "CVE-2026-0001",+ PkgName: "openssl",+ InstalledVersion: "1.0.0",+ Severity: vulnerability.SeverityHigh,+ },+ }+ rawVulns, err := json.Marshal(vulns)+ require.NoError(t, err)++ scan := models.VulnerabilityScanRecord{+ ID: "sha256:list-all-1",+ ImageName: "nginx:latest",+ Status: models.ScanStatusCompleted,+ ScanTime: time.Now(),+ HighCount: 1,+ TotalCount: 1,+ Vulnerabilities: models.StringSlice{string(rawVulns)},+ }+ require.NoError(t, db.Create(&scan).Error)++ ignore := models.VulnerabilityIgnore{+ EnvironmentID: "env-1",+ ImageID: scan.ID,+ VulnerabilityID: "CVE-2026-0001",+ PkgName: "openssl",+ InstalledVersion: "1.0.0",+ Reason: new("accepted risk"),+ CreatedBy: "tester",+ }+ require.NoError(t, db.Create(&ignore).Error)++ items, page, err := svc.ListAllVulnerabilities(ctx, "env-1", pagination.QueryParams{+ Params: pagination.Params{Start: 0, Limit: 20},+ })+ require.NoError(t, err)+ require.Empty(t, items)+ require.Equal(t, int64(0), page.TotalItems)+ require.Equal(t, int64(0), page.GrandTotalItems)+}++func TestVulnerabilityService_ListAllVulnerabilityImageOptions_SeverityFilter(t *testing.T) {+ ctx := context.Background()+ db := setupVulnerabilityScanTestDB(t)+ svc := &VulnerabilityService{db: db}++ now := time.Now()+ records := []models.VulnerabilityScanRecord{+ {ID: "sha256:critical-a", ImageName: "zeta:latest", Status: models.ScanStatusCompleted, ScanTime: now, CriticalCount: 2, TotalCount: 2},+ {ID: "sha256:high-a", ImageName: "alpha:latest", Status: models.ScanStatusCompleted, ScanTime: now, HighCount: 4, TotalCount: 4},+ {ID: "sha256:critical-b", ImageName: "", Status: models.ScanStatusCompleted, ScanTime: now, CriticalCount: 1, TotalCount: 1},+ {ID: "sha256:critical-c", ImageName: "zeta:latest", Status: models.ScanStatusCompleted, ScanTime: now, CriticalCount: 1, TotalCount: 1},+ {ID: "sha256:failed", ImageName: "failed:latest", Status: models.ScanStatusFailed, ScanTime: now, CriticalCount: 9, TotalCount: 9},+ {ID: "sha256:zero", ImageName: "zero:latest", Status: models.ScanStatusCompleted, ScanTime: now, CriticalCount: 1, TotalCount: 0},+ }++ for i := range records {+ require.NoError(t, db.Create(&records[i]).Error)+ }++ items, err := svc.ListAllVulnerabilityImageOptions(ctx, "critical")+ require.NoError(t, err)+ require.Equal(t, []string{"sha256:critical-b", "zeta:latest"}, items)+}++func TestVulnerabilityService_ListAllVulnerabilityImageOptions_InvalidSeverityAndNoFilter(t *testing.T) {+ ctx := context.Background()+ db := setupVulnerabilityScanTestDB(t)+ svc := &VulnerabilityService{db: db}++ now := time.Now()+ records := []models.VulnerabilityScanRecord{+ {ID: "sha256:one", ImageName: "beta:latest", Status: models.ScanStatusCompleted, ScanTime: now, HighCount: 1, TotalCount: 1},+ {ID: "sha256:two", ImageName: "", Status: models.ScanStatusCompleted, ScanTime: now, MediumCount: 2, TotalCount: 2},+ {ID: "sha256:three", ImageName: "alpha:latest", Status: models.ScanStatusCompleted, ScanTime: now, LowCount: 1, TotalCount: 1},+ }++ for i := range records {+ require.NoError(t, db.Create(&records[i]).Error)+ }++ invalid, err := svc.ListAllVulnerabilityImageOptions(ctx, "not-a-severity")+ require.NoError(t, err)+ require.Empty(t, invalid)++ all, err := svc.ListAllVulnerabilityImageOptions(ctx, "")+ require.NoError(t, err)+ require.Equal(t, []string{"alpha:latest", "beta:latest", "sha256:two"}, all)+}diff --git a/backend/internal/vulnerability/vulnerability.go b/backend/internal/vulnerability/vulnerability.gonew file mode 100644index 000000000..10619e8e9--- /dev/null+++ b/backend/internal/vulnerability/vulnerability.go@@ -0,0 +1,62 @@+package vulnerability++import (+ "sync"+ "time"++ "github.com/getarcaneapp/arcane/backend/v2/internal/registry"++ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/vuln"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+)++const (+ trivyCacheVolumeName = "arcane-trivy-cache"+ trivyCacheSlotRootDir = vuln.CacheMountTarget + "/arcane-scan-slots"+ scanStaleTimeout = 30 * time.Minute+ defaultTrivyCPULimitCores = 1.0+ defaultTrivyMemoryLimitMB int64 = 0+ defaultScanSaveRetryAttempts = 3+ defaultScanSaveRetryDelay = 200 * time.Millisecond+ defaultScanSaveRetryMaxDelay = 2 * time.Second+)++// VulnerabilityService handles vulnerability scanning of container images+type VulnerabilityService struct {+ db *database.DB+ dockerService *docker.DockerClientService+ eventService *event.EventService+ settingsService *settings.SettingsService+ notificationService *notification.NotificationService+ activityService *activity.ActivityService+ registryService *registry.ContainerRegistryService+ // scanLocks provides per-image locking to allow concurrent scans of different+ // images while preventing duplicate scans of the same image.+ scanLocks utils.KeyedMutex+ // trivyScanSlots limits concurrent Trivy scans (manual + scheduled) and+ // provides deterministic slot IDs for shared-cache sharding.+ trivyScanSlots chan int+ trivyScanMu sync.Mutex+ // scanPhases tracks live scan phases in-memory for in-progress scans.+ scanPhases sync.Map // map[string]vulnerability.ScanPhase+}++// NewVulnerabilityService creates a new VulnerabilityService instance+func NewVulnerabilityService(db *database.DB, dockerService *docker.DockerClientService, eventService *event.EventService, settingsService *settings.SettingsService, notificationService *notification.NotificationService, activityService *activity.ActivityService, registryService *registry.ContainerRegistryService) *VulnerabilityService {+ return &VulnerabilityService{+ db: db,+ dockerService: dockerService,+ eventService: eventService,+ settingsService: settingsService,+ notificationService: notificationService,+ activityService: activityService,+ registryService: registryService,+ trivyScanSlots: nil,+ }+}diff --git a/backend/internal/vulnerability/vulnerability_ignores.go b/backend/internal/vulnerability/vulnerability_ignores.gonew file mode 100644index 000000000..3a35d5228--- /dev/null+++ b/backend/internal/vulnerability/vulnerability_ignores.go@@ -0,0 +1,255 @@+package vulnerability++import (+ "context"+ "log/slog"+ "strings"++ "emperror.dev/errors"++ "gorm.io/gorm"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ "github.com/getarcaneapp/arcane/types/v2/vulnerability"+)++func vulnerabilityIgnoreKeyInternal(imageID, vulnerabilityID, pkgName, installedVersion string) string {+ return imageID + ":" + vulnerabilityID + ":" + pkgName + ":" + installedVersion+}++func (s *VulnerabilityService) getIgnoredVulnerabilityKeySetByEnvironmentInternal(ctx context.Context, envID string) (map[string]struct{}, error) {+ if s.db == nil {+ return nil, nil+ }++ var ignores []models.VulnerabilityIgnore+ if err := s.db.WithContext(ctx).Where("environment_id = ?", envID).Find(&ignores).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to get ignore records")+ }++ if len(ignores) == 0 {+ return nil, nil+ }++ ignoredKeys := make(map[string]struct{}, len(ignores))+ for _, ignore := range ignores {+ key := vulnerabilityIgnoreKeyInternal(ignore.ImageID, ignore.VulnerabilityID, ignore.PkgName, ignore.InstalledVersion)+ ignoredKeys[key] = struct{}{}+ }++ return ignoredKeys, nil+}++func (s *VulnerabilityService) filterIgnoredVulnerabilitiesForImage(+ ctx context.Context,+ imageID string,+ vulns []vulnerability.Vulnerability,+) ([]vulnerability.Vulnerability, error) {+ if s.db == nil || len(vulns) == 0 || imageID == "" {+ return vulns, nil+ }++ var ignores []models.VulnerabilityIgnore+ if err := s.db.WithContext(ctx).Where("image_id = ?", imageID).Find(&ignores).Error; err != nil {+ return nil, err+ }+ if len(ignores) == 0 {+ return vulns, nil+ }++ ignoredKeys := make(map[string]struct{}, len(ignores))+ for _, ignore := range ignores {+ key := vulnerabilityIgnoreKeyInternal(ignore.ImageID, ignore.VulnerabilityID, ignore.PkgName, ignore.InstalledVersion)+ ignoredKeys[key] = struct{}{}+ }++ filtered := make([]vulnerability.Vulnerability, 0, len(vulns))+ for _, vulnEntry := range vulns {+ key := vulnerabilityIgnoreKeyInternal(imageID, vulnEntry.VulnerabilityID, vulnEntry.PkgName, vulnEntry.InstalledVersion)+ if _, isIgnored := ignoredKeys[key]; isIgnored {+ continue+ }+ filtered = append(filtered, vulnEntry)+ }++ return filtered, nil+}++// IgnoreVulnerability creates a new ignore record for a vulnerability+func (s *VulnerabilityService) IgnoreVulnerability(ctx context.Context, envID string, payload *vulnerability.IgnorePayload) (*models.VulnerabilityIgnore, error) {+ if s.db == nil {+ return nil, errors.New("database not available")+ }++ // Check if already ignored (composite key check)+ var existing models.VulnerabilityIgnore+ err := s.db.WithContext(ctx).Where(+ "environment_id = ? AND image_id = ? AND vulnerability_id = ? AND pkg_name = ? AND installed_version = ?",+ envID, payload.ImageID, payload.VulnerabilityID, payload.PkgName, payload.InstalledVersion,+ ).First(&existing).Error++ if err == nil {+ return nil, errors.New("vulnerability is already ignored")+ }++ if !errors.Is(err, gorm.ErrRecordNotFound) {+ return nil, errors.WrapIf(err, "failed to check existing ignore")+ }++ ignore := &models.VulnerabilityIgnore{+ EnvironmentID: envID,+ ImageID: payload.ImageID,+ VulnerabilityID: payload.VulnerabilityID,+ PkgName: payload.PkgName,+ InstalledVersion: payload.InstalledVersion,+ Reason: payload.Reason,+ CreatedBy: payload.CreatedBy,+ }++ if err := s.db.WithContext(ctx).Create(ignore).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to create ignore record")+ }++ slog.InfoContext(ctx, "vulnerability ignored",+ "vulnerability_id", payload.VulnerabilityID,+ "image_id", payload.ImageID,+ "pkg_name", payload.PkgName,+ )++ return ignore, nil+}++// UnignoreVulnerability removes an ignore record+func (s *VulnerabilityService) UnignoreVulnerability(ctx context.Context, envID string, ignoreID string) error {+ if s.db == nil {+ return errors.New("database not available")+ }++ result := s.db.WithContext(ctx).Where("id = ? AND environment_id = ?", ignoreID, envID).Delete(&models.VulnerabilityIgnore{})+ if result.Error != nil {+ return errors.WrapIf(result.Error, "failed to delete ignore record")+ }++ if result.RowsAffected == 0 {+ return errors.New("ignore record not found")+ }++ slog.InfoContext(ctx, "vulnerability unignored", "ignore_id", ignoreID)+ return nil+}++func applyIgnoredVulnerabilitiesSort(query *gorm.DB, sort string) *gorm.DB {+ if sort == "" {+ return query.Order("created_at DESC")+ }++ for part := range strings.SplitSeq(sort, ",") {+ part = strings.TrimSpace(part)+ if part == "" {+ continue+ }++ desc := false+ switch {+ case strings.HasPrefix(part, "-"):+ desc = true+ part = strings.TrimPrefix(part, "-")+ case strings.HasPrefix(part, "+"):+ part = strings.TrimPrefix(part, "+")+ }++ query = applyIgnoredVulnerabilitiesSortPart(query, part, desc)+ }++ return query+}++func applyIgnoredVulnerabilitiesSortPart(query *gorm.DB, part string, desc bool) *gorm.DB {+ direction := "ASC"+ if desc {+ direction = "DESC"+ }++ switch part {+ case "createdAt":+ return query.Order("created_at " + direction)+ case "vulnerabilityId":+ return query.Order("vulnerability_id " + direction)+ default:+ return query.Order("created_at DESC")+ }+}++func mapIgnoredVulnerabilities(ignores []models.VulnerabilityIgnore) []vulnerability.IgnoredVulnerability {+ result := make([]vulnerability.IgnoredVulnerability, len(ignores))+ for i, ignore := range ignores {+ result[i] = vulnerability.IgnoredVulnerability{+ ID: ignore.ID,+ EnvironmentID: ignore.EnvironmentID,+ ImageID: ignore.ImageID,+ VulnerabilityID: ignore.VulnerabilityID,+ PkgName: ignore.PkgName,+ InstalledVersion: ignore.InstalledVersion,+ Reason: ignore.Reason,+ CreatedBy: ignore.CreatedBy,+ CreatedAt: ignore.CreatedAt,+ }+ }+ return result+}++func buildIgnoredVulnerabilitiesResponse(total int64, params pagination.QueryParams, ignores []models.VulnerabilityIgnore) pagination.Response {+ filtered := pagination.FilterResult[models.VulnerabilityIgnore]{+ Items: ignores,+ TotalCount: total,+ TotalAvailable: total,+ }+ return pagination.BuildResponse(filtered.TotalCount, filtered.TotalAvailable, params)+}++// ListIgnoredVulnerabilities returns a list of ignored vulnerabilities for an environment+func (s *VulnerabilityService) ListIgnoredVulnerabilities(ctx context.Context, envID string, params pagination.QueryParams) ([]vulnerability.IgnoredVulnerability, pagination.Response, error) {+ if params.Limit == 0 {+ params.Limit = 20+ }++ if s.db == nil {+ return []vulnerability.IgnoredVulnerability{}, pagination.BuildResponse(0, 0, params), nil+ }++ var ignores []models.VulnerabilityIgnore+ query := s.db.WithContext(ctx).Where("environment_id = ?", envID)+ query = applyIgnoredVulnerabilitiesSort(query, params.Sort)++ // Count total+ var total int64+ if err := query.Model(&models.VulnerabilityIgnore{}).Count(&total).Error; err != nil {+ return nil, pagination.Response{}, errors.WrapIf(err, "failed to count ignored vulnerabilities")+ }++ // Apply pagination+ offset := params.Start+ if err := query.Offset(offset).Limit(params.Limit).Find(&ignores).Error; err != nil {+ return nil, pagination.Response{}, errors.WrapIf(err, "failed to list ignored vulnerabilities")+ }++ result := mapIgnoredVulnerabilities(ignores)+ response := buildIgnoredVulnerabilitiesResponse(total, params, ignores)++ return result, response, nil+}++// GetIgnoreRecordsForImage retrieves all ignore records for a specific image+func (s *VulnerabilityService) GetIgnoreRecordsForImage(ctx context.Context, envID string, imageID string) ([]models.VulnerabilityIgnore, error) {+ if s.db == nil {+ return nil, nil+ }++ var ignores []models.VulnerabilityIgnore+ if err := s.db.WithContext(ctx).Where("environment_id = ? AND image_id = ?", envID, imageID).Find(&ignores).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to get ignore records")+ }++ return ignores, nil+}diff --git a/backend/internal/vulnerability/vulnerability_query.go b/backend/internal/vulnerability/vulnerability_query.gonew file mode 100644index 000000000..75ddd1196--- /dev/null+++ b/backend/internal/vulnerability/vulnerability_query.go@@ -0,0 +1,626 @@+package vulnerability++import (+ "context"+ "encoding/json/v2"+ "log/slog"+ "math"+ "sort"+ "strings"++ "emperror.dev/errors"++ "github.com/samber/mo"+ "gorm.io/gorm"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ "github.com/getarcaneapp/arcane/types/v2/vulnerability"+)++// GetScanResult retrieves the most recent scan result for an image+func (s *VulnerabilityService) GetScanResult(ctx context.Context, imageID string) (*vulnerability.ScanResult, error) {+ if s.db == nil {+ return nil, nil+ }++ var record models.VulnerabilityScanRecord+ err := s.db.WithContext(ctx).Where("id = ?", imageID).First(&record).Error+ if err != nil {+ if errors.Is(err, gorm.ErrRecordNotFound) {+ return nil, nil+ }+ return nil, errors.WrapIf(err, "failed to get scan result")+ }++ s.markStaleScanIfNeeded(ctx, &record)++ return s.convertRecordToResult(&record)+}++// GetScanSummary retrieves just the summary for an image (for list views)+func (s *VulnerabilityService) GetScanSummary(ctx context.Context, imageID string) (*vulnerability.ScanSummary, error) {+ if s.db == nil {+ return nil, nil+ }++ var record models.VulnerabilityScanRecord+ err := s.db.WithContext(ctx).+ Select("id", "scan_time", "status", "critical_count", "high_count", "medium_count", "low_count", "unknown_count", "total_count", "error").+ Where("id = ?", imageID).+ First(&record).Error+ if err != nil {+ if errors.Is(err, gorm.ErrRecordNotFound) {+ return nil, nil+ }+ return nil, errors.WrapIf(err, "failed to get scan summary")+ }++ s.markStaleScanIfNeeded(ctx, &record)++ summary := &vulnerability.ScanSummary{+ ImageID: record.ID,+ ScanTime: record.ScanTime,+ Status: vulnerability.ScanStatus(record.Status),+ Summary: &vulnerability.SeveritySummary{+ Critical: record.CriticalCount,+ High: record.HighCount,+ Medium: record.MediumCount,+ Low: record.LowCount,+ Unknown: record.UnknownCount,+ Total: record.TotalCount,+ },+ Error: mo.PointerToOption(record.Error).OrEmpty(),+ }+ s.applyScanPhaseToSummaryInternal(summary)++ return summary, nil+}++// ListVulnerabilities returns a paginated, filtered list of vulnerabilities for an image.+func (s *VulnerabilityService) ListVulnerabilities(ctx context.Context, imageID string, params pagination.QueryParams) ([]vulnerability.Vulnerability, pagination.Response, error) {+ result, err := s.GetScanResult(ctx, imageID)+ if err != nil {+ return nil, pagination.Response{}, errors.WrapIf(err, "failed to get scan result")+ }+ if result == nil {+ if params.Limit <= 0 {+ params.Limit = 20+ }+ return []vulnerability.Vulnerability{}, pagination.BuildResponse(0, 0, params), nil+ }++ config := pagination.Config[vulnerability.Vulnerability]{+ SearchAccessors: []pagination.SearchAccessor[vulnerability.Vulnerability]{+ func(item vulnerability.Vulnerability) (string, error) { return item.VulnerabilityID, nil },+ func(item vulnerability.Vulnerability) (string, error) { return item.PkgName, nil },+ func(item vulnerability.Vulnerability) (string, error) { return item.InstalledVersion, nil },+ func(item vulnerability.Vulnerability) (string, error) { return item.FixedVersion, nil },+ func(item vulnerability.Vulnerability) (string, error) { return item.Title, nil },+ },+ SortBindings: []pagination.SortBinding[vulnerability.Vulnerability]{+ {+ Key: "vulnerabilityId",+ Fn: func(a, b vulnerability.Vulnerability) int {+ return strings.Compare(a.VulnerabilityID, b.VulnerabilityID)+ },+ },+ {+ Key: "pkgName",+ Fn: func(a, b vulnerability.Vulnerability) int { return strings.Compare(a.PkgName, b.PkgName) },+ },+ {+ Key: "severity",+ Fn: func(a, b vulnerability.Vulnerability) int {+ return severityRankInternal(a.Severity) - severityRankInternal(b.Severity)+ },+ },+ {+ Key: "installedVersion",+ Fn: func(a, b vulnerability.Vulnerability) int {+ return strings.Compare(a.InstalledVersion, b.InstalledVersion)+ },+ },+ {+ Key: "fixedVersion",+ Fn: func(a, b vulnerability.Vulnerability) int { return strings.Compare(a.FixedVersion, b.FixedVersion) },+ },+ },+ FilterAccessors: []pagination.FilterAccessor[vulnerability.Vulnerability]{+ {+ Key: "severity",+ Fn: func(item vulnerability.Vulnerability, value string) bool {+ return strings.EqualFold(string(item.Severity), value)+ },+ },+ },+ }++ filtered := pagination.SearchOrderAndPaginate(result.Vulnerabilities, params, config)+ response := pagination.BuildResponse(filtered.TotalCount, filtered.TotalAvailable, params)++ return filtered.Items, response, nil+}++// GetEnvironmentSummary returns aggregated vulnerability counts across all images.+func (s *VulnerabilityService) GetEnvironmentSummary(ctx context.Context) (*vulnerability.EnvironmentVulnerabilitySummary, error) {+ if s.db != nil && s.dockerService != nil {+ if deleted, err := s.CleanupOrphanedScanRecords(ctx); err != nil {+ slog.WarnContext(ctx, "failed to cleanup orphaned vulnerability scan records", "error", err)+ } else if deleted > 0 {+ slog.DebugContext(ctx, "cleaned up orphaned vulnerability scan records", "deleted", deleted)+ }+ }++ summary := &vulnerability.EnvironmentVulnerabilitySummary{+ Summary: &vulnerability.SeveritySummary{},+ }++ if s.dockerService != nil {+ _, _, _, total, err := s.dockerService.GetAllImages(ctx)+ if err != nil {+ slog.WarnContext(ctx, "failed to list images for vulnerability summary", "error", err)+ } else {+ summary.TotalImages = total+ }+ }++ if s.db == nil {+ return summary, nil+ }++ type aggregate struct {+ Critical int `gorm:"column:critical"`+ High int `gorm:"column:high"`+ Medium int `gorm:"column:medium"`+ Low int `gorm:"column:low"`+ Unknown int `gorm:"column:unknown"`+ Total int `gorm:"column:total"`+ Scanned int `gorm:"column:scanned"`+ }++ var agg aggregate+ err := s.db.WithContext(ctx).+ Model(&models.VulnerabilityScanRecord{}).+ Select(+ "COALESCE(SUM(critical_count), 0) AS critical",+ "COALESCE(SUM(high_count), 0) AS high",+ "COALESCE(SUM(medium_count), 0) AS medium",+ "COALESCE(SUM(low_count), 0) AS low",+ "COALESCE(SUM(unknown_count), 0) AS unknown",+ "COALESCE(SUM(total_count), 0) AS total",+ "COUNT(*) AS scanned",+ ).+ Where("status = ?", models.ScanStatusCompleted).+ Scan(&agg).Error+ if err != nil {+ return nil, errors.WrapIf(err, "failed to aggregate vulnerability summary")+ }++ summary.ScannedImages = agg.Scanned+ summary.Summary = &vulnerability.SeveritySummary{+ Critical: agg.Critical,+ High: agg.High,+ Medium: agg.Medium,+ Low: agg.Low,+ Unknown: agg.Unknown,+ Total: agg.Total,+ }++ return summary, nil+}++// ActionableCountExcludingIgnored counts critical and high severity vulnerabilities+// across all completed scans, excluding any that have been ignored. This is used by the dashboard+// to show an accurate action item count that reflects ignore decisions.+func (s *VulnerabilityService) ActionableCountExcludingIgnored(ctx context.Context) (int, error) {+ if s.db == nil {+ return 0, nil+ }++ var records []models.VulnerabilityScanRecord+ err := s.db.WithContext(ctx).+ Select("id", "vulnerabilities").+ Where("status = ?", models.ScanStatusCompleted).+ Where("critical_count + high_count > 0").+ Find(&records).Error+ if err != nil {+ return 0, errors.WrapIf(err, "failed to list vulnerability scans")+ }++ if len(records) == 0 {+ return 0, nil+ }++ // Load all ignore records (not scoped to a single environment, since an+ // Arcane instance serves one environment and the dashboard does not carry+ // an environment ID through the service layer today).+ var ignores []models.VulnerabilityIgnore+ if err := s.db.WithContext(ctx).Find(&ignores).Error; err != nil {+ return 0, errors.WrapIf(err, "failed to get ignore records")+ }++ ignoredKeys := make(map[string]struct{}, len(ignores))+ for _, ig := range ignores {+ ignoredKeys[vulnerabilityIgnoreKeyInternal(ig.ImageID, ig.VulnerabilityID, ig.PkgName, ig.InstalledVersion)] = struct{}{}+ }++ count := 0+ for _, record := range records {+ if len(record.Vulnerabilities) == 0 || record.Vulnerabilities[0] == "" {+ continue+ }++ var vulns []vulnerability.Vulnerability+ if err := json.Unmarshal([]byte(record.Vulnerabilities[0]), &vulns); err != nil {+ slog.WarnContext(ctx, "failed to unmarshal vulnerabilities", "error", err, "image_id", record.ID)+ continue+ }++ for _, vulnEntry := range vulns {+ if vulnEntry.Severity != vulnerability.SeverityCritical && vulnEntry.Severity != vulnerability.SeverityHigh {+ continue+ }+ if len(ignoredKeys) > 0 {+ key := vulnerabilityIgnoreKeyInternal(record.ID, vulnEntry.VulnerabilityID, vulnEntry.PkgName, vulnEntry.InstalledVersion)+ if _, isIgnored := ignoredKeys[key]; isIgnored {+ continue+ }+ }+ count+++ }+ }++ return count, nil+}++// ListAllVulnerabilities returns a paginated list of vulnerabilities across all scanned images.+func (s *VulnerabilityService) ListAllVulnerabilities(ctx context.Context, envID string, params pagination.QueryParams) ([]vulnerability.VulnerabilityWithImage, pagination.Response, error) {+ if params.Limit == 0 {+ params.Limit = 20+ }++ if s.db == nil {+ return []vulnerability.VulnerabilityWithImage{}, pagination.BuildResponse(0, 0, params), nil+ }++ recordsQuery := s.db.WithContext(ctx).+ Select("id", "image_name", "vulnerabilities", "total_count").+ Where("status = ?", models.ScanStatusCompleted).+ Where("total_count > 0")+ recordsQuery, hasPossibleMatches := applyListAllVulnerabilityRecordPrefiltersInternal(recordsQuery, params.Filters).Get()+ if !hasPossibleMatches {+ filtered := pagination.FilterResult[vulnerability.VulnerabilityWithImage]{+ Items: []vulnerability.VulnerabilityWithImage{},+ TotalCount: 0,+ TotalAvailable: 0,+ }+ response := pagination.BuildResponse(filtered.TotalCount, filtered.TotalAvailable, params)+ return filtered.Items, response, nil+ }++ var records []models.VulnerabilityScanRecord+ if err := recordsQuery.Find(&records).Error; err != nil {+ return nil, pagination.Response{}, errors.WrapIf(err, "failed to list vulnerability scans")+ }++ ignoredKeys, err := s.getIgnoredVulnerabilityKeySetByEnvironmentInternal(ctx, envID)+ if err != nil {+ slog.WarnContext(ctx, "failed to filter ignored vulnerabilities", "error", err)+ }++ items := make([]vulnerability.VulnerabilityWithImage, 0, estimatedVulnerabilityRecordCountInternal(records))+ for _, record := range records {+ if len(record.Vulnerabilities) == 0 || record.Vulnerabilities[0] == "" {+ continue+ }++ var vulns []vulnerability.Vulnerability+ if err := json.Unmarshal([]byte(record.Vulnerabilities[0]), &vulns); err != nil {+ slog.WarnContext(ctx, "failed to unmarshal vulnerabilities", "error", err, "image_id", record.ID)+ continue+ }++ for _, vulnEntry := range vulns {+ if len(ignoredKeys) > 0 {+ key := vulnerabilityIgnoreKeyInternal(record.ID, vulnEntry.VulnerabilityID, vulnEntry.PkgName, vulnEntry.InstalledVersion)+ if _, isIgnored := ignoredKeys[key]; isIgnored {+ continue+ }+ }++ items = append(items, vulnerability.VulnerabilityWithImage{+ Vulnerability: vulnEntry,+ ImageID: record.ID,+ ImageName: record.ImageName,+ })+ }+ }++ config := pagination.Config[vulnerability.VulnerabilityWithImage]{+ SearchAccessors: []pagination.SearchAccessor[vulnerability.VulnerabilityWithImage]{+ func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.VulnerabilityID, nil },+ func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.PkgName, nil },+ func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.InstalledVersion, nil },+ func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.FixedVersion, nil },+ func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.Title, nil },+ func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.ImageName, nil },+ func(item vulnerability.VulnerabilityWithImage) (string, error) { return item.ImageID, nil },+ },+ SortBindings: []pagination.SortBinding[vulnerability.VulnerabilityWithImage]{+ {+ Key: "vulnerabilityId",+ Fn: func(a, b vulnerability.VulnerabilityWithImage) int {+ return strings.Compare(a.VulnerabilityID, b.VulnerabilityID)+ },+ },+ {+ Key: "pkgName",+ Fn: func(a, b vulnerability.VulnerabilityWithImage) int { return strings.Compare(a.PkgName, b.PkgName) },+ },+ {+ Key: "severity",+ Fn: func(a, b vulnerability.VulnerabilityWithImage) int {+ return severityRankInternal(a.Severity) - severityRankInternal(b.Severity)+ },+ },+ {+ Key: "installedVersion",+ Fn: func(a, b vulnerability.VulnerabilityWithImage) int {+ return strings.Compare(a.InstalledVersion, b.InstalledVersion)+ },+ },+ {+ Key: "fixedVersion",+ Fn: func(a, b vulnerability.VulnerabilityWithImage) int {+ return strings.Compare(a.FixedVersion, b.FixedVersion)+ },+ },+ {+ Key: "imageName",+ Fn: func(a, b vulnerability.VulnerabilityWithImage) int { return strings.Compare(a.ImageName, b.ImageName) },+ },+ },+ FilterAccessors: []pagination.FilterAccessor[vulnerability.VulnerabilityWithImage]{+ {+ Key: "severity",+ Fn: func(item vulnerability.VulnerabilityWithImage, value string) bool {+ return strings.EqualFold(string(item.Severity), value)+ },+ },+ {+ Key: "imageName",+ Fn: func(item vulnerability.VulnerabilityWithImage, value string) bool {+ itemLower := strings.ToLower(item.ImageName)+ for part := range strings.SplitSeq(value, ",") {+ if part = strings.TrimSpace(part); part != "" && strings.Contains(itemLower, strings.ToLower(part)) {+ return true+ }+ }+ return false+ },+ },+ },+ }++ filtered := pagination.SearchOrderAndPaginate(items, params, config)+ response := pagination.BuildResponse(filtered.TotalCount, filtered.TotalAvailable, params)++ return filtered.Items, response, nil+}++// ListAllVulnerabilityImageOptions returns unique image names (or image IDs when name is empty)+// for vulnerability filtering, optionally constrained by severity.+func (s *VulnerabilityService) ListAllVulnerabilityImageOptions(ctx context.Context, severityFilter string) ([]string, error) {+ if s.db == nil {+ return []string{}, nil+ }++ query := s.db.WithContext(ctx).+ Model(&models.VulnerabilityScanRecord{}).+ Select("id", "image_name").+ Where("status = ?", models.ScanStatusCompleted).+ Where("total_count > 0")++ if strings.TrimSpace(severityFilter) != "" {+ var hasPossibleMatches bool+ query, hasPossibleMatches = applyListAllVulnerabilityRecordPrefiltersInternal(query, map[string]string{+ "severity": severityFilter,+ }).Get()+ if !hasPossibleMatches {+ return []string{}, nil+ }+ }++ var records []models.VulnerabilityScanRecord+ if err := query.Find(&records).Error; err != nil {+ return nil, errors.WrapIf(err, "failed to list vulnerability image options")+ }++ seen := make(map[string]struct{}, len(records))+ options := make([]string, 0, len(records))+ for _, record := range records {+ label := strings.TrimSpace(record.ImageName)+ if label == "" {+ label = strings.TrimSpace(record.ID)+ }+ if label == "" {+ continue+ }+ if _, exists := seen[label]; exists {+ continue+ }++ seen[label] = struct{}{}+ options = append(options, label)+ }++ sort.Strings(options)+ return options, nil+}++func applyListAllVulnerabilityRecordPrefiltersInternal(query *gorm.DB, filters map[string]string) mo.Option[*gorm.DB] {+ if query == nil || len(filters) == 0 {+ return mo.Some(query)+ }++ if severityFilter, ok := filters["severity"]; ok {+ severityColumns := severityCountColumnsForFilterInternal(severityFilter)+ if len(severityColumns) == 0 {+ return mo.None[*gorm.DB]()+ }++ parts := make([]string, 0, len(severityColumns))+ args := make([]any, 0, len(severityColumns))+ for _, col := range severityColumns {+ parts = append(parts, col+" > ?")+ args = append(args, 0)+ }+ query = query.Where("("+strings.Join(parts, " OR ")+")", args...)+ }++ if imageNameFilter, ok := filters["imageName"]; ok {+ imageNameParts := splitNonEmptyCSVInternal(imageNameFilter)+ if len(imageNameParts) == 0 {+ return mo.None[*gorm.DB]()+ }++ clauses := make([]string, 0, len(imageNameParts))+ args := make([]any, 0, len(imageNameParts))+ for _, part := range imageNameParts {+ clauses = append(clauses, "LOWER(image_name) LIKE ?")+ args = append(args, "%"+strings.ToLower(part)+"%")+ }+ query = query.Where("("+strings.Join(clauses, " OR ")+")", args...)+ }++ return mo.Some(query)+}++func severityCountColumnsForFilterInternal(value string) []string {+ parts := splitNonEmptyCSVInternal(value)+ if len(parts) == 0 {+ return nil+ }++ columns := make([]string, 0, len(parts))+ seen := make(map[string]struct{}, len(parts))+ for _, part := range parts {+ var col string+ switch {+ case strings.EqualFold(part, string(vulnerability.SeverityCritical)):+ col = "critical_count"+ case strings.EqualFold(part, string(vulnerability.SeverityHigh)):+ col = "high_count"+ case strings.EqualFold(part, string(vulnerability.SeverityMedium)):+ col = "medium_count"+ case strings.EqualFold(part, string(vulnerability.SeverityLow)):+ col = "low_count"+ case strings.EqualFold(part, string(vulnerability.SeverityUnknown)):+ col = "unknown_count"+ default:+ continue+ }++ if _, exists := seen[col]; exists {+ continue+ }++ seen[col] = struct{}{}+ columns = append(columns, col)+ }++ return columns+}++func splitNonEmptyCSVInternal(value string) []string {+ parts := make([]string, 0)+ for part := range strings.SplitSeq(value, ",") {+ part = strings.TrimSpace(part)+ if part == "" {+ continue+ }+ parts = append(parts, part)+ }+ return parts+}++func estimatedVulnerabilityRecordCountInternal(records []models.VulnerabilityScanRecord) int {+ if len(records) == 0 {+ return 0+ }++ total := 0+ for i := range records {+ if records[i].TotalCount <= 0 {+ continue+ }++ if total > math.MaxInt-records[i].TotalCount {+ return math.MaxInt+ }+ total += records[i].TotalCount+ }++ return total+}++func severityRankInternal(severity vulnerability.Severity) int {+ switch severity {+ case vulnerability.SeverityCritical:+ return 4+ case vulnerability.SeverityHigh:+ return 3+ case vulnerability.SeverityMedium:+ return 2+ case vulnerability.SeverityLow:+ return 1+ case vulnerability.SeverityUnknown:+ return 0+ default:+ return 0+ }+}++// GetScanSummariesByImageIDs retrieves scan summaries for multiple images+func (s *VulnerabilityService) GetScanSummariesByImageIDs(ctx context.Context, imageIDs []string) (map[string]*vulnerability.ScanSummary, error) {+ if s.db == nil || len(imageIDs) == 0 {+ return make(map[string]*vulnerability.ScanSummary), nil+ }++ var records []models.VulnerabilityScanRecord+ err := s.db.WithContext(ctx).+ Select("id", "scan_time", "status", "critical_count", "high_count", "medium_count", "low_count", "unknown_count", "total_count", "error").+ Where("id IN ?", imageIDs).+ Find(&records).Error+ if err != nil {+ return nil, errors.WrapIf(err, "failed to get scan summaries")+ }++ result := make(map[string]*vulnerability.ScanSummary, len(records))+ for _, record := range records {+ s.markStaleScanIfNeeded(ctx, &record)+ var errStr string+ if record.Error != nil {+ errStr = *record.Error+ }++ result[record.ID] = &vulnerability.ScanSummary{+ ImageID: record.ID,+ ScanTime: record.ScanTime,+ Status: vulnerability.ScanStatus(record.Status),+ Summary: &vulnerability.SeveritySummary{+ Critical: record.CriticalCount,+ High: record.HighCount,+ Medium: record.MediumCount,+ Low: record.LowCount,+ Unknown: record.UnknownCount,+ Total: record.TotalCount,+ },+ Error: errStr,+ }+ s.applyScanPhaseToSummaryInternal(result[record.ID])+ }++ return result, nil+}diff --git a/backend/internal/vulnerability/vulnerability_scan.go b/backend/internal/vulnerability/vulnerability_scan.gonew file mode 100644index 000000000..abc106bf1--- /dev/null+++ b/backend/internal/vulnerability/vulnerability_scan.go@@ -0,0 +1,624 @@+package vulnerability++import (+ "context"+ "fmt"+ "log/slog"+ "strings"+ "time"++ "emperror.dev/errors"++ imagetypes "github.com/moby/moby/api/types/image"+ "github.com/moby/moby/client"+ "github.com/samber/mo"++ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ activitylib "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/activity"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils"+ "github.com/getarcaneapp/arcane/types/v2/vulnerability"+)++func (s *VulnerabilityService) setScanPhaseInternal(imageID string, phase vulnerability.ScanPhase) {+ if imageID == "" || phase == "" {+ return+ }+ s.scanPhases.Store(imageID, phase)+}++func (s *VulnerabilityService) clearScanPhaseInternal(imageID string) {+ if imageID == "" {+ return+ }+ s.scanPhases.Delete(imageID)+}++func (s *VulnerabilityService) getScanPhaseInternal(imageID string) vulnerability.ScanPhase {+ if imageID == "" {+ return ""+ }++ phase, ok := s.scanPhases.Load(imageID)+ if !ok {+ return ""+ }++ if typed, ok := phase.(vulnerability.ScanPhase); ok {+ return typed+ }++ if str, ok := phase.(string); ok {+ return vulnerability.ScanPhase(str)+ }++ return ""+}++func (s *VulnerabilityService) resolveScanPhaseInternal(imageID string, status vulnerability.ScanStatus) vulnerability.ScanPhase {+ if status != vulnerability.ScanStatusPending && status != vulnerability.ScanStatusScanning {+ return ""+ }++ if phase := s.getScanPhaseInternal(imageID); phase != "" {+ return phase+ }++ if status == vulnerability.ScanStatusPending {+ return vulnerability.ScanPhaseCreatingContainer+ }++ return vulnerability.ScanPhaseScanningImage+}++func (s *VulnerabilityService) applyScanPhaseToSummaryInternal(summary *vulnerability.ScanSummary) {+ if summary == nil {+ return+ }++ summary.ScanPhase = s.resolveScanPhaseInternal(summary.ImageID, summary.Status)+}++func (s *VulnerabilityService) applyScanPhaseToResultInternal(result *vulnerability.ScanResult) {+ if result == nil {+ return+ }++ result.ScanPhase = s.resolveScanPhaseInternal(result.ImageID, result.Status)+}++// ScanImage scans an image for vulnerabilities using Trivy+func (s *VulnerabilityService) ScanImage(ctx context.Context, envID string, imageID string, user models.User) (*vulnerability.ScanResult, error) {+ scanCtx := utils.ActivityRuntimeContext(ctx, nil)++ // Get Docker client to inspect the image+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to connect to Docker")+ }++ // Inspect the image to get details+ imageInspect, err := dockerClient.ImageInspect(scanCtx, imageID)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to inspect image")+ }++ // Determine image name to scan+ imageName := imageID+ if len(imageInspect.RepoTags) > 0 {+ imageName = imageInspect.RepoTags[0]+ } else if len(imageInspect.RepoDigests) > 0 {+ imageName = imageInspect.RepoDigests[0]+ }++ slog.InfoContext(scanCtx,+ "vulnerability scan queued",+ "environmentId", envID,+ "imageId", imageID,+ "imageName", imageName,+ )+ activityID := s.startVulnerabilityScanActivityInternal(scanCtx, envID, imageID, imageName, &user)+ // Make the scan cancelable: cancelling the activity cancels scanCtx, which+ // unblocks the Trivy container wait and triggers its (cancel-safe) teardown.+ scanCtx = s.activityService.Track(scanCtx, activityID)+ s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseCreatingContainer)++ // Create pending scan record+ pendingResult := &vulnerability.ScanResult{+ ImageID: imageID,+ ImageName: imageName,+ ScanTime: time.Now(),+ Status: vulnerability.ScanStatusScanning,+ ActivityID: mo.EmptyableToOption(strings.TrimSpace(activityID)).ToPointer(),+ }+ s.applyScanPhaseToResultInternal(pendingResult)+ if saveErr := s.saveScanResultWithRetryInternal(scanCtx, pendingResult); saveErr != nil {+ slog.WarnContext(scanCtx,+ "failed to save pending scan result",+ "error", saveErr,+ "imageId", imageID,+ "imageName", imageName,+ )+ } else {+ slog.DebugContext(scanCtx,+ "saved pending vulnerability scan record",+ "imageId", imageID,+ "imageName", imageName,+ "scanTime", pendingResult.ScanTime,+ )+ }++ go func(bgCtx context.Context, scanEnvID, imgID, imgName string, scanUser models.User, scanActivityID string) {+ s.scanImageInBackgroundInternal(bgCtx, scanEnvID, imgID, imgName, scanUser, scanActivityID)+ }(scanCtx, envID, imageID, imageName, user, activityID)++ return pendingResult, nil+}++func (s *VulnerabilityService) markStaleScanIfNeeded(ctx context.Context, record *models.VulnerabilityScanRecord) {+ if record == nil {+ return+ }+ if record.Status != models.ScanStatusScanning && record.Status != models.ScanStatusPending {+ return+ }+ if time.Since(record.ScanTime) <= scanStaleTimeout {+ return+ }++ errMsg := "Scan timed out or was interrupted. Please retry."+ record.Status = models.ScanStatusFailed+ record.Error = &errMsg+ record.Duration = time.Since(record.ScanTime).Milliseconds()+ s.clearScanPhaseInternal(record.ID)++ if s.db == nil {+ return+ }++ if err := s.db.WithContext(ctx).+ Model(&models.VulnerabilityScanRecord{}).+ Where("id = ?", record.ID).+ Updates(map[string]any{+ "status": record.Status,+ "error": errMsg,+ "duration": record.Duration,+ }).Error; err != nil {+ slog.WarnContext(ctx, "failed to mark stale scan as failed", "error", err, "scan_id", record.ID)+ return+ }+}++func (s *VulnerabilityService) scanImageInBackgroundInternal(ctx context.Context, envID string, imageID, imageName string, user models.User, activityID string) {+ defer s.clearScanPhaseInternal(imageID)+ s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseCreatingContainer)+ s.updateVulnerabilityScanActivityInternal(ctx, activityID, vulnerability.ScanPhaseCreatingContainer, 10, "Preparing vulnerability scanner")++ slog.DebugContext(ctx,+ "starting background vulnerability scan",+ "environmentId", envID,+ "imageId", imageID,+ "imageName", imageName,+ )++ trivyImage, err := s.ensureTrivyImageInternal(ctx)+ if err != nil {+ s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseStoringResults)+ s.updateVulnerabilityScanActivityInternal(ctx, activityID, vulnerability.ScanPhaseStoringResults, 90, "Unable to prepare vulnerability scanner")+ result := &vulnerability.ScanResult{+ ImageID: imageID,+ ImageName: imageName,+ ScanTime: time.Now(),+ Status: vulnerability.ScanStatusFailed,+ ActivityID: mo.EmptyableToOption(strings.TrimSpace(activityID)).ToPointer(),+ Error: "Trivy scanner is not available: " + err.Error(),+ }+ if saveErr := s.saveScanResultWithRetryInternal(ctx, result); saveErr != nil {+ slog.WarnContext(ctx, "failed to save scan result", "error", saveErr)+ }+ slog.WarnContext(ctx,+ "background vulnerability scan failed before execution",+ "environmentId", envID,+ "imageId", imageID,+ "imageName", imageName,+ "error", result.Error,+ )+ s.completeVulnerabilityScanActivityInternal(ctx, activityID, false, result.Error)+ return+ }++ s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseScanningImage)+ s.updateVulnerabilityScanActivityInternal(ctx, activityID, vulnerability.ScanPhaseScanningImage, 45, "Scanning image for vulnerabilities")+ startTime := time.Now()+ result, err := s.runTrivyScan(ctx, trivyImage, imageName, imageID)+ duration := time.Since(startTime).Milliseconds()++ if err != nil {+ s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseStoringResults)+ s.updateVulnerabilityScanActivityInternal(ctx, activityID, vulnerability.ScanPhaseStoringResults, 90, "Storing failed scan result")+ failedResult := &vulnerability.ScanResult{+ ImageID: imageID,+ ImageName: imageName,+ ScanTime: time.Now(),+ Status: vulnerability.ScanStatusFailed,+ ActivityID: mo.EmptyableToOption(strings.TrimSpace(activityID)).ToPointer(),+ Error: err.Error(),+ Duration: duration,+ }+ if saveErr := s.saveScanResultWithRetryInternal(ctx, failedResult); saveErr != nil {+ slog.WarnContext(ctx, "failed to save failed scan result", "error", saveErr)+ }+ slog.WarnContext(ctx,+ "background vulnerability scan failed",+ "environmentId", envID,+ "imageId", imageID,+ "imageName", imageName,+ "durationMs", duration,+ "error", err,+ )+ s.logScanEvent(ctx, envID, imageID, imageName, user, false, err.Error())+ s.completeVulnerabilityScanActivityInternal(ctx, activityID, false, err.Error())+ return+ }++ result.Duration = duration+ result.ActivityID = mo.EmptyableToOption(strings.TrimSpace(activityID)).ToPointer()+ s.ensureSummary(result)+ s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseStoringResults)+ s.updateVulnerabilityScanActivityInternal(ctx, activityID, vulnerability.ScanPhaseStoringResults, 90, "Storing vulnerability results")+ if saveErr := s.saveScanResultWithRetryInternal(ctx, result); saveErr != nil {+ slog.WarnContext(ctx, "failed to save scan result", "error", saveErr)+ }++ slog.InfoContext(ctx,+ "background vulnerability scan completed",+ "environmentId", envID,+ "imageId", imageID,+ "imageName", imageName,+ "durationMs", duration,+ "vulnerabilities", result.Summary.Total,+ )++ s.logScanEvent(ctx, envID, imageID, imageName, user, true, "")+ s.completeVulnerabilityScanActivityInternal(ctx, activityID, true, "")+}++func (s *VulnerabilityService) startVulnerabilityScanActivityInternal(ctx context.Context, envID, imageID, imageName string, user *models.User) string {+ if s.activityService == nil {+ return ""+ }++ activity, err := s.activityService.StartActivity(ctx, activity.StartActivityRequest{+ EnvironmentID: envID,+ Type: models.ActivityTypeVulnerabilityScan,+ ResourceType: new("image"),+ ResourceID: &imageID,+ ResourceName: &imageName,+ StartedBy: user,+ Progress: new(0),+ Step: string(vulnerability.ScanPhaseCreatingContainer),+ LatestMessage: "Vulnerability scan queued",+ })+ if err != nil {+ slog.WarnContext(ctx, "failed to create vulnerability scan activity", "error", err, "imageId", imageID)+ return ""+ }+ return activity.ID+}++func (s *VulnerabilityService) updateVulnerabilityScanActivityInternal(ctx context.Context, activityID string, phase vulnerability.ScanPhase, progress int, message string) {+ if s.activityService == nil || activityID == "" {+ return+ }++ if _, err := s.activityService.AppendMessage(ctx, activityID, activity.AppendActivityMessageRequest{+ Level: models.ActivityMessageLevelInfo,+ Message: message,+ Progress: &progress,+ Step: string(phase),+ }); err != nil {+ slog.DebugContext(ctx, "failed to append vulnerability scan activity message", "activityId", activityID, "error", err)+ }+}++func (s *VulnerabilityService) completeVulnerabilityScanActivityInternal(ctx context.Context, activityID string, success bool, errMessage string) {+ if s.activityService == nil || activityID == "" {+ return+ }++ status := models.ActivityStatusSuccess+ message := "Vulnerability scan completed"+ var errorPtr *string+ if !success {+ if activitylib.CancelledByContext(ctx) {+ status = models.ActivityStatusCancelled+ message = "Vulnerability scan cancelled"+ } else {+ status = models.ActivityStatusFailed+ message = "Vulnerability scan failed"+ if strings.TrimSpace(errMessage) != "" {+ errorPtr = &errMessage+ message = errMessage+ }+ }+ }++ if _, err := s.activityService.CompleteActivity(utils.ActivityRuntimeContext(ctx, nil), activityID, status, message, errorPtr); err != nil {+ slog.DebugContext(ctx, "failed to complete vulnerability scan activity", "activityId", activityID, "error", err)+ }+}++type scheduledScanNotificationSummary struct {+ totalFixable int+ imagesWithFixable int+ severityCounts map[string]int+ sampleCVEs []string+ seenCVEs map[string]struct{}+}++func newScheduledScanNotificationSummary() *scheduledScanNotificationSummary {+ return &scheduledScanNotificationSummary{+ severityCounts: map[string]int{+ "CRITICAL": 0,+ "HIGH": 0,+ "MEDIUM": 0,+ "LOW": 0,+ "UNKNOWN": 0,+ },+ sampleCVEs: make([]string, 0, 5),+ seenCVEs: make(map[string]struct{}),+ }+}++func scheduledScanImageTargetInternal(img imagetypes.Summary) (imageID, imageName string, shouldScan bool) {+ imageID = img.ID+ imageName = imageID+ if len(img.RepoTags) > 0 {+ imageName = img.RepoTags[0]+ } else if len(img.RepoDigests) > 0 {+ imageName = img.RepoDigests[0]+ }++ // Skip intermediate / dangling images with no usable name.+ if imageName == "<none>:<none>" || imageName == imageID {+ return imageID, imageName, false+ }++ return imageID, imageName, true+}++func (s *VulnerabilityService) runScheduledImageScanInternal(+ ctx context.Context,+ containerID, imageName, imageID, scannerVersion string,+) (*vulnerability.ScanResult, int64, error) {+ startTime := time.Now()+ _, releaseSlot, slotErr := s.acquireTrivyScanSlotInternal(ctx)+ if slotErr != nil {+ return nil, 0, slotErr+ }++ var result *vulnerability.ScanResult+ var scanErr error+ func() {+ defer releaseSlot()+ result, scanErr = s.execTrivyScanInContainer(ctx, containerID, imageName, imageID, scannerVersion)+ }()++ return result, time.Since(startTime).Milliseconds(), scanErr+}++func (s *VulnerabilityService) saveScheduledFailedScanInternal(+ ctx context.Context,+ envID, imageID, imageName string,+ user models.User,+ duration int64,+ scanErr error,+) {+ failedResult := &vulnerability.ScanResult{+ ImageID: imageID,+ ImageName: imageName,+ ScanTime: time.Now(),+ Status: vulnerability.ScanStatusFailed,+ Error: scanErr.Error(),+ Duration: duration,+ }+ if saveErr := s.saveScanResultWithRetryInternal(ctx, failedResult); saveErr != nil {+ slog.WarnContext(ctx, "failed to save failed scan result", "error", saveErr)+ }+ s.logScheduledScanEvent(ctx, envID, imageID, imageName, user, false, scanErr.Error())+}++func (s *VulnerabilityService) saveScheduledSuccessfulScanInternal(+ ctx context.Context,+ envID, imageID, imageName string,+ user models.User,+ result *vulnerability.ScanResult,+ duration int64,+) {+ result.Duration = duration+ s.ensureSummary(result)+ if saveErr := s.saveScanResultWithRetryInternal(ctx, result); saveErr != nil {+ slog.WarnContext(ctx, "failed to save scan result", "error", saveErr)+ }+ s.logScheduledScanEvent(ctx, envID, imageID, imageName, user, true, "")+}++func (s *VulnerabilityService) collectScheduledFixableSummaryInternal(+ ctx context.Context,+ result *vulnerability.ScanResult,+ summary *scheduledScanNotificationSummary,+) {+ fixableVulns := s.fixedVulnerabilitiesForNotifications(ctx, result)+ if len(fixableVulns) == 0 {+ return+ }++ summary.imagesWithFixable+++ for i := range fixableVulns {+ v := fixableVulns[i]+ summary.totalFixable++++ sev := strings.ToUpper(strings.TrimSpace(string(v.Severity)))+ if _, ok := summary.severityCounts[sev]; !ok {+ sev = "UNKNOWN"+ }+ summary.severityCounts[sev]++++ cveID := strings.TrimSpace(v.VulnerabilityID)+ if cveID == "" {+ continue+ }+ if _, alreadyAdded := summary.seenCVEs[cveID]; alreadyAdded {+ continue+ }+ summary.seenCVEs[cveID] = struct{}{}+ if len(summary.sampleCVEs) < 5 {+ summary.sampleCVEs = append(summary.sampleCVEs, cveID)+ }+ }+}++func (s *VulnerabilityService) sendScheduledVulnerabilitySummaryInternal(+ ctx context.Context,+ scanned int,+ summary *scheduledScanNotificationSummary,+) {+ if summary.totalFixable == 0 || s.notificationService == nil {+ return+ }++ summaryDate := time.Now().UTC().Format("2006-01-02")+ payload := notification.VulnerabilityNotificationPayload{+ CVEID: "Daily Summary - " + summaryDate,+ Severity: fmt.Sprintf("Critical:%d High:%d Medium:%d Low:%d Unknown:%d", summary.severityCounts["CRITICAL"], summary.severityCounts["HIGH"], summary.severityCounts["MEDIUM"], summary.severityCounts["LOW"], summary.severityCounts["UNKNOWN"]),+ ImageName: fmt.Sprintf("%d image(s) scanned, %d with fixable vulnerabilities", scanned, summary.imagesWithFixable),+ FixedVersion: fmt.Sprintf("%d fixable vulnerability record(s)", summary.totalFixable),+ }+ if len(summary.sampleCVEs) > 0 {+ payload.PkgName = strings.Join(summary.sampleCVEs, ", ")+ }++ if err := s.notificationService.SendVulnerabilityNotification(ctx, payload); err != nil {+ slog.WarnContext(ctx, "failed to send daily vulnerability summary notification", "error", err)+ }+}++// ScanAllImages scans all Docker images for vulnerabilities. It is intended+// for use by the scheduled vulnerability scan job. A single long-running Trivy+// container is created and reused for every image via docker exec, which avoids+// the overhead of creating/destroying a container per scan. The caller-supplied+// user is recorded in the event log.+func (s *VulnerabilityService) ScanAllImages(ctx context.Context, envID string, user models.User) (scanned, failed int, err error) {+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return 0, 0, errors.WrapIf(err, "failed to connect to Docker")+ }++ imageList, err := dockerClient.ImageList(ctx, client.ImageListOptions{})+ if err != nil {+ return 0, 0, errors.WrapIf(err, "failed to list images")+ }+ images := imageList.Items++ containerID, cleanup, err := s.setupTrivyBatchContainer(ctx)+ if err != nil {+ return 0, 0, err+ }+ defer cleanup()++ scannerVersion := s.GetTrivyVersion(ctx)+ notificationSummary := newScheduledScanNotificationSummary()++ for _, img := range images {+ if ctx.Err() != nil {+ return scanned, failed, ctx.Err()+ }++ switch s.scanSingleImage(ctx, containerID, img, scannerVersion, envID, user, notificationSummary) {+ case scanStatusSuccess:+ scanned+++ case scanStatusFailed:+ failed+++ case scanStatusSkipped:+ // no-op+ }+ }++ s.sendScheduledVulnerabilitySummaryInternal(ctx, scanned, notificationSummary)++ return scanned, failed, nil+}++func (s *VulnerabilityService) fixedVulnerabilitiesForNotifications(+ ctx context.Context,+ result *vulnerability.ScanResult,+) []vulnerability.Vulnerability {+ if result == nil || result.Status != vulnerability.ScanStatusCompleted || len(result.Vulnerabilities) == 0 {+ return nil+ }++ vulnerabilities := result.Vulnerabilities+ filtered, err := s.filterIgnoredVulnerabilitiesForImage(ctx, result.ImageID, vulnerabilities)+ if err != nil {+ slog.WarnContext(ctx, "failed to filter ignored vulnerabilities for notifications", "error", err)+ } else {+ vulnerabilities = filtered+ }++ fixable := make([]vulnerability.Vulnerability, 0, len(vulnerabilities))+ for i := range vulnerabilities {+ v := vulnerabilities[i]+ if strings.TrimSpace(v.FixedVersion) == "" {+ continue+ }+ fixable = append(fixable, v)+ }++ return fixable+}++// logScanEvent logs a vulnerability scan event+func (s *VulnerabilityService) logScanEvent(ctx context.Context, envID string, imageID, imageName string, user models.User, success bool, errMsg string) {+ metadata := models.JSON{+ "action": "vulnerability_scan",+ "imageId": imageID,+ "imageName": imageName,+ "success": success,+ }+ if errMsg != "" {+ metadata["error"] = errMsg+ }++ environmentID := envID+ if environmentID == "" {+ environmentID = "0"+ }++ eventType := models.EventTypeImageScan+ if err := s.eventService.LogImageEvent(ctx, eventType, imageID, imageName, user.ID, user.Username, environmentID, metadata); err != nil {+ slog.WarnContext(ctx, "failed to log vulnerability scan event", "error", err)+ }+}++// logScheduledScanEvent logs a vulnerability scan event using the dedicated scheduled scan event type.+func (s *VulnerabilityService) logScheduledScanEvent(ctx context.Context, envID string, imageID, imageName string, user models.User, success bool, errMsg string) {+ metadata := models.JSON{+ "action": "scheduled_vulnerability_scan",+ "imageId": imageID,+ "imageName": imageName,+ "success": success,+ }+ if errMsg != "" {+ metadata["error"] = errMsg+ }++ environmentID := envID+ if environmentID == "" {+ environmentID = "0"+ }++ if err := s.eventService.LogImageEvent(ctx, models.EventTypeImageVulnerabilityScan, imageID, imageName, user.ID, user.Username, environmentID, metadata); err != nil {+ slog.WarnContext(ctx, "failed to log scheduled vulnerability scan event", "error", err)+ }+}diff --git a/backend/internal/vulnerability/vulnerability_store.go b/backend/internal/vulnerability/vulnerability_store.gonew file mode 100644index 000000000..44f922786--- /dev/null+++ b/backend/internal/vulnerability/vulnerability_store.go@@ -0,0 +1,327 @@+package vulnerability++import (+ "context"+ "encoding/json/v2"+ "fmt"+ "log/slog"+ "strings"+ "time"++ "emperror.dev/errors"++ "github.com/moby/moby/client"+ "gorm.io/gorm"+ "gorm.io/gorm/clause"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/dbutil"+ "github.com/getarcaneapp/arcane/types/v2/vulnerability"+)++// DeleteScanResult deletes the scan result for an image+func (s *VulnerabilityService) DeleteScanResult(ctx context.Context, imageID string) error {+ if s.db == nil {+ return nil+ }++ return s.db.WithContext(ctx).Where("id = ?", imageID).Delete(&models.VulnerabilityScanRecord{}).Error+}++// DeleteScanResultsByImageIDs deletes scan results for multiple images in one query.+func (s *VulnerabilityService) DeleteScanResultsByImageIDs(ctx context.Context, imageIDs []string) error {+ if s.db == nil || len(imageIDs) == 0 {+ return nil+ }++ return s.db.WithContext(ctx).Where("id IN ?", imageIDs).Delete(&models.VulnerabilityScanRecord{}).Error+}++// CleanupOrphanedScanRecords removes vulnerability scan records for images that+// no longer exist in Docker. This keeps "images scanned" counts in sync (e.g.+// avoids "5/3" when images were deleted after being scanned). Safe to call+// even when no images exist; returns the number of records deleted.+func (s *VulnerabilityService) CleanupOrphanedScanRecords(ctx context.Context) (deleted int64, err error) {+ if s.db == nil {+ return 0, nil+ }++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return 0, errors.WrapIf(err, "failed to connect to Docker")+ }++ imageList, err := dockerClient.ImageList(ctx, client.ImageListOptions{})+ if err != nil {+ return 0, errors.WrapIf(err, "failed to list images")+ }+ images := imageList.Items++ currentIDs := make(map[string]struct{}, len(images))+ for _, img := range images {+ currentIDs[img.ID] = struct{}{}+ }++ // Build list for NOT IN. If no images, we delete all scan records.+ var ids []string+ if len(currentIDs) > 0 {+ ids = make([]string, 0, len(currentIDs))+ for id := range currentIDs {+ ids = append(ids, id)+ }+ }++ var result *gorm.DB+ if len(ids) == 0 {+ result = s.db.WithContext(ctx).Where("1 = 1").Delete(&models.VulnerabilityScanRecord{})+ } else {+ result = s.db.WithContext(ctx).Where("id NOT IN ?", ids).Delete(&models.VulnerabilityScanRecord{})+ }+ if result.Error != nil {+ return 0, result.Error+ }++ return result.RowsAffected, nil+}++func (s *VulnerabilityService) saveScanResultWithRetryInternal(+ ctx context.Context,+ result *vulnerability.ScanResult,+) error {+ const (+ maxAttempts = defaultScanSaveRetryAttempts+ retryDelay = defaultScanSaveRetryDelay+ )++ var lastErr error+ for attempt := 1; attempt <= maxAttempts; attempt++ {+ lastErr = s.saveScanResult(ctx, result)+ if lastErr == nil {+ if attempt > 1 {+ slog.DebugContext(ctx,+ "save scan result succeeded after retry",+ "attempt", attempt,+ "maxAttempts", maxAttempts,+ "imageId", result.ImageID,+ "status", result.Status,+ )+ }+ return nil+ }++ if attempt < maxAttempts {+ if !dbutil.IsRetryableWriteError(lastErr) {+ break+ }++ delay := min(retryDelay*time.Duration(1<<(attempt-1)), defaultScanSaveRetryMaxDelay)++ timer := time.NewTimer(delay)+ select {+ case <-ctx.Done():+ timer.Stop()+ s.saveScanResultStatusFallbackInternal(ctx, result, ctx.Err())+ return ctx.Err()+ case <-timer.C:+ }+ }+ }++ // Ensure scan status does not stay stuck as scanning when full result+ // persistence fails. Best-effort minimal status write.+ s.saveScanResultStatusFallbackInternal(ctx, result, lastErr)++ return lastErr+}++func (s *VulnerabilityService) saveScanResultStatusFallbackInternal(ctx context.Context, result *vulnerability.ScanResult, originalErr error) {+ if s.db == nil || result == nil || result.ImageID == "" {+ return+ }++ var summary *vulnerability.SeveritySummary+ if result.Summary != nil {+ summary = result.Summary+ } else {+ summary = &vulnerability.SeveritySummary{}+ }++ updates := map[string]any{+ "image_name": result.ImageName,+ "status": string(result.Status),+ "scan_time": result.ScanTime,+ "duration": result.Duration,+ "critical_count": summary.Critical,+ "high_count": summary.High,+ "medium_count": summary.Medium,+ "low_count": summary.Low,+ "unknown_count": summary.Unknown,+ "total_count": summary.Total,+ "scanner_version": result.ScannerVersion,+ }++ errMsg := strings.TrimSpace(result.Error)+ if errMsg == "" && originalErr != nil {+ errMsg = fmt.Sprintf("failed to persist full scan result: %v", originalErr)+ }+ if errMsg == "" {+ updates["error"] = nil+ } else {+ updates["error"] = errMsg+ }++ if err := s.db.WithContext(ctx).+ Model(&models.VulnerabilityScanRecord{}).+ Where("id = ?", result.ImageID).+ Updates(updates).Error; err != nil {+ slog.WarnContext(ctx,+ "failed fallback scan status persistence",+ "imageId", result.ImageID,+ "status", result.Status,+ "error", err,+ "originalError", originalErr,+ )+ return+ }++ slog.WarnContext(ctx,+ "persisted fallback scan status after full save failure",+ "imageId", result.ImageID,+ "status", result.Status,+ "originalError", originalErr,+ )+}++// saveScanResult saves the scan result to the database+func (s *VulnerabilityService) saveScanResult(ctx context.Context, result *vulnerability.ScanResult) error {+ if s.db == nil {+ return nil+ }++ s.ensureSummary(result)++ // Convert vulnerabilities to JSON+ var vulnJSON models.StringSlice+ if len(result.Vulnerabilities) > 0 {+ vulnBytes, err := json.Marshal(result.Vulnerabilities)+ if err != nil {+ return errors.WrapIf(err, "failed to marshal vulnerabilities")+ }+ vulnJSON = models.StringSlice{string(vulnBytes)}+ }++ var errPtr *string+ if result.Error != "" {+ errPtr = &result.Error+ }++ var summary *vulnerability.SeveritySummary+ if result.Summary != nil {+ summary = result.Summary+ } else {+ summary = &vulnerability.SeveritySummary{}+ }++ record := models.VulnerabilityScanRecord{+ ID: result.ImageID,+ ImageName: result.ImageName,+ Status: string(result.Status),+ ScanTime: result.ScanTime,+ Duration: result.Duration,+ CriticalCount: summary.Critical,+ HighCount: summary.High,+ MediumCount: summary.Medium,+ LowCount: summary.Low,+ UnknownCount: summary.Unknown,+ TotalCount: summary.Total,+ Vulnerabilities: vulnJSON,+ Error: errPtr,+ ScannerVersion: result.ScannerVersion,+ }++ // Upsert the record+ return s.db.WithContext(ctx).Clauses(clause.OnConflict{+ Columns: []clause.Column{{Name: "id"}},+ UpdateAll: true,+ }).Create(&record).Error+}++// convertRecordToResult converts a database record to a ScanResult+func (s *VulnerabilityService) convertRecordToResult(record *models.VulnerabilityScanRecord) (*vulnerability.ScanResult, error) {+ result := &vulnerability.ScanResult{+ ImageID: record.ID,+ ImageName: record.ImageName,+ ScanTime: record.ScanTime,+ Status: vulnerability.ScanStatus(record.Status),+ Duration: record.Duration,+ Summary: &vulnerability.SeveritySummary{+ Critical: record.CriticalCount,+ High: record.HighCount,+ Medium: record.MediumCount,+ Low: record.LowCount,+ Unknown: record.UnknownCount,+ Total: record.TotalCount,+ },+ ScannerVersion: record.ScannerVersion,+ }++ if record.Error != nil {+ result.Error = *record.Error+ }++ // Parse vulnerabilities from JSON+ if len(record.Vulnerabilities) > 0 && record.Vulnerabilities[0] != "" {+ var vulns []vulnerability.Vulnerability+ if err := json.Unmarshal([]byte(record.Vulnerabilities[0]), &vulns); err != nil {+ slog.Warn("failed to unmarshal vulnerabilities", "error", err)+ } else {+ result.Vulnerabilities = vulns+ }+ }++ s.ensureSummary(result)+ s.applyScanPhaseToResultInternal(result)++ return result, nil+}++func (s *VulnerabilityService) ensureSummary(result *vulnerability.ScanResult) {+ if result == nil {+ return+ }++ if len(result.Vulnerabilities) == 0 {+ if result.Summary == nil {+ result.Summary = &vulnerability.SeveritySummary{}+ }+ return+ }++ if result.Summary != nil && result.Summary.Total > 0 {+ return+ }++ result.Summary = buildSeveritySummaryFromVulnerabilitiesInternal(result.Vulnerabilities)+}++func buildSeveritySummaryFromVulnerabilitiesInternal(vulns []vulnerability.Vulnerability) *vulnerability.SeveritySummary {+ summary := &vulnerability.SeveritySummary{}+ for _, vulnEntry := range vulns {+ switch vulnEntry.Severity {+ case vulnerability.SeverityCritical:+ summary.Critical+++ case vulnerability.SeverityHigh:+ summary.High+++ case vulnerability.SeverityMedium:+ summary.Medium+++ case vulnerability.SeverityLow:+ summary.Low+++ case vulnerability.SeverityUnknown:+ summary.Unknown+++ default:+ summary.Unknown+++ }+ summary.Total+++ }+ return summary+}diff --git a/backend/internal/vulnerability/vulnerability_trivy.go b/backend/internal/vulnerability/vulnerability_trivy.gonew file mode 100644index 000000000..e12bef823--- /dev/null+++ b/backend/internal/vulnerability/vulnerability_trivy.go@@ -0,0 +1,1226 @@+package vulnerability++import (+ "context"+ "fmt"+ "io"+ "log/slog"+ "os"+ "strings"+ "sync"+ "time"++ "emperror.dev/errors"++ containertypes "github.com/moby/moby/api/types/container"+ imagetypes "github.com/moby/moby/api/types/image"+ mounttypes "github.com/moby/moby/api/types/mount"+ "github.com/moby/moby/client"+ "github.com/samber/mo"++ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/vuln"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/projects"+ "github.com/getarcaneapp/arcane/types/v2/vulnerability"+ "go.getarcane.app/sys/cgroup"+)++type trivyRuntimeSecurityInternal struct {+ securityOpts []string+ privileged bool+}++func (s *VulnerabilityService) acquireTrivyScanSlotInternal(ctx context.Context) (int, func(), error) {+ limit := s.getTrivyConcurrentScanContainersInternal()+ slotCh := s.getTrivyScanSlotChannelInternal(limit)+ if slotCh == nil {+ return 0, func() {}, nil+ }++ select {+ case slotID := <-slotCh:+ return slotID, func() { slotCh <- slotID }, nil+ case <-ctx.Done():+ return 0, nil, ctx.Err()+ }+}++func (s *VulnerabilityService) getTrivyConcurrentScanContainersInternal() int {+ if s.settingsService == nil {+ return 1+ }++ cfg := s.settingsService.GetSettingsConfig()+ if cfg == nil {+ return 1+ }++ limit := cfg.TrivyConcurrentScanContainers.AsInt()+ if limit < 1 {+ return 1+ }++ return limit+}++func (s *VulnerabilityService) getTrivyScanSlotChannelInternal(limit int) chan int {+ if limit < 1 {+ limit = 1+ }++ createSlots := func(size int) chan int {+ slots := make(chan int, size)+ for i := range size {+ slots <- i+ }+ return slots+ }++ s.trivyScanMu.Lock()+ defer s.trivyScanMu.Unlock()++ if s.trivyScanSlots == nil {+ s.trivyScanSlots = createSlots(limit)+ return s.trivyScanSlots+ }++ if cap(s.trivyScanSlots) == limit {+ return s.trivyScanSlots+ }++ // Apply capacity changes only when idle to avoid splitting tokens across channels.+ if len(s.trivyScanSlots) == cap(s.trivyScanSlots) {+ s.trivyScanSlots = createSlots(limit)+ }++ return s.trivyScanSlots+}++func (s *VulnerabilityService) buildTrivyRegistryConfigInternal(ctx context.Context) []byte {+ if s.registryService == nil {+ return nil+ }++ authConfigs, err := s.registryService.GetAllRegistryAuthConfigs(ctx)+ if err != nil {+ slog.WarnContext(ctx, "failed to load registry credentials for trivy scan; continuing anonymously", "error", err)+ return nil+ }++ configJSON, err := vuln.BuildDockerConfigJSON(authConfigs)+ if err != nil {+ slog.WarnContext(ctx, "failed to build trivy registry config; continuing anonymously", "error", err)+ return nil+ }++ return configJSON+}++type scanResultStatus int++const (+ scanStatusSkipped scanResultStatus = iota+ scanStatusSuccess+ scanStatusFailed+)++func (s *VulnerabilityService) setupTrivyBatchContainer(ctx context.Context) (string, func(), error) {+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return "", nil, errors.WrapIf(err, "failed to connect to Docker")+ }++ trivyImage, err := s.ensureTrivyImageInternal(ctx)+ if err != nil {+ return "", nil, errors.WrapIf(err, "trivy scanner not available")+ }++ serverURL, _ := s.getTrivyServerConfig()+ var cacheVolume string+ if serverURL == "" {+ cacheVolume, err = s.ensureTrivyCacheVolumeInternal(ctx)+ if err != nil {+ return "", nil, errors.WrapIf(err, "failed to ensure trivy cache volume")+ }+ }++ containerID, err := s.createBatchTrivyContainer(ctx, trivyImage, cacheVolume)+ if err != nil {+ return "", nil, errors.WrapIf(err, "failed to create batch trivy container")+ }++ cleanup := func() {+ vuln.RemoveContainer(ctx, dockerClient, containerID, "failed to remove batch trivy container")+ }++ return containerID, cleanup, nil+}++func (s *VulnerabilityService) scanSingleImage(+ ctx context.Context,+ containerID string,+ img imagetypes.Summary,+ scannerVersion, envID string,+ user models.User,+ notificationSummary *scheduledScanNotificationSummary,+) scanResultStatus {+ imageID, imageName, shouldScan := scheduledScanImageTargetInternal(img)+ if !shouldScan {+ return scanStatusSkipped+ }++ slog.InfoContext(ctx, "scheduled vulnerability scan: scanning image", "image", imageName, "imageId", imageID)++ result, duration, scanErr := s.runScheduledImageScanInternal(ctx, containerID, imageName, imageID, scannerVersion)+ if scanErr != nil {+ s.saveScheduledFailedScanInternal(ctx, envID, imageID, imageName, user, duration, scanErr)+ return scanStatusFailed+ }++ s.saveScheduledSuccessfulScanInternal(ctx, envID, imageID, imageName, user, result, duration)+ s.collectScheduledFixableSummaryInternal(ctx, result, notificationSummary)+ return scanStatusSuccess+}++// createBatchTrivyContainer creates a long-running Trivy container that stays+// alive so we can exec individual scan commands into it. This avoids the+// overhead of creating/starting/removing a container for every image.+func (s *VulnerabilityService) createBatchTrivyContainer(ctx context.Context, trivyImage, cacheVolume string) (string, error) {+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return "", errors.WrapIf(err, "failed to connect to Docker")+ }++ runtimeOptions, err := s.resolveTrivyRuntimeOptionsInternal(ctx, dockerClient)+ if err != nil {+ return "", err+ }++ registryConfigJSON := s.buildTrivyRegistryConfigInternal(ctx)+ env := append([]string(nil), runtimeOptions.ContainerEnv...)+ if len(registryConfigJSON) > 0 {+ env = append(env, "DOCKER_CONFIG="+vuln.RegistryConfigDir)+ // ECR tokens are refreshed when the batch container is created; a multi-hour+ // batch could outlive the token and fall back to anonymous pulls.+ }++ config := &containertypes.Config{+ Image: trivyImage,+ Entrypoint: []string{"sh", "-c", "trap 'exit 0' TERM; while :; do sleep 1; done"},+ Env: env,+ Labels: map[string]string{+ libarcane.InternalResourceLabel: "true",+ },+ }++ runtimeSecurity := s.getTrivyRuntimeSecurityInternal().OrEmpty()+ securityOpts, privileged := runtimeSecurity.securityOpts, runtimeSecurity.privileged+ hostConfig := vuln.BuildBatchHostConfig(cacheVolume, runtimeOptions, securityOpts, privileged)++ resources, cpuSet, applyLimits := s.getTrivyContainerResourceOptionsInternal(ctx, dockerClient)+ vuln.ApplyContainerResources(hostConfig, resources, cpuSet, applyLimits)++ containerID, err := s.createAndStartTrivyContainerInternal(ctx, dockerClient, "batch", config, hostConfig, registryConfigJSON)+ if err != nil {+ return "", err+ }++ slog.InfoContext(ctx, "batch trivy container started", "containerId", containerID)++ return containerID, nil+}++// execTrivyScanInContainer runs a trivy scan for a single image inside an+// already-running container via docker exec.+func (s *VulnerabilityService) execTrivyScanInContainer(ctx context.Context, containerID, imageName, imageID, scannerVersion string) (*vulnerability.ScanResult, error) {+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to connect to Docker")+ }++ stdoutFile, err := vuln.CreateLogTempFile("trivy-exec-stdout-*")+ if err != nil {+ return nil, errors.WrapIf(err, "failed to create trivy exec stdout temp file")+ }++ stderrFile, err := vuln.CreateLogTempFile("trivy-exec-stderr-*")+ if err != nil {+ vuln.CleanupLogTempFiles(ctx, stdoutFile)+ return nil, errors.WrapIf(err, "failed to create trivy exec stderr temp file")+ }+ defer vuln.CleanupLogTempFiles(ctx, stdoutFile, stderrFile)++ trivyTimeout := s.getTrivyScanTimeoutArg()+ outputPath := vuln.NewOutputPath()+ defer vuln.CleanupOutputFileInContainer(ctx, dockerClient, containerID, outputPath)++ serverURL, serverToken := s.getTrivyServerConfig()+ execCmd := make([]string, 0, 16)+ execCmd = append(execCmd, "trivy", "image", "--format", "json", "--quiet", "--output", outputPath, "--timeout", trivyTimeout)+ execCmd = append(execCmd, vuln.ScanSourceArgs(serverURL)...)+ execCmd = append(execCmd, vuln.IgnoreUnfixedArgs(s.getTrivyIgnoreUnfixed())...)+ execCmd = append(execCmd, imageName)++ exitCode, err := dockerutils.ExecInContainer(ctx, dockerClient, containerID, client.ExecCreateOptions{+ Cmd: execCmd,+ Env: vuln.ServerTokenEnv(serverToken),+ AttachStdout: true,+ AttachStderr: true,+ }, stdoutFile, stderrFile)+ if err != nil {+ return nil, err+ }++ if exitCode != 0 {+ errMsg := vuln.ReadTempFileExcerpt(stderrFile)+ if errMsg == "" {+ errMsg = vuln.ReadTempFileExcerpt(stdoutFile)+ }+ if errMsg == "" {+ errMsg = fmt.Sprintf("exit status %d", exitCode)+ }+ slog.WarnContext(ctx,+ "trivy scan diagnostics",+ "image", imageName,+ "imageId", imageID,+ "reason", "exec non-zero exit",+ "exitCode", int64(exitCode),+ "stdoutBytes", vuln.TempFileSize(stdoutFile),+ "stderrBytes", vuln.TempFileSize(stderrFile),+ )+ return nil, vuln.FormatExitError(int64(exitCode), errMsg, imageName)+ }++ trivyOutput, err := vuln.ReadOutputFromContainerFile(ctx, dockerClient, containerID, outputPath)+ if err != nil {+ slog.WarnContext(ctx,+ "failed to read trivy output file from batch container; falling back to stdout parsing",+ "containerId", containerID,+ "outputPath", outputPath,+ "error", err,+ )+ trivyOutput = nil+ }++ var trivyReport *vulnerability.TrivyReport+ if len(trivyOutput) > 0 {+ trivyReport, err = vuln.DecodeReportFromBytes(trivyOutput)+ } else {+ trivyReport, err = vuln.DecodeReportFromFile(stdoutFile)+ }+ if err != nil {+ if errors.Is(err, io.EOF) {+ errMsg := vuln.ReadTempFileExcerpt(stderrFile)+ if errMsg == "" {+ errMsg = "trivy scan produced no output"+ }+ slog.WarnContext(ctx,+ "trivy scan diagnostics",+ "image", imageName,+ "imageId", imageID,+ "reason", "exec empty output",+ "exitCode", int64(exitCode),+ "stdoutBytes", vuln.TempFileSize(stdoutFile),+ "stderrBytes", vuln.TempFileSize(stderrFile),+ )+ return nil, errors.Errorf("trivy scan failed: %s", errMsg)+ }++ slog.WarnContext(ctx,+ "trivy scan diagnostics",+ "image", imageName,+ "imageId", imageID,+ "reason", "exec parse error",+ "exitCode", int64(exitCode),+ "stdoutBytes", vuln.TempFileSize(stdoutFile),+ "stderrBytes", vuln.TempFileSize(stderrFile),+ )+ return nil, errors.WrapIf(err, "failed to parse trivy output")+ }++ if trivyReport == nil {+ errMsg := vuln.ReadTempFileExcerpt(stderrFile)+ if errMsg == "" {+ errMsg = "trivy scan produced no output"+ }+ slog.WarnContext(ctx,+ "trivy scan diagnostics",+ "image", imageName,+ "imageId", imageID,+ "reason", "exec nil report",+ "exitCode", int64(exitCode),+ "stdoutBytes", vuln.TempFileSize(stdoutFile),+ "stderrBytes", vuln.TempFileSize(stderrFile),+ )+ return nil, errors.Errorf("trivy scan failed: %s", errMsg)+ }++ result := vulnerability.ConvertTrivyReportToScanResult(trivyReport, imageID, time.Now(), 0)+ result.ScannerVersion = scannerVersion++ return result, nil+}++// GetTrivyVersion returns the Trivy version from the configured scanner image.+func (s *VulnerabilityService) GetTrivyVersion(ctx context.Context) string {+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return ""+ }++ trivyImage, err := s.ensureTrivyImageInternal(ctx)+ if err != nil {+ return ""+ }++ config := &containertypes.Config{+ Image: trivyImage,+ Entrypoint: []string{"trivy"},+ Cmd: []string{"--version"},+ Labels: map[string]string{+ libarcane.InternalResourceLabel: "true",+ },+ }++ hostConfig := &containertypes.HostConfig{+ AutoRemove: true,+ }++ resp, err := dockerClient.ContainerCreate(ctx, client.ContainerCreateOptions{+ Config: config,+ HostConfig: hostConfig,+ })+ if err != nil {+ return ""+ }++ if _, err := dockerClient.ContainerStart(ctx, resp.ID, client.ContainerStartOptions{}); err != nil {+ _, _ = dockerClient.ContainerRemove(ctx, resp.ID, client.ContainerRemoveOptions{Force: true})+ return ""+ }++ logs, err := dockerClient.ContainerLogs(ctx, resp.ID, client.ContainerLogsOptions{+ ShowStdout: true,+ ShowStderr: true,+ Follow: true,+ })+ if err != nil {+ _, _ = dockerClient.ContainerRemove(ctx, resp.ID, client.ContainerRemoveOptions{Force: true})+ return ""+ }+ var closeLogsOnce sync.Once+ closeLogs := func() {+ closeLogsOnce.Do(func() {+ if closeErr := logs.Close(); closeErr != nil && !dockerutils.IsExpectedStreamEndError(closeErr) {+ slog.WarnContext(ctx, "failed to close trivy version logs stream", "error", closeErr)+ }+ })+ }+ defer closeLogs()++ stdoutFile, err := vuln.CreateLogTempFile("trivy-version-stdout-*")+ if err != nil {+ return ""+ }++ stderrFile, err := vuln.CreateLogTempFile("trivy-version-stderr-*")+ if err != nil {+ vuln.CleanupLogTempFiles(ctx, stdoutFile)+ return ""+ }+ defer vuln.CleanupLogTempFiles(ctx, stdoutFile, stderrFile)++ logDone := dockerutils.StartStdCopy(logs, stdoutFile, stderrFile)++ waitResult := dockerClient.ContainerWait(ctx, resp.ID, client.ContainerWaitOptions{Condition: containertypes.WaitConditionNotRunning})+ var waitResp containertypes.WaitResponse+ select {+ case err := <-waitResult.Error:+ if err != nil {+ return ""+ }+ case waitResp = <-waitResult.Result:+ }++ closeLogs()++ if err := dockerutils.WaitStdCopy(logDone); err != nil {+ return ""+ }++ if waitResp.StatusCode != 0 {+ return ""+ }++ return vuln.ParseVersion(vuln.ReadTempFileExcerpt(stdoutFile))+}++func (s *VulnerabilityService) getTrivyImageRef() string {+ return settings.DefaultTrivyImage+}++func (s *VulnerabilityService) getConfiguredTrivyNetworkModeInternal() string {+ if s.settingsService == nil {+ return ""+ }++ cfg := s.settingsService.GetSettingsConfig()+ if cfg == nil {+ return ""+ }++ return strings.TrimSpace(cfg.TrivyNetwork.Value)+}++func (s *VulnerabilityService) getEffectiveDockerHostInternal() string {+ if s.dockerService != nil {+ if host := strings.TrimSpace(s.dockerService.DockerHost()); host != "" {+ return host+ }+ }++ if s.settingsService != nil {+ if cfg := s.settingsService.GetSettingsConfig(); cfg != nil {+ if host := strings.TrimSpace(cfg.DockerHost.Value); host != "" {+ return host+ }+ }+ }++ return vuln.DefaultDockerHostURI+}++func (s *VulnerabilityService) resolveTrivyNetworkModeInternal(ctx context.Context, dockerClient *client.Client) string {+ if configuredNetwork := s.getConfiguredTrivyNetworkModeInternal(); configuredNetwork != "" {+ return configuredNetwork+ }++ inspect, err := libarcane.InspectCurrentArcaneContainer(ctx, dockerClient)+ if err == nil {+ return vuln.SelectAutoNetworkMode(inspect)+ }++ slog.DebugContext(ctx, "failed to auto-detect Trivy network; falling back to bridge", "error", err)+ return vuln.DefaultNetworkMode+}++func (s *VulnerabilityService) resolveTrivyRuntimeOptionsInternal(ctx context.Context, dockerClient *client.Client) (vuln.RuntimeOptions, error) {+ options := vuln.RuntimeOptions{+ NetworkMode: s.resolveTrivyNetworkModeInternal(ctx, dockerClient),+ }++ dockerHost := s.getEffectiveDockerHostInternal()+ scheme, socketPath, err := vuln.ParseDockerHost(dockerHost)+ if err != nil {+ return vuln.RuntimeOptions{}, errors.WrapIf(err, "failed to resolve trivy docker host")+ }++ options.ContainerEnv = vuln.BuildDockerHostEnv(dockerHost)+ if scheme != "unix" {+ return options, nil+ }++ socketSource, err := vuln.ResolveUnixSocketSource(+ ctx,+ socketPath,+ func(ctx context.Context, containerPath string) (string, error) {+ return projects.GetHostPathForContainerPath(ctx, dockerClient, containerPath)+ },+ func() bool {+ _, err := cgroup.CurrentContainerID()+ return err == nil+ },+ )+ if err != nil {+ return vuln.RuntimeOptions{}, errors.WrapIf(err, "failed to resolve trivy docker host")+ }++ options.Mounts = append(options.Mounts, mounttypes.Mount{+ Type: mounttypes.TypeBind,+ Source: socketSource,+ Target: socketPath,+ })++ return options, nil+}++func (s *VulnerabilityService) getTrivyRuntimeSecurityInternal() mo.Option[trivyRuntimeSecurityInternal] {+ if s.settingsService == nil {+ return mo.None[trivyRuntimeSecurityInternal]()+ }++ cfg := s.settingsService.GetSettingsConfig()+ if cfg == nil {+ return mo.None[trivyRuntimeSecurityInternal]()+ }++ return mo.Some(trivyRuntimeSecurityInternal{+ securityOpts: vuln.ParseSecurityOpts(cfg.TrivySecurityOpts.Value),+ privileged: cfg.TrivyPrivileged.IsTrue(),+ })+}++func (s *VulnerabilityService) getTrivyScanTimeoutArg() string {+ timeoutSeconds := 0+ if s.settingsService != nil {+ timeoutSeconds = s.settingsService.GetSettingsConfig().TrivyScanTimeout.AsInt()+ }++ return timeouts.GetDuration(timeoutSeconds, timeouts.DefaultTrivyScan).String()+}++func (s *VulnerabilityService) getTrivySingleScanCacheDirForSlotInternal(slotID int) string {+ if slotID < 0 {+ slotID = 0+ }++ // Use a fixed cache directory per concurrency slot on the shared cache+ // volume. This avoids Trivy fs-cache lock contention while bounding cache+ // growth to the configured slot count.+ return fmt.Sprintf("%s/slot-%d", trivyCacheSlotRootDir, slotID)+}++func (s *VulnerabilityService) ensureTrivyImageInternal(ctx context.Context) (string, error) {+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return "", errors.WrapIf(err, "failed to connect to Docker")+ }++ trivyImage := s.getTrivyImageRef()+ if _, err := dockerClient.ImageInspect(ctx, trivyImage); err == nil {+ return trivyImage, nil+ }++ pullTimeoutSeconds := 0+ if s.settingsService != nil && s.settingsService.GetSettingsConfig() != nil {+ pullTimeoutSeconds = s.settingsService.GetSettingsConfig().DockerImagePullTimeout.AsInt()+ }++ pullCtx, pullCancel := context.WithTimeout(ctx, timeouts.GetDuration(pullTimeoutSeconds, timeouts.DefaultDockerImagePull))+ defer pullCancel()++ pullReader, err := dockerClient.ImagePull(pullCtx, trivyImage, client.ImagePullOptions{})+ if err != nil {+ if errors.Is(pullCtx.Err(), context.DeadlineExceeded) {+ return "", errors.Errorf("scanner image pull timed out for %s (increase DOCKER_IMAGE_PULL_TIMEOUT or setting)", trivyImage)+ }+ return "", errors.WrapIff(err, "pull scanner image %s", trivyImage)+ }+ if err := dockerutils.RenderJSONMessageStream(pullReader, io.Discard); err != nil {+ _ = pullReader.Close()+ return "", errors.WrapIf(err, "failed to complete scanner image pull")+ }+ _ = pullReader.Close()++ return trivyImage, nil+}++func (s *VulnerabilityService) ensureTrivyCacheVolumeInternal(ctx context.Context) (string, error) {+ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return "", err+ }++ if _, err := dockerClient.VolumeInspect(ctx, trivyCacheVolumeName, client.VolumeInspectOptions{}); err == nil {+ return trivyCacheVolumeName, nil+ }++ _, err = dockerClient.VolumeCreate(ctx, client.VolumeCreateOptions{+ Name: trivyCacheVolumeName,+ Labels: map[string]string{+ libarcane.InternalResourceLabel: "true",+ },+ })+ if err != nil {+ return "", errors.WrapIf(err, "failed to create trivy cache volume")+ }++ return trivyCacheVolumeName, nil+}++// runTrivyScan executes Trivy scan on an image+func (s *VulnerabilityService) getTrivyConfigFiles() (configContent, ignoreContent string) {+ if s.settingsService == nil {+ return "", ""+ }++ cfg := s.settingsService.GetSettingsConfig()+ if cfg == nil {+ return "", ""+ }++ return cfg.TrivyConfig.Value, cfg.TrivyIgnore.Value+}++// getTrivyServerConfig returns the remote Trivy server URL and optional token when+// client/server mode is enabled, or empty strings for standalone (local DB) scanning.+func (s *VulnerabilityService) getTrivyServerConfig() (serverURL, token string) {+ if s.settingsService == nil {+ return "", ""+ }++ cfg := s.settingsService.GetSettingsConfig()+ if cfg == nil || !cfg.TrivyServerEnabled.IsTrue() {+ return "", ""+ }++ return strings.TrimSpace(cfg.TrivyServerUrl.Value), strings.TrimSpace(cfg.TrivyServerToken.Value)+}++// getTrivyIgnoreUnfixed reports whether scans should only return vulnerabilities+// that have a known fix (--ignore-unfixed).+func (s *VulnerabilityService) getTrivyIgnoreUnfixed() bool {+ if s.settingsService == nil {+ return false+ }++ cfg := s.settingsService.GetSettingsConfig()+ return cfg != nil && cfg.TrivyIgnoreUnfixed.IsTrue()+}++func (s *VulnerabilityService) createTrivyConfigTempFile(ctx context.Context, content string) mo.Option[string] {+ configFile, err := os.CreateTemp("", "trivy-config-*.yaml")+ if err != nil {+ slog.WarnContext(ctx, "failed to create trivy config temp file", "error", err)+ return mo.None[string]()+ }+ if _, err := configFile.WriteString(content); err != nil {+ slog.WarnContext(ctx, "failed to write trivy config", "error", err)+ _ = configFile.Close()+ _ = os.Remove(configFile.Name())+ return mo.None[string]()+ }+ _ = configFile.Close()+ return mo.Some(configFile.Name())+}++func (s *VulnerabilityService) createTrivyIgnoreTempFile(ctx context.Context, content string) mo.Option[string] {+ ignoreFile, err := os.CreateTemp("", "trivy-ignore-*")+ if err != nil {+ slog.WarnContext(ctx, "failed to create trivy ignore temp file", "error", err)+ return mo.None[string]()+ }+ if _, err := ignoreFile.WriteString(content); err != nil {+ slog.WarnContext(ctx, "failed to write trivy ignore", "error", err)+ _ = ignoreFile.Close()+ _ = os.Remove(ignoreFile.Name())+ return mo.None[string]()+ }+ _ = ignoreFile.Close()+ return mo.Some(ignoreFile.Name())+}++// trivyScanCacheBackendArgsForArchInternal returns extra Trivy flags that switch+// the scan cache to Trivy's documented in-memory backend on 32-bit architectures.+func (s *VulnerabilityService) buildTrivyCommandArgs(+ ctx context.Context,+ imageName string,+ configContent string,+ ignoreContent string,+ cacheDir string,+ outputPath string,+) ([]string, []string) {+ serverURL, _ := s.getTrivyServerConfig()+ cmdArgs := []string{"image", "--format", "json", "--quiet", "--timeout", s.getTrivyScanTimeoutArg()}+ cmdArgs = append(cmdArgs, vuln.ScanSourceArgs(serverURL)...)+ cmdArgs = append(cmdArgs, vuln.IgnoreUnfixedArgs(s.getTrivyIgnoreUnfixed())...)+ var tempFiles []string++ if strings.TrimSpace(configContent) != "" {+ if tempFile, ok := s.createTrivyConfigTempFile(ctx, configContent).Get(); ok {+ tempFiles = append(tempFiles, tempFile)+ cmdArgs = append(cmdArgs, "--config", "/tmp/trivy-config.yaml")+ }+ }++ if strings.TrimSpace(ignoreContent) != "" {+ if tempFile, ok := s.createTrivyIgnoreTempFile(ctx, ignoreContent).Get(); ok {+ tempFiles = append(tempFiles, tempFile)+ cmdArgs = append(cmdArgs, "--ignorefile", "/tmp/trivy-ignore")+ }+ }++ if strings.TrimSpace(cacheDir) != "" {+ cmdArgs = append(cmdArgs, "--cache-dir", cacheDir)+ }++ if strings.TrimSpace(outputPath) != "" {+ cmdArgs = append(cmdArgs, "--output", outputPath)+ }++ cmdArgs = append(cmdArgs, imageName)++ return cmdArgs, tempFiles+}++func (s *VulnerabilityService) createAndStartTrivyContainerInternal(+ ctx context.Context,+ dockerClient *client.Client,+ scope string,+ config *containertypes.Config,+ hostConfig *containertypes.HostConfig,+ registryConfigJSON []byte,+) (string, error) {+ vuln.LogContainerStartupRequest(ctx, scope, config, hostConfig)++ createCtx, createCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))+ defer createCancel()++ resp, err := dockerClient.ContainerCreate(createCtx, client.ContainerCreateOptions{+ Config: config,+ HostConfig: hostConfig,+ })+ if err != nil {+ slog.WarnContext(ctx, "failed to create trivy container", "scope", scope, "error", err)+ return "", errors.WrapIf(err, "failed to create container")+ }++ if len(registryConfigJSON) > 0 {+ if err := vuln.CopyRegistryConfigToContainer(ctx, dockerClient, resp.ID, registryConfigJSON); err != nil {+ // DOCKER_CONFIG may still point at the target directory. When the+ // config file is absent, Trivy falls back to anonymous registry access.+ slog.WarnContext(ctx,+ "failed to inject registry credentials into trivy container; continuing anonymously",+ "scope", scope,+ "containerId", resp.ID,+ "error", err,+ )+ }+ }++ startCtx, startCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))+ defer startCancel()++ if _, err := dockerClient.ContainerStart(startCtx, resp.ID, client.ContainerStartOptions{}); err != nil {+ stdoutLog, stderrLog := vuln.ReadStartupLogs(ctx, dockerClient, resp.ID)+ slog.WarnContext(ctx,+ "failed to start trivy container",+ "scope", scope,+ "containerId", resp.ID,+ "error", err,+ "stdout", stdoutLog,+ "stderr", stderrLog,+ )++ vuln.RemoveContainer(ctx, dockerClient, resp.ID, "failed to cleanup trivy container after start failure")+ return "", errors.WrapIf(err, "failed to start container")+ }++ return resp.ID, nil+}++func (s *VulnerabilityService) getTrivyContainerResourcesInternal() mo.Option[containertypes.Resources] {+ var cfg *models.Settings+ if s.settingsService != nil {+ cfg = s.settingsService.GetSettingsConfig()+ }++ return buildTrivyContainerResourcesInternal(cfg)+}++func (s *VulnerabilityService) getTrivyContainerResourceOptionsInternal(ctx context.Context, dockerClient *client.Client) (containertypes.Resources, string, bool) {+ resources, applyLimits := s.getTrivyContainerResourcesInternal().Get()+ if !applyLimits || dockerClient == nil {+ return resources, "", applyLimits+ }++ infoCtx, infoCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))+ defer infoCancel()++ slog.DebugContext(ctx, "inspecting docker host for trivy resource strategy")++ info, err := dockerClient.Info(infoCtx, client.InfoOptions{})+ if err != nil {+ if errors.Is(infoCtx.Err(), context.DeadlineExceeded) || errors.Is(err, context.DeadlineExceeded) {+ slog.WarnContext(ctx,+ "timed out inspecting docker info for trivy resource strategy; falling back to default limits",+ "timeout", timeouts.DefaultDockerAPI.String(),+ "error", err,+ )+ } else {+ slog.DebugContext(ctx, "failed to inspect docker info for trivy resource strategy", "error", err)+ }+ return resources, "", applyLimits+ }++ slog.DebugContext(ctx,+ "resolved docker host info for trivy resource strategy",+ "operatingSystem", info.Info.OperatingSystem,+ "ncpu", info.Info.NCPU,+ )++ if !vuln.IsSynologyDockerHost(info.Info.OperatingSystem) {+ return resources, "", applyLimits+ }++ cpuSet := vuln.BuildCPUSet(resources.NanoCPUs, info.Info.NCPU)+ resources.NanoCPUs = 0++ if cpuSet != "" {+ slog.InfoContext(ctx,+ "detected Synology Docker host; applying Trivy CPU limit via cpuset",+ "operatingSystem", info.Info.OperatingSystem,+ "cpuset", cpuSet,+ )+ } else {+ slog.InfoContext(ctx,+ "detected Synology Docker host; skipping Trivy NanoCPUs limit",+ "operatingSystem", info.Info.OperatingSystem,+ )+ }++ return resources, cpuSet, applyLimits+}++func buildTrivyContainerResourcesInternal(cfg *models.Settings) mo.Option[containertypes.Resources] {+ limitsEnabled := true+ cpuLimitCores := defaultTrivyCPULimitCores+ memoryLimitMB := defaultTrivyMemoryLimitMB++ if cfg != nil {+ limitsEnabled = cfg.TrivyResourceLimitsEnabled.IsTrue()+ cpuLimitCores = vuln.ParseCPULimit(cfg.TrivyCpuLimit.Value, defaultTrivyCPULimitCores)+ memoryLimitMB = vuln.ParseMemoryLimitMB(cfg.TrivyMemoryLimitMb.Value, defaultTrivyMemoryLimitMB)+ }++ if !limitsEnabled {+ return mo.None[containertypes.Resources]()+ }++ resources := containertypes.Resources{}+ if cpuLimitCores > 0 {+ resources.NanoCPUs = int64(cpuLimitCores * float64(vuln.NanoCPUsPerCore))+ }++ if memoryLimitMB > 0 {+ memoryBytes := memoryLimitMB * vuln.BytesPerMB+ resources.Memory = memoryBytes+ resources.MemorySwap = memoryBytes+ }++ return mo.Some(resources)+}++func (s *VulnerabilityService) runTrivyContainer(+ ctx context.Context,+ dockerClient *client.Client,+ imageID string,+ config *containertypes.Config,+ hostConfig *containertypes.HostConfig,+ registryConfigJSON []byte,+) (*os.File, *os.File, int64, int64, string, error) {+ stdoutFile, err := vuln.CreateLogTempFile("trivy-stdout-*")+ if err != nil {+ return nil, nil, 0, 0, "", errors.WrapIf(err, "failed to create trivy stdout temp file")+ }++ stderrFile, err := vuln.CreateLogTempFile("trivy-stderr-*")+ if err != nil {+ vuln.CleanupLogTempFiles(ctx, stdoutFile)+ return nil, nil, 0, 0, "", errors.WrapIf(err, "failed to create trivy stderr temp file")+ }++ success := false+ containerID := ""+ defer func() {+ if !success {+ vuln.RemoveContainer(ctx, dockerClient, containerID, "failed to cleanup trivy scan container")+ vuln.CleanupLogTempFiles(ctx, stdoutFile, stderrFile)+ }+ }()++ containerID, err = s.createAndStartTrivyContainerInternal(ctx, dockerClient, "single-scan", config, hostConfig, registryConfigJSON)+ if err != nil {+ return nil, nil, 0, 0, "", errors.WrapIf(err, "failed to prepare trivy scan container")+ }++ slog.DebugContext(ctx, "started trivy scan container", "containerId", containerID)+ s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseScanningImage)++ logsCtx, logsCancel := context.WithCancel(ctx)+ logs, err := dockerClient.ContainerLogs(logsCtx, containerID, client.ContainerLogsOptions{+ ShowStdout: true,+ ShowStderr: true,+ Follow: true,+ })+ if err != nil {+ logsCancel()+ return nil, nil, 0, 0, "", errors.WrapIf(err, "failed to stream trivy logs")+ }++ var closeLogsOnce sync.Once+ closeLogs := func() {+ closeLogsOnce.Do(func() {+ logsCancel()+ if closeErr := logs.Close(); closeErr != nil && !dockerutils.IsExpectedStreamEndError(closeErr) {+ slog.WarnContext(ctx, "failed to close trivy scan logs stream", "error", closeErr)+ }+ })+ }+ defer closeLogs()++ logDone := dockerutils.StartStdCopy(logs, stdoutFile, stderrFile)++ startTime := time.Now()+ waitTimeout := vuln.ContainerWaitTimeout(s.getTrivyScanTimeoutArg())+ waitCtx, waitCancel := context.WithTimeout(ctx, waitTimeout)+ defer waitCancel()++ statusCode, err := s.waitForTrivyContainer(waitCtx, dockerClient, containerID)+ if err != nil {+ closeLogs()+ if logErr := dockerutils.WaitStdCopy(logDone); logErr != nil {+ slog.DebugContext(ctx, "trivy log stream ended with error while handling wait failure", "error", logErr)+ }+ return nil, nil, 0, 0, "", err+ }++ // Wait for the log copy goroutine to complete naturally. On some Docker+ // variants (e.g. Synology with Docker API 1.43), the daemon may not have+ // fully flushed container stdout before ContainerWait returns. Cancelling+ // the stream immediately after ContainerWait can truncate the JSON output+ // and cause "unexpected EOF" or empty-output decode errors.+ //+ // Strategy: wait up to logDrainDeadline for the stream to close on its own,+ // then force-close it (for Docker variants that keep the stream open after+ // container exit, e.g. older Docker builds that never send EOF).+ const logDrainDeadline = 30 * time.Second+ logDrainTimer := time.NewTimer(logDrainDeadline)+ defer logDrainTimer.Stop()++ select {+ case logErr := <-logDone:+ closeLogs() // clean up context / connection+ if logErr != nil && !dockerutils.IsExpectedStreamEndError(logErr) {+ return nil, nil, 0, 0, "", errors.WrapIf(logErr, "failed to read trivy logs")+ }+ case <-logDrainTimer.C:+ // Stream did not close on its own; force-close so the goroutine exits.+ slog.DebugContext(ctx, "trivy log stream did not close after container exit; force-closing", "containerId", containerID)+ closeLogs()+ if logErr := dockerutils.WaitStdCopy(logDone); logErr != nil {+ return nil, nil, 0, 0, "", errors.WrapIf(logErr, "failed to read trivy logs")+ }+ }++ duration := time.Since(startTime).Milliseconds()+ success = true+ return stdoutFile, stderrFile, duration, statusCode, containerID, nil+}++func (s *VulnerabilityService) waitForTrivyContainer(+ ctx context.Context,+ dockerClient *client.Client,+ containerID string,+) (int64, error) {+ waitResult := dockerClient.ContainerWait(ctx, containerID, client.ContainerWaitOptions{Condition: containertypes.WaitConditionNotRunning})++ statusCode, err := vuln.AwaitContainerWaitResponse(ctx, waitResult.Result, waitResult.Error)+ if err != nil {+ if errors.Is(err, context.DeadlineExceeded) {+ return 0, errors.WrapIf(err, "trivy container wait timed out")+ }+ if errors.Is(err, context.Canceled) {+ return 0, errors.WrapIf(err, "scan cancelled")+ }+ return 0, errors.WrapIf(err, "trivy container wait failed")+ }++ return statusCode, nil+}++func (s *VulnerabilityService) runTrivyScan(ctx context.Context, trivyImage string, imageName string, imageID string) (*vulnerability.ScanResult, error) {+ slog.DebugContext(ctx,+ "running trivy scan",+ "imageId", imageID,+ "imageName", imageName,+ "trivyImage", trivyImage,+ )++ slotID, releaseSlot, err := s.acquireTrivyScanSlotInternal(ctx)+ if err != nil {+ return nil, err+ }+ defer releaseSlot()++ // Use per-image locking to allow concurrent scans of different images+ defer s.scanLocks.Lock(imageID)()++ dockerClient, err := s.dockerService.GetClient(ctx)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to connect to Docker")+ }++ serverURL, serverToken := s.getTrivyServerConfig()+ var cacheVolume string+ if serverURL == "" {+ cacheVolume, err = s.ensureTrivyCacheVolumeInternal(ctx)+ if err != nil {+ return nil, err+ }+ }++ // Get Trivy config files if they exist+ configContent, ignoreContent := s.getTrivyConfigFiles()++ var cacheDir string+ if serverURL == "" {+ cacheDir = s.getTrivySingleScanCacheDirForSlotInternal(slotID)+ }+ if cacheDir != "" {+ slog.DebugContext(ctx,+ "using slot-based trivy cache dir for scan",+ "slotId", slotID,+ "cacheDir", cacheDir,+ "concurrentScanContainers", s.getTrivyConcurrentScanContainersInternal(),+ )+ }++ outputPath := vuln.NewOutputPath()++ cmdArgs, tempFiles := s.buildTrivyCommandArgs(ctx, imageName, configContent, ignoreContent, cacheDir, outputPath)+ defer vuln.CleanupTempFiles(ctx, tempFiles)++ runtimeOptions, err := s.resolveTrivyRuntimeOptionsInternal(ctx, dockerClient)+ if err != nil {+ return nil, err+ }++ registryConfigJSON := s.buildTrivyRegistryConfigInternal(ctx)+ env := append([]string(nil), runtimeOptions.ContainerEnv...)+ if len(registryConfigJSON) > 0 {+ env = append(env, "DOCKER_CONFIG="+vuln.RegistryConfigDir)+ }+ if serverToken != "" {+ env = append(env, vuln.ServerTokenEnv(serverToken)...)+ }++ config := vuln.BuildContainerConfig(trivyImage, cmdArgs, env)+ resources, cpuSet, applyLimits := s.getTrivyContainerResourceOptionsInternal(ctx, dockerClient)+ runtimeSecurity := s.getTrivyRuntimeSecurityInternal().OrEmpty()+ securityOpts, privileged := runtimeSecurity.securityOpts, runtimeSecurity.privileged+ hostConfig := vuln.BuildHostConfig(+ cacheVolume,+ tempFiles,+ resources,+ cpuSet,+ applyLimits,+ runtimeOptions,+ securityOpts,+ privileged,+ )+ s.setScanPhaseInternal(imageID, vulnerability.ScanPhaseCreatingContainer)++ stdoutFile, stderrFile, duration, statusCode, containerID, err := s.runTrivyContainer(ctx, dockerClient, imageID, config, hostConfig, registryConfigJSON)+ if err != nil {+ return nil, err+ }+ defer vuln.CleanupLogTempFiles(ctx, stdoutFile, stderrFile)+ defer vuln.RemoveContainer(ctx, dockerClient, containerID, "failed to cleanup trivy scan container")++ if statusCode != 0 {+ errMsg := vuln.ReadTempFileExcerpt(stderrFile)+ if errMsg == "" {+ errMsg = vuln.ReadTempFileExcerpt(stdoutFile)+ }+ if errMsg == "" {+ errMsg = fmt.Sprintf("exit status %d", statusCode)+ }+ slog.WarnContext(ctx,+ "trivy scan diagnostics",+ "image", imageName,+ "imageId", imageID,+ "reason", "container non-zero exit",+ "exitCode", statusCode,+ "stdoutBytes", vuln.TempFileSize(stdoutFile),+ "stderrBytes", vuln.TempFileSize(stderrFile),+ )+ return nil, vuln.FormatExitError(statusCode, errMsg, imageName)+ }++ trivyOutput, outputErr := vuln.ReadOutputFromContainerFile(ctx, dockerClient, containerID, outputPath)+ if outputErr != nil {+ slog.WarnContext(ctx,+ "failed to read trivy output file from single-scan container; falling back to stdout parsing",+ "containerId", containerID,+ "outputPath", outputPath,+ "error", outputErr,+ )+ }++ var trivyReport *vulnerability.TrivyReport+ if len(trivyOutput) > 0 {+ trivyReport, err = vuln.DecodeReportFromBytes(trivyOutput)+ } else {+ trivyReport, err = vuln.DecodeReportFromFile(stdoutFile)+ }+ if err != nil {+ if errors.Is(err, io.EOF) {+ errMsg := vuln.ReadTempFileExcerpt(stderrFile)+ if errMsg == "" {+ errMsg = "trivy scan produced no output"+ }+ slog.WarnContext(ctx,+ "trivy scan diagnostics",+ "image", imageName,+ "imageId", imageID,+ "reason", "container empty output",+ "exitCode", statusCode,+ "stdoutBytes", vuln.TempFileSize(stdoutFile),+ "stderrBytes", vuln.TempFileSize(stderrFile),+ )+ return nil, errors.Errorf("trivy scan failed: %s", errMsg)+ }++ slog.WarnContext(ctx,+ "trivy scan diagnostics",+ "image", imageName,+ "imageId", imageID,+ "reason", "container parse error",+ "exitCode", statusCode,+ "stdoutBytes", vuln.TempFileSize(stdoutFile),+ "stderrBytes", vuln.TempFileSize(stderrFile),+ )+ return nil, errors.WrapIf(err, "failed to parse trivy output")+ }++ if trivyReport == nil {+ errMsg := vuln.ReadTempFileExcerpt(stderrFile)+ if errMsg == "" {+ errMsg = "trivy scan produced no output"+ }+ slog.WarnContext(ctx,+ "trivy scan diagnostics",+ "image", imageName,+ "imageId", imageID,+ "reason", "container nil report",+ "exitCode", statusCode,+ "stdoutBytes", vuln.TempFileSize(stdoutFile),+ "stderrBytes", vuln.TempFileSize(stderrFile),+ )+ return nil, errors.Errorf("trivy scan failed: %s", errMsg)+ }++ // Convert Trivy report to our ScanResult format+ result := vulnerability.ConvertTrivyReportToScanResult(trivyReport, imageID, time.Now(), duration)+ result.ScannerVersion = s.GetTrivyVersion(ctx)+ totalVulnerabilities := 0+ if result.Summary != nil {+ totalVulnerabilities = result.Summary.Total+ }++ slog.DebugContext(ctx,+ "trivy scan finished",+ "imageId", imageID,+ "imageName", imageName,+ "durationMs", duration,+ "status", result.Status,+ "vulnerabilities", totalVulnerabilities,+ )++ return result, nil+}diff --git a/backend/api/handlers/webhooks.go b/backend/internal/webhook/handler.gosimilarity index 86%rename from backend/api/handlers/webhooks.gorename to backend/internal/webhook/handler.goindex 8267a6a68..f70d64a5c 100644--- a/backend/api/handlers/webhooks.go+++ b/backend/internal/webhook/handler.go@@ -1,4 +1,4 @@-package handlers+package webhookimport ("context"@@ -7,16 +7,16 @@ import ("emperror.dev/errors""github.com/danielgtaylor/huma/v2"- humamw "github.com/getarcaneapp/arcane/backend/v2/api/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services""github.com/getarcaneapp/arcane/backend/v2/pkg/authz"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/handlerutil""github.com/getarcaneapp/arcane/types/v2/base""github.com/getarcaneapp/arcane/types/v2/webhook")type WebhookHandler struct {- webhookService *services.WebhookService+ webhookService *WebhookService}// --- Input/Output types ---@@ -58,47 +58,47 @@ type UpdateWebhookOutput struct {}// RegisterWebhooks registers the authenticated CRUD routes for webhook management.-func RegisterWebhooks(api huma.API, webhookService *services.WebhookService) {+func RegisterWebhooks(api huma.API, webhookService *WebhookService) {h := &WebhookHandler{webhookService: webhookService}- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "list-webhooks",Method: http.MethodGet,Path: "/environments/{id}/webhooks",Summary: "List webhooks",Description: "List all webhooks configured for this environment",Tags: []string{"Webhooks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermWebhooksList, h.ListWebhooks)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "create-webhook",Method: http.MethodPost,Path: "/environments/{id}/webhooks",Summary: "Create webhook",Description: "Create a webhook that triggers a container or stack update. The token is only returned once.",Tags: []string{"Webhooks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermWebhooksCreate, h.CreateWebhook)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "update-webhook",Method: http.MethodPatch,Path: "/environments/{id}/webhooks/{webhookId}",Summary: "Update webhook",Description: "Update a webhook's enabled state",Tags: []string{"Webhooks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermWebhooksUpdate, h.UpdateWebhook)- humamw.RegisterWithPermission(api, huma.Operation{+ middleware.RegisterWithPermission(api, huma.Operation{OperationID: "delete-webhook",Method: http.MethodDelete,Path: "/environments/{id}/webhooks/{webhookId}",Summary: "Delete webhook",Description: "Delete a webhook by ID",Tags: []string{"Webhooks"},- Security: defaultOperationSecurityInternal(),+ Security: handlerutil.DefaultOperationSecurity(),}, authz.PermWebhooksDelete, h.DeleteWebhook)}@@ -138,13 +138,13 @@ func (h *WebhookHandler) CreateWebhook(ctx context.Context, input *CreateWebhookactor,)if err != nil {- if errors.Is(err, services.ErrWebhookInvalidType) {+ if errors.Is(err, ErrWebhookInvalidType) {return nil, huma.Error400BadRequest("invalid target type, must be 'container', 'project', 'updater', or 'gitops'")}- if errors.Is(err, services.ErrWebhookMissingTarget) {+ if errors.Is(err, ErrWebhookMissingTarget) {return nil, huma.Error400BadRequest("target ID is required for container, project, and gitops webhook types")}- if errors.Is(err, services.ErrWebhookInvalidAction) {+ if errors.Is(err, ErrWebhookInvalidAction) {return nil, huma.Error400BadRequest("invalid action type for target type")}return nil, huma.Error500InternalServerError("failed to create webhook")@@ -179,7 +179,7 @@ func (h *WebhookHandler) UpdateWebhook(ctx context.Context, input *UpdateWebhookwh, err := h.webhookService.UpdateWebhook(ctx, input.WebhookID, input.EnvironmentID, input.Body.Enabled, actor)if err != nil {- if errors.Is(err, services.ErrWebhookNotFound) {+ if errors.Is(err, ErrWebhookNotFound) {return nil, huma.Error404NotFound("webhook not found")}return nil, huma.Error500InternalServerError("failed to update webhook")@@ -199,7 +199,7 @@ func (h *WebhookHandler) DeleteWebhook(ctx context.Context, input *DeleteWebhook}if err := h.webhookService.DeleteWebhook(ctx, input.WebhookID, input.EnvironmentID, actor); err != nil {- if errors.Is(err, services.ErrWebhookNotFound) {+ if errors.Is(err, ErrWebhookNotFound) {return nil, huma.Error404NotFound("webhook not found")}return nil, huma.Error500InternalServerError("failed to delete webhook")diff --git a/backend/internal/webhook/module.go b/backend/internal/webhook/module.gonew file mode 100644index 000000000..62617600b--- /dev/null+++ b/backend/internal/webhook/module.go@@ -0,0 +1,57 @@+// Package webhook owns inbound webhook targets: the tokens that authenticate+// them, the actions they trigger, and the HTTP surface that manages them.+package webhook++import (+ "github.com/danielgtaylor/huma/v2"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitops"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/updater"++ "github.com/getarcaneapp/arcane/backend/v2/internal/container"+ "github.com/getarcaneapp/arcane/backend/v2/internal/database"+)++// Dependencies are the collaborators the webhook domain needs.+type Dependencies struct {+ DB *database.DB+ Container *container.ContainerService+ Updater *updater.UpdaterService+ Project *project.ProjectService+ GitOpsSync *gitops.GitOpsSyncService+ Event *event.EventService+ Environment *environment.EnvironmentService+}++// Module wires the webhook domain and mounts its routes.+type Module struct {+ service *WebhookService+}++// New builds the webhook domain from its dependencies.+func New(deps Dependencies) *Module {+ return &Module{+ service: NewWebhookService(deps.DB, deps.Container, deps.Updater, deps.Project, deps.GitOpsSync, deps.Event, deps.Environment),+ }+}++// Service exposes the webhook service to collaborators that trigger webhooks+// outside the HTTP surface.+func (m *Module) Service() *WebhookService {+ if m == nil {+ return nil+ }+ return m.service+}++// RegisterRoutes mounts the webhook endpoints. A nil module still registers, so+// OpenAPI spec generation can discover the routes without a service graph.+func (m *Module) RegisterRoutes(api huma.API) {+ if m == nil {+ RegisterWebhooks(api, nil)+ return+ }+ RegisterWebhooks(api, m.service)+}diff --git a/backend/internal/services/webhook_service.go b/backend/internal/webhook/webhook.gosimilarity index 90%rename from backend/internal/services/webhook_service.gorename to backend/internal/webhook/webhook.goindex e226edf6b..df865edaa 100644--- a/backend/internal/services/webhook_service.go+++ b/backend/internal/webhook/webhook.go@@ -1,4 +1,4 @@-package services+package webhookimport ("context"@@ -12,13 +12,20 @@ import ("strings""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/gitops"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/updater"+"emperror.dev/errors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/container""github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models""github.com/getarcaneapp/arcane/types/v2""github.com/getarcaneapp/arcane/types/v2/base"- "github.com/getarcaneapp/arcane/types/v2/updater"+ updatertypes "github.com/getarcaneapp/arcane/types/v2/updater"webhooktypes "github.com/getarcaneapp/arcane/types/v2/webhook"libcrypto "go.getarcane.app/sys/crypto""gorm.io/gorm"@@ -41,15 +48,15 @@ const (type WebhookService struct {db *database.DB- containerService *ContainerService- updaterService *UpdaterService- projectService *ProjectService- gitOpsSyncService *GitOpsSyncService- eventService *EventService- environmentService *EnvironmentService+ containerService *container.ContainerService+ updaterService *updater.UpdaterService+ projectService *project.ProjectService+ gitOpsSyncService *gitops.GitOpsSyncService+ eventService *event.EventService+ environmentService *environment.EnvironmentService}-func NewWebhookService(db *database.DB, containerService *ContainerService, updaterService *UpdaterService, projectService *ProjectService, gitOpsSyncService *GitOpsSyncService, eventService *EventService, environmentService *EnvironmentService) *WebhookService {+func NewWebhookService(db *database.DB, containerService *container.ContainerService, updaterService *updater.UpdaterService, projectService *project.ProjectService, gitOpsSyncService *gitops.GitOpsSyncService, eventService *event.EventService, environmentService *environment.EnvironmentService) *WebhookService {return &WebhookService{db: db,containerService: containerService,@@ -214,7 +221,7 @@ func (s *WebhookService) CreateWebhook(ctx context.Context, name, targetType, ac}if s.eventService != nil {- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeWebhookCreate,Severity: models.EventSeveritySuccess,Title: "Webhook created: " + wh.Name,@@ -356,7 +363,7 @@ func (s *WebhookService) DeleteWebhook(ctx context.Context, id, environmentID st}if s.eventService != nil {- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeWebhookDelete,Severity: models.EventSeverityInfo,Title: "Webhook deleted: " + wh.Name,@@ -391,7 +398,7 @@ func (s *WebhookService) UpdateWebhook(ctx context.Context, id, environmentID st}if s.eventService != nil {- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeWebhookUpdate,Severity: models.EventSeveritySuccess,Title: "Webhook updated: " + wh.Name,@@ -410,7 +417,7 @@ func (s *WebhookService) UpdateWebhook(ctx context.Context, id, environmentID st// TriggerByToken looks up a webhook by its raw token and executes the configured action.// Returns an updater result for "updater" webhooks; nil for "project" and "gitops".-func (s *WebhookService) TriggerByToken(ctx context.Context, rawToken string) (*updater.Result, error) {+func (s *WebhookService) TriggerByToken(ctx context.Context, rawToken string) (*updatertypes.Result, error) {prefix, err := parseWebhookPrefixInternal(rawToken)if err != nil {return nil, ErrWebhookInvalid@@ -439,7 +446,7 @@ func (s *WebhookService) TriggerByToken(ctx context.Context, rawToken string) (*return nil, ErrWebhookDisabled}- var result *updater.Result+ var result *updatertypes.ResultactionType, err := resolveWebhookActionTypeInternal(wh.TargetType, wh.ActionType)if err != nil {return nil, err@@ -459,7 +466,7 @@ func (s *WebhookService) TriggerByToken(ctx context.Context, rawToken string) (*return result, nil}-func (s *WebhookService) executeWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updater.Result, error) {+func (s *WebhookService) executeWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updatertypes.Result, error) {// Webhooks are stored and triggered on the manager; actions against remote// environments are forwarded through the environment proxy.if isRemoteWebhookEnvironmentInternal(wh.EnvironmentID) {@@ -483,7 +490,7 @@ func (s *WebhookService) executeWebhookActionInternal(ctx context.Context, wh *m// remoteWebhookRequestInternal maps a webhook action to the env-scoped API// request forwarded to the remote environment. Paths use the agent-local// environment ID, matching the environment proxy's path-rewriting convention.-// wantResult reports whether the response carries an updater.Result payload.+// wantResult reports whether the response carries an updatertypes.Result payload.func remoteWebhookRequestInternal(wh *models.Webhook, actionType string) (method, path string, wantResult bool, err error) {apiPrefix := "/api/environments/" + types.LocalDockerEnvironmentID@@ -528,7 +535,7 @@ func remoteWebhookRequestInternal(wh *models.Webhook, actionType string) (method}}-func (s *WebhookService) executeRemoteWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updater.Result, error) {+func (s *WebhookService) executeRemoteWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updatertypes.Result, error) {if s.environmentService == nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, wh.TargetType, actionType, errors.New("environment service not available"))}@@ -538,9 +545,9 @@ func (s *WebhookService) executeRemoteWebhookActionInternal(ctx context.Context,return nil, err}- var result *updater.Result+ var result *updatertypes.Resultif wantResult {- var out base.ApiResponse[*updater.Result]+ var out base.ApiResponse[*updatertypes.Result]if err := s.environmentService.ProxyJSONRequest(ctx, wh.EnvironmentID, method, path, nil, &out); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, wh.TargetType, actionType, err)}@@ -555,7 +562,7 @@ func (s *WebhookService) executeRemoteWebhookActionInternal(ctx context.Context,return result, nil}-func (s *WebhookService) executeContainerWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updater.Result, error) {+func (s *WebhookService) executeContainerWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updatertypes.Result, error) {containerID, err := s.resolveContainerWebhookTargetIDInternal(ctx, wh)if err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "container", actionType, err)@@ -569,22 +576,22 @@ func (s *WebhookService) executeContainerWebhookActionInternal(ctx context.Conte}return result, nilcase models.WebhookActionTypeStart:- if err := s.containerService.StartContainer(ctx, containerID, systemUser); err != nil {+ if err := s.containerService.StartContainer(ctx, containerID, models.SystemUser); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "container", actionType, err)}return nil, nilcase models.WebhookActionTypeStop:- if err := s.containerService.StopContainer(ctx, containerID, systemUser); err != nil {+ if err := s.containerService.StopContainer(ctx, containerID, models.SystemUser); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "container", actionType, err)}return nil, nilcase models.WebhookActionTypeRestart:- if err := s.containerService.RestartContainer(ctx, containerID, systemUser); err != nil {+ if err := s.containerService.RestartContainer(ctx, containerID, models.SystemUser); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "container", actionType, err)}return nil, nilcase models.WebhookActionTypeRedeploy:- if _, err := s.containerService.RedeployContainer(ctx, containerID, systemUser); err != nil {+ if _, err := s.containerService.RedeployContainer(ctx, containerID, models.SystemUser); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "container", actionType, err)}return nil, nil@@ -656,30 +663,30 @@ func (s *WebhookService) syncWebhookContainerTargetInternal(ctx context.Context,_ = s.db.WithContext(ctx).Model(wh).Updates(updates).Error}-func (s *WebhookService) executeProjectWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updater.Result, error) {+func (s *WebhookService) executeProjectWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updatertypes.Result, error) {switch actionType {case models.WebhookActionTypeUpdate:- if err := s.projectService.UpdateProjectServices(ctx, wh.TargetID, nil, systemUser); err != nil {+ if err := s.projectService.UpdateProjectServices(ctx, wh.TargetID, nil, models.SystemUser); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "project", actionType, err)}return nil, nilcase models.WebhookActionTypeUp:- if err := s.projectService.DeployProject(ctx, wh.TargetID, systemUser, nil); err != nil {+ if err := s.projectService.DeployProject(ctx, wh.TargetID, models.SystemUser, nil); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "project", actionType, err)}return nil, nilcase models.WebhookActionTypeDown:- if err := s.projectService.DownProject(ctx, wh.TargetID, systemUser); err != nil {+ if err := s.projectService.DownProject(ctx, wh.TargetID, models.SystemUser); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "project", actionType, err)}return nil, nilcase models.WebhookActionTypeRestart:- if err := s.projectService.RestartProject(ctx, wh.TargetID, nil, systemUser); err != nil {+ if err := s.projectService.RestartProject(ctx, wh.TargetID, nil, models.SystemUser); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "project", actionType, err)}return nil, nilcase models.WebhookActionTypeRedeploy:- if err := s.projectService.RedeployProject(ctx, wh.TargetID, systemUser, nil); err != nil {+ if err := s.projectService.RedeployProject(ctx, wh.TargetID, models.SystemUser, nil); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "project", actionType, err)}return nil, nil@@ -688,12 +695,12 @@ func (s *WebhookService) executeProjectWebhookActionInternal(ctx context.Context}}-func (s *WebhookService) executeUpdaterWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updater.Result, error) {+func (s *WebhookService) executeUpdaterWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updatertypes.Result, error) {if actionType != models.WebhookActionTypeRun {return nil, ErrWebhookInvalidAction}- result, err := s.updaterService.ApplyPending(ctx, updater.Options{})+ result, err := s.updaterService.ApplyPending(ctx, updatertypes.Options{})if err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "updater", actionType, err)}@@ -701,12 +708,12 @@ func (s *WebhookService) executeUpdaterWebhookActionInternal(ctx context.Contextreturn result, nil}-func (s *WebhookService) executeGitOpsWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updater.Result, error) {+func (s *WebhookService) executeGitOpsWebhookActionInternal(ctx context.Context, wh *models.Webhook, actionType string) (*updatertypes.Result, error) {if actionType != models.WebhookActionTypeSync {return nil, ErrWebhookInvalidAction}- if _, err := s.gitOpsSyncService.PerformSync(ctx, wh.EnvironmentID, wh.TargetID, systemUser); err != nil {+ if _, err := s.gitOpsSyncService.PerformSync(ctx, wh.EnvironmentID, wh.TargetID, models.SystemUser); err != nil {return nil, s.wrapWebhookActionErrorInternal(ctx, wh, "gitops", actionType, err)}@@ -731,7 +738,7 @@ func (s *WebhookService) logWebhookEventInternal(ctx context.Context, wh *modelsif errMsg != "" {description = errMsg}- _, _ = s.eventService.CreateEvent(ctx, CreateEventRequest{+ _, _ = s.eventService.CreateEvent(ctx, event.CreateEventRequest{Type: models.EventTypeWebhookTrigger,Severity: severity,Title: title,diff --git a/backend/internal/services/webhook_service_test.go b/backend/internal/webhook/webhook_test.gosimilarity index 99%rename from backend/internal/services/webhook_service_test.gorename to backend/internal/webhook/webhook_test.goindex 2fe522c49..c0a60d7c9 100644--- a/backend/internal/services/webhook_service_test.go+++ b/backend/internal/webhook/webhook_test.go@@ -1,4 +1,4 @@-package services+package webhookimport ("context"diff --git a/backend/pkg/authz/access_policy_test.go b/backend/pkg/authz/access_policy_test.goindex 5d22bd008..a656f8235 100644--- a/backend/pkg/authz/access_policy_test.go+++ b/backend/pkg/authz/access_policy_test.go@@ -8,7 +8,8 @@ import ("strings""testing"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/search"+"github.com/getarcaneapp/arcane/backend/v2/pkg/authz""github.com/stretchr/testify/require")@@ -160,7 +161,7 @@ func TestAccessSurfaceCategoryRegistryCoversBackendCategories(t *testing.T) {"security": {},}- for _, category := range services.NewSettingsSearchService().GetSettingsCategories() {+ for _, category := range search.NewSettingsSearchService().GetSettingsCategories() {if _, hidden := hiddenSettingsCategories[category.ID]; hidden {continue}@@ -168,7 +169,7 @@ func TestAccessSurfaceCategoryRegistryCoversBackendCategories(t *testing.T) {require.True(t, ok, "settings category %s must have an access surface", category.ID)}- for _, category := range services.NewCustomizeSearchService().GetCustomizeCategories() {+ for _, category := range search.NewCustomizeSearchService().GetCustomizeCategories() {_, ok := findAccessSurfaceInternal("customize.category." + category.ID)require.True(t, ok, "customize category %s must have an access surface", category.ID)}diff --git a/backend/pkg/authz/permissions.go b/backend/pkg/authz/permissions.goindex a7e66566d..9e921eb04 100644--- a/backend/pkg/authz/permissions.go+++ b/backend/pkg/authz/permissions.go@@ -27,7 +27,7 @@ const (// PermRolesList and the role permissions below cover role management// (Create / Update / Delete) and role assignment to users. They are reserved// for global admins and intentionally not exposed as delegated permissions —- // see backend/api/middleware/role.go::RequireGlobalAdmin. Likewise, managing+ // see backend/internal/middleware/role.go::RequireGlobalAdmin. Likewise, managing// OIDC group → role mappings is admin-only because it is effectively another// path for granting role assignments.PermRolesList = "roles:list"diff --git a/backend/pkg/dockerutil/exec.go b/backend/pkg/dockerutil/exec.gonew file mode 100644index 000000000..f340bb097--- /dev/null+++ b/backend/pkg/dockerutil/exec.go@@ -0,0 +1,78 @@+package docker++import (+ "context"+ "io"+ "net"+ "strings"++ "emperror.dev/errors"+ "github.com/moby/moby/api/pkg/stdcopy"+ "github.com/moby/moby/client"+)++// IsExpectedStreamEndError reports whether err is the ordinary way a Docker+// attach/log stream ends rather than a real failure.+func IsExpectedStreamEndError(err error) bool {+ if err == nil {+ return false+ }++ if errors.Is(err, io.EOF) || errors.Is(err, net.ErrClosed) || errors.Is(err, context.Canceled) {+ return true+ }++ errMsg := strings.ToLower(err.Error())+ return strings.Contains(errMsg, "use of closed network connection") ||+ strings.Contains(errMsg, "context canceled") ||+ strings.Contains(errMsg, "broken pipe") ||+ strings.Contains(errMsg, "connection reset by peer")+}++// StartStdCopy demultiplexes a Docker stream into stdout/stderr on a goroutine+// and reports the copy result on the returned channel. Pair with WaitStdCopy.+func StartStdCopy(stream io.Reader, stdout, stderr io.Writer) <-chan error {+ done := make(chan error, 1)+ go func() {+ _, err := stdcopy.StdCopy(stdout, stderr, stream)+ done <- err+ }()+ return done+}++// WaitStdCopy waits for a StartStdCopy goroutine, treating an ordinary stream+// end as success.+func WaitStdCopy(done <-chan error) error {+ err := <-done+ if err == nil || IsExpectedStreamEndError(err) {+ return nil+ }+ return err+}++// ExecInContainer runs an exec in containerID, demultiplexing its output into+// stdout and stderr, and returns the exec's exit code. A non-zero exit code is+// not an error — callers decide how to report it.+func ExecInContainer(ctx context.Context, dockerClient *client.Client, containerID string, opts client.ExecCreateOptions, stdout, stderr io.Writer) (int, error) {+ execResp, err := dockerClient.ExecCreate(ctx, containerID, opts)+ if err != nil {+ return 0, errors.WrapIf(err, "failed to create exec")+ }++ attachResp, err := dockerClient.ExecAttach(ctx, execResp.ID, client.ExecAttachOptions{})+ if err != nil {+ return 0, errors.WrapIf(err, "failed to attach to exec")+ }+ defer attachResp.Close()++ if err := WaitStdCopy(StartStdCopy(attachResp.Reader, stdout, stderr)); err != nil {+ return 0, errors.WrapIf(err, "failed to read exec output")+ }++ inspect, err := dockerClient.ExecInspect(ctx, execResp.ID, client.ExecInspectOptions{})+ if err != nil {+ return 0, errors.WrapIf(err, "failed to inspect exec")+ }++ return inspect.ExitCode, nil+}diff --git a/backend/pkg/dockerutil/exec_test.go b/backend/pkg/dockerutil/exec_test.gonew file mode 100644index 000000000..77e937f49--- /dev/null+++ b/backend/pkg/dockerutil/exec_test.go@@ -0,0 +1,34 @@+package docker++import (+ "errors"+ "fmt"+ "io"+ "net"+ "testing"++ "github.com/stretchr/testify/require"+)++func TestIsExpectedStreamEndError(t *testing.T) {+ tests := []struct {+ name string+ err error+ want bool+ }{+ {name: "nil", err: nil, want: false},+ {name: "eof", err: io.EOF, want: true},+ {name: "wrapped eof", err: fmt.Errorf("wrapped: %w", io.EOF), want: true},+ {name: "net err closed", err: net.ErrClosed, want: true},+ {name: "closed network connection", err: errors.New("read tcp 127.0.0.1:1234->127.0.0.1:5678: use of closed network connection"), want: true},+ {name: "broken pipe", err: errors.New("write: broken pipe"), want: true},+ {name: "connection reset by peer", err: errors.New("read: connection reset by peer"), want: true},+ {name: "unexpected", err: errors.New("some other error"), want: false},+ }++ for _, tt := range tests {+ t.Run(tt.name, func(t *testing.T) {+ require.Equal(t, tt.want, IsExpectedStreamEndError(tt.err))+ })+ }+}diff --git a/backend/pkg/dockerutil/mount_utils.go b/backend/pkg/dockerutil/mount_utils.goindex 6a8566609..525a18b26 100644--- a/backend/pkg/dockerutil/mount_utils.go+++ b/backend/pkg/dockerutil/mount_utils.go@@ -13,28 +13,6 @@ import ("go.getarcane.app/sys/cgroup")-// CurrentContainerInspectTarget resolves the identifier used to inspect Arcane's current container.-func CurrentContainerInspectTarget(currentContainerID func() (string, error), hostname func() (string, error)) (string, error) {- if currentContainerID != nil {- if containerID, err := currentContainerID(); err == nil {- if containerID = strings.TrimSpace(containerID); containerID != "" {- return containerID, nil- }- }- }-- if hostname == nil {- hostname = os.Hostname- }-- value, err := hostname()- if err != nil {- return "", err- }-- return strings.TrimSpace(value), nil-}-// MountForCurrentContainerSubpath inspects the current container, finds the// existing mount whose destination covers containerPath, and returns a Mount// suitable for use in another container creation that exposes the same data@@ -45,7 +23,7 @@ func MountForCurrentContainerSubpath(ctx context.Context, dockerCli *client.Clieif dockerCli == nil {return nil, nil}- inspectTarget, err := CurrentContainerInspectTarget(cgroup.CurrentContainerID, os.Hostname)+ inspectTarget, err := libarcane.CurrentContainerInspectTarget(cgroup.CurrentContainerID, os.Hostname)if err != nil {return nil, err}diff --git a/backend/pkg/dockerutil/mount_utils_test.go b/backend/pkg/dockerutil/mount_utils_test.goindex f75e9c7e3..6ebd015c4 100644--- a/backend/pkg/dockerutil/mount_utils_test.go+++ b/backend/pkg/dockerutil/mount_utils_test.go@@ -1,7 +1,6 @@package dockerimport (- "errors""testing"containertypes "github.com/moby/moby/api/types/container"@@ -9,49 +8,6 @@ import ("github.com/stretchr/testify/require")-func TestCurrentContainerInspectTarget(t *testing.T) {- t.Run("prefers detected container id over hostname", func(t *testing.T) {- target, err := CurrentContainerInspectTarget(- func() (string, error) { return "0123456789ab", nil },- func() (string, error) { return "rpi4", nil },- )-- require.NoError(t, err)- require.Equal(t, "0123456789ab", target)- })-- t.Run("falls back to hostname when container id unavailable", func(t *testing.T) {- target, err := CurrentContainerInspectTarget(- func() (string, error) { return "", errors.New("no container id found") },- func() (string, error) { return "rpi4", nil },- )-- require.NoError(t, err)- require.Equal(t, "rpi4", target)- })-- t.Run("trims whitespace from detected container id", func(t *testing.T) {- target, err := CurrentContainerInspectTarget(- func() (string, error) { return " 0123456789ab ", nil },- func() (string, error) { return "rpi4", nil },- )-- require.NoError(t, err)- require.Equal(t, "0123456789ab", target)- })-- t.Run("returns hostname error when fallback fails", func(t *testing.T) {- target, err := CurrentContainerInspectTarget(- func() (string, error) { return "", errors.New("no container id found") },- func() (string, error) { return "", errors.New("hostname unavailable") },- )-- require.Error(t, err)- require.Contains(t, err.Error(), "hostname unavailable")- require.Empty(t, target)- })-}-func TestMountForDestination(t *testing.T) {tests := []struct {name stringdiff --git a/backend/pkg/libarcane/edge/tls.go b/backend/pkg/libarcane/edge/tls.goindex 9c928343e..0d3b60088 100644--- a/backend/pkg/libarcane/edge/tls.go+++ b/backend/pkg/libarcane/edge/tls.go@@ -25,7 +25,6 @@ import ("emperror.dev/errors"- "github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"certgen "github.com/getarcaneapp/arcane/cli/v2/pkg/generate""go.getarcane.app/sys/atomic"@@ -126,8 +125,7 @@ func PrepareManagerMTLSAssetsWithContext(ctx context.Context, cfg *Config) errorreturn nil}-// GeneratedManagerMTLSCAPath returns the configured or Arcane-managed manager CA path without creating assets.-func GeneratedManagerMTLSCAPath(cfg *Config) (string, error) {+func generatedManagerMTLSCAPathInternal(cfg *Config) (string, error) {if cfg == nil {return "", errors.New("edge config is required")}@@ -141,6 +139,25 @@ func GeneratedManagerMTLSCAPath(cfg *Config) (string, error) {return filepath.Join(assetsDir, generatedMTLSCACertFileName), nil}+// AvailableManagerMTLSCAPath resolves an existing manager CA certificate.+func AvailableManagerMTLSCAPath(cfg *Config) (string, error) {+ if cfg == nil {+ return "", errors.New("edge config is required")+ }+ if NormalizeEdgeMTLSMode(cfg.EdgeMTLSMode) == EdgeMTLSModeDisabled {+ return "", errors.New("edge mTLS is disabled")+ }++ caPath, err := generatedManagerMTLSCAPathInternal(cfg)+ if err != nil {+ return "", errors.WrapIf(err, "resolve edge mTLS CA path")+ }+ if _, err := os.Stat(caPath); err != nil {+ return "", errors.WrapIf(err, "stat edge mTLS CA")+ }+ return caPath, nil+}+// GenerateManagerClientMTLSAssetsWithContext creates or loads the generated CA and per-environment client certificate bundle.func GenerateManagerClientMTLSAssetsWithContext(ctx context.Context, cfg *Config, envID string, envName string) (*GeneratedMTLSAssets, error) {if !shouldUseGeneratedManagerCAInternal(cfg) {@@ -345,13 +362,13 @@ func enrollAgentMTLSAssetsInternal(ctx context.Context, cfg *Config, assetsDir,return errors.New("edge mTLS enrollment response did not include any files")}- if err := os.MkdirAll(assetsDir, common.DirPerm); err != nil {+ if err := os.MkdirAll(assetsDir, utils.DirPerm); err != nil {return errors.WrapIf(err, "failed to create edge mTLS asset dir")}for _, file := range enrollResp.Files {fileName := filepath.Base(file.Name)targetPath := filepath.Join(assetsDir, fileName)- perm := common.FilePerm+ perm := utils.FilePermif strings.TrimSpace(file.Permissions) == "0600" {perm = 0o600}@@ -683,7 +700,7 @@ func edgeAgentMTLSAssetsDirInternal(cfg *Config) (string, error) {}func ensureManagerCAInternal(ctx context.Context, assetsDir string) (string, string, bool, error) {- if err := os.MkdirAll(assetsDir, common.DirPerm); err != nil {+ if err := os.MkdirAll(assetsDir, utils.DirPerm); err != nil {return "", "", false, errors.WrapIf(err, "failed to create edge mTLS assets dir")}@@ -716,7 +733,7 @@ func ensureManagerCAInternal(ctx context.Context, assetsDir string) (string, strreturn "", "", false, errors.WrapIf(err, "failed to create CA certificate")}- if err := writePEMFileInternal(caCertPath, "CERTIFICATE", certDER, common.FilePerm); err != nil {+ if err := writePEMFileInternal(caCertPath, "CERTIFICATE", certDER, utils.FilePerm); err != nil {return "", "", false, err}caKeyDER, err := x509.MarshalECPrivateKey(privateKey)@@ -776,7 +793,7 @@ func ensureClientCertificateInternal(ctx context.Context, assetsDir string, envIsafeEnvID := generatedAssetNameSanitizer.ReplaceAllString(strings.TrimSpace(envID), "_")clientDir := filepath.Join(assetsDir, generatedClientMTLSSubdir, safeEnvID)- if err := os.MkdirAll(clientDir, common.DirPerm); err != nil {+ if err := os.MkdirAll(clientDir, utils.DirPerm); err != nil {return "", "", false, errors.WrapIf(err, "failed to create client cert dir")}unlock, err := lockEdgeMTLSPathInternal(ctx, clientDir, ".client.lock")@@ -814,7 +831,7 @@ func ensureClientCertificateInternal(ctx context.Context, assetsDir string, envIreturn "", "", false, errors.WrapIf(err, "failed to create client certificate")}- if err := writePEMFileInternal(clientCertPath, "CERTIFICATE", certDER, common.FilePerm); err != nil {+ if err := writePEMFileInternal(clientCertPath, "CERTIFICATE", certDER, utils.FilePerm); err != nil {return "", "", false, err}clientKeyDER, err := x509.MarshalECPrivateKey(privateKey)diff --git a/backend/pkg/libarcane/edge/tls_test.go b/backend/pkg/libarcane/edge/tls_test.goindex e822de6ae..7c5799a70 100644--- a/backend/pkg/libarcane/edge/tls_test.go+++ b/backend/pkg/libarcane/edge/tls_test.go@@ -19,7 +19,7 @@ import ("testing""time"- "github.com/getarcaneapp/arcane/backend/v2/internal/common"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils""github.com/stretchr/testify/require"libcrypto "go.getarcane.app/sys/crypto""google.golang.org/grpc/credentials"@@ -192,7 +192,7 @@ func TestEnsureAgentMTLSAssets_UsesDownloadedCAPathWhenPresent(t *testing.T) {require.NotNil(t, generated)for _, file := range generated.Files {targetPath := filepath.Join(assetsDir, filepath.Base(file.Name))- perm := common.FilePerm+ perm := utils.FilePermif file.Permissions == "0600" {perm = 0o600}@@ -432,7 +432,7 @@ func TestAgentMTLSAssetsNeedEnrollmentInternal_RenewsExpiredCertificate(t *testifor _, file := range assets.Files {targetPath := filepath.Join(assetsDir, filepath.Base(file.Name))- perm := common.FilePerm+ perm := utils.FilePermif file.Permissions == "0600" {perm = 0o600}diff --git a/backend/pkg/libarcane/internal_resource.go b/backend/pkg/libarcane/internal_resource.goindex a2325209d..87944a39e 100644--- a/backend/pkg/libarcane/internal_resource.go+++ b/backend/pkg/libarcane/internal_resource.go@@ -1,6 +1,17 @@package libarcane-import "strings"+import (+ "context"+ "log/slog"+ "os"+ "strings"++ "emperror.dev/errors"+ "github.com/moby/moby/api/types/container"+ "github.com/moby/moby/client"+ "go.getarcane.app/sys/cgroup"+ "go.getarcane.app/updater/labels"+)// InternalResourceLabel marks containers used for Arcane utilities, e.g. temp containers used for viewing volume files.const InternalResourceLabel = "com.getarcaneapp.internal.resource"@@ -19,3 +30,90 @@ func IsInternalContainer(labels map[string]string) bool {}return false}++// LabelArcaneUpgrader marks the short-lived helper container that performs an+// Arcane self-upgrade. It carries the Arcane label too, so container lookups+// must skip it to avoid mistaking it for Arcane itself.+const LabelArcaneUpgrader = "com.getarcaneapp.arcane.upgrader"++// FindArcaneContainerIDByLabel locates Arcane's own container through the+// Arcane label. Running containers win; otherwise the first match is returned.+// The upgrader helper is never a candidate. Returns "" when nothing matches.+func FindArcaneContainerIDByLabel(ctx context.Context, dockerClient *client.Client) string {+ if dockerClient == nil {+ return ""+ }++ filters := make(client.Filters)+ filters = filters.Add("label", labels.LabelArcane+"=true")++ list, err := dockerClient.ContainerList(ctx, client.ContainerListOptions{All: true, Filters: filters})+ if err != nil {+ slog.DebugContext(ctx, "find arcane container by label: list failed", "error", err)+ return ""+ }++ var fallbackID string+ for _, c := range list.Items {+ if v, ok := c.Labels[LabelArcaneUpgrader]; ok && strings.EqualFold(strings.TrimSpace(v), "true") {+ continue+ }+ if c.State == container.StateRunning {+ return c.ID+ }+ if fallbackID == "" {+ fallbackID = c.ID+ }+ }++ slog.DebugContext(ctx, "find arcane container by label", "candidates", len(list.Items), "selected", fallbackID)+ return fallbackID+}++// InspectCurrentArcaneContainer inspects Arcane's own container, preferring+// cgroup/hostname self-detection and falling back to the Arcane label.+func InspectCurrentArcaneContainer(ctx context.Context, dockerClient *client.Client) (*container.InspectResponse, error) {+ if dockerClient == nil {+ return nil, errors.New("docker client is not available")+ }++ if target, err := CurrentContainerInspectTarget(cgroup.CurrentContainerID, os.Hostname); err == nil && target != "" {+ if inspect, inspectErr := ContainerInspectWithCompatibility(ctx, dockerClient, target, client.ContainerInspectOptions{}); inspectErr == nil {+ return &inspect.Container, nil+ }+ }++ containerID := FindArcaneContainerIDByLabel(ctx, dockerClient)+ if containerID == "" {+ return nil, errors.New("could not detect Arcane container")+ }++ inspect, err := ContainerInspectWithCompatibility(ctx, dockerClient, containerID, client.ContainerInspectOptions{})+ if err != nil {+ return nil, errors.WrapIf(err, "inspect Arcane container")+ }++ return &inspect.Container, nil+}++// CurrentContainerInspectTarget resolves the identifier used to inspect Arcane's current container.+func CurrentContainerInspectTarget(currentContainerID func() (string, error), hostname func() (string, error)) (string, error) {+ if currentContainerID != nil {+ if containerID, err := currentContainerID(); err == nil {+ if containerID = strings.TrimSpace(containerID); containerID != "" {+ return containerID, nil+ }+ }+ }++ if hostname == nil {+ hostname = os.Hostname+ }++ value, err := hostname()+ if err != nil {+ return "", err+ }++ return strings.TrimSpace(value), nil+}diff --git a/backend/pkg/libarcane/internal_resource_test.go b/backend/pkg/libarcane/internal_resource_test.gonew file mode 100644index 000000000..34302de73--- /dev/null+++ b/backend/pkg/libarcane/internal_resource_test.go@@ -0,0 +1,51 @@+package libarcane++import (+ "testing"++ "emperror.dev/errors"+ "github.com/stretchr/testify/require"+)++func TestCurrentContainerInspectTarget(t *testing.T) {+ t.Run("prefers detected container id over hostname", func(t *testing.T) {+ target, err := CurrentContainerInspectTarget(+ func() (string, error) { return "0123456789ab", nil },+ func() (string, error) { return "rpi4", nil },+ )++ require.NoError(t, err)+ require.Equal(t, "0123456789ab", target)+ })++ t.Run("falls back to hostname when container id unavailable", func(t *testing.T) {+ target, err := CurrentContainerInspectTarget(+ func() (string, error) { return "", errors.New("no container id found") },+ func() (string, error) { return "rpi4", nil },+ )++ require.NoError(t, err)+ require.Equal(t, "rpi4", target)+ })++ t.Run("trims whitespace from detected container id", func(t *testing.T) {+ target, err := CurrentContainerInspectTarget(+ func() (string, error) { return " 0123456789ab ", nil },+ func() (string, error) { return "rpi4", nil },+ )++ require.NoError(t, err)+ require.Equal(t, "0123456789ab", target)+ })++ t.Run("returns hostname error when fallback fails", func(t *testing.T) {+ target, err := CurrentContainerInspectTarget(+ func() (string, error) { return "", errors.New("no container id found") },+ func() (string, error) { return "", errors.New("hostname unavailable") },+ )++ require.Error(t, err)+ require.Contains(t, err.Error(), "hostname unavailable")+ require.Empty(t, target)+ })+}diff --git a/backend/pkg/libarcane/volumehelper/container_files.go b/backend/pkg/libarcane/volumehelper/container_files.gonew file mode 100644index 000000000..af50d4f23--- /dev/null+++ b/backend/pkg/libarcane/volumehelper/container_files.go@@ -0,0 +1,93 @@+package volumehelper++import (+ "archive/tar"+ "context"+ "io"+ "strings"++ "emperror.dev/errors"+ "github.com/moby/moby/api/types/container"+ "github.com/moby/moby/client"++ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"+)++type cleanupReadCloserInternal struct {+ io.Reader+ io.Closer++ cleanup func()+}++func IsLegacyHelperContainer(c container.Summary) bool {+ if !libarcane.IsInternalContainer(c.Labels) {+ return false+ }++ command := strings.ToLower(c.Command)+ if !strings.Contains(command, "sleep") || !strings.Contains(command, "infinity") {+ return false+ }++ for _, m := range c.Mounts {+ if m.Destination == "/volume" {+ return true+ }+ }++ return false+}++func IsHelperContainer(c container.Summary) bool {+ if IsLegacyHelperContainer(c) {+ return true+ }+ if !libarcane.IsInternalContainer(c.Labels) {+ return false+ }++ return strings.EqualFold(c.Labels[ContainerLabel], "true")+}++func (c *cleanupReadCloserInternal) Close() error {+ err := c.Closer.Close()+ c.cleanup()+ return err+}++func DownloadFileFromContainer(+ ctx context.Context,+ dockerClient *client.Client,+ containerID string,+ containerPath string,+ cleanup func(),+) (io.ReadCloser, int64, error) {+ copyResult, err := dockerClient.CopyFromContainer(ctx, containerID, client.CopyFromContainerOptions{+ SourcePath: containerPath,+ })+ if err != nil {+ cleanup()+ return nil, 0, errors.WrapIf(err, "failed to download")+ }+ reader := copyResult.Content++ tr := tar.NewReader(reader)+ hdr, err := tr.Next()+ if err != nil {+ _ = reader.Close()+ cleanup()+ return nil, 0, errors.WrapIf(err, "failed to read tar stream")+ }+ if hdr.FileInfo().IsDir() {+ _ = reader.Close()+ cleanup()+ return nil, 0, errors.New("path is a directory")+ }++ return &cleanupReadCloserInternal{+ Reader: tr,+ Closer: reader,+ cleanup: cleanup,+ }, hdr.Size, nil+}diff --git a/backend/pkg/libarcane/volumehelper/container_files_test.go b/backend/pkg/libarcane/volumehelper/container_files_test.gonew file mode 100644index 000000000..c17df33ca--- /dev/null+++ b/backend/pkg/libarcane/volumehelper/container_files_test.go@@ -0,0 +1,101 @@+package volumehelper++import (+ "testing"++ "github.com/moby/moby/api/types/container"+ "github.com/stretchr/testify/require"++ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane"+)++func TestIsLegacyVolumeHelperContainerInternal(t *testing.T) {+ tests := []struct {+ name string+ summary container.Summary+ want bool+ }{+ {+ name: "legacy helper signature matches",+ summary: container.Summary{+ Labels: map[string]string{+ libarcane.InternalResourceLabel: "true",+ },+ Command: "sleep infinity",+ Mounts: []container.MountPoint{+ {Destination: "/volume"},+ },+ },+ want: true,+ },+ {+ name: "internal trivy-like helper is not treated as legacy volume helper",+ summary: container.Summary{+ Labels: map[string]string{+ libarcane.InternalResourceLabel: "true",+ },+ Command: "trivy image --quiet alpine:latest",+ Mounts: []container.MountPoint{+ {Destination: "/var/run/docker.sock"},+ },+ },+ want: false,+ },+ }++ for _, tt := range tests {+ t.Run(tt.name, func(t *testing.T) {+ require.Equal(t, tt.want, IsLegacyHelperContainer(tt.summary))+ })+ }+}++func TestIsVolumeHelperContainerInternal_UsesExplicitHelperLabel(t *testing.T) {+ tests := []struct {+ name string+ summary container.Summary+ want bool+ }{+ {+ name: "new helper label matches",+ summary: container.Summary{+ Labels: map[string]string{+ libarcane.InternalResourceLabel: "true",+ ContainerLabel: "true",+ },+ },+ want: true,+ },+ {+ name: "generic internal volume mount does not match",+ summary: container.Summary{+ Labels: map[string]string{+ libarcane.InternalResourceLabel: "true",+ },+ Mounts: []container.MountPoint{+ {Destination: "/volume"},+ },+ },+ want: false,+ },+ {+ name: "legacy helper still matches",+ summary: container.Summary{+ Labels: map[string]string{+ libarcane.InternalResourceLabel: "true",+ },+ Command: "sleep infinity",+ Mounts: []container.MountPoint{+ {Destination: "/volume"},+ },+ },+ want: true,+ },+ }++ for _, tt := range tests {+ t.Run(tt.name, func(t *testing.T) {+ require.Equal(t, tt.want, IsHelperContainer(tt.summary))+ })+ }+}diff --git a/backend/pkg/libarcane/volumes/volume_rename_test.go b/backend/pkg/libarcane/volumes/volume_rename_test.goindex e62507e34..d4bbebc99 100644--- a/backend/pkg/libarcane/volumes/volume_rename_test.go+++ b/backend/pkg/libarcane/volumes/volume_rename_test.go@@ -13,10 +13,29 @@ import ("github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""github.com/moby/moby/api/types/container""github.com/moby/moby/api/types/volume"+ "github.com/moby/moby/client""github.com/stretchr/testify/assert""github.com/stretchr/testify/require")+func newTestDockerClient(t *testing.T, server *httptest.Server) *client.Client {+ t.Helper()++ httpClient := server.Client()+ cli, err := client.New(+ client.WithHost(server.URL),+ client.WithAPIVersion("1.41"),+ client.WithHTTPClient(httpClient),+ )+ require.NoError(t, err)++ t.Cleanup(func() {+ _ = cli.Close()+ })++ return cli+}+func TestBuildProjectRenamedVolumeConfigInternal(t *testing.T) {source := composetypes.VolumeConfig{Name: "nginx_data",diff --git a/backend/pkg/libarcane/volumes/volume_rename_test_helpers_test.go b/backend/pkg/libarcane/volumes/volume_rename_test_helpers_test.godeleted file mode 100644index 76e557891..000000000--- a/backend/pkg/libarcane/volumes/volume_rename_test_helpers_test.go+++ /dev/null@@ -1,27 +0,0 @@-package volumes--import (- "net/http/httptest"- "testing"-- "github.com/moby/moby/client"- "github.com/stretchr/testify/require"-)--func newTestDockerClient(t *testing.T, server *httptest.Server) *client.Client {- t.Helper()-- httpClient := server.Client()- cli, err := client.New(- client.WithHost(server.URL),- client.WithAPIVersion("1.41"),- client.WithHTTPClient(httpClient),- )- require.NoError(t, err)-- t.Cleanup(func() {- _ = cli.Close()- })-- return cli-}diff --git a/backend/pkg/libarcane/vuln/driver.go b/backend/pkg/libarcane/vuln/driver.gonew file mode 100644index 000000000..84e304d6c--- /dev/null+++ b/backend/pkg/libarcane/vuln/driver.go@@ -0,0 +1,860 @@+package vuln++import (+ "archive/tar"+ "bytes"+ "context"+ "encoding/json/jsontext"+ "encoding/json/v2"+ "fmt"+ "io"+ "log/slog"+ "math"+ "os"+ "path/filepath"+ "sort"+ "strconv"+ "strings"+ "time"++ "emperror.dev/errors"+ cerrdefs "github.com/containerd/errdefs"+ dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"+ "github.com/moby/moby/api/pkg/stdcopy"+ containertypes "github.com/moby/moby/api/types/container"+ mounttypes "github.com/moby/moby/api/types/mount"+ "github.com/moby/moby/client"++ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"+ "github.com/getarcaneapp/arcane/types/v2/vulnerability"+)++const (+ // DefaultNetworkMode is the network the scan container joins when nothing+ // more specific is configured or detected.+ DefaultNetworkMode = "bridge"+ // CacheMountTarget is where the shared Trivy cache volume is mounted inside+ // the scan container.+ CacheMountTarget = "/root/.cache"+ // RegistryConfigDir is where the generated registry auth config lands inside+ // the scan container.+ RegistryConfigDir = "/tmp/arcane-registry-auth"++ // NanoCPUsPerCore and BytesPerMB convert configured limits to Docker units.+ NanoCPUsPerCore = int64(1_000_000_000)+ BytesPerMB = int64(1024 * 1024)++ outputPathPrefixInternal = "/tmp/arcane-trivy-result-"+ registryConfigCopyDestInternal = "/tmp"+ registryConfigTarNameInternal = "arcane-registry-auth/config.json"+ errorExcerptSizeInternal = int64(32 * 1024)+)++type RuntimeOptions struct {+ ContainerEnv []string+ Mounts []mounttypes.Mount+ NetworkMode string+}++func ResolveUnixSocketSource(+ ctx context.Context,+ socketPath string,+ discoverHostPath func(context.Context, string) (string, error),+ isRunningInDocker func() bool,+) (string, error) {+ socketPath = strings.TrimSpace(socketPath)+ if socketPath == "" {+ return "", errors.New("unix docker host is missing a socket path")+ }++ if discoverHostPath != nil {+ hostPath, err := discoverHostPath(ctx, socketPath)+ if err == nil && strings.TrimSpace(hostPath) != "" {+ return strings.TrimSpace(hostPath), nil+ }+ if err != nil && (isRunningInDocker == nil || isRunningInDocker()) {+ return "", errors.WrapIff(err, "failed to resolve socket path %q", socketPath)+ }+ }++ if isRunningInDocker != nil && isRunningInDocker() {+ return "", errors.Errorf("failed to resolve socket path %q to a daemon-visible host path", socketPath)+ }++ return socketPath, nil+}++func SelectAutoNetworkMode(inspect *containertypes.InspectResponse) string {+ if inspect == nil {+ return DefaultNetworkMode+ }++ if inspect.HostConfig != nil {+ networkMode := strings.TrimSpace(string(inspect.HostConfig.NetworkMode))+ if networkMode != "" && networkMode != "default" && networkMode != DefaultNetworkMode &&+ !containertypes.NetworkMode(networkMode).IsContainer() {+ return networkMode+ }+ }++ if inspect.NetworkSettings != nil && len(inspect.NetworkSettings.Networks) > 0 {+ networkNames := make([]string, 0, len(inspect.NetworkSettings.Networks))+ for networkName := range inspect.NetworkSettings.Networks {+ networkName = strings.TrimSpace(networkName)+ if networkName != "" {+ networkNames = append(networkNames, networkName)+ }+ }+ sort.Strings(networkNames)++ for _, networkName := range networkNames {+ if !dockerutils.IsDefaultNetwork(networkName) {+ return networkName+ }+ }++ for _, networkName := range networkNames {+ if containertypes.NetworkMode(networkName).IsContainer() {+ continue+ }+ if networkName == "host" || networkName == "none" || networkName == DefaultNetworkMode {+ return networkName+ }+ }+ }++ if inspect.HostConfig != nil {+ networkMode := strings.TrimSpace(string(inspect.HostConfig.NetworkMode))+ if networkMode != "" && networkMode != "default" && !containertypes.NetworkMode(networkMode).IsContainer() {+ return networkMode+ }+ }++ return DefaultNetworkMode+}++func NewOutputPath() string {+ return outputPathPrefixInternal + strconv.FormatInt(time.Now().UnixNano(), 10) + ".json"+}++func CleanupTempFiles(ctx context.Context, tempFiles []string) {+ for _, f := range tempFiles {+ if err := os.Remove(f); err != nil {+ slog.WarnContext(ctx, "failed to remove trivy temp file", "path", f, "error", err)+ }+ }+}++func BuildBatchHostConfig(+ cacheVolume string,+ runtimeOptions RuntimeOptions,+ securityOpts []string,+ privileged bool,+) *containertypes.HostConfig {+ mounts := append([]mounttypes.Mount{}, runtimeOptions.Mounts...)+ if strings.TrimSpace(cacheVolume) != "" {+ mounts = append(mounts, mounttypes.Mount{+ Type: mounttypes.TypeVolume,+ Source: cacheVolume,+ Target: CacheMountTarget,+ })+ }++ hostConfig := &containertypes.HostConfig{+ NetworkMode: containertypes.NetworkMode(SelectDefaultNetworkMode(runtimeOptions.NetworkMode)),+ Mounts: mounts,+ }++ ApplyRuntimeSecurity(hostConfig, securityOpts, privileged)+ return hostConfig+}++func BuildHostConfig(+ cacheVolume string,+ tempFiles []string,+ resources containertypes.Resources,+ cpuSet string,+ applyLimits bool,+ runtimeOptions RuntimeOptions,+ securityOpts []string,+ privileged bool,+) *containertypes.HostConfig {+ mounts := append([]mounttypes.Mount{}, runtimeOptions.Mounts...)+ if strings.TrimSpace(cacheVolume) != "" {+ mounts = append(mounts, mounttypes.Mount{+ Type: mounttypes.TypeVolume,+ Source: cacheVolume,+ Target: CacheMountTarget,+ })+ }++ hostConfig := &containertypes.HostConfig{+ NetworkMode: containertypes.NetworkMode(SelectDefaultNetworkMode(runtimeOptions.NetworkMode)),+ // Keep single-scan containers until explicit cleanup so we can reliably+ // wait for completion and collect logs across Docker variants.+ AutoRemove: false,+ Mounts: mounts,+ }++ ApplyRuntimeSecurity(hostConfig, securityOpts, privileged)+ ApplyContainerResources(hostConfig, resources, cpuSet, applyLimits)++ addTempFileMountsInternal(hostConfig, tempFiles)+ return hostConfig+}++func SelectDefaultNetworkMode(networkMode string) string {+ networkMode = strings.TrimSpace(networkMode)+ if networkMode == "" {+ return DefaultNetworkMode+ }+ return networkMode+}++func ApplyRuntimeSecurity(hostConfig *containertypes.HostConfig, securityOpts []string, privileged bool) {+ if hostConfig == nil {+ return+ }++ if len(securityOpts) > 0 {+ hostConfig.SecurityOpt = append([]string(nil), securityOpts...)+ }+ hostConfig.Privileged = privileged+}++func ApplyContainerResources(hostConfig *containertypes.HostConfig, resources containertypes.Resources, cpuSet string, applyLimits bool) {+ if hostConfig == nil || !applyLimits {+ return+ }++ if cpuSet != "" {+ resources.NanoCPUs = 0+ resources.CpusetCpus = cpuSet+ }++ hostConfig.Resources = resources+}++func addTempFileMountsInternal(hostConfig *containertypes.HostConfig, tempFiles []string) {+ for _, tempFile := range tempFiles {+ switch {+ case strings.Contains(tempFile, "trivy-config"):+ hostConfig.Mounts = append(hostConfig.Mounts, mounttypes.Mount{+ Type: mounttypes.TypeBind,+ Source: tempFile,+ Target: "/tmp/trivy-config.yaml",+ ReadOnly: true,+ })+ case strings.Contains(tempFile, "trivy-ignore"):+ hostConfig.Mounts = append(hostConfig.Mounts, mounttypes.Mount{+ Type: mounttypes.TypeBind,+ Source: tempFile,+ Target: "/tmp/trivy-ignore",+ ReadOnly: true,+ })+ }+ }+}++func CreateLogTempFile(prefix string) (*os.File, error) {+ // Try the default system temp dir first.+ file, primaryErr := os.CreateTemp("", prefix)+ if primaryErr == nil {+ return file, nil+ }+ // /tmp is not writable (e.g. hardened container). Fall back to a+ // per-user cache directory so Trivy scans can still run.+ cacheDir, err := os.UserCacheDir()+ if err != nil {+ return nil, errors.WrapIff(err, "failed to create trivy temp file (primary: %s) and user cache dir unavailable", primaryErr.Error())+ }+ fallbackDir := filepath.Join(cacheDir, "arcane", "trivy-tmp")+ if err := os.MkdirAll(fallbackDir, 0o700); err != nil {+ return nil, errors.WrapIff(err, "failed to create trivy fallback temp dir %s (primary: %s)", fallbackDir, primaryErr.Error())+ }+ fallbackFile, err := os.CreateTemp(fallbackDir, prefix)+ if err != nil {+ return nil, errors.WrapIff(err, "failed to create trivy temp file in fallback dir %s (primary: %s)", fallbackDir, primaryErr.Error())+ }+ return fallbackFile, nil+}++func CleanupLogTempFiles(ctx context.Context, files ...*os.File) {+ for _, file := range files {+ if file == nil {+ continue+ }++ path := file.Name()+ if err := file.Close(); err != nil {+ slog.WarnContext(ctx, "failed to close trivy temp file", "path", path, "error", err)+ }++ if path == "" {+ continue+ }++ if err := os.Remove(path); err != nil {+ slog.WarnContext(ctx, "failed to remove trivy temp file", "path", path, "error", err)+ }+ }+}++func MountSummaries(hostConfig *containertypes.HostConfig) []string {+ if hostConfig == nil || len(hostConfig.Mounts) == 0 {+ return nil+ }++ summaries := make([]string, 0, len(hostConfig.Mounts))+ for _, mount := range hostConfig.Mounts {+ source := mount.Source+ if source == "" {+ source = string(mount.Type)+ }+ summaries = append(summaries, fmt.Sprintf("%s:%s", source, mount.Target))+ }++ return summaries+}++func LogContainerStartupRequest(ctx context.Context, scope string, config *containertypes.Config, hostConfig *containertypes.HostConfig) {+ if config == nil {+ return+ }++ resources := containertypes.Resources{}+ autoRemove := false+ var mounts []string+ networkMode := ""+ var securityOpts []string+ privileged := false+ if hostConfig != nil {+ resources = hostConfig.Resources+ autoRemove = hostConfig.AutoRemove+ mounts = MountSummaries(hostConfig)+ networkMode = string(hostConfig.NetworkMode)+ securityOpts = append([]string(nil), hostConfig.SecurityOpt...)+ privileged = hostConfig.Privileged+ }++ slog.DebugContext(ctx,+ "preparing trivy container startup",+ "scope", scope,+ "image", config.Image,+ "cmd", config.Cmd,+ "entrypoint", config.Entrypoint,+ "networkMode", networkMode,+ "securityOpts", securityOpts,+ "privileged", privileged,+ "autoRemove", autoRemove,+ "mounts", mounts,+ "nanoCPUs", resources.NanoCPUs,+ "cpuset", resources.CpusetCpus,+ "memory", resources.Memory,+ "memorySwap", resources.MemorySwap,+ )+}++func TruncateLogOutput(content string, maxBytes int64) string {+ content = strings.TrimSpace(content)+ if content == "" || maxBytes <= 0 {+ return content+ }++ raw := []byte(content)+ if int64(len(raw)) <= maxBytes {+ return content+ }++ return strings.TrimSpace(string(raw[:maxBytes])) + " ...<truncated>"+}++func ReadStartupLogs(ctx context.Context, dockerClient *client.Client, containerID string) (string, string) {+ if dockerClient == nil || containerID == "" {+ return "", ""+ }++ logsCtx, logsCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))+ defer logsCancel()++ logs, err := dockerClient.ContainerLogs(logsCtx, containerID, client.ContainerLogsOptions{+ ShowStdout: true,+ ShowStderr: true,+ Tail: "200",+ })+ if err != nil {+ slog.DebugContext(ctx, "failed to read trivy startup logs", "containerId", containerID, "error", err)+ return "", ""+ }+ defer func() {+ if closeErr := logs.Close(); closeErr != nil {+ slog.WarnContext(ctx, "failed to close trivy startup logs stream", "containerId", containerID, "error", closeErr)+ }+ }()++ var stdoutBuf bytes.Buffer+ var stderrBuf bytes.Buffer+ if _, err := stdcopy.StdCopy(&stdoutBuf, &stderrBuf, logs); err != nil && !dockerutils.IsExpectedStreamEndError(err) {+ slog.DebugContext(ctx, "failed to decode trivy startup logs", "containerId", containerID, "error", err)+ }++ stdoutLog := TruncateLogOutput(stdoutBuf.String(), errorExcerptSizeInternal)+ stderrLog := TruncateLogOutput(stderrBuf.String(), errorExcerptSizeInternal)+ return stdoutLog, stderrLog+}++func RemoveContainer(ctx context.Context, dockerClient *client.Client, containerID, warningMessage string) {+ if dockerClient == nil || containerID == "" {+ return+ }++ cleanupCtx := context.WithoutCancel(ctx)+ cleanupCtx, cleanupCancel := context.WithTimeout(cleanupCtx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))+ defer cleanupCancel()++ if _, err := dockerClient.ContainerRemove(cleanupCtx, containerID, client.ContainerRemoveOptions{Force: true}); err != nil && !cerrdefs.IsNotFound(err) {+ slog.WarnContext(cleanupCtx, warningMessage, "containerId", containerID, "error", err)+ }+}++func CopyRegistryConfigToContainer(ctx context.Context, dockerClient *client.Client, containerID string, configJSON []byte) error {+ if dockerClient == nil || containerID == "" || len(configJSON) == 0 {+ return nil+ }++ var buf bytes.Buffer+ tw := tar.NewWriter(&buf)++ if err := tw.WriteHeader(&tar.Header{Name: "arcane-registry-auth/", Mode: 0o755, Typeflag: tar.TypeDir}); err != nil {+ return err+ }+ if err := tw.WriteHeader(&tar.Header{Name: registryConfigTarNameInternal, Mode: 0o644, Size: int64(len(configJSON))}); err != nil {+ return err+ }+ if _, err := tw.Write(configJSON); err != nil {+ return err+ }+ if err := tw.Close(); err != nil {+ return err+ }++ copyCtx, copyCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))+ defer copyCancel()++ _, err := dockerClient.CopyToContainer(copyCtx, containerID, client.CopyToContainerOptions{+ DestinationPath: registryConfigCopyDestInternal,+ Content: &buf,+ })+ return err+}++func ParseCPULimit(raw string, fallback float64) float64 {+ raw = strings.TrimSpace(raw)+ if raw == "" {+ return fallback+ }++ limit, err := strconv.ParseFloat(raw, 64)+ if err != nil {+ return fallback+ }++ if limit < 0 {+ return 0+ }++ return limit+}++func ParseMemoryLimitMB(raw string, fallback int64) int64 {+ raw = strings.TrimSpace(raw)+ if raw == "" {+ return fallback+ }++ limit, err := strconv.ParseInt(raw, 10, 64)+ if err != nil {+ return fallback+ }++ if limit < 0 {+ return 0+ }++ return limit+}++func ContainerWaitTimeout(trivyTimeoutArg string) time.Duration {+ trivyTimeoutArg = strings.TrimSpace(trivyTimeoutArg)+ if trivyTimeoutArg == "" {+ return timeouts.DefaultTrivyScan + timeouts.DefaultDockerAPI+ }++ timeout, err := time.ParseDuration(trivyTimeoutArg)+ if err != nil || timeout <= 0 {+ return timeouts.DefaultTrivyScan + timeouts.DefaultDockerAPI+ }++ return timeout + timeouts.DefaultDockerAPI+}++func IsSynologyDockerHost(operatingSystem string) bool {+ // Docker info may return a multi-line OS string on some Synology models.+ // e.g. DS925+ (Kernel 5.x) returns "Synology NAS\n (containerized)".+ // Check each line independently so a match on any line is sufficient.+ for line := range strings.SplitSeq(operatingSystem, "\n") {+ if strings.Contains(strings.ToLower(strings.TrimSpace(line)), "synology") {+ return true+ }+ }+ return false+}++func BuildCPUSet(nanoCPUs int64, hostCPUs int) string {+ if nanoCPUs <= 0 {+ return ""+ }++ if hostCPUs <= 0 {+ hostCPUs = 1+ }++ requestedCPUs := min(max(int(math.Floor(float64(nanoCPUs)/float64(NanoCPUsPerCore))), 1), hostCPUs)++ if requestedCPUs == 1 {+ return "0"+ }++ return fmt.Sprintf("0-%d", requestedCPUs-1)+}++func TempFileSize(file *os.File) int64 {+ if file == nil {+ return 0+ }++ stat, err := file.Stat()+ if err != nil {+ return 0+ }++ return stat.Size()+}++func ReadTempFileExcerpt(file *os.File) string {+ if file == nil {+ return ""+ }++ if _, err := file.Seek(0, io.SeekStart); err != nil {+ return ""+ }++ content, err := io.ReadAll(io.LimitReader(file, errorExcerptSizeInternal))+ if err != nil {+ return ""+ }++ return strings.TrimSpace(string(content))+}++func ReadOutputFromContainerFile(ctx context.Context, dockerClient *client.Client, containerID, outputPath string) ([]byte, error) {+ if dockerClient == nil {+ return nil, errors.New("docker client is nil")+ }+ if containerID == "" {+ return nil, errors.New("container id is empty")+ }+ if strings.TrimSpace(outputPath) == "" {+ return nil, errors.New("trivy output path is empty")+ }++ copyCtx, copyCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))+ defer copyCancel()++ copyResult, err := dockerClient.CopyFromContainer(copyCtx, containerID, client.CopyFromContainerOptions{SourcePath: outputPath})+ if err != nil {+ return nil, errors.WrapIf(err, "copy trivy output file")+ }+ archiveReader := copyResult.Content+ defer func() {+ if closeErr := archiveReader.Close(); closeErr != nil && !dockerutils.IsExpectedStreamEndError(closeErr) {+ slog.WarnContext(ctx,+ "failed to close trivy output archive stream",+ "containerId", containerID,+ "outputPath", outputPath,+ "error", closeErr,+ )+ }+ }()++ rawOutput, err := extractFileFromContainerArchiveInternal(archiveReader)+ if err != nil {+ return nil, errors.WrapIf(err, "extract trivy output file")+ }++ if len(bytes.TrimSpace(rawOutput)) == 0 {+ return nil, io.EOF+ }++ return rawOutput, nil+}++func CleanupOutputFileInContainer(ctx context.Context, dockerClient *client.Client, containerID, outputPath string) {+ if dockerClient == nil || containerID == "" || strings.TrimSpace(outputPath) == "" {+ return+ }++ execCtx, execCancel := context.WithTimeout(ctx, timeouts.GetDuration(0, timeouts.DefaultDockerAPI))+ defer execCancel()++ execResp, err := dockerClient.ExecCreate(execCtx, containerID, client.ExecCreateOptions{+ Cmd: []string{"rm", "-f", outputPath},+ })+ if err != nil {+ slog.DebugContext(ctx,+ "failed to create cleanup exec for trivy output file",+ "containerId", containerID,+ "outputPath", outputPath,+ "error", err,+ )+ return+ }++ if _, err := dockerClient.ExecStart(execCtx, execResp.ID, client.ExecStartOptions{}); err != nil {+ slog.DebugContext(ctx,+ "failed to start cleanup exec for trivy output file",+ "containerId", containerID,+ "outputPath", outputPath,+ "error", err,+ )+ return+ }+}++func extractFileFromContainerArchiveInternal(archiveReader io.Reader) ([]byte, error) {+ if archiveReader == nil {+ return nil, errors.New("container archive reader is nil")+ }++ tarReader := tar.NewReader(archiveReader)+ for {+ header, err := tarReader.Next()+ if errors.Is(err, io.EOF) {+ break+ }+ if err != nil {+ return nil, errors.WrapIf(err, "read tar header")+ }++ if header == nil || header.Typeflag == tar.TypeDir {+ continue+ }++ if header.Typeflag != tar.TypeReg {+ continue+ }++ data, err := io.ReadAll(tarReader)+ if err != nil {+ return nil, errors.WrapIf(err, "read tar file content")+ }++ return data, nil+ }++ return nil, errors.New("container archive did not include a file")+}++func DecodeReportFromFile(file *os.File) (*vulnerability.TrivyReport, error) {+ if file == nil {+ return nil, errors.New("trivy output file is nil")+ }++ if _, err := file.Seek(0, io.SeekStart); err != nil {+ return nil, errors.WrapIf(err, "failed to seek trivy output file")+ }++ rawOutput, err := io.ReadAll(file)+ if err != nil {+ return nil, errors.WrapIf(err, "failed to read trivy output file")+ }++ return DecodeReportFromBytes(rawOutput)+}++func DecodeReportFromBytes(rawOutput []byte) (*vulnerability.TrivyReport, error) {+ trimmedOutput := bytes.TrimSpace(rawOutput)+ if len(trimmedOutput) == 0 {+ return nil, io.EOF+ }++ report, strictErr := decodeSingleReportInternal(trimmedOutput)+ if strictErr == nil {+ return report, nil+ }++ // Trivy output can occasionally include non-JSON log lines or additional JSON+ // objects around the report depending on Docker variant / stream behavior.+ // Extract top-level JSON objects and return the first valid Trivy report.+ jsonCandidates := extractTopLevelJSONObjectCandidatesInternal(trimmedOutput)+ if len(jsonCandidates) == 0 {+ return nil, strictErr+ }++ lastErr := strictErr+ for _, candidate := range jsonCandidates {+ report, err := decodeSingleReportInternal(candidate)+ if err == nil {+ return report, nil+ }+ lastErr = err+ }++ return nil, lastErr+}++func decodeSingleReportInternal(rawOutput []byte) (*vulnerability.TrivyReport, error) {+ decoder := jsontext.NewDecoder(bytes.NewReader(rawOutput))++ var report vulnerability.TrivyReport+ if err := json.UnmarshalDecode(decoder, &report); err != nil {+ return nil, err+ }++ if !isLikelyReportInternal(&report) {+ return nil, errors.New("decoded JSON object is not a trivy report")+ }++ if hasTrailingNonWhitespaceInternal(rawOutput, decoder.InputOffset()) {+ return nil, errors.New("trivy output contains trailing data")+ }++ return &report, nil+}++func hasTrailingNonWhitespaceInternal(rawOutput []byte, offset int64) bool {+ if offset < 0 {+ return len(bytes.TrimSpace(rawOutput)) > 0+ }++ if offset >= int64(len(rawOutput)) {+ return false+ }++ return len(bytes.TrimSpace(rawOutput[offset:])) > 0+}++func isLikelyReportInternal(report *vulnerability.TrivyReport) bool {+ if report == nil {+ return false+ }++ return report.SchemaVersion > 0 ||+ strings.TrimSpace(report.ArtifactName) != "" ||+ strings.TrimSpace(report.ArtifactType) != "" ||+ len(report.Results) > 0+}++func extractTopLevelJSONObjectCandidatesInternal(rawOutput []byte) [][]byte {+ if len(rawOutput) == 0 {+ return nil+ }++ candidates := make([][]byte, 0, 4)+ start := -1+ depth := 0+ inString := false+ escaped := false++ for i, b := range rawOutput {+ if start == -1 {+ if b == '{' {+ start = i+ depth = 1+ inString = false+ escaped = false+ }+ continue+ }++ if inString {+ if escaped {+ escaped = false+ continue+ }++ switch b {+ case '\\':+ escaped = true+ case '"':+ inString = false+ }+ continue+ }++ switch b {+ case '"':+ inString = true+ case '{':+ depth+++ case '}':+ depth--+ if depth == 0 {+ candidate := bytes.TrimSpace(rawOutput[start : i+1])+ if len(candidate) > 0 {+ candidates = append(candidates, candidate)+ }+ start = -1+ }+ }+ }++ return candidates+}++func FormatExitError(exitCode int64, errMsg, imageName string) error {+ errMsg = strings.TrimSpace(errMsg)++ if exitCode == 137 {+ if errMsg == "" || errMsg == fmt.Sprintf("exit status %d", exitCode) {+ return errors.Errorf("trivy scan failed: process killed with exit status 137 (likely out of memory while scanning %s)", imageName)+ }+ return errors.Errorf("trivy scan failed: %s (process killed with exit status 137; likely out of memory)", errMsg)+ }++ if strings.Contains(strings.ToLower(errMsg), "deadline exceeded") {+ return errors.Errorf("trivy scan timed out for %s (increase TRIVY_SCAN_TIMEOUT or trivyScanTimeout setting)", imageName)+ }++ return errors.Errorf("trivy scan failed: %s", errMsg)+}++func AwaitContainerWaitResponse(+ ctx context.Context,+ statusCh <-chan containertypes.WaitResponse,+ errCh <-chan error,+) (int64, error) {+ for statusCh != nil || errCh != nil {+ select {+ case waitResp, ok := <-statusCh:+ if !ok {+ statusCh = nil+ continue+ }+ return waitResp.StatusCode, nil+ case err, ok := <-errCh:+ if !ok {+ errCh = nil+ continue+ }+ if err != nil {+ return 0, err+ }+ case <-ctx.Done():+ return 0, ctx.Err()+ }+ }++ return 0, errors.New("trivy container wait ended without status")+}diff --git a/backend/pkg/libarcane/vuln/driver_test.go b/backend/pkg/libarcane/vuln/driver_test.gonew file mode 100644index 000000000..863c1e420--- /dev/null+++ b/backend/pkg/libarcane/vuln/driver_test.go@@ -0,0 +1,600 @@+package vuln++import (+ "context"+ "encoding/json"+ "fmt"+ "os"+ "strings"+ "testing"+ "time"++ "emperror.dev/errors"+ containertypes "github.com/moby/moby/api/types/container"+ mounttypes "github.com/moby/moby/api/types/mount"+ networktypes "github.com/moby/moby/api/types/network"+ "github.com/stretchr/testify/require"++ "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/timeouts"+ "github.com/getarcaneapp/arcane/types/v2/vulnerability"+)++func TestDecodeTrivyReportFromFileInternal_LargePayload(t *testing.T) {+ tmpFile, err := os.CreateTemp("", "trivy-report-*.json")+ require.NoError(t, err)+ defer func() {+ require.NoError(t, os.Remove(tmpFile.Name()))+ }()+ defer func() {+ require.NoError(t, tmpFile.Close())+ }()++ const vulnCount = 4000+ vulns := make([]vulnerability.TrivyVulnerability, 0, vulnCount)+ for i := range vulnCount {+ vulns = append(vulns, vulnerability.TrivyVulnerability{+ VulnerabilityID: fmt.Sprintf("CVE-2026-%04d", i),+ PkgName: fmt.Sprintf("pkg-%d", i),+ InstalledVersion: "1.0.0",+ FixedVersion: "1.0.1",+ Severity: "HIGH",+ })+ }++ report := vulnerability.TrivyReport{+ SchemaVersion: 2,+ ArtifactName: "example/image:latest",+ ArtifactType: "container_image",+ Results: []vulnerability.TrivyResults{+ {+ Target: "alpine:3.20",+ Class: "os-pkgs",+ Type: "alpine",+ Vulnerabilities: vulns,+ },+ },+ }++ encoder := json.NewEncoder(tmpFile)+ require.NoError(t, encoder.Encode(report))++ decoded, err := DecodeReportFromFile(tmpFile)+ require.NoError(t, err)+ require.NotNil(t, decoded)+ require.Equal(t, report.ArtifactName, decoded.ArtifactName)+ require.Len(t, decoded.Results, 1)+ require.Len(t, decoded.Results[0].Vulnerabilities, vulnCount)+}++func TestDecodeTrivyReportFromFileInternal_RecoversFromPrefixedAndTrailingNoise(t *testing.T) {+ tmpFile, err := os.CreateTemp("", "trivy-report-noisy-*.json")+ require.NoError(t, err)+ defer func() {+ require.NoError(t, os.Remove(tmpFile.Name()))+ }()+ defer func() {+ require.NoError(t, tmpFile.Close())+ }()++ report := vulnerability.TrivyReport{+ SchemaVersion: 2,+ ArtifactName: "ghcr.io/getarcaneapp/example:latest",+ ArtifactType: "container_image",+ Results: []vulnerability.TrivyResults{+ {Target: "alpine:3.20", Class: "os-pkgs", Type: "alpine"},+ },+ }++ reportBytes, err := json.Marshal(report)+ require.NoError(t, err)++ _, err = tmpFile.WriteString("WARN scanner warming cache\n")+ require.NoError(t, err)+ _, err = tmpFile.Write(reportBytes)+ require.NoError(t, err)+ _, err = tmpFile.WriteString("\nINFO scan completed\n")+ require.NoError(t, err)++ decoded, err := DecodeReportFromFile(tmpFile)+ require.NoError(t, err)+ require.NotNil(t, decoded)+ require.Equal(t, report.ArtifactName, decoded.ArtifactName)+ require.Equal(t, report.SchemaVersion, decoded.SchemaVersion)+}++func TestDecodeTrivyReportFromFileInternal_SelectsReportFromMultipleJSONObjects(t *testing.T) {+ tmpFile, err := os.CreateTemp("", "trivy-report-multi-json-*.json")+ require.NoError(t, err)+ defer func() {+ require.NoError(t, os.Remove(tmpFile.Name()))+ }()+ defer func() {+ require.NoError(t, tmpFile.Close())+ }()++ report := vulnerability.TrivyReport{+ SchemaVersion: 2,+ ArtifactName: "ghcr.io/getarcaneapp/example:stable",+ ArtifactType: "container_image",+ Results: []vulnerability.TrivyResults{+ {Target: "debian:12", Class: "os-pkgs", Type: "debian"},+ },+ }++ reportBytes, err := json.Marshal(report)+ require.NoError(t, err)++ _, err = tmpFile.WriteString(`{"action":"vulnerability_scan","success":false}` + "\n")+ require.NoError(t, err)+ _, err = tmpFile.Write(reportBytes)+ require.NoError(t, err)++ decoded, err := DecodeReportFromFile(tmpFile)+ require.NoError(t, err)+ require.NotNil(t, decoded)+ require.Equal(t, report.ArtifactName, decoded.ArtifactName)+ require.Equal(t, report.SchemaVersion, decoded.SchemaVersion)+}++func TestDecodeTrivyReportFromFileInternal_HandlesConcatenatedJSONObjects(t *testing.T) {+ tmpFile, err := os.CreateTemp("", "trivy-report-concatenated-*.json")+ require.NoError(t, err)+ defer func() {+ require.NoError(t, os.Remove(tmpFile.Name()))+ }()+ defer func() {+ require.NoError(t, tmpFile.Close())+ }()++ report := vulnerability.TrivyReport{+ SchemaVersion: 2,+ ArtifactName: "ghcr.io/getarcaneapp/example:concat",+ ArtifactType: "container_image",+ Results: []vulnerability.TrivyResults{+ {Target: "ubuntu:24.04", Class: "os-pkgs", Type: "ubuntu"},+ },+ }++ reportBytes, err := json.Marshal(report)+ require.NoError(t, err)++ _, err = tmpFile.Write(reportBytes)+ require.NoError(t, err)+ _, err = tmpFile.WriteString(`{"message":"done"}`)+ require.NoError(t, err)++ decoded, err := DecodeReportFromFile(tmpFile)+ require.NoError(t, err)+ require.NotNil(t, decoded)+ require.Equal(t, report.ArtifactName, decoded.ArtifactName)+ require.Equal(t, report.SchemaVersion, decoded.SchemaVersion)+}++func TestCleanupTrivyLogTempFilesInternal_RemovesFiles(t *testing.T) {+ stdoutFile, err := os.CreateTemp("", "trivy-stdout-test-*")+ require.NoError(t, err)+ stderrFile, err := os.CreateTemp("", "trivy-stderr-test-*")+ require.NoError(t, err)++ stdoutPath := stdoutFile.Name()+ stderrPath := stderrFile.Name()++ _, err = stdoutFile.WriteString("stdout")+ require.NoError(t, err)+ _, err = stderrFile.WriteString("stderr")+ require.NoError(t, err)++ CleanupLogTempFiles(context.Background(), stdoutFile, stderrFile)++ _, err = os.Stat(stdoutPath)+ require.Error(t, err)+ require.True(t, os.IsNotExist(err))++ _, err = os.Stat(stderrPath)+ require.Error(t, err)+ require.True(t, os.IsNotExist(err))+}++func TestBuildTrivyHostConfig_ExcludesResourcesWhenLimitsDisabled(t *testing.T) {+ resources := containertypes.Resources{+ NanoCPUs: int64(2_000_000_000),+ Memory: int64(1_073_741_824),+ MemorySwap: int64(1_073_741_824),+ }++ hostConfig := BuildHostConfig(+ "cache-volume",+ nil,+ resources,+ "",+ false,+ RuntimeOptions{NetworkMode: "bridge"},+ nil,+ false,+ )+ require.Equal(t, containertypes.NetworkMode("bridge"), hostConfig.NetworkMode)+ require.Equal(t, int64(0), hostConfig.NanoCPUs)+ require.Equal(t, int64(0), hostConfig.Memory)+ require.Equal(t, int64(0), hostConfig.MemorySwap)+ require.Empty(t, hostConfig.SecurityOpt)+ require.False(t, hostConfig.Privileged)+}++func TestBuildTrivyHostConfig_IncludesResourcesWhenLimitsEnabled(t *testing.T) {+ resources := containertypes.Resources{+ NanoCPUs: int64(2_000_000_000),+ Memory: int64(1_073_741_824),+ MemorySwap: int64(1_073_741_824),+ }++ hostConfig := BuildHostConfig(+ "cache-volume",+ nil,+ resources,+ "",+ true,+ RuntimeOptions{NetworkMode: "arcane-external"},+ nil,+ false,+ )+ require.Equal(t, containertypes.NetworkMode("arcane-external"), hostConfig.NetworkMode)+ require.Equal(t, resources.NanoCPUs, hostConfig.NanoCPUs)+ require.Equal(t, resources.Memory, hostConfig.Memory)+ require.Equal(t, resources.MemorySwap, hostConfig.MemorySwap)+}++func TestBuildTrivyHostConfig_UsesCPUSetAndClearsNanoCPUs(t *testing.T) {+ resources := containertypes.Resources{+ NanoCPUs: int64(2_000_000_000),+ Memory: int64(1_073_741_824),+ MemorySwap: int64(1_073_741_824),+ }++ hostConfig := BuildHostConfig(+ "cache-volume",+ nil,+ resources,+ "0-1",+ true,+ RuntimeOptions{NetworkMode: "bridge"},+ nil,+ false,+ )+ require.Equal(t, containertypes.NetworkMode("bridge"), hostConfig.NetworkMode)+ require.Equal(t, int64(0), hostConfig.NanoCPUs)+ require.Equal(t, "0-1", hostConfig.CpusetCpus)+ require.Equal(t, resources.Memory, hostConfig.Memory)+ require.Equal(t, resources.MemorySwap, hostConfig.MemorySwap)+}++func TestBuildTrivyHostConfig_AppliesRuntimeSecurity(t *testing.T) {+ hostConfig := BuildHostConfig(+ "cache-volume",+ nil,+ containertypes.Resources{},+ "",+ false,+ RuntimeOptions{NetworkMode: "bridge"},+ []string{"label=disable", "label=type:container_runtime_t"},+ true,+ )++ require.Equal(t, []string{"label=disable", "label=type:container_runtime_t"}, hostConfig.SecurityOpt)+ require.True(t, hostConfig.Privileged)+}++func TestBuildTrivyHostConfig_SkipsCacheMountWhenVolumeEmpty(t *testing.T) {+ hostConfig := BuildHostConfig(+ "",+ nil,+ containertypes.Resources{},+ "",+ false,+ RuntimeOptions{NetworkMode: "bridge"},+ nil,+ false,+ )++ require.Empty(t, hostConfig.Mounts)++ socketMount := mounttypes.Mount{+ Type: mounttypes.TypeBind,+ Source: "/run/user/1000/podman/podman.sock",+ Target: "/run/user/1000/podman/podman.sock",+ }+ hostConfig = BuildHostConfig(+ "",+ nil,+ containertypes.Resources{},+ "",+ false,+ RuntimeOptions{+ NetworkMode: "bridge",+ Mounts: []mounttypes.Mount{socketMount},+ },+ nil,+ false,+ )++ require.Len(t, hostConfig.Mounts, 1)+ require.Equal(t, socketMount.Source, hostConfig.Mounts[0].Source)+}++func TestBuildTrivyBatchHostConfig_AppliesRuntimeSecurity(t *testing.T) {+ hostConfig := BuildBatchHostConfig(+ "cache-volume",+ RuntimeOptions{NetworkMode: "arcane-external"},+ []string{"label=disable"},+ true,+ )++ require.Equal(t, containertypes.NetworkMode("arcane-external"), hostConfig.NetworkMode)+ require.Equal(t, []string{"label=disable"}, hostConfig.SecurityOpt)+ require.True(t, hostConfig.Privileged)+ require.Len(t, hostConfig.Mounts, 1)+ require.Equal(t, "cache-volume", hostConfig.Mounts[0].Source)+}++func TestBuildTrivyBatchHostConfig_SkipsCacheMountWhenVolumeEmpty(t *testing.T) {+ hostConfig := BuildBatchHostConfig(+ "",+ RuntimeOptions{NetworkMode: "arcane-external"},+ nil,+ false,+ )++ require.Empty(t, hostConfig.Mounts)+}++func TestBuildTrivyHostConfig_IncludesInheritedSocketMount(t *testing.T) {+ hostConfig := BuildHostConfig(+ "cache-volume",+ nil,+ containertypes.Resources{},+ "",+ false,+ RuntimeOptions{+ NetworkMode: "bridge",+ Mounts: []mounttypes.Mount{+ {+ Type: mounttypes.TypeBind,+ Source: "/run/user/1000/podman/podman.sock",+ Target: "/run/user/1000/podman/podman.sock",+ },+ },+ },+ nil,+ false,+ )++ require.Len(t, hostConfig.Mounts, 2)+ require.Equal(t, "/run/user/1000/podman/podman.sock", hostConfig.Mounts[0].Source)+ require.Equal(t, "/run/user/1000/podman/podman.sock", hostConfig.Mounts[0].Target)+ require.Equal(t, "cache-volume", hostConfig.Mounts[1].Source)+}++func TestApplyTrivyRuntimeSecurityInternal(t *testing.T) {+ hostConfig := &containertypes.HostConfig{}++ ApplyRuntimeSecurity(hostConfig, []string{"label=disable"}, true)++ require.Equal(t, []string{"label=disable"}, hostConfig.SecurityOpt)+ require.True(t, hostConfig.Privileged)+}++func TestSelectDefaultTrivyNetworkModeInternal(t *testing.T) {+ require.Equal(t, DefaultNetworkMode, SelectDefaultNetworkMode(""))+ require.Equal(t, DefaultNetworkMode, SelectDefaultNetworkMode(" \t\n "))+ require.Equal(t, "arcane-external", SelectDefaultNetworkMode("arcane-external"))+}++func TestResolveTrivyUnixSocketSourceInternal(t *testing.T) {+ t.Run("uses resolved host path", func(t *testing.T) {+ source, err := ResolveUnixSocketSource(+ context.Background(),+ "/run/user/1000/podman/podman.sock",+ func(context.Context, string) (string, error) {+ return "/host/podman/podman.sock", nil+ },+ func() bool { return true },+ )++ require.NoError(t, err)+ require.Equal(t, "/host/podman/podman.sock", source)+ })++ t.Run("fails when socket cannot be resolved in docker", func(t *testing.T) {+ _, err := ResolveUnixSocketSource(+ context.Background(),+ "/run/user/1000/podman/podman.sock",+ func(context.Context, string) (string, error) {+ return "", nil+ },+ func() bool { return true },+ )++ require.Error(t, err)+ require.Contains(t, err.Error(), "daemon-visible host path")+ })++ t.Run("falls back to original path outside docker", func(t *testing.T) {+ source, err := ResolveUnixSocketSource(+ context.Background(),+ "/var/run/docker.sock",+ func(context.Context, string) (string, error) {+ return "", errors.New("not running in docker")+ },+ func() bool { return false },+ )++ require.NoError(t, err)+ require.Equal(t, "/var/run/docker.sock", source)+ })+}++func TestSelectTrivyAutoNetworkModeInternal(t *testing.T) {+ t.Run("prefers explicit host network mode", func(t *testing.T) {+ mode := SelectAutoNetworkMode(&containertypes.InspectResponse{+ HostConfig: &containertypes.HostConfig{NetworkMode: "host"},+ })++ require.Equal(t, "host", mode)+ })++ t.Run("prefers attached custom network over bridge", func(t *testing.T) {+ mode := SelectAutoNetworkMode(&containertypes.InspectResponse{+ HostConfig: &containertypes.HostConfig{NetworkMode: "bridge"},+ NetworkSettings: &containertypes.NetworkSettings{+ Networks: map[string]*networktypes.EndpointSettings{+ "bridge": {},+ "arcane-internal": {},+ },+ },+ })++ require.Equal(t, "arcane-internal", mode)+ })++ t.Run("falls back to bridge when no custom network is attached", func(t *testing.T) {+ mode := SelectAutoNetworkMode(&containertypes.InspectResponse{+ HostConfig: &containertypes.HostConfig{NetworkMode: "bridge"},+ NetworkSettings: &containertypes.NetworkSettings{+ Networks: map[string]*networktypes.EndpointSettings{+ "bridge": {},+ },+ },+ })++ require.Equal(t, "bridge", mode)+ })+}++func TestIsSynologyDockerHostInternal(t *testing.T) {+ // DS220+ (Kernel 4.x): single-line+ require.True(t, IsSynologyDockerHost("Synology NAS"))+ // DS925+ (Kernel 5.x): OS field contains a newline before "(containerized)"+ require.True(t, IsSynologyDockerHost("Synology NAS\n (containerized)"))+ // Variation: Windows-style line ending+ require.True(t, IsSynologyDockerHost("Synology NAS\r\n (containerized)"))+ // Case-insensitive+ require.True(t, IsSynologyDockerHost("synology nas"))+ // Padded with spaces+ require.True(t, IsSynologyDockerHost(" Synology NAS "))+ // Empty / non-Synology+ require.False(t, IsSynologyDockerHost(""))+ require.False(t, IsSynologyDockerHost("Ubuntu 24.04.1 LTS"))+ require.False(t, IsSynologyDockerHost("Ubuntu 24.04.1 LTS\n (containerized)"))+}++func TestBuildTrivyCPUSetInternal(t *testing.T) {+ require.Empty(t, BuildCPUSet(0, 4))+ require.Equal(t, "0", BuildCPUSet(1_000_000_000, 4))+ require.Equal(t, "0-1", BuildCPUSet(2_500_000_000, 4))+ require.Equal(t, "0-1", BuildCPUSet(3_000_000_000, 2))+ require.Equal(t, "0", BuildCPUSet(1_000_000_000, 0))+}++func TestTrivyMountSummariesInternal(t *testing.T) {+ require.Nil(t, MountSummaries(nil))++ hostConfig := &containertypes.HostConfig{+ Mounts: []mounttypes.Mount{+ {+ Type: mounttypes.TypeBind,+ Source: "/var/run/docker.sock",+ Target: "/var/run/docker.sock",+ },+ {+ Type: mounttypes.TypeVolume,+ Source: "arcane-trivy-cache",+ Target: "/root/.cache",+ },+ {+ Type: mounttypes.TypeTmpfs,+ Target: "/tmp",+ },+ },+ }++ require.Equal(t,+ []string{+ "/var/run/docker.sock:/var/run/docker.sock",+ "arcane-trivy-cache:/root/.cache",+ "tmpfs:/tmp",+ },+ MountSummaries(hostConfig),+ )+}++func TestTruncateTrivyLogOutputInternal(t *testing.T) {+ require.Empty(t, TruncateLogOutput("", 32))+ require.Equal(t, "keep me", TruncateLogOutput(" keep me ", 32))++ longOutput := strings.Repeat("x", 80)+ truncated := TruncateLogOutput(longOutput, 16)+ require.Equal(t, strings.Repeat("x", 16)+" ...<truncated>", truncated)+}++func TestTrivyContainerWaitTimeoutInternal(t *testing.T) {+ require.Equal(t, timeouts.DefaultTrivyScan+timeouts.DefaultDockerAPI, ContainerWaitTimeout(""))+ require.Equal(t, 5*time.Minute+timeouts.DefaultDockerAPI, ContainerWaitTimeout("5m"))+ require.Equal(t, timeouts.DefaultTrivyScan+timeouts.DefaultDockerAPI, ContainerWaitTimeout("not-a-duration"))+ require.Equal(t, timeouts.DefaultTrivyScan+timeouts.DefaultDockerAPI, ContainerWaitTimeout("0s"))+}++func TestAwaitContainerWaitResponseInternal_Status(t *testing.T) {+ statusCh := make(chan containertypes.WaitResponse, 1)+ errCh := make(chan error)+ statusCh <- containertypes.WaitResponse{StatusCode: 12}++ status, err := AwaitContainerWaitResponse(context.Background(), statusCh, errCh)+ require.NoError(t, err)+ require.Equal(t, int64(12), status)+}++func TestAwaitContainerWaitResponseInternal_Error(t *testing.T) {+ statusCh := make(chan containertypes.WaitResponse)+ errCh := make(chan error, 1)+ errCh <- errors.New("boom")++ status, err := AwaitContainerWaitResponse(context.Background(), statusCh, errCh)+ require.Error(t, err)+ require.Contains(t, err.Error(), "boom")+ require.Equal(t, int64(0), status)+}++func TestAwaitContainerWaitResponseInternal_ClosedErrorChannelStillReadsStatus(t *testing.T) {+ statusCh := make(chan containertypes.WaitResponse, 1)+ errCh := make(chan error)++ close(errCh)+ statusCh <- containertypes.WaitResponse{StatusCode: 7}++ status, err := AwaitContainerWaitResponse(context.Background(), statusCh, errCh)+ require.NoError(t, err)+ require.Equal(t, int64(7), status)+}++func TestAwaitContainerWaitResponseInternal_ContextDone(t *testing.T) {+ statusCh := make(chan containertypes.WaitResponse)+ errCh := make(chan error)+ ctx, cancel := context.WithTimeout(context.Background(), 10*time.Millisecond)+ defer cancel()++ status, err := AwaitContainerWaitResponse(ctx, statusCh, errCh)+ require.Error(t, err)+ require.ErrorIs(t, err, context.DeadlineExceeded)+ require.Equal(t, int64(0), status)+}++func TestAwaitContainerWaitResponseInternal_NoStatus(t *testing.T) {+ statusCh := make(chan containertypes.WaitResponse)+ errCh := make(chan error)+ close(statusCh)+ close(errCh)++ status, err := AwaitContainerWaitResponse(context.Background(), statusCh, errCh)+ require.Error(t, err)+ require.Contains(t, err.Error(), "ended without status")+ require.Equal(t, int64(0), status)+}diff --git a/backend/pkg/libarcane/vuln/trivy.go b/backend/pkg/libarcane/vuln/trivy.goindex 21880f5e8..c7b14e428 100644--- a/backend/pkg/libarcane/vuln/trivy.go+++ b/backend/pkg/libarcane/vuln/trivy.go@@ -1,6 +1,8 @@-// Package vuln provides stateless helpers for building and parsing the Trivy-// scanner invocations used by the vulnerability service. Functions here have no-// dependency on database or service state so they can be unit-tested in isolation.+// Package vuln drives Trivy scans: building and parsing scanner invocations,+// plus the container-level mechanics of running one — runtime/network+// detection, container spec and resources, temp files, log copy and tar+// extraction, and report decoding. Nothing here depends on Arcane's database or+// service graph, so it is unit-testable in isolation.package vulnimport (diff --git a/backend/pkg/libarcane/ws/hub_test.go b/backend/pkg/libarcane/ws/hub_test.goindex d2907f5d2..16c39caf0 100644--- a/backend/pkg/libarcane/ws/hub_test.go+++ b/backend/pkg/libarcane/ws/hub_test.go@@ -45,6 +45,12 @@ func newTestWSPair(t *testing.T) (clientConn *websocket.Conn, serverConn *websoc}}+func (h *Hub) ClientCount() int {+ h.mu.RLock()+ defer h.mu.RUnlock()+ return len(h.clients)+}+func TestNewHub(t *testing.T) {h := NewHub(10)require.NotNil(t, h)diff --git a/backend/pkg/libarcane/ws/hub_test_helper_test.go b/backend/pkg/libarcane/ws/hub_test_helper_test.godeleted file mode 100644index d7e7347e4..000000000--- a/backend/pkg/libarcane/ws/hub_test_helper_test.go+++ /dev/null@@ -1,7 +0,0 @@-package ws--func (h *Hub) ClientCount() int {- h.mu.RLock()- defer h.mu.RUnlock()- return len(h.clients)-}diff --git a/backend/pkg/pagination/pagination_test.go b/backend/pkg/pagination/pagination_test.goindex cb41a3b82..a53a566fc 100644--- a/backend/pkg/pagination/pagination_test.go+++ b/backend/pkg/pagination/pagination_test.go@@ -6,7 +6,7 @@ import ("github.com/stretchr/testify/assert")-func TestBuildResponseFromFilterResult(t *testing.T) {+func TestBuildResponse(t *testing.T) {tests := []struct {name stringresult FilterResult[int]@@ -122,7 +122,7 @@ func TestBuildResponseFromFilterResult(t *testing.T) {for _, tt := range tests {t.Run(tt.name, func(t *testing.T) {- result := BuildResponseFromFilterResult(tt.result, tt.params)+ result := BuildResponse(tt.result.TotalCount, tt.result.TotalAvailable, tt.params)assert.Equal(t, tt.expected, result)})}diff --git a/backend/pkg/pagination/types.go b/backend/pkg/pagination/types.goindex b780547f6..39a759e4b 100644--- a/backend/pkg/pagination/types.go+++ b/backend/pkg/pagination/types.go@@ -8,11 +8,10 @@ type Response struct {GrandTotalItems int64 `json:"grandTotalItems,omitempty"`}-// BuildResponseFromFilterResult creates a pagination Response from a FilterResult.+// BuildResponse creates a pagination Response from raw counts.// Handles the special case where limit = -1 means "show all".-func BuildResponseFromFilterResult[T any](result FilterResult[T], params QueryParams) Response {+func BuildResponse(totalCount int64, totalAvailable int64, params QueryParams) Response {limit := params.Limit- totalCount := result.TotalCountvar totalPages int64var page int@@ -44,6 +43,6 @@ func BuildResponseFromFilterResult[T any](result FilterResult[T], params QueryPaTotalItems: totalCount,CurrentPage: page,ItemsPerPage: itemsPerPage,- GrandTotalItems: result.TotalAvailable,+ GrandTotalItems: totalAvailable,}}diff --git a/backend/pkg/projects/compose_override.go b/backend/pkg/projects/compose_override.goindex 2023b2df9..cd0e85802 100644--- a/backend/pkg/projects/compose_override.go+++ b/backend/pkg/projects/compose_override.go@@ -138,3 +138,43 @@ func WriteComposeOverrideFile(projectsRoot, dir string, content *string, fileNamreturn nil}++// ExistingOrDefaultOverrideName returns the override file name already present+// in dir, falling back to DefaultComposeOverrideFileName.+func ExistingOrDefaultOverrideName(dir string) string {+ if overridePath := DetectComposeOverrideFile(dir); overridePath != "" {+ return filepath.Base(overridePath)+ }+ return DefaultComposeOverrideFileName+}++// ResolveEffectiveOverrideForValidation resolves the override content compose+// validation should see: the on-disk override when the caller supplied none,+// nothing when the caller explicitly blanked it, otherwise the caller's content.+func ResolveEffectiveOverrideForValidation(dir string, overrideContent *string) (*string, string) {+ switch {+ case overrideContent == nil:+ overridePath := DetectComposeOverrideFile(dir)+ if overridePath == "" {+ return nil, ""+ }+ onDisk := ReadComposeOverrideContent(dir)+ return &onDisk, filepath.Base(overridePath)+ case strings.TrimSpace(*overrideContent) == "":+ return nil, ""+ default:+ return overrideContent, ExistingOrDefaultOverrideName(dir)+ }+}++// ApplyOverrideFileChange writes, clears, or leaves the override file alone+// depending on whether overrideContent is nil, blank, or set.+func ApplyOverrideFileChange(projectsRoot, dir string, overrideContent *string) error {+ if overrideContent == nil {+ return nil+ }+ if strings.TrimSpace(*overrideContent) == "" {+ return WriteComposeOverrideFile(projectsRoot, dir, nil, "")+ }+ return WriteComposeOverrideFile(projectsRoot, dir, overrideContent, ExistingOrDefaultOverrideName(dir))+}diff --git a/backend/pkg/projects/container_mounts.go b/backend/pkg/projects/container_mounts.goindex 7656dac0b..c665f84d1 100644--- a/backend/pkg/projects/container_mounts.go+++ b/backend/pkg/projects/container_mounts.go@@ -8,7 +8,6 @@ import (mounttypes "github.com/moby/moby/api/types/mount""github.com/moby/moby/client"- dockerutils "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""go.getarcane.app/sys/cgroup")@@ -23,7 +22,7 @@ func GetCurrentContainerMounts(ctx context.Context, dockerCli *client.Client) ([}// Prefer robust current-container detection and fall back to hostname.- inspectTarget, err := dockerutils.CurrentContainerInspectTarget(cgroup.CurrentContainerID, os.Hostname)+ inspectTarget, err := libarcane.CurrentContainerInspectTarget(cgroup.CurrentContainerID, os.Hostname)if err != nil {return nil, err}diff --git a/backend/pkg/projects/env_validation.go b/backend/pkg/projects/env_validation.gonew file mode 100644index 000000000..56c7ced6d--- /dev/null+++ b/backend/pkg/projects/env_validation.go@@ -0,0 +1,97 @@+package projects++import (+ "maps"+ "os"+ "path/filepath"+ "strings"++ "emperror.dev/errors"+ "github.com/compose-spec/compose-go/v2/dotenv"+)++// BuildValidationEnvironment resolves the env a compose file is validated+// against. It matches the project-visible env sources without inheriting the+// Arcane process environment, which may contain unrelated secrets.+func BuildValidationEnvironment(projectsDirectory, projectPath string, effectiveEnvContent *string) (EnvMap, error) {+ fullEnvMap := make(EnvMap)+ if absWorkdir, absErr := filepath.Abs(projectPath); absErr == nil {+ fullEnvMap["PWD"] = absWorkdir+ } else {+ fullEnvMap["PWD"] = projectPath+ }++ globalEnvPath := filepath.Join(projectsDirectory, GlobalEnvFileName)+ globalEnv, err := ParseValidationEnvFile(globalEnvPath, fullEnvMap)+ if err != nil {+ return nil, errors.WrapIf(err, "parse global env file")+ }+ maps.Copy(fullEnvMap, globalEnv)++ if effectiveEnvContent != nil {+ projectEnv, err := ParseValidationEnvContent(*effectiveEnvContent, fullEnvMap)+ if err != nil {+ return nil, errors.WrapIf(err, "parse provided env content")+ }+ maps.Copy(fullEnvMap, projectEnv)+ return fullEnvMap, nil+ }++ projectEnvPath := filepath.Join(projectPath, ".env")+ projectEnv, err := ParseValidationEnvFile(projectEnvPath, fullEnvMap)+ if err != nil {+ return nil, errors.WrapIf(err, "parse project env file")+ }+ maps.Copy(fullEnvMap, projectEnv)++ return fullEnvMap, nil+}++// ParseValidationEnvFile parses one env file against contextEnv. A missing file+// is not an error.+func ParseValidationEnvFile(path string, contextEnv EnvMap) (EnvMap, error) {+ if _, err := os.Stat(path); err != nil {+ if os.IsNotExist(err) {+ return nil, nil+ }+ return nil, errors.WrapIf(err, "stat file")+ }++ content, err := os.ReadFile(path)+ if err != nil {+ return nil, errors.WrapIf(err, "read file")+ }++ return ParseValidationEnvContent(string(content), contextEnv)+}++// ParseValidationEnvContent parses env content, resolving interpolations+// against contextEnv rather than the process environment.+func ParseValidationEnvContent(content string, contextEnv EnvMap) (EnvMap, error) {+ lookupFn := func(key string) (string, bool) {+ value, ok := contextEnv[key]+ return value, ok+ }++ envMap, err := dotenv.ParseWithLookup(strings.NewReader(content), lookupFn)+ if err != nil {+ return nil, errors.WrapIf(err, "parse env")+ }++ return envMap, nil+}++// ValidateIncludePathsForContent rejects compose content whose includes resolve+// outside the project directory.+func ValidateIncludePathsForContent(projectPath, composeContent string, envMap EnvMap) error {+ includes, err := ParseIncludesFromContent(filepath.Join(projectPath, "compose.yaml"), []byte(composeContent), envMap, false)+ if err != nil {+ return err+ }+ for _, inc := range includes {+ if _, err := ValidateIncludePathForWrite(projectPath, inc.Path); err != nil {+ return errors.WrapIff(err, "include path %q is outside project directory", inc.RelativePath)+ }+ }+ return nil+}diff --git a/backend/pkg/projects/fs_backup.go b/backend/pkg/projects/fs_backup.goindex a79c18238..5fa4f30a1 100644--- a/backend/pkg/projects/fs_backup.go+++ b/backend/pkg/projects/fs_backup.go@@ -8,6 +8,7 @@ import ("strings""emperror.dev/errors"+ "github.com/getarcaneapp/arcane/types/v2/project""go.getarcane.app/sys/atomic")@@ -482,3 +483,84 @@ func restoreTopLevelFilesInternal(backupRoot, projRoot *os.Root, backup *Project}return nil}++// BuildUpdateBackupScope derives the backup scope for one project update from+// the content and file changes it will apply.+func BuildUpdateBackupScope(projectPath string, composeContent, envContent, overrideContent *string, fileChanges []project.ProjectFileChange) ProjectUpdateBackupScope {+ scope := ProjectUpdateBackupScope{+ TopLevelFiles: composeContent != nil || envContent != nil || overrideContent != nil,+ }++ for _, change := range fileChanges {+ rel, err := NormalizeProjectRelativePath(change.RelativePath)+ if err != nil {+ continue+ }++ var dest string+ switch change.Operation {+ case project.FileOpRename:+ newName, nameErr := ValidateProjectFileName(change.NewName)+ if nameErr != nil {+ continue+ }+ dest = path.Join(path.Dir(rel), newName)+ case project.FileOpMove:+ parent := strings.TrimSpace(change.NewParentPath)+ if parent != "" {+ normalizedParent, parentErr := NormalizeProjectRelativePath(parent)+ if parentErr != nil {+ continue+ }+ parent = normalizedParent+ }+ dest = path.Join(parent, path.Base(rel))+ default:+ scope.Paths = append(scope.Paths, rel)+ continue+ }++ // Rename/move of an existing directory rolls back via an inverse+ // rename — never a copy of a potentially huge tree.+ if info, statErr := os.Lstat(filepath.Join(projectPath, filepath.FromSlash(rel))); statErr == nil && info.IsDir() {+ scope.RenamedDirs = append(scope.RenamedDirs, [2]string{rel, dest})+ } else {+ scope.Paths = append(scope.Paths, rel, dest)+ }+ }++ demoteInterferingRenamedDirsInternal(&scope)+ return scope+}++// demoteInterferingRenamedDirsInternal downgrades a directory rename to a full+// copy when another change in the same batch touches its source or destination+// subtree (e.g. rename a -> b then delete b, or update_file b/x). The inverse+// rename alone cannot roll those back: the destination may be mutated or gone+// by the time the batch fails, so the original directory must be backed up.+func demoteInterferingRenamedDirsInternal(scope *ProjectUpdateBackupScope) {+ overlaps := func(a, b string) bool {+ return a == b || strings.HasPrefix(a, b+"/") || strings.HasPrefix(b, a+"/")+ }+ touchesPair := func(p string, pair [2]string) bool {+ return overlaps(p, pair[0]) || overlaps(p, pair[1])+ }++ for i := 0; i < len(scope.RenamedDirs); i++ {+ pair := scope.RenamedDirs[i]+ conflict := slices.ContainsFunc(scope.Paths, func(p string) bool { return touchesPair(p, pair) })+ if !conflict {+ for j, other := range scope.RenamedDirs {+ if j != i && (touchesPair(other[0], pair) || touchesPair(other[1], pair)) {+ conflict = true+ break+ }+ }+ }+ if conflict {+ scope.Paths = append(scope.Paths, pair[0], pair[1])+ scope.RenamedDirs = slices.Delete(scope.RenamedDirs, i, i+1)+ i--+ }+ }+}diff --git a/backend/pkg/projects/ports.go b/backend/pkg/projects/ports.gonew file mode 100644index 000000000..5833ea4f7--- /dev/null+++ b/backend/pkg/projects/ports.go@@ -0,0 +1,36 @@+package projects++import (+ "fmt"++ "github.com/docker/compose/v5/pkg/api"+ "github.com/moby/moby/api/types/container"+)++// FormatPorts renders compose port publishers as "published:target/proto"+// (or "target/proto" when the port is not published).+func FormatPorts(publishers []api.PortPublisher) []string {+ var ports []string+ for _, pub := range publishers {+ if pub.PublishedPort > 0 {+ ports = append(ports, fmt.Sprintf("%d:%d/%s", pub.PublishedPort, pub.TargetPort, pub.Protocol))+ } else {+ ports = append(ports, fmt.Sprintf("%d/%s", pub.TargetPort, pub.Protocol))+ }+ }+ return ports+}++// FormatDockerPorts renders container port summaries in the same shape as+// FormatPorts, for containers Arcane sees outside a compose project.+func FormatDockerPorts(ports []container.PortSummary) []string {+ var res []string+ for _, p := range ports {+ if p.PublicPort == 0 {+ res = append(res, fmt.Sprintf("%d/%s", p.PrivatePort, p.Type))+ } else {+ res = append(res, fmt.Sprintf("%d:%d/%s", p.PublicPort, p.PrivatePort, p.Type))+ }+ }+ return res+}diff --git a/backend/pkg/projects/ports_test.go b/backend/pkg/projects/ports_test.gonew file mode 100644index 000000000..82a1e9074--- /dev/null+++ b/backend/pkg/projects/ports_test.go@@ -0,0 +1,43 @@+package projects++import (+ "testing"++ "github.com/moby/moby/api/types/container"+ "github.com/stretchr/testify/assert"+)++func TestFormatDockerPorts(t *testing.T) {+ tests := []struct {+ name string+ input []container.PortSummary+ expected []string+ }{+ {+ name: "public port",+ input: []container.PortSummary{+ {PublicPort: 8080, PrivatePort: 80, Type: "tcp"},+ },+ expected: []string{"8080:80/tcp"},+ },+ {+ name: "private only",+ input: []container.PortSummary{+ {PrivatePort: 80, Type: "tcp"},+ },+ expected: []string{"80/tcp"},+ },+ {+ name: "empty",+ input: []container.PortSummary{},+ expected: nil,+ },+ }++ for _, tt := range tests {+ t.Run(tt.name, func(t *testing.T) {+ got := FormatDockerPorts(tt.input)+ assert.Equal(t, tt.expected, got)+ })+ }+}diff --git a/backend/pkg/projects/service_selection.go b/backend/pkg/projects/service_selection.gonew file mode 100644index 000000000..83b6dc6cc--- /dev/null+++ b/backend/pkg/projects/service_selection.go@@ -0,0 +1,93 @@+package projects++import (+ "strings"++ composetypes "github.com/compose-spec/compose-go/v2/types"+)++// NormalizeBuildSelections turns a user-supplied service list into a lookup set,+// dropping blanks. An empty set means "every service".+func NormalizeBuildSelections(services []string) map[string]struct{} {+ selected := map[string]struct{}{}+ for _, name := range services {+ name = strings.TrimSpace(name)+ if name == "" {+ continue+ }+ selected[name] = struct{}{}+ }+ return selected+}++// ServiceSelected reports whether name is in the selection built by+// NormalizeBuildSelections. An empty selection selects everything.+func ServiceSelected(selected map[string]struct{}, name string) bool {+ if len(selected) == 0 {+ return true+ }+ _, ok := selected[name]+ return ok+}++// EnsureServiceImage gives a build-only service a deterministic local image tag+// so the built image has a stable name to deploy from. It reports whether the+// service config was changed.+func EnsureServiceImage(projectID, projectName, serviceName string, svc composetypes.ServiceConfig) (string, composetypes.ServiceConfig, bool) {+ imageName := strings.TrimSpace(svc.Image)+ if imageName == "" {+ imageName = BuildLocalImageTag(projectID, projectName, serviceName)+ svc.Image = imageName+ return imageName, svc, true+ }+ return imageName, svc, false+}++// PrepareDeployServiceConfig resolves the image a service deploys as, naming+// build-only services via EnsureServiceImage. It reports whether the service+// config was changed.+func PrepareDeployServiceConfig(projectID, projectName, serviceName string, svc composetypes.ServiceConfig) (composetypes.ServiceConfig, string, bool) {+ if svc.Build == nil {+ return svc, strings.TrimSpace(svc.Image), false+ }++ resolvedImage, updatedSvc, updated := EnsureServiceImage(projectID, projectName, serviceName, svc)+ return updatedSvc, resolvedImage, updated+}++// ShouldPullDeployImage reports whether a deploy must pull, given the resolved+// pull decision and whether the image is already present locally.+func ShouldPullDeployImage(decision DeployImageDecision, exists bool) bool {+ return decision.PullAlways || (decision.PullIfMissing && !exists)+}++// SelectedImageRefs returns the distinct pullable image refs of the selected+// services — build services are excluded, since their images come from a build.+func SelectedImageRefs(compProj *composetypes.Project, servicesToUpdate []string) []string {+ if compProj == nil {+ return nil+ }++ selected := NormalizeBuildSelections(servicesToUpdate)+ refs := make([]string, 0, len(compProj.Services))+ seen := make(map[string]struct{}, len(compProj.Services))++ for name, svc := range compProj.Services {+ if !ServiceSelected(selected, name) || svc.Build != nil {+ continue+ }++ imageRef := strings.TrimSpace(svc.Image)+ if imageRef == "" {+ continue+ }+ if _, exists := seen[imageRef]; exists {+ continue+ }++ seen[imageRef] = struct{}{}+ refs = append(refs, imageRef)+ }++ return refs+}diff --git a/backend/pkg/scheduler/activity_sweep_job.go b/backend/pkg/scheduler/activity_sweep_job.goindex 011346796..b73231ec5 100644--- a/backend/pkg/scheduler/activity_sweep_job.go+++ b/backend/pkg/scheduler/activity_sweep_job.go@@ -4,7 +4,7 @@ import ("context""log/slog"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity")const ActivitySweepJobName = "activity-sweep"@@ -15,10 +15,10 @@ const ActivitySweepJobName = "activity-sweep"// an internal job: it has no job_metadata entry and is invisible in the Jobs// UI.type ActivitySweepJob struct {- activityService *services.ActivityService+ activityService *activity.ActivityService}-func NewActivitySweepJob(activityService *services.ActivityService) *ActivitySweepJob {+func NewActivitySweepJob(activityService *activity.ActivityService) *ActivitySweepJob {return &ActivitySweepJob{activityService: activityService}}diff --git a/backend/pkg/scheduler/actor_test.go b/backend/pkg/scheduler/actor_test.goindex bec3b5573..8c30e74f7 100644--- a/backend/pkg/scheduler/actor_test.go+++ b/backend/pkg/scheduler/actor_test.go@@ -7,7 +7,7 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/database"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/stretchr/testify/require""go.uber.org/fx/fxtest")@@ -45,7 +45,7 @@ func newJobSchedulerForTestInternal(t testing.TB, ctx context.Context, locationreturn scheduler}-func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*services.SettingsService, error) {+func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *database.DB) (*settings.SettingsService, error) {t.Helper()lifecycle := fxtest.NewLifecycle(t)runtime, err := actors.NewRuntime(t.Context(), lifecycle)@@ -61,5 +61,5 @@ func newSettingsServiceForTestInternal(t testing.TB, ctx context.Context, db *darequire.NoError(t, effects.Stop(stopCtx))require.NoError(t, lifecycle.Stop(stopCtx))})- return services.NewSettingsService(ctx, db, executor, effects)+ return settings.NewSettingsService(ctx, db, executor, effects)}diff --git a/backend/pkg/scheduler/analytics_job.go b/backend/pkg/scheduler/analytics_job.goindex 1c0287593..e37bb3ef9 100644--- a/backend/pkg/scheduler/analytics_job.go+++ b/backend/pkg/scheduler/analytics_job.go@@ -15,7 +15,8 @@ import ("github.com/cenkalti/backoff/v5""github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings")const (@@ -28,8 +29,8 @@ const ()type AnalyticsJob struct {- settingsService *services.SettingsService- kvService *services.KVService+ settingsService *settings.SettingsService+ kvService *kv.KVServicehttpClient *http.ClientheartbeatURL stringcfg *config.Config@@ -38,8 +39,8 @@ type AnalyticsJob struct {}func NewAnalyticsJob(- settingsService *services.SettingsService,- kvService *services.KVService,+ settingsService *settings.SettingsService,+ kvService *kv.KVService,httpClient *http.Client,cfg *config.Config,) *AnalyticsJob {diff --git a/backend/pkg/scheduler/analytics_job_test.go b/backend/pkg/scheduler/analytics_job_test.goindex 46229dcb1..81b592546 100644--- a/backend/pkg/scheduler/analytics_job_test.go+++ b/backend/pkg/scheduler/analytics_job_test.go@@ -17,12 +17,13 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/kv""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/stretchr/testify/assert")-func setupAnalyticsStateServicesInternal(t *testing.T) (*database.DB, *services.SettingsService, *services.KVService) {+func setupAnalyticsStateServicesInternal(t *testing.T) (*database.DB, *settings.SettingsService, *kv.KVService) {t.Helper()ctx := context.Background()db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})@@ -35,7 +36,7 @@ func setupAnalyticsStateServicesInternal(t *testing.T) (*database.DB, *services.require.NoError(t, err)require.NoError(t, settingsService.SetStringSetting(ctx, "instanceId", "test-instance"))- return wrappedDB, settingsService, services.NewKVService(wrappedDB)+ return wrappedDB, settingsService, kv.NewKVService(wrappedDB)}func newHeartbeatServer(t *testing.T) (*httptest.Server, <-chan []byte, *atomic.Int32) {@@ -161,7 +162,7 @@ func TestAnalyticsJob_Run_SkipsWithinHeartbeatWindowAfterRestart(t *testing.T) {reloadedSettingsService, err := newSettingsServiceForTestInternal(t, ctx, wrappedDB)require.NoError(t, err)- restartedJob := NewAnalyticsJob(reloadedSettingsService, services.NewKVService(wrappedDB), server.Client(), cfg)+ restartedJob := NewAnalyticsJob(reloadedSettingsService, kv.NewKVService(wrappedDB), server.Client(), cfg)restartedJob.heartbeatURL = server.URLrestartedJob.now = func() time.Time { return firstAttemptAt.Add(19 * time.Minute) }restartedJob.Run(ctx)diff --git a/backend/pkg/scheduler/auto_heal_job.go b/backend/pkg/scheduler/auto_heal_job.goindex 6dbeda741..605d38087 100644--- a/backend/pkg/scheduler/auto_heal_job.go+++ b/backend/pkg/scheduler/auto_heal_job.go@@ -8,6 +8,9 @@ import ("sync""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+"emperror.dev/errors""github.com/moby/moby/api/types/container""github.com/moby/moby/client"@@ -15,8 +18,9 @@ import ("golang.org/x/sync/errgroup""github.com/getarcaneapp/arcane/backend/v2/internal/actors"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane""github.com/getarcaneapp/arcane/backend/v2/pkg/utils"@@ -33,10 +37,10 @@ type restartRecord struct {}type AutoHealJob struct {- dockerClientService *services.DockerClientService- settingsService *services.SettingsService- eventService *services.EventService- notificationService *services.NotificationService+ dockerClientService *docker.DockerClientService+ settingsService *settings.SettingsService+ eventService *event.EventService+ notificationService *notification.NotificationServiceadmissionGate *actors.Gate[actors.AdmissionKey]mu sync.Mutex@@ -53,10 +57,10 @@ type AutoHealJob struct {}func NewAutoHealJob(- dockerClientService *services.DockerClientService,- settingsService *services.SettingsService,- eventService *services.EventService,- notificationService *services.NotificationService,+ dockerClientService *docker.DockerClientService,+ settingsService *settings.SettingsService,+ eventService *event.EventService,+ notificationService *notification.NotificationService,admissionGate *actors.Gate[actors.AdmissionKey],) (*AutoHealJob, error) {if admissionGate == nil {@@ -119,7 +123,7 @@ func (j *AutoHealJob) Run(ctx context.Context) {return}- containerList, err := j.listContainersInternal(ctx, dockerClient)+ containerList, err := j.ListContainers(ctx, dockerClient)if err != nil {slog.ErrorContext(ctx, "auto-heal failed to list containers", "error", err)return@@ -418,7 +422,7 @@ func (j *AutoHealJob) getDockerClientInternal(ctx context.Context) (*client.Cliereturn j.dockerClientService.GetClient(ctx)}-func (j *AutoHealJob) listContainersInternal(ctx context.Context, dockerClient *client.Client) ([]container.Summary, error) {+func (j *AutoHealJob) ListContainers(ctx context.Context, dockerClient *client.Client) ([]container.Summary, error) {if j.listContainers != nil {return j.listContainers(ctx, dockerClient)}diff --git a/backend/pkg/scheduler/auto_update_job.go b/backend/pkg/scheduler/auto_update_job.goindex 50cdcf331..2bd847e5d 100644--- a/backend/pkg/scheduler/auto_update_job.go+++ b/backend/pkg/scheduler/auto_update_job.go@@ -4,10 +4,12 @@ import ("context""log/slog"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/updater"+"emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/actors"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/types/v2/updater"+ updatertypes "github.com/getarcaneapp/arcane/types/v2/updater")const autoUpdateAdmissionScopeInternal = "auto-update"@@ -15,21 +17,21 @@ const autoUpdateAdmissionScopeInternal = "auto-update"// pendingUpdateApplierInternal is the slice of UpdaterService the job needs,// kept as an interface so the overlap guard is testable with a fake.type pendingUpdateApplierInternal interface {- ApplyPending(ctx context.Context, options updater.Options) (*updater.Result, error)+ ApplyPending(ctx context.Context, options updatertypes.Options) (*updatertypes.Result, error)}type AutoUpdateJob struct {updaterService pendingUpdateApplierInternal- settingsService *services.SettingsService+ settingsService *settings.SettingsServiceadmissionGate *actors.Gate[actors.AdmissionKey]}-func NewAutoUpdateJob(updaterService *services.UpdaterService, settingsService *services.SettingsService, admissionGate *actors.Gate[actors.AdmissionKey]) (*AutoUpdateJob, error) {+func NewAutoUpdateJob(updaterModule *updater.Module, settingsService *settings.SettingsService, admissionGate *actors.Gate[actors.AdmissionKey]) (*AutoUpdateJob, error) {if admissionGate == nil {return nil, errors.New("auto-update admission gate unavailable")}return &AutoUpdateJob{- updaterService: updaterService,+ updaterService: updaterModule.Service(),settingsService: settingsService,admissionGate: admissionGate,}, nil@@ -75,7 +77,7 @@ func (j *AutoUpdateJob) Run(ctx context.Context) {slog.InfoContext(ctx, "auto-update run started")- result, err := j.updaterService.ApplyPending(ctx, updater.Options{})+ result, err := j.updaterService.ApplyPending(ctx, updatertypes.Options{})if err != nil {slog.ErrorContext(ctx, "auto-update run failed", "err", err)returndiff --git a/backend/pkg/scheduler/docker_client_refresh_job.go b/backend/pkg/scheduler/docker_client_refresh_job.goindex 21c02de7d..b3ab4cc64 100644--- a/backend/pkg/scheduler/docker_client_refresh_job.go+++ b/backend/pkg/scheduler/docker_client_refresh_job.go@@ -4,7 +4,9 @@ import ("context""log/slog"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"++ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/robfig/cron/v3")@@ -14,12 +16,12 @@ const dockerClientRefreshDefaultSchedule = "*/30 * * * * *"// DockerClientRefreshJob keeps the cached Docker client aligned with the daemon// API version after daemon restarts or upgrades.type DockerClientRefreshJob struct {- dockerClientService *services.DockerClientService- settingsService *services.SettingsService+ dockerClientService *docker.DockerClientService+ settingsService *settings.SettingsService}// NewDockerClientRefreshJob creates the scheduled Docker client refresh job.-func NewDockerClientRefreshJob(dockerClientService *services.DockerClientService, settingsService *services.SettingsService) *DockerClientRefreshJob {+func NewDockerClientRefreshJob(dockerClientService *docker.DockerClientService, settingsService *settings.SettingsService) *DockerClientRefreshJob {return &DockerClientRefreshJob{dockerClientService: dockerClientService,settingsService: settingsService,diff --git a/backend/pkg/scheduler/entityjobs/entityjobs.go b/backend/pkg/scheduler/entityjobs/entityjobs.gonew file mode 100644index 000000000..3c854880f--- /dev/null+++ b/backend/pkg/scheduler/entityjobs/entityjobs.go@@ -0,0 +1,107 @@+// Package entityjobs holds the per-entity dynamic-job registry shared by the+// services that schedule one job per database row (GitOps syncs, environment+// health checks). It lives beside the scheduler rather than inside it because+// package scheduler imports the services that would use this registry.+package entityjobs++import (+ "context"+ "log/slog"++ "emperror.dev/errors"+ schedulertypes "github.com/getarcaneapp/arcane/types/v2/scheduler"++ "github.com/getarcaneapp/arcane/backend/v2/internal/actors"+)++// GitOpsSyncJobPrefix is shared by the GitOps registry and environment cleanup.+const GitOpsSyncJobPrefix = "gitops-sync:"++// Registry owns one dynamic scheduler job per entity ID, plus the admission+// gate that keeps a single run of any one entity in flight at a time.+//+// The scheduler and the app lifecycle context arrive post-construction via+// SetScheduler: the manager wires them during bootstrap, and agent mode leaves+// them nil so every registration becomes a no-op.+type Registry struct {+ jobPrefix string+ admissionScope string++ lifecycleCtx context.Context+ scheduler schedulertypes.DynamicScheduler+ admissionGate *actors.Gate[actors.AdmissionKey]+}++// New creates a registry whose job names are jobPrefix+entityID and whose+// admission keys are scoped to admissionScope.+func New(jobPrefix, admissionScope string) *Registry {+ return &Registry{jobPrefix: jobPrefix, admissionScope: admissionScope}+}++// SetScheduler injects the job scheduler, the admission gate and the app+// lifecycle context. The lifecycle context is what scheduled runs execute on,+// so they outlive the request or bootstrap goroutine that registered them.+func (r *Registry) SetScheduler(ctx context.Context, scheduler schedulertypes.DynamicScheduler, admissionGate *actors.Gate[actors.AdmissionKey]) error { //nolint:contextcheck // scheduled runs must capture the app lifecycle context, not request contexts+ if scheduler == nil || admissionGate == nil {+ return errors.Errorf("%s scheduler dependencies unavailable", r.admissionScope)+ }+ if ctx == nil {+ ctx = context.Background()+ }+ r.lifecycleCtx = ctx+ r.scheduler = scheduler+ r.admissionGate = admissionGate+ return nil+}++// Enabled reports whether a scheduler has been injected.+func (r *Registry) Enabled() bool { return r.scheduler != nil }++// Scheduler exposes the injected scheduler for the rare caller that must touch+// a job this registry does not own (nil when none was injected).+func (r *Registry) Scheduler() schedulertypes.DynamicScheduler { return r.scheduler }++// Context returns the context scheduled work should run on: the app lifecycle+// context when one was injected, otherwise a cancel-detached copy of ctx.+func (r *Registry) Context(ctx context.Context) context.Context {+ if r.lifecycleCtx != nil {+ return r.lifecycleCtx+ }+ if ctx != nil {+ return context.WithoutCancel(ctx)+ }+ return context.Background()+}++// JobName returns the scheduler job name for entityID.+func (r *Registry) JobName(entityID string) string { return r.jobPrefix + entityID }++// Register adds (or replaces) the dynamic job for entityID.+func (r *Registry) Register(ctx context.Context, entityID string, schedule func(context.Context) string, run func(context.Context)) {+ if r.scheduler == nil {+ return+ }+ schedulerCtx := r.Context(ctx)+ job := &schedulertypes.GenericJob{+ JobName: r.JobName(entityID),+ ScheduleFn: schedule,+ RunFn: run,+ }+ if err := r.scheduler.AddJob(schedulerCtx, job); err != nil {+ slog.ErrorContext(schedulerCtx, "failed to register scheduled job", "job", job.JobName, "error", err)+ }+}++// Unregister removes the dynamic job for entityID.+func (r *Registry) Unregister(ctx context.Context, entityID string) {+ if r.scheduler == nil {+ return+ }+ r.scheduler.RemoveJob(r.Context(ctx), r.JobName(entityID))+}++// TryAcquire admits at most one in-flight run per entity ID. It refuses+// immediately when a run is already active.+func (r *Registry) TryAcquire(ctx context.Context, entityID string) (*actors.Lease[actors.AdmissionKey], bool, error) {+ return r.admissionGate.TryAcquire(ctx, actors.AdmissionKey{Scope: r.admissionScope, ID: entityID})+}diff --git a/backend/pkg/scheduler/event_cleanup_job.go b/backend/pkg/scheduler/event_cleanup_job.goindex 30952499f..2444d0fa1 100644--- a/backend/pkg/scheduler/event_cleanup_job.go+++ b/backend/pkg/scheduler/event_cleanup_job.go@@ -5,18 +5,20 @@ import ("log/slog""time"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/activity"+ "github.com/getarcaneapp/arcane/backend/v2/internal/event"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings")const EventCleanupJobName = "event-cleanup"type EventCleanupJob struct {- eventService *services.EventService- activityService *services.ActivityService- settingsService *services.SettingsService+ eventService *event.EventService+ activityService *activity.ActivityService+ settingsService *settings.SettingsService}-func NewEventCleanupJob(eventService *services.EventService, activityService *services.ActivityService, settingsService *services.SettingsService) *EventCleanupJob {+func NewEventCleanupJob(eventService *event.EventService, activityService *activity.ActivityService, settingsService *settings.SettingsService) *EventCleanupJob {return &EventCleanupJob{eventService: eventService,activityService: activityService,diff --git a/backend/pkg/scheduler/expired_sessions_cleanup_job.go b/backend/pkg/scheduler/expired_sessions_cleanup_job.goindex 7b84fa57b..feec763bc 100644--- a/backend/pkg/scheduler/expired_sessions_cleanup_job.go+++ b/backend/pkg/scheduler/expired_sessions_cleanup_job.go@@ -5,17 +5,18 @@ import ("log/slog""time"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/session"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings")const ExpiredSessionsCleanupJobName = "expired-sessions-cleanup"type ExpiredSessionsCleanupJob struct {- sessionService *services.SessionService- settingsService *services.SettingsService+ sessionService *session.SessionService+ settingsService *settings.SettingsService}-func NewExpiredSessionsCleanupJob(sessionService *services.SessionService, settingsService *services.SettingsService) *ExpiredSessionsCleanupJob {+func NewExpiredSessionsCleanupJob(sessionService *session.SessionService, settingsService *settings.SettingsService) *ExpiredSessionsCleanupJob {return &ExpiredSessionsCleanupJob{sessionService: sessionService,settingsService: settingsService,diff --git a/backend/pkg/scheduler/filesystem_watcher_job.go b/backend/pkg/scheduler/filesystem_watcher_job.goindex 40ffb3841..849524737 100644--- a/backend/pkg/scheduler/filesystem_watcher_job.go+++ b/backend/pkg/scheduler/filesystem_watcher_job.go@@ -8,15 +8,17 @@ import ("emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/actors"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/template""github.com/getarcaneapp/arcane/backend/v2/pkg/fswatch""github.com/getarcaneapp/arcane/backend/v2/pkg/projects")type FilesystemWatcherJob struct {- projectService *services.ProjectService- templateService *services.TemplateService- settingsService *services.SettingsService+ projectService *project.ProjectService+ templateService *template.TemplateService+ settingsService *settings.SettingsServiceprojectScanDepth intlifecycleCtx context.ContextprojectsWatcher *actors.Resource[*fswatch.Watcher]@@ -26,9 +28,9 @@ type FilesystemWatcherJob struct {func NewFilesystemWatcherJob(ctx context.Context,actorRuntime *actors.Runtime,- projectService *services.ProjectService,- templateService *services.TemplateService,- settingsService *services.SettingsService,+ projectService *project.ProjectService,+ templateService *template.TemplateService,+ settingsService *settings.SettingsService,projectScanDepth int,) (*FilesystemWatcherJob, error) {projectsWatcher, err := actors.NewResource(ctx, actorRuntime, "filesystem-watcher", "projects", 3, (*fswatch.Watcher).Stop)diff --git a/backend/pkg/scheduler/image_update_watcher.go b/backend/pkg/scheduler/image_update_watcher.goindex bc4e89e2d..41d3cc853 100644--- a/backend/pkg/scheduler/image_update_watcher.go+++ b/backend/pkg/scheduler/image_update_watcher.go@@ -7,15 +7,20 @@ import ("sync/atomic""time"+ "github.com/getarcaneapp/arcane/backend/v2/internal/docker"+ "github.com/getarcaneapp/arcane/backend/v2/internal/environment"+ "github.com/getarcaneapp/arcane/backend/v2/internal/imageupdate"+ "github.com/getarcaneapp/arcane/backend/v2/internal/project"+"emperror.dev/errors""github.com/getarcaneapp/arcane/backend/v2/internal/actors""github.com/getarcaneapp/arcane/backend/v2/internal/common""github.com/getarcaneapp/arcane/backend/v2/internal/config"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- docker "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ dockerutil "github.com/getarcaneapp/arcane/backend/v2/pkg/dockerutil""github.com/getarcaneapp/arcane/types/v2/containerregistry"- "github.com/getarcaneapp/arcane/types/v2/imageupdate"+ imageupdatetypes "github.com/getarcaneapp/arcane/types/v2/imageupdate""github.com/moby/moby/api/types/events""go.getarcane.app/streams/bus")@@ -29,7 +34,7 @@ const ()type imageUpdateScannerInternal interface {- CheckAllImages(ctx context.Context, limit int, externalCreds []containerregistry.Credential) (map[string]*imageupdate.Response, error)+ CheckAllImages(ctx context.Context, limit int, externalCreds []containerregistry.Credential) (map[string]*imageupdatetypes.Response, error)}type registryCredentialLoaderInternal interface {@@ -92,7 +97,7 @@ type ImageUpdateWatcher struct {}// NewImageUpdateWatcher constructs the image update watcher from the existing services.-func NewImageUpdateWatcher(runtime *actors.Runtime, cfg *config.Config, imageUpdateService *services.ImageUpdateService, settingsService *services.SettingsService, environmentService *services.EnvironmentService, dockerService *services.DockerClientService, projectService *services.ProjectService) (*ImageUpdateWatcher, error) {+func NewImageUpdateWatcher(runtime *actors.Runtime, cfg *config.Config, imageUpdateService *imageupdate.ImageUpdateService, settingsService *settings.SettingsService, environmentService *environment.EnvironmentService, dockerService *docker.DockerClientService, projectService *project.ProjectService) (*ImageUpdateWatcher, error) {if runtime == nil || imageUpdateService == nil || settingsService == nil || environmentService == nil || dockerService == nil || projectService == nil {return nil, errors.New("image update watcher dependencies unavailable")}@@ -194,7 +199,7 @@ func (w *ImageUpdateWatcher) Start(ctx context.Context) error {eventCh = nilcontinue}- if message.Action == docker.ImageStateResyncAction || !w.settingsService.GetBoolSetting(ctx, "imageEventWatcherEnabled", false) {+ if message.Action == dockerutil.ImageStateResyncAction || !w.settingsService.GetBoolSetting(ctx, "imageEventWatcherEnabled", false) {continue}w.Trigger()diff --git a/backend/pkg/scheduler/job_schedule_integration_test.go b/backend/pkg/scheduler/job_schedule_integration_test.goindex 15e33253e..9612f22e9 100644--- a/backend/pkg/scheduler/job_schedule_integration_test.go+++ b/backend/pkg/scheduler/job_schedule_integration_test.go@@ -7,8 +7,9 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/config""github.com/getarcaneapp/arcane/backend/v2/internal/database"+ "github.com/getarcaneapp/arcane/backend/v2/internal/job""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/types/v2/jobschedule""github.com/libtnb/sqlite""github.com/stretchr/testify/require"@@ -25,7 +26,7 @@ func TestDeprecatedImagePollingSchedulePersistsWithoutRuntimeJob(t *testing.T) {lifecycleCtx, cancelLifecycle := context.WithCancel(context.Background())jobScheduler := newJobSchedulerForTestInternal(t, lifecycleCtx, appConfig.GetLocation())- jobService := services.NewJobService(db, settingsService, appConfig)+ jobService := job.NewJobService(db, settingsService, appConfig)jobService.SetScheduler(lifecycleCtx, jobScheduler)require.NoError(t, jobScheduler.StartScheduler())@@ -64,7 +65,7 @@ func TestDeprecatedImagePollingSchedulePersistsWithoutRuntimeJob(t *testing.T) {restartDB, restartSettingsService := openJobScheduleTestDatabaseInternal(t, ctx, databasePath)restartLifecycleCtx, cancelRestartLifecycle := context.WithCancel(context.Background())restartScheduler := newJobSchedulerForTestInternal(t, restartLifecycleCtx, appConfig.GetLocation())- restartJobService := services.NewJobService(restartDB, restartSettingsService, appConfig)+ restartJobService := job.NewJobService(restartDB, restartSettingsService, appConfig)restartJobService.SetScheduler(restartLifecycleCtx, restartScheduler)require.NoError(t, restartScheduler.StartScheduler())t.Cleanup(func() {@@ -98,7 +99,7 @@ func TestDeprecatedImagePollingSchedulePersistsWithoutRuntimeJob(t *testing.T) {require.Equal(t, disabledSchedule, disabledStatus.Schedule)}-func openJobScheduleTestDatabaseInternal(t *testing.T, ctx context.Context, databasePath string) (*database.DB, *services.SettingsService) {+func openJobScheduleTestDatabaseInternal(t *testing.T, ctx context.Context, databasePath string) (*database.DB, *settings.SettingsService) {t.Helper()gormDB, err := gorm.Open(sqlite.Open(databasePath), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})diff --git a/backend/pkg/scheduler/scheduled_prune_job.go b/backend/pkg/scheduler/scheduled_prune_job.goindex 4a8517b5e..bb701b80a 100644--- a/backend/pkg/scheduler/scheduled_prune_job.go+++ b/backend/pkg/scheduler/scheduled_prune_job.go@@ -4,22 +4,25 @@ import ("context""log/slog"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"- "github.com/getarcaneapp/arcane/types/v2/system"+ "github.com/getarcaneapp/arcane/backend/v2/internal/notification"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ systemtypes "github.com/getarcaneapp/arcane/types/v2/system"++ "github.com/getarcaneapp/arcane/backend/v2/internal/system""github.com/robfig/cron/v3")const ScheduledPruneJobName = "scheduled-prune"type ScheduledPruneJob struct {- systemService *services.SystemService- settingsService *services.SettingsService- notificationService *services.NotificationService+ systemService *system.SystemService+ settingsService *settings.SettingsService+ notificationService *notification.NotificationService}-func NewScheduledPruneJob(systemService *services.SystemService, settingsService *services.SettingsService, notificationService *services.NotificationService) *ScheduledPruneJob {+func NewScheduledPruneJob(systemModule *system.Module, settingsService *settings.SettingsService, notificationService *notification.NotificationService) *ScheduledPruneJob {return &ScheduledPruneJob{- systemService: systemService,+ systemService: systemModule.Service(),settingsService: settingsService,notificationService: notificationService,}@@ -104,8 +107,8 @@ func (j *ScheduledPruneJob) Reschedule(ctx context.Context) error {return nil}-func buildScheduledPruneRequestInternal(ctx context.Context, settingsService *services.SettingsService) system.PruneAllRequest {- return system.PruneAllRequest{+func buildScheduledPruneRequestInternal(ctx context.Context, settingsService *settings.SettingsService) systemtypes.PruneAllRequest {+ return systemtypes.PruneAllRequest{Containers: buildScheduledContainerPruneOptionsInternal(ctx, settingsService),Images: buildScheduledImagePruneOptionsInternal(ctx, settingsService),Volumes: buildScheduledVolumePruneOptionsInternal(ctx, settingsService),@@ -114,63 +117,63 @@ func buildScheduledPruneRequestInternal(ctx context.Context, settingsService *se}}-func hasScheduledPruneTargetsInternal(req system.PruneAllRequest) bool {+func hasScheduledPruneTargetsInternal(req systemtypes.PruneAllRequest) bool {return req.Containers != nil || req.Images != nil || req.Volumes != nil || req.Networks != nil || req.BuildCache != nil}-func buildScheduledContainerPruneOptionsInternal(ctx context.Context, settingsService *services.SettingsService) *system.PruneContainersOptions {+func buildScheduledContainerPruneOptionsInternal(ctx context.Context, settingsService *settings.SettingsService) *systemtypes.PruneContainersOptions {mode := settingsService.GetStringSetting(ctx, "pruneContainerMode", "stopped")- if mode == "" || mode == string(system.PruneContainerModeNone) {+ if mode == "" || mode == string(systemtypes.PruneContainerModeNone) {return nil}- return &system.PruneContainersOptions{- Mode: system.PruneContainerMode(mode),+ return &systemtypes.PruneContainersOptions{+ Mode: systemtypes.PruneContainerMode(mode),Until: settingsService.GetStringSetting(ctx, "pruneContainerUntil", ""),}}-func buildScheduledImagePruneOptionsInternal(ctx context.Context, settingsService *services.SettingsService) *system.PruneImagesOptions {+func buildScheduledImagePruneOptionsInternal(ctx context.Context, settingsService *settings.SettingsService) *systemtypes.PruneImagesOptions {mode := settingsService.GetStringSetting(ctx, "pruneImageMode", "dangling")- if mode == "" || mode == string(system.PruneImageModeNone) {+ if mode == "" || mode == string(systemtypes.PruneImageModeNone) {return nil}- return &system.PruneImagesOptions{- Mode: system.PruneImageMode(mode),+ return &systemtypes.PruneImagesOptions{+ Mode: systemtypes.PruneImageMode(mode),Until: settingsService.GetStringSetting(ctx, "pruneImageUntil", ""),}}-func buildScheduledVolumePruneOptionsInternal(ctx context.Context, settingsService *services.SettingsService) *system.PruneVolumesOptions {+func buildScheduledVolumePruneOptionsInternal(ctx context.Context, settingsService *settings.SettingsService) *systemtypes.PruneVolumesOptions {mode := settingsService.GetStringSetting(ctx, "pruneVolumeMode", "none")- if mode == "" || mode == string(system.PruneVolumeModeNone) {+ if mode == "" || mode == string(systemtypes.PruneVolumeModeNone) {return nil}- return &system.PruneVolumesOptions{Mode: system.PruneVolumeMode(mode)}+ return &systemtypes.PruneVolumesOptions{Mode: systemtypes.PruneVolumeMode(mode)}}-func buildScheduledNetworkPruneOptionsInternal(ctx context.Context, settingsService *services.SettingsService) *system.PruneNetworksOptions {+func buildScheduledNetworkPruneOptionsInternal(ctx context.Context, settingsService *settings.SettingsService) *systemtypes.PruneNetworksOptions {mode := settingsService.GetStringSetting(ctx, "pruneNetworkMode", "unused")- if mode == "" || mode == string(system.PruneNetworkModeNone) {+ if mode == "" || mode == string(systemtypes.PruneNetworkModeNone) {return nil}- return &system.PruneNetworksOptions{- Mode: system.PruneNetworkMode(mode),+ return &systemtypes.PruneNetworksOptions{+ Mode: systemtypes.PruneNetworkMode(mode),Until: settingsService.GetStringSetting(ctx, "pruneNetworkUntil", ""),}}-func buildScheduledBuildCachePruneOptionsInternal(ctx context.Context, settingsService *services.SettingsService) *system.PruneBuildCacheOptions {+func buildScheduledBuildCachePruneOptionsInternal(ctx context.Context, settingsService *settings.SettingsService) *systemtypes.PruneBuildCacheOptions {mode := settingsService.GetStringSetting(ctx, "pruneBuildCacheMode", "none")- if mode == "" || mode == string(system.PruneBuildCacheModeNone) {+ if mode == "" || mode == string(systemtypes.PruneBuildCacheModeNone) {return nil}- return &system.PruneBuildCacheOptions{- Mode: system.PruneBuildCacheMode(mode),+ return &systemtypes.PruneBuildCacheOptions{+ Mode: systemtypes.PruneBuildCacheMode(mode),Until: settingsService.GetStringSetting(ctx, "pruneBuildCacheUntil", ""),}}diff --git a/backend/pkg/scheduler/scheduled_prune_job_test.go b/backend/pkg/scheduler/scheduled_prune_job_test.goindex a7cc2c63d..a79393831 100644--- a/backend/pkg/scheduler/scheduled_prune_job_test.go+++ b/backend/pkg/scheduler/scheduled_prune_job_test.go@@ -10,11 +10,11 @@ import ("github.com/getarcaneapp/arcane/backend/v2/internal/database""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings""github.com/getarcaneapp/arcane/types/v2/system")-func setupScheduledPruneSettingsServiceInternal(t *testing.T) *services.SettingsService {+func setupScheduledPruneSettingsServiceInternal(t *testing.T) *settings.SettingsService {t.Helper()db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})diff --git a/backend/pkg/scheduler/volume_helper_reaper_job.go b/backend/pkg/scheduler/volume_helper_reaper_job.goindex beca81c66..9be277f3f 100644--- a/backend/pkg/scheduler/volume_helper_reaper_job.go+++ b/backend/pkg/scheduler/volume_helper_reaper_job.go@@ -5,7 +5,8 @@ import ("log/slog""time"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/volume")const (@@ -22,13 +23,13 @@ const (// containers. The run frequency is fixed (every 5 minutes); how stale a helper must// be to be pruned is driven by the volumeBrowserHelperIdleTimeout setting.type PruningVolumeHelperJob struct {- volumeService *services.VolumeService- settingsService *services.SettingsService+ volumeService *volume.VolumeService+ settingsService *settings.SettingsService}-func NewPruningVolumeHelperJob(volumeService *services.VolumeService, settingsService *services.SettingsService) *PruningVolumeHelperJob {+func NewPruningVolumeHelperJob(volumeModule *volume.Module, settingsService *settings.SettingsService) *PruningVolumeHelperJob {return &PruningVolumeHelperJob{- volumeService: volumeService,+ volumeService: volumeModule.Service(),settingsService: settingsService,}}diff --git a/backend/pkg/scheduler/vulnerability_scan_job.go b/backend/pkg/scheduler/vulnerability_scan_job.goindex f00fd7959..5d5160eeb 100644--- a/backend/pkg/scheduler/vulnerability_scan_job.go+++ b/backend/pkg/scheduler/vulnerability_scan_job.go@@ -5,7 +5,8 @@ import ("log/slog""github.com/getarcaneapp/arcane/backend/v2/internal/models"- "github.com/getarcaneapp/arcane/backend/v2/internal/services"+ "github.com/getarcaneapp/arcane/backend/v2/internal/settings"+ "github.com/getarcaneapp/arcane/backend/v2/internal/vulnerability""github.com/getarcaneapp/arcane/types/v2""github.com/robfig/cron/v3")@@ -20,12 +21,12 @@ var vulnerabilityScanSystemUser = models.User{// VulnerabilityScanJob periodically scans all Docker images for vulnerabilities// using Trivy. It is opt-in via the "vulnerabilityScanEnabled" setting.type VulnerabilityScanJob struct {- vulnerabilityService *services.VulnerabilityService- settingsService *services.SettingsService+ vulnerabilityService *vulnerability.VulnerabilityService+ settingsService *settings.SettingsService}// NewVulnerabilityScanJob creates a new VulnerabilityScanJob.-func NewVulnerabilityScanJob(vulnerabilityService *services.VulnerabilityService, settingsService *services.SettingsService) *VulnerabilityScanJob {+func NewVulnerabilityScanJob(vulnerabilityService *vulnerability.VulnerabilityService, settingsService *settings.SettingsService) *VulnerabilityScanJob {return &VulnerabilityScanJob{vulnerabilityService: vulnerabilityService,settingsService: settingsService,diff --git a/backend/pkg/utils/dbutil/dbutil.go b/backend/pkg/utils/dbutil/dbutil.goindex 770d5bf8e..83b750267 100644--- a/backend/pkg/utils/dbutil/dbutil.go+++ b/backend/pkg/utils/dbutil/dbutil.go@@ -4,12 +4,28 @@ package dbutilimport ("context"+ "strings""emperror.dev/errors""gorm.io/gorm")+// IsRetryableWriteError reports whether a database write failed because the+// database was temporarily busy or timed out.+func IsRetryableWriteError(err error) bool {+ if err == nil {+ return false+ }++ errMsg := strings.ToLower(strings.TrimSpace(err.Error()))+ return strings.Contains(errMsg, "database is locked") ||+ strings.Contains(errMsg, "database table is locked") ||+ strings.Contains(errMsg, "database is busy") ||+ strings.Contains(errMsg, "resource busy") ||+ strings.Contains(errMsg, "timeout")+}+// FirstWhere fetches a single row of T matching the provided WHERE clause.// It maps gorm.ErrRecordNotFound to notFound (when non-nil) and otherwise wraps// the error with a generic "failed to query <T>" prefix.diff --git a/backend/pkg/utils/handlerutil/handlerutil.go b/backend/pkg/utils/handlerutil/handlerutil.gonew file mode 100644index 000000000..0d04d3c7b--- /dev/null+++ b/backend/pkg/utils/handlerutil/handlerutil.go@@ -0,0 +1,147 @@+package handlerutil++import (+ "context"+ "mime/multipart"+ "strings"++ "emperror.dev/errors"++ "github.com/danielgtaylor/huma/v2"+ "github.com/getarcaneapp/arcane/backend/v2/internal/middleware"+ "github.com/getarcaneapp/arcane/backend/v2/internal/models"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/pagination"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/utils/mapper"+ "github.com/getarcaneapp/arcane/types/v2/auth"+ "github.com/getarcaneapp/arcane/types/v2/base"+)++// ActivityAppContext carries the app lifecycle context through handler registration.+type ActivityAppContext struct {+ ctx context.Context+}++// NewActivityAppContext wraps the app lifecycle context for handler constructors.+func NewActivityAppContext(ctx context.Context) ActivityAppContext {+ return ActivityAppContext{ctx: ctx}+}++// Context returns the wrapped app lifecycle context.+func (c ActivityAppContext) Context() context.Context {+ return c.ctx+}++// PaginationParams converts query parameters to pagination.QueryParams.+// A limit of -1 means "show all items" (no pagination).+func PaginationParams(start, limit int, sortCol, sortDir, search string) pagination.QueryParams {+ // limit = -1 means "show all" and 0 means "no pagination"; only invalid values default to 20+ if limit < -1 {+ limit = 20+ }++ return pagination.QueryParams{+ SearchQuery: pagination.SearchQuery{+ Search: strings.TrimSpace(search),+ },+ SortParams: pagination.SortParams{+ Sort: strings.TrimSpace(sortCol),+ Order: pagination.SortOrder(sortDir),+ },+ Params: pagination.Params{+ Start: start,+ Limit: limit,+ },+ Filters: make(map[string]string),+ }+}++// PaginationResponse converts the pagination package's response into the API response shape.+func PaginationResponse(p pagination.Response) base.PaginationResponse {+ return base.PaginationResponse{+ TotalPages: p.TotalPages,+ TotalItems: p.TotalItems,+ CurrentPage: p.CurrentPage,+ ItemsPerPage: p.ItemsPerPage,+ GrandTotalItems: p.GrandTotalItems,+ }+}++// RequireUser returns the authenticated user from the request context or a 401 error.+func RequireUser(ctx context.Context) (*models.User, error) {+ user, exists := models.CurrentUserFromContext(ctx)+ if !exists || user == nil {+ return nil, huma.Error401Unauthorized("Not authenticated")+ }+ return user, nil+}++func OpenUploadedFile(form multipart.Form) (multipart.File, *multipart.FileHeader, error) {+ files := form.File["file"]+ if len(files) == 0 {+ return nil, nil, huma.Error400BadRequest("No file uploaded")+ }++ fileHeader := files[0]+ file, err := fileHeader.Open()+ if err != nil {+ return nil, nil, huma.Error500InternalServerError(errors.WithMessage(err, "Failed to read upload").Error())+ }++ return file, fileHeader, nil+}++func RegisterSecured[I, O any](+ api huma.API,+ op huma.Operation,+ permission string,+ handler func(context.Context, *I) (*O, error),+) {+ op.Security = DefaultOperationSecurity()+ middleware.RegisterWithPermission(api, op, permission, handler)+}++func Operation(operationID, method, path, summary, description string, tags ...string) huma.Operation {+ return huma.Operation{+ OperationID: operationID,+ Method: method,+ Path: path,+ Summary: summary,+ Description: description,+ Tags: tags,+ }+}++func CurrentActor(ctx context.Context) models.User {+ actor := models.User{}+ if currentUser, exists := models.CurrentUserFromContext(ctx); exists && currentUser != nil {+ actor = *currentUser+ }+ return actor+}++func MapOneAPIResponse[S any, D any](source S, mappingMessage func(error) string) (base.ApiResponse[D], error) {+ out, err := mapper.MapOne[S, D](source)+ if err != nil {+ return base.ApiResponse[D]{}, huma.Error500InternalServerError(mappingMessage(err))+ }++ return base.ApiResponse[D]{+ Success: true,+ Data: out,+ }, nil+}++func DefaultOperationSecurity() []map[string][]string {+ return []map[string][]string{+ {"BearerAuth": {}},+ {"ApiKeyAuth": {}},+ }+}++// SessionMetaFromContext builds the session metadata recorded on login.+func SessionMetaFromContext(ctx context.Context, userAgent string) auth.SessionMeta {+ return auth.SessionMeta{+ UserAgent: userAgent,+ IPAddress: middleware.GetRemoteAddrFromContext(ctx),+ }+}diff --git a/backend/pkg/utils/handlerutil/remote_proxy.go b/backend/pkg/utils/handlerutil/remote_proxy.gonew file mode 100644index 000000000..11bc6a52a--- /dev/null+++ b/backend/pkg/utils/handlerutil/remote_proxy.go@@ -0,0 +1,63 @@+package handlerutil++import (+ "context"+ "encoding/json/v2"+ stderrors "errors"++ "github.com/danielgtaylor/huma/v2"+ "github.com/getarcaneapp/arcane/backend/v2/pkg/remenv"+)++// RemoteJSONProxy is the remote environment transport operation used by domain handlers.+type RemoteJSONProxy func(context.Context, string, string, string, []byte, any) error++// ProxyRemoteJSON marshals a request, invokes an environment proxy, and translates transport errors for Huma.+func ProxyRemoteJSON[T any](+ ctx context.Context,+ proxy RemoteJSONProxy,+ environmentID string,+ method string,+ path string,+ requestBody any,+) (*T, error) {+ body, err := marshalRemoteRequestBodyInternal(requestBody)+ if err != nil {+ return nil, huma.Error500InternalServerError("failed to marshal request body: " + err.Error())+ }++ var output T+ if err := proxy(ctx, environmentID, method, path, body, &output); err != nil {+ return nil, TranslateRemoteProxyError(err)+ }++ return &output, nil+}++func marshalRemoteRequestBodyInternal(requestBody any) ([]byte, error) {+ switch value := requestBody.(type) {+ case nil:+ return nil, nil+ case []byte:+ return value, nil+ default:+ return json.Marshal(value)+ }+}++// TranslateRemoteProxyError converts environment transport failures into Huma errors.+func TranslateRemoteProxyError(err error) error {+ if transportErr, ok := stderrors.AsType[*remenv.TransportError](err); ok {+ return huma.Error502BadGateway("failed to proxy request to environment: " + transportErr.Error())+ }++ if statusErr, ok := stderrors.AsType[*remenv.StatusError](err); ok {+ return huma.NewError(statusErr.StatusCode, "environment returned error: "+string(statusErr.Body), nil)+ }++ if decodeErr, ok := stderrors.AsType[*remenv.DecodeError](err); ok {+ return huma.Error500InternalServerError("failed to decode environment response: " + decodeErr.Error())+ }++ return huma.Error500InternalServerError("failed to proxy request to environment: " + err.Error())+}diff --git a/backend/pkg/utils/httpx/base_url.go b/backend/pkg/utils/httpx/base_url.gonew file mode 100644index 000000000..3aa967d15--- /dev/null+++ b/backend/pkg/utils/httpx/base_url.go@@ -0,0 +1,16 @@+package httpx++import "strings"++// NormalizeBaseURL validates an outbound HTTP URL and strips endpoint-specific components.+func NormalizeBaseURL(rawURL string) (string, error) {+ parsed, err := ValidateOutboundHTTPURL(rawURL)+ if err != nil {+ return "", err+ }++ parsed.RawQuery = ""+ parsed.Fragment = ""+ parsed.Path = strings.TrimRight(parsed.Path, "/")+ return parsed.String(), nil+}diff --git a/backend/pkg/utils/imageref/update_lookup.go b/backend/pkg/utils/imageref/update_lookup.gonew file mode 100644index 000000000..535d3b0a3--- /dev/null+++ b/backend/pkg/utils/imageref/update_lookup.go@@ -0,0 +1,54 @@+// Package imageref contains shared image-reference parsing operations.+package imageref++import (+ "fmt"+ "strings"++ ref "github.com/distribution/reference"+ registryauth "github.com/getarcaneapp/arcane/backend/v2/pkg/libarcane/registryauth"+)++// ParseUpdateLookup normalizes an image reference into the repository and tag+// candidates used to match persisted update records.+func ParseUpdateLookup(imageRef string) (originalRef, tag string, repositoryCandidates map[string]struct{}, ok bool) {+ trimmedRef := strings.TrimSpace(imageRef)+ if trimmedRef == "" {+ return "", "", nil, false+ }++ named, err := ref.ParseNormalizedNamed(trimmedRef)+ if err != nil {+ return "", "", nil, false+ }++ tag = "latest"+ if tagged, taggedOK := named.(ref.NamedTagged); taggedOK {+ tag = strings.TrimSpace(tagged.Tag())+ }+ if tag == "" {+ tag = "latest"+ }++ registryHost := registryauth.NormalizeRegistryForComparison(ref.Domain(named))+ repositoryPath := strings.TrimSpace(ref.Path(named))+ familiarRepository := strings.TrimSpace(ref.FamiliarName(named))++ repositoryCandidates = make(map[string]struct{})+ for _, candidate := range []string{+ repositoryPath,+ familiarRepository,+ fmt.Sprintf("%s/%s", registryHost, repositoryPath),+ } {+ candidate = strings.TrimSpace(candidate)+ if candidate != "" {+ repositoryCandidates[candidate] = struct{}{}+ }+ }++ if registryHost == "docker.io" && strings.HasPrefix(repositoryPath, "library/") {+ repositoryCandidates[strings.TrimPrefix(repositoryPath, "library/")] = struct{}{}+ }++ return trimmedRef, tag, repositoryCandidates, true+}diff --git a/backend/internal/logging/error_handler.go b/backend/pkg/utils/logging/error_handler.gosimilarity index 84%rename from backend/internal/logging/error_handler.gorename to backend/pkg/utils/logging/error_handler.goindex d8af575db..9c9273c55 100644--- a/backend/internal/logging/error_handler.go+++ b/backend/pkg/utils/logging/error_handler.go@@ -30,8 +30,3 @@ func (SlogErrorHandler) HandleContext(ctx context.Context, err error) {}slog.ErrorContext(ctx, "Unhandled error", attrs...)}--// NewSlogErrorHandler creates the application error handler.-func NewSlogErrorHandler() *SlogErrorHandler {- return &SlogErrorHandler{}-}diff --git a/backend/proto/tunnel/v1/tunnel.pb.go b/backend/proto/tunnel/v1/tunnel.pb.goindex a16bfc45e..0e2a2d958 100644--- a/backend/proto/tunnel/v1/tunnel.pb.go+++ b/backend/proto/tunnel/v1/tunnel.pb.go@@ -7,11 +7,12 @@package tunnelpbimport (- protoreflect "google.golang.org/protobuf/reflect/protoreflect"- protoimpl "google.golang.org/protobuf/runtime/protoimpl"reflect "reflect"sync "sync"unsafe "unsafe"++ protoreflect "google.golang.org/protobuf/reflect/protoreflect"+ protoimpl "google.golang.org/protobuf/runtime/protoimpl")const (diff --git a/backend/proto/tunnel/v1/tunnel_grpc.pb.go b/backend/proto/tunnel/v1/tunnel_grpc.pb.goindex eba05fd77..373719642 100644--- a/backend/proto/tunnel/v1/tunnel_grpc.pb.go+++ b/backend/proto/tunnel/v1/tunnel_grpc.pb.go@@ -8,6 +8,7 @@ package tunnelpbimport (context "context"+grpc "google.golang.org/grpc"codes "google.golang.org/grpc/codes"status "google.golang.org/grpc/status"diff --git a/types/containerregistry/container_registry.go b/types/containerregistry/container_registry.goindex 3f079e5b0..9b2e6aaa7 100644--- a/types/containerregistry/container_registry.go+++ b/types/containerregistry/container_registry.go@@ -246,6 +246,16 @@ type Credential struct {Enabled bool `json:"enabled"`}+// DigestResult describes a registry digest lookup and the authentication path+// used to complete it.+type DigestResult struct {+ Digest string+ AuthMethod string+ AuthUsername string+ AuthRegistry string+ UsedCredential bool+}+type SyncRequest struct {// Registries is a list of container registries to sync.//diff --git a/types/lifecycle/lifecycle.go b/types/lifecycle/lifecycle.gonew file mode 100644index 000000000..ceeb9a2b2--- /dev/null+++ b/types/lifecycle/lifecycle.go@@ -0,0 +1,9 @@+// Package lifecycle contains lifecycle-hook values shared across Arcane modules.+package lifecycle++// ExtraMount is one configured bind mount for a lifecycle runner container.+type ExtraMount struct {+ Source string+ Target string+ Readonly bool+}