Multiplayer agent harness for work.
Refuse a porter publish up front when no apps domain is set
Josh France3f071cabcee5
diff --git a/docs/porter.md b/docs/porter.mdindex b27bde389..2bf96a791 100644--- a/docs/porter.md+++ b/docs/porter.md@@ -29,16 +29,19 @@ for f in porter/apps/*.yaml; do porter apply -f "$f"; donePorter runs the services and assigns each **public** web service an `onporter.run`hostname with a Let's Encrypt certificate, so this path needs no DNS record, no TLScertificate, and no ingress controller. Only the portal is meant to be Internet-facing;-auth, web-ui, and admin carry `private: true` so Porter creates no ingress for them and-they are reachable only at their in-cluster address.+core, auth, web-ui, and admin carry `private: true` so Porter creates no ingress for them+and they are reachable only at their in-cluster address.Two things `porter apply` will not do for you:- **Building on an Apple Silicon machine fails.** `build.method: docker` builds locallyfor `linux/amd64` and the legacy builder cannot cross-compile: the apply dies with`image ... does not provide the specified platform (linux/amd64)`. Build and push with- `docker buildx build --platform linux/amd64 --push`, then point the app at the pushed- image with an `image: {repository, tag}` block, or use `porter apply --remote`.+ `docker buildx build --platform linux/amd64 --push` to the registry Porter already+ connected for the project (`porter registry list` prints it), then point the app at the+ pushed image with an `image: {repository, tag}` block. `porter apply --remote` builds+ server-side but is an opt-in project feature and fails with `remote build is not+enabled for this project` until Porter enables it.- **Env groups created by `porter env create` are not visible to `porter apply`.** Boththe `envGroups:` key and `--attach-env-groups` fail with `internal: unable to findlatest environment with provided name`, because `porter env create` makes a@@ -49,15 +52,24 @@ latest environment with provided name`, because `porter env create` makes aNothing generates the inter-service wiring for you — `cli/src/services.ts` has `fly` and`docker` targets but no Porter one — so set it by hand on each app:-| Service | Wiring |-| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |-| all but core | `CORE_API_URL=http://qm-core-core.<namespace>.svc.cluster.local:8080`, `CORE_ORG_ID`, `CORE_SIGNING_SECRET`, `PORTAL_IDENTITY_SECRET` |-| core | `ORG_ID`, `MODEL_PROVIDER`, `HARNESS`, `SANDBOX_BACKEND`, `PUBLIC_WEB_URL`/`WEB_UI_PUBLIC_URL` (the portal's hostname), `CAPABILITY_SECRET`, `CONNECTOR_SECRET_KEY`, `SKILL_SIGNING_SECRET`, and `DATABASE_URL` when `SESSION_STORE`/`RUN_STORE` are `postgres` |-| portal | `PORTAL_PUBLIC_URL`, `PORTAL_SESSION_SECRET`, `WEB_UI_UPSTREAM`, `ADMIN_UPSTREAM`, `AUTH_BROKER_UPSTREAM`, `AUTH_BROKER_PREFIX=/idp`, and the `OIDC_*` set from `brokerWiring` in `cli/src/services.ts` |-| auth | `AUTH_ISSUER=<portal>/idp`, `AUTH_CLIENT_ID`, `AUTH_CLIENT_SECRET`, `AUTH_REDIRECT_URI=<portal>/auth/callback`, `AUTH_TOKEN_SECRET`, `AUTH_SIGNING_JWK` (a P-256 private JWK), and a `RESEND_API_KEY` or SMTP credentials — without a mail transport nobody can sign in |+| Service | Wiring |+| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| all | `CORE_ORG_ID`/`ORG_ID`, `CORE_SIGNING_SECRET`, `PORTAL_IDENTITY_SECRET` |+| all but core | `CORE_API_URL=http://qm-core-core.<namespace>.svc.cluster.local:8080` |+| core | `MODEL_PROVIDER`, `HARNESS`, `SANDBOX_BACKEND`, `PUBLIC_WEB_URL`/`WEB_UI_PUBLIC_URL` (the portal's hostname), `CAPABILITY_SECRET`, `CONNECTOR_SECRET_KEY`, `SKILL_SIGNING_SECRET`, and `DATABASE_URL` when `SESSION_STORE`/`RUN_STORE` are `postgres` |+| portal | `PORTAL_PUBLIC_URL`, `PORTAL_SESSION_SECRET`, `WEB_UI_UPSTREAM`, `ADMIN_UPSTREAM`, `AUTH_BROKER_UPSTREAM`, `AUTH_BROKER_PREFIX=/idp`, the `OIDC_*` set from `brokerWiring` in `cli/src/services.ts` (including `OIDC_CLIENT_SECRET`), and `OIDC_ALLOWED_EMAILS` or `OIDC_ALLOWED_EMAIL_DOMAIN` — production refuses to boot without an allow-list |+| admin | `ADMIN_BASE_PATH=/admin` |+| auth | `AUTH_ISSUER=<portal>/idp`, `AUTH_CLIENT_ID`, `AUTH_CLIENT_SECRET`, `AUTH_REDIRECT_URI=<portal>/auth/callback`, `AUTH_TOKEN_SECRET`, `AUTH_SIGNING_JWK` (a P-256 private JWK), `AUTH_EMAIL_FROM`, and `AUTH_EMAIL_TRANSPORT` with a `RESEND_API_KEY` or SMTP credentials — without a mail transport nobody can sign in |++`src/deployment/secret-schema.ts` is the authoritative list of what each service+requires; when a boot refusal names a variable this table doesn't, that file is the place+to look.The portal's hostname is only knowable after its first apply, and core, portal, and auth-all need it, so expect to apply twice: once to mint the hostname, once to wire it in.+all need it, so expect to apply twice: once to mint the hostname, once to wire it in. The+CLI does not print the assigned hostname — read it from the app's page in the dashboard,+or from the ingress itself: `porter kubectl --print-kubeconfig` writes a short-lived+kubeconfig, and `kubectl get ingress` lists every assigned host.Porter's own Postgres datastores are created from the dashboard. An RDS instance in thecluster's VPC works too, but Postgres 17 defaults to `rds.force_ssl=1`, so the URL needs@@ -72,8 +84,9 @@ one from before Porter support landed rejects `SANDBOX_BACKEND=porter` at startu## Giving published apps stable hostnames-Apps the agent deploys are reachable only if the cluster gives them an address. Without-one, `apply` fails with `the cluster assigned no hostname`, and core warns at startup.+Apps the agent deploys are reachable only if the cluster gives them an address. With+`PORTER_DEPLOY_APPS_DOMAIN` unset, core warns at startup and every publish is refused up+front, before anything is created.The address comes from Porter's **sandbox ingress**, which is separate from the ingressthat serves Porter apps and is enabled per cluster from the dashboard. A cluster withoutdiff --git a/porter/apps/core.yaml b/porter/apps/core.yamlindex 508240a9a..8e8102731 100644--- a/porter/apps/core.yaml+++ b/porter/apps/core.yaml@@ -7,6 +7,7 @@ build:services:- name: coretype: web+ private: truerun: node src/index.tsinstances: 1cpuCores: 1diff --git a/src/config.ts b/src/config.tsindex 98db43de6..28fd74f78 100644--- a/src/config.ts+++ b/src/config.ts@@ -772,7 +772,7 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): Config {}if (env.DEPLOY_PROVIDER === "porter" && !env.PORTER_DEPLOY_APPS_DOMAIN) {console.warn(- "[config] DEPLOY_PROVIDER=porter without PORTER_DEPLOY_APPS_DOMAIN — published apps only get a URL if the cluster assigns one, so a deploy can succeed with no reachable address; point a wildcard DNS record at the cluster ingress and set PORTER_DEPLOY_APPS_DOMAIN to give every app a stable hostname.",+ "[config] DEPLOY_PROVIDER=porter without PORTER_DEPLOY_APPS_DOMAIN — every publish will be refused because the app would have no address; enable sandbox ingress on the cluster, point a wildcard DNS record at it, and set PORTER_DEPLOY_APPS_DOMAIN.",);}for (const [selected, label] of [diff --git a/src/deploy/porter-deploy-provider.ts b/src/deploy/porter-deploy-provider.tsindex 990037e8a..80a117d82 100644--- a/src/deploy/porter-deploy-provider.ts+++ b/src/deploy/porter-deploy-provider.ts@@ -154,6 +154,11 @@ export function createPorterDeployProvider(opts: PorterDeployProviderOptions): Dapply: (d, version) =>serialized(d, async () => {+ if (!appsDomain) {+ throw new Error(+ `porter deploy ${d.id}: PORTER_DEPLOY_APPS_DOMAIN is not set, so the app would have no address — enable sandbox ingress on the cluster in the Porter dashboard, point a wildcard DNS record at it, and set PORTER_DEPLOY_APPS_DOMAIN`,+ );+ }resolveCache.delete(d.id);const { id: volumeId } = await ensurePorterVolume(client, volumeName(d));await retireBodies(d, true);@@ -181,12 +186,7 @@ export function createPorterDeployProvider(opts: PorterDeployProviderOptions): D});try {await waitPorterRunning(name, sb);- const host = (await sb.refresh()).host || domainOf(d);- if (!host) {- throw new Error(- `porter deploy ${d.id}: the cluster assigned no hostname — it has no sandbox ingress configured; set PORTER_DEPLOY_APPS_DOMAIN or configure the cluster ingress`,- );- }+ const host = (await sb.refresh()).host || domainOf(d)!;await materialize(sb.id, version);await startApp(sb.id, version);await store.put(d.id, { deploymentId: d.id, sandboxId: sb.id, name, host, createdAtMs: Date.now() });diff --git a/test/porter-deploy-provider.test.ts b/test/porter-deploy-provider.test.tsindex 3d5bf4146..d2542b337 100644--- a/test/porter-deploy-provider.test.ts+++ b/test/porter-deploy-provider.test.ts@@ -249,7 +249,7 @@ test("a body that vanished between list and terminate does not break destroy orassert.match(await fetchText("/"), /^hello from/);});-test("without an apps domain the cluster-assigned hostname becomes the URL", async () => {+test("without an apps domain apply refuses up front and creates nothing", async () => {const bare = createPorterDeployProvider({namePrefix: "qmt",appPort,@@ -257,13 +257,20 @@ test("without an apps domain the cluster-assigned hostname becomes the URL", asyclient: fake.client,store,});- const d = deployment("dep-8");- const endpoint = await bare.apply(d, version({ "server.js": SERVER }, "node server.js"));- const body = fake.bodies().find((b) => b.phase === "running")!;- assert.deepEqual(body.networking, [{ port: appPort }]);- assert.match(endpoint.host, /^qmt-app-dep-8-[a-z0-9]{5}\.fake\.test$/);+ await assert.rejects(+ bare.apply(deployment("dep-8"), version({ "server.js": SERVER }, "node server.js")),+ /PORTER_DEPLOY_APPS_DOMAIN is not set/,+ );+ assert.equal(fake.bodies().length, 0);+ assert.equal(fake.volumeNames().length, 0);+});++test("a cluster-assigned hostname wins over the derived domain", async () => {+ const d = deployment("dep-8b", "kept");+ const endpoint = await provider.apply(d, version({ "server.js": SERVER }, "node server.js"));+ assert.equal(endpoint.host, "kept.apps.test");assert.equal(endpoint.publicUrl, `https://${endpoint.host}/`);- assert.deepEqual(await bare.resolveEndpoint!({ ...d, endpoint }, version({}, "")), endpoint);+ assert.deepEqual(await provider.resolveEndpoint!({ ...d, endpoint }, version({}, "")), endpoint);assert.match(await fetchText("/"), /^hello from/);});