Skip to content

armgit / qm

Public Imported from GitHub
0

Multiplayer agent harness for work.

Refuse a porter publish up front when no apps domain is set

Josh France3f071cabcee5
diff --git a/docs/porter.md b/docs/porter.md
index b27bde389..2bf96a791 100644
--- a/docs/porter.md
+++ b/docs/porter.md
@@ -29,16 +29,19 @@ for f in porter/apps/*.yaml; do porter apply -f "$f"; done
Porter runs the services and assigns each **public** web service an `onporter.run`
hostname with a Let's Encrypt certificate, so this path needs no DNS record, no TLS
certificate, and no ingress controller. Only the portal is meant to be Internet-facing;
-auth, web-ui, and admin carry `private: true` so Porter creates no ingress for them and
-they are reachable only at their in-cluster address.
+core, auth, web-ui, and admin carry `private: true` so Porter creates no ingress for them
+and they are reachable only at their in-cluster address.
Two things `porter apply` will not do for you:
- **Building on an Apple Silicon machine fails.** `build.method: docker` builds locally
for `linux/amd64` and the legacy builder cannot cross-compile: the apply dies with
`image ... does not provide the specified platform (linux/amd64)`. Build and push with
- `docker buildx build --platform linux/amd64 --push`, then point the app at the pushed
- image with an `image: {repository, tag}` block, or use `porter apply --remote`.
+ `docker buildx build --platform linux/amd64 --push` to the registry Porter already
+ connected for the project (`porter registry list` prints it), then point the app at the
+ pushed image with an `image: {repository, tag}` block. `porter apply --remote` builds
+ server-side but is an opt-in project feature and fails with `remote build is not
+enabled for this project` until Porter enables it.
- **Env groups created by `porter env create` are not visible to `porter apply`.** Both
the `envGroups:` key and `--attach-env-groups` fail with `internal: unable to find
latest environment with provided name`, because `porter env create` makes a
@@ -49,15 +52,24 @@ latest environment with provided name`, because `porter env create` makes a
Nothing generates the inter-service wiring for you — `cli/src/services.ts` has `fly` and
`docker` targets but no Porter one — so set it by hand on each app:
-| Service | Wiring |
-| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| all but core | `CORE_API_URL=http://qm-core-core.<namespace>.svc.cluster.local:8080`, `CORE_ORG_ID`, `CORE_SIGNING_SECRET`, `PORTAL_IDENTITY_SECRET` |
-| core | `ORG_ID`, `MODEL_PROVIDER`, `HARNESS`, `SANDBOX_BACKEND`, `PUBLIC_WEB_URL`/`WEB_UI_PUBLIC_URL` (the portal's hostname), `CAPABILITY_SECRET`, `CONNECTOR_SECRET_KEY`, `SKILL_SIGNING_SECRET`, and `DATABASE_URL` when `SESSION_STORE`/`RUN_STORE` are `postgres` |
-| portal | `PORTAL_PUBLIC_URL`, `PORTAL_SESSION_SECRET`, `WEB_UI_UPSTREAM`, `ADMIN_UPSTREAM`, `AUTH_BROKER_UPSTREAM`, `AUTH_BROKER_PREFIX=/idp`, and the `OIDC_*` set from `brokerWiring` in `cli/src/services.ts` |
-| auth | `AUTH_ISSUER=<portal>/idp`, `AUTH_CLIENT_ID`, `AUTH_CLIENT_SECRET`, `AUTH_REDIRECT_URI=<portal>/auth/callback`, `AUTH_TOKEN_SECRET`, `AUTH_SIGNING_JWK` (a P-256 private JWK), and a `RESEND_API_KEY` or SMTP credentials — without a mail transport nobody can sign in |
+| Service | Wiring |
+| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| all | `CORE_ORG_ID`/`ORG_ID`, `CORE_SIGNING_SECRET`, `PORTAL_IDENTITY_SECRET` |
+| all but core | `CORE_API_URL=http://qm-core-core.<namespace>.svc.cluster.local:8080` |
+| core | `MODEL_PROVIDER`, `HARNESS`, `SANDBOX_BACKEND`, `PUBLIC_WEB_URL`/`WEB_UI_PUBLIC_URL` (the portal's hostname), `CAPABILITY_SECRET`, `CONNECTOR_SECRET_KEY`, `SKILL_SIGNING_SECRET`, and `DATABASE_URL` when `SESSION_STORE`/`RUN_STORE` are `postgres` |
+| portal | `PORTAL_PUBLIC_URL`, `PORTAL_SESSION_SECRET`, `WEB_UI_UPSTREAM`, `ADMIN_UPSTREAM`, `AUTH_BROKER_UPSTREAM`, `AUTH_BROKER_PREFIX=/idp`, the `OIDC_*` set from `brokerWiring` in `cli/src/services.ts` (including `OIDC_CLIENT_SECRET`), and `OIDC_ALLOWED_EMAILS` or `OIDC_ALLOWED_EMAIL_DOMAIN` — production refuses to boot without an allow-list |
+| admin | `ADMIN_BASE_PATH=/admin` |
+| auth | `AUTH_ISSUER=<portal>/idp`, `AUTH_CLIENT_ID`, `AUTH_CLIENT_SECRET`, `AUTH_REDIRECT_URI=<portal>/auth/callback`, `AUTH_TOKEN_SECRET`, `AUTH_SIGNING_JWK` (a P-256 private JWK), `AUTH_EMAIL_FROM`, and `AUTH_EMAIL_TRANSPORT` with a `RESEND_API_KEY` or SMTP credentials — without a mail transport nobody can sign in |
+
+`src/deployment/secret-schema.ts` is the authoritative list of what each service
+requires; when a boot refusal names a variable this table doesn't, that file is the place
+to look.
The portal's hostname is only knowable after its first apply, and core, portal, and auth
-all need it, so expect to apply twice: once to mint the hostname, once to wire it in.
+all need it, so expect to apply twice: once to mint the hostname, once to wire it in. The
+CLI does not print the assigned hostname — read it from the app's page in the dashboard,
+or from the ingress itself: `porter kubectl --print-kubeconfig` writes a short-lived
+kubeconfig, and `kubectl get ingress` lists every assigned host.
Porter's own Postgres datastores are created from the dashboard. An RDS instance in the
cluster's VPC works too, but Postgres 17 defaults to `rds.force_ssl=1`, so the URL needs
@@ -72,8 +84,9 @@ one from before Porter support landed rejects `SANDBOX_BACKEND=porter` at startu
## Giving published apps stable hostnames
-Apps the agent deploys are reachable only if the cluster gives them an address. Without
-one, `apply` fails with `the cluster assigned no hostname`, and core warns at startup.
+Apps the agent deploys are reachable only if the cluster gives them an address. With
+`PORTER_DEPLOY_APPS_DOMAIN` unset, core warns at startup and every publish is refused up
+front, before anything is created.
The address comes from Porter's **sandbox ingress**, which is separate from the ingress
that serves Porter apps and is enabled per cluster from the dashboard. A cluster without
diff --git a/porter/apps/core.yaml b/porter/apps/core.yaml
index 508240a9a..8e8102731 100644
--- a/porter/apps/core.yaml
+++ b/porter/apps/core.yaml
@@ -7,6 +7,7 @@ build:
services:
- name: core
type: web
+ private: true
run: node src/index.ts
instances: 1
cpuCores: 1
diff --git a/src/config.ts b/src/config.ts
index 98db43de6..28fd74f78 100644
--- a/src/config.ts
+++ b/src/config.ts
@@ -772,7 +772,7 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): Config {
}
if (env.DEPLOY_PROVIDER === "porter" && !env.PORTER_DEPLOY_APPS_DOMAIN) {
console.warn(
- "[config] DEPLOY_PROVIDER=porter without PORTER_DEPLOY_APPS_DOMAIN — published apps only get a URL if the cluster assigns one, so a deploy can succeed with no reachable address; point a wildcard DNS record at the cluster ingress and set PORTER_DEPLOY_APPS_DOMAIN to give every app a stable hostname.",
+ "[config] DEPLOY_PROVIDER=porter without PORTER_DEPLOY_APPS_DOMAIN — every publish will be refused because the app would have no address; enable sandbox ingress on the cluster, point a wildcard DNS record at it, and set PORTER_DEPLOY_APPS_DOMAIN.",
);
}
for (const [selected, label] of [
diff --git a/src/deploy/porter-deploy-provider.ts b/src/deploy/porter-deploy-provider.ts
index 990037e8a..80a117d82 100644
--- a/src/deploy/porter-deploy-provider.ts
+++ b/src/deploy/porter-deploy-provider.ts
@@ -154,6 +154,11 @@ export function createPorterDeployProvider(opts: PorterDeployProviderOptions): D
apply: (d, version) =>
serialized(d, async () => {
+ if (!appsDomain) {
+ throw new Error(
+ `porter deploy ${d.id}: PORTER_DEPLOY_APPS_DOMAIN is not set, so the app would have no address — enable sandbox ingress on the cluster in the Porter dashboard, point a wildcard DNS record at it, and set PORTER_DEPLOY_APPS_DOMAIN`,
+ );
+ }
resolveCache.delete(d.id);
const { id: volumeId } = await ensurePorterVolume(client, volumeName(d));
await retireBodies(d, true);
@@ -181,12 +186,7 @@ export function createPorterDeployProvider(opts: PorterDeployProviderOptions): D
});
try {
await waitPorterRunning(name, sb);
- const host = (await sb.refresh()).host || domainOf(d);
- if (!host) {
- throw new Error(
- `porter deploy ${d.id}: the cluster assigned no hostname — it has no sandbox ingress configured; set PORTER_DEPLOY_APPS_DOMAIN or configure the cluster ingress`,
- );
- }
+ const host = (await sb.refresh()).host || domainOf(d)!;
await materialize(sb.id, version);
await startApp(sb.id, version);
await store.put(d.id, { deploymentId: d.id, sandboxId: sb.id, name, host, createdAtMs: Date.now() });
diff --git a/test/porter-deploy-provider.test.ts b/test/porter-deploy-provider.test.ts
index 3d5bf4146..d2542b337 100644
--- a/test/porter-deploy-provider.test.ts
+++ b/test/porter-deploy-provider.test.ts
@@ -249,7 +249,7 @@ test("a body that vanished between list and terminate does not break destroy or
assert.match(await fetchText("/"), /^hello from/);
});
-test("without an apps domain the cluster-assigned hostname becomes the URL", async () => {
+test("without an apps domain apply refuses up front and creates nothing", async () => {
const bare = createPorterDeployProvider({
namePrefix: "qmt",
appPort,
@@ -257,13 +257,20 @@ test("without an apps domain the cluster-assigned hostname becomes the URL", asy
client: fake.client,
store,
});
- const d = deployment("dep-8");
- const endpoint = await bare.apply(d, version({ "server.js": SERVER }, "node server.js"));
- const body = fake.bodies().find((b) => b.phase === "running")!;
- assert.deepEqual(body.networking, [{ port: appPort }]);
- assert.match(endpoint.host, /^qmt-app-dep-8-[a-z0-9]{5}\.fake\.test$/);
+ await assert.rejects(
+ bare.apply(deployment("dep-8"), version({ "server.js": SERVER }, "node server.js")),
+ /PORTER_DEPLOY_APPS_DOMAIN is not set/,
+ );
+ assert.equal(fake.bodies().length, 0);
+ assert.equal(fake.volumeNames().length, 0);
+});
+
+test("a cluster-assigned hostname wins over the derived domain", async () => {
+ const d = deployment("dep-8b", "kept");
+ const endpoint = await provider.apply(d, version({ "server.js": SERVER }, "node server.js"));
+ assert.equal(endpoint.host, "kept.apps.test");
assert.equal(endpoint.publicUrl, `https://${endpoint.host}/`);
- assert.deepEqual(await bare.resolveEndpoint!({ ...d, endpoint }, version({}, "")), endpoint);
+ assert.deepEqual(await provider.resolveEndpoint!({ ...d, endpoint }, version({}, "")), endpoint);
assert.match(await fetchText("/"), /^hello from/);
});