Skip to content

Open-source manufacturing ERP, MES and QMS. Quoting, MRP, inventory, shop floor, quality and lot/serial traceability on one Postgres schema, with a REST API and MCP server. Self-host or use Carbon Cloud.

ci: deploy the assembler from build-images, drop assembler.yml

Sidwebworksaf2e945073a7
diff --git a/.claude/rules/sst-deployment-infrastructure.md b/.claude/rules/sst-deployment-infrastructure.md
index 4e8ad1e55d..162ce820be 100644
--- a/.claude/rules/sst-deployment-infrastructure.md
+++ b/.claude/rules/sst-deployment-infrastructure.md
@@ -52,15 +52,26 @@ alternative deployment, not a replacement. Both build from the same root `Docker
Every push to `main` runs `build-images.yml`, which builds or re-tags every image
for the commit (`carbon/<image>:<sha>` in `vars.IMAGE_REGISTRY`). The deploy is a
job of that same run, chained with `needs:` — there is no polling:
-- `apps` builds `carbon/erp` and `carbon/mes`; `services` builds `bootstrap`,
- `edge-functions` and `assembler` with the same steps (a YAML anchor). Split so
- the deploy needs only the app images: a failed assembler build does not hold
- back a release.
-- `changes` diffs `github.event.before..sha` over `apps/erp`, `apps/mes`,
- `packages`, `docs/content`; a new branch or force-push (no usable `before`)
- counts as changed. A manual run of `build-images.yml` never deploys.
-- `deploy` (`needs: [apps, changes]`) calls `deploy.yml` (`workflow_call`,
- `secrets: inherit`). A failed or cancelled app image skips it at once.
+- `apps` builds `carbon/erp` and `carbon/mes`; `services` builds `bootstrap` and
+ `edge-functions`; `assembler` builds `carbon/assembler` — all with the same steps
+ (a YAML anchor). Split so each deploy needs only its own images: a failed
+ assembler build does not hold back an app release.
+- `changes` diffs `github.event.before..sha` twice: over the app inputs
+ (`apps/erp`, `apps/mes`, `packages`, `docs/content`, …) for `deploy`, and over
+ `apps/assembler`, `crates` and the Cargo files for `assembler`. A new branch or
+ force-push (no usable `before`) counts as changed. A manual run of
+ `build-images.yml` never deploys.
+- `deploy` (`needs: [apps, changes]`) calls `deploy.yml` with `apps: true`;
+ `deploy-assembler` (`needs: [assembler, changes]`) calls it with
+ `assembler: true` (`workflow_call`, `secrets: inherit`). A failed or cancelled
+ image skips its deploy at once.
+- `deploy.yml`'s `assembler` job runs one leg per AWS account (prod with
+ `ASSEMBLER_AWS_*`, govcloud with `AWS_*`). A Lambda pulls only from a private
+ ECR repository in its own account, so each leg copies
+ `${IMAGE_REGISTRY}/carbon/assembler:<sha>` into its account's `carbon/assembler`
+ (pull and push, keeping one linux/amd64 manifest) and then runs
+ `pnpm --filter ci ci:assembler` (`ci/src/assembler.ts`, which deploys
+ `apps/assembler/sst.config.ts` for that account's workspaces).
- Image concurrency is per image and per job (`build-images-<image>`), not per
workflow, so a deploy still running does not hold up the next commit's builds.
- The services pull `${IMAGE_REGISTRY}/carbon/<app>:${IMAGE_TAG}`. With
diff --git a/.github/workflows/assembler.yml b/.github/workflows/assembler.yml
deleted file mode 100644
index d6e7cd89b0..0000000000
--- a/.github/workflows/assembler.yml
+++ /dev/null
@@ -1,203 +0,0 @@
-name: Deploy Assembler
-
-# Builds carbon/assembler and mirrors it to BOTH the prod (ASSEMBLER_AWS_*) and
-# govcloud (shared AWS_*) ECRs — a Lambda pulls same-account, so each account needs
-# its own copy. Links the prebuilt carbon/occt base, rebuilt on demand only
-# (workflow_dispatch, ~30 min compile). provenance/sbom stay off: Lambda rejects the
-# OCI image index attestations produce.
-# Prereq: carbon/assembler + carbon/occt ECR repos in prod; carbon/assembler in govcloud.
-
-on:
- push:
- branches:
- - main
- paths:
- - "apps/assembler/**"
- - "crates/**"
- workflow_dispatch:
- inputs:
- build_occt_base:
- description: "Also rebuild the carbon/occt base image (slow, ~30 min)"
- type: boolean
- default: false
- deploy:
- description: "Deploy the Lambda (+ optional ECS service) after building"
- type: boolean
- default: false
-
-jobs:
- build:
- name: Build assembler image
- # Push builds only fire when repo var ASSEMBLER_CI_ENABLED is 'true'; dispatch
- # always runs.
- if: ${{ github.event_name == 'workflow_dispatch' || vars.ASSEMBLER_CI_ENABLED == 'true' }}
- runs-on: ubuntu-latest
- steps:
- - name: Checkout code
- uses: actions/checkout@v7
-
- # Log into both ECRs up front — different accounts and hosts, so both
- # credentials coexist in ~/.docker/config.json for the pushes below.
- - name: Login to prod ECR
- id: prod-ecr
- uses: aws-actions/amazon-ecr-login@v2
- env:
- AWS_ACCESS_KEY_ID: ${{ secrets.ASSEMBLER_AWS_ACCESS_KEY_ID }}
- AWS_SECRET_ACCESS_KEY: ${{ secrets.ASSEMBLER_AWS_SECRET_ACCESS_KEY }}
- AWS_REGION: ${{ secrets.ASSEMBLER_AWS_REGION }}
-
- - name: Login to govcloud ECR
- id: gov-ecr
- uses: aws-actions/amazon-ecr-login@v2
- env:
- AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
- AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- AWS_REGION: ${{ secrets.AWS_REGION }}
-
- - name: Set up Docker Buildx
- uses: docker/setup-buildx-action@v3
-
- # OCCT base on demand. 8.0.0-p1 is immutable, so check each registry and push
- # only where it's missing (re-pushing an existing tag errors).
- - name: Build and mirror OCCT base
- if: ${{ github.event_name == 'workflow_dispatch' && inputs.build_occt_base }}
- run: |
- PROD_TAG="${{ steps.prod-ecr.outputs.registry }}/carbon/occt:8.0.0-p1"
- GOV_TAG="${{ steps.gov-ecr.outputs.registry }}/carbon/occt:8.0.0-p1"
-
- prod_has() { AWS_ACCESS_KEY_ID="${{ secrets.ASSEMBLER_AWS_ACCESS_KEY_ID }}" \
- AWS_SECRET_ACCESS_KEY="${{ secrets.ASSEMBLER_AWS_SECRET_ACCESS_KEY }}" \
- AWS_DEFAULT_REGION="${{ secrets.ASSEMBLER_AWS_REGION }}" \
- aws ecr describe-images --repository-name carbon/occt --image-ids imageTag=8.0.0-p1 >/dev/null 2>&1; }
- gov_has() { AWS_ACCESS_KEY_ID="${{ secrets.AWS_ACCESS_KEY_ID }}" \
- AWS_SECRET_ACCESS_KEY="${{ secrets.AWS_SECRET_ACCESS_KEY }}" \
- AWS_DEFAULT_REGION="${{ secrets.AWS_REGION }}" \
- aws ecr describe-images --repository-name carbon/occt --image-ids imageTag=8.0.0-p1 >/dev/null 2>&1; }
-
- prod_has && prod=1 || prod=0
- gov_has && gov=1 || gov=0
-
- if [ "$prod" = 1 ] && [ "$gov" = 1 ]; then
- echo "carbon/occt:8.0.0-p1 already in both registries — skipping build"
- exit 0
- fi
-
- docker buildx build apps/assembler \
- --file apps/assembler/occt.Dockerfile \
- --platform linux/amd64 --provenance=false --sbom=false \
- --cache-from type=gha --cache-to type=gha,mode=max \
- --tag "$PROD_TAG" --tag "$GOV_TAG" --load
-
- if [ "$prod" = 0 ]; then docker push "$PROD_TAG"; else echo "occt already in prod — not re-pushing"; fi
- if [ "$gov" = 0 ]; then docker push "$GOV_TAG"; else echo "occt already in govcloud — not re-pushing"; fi
-
- # Fail fast: the assembler build's FROM links the prod OCCT base.
- - name: Require prod OCCT base image
- env:
- AWS_ACCESS_KEY_ID: ${{ secrets.ASSEMBLER_AWS_ACCESS_KEY_ID }}
- AWS_SECRET_ACCESS_KEY: ${{ secrets.ASSEMBLER_AWS_SECRET_ACCESS_KEY }}
- AWS_DEFAULT_REGION: ${{ secrets.ASSEMBLER_AWS_REGION }}
- run: |
- aws ecr describe-images --repository-name carbon/occt \
- --image-ids imageTag=8.0.0-p1 >/dev/null 2>&1 || {
- echo "carbon/occt:8.0.0-p1 is not in prod ECR."
- echo "Dispatch this workflow once with build_occt_base=true to build it."
- exit 1
- }
-
- # Repo-root context (cargo workspace). Build once (load), scan, then push to
- # both registries.
- - name: Build assembler image
- uses: docker/build-push-action@v5
- with:
- context: .
- file: apps/assembler/Dockerfile
- build-args: |
- OCCT_IMAGE=${{ steps.prod-ecr.outputs.registry }}/carbon/occt:8.0.0-p1
- load: true
- tags: |
- ${{ steps.prod-ecr.outputs.registry }}/carbon/assembler:latest
- ${{ steps.prod-ecr.outputs.registry }}/carbon/assembler:${{ github.sha }}
- ${{ steps.gov-ecr.outputs.registry }}/carbon/assembler:latest
- ${{ steps.gov-ecr.outputs.registry }}/carbon/assembler:${{ github.sha }}
- cache-from: type=gha
- cache-to: type=gha,mode=max
- platforms: linux/amd64
- provenance: false
- sbom: false
-
- # Report-only (exit-code 0). Flip to '1' to fail on CRITICAL/HIGH once triaged.
- - name: Scan image (Trivy)
- uses: aquasecurity/trivy-action@master
- with:
- image-ref: ${{ steps.prod-ecr.outputs.registry }}/carbon/assembler:${{ github.sha }}
- severity: CRITICAL,HIGH
- ignore-unfixed: true
- exit-code: "0"
-
- - name: Push assembler image (prod + govcloud)
- run: |
- docker push ${{ steps.prod-ecr.outputs.registry }}/carbon/assembler:latest
- docker push ${{ steps.prod-ecr.outputs.registry }}/carbon/assembler:${{ github.sha }}
- docker push ${{ steps.gov-ecr.outputs.registry }}/carbon/assembler:latest
- docker push ${{ steps.gov-ecr.outputs.registry }}/carbon/assembler:${{ github.sha }}
-
- # A matrix leg per account (prod + govcloud) so one build deploys both in
- # parallel. Each leg deploys only its own account's rows (see
- # DEPLOY_AWS_ACCOUNT_ID below). Runs on a main push or dispatch with deploy=true.
- deploy:
- name: Deploy assembler (${{ matrix.env.stage }})
- if: ${{ needs.build.result == 'success' && (github.event_name == 'push' || inputs.deploy) }}
- needs: build
- runs-on: ubuntu-latest
- strategy:
- fail-fast: false
- matrix:
- env:
- - stage: prod
- access_key_id: ASSEMBLER_AWS_ACCESS_KEY_ID
- secret_access_key: ASSEMBLER_AWS_SECRET_ACCESS_KEY
- region: ASSEMBLER_AWS_REGION
- - stage: govcloud
- access_key_id: AWS_ACCESS_KEY_ID
- secret_access_key: AWS_SECRET_ACCESS_KEY
- region: AWS_REGION
- env:
- SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }}
- SUPABASE_SERVICE_ROLE_KEY: ${{ secrets.SUPABASE_SERVICE_ROLE_KEY }}
- SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
- steps:
- - uses: actions/checkout@v7
-
- - name: Configure AWS credentials
- uses: aws-actions/configure-aws-credentials@v6
- with:
- aws-access-key-id: ${{ secrets[matrix.env.access_key_id] }}
- aws-secret-access-key: ${{ secrets[matrix.env.secret_access_key] }}
- aws-region: ${{ secrets[matrix.env.region] }}
-
- # ci/src/assembler.ts uses this to skip rows in other accounts, so the two
- # legs never touch each other's workspaces.
- - name: Resolve deploy account
- run: echo "DEPLOY_AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)" >> "$GITHUB_ENV"
-
- - uses: pnpm/action-setup@v6
- - uses: actions/setup-node@v4
- with:
- node-version: "22"
- cache: pnpm
-
- # SST v3 runs on Pulumi; pin the version deploy.yml uses.
- - name: Install compatible Pulumi version
- run: |
- curl -fsSL https://get.pulumi.com | sh -s -- --version 3.212.0
- echo "$HOME/.pulumi/bin" >> $GITHUB_PATH
-
- - name: Install dependencies
- run: pnpm install --frozen-lockfile
-
- - name: Deploy
- env:
- IMAGE_TAG: ${{ github.sha }}
- STAGE: ${{ matrix.env.stage }}
- run: pnpm --filter ci ci:assembler
diff --git a/.github/workflows/build-images.yml b/.github/workflows/build-images.yml
index 7c7c1006d8..a9588547a3 100644
--- a/.github/workflows/build-images.yml
+++ b/.github/workflows/build-images.yml
@@ -28,8 +28,9 @@ env:
jobs:
# erp and mes are their own job so the deploy can need exactly them: a
- # failed assembler image does not hold back an app release. Both jobs run
- # the same steps (the anchor below).
+ # failed assembler image does not hold back an app release, and the
+ # assembler is its own job so its deploy needs only its image. Every image
+ # job runs the same steps (the anchor below).
apps:
name: carbon/${{ matrix.image }}
if: vars.IMAGE_REGISTRY != ''
@@ -142,17 +143,30 @@ jobs:
strategy:
fail-fast: false
matrix:
- image: [bootstrap, edge-functions, assembler]
+ image: [bootstrap, edge-functions]
steps: *image-steps
- # Whether this push changed anything the apps are built from. Runs beside
- # the builds; a manual run of this workflow never deploys.
+ assembler:
+ name: carbon/assembler
+ if: vars.IMAGE_REGISTRY != ''
+ runs-on: ubuntu-24.04
+ concurrency:
+ group: build-images-${{ matrix.image }}
+ cancel-in-progress: false
+ strategy:
+ matrix:
+ image: [assembler]
+ steps: *image-steps
+
+ # Whether this push changed anything the apps, or the assembler, are built
+ # from. Runs beside the builds; a manual run of this workflow never deploys.
changes:
name: Deployable changes
if: github.event_name == 'push' && vars.IMAGE_REGISTRY != ''
runs-on: ubuntu-24.04
outputs:
deploy: ${{ steps.diff.outputs.deploy }}
+ assembler: ${{ steps.diff.outputs.assembler }}
steps:
- uses: actions/checkout@v7
- id: diff
@@ -163,15 +177,19 @@ jobs:
# A new branch or a force-push leaves no usable BEFORE: deploy.
if [[ $BEFORE =~ ^0+$ ]] || ! git fetch --quiet --depth=1 origin "$BEFORE"; then
echo "deploy=true" >> "$GITHUB_OUTPUT"
+ echo "assembler=true" >> "$GITHUB_OUTPUT"
+ exit 0
+ fi
+ changed() { ! git diff --quiet "$BEFORE" "$SHA" -- "$@"; }
# The erp/mes inputs of .github/scripts/build-images.sh, plus the
# docs the ERP bundles: a lockfile-only push rebuilds the images too.
- elif git diff --quiet "$BEFORE" "$SHA" -- Dockerfile .dockerignore \
- package.json pnpm-lock.yaml pnpm-workspace.yaml .npmrc turbo.json \
- lingui.config.js apps/erp apps/mes packages patches scripts docs/content; then
- echo "deploy=false" >> "$GITHUB_OUTPUT"
- else
- echo "deploy=true" >> "$GITHUB_OUTPUT"
- fi
+ changed Dockerfile .dockerignore package.json pnpm-lock.yaml \
+ pnpm-workspace.yaml .npmrc turbo.json lingui.config.js apps/erp \
+ apps/mes packages patches scripts docs/content &&
+ echo "deploy=true" >> "$GITHUB_OUTPUT" || echo "deploy=false" >> "$GITHUB_OUTPUT"
+ # The assembler inputs of the same script.
+ changed .dockerignore apps/assembler crates Cargo.toml Cargo.lock &&
+ echo "assembler=true" >> "$GITHUB_OUTPUT" || echo "assembler=false" >> "$GITHUB_OUTPUT"
# Starts only once carbon/erp and carbon/mes exist for this commit, and is
# skipped outright when either failed or was cancelled — a dependency, not
@@ -181,4 +199,16 @@ jobs:
needs: [apps, changes]
if: needs.changes.outputs.deploy == 'true'
uses: ./.github/workflows/deploy.yml
+ with:
+ apps: true
+ secrets: inherit
+
+ # The same for carbon/assembler, deployed only when its files changed.
+ deploy-assembler:
+ name: Deploy assembler
+ needs: [assembler, changes]
+ if: needs.changes.outputs.assembler == 'true'
+ uses: ./.github/workflows/deploy.yml
+ with:
+ assembler: true
secrets: inherit
diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
index aca41cb50c..9ceadccf16 100644
--- a/.github/workflows/deploy.yml
+++ b/.github/workflows/deploy.yml
@@ -1,15 +1,37 @@
name: Deploy Apps
-# Called by build-images.yml once this commit's carbon/erp and carbon/mes
-# images are pushed. Run by hand, it deploys the images of the commit it is
+# Called by build-images.yml once this commit's images are pushed: with
+# `apps` after carbon/erp and carbon/mes, with `assembler` after
+# carbon/assembler. Run by hand, it deploys the images of the commit it is
# run on, which must already exist.
on:
workflow_call:
+ inputs:
+ apps:
+ type: boolean
+ default: false
+ assembler:
+ type: boolean
+ default: false
workflow_dispatch:
+ inputs:
+ apps:
+ description: "Deploy ERP and MES"
+ type: boolean
+ default: true
+ assembler:
+ description: "Deploy the assembler"
+ type: boolean
+ default: false
+
+permissions:
+ contents: read
+ id-token: write # the image role's public.ecr.aws login
jobs:
deploy:
name: Deploy
+ if: inputs.apps
runs-on: ubuntu-latest
# One deploy at a time. A newer commit waiting replaces an older one that
# had not started yet: it deploys the newer images anyway.
@@ -57,3 +79,99 @@ jobs:
needs: deploy
uses: ./.github/workflows/inngest.yml
secrets: inherit
+
+ # One leg per AWS account (prod and govcloud); each deploys only its own
+ # account's workspaces (DEPLOY_AWS_ACCOUNT_ID, read by ci/src/assembler.ts).
+ assembler:
+ name: Deploy assembler (${{ matrix.env.stage }})
+ if: inputs.assembler
+ runs-on: ubuntu-latest
+ concurrency:
+ group: deploy-assembler-${{ matrix.env.stage }}
+ cancel-in-progress: false
+ strategy:
+ fail-fast: false
+ matrix:
+ env:
+ - stage: prod
+ access_key_id: ASSEMBLER_AWS_ACCESS_KEY_ID
+ secret_access_key: ASSEMBLER_AWS_SECRET_ACCESS_KEY
+ region: ASSEMBLER_AWS_REGION
+ - stage: govcloud
+ access_key_id: AWS_ACCESS_KEY_ID
+ secret_access_key: AWS_SECRET_ACCESS_KEY
+ region: AWS_REGION
+ env:
+ IMAGE_TAG: ${{ github.sha }}
+ SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }}
+ SUPABASE_SERVICE_ROLE_KEY: ${{ secrets.SUPABASE_SERVICE_ROLE_KEY }}
+ SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
+ steps:
+ - uses: actions/checkout@v7
+
+ # build-images.yml pushes to vars.IMAGE_REGISTRY. An authenticated pull
+ # from public.ecr.aws counts against the account, not the runner's
+ # shared quota; without the login the pull stays anonymous.
+ - if: vars.IMAGE_AWS_REGION != ''
+ uses: aws-actions/configure-aws-credentials@v6
+ continue-on-error: true
+ with:
+ aws-region: ${{ vars.IMAGE_AWS_REGION }}
+ role-to-assume: ${{ vars.IMAGE_AWS_ROLE_ARN }}
+ aws-access-key-id: ${{ secrets.IMAGE_AWS_ACCESS_KEY_ID }}
+ aws-secret-access-key: ${{ secrets.IMAGE_AWS_SECRET_ACCESS_KEY }}
+ - if: vars.IMAGE_AWS_REGION != ''
+ uses: docker/login-action@v4
+ continue-on-error: true
+ with:
+ registry: public.ecr.aws
+ env:
+ AWS_REGION: us-east-1
+
+ - name: Configure AWS credentials
+ uses: aws-actions/configure-aws-credentials@v6
+ with:
+ # Drops the image role's session token, which would otherwise ride
+ # along with these keys.
+ unset-current-credentials: true
+ aws-access-key-id: ${{ secrets[matrix.env.access_key_id] }}
+ aws-secret-access-key: ${{ secrets[matrix.env.secret_access_key] }}
+ aws-region: ${{ secrets[matrix.env.region] }}
+
+ - name: Resolve deploy account
+ run: echo "DEPLOY_AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)" >> "$GITHUB_ENV"
+
+ - id: ecr
+ uses: aws-actions/amazon-ecr-login@v2
+
+ # A Lambda pulls its image from a private ECR repository in its own
+ # account, so copy the commit's image there. Pull and push rather than
+ # re-tag across registries: the copy stays one linux/amd64 manifest,
+ # the only shape Lambda accepts.
+ - name: Copy the image into this account
+ env:
+ FROM: ${{ vars.IMAGE_REGISTRY }}/carbon/assembler:${{ github.sha }}
+ TO: ${{ steps.ecr.outputs.registry }}/carbon/assembler:${{ github.sha }}
+ run: |
+ docker pull --platform linux/amd64 "$FROM"
+ docker tag "$FROM" "$TO"
+ docker push "$TO"
+
+ - uses: pnpm/action-setup@v6
+ - uses: actions/setup-node@v4
+ with:
+ node-version: "22"
+ cache: pnpm
+
+ - name: Install compatible Pulumi version
+ run: |
+ curl -fsSL https://get.pulumi.com | sh -s -- --version 3.212.0
+ echo "$HOME/.pulumi/bin" >> $GITHUB_PATH
+
+ - name: Install dependencies
+ run: pnpm install --frozen-lockfile
+
+ - name: Deploy
+ env:
+ STAGE: ${{ matrix.env.stage }}
+ run: pnpm --filter ci ci:assembler
diff --git a/apps/assembler/sst.config.ts b/apps/assembler/sst.config.ts
index 9bac7767c7..b021994a65 100644
--- a/apps/assembler/sst.config.ts
+++ b/apps/assembler/sst.config.ts
@@ -38,7 +38,8 @@ export default $config({
const account = process.env.AWS_ACCOUNT_ID;
const region = process.env.AWS_REGION;
const imageTag = process.env.IMAGE_TAG ?? "latest";
- // Same image for both runtimes (built once by .github/workflows/assembler.yml).
+ // Same image for both runtimes: build-images.yml builds it and deploy.yml
+ // copies it into this account's ECR.
const image = `${account}.dkr.ecr.${region}.amazonaws.com/carbon/assembler:${imageTag}`;
// Shared runtime env (same knobs on Lambda and the ECS service).