Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
ci: pin plugin preview donor to PR head (#1631)
Ian Oberste025978044aa
diff --git a/.github/workflows/docker-publish-on-pr.yml b/.github/workflows/docker-publish-on-pr.ymlindex de010703..7d43487b 100644--- a/.github/workflows/docker-publish-on-pr.yml+++ b/.github/workflows/docker-publish-on-pr.yml@@ -30,15 +30,56 @@ jobs:id: guardenv:GH_TOKEN: ${{ github.token }}+ EVENT_HEAD: ${{ github.event.pull_request.head.sha }}+ PR_NUMBER: ${{ github.event.pull_request.number }}+ REPOSITORY: ${{ github.repository }}run: |- files=$(gh pr view "${{ github.event.pull_request.number }}" \- --repo "${{ github.repository }}" --json files --jq '.files[].path')- echo "Changed files:"; echo "$files"- if echo "$files" | grep -qE '^(Dockerfile([.].*)?|\.dockerignore|\.github/)'; then+ skip_preview() {echo "skip=true" >> "$GITHUB_OUTPUT"- echo "::warning::PR modifies the build/CI surface; auto-preview skipped pending manual maintainer build."+ echo "plugin_changed=false" >> "$GITHUB_OUTPUT"+ echo "skip_reason=$1" >> "$GITHUB_OUTPUT"+ echo "::warning::$1"+ exit 0+ }++ pr=$(gh pr view "$PR_NUMBER" --repo "$REPOSITORY" \+ --json changedFiles,headRefOid)+ head_before=$(printf '%s' "$pr" | jq -r '.headRefOid')+ expected_files=$(printf '%s' "$pr" | jq -r '.changedFiles')+ if [ "$head_before" != "$EVENT_HEAD" ]; then+ skip_preview "PR head changed while this run was queued; the newer synchronize event will publish the preview."+ fi++ files=$(gh api --paginate --slurp \+ "repos/$REPOSITORY/pulls/$PR_NUMBER/files?per_page=100")+ head_after=$(gh pr view "$PR_NUMBER" --repo "$REPOSITORY" \+ --json headRefOid --jq '.headRefOid')+ if [ "$head_after" != "$EVENT_HEAD" ]; then+ skip_preview "PR head changed while files were being inspected; the newer synchronize event will publish the preview."+ fi++ fetched_files=$(printf '%s' "$files" | jq '[.[][]] | length')+ if [ "$fetched_files" -ne "$expected_files" ]; then+ skip_preview "GitHub returned an incomplete changed-file list; refusing to publish a preview."+ fi++ echo "Changed files:"+ printf '%s' "$files" | jq -r '.[][] | .filename'++ if printf '%s' "$files" | jq -e \+ 'any(.[][]; .filename | test("^(Dockerfile([.].*)?|[.]dockerignore|[.]github/)"))' \+ >/dev/null; then+ skip_preview "PR modifies the build/CI surface; auto-preview skipped pending manual maintainer build."+ fi++ echo "skip=false" >> "$GITHUB_OUTPUT"+ echo "skip_reason=" >> "$GITHUB_OUTPUT"+ if printf '%s' "$files" | jq -e \+ 'any(.[][]; .filename | test("^contrib/mysql-iam-plugin/"))' \+ >/dev/null; then+ echo "plugin_changed=true" >> "$GITHUB_OUTPUT"else- echo "skip=false" >> "$GITHUB_OUTPUT"+ echo "plugin_changed=false" >> "$GITHUB_OUTPUT"fi- name: Comment that auto-preview was skipped@@ -48,17 +89,15 @@ jobs:token: ${{ secrets.GITHUB_TOKEN }}issue-number: ${{ github.event.pull_request.number }}body: |- Auto-preview build skipped: this PR modifies the build/CI surface- (`Dockerfile`, `.dockerignore`, or files under `.github/`). A maintainer- will review and build the preview image manually before it is published.+ Auto-preview build skipped: ${{ steps.guard.outputs.skip_reason }}# checkout@v6 now refuses fork-PR checkouts from pull_request_target# unless explicitly allowed, which has failed this job for every fork PR# since 2026-07-20. Checking out the PR head is intentional here: the# guard above keeps PRs that touch the build/CI surface from reaching- # this step, and the build passes no secrets (only MINISTACK_VERSION as- # a build-arg). persist-credentials is disabled so the untrusted tree- # never holds the repo token — no later step needs git credentials.+ # this step, and the builds receive only non-secret version and image+ # reference arguments. persist-credentials is disabled so the untrusted+ # tree never holds the repo token — no later step needs git credentials.- uses: actions/checkout@v6if: steps.guard.outputs.skip != 'true'with:@@ -93,6 +132,56 @@ jobs:username: ${{ secrets.DOCKERHUB_USERNAME }}password: ${{ secrets.DOCKERHUB_TOKEN }}+ # ── Full (Debian/glibc) image — DuckDB-backed Athena, psycopg2, pymysql.+ # Built automatically for plugin-source changes so the regular preview can+ # consume artifacts from this exact PR head. The `full-preview` label remains+ # the opt-in path for other DuckDB/Postgres/MySQL end-to-end changes.+ - name: Extract metadata (full)+ id: meta_full+ if: >-+ steps.guard.outputs.skip != 'true' &&+ (steps.guard.outputs.plugin_changed == 'true' ||+ contains(github.event.pull_request.labels.*.name, 'full-preview'))+ uses: docker/metadata-action@v6+ with:+ images: |+ ${{ env.DOCKERHUB_IMAGE_ORG }}+ flavor: |+ latest=false+ tags: |+ type=ref,event=pr,suffix=-${{ steps.short_sha.outputs.short_sha }}-full++ - name: Build and push (full)+ id: build_full+ if: >-+ steps.guard.outputs.skip != 'true' &&+ (steps.guard.outputs.plugin_changed == 'true' ||+ contains(github.event.pull_request.labels.*.name, 'full-preview'))+ uses: docker/build-push-action@v7+ with:+ context: .+ file: Dockerfile.full+ platforms: linux/amd64,linux/arm64+ push: true+ tags: ${{ steps.meta_full.outputs.tags }}+ labels: ${{ steps.meta_full.outputs.labels }}+ build-args: |+ MINISTACK_VERSION=pr-${{ github.event.pull_request.number }}-${{ steps.short_sha.outputs.short_sha }}+ # Read-only cache on pull_request_target (see note on the regular build above).+ cache-from: type=gha,scope=full-pr++ - name: Resolve plugin donor+ id: plugin_donor+ if: steps.guard.outputs.skip != 'true'+ env:+ FULL_DIGEST: ${{ steps.build_full.outputs.digest }}+ run: |+ if [ -n "$FULL_DIGEST" ]; then+ echo "image=${DOCKERHUB_IMAGE_ORG}@${FULL_DIGEST}" >> "$GITHUB_OUTPUT"+ else+ echo "image=ghcr.io/ministackorg/ministack:full" >> "$GITHUB_OUTPUT"+ fi+# ── Regular (Alpine) image ──- name: Extract metadata (regular)id: meta@@ -105,6 +194,7 @@ jobs:type=ref,event=pr,suffix=-${{ steps.short_sha.outputs.short_sha }}- name: Build and push (regular)+ id: build_regularif: steps.guard.outputs.skip != 'true'uses: docker/build-push-action@v7with:@@ -116,42 +206,54 @@ jobs:labels: ${{ steps.meta.outputs.labels }}build-args: |MINISTACK_VERSION=pr-${{ github.event.pull_request.number }}-${{ steps.short_sha.outputs.short_sha }}+ PLUGIN_DONOR_IMAGE=${{ steps.plugin_donor.outputs.image }}# pull_request_target receives a read-only Actions cache as of GitHub's# 2026-06-26 change (read-only cache for untrusted triggers), so a cache-to# write fails the build with "failed to reserve cache". Restore only here;# the cache is populated by a trusted (push) workflow.cache-from: type=gha,scope=regular-pr- # ── Full (Debian/glibc) image — DuckDB-backed Athena, psycopg2, pymysql.- # Only built when the PR carries the `full-preview` label (opt-in to keep- # default PR cost down). Add the label to any PR that touches the full- # variant or needs DuckDB/Postgres/MySQL paths verified end-to-end.- - name: Extract metadata (full)- id: meta_full- if: steps.guard.outputs.skip != 'true' && contains(github.event.pull_request.labels.*.name, 'full-preview')- uses: docker/metadata-action@v6- with:- images: |- ${{ env.DOCKERHUB_IMAGE_ORG }}- flavor: |- latest=false- tags: |- type=ref,event=pr,suffix=-${{ steps.short_sha.outputs.short_sha }}-full+ - name: Verify full and regular plugin artifacts match+ if: >-+ steps.guard.outputs.skip != 'true' &&+ steps.build_full.outputs.digest != '' &&+ steps.build_regular.outputs.digest != ''+ env:+ FULL_IMAGE: ${{ env.DOCKERHUB_IMAGE_ORG }}@${{ steps.build_full.outputs.digest }}+ REGULAR_IMAGE: ${{ env.DOCKERHUB_IMAGE_ORG }}@${{ steps.build_regular.outputs.digest }}+ run: |+ set -eu+ scratch=$(mktemp -d)+ full_container=+ regular_container=+ cleanup() {+ [ -z "$full_container" ] || docker rm "$full_container" >/dev/null 2>&1 || true+ [ -z "$regular_container" ] || docker rm "$regular_container" >/dev/null 2>&1 || true+ rm -rf "$scratch"+ }+ trap cleanup EXIT- - name: Build and push (full)- if: steps.guard.outputs.skip != 'true' && contains(github.event.pull_request.labels.*.name, 'full-preview')- uses: docker/build-push-action@v7- with:- context: .- file: Dockerfile.full- platforms: linux/amd64,linux/arm64- push: true- tags: ${{ steps.meta_full.outputs.tags }}- labels: ${{ steps.meta_full.outputs.labels }}- build-args: |- MINISTACK_VERSION=pr-${{ github.event.pull_request.number }}-${{ steps.short_sha.outputs.short_sha }}- # Read-only cache on pull_request_target (see note on the regular build above).- cache-from: type=gha,scope=full-pr+ for platform in linux/amd64 linux/arm64; do+ arch=${platform#linux/}+ rm -rf "$scratch/full" "$scratch/regular"+ mkdir -p "$scratch/full" "$scratch/regular"++ docker pull --platform "$platform" "$FULL_IMAGE"+ full_container=$(docker create --platform "$platform" "$FULL_IMAGE")+ docker cp "$full_container:/opt/ministack/mysql-plugins/." "$scratch/full"++ docker pull --platform "$platform" "$REGULAR_IMAGE"+ regular_container=$(docker create --platform "$platform" "$REGULAR_IMAGE")+ docker cp "$regular_container:/opt/ministack/mysql-plugins/." "$scratch/regular"++ test -f "$scratch/full/8.0/$arch/aws_auth_plugin.so"+ test -f "$scratch/full/8.4/$arch/aws_auth_plugin.so"+ diff -qr "$scratch/full" "$scratch/regular"++ docker rm "$full_container" "$regular_container" >/dev/null+ full_container=+ regular_container=+ done- name: Comment on PR with Docker image infoif: steps.guard.outputs.skip != 'true'@@ -162,3 +264,4 @@ jobs:body: |Docker image for this PR has been published: `${{ steps.meta.outputs.tags }}`${{ steps.meta_full.outputs.tags && format('Full variant: `{0}`', steps.meta_full.outputs.tags) || '' }}+ Plugin donor used by the regular variant: `${{ steps.plugin_donor.outputs.image }}`diff --git a/Dockerfile b/Dockerfileindex 889ec026..6838888e 100644--- a/Dockerfile+++ b/Dockerfile@@ -1,6 +1,7 @@# Non-release builds intentionally use the maintainer-requested fallback of the latest# published full image. Release builds override this in docker-publish.yml with the-# same-release full-image digest; plugin-source PRs use the full-preview label path.+# same-release full-image digest; plugin-source PRs build and consume a head-matched+# full-preview digest in docker-publish-on-pr.yml.ARG PLUGIN_DONOR_IMAGE=ghcr.io/ministackorg/ministack:fullFROM python:3.13-alpine AS builderdiff --git a/README.md b/README.mdindex 1e416656..aec77991 100644--- a/README.md+++ b/README.md@@ -567,7 +567,7 @@ MiniStack expands AWS SAM transforms (`Transform: AWS::Serverless-2016-10-31`) i| Service | Operations | Notes ||---------|-----------|-------|| **ECS** | CreateCluster, DeleteCluster, DescribeClusters, ListClusters, UpdateCluster, UpdateClusterSettings, PutClusterCapacityProviders, RegisterTaskDefinition, DeregisterTaskDefinition, DescribeTaskDefinition, ListTaskDefinitions, ListTaskDefinitionFamilies, DeleteTaskDefinitions, CreateService, DeleteService, DescribeServices, UpdateService, ListServices, ListServicesByNamespace, RunTask, StopTask, DescribeTasks, ListTasks, ExecuteCommand, UpdateTaskProtection, GetTaskProtection, CreateCapacityProvider, UpdateCapacityProvider, DeleteCapacityProvider, DescribeCapacityProviders, TagResource, UntagResource, ListTagsForResource, ListAccountSettings, PutAccountSetting, PutAccountSettingDefault, DeleteAccountSetting, PutAttributes, DeleteAttributes, ListAttributes, DescribeServiceDeployments, ListServiceDeployments, DescribeServiceRevisions, SubmitTaskStateChange, SubmitContainerStateChange, SubmitAttachmentStateChanges, DiscoverPollEndpoint | 47 actions; `RunTask` starts real Docker containers via Docker socket; full Terraform ECS coverage |-| **RDS** | CreateDBInstance, DeleteDBInstance, DescribeDBInstances, ModifyDBInstance, StartDBInstance, StopDBInstance, RebootDBInstance, CreateDBInstanceReadReplica, RestoreDBInstanceFromDBSnapshot, CreateDBCluster, DeleteDBCluster, DescribeDBClusters, ModifyDBCluster, StartDBCluster, StopDBCluster, FailoverDBCluster, CreateDBSnapshot, DeleteDBSnapshot, DescribeDBSnapshots, CreateDBClusterSnapshot, DescribeDBClusterSnapshots, DeleteDBClusterSnapshot, CreateDBSubnetGroup, DeleteDBSubnetGroup, DescribeDBSubnetGroups, ModifyDBSubnetGroup, CreateDBParameterGroup, DeleteDBParameterGroup, DescribeDBParameterGroups, DescribeDBParameters, ModifyDBParameterGroup, CreateDBClusterParameterGroup, DescribeDBClusterParameterGroups, DeleteDBClusterParameterGroup, DescribeDBClusterParameters, ModifyDBClusterParameterGroup, CreateOptionGroup, DeleteOptionGroup, DescribeOptionGroups, DescribeOptionGroupOptions, ListTagsForResource, AddTagsToResource, RemoveTagsFromResource, DescribeDBEngineVersions, DescribeOrderableDBInstanceOptions, CreateGlobalCluster, DescribeGlobalClusters, DeleteGlobalCluster, RemoveFromGlobalCluster, ModifyGlobalCluster, CreateDBProxy, DescribeDBProxies, ModifyDBProxy, DeleteDBProxy, CreateDBProxyEndpoint, DescribeDBProxyEndpoints, ModifyDBProxyEndpoint, DeleteDBProxyEndpoint, DescribeDBProxyTargetGroups, ModifyDBProxyTargetGroup, RegisterDBProxyTargets, DeregisterDBProxyTargets, DescribeDBProxyTargets | `CreateDBInstance` spins up real Postgres/MySQL Docker container, returns actual `host:port` endpoint; a cluster's endpoint is its stable AWS-shaped name, registered as a Docker network alias on the backing container (on user-defined networks; the default bridge keeps the address form), so a stored endpoint keeps resolving after the container is replaced; Aurora MySQL 8.0/8.4 compatibility objects (`AWSAuthenticationPlugin`, RDS/Aurora stored procedures, predefined S3 roles) install automatically for supported MySQL images so Aurora provisioning workflows run — the plugin rejects authentication (non-enforcing); disable with `MINISTACK_MYSQL_IAM_AUTH=off`. RDS Proxy is control plane only — proxies, proxy endpoints, the default target group and registered targets are stored and returned with the AWS shapes, but nothing listens on the proxy endpoint hostname; clients keep connecting to the instance or cluster endpoint |+| **RDS** | CreateDBInstance, DeleteDBInstance, DescribeDBInstances, ModifyDBInstance, StartDBInstance, StopDBInstance, RebootDBInstance, CreateDBInstanceReadReplica, RestoreDBInstanceFromDBSnapshot, CreateDBCluster, DeleteDBCluster, DescribeDBClusters, ModifyDBCluster, StartDBCluster, StopDBCluster, FailoverDBCluster, CreateDBSnapshot, DeleteDBSnapshot, DescribeDBSnapshots, CreateDBClusterSnapshot, DescribeDBClusterSnapshots, DeleteDBClusterSnapshot, CreateDBSubnetGroup, DeleteDBSubnetGroup, DescribeDBSubnetGroups, ModifyDBSubnetGroup, CreateDBParameterGroup, DeleteDBParameterGroup, DescribeDBParameterGroups, DescribeDBParameters, ModifyDBParameterGroup, CreateDBClusterParameterGroup, DescribeDBClusterParameterGroups, DeleteDBClusterParameterGroup, DescribeDBClusterParameters, ModifyDBClusterParameterGroup, CreateOptionGroup, DeleteOptionGroup, DescribeOptionGroups, DescribeOptionGroupOptions, ListTagsForResource, AddTagsToResource, RemoveTagsFromResource, DescribeDBEngineVersions, DescribeOrderableDBInstanceOptions, CreateGlobalCluster, DescribeGlobalClusters, DeleteGlobalCluster, RemoveFromGlobalCluster, ModifyGlobalCluster, CreateDBProxy, DescribeDBProxies, ModifyDBProxy, DeleteDBProxy, CreateDBProxyEndpoint, DescribeDBProxyEndpoints, ModifyDBProxyEndpoint, DeleteDBProxyEndpoint, DescribeDBProxyTargetGroups, ModifyDBProxyTargetGroup, RegisterDBProxyTargets, DeregisterDBProxyTargets, DescribeDBProxyTargets | `CreateDBInstance` spins up real Postgres/MySQL Docker container, returns actual `host:port` endpoint; a cluster's endpoint is its stable AWS-shaped name, registered as a Docker network alias on the backing container (on user-defined networks; the default bridge keeps the address form), so a stored endpoint keeps resolving after the container is replaced; Aurora MySQL 8.0/8.4 compatibility objects (`AWSAuthenticationPlugin`, RDS/Aurora stored procedures, predefined S3 roles) install automatically for supported MySQL images so Aurora provisioning workflows run — the plugin rejects authentication (non-enforcing). RDS Proxy is control plane only — proxies, proxy endpoints, the default target group and registered targets are stored and returned with the AWS shapes, but nothing listens on the proxy endpoint hostname; clients keep connecting to the instance or cluster endpoint || **Aurora DSQL** | CreateCluster, GetCluster, UpdateCluster, DeleteCluster, ListClusters, PutClusterPolicy, GetClusterPolicy, DeleteClusterPolicy, CreateStream, GetStream, ListStreams, DeleteStream, GetVpcEndpointServiceName, TagResource, UntagResource, ListTagsForResource | `CreateCluster` spins up a real `postgres:16-alpine` Docker container fronted by a validating PG wire-protocol proxy that **fails like real DSQL** — rejects unsupported SQL (`serial`, extensions, temp tables, ...), accepts `CREATE INDEX ASYNC` + `sys.jobs`; endpoint is `localhost:<port>`; any password accepted || **ElastiCache** | CreateCacheCluster, DeleteCacheCluster, DescribeCacheClusters, ModifyCacheCluster, RebootCacheCluster, CreateReplicationGroup, DeleteReplicationGroup, DescribeReplicationGroups, ModifyReplicationGroup, IncreaseReplicaCount, DecreaseReplicaCount, CreateCacheSubnetGroup, DescribeCacheSubnetGroups, ModifyCacheSubnetGroup, DeleteCacheSubnetGroup, CreateCacheParameterGroup, DescribeCacheParameterGroups, ModifyCacheParameterGroup, ResetCacheParameterGroup, DeleteCacheParameterGroup, DescribeCacheParameters, DescribeCacheEngineVersions, CreateUser, DescribeUsers, DeleteUser, ModifyUser, CreateUserGroup, DescribeUserGroups, DeleteUserGroup, ModifyUserGroup, ListTagsForResource, AddTagsToResource, RemoveTagsFromResource, CreateSnapshot, DeleteSnapshot, DescribeSnapshots, DescribeEvents | `CreateCacheCluster` spins up a real Redis, Valkey, or Memcached Docker container (`Engine=redis`/`valkey`/`memcached`) || **Glue** | CreateDatabase, DeleteDatabase, GetDatabase, GetDatabases, UpdateDatabase, CreateTable, DeleteTable, GetTable, GetTables, UpdateTable, BatchDeleteTable, CreatePartition, DeletePartition, GetPartition, GetPartitions, BatchCreatePartition, BatchGetPartition, CreatePartitionIndex, GetPartitionIndexes, GetColumnStatisticsForTable, UpdateColumnStatisticsForTable, DeleteColumnStatisticsForTable, GetColumnStatisticsForPartition, UpdateColumnStatisticsForPartition, DeleteColumnStatisticsForPartition, CreateConnection, DeleteConnection, GetConnection, GetConnections, CreateCrawler, DeleteCrawler, GetCrawler, GetCrawlers, UpdateCrawler, StartCrawler, StopCrawler, GetCrawlerMetrics, CreateJob, DeleteJob, GetJob, GetJobs, UpdateJob, StartJobRun, GetJobRun, GetJobRuns, BatchStopJobRun, CreateTrigger, GetTrigger, DeleteTrigger, UpdateTrigger, StartTrigger, StopTrigger, ListTriggers, BatchGetTriggers, GetTriggers, CreateWorkflow, GetWorkflow, DeleteWorkflow, UpdateWorkflow, StartWorkflowRun, CreateUserDefinedFunction, UpdateUserDefinedFunction, DeleteUserDefinedFunction, GetUserDefinedFunction, GetUserDefinedFunctions, CreateSecurityConfiguration, DeleteSecurityConfiguration, GetSecurityConfiguration, GetSecurityConfigurations, CreateClassifier, GetClassifier, GetClassifiers, DeleteClassifier, TagResource, UntagResource, GetTags | Python shell jobs actually execute via subprocess; Spark jobs run on the official `amazon/aws-glue-libs` PySpark image (`GlueVersion: 4.0` / `3.0`) on MiniStack's Docker network; Glue-backed Iceberg REST catalog at `/iceberg` (signed with the `glue` credential scope, prefix `catalogs/{catalog}`) — read path for tables whose `metadata_location` parameter points at a metadata.json on MiniStack S3, e.g. DuckDB `ATTACH ... (TYPE iceberg, ENDPOINT 'localhost:4566/iceberg', AUTHORIZATION_TYPE 'sigv4')` with `USE_SSL=1` |