Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

feat(dsql): foreign keys, `FOR KEY SHARE`, and CDC streams (#1567)

Edoardo Viviani3bee75f39b65
diff --git a/CHANGELOG.md b/CHANGELOG.md
index d02a947d..75c6eca3 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,16 @@ All notable changes to MiniStack will be documented here.
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
Versioning follows [Semantic Versioning](https://semver.org/).
+## [Unreleased]
+
+### Added
+- **Aurora DSQL — foreign key constraints** — every `REFERENCES` / `FOREIGN KEY` clause was refused `0A000`, so a schema with referential integrity could not be created at all; Aurora DSQL has supported foreign keys since 2026-08-26. They now reach the backend as written — the five referential actions, `MATCH FULL` / `MATCH SIMPLE`, self-references, composite keys and deferrability — with the two DSQL-specific rules enforced, measured against a live cluster (eu-central-1, 2026-08-30): `ALTER TABLE ... ADD CONSTRAINT ... FOREIGN KEY` must use `NOT VALID` (as `CHECK` already did, with `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` validating it as a job afterwards), and `DEFERRABLE` is refused on any other kind of constraint, in the service's three wordings (`DEFERRABLE constraint not supported`, `CHECK constraints cannot be marked DEFERRABLE`, and `ALTER TABLE / ADD CONSTRAINT UNIQUE / DEFERRED / INITIALLY DEFERRED not supported`), DSQL restricting it to foreign keys; `NOT DEFERRABLE` and `INITIALLY IMMEDIATE` declare no deferral and are accepted anywhere. Referential integrity is the backend's, so a referencing write racing a change to the referenced key waits on a row lock instead of failing the loser with `40001` (OC000). Contributed by @vivedo.
+- **Aurora DSQL — `SELECT ... FOR KEY SHARE`** — the clause was refused `0A000` alongside `FOR SHARE` and `FOR NO KEY UPDATE`, which matched a live cluster in August but not the service since 2026-08-25; it is now forwarded, and only `FOR SHARE` and `FOR NO KEY UPDATE` are refused. Contributed by @vivedo.
+- **Aurora DSQL — change data capture streams, and `GetVpcEndpointServiceName`** — `CreateStream` and the three other CDC operations answered `ValidationException` (`Unknown path`), so an application that provisions a stream alongside its cluster could not be pointed at MiniStack at all; CDC went GA on 2026-07-08. Streams now create, describe, list, page, tag and delete with the API's shapes and defaults — `clientToken` idempotency, the 5-per-cluster quota as `ServiceQuotaExceededException` (`You have reached the stream limit.`), and a cluster's streams going with it on delete. Metadata only: no change record reaches the Kinesis target, so a stream is `ACTIVE` on creation unless its target is one MiniStack can see is broken — an absent Kinesis stream, or a role that cannot be assumed under `AUTH=true` — which lands `FAILED` carrying the service's `statusReason` error code. `GetVpcEndpointServiceName`, missing since the emulator landed, answers a stable per-cluster `com.amazonaws.<region>.dsql-<suffix>`. Contributed by @vivedo.
+
+### Fixed
+- **Aurora DSQL — the `ALTER TABLE` refusals answer what the service answers** — measuring the emulator against a live cluster (eu-central-1, 2026-08-30) found four statements answered differently. A refused `ADD CONSTRAINT` — `PRIMARY KEY`, an inline `UNIQUE`, `EXCLUDE`, or a `CHECK`/`FOREIGN KEY` without `NOT VALID` — drew three invented messages where DSQL answers one, `unsupported ALTER TABLE ADD CONSTRAINT statement`. `ADD COLUMN` refused only `DEFAULT` and `NOT NULL` and let `REFERENCES`, `UNIQUE`, `CHECK`, `GENERATED ... STORED` and an inline identity through, where DSQL takes `name type [STORAGE mode]` and nothing else (`ALTER TABLE ADD COLUMN with constraint not supported`, a bare `NULL` apart, and `COLLATE clause not supported` for a collation); `ALTER COLUMN ... ADD GENERATED AS IDENTITY` on an existing column is unaffected. `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` over rows that break the constraint relayed the backend's `23503` at submit time, where DSQL hands back a `job_id` and fails the job — `sys.jobs` now reports `failed` with the violation as `details` and `sys.wait_for_job` answers false. Adding several columns in one statement also read the second action as part of the first column's type. Contributed by @vivedo.
+
## [1.5.3] — 2026-08-28
### Added
diff --git a/README.md b/README.md
index b61b4b79..0896c1c8 100644
--- a/README.md
+++ b/README.md
@@ -565,7 +565,7 @@ MiniStack expands AWS SAM transforms (`Transform: AWS::Serverless-2016-10-31`) i
|---------|-----------|-------|
| **ECS** | CreateCluster, DeleteCluster, DescribeClusters, ListClusters, UpdateCluster, UpdateClusterSettings, PutClusterCapacityProviders, RegisterTaskDefinition, DeregisterTaskDefinition, DescribeTaskDefinition, ListTaskDefinitions, ListTaskDefinitionFamilies, DeleteTaskDefinitions, CreateService, DeleteService, DescribeServices, UpdateService, ListServices, ListServicesByNamespace, RunTask, StopTask, DescribeTasks, ListTasks, ExecuteCommand, UpdateTaskProtection, GetTaskProtection, CreateCapacityProvider, UpdateCapacityProvider, DeleteCapacityProvider, DescribeCapacityProviders, TagResource, UntagResource, ListTagsForResource, ListAccountSettings, PutAccountSetting, PutAccountSettingDefault, DeleteAccountSetting, PutAttributes, DeleteAttributes, ListAttributes, DescribeServiceDeployments, ListServiceDeployments, DescribeServiceRevisions, SubmitTaskStateChange, SubmitContainerStateChange, SubmitAttachmentStateChanges, DiscoverPollEndpoint | 47 actions; `RunTask` starts real Docker containers via Docker socket; full Terraform ECS coverage |
| **RDS** | CreateDBInstance, DeleteDBInstance, DescribeDBInstances, ModifyDBInstance, StartDBInstance, StopDBInstance, RebootDBInstance, CreateDBInstanceReadReplica, RestoreDBInstanceFromDBSnapshot, CreateDBCluster, DeleteDBCluster, DescribeDBClusters, ModifyDBCluster, StartDBCluster, StopDBCluster, FailoverDBCluster, CreateDBSnapshot, DeleteDBSnapshot, DescribeDBSnapshots, CreateDBClusterSnapshot, DescribeDBClusterSnapshots, DeleteDBClusterSnapshot, CreateDBSubnetGroup, DeleteDBSubnetGroup, DescribeDBSubnetGroups, ModifyDBSubnetGroup, CreateDBParameterGroup, DeleteDBParameterGroup, DescribeDBParameterGroups, DescribeDBParameters, ModifyDBParameterGroup, CreateDBClusterParameterGroup, DescribeDBClusterParameterGroups, DeleteDBClusterParameterGroup, DescribeDBClusterParameters, ModifyDBClusterParameterGroup, CreateOptionGroup, DeleteOptionGroup, DescribeOptionGroups, DescribeOptionGroupOptions, ListTagsForResource, AddTagsToResource, RemoveTagsFromResource, DescribeDBEngineVersions, DescribeOrderableDBInstanceOptions, CreateGlobalCluster, DescribeGlobalClusters, DeleteGlobalCluster, RemoveFromGlobalCluster, ModifyGlobalCluster, CreateDBProxy, DescribeDBProxies, ModifyDBProxy, DeleteDBProxy, CreateDBProxyEndpoint, DescribeDBProxyEndpoints, ModifyDBProxyEndpoint, DeleteDBProxyEndpoint, DescribeDBProxyTargetGroups, ModifyDBProxyTargetGroup, RegisterDBProxyTargets, DeregisterDBProxyTargets, DescribeDBProxyTargets | `CreateDBInstance` spins up real Postgres/MySQL Docker container, returns actual `host:port` endpoint; Aurora MySQL 8.0/8.4 compatibility objects (`AWSAuthenticationPlugin`, RDS/Aurora stored procedures, predefined S3 roles) install automatically for supported MySQL images so Aurora provisioning workflows run — the plugin rejects authentication (non-enforcing); disable with `MINISTACK_MYSQL_IAM_AUTH=off`. RDS Proxy is control plane only — proxies, proxy endpoints, the default target group and registered targets are stored and returned with the AWS shapes, but nothing listens on the proxy endpoint hostname; clients keep connecting to the instance or cluster endpoint |
-| **Aurora DSQL** | CreateCluster, GetCluster, UpdateCluster, DeleteCluster, ListClusters, PutClusterPolicy, GetClusterPolicy, DeleteClusterPolicy, TagResource, UntagResource, ListTagsForResource | `CreateCluster` spins up a real `postgres:16-alpine` Docker container fronted by a validating PG wire-protocol proxy that **fails like real DSQL** — rejects unsupported SQL (`serial`, foreign keys, extensions, ...), accepts `CREATE INDEX ASYNC` + `sys.jobs`; endpoint is `localhost:<port>`; any password accepted |
+| **Aurora DSQL** | CreateCluster, GetCluster, UpdateCluster, DeleteCluster, ListClusters, PutClusterPolicy, GetClusterPolicy, DeleteClusterPolicy, CreateStream, GetStream, ListStreams, DeleteStream, GetVpcEndpointServiceName, TagResource, UntagResource, ListTagsForResource | `CreateCluster` spins up a real `postgres:16-alpine` Docker container fronted by a validating PG wire-protocol proxy that **fails like real DSQL** — rejects unsupported SQL (`serial`, extensions, temp tables, ...), accepts `CREATE INDEX ASYNC` + `sys.jobs`; endpoint is `localhost:<port>`; any password accepted |
| **ElastiCache** | CreateCacheCluster, DeleteCacheCluster, DescribeCacheClusters, ModifyCacheCluster, RebootCacheCluster, CreateReplicationGroup, DeleteReplicationGroup, DescribeReplicationGroups, ModifyReplicationGroup, IncreaseReplicaCount, DecreaseReplicaCount, CreateCacheSubnetGroup, DescribeCacheSubnetGroups, ModifyCacheSubnetGroup, DeleteCacheSubnetGroup, CreateCacheParameterGroup, DescribeCacheParameterGroups, ModifyCacheParameterGroup, ResetCacheParameterGroup, DeleteCacheParameterGroup, DescribeCacheParameters, DescribeCacheEngineVersions, CreateUser, DescribeUsers, DeleteUser, ModifyUser, CreateUserGroup, DescribeUserGroups, DeleteUserGroup, ModifyUserGroup, ListTagsForResource, AddTagsToResource, RemoveTagsFromResource, CreateSnapshot, DeleteSnapshot, DescribeSnapshots, DescribeEvents | `CreateCacheCluster` spins up a real Redis, Valkey, or Memcached Docker container (`Engine=redis`/`valkey`/`memcached`) |
| **Glue** | CreateDatabase, DeleteDatabase, GetDatabase, GetDatabases, UpdateDatabase, CreateTable, DeleteTable, GetTable, GetTables, UpdateTable, BatchDeleteTable, CreatePartition, DeletePartition, GetPartition, GetPartitions, BatchCreatePartition, BatchGetPartition, CreatePartitionIndex, GetPartitionIndexes, GetColumnStatisticsForTable, UpdateColumnStatisticsForTable, DeleteColumnStatisticsForTable, GetColumnStatisticsForPartition, UpdateColumnStatisticsForPartition, DeleteColumnStatisticsForPartition, CreateConnection, DeleteConnection, GetConnection, GetConnections, CreateCrawler, DeleteCrawler, GetCrawler, GetCrawlers, UpdateCrawler, StartCrawler, StopCrawler, GetCrawlerMetrics, CreateJob, DeleteJob, GetJob, GetJobs, UpdateJob, StartJobRun, GetJobRun, GetJobRuns, BatchStopJobRun, CreateTrigger, GetTrigger, DeleteTrigger, UpdateTrigger, StartTrigger, StopTrigger, ListTriggers, BatchGetTriggers, GetTriggers, CreateWorkflow, GetWorkflow, DeleteWorkflow, UpdateWorkflow, StartWorkflowRun, CreateUserDefinedFunction, UpdateUserDefinedFunction, DeleteUserDefinedFunction, GetUserDefinedFunction, GetUserDefinedFunctions, CreateSecurityConfiguration, DeleteSecurityConfiguration, GetSecurityConfiguration, GetSecurityConfigurations, CreateClassifier, GetClassifier, GetClassifiers, DeleteClassifier, TagResource, UntagResource, GetTags | Python shell jobs actually execute via subprocess; Spark jobs run on the official `amazon/aws-glue-libs` PySpark image (`GlueVersion: 4.0` / `3.0`) on MiniStack's Docker network; Glue-backed Iceberg REST catalog at `/iceberg` (signed with the `glue` credential scope, prefix `catalogs/{catalog}`) — read path for tables whose `metadata_location` parameter points at a metadata.json on MiniStack S3, e.g. DuckDB `ATTACH ... (TYPE iceberg, ENDPOINT 'localhost:4566/iceberg', AUTHORIZATION_TYPE 'sigv4')` with `USE_SSL=1` |
| **S3 Tables** | CreateTableBucket, ListTableBuckets, GetTableBucket, DeleteTableBucket, CreateNamespace, ListNamespaces, GetNamespace, DeleteNamespace, CreateTable, ListTables, GetTable, DeleteTable, GetTableMetadataLocation, UpdateTableMetadataLocation | Iceberg-format tables; embedded Iceberg REST catalog at `/iceberg` so Spark jobs (`spark.sql.catalog.*.type=rest`) can create / load / commit without an external catalog; data files in S3, metadata in memory |
@@ -649,13 +649,15 @@ only `DeleteDBCluster` removes the container and its storage.
Each `CreateCluster` returns an `ACTIVE` cluster with an endpoint. Set `DSQL_STRICT=1` to additionally spin up a real `postgres:16-alpine` container per cluster (capped by a fixed port window off `DSQL_BASE_PORT`, default 30 ports) behind an in-process Postgres wire-protocol proxy.
-**The proxy fails like real DSQL**: SQL that DSQL rejects fails here too (`serial` columns, foreign keys, `CREATE EXTENSION`, `TRUNCATE`, temp tables, `ALTER COLUMN TYPE`, `SET NOT NULL`, materialized views, partitioned tables, ...), index creation is always asynchronous — plain `CREATE INDEX` is refused (`unsupported mode. please use CREATE INDEX ASYNC.`), as are `CONCURRENTLY`, `USING` and partial (`WHERE`) indexes — while expression index keys are supported as long as the functions used are immutable (violations fail with `42P17`, same as the backend; `INCLUDE` columns can't be expressions), `CREATE INDEX ASYNC` and `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` return a `job_id`, `sys.jobs` / `sys.wait_for_job` are emulated, CHECK constraints via ALTER TABLE require `NOT VALID`, `ALTER TABLE ... DROP COLUMN` works (several columns in one statement included) except on a primary key column, identity columns are bigint-only (CACHE 1 or ≥ 65536), index limits are enforced (8 key columns/index, 24/table), transactions allow only one DDL statement with no DDL+DML mixing, `SELECT ... FOR UPDATE` is the only locking clause supported (`FOR SHARE`, `FOR KEY SHARE` and `FOR NO KEY UPDATE` are refused, at any nesting depth; the predicate itself is unrestricted), a transaction that outlives another session's DDL fails once with `40001` (OC001) and succeeds on retry, and the transaction limits (3,000 rows / 10 MiB / 5 min) are enforced.
+**The proxy fails like real DSQL**: SQL that DSQL rejects fails here too (`serial` columns, `CREATE EXTENSION`, `TRUNCATE`, temp tables, `ALTER COLUMN TYPE`, `SET NOT NULL`, materialized views, partitioned tables, ...), index creation is always asynchronous — plain `CREATE INDEX` is refused (`unsupported mode. please use CREATE INDEX ASYNC.`), as are `CONCURRENTLY`, `USING` and partial (`WHERE`) indexes — while expression index keys are supported as long as the functions used are immutable (violations fail with `42P17`, same as the backend; `INCLUDE` columns can't be expressions), `CREATE INDEX ASYNC` and `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` return a `job_id`, `sys.jobs` / `sys.wait_for_job` are emulated (data that does not satisfy the constraint fails the job, not the submit — `sys.jobs` reports it `failed` with the violation as `details`), CHECK and FOREIGN KEY constraints via ALTER TABLE require `NOT VALID` and every other `ADD CONSTRAINT` form is refused, `ADD COLUMN` takes a name, a type and an optional `STORAGE` mode and nothing else, foreign keys are supported in full (all five referential actions, `MATCH FULL`/`MATCH SIMPLE`, deferrable — `DEFERRABLE` on any other kind of constraint is refused), `ALTER TABLE ... DROP COLUMN` works (several columns in one statement included) except on a primary key column, identity columns are bigint-only (CACHE 1 or ≥ 65536), index limits are enforced (8 key columns/index, 24/table), transactions allow only one DDL statement with no DDL+DML mixing, `SELECT ... FOR UPDATE` and `FOR KEY SHARE` are the only locking clauses supported (`FOR SHARE` and `FOR NO KEY UPDATE` are refused, at any nesting depth; the predicate itself is unrestricted), a transaction that outlives another session's DDL fails once with `40001` (OC001) and succeeds on retry, and the transaction limits (3,000 rows / 10 MiB / 5 min) are enforced.
+
+CDC streams are metadata only: `CreateStream` validates its Kinesis target definition, enforces the 5-streams-per-cluster quota (`ServiceQuotaExceededException`) and reports `ACTIVE` right away, but no change record is ever delivered — a target whose Kinesis stream is absent (or, with `AUTH=true`, whose role cannot be assumed) lands `FAILED` with the service's `statusReason` code instead. Deleting a cluster deletes its streams.
`GetCluster` returns an `endpoint` of the form `localhost:<port>` (ports allocated from `DSQL_BASE_PORT`, default `25432`). Any user/password is accepted — IAM auth tokens from `generate_db_connect_auth_token` work but aren't validated. Connect with e.g. `psql "postgresql://admin@localhost:25432/postgres"` and enter any test password when prompted (no TLS — use `sslmode=disable`/`prefer`; with psycopg2 use `autocommit=True`).
**Both wire protocols are validated**: the simple protocol (psql, psycopg2) and the extended `Parse`/`Bind`/`Execute` protocol used by pgjdbc, pgx, asyncpg, psycopg3 and every ORM built on prepared statements. Transaction state is read from the backend's `ReadyForQuery` status byte rather than inferred from statement text, so `BEGIN`/`COMMIT` sent over either protocol keeps the transaction rules accurate. A statement the proxy rejects inside a transaction aborts that block exactly as a real error would — later statements fail with `25P02` and `COMMIT` reports `ROLLBACK`.
-Limitations: SQL analysis is heuristic regex, not a real parser (statement splitting, comment stripping and parenthesis matching are lexer-aware, but clause-level rules are pattern matches); the proxy's immutability check for index expressions is a denylist of well-known volatile functions — the backend catalog enforces the rest; transaction row counting is static (VALUES tuples only — `INSERT ... SELECT`/UPDATE/DELETE row counts aren't tracked); OC001 emulation is optimistic (catalog version bumps on forwarded DDL, not on successful commit); the 1 KiB PK/index key-size limit is not enforced; and in a pipelined extended-protocol batch (several statements before one `Sync`) the rules that need to inspect backend state — the 24-index limit and the `FOR UPDATE` key check — are skipped, while `CREATE INDEX ASYNC` is rejected as it is in multi-statement `Q` batches. Error messages match live Aurora DSQL where verified, otherwise Postgres-style approximations (`0A000`, `25006`).
+Limitations: SQL analysis is heuristic regex, not a real parser (statement splitting, comment stripping and parenthesis matching are lexer-aware, but clause-level rules are pattern matches); the proxy's immutability check for index expressions is a denylist of well-known volatile functions — the backend catalog enforces the rest; transaction row counting is static (VALUES tuples only — `INSERT ... SELECT`/UPDATE/DELETE row counts aren't tracked); OC001 emulation is optimistic (catalog version bumps on forwarded DDL, not on successful commit); foreign keys are enforced by the backend, so a referencing write racing a change to the referenced key waits on a row lock instead of failing the loser with `40001` (OC000) the way DSQL's optimistic resolution does; the 1 KiB PK/index key-size limit is not enforced; and in a pipelined extended-protocol batch (several statements before one `Sync`) the rules that need to inspect backend state — the 24-index limit and the `FOR UPDATE` key check — are skipped, while `CREATE INDEX ASYNC` is rejected as it is in multi-statement `Q` batches. Error messages match live Aurora DSQL where verified, otherwise Postgres-style approximations (`0A000`, `25006`).
---
diff --git a/ministack/core/pgproxy.py b/ministack/core/pgproxy.py
index 338bba93..28184b25 100644
--- a/ministack/core/pgproxy.py
+++ b/ministack/core/pgproxy.py
@@ -29,6 +29,9 @@ Documented limitations:
same relation supersedes (``SELECT * FROM (SELECT id FROM t FOR UPDATE) x
FOR SHARE``). Both are no-ops; the proxy refuses them anyway rather than
model the planner's strongest-lock-wins merge.
+- Foreign keys are enforced by the backend, so a referencing write and a
+ concurrent change to the referenced key wait on a row lock where DSQL
+ resolves them optimistically and fails the loser with ``40001`` (OC000).
- Transaction row counting is static (``VALUES`` tuples only); ``INSERT ...
SELECT`` / ``UPDATE`` / ``DELETE`` affected-row counts are not tracked.
- OC001 emulation is optimistic: the catalog version bumps on forwarded DDL,
@@ -163,12 +166,13 @@ def _bump_catalog(cluster_id):
return _catalog_versions[cluster_id]
-def _register_job(cluster_id, object_name, job_type="INDEX_BUILD"):
+def _register_job(cluster_id, object_name, job_type="INDEX_BUILD",
+ status="completed", details=""):
now = datetime.now(timezone.utc).isoformat()
job = {
"job_id": "".join(secrets.choice(_ID_ALPHABET) for _ in range(26)),
- "status": "completed",
- "details": "",
+ "status": status,
+ "details": details,
"job_type": job_type,
"class_id": "1259",
"object_id": str(16384 + secrets.randbelow(100000)),
@@ -667,24 +671,13 @@ def _check_denylist(sql):
if re.match(r"\s*(CREATE|ALTER)\s+TABLE\b", sql, re.I) or re.match(
r"\s*CREATE\s+(TEMP|TEMPORARY|UNLOGGED)\s+TABLE\b", sql, re.I
):
- if re.search(r"\bFOREIGN\s+KEY\b|\bREFERENCES\b", sql, re.I):
- return DsqlError("0A000", "foreign key constraints are not supported")
if re.search(r"\bPARTITION\s+BY\b|\bPARTITION\s+OF\b|\bINHERITS\b", sql, re.I):
return DsqlError("0A000", "table partitioning is not supported")
- if re.search(r"\bEXCLUDE\b", sql, re.I):
- return DsqlError("0A000", "the EXCLUDE constraint is not supported")
- if re.match(r"\s*ALTER\s+TABLE\b", sql, re.I):
- add = re.search(r"\bADD\s+COLUMN\b", sql, re.I)
- # STORAGE DEFAULT is a storage option and BY DEFAULT an identity
- # option — neither is an inline column default. Generated columns
- # (GENERATED ALWAYS AS ... STORED / identity) are allowed.
- if add and re.search(
- r"(?<!STORAGE )(?<!BY )\bDEFAULT\b|\bNOT\s+NULL\b", sql[add.end():], re.I
+ if re.search(r"\bEXCLUDE\b", sql, re.I) and not re.match(
+ r"\s*ALTER\s+TABLE\b", sql, re.I
):
- return DsqlError(
- "0A000",
- "ALTER TABLE ADD COLUMN with DEFAULT or NOT NULL is not supported",
- )
+ # On ALTER TABLE it is an ADD CONSTRAINT form, refused as one.
+ return DsqlError("0A000", "the EXCLUDE constraint is not supported")
return None
@@ -836,9 +829,15 @@ def _check_column_types(sql):
return None
m = re.match(r"\s*ALTER\s+TABLE\b", sql, re.I)
if m:
- add = re.search(r"\bADD\s+COLUMN\s+(?:IF\s+NOT\s+EXISTS\s+)?([\w\"]+)\s+", sql, re.I)
- if add:
- tail = sql[add.end():]
+ # One action per part: a statement can add several columns at once.
+ for part in _split_top_level(sql):
+ add = re.search(
+ r"\bADD\s+COLUMN\s+(?:IF\s+NOT\s+EXISTS\s+)?([\w\"]+)\s+",
+ part, re.I,
+ )
+ if not add:
+ continue
+ tail = part[add.end():]
tokens = tail.split()
raw_type = _type_from_tokens(tokens)
err = _check_type(raw_type)
@@ -857,9 +856,10 @@ def _check_column_types(sql):
# columns, but not a primary key column — needs a catalog probe, so it lives in
# _check_drop_column rather than here), SET/DROP
# DEFAULT, DROP NOT NULL, DROP EXPRESSION, identity actions, SET STORAGE,
-# ADD CONSTRAINT ... CHECK ... NOT VALID, ADD CONSTRAINT ... UNIQUE USING
-# INDEX, DROP CONSTRAINT, RENAME, SET SCHEMA, OWNER TO, and the async
-# VALIDATE CONSTRAINT form (handled as a Rewrite in validate()).
+# ADD CONSTRAINT ... CHECK / FOREIGN KEY ... NOT VALID, ADD CONSTRAINT ...
+# UNIQUE USING INDEX, ALTER CONSTRAINT, DROP CONSTRAINT, RENAME, SET SCHEMA,
+# OWNER TO, and the async VALIDATE CONSTRAINT form (handled as a Rewrite in
+# validate()).
def _check_alter_table(sql):
if not re.match(r"\s*ALTER\s+TABLE\b", sql, re.I):
return None
@@ -874,24 +874,23 @@ def _check_alter_table(sql):
add = re.search(r"\bADD\s+(?:CONSTRAINT\s+[\w\"]+\s+)?(\w+)", sql, re.I)
if add and add.group(1).upper() != "COLUMN":
what = add.group(1).upper()
- if what == "PRIMARY":
- return DsqlError(
- "0A000", "ADD CONSTRAINT PRIMARY KEY is not supported"
- )
- if what == "UNIQUE" and not re.search(
- r"\bUNIQUE\s+USING\s+INDEX\b", sql, re.I
- ):
- return DsqlError(
- "0A000",
- "ADD CONSTRAINT UNIQUE (...) is not supported — use "
- "CREATE UNIQUE INDEX ASYNC, then ADD CONSTRAINT ... "
- "UNIQUE USING INDEX",
- )
- if what == "CHECK" and not re.search(r"\bNOT\s+VALID\b", sql, re.I):
+ # Only two ADD CONSTRAINT forms exist: a CHECK or FOREIGN KEY added
+ # NOT VALID, and UNIQUE USING INDEX. Every other one draws the same
+ # message, whatever the constraint (measured, eu-central-1).
+ refused = (
+ what in ("PRIMARY", "EXCLUDE")
+ or (what == "UNIQUE"
+ and not re.search(r"\bUNIQUE\s+USING\s+INDEX\b", sql, re.I))
+ or (what in ("CHECK", "FOREIGN")
+ and not re.search(r"\bNOT\s+VALID\b", sql, re.I))
+ )
+ if refused:
return DsqlError(
- "0A000",
- "CHECK constraints added via ALTER TABLE must use NOT VALID",
+ "0A000", "unsupported ALTER TABLE ADD CONSTRAINT statement"
)
+ err = _check_add_column(sql)
+ if err:
+ return err
# ADD GENERATED ... AS IDENTITY requires an explicit CACHE value.
if re.search(r"\bADD\s+GENERATED\b", sql, re.I) and re.search(
r"\bIDENTITY\b", sql, re.I
@@ -904,6 +903,101 @@ def _check_alter_table(sql):
return None
+# ALTER TABLE ADD COLUMN takes a name, a type and an optional STORAGE mode —
+# nothing else. Every column constraint draws one message, a bare NULL (which
+# constrains nothing) and COLLATE apart (measured, eu-central-1 2026-08-30).
+_ADD_COLUMN_RE = re.compile(
+ rf"\bADD\s+(?:COLUMN\s+)?(?:IF\s+NOT\s+EXISTS\s+)?({_IDENT})\s+(.+)$",
+ re.I | re.S,
+)
+
+
+def _check_add_column(sql):
+ for part in _split_top_level(sql):
+ # ALTER COLUMN x ADD GENERATED ... is an identity action, not a column.
+ if re.search(r"\bALTER\s+(?:COLUMN\s+)?[\w\"]+\s+ADD\b", part, re.I):
+ continue
+ m = _ADD_COLUMN_RE.search(part)
+ if not m:
+ continue
+ name, quoted = fold_identifier(m.group(1))
+ if not quoted and name in _TABLE_CONSTRAINT_STARTERS:
+ continue # ADD <table constraint>, handled above
+ tokens = m.group(2).split()
+ raw_type = _type_from_tokens(tokens)
+ tail = tokens[len(raw_type.split()):]
+ i = 0
+ while i < len(tail):
+ word = tail[i].upper().rstrip(",")
+ if word == "NULL":
+ i += 1
+ elif word == "STORAGE":
+ i += 2
+ elif word == "COLLATE":
+ return DsqlError("0A000", "COLLATE clause not supported")
+ else:
+ return DsqlError(
+ "0A000",
+ "ALTER TABLE ADD COLUMN with constraint not supported",
+ )
+ return None
+
+
+# --- Foreign keys (AWS create-table-syntax-support, 2026-08-26) -------------
+
+# A foreign key is otherwise plain PostgreSQL — the referential actions, both
+# match types and deferrability all behave as they do on the backend, so the
+# only DSQL-specific rules are that ALTER TABLE must add one NOT VALID
+# (_check_alter_table) and that DEFERRABLE applies to foreign keys alone.
+# Cheap gate: only tokenize a statement that could carry a deferral.
+_DEFERRABLE_RE = re.compile(r"\bDEFERRABLE\b|\bINITIALLY\s+DEFERRED\b", re.I)
+
+
+def _check_deferrable(sql):
+ if not _DEFERRABLE_RE.search(sql):
+ return None
+ m = re.match(
+ r"\s*CREATE\s+TABLE\s+(?:IF\s+NOT\s+EXISTS\s+)?[\w\".]+\s*", sql, re.I
+ )
+ is_alter = bool(re.match(r"\s*ALTER\s+TABLE\b", sql, re.I))
+ if m and sql[m.end():].lstrip().startswith("("):
+ start = sql.index("(", m.end())
+ end = _match_paren(sql, start)
+ if end < 0:
+ return None
+ body = sql[start + 1 : end]
+ elif is_alter:
+ body = sql
+ else:
+ return None
+ for part in _split_top_level(body):
+ words = [t for k, t in _sql_tokens(part) if k == "name"]
+ pairs = list(zip(words, words[1:]))
+ # NOT DEFERRABLE is the default, so only an actual deferral counts.
+ if not any(
+ w == "deferrable" and (i == 0 or words[i - 1] != "not")
+ for i, w in enumerate(words)
+ ) and ("initially", "deferred") not in pairs:
+ continue
+ # ALTER CONSTRAINT only ever targets a foreign key constraint.
+ if ("alter", "constraint") in pairs:
+ continue
+ if "references" in words or ("foreign", "key") in pairs:
+ continue
+ if "check" in words:
+ return DsqlError(
+ "0A000", "CHECK constraints cannot be marked DEFERRABLE"
+ )
+ if is_alter and "unique" in words:
+ return DsqlError(
+ "0A000",
+ "ALTER TABLE / ADD CONSTRAINT UNIQUE / DEFERRED / "
+ "INITIALLY DEFERRED not supported",
+ )
+ return DsqlError("0A000", "DEFERRABLE constraint not supported")
+ return None
+
+
# IDENTITY columns and sequences: CACHE must be 1 or >= 65536.
def _check_cache_value(sql):
if not re.match(r"\s*(CREATE|ALTER)\s+(TABLE|SEQUENCE)\b", sql, re.I):
@@ -1089,19 +1183,17 @@ def match_wait_for_job(sql):
# --- Locking clauses (measured against Aurora DSQL, 2026-08-18) -------------
-# DSQL supports one locking clause, FOR UPDATE, with its optional
-# OF / NOWAIT / SKIP LOCKED tail. It places no restriction on the query the
-# clause locks — no WHERE clause, a non-key column, an inequality, IN/OR, a
-# join and a table without a primary key all lock fine.
-_UNSUPPORTED_LOCKS = (("no", "key", "update"), ("key", "share"), ("share",))
+# DSQL supports two locking clauses, FOR UPDATE and FOR KEY SHARE (added
+# 2026-08-25), each with its optional OF / NOWAIT / SKIP LOCKED tail. Neither
+# restricts the query the clause locks — no WHERE clause, a non-key column, an
+# inequality, IN/OR, a join and a table without a primary key all lock fine.
+_UNSUPPORTED_LOCKS = (("no", "key", "update"), ("share",))
# Cheap gate: only tokenize a statement that could carry one of those.
-_LOCK_CLAUSE_RE = re.compile(
- r"\bFOR\s+(?:NO\s+KEY\s+UPDATE|(?:KEY\s+)?SHARE)\b", re.I
-)
+_LOCK_CLAUSE_RE = re.compile(r"\bFOR\s+(?:NO\s+KEY\s+UPDATE|SHARE)\b", re.I)
def _check_locking_clause(sql):
- """Refuse every locking clause but FOR UPDATE.
+ """Refuse every locking clause but FOR UPDATE and FOR KEY SHARE.
The check reads the statement's tokens, so ``FOR SHARE`` inside a string
literal is text rather than a clause. A clause nested in a CTE or subquery
@@ -1116,8 +1208,7 @@ def _check_locking_clause(sql):
continue
words = tuple(t for k, t in toks[i + 1 : i + 4] if k == "name")
if any(words[: len(lock)] == lock for lock in _UNSUPPORTED_LOCKS):
- # Verbatim from the service, which names FOR KEY SHARE as
- # supported and then refuses it too.
+ # Verbatim from the service.
return DsqlError(
"0A000",
"locking clauses other than FOR UPDATE/FOR KEY SHARE "
@@ -1183,6 +1274,9 @@ def validate(sql, txn_state=None):
if err:
return err
err = _check_alter_table(s)
+ if err:
+ return err
+ err = _check_deferrable(s)
if err:
return err
err = _check_cache_value(s)
@@ -1604,11 +1698,32 @@ def _synth_columns(kind, result):
return result["cols"] if kind == "rows" else ["job_id"]
+def _error_payload_fields(payload):
+ """SQLSTATE and primary message out of an ErrorResponse payload."""
+ fields = {}
+ for field in payload.split(b"\0"):
+ if field:
+ fields[field[:1]] = field[1:].decode("utf-8", "replace")
+ return fields.get(b"C", ""), fields.get(b"M", "")
+
+
async def _run_rewrite(conn, b_writer, rewrite):
"""Execute a rewritten ASYNC statement; return (job, backend_error)."""
frames = await _run_backend_capture(conn, b_writer, rewrite.sql)
backend_err = next((p for t, p in frames if t == b"E"), None)
if backend_err is not None:
+ sqlstate, message = _error_payload_fields(backend_err)
+ # Data that does not satisfy the constraint (or a duplicate key for a
+ # unique index) fails the job, not its submission: DSQL answers with a
+ # job_id and reports the failure through sys.jobs. Anything else — an
+ # unresolvable relation, a syntax error — is refused on the spot. In a
+ # transaction the backend is left aborted, so relay there either way.
+ if sqlstate.startswith("23") and not conn.txn.in_txn:
+ job = _register_job(
+ conn.cluster_id, rewrite.object_name, rewrite.job_type,
+ status="failed", details=message,
+ )
+ return job, None
return None, backend_err
conn.catalog_version = _bump_catalog(conn.cluster_id)
job = _register_job(conn.cluster_id, rewrite.object_name, rewrite.job_type)
diff --git a/ministack/services/dsql.py b/ministack/services/dsql.py
index e8cd7583..0b3bc273 100644
--- a/ministack/services/dsql.py
+++ b/ministack/services/dsql.py
@@ -2,7 +2,8 @@
Amazon Aurora DSQL Emulator (control plane + data plane wiring).
REST-JSON API (no X-Amz-Target; routed on method + path). Covers cluster
-lifecycle, tags, and cluster policies.
+lifecycle, tags, cluster policies, and CDC streams (metadata only — no change
+records are delivered to the Kinesis target).
Data plane: with ``DSQL_STRICT=1`` and a Docker daemon available, each
cluster gets a real Postgres container (RDS idiom, capped by a fixed port
@@ -13,6 +14,7 @@ the cluster goes ACTIVE metadata-only — nothing listens on its port.
import asyncio
import copy
+import hashlib
import json
import logging
import os
@@ -46,7 +48,9 @@ DSQL_STRICT = os.environ.get("DSQL_STRICT", "0").lower() in ("1", "true", "yes")
_clusters = AccountScopedDict() # identifier -> cluster dict
_client_tokens = AccountScopedDict() # clientToken -> identifier (create idempotency)
-_tags = AccountScopedDict() # cluster arn -> {key: value}
+_tags = AccountScopedDict() # cluster/stream arn -> {key: value}
+_streams = AccountScopedDict() # "{clusterId}/{streamId}" -> stream dict
+_stream_tokens = AccountScopedDict() # clientToken -> stream key (create idempotency)
_port_counter = [BASE_PORT]
_port_lock = threading.Lock()
@@ -583,6 +587,10 @@ def _delete_cluster(identifier, query_params):
cluster["status"] = "DELETING"
resp = _lifecycle_response(cluster)
_tags.pop(cluster["arn"], None)
+ for key, stream in list(_streams.items()):
+ if stream["clusterIdentifier"] == identifier:
+ _tags.pop(stream["arn"], None)
+ del _streams[key]
del _clusters[identifier]
if client_token:
# Cache the entire response so an idempotent retry succeeds.
@@ -717,14 +725,249 @@ def _delete_cluster_policy(identifier, query_params):
return json_response({"policyVersion": deleted_version})
+# ---------------------------------------------------------------------------
+# CDC streams
+# ---------------------------------------------------------------------------
+
+# A cluster allows 5 CDC streams (AWS cluster quotas). Only the Kinesis target,
+# UNORDERED ordering and the JSON format exist in the API today.
+_MAX_STREAMS_PER_CLUSTER = 5
+_STREAM_ORDERINGS = ("UNORDERED",)
+_STREAM_FORMATS = ("JSON",)
+
+
+def _stream_arn(cluster_identifier, stream_identifier, account_id=None, region=None):
+ cluster = _cluster_arn(cluster_identifier, account_id, region)
+ return f"{cluster}/stream/{stream_identifier}"
+
+
+def _stream_key(cluster_identifier, stream_identifier):
+ return f"{cluster_identifier}/{stream_identifier}"
+
+
+def _stream_not_found(cluster_identifier, stream_identifier):
+ return _not_found(
+ _stream_key(cluster_identifier, stream_identifier), resource_type="stream"
+ )
+
+
+def _stream_summary(stream):
+ """The five members ListStreams and DeleteStream report."""
+ return {
+ "clusterIdentifier": stream["clusterIdentifier"],
+ "streamIdentifier": stream["streamIdentifier"],
+ "arn": stream["arn"],
+ "status": stream["status"],
+ "creationTime": stream["creationTime"],
+ }
+
+
+def _stream_response(stream, include_target=False):
+ resp = _stream_summary(stream)
+ resp["ordering"] = stream["ordering"]
+ resp["format"] = stream["format"]
+ if include_target:
+ resp["targetDefinition"] = stream["targetDefinition"]
+ if stream.get("statusReason"):
+ resp["statusReason"] = stream["statusReason"]
+ resp["tags"] = _tags.get(stream["arn"], {})
+ return resp
+
+
+def _check_target_definition(target):
+ """ValidationException for a target the API's union shape can't hold."""
+ if not isinstance(target, dict) or len(target) != 1 or "kinesis" not in target:
+ return _validation(
+ "targetDefinition must set exactly one member: kinesis",
+ reason="fieldValidationFailed", field="targetDefinition",
+ )
+ kinesis_target = target["kinesis"]
+ if not isinstance(kinesis_target, dict):
+ return _validation(
+ "targetDefinition.kinesis must be a structure",
+ reason="fieldValidationFailed", field="targetDefinition.kinesis",
+ )
+ for member in ("streamArn", "roleArn"):
+ if not kinesis_target.get(member):
+ return _validation(
+ f"targetDefinition.kinesis.{member} is required",
+ reason="fieldValidationFailed",
+ field=f"targetDefinition.kinesis.{member}",
+ )
+ return None
+
+
+def _target_failure(target):
+ """The failure real DSQL reports through statusReason, or None.
+
+ Delivery is not emulated, so the target is judged once at create time:
+ a role that cannot be assumed (AUTH=true only, as everywhere else) and a
+ Kinesis stream that isn't there are the two cases MiniStack can see.
+ """
+ kinesis_target = target["kinesis"]
+ from ministack.core.iam_evaluator import validate_role_arn
+
+ if validate_role_arn(kinesis_target["roleArn"]):
+ return "ROLE_ACCESS_DENIED"
+ from ministack.services import kinesis
+
+ if kinesis._resolve_stream_by_arn(kinesis_target["streamArn"]) is None:
+ return "KINESIS_STREAM_NOT_FOUND"
+ return None
+
+
+def _create_stream(cluster_identifier, data):
+ cluster = _clusters.get(cluster_identifier)
+ if not cluster:
+ return _not_found(cluster_identifier)
+
+ client_token = data.get("clientToken")
+ if client_token:
+ existing_key = _stream_tokens.get(client_token)
+ if isinstance(existing_key, str) and existing_key in _streams:
+ return json_response(_stream_response(_streams[existing_key]))
+
+ err = _check_target_definition(data.get("targetDefinition"))
+ if err:
+ return err
+ ordering = data.get("ordering")
+ if ordering not in _STREAM_ORDERINGS:
+ return _validation(
+ f"ordering must be one of: {', '.join(_STREAM_ORDERINGS)}",
+ reason="fieldValidationFailed", field="ordering",
+ )
+ fmt = data.get("format")
+ if fmt not in _STREAM_FORMATS:
+ return _validation(
+ f"format must be one of: {', '.join(_STREAM_FORMATS)}",
+ reason="fieldValidationFailed", field="format",
+ )
+
+ existing = [
+ st for st in _streams.values()
+ if st["clusterIdentifier"] == cluster_identifier
+ ]
+ if len(existing) >= _MAX_STREAMS_PER_CLUSTER:
+ return _error(
+ "ServiceQuotaExceededException", "You have reached the stream limit.",
+ 402, resourceId=cluster_identifier, resourceType="stream",
+ serviceCode="dsql", quotaCode="StreamsPerCluster",
+ )
+
+ identifier = _new_identifier()
+ arn = _stream_arn(cluster_identifier, identifier)
+ failure = _target_failure(data["targetDefinition"])
+ stream = {
+ "clusterIdentifier": cluster_identifier,
+ "streamIdentifier": identifier,
+ "arn": arn,
+ # Nothing is delivered, so a stream that has a target to deliver to
+ # is ACTIVE straight away rather than going through CREATING.
+ "status": "FAILED" if failure else "ACTIVE",
+ "creationTime": int(time.time()),
+ "ordering": ordering,
+ "format": fmt,
+ "targetDefinition": copy.deepcopy(data["targetDefinition"]),
+ }
+ if failure:
+ stream["statusReason"] = {"error": failure, "updatedAt": stream["creationTime"]}
+
+ _streams[_stream_key(cluster_identifier, identifier)] = stream
+ if client_token:
+ _stream_tokens[client_token] = _stream_key(cluster_identifier, identifier)
+ if data.get("tags"):
+ _tags[arn] = dict(data["tags"])
+
+ return json_response(_stream_response(stream))
+
+
+def _get_stream(cluster_identifier, stream_identifier):
+ if cluster_identifier not in _clusters:
+ return _not_found(cluster_identifier)
+ stream = _streams.get(_stream_key(cluster_identifier, stream_identifier))
+ if not stream:
+ return _stream_not_found(cluster_identifier, stream_identifier)
+ return json_response(_stream_response(stream, include_target=True))
+
+
+def _list_streams(cluster_identifier, query_params):
+ if cluster_identifier not in _clusters:
+ return _not_found(cluster_identifier)
+
+ raw_max = _qp(query_params, "max-results", "maxResults")
+ next_token = _qp(query_params, "next-token", "nextToken")
+ try:
+ max_results = int(raw_max) if raw_max else 100
+ except (TypeError, ValueError):
+ return _validation(
+ f"Invalid maxResults: {raw_max}",
+ reason="fieldValidationFailed", field="maxResults",
+ )
+ if max_results < 1 or max_results > 100:
+ return _validation(
+ "maxResults must be between 1 and 100",
+ reason="fieldValidationFailed", field="maxResults",
+ )
+ start = 0
+ if next_token:
+ try:
+ start = int(next_token)
+ if start < 0:
+ raise ValueError
+ except (TypeError, ValueError):
+ return _validation(
+ f"Invalid nextToken: {next_token}",
+ reason="fieldValidationFailed", field="nextToken",
+ )
+
+ summaries = [
+ _stream_summary(st) for st in _streams.values()
+ if st["clusterIdentifier"] == cluster_identifier
+ ]
+ resp = {"streams": summaries[start : start + max_results]}
+ if start + max_results < len(summaries):
+ resp["nextToken"] = str(start + max_results)
+ return json_response(resp)
+
+
+def _delete_stream(cluster_identifier, stream_identifier, query_params):
+ if cluster_identifier not in _clusters:
+ return _not_found(cluster_identifier)
+ key = _stream_key(cluster_identifier, stream_identifier)
+ stream = _streams.get(key)
+ if not stream:
+ return _stream_not_found(cluster_identifier, stream_identifier)
+ stream["status"] = "DELETING"
+ resp = json_response(_stream_summary(stream))
+ _tags.pop(stream["arn"], None)
+ del _streams[key]
+ return resp
+
+
+def _get_vpc_endpoint_service_name(identifier):
+ cluster = _clusters.get(identifier)
+ if not cluster:
+ return _not_found(identifier)
+ # The service name's suffix is per-cluster and stable, as on AWS.
+ suffix = hashlib.sha256(identifier.encode()).hexdigest()[:6]
+ return json_response({
+ "serviceName": f"com.amazonaws.{get_region()}.dsql-{suffix}",
+ })
+
+
# ---------------------------------------------------------------------------
# Tags
# ---------------------------------------------------------------------------
def _tags_for_arn(arn):
- """Return tags for a known cluster ARN, or a 404 response."""
+ """Return tags for a known cluster or stream ARN, or a 404 response."""
identifier = _identifier_from_arn(arn)
+ if "/stream/" in identifier:
+ cluster_identifier, _, stream_identifier = identifier.partition("/stream/")
+ if _stream_key(cluster_identifier, stream_identifier) not in _streams:
+ return None, _not_found(arn, resource_type="stream")
+ return _tags.get(arn, {}), None
if not identifier or identifier not in _clusters:
return None, _not_found(arn, resource_type="cluster")
return _tags.get(arn, {}), None
@@ -790,6 +1033,38 @@ def _handle_request_sync(method, path, headers, body, query_params):
return _list_tags_for_resource(arn)
return _validation(f"Unknown method for /tags: {method}", reason="unknownOperation")
+ if path.startswith("/stream/"):
+ parts = path[len("/stream/") :].split("/")
+ if len(parts) == 1 and parts[0]:
+ if method == "POST":
+ return _create_stream(parts[0], data)
+ if method == "GET":
+ return _list_streams(parts[0], query_params)
+ return _validation(
+ f"Unknown method for /stream/{{id}}: {method}",
+ reason="unknownOperation",
+ )
+ if len(parts) == 2 and all(parts):
+ if method == "GET":
+ return _get_stream(parts[0], parts[1])
+ if method == "DELETE":
+ return _delete_stream(parts[0], parts[1], query_params)
+ return _validation(
+ f"Unknown method for /stream/{{id}}/{{streamId}}: {method}",
+ reason="unknownOperation",
+ )
+ return _validation(f"Unknown path: {method} {path}", reason="unknownOperation")
+
+ if path.startswith("/clusters/") and path.endswith(
+ "/vpc-endpoint-service-name"
+ ):
+ identifier = path[len("/clusters/") : -len("/vpc-endpoint-service-name")]
+ if method == "GET":
+ return _get_vpc_endpoint_service_name(identifier)
+ return _validation(
+ f"Unknown method for {path}: {method}", reason="unknownOperation"
+ )
+
if path == "/cluster":
if method == "POST":
return _create_cluster(data)
@@ -843,6 +1118,8 @@ def get_state():
"clusters": clusters,
"client_tokens": copy.deepcopy(dict(_client_tokens._data)),
"tags": copy.deepcopy(dict(_tags._data)),
+ "streams": copy.deepcopy(dict(_streams._data)),
+ "stream_tokens": copy.deepcopy(dict(_stream_tokens._data)),
"port_counter": _port_counter[0],
}
@@ -859,6 +1136,12 @@ def restore_state(data):
tags = data.get("tags", {})
if tags:
_tags._data.update(tags)
+ streams = data.get("streams", {})
+ if streams:
+ _streams._data.update(streams)
+ stream_tokens = data.get("stream_tokens", {})
+ if stream_tokens:
+ _stream_tokens._data.update(stream_tokens)
saved_counter = data.get("port_counter")
if isinstance(saved_counter, int) and saved_counter > _port_counter[0]:
_port_counter[0] = saved_counter
@@ -924,6 +1207,8 @@ def reset():
_clusters.clear()
_client_tokens.clear()
_tags.clear()
+ _streams.clear()
+ _stream_tokens.clear()
_port_counter[0] = BASE_PORT
diff --git a/tests/test_dsql.py b/tests/test_dsql.py
index 3cad769b..b2f96f7e 100644
--- a/tests/test_dsql.py
+++ b/tests/test_dsql.py
@@ -413,6 +413,198 @@ class TestClusterPolicy:
_cleanup(dsql, identifier)
+
+
+class TestStreams:
+ """CDC streams (GA 2026-07-08): metadata only — nothing is delivered."""
+
+ ROLE_ARN = "arn:aws:iam::000000000000:role/dsql-cdc"
+
+ def _target(self, kin, name):
+ try:
+ kin.create_stream(StreamName=name, ShardCount=1)
+ except ClientError as e:
+ if e.response["Error"]["Code"] != "ResourceInUseException":
+ raise
+ arn = kin.describe_stream(StreamName=name)["StreamDescription"]["StreamARN"]
+ return {"kinesis": {"streamArn": arn, "roleArn": self.ROLE_ARN}}
+
+ def test_stream_lifecycle(self, dsql, kin):
+ identifier = _create(dsql)
+ try:
+ target = self._target(kin, "dsql-cdc-life")
+ created = dsql.create_stream(
+ clusterIdentifier=identifier,
+ targetDefinition=target,
+ ordering="UNORDERED",
+ format="JSON",
+ tags={"env": "test"},
+ )
+ stream_id = created["streamIdentifier"]
+ assert ID_RE.match(stream_id)
+ assert created["status"] == "ACTIVE"
+ assert created["arn"] == (
+ f"arn:aws:dsql:us-east-1:000000000000:cluster/{identifier}"
+ f"/stream/{stream_id}"
+ )
+
+ got = dsql.get_stream(
+ clusterIdentifier=identifier, streamIdentifier=stream_id
+ )
+ assert got["targetDefinition"] == target
+ assert got["ordering"] == "UNORDERED"
+ assert got["format"] == "JSON"
+ assert got["tags"] == {"env": "test"}
+ assert "statusReason" not in got
+
+ listed = dsql.list_streams(clusterIdentifier=identifier)["streams"]
+ assert [st["streamIdentifier"] for st in listed] == [stream_id]
+
+ deleted = dsql.delete_stream(
+ clusterIdentifier=identifier, streamIdentifier=stream_id
+ )
+ assert deleted["status"] == "DELETING"
+ with pytest.raises(ClientError) as exc:
+ dsql.get_stream(
+ clusterIdentifier=identifier, streamIdentifier=stream_id
+ )
+ assert exc.value.response["Error"]["Code"] == "ResourceNotFoundException"
+ finally:
+ _cleanup(dsql, identifier)
+
+ def test_stream_tags_and_client_token(self, dsql, kin):
+ identifier = _create(dsql)
+ try:
+ target = self._target(kin, "dsql-cdc-tags")
+ created = dsql.create_stream(
+ clusterIdentifier=identifier,
+ targetDefinition=target,
+ ordering="UNORDERED",
+ format="JSON",
+ clientToken="tok-cdc",
+ )
+ repeat = dsql.create_stream(
+ clusterIdentifier=identifier,
+ targetDefinition=target,
+ ordering="UNORDERED",
+ format="JSON",
+ clientToken="tok-cdc",
+ )
+ assert repeat["streamIdentifier"] == created["streamIdentifier"]
+
+ dsql.tag_resource(resourceArn=created["arn"], tags={"team": "data"})
+ assert dsql.list_tags_for_resource(resourceArn=created["arn"])["tags"] == {
+ "team": "data"
+ }
+ dsql.untag_resource(resourceArn=created["arn"], tagKeys=["team"])
+ assert dsql.list_tags_for_resource(resourceArn=created["arn"])["tags"] == {}
+ finally:
+ _cleanup(dsql, identifier)
+
+ def test_missing_kinesis_target_lands_failed(self, dsql):
+ identifier = _create(dsql)
+ try:
+ created = dsql.create_stream(
+ clusterIdentifier=identifier,
+ targetDefinition={"kinesis": {
+ "streamArn": "arn:aws:kinesis:us-east-1:000000000000:stream/absent",
+ "roleArn": self.ROLE_ARN,
+ }},
+ ordering="UNORDERED",
+ format="JSON",
+ )
+ assert created["status"] == "FAILED"
+ got = dsql.get_stream(
+ clusterIdentifier=identifier,
+ streamIdentifier=created["streamIdentifier"],
+ )
+ assert got["statusReason"]["error"] == "KINESIS_STREAM_NOT_FOUND"
+ finally:
+ _cleanup(dsql, identifier)
+
+ def test_stream_limit_per_cluster(self, dsql, kin):
+ identifier = _create(dsql)
+ try:
+ target = self._target(kin, "dsql-cdc-limit")
+ for _ in range(5):
+ dsql.create_stream(
+ clusterIdentifier=identifier,
+ targetDefinition=target,
+ ordering="UNORDERED",
+ format="JSON",
+ )
+ with pytest.raises(ClientError) as exc:
+ dsql.create_stream(
+ clusterIdentifier=identifier,
+ targetDefinition=target,
+ ordering="UNORDERED",
+ format="JSON",
+ )
+ assert exc.value.response["Error"]["Code"] == "ServiceQuotaExceededException"
+ assert "stream limit" in exc.value.response["Error"]["Message"]
+ finally:
+ _cleanup(dsql, identifier)
+
+ def test_stream_on_unknown_cluster(self, dsql):
+ unknown = "z" * 26
+ with pytest.raises(ClientError) as exc:
+ dsql.list_streams(clusterIdentifier=unknown)
+ assert exc.value.response["Error"]["Code"] == "ResourceNotFoundException"
+
+ def test_deleting_a_cluster_takes_its_streams(self, dsql, kin):
+ identifier = _create(dsql)
+ target = self._target(kin, "dsql-cdc-gone")
+ created = dsql.create_stream(
+ clusterIdentifier=identifier,
+ targetDefinition=target,
+ ordering="UNORDERED",
+ format="JSON",
+ )
+ _cleanup(dsql, identifier)
+ with pytest.raises(ClientError) as exc:
+ dsql.get_stream(
+ clusterIdentifier=identifier,
+ streamIdentifier=created["streamIdentifier"],
+ )
+ assert exc.value.response["Error"]["Code"] == "ResourceNotFoundException"
+
+ def test_stream_active_waiter(self, dsql, kin):
+ identifier = _create(dsql)
+ try:
+ target = self._target(kin, "dsql-cdc-waiter")
+ created = dsql.create_stream(
+ clusterIdentifier=identifier,
+ targetDefinition=target,
+ ordering="UNORDERED",
+ format="JSON",
+ )
+ dsql.get_waiter("stream_active").wait(
+ clusterIdentifier=identifier,
+ streamIdentifier=created["streamIdentifier"],
+ WaiterConfig={"Delay": 1, "MaxAttempts": 3},
+ )
+ finally:
+ _cleanup(dsql, identifier)
+
+
+class TestVpcEndpointServiceName:
+ def test_service_name_is_stable_per_cluster(self, dsql):
+ identifier = _create(dsql)
+ try:
+ first = dsql.get_vpc_endpoint_service_name(identifier=identifier)
+ assert re.match(
+ r"^com\.amazonaws\.us-east-1\.dsql-[a-f0-9]{6}$", first["serviceName"]
+ )
+ second = dsql.get_vpc_endpoint_service_name(identifier=identifier)
+ assert second["serviceName"] == first["serviceName"]
+ finally:
+ _cleanup(dsql, identifier)
+
+ def test_unknown_cluster(self, dsql):
+ with pytest.raises(ClientError) as exc:
+ dsql.get_vpc_endpoint_service_name(identifier="z" * 26)
+ assert exc.value.response["Error"]["Code"] == "ResourceNotFoundException"
+
class TestBotoParity:
"""Waiters, paginators, and structured error members (botocore model)."""
@@ -659,30 +851,32 @@ class TestDenylist:
assert "is not supported" in err.message
@pytest.mark.parametrize(
- "sql",
- [
- "CREATE TABLE t (a int, b int REFERENCES other (id))",
- "CREATE TABLE t (a int, FOREIGN KEY (a) REFERENCES other (id))",
- "ALTER TABLE t ADD CONSTRAINT fk FOREIGN KEY (a) REFERENCES o (id)",
- ],
- )
- def test_foreign_keys_denied(self, sql):
- err = pgproxy.validate(sql, pgproxy.TxnState())
- assert isinstance(err, pgproxy.DsqlError), sql
- assert err.sqlstate == "0A000"
- assert "foreign key" in err.message
-
- @pytest.mark.parametrize(
- "sql",
+ "sql,message",
[
- "ALTER TABLE t ADD COLUMN c int DEFAULT 0",
- "ALTER TABLE t ADD COLUMN c text NOT NULL",
+ ("ALTER TABLE t ADD COLUMN c int DEFAULT 0",
+ "ALTER TABLE ADD COLUMN with constraint not supported"),
+ ("ALTER TABLE t ADD COLUMN c text NOT NULL",
+ "ALTER TABLE ADD COLUMN with constraint not supported"),
+ ("ALTER TABLE t ADD COLUMN c int UNIQUE",
+ "ALTER TABLE ADD COLUMN with constraint not supported"),
+ ("ALTER TABLE t ADD COLUMN c int CHECK (c > 0)",
+ "ALTER TABLE ADD COLUMN with constraint not supported"),
+ ("ALTER TABLE t ADD COLUMN c int REFERENCES o",
+ "ALTER TABLE ADD COLUMN with constraint not supported"),
+ ("ALTER TABLE t ADD COLUMN c text GENERATED ALWAYS AS (lower(e)) STORED",
+ "ALTER TABLE ADD COLUMN with constraint not supported"),
+ ("ALTER TABLE t ADD COLUMN c bigint GENERATED BY DEFAULT AS IDENTITY "
+ "(CACHE 1)",
+ "ALTER TABLE ADD COLUMN with constraint not supported"),
+ ('ALTER TABLE t ADD COLUMN c text COLLATE "C"',
+ "COLLATE clause not supported"),
],
)
- def test_add_column_default_not_null_denied(self, sql):
+ def test_add_column_takes_no_constraint(self, sql, message):
err = pgproxy.validate(sql, pgproxy.TxnState())
assert isinstance(err, pgproxy.DsqlError), sql
assert err.sqlstate == "0A000"
+ assert err.message == message, sql
def test_create_function_language_sql_allowed(self):
sql = "CREATE FUNCTION f() RETURNS int LANGUAGE sql AS $$ SELECT 1 $$"
@@ -798,8 +992,9 @@ class TestAlterTableSubset:
"ALTER TABLE t ALTER COLUMN c ADD GENERATED BY DEFAULT AS IDENTITY (CACHE 1)",
"ALTER TABLE t ADD COLUMN c integer",
"ALTER TABLE t ADD COLUMN j jsonb STORAGE PLAIN",
- "ALTER TABLE t ADD COLUMN c bigint GENERATED BY DEFAULT AS IDENTITY (CACHE 1)",
- "ALTER TABLE t ADD COLUMN cl text GENERATED ALWAYS AS (lower(email)) STORED",
+ "ALTER TABLE t ADD COLUMN c int NULL",
+ "ALTER TABLE t ADD COLUMN IF NOT EXISTS c numeric(18,6)",
+ "ALTER TABLE t ADD COLUMN a int, ADD COLUMN b text",
],
)
def test_supported_actions_allowed(self, sql):
@@ -842,6 +1037,124 @@ class TestAlterTableSubset:
assert result.object_name == "public.t"
+
+
+class TestForeignKeys:
+ """Foreign keys, supported since 2026-08-26 (AWS release notes)."""
+
+ @pytest.mark.parametrize(
+ "sql",
+ [
+ "CREATE TABLE orders (id int PRIMARY KEY, p int REFERENCES products)",
+ "CREATE TABLE orders (id int PRIMARY KEY, "
+ "p int REFERENCES products (product_no))",
+ "CREATE TABLE orders (id int PRIMARY KEY, "
+ "p int CONSTRAINT fk_product REFERENCES products)",
+ "CREATE TABLE shipments (id int PRIMARY KEY, w int, p int, "
+ "FOREIGN KEY (w, p) REFERENCES inventory (warehouse_id, product_no))",
+ # Self-referencing.
+ "CREATE TABLE tree (id int PRIMARY KEY, parent int REFERENCES tree)",
+ # Every referential action DSQL supports.
+ "CREATE TABLE o (p int REFERENCES products ON DELETE NO ACTION)",
+ "CREATE TABLE o (p int REFERENCES products ON DELETE RESTRICT)",
+ "CREATE TABLE o (p int REFERENCES products ON DELETE CASCADE)",
+ "CREATE TABLE o (p int REFERENCES products ON DELETE SET NULL)",
+ "CREATE TABLE o (p int REFERENCES products ON DELETE SET DEFAULT)",
+ "CREATE TABLE o (p int REFERENCES products "
+ "ON DELETE CASCADE ON UPDATE CASCADE)",
+ # Match types and deferrability.
+ "CREATE TABLE s (w int, p int, FOREIGN KEY (w, p) "
+ "REFERENCES inventory (w, p) MATCH FULL)",
+ "CREATE TABLE s (w int, p int, FOREIGN KEY (w, p) "
+ "REFERENCES inventory (w, p) MATCH SIMPLE)",
+ "CREATE TABLE o (p int REFERENCES products DEFERRABLE)",
+ "CREATE TABLE o (p int REFERENCES products "
+ "DEFERRABLE INITIALLY DEFERRED)",
+ "CREATE TABLE o (p int REFERENCES products NOT DEFERRABLE)",
+ "ALTER TABLE o ALTER CONSTRAINT fk DEFERRABLE INITIALLY DEFERRED",
+ "ALTER TABLE o ADD CONSTRAINT fk FOREIGN KEY (p) "
+ "REFERENCES products NOT VALID",
+ "ALTER TABLE o ADD FOREIGN KEY (p) REFERENCES products "
+ "ON DELETE CASCADE NOT VALID",
+ "ALTER TABLE o ADD CONSTRAINT fk FOREIGN KEY (p) "
+ "REFERENCES products DEFERRABLE INITIALLY DEFERRED NOT VALID",
+ "ALTER TABLE o DROP CONSTRAINT fk",
+ "SET CONSTRAINTS ALL DEFERRED",
+ ],
+ )
+ def test_allowed(self, sql):
+ assert pgproxy.validate(sql, pgproxy.TxnState()) is None, sql
+
+ @pytest.mark.parametrize(
+ "sql",
+ [
+ "ALTER TABLE o ADD CONSTRAINT fk FOREIGN KEY (p) REFERENCES products",
+ "ALTER TABLE o ADD FOREIGN KEY (p) REFERENCES products ON DELETE CASCADE",
+ ],
+ )
+ def test_alter_table_requires_not_valid(self, sql):
+ err = pgproxy.validate(sql, pgproxy.TxnState())
+ assert isinstance(err, pgproxy.DsqlError), sql
+ assert err.sqlstate == "0A000"
+ assert err.message == "unsupported ALTER TABLE ADD CONSTRAINT statement"
+
+ @pytest.mark.parametrize(
+ "sql,message",
+ [
+ ("CREATE TABLE t (a int, UNIQUE (a) DEFERRABLE)",
+ "DEFERRABLE constraint not supported"),
+ ("CREATE TABLE t (a int PRIMARY KEY DEFERRABLE INITIALLY DEFERRED)",
+ "DEFERRABLE constraint not supported"),
+ # INITIALLY DEFERRED without DEFERRABLE is refused the same way.
+ ("CREATE TABLE t (a int, UNIQUE (a) INITIALLY DEFERRED)",
+ "DEFERRABLE constraint not supported"),
+ # ADD CONSTRAINT ... UNIQUE has its own wording again.
+ ("ALTER TABLE t ADD CONSTRAINT u UNIQUE USING INDEX idx DEFERRABLE",
+ "ALTER TABLE / ADD CONSTRAINT UNIQUE / DEFERRED / "
+ "INITIALLY DEFERRED not supported"),
+ ("ALTER TABLE t ADD CONSTRAINT u UNIQUE USING INDEX idx "
+ "INITIALLY DEFERRED",
+ "ALTER TABLE / ADD CONSTRAINT UNIQUE / DEFERRED / "
+ "INITIALLY DEFERRED not supported"),
+ ("ALTER TABLE t ADD CONSTRAINT c CHECK (a > 0) DEFERRABLE NOT VALID",
+ "CHECK constraints cannot be marked DEFERRABLE"),
+ # A CHECK constraint has its own wording.
+ ("CREATE TABLE t (a int, CONSTRAINT c CHECK (a > 0) "
+ "DEFERRABLE INITIALLY IMMEDIATE)",
+ "CHECK constraints cannot be marked DEFERRABLE"),
+ ],
+ )
+ def test_deferrable_on_a_non_foreign_key_denied(self, sql, message):
+ err = pgproxy.validate(sql, pgproxy.TxnState())
+ assert isinstance(err, pgproxy.DsqlError), sql
+ assert err.sqlstate == "0A000"
+ assert err.message == message
+
+ @pytest.mark.parametrize(
+ "sql",
+ [
+ # NOT DEFERRABLE is the default, so it is accepted anywhere.
+ "CREATE TABLE t (a int, UNIQUE (a) NOT DEFERRABLE)",
+ "CREATE TABLE t (a int PRIMARY KEY NOT DEFERRABLE)",
+ "CREATE TABLE t (a int, CHECK (a > 0) NOT DEFERRABLE)",
+ # A column carrying both constraints defers the foreign key.
+ "CREATE TABLE t (p int UNIQUE REFERENCES o DEFERRABLE)",
+ "CREATE TABLE t (a int UNIQUE, p int REFERENCES o "
+ "DEFERRABLE INITIALLY DEFERRED)",
+ # INITIALLY IMMEDIATE declares no deferral either.
+ "ALTER TABLE t ADD CONSTRAINT u UNIQUE USING INDEX idx "
+ "INITIALLY IMMEDIATE",
+ "ALTER TABLE t ADD CONSTRAINT u UNIQUE USING INDEX idx NOT DEFERRABLE",
+ "ALTER TABLE t ADD CONSTRAINT c CHECK (a > 0) NOT DEFERRABLE NOT VALID",
+ "ALTER TABLE t ADD CONSTRAINT f FOREIGN KEY (p) REFERENCES o "
+ "NOT VALID DEFERRABLE",
+ # The word as data, not as a constraint option.
+ "CREATE TABLE t (a int, s text, CHECK (s <> 'DEFERRABLE'))",
+ ],
+ )
+ def test_deferrable_accepted(self, sql):
+ assert pgproxy.validate(sql, pgproxy.TxnState()) is None, sql
+
class TestIdentityAndSequences:
@pytest.mark.parametrize(
"sql",
@@ -1247,8 +1560,8 @@ class TestLockingClauses:
not the predicate. Measured against a live cluster (eu-central-1,
2026-08-18, server_version 16.15): FOR UPDATE locks whatever the query
selects — no WHERE clause, a non-key column, an inequality, IN/OR, a join,
- a table with no primary key — while FOR NO KEY UPDATE, FOR SHARE and
- FOR KEY SHARE are all refused with 0A000."""
+ a table with no primary key — while FOR NO KEY UPDATE and FOR SHARE are
+ refused with 0A000. FOR KEY SHARE joined FOR UPDATE on 2026-08-25."""
MESSAGE = "locking clauses other than FOR UPDATE/FOR KEY SHARE are not supported"
@@ -1269,6 +1582,11 @@ class TestLockingClauses:
"SELECT s FROM t WHERE id = 1 FOR UPDATE SKIP LOCKED",
"SELECT s FROM t WHERE id = 1 FOR UPDATE OF t NOWAIT",
"SELECT * FROM (SELECT 1 FOR UPDATE) x FOR UPDATE",
+ # FOR KEY SHARE, supported since 2026-08-25.
+ "SELECT s FROM t WHERE id = 1 FOR KEY SHARE",
+ "SELECT s FROM t FOR KEY SHARE NOWAIT",
+ "SELECT * FROM (SELECT s FROM t FOR KEY SHARE) y",
+ "WITH x AS (SELECT s FROM t FOR KEY SHARE) SELECT * FROM x",
# The words only as text — not a locking clause at all.
"SELECT s FROM t WHERE s = 'FOR SHARE'",
"SELECT 'FOR NO KEY UPDATE' AS lit",
@@ -1284,13 +1602,12 @@ class TestLockingClauses:
"sql",
[
"SELECT s FROM t WHERE id = 1 FOR SHARE",
- "SELECT s FROM t WHERE id = 1 FOR KEY SHARE",
"SELECT s FROM t WHERE id = 1 FOR NO KEY UPDATE",
"SELECT s FROM t for share",
"SELECT s FROM t FOR SHARE NOWAIT",
# Any nesting depth: DSQL refuses it inside a CTE or a subquery.
"WITH x AS (SELECT s FROM t WHERE id = 1 FOR SHARE) SELECT * FROM x",
- "SELECT * FROM (SELECT s FROM t FOR KEY SHARE) y",
+ "SELECT * FROM (SELECT s FROM t FOR SHARE) y",
# A supported clause earlier in the statement must not hide one
# that still reaches a relation (both verified rejected by DSQL).
"SELECT x.id FROM (SELECT id FROM t FOR UPDATE) x, "
@@ -1596,8 +1913,8 @@ class TestExtendedProtocol:
"sql,expected",
[
("CREATE TABLE xp_serial (id serial)", "serial"),
- ("CREATE TABLE xp_fk (id int, p int REFERENCES xp_serial (id))",
- "foreign key"),
+ ("CREATE TABLE xp_uq (id int, UNIQUE (id) DEFERRABLE)",
+ "DEFERRABLE"),
("CREATE EXTENSION pgcrypto", "CREATE EXTENSION"),
("CREATE TEMP TABLE xp_tmp (id int)", "temporary"),
("TRUNCATE xp_base", "TRUNCATE"),
@@ -1778,7 +2095,7 @@ class TestLockingReads:
"""A locking read has to reach the backend whatever its predicate looks
like — an ORM quotes every identifier, and DSQL itself places no
restriction on the predicate (measured, eu-central-1, 2026-08-18). What it
- does refuse is a lock strength other than FOR UPDATE."""
+ does refuse is a lock strength other than FOR UPDATE/FOR KEY SHARE."""
UUID = "11111111-1111-1111-1111-111111111111"
MESSAGE = "locking clauses other than FOR UPDATE/FOR KEY SHARE are not supported"
@@ -1901,9 +2218,7 @@ class TestLockingReads:
result = c.simple(sql)
assert result.ok, f"{sql} -> {result.sqlstate} {result.message}"
- @pytest.mark.parametrize(
- "clause", ["FOR SHARE", "FOR KEY SHARE", "FOR NO KEY UPDATE"]
- )
+ @pytest.mark.parametrize("clause", ["FOR SHARE", "FOR NO KEY UPDATE"])
def test_other_lock_strengths_rejected(self, dsql_proxy, clause):
with _WireClient(dsql_proxy) as c:
self._setup(c, "fu_l")
@@ -1913,6 +2228,15 @@ class TestLockingReads:
assert result.sqlstate == "0A000", sql
assert result.message == self.MESSAGE, sql
+ def test_key_share_reaches_the_backend(self, dsql_proxy):
+ with _WireClient(dsql_proxy) as c:
+ self._setup(c, "fu_ks")
+ sql = f"SELECT s FROM fu_ks WHERE id = '{self.UUID}' FOR KEY SHARE"
+ for send in (c.simple, c.extended):
+ result = send(sql)
+ assert result.ok, f"{sql} -> {result.sqlstate} {result.message}"
+ assert result.rows == [("a",)], sql
+
def test_lock_strength_in_a_literal_is_not_a_clause(self, dsql_proxy):
with _WireClient(dsql_proxy) as c:
self._setup(c, "fu_lit")
@@ -2057,17 +2381,64 @@ class TestLiveProxy:
finally:
conn.close()
- def test_foreign_key_rejected(self, dsql_proxy):
+ def test_foreign_key_enforced(self, dsql_proxy):
+ """Foreign keys, supported since 2026-08-26, reach the backend and
+ enforce referential integrity."""
psycopg2 = pytest.importorskip("psycopg2")
conn = _pg_connect(dsql_proxy)
try:
cur = conn.cursor()
+ cur.execute("CREATE TABLE fk_parent (id int PRIMARY KEY, name text)")
+ cur.execute(
+ "CREATE TABLE fk_child (id int PRIMARY KEY, "
+ "p int REFERENCES fk_parent ON DELETE CASCADE)"
+ )
+ cur.execute("INSERT INTO fk_parent VALUES (1, 'a')")
+ cur.execute("INSERT INTO fk_child VALUES (10, 1)")
+ with pytest.raises(psycopg2.Error) as exc:
+ cur.execute("INSERT INTO fk_child VALUES (11, 99)")
+ assert exc.value.pgcode == "23503"
+ # ON DELETE CASCADE removes the referencing row with the parent.
+ cur.execute("DELETE FROM fk_parent WHERE id = 1")
+ cur.execute("SELECT count(*) FROM fk_child")
+ assert cur.fetchone()[0] == 0
+ cur.execute("DROP TABLE fk_child")
+ cur.execute("DROP TABLE fk_parent")
+ finally:
+ conn.close()
+
+ def test_foreign_key_via_alter_table_requires_not_valid(self, dsql_proxy):
+ psycopg2 = pytest.importorskip("psycopg2")
+ conn = _pg_connect(dsql_proxy)
+ try:
+ cur = conn.cursor()
+ cur.execute("CREATE TABLE fka_parent (id int PRIMARY KEY)")
+ cur.execute("CREATE TABLE fka_child (id int PRIMARY KEY, p int)")
+ cur.execute("INSERT INTO fka_parent VALUES (1)")
+ cur.execute("INSERT INTO fka_child VALUES (10, 1)")
with pytest.raises(psycopg2.Error) as exc:
cur.execute(
- "CREATE TABLE fk_live (a int REFERENCES other_table (id))"
+ "ALTER TABLE fka_child ADD CONSTRAINT fk "
+ "FOREIGN KEY (p) REFERENCES fka_parent"
)
assert exc.value.pgcode == "0A000"
- assert "foreign key" in str(exc.value)
+ assert "unsupported ALTER TABLE ADD CONSTRAINT" in str(exc.value)
+ cur.execute(
+ "ALTER TABLE fka_child ADD CONSTRAINT fk "
+ "FOREIGN KEY (p) REFERENCES fka_parent NOT VALID"
+ )
+ # Async validation of a NOT VALID foreign key registers a job.
+ cur.execute("ALTER TABLE ASYNC fka_child VALIDATE CONSTRAINT fk")
+ job_id = cur.fetchone()[0]
+ cur.execute(
+ "SELECT job_type, status FROM sys.jobs WHERE job_id = %s" % repr(job_id)
+ )
+ assert cur.fetchone() == ("VALIDATE_CONSTRAINT", "completed")
+ with pytest.raises(psycopg2.Error) as exc:
+ cur.execute("INSERT INTO fka_child VALUES (11, 99)")
+ assert exc.value.pgcode == "23503"
+ cur.execute("DROP TABLE fka_child")
+ cur.execute("DROP TABLE fka_parent")
finally:
conn.close()
@@ -2141,7 +2512,7 @@ class TestLiveProxy:
"ALTER TABLE chk_live ADD CONSTRAINT chk_a CHECK (a > 0)"
)
assert exc.value.pgcode == "0A000"
- assert "NOT VALID" in str(exc.value)
+ assert "unsupported ALTER TABLE ADD CONSTRAINT" in str(exc.value)
cur.execute(
"ALTER TABLE chk_live ADD CONSTRAINT chk_a CHECK (a > 0) NOT VALID"
)
@@ -2159,6 +2530,41 @@ class TestLiveProxy:
finally:
conn.close()
+ def test_validate_constraint_over_violating_rows_fails_the_job(self, dsql_proxy):
+ """Existing data that breaks the constraint fails the job, not the
+ submit: DSQL answers with a job_id and reports it through sys.jobs
+ (measured, eu-central-1 2026-08-30)."""
+ psycopg2 = pytest.importorskip("psycopg2")
+ conn = _pg_connect(dsql_proxy)
+ try:
+ cur = conn.cursor()
+ cur.execute("CREATE TABLE vj_parent (id int PRIMARY KEY)")
+ cur.execute("CREATE TABLE vj_child (id int PRIMARY KEY, p int)")
+ cur.execute("INSERT INTO vj_parent VALUES (1)")
+ cur.execute("INSERT INTO vj_child VALUES (1, 999)")
+ cur.execute(
+ "ALTER TABLE vj_child ADD CONSTRAINT vj_fk FOREIGN KEY (p) "
+ "REFERENCES vj_parent NOT VALID"
+ )
+ cur.execute("ALTER TABLE ASYNC vj_child VALIDATE CONSTRAINT vj_fk")
+ job_id = cur.fetchone()[0]
+ cur.execute(
+ "SELECT status, details FROM sys.jobs WHERE job_id = %s" % repr(job_id)
+ )
+ status, details = cur.fetchone()
+ assert status == "failed"
+ assert "violates foreign key constraint" in details
+ cur.execute(f"SELECT sys.wait_for_job('{job_id}')")
+ assert cur.fetchone()[0] is False
+ # A NOT VALID constraint still holds against new rows.
+ with pytest.raises(psycopg2.Error) as exc:
+ cur.execute("INSERT INTO vj_child VALUES (2, 998)")
+ assert exc.value.pgcode == "23503"
+ cur.execute("DROP TABLE vj_child")
+ cur.execute("DROP TABLE vj_parent")
+ finally:
+ conn.close()
+
def test_max_24_indexes_per_table(self, dsql_proxy):
psycopg2 = pytest.importorskip("psycopg2")
conn = _pg_connect(dsql_proxy)
@@ -2213,10 +2619,11 @@ class TestLiveProxy:
cur.execute(sql)
cur.fetchall()
# Any other lock strength is not supported.
+ cur.execute("SELECT * FROM fu_live WHERE id = 1 FOR KEY SHARE")
+ cur.fetchall()
for sql in (
"SELECT * FROM fu_live WHERE id = 1 FOR SHARE",
"SELECT * FROM fu_live WHERE id = 1 FOR NO KEY UPDATE",
- "SELECT * FROM fu_live WHERE id = 1 FOR KEY SHARE",
):
with pytest.raises(psycopg2.Error) as exc:
cur.execute(sql)