Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
release v1.5.28 (#2225)
Nahuel Nucerac6eedf298651
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 56a36381..4da11f9d 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,8 @@ Versioning follows [Semantic Versioning](https://semver.org/).## [Unreleased]+## [1.5.28] — 2026-10-16+### Added- **RDS — `DescribeEvents` with reboot events** — `DescribeEvents` returns the `availability` event `DB instance restarted.` once a `RebootDBInstance` has finished, filtered by source, source type, category and time, paged with `Marker` and kept for 14 days; other RDS lifecycle events are not recorded yet. Contributed by @jayjanssen.- **S3 Control — Multi-Region Access Points** — `CreateMultiRegionAccessPoint`, `DeleteMultiRegionAccessPoint`, `PutMultiRegionAccessPointPolicy`, `DescribeMultiRegionAccessPointOperation`, `GetMultiRegionAccessPoint`, `GetMultiRegionAccessPointPolicy`, `ListMultiRegionAccessPoints`, `GetMultiRegionAccessPointRoutes` and `SubmitMultiRegionAccessPointRoutes`. The async operations apply at once and `Describe` reports `SUCCEEDED`, or `FAILED` with AWS's `ErrorDetails` for a missing bucket, two buckets in one region or a name already in use. Management is served from `us-west-2` and route operations from `us-east-1`, `us-west-2`, `eu-west-1`, `ap-northeast-1` and `ap-southeast-2`, as on AWS; other regions get `301 PermanentRedirect`. `DeleteBucket` on a member returns `BucketHasAccessPointsAttached`. Under `AUTH=true` each operation needs its `s3:` IAM action and `x-amz-account-id` must name the caller's account. Access points from `AWS::S3::MultiRegionAccessPoint` appear in the API. Contributed by @fabio-andre-rodrigues.@@ -78,7 +80,8 @@ Versioning follows [Semantic Versioning](https://semver.org/).- **IAM — caller and transport condition keys under `AUTH=true`** — identity policies saw none of the caller's keys and a fixed `127.0.0.1` over plain HTTP; they now see `aws:username`, `aws:userid`, `aws:PrincipalTag/*` (the user's or role's tags, `AssumeRole` session tags taking precedence; in `rds-db:connect` too), for temporary credentials `aws:MultiFactorAuthPresent` and `aws:TokenIssueTime`, the client's address as `aws:SourceIp` and `aws:SecureTransport` as `true` over HTTPS or behind a proxy that sends `X-Forwarded-Proto: https`, and `AssumedRoleId` starts with the role's `RoleId` with or without `AUTH`. Contributed by @iot-rocket.- **S3 — requests without credentials are anonymous under `AUTH=true`** — an unsigned request (no `Authorization` header, presigned query or POST form credential) ran as the root key, so a private bucket in the caller's own account answered 200 to anyone; it is now anonymous, and only the owner's public grants let it in: an `AllUsers` ACL grant or a bucket policy that allows `*`, unless Public Access Block shuts them off. Anything else answers 403 `AccessDenied`. A `public-read` object ACL is now also found for an unversioned object, which the grant check missed for cross-account callers too. Without `AUTH` nothing changes. Contributed by @prandogabriel.- **Cognito — `InboundFederation` `userName`** — the trigger received `<provider>_<NameID>` for SAML and `<provider>_<sub>` for OIDC; it now receives the bare SAML `NameID` or OIDC `sub`. Contributed by @kjdev.-- **S3 Tables — `CreateTable` keeps Iceberg column types** — a `schema` or `schemaV2` field type other than `string`, `int`, `long`, `boolean`, `date`, `timestamp`, `float` or `double` was stored as `string`, and `timestamp` as `timestamptz`; types such as `decimal(19, 2)`, `uuid`, `fixed[16]` and nested `list`, `struct` and `map` are now stored as given. Reported by @anatollius.+- **S3 Tables — `CreateTable` keeps Iceberg column types** — a `schema` or `schemaV2` field type other than `string`, `int`, `long`, `boolean`, `date`, `timestamp`, `float` or `double` was stored as `string`, and `timestamp` as `timestamptz`; every Iceberg v2 type, nested `list`, `struct` and `map` included, is now stored as AWS stores it (lowercase, `decimal(P, S)`), and any other type answers `BadRequestException` `The specified metadata is not valid`. Reported by @anatollius.+- **S3 Tables — the Iceberg REST catalog takes the table bucket prefix** — a request without the `{prefix}` searched every table bucket, so two buckets with the same namespace and table name answered with the first one; such a request now answers 404 `<UnknownOperationException/>`, `GET /v1/config` without a table bucket ARN `warehouse` answers 400 `The specified table bucket ARN is not valid.`, and the url-encoded ARN comes back as `prefix` in `defaults`.## [1.5.27] — 2026-10-13diff --git a/ministack/services/cloudformation/provisioners.py b/ministack/services/cloudformation/provisioners.pyindex 19a0b85a..cad6328f 100644--- a/ministack/services/cloudformation/provisioners.py+++ b/ministack/services/cloudformation/provisioners.py@@ -12342,10 +12342,14 @@ def _s3tables_table_create(logical_id, props, stack_name):schema_field_list = (props.get("IcebergMetadata", {}).get("IcebergSchema", {}).get("SchemaFieldList", []))- schema_fields = [- {"name": f["Name"], "type": f.get("Type", "string"), "required": f.get("Required", False)}- for f in schema_field_list- ]+ try:+ schema_fields = [+ {"name": f["Name"], "type": _s3tables._iceberg_field_type(f.get("Type", "string")),+ "required": f.get("Required", False)}+ for f in schema_field_list+ ]+ except ValueError:+ raise ValueError("The specified metadata is not valid") from Noneiceberg_metadata = _s3tables._initial_iceberg_metadata(table_name, schema_fields, location)metadata_location = f"s3://{bucket_name}/{namespace}/{table_name}/metadata/v0.metadata.json"table_arn = _s3tables._table_arn(bucket_arn, namespace, table_name)diff --git a/ministack/services/s3tables.py b/ministack/services/s3tables.pyindex db6e8741..232187a9 100644--- a/ministack/services/s3tables.py+++ b/ministack/services/s3tables.py@@ -35,7 +35,7 @@ import loggingimport osimport reimport time-from urllib.parse import unquote+from urllib.parse import quote, unquotefrom ministack.core.arn import ArnParseError, parse_arnfrom ministack.core.responses import (@@ -186,6 +186,42 @@ def _set_bucket_region_value(store, bucket_arn, key, value):store.set_scoped(spec.account_id, spec.region, key, value)+_ICEBERG_V2_PRIMITIVES = {+ "boolean", "int", "long", "float", "double", "date", "time", "timestamp", "timestamptz", "string", "uuid", "binary",+}+_DECIMAL_TYPE_RE = re.compile(r"decimal\(\s*(\d+)\s*,\s*(\d+)\s*\)")+_FIXED_TYPE_RE = re.compile(r"fixed\[\s*(\d+)\s*\]")+++def _iceberg_field_type(kind):+ """An Iceberg v2 type as the table metadata stores it; ValueError when invalid."""+ if isinstance(kind, dict):+ nested = dict(kind)+ if kind.get("type") == "struct" and isinstance(kind.get("fields"), list):+ nested["fields"] = [{**f, "type": _iceberg_field_type(f.get("type"))} for f in kind["fields"]]+ elif kind.get("type") == "list" and "element" in kind:+ nested["element"] = _iceberg_field_type(kind["element"])+ elif kind.get("type") == "map" and "key" in kind and "value" in kind:+ nested["key"] = _iceberg_field_type(kind["key"])+ nested["value"] = _iceberg_field_type(kind["value"])+ else:+ raise ValueError(kind)+ return nested+ if not isinstance(kind, str):+ raise ValueError(kind)+ lowered = kind.strip().lower()+ if lowered in _ICEBERG_V2_PRIMITIVES:+ return lowered+ if match := _DECIMAL_TYPE_RE.fullmatch(lowered):+ precision, scale = int(match.group(1)), int(match.group(2))+ if precision > 38:+ raise ValueError(kind)+ return f"decimal({precision}, {scale})"+ if match := _FIXED_TYPE_RE.fullmatch(lowered):+ return f"fixed[{int(match.group(1))}]"+ raise ValueError(kind)++def _initial_iceberg_metadata(table_name, schema_fields, location):table_uuid = new_uuid()fields = []@@ -372,8 +408,12 @@ def _create_table(bucket_arn, namespace, data):metadata = data.get("metadata", {})iceberg_meta = metadata.get("iceberg", {})schema_def = iceberg_meta.get("schema", iceberg_meta.get("schemaV2", {}))- for f in schema_def.get("field", schema_def.get("fields", [])):- schema_fields.append({"name": f["name"], "type": f.get("type", "string"), "required": f.get("required", False)})+ try:+ for f in schema_def.get("field", schema_def.get("fields", [])):+ schema_fields.append({"name": f["name"], "type": _iceberg_field_type(f.get("type", "string")),+ "required": f.get("required", False)})+ except ValueError:+ return error_response_json("BadRequestException", "The specified metadata is not valid", 400)logger.debug("S3Tables: create_table %s/%s schema_fields=%d metadata_keys=%s",namespace,@@ -396,7 +436,7 @@ def _create_table(bucket_arn, namespace, data):{"id": f.get("id", i + 1),"name": f["name"],- "type": f.get("type", "string"),+ "type": schema_fields[i]["type"],"required": f.get("required", False),})@@ -526,6 +566,9 @@ def _update_table_metadata_location(bucket_arn, namespace, table_name, data):# ── Iceberg REST catalog (data plane for Spark) ───────────+_TABLE_BUCKET_ARN_RE = re.compile(r"arn:aws[\w-]*:s3tables:[\w-]+:\d{12}:bucket/[a-z0-9-]+")++def _iceberg_config(query_params=None, headers=None):warehouse = ""if query_params:@@ -545,8 +588,19 @@ def _iceberg_config(query_params=None, headers=None):"s3.path-style-access": "true",}overrides = {}- if warehouse:+ if ":s3tablescatalog/" in warehouse:overrides["prefix"] = warehouse+ elif _TABLE_BUCKET_ARN_RE.fullmatch(warehouse):+ defaults.update({+ "write.object-storage.partitioned-paths": "false",+ "s3.delete-enabled": "false",+ "io-impl": "org.apache.iceberg.aws.s3.S3FileIO",+ "write.object-storage.enabled": "true",+ "prefix": quote(warehouse, safe=""),+ "rest-metrics-reporting-enabled": "false",+ })+ else:+ return _iceberg_error("The specified table bucket ARN is not valid.", "bad_request", 400)return json_response({"defaults": defaults, "overrides": overrides})@@ -1044,6 +1098,10 @@ def _iceberg_create_table(namespace, data, allow_cross_region, bucket_filter=Non# ── Iceberg REST router ───────────────────────────────────+def _unknown_operation():+ return 404, {"Content-Type": "application/xml"}, b"<UnknownOperationException/>"++async def _handle_iceberg_request(method, path, headers, body, query_params):parts = [p for p in path.strip("/").split("/") if p]allow_cross_region = _iceberg_allows_cross_region(headers)@@ -1062,7 +1120,9 @@ async def _handle_iceberg_request(method, path, headers, body, query_params):if parts[1] == "v1" and method == "POST" and "transactions" in parts[2:]:data = json.loads(body) if body else {}tx_idx = parts.index("transactions", 2)- tx_filter = _bucket_scope_from_prefix(unquote("/".join(parts[2:tx_idx]))) if tx_idx > 2 else None+ if tx_idx == 2:+ return _unknown_operation()+ tx_filter = _bucket_scope_from_prefix(unquote("/".join(parts[2:tx_idx])))for change in data.get("table-changes", []):ident = change.get("identifier", {})ns = ident.get("namespace", [""])@@ -1076,18 +1136,8 @@ async def _handle_iceberg_request(method, path, headers, body, query_params):if len(parts) < 3:return None- # Support two URL formats:- # Standard Iceberg REST: /iceberg/v1/{prefix}/namespaces/...- # S3 Tables (no prefix): /iceberg/v1/namespaces/... (warehouse in query param)- #- # The prefix carries the warehouse value from /v1/config, which for S3- # Tables is ``{account}:s3tablescatalog/{bucket-name}``. Extract the- # bucket name to scope table lookups -- without it, tables with the same- # name in different buckets collide and the wrong schema is returned.- # The prefix from /v1/config can contain ``/`` (e.g.- # ``000000000000:s3tablescatalog/bucket-name``), which splits into- # multiple URL segments. Scan for ``namespaces`` and treat everything- # between ``v1`` and ``namespaces`` as the prefix.+ # Every route but /v1/config takes the {prefix}, which "for S3 Tables is+ # always your url-encoded table bucket ARN"; it can span several segments.bucket_filter = Nonens_idx = Nonefor i in range(2, len(parts)):@@ -1096,8 +1146,9 @@ async def _handle_iceberg_request(method, path, headers, body, query_params):breakif ns_idx is None:return None- if ns_idx > 2:- bucket_filter = _bucket_scope_from_prefix(unquote("/".join(parts[2:ns_idx])))+ if ns_idx == 2:+ return _unknown_operation()+ bucket_filter = _bucket_scope_from_prefix(unquote("/".join(parts[2:ns_idx])))rest = parts[ns_idx + 1 :] # segments after "namespaces"diff --git a/pyproject.toml b/pyproject.tomlindex d467664a..2628f447 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"[project]name = "ministack"-version = "1.5.27"+version = "1.5.28"description = "Free, open-source local AWS emulator — drop-in LocalStack replacement"readme = "README.md"license = { file = "LICENSE" }diff --git a/tests/test_cfn.py b/tests/test_cfn.pyindex 6aefd4a8..d7750b93 100644--- a/tests/test_cfn.py+++ b/tests/test_cfn.py@@ -18523,7 +18523,7 @@ def test_cfn_s3tables_table_schema_from_iceberg_metadata(cfn, s3tables):stack = _wait_stack(cfn, stack_name)assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")- resp = _cfn_iceberg_json("/iceberg/v1/namespaces/myns/tables/mytable")+ resp = _cfn_iceberg_json("/iceberg/v1/arn%3Aaws%3As3tables%3Aus-east-1%3A000000000000%3Abucket%2Fcfn-s3tables-schema-test/namespaces/myns/tables/mytable")fields = resp.get("metadata", {}).get("schemas", [{}])[0].get("fields", [])field_names = {f["name"] for f in fields}assert field_names == {"id", "value"}, f"expected columns id/value, got {field_names}"diff --git a/tests/test_s3tables.py b/tests/test_s3tables.pyindex 563f5430..78b4f960 100644--- a/tests/test_s3tables.py+++ b/tests/test_s3tables.py@@ -495,62 +495,54 @@ def test_iceberg_rest_error_uses_spec_envelope():assert body["error"]["message"]-def test_s3tables_iceberg_catalog_no_prefix_url_format(s3tables):- """S3 Tables uses /iceberg/v1/namespaces/... (no catalog prefix in path,- warehouse in query param) — the format DuckDB sends with ENDPOINT_TYPE s3_tables- or an explicit ENDPOINT pointing at the catalog root."""- bucket_name = f"tb-noprefix-{_uuid_mod.uuid4().hex[:6]}"- bucket_arn = s3tables.create_table_bucket(name=bucket_name)["arn"]+def test_s3tables_iceberg_routes_without_prefix_are_unknown(s3tables):+ """Every route but /v1/config takes the table bucket ARN prefix; real AWS+ answers one without it 404 <UnknownOperationException/>."""+ bucket_arn = s3tables.create_table_bucket(name=f"tb-noprefix-{_uuid_mod.uuid4().hex[:6]}")["arn"]ns = f"ns_{_uuid_mod.uuid4().hex[:6]}"table = f"t_{_uuid_mod.uuid4().hex[:6]}"try:s3tables.create_namespace(tableBucketARN=bucket_arn, namespace=[ns])s3tables.create_table(- tableBucketARN=bucket_arn,- namespace=ns,- name=table,- format="ICEBERG",+ tableBucketARN=bucket_arn, namespace=ns, name=table, format="ICEBERG",metadata={"iceberg": {"schema": {"fields": [{"name": "id", "type": "long"}]}}},)-- # List namespaces — no prefix- resp = _iceberg_json("/iceberg/v1/namespaces")- ns_names = [- (n[0] if isinstance(n, list) else n)- for n in resp.get("namespaces", [])- ]- assert ns in ns_names-- # List tables — no prefix- resp = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables")- assert {"namespace": [ns], "name": table} in resp.get("identifiers", [])-- # Load table — no prefix- resp = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")- assert resp.get("metadata-location", "").startswith(f"s3://{bucket_name}/{ns}/{table}/")- assert resp.get("metadata", {}).get("table-uuid")+ for method, path in (+ ("GET", "/iceberg/v1/namespaces"),+ ("GET", f"/iceberg/v1/namespaces/{ns}/tables"),+ ("GET", f"/iceberg/v1/namespaces/{ns}/tables/{table}"),+ ("POST", f"/iceberg/v1/namespaces/{ns}/tables"),+ ("POST", "/iceberg/v1/transactions/commit"),+ ):+ req = urllib.request.Request(f"{_ENDPOINT}{path}", method=method, data=b"{}" if method == "POST" else None)+ with pytest.raises(urllib.error.HTTPError) as exc:+ urllib.request.urlopen(req)+ assert (exc.value.code, exc.value.read()) == (404, b"<UnknownOperationException/>")+ loaded = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")+ assert loaded["metadata-location"].startswith(f"s3://{bucket_arn.rsplit('/', 1)[1]}/{ns}/{table}/")finally:- try:- s3tables.delete_table(tableBucketARN=bucket_arn, namespace=ns, name=table)- except Exception:- pass- try:- s3tables.delete_namespace(tableBucketARN=bucket_arn, namespace=ns)- except Exception:- pass+ s3tables.delete_table(tableBucketARN=bucket_arn, namespace=ns, name=table)+ s3tables.delete_namespace(tableBucketARN=bucket_arn, namespace=ns)s3tables.delete_table_bucket(tableBucketARN=bucket_arn)def test_s3tables_iceberg_config_returns_s3_defaults():- """GET /iceberg/v1/config must return S3 connection defaults so clients- don't need out-of-band credential / endpoint configuration."""- resp = _iceberg_json("/iceberg/v1/config")- defaults = resp.get("defaults", {})- assert "s3.endpoint" in defaults, "s3.endpoint missing from config defaults"- assert "s3.access-key-id" in defaults, "s3.access-key-id missing from config defaults"- assert "s3.secret-access-key" in defaults, "s3.secret-access-key missing from config defaults"- assert "s3.path-style-access" in defaults, "s3.path-style-access missing from config defaults"- assert "client.region" in defaults, "client.region missing from config defaults"+ """GET /iceberg/v1/config hands back the url-encoded table bucket ARN as the+ prefix in defaults, as AWS does, plus the S3 connection defaults."""+ arn = "arn:aws:s3tables:us-east-1:000000000000:bucket/cfg-bucket"+ resp = _iceberg_json(f"/iceberg/v1/config?warehouse={_arn_prefix(arn)}")+ defaults = resp["defaults"]+ assert defaults["prefix"] == _arn_prefix(arn)+ assert resp["overrides"] == {}+ for key in ("s3.endpoint", "s3.access-key-id", "s3.secret-access-key", "s3.path-style-access", "client.region"):+ assert key in defaults++[email protected]("query", ["", "?warehouse=arn:aws:s3tables:us-east-1:000000000000:bucket"])+def test_s3tables_iceberg_config_needs_a_table_bucket_arn(query):+ status, body = _iceberg_call(f"/iceberg/v1/config{query}")+ assert (status, body) == (400, {"error": {+ "message": "The specified table bucket ARN is not valid.", "type": "bad_request", "code": 400}})def test_s3tables_iceberg_load_table_includes_region(s3tables):@@ -564,7 +556,7 @@ def test_s3tables_iceberg_load_table_includes_region(s3tables):s3tables.create_namespace(tableBucketARN=bucket_arn, namespace=[ns])s3tables.create_table(tableBucketARN=bucket_arn, namespace=ns, name=table, format="ICEBERG")- resp = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ resp = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")config = resp.get("config", {})assert "s3.region" in config, "s3.region missing from LoadTable config"assert "client.region" in config, "client.region missing from LoadTable config"@@ -593,7 +585,7 @@ def test_s3tables_iceberg_transactions_commit(s3tables):snapshot_id = 1234567890resp = _iceberg_json(- "/iceberg/v1/transactions/commit",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/transactions/commit",method="POST",payload={"table-changes": [@@ -626,7 +618,7 @@ def test_s3tables_iceberg_transactions_commit(s3tables):assert resp is not None# Verify the snapshot was recorded in the table metadata- loaded = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ loaded = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")metadata = loaded.get("metadata", {})snap_ids = [s.get("snapshot-id") for s in metadata.get("snapshots", [])]assert snapshot_id in snap_ids, f"snapshot {snapshot_id} not found after commit"@@ -658,21 +650,21 @@ def test_s3tables_iceberg_upgrade_format_version(s3tables):s3tables.create_namespace(tableBucketARN=bucket_arn, namespace=[ns])s3tables.create_table(tableBucketARN=bucket_arn, namespace=ns, name=table, format="ICEBERG")- loaded = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ loaded = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")assert loaded["metadata"]["format-version"] == 2resp = _iceberg_json(- f"/iceberg/v1/namespaces/{ns}/tables/{table}", method="POST",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}", method="POST",payload={"requirements": [],"updates": [{"action": "upgrade-format-version", "format-version": 3}]},)assert resp["metadata"]["format-version"] == 3- loaded = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ loaded = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")assert loaded["metadata"]["format-version"] == 3# Re-asserting the version the table already has is a no-op commit.resp = _iceberg_json(- f"/iceberg/v1/namespaces/{ns}/tables/{table}", method="POST",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}", method="POST",payload={"requirements": [],"updates": [{"action": "upgrade-format-version", "format-version": 3}]},)@@ -681,13 +673,13 @@ def test_s3tables_iceberg_upgrade_format_version(s3tables):# A downgrade is refused, and the table keeps its version.with pytest.raises(urllib.error.HTTPError) as exc_info:_iceberg_json(- f"/iceberg/v1/namespaces/{ns}/tables/{table}", method="POST",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}", method="POST",payload={"requirements": [],"updates": [{"action": "upgrade-format-version", "format-version": 2}]},)assert exc_info.value.code == 400assert b"downgrade" in exc_info.value.read()- loaded = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ loaded = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")assert loaded["metadata"]["format-version"] == 3finally:try:@@ -716,14 +708,14 @@ def test_s3tables_iceberg_create_table_rejects_resend_instead_of_wiping_data(s3ts3tables.create_namespace(tableBucketARN=bucket_arn, namespace=[ns])_iceberg_json(- f"/iceberg/v1/namespaces/{ns}/tables",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables",method="POST",payload={"name": table, "schema": {"type": "struct", "schema-id": 0, "fields": []}},)# Commit a snapshot so there's real state a resent create could destroy.snapshot_id = 42_iceberg_json(- f"/iceberg/v1/namespaces/{ns}/tables/{table}",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}",method="POST",payload={"requirements": [],@@ -746,7 +738,7 @@ def test_s3tables_iceberg_create_table_rejects_resend_instead_of_wiping_data(s3t# response was lost.try:_iceberg_json(- f"/iceberg/v1/namespaces/{ns}/tables",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables",method="POST",payload={"name": table, "schema": {"type": "struct", "schema-id": 0, "fields": []}},)@@ -757,7 +749,7 @@ def test_s3tables_iceberg_create_table_rejects_resend_instead_of_wiping_data(s3tassert body["error"]["type"] == "AlreadyExistsException"# The original snapshot must still be there -- not wiped by the resend.- loaded = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ loaded = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")snap_ids = [s.get("snapshot-id") for s in loaded["metadata"]["snapshots"]]assert snapshot_id in snap_ids, "resent create-table must not wipe existing table state"finally:@@ -786,7 +778,7 @@ def test_s3tables_iceberg_add_schema_advances_last_column_id(s3tables):s3tables.create_table(tableBucketARN=bucket_arn, namespace=ns, name=table, format="ICEBERG")_iceberg_json(- f"/iceberg/v1/namespaces/{ns}/tables/{table}",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}",method="POST",payload={"requirements": [],@@ -802,11 +794,11 @@ def test_s3tables_iceberg_add_schema_advances_last_column_id(s3tables):],},)- first = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ first = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")assert first["metadata"]["last-column-id"] == 1_iceberg_json(- f"/iceberg/v1/namespaces/{ns}/tables/{table}",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}",method="POST",payload={"requirements": [],@@ -825,7 +817,7 @@ def test_s3tables_iceberg_add_schema_advances_last_column_id(s3tables):],},)- second = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ second = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")metadata = second["metadata"]assert metadata["last-column-id"] == 2@@ -858,7 +850,7 @@ def test_s3tables_iceberg_add_spec_uses_real_wire_action_name(s3tables):s3tables.create_table(tableBucketARN=bucket_arn, namespace=ns, name=table, format="ICEBERG")_iceberg_json(- f"/iceberg/v1/namespaces/{ns}/tables/{table}",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}",method="POST",payload={"requirements": [],@@ -874,7 +866,7 @@ def test_s3tables_iceberg_add_spec_uses_real_wire_action_name(s3tables):],},)- loaded = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ loaded = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")metadata = loaded["metadata"]spec_ids = [s["spec-id"] for s in metadata["partition-specs"]]assert 1 in spec_ids, "add-spec must append the new partition spec"@@ -906,7 +898,7 @@ def test_s3tables_iceberg_create_table_honors_requested_partition_spec(s3tables)s3tables.create_namespace(tableBucketARN=bucket_arn, namespace=[ns])created = _iceberg_json(- f"/iceberg/v1/namespaces/{ns}/tables",+ f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables",method="POST",payload={"name": table,@@ -965,9 +957,9 @@ def test_s3tables_iceberg_add_schema_is_idempotent_by_schema_id(s3tables):# Send the identical commit twice, simulating Spark re-declaring its# current schema on a second, unrelated write.for _ in range(2):- _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}", method="POST", payload=payload)+ _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}", method="POST", payload=payload)- loaded = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ loaded = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")schema_ids = [s.get("schema-id") for s in loaded["metadata"]["schemas"]]assert schema_ids.count(0) == 1, "resent add-schema must not duplicate the schema-id entry"finally:@@ -1008,9 +1000,9 @@ def test_s3tables_iceberg_add_spec_is_idempotent_by_spec_id(s3tables):],}for _ in range(2):- _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}", method="POST", payload=payload)+ _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}", method="POST", payload=payload)- loaded = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ loaded = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")spec_ids = [s.get("spec-id") for s in loaded["metadata"]["partition-specs"]]assert spec_ids.count(1) == 1, "resent add-spec must not duplicate the spec-id entry"finally:@@ -1051,9 +1043,9 @@ def test_s3tables_iceberg_add_sort_order_is_idempotent_by_order_id(s3tables):],}for _ in range(2):- _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}", method="POST", payload=payload)+ _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}", method="POST", payload=payload)- loaded = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ loaded = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")order_ids = [o.get("order-id") for o in loaded["metadata"]["sort-orders"]]assert order_ids.count(1) == 1, "resent add-sort-order must not duplicate the order-id entry"finally:@@ -1093,7 +1085,7 @@ def test_s3tables_iceberg_load_table_serves_the_numerically_latest_metadata(s3tas3tables.create_namespace(tableBucketARN=bucket_arn, namespace=[ns])s3tables.create_table(tableBucketARN=bucket_arn, namespace=ns, name=table, format="ICEBERG")- base = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")["metadata"]+ base = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")["metadata"]for v in range(1, 12):doc = dict(base, **{"last-sequence-number": v})s3.put_object(@@ -1102,7 +1094,7 @@ def test_s3tables_iceberg_load_table_serves_the_numerically_latest_metadata(s3taBody=json.dumps(doc).encode(),)- resp = _iceberg_json(f"/iceberg/v1/namespaces/{ns}/tables/{table}")+ resp = _iceberg_json(f"/iceberg/v1/{_arn_prefix(bucket_arn)}/namespaces/{ns}/tables/{table}")assert resp["metadata-location"].endswith("/v11.metadata.json"), resp["metadata-location"]assert resp["metadata"]["last-sequence-number"] == 11finally:@@ -1351,7 +1343,10 @@ def test_s3tables_table_subresources_and_rename(s3tables):def test_s3tables_create_table_keeps_iceberg_types(s3tables):bucket = s3tables.create_table_bucket(name=f"types-{_uuid_mod.uuid4().hex[:8]}")["arn"]s3tables.create_namespace(tableBucketARN=bucket, namespace=["ns"])- primitives = ["decimal(19, 2)", "timestamp", "timestamptz", "time", "uuid", "fixed[16]", "binary", "long"]+ primitives = ["decimal(19, 2)", "decimal(19,2)", "timestamp", "timestamptz", "time", "uuid", "fixed[16]", "binary",+ "STRING"]+ stored = ["decimal(19, 2)", "decimal(19, 2)", "timestamp", "timestamptz", "time", "uuid", "fixed[16]", "binary",+ "string"]nested = {"type": "list", "element-id": 3, "element": "string", "element-required": False}try:s3tables.create_table(@@ -1367,12 +1362,32 @@ def test_s3tables_create_table_keeps_iceberg_types(s3tables):)prefix = _arn_prefix(bucket)flat = _iceberg_json(f"/iceberg/v1/{prefix}/namespaces/ns/tables/flat")["metadata"]- assert [f["type"] for f in flat["schemas"][0]["fields"]] == primitives+ assert [f["type"] for f in flat["schemas"][0]["fields"]] == storedfields = _iceberg_json(f"/iceberg/v1/{prefix}/namespaces/ns/tables/nested")["metadata"]["schemas"][0]["fields"]- assert [f["type"] for f in fields] == ["decimal(9,2)", nested]+ assert [f["type"] for f in fields] == ["decimal(9, 2)", nested]finally:for name in ("flat", "nested"):with contextlib.suppress(ClientError):s3tables.delete_table(tableBucketARN=bucket, namespace="ns", name=name)s3tables.delete_namespace(tableBucketARN=bucket, namespace="ns")s3tables.delete_table_bucket(tableBucketARN=bucket)++[email protected]("kind", ["timestamp_ns", "varchar", "decimal(x,2)"])+def test_s3tables_create_table_refuses_an_invalid_iceberg_type(s3tables, kind):+ bucket = s3tables.create_table_bucket(name=f"badtype-{_uuid_mod.uuid4().hex[:8]}")["arn"]+ s3tables.create_namespace(tableBucketARN=bucket, namespace=["ns"])+ try:+ with pytest.raises(ClientError) as exc:+ s3tables.create_table(+ tableBucketARN=bucket, namespace="ns", name="t", format="ICEBERG",+ metadata={"iceberg": {"schema": {"fields": [{"name": "c", "type": kind}]}}},+ )+ error = exc.value.response+ assert error["ResponseMetadata"]["HTTPStatusCode"] == 400+ assert (error["Error"]["Code"], error["Error"]["Message"]) == (+ "BadRequestException", "The specified metadata is not valid")+ assert s3tables.list_tables(tableBucketARN=bucket)["tables"] == []+ finally:+ s3tables.delete_namespace(tableBucketARN=bucket, namespace="ns")+ s3tables.delete_table_bucket(tableBucketARN=bucket)