Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
prerelease (#1840)
Nahuel Nuceraff9015207854
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex acac654f..3e24dd58 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -9,22 +9,25 @@ Versioning follows [Semantic Versioning](https://semver.org/).### Added-- **SNS — `AddPermission` and `RemovePermission`** — they append to and prune the topic's `Policy` attribute with the AWS statement shape (bare account ids, lowercase `sns:` actions), mirroring the long-standing SQS pair, and a duplicate label is `InvalidParameterException`.+- **Cognito — OIDC discovery through the real issuer host (`USE_SSL=1`)** — the generated certificate now names `cognito-idp.<region>.amazonaws.com` for every region, the gateway also listens on 443 when it can, and Lambda containers resolve those hosts to MiniStack and trust its certificate. With an `/etc/hosts` entry and the certificate trusted, an unmodified client follows the token's `iss` to MiniStack while `iss` stays exactly as AWS issues it. Reported by @epcap90.+- **Bedrock AgentCore — `InvokeAgentRuntime` runs the runtime's container** — with Docker available (full image), the runtime's `containerConfiguration.containerUri` image is started once, checked on `/ping`, and invocations are forwarded to `/invocations` on port 8080; worker and container failures answer `RuntimeClientError` (424). The container is removed on update, delete or reset. Without Docker, or for a code artifact, the deterministic echo remains. Contributed by @pingedbrain.+- **SNS — `AddPermission` and `RemovePermission`** — they add and remove labelled statements in the topic's `Policy`; a duplicate label answers `InvalidParameter`. Contributed by @AdrianAcala.### Fixed-- **SQS/SNS — queue and topic policies are enforced under `AUTH=true`** — the `Policy` attributes were stored but never evaluated, so S3→SQS and SNS→SQS deliveries landed and cross-account send/publish failed regardless of the policy. Same-account calls now pass unless the policy carries an explicit Deny; cross-account calls need an explicit Allow (`AccessDenied` for SQS, `AuthorizationError` for SNS); S3 and SNS deliveries are dropped unless the destination policy Allows the calling service with its `aws:SourceArn`/`aws:SourceAccount` conditions, exactly like real AWS — including queues and topics with no policy at all. With auth disabled every path stays permissive. `GetQueueUrl` accepts `QueueOwnerAWSAccountId`.-- **S3 — notification destinations are validated at PUT time** — configuring an SQS/SNS target whose policy does not let S3 publish now fails the whole `PutBucketNotificationConfiguration` with `InvalidArgument` ("Unable to validate the following destination configurations"), like real S3, when `AUTH=true`; with auth disabled the probe checks existence only. SQS/SNS targets may live in another account when their policy allows the delivery. The `SkipDestinationValidation` option skips the probe (and the test event).+- **SQS/SNS — queue and topic policies are enforced under `AUTH=true`** — the `Policy` attributes were stored but never evaluated. Same-account calls pass unless the policy explicitly denies them; cross-account calls need an explicit Allow (`AccessDenied` for SQS, `AuthorizationError` for SNS). S3→SQS, S3→SNS and SNS→SQS deliveries need the destination policy to allow the calling service. With `AUTH=false` every path stays permissive. `GetQueueUrl` accepts `QueueOwnerAWSAccountId`. Contributed by @AdrianAcala.+- **S3 — notification destinations are validated at PUT time** — under `AUTH=true`, `PutBucketNotificationConfiguration` fails with `InvalidArgument` ("Unable to validate the following destination configurations") when an SQS/SNS destination's policy does not let S3 publish; with `AUTH=false` only existence is checked. Cross-account SQS/SNS destinations are accepted, and `SkipDestinationValidation` skips the check. Contributed by @AdrianAcala.- **Bedrock AgentCore — opt-in Docker runtime** — set `MINISTACK_AGENTCORE_DOCKER=1` to start the declared `containerConfiguration.containerUri` image on first invocation, connect via a localhost port when MiniStack runs on the host or via MiniStack's Docker network when containerized, call `/ping`, and forward `InvokeAgentRuntime` to `/invocations` on port 8080. The container is reused and removed on update, delete, or reset. Container and worker HTTP errors become `RuntimeClientError` (424); caller credentials are not forwarded. Without Docker mode, the existing deterministic echo remains for SDK tests. This covers HTTP only and does not emulate AWS IAM/VPC isolation or model inference.- **CloudWatch Logs — delivery destination policy** — `PutDeliveryDestinationPolicy`, `GetDeliveryDestinationPolicy`, `DeleteDeliveryDestinationPolicy`. Required by an operator that authorizes `delivery.logs.amazonaws.com` to create deliveries to a `DeliveryDestination` (e.g. CloudFront access-log delivery to a log group) — `PutDeliveryDestinationPolicy` previously had no route, blocking the delivery from ever being created. Contributed by @fabio-andre-rodrigues.-- **S3 — object annotations** — `PutObjectAnnotation`, `GetObjectAnnotation`, `ListObjectAnnotations` (prefix, `max-annotation-results`, continuation tokens) and `DeleteObjectAnnotation`: named UTF-8 payloads of 1 byte to 1 MiB, up to 1,000 per object version, with S3's naming rules (`InvalidAnnotationName`, `AnnotationNameTooLong`), `AnnotationLimitExceeded`, `UnsupportedMediaType` and `x-amz-object-if-match`. Each belongs to one object version: a new version starts with none, deleting a version deletes its annotations, a delete marker keeps them, and an unversioned overwrite or delete drops them; writing one changes neither the object's ETag nor its version. `CopyObject` carries them unless `x-amz-object-annotation-directive: EXCLUDE`, and recomputes their checksums when the copy names a new algorithm. Payload checksums are verified (`BadDigest`) and default to CRC64NVME; CRC32, CRC32C, CRC64NVME, SHA1, SHA256, SHA512, MD5 and XXHASH64 are computed, while XXHASH3 and XXHASH128 are refused rather than stored unverified. Annotations take the object's encryption and are refused on SSE-C objects, replicate with the object (`COMPLETED` / `REPLICA`), respect Object Lock on every write (governance retention needs `x-amz-bypass-governance-retention` to put, overwrite or delete one; compliance retention and a legal hold refuse all three, bypass or not, each as 403 `AccessDenied` naming retention or the legal hold, as observed on AWS), persist per version with `S3_PERSIST=1`, and send `s3:ObjectAnnotation:Put` / `s3:ObjectAnnotation:Delete` events with the record's `objectAnnotation` block; a `CopyObject` event reports `hasObjectAnnotation`. Previously a `PUT ?annotation` fell through to `PutObject` and replaced the object with the payload. The S3 Metadata annotation table (`UpdateBucketMetadataAnnotationTableConfiguration`) is not included: MiniStack has no S3 Metadata configuration for it to belong to. Checked against the S3 API Reference, the S3 User Guide's "Annotating your objects" page and the botocore model; the Object Lock behaviour was validated against AWS (ap-southeast-2).+- **S3 — object annotations** — `PutObjectAnnotation`, `GetObjectAnnotation`, `ListObjectAnnotations` and `DeleteObjectAnnotation`: up to 1,000 named UTF-8 payloads (1 byte to 1 MiB) per object version, with S3's naming rules and errors. `CopyObject` carries them unless `x-amz-object-annotation-directive: EXCLUDE`; checksums default to CRC64NVME (XXHASH3 and XXHASH128 are refused); annotations take the object's encryption (refused on SSE-C), replicate, persist with `S3_PERSIST=1`, and send `s3:ObjectAnnotation:Put` / `s3:ObjectAnnotation:Delete` events. Under `AUTH=true`, Object Lock refuses annotation writes with 403 `AccessDenied` (governance retention unless `x-amz-bypass-governance-retention: true`, which also needs `s3:BypassGovernanceRetention`; compliance retention and a legal hold always), and the operations authorize as `s3:PutObjectAnnotation`, `s3:GetObjectAnnotation`, `s3:ListObjectAnnotations` and `s3:DeleteObjectAnnotation`. Previously a `PUT ?annotation` replaced the object with the payload. Contributed by @BoLaMN.### Fixed-- **API Gateway v1 — REQUEST authorizer events carry `requestContext.identity`** — the event sent to a REQUEST Lambda authorizer had no `identity` block, so an authorizer reading `event.requestContext.identity.sourceIp` or `.userAgent` crashed and every request answered 500 or its own fallback Deny. It now reports `sourceIp` (first `X-Forwarded-For` hop) and `userAgent`, the same values the `AWS_PROXY` integration event already carries. Checked against the API Gateway Developer Guide ("Input to an API Gateway Lambda authorizer", whose REQUEST example includes `requestContext.identity.sourceIp`, and the `$context.identity.userAgent` context variable); a production REQUEST authorizer that reads `identity.userAgent` works on AWS. Not validated against a captured AWS event.+- **API Gateway v1 — REQUEST authorizer events carry `requestContext.identity`** — the event had no `identity` block, so an authorizer reading `sourceIp` or `userAgent` crashed. It now carries both, as the `AWS_PROXY` integration event does. Contributed by @yosriady.- **DynamoDB — `DescribeTable` accepts a table ARN** — `TableName` given as the table's ARN answered `ResourceNotFoundException`, where AWS accepts either form. It now resolves through the same ARN normalization the other table operations use, and an ARN from another account or region still answers `ResourceNotFoundException`. This unblocks OpenSearch Data Prepper's DynamoDB source, which always describes the table by ARN.-- **S3 — event notification records match S3's** — the object block carries the key URL-encoded (`red flower.jpg` → `red+flower.jpg`), the `versionId` the write made or removed, and a `sequencer` that grows with every create and delete (it was always `"0"`); records are `eventVersion` `2.6`. A delete that leaves a delete marker is `ObjectRemoved:DeleteMarkerCreated` (it was `ObjectRemoved:Delete`, so a subscription to permanent deletes received markers), and `DeleteObjects` sends an event for each object it removes (it sent none). Checked against the S3 User Guide's "Event message structure" and "Event notification types and destinations" pages; not validated against a real AWS account.-- **STS — `GetCallerIdentity` resolves IAM-user callers** — keys created with `CreateAccessKey` returned the `root` ARN; they now return the user's ARN and ID in both XML and JSON protocols. Under `AUTH=true`, unknown and inactive keys are rejected instead of reported as root.+- **S3 — event notification records match S3's** — the key is URL-encoded (`red flower.jpg` → `red+flower.jpg`), `versionId` names the version the write made or removed, `sequencer` grows with every create and delete (it was always `"0"`), and `eventVersion` is `2.6`. A delete that leaves a delete marker is `ObjectRemoved:DeleteMarkerCreated` (it was `ObjectRemoved:Delete`), and `DeleteObjects` sends an event for each object it removes (it sent none). Contributed by @BoLaMN.+- **STS — `GetCallerIdentity` resolves IAM-user callers** — keys created with `CreateAccessKey` returned the `root` ARN; they now return the user's ARN and ID. Under `AUTH=true`, unknown and inactive keys are refused with 403 `InvalidClientTokenId` instead of reported as root. Contributed by @AdrianAcala.+- **STS — an unknown access key answers `InvalidClientTokenId`** — under `AUTH=true`, STS operations refused an unknown key with `UnrecognizedClientException`; real STS answers `InvalidClientTokenId`.## [1.5.17] — 2026-09-25diff --git a/README.md b/README.mdindex af9dc11c..8dbfdd73 100644--- a/README.md+++ b/README.md@@ -432,7 +432,7 @@ subnet = ec2.create_subnet(| **Bedrock Runtime** | Converse, ConverseStream, InvokeModel, InvokeModelWithResponseStream, ApplyGuardrail, StartAsyncInvoke, GetAsyncInvoke, ListAsyncInvokes | Real eventstream wire format on streaming operations; **guardrails enforce their deterministic policies** — word policy, sensitive-information regexes, and the pattern-matchable PII types with `BLOCK`/`ANONYMIZE` per input/output stage — on `ApplyGuardrail` and on `Converse`/`ConverseStream` via `guardrailConfig` (`GUARDRAIL_INTERVENED`, blocked messaging, per-policy assessments, `ResourceNotFoundException` for an unknown id); the model-graded policies (topic, content, contextual grounding, `PROFANITY`) are not evaluated; deterministic family-aware mock responses selected by model ID prefix (`anthropic.*`, `amazon.titan*`, `amazon.nova*`, `meta.llama*`, `mistral.*`, `cohere.*`, `ai21.*`); verified against botocore `bedrock-runtime-2023-09-30` || **Bedrock Agent** | 72 operations — agents (versions, aliases, action groups, collaborators, agent knowledge bases, PrepareAgent), knowledge bases (data sources, ingestion jobs — `StartIngestionJob` reads the S3 data source and reports real statistics, `FAILED` with `failureReasons` when the source is unreadable — documents), flows (aliases, versions, ValidateFlowDefinition, PrepareFlow), prompts with versions, tagging | All 72 operations verified against botocore `bedrock-agent-2023-06-05` || **Bedrock Agent Runtime** | InvokeAgent, InvokeInlineAgent, GetAgentMemory, DeleteAgentMemory, Retrieve, RetrieveAndGenerate, RetrieveAndGenerateStream, Rerank, CreateSession, GetSession, UpdateSession, DeleteSession, EndSession, ListSessions, CreateInvocation, ListInvocations, PutInvocationStep, GetInvocationStep, ListInvocationSteps, InvokeFlow, StartFlowExecution, StopFlowExecution, GetFlowExecution, ListFlowExecutions, ListFlowExecutionEvents, GetExecutionFlowSnapshot, OptimizePrompt | All 31 operations verified against botocore `bedrock-agent-runtime-2023-07-26`; eventstream responses on streaming operations; `Retrieve` searches the documents Knowledge Base ingestion stored (lexical scoring — no embeddings) and returns content, `s3Location` and score; `InvokeAgent` answers a canned reply without orchestration and emits no trace events — it never fabricates an `ORCHESTRATE` trace |-| **Bedrock AgentCore** | Control plane (`bedrock-agentcore-control`): CreateAgentRuntime, GetAgentRuntime, ListAgentRuntimes, UpdateAgentRuntime, DeleteAgentRuntime, ListAgentRuntimeVersions, CreateAgentRuntimeEndpoint, GetAgentRuntimeEndpoint, ListAgentRuntimeEndpoints, UpdateAgentRuntimeEndpoint, DeleteAgentRuntimeEndpoint. Data plane (`bedrock-agentcore`): InvokeAgentRuntime | Agent-runtime lifecycle with `CREATING`/`UPDATING`→`READY` transitions, versioned ARNs (`agent/{uuid}:{version}`, `agentEndpoint/{uuid}`); request/response shapes verified against botocore `bedrock-agentcore-control` / `bedrock-agentcore`; account- and region-scoped state; `InvokeAgentRuntime` echoes by default or, with `MINISTACK_AGENTCORE_DOCKER=1`, starts the declared container image and forwards HTTP invocations to its port 8080 (no model inference in MiniStack) |+| **Bedrock AgentCore** | Control plane (`bedrock-agentcore-control`): CreateAgentRuntime, GetAgentRuntime, ListAgentRuntimes, UpdateAgentRuntime, DeleteAgentRuntime, ListAgentRuntimeVersions, CreateAgentRuntimeEndpoint, GetAgentRuntimeEndpoint, ListAgentRuntimeEndpoints, UpdateAgentRuntimeEndpoint, DeleteAgentRuntimeEndpoint. Data plane (`bedrock-agentcore`): InvokeAgentRuntime | Agent-runtime lifecycle with `CREATING`/`UPDATING`→`READY` transitions, versioned ARNs (`agent/{uuid}:{version}`, `agentEndpoint/{uuid}`); request/response shapes verified against botocore `bedrock-agentcore-control` / `bedrock-agentcore`; account- and region-scoped state; `InvokeAgentRuntime` runs the runtime's `containerUri` image in Docker (full image), waits for `/ping` and forwards to `/invocations` on port 8080, with container errors as `RuntimeClientError` (424); without Docker (slim image) or for a code artifact it answers a deterministic echo || **Transcribe** | StartTranscriptionJob, GetTranscriptionJob, ListTranscriptionJobs, DeleteTranscriptionJob | Batch transcription jobs only; streaming is a separate endpoint (`transcribestreaming`). All four operations verified against botocore `transcribe-2017-10-26`. A job starts `IN_PROGRESS` with `StartTime` set, as on AWS, and reaches `COMPLETED` over two seconds; `QUEUED`, which AWS reaches only for a request that opted into job queueing (`JobExecutionSettings.AllowDeferredExecution`) while the account is at its concurrent job limit, is not modelled; the media object is read from MiniStack S3 via an `s3://`, path-style or virtual-hosted HTTPS URI, and an unreadable URI fails the job with a `FailureReason` rather than erroring at `Start`, as on AWS; the transcript is written back to S3 in the real result format (`results.transcripts`, word-level `results.items`, `results.audio_segments`, plus `results.segments` and `results.speaker_labels` when the matching `Settings` are set) at `OutputBucketName`/`OutputKey`, or to an auto-created `ministack-transcribe-output` bucket. `IdentifyLanguage` and `IdentifyMultipleLanguages` resolve a language on completion, `ContentRedaction` controls whether a `redacted-` copy replaces or accompanies the transcript, and `Subtitles` writes WebVTT and SubRip files, redacted alongside the transcript when redaction is on. Reaching a terminal state publishes a `Transcribe Job State Change` event, so EventBridge rules downstream of a job fire. There is no speech recognition, so the transcript is deterministic canned text keyed on a digest of the media URI || **Translate** | StartTextTranslationJob, DescribeTextTranslationJob, ListTextTranslationJobs, StopTextTranslationJob | Batch text translation jobs only; the synchronous `TranslateText` / `TranslateDocument` pair and the custom terminology / parallel data families are not implemented. All four operations verified against botocore `translate-2017-07-01`. Jobs walk `SUBMITTED` → `IN_PROGRESS` → `COMPLETED` over `TRANSLATE_JOB_RUN_SECONDS` (default 2, `0` completes immediately); every document under `InputDataConfig.S3Uri` is read from MiniStack S3 and written back to the rewritten output location AWS uses — `OutputDataConfig.S3Uri` is reported as `s3://<bucket>/<prefix><account-id>-TranslateText-<JobId>/` — as `<target-code>.<input name>`, with a `details/<code>.auxiliary-translation-details.json` summary per target language. `text/plain` is translated line by line, `text/html` keeps its markup and translates only text nodes, `application/x-xliff+xml` gets a populated `<target>` in every `<trans-unit>` plus `target-language` on every `<file>`, and the three Office ZIP types are copied through. A document that can't be read as its content type is counted in `JobDetails` and the job ends `COMPLETED_WITH_ERROR`; an empty or unreadable input location fails the job with a `Message` rather than erroring at `Start`. `ClientToken` is idempotent, `StopTextTranslationJob` reaches a running worker, and reaching a terminal state publishes a `Translate TextTranslationJob State Change` event, so EventBridge rules downstream of a job fire. There is no machine translation, so the output is deterministic canned text derived from the source || **MSK** | CreateCluster, ListClusters, DescribeCluster, DeleteCluster, GetBootstrapBrokers, ListNodes, CreateConfiguration, ListConfigurations, DescribeConfiguration, ListConfigurationRevisions, DescribeConfigurationRevision, BatchAssociateScramSecret, BatchDisassociateScramSecret, ListScramSecrets, TagResource, UntagResource, ListTagsForResource | Kafka control plane; `GetBootstrapBrokers` honors `MINISTACK_MSK_BOOTSTRAP` so clients route to a real broker you bring (Redpanda, Kafka, KRaft) — the Kafka wire protocol itself is not emulated |@@ -865,7 +865,6 @@ end-to-end without any client config.| `MINISTACK_IMDS_V2_REQUIRED` | `0` | Reject token-less GETs on `/latest/meta-data/...`. When set, callers must first `PUT /latest/api/token` and pass the token as `X-aws-ec2-metadata-token`, matching real-AWS hop-limit-1 IMDSv2-only instances || `MINISTACK_BEDROCK_PROXY_URL` | _(unset)_ | OpenAI-compatible `/chat/completions` endpoint (Ollama, llama.cpp, vLLM). When set, Bedrock `Converse`/`InvokeModel` prompts are translated to OpenAI shape, forwarded, and translated back to Bedrock shape — real completions through the Bedrock API. Falls back to the deterministic mock on connection error || `MINISTACK_BEDROCK_PROXY_TIMEOUT_SECONDS` | `30` | Upstream request timeout for the Bedrock proxy |-| `MINISTACK_AGENTCORE_DOCKER` | _(unset)_ | Set to `1` to run the image in `agentRuntimeArtifact.containerConfiguration.containerUri` on first `InvokeAgentRuntime`. When MiniStack runs on the host, it publishes the worker's port 8080 on localhost; when MiniStack runs in Docker, it joins the worker to MiniStack's Docker network. It calls `/ping`, then forwards to `/invocations`. The container is reused until the runtime is updated, deleted, or MiniStack resets. Requires a reachable Docker daemon and an image Docker can pull or find locally. Container and HTTP errors return `RuntimeClientError` (424). Without this opt-in, the existing deterministic echo remains available for control-plane and SDK tests. This covers HTTP; IAM, VPC isolation, MCP, A2A and WebSocket are not emulated. **Keep the MiniStack API private when this is enabled:** anyone who can create and invoke a runtime can cause Docker to start an image of their choosing. Use only trusted images. || `MINISTACK_MSK_BOOTSTRAP` | _(unset)_ | Plaintext bootstrap broker string returned by MSK `GetBootstrapBrokers` — point it at a real broker you bring (Redpanda, Kafka, KRaft). Unset → placeholder endpoint (control-plane only) || `MINISTACK_MSK_BOOTSTRAP_TLS` | _(unset)_ | TLS bootstrap string (`BootstrapBrokerStringTls`) || `MINISTACK_MSK_BOOTSTRAP_SASL_SCRAM` | _(unset)_ | SASL/SCRAM bootstrap string (`BootstrapBrokerStringSaslScram`) |@@ -876,6 +875,25 @@ end-to-end without any client config.API Gateway v1/v2 HTTP proxy forwarding uses non-blocking event-loop semantics by offloading the upstream socket call to a worker thread. This preserves AWS-compatible response behavior while preventing long-running proxy calls from stalling unrelated requests (for example, parallel DynamoDB operations). The same non-blocking path is used for JWT **JWKS** fetches. Tune wall-clock limits with `MINISTACK_APIGW_PROXY_TIMEOUT_SECONDS` and `MINISTACK_APIGW_JWKS_TIMEOUT_SECONDS` at the bottom of the [Configuration](#configuration) table above.+### Cognito OIDC discovery (`USE_SSL=1`)++A Cognito token's `iss` is `https://cognito-idp.<region>.amazonaws.com/<poolId>`, exactly as on AWS. To let an unmodified client (Spring, Vault, `aws-jwt-verify`) follow it to MiniStack:++```bash+docker run -d --name ministack -p 443:4566 -p 4566:4566 -e USE_SSL=1 ministackorg/ministack+docker cp ministack:/tmp/ministack-tls/server.crt ./ministack-ca.pem+```++Then point the issuer host at MiniStack in `/etc/hosts` (`127.0.0.1 cognito-idp.us-east-1.amazonaws.com`) and trust `ministack-ca.pem` in your client's runtime. The OS keychain is not enough for these:++| Runtime | Trust step |+|---|---|+| Java | `keytool -importcert -file ministack-ca.pem -alias ministack -cacerts -storepass changeit` |+| Node | `NODE_EXTRA_CA_CERTS=./ministack-ca.pem` |+| Python (`requests`, boto3) | `REQUESTS_CA_BUNDLE` / `AWS_CA_BUNDLE` pointing at a bundle of your system roots plus `ministack-ca.pem` |++The generated certificate covers every region's issuer host. Lambda containers MiniStack starts get the hosts entries and the trust settings automatically. Remove the `/etc/hosts` line before calling the real Cognito.+### Startup ScriptsMiniStack supports two types of init scripts, with LocalStack-compatible paths:diff --git a/ministack/app.py b/ministack/app.pyindex 2ea6624b..41893cdc 100644--- a/ministack/app.py+++ b/ministack/app.py@@ -3239,6 +3239,16 @@ def main():logging.getLogger("hypercorn.access").addFilter(_HealthLogFilter())+ # Raise the soft fd limit to the hard one so connection bursts don't hit 1024.+ try:+ import resource++ _soft, _hard = resource.getrlimit(resource.RLIMIT_NOFILE)+ if _soft < _hard:+ resource.setrlimit(resource.RLIMIT_NOFILE, (_hard, _hard))+ except (ImportError, ValueError, OSError):+ pass+config = HypercornConfig()config.bind = [f"{bind_host}:{port}"]config.keep_alive_timeout = 75diff --git a/ministack/core/iam_actions.py b/ministack/core/iam_actions.pyindex cd6c9301..e298764e 100644--- a/ministack/core/iam_actions.py+++ b/ministack/core/iam_actions.py@@ -244,6 +244,8 @@ _S3_OBJECT_QUERY_ACTIONS: dict[str, dict[str, str]] = {"retention": {"GET": "GetObjectRetention", "PUT": "PutObjectRetention"},"legal-hold": {"GET": "GetObjectLegalHold", "PUT": "PutObjectLegalHold"},"attributes": {"GET": "GetObjectAttributes"},+ "annotation": {"GET": "GetObjectAnnotation", "PUT": "PutObjectAnnotation",+ "DELETE": "DeleteObjectAnnotation"},"select": {"POST": "GetObject"}, # SelectObjectContent reads the object"torrent": {"GET": "GetObject"},}@@ -282,6 +284,9 @@ def _s3_action(method: str, path: str, query_params: dict) -> str | None:breakif action is None:action = _S3_ACTIONS.get((method, depth))+ # GET ?annotation without annotationName is ListObjectAnnotations.+ if action == "GetObjectAnnotation" and "annotationName" not in query_params:+ action = "ListObjectAnnotations"if depth == 2 and action and _query_param(query_params, "versionId"):action = _S3_VERSIONED_ACTIONS.get(action, action)@@ -290,7 +295,8 @@ def _s3_action(method: str, path: str, query_params: dict) -> str | None:# Operations that take x-amz-bypass-governance-retention; when the header says# true they also need s3:BypassGovernanceRetention on the object.-_S3_GOVERNANCE_BYPASS_ACTIONS = frozenset({"DeleteObject", "DeleteObjectVersion", "PutObjectRetention"})+_S3_GOVERNANCE_BYPASS_ACTIONS = frozenset({"DeleteObject", "DeleteObjectVersion", "PutObjectRetention",+ "PutObjectAnnotation", "DeleteObjectAnnotation"})def _s3_source_object(headers: dict) -> tuple[str, str] | None:diff --git a/ministack/core/iam_evaluator.py b/ministack/core/iam_evaluator.pyindex 0994c7a3..98cb4b4c 100644--- a/ministack/core/iam_evaluator.py+++ b/ministack/core/iam_evaluator.py@@ -606,6 +606,9 @@ _request_caller_arn: contextvars.ContextVar[str] = contextvars.ContextVar(def pin_request_caller(headers: dict, query_params: dict | None = None) -> None:"""Remember the caller's principal ARN for this request's policy checks."""+ from ministack.app import AUTH+ if not AUTH:+ return_request_caller_arn.set(request_caller_arn(headers, query_params))diff --git a/ministack/core/tls.py b/ministack/core/tls.pyindex ca4abbc3..f193477c 100644--- a/ministack/core/tls.py+++ b/ministack/core/tls.py@@ -15,11 +15,7 @@ import tempfiledef _write_atomic(path: str, data: bytes) -> "str | None":- """Write `data` to `path` via a rename, or None on failure.-- Containers spawn concurrently and mount these files, so a reader must never- see a half-written one.- """+ """Write `data` to `path` via an atomic rename; None on failure."""try:handle, staging = tempfile.mkstemp(dir=os.path.dirname(path))with os.fdopen(handle, "wb") as out:@@ -40,12 +36,16 @@ def cognito_idp_host() -> str:return f"cognito-idp.{region}.amazonaws.com"-def ca_bundle_path(cert_path: str) -> "str | None":- """Public roots plus `cert_path`, or None.+def cognito_idp_hosts() -> "list[str]":+ """Every region's Cognito issuer host, so a pool in any region validates."""+ from ministack.services.account import _REGIONS_LIST- AWS_CA_BUNDLE and REQUESTS_CA_BUNDLE replace the trust store rather than- adding to it, so ours alone would cost the caller every other endpoint.- """+ hosts = [cognito_idp_host()] + [f"cognito-idp.{r}.amazonaws.com" for r in _REGIONS_LIST]+ return list(dict.fromkeys(hosts))+++def ca_bundle_path(cert_path: str) -> "str | None":+ """System roots plus `cert_path` (the *_CA_BUNDLE vars replace the store), or None."""import sslpaths = ssl.get_default_verify_paths()@@ -63,10 +63,7 @@ JAVA_TRUSTSTORE_PASSWORD = "changeit"def java_truststore_path(cert_path: str) -> "str | None":- """The same trust as `ca_bundle_path` in the PKCS12 a JVM reads, or None.-- Written here rather than with `keytool` so no JDK is needed on this side.- """+ """The same trust as a PKCS12 truststore for the JVM, or None."""try:from cryptography import x509from cryptography.hazmat.primitives.serialization import (@@ -98,8 +95,8 @@ def java_truststore_path(cert_path: str) -> "str | None":return _write_atomic(os.path.join(os.path.dirname(cert_path), "truststore.p12"), blob)-def _cert_names(cert_path: str, name: str) -> bool:- """Whether the certificate at `cert_path` carries `name` as a SAN."""+def _cert_names(cert_path: str, names: "list[str]") -> bool:+ """Whether the certificate at `cert_path` carries every name in `names` as a SAN."""try:out = subprocess.run(["openssl", "x509", "-in", cert_path, "-noout", "-ext", "subjectAltName"],@@ -107,7 +104,7 @@ def _cert_names(cert_path: str, name: str) -> bool:)except OSError:return True # No openssl to check with; leave the cached cert alone.- return name in (out.stdout or "")+ return all(f"DNS:{name}" in (out.stdout or "") for name in names)def resolve_tls_material() -> "tuple[str, str]":@@ -135,8 +132,8 @@ def resolve_tls_material() -> "tuple[str, str]":cert_path = os.path.join(tls_dir, "server.crt")key_path = os.path.join(tls_dir, "server.key")if (os.path.exists(cert_path) and os.path.exists(key_path)- and not _cert_names(cert_path, cognito_idp_host())):- # Cached by an older build or another region: cannot serve that name.+ and not _cert_names(cert_path, cognito_idp_hosts())):+ # Cached by an older build: regenerate so every issuer host is covered.os.remove(cert_path)os.remove(key_path)if not (os.path.exists(cert_path) and os.path.exists(key_path)):@@ -147,7 +144,8 @@ def resolve_tls_material() -> "tuple[str, str]":"-subj", "/CN=ministack-local/O=MiniStack","-addext","subjectAltName=DNS:localhost,DNS:ministack,"- f"DNS:{cognito_idp_host()},IP:127.0.0.1,IP:0:0:0:0:0:0:0:1",+ + "".join(f"DNS:{host}," for host in cognito_idp_hosts())+ + "IP:127.0.0.1,IP:0:0:0:0:0:0:0:1",], check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)os.chmod(cert_path, 0o600)os.chmod(key_path, 0o600)diff --git a/ministack/services/bedrock_agentcore.py b/ministack/services/bedrock_agentcore.pyindex 89f375b8..3f0d6516 100644--- a/ministack/services/bedrock_agentcore.py+++ b/ministack/services/bedrock_agentcore.py@@ -426,7 +426,8 @@ def _invoke_agent_runtime(runtime_arn, headers, body):session_id = (headers.get("x-amzn-bedrock-agentcore-runtime-session-id")or new_uuid())content_type = headers.get("content-type", "application/json")- if os.environ.get("MINISTACK_AGENTCORE_DOCKER") == "1":+ artifact = runtime.get("agentRuntimeArtifact", {}).get("containerConfiguration", {})+ if artifact.get("containerUri") and _docker_client() is not None:try:url = _container_invocations_url(runtime)except (RuntimeError, ValueError) as error:@@ -468,6 +469,23 @@ def _local_open(request, *, timeout):return _LOCAL_HTTP.open(request, timeout=timeout)+_docker = None+++def _docker_client():+ """A Docker client when the SDK is installed and the daemon answers, else None."""+ global _docker+ if _docker is None:+ try:+ import docker+ client = docker.from_env(timeout=60)+ client.ping()+ _docker = client+ except Exception:+ return None+ return _docker++def _container_key(runtime_id):return get_account_id(), get_region(), runtime_id@@ -519,13 +537,15 @@ def _container_invocations_url(runtime):raise RuntimeError(f"Could not inspect runtime container: {error}") from errorif container is None:client = None- labels = {"ministack": "agentcore",- "ministack.agentcore.runtime": runtime["agentRuntimeId"],- "ministack.agentcore.account": key[0],- "ministack.agentcore.region": key[1]}+ from ministack.core.container_reaper import own_labels++ labels = own_labels("agentcore", **{+ "ministack.agentcore.runtime": runtime["agentRuntimeId"],+ "ministack.agentcore.account": key[0],+ "ministack.agentcore.region": key[1]})try:import docker- client = docker.from_env(timeout=60)+ client = _docker_client() or docker.from_env(timeout=60)run_kwargs = {"environment": runtime.get("environmentVariables", {}),"labels": labels,diff --git a/ministack/services/lambda_svc.py b/ministack/services/lambda_svc.pyindex 800db387..b55cdb5c 100644--- a/ministack/services/lambda_svc.py+++ b/ministack/services/lambda_svc.py@@ -4089,11 +4089,7 @@ _CONTAINER_TRUSTSTORE_PATH = "/var/ministack/truststore.p12"def _wire_cognito_issuer_host(run_kwargs, container_env, mounts, runtime=""):- """Resolve the host a Cognito token's `iss` names to the gateway, and trust it.-- A verifier built from the pool id alone has no other way to reach us. Only- under USE_SSL=1: `iss` is https, so plain HTTP has nothing to resolve to.- """+ """Under USE_SSL=1, resolve the Cognito issuer hosts to the gateway and trust its cert."""from ministack.core import tls as _tlsif not _tls.use_ssl_enabled():@@ -4102,8 +4098,9 @@ def _wire_cognito_issuer_host(run_kwargs, container_env, mounts, runtime=""):cert_path, _key_path = _tls.resolve_tls_material()except SystemExit:return- run_kwargs.setdefault("extra_hosts", {}).setdefault(- _tls.cognito_idp_host(), "host-gateway")+ extra_hosts = run_kwargs.setdefault("extra_hosts", {})+ for host in _tls.cognito_idp_hosts():+ extra_hosts.setdefault(host, "host-gateway")if not os.path.exists(cert_path):return# NODE_EXTRA_CA_CERTS adds to node's roots; the other two replace the store.@@ -4904,8 +4901,7 @@ def _inflight_key(config: dict) -> str:account, region = _account_region_from_function_config(config)except Exception:account, region = get_account_id(), get_region()- # Reserved concurrency belongs to the function, not an individual- # published version or alias. All versions therefore share one counter.+ # Reserved concurrency is shared across all versions and aliases.return f"{account}:{region}:{config.get('FunctionName', '?')}"diff --git a/ministack/services/s3.py b/ministack/services/s3.pyindex bed909ae..09c1fc4b 100644--- a/ministack/services/s3.py+++ b/ministack/services/s3.py@@ -144,8 +144,6 @@ _bucket_request_payment_config = AccountScopedDict()_object_tags = AccountScopedDict()_object_acl = AccountScopedDict() # (bucket, key, version_id) -> stored ACL XML string-# (bucket, key, version_id) -> {annotation name: {payload, size, etag, last_modified, checksums,-# replication_status}}; version_id is None for the object without one, as for tags._object_annotations = AccountScopedDict()_object_versions = AccountScopedDict() # (bucket, key) -> [{version_id, obj_record}, ...]@@ -3433,30 +3431,20 @@ def _s3_event_to_eventbridge(event_name: str) -> tuple[str, str | None]:return detail_type, reason-# The notification record's `sequencer`: a hexadecimal string that grows with-# every create and delete, so "the event notification with the greater-# `sequencer` hexadecimal value is the event that occurred later" for one key-# (S3 User Guide, "Event message structure"). Taken when the write happens,-# not when the background thread builds the record, so it orders the writes.-# Seeded from the clock so values keep growing across a restart.+# Clock-seeded so sequencers keep growing across restarts._event_sequence = itertools.count(time.time_ns() // 1000)_EVENT_SEQUENCED_FAMILIES = ("s3:ObjectCreated:", "s3:ObjectRemoved:")def _next_event_sequencer(event_name: str) -> str | None:- """A sequencer for an object create or delete event; None for the others,- which the record carries no sequencer for ("only used with PUT and DELETE- requests")."""+ """Sequencer for create/delete events only; None otherwise."""if not event_name.startswith(_EVENT_SEQUENCED_FAMILIES):return Nonereturn f"{next(_event_sequence):018X}"def _event_object_key(key: str) -> str:- """The key as a notification record carries it: "The object key name value- is URL encoded. For example, `red flower.jpg` becomes `red+flower.jpg`"- (S3 User Guide, "Event message structure"); slashes are left as they are.- EventBridge details carry the key as stored."""+ """URL-encoded key for notification records (EventBridge keeps it raw)."""return _url_quote_plus(key, safe="/")@@ -3473,13 +3461,7 @@ def _fire_s3_event(object_extra: dict | None = None,record_extra: dict | None = None,) -> None:- """Build and deliver an S3 event notification. Best-effort — errors are logged.-- `version_id` is the version the write made or removed ("null or not present- if the bucket isn't versioning-enabled"), `sequencer` the one taken when it- happened, `object_extra` fields the record's `s3.object` gains for this- event (`hasObjectAnnotation` on a copy), and `record_extra` fields the record- itself gains (`objectAnnotation` on an annotation event)."""+ """Build and deliver an S3 event notification. Best-effort — errors are logged."""try:configs = _parse_notification_config(bucket_name)raw_xml = _bucket_notifications.get(bucket_name, "")@@ -4634,11 +4616,7 @@ def _record_object_version(bucket_name: str, key: str, prior_obj: dict | None, ooverwrite), preserved as the null version when it predates versioning."""versioning = _bucket_versioning.get(bucket_name)if versioning != "Enabled":- # An unversioned or suspended write replaces the object that has no- # version id, and "Overwriting an object replaces its annotations with- # whatever annotations the new version has": none, until CopyObject or- # PutObjectAnnotation gives it some. An Enabled write mints a new- # version, which has none of its own.+ # Overwriting the null version drops its annotations._object_annotations.pop((bucket_name, key, None), None)if versioning not in ("Enabled", "Suspended"):return None@@ -4756,8 +4734,6 @@ def _delete_object_version(bucket: dict, bucket_name: str, key: str, version_id:# Per-version tags and ACLs travel with the version being removed._object_tags.pop((bucket_name, key, version_id), None)_object_acl.pop((bucket_name, key, version_id), None)- # "Deleting a specific version ID deletes that version and all associated- # annotations." The null version's are keyed None, as its tags are read._object_annotations.pop((bucket_name, key, None if version_id == "null" else version_id), None)if not versions:@@ -4857,8 +4833,6 @@ def _delete_object(bucket_name: str, key: str, headers: dict | None = None, querexisted = key in bucket["objects"]bucket["objects"].pop(key, None)if existed:- # "s3:ObjectRemoved:DeleteMarkerCreated" is its own event type:- # s3:ObjectRemoved:Delete is for an object or version that is gone._fire_s3_event_async(bucket_name,key,@@ -5014,10 +4988,7 @@ def _copy_object(bucket_name: str, dest_key: str, headers: dict):if canned_acl and canned_acl not in _CANNED_OBJECT_ACLS:return _error("InvalidArgument", f"Invalid x-amz-acl value: {canned_acl}", 400)- # "CopyObject – Copies annotations from the source object by default",- # COPY or EXCLUDE (x-amz-object-annotation-directive, the botocore model's- # AnnotationDirective). A new checksum algorithm on the copy "applies to- # both the object and its annotations", so it must be one they can take.+ # A new checksum algorithm on the copy also applies to the annotations.annotation_directive = headers.get("x-amz-object-annotation-directive", "COPY").upper()if annotation_directive not in ("COPY", "EXCLUDE"):return _error("InvalidArgument", f"Invalid annotation directive: {annotation_directive}", 400)@@ -5152,8 +5123,6 @@ def _copy_object(bucket_name: str, dest_key: str, headers: dict):if src_tags:pending_dest_tags = dict(src_tags)- # Annotations of the version copied, for the version the copy makes. Their- # checksums are the source's unless the copy names another algorithm.pending_dest_annotations: dict | None = Noneif annotation_directive == "COPY":src_annotations = _object_annotations.get((src_bucket_name, src_key, src_obj.get("version_id")))@@ -5197,8 +5166,6 @@ def _copy_object(bucket_name: str, dest_key: str, headers: dict):size=dest_obj["size"],etag=new_etag,version_id=version_id,- # "If the copied object has annotations, the event includes a- # hasObjectAnnotation field set to true."object_extra={"hasObjectAnnotation": bool(pending_dest_annotations)},)@@ -5371,26 +5338,12 @@ def _delete_object_tagging(bucket_name: str, key: str, query_params: dict | None# ---------------------------------------------------------------------------# Object annotations# ----------------------------------------------------------------------------# PutObjectAnnotation, GetObjectAnnotation, ListObjectAnnotations and-# DeleteObjectAnnotation. Evidence: the S3 API Reference pages for the four-# operations, the S3 User Guide's "Annotating your objects", "Event-# notification types and destinations" and "Event message structure" pages,-# and the botocore s3 model (1.43.63). Not validated against a real AWS-# account. Where the documentation is silent the choice is marked inference.-#-# An annotation belongs to one object version: a new version starts with none,-# deleting a version deletes its annotations, and a delete marker leaves the-# annotations of the version beneath it alone. Writing one changes nothing-# about the object (not its ETag, not its version) and sends only the-# s3:ObjectAnnotation:* events._ANNOTATION_MAX_PER_VERSION = 1000_ANNOTATION_MAX_NAME_BYTES = 512_ANNOTATION_MAX_PAYLOAD = 1024 * 1024_ANNOTATION_NAME_PUNCTUATION = frozenset("0123456789_.-")-# The algorithms an annotation takes. MiniStack computes all but the two-# XXHASH3 variants, which have no stdlib or small-table implementation; like-# PutObject's unverifiable algorithms they are refused, never stored unchecked.+# XXHASH3/XXHASH128 have no stdlib implementation and are refused._ANNOTATION_CHECKSUM_ALGORITHMS = ("CRC32","CRC32C",@@ -5404,9 +5357,7 @@ _ANNOTATION_CHECKSUM_ALGORITHMS = ("XXHASH128",)-# XXH64 (the xxHash specification, "XXH64 algorithm description"): plain-# 64-bit arithmetic, so it stays in the stdlib. Check values in the tests:-# xxh64(b"") == 0xEF46DB3751D8E999, xxh64(b"abc") == 0x44BC2CF5AD770999.+# XXH64 per the xxHash spec._XXH64_P1 = 0x9E3779B185EBCA87_XXH64_P2 = 0xC2B2AE3D27D4EB4F_XXH64_P3 = 0x165667B19E3779F9@@ -5463,8 +5414,7 @@ def _xxh64(data: bytes, seed: int = 0) -> int:def _annotation_checksum(algorithm: str, data: bytes) -> str | None:- """Base64 checksum of an annotation payload, or None for an algorithm that- is not an annotation's or that MiniStack cannot compute."""+ """Base64 checksum of a payload, or None if the algorithm is unsupported."""algo = (algorithm or "").upper().replace("_", "")if algo not in _ANNOTATION_CHECKSUM_ALGORITHMS:return None@@ -5487,14 +5437,7 @@ def _annotation_checksum_unsupported(algorithm: str) -> tuple:def _resolve_annotation_checksums(payload: bytes, headers: dict):- """The checksum stored with an annotation, and the refusal for a bad one.-- One algorithm per payload: the one the client sent a value for, or named- in x-amz-sdk-checksum-algorithm, or else CRC64NVME ("If the annotation- doesn't have a specified checksum algorithm or checksum value, Amazon S3- uses the CRC-64/NVME algorithm"). A value sent is checked against the- payload (BadDigest), as PutObject checks its body; so is Content-MD5.- Returns ``(checksums, error_or_None)``."""+ """Return ``(checksums, error_or_None)``; CRC64NVME when none is named."""provided = {alg: headers[f"x-amz-checksum-{alg.lower()}"]for alg in _ANNOTATION_CHECKSUM_ALGORITHMS@@ -5529,9 +5472,7 @@ def _resolve_annotation_checksums(payload: bytes, headers: dict):def _annotation_name_error(name: str) -> tuple | None:- """The annotation naming rules: 1 to 512 bytes of letters (any language),- digits, underscore, period and hyphen, not all whitespace, not starting- with "aws" or "s3" in any case (S3 User Guide, "Annotation naming rules")."""+ """Validate an annotation name against S3's naming rules."""if not name or not name.strip():return _error("InvalidAnnotationName", "The annotation name you provided is invalid.", 400)if len(name.encode("utf-8")) > _ANNOTATION_MAX_NAME_BYTES:@@ -5544,15 +5485,7 @@ def _annotation_name_error(name: str) -> tuple | None:def _annotation_target(bucket_name: str, key: str, query_params: dict):- """The object version an annotation operation acts on.-- Returns ``((bucket, version_key, record), None)`` or ``(None, error)``:- `version_key` keys the version's annotations (None for the object without- a version id, as for tags) and `record` is the current-object record or- version entry, for its ETag and encryption. With no ``versionId`` it is- the current object; a key whose current version is a delete marker is- NoSuchKey, as a GetObject of it is. A ``versionId`` naming a delete marker- is refused as a read of one is (inference: the documentation is silent)."""+ """Resolve the object version an annotation call acts on."""bucket = _ensure_bucket(bucket_name)if bucket is None:return None, _no_such_bucket(bucket_name)@@ -5578,9 +5511,7 @@ def _annotation_target(bucket_name: str, key: str, query_params: dict):def _annotation_version_header(bucket_name: str, version_key: str | None) -> dict:- """x-amz-object-version-id: the version annotated. The object without a- version id is "null" in a bucket that has had versioning; in one that- never has, the header is left off, as x-amz-version-id is."""+ """x-amz-object-version-id for the annotated version."""if version_key:return {"x-amz-object-version-id": version_key}if _bucket_versioning.get(bucket_name) in ("Enabled", "Suspended"):@@ -5589,9 +5520,7 @@ def _annotation_version_header(bucket_name: str, version_key: str | None) -> dicdef _annotation_sse_headers(record: dict) -> dict:- """Annotations take the parent object's encryption: SSE-KMS and DSSE-KMS- with the object's key, and SSE-S3 otherwise, including for an object with- no server-side encryption (S3 User Guide, "Encryption")."""+ """Annotations inherit the object's encryption; SSE-S3 when it has none."""sse = _stored_sse_headers(record)out = {"x-amz-server-side-encryption": sse.get("x-amz-server-side-encryption") or "AES256"}if sse.get("x-amz-server-side-encryption-aws-kms-key-id"):@@ -5600,8 +5529,7 @@ def _annotation_sse_headers(record: dict) -> dict:def _annotation_if_match(headers: dict, record: dict) -> tuple | None:- """x-amz-object-if-match: the operation "only succeeds if the object's ETag- matches the provided value"; it checks the object, never the annotation."""+ """x-amz-object-if-match checks the object's ETag."""expected = headers.get("x-amz-object-if-match")if expected and expected != "*" and expected.strip('"') != (record.get("etag") or "").strip('"'):return _error("PreconditionFailed", "At least one of the pre-conditions you specified did not hold", 412)@@ -5616,13 +5544,10 @@ def _annotation_checksum_headers(entry: dict) -> dict:def _annotation_lock_error(bucket_name: str, key: str, headers: dict) -> tuple | None:- """Object Lock over an annotation write (put, overwrite or delete), as AWS answers it.-- Observed on AWS (ap-southeast-2, 2026-09-28): governance retention refuses- both unless x-amz-bypass-governance-retention is true; compliance retention- and a legal hold refuse both whatever the header says. Each refusal is 403- AccessDenied, naming retention or the legal hold.- """+ """Object Lock refusal for an annotation write (observed on AWS)."""+ from ministack.app import AUTH+ if not AUTH:+ return Noneif _object_legal_hold.get((bucket_name, key)) == "ON":return _error("AccessDenied", "Access Denied because object protected by object lock legal hold.", 403)retention = _object_retention.get((bucket_name, key))@@ -5681,9 +5606,7 @@ def _put_object_annotation(bucket_name: str, key: str, body: bytes, headers: dic"last_modified": now_iso(),"checksums": checksums,}- # "If you have S3 Replication configured on your bucket, Amazon S3- # replicates annotations automatically": onto the replica of the version,- # when it is the one this bucket replicated.+ # Annotations replicate onto the version's replica.current = bucket["objects"].get(key)if current is not None and current.get("version_id") == version_key and current.get("_replica"):replica_bucket, replica_version = current["_replica"]@@ -5772,8 +5695,7 @@ def _list_object_annotations(bucket_name: str, key: str, query_params: dict):except (ValueError, UnicodeDecodeError):return _error("InvalidArgument", "The continuation token provided is incorrect.", 400)- # Names in UTF-8 binary order, as S3 lists keys (inference: the operation- # documents no order); the token is the last name returned.+ # UTF-8 binary order (undocumented); the token is the last name returned.annotations = _object_annotations.get((bucket_name, key, version_key)) or {}names = sorted((n for n in annotations if n.startswith(prefix) and (not after or n.encode() > after.encode())),@@ -5827,8 +5749,7 @@ def _delete_object_annotation(bucket_name: str, key: str, headers: dict, query_pstore_key = (bucket_name, key, version_key)annotations = dict(_object_annotations.get(store_key) or {})version_headers = _annotation_version_header(bucket_name, version_key)- # Deleting an annotation that is not there succeeds, as the operation- # documents no NoSuchAnnotation; only a real delete sends the event.+ # A missing annotation still answers 204; only a real delete fires the event.if annotations.pop(name, None) is not None:if annotations:_object_annotations[store_key] = annotations@@ -6635,8 +6556,7 @@ def _delete_objects(bucket_name: str, body: bytes, headers: dict = None):# S3 reports the delete as successful even if the version was absent._found, was_marker = _delete_object_version(bucket, bucket_name, k, version_id)deleted.append({"key": k, "version_id": version_id, "was_marker": was_marker})- # "notification when an object or a batch of objects is removed":- # each entry sends what the single-object DELETE would.+ # Each entry sends the single-object DELETE's event.if _found:_fire_s3_event_async(bucket_name, k, "s3:ObjectRemoved:Delete", version_id=version_id)elif _bucket_versioning.get(bucket_name) in ("Enabled", "Suspended"):diff --git a/ministack/services/sns.py b/ministack/services/sns.pyindex 031834f0..b4ec96e2 100644--- a/ministack/services/sns.py+++ b/ministack/services/sns.py@@ -531,15 +531,15 @@ def _add_permission(params):topic_arn = topic["arn"]label = _p(params, "Label")if not label:- return _error("InvalidParameterException",+ return _error("InvalidParameter","Label is required", 400)account_ids = _member_list(params, "AWSAccountId")actions = _member_list(params, "ActionName")if not account_ids:- return _error("InvalidParameterException",+ return _error("InvalidParameter","AWSAccountId is required", 400)if not actions:- return _error("InvalidParameterException",+ return _error("InvalidParameter","ActionName is required", 400)raw = topic["attributes"].get("Policy") or ""@@ -555,7 +555,7 @@ def _add_permission(params):policy["Statement"] = statementsif any(s.get("Sid") == label for s in statements):- return _error("InvalidParameterException",+ return _error("InvalidParameter",f"Value {label} for parameter Label is invalid. "f"Reason: Already exists.", 400)@@ -576,7 +576,7 @@ def _remove_permission(params):return errlabel = _p(params, "Label")if not label:- return _error("InvalidParameterException",+ return _error("InvalidParameter","Label is required", 400)raw = topic["attributes"].get("Policy") or ""diff --git a/ministack/services/sqs.py b/ministack/services/sqs.pyindex 653eeb8c..3acfc3c2 100644--- a/ministack/services/sqs.py+++ b/ministack/services/sqs.py@@ -146,8 +146,7 @@ def _enforce_queue_policy(q: dict, iam_action: str) -> None:if not resource_policy_allows(q["attributes"].get("Policy") or "", ctx,owner is None or owner == caller):raise _Err("AccessDenied",- f"User: {ctx.principal_arn} is not authorized to perform: "- f"{iam_action} on resource: {queue_arn}", 403)+ f"Access to the resource https://sqs.{get_region()}.amazonaws.com/ is denied.", 403)def queue_policy_allows(queue_arn: str, service: str,diff --git a/ministack/services/sts.py b/ministack/services/sts.pyindex 613368ce..7d091320 100644--- a/ministack/services/sts.py+++ b/ministack/services/sts.py@@ -92,7 +92,8 @@ def _caller_identity(def _credential_error_response(error: CredentialResolutionError):- code = "ExpiredToken" if error.code == "ExpiredTokenException" else error.code+ code = {"ExpiredTokenException": "ExpiredToken",+ "UnrecognizedClientException": "InvalidClientTokenId"}.get(error.code, error.code)return _error(403, code, error.message, ns="sts")diff --git a/tests/test_bedrock_agentcore.py b/tests/test_bedrock_agentcore.pyindex 6f3e84fd..6c0c3eb1 100644--- a/tests/test_bedrock_agentcore.py+++ b/tests/test_bedrock_agentcore.py@@ -3,10 +3,15 @@Covers the v1 surface: agent-runtime CRUD, runtime-endpoint CRUD,InvokeAgentRuntime (deterministic echo), region isolation, and validation."""+import asyncioimport jsonimport os+import sys+import threading+import typesimport urllib.requestimport uuid as _uuid_mod+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServerimport boto3import pytest@@ -14,6 +19,9 @@ from botocore.config import Configfrom botocore.exceptions import ClientErrorfrom botocore.utils import parse_timestamp+from ministack.core.responses import StreamingResponse+from ministack.services import bedrock_agentcore as agentcore+ENDPOINT = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566")_ARTIFACT = {"containerConfiguration": {"containerUri": "0.dkr.ecr.us-east-1.amazonaws.com/agent:latest"}}@@ -32,9 +40,14 @@ def _client(service, region="us-east-1"):)-def _create(ctl, name):+_CODE_ARTIFACT = {"codeConfiguration": {+ "code": {"s3": {"bucket": "agent-code", "prefix": "agent.zip"}},+ "runtime": "PYTHON_3_12", "entryPoint": ["main.py"]}}+++def _create(ctl, name, artifact=_ARTIFACT):return ctl.create_agent_runtime(- agentRuntimeName=name, agentRuntimeArtifact=_ARTIFACT,+ agentRuntimeName=name, agentRuntimeArtifact=artifact,roleArn=_ROLE, networkConfiguration=_NET,)@@ -111,7 +124,8 @@ def test_agentcore_endpoint_lifecycle():def test_agentcore_invoke_returns_deterministic_echo():ctl = _client("bedrock-agentcore-control")rt = _client("bedrock-agentcore")- rid_resp = _create(ctl, f"rt_{_uuid_mod.uuid4().hex[:8]}")+ # A code artifact never starts a container, so this echoes with or without Docker.+ rid_resp = _create(ctl, f"rt_{_uuid_mod.uuid4().hex[:8]}", _CODE_ARTIFACT)arn = rid_resp["agentRuntimeArn"]try:resp = rt.invoke_agent_runtime(@@ -217,3 +231,330 @@ def test_agentcore_timestamps_are_iso8601_strings_on_the_wire():where="ListAgentRuntimeEndpoints.")finally:ctl.delete_agent_runtime(agentRuntimeId=rid)+++# --- InvokeAgentRuntime against the runtime container (in-process) ---++def _runtime(name="container_test", image="example.local/worker:1"):+ status, _, body = agentcore._create_agent_runtime(json.dumps({+ "agentRuntimeName": name,+ "agentRuntimeArtifact": {"containerConfiguration": {"containerUri": image}},+ "roleArn": "arn:aws:iam::000000000000:role/agentcore",+ "networkConfiguration": {"networkMode": "PUBLIC"},+ "environmentVariables": {"WORKER_MODE": "test"},+ }).encode())+ assert status == 200+ return json.loads(body)+++def _invoke(arn, headers=None, body=b"{}"):+ return asyncio.run(agentcore.handle_request(+ "POST", f"/runtimes/{arn}/invocations",+ headers or {"content-type": "application/json"}, body, {},+ ))+++def _fake_docker(monkeypatch, port, *, fail=False, network=None,+ fail_on_start=False, fail_after_create=False, missing_image=False):+ started = []+ removed = []+ created = []+ pulled = []++ class ImageNotFound(Exception):+ pass++ class Container:+ status = "running"+ attrs = {"NetworkSettings": {+ "Ports": {"8080/tcp": [{"HostPort": str(port)}]},+ "Networks": {network: {"IPAddress": "127.0.0.1"}} if network else {},+ }}++ def reload(self):+ pass++ def start(self):+ if fail_on_start:+ raise TimeoutError("Docker start timed out")++ def remove(self, force=False):+ removed.append(force)++ class Containers:+ def get(self, _name):+ if not network:+ raise RuntimeError("MiniStack is not in Docker")+ return Container()++ def create(self, image, **kwargs):+ if fail:+ raise RuntimeError("image unavailable")+ if missing_image and not pulled:+ raise ImageNotFound(image)+ started.append((image, kwargs))+ container = Container()+ created.append(container)+ if fail_after_create:+ raise TimeoutError("Docker create timed out after allocation")+ return container++ def list(self, **_kwargs):+ return created++ fake_docker = types.SimpleNamespace(+ errors=types.SimpleNamespace(ImageNotFound=ImageNotFound),+ from_env=lambda **_kwargs: types.SimpleNamespace(+ containers=Containers(),+ images=types.SimpleNamespace(pull=lambda image: pulled.append(image)),+ ping=lambda: True,+ ),+ )+ monkeypatch.setitem(sys.modules, "docker", fake_docker)+ from ministack.services import bedrock_agentcore+ monkeypatch.setattr(bedrock_agentcore, "_docker", None)+ return started, removed+++def _worker(response_status=200):+ requests = []++ class Worker(BaseHTTPRequestHandler):+ def do_GET(self):+ self.send_response(200 if self.path == "/ping" else 404)+ self.end_headers()++ def do_POST(self):+ requests.append((self.path, dict(self.headers),+ self.rfile.read(int(self.headers["Content-Length"]))))+ payload = b'{"evidence":[{"id":"deployment-1"}]}'+ self.send_response(response_status)+ self.send_header("Content-Type", "application/json")+ self.send_header("Content-Length", str(len(payload)))+ self.end_headers()+ self.wfile.write(payload)++ def log_message(self, *_args):+ pass++ server = ThreadingHTTPServer(("127.0.0.1", 0), Worker)+ thread = threading.Thread(target=server.serve_forever, daemon=True)+ thread.start()+ return server, thread, requests+++async def _discard(_message):+ pass+++def test_container_image_invoked_and_removed(monkeypatch):+ server, thread, requests = _worker()+ started, removed = _fake_docker(monkeypatch, server.server_port)+ runtime = _runtime()+ try:+ headers = {"content-type": "application/json", "authorization": "secret",+ "x-amzn-bedrock-agentcore-runtime-session-id": "session-1"}+ status, response_headers, body = _invoke(runtime["agentRuntimeArn"], headers,+ b'{"prompt":"why?"}')+ assert status == 200+ assert isinstance(body, StreamingResponse)+ messages = []++ async def send(message):+ messages.append(message)++ asyncio.run(body.runner(send, None))+ assert json.loads(b"".join(m["body"] for m in messages)) == {+ "evidence": [{"id": "deployment-1"}]}+ assert response_headers["x-amzn-bedrock-agentcore-runtime-session-id"] == "session-1"+ assert started[0][0] == "example.local/worker:1"+ assert started[0][1]["ports"] == {"8080/tcp": ("127.0.0.1", None)}+ assert started[0][1]["environment"] == {"WORKER_MODE": "test"}+ from ministack.core.container_reaper import INSTANCE_LABEL+ assert started[0][1]["labels"]["ministack"] == "agentcore"+ assert INSTANCE_LABEL in started[0][1]["labels"]+ path, forwarded, payload = requests[0]+ assert path == "/invocations" and payload == b'{"prompt":"why?"}'+ assert "authorization" not in {key.lower() for key in forwarded}+ _, _, second_body = _invoke(runtime["agentRuntimeArn"])+ asyncio.run(second_body.runner(_discard, None))+ assert len(started) == 1+ finally:+ agentcore._delete_agent_runtime(runtime["agentRuntimeId"])+ server.shutdown()+ server.server_close()+ thread.join()+ assert removed == [True]+++def test_container_joins_ministack_network(monkeypatch):+ started, _ = _fake_docker(monkeypatch, 8080, network="ministack-net")+ class Healthy:+ def __enter__(self):+ return self++ def __exit__(self, *_args):+ pass++ monkeypatch.setattr(agentcore, "_local_open", lambda *_args, **_kwargs: Healthy())+ runtime = _runtime("network_test")+ try:+ url = agentcore._container_invocations_url(agentcore._runtimes[runtime["agentRuntimeId"]])+ assert url == "http://127.0.0.1:8080/invocations"+ assert started[0][1]["network"] == "ministack-net"+ assert "ports" not in started[0][1]+ finally:+ agentcore._delete_agent_runtime(runtime["agentRuntimeId"])+++def test_worker_error_is_runtime_client_error(monkeypatch):+ server, thread, _ = _worker(400)+ _fake_docker(monkeypatch, server.server_port)+ runtime = _runtime("worker_error")+ try:+ status, headers, body = _invoke(runtime["agentRuntimeArn"])+ assert status == 424+ assert headers["x-amzn-errortype"] == "RuntimeClientError"+ assert json.loads(body)["__type"] == "RuntimeClientError"+ finally:+ agentcore._delete_agent_runtime(runtime["agentRuntimeId"])+ server.shutdown()+ server.server_close()+ thread.join()+++def test_container_start_failure_is_not_echo(monkeypatch):+ _fake_docker(monkeypatch, 1, fail=True)+ runtime = _runtime("image_failure")+ try:+ status, headers, body = _invoke(runtime["agentRuntimeArn"])+ assert status == 424+ assert headers["x-amzn-errortype"] == "RuntimeClientError"+ assert b"image unavailable" in body+ finally:+ agentcore._delete_agent_runtime(runtime["agentRuntimeId"])+++def test_container_is_removed_if_start_times_out(monkeypatch):+ _, removed = _fake_docker(monkeypatch, 1, fail_on_start=True)+ runtime = _runtime("start_timeout")+ try:+ status, _, _ = _invoke(runtime["agentRuntimeArn"])+ assert status == 424+ assert removed == [True]+ finally:+ agentcore._delete_agent_runtime(runtime["agentRuntimeId"])+++def test_missing_image_is_pulled_then_started(monkeypatch):+ server, thread, _ = _worker()+ started, _ = _fake_docker(monkeypatch, server.server_port, missing_image=True)+ runtime = _runtime("pull_image")+ try:+ status, _, response = _invoke(runtime["agentRuntimeArn"])+ assert status == 200+ asyncio.run(response.runner(_discard, None))+ assert [image for image, _ in started] == ["example.local/worker:1"]+ finally:+ agentcore._delete_agent_runtime(runtime["agentRuntimeId"])+ server.shutdown()+ server.server_close()+ thread.join()+++def test_orphan_is_removed_if_create_times_out(monkeypatch):+ _, removed = _fake_docker(monkeypatch, 1, fail_after_create=True)+ runtime = _runtime("create_timeout")+ try:+ status, _, _ = _invoke(runtime["agentRuntimeArn"])+ assert status == 424+ assert removed == [True]+ finally:+ agentcore._delete_agent_runtime(runtime["agentRuntimeId"])+++def test_connection_reset_maps_to_runtime_client_error(monkeypatch):+ server, thread, _ = _worker()+ _fake_docker(monkeypatch, server.server_port)+ runtime = _runtime("connection_reset")+ try:+ agentcore._container_invocations_url(agentcore._runtimes[runtime["agentRuntimeId"]])+ monkeypatch.setattr(agentcore, "_local_open", lambda *_args, **_kwargs:+ (_ for _ in ()).throw(ConnectionResetError("peer reset")))+ status, headers, _ = _invoke(runtime["agentRuntimeArn"])+ assert status == 424+ assert headers["x-amzn-errortype"] == "RuntimeClientError"+ finally:+ agentcore._delete_agent_runtime(runtime["agentRuntimeId"])+ server.shutdown()+ server.server_close()+ thread.join()+++def test_stream_failure_leaves_response_truncated(monkeypatch):+ class BrokenResponse:+ status = 200+ headers = {"Content-Type": "text/event-stream"}+ closed = False+ reads = 0++ def read1(self, _size):+ self.reads += 1+ if self.reads == 1:+ return b"first\n"+ raise ConnectionResetError("stream interrupted")++ def close(self):+ self.closed = True++ response = BrokenResponse()+ monkeypatch.setattr(agentcore, "_local_open", lambda *_args, **_kwargs: response)+ status, _, stream = agentcore._invoke_container(+ "http://127.0.0.1:8080/invocations", b"{}", {}, "application/json", "session-1")+ assert status == 200+ frames = []++ async def send(frame):+ frames.append(frame)++ asyncio.run(stream.runner(send, None))+ assert frames == [{"type": "http.response.body", "body": b"first\n", "more_body": True}]+ assert response.closed+++def test_update_restarts_image(monkeypatch):+ server, thread, _ = _worker()+ started, removed = _fake_docker(monkeypatch, server.server_port)+ runtime = _runtime("update_image")+ try:+ status, _, response = _invoke(runtime["agentRuntimeArn"])+ assert status == 200+ asyncio.run(response.runner(_discard, None))+ agentcore._update_agent_runtime(runtime["agentRuntimeId"], json.dumps({+ "agentRuntimeArtifact": {"containerConfiguration": {"containerUri": "example.local/worker:2"}},+ "roleArn": "arn:aws:iam::000000000000:role/agentcore",+ "networkConfiguration": {"networkMode": "PUBLIC"},+ }).encode())+ assert removed == [True]+ current = agentcore._runtimes[runtime["agentRuntimeId"]]+ status, _, response = _invoke(current["agentRuntimeArn"])+ assert status == 200+ assert [image for image, _ in started] == ["example.local/worker:1", "example.local/worker:2"]+ asyncio.run(response.runner(_discard, None))+ finally:+ agentcore._delete_agent_runtime(runtime["agentRuntimeId"])+ server.shutdown()+ server.server_close()+ thread.join()+++def test_without_docker_the_invocation_echoes(monkeypatch):+ monkeypatch.setitem(sys.modules, "docker", None)+ monkeypatch.setattr(agentcore, "_docker", None)+ runtime = _runtime(name="no_docker_echo")+ try:+ status, _, body = _invoke(runtime["agentRuntimeArn"], body=b'{"prompt":"hi"}')+ assert status == 200+ assert json.loads(body)["input"] == {"prompt": "hi"}+ finally:+ agentcore._delete_agent_runtime(runtime["agentRuntimeId"])diff --git a/tests/test_bedrock_agentcore_container.py b/tests/test_bedrock_agentcore_container.pydeleted file mode 100644index 5cba2cc4..00000000--- a/tests/test_bedrock_agentcore_container.py+++ /dev/null@@ -1,319 +0,0 @@-"""AgentCore container invocation contract, without a live AWS account."""--import asyncio-import json-import sys-import threading-import types-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer--from ministack.core.responses import StreamingResponse-from ministack.services import bedrock_agentcore as agentcore---def _runtime(name="container_test", image="example.local/worker:1"):- status, _, body = agentcore._create_agent_runtime(json.dumps({- "agentRuntimeName": name,- "agentRuntimeArtifact": {"containerConfiguration": {"containerUri": image}},- "roleArn": "arn:aws:iam::000000000000:role/agentcore",- "networkConfiguration": {"networkMode": "PUBLIC"},- "environmentVariables": {"WORKER_MODE": "test"},- }).encode())- assert status == 200- return json.loads(body)---def _invoke(arn, headers=None, body=b"{}"):- return asyncio.run(agentcore.handle_request(- "POST", f"/runtimes/{arn}/invocations",- headers or {"content-type": "application/json"}, body, {},- ))---def _fake_docker(monkeypatch, port, *, fail=False, network=None,- fail_on_start=False, fail_after_create=False, missing_image=False):- started = []- removed = []- created = []- pulled = []-- class ImageNotFound(Exception):- pass-- class Container:- status = "running"- attrs = {"NetworkSettings": {- "Ports": {"8080/tcp": [{"HostPort": str(port)}]},- "Networks": {network: {"IPAddress": "127.0.0.1"}} if network else {},- }}-- def reload(self):- pass-- def start(self):- if fail_on_start:- raise TimeoutError("Docker start timed out")-- def remove(self, force=False):- removed.append(force)-- class Containers:- def get(self, _name):- if not network:- raise RuntimeError("MiniStack is not in Docker")- return Container()-- def create(self, image, **kwargs):- if fail:- raise RuntimeError("image unavailable")- if missing_image and not pulled:- raise ImageNotFound(image)- started.append((image, kwargs))- container = Container()- created.append(container)- if fail_after_create:- raise TimeoutError("Docker create timed out after allocation")- return container-- def list(self, **_kwargs):- return created-- fake_docker = types.SimpleNamespace(- errors=types.SimpleNamespace(ImageNotFound=ImageNotFound),- from_env=lambda **_kwargs: types.SimpleNamespace(- containers=Containers(),- images=types.SimpleNamespace(pull=lambda image: pulled.append(image)),- ),- )- monkeypatch.setitem(sys.modules, "docker", fake_docker)- monkeypatch.setenv("MINISTACK_AGENTCORE_DOCKER", "1")- return started, removed---def _worker(response_status=200):- requests = []-- class Worker(BaseHTTPRequestHandler):- def do_GET(self):- self.send_response(200 if self.path == "/ping" else 404)- self.end_headers()-- def do_POST(self):- requests.append((self.path, dict(self.headers),- self.rfile.read(int(self.headers["Content-Length"]))))- payload = b'{"evidence":[{"id":"deployment-1"}]}'- self.send_response(response_status)- self.send_header("Content-Type", "application/json")- self.send_header("Content-Length", str(len(payload)))- self.end_headers()- self.wfile.write(payload)-- def log_message(self, *_args):- pass-- server = ThreadingHTTPServer(("127.0.0.1", 0), Worker)- thread = threading.Thread(target=server.serve_forever, daemon=True)- thread.start()- return server, thread, requests---async def _discard(_message):- pass---def test_container_image_invoked_and_removed(monkeypatch):- server, thread, requests = _worker()- started, removed = _fake_docker(monkeypatch, server.server_port)- runtime = _runtime()- try:- headers = {"content-type": "application/json", "authorization": "secret",- "x-amzn-bedrock-agentcore-runtime-session-id": "session-1"}- status, response_headers, body = _invoke(runtime["agentRuntimeArn"], headers,- b'{"prompt":"why?"}')- assert status == 200- assert isinstance(body, StreamingResponse)- messages = []-- async def send(message):- messages.append(message)-- asyncio.run(body.runner(send, None))- assert json.loads(b"".join(m["body"] for m in messages)) == {- "evidence": [{"id": "deployment-1"}]}- assert response_headers["x-amzn-bedrock-agentcore-runtime-session-id"] == "session-1"- assert started[0][0] == "example.local/worker:1"- assert started[0][1]["ports"] == {"8080/tcp": ("127.0.0.1", None)}- assert started[0][1]["environment"] == {"WORKER_MODE": "test"}- path, forwarded, payload = requests[0]- assert path == "/invocations" and payload == b'{"prompt":"why?"}'- assert "authorization" not in {key.lower() for key in forwarded}- _, _, second_body = _invoke(runtime["agentRuntimeArn"])- asyncio.run(second_body.runner(_discard, None))- assert len(started) == 1- finally:- agentcore._delete_agent_runtime(runtime["agentRuntimeId"])- server.shutdown()- server.server_close()- thread.join()- assert removed == [True]---def test_container_joins_ministack_network(monkeypatch):- started, _ = _fake_docker(monkeypatch, 8080, network="ministack-net")- class Healthy:- def __enter__(self):- return self-- def __exit__(self, *_args):- pass-- monkeypatch.setattr(agentcore, "_local_open", lambda *_args, **_kwargs: Healthy())- runtime = _runtime("network_test")- try:- url = agentcore._container_invocations_url(agentcore._runtimes[runtime["agentRuntimeId"]])- assert url == "http://127.0.0.1:8080/invocations"- assert started[0][1]["network"] == "ministack-net"- assert "ports" not in started[0][1]- finally:- agentcore._delete_agent_runtime(runtime["agentRuntimeId"])---def test_worker_error_is_runtime_client_error(monkeypatch):- server, thread, _ = _worker(400)- _fake_docker(monkeypatch, server.server_port)- runtime = _runtime("worker_error")- try:- status, headers, body = _invoke(runtime["agentRuntimeArn"])- assert status == 424- assert headers["x-amzn-errortype"] == "RuntimeClientError"- assert json.loads(body)["__type"] == "RuntimeClientError"- finally:- agentcore._delete_agent_runtime(runtime["agentRuntimeId"])- server.shutdown()- server.server_close()- thread.join()---def test_container_start_failure_is_not_echo(monkeypatch):- _fake_docker(monkeypatch, 1, fail=True)- runtime = _runtime("image_failure")- try:- status, headers, body = _invoke(runtime["agentRuntimeArn"])- assert status == 424- assert headers["x-amzn-errortype"] == "RuntimeClientError"- assert b"image unavailable" in body- finally:- agentcore._delete_agent_runtime(runtime["agentRuntimeId"])---def test_container_is_removed_if_start_times_out(monkeypatch):- _, removed = _fake_docker(monkeypatch, 1, fail_on_start=True)- runtime = _runtime("start_timeout")- try:- status, _, _ = _invoke(runtime["agentRuntimeArn"])- assert status == 424- assert removed == [True]- finally:- agentcore._delete_agent_runtime(runtime["agentRuntimeId"])---def test_missing_image_is_pulled_then_started(monkeypatch):- server, thread, _ = _worker()- started, _ = _fake_docker(monkeypatch, server.server_port, missing_image=True)- runtime = _runtime("pull_image")- try:- status, _, response = _invoke(runtime["agentRuntimeArn"])- assert status == 200- asyncio.run(response.runner(_discard, None))- assert [image for image, _ in started] == ["example.local/worker:1"]- finally:- agentcore._delete_agent_runtime(runtime["agentRuntimeId"])- server.shutdown()- server.server_close()- thread.join()---def test_orphan_is_removed_if_create_times_out(monkeypatch):- _, removed = _fake_docker(monkeypatch, 1, fail_after_create=True)- runtime = _runtime("create_timeout")- try:- status, _, _ = _invoke(runtime["agentRuntimeArn"])- assert status == 424- assert removed == [True]- finally:- agentcore._delete_agent_runtime(runtime["agentRuntimeId"])---def test_connection_reset_maps_to_runtime_client_error(monkeypatch):- server, thread, _ = _worker()- _fake_docker(monkeypatch, server.server_port)- runtime = _runtime("connection_reset")- try:- agentcore._container_invocations_url(agentcore._runtimes[runtime["agentRuntimeId"]])- monkeypatch.setattr(agentcore, "_local_open", lambda *_args, **_kwargs:- (_ for _ in ()).throw(ConnectionResetError("peer reset")))- status, headers, _ = _invoke(runtime["agentRuntimeArn"])- assert status == 424- assert headers["x-amzn-errortype"] == "RuntimeClientError"- finally:- agentcore._delete_agent_runtime(runtime["agentRuntimeId"])- server.shutdown()- server.server_close()- thread.join()---def test_stream_failure_leaves_response_truncated(monkeypatch):- class BrokenResponse:- status = 200- headers = {"Content-Type": "text/event-stream"}- closed = False- reads = 0-- def read1(self, _size):- self.reads += 1- if self.reads == 1:- return b"first\n"- raise ConnectionResetError("stream interrupted")-- def close(self):- self.closed = True-- response = BrokenResponse()- monkeypatch.setattr(agentcore, "_local_open", lambda *_args, **_kwargs: response)- status, _, stream = agentcore._invoke_container(- "http://127.0.0.1:8080/invocations", b"{}", {}, "application/json", "session-1")- assert status == 200- frames = []-- async def send(frame):- frames.append(frame)-- asyncio.run(stream.runner(send, None))- assert frames == [{"type": "http.response.body", "body": b"first\n", "more_body": True}]- assert response.closed---def test_update_restarts_image(monkeypatch):- server, thread, _ = _worker()- started, removed = _fake_docker(monkeypatch, server.server_port)- runtime = _runtime("update_image")- try:- status, _, response = _invoke(runtime["agentRuntimeArn"])- assert status == 200- asyncio.run(response.runner(_discard, None))- agentcore._update_agent_runtime(runtime["agentRuntimeId"], json.dumps({- "agentRuntimeArtifact": {"containerConfiguration": {"containerUri": "example.local/worker:2"}},- "roleArn": "arn:aws:iam::000000000000:role/agentcore",- "networkConfiguration": {"networkMode": "PUBLIC"},- }).encode())- assert removed == [True]- current = agentcore._runtimes[runtime["agentRuntimeId"]]- status, _, response = _invoke(current["agentRuntimeArn"])- assert status == 200- assert [image for image, _ in started] == ["example.local/worker:1", "example.local/worker:2"]- asyncio.run(response.runner(_discard, None))- finally:- agentcore._delete_agent_runtime(runtime["agentRuntimeId"])- server.shutdown()- server.server_close()- thread.join()diff --git a/tests/test_lambda.py b/tests/test_lambda.pyindex 9621da90..aa41337d 100644--- a/tests/test_lambda.py+++ b/tests/test_lambda.py@@ -13708,8 +13708,8 @@ def test_cognito_issuer_host_resolves_to_the_gateway_under_use_ssl(monkeypatch,run_kwargs, container_env, mounts = _wire(monkeypatch, USE_SSL="1", TMPDIR=str(tmp_path), MINISTACK_REGION="us-east-1")- assert run_kwargs["extra_hosts"] == {- "cognito-idp.us-east-1.amazonaws.com": "host-gateway"}+ assert run_kwargs["extra_hosts"]["cognito-idp.us-east-1.amazonaws.com"] == "host-gateway"+ assert run_kwargs["extra_hosts"]["cognito-idp.eu-west-2.amazonaws.com"] == "host-gateway"# NODE_EXTRA_CA_CERTS adds to node's roots, so it takes the bare certificate.assert container_env["NODE_EXTRA_CA_CERTS"] == lsvc._CONTAINER_CA_PATH# The other two replace the trust store, so they take the public roots withdiff --git a/tests/test_messaging_policies.py b/tests/test_messaging_policies.pydeleted file mode 100644index 6c5efbae..00000000--- a/tests/test_messaging_policies.py+++ /dev/null@@ -1,654 +0,0 @@-"""SQS queue-policy and SNS topic-policy enforcement.--The Policy attributes were stored (CRUD) but never evaluated. Under AUTH=true-they are now enforced:--- Same-account API calls pass unless the policy carries an explicit Deny.-- Cross-account API calls need an explicit Allow (AccessDenied /- AuthorizationError otherwise).-- S3→SQS, S3→SNS and SNS→SQS deliveries need an explicit Allow for the- calling service in the destination policy (with its SourceArn/SourceAccount- conditions); without one the delivery is dropped, like real AWS.-- S3 refuses a notification config whose SQS/SNS destination denies it- (InvalidArgument), unless SkipDestinationValidation skips the probe.--With AUTH=false every path stays permissive, like the rest of the identity-layer. The shared CI server runs with auth disabled, so these tests drive the-service modules in-process with AUTH monkeypatched (the test_cfn capabilities-pattern) instead of going over HTTP.-"""--import json-import re-import uuid--import pytest-from conftest import (- REGION,- sqs_policy_allow_s3,- sqs_policy_allow_sns,-)--from ministack.core.iam_evaluator import (- EvalContext,- evaluate_resource_policy,-)-from ministack.core.responses import request_scope-from ministack.services import s3 as s3_svc-from ministack.services import sns as sns_svc-from ministack.services import sqs as sqs_svc--ACCT_A = "111111111111"-ACCT_B = "222222222222"---def _reset_messaging_state():- from ministack.core.iam_evaluator import _request_caller_arn-- sqs_svc.reset()- sns_svc.reset()- s3_svc.reset()- _request_caller_arn.set("")---def auth(monkeypatch):- """AUTH=true with clean in-process SQS/SNS/S3 state."""- import ministack.app as app_mod-- monkeypatch.setattr(app_mod, "AUTH", True)- _reset_messaging_state()- yield- _reset_messaging_state()---def noauth(monkeypatch):- """AUTH=false with clean in-process SQS/SNS/S3 state."""- import ministack.app as app_mod-- monkeypatch.setattr(app_mod, "AUTH", False)- _reset_messaging_state()- yield- _reset_messaging_state()---def _uniq(prefix):- return f"{prefix}-{uuid.uuid4().hex[:8]}"---def _as(account, fn, *args, **kwargs):- """Call an in-process service function as *account* in the test region."""- with request_scope(account, REGION):- return fn(*args, **kwargs)---def _resp_detail(resp):- """``(status, code, message)`` of an in-process SNS/S3 response tuple."""- status, _headers, body = resp- text = body.decode("utf-8", errors="replace") if isinstance(body, bytes) else str(body)- code = re.search(r"<Code>([^<]*)</Code>", text)- msg = re.search(r"<Message>([^<]*)</Message>", text)- return (status, code.group(1) if code else "",- msg.group(1) if msg else "")---def _resp_code(resp):- """``(status, code)`` of an in-process SNS/S3 response tuple."""- status, code, _ = _resp_detail(resp)- return status, code---def _qarn(account, name):- return f"arn:aws:sqs:{REGION}:{account}:{name}"---def _tarn(account, name):- return f"arn:aws:sns:{REGION}:{account}:{name}"---def _make_queue(account, name, policy=None):- url = _as(account, sqs_svc._act_create_queue,- {"QueueName": name}, "")["QueueUrl"]- if policy is not None:- _as(account, sqs_svc._act_set_queue_attributes,- {"QueueUrl": url,- "Attributes": {"Policy": json.dumps(policy)}}, url)- return url---def _queue_bodies(arn):- q = sqs_svc._queue_by_arn(arn)- assert q is not None- return [m["body"] for m in q["messages"]]---def _deny_all(arn, action):- return {"Version": "2012-10-17", "Statement": [{- "Sid": "deny", "Effect": "Deny", "Principal": {"AWS": "*"},- "Action": action, "Resource": arn}]}---def _allow_account(arn, account, action):- return {"Version": "2012-10-17", "Statement": [{- "Sid": "x", "Effect": "Allow", "Principal": {"AWS": account},- "Action": action, "Resource": arn}]}---def _make_topic(account, name):- assert _resp_code(_as(account, sns_svc._create_topic,- {"Name": name})) == (200, "")- return _tarn(account, name)---def _set_topic_policy(account, arn, policy):- assert _resp_code(_as(account, sns_svc._set_topic_attributes, {- "TopicArn": arn, "AttributeName": "Policy",- "AttributeValue": json.dumps(policy)})) == (200, "")---def _topic_policy(arn):- topic = sns_svc._topic_by_arn_any_scope(arn)- assert topic is not None- return json.loads(topic["attributes"]["Policy"])---def _make_bucket(account, name):- assert _resp_code(_as(account, s3_svc._create_bucket,- name, b"", {})) == (200, "")---def _notif_xml(kind, arn):- tag = "Queue" if kind == "sqs" else "Topic"- return (- "<NotificationConfiguration>"- f"<{tag}Configuration><{tag}>{arn}</{tag}>"- "<Event>s3:ObjectCreated:*</Event>"- f"</{tag}Configuration>"- "</NotificationConfiguration>"- ).encode()---def _put_notif(account, bucket, kind, arn, skip=False):- headers = {"x-amz-skip-destination-validation": "true"} if skip else {}- return _as(account, s3_svc._put_bucket_notification,- bucket, _notif_xml(kind, arn), headers)---# ----------------------------------------------------------------------------# Unit: evaluate_resource_policy Service principals-# -----------------------------------------------------------------------------def _ctx(action="sqs:SendMessage", resource="arn:aws:sqs:us-east-1:1:q",- source_arn="", source_account="", principal="arn:aws:iam::1:root"):- return EvalContext(- principal_arn=principal,- principal_type="Service",- principal_account="1",- action=action,- resource_arn=resource,- region="us-east-1",- service_context={"aws:sourcearn": source_arn,- "aws:sourceaccount": source_account,- "aws:sourceowner": source_account},- )---def _svc_policy():- return {- "Version": "2012-10-17",- "Statement": [{- "Effect": "Allow",- "Principal": {"Service": "s3.amazonaws.com"},- "Action": "sqs:SendMessage",- "Resource": "arn:aws:sqs:us-east-1:1:q",- "Condition": {"ArnLike": {"aws:SourceArn": "arn:aws:s3:*:*:bkt"}},- }],- }---class TestEvaluateResourcePolicyService:- def test_service_principal_allows_matching_service(self):- assert evaluate_resource_policy(- _svc_policy(), _ctx(source_arn="arn:aws:s3:::bkt"),- service="s3.amazonaws.com").decision == "Allow"-- def test_service_principal_ignores_other_service(self):- assert evaluate_resource_policy(- _svc_policy(), _ctx(source_arn="arn:aws:s3:::bkt"),- service="sns.amazonaws.com").decision == "ImplicitDeny"-- def test_service_principal_ignored_without_service_arg(self):- # Legacy callers (Lambda layer policies, S3 bucket policies) keep the- # old behaviour: Service entries never match.- assert evaluate_resource_policy(- _svc_policy(), _ctx(source_arn="arn:aws:s3:::bkt")).decision == "ImplicitDeny"-- def test_wrong_source_arn_denies(self):- assert evaluate_resource_policy(- _svc_policy(), _ctx(source_arn="arn:aws:s3:::other"),- service="s3.amazonaws.com").decision == "ImplicitDeny"-- def test_aws_principal_still_matches(self):- doc = {"Statement": [{- "Effect": "Allow", "Principal": {"AWS": "222222222222"},- "Action": "sqs:SendMessage", "Resource": "*"}]}- ctx = _ctx(principal="arn:aws:iam::222222222222:root")- assert evaluate_resource_policy(doc, ctx).decision == "Allow"-- def test_star_principal_matches(self):- doc = {"Statement": [{- "Effect": "Allow", "Principal": "*",- "Action": "sqs:SendMessage", "Resource": "*"}]}- assert evaluate_resource_policy(doc, _ctx()).decision == "Allow"---# ----------------------------------------------------------------------------# Cross-account SQS-# -----------------------------------------------------------------------------class TestCrossAccountSqs:- def test_send_denied_without_policy(self, auth):- url = _make_queue(ACCT_A, _uniq("xpol-q"))- with pytest.raises(sqs_svc._Err) as exc:- _as(ACCT_B, sqs_svc._act_send_message,- {"QueueUrl": url, "MessageBody": "no-policy"}, url)- assert exc.value.code == "AccessDenied"- assert exc.value.status == 403-- def test_send_allowed_with_policy(self, auth):- name = _uniq("xpol-q")- url = _make_queue(ACCT_A, name, _allow_account(- _qarn(ACCT_A, name), ACCT_B, "sqs:SendMessage"))- _as(ACCT_B, sqs_svc._act_send_message,- {"QueueUrl": url, "MessageBody": "with-policy"}, url)- assert _queue_bodies(_qarn(ACCT_A, name)) == ["with-policy"]-- def test_send_allowed_via_add_permission(self, auth):- name = _uniq("xpol-q")- url = _make_queue(ACCT_A, name)- _as(ACCT_A, sqs_svc._act_add_permission,- {"QueueUrl": url, "Label": "xacct",- "AWSAccountIds": [ACCT_B], "Actions": ["SendMessage"]}, url)- _as(ACCT_B, sqs_svc._act_send_message,- {"QueueUrl": url, "MessageBody": "via-add-permission"}, url)- assert _queue_bodies(_qarn(ACCT_A, name)) == ["via-add-permission"]-- def test_explicit_deny_blocks_same_account(self, auth):- name = _uniq("xpol-q")- url = _make_queue(ACCT_A, name, _deny_all(- _qarn(ACCT_A, name), "sqs:SendMessage"))- with pytest.raises(sqs_svc._Err) as exc:- _as(ACCT_A, sqs_svc._act_send_message,- {"QueueUrl": url, "MessageBody": "deny-me"}, url)- assert exc.value.code == "AccessDenied"-- def test_get_queue_url_with_owner_account(self, auth):- name = _uniq("xpol-q")- url = _make_queue(ACCT_A, name, _allow_account(- _qarn(ACCT_A, name), ACCT_B, "sqs:GetQueueUrl"))- found = _as(ACCT_B, sqs_svc._act_get_queue_url,- {"QueueName": name,- "QueueOwnerAWSAccountId": ACCT_A}, "")["QueueUrl"]- assert found == url-- def test_get_queue_url_cross_account_denied_without_policy(self, auth):- name = _uniq("xpol-q")- _make_queue(ACCT_A, name)- with pytest.raises(sqs_svc._Err) as exc:- _as(ACCT_B, sqs_svc._act_get_queue_url,- {"QueueName": name,- "QueueOwnerAWSAccountId": ACCT_A}, "")- assert exc.value.code == "AccessDenied"- assert exc.value.status == 403-- def test_get_queue_url_same_account_denied_by_explicit_deny(self, auth):- name = _uniq("xpol-q")- _make_queue(ACCT_A, name, _deny_all(- _qarn(ACCT_A, name), "sqs:GetQueueUrl"))- with pytest.raises(sqs_svc._Err) as exc:- _as(ACCT_A, sqs_svc._act_get_queue_url,- {"QueueName": name}, "")- assert exc.value.code == "AccessDenied"- assert exc.value.status == 403-- def test_get_queue_url_same_account_passes_without_policy(self, auth):- name = _uniq("xpol-q")- url = _make_queue(ACCT_A, name)- assert _as(ACCT_A, sqs_svc._act_get_queue_url,- {"QueueName": name}, "") == {"QueueUrl": url}---# ----------------------------------------------------------------------------# Cross-account SNS (+ AddPermission/RemovePermission)-# -----------------------------------------------------------------------------class TestCrossAccountSns:- def test_publish_denied_by_default(self, auth):- arn = _make_topic(ACCT_A, _uniq("xpol-t"))- assert _resp_code(_as(ACCT_B, sns_svc._publish, {- "TopicArn": arn, "Message": "no-grant"})) == (403, "AuthorizationError")-- def test_publish_allowed_with_policy(self, auth):- name = _uniq("xpol-t")- arn = _make_topic(ACCT_A, name)- _set_topic_policy(ACCT_A, arn, _allow_account(- arn, ACCT_B, "sns:Publish"))- assert _resp_code(_as(ACCT_B, sns_svc._publish, {- "TopicArn": arn, "Message": "granted"})) == (200, "")- topic = sns_svc._topic_by_arn_any_scope(arn)- assert [m["message"] for m in topic["messages"]] == ["granted"]-- def test_explicit_deny_blocks_same_account(self, auth):- name = _uniq("xpol-t")- arn = _make_topic(ACCT_A, name)- _set_topic_policy(ACCT_A, arn, _deny_all(arn, "sns:Publish"))- assert _resp_code(_as(ACCT_A, sns_svc._publish, {- "TopicArn": arn, "Message": "deny-me"})) == (403, "AuthorizationError")-- def test_add_permission_grants_cross_account_publish(self, auth):- arn = _make_topic(ACCT_A, _uniq("xpol-t"))- assert _resp_code(_as(ACCT_A, sns_svc._add_permission, {- "TopicArn": arn, "Label": "perm-1",- "AWSAccountId.member.1": ACCT_B,- "ActionName.member.1": "Publish"})) == (200, "")- stmt = next(s for s in _topic_policy(arn)["Statement"]- if s.get("Sid") == "perm-1")- assert ACCT_B in stmt["Principal"]["AWS"]- assert "sns:Publish" in stmt["Action"]- assert _resp_code(_as(ACCT_B, sns_svc._publish, {- "TopicArn": arn,- "Message": "via-add-permission"})) == (200, "")-- def test_add_permission_rejects_duplicate_label(self, auth):- arn = _make_topic(ACCT_A, _uniq("xpol-t"))- params = {"TopicArn": arn, "Label": "dup",- "AWSAccountId.member.1": ACCT_B,- "ActionName.member.1": "Publish"}- assert _resp_code(_as(ACCT_A, sns_svc._add_permission, params)) == (200, "")- assert _resp_code(- _as(ACCT_A, sns_svc._add_permission, params)) == (- 400, "InvalidParameterException")-- def test_cross_account_tag_access_needs_grant(self, auth):- name = _uniq("xpol-t")- arn = _make_topic(ACCT_A, name)- assert _resp_code(_as(ACCT_B, sns_svc._list_tags_for_resource, {- "ResourceArn": arn})) == (403, "AuthorizationError")- _set_topic_policy(ACCT_A, arn, _allow_account(- arn, ACCT_B, ["sns:TagResource", "sns:ListTagsForResource"]))- assert _resp_code(_as(ACCT_B, sns_svc._tag_resource, {- "ResourceArn": arn, "Tags.member.1.Key": "k",- "Tags.member.1.Value": "v"})) == (200, "")- assert _resp_code(_as(ACCT_B, sns_svc._list_tags_for_resource, {- "ResourceArn": arn})) == (200, "")- assert sns_svc._topic_by_arn_any_scope(arn)["tags"] == {"k": "v"}-- def test_remove_permission_drops_statement(self, auth):- arn = _make_topic(ACCT_A, _uniq("xpol-t"))- _as(ACCT_A, sns_svc._add_permission, {- "TopicArn": arn, "Label": "drop",- "AWSAccountId.member.1": ACCT_B,- "ActionName.member.1": "Publish"})- assert _resp_code(_as(ACCT_B, sns_svc._publish, {- "TopicArn": arn, "Message": "before"})) == (200, "")- assert _resp_code(_as(ACCT_A, sns_svc._remove_permission, {- "TopicArn": arn, "Label": "drop"})) == (200, "")- assert all(s.get("Sid") != "drop"- for s in _topic_policy(arn)["Statement"])- assert _resp_code(_as(ACCT_B, sns_svc._publish, {- "TopicArn": arn, "Message": "after"})) == (403, "AuthorizationError")---# ----------------------------------------------------------------------------# S3 notifications gated on the destination policy-# -----------------------------------------------------------------------------def _bucket_and_queue(tag):- bucket, qname = _uniq(f"{tag}-bkt"), _uniq(f"{tag}-q")- _make_bucket(ACCT_A, bucket)- url = _make_queue(ACCT_A, qname)- return bucket, url, _qarn(ACCT_A, qname)---def _assert_put_rejected(bucket, kind, arn):- status, code, msg = _resp_detail(_put_notif(ACCT_A, bucket, kind, arn))- assert status == 400- assert code == "InvalidArgument"- assert "Unable to validate the following destination configurations" in msg---def _bucket_topic_queue(tag):- bucket = _uniq(f"{tag}-bkt")- _make_bucket(ACCT_A, bucket)- topic = _make_topic(ACCT_A, _uniq(f"{tag}-t"))- qname = _uniq(f"{tag}-q")- qurl = _make_queue(ACCT_A, qname,- sqs_policy_allow_sns(_qarn(ACCT_A, qname), topic))- assert _resp_code(_as(ACCT_A, sns_svc._subscribe, {- "TopicArn": topic, "Protocol": "sqs",- "Endpoint": _qarn(ACCT_A, qname)})) == (200, "")- return bucket, topic, qurl, _qarn(ACCT_A, qname)---class TestS3NotificationPolicies:- def test_s3_put_rejected_without_queue_policy(self, auth):- bucket, _url, arn = _bucket_and_queue("xpol")- _assert_put_rejected(bucket, "sqs", arn)-- def test_s3_put_rejected_by_unrelated_policy(self, auth):- bucket, url, arn = _bucket_and_queue("xpol")- _as(ACCT_A, sqs_svc._act_set_queue_attributes,- {"QueueUrl": url, "Attributes": {"Policy": json.dumps(- _allow_account(arn, ACCT_B, "sqs:SendMessage"))}}, url)- _assert_put_rejected(bucket, "sqs", arn)-- def test_s3_put_rejected_by_wrong_source_arn(self, auth):- bucket, url, arn = _bucket_and_queue("xpol")- _as(ACCT_A, sqs_svc._act_set_queue_attributes,- {"QueueUrl": url, "Attributes": {"Policy": json.dumps(- sqs_policy_allow_s3(arn, "no-such-bucket", ACCT_A))}}, url)- _assert_put_rejected(bucket, "sqs", arn)-- def test_s3_to_sqs_delivered_with_policy(self, auth):- bucket, url, arn = _bucket_and_queue("xpol")- _as(ACCT_A, sqs_svc._act_set_queue_attributes,- {"QueueUrl": url, "Attributes": {"Policy": json.dumps(- sqs_policy_allow_s3(arn, bucket, ACCT_A))}}, url)- assert _resp_code(_put_notif(ACCT_A, bucket, "sqs", arn)) == (200, "")- bodies = _queue_bodies(arn)- assert len(bodies) == 1- assert "s3:TestEvent" in bodies[0]-- def test_s3_skip_validation_stores_but_delivery_blocked(self, auth):- bucket, _url, arn = _bucket_and_queue("xpol")- assert _resp_code(- _put_notif(ACCT_A, bucket, "sqs", arn, skip=True)) == (200, "")- assert _queue_bodies(arn) == []- _as(ACCT_A, s3_svc._deliver_event_to_sqs,- arn, {"Records": []}, REGION, bucket_name=bucket)- assert _queue_bodies(arn) == []-- def test_s3_to_sns_default_policy_delivers(self, auth):- bucket, topic, _qurl, qarn = _bucket_topic_queue("xpol")- assert _resp_code(_put_notif(ACCT_A, bucket, "sns", topic)) == (200, "")- bodies = _queue_bodies(qarn)- assert len(bodies) == 1- assert "s3:TestEvent" in bodies[0]-- def test_s3_put_rejected_by_topic_policy_without_s3(self, auth):- bucket, topic, _qurl, _qarn = _bucket_topic_queue("xpol")- _set_topic_policy(ACCT_A, topic, _allow_account(- topic, ACCT_A, "sns:Publish"))- _assert_put_rejected(bucket, "sns", topic)-- def test_s3_to_sns_skip_validation_delivery_blocked(self, auth):- bucket, topic, _qurl, qarn = _bucket_topic_queue("xpol")- _set_topic_policy(ACCT_A, topic, _allow_account(- topic, ACCT_A, "sns:Publish"))- assert _resp_code(- _put_notif(ACCT_A, bucket, "sns", topic, skip=True)) == (200, "")- _as(ACCT_A, s3_svc._deliver_event_to_sns,- topic, {"Records": []}, REGION, bucket_name=bucket)- assert _queue_bodies(qarn) == []-- def test_s3_to_cross_account_sns_topic_delivered_with_policy(self, auth):- bucket = _uniq("xpol-xbkt")- _make_bucket(ACCT_A, bucket)- topic = _make_topic(ACCT_B, _uniq("xpol-xt"))- _set_topic_policy(ACCT_B, topic, {- "Version": "2012-10-17", "Statement": [{- "Sid": "s3-xacct", "Effect": "Allow",- "Principal": {"Service": "s3.amazonaws.com"},- "Action": "sns:Publish", "Resource": topic,- "Condition": {- "ArnLike": {"aws:SourceArn": f"arn:aws:s3:::{bucket}"},- "StringEquals": {"aws:SourceAccount": ACCT_A}}}]})- qname = _uniq("xpol-xq")- _make_queue(ACCT_B, qname, sqs_policy_allow_sns(- _qarn(ACCT_B, qname), topic))- assert _resp_code(_as(ACCT_B, sns_svc._subscribe, {- "TopicArn": topic, "Protocol": "sqs",- "Endpoint": _qarn(ACCT_B, qname)})) == (200, "")- # The topic lives in another account, but its policy allows this- # bucket's deliveries, so the PUT validates instead of failing.- assert _resp_code(_put_notif(ACCT_A, bucket, "sns", topic)) == (200, "")- bodies = _queue_bodies(_qarn(ACCT_B, qname))- assert len(bodies) == 1- assert "s3:TestEvent" in bodies[0]-- def test_s3_to_cross_account_sqs_queue_delivered_with_policy(self, auth):- bucket = _uniq("xpol-xbkt")- _make_bucket(ACCT_A, bucket)- qname = _uniq("xpol-xq")- _make_queue(ACCT_B, qname, sqs_policy_allow_s3(- _qarn(ACCT_B, qname), bucket, ACCT_A))- assert _resp_code(- _put_notif(ACCT_A, bucket, "sqs", _qarn(ACCT_B, qname))) == (200, "")- bodies = _queue_bodies(_qarn(ACCT_B, qname))- assert len(bodies) == 1- assert "s3:TestEvent" in bodies[0]-- def test_s3_to_cross_account_topic_rejected_without_policy(self, auth):- bucket = _uniq("xpol-xbkt")- _make_bucket(ACCT_A, bucket)- topic = _make_topic(ACCT_B, _uniq("xpol-xt"))- _assert_put_rejected(bucket, "sns", topic)---# ----------------------------------------------------------------------------# SNS→SQS fanout gated on the queue policy-# -----------------------------------------------------------------------------def _topic_and_queue(tag):- topic = _make_topic(ACCT_A, _uniq(f"{tag}-t"))- qname = _uniq(f"{tag}-q")- qurl = _make_queue(ACCT_A, qname)- assert _resp_code(_as(ACCT_A, sns_svc._subscribe, {- "TopicArn": topic, "Protocol": "sqs",- "Endpoint": _qarn(ACCT_A, qname)})) == (200, "")- return topic, qurl, _qarn(ACCT_A, qname)---class TestSnsFanoutPolicies:- def test_fanout_blocked_without_policy(self, auth):- topic, _qurl, qarn = _topic_and_queue("xpol-fan")- assert _resp_code(_as(ACCT_A, sns_svc._publish, {- "TopicArn": topic, "Message": "no-policy"})) == (200, "")- assert _queue_bodies(qarn) == []-- def test_fanout_blocked_by_wrong_source_arn(self, auth):- topic, qurl, qarn = _topic_and_queue("xpol-fan")- _as(ACCT_A, sqs_svc._act_set_queue_attributes,- {"QueueUrl": qurl, "Attributes": {"Policy": json.dumps(- sqs_policy_allow_sns(- qarn, f"arn:aws:sns:{REGION}:{ACCT_A}:wrong-topic"))}},- qurl)- assert _resp_code(_as(ACCT_A, sns_svc._publish, {- "TopicArn": topic, "Message": "wrong-source"})) == (200, "")- assert _queue_bodies(qarn) == []-- def test_fanout_delivered_with_policy(self, auth):- topic, qurl, qarn = _topic_and_queue("xpol-fan")- _as(ACCT_A, sqs_svc._act_set_queue_attributes,- {"QueueUrl": qurl, "Attributes": {"Policy": json.dumps(- sqs_policy_allow_sns(qarn, topic))}}, qurl)- assert _resp_code(_as(ACCT_A, sns_svc._publish, {- "TopicArn": topic, "Message": "with-policy"})) == (200, "")- bodies = _queue_bodies(qarn)- assert len(bodies) == 1- assert "with-policy" in bodies[0]-- def test_cross_account_fanout_needs_queue_policy(self, auth):- topic = _make_topic(ACCT_A, _uniq("xpol-xfan"))- _set_topic_policy(ACCT_A, topic, _allow_account(- topic, ACCT_B, "sns:Subscribe"))- qname = _uniq("xpol-xfanq")- qurl = _make_queue(ACCT_B, qname)- assert _resp_code(_as(ACCT_B, sns_svc._subscribe, {- "TopicArn": topic, "Protocol": "sqs",- "Endpoint": _qarn(ACCT_B, qname)})) == (200, "")- assert _resp_code(_as(ACCT_A, sns_svc._publish, {- "TopicArn": topic, "Message": "no-queue-policy"})) == (200, "")- assert _queue_bodies(_qarn(ACCT_B, qname)) == []-- _as(ACCT_B, sqs_svc._act_set_queue_attributes,- {"QueueUrl": qurl, "Attributes": {"Policy": json.dumps(- sqs_policy_allow_sns(_qarn(ACCT_B, qname), topic))}}, qurl)- assert _resp_code(_as(ACCT_A, sns_svc._publish, {- "TopicArn": topic, "Message": "with-queue-policy"})) == (200, "")- bodies = _queue_bodies(_qarn(ACCT_B, qname))- assert len(bodies) == 1- assert "with-queue-policy" in bodies[0]-- def test_cross_account_subscribe_denied_without_policy(self, auth):- topic = _make_topic(ACCT_A, _uniq("xpol-xfan"))- assert _resp_code(_as(ACCT_B, sns_svc._subscribe, {- "TopicArn": topic, "Protocol": "email",- "Endpoint": "[email protected]"})) == (403, "AuthorizationError")---# ----------------------------------------------------------------------------# AUTH=false stays permissive-# -----------------------------------------------------------------------------class TestAuthDisabledIsPermissive:- def test_cross_account_send_passes_without_policy(self, noauth):- name = _uniq("xpol-q")- url = _make_queue(ACCT_A, name)- _as(ACCT_B, sqs_svc._act_send_message,- {"QueueUrl": url, "MessageBody": "open"}, url)- assert _queue_bodies(_qarn(ACCT_A, name)) == ["open"]-- def test_cross_account_publish_passes_without_policy(self, noauth):- arn = _make_topic(ACCT_A, _uniq("xpol-t"))- assert _resp_code(_as(ACCT_B, sns_svc._publish, {- "TopicArn": arn, "Message": "open"})) == (200, "")-- def test_explicit_deny_ignored(self, noauth):- name = _uniq("xpol-q")- url = _make_queue(ACCT_A, name, _deny_all(- _qarn(ACCT_A, name), "sqs:SendMessage"))- _as(ACCT_A, sqs_svc._act_send_message,- {"QueueUrl": url, "MessageBody": "deny-ignored"}, url)- assert _queue_bodies(_qarn(ACCT_A, name)) == ["deny-ignored"]-- def test_get_queue_url_ignores_deny(self, noauth):- name = _uniq("xpol-q")- url = _make_queue(ACCT_A, name, _deny_all(- _qarn(ACCT_A, name), "sqs:GetQueueUrl"))- assert _as(ACCT_A, sqs_svc._act_get_queue_url,- {"QueueName": name}, "") == {"QueueUrl": url}-- def test_s3_put_and_delivery_without_policy(self, noauth):- bucket, _url, arn = _bucket_and_queue("xpol")- assert _resp_code(_put_notif(ACCT_A, bucket, "sqs", arn)) == (200, "")- bodies = _queue_bodies(arn)- assert len(bodies) == 1- assert "s3:TestEvent" in bodies[0]-- def test_fanout_without_policy(self, noauth):- topic, _qurl, qarn = _topic_and_queue("xpol-fan")- assert _resp_code(_as(ACCT_A, sns_svc._publish, {- "TopicArn": topic, "Message": "open"})) == (200, "")- assert len(_queue_bodies(qarn)) == 1diff --git a/tests/test_s3.py b/tests/test_s3.pyindex b161593e..150de7b9 100644--- a/tests/test_s3.py+++ b/tests/test_s3.py@@ -1,7 +1,11 @@+import base64+import hashlibimport jsonimport osimport re+import structimport time+import uuidimport uuid as _uuid_modfrom datetime import datetimefrom urllib.parse import urlparse@@ -3614,11 +3618,13 @@ def s3_persist(tmp_path, monkeypatch):buckets_before = set(s3mod._buckets._data)versions_before = set(s3mod._object_versions._data)versioning_before = set(s3mod._bucket_versioning._data)+ annotations_before = set(s3mod._object_annotations._data)yield s3modfor store, before in ((s3mod._buckets._data, buckets_before),(s3mod._object_versions._data, versions_before),(s3mod._bucket_versioning._data, versioning_before),+ (s3mod._object_annotations._data, annotations_before),):for key in set(store) - before:store.pop(key, None)@@ -6998,3 +7004,785 @@ def test_s3_notification_keeps_an_explicit_id(s3, sqs):)assert s3.get_bucket_notification_configuration(Bucket=bucket)["QueueConfigurations"][0]["Id"] == "mine"+++# --- Object annotations ---++def _bucket(s3, versioned=False, **create):+ name = f"annot-{uuid.uuid4().hex[:12]}"+ s3.create_bucket(Bucket=name, **create)+ if versioned:+ s3.put_bucket_versioning(Bucket=name, VersioningConfiguration={"Status": "Enabled"})+ return name+++def _code(exc):+ return exc.value.response["Error"]["Code"]+++def _status(exc):+ return exc.value.response["ResponseMetadata"]["HTTPStatusCode"]+++# ── Put / Get ──────────────────────────────────────────────────────────+++def test_put_and_get_an_annotation(s3):+ bucket = _bucket(s3, versioned=True)+ payload = b'{"archive":"Images"}'+ put = s3.put_object(Bucket=bucket, Key="photo.jpg", Body=b"jpeg")+ out = s3.put_object_annotation(Bucket=bucket, Key="photo.jpg", AnnotationName="nexus.system", AnnotationPayload=payload)+ assert out["ObjectVersionId"] == put["VersionId"]+ assert out["ETag"] == f'"{hashlib.md5(payload).hexdigest()}"'+ got = s3.get_object_annotation(Bucket=bucket, Key="photo.jpg", AnnotationName="nexus.system")+ assert got["AnnotationPayload"].read() == payload+ assert got["ObjectVersionId"] == put["VersionId"]+ assert got["ContentLength"] == len(payload)+ assert got["ServerSideEncryption"] == "AES256"+++def test_an_annotation_leaves_the_object_alone(s3):+ """"Adding, updating, or removing an annotation does not modify the parent+ object's ETag", and makes no new version."""+ bucket = _bucket(s3, versioned=True)+ put = s3.put_object(Bucket=bucket, Key="k", Body=b"body")+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")+ head = s3.head_object(Bucket=bucket, Key="k")+ assert head["ETag"] == put["ETag"] and head["VersionId"] == put["VersionId"]+ assert len(s3.list_object_versions(Bucket=bucket)["Versions"]) == 1+ assert s3.get_object(Bucket=bucket, Key="k")["Body"].read() == b"body"+++def test_putting_again_replaces_the_payload(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"first")+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"second")+ got = s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+ assert got["AnnotationPayload"].read() == b"second"+++def test_get_of_a_missing_annotation(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ with pytest.raises(ClientError) as exc:+ s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="absent")+ assert (_code(exc), _status(exc)) == ("NoSuchAnnotation", 404)+++def test_annotation_on_a_missing_key_or_bucket(s3):+ bucket = _bucket(s3)+ with pytest.raises(ClientError) as exc:+ s3.put_object_annotation(Bucket=bucket, Key="nope", AnnotationName="a", AnnotationPayload=b"1")+ assert _code(exc) == "NoSuchKey"+ with pytest.raises(ClientError) as exc:+ s3.get_object_annotation(Bucket=f"missing-{uuid.uuid4().hex[:8]}", Key="k", AnnotationName="a")+ assert _code(exc) == "NoSuchBucket"+++ "name, code",+ [+ ("aws.reserved", "InvalidAnnotationName"),+ ("AWSthing", "InvalidAnnotationName"),+ ("s3-thing", "InvalidAnnotationName"),+ ("S3", "InvalidAnnotationName"),+ ("has space", "InvalidAnnotationName"),+ ("slash/name", "InvalidAnnotationName"),+ (" ", "InvalidAnnotationName"),+ ("n" * 513, "AnnotationNameTooLong"),+ ],+)+def test_annotation_naming_rules(s3, name, code):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ with pytest.raises(ClientError) as exc:+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName=name, AnnotationPayload=b"1")+ assert (_code(exc), _status(exc)) == (code, 400)+++def test_names_may_use_letters_of_any_language(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ for name in ("données_1.2-3", "名前", "n" * 512):+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName=name, AnnotationPayload=b"1")+++def test_payload_limits(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="max", AnnotationPayload=b"a" * (1024 * 1024))+ with pytest.raises(ClientError) as exc:+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="big", AnnotationPayload=b"a" * (1024 * 1024 + 1))+ assert _code(exc) == "InvalidRequest"+ with pytest.raises(ClientError) as exc:+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="empty", AnnotationPayload=b"")+ assert _code(exc) == "InvalidRequest"+++def test_payload_must_be_utf8(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ with pytest.raises(ClientError) as exc:+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="bin", AnnotationPayload=b"\xff\xfe")+ assert (_code(exc), _status(exc)) == ("UnsupportedMediaType", 415)+++def test_a_version_holds_at_most_1000_annotations(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ for i in range(1000):+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName=f"n{i}", AnnotationPayload=b"1")+ with pytest.raises(ClientError) as exc:+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="n1000", AnnotationPayload=b"1")+ assert _code(exc) == "AnnotationLimitExceeded"+ # Replacing one of the thousand is not a new one.+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="n0", AnnotationPayload=b"2")+++def test_object_if_match(s3):+ bucket = _bucket(s3)+ etag = s3.put_object(Bucket=bucket, Key="k", Body=b"x")["ETag"]+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1", ObjectIfMatch=etag)+ with pytest.raises(ClientError) as exc:+ s3.put_object_annotation(+ Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"2", ObjectIfMatch='"deadbeef"'+ )+ assert _status(exc) == 412+ with pytest.raises(ClientError) as exc:+ s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", ObjectIfMatch='"deadbeef"')+ assert _status(exc) == 412+ s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", ObjectIfMatch=etag)+++# ── Checksums ──────────────────────────────────────────────────────────+++def test_default_checksum_is_crc64nvme(s3):+ """"If the annotation doesn't have a specified checksum algorithm or+ checksum value, Amazon S3 uses the CRC-64/NVME algorithm"."""+ plain = make_client("s3", {"request_checksum_calculation": "when_required"})+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ out = plain.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"123456789")+ expected = base64.b64encode(struct.pack(">Q", _crc64nvme_bitwise(b"123456789"))).decode()+ assert out["ChecksumCRC64NVME"] == expected+ got = s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", ChecksumMode="ENABLED")+ assert got["ChecksumCRC64NVME"] == expected+ # Only when asked: a client that does not send x-amz-checksum-mode gets none back.+ unasked = make_client("s3", {"response_checksum_validation": "when_required"})+ assert "ChecksumCRC64NVME" not in unasked.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+++ "member, digest",+ [+ ("ChecksumSHA256", lambda b: hashlib.sha256(b).digest()),+ ("ChecksumSHA512", lambda b: hashlib.sha512(b).digest()),+ ("ChecksumMD5", lambda b: hashlib.md5(b).digest()),+ # XXH64's published check value for b"abc" (the xxHash reference).+ ("ChecksumXXHASH64", lambda b: struct.pack(">Q", 0x44BC2CF5AD770999)),+ ],+)+def test_a_checksum_sent_is_verified(s3, member, digest):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ good = base64.b64encode(digest(b"abc")).decode()+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"abc", **{member: good})+ got = s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", ChecksumMode="ENABLED")+ assert got[member] == good and got["ChecksumType"] == "FULL_OBJECT"+ bad = base64.b64encode(bytes(x ^ 0xFF for x in digest(b"abc"))).decode()+ with pytest.raises(ClientError) as exc:+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"abc", **{member: bad})+ assert _code(exc) == "BadDigest"+++def test_xxh64_check_values():+ from ministack.services import s3 as s3mod++ assert s3mod._xxh64(b"") == 0xEF46DB3751D8E999+ assert s3mod._xxh64(b"abc") == 0x44BC2CF5AD770999+ # Past the 32-byte stripe loop, against the reference implementation's output.+ assert s3mod._xxh64(b"Nobody inspects the spammish repetition") == 0xFBCEA83C8A378BF1+++def test_an_algorithm_ministack_cannot_compute_is_refused(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ with pytest.raises(ClientError) as exc:+ s3.put_object_annotation(+ Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"abc", ChecksumXXHASH3="AAAAAAAAAAA="+ )+ assert _code(exc) == "InvalidRequest"+++# ── Encryption ─────────────────────────────────────────────────────────+++def test_annotations_take_the_objects_encryption(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x", ServerSideEncryption="aws:kms", SSEKMSKeyId="alias/annot")+ out = s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")+ assert out["ServerSideEncryption"] == "aws:kms"+ got = s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+ assert got["ServerSideEncryption"] == "aws:kms"+++def test_sse_c_objects_cannot_have_annotations(s3):+ bucket = _bucket(s3)+ key = os.urandom(32)+ s3.put_object(+ Bucket=bucket,+ Key="k",+ Body=b"x",+ SSECustomerAlgorithm="AES256",+ SSECustomerKey=key,+ )+ with pytest.raises(ClientError) as exc:+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")+ assert _code(exc) == "InvalidRequest"+++# ── Versions ───────────────────────────────────────────────────────────+++def test_annotations_belong_to_one_version(s3):+ bucket = _bucket(s3, versioned=True)+ v1 = s3.put_object(Bucket=bucket, Key="k", Body=b"one")["VersionId"]+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"on v1")+ v2 = s3.put_object(Bucket=bucket, Key="k", Body=b"two")["VersionId"]+ # "Creating a new version does not copy annotations from the previous version."+ with pytest.raises(ClientError) as exc:+ s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+ assert _code(exc) == "NoSuchAnnotation"+ got = s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=v1)+ assert got["AnnotationPayload"].read() == b"on v1" and got["ObjectVersionId"] == v1+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"on v2")+ assert s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=v2)["AnnotationPayload"].read() == b"on v2"+ assert s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=v1)["AnnotationPayload"].read() == b"on v1"+++def test_a_delete_marker_keeps_the_versions_annotations(s3):+ bucket = _bucket(s3, versioned=True)+ v1 = s3.put_object(Bucket=bucket, Key="k", Body=b"one")["VersionId"]+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"kept")+ marker = s3.delete_object(Bucket=bucket, Key="k")["VersionId"]+ with pytest.raises(ClientError) as exc:+ s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+ assert _code(exc) == "NoSuchKey"+ with pytest.raises(ClientError) as exc:+ s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=marker)+ assert _status(exc) == 405+ assert s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=v1)["AnnotationPayload"].read() == b"kept"+ # Removing the marker brings the version back with its annotations.+ s3.delete_object(Bucket=bucket, Key="k", VersionId=marker)+ assert s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")["AnnotationPayload"].read() == b"kept"+++def test_deleting_a_version_deletes_its_annotations(s3):+ bucket = _bucket(s3, versioned=True)+ v1 = s3.put_object(Bucket=bucket, Key="k", Body=b"one")["VersionId"]+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")+ s3.put_object(Bucket=bucket, Key="k", Body=b"two")+ s3.delete_object(Bucket=bucket, Key="k", VersionId=v1)+ with pytest.raises(ClientError) as exc:+ s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=v1)+ assert _code(exc) == "NoSuchVersion"+++def test_unversioned_overwrite_and_delete_drop_annotations(s3):+ """"In a non-versioned bucket, if you delete or overwrite the object, the+ annotations are deleted with it.\""""+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"one")+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")+ s3.put_object(Bucket=bucket, Key="k", Body=b"two")+ with pytest.raises(ClientError) as exc:+ s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+ assert _code(exc) == "NoSuchAnnotation"+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="b", AnnotationPayload=b"1")+ s3.delete_object(Bucket=bucket, Key="k")+ s3.put_object(Bucket=bucket, Key="k", Body=b"three")+ assert s3.list_object_annotations(Bucket=bucket, Key="k")["Annotations"] == []+++def test_batch_delete_drops_annotations(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"one")+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")+ s3.delete_objects(Bucket=bucket, Delete={"Objects": [{"Key": "k"}]})+ s3.put_object(Bucket=bucket, Key="k", Body=b"two")+ assert s3.list_object_annotations(Bucket=bucket, Key="k")["Annotations"] == []+++def test_annotations_need_no_restore_for_archived_objects(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="cold", Body=b"x", StorageClass="GLACIER")+ s3.put_object_annotation(Bucket=bucket, Key="cold", AnnotationName="a", AnnotationPayload=b"1")+ assert s3.get_object_annotation(Bucket=bucket, Key="cold", AnnotationName="a")["AnnotationPayload"].read() == b"1"+++# ── List ───────────────────────────────────────────────────────────────+++def test_list_annotations(s3):+ bucket = _bucket(s3, versioned=True)+ version = s3.put_object(Bucket=bucket, Key="k", Body=b"x")["VersionId"]+ for name, payload in (("ml.label", b"cat"), ("etl.status", b"done"), ("ml.score", b"0.97")):+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName=name, AnnotationPayload=payload, ChecksumAlgorithm="SHA256")+ out = s3.list_object_annotations(Bucket=bucket, Key="k")+ assert [a["AnnotationName"] for a in out["Annotations"]] == ["etl.status", "ml.label", "ml.score"]+ assert out["AnnotationCount"] == 3 and out["ObjectVersionId"] == version+ assert out["Bucket"] == bucket and out["Key"] == "k"+ entry = out["Annotations"][1]+ assert entry["Size"] == 3 and entry["ETag"] == f'"{hashlib.md5(b"cat").hexdigest()}"'+ assert entry["ChecksumAlgorithm"] == ["SHA256"] and "LastModified" in entry+ assert "NextContinuationToken" not in out+++def test_list_annotations_by_prefix_and_page(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ for i in range(5):+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName=f"page.{i}", AnnotationPayload=b"1")+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="other", AnnotationPayload=b"1")+ first = s3.list_object_annotations(Bucket=bucket, Key="k", AnnotationPrefix="page.", MaxAnnotationResults=2)+ assert [a["AnnotationName"] for a in first["Annotations"]] == ["page.0", "page.1"]+ assert first["AnnotationPrefix"] == "page." and first["MaxAnnotationResults"] == 2+ token = first["NextContinuationToken"]+ rest = s3.list_object_annotations(Bucket=bucket, Key="k", AnnotationPrefix="page.", ContinuationToken=token)+ assert [a["AnnotationName"] for a in rest["Annotations"]] == ["page.2", "page.3", "page.4"]+ assert rest["ContinuationToken"] == token and "NextContinuationToken" not in rest+++def test_list_rejects_a_bad_page_size_or_prefix(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ plain = make_client("s3", {"parameter_validation": False})+ with pytest.raises(ClientError) as exc:+ plain.list_object_annotations(Bucket=bucket, Key="k", MaxAnnotationResults=1001)+ assert _code(exc) == "InvalidArgument"+ with pytest.raises(ClientError) as exc:+ s3.list_object_annotations(Bucket=bucket, Key="k", AnnotationPrefix="p" * 513)+ assert _code(exc) == "InvalidPrefix"+++# ── Delete ─────────────────────────────────────────────────────────────+++def test_delete_an_annotation(s3):+ bucket = _bucket(s3, versioned=True)+ version = s3.put_object(Bucket=bucket, Key="k", Body=b"x")["VersionId"]+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")+ out = s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+ assert out["ResponseMetadata"]["HTTPStatusCode"] == 204 and out["ObjectVersionId"] == version+ with pytest.raises(ClientError) as exc:+ s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+ assert _code(exc) == "NoSuchAnnotation"+ # Deleting one that is not there is not an error.+ s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+++# Observed on AWS (ap-southeast-2, 2026-09-28): annotations are writes to the object version, so Object Lock guards+# PutObjectAnnotation and DeleteObjectAnnotation alike. Governance retention yields to+# x-amz-bypass-governance-retention; compliance retention and a legal hold refuse both, bypass or not.+++RETENTION = "Access Denied because object protected by object lock retention."+LEGAL_HOLD = "Access Denied because object protected by object lock legal hold."+++def _lock_call(method, path, headers=None, body=b"", query=None):+ import asyncio++ from ministack.core.responses import request_scope+ from ministack.services import s3 as s3mod++ with request_scope("000000000000", "us-east-1"):+ return asyncio.run(s3mod.handle_request(method, path, headers or {}, body, query or {}))+++def _locked_in_process(**lock_headers):+ bucket = f"ann-lock-{uuid.uuid4().hex[:10]}"+ assert _lock_call("PUT", f"/{bucket}", {"x-amz-bucket-object-lock-enabled": "true"})[0] == 200+ assert _lock_call("PUT", f"/{bucket}/k", lock_headers, b"x")[0] == 200+ return bucket+++def _annotation_write(bucket, method, bypass=False):+ headers = {"x-amz-bypass-governance-retention": "true"} if bypass else {}+ query = {"annotation": [""], "annotationName": ["a"]}+ return _lock_call(method, f"/{bucket}/k", headers, b"1" if method == "PUT" else b"", query)+++def _assert_refused(response, message):+ status, _, body = response+ assert status == 403+ assert b"<Code>AccessDenied</Code>" in body and message.encode() in body+++def _until():+ return time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(time.time() + 3600))+++def test_governance_retention_needs_the_bypass_to_put_or_delete_an_annotation(monkeypatch):+ import ministack.app as app_mod++ monkeypatch.setattr(app_mod, "AUTH", True)+ bucket = _locked_in_process(**{"x-amz-object-lock-mode": "GOVERNANCE",+ "x-amz-object-lock-retain-until-date": _until()})+ _assert_refused(_annotation_write(bucket, "PUT"), RETENTION)+ assert _annotation_write(bucket, "PUT", bypass=True)[0] == 200+ _assert_refused(_annotation_write(bucket, "DELETE"), RETENTION)+ assert _annotation_write(bucket, "DELETE", bypass=True)[0] == 204++[email protected]("lock_headers,message", [+ ({"x-amz-object-lock-mode": "COMPLIANCE"}, RETENTION),+ ({"x-amz-object-lock-legal-hold": "ON"}, LEGAL_HOLD),+])+def test_compliance_and_legal_hold_refuse_annotation_writes_even_with_the_bypass(monkeypatch, lock_headers, message):+ import ministack.app as app_mod++ monkeypatch.setattr(app_mod, "AUTH", True)+ if "x-amz-object-lock-mode" in lock_headers:+ lock_headers = dict(lock_headers, **{"x-amz-object-lock-retain-until-date": _until()})+ bucket = _locked_in_process(**lock_headers)+ for method in ("PUT", "DELETE"):+ for bypass in (False, True):+ _assert_refused(_annotation_write(bucket, method, bypass), message)+++def test_object_lock_does_not_refuse_annotation_writes_without_auth(monkeypatch):+ import ministack.app as app_mod++ monkeypatch.setattr(app_mod, "AUTH", False)+ bucket = _locked_in_process(**{"x-amz-object-lock-legal-hold": "ON"})+ assert _annotation_write(bucket, "PUT")[0] == 200+ assert _annotation_write(bucket, "DELETE")[0] == 204+++def _locked(s3, **lock):+ bucket = _bucket(s3, ObjectLockEnabledForBucket=True)+ s3.put_object(Bucket=bucket, Key="k", Body=b"x", **lock)+ return bucket+++def test_an_unlocked_object_in_a_lock_enabled_bucket_takes_annotations_freely(s3):+ bucket = _locked(s3)+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"2")+ s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+++# ── CopyObject ─────────────────────────────────────────────────────────+++def test_copy_carries_annotations_by_default(s3):+ src = _bucket(s3, versioned=True)+ dst = _bucket(s3, versioned=True)+ s3.put_object(Bucket=src, Key="landing/x.jpg", Body=b"x")+ s3.put_object_annotation(Bucket=src, Key="landing/x.jpg", AnnotationName="nexus.system", AnnotationPayload=b"{}")+ copied = s3.copy_object(Bucket=dst, Key="assets/x.jpg", CopySource={"Bucket": src, "Key": "landing/x.jpg"})+ got = s3.get_object_annotation(Bucket=dst, Key="assets/x.jpg", AnnotationName="nexus.system")+ assert got["AnnotationPayload"].read() == b"{}" and got["ObjectVersionId"] == copied["VersionId"]+++def test_copy_of_a_named_version_carries_that_versions_annotations(s3):+ bucket = _bucket(s3, versioned=True)+ v1 = s3.put_object(Bucket=bucket, Key="k", Body=b"one")["VersionId"]+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"v1")+ s3.put_object(Bucket=bucket, Key="k", Body=b"two")+ s3.copy_object(Bucket=bucket, Key="copy", CopySource={"Bucket": bucket, "Key": "k", "VersionId": v1})+ assert s3.get_object_annotation(Bucket=bucket, Key="copy", AnnotationName="a")["AnnotationPayload"].read() == b"v1"+++def test_copy_with_exclude_leaves_them_behind(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="src", Body=b"x")+ s3.put_object_annotation(Bucket=bucket, Key="src", AnnotationName="a", AnnotationPayload=b"1")+ s3.copy_object(Bucket=bucket, Key="dst", CopySource={"Bucket": bucket, "Key": "src"}, AnnotationDirective="EXCLUDE")+ assert s3.list_object_annotations(Bucket=bucket, Key="dst")["Annotations"] == []+++def test_copy_rejects_an_unknown_directive(s3):+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="src", Body=b"x")+ plain = make_client("s3", {"parameter_validation": False})+ with pytest.raises(ClientError) as exc:+ plain.copy_object(Bucket=bucket, Key="dst", CopySource={"Bucket": bucket, "Key": "src"}, AnnotationDirective="REPLACE")+ assert _code(exc) == "InvalidArgument"+++def test_copy_with_a_new_algorithm_rechecksums_the_annotations(s3):+ """"If you specify a different checksum algorithm in the copy request, the+ new algorithm applies to both the object and its annotations.\""""+ bucket = _bucket(s3)+ s3.put_object(Bucket=bucket, Key="src", Body=b"x")+ s3.put_object_annotation(Bucket=bucket, Key="src", AnnotationName="a", AnnotationPayload=b"abc")+ s3.copy_object(Bucket=bucket, Key="dst", CopySource={"Bucket": bucket, "Key": "src"}, ChecksumAlgorithm="SHA256")+ got = s3.get_object_annotation(Bucket=bucket, Key="dst", AnnotationName="a", ChecksumMode="ENABLED")+ assert got["ChecksumSHA256"] == base64.b64encode(hashlib.sha256(b"abc").digest()).decode()+++# ── Replication ────────────────────────────────────────────────────────+++def test_annotations_replicate(s3):+ src = _bucket(s3, versioned=True)+ dst = _bucket(s3, versioned=True)+ s3.put_bucket_replication(+ Bucket=src,+ ReplicationConfiguration={+ "Role": "arn:aws:iam::000000000000:role/repl",+ "Rules": [{"ID": "r1", "Status": "Enabled", "Prefix": "", "Destination": {"Bucket": f"arn:aws:s3:::{dst}"}}],+ },+ )+ s3.put_object(Bucket=src, Key="k", Body=b"x")+ s3.put_object_annotation(Bucket=src, Key="k", AnnotationName="a", AnnotationPayload=b"1")+ assert s3.get_object_annotation(Bucket=src, Key="k", AnnotationName="a")["ReplicationStatus"] == "COMPLETED"+ replica = s3.get_object_annotation(Bucket=dst, Key="k", AnnotationName="a")+ assert replica["AnnotationPayload"].read() == b"1" and replica["ReplicationStatus"] == "REPLICA"+ listed = s3.list_object_annotations(Bucket=dst, Key="k")["Annotations"]+ assert [(a["AnnotationName"], a["ReplicationStatus"]) for a in listed] == [("a", "REPLICA")]+++# ── Event notifications ────────────────────────────────────────────────+++def _notified(s3, sqs, events, versioned=True):+ bucket = _bucket(s3, versioned=versioned)+ url = sqs.create_queue(QueueName=bucket)["QueueUrl"]+ arn = sqs.get_queue_attributes(QueueUrl=url, AttributeNames=["QueueArn"])["Attributes"]["QueueArn"]+ s3.put_bucket_notification_configuration(+ Bucket=bucket, NotificationConfiguration={"QueueConfigurations": [{"QueueArn": arn, "Events": events}]}+ )+ return bucket, url+++def _records(sqs, url, expected, timeout=10):+ records = []+ deadline = time.time() + timeout+ while len(records) < expected and time.time() < deadline:+ for m in sqs.receive_message(QueueUrl=url, MaxNumberOfMessages=10, WaitTimeSeconds=1).get("Messages", []):+ records.extend(json.loads(m["Body"]).get("Records", []))+ sqs.delete_message(QueueUrl=url, ReceiptHandle=m["ReceiptHandle"])+ return records+++def test_annotation_events(s3, sqs):+ bucket, url = _notified(s3, sqs, ["s3:ObjectAnnotation:*", "s3:ObjectCreated:*"])+ version = s3.put_object(Bucket=bucket, Key="k", Body=b"x")["VersionId"]+ assert [r["eventName"] for r in _records(sqs, url, 1)] == ["ObjectCreated:Put"]+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"abc")+ s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+ records = sorted(_records(sqs, url, 2), key=lambda r: r["eventName"], reverse=True)+ # "PutObjectAnnotation and DeleteObjectAnnotation send only annotation-specific events."+ assert [r["eventName"] for r in records] == ["ObjectAnnotation:Put", "ObjectAnnotation:Delete"]+ put, delete = records+ assert put["objectAnnotation"] == [{"name": "a", "size": 3, "eTag": hashlib.md5(b"abc").hexdigest()}]+ assert delete["objectAnnotation"] == [{"name": "a"}]+ assert put["s3"]["object"]["versionId"] == version+ assert "sequencer" not in put["s3"]["object"]+++def test_get_and_list_send_no_events(s3, sqs):+ bucket, url = _notified(s3, sqs, ["s3:ObjectAnnotation:*"])+ s3.put_object(Bucket=bucket, Key="k", Body=b"x")+ s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")+ assert len(_records(sqs, url, 1)) == 1+ s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")+ s3.list_object_annotations(Bucket=bucket, Key="k")+ assert _records(sqs, url, 1, timeout=2) == []+++def test_copy_event_says_whether_annotations_came(s3, sqs):+ bucket, url = _notified(s3, sqs, ["s3:ObjectCreated:Copy"])+ s3.put_object(Bucket=bucket, Key="src", Body=b"x")+ s3.put_object_annotation(Bucket=bucket, Key="src", AnnotationName="a", AnnotationPayload=b"1")+ s3.copy_object(Bucket=bucket, Key="with", CopySource={"Bucket": bucket, "Key": "src"})+ s3.copy_object(Bucket=bucket, Key="without", CopySource={"Bucket": bucket, "Key": "src"}, AnnotationDirective="EXCLUDE")+ flags = {r["s3"]["object"]["key"]: r["s3"]["object"]["hasObjectAnnotation"] for r in _records(sqs, url, 2)}+ assert flags == {"with": True, "without": False}+++# ── Persistence ────────────────────────────────────────────────────────+++def test_annotations_survive_a_restart(s3_persist, tmp_path):+ s3mod = s3_persist+ bucket = "qa-annot-persist"+ s3mod._create_bucket(bucket, b"")+ s3mod._bucket_versioning[bucket] = "Enabled"+ v1 = s3mod._put_object(bucket, "k", b"one", {})[1]["x-amz-version-id"]+ s3mod._put_object_annotation(bucket, "k", b"on v1", {}, {"annotationName": ["a"]})+ v2 = s3mod._put_object(bucket, "k", b"two", {})[1]["x-amz-version-id"]+ s3mod._put_object_annotation(bucket, "k", b"on v2", {}, {"annotationName": ["a"]})++ s3mod._buckets._data.pop(("000000000000", bucket), None)+ for store in (s3mod._object_versions._data, s3mod._object_annotations._data):+ for k in [k for k in store if k[0] == "000000000000" and k[1][0] == bucket]:+ store.pop(k, None)+ s3mod._load_persisted_bucket("000000000000", bucket, os.path.join(str(tmp_path), "000000000000", bucket))++ assert s3mod._object_annotations[(bucket, "k", v1)]["a"]["payload"] == "on v1"+ assert s3mod._object_annotations[(bucket, "k", v2)]["a"]["payload"] == "on v2"+++# --- Event notification records ---++def _queue(sqs, name):+ url = sqs.create_queue(QueueName=name)["QueueUrl"]+ arn = sqs.get_queue_attributes(QueueUrl=url, AttributeNames=["QueueArn"])["Attributes"]["QueueArn"]+ return url, arn+++def _event_records(sqs, url, expected, timeout=10):+ """Every S3 record delivered to the queue, waiting until `expected` have arrived."""+ records = []+ deadline = time.time() + timeout+ while len(records) < expected and time.time() < deadline:+ resp = sqs.receive_message(QueueUrl=url, MaxNumberOfMessages=10, WaitTimeSeconds=1)+ for m in resp.get("Messages", []):+ body = json.loads(m["Body"])+ records.extend(body.get("Records", []))+ sqs.delete_message(QueueUrl=url, ReceiptHandle=m["ReceiptHandle"])+ return records+++def notified(s3, sqs):+ """A bucket whose object events go to a fresh queue: (bucket, queue url, configure)."""+ suffix = uuid.uuid4().hex[:8]+ bucket = f"evt-records-{suffix}"+ s3.create_bucket(Bucket=bucket)+ url, arn = _queue(sqs, f"evt-records-{suffix}")++ def configure(*events, versioned=False):+ if versioned:+ s3.put_bucket_versioning(Bucket=bucket, VersioningConfiguration={"Status": "Enabled"})+ s3.put_bucket_notification_configuration(+ Bucket=bucket,+ NotificationConfiguration={"QueueConfigurations": [{"QueueArn": arn, "Events": list(events)}]},+ )+ # The configuration's own s3:TestEvent carries no Records; drain it.+ _event_records(sqs, url, 0, timeout=1)++ return bucket, url, configure+++def test_record_key_is_url_encoded(s3, sqs, notified):+ bucket, url, configure = notified+ configure("s3:ObjectCreated:*")+ s3.put_object(Bucket=bucket, Key="photos/red flower+(1).jpg", Body=b"x")+ [record] = _event_records(sqs, url, 1)+ # "`red flower.jpg` becomes `red+flower.jpg`"; slashes stay, a literal + is escaped.+ assert record["s3"]["object"]["key"] == "photos/red+flower%2B%281%29.jpg"+ assert record["eventVersion"] == "2.6"+++def test_record_carries_the_version_the_put_made(s3, sqs, notified):+ bucket, url, configure = notified+ configure("s3:ObjectCreated:*", versioned=True)+ first = s3.put_object(Bucket=bucket, Key="doc.txt", Body=b"one")["VersionId"]+ second = s3.put_object(Bucket=bucket, Key="doc.txt", Body=b"two")["VersionId"]+ records = _event_records(sqs, url, 2)+ assert sorted(r["s3"]["object"]["versionId"] for r in records) == sorted([first, second])+++def test_record_has_no_version_in_an_unversioned_bucket(s3, sqs, notified):+ bucket, url, configure = notified+ configure("s3:ObjectCreated:*")+ s3.put_object(Bucket=bucket, Key="plain.txt", Body=b"x")+ [record] = _event_records(sqs, url, 1)+ assert "versionId" not in record["s3"]["object"]+++def test_sequencer_grows_with_each_write(s3, sqs, notified):+ bucket, url, configure = notified+ configure("s3:ObjectCreated:*", "s3:ObjectRemoved:*", versioned=True)+ v1 = s3.put_object(Bucket=bucket, Key="seq.txt", Body=b"1")["VersionId"]+ v2 = s3.put_object(Bucket=bucket, Key="seq.txt", Body=b"2")["VersionId"]+ marker = s3.delete_object(Bucket=bucket, Key="seq.txt")["VersionId"]+ by_version = {r["s3"]["object"]["versionId"]: r["s3"]["object"]["sequencer"] for r in _event_records(sqs, url, 3)}+ # Compared as S3 says to: left-pad the shorter with zeros, then compare.+ width = max(len(v) for v in by_version.values())+ ordered = [by_version[v].rjust(width, "0") for v in (v1, v2, marker)]+ assert ordered == sorted(ordered) and len(set(ordered)) == 3+ assert all(int(s, 16) >= 0 for s in ordered)+++def test_delete_without_version_reports_the_marker(s3, sqs, notified):+ bucket, url, configure = notified+ configure("s3:ObjectRemoved:*", versioned=True)+ s3.put_object(Bucket=bucket, Key="gone.txt", Body=b"x")+ marker = s3.delete_object(Bucket=bucket, Key="gone.txt")["VersionId"]+ [record] = _event_records(sqs, url, 1)+ assert record["eventName"] == "ObjectRemoved:DeleteMarkerCreated"+ assert record["s3"]["object"]["versionId"] == marker+++def test_marker_is_not_an_object_removed_delete(s3, sqs, notified):+ bucket, url, configure = notified+ # Subscribed to permanent deletes only: a marker is its own event type.+ configure("s3:ObjectRemoved:Delete", versioned=True)+ version = s3.put_object(Bucket=bucket, Key="kept.txt", Body=b"x")["VersionId"]+ s3.delete_object(Bucket=bucket, Key="kept.txt")+ s3.delete_object(Bucket=bucket, Key="kept.txt", VersionId=version)+ [record] = _event_records(sqs, url, 1)+ assert record["eventName"] == "ObjectRemoved:Delete"+ assert record["s3"]["object"]["versionId"] == version+ assert _event_records(sqs, url, 1, timeout=2) == []+++def test_copy_record_names_the_new_version(s3, sqs, notified):+ bucket, url, configure = notified+ configure("s3:ObjectCreated:Copy", versioned=True)+ s3.put_object(Bucket=bucket, Key="src.txt", Body=b"x")+ copied = s3.copy_object(Bucket=bucket, Key="dst.txt", CopySource={"Bucket": bucket, "Key": "src.txt"})+ [record] = _event_records(sqs, url, 1)+ assert record["eventName"] == "ObjectCreated:Copy"+ assert record["s3"]["object"]["versionId"] == copied["VersionId"]+++def test_multipart_record_names_the_new_version(s3, sqs, notified):+ bucket, url, configure = notified+ configure("s3:ObjectCreated:CompleteMultipartUpload", versioned=True)+ upload = s3.create_multipart_upload(Bucket=bucket, Key="big.bin")["UploadId"]+ part = s3.upload_part(Bucket=bucket, Key="big.bin", UploadId=upload, PartNumber=1, Body=b"x" * 16)+ done = s3.complete_multipart_upload(+ Bucket=bucket,+ Key="big.bin",+ UploadId=upload,+ MultipartUpload={"Parts": [{"PartNumber": 1, "ETag": part["ETag"]}]},+ )+ [record] = _event_records(sqs, url, 1)+ assert record["s3"]["object"]["versionId"] == done["VersionId"]+++def test_delete_objects_sends_an_event_per_object(s3, sqs, notified):+ bucket, url, configure = notified+ configure("s3:ObjectRemoved:*", versioned=True)+ for k in ("a.txt", "b.txt"):+ s3.put_object(Bucket=bucket, Key=k, Body=b"x")+ version = s3.put_object(Bucket=bucket, Key="c.txt", Body=b"x")["VersionId"]+ s3.delete_objects(+ Bucket=bucket,+ Delete={"Objects": [{"Key": "a.txt"}, {"Key": "b.txt"}, {"Key": "c.txt", "VersionId": version}]},+ )+ records = _event_records(sqs, url, 3)+ got = sorted((r["s3"]["object"]["key"], r["eventName"]) for r in records)+ assert got == [+ ("a.txt", "ObjectRemoved:DeleteMarkerCreated"),+ ("b.txt", "ObjectRemoved:DeleteMarkerCreated"),+ ("c.txt", "ObjectRemoved:Delete"),+ ]+++def test_delete_objects_in_an_unversioned_bucket(s3, sqs, notified):+ bucket, url, configure = notified+ configure("s3:ObjectRemoved:*")+ s3.put_object(Bucket=bucket, Key="u.txt", Body=b"x")+ s3.delete_objects(Bucket=bucket, Delete={"Objects": [{"Key": "u.txt"}, {"Key": "never-there.txt"}]})+ [record] = _event_records(sqs, url, 1)+ assert (record["s3"]["object"]["key"], record["eventName"]) == ("u.txt", "ObjectRemoved:Delete")diff --git a/tests/test_s3_annotations.py b/tests/test_s3_annotations.pydeleted file mode 100644index 943324e4..00000000--- a/tests/test_s3_annotations.py+++ /dev/null@@ -1,659 +0,0 @@-"""S3 object annotations: PutObjectAnnotation, GetObjectAnnotation,-ListObjectAnnotations and DeleteObjectAnnotation, and how annotations follow-object versions, CopyObject, deletes, replication, encryption, Object Lock,-restarts and event notifications.--Evidence: the S3 API Reference pages for the four operations, the S3 User-Guide's "Annotating your objects", "Event notification types and-destinations" and "Event message structure" pages, and the botocore s3 model.-Not validated against a real AWS account."""--import base64-import hashlib-import json-import os-import struct-import time-import uuid--import pytest-from botocore.exceptions import ClientError-from conftest import make_client---def _bucket(s3, versioned=False, **create):- name = f"annot-{uuid.uuid4().hex[:12]}"- s3.create_bucket(Bucket=name, **create)- if versioned:- s3.put_bucket_versioning(Bucket=name, VersioningConfiguration={"Status": "Enabled"})- return name---def _code(exc):- return exc.value.response["Error"]["Code"]---def _status(exc):- return exc.value.response["ResponseMetadata"]["HTTPStatusCode"]---def _crc64nvme_bitwise(data: bytes) -> int:- """Bit-at-a-time CRC-64/NVME, independent of the server's table."""- crc = 0xFFFFFFFFFFFFFFFF- for byte in data:- crc ^= byte- for _ in range(8):- crc = (crc >> 1) ^ 0x9A6C9329AC4BC9B5 if crc & 1 else crc >> 1- return crc ^ 0xFFFFFFFFFFFFFFFF---# ── Put / Get ──────────────────────────────────────────────────────────---def test_put_and_get_an_annotation(s3):- bucket = _bucket(s3, versioned=True)- payload = b'{"archive":"Images"}'- put = s3.put_object(Bucket=bucket, Key="photo.jpg", Body=b"jpeg")- out = s3.put_object_annotation(Bucket=bucket, Key="photo.jpg", AnnotationName="nexus.system", AnnotationPayload=payload)- assert out["ObjectVersionId"] == put["VersionId"]- assert out["ETag"] == f'"{hashlib.md5(payload).hexdigest()}"'- got = s3.get_object_annotation(Bucket=bucket, Key="photo.jpg", AnnotationName="nexus.system")- assert got["AnnotationPayload"].read() == payload- assert got["ObjectVersionId"] == put["VersionId"]- assert got["ContentLength"] == len(payload)- assert got["ServerSideEncryption"] == "AES256"---def test_an_annotation_leaves_the_object_alone(s3):- """"Adding, updating, or removing an annotation does not modify the parent- object's ETag", and makes no new version."""- bucket = _bucket(s3, versioned=True)- put = s3.put_object(Bucket=bucket, Key="k", Body=b"body")- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")- head = s3.head_object(Bucket=bucket, Key="k")- assert head["ETag"] == put["ETag"] and head["VersionId"] == put["VersionId"]- assert len(s3.list_object_versions(Bucket=bucket)["Versions"]) == 1- assert s3.get_object(Bucket=bucket, Key="k")["Body"].read() == b"body"---def test_putting_again_replaces_the_payload(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"first")- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"second")- got = s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")- assert got["AnnotationPayload"].read() == b"second"---def test_get_of_a_missing_annotation(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- with pytest.raises(ClientError) as exc:- s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="absent")- assert (_code(exc), _status(exc)) == ("NoSuchAnnotation", 404)---def test_annotation_on_a_missing_key_or_bucket(s3):- bucket = _bucket(s3)- with pytest.raises(ClientError) as exc:- s3.put_object_annotation(Bucket=bucket, Key="nope", AnnotationName="a", AnnotationPayload=b"1")- assert _code(exc) == "NoSuchKey"- with pytest.raises(ClientError) as exc:- s3.get_object_annotation(Bucket=f"missing-{uuid.uuid4().hex[:8]}", Key="k", AnnotationName="a")- assert _code(exc) == "NoSuchBucket"--- "name, code",- [- ("aws.reserved", "InvalidAnnotationName"),- ("AWSthing", "InvalidAnnotationName"),- ("s3-thing", "InvalidAnnotationName"),- ("S3", "InvalidAnnotationName"),- ("has space", "InvalidAnnotationName"),- ("slash/name", "InvalidAnnotationName"),- (" ", "InvalidAnnotationName"),- ("n" * 513, "AnnotationNameTooLong"),- ],-)-def test_annotation_naming_rules(s3, name, code):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- with pytest.raises(ClientError) as exc:- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName=name, AnnotationPayload=b"1")- assert (_code(exc), _status(exc)) == (code, 400)---def test_names_may_use_letters_of_any_language(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- for name in ("données_1.2-3", "名前", "n" * 512):- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName=name, AnnotationPayload=b"1")---def test_payload_limits(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="max", AnnotationPayload=b"a" * (1024 * 1024))- with pytest.raises(ClientError) as exc:- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="big", AnnotationPayload=b"a" * (1024 * 1024 + 1))- assert _code(exc) == "InvalidRequest"- with pytest.raises(ClientError) as exc:- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="empty", AnnotationPayload=b"")- assert _code(exc) == "InvalidRequest"---def test_payload_must_be_utf8(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- with pytest.raises(ClientError) as exc:- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="bin", AnnotationPayload=b"\xff\xfe")- assert (_code(exc), _status(exc)) == ("UnsupportedMediaType", 415)---def test_a_version_holds_at_most_1000_annotations(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- for i in range(1000):- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName=f"n{i}", AnnotationPayload=b"1")- with pytest.raises(ClientError) as exc:- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="n1000", AnnotationPayload=b"1")- assert _code(exc) == "AnnotationLimitExceeded"- # Replacing one of the thousand is not a new one.- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="n0", AnnotationPayload=b"2")---def test_object_if_match(s3):- bucket = _bucket(s3)- etag = s3.put_object(Bucket=bucket, Key="k", Body=b"x")["ETag"]- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1", ObjectIfMatch=etag)- with pytest.raises(ClientError) as exc:- s3.put_object_annotation(- Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"2", ObjectIfMatch='"deadbeef"'- )- assert _status(exc) == 412- with pytest.raises(ClientError) as exc:- s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", ObjectIfMatch='"deadbeef"')- assert _status(exc) == 412- s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", ObjectIfMatch=etag)---# ── Checksums ──────────────────────────────────────────────────────────---def test_default_checksum_is_crc64nvme(s3):- """"If the annotation doesn't have a specified checksum algorithm or- checksum value, Amazon S3 uses the CRC-64/NVME algorithm"."""- plain = make_client("s3", {"request_checksum_calculation": "when_required"})- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- out = plain.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"123456789")- expected = base64.b64encode(struct.pack(">Q", _crc64nvme_bitwise(b"123456789"))).decode()- assert out["ChecksumCRC64NVME"] == expected- got = s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", ChecksumMode="ENABLED")- assert got["ChecksumCRC64NVME"] == expected- # Only when asked: a client that does not send x-amz-checksum-mode gets none back.- unasked = make_client("s3", {"response_checksum_validation": "when_required"})- assert "ChecksumCRC64NVME" not in unasked.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")--- "member, digest",- [- ("ChecksumSHA256", lambda b: hashlib.sha256(b).digest()),- ("ChecksumSHA512", lambda b: hashlib.sha512(b).digest()),- ("ChecksumMD5", lambda b: hashlib.md5(b).digest()),- # XXH64's published check value for b"abc" (the xxHash reference).- ("ChecksumXXHASH64", lambda b: struct.pack(">Q", 0x44BC2CF5AD770999)),- ],-)-def test_a_checksum_sent_is_verified(s3, member, digest):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- good = base64.b64encode(digest(b"abc")).decode()- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"abc", **{member: good})- got = s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", ChecksumMode="ENABLED")- assert got[member] == good and got["ChecksumType"] == "FULL_OBJECT"- bad = base64.b64encode(bytes(x ^ 0xFF for x in digest(b"abc"))).decode()- with pytest.raises(ClientError) as exc:- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"abc", **{member: bad})- assert _code(exc) == "BadDigest"---def test_xxh64_check_values():- from ministack.services import s3 as s3mod-- assert s3mod._xxh64(b"") == 0xEF46DB3751D8E999- assert s3mod._xxh64(b"abc") == 0x44BC2CF5AD770999- # Past the 32-byte stripe loop, against the reference implementation's output.- assert s3mod._xxh64(b"Nobody inspects the spammish repetition") == 0xFBCEA83C8A378BF1---def test_an_algorithm_ministack_cannot_compute_is_refused(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- with pytest.raises(ClientError) as exc:- s3.put_object_annotation(- Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"abc", ChecksumXXHASH3="AAAAAAAAAAA="- )- assert _code(exc) == "InvalidRequest"---# ── Encryption ─────────────────────────────────────────────────────────---def test_annotations_take_the_objects_encryption(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x", ServerSideEncryption="aws:kms", SSEKMSKeyId="alias/annot")- out = s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")- assert out["ServerSideEncryption"] == "aws:kms"- got = s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")- assert got["ServerSideEncryption"] == "aws:kms"---def test_sse_c_objects_cannot_have_annotations(s3):- bucket = _bucket(s3)- key = os.urandom(32)- s3.put_object(- Bucket=bucket,- Key="k",- Body=b"x",- SSECustomerAlgorithm="AES256",- SSECustomerKey=key,- )- with pytest.raises(ClientError) as exc:- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")- assert _code(exc) == "InvalidRequest"---# ── Versions ───────────────────────────────────────────────────────────---def test_annotations_belong_to_one_version(s3):- bucket = _bucket(s3, versioned=True)- v1 = s3.put_object(Bucket=bucket, Key="k", Body=b"one")["VersionId"]- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"on v1")- v2 = s3.put_object(Bucket=bucket, Key="k", Body=b"two")["VersionId"]- # "Creating a new version does not copy annotations from the previous version."- with pytest.raises(ClientError) as exc:- s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")- assert _code(exc) == "NoSuchAnnotation"- got = s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=v1)- assert got["AnnotationPayload"].read() == b"on v1" and got["ObjectVersionId"] == v1- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"on v2")- assert s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=v2)["AnnotationPayload"].read() == b"on v2"- assert s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=v1)["AnnotationPayload"].read() == b"on v1"---def test_a_delete_marker_keeps_the_versions_annotations(s3):- bucket = _bucket(s3, versioned=True)- v1 = s3.put_object(Bucket=bucket, Key="k", Body=b"one")["VersionId"]- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"kept")- marker = s3.delete_object(Bucket=bucket, Key="k")["VersionId"]- with pytest.raises(ClientError) as exc:- s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")- assert _code(exc) == "NoSuchKey"- with pytest.raises(ClientError) as exc:- s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=marker)- assert _status(exc) == 405- assert s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=v1)["AnnotationPayload"].read() == b"kept"- # Removing the marker brings the version back with its annotations.- s3.delete_object(Bucket=bucket, Key="k", VersionId=marker)- assert s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")["AnnotationPayload"].read() == b"kept"---def test_deleting_a_version_deletes_its_annotations(s3):- bucket = _bucket(s3, versioned=True)- v1 = s3.put_object(Bucket=bucket, Key="k", Body=b"one")["VersionId"]- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")- s3.put_object(Bucket=bucket, Key="k", Body=b"two")- s3.delete_object(Bucket=bucket, Key="k", VersionId=v1)- with pytest.raises(ClientError) as exc:- s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", VersionId=v1)- assert _code(exc) == "NoSuchVersion"---def test_unversioned_overwrite_and_delete_drop_annotations(s3):- """"In a non-versioned bucket, if you delete or overwrite the object, the- annotations are deleted with it.\""""- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"one")- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")- s3.put_object(Bucket=bucket, Key="k", Body=b"two")- with pytest.raises(ClientError) as exc:- s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")- assert _code(exc) == "NoSuchAnnotation"- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="b", AnnotationPayload=b"1")- s3.delete_object(Bucket=bucket, Key="k")- s3.put_object(Bucket=bucket, Key="k", Body=b"three")- assert s3.list_object_annotations(Bucket=bucket, Key="k")["Annotations"] == []---def test_batch_delete_drops_annotations(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"one")- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")- s3.delete_objects(Bucket=bucket, Delete={"Objects": [{"Key": "k"}]})- s3.put_object(Bucket=bucket, Key="k", Body=b"two")- assert s3.list_object_annotations(Bucket=bucket, Key="k")["Annotations"] == []---def test_annotations_need_no_restore_for_archived_objects(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="cold", Body=b"x", StorageClass="GLACIER")- s3.put_object_annotation(Bucket=bucket, Key="cold", AnnotationName="a", AnnotationPayload=b"1")- assert s3.get_object_annotation(Bucket=bucket, Key="cold", AnnotationName="a")["AnnotationPayload"].read() == b"1"---# ── List ───────────────────────────────────────────────────────────────---def test_list_annotations(s3):- bucket = _bucket(s3, versioned=True)- version = s3.put_object(Bucket=bucket, Key="k", Body=b"x")["VersionId"]- for name, payload in (("ml.label", b"cat"), ("etl.status", b"done"), ("ml.score", b"0.97")):- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName=name, AnnotationPayload=payload, ChecksumAlgorithm="SHA256")- out = s3.list_object_annotations(Bucket=bucket, Key="k")- assert [a["AnnotationName"] for a in out["Annotations"]] == ["etl.status", "ml.label", "ml.score"]- assert out["AnnotationCount"] == 3 and out["ObjectVersionId"] == version- assert out["Bucket"] == bucket and out["Key"] == "k"- entry = out["Annotations"][1]- assert entry["Size"] == 3 and entry["ETag"] == f'"{hashlib.md5(b"cat").hexdigest()}"'- assert entry["ChecksumAlgorithm"] == ["SHA256"] and "LastModified" in entry- assert "NextContinuationToken" not in out---def test_list_annotations_by_prefix_and_page(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- for i in range(5):- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName=f"page.{i}", AnnotationPayload=b"1")- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="other", AnnotationPayload=b"1")- first = s3.list_object_annotations(Bucket=bucket, Key="k", AnnotationPrefix="page.", MaxAnnotationResults=2)- assert [a["AnnotationName"] for a in first["Annotations"]] == ["page.0", "page.1"]- assert first["AnnotationPrefix"] == "page." and first["MaxAnnotationResults"] == 2- token = first["NextContinuationToken"]- rest = s3.list_object_annotations(Bucket=bucket, Key="k", AnnotationPrefix="page.", ContinuationToken=token)- assert [a["AnnotationName"] for a in rest["Annotations"]] == ["page.2", "page.3", "page.4"]- assert rest["ContinuationToken"] == token and "NextContinuationToken" not in rest---def test_list_rejects_a_bad_page_size_or_prefix(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- plain = make_client("s3", {"parameter_validation": False})- with pytest.raises(ClientError) as exc:- plain.list_object_annotations(Bucket=bucket, Key="k", MaxAnnotationResults=1001)- assert _code(exc) == "InvalidArgument"- with pytest.raises(ClientError) as exc:- s3.list_object_annotations(Bucket=bucket, Key="k", AnnotationPrefix="p" * 513)- assert _code(exc) == "InvalidPrefix"---# ── Delete ─────────────────────────────────────────────────────────────---def test_delete_an_annotation(s3):- bucket = _bucket(s3, versioned=True)- version = s3.put_object(Bucket=bucket, Key="k", Body=b"x")["VersionId"]- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")- out = s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")- assert out["ResponseMetadata"]["HTTPStatusCode"] == 204 and out["ObjectVersionId"] == version- with pytest.raises(ClientError) as exc:- s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")- assert _code(exc) == "NoSuchAnnotation"- # Deleting one that is not there is not an error.- s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")---# Observed on AWS (ap-southeast-2, 2026-09-28): annotations are writes to the object version, so Object Lock guards-# PutObjectAnnotation and DeleteObjectAnnotation alike. Governance retention yields to-# x-amz-bypass-governance-retention; compliance retention and a legal hold refuse both, bypass or not.---def _bypassing(operation):- raw = make_client("s3")-- def bypass(request, **_):- request.headers["x-amz-bypass-governance-retention"] = "true"-- raw.meta.events.register(f"before-sign.s3.{operation}", bypass)- return raw---def _refused(call, message):- with pytest.raises(ClientError) as exc:- call()- assert _code(exc) == "AccessDenied"- assert exc.value.response["ResponseMetadata"]["HTTPStatusCode"] == 403- assert exc.value.response["Error"]["Message"] == message---def _locked(s3, **lock):- bucket = _bucket(s3, ObjectLockEnabledForBucket=True)- s3.put_object(Bucket=bucket, Key="k", Body=b"x", **lock)- return bucket---def _until():- return time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(time.time() + 3600))---RETENTION = "Access Denied because object protected by object lock retention."-LEGAL_HOLD = "Access Denied because object protected by object lock legal hold."---def test_governance_retention_needs_the_bypass_to_put_or_delete_an_annotation(s3):- bucket = _locked(s3, ObjectLockMode="GOVERNANCE", ObjectLockRetainUntilDate=_until())- _refused(lambda: s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1"), RETENTION)- _bypassing("PutObjectAnnotation").put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")- _refused(lambda: s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"2"), RETENTION)- _refused(lambda: s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a"), RETENTION)- _bypassing("DeleteObjectAnnotation").delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")---def test_compliance_retention_refuses_annotation_writes_even_with_the_bypass(s3):- bucket = _locked(s3, ObjectLockMode="COMPLIANCE", ObjectLockRetainUntilDate=_until())- def put(c):- return c.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")-- def delete(c):- return c.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")-- _refused(lambda: put(s3), RETENTION)- _refused(lambda: put(_bypassing("PutObjectAnnotation")), RETENTION)- _refused(lambda: delete(s3), RETENTION)- _refused(lambda: delete(_bypassing("DeleteObjectAnnotation")), RETENTION)---def test_a_legal_hold_refuses_annotation_writes_even_with_the_bypass(s3):- bucket = _locked(s3, ObjectLockLegalHoldStatus="ON")- def put(c):- return c.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")-- def delete(c):- return c.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")-- _refused(lambda: put(s3), LEGAL_HOLD)- _refused(lambda: put(_bypassing("PutObjectAnnotation")), LEGAL_HOLD)- _refused(lambda: delete(s3), LEGAL_HOLD)- _refused(lambda: delete(_bypassing("DeleteObjectAnnotation")), LEGAL_HOLD)---def test_an_unlocked_object_in_a_lock_enabled_bucket_takes_annotations_freely(s3):- bucket = _locked(s3)- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"2")- s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")---# ── CopyObject ─────────────────────────────────────────────────────────---def test_copy_carries_annotations_by_default(s3):- src = _bucket(s3, versioned=True)- dst = _bucket(s3, versioned=True)- s3.put_object(Bucket=src, Key="landing/x.jpg", Body=b"x")- s3.put_object_annotation(Bucket=src, Key="landing/x.jpg", AnnotationName="nexus.system", AnnotationPayload=b"{}")- copied = s3.copy_object(Bucket=dst, Key="assets/x.jpg", CopySource={"Bucket": src, "Key": "landing/x.jpg"})- got = s3.get_object_annotation(Bucket=dst, Key="assets/x.jpg", AnnotationName="nexus.system")- assert got["AnnotationPayload"].read() == b"{}" and got["ObjectVersionId"] == copied["VersionId"]---def test_copy_of_a_named_version_carries_that_versions_annotations(s3):- bucket = _bucket(s3, versioned=True)- v1 = s3.put_object(Bucket=bucket, Key="k", Body=b"one")["VersionId"]- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"v1")- s3.put_object(Bucket=bucket, Key="k", Body=b"two")- s3.copy_object(Bucket=bucket, Key="copy", CopySource={"Bucket": bucket, "Key": "k", "VersionId": v1})- assert s3.get_object_annotation(Bucket=bucket, Key="copy", AnnotationName="a")["AnnotationPayload"].read() == b"v1"---def test_copy_with_exclude_leaves_them_behind(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="src", Body=b"x")- s3.put_object_annotation(Bucket=bucket, Key="src", AnnotationName="a", AnnotationPayload=b"1")- s3.copy_object(Bucket=bucket, Key="dst", CopySource={"Bucket": bucket, "Key": "src"}, AnnotationDirective="EXCLUDE")- assert s3.list_object_annotations(Bucket=bucket, Key="dst")["Annotations"] == []---def test_copy_rejects_an_unknown_directive(s3):- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="src", Body=b"x")- plain = make_client("s3", {"parameter_validation": False})- with pytest.raises(ClientError) as exc:- plain.copy_object(Bucket=bucket, Key="dst", CopySource={"Bucket": bucket, "Key": "src"}, AnnotationDirective="REPLACE")- assert _code(exc) == "InvalidArgument"---def test_copy_with_a_new_algorithm_rechecksums_the_annotations(s3):- """"If you specify a different checksum algorithm in the copy request, the- new algorithm applies to both the object and its annotations.\""""- bucket = _bucket(s3)- s3.put_object(Bucket=bucket, Key="src", Body=b"x")- s3.put_object_annotation(Bucket=bucket, Key="src", AnnotationName="a", AnnotationPayload=b"abc")- s3.copy_object(Bucket=bucket, Key="dst", CopySource={"Bucket": bucket, "Key": "src"}, ChecksumAlgorithm="SHA256")- got = s3.get_object_annotation(Bucket=bucket, Key="dst", AnnotationName="a", ChecksumMode="ENABLED")- assert got["ChecksumSHA256"] == base64.b64encode(hashlib.sha256(b"abc").digest()).decode()---# ── Replication ────────────────────────────────────────────────────────---def test_annotations_replicate(s3):- src = _bucket(s3, versioned=True)- dst = _bucket(s3, versioned=True)- s3.put_bucket_replication(- Bucket=src,- ReplicationConfiguration={- "Role": "arn:aws:iam::000000000000:role/repl",- "Rules": [{"ID": "r1", "Status": "Enabled", "Prefix": "", "Destination": {"Bucket": f"arn:aws:s3:::{dst}"}}],- },- )- s3.put_object(Bucket=src, Key="k", Body=b"x")- s3.put_object_annotation(Bucket=src, Key="k", AnnotationName="a", AnnotationPayload=b"1")- assert s3.get_object_annotation(Bucket=src, Key="k", AnnotationName="a")["ReplicationStatus"] == "COMPLETED"- replica = s3.get_object_annotation(Bucket=dst, Key="k", AnnotationName="a")- assert replica["AnnotationPayload"].read() == b"1" and replica["ReplicationStatus"] == "REPLICA"- listed = s3.list_object_annotations(Bucket=dst, Key="k")["Annotations"]- assert [(a["AnnotationName"], a["ReplicationStatus"]) for a in listed] == [("a", "REPLICA")]---# ── Event notifications ────────────────────────────────────────────────---def _notified(s3, sqs, events, versioned=True):- bucket = _bucket(s3, versioned=versioned)- url = sqs.create_queue(QueueName=bucket)["QueueUrl"]- arn = sqs.get_queue_attributes(QueueUrl=url, AttributeNames=["QueueArn"])["Attributes"]["QueueArn"]- s3.put_bucket_notification_configuration(- Bucket=bucket, NotificationConfiguration={"QueueConfigurations": [{"QueueArn": arn, "Events": events}]}- )- return bucket, url---def _records(sqs, url, expected, timeout=10):- records = []- deadline = time.time() + timeout- while len(records) < expected and time.time() < deadline:- for m in sqs.receive_message(QueueUrl=url, MaxNumberOfMessages=10, WaitTimeSeconds=1).get("Messages", []):- records.extend(json.loads(m["Body"]).get("Records", []))- sqs.delete_message(QueueUrl=url, ReceiptHandle=m["ReceiptHandle"])- return records---def test_annotation_events(s3, sqs):- bucket, url = _notified(s3, sqs, ["s3:ObjectAnnotation:*", "s3:ObjectCreated:*"])- version = s3.put_object(Bucket=bucket, Key="k", Body=b"x")["VersionId"]- assert [r["eventName"] for r in _records(sqs, url, 1)] == ["ObjectCreated:Put"]- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"abc")- s3.delete_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")- records = sorted(_records(sqs, url, 2), key=lambda r: r["eventName"], reverse=True)- # "PutObjectAnnotation and DeleteObjectAnnotation send only annotation-specific events."- assert [r["eventName"] for r in records] == ["ObjectAnnotation:Put", "ObjectAnnotation:Delete"]- put, delete = records- assert put["objectAnnotation"] == [{"name": "a", "size": 3, "eTag": hashlib.md5(b"abc").hexdigest()}]- assert delete["objectAnnotation"] == [{"name": "a"}]- assert put["s3"]["object"]["versionId"] == version- assert "sequencer" not in put["s3"]["object"]---def test_get_and_list_send_no_events(s3, sqs):- bucket, url = _notified(s3, sqs, ["s3:ObjectAnnotation:*"])- s3.put_object(Bucket=bucket, Key="k", Body=b"x")- s3.put_object_annotation(Bucket=bucket, Key="k", AnnotationName="a", AnnotationPayload=b"1")- assert len(_records(sqs, url, 1)) == 1- s3.get_object_annotation(Bucket=bucket, Key="k", AnnotationName="a")- s3.list_object_annotations(Bucket=bucket, Key="k")- assert _records(sqs, url, 1, timeout=2) == []---def test_copy_event_says_whether_annotations_came(s3, sqs):- bucket, url = _notified(s3, sqs, ["s3:ObjectCreated:Copy"])- s3.put_object(Bucket=bucket, Key="src", Body=b"x")- s3.put_object_annotation(Bucket=bucket, Key="src", AnnotationName="a", AnnotationPayload=b"1")- s3.copy_object(Bucket=bucket, Key="with", CopySource={"Bucket": bucket, "Key": "src"})- s3.copy_object(Bucket=bucket, Key="without", CopySource={"Bucket": bucket, "Key": "src"}, AnnotationDirective="EXCLUDE")- flags = {r["s3"]["object"]["key"]: r["s3"]["object"]["hasObjectAnnotation"] for r in _records(sqs, url, 2)}- assert flags == {"with": True, "without": False}---# ── Persistence ────────────────────────────────────────────────────────---def s3_persist(tmp_path, monkeypatch):- """S3 in this process with persistence on, over a throwaway DATA_DIR."""- from ministack.core import responses as respmod- from ministack.services import s3 as s3mod-- monkeypatch.setattr(s3mod, "DATA_DIR", str(tmp_path))- monkeypatch.setattr(s3mod, "S3_PERSIST", True)- monkeypatch.setattr(s3mod, "get_account_id", lambda: "000000000000")- monkeypatch.setattr(respmod, "get_account_id", lambda: "000000000000")- stores = (s3mod._buckets._data, s3mod._object_versions._data, s3mod._bucket_versioning._data, s3mod._object_annotations._data)- before = [set(store) for store in stores]- yield s3mod- for store, keys in zip(stores, before):- for key in set(store) - keys:- store.pop(key, None)---def test_annotations_survive_a_restart(s3_persist, tmp_path):- s3mod = s3_persist- bucket = "qa-annot-persist"- s3mod._create_bucket(bucket, b"")- s3mod._bucket_versioning[bucket] = "Enabled"- v1 = s3mod._put_object(bucket, "k", b"one", {})[1]["x-amz-version-id"]- s3mod._put_object_annotation(bucket, "k", b"on v1", {}, {"annotationName": ["a"]})- v2 = s3mod._put_object(bucket, "k", b"two", {})[1]["x-amz-version-id"]- s3mod._put_object_annotation(bucket, "k", b"on v2", {}, {"annotationName": ["a"]})-- s3mod._buckets._data.pop(("000000000000", bucket), None)- for store in (s3mod._object_versions._data, s3mod._object_annotations._data):- for k in [k for k in store if k[0] == "000000000000" and k[1][0] == bucket]:- store.pop(k, None)- s3mod._load_persisted_bucket("000000000000", bucket, os.path.join(str(tmp_path), "000000000000", bucket))-- assert s3mod._object_annotations[(bucket, "k", v1)]["a"]["payload"] == "on v1"- assert s3mod._object_annotations[(bucket, "k", v2)]["a"]["payload"] == "on v2"diff --git a/tests/test_s3_event_records.py b/tests/test_s3_event_records.pydeleted file mode 100644index 16faa583..00000000--- a/tests/test_s3_event_records.py+++ /dev/null@@ -1,170 +0,0 @@-"""The S3 notification record's object block, per the S3 User Guide's-"Event message structure" and "Event notification types and destinations"-pages: a URL-encoded key, the version the write made or removed, a sequencer-that grows with each create and delete, eventVersion 2.6, a delete marker-reported as ObjectRemoved:DeleteMarkerCreated, and one event per object a-DeleteObjects removes. Evidence: official AWS documentation, not a run against-a real AWS account."""--import json-import time-import uuid--import pytest---def _queue(sqs, name):- url = sqs.create_queue(QueueName=name)["QueueUrl"]- arn = sqs.get_queue_attributes(QueueUrl=url, AttributeNames=["QueueArn"])["Attributes"]["QueueArn"]- return url, arn---def _records(sqs, url, expected, timeout=10):- """Every S3 record delivered to the queue, waiting until `expected` have arrived."""- records = []- deadline = time.time() + timeout- while len(records) < expected and time.time() < deadline:- resp = sqs.receive_message(QueueUrl=url, MaxNumberOfMessages=10, WaitTimeSeconds=1)- for m in resp.get("Messages", []):- body = json.loads(m["Body"])- records.extend(body.get("Records", []))- sqs.delete_message(QueueUrl=url, ReceiptHandle=m["ReceiptHandle"])- return records---def notified(s3, sqs):- """A bucket whose object events go to a fresh queue: (bucket, queue url, configure)."""- suffix = uuid.uuid4().hex[:8]- bucket = f"evt-records-{suffix}"- s3.create_bucket(Bucket=bucket)- url, arn = _queue(sqs, f"evt-records-{suffix}")-- def configure(*events, versioned=False):- if versioned:- s3.put_bucket_versioning(Bucket=bucket, VersioningConfiguration={"Status": "Enabled"})- s3.put_bucket_notification_configuration(- Bucket=bucket,- NotificationConfiguration={"QueueConfigurations": [{"QueueArn": arn, "Events": list(events)}]},- )- # The configuration's own s3:TestEvent carries no Records; drain it.- _records(sqs, url, 0, timeout=1)-- return bucket, url, configure---def test_record_key_is_url_encoded(s3, sqs, notified):- bucket, url, configure = notified- configure("s3:ObjectCreated:*")- s3.put_object(Bucket=bucket, Key="photos/red flower+(1).jpg", Body=b"x")- [record] = _records(sqs, url, 1)- # "`red flower.jpg` becomes `red+flower.jpg`"; slashes stay, a literal + is escaped.- assert record["s3"]["object"]["key"] == "photos/red+flower%2B%281%29.jpg"- assert record["eventVersion"] == "2.6"---def test_record_carries_the_version_the_put_made(s3, sqs, notified):- bucket, url, configure = notified- configure("s3:ObjectCreated:*", versioned=True)- first = s3.put_object(Bucket=bucket, Key="doc.txt", Body=b"one")["VersionId"]- second = s3.put_object(Bucket=bucket, Key="doc.txt", Body=b"two")["VersionId"]- records = _records(sqs, url, 2)- assert sorted(r["s3"]["object"]["versionId"] for r in records) == sorted([first, second])---def test_record_has_no_version_in_an_unversioned_bucket(s3, sqs, notified):- bucket, url, configure = notified- configure("s3:ObjectCreated:*")- s3.put_object(Bucket=bucket, Key="plain.txt", Body=b"x")- [record] = _records(sqs, url, 1)- assert "versionId" not in record["s3"]["object"]---def test_sequencer_grows_with_each_write(s3, sqs, notified):- bucket, url, configure = notified- configure("s3:ObjectCreated:*", "s3:ObjectRemoved:*", versioned=True)- v1 = s3.put_object(Bucket=bucket, Key="seq.txt", Body=b"1")["VersionId"]- v2 = s3.put_object(Bucket=bucket, Key="seq.txt", Body=b"2")["VersionId"]- marker = s3.delete_object(Bucket=bucket, Key="seq.txt")["VersionId"]- by_version = {r["s3"]["object"]["versionId"]: r["s3"]["object"]["sequencer"] for r in _records(sqs, url, 3)}- # Compared as S3 says to: left-pad the shorter with zeros, then compare.- width = max(len(v) for v in by_version.values())- ordered = [by_version[v].rjust(width, "0") for v in (v1, v2, marker)]- assert ordered == sorted(ordered) and len(set(ordered)) == 3- assert all(int(s, 16) >= 0 for s in ordered)---def test_delete_without_version_reports_the_marker(s3, sqs, notified):- bucket, url, configure = notified- configure("s3:ObjectRemoved:*", versioned=True)- s3.put_object(Bucket=bucket, Key="gone.txt", Body=b"x")- marker = s3.delete_object(Bucket=bucket, Key="gone.txt")["VersionId"]- [record] = _records(sqs, url, 1)- assert record["eventName"] == "ObjectRemoved:DeleteMarkerCreated"- assert record["s3"]["object"]["versionId"] == marker---def test_marker_is_not_an_object_removed_delete(s3, sqs, notified):- bucket, url, configure = notified- # Subscribed to permanent deletes only: a marker is its own event type.- configure("s3:ObjectRemoved:Delete", versioned=True)- version = s3.put_object(Bucket=bucket, Key="kept.txt", Body=b"x")["VersionId"]- s3.delete_object(Bucket=bucket, Key="kept.txt")- s3.delete_object(Bucket=bucket, Key="kept.txt", VersionId=version)- [record] = _records(sqs, url, 1)- assert record["eventName"] == "ObjectRemoved:Delete"- assert record["s3"]["object"]["versionId"] == version- assert _records(sqs, url, 1, timeout=2) == []---def test_copy_record_names_the_new_version(s3, sqs, notified):- bucket, url, configure = notified- configure("s3:ObjectCreated:Copy", versioned=True)- s3.put_object(Bucket=bucket, Key="src.txt", Body=b"x")- copied = s3.copy_object(Bucket=bucket, Key="dst.txt", CopySource={"Bucket": bucket, "Key": "src.txt"})- [record] = _records(sqs, url, 1)- assert record["eventName"] == "ObjectCreated:Copy"- assert record["s3"]["object"]["versionId"] == copied["VersionId"]---def test_multipart_record_names_the_new_version(s3, sqs, notified):- bucket, url, configure = notified- configure("s3:ObjectCreated:CompleteMultipartUpload", versioned=True)- upload = s3.create_multipart_upload(Bucket=bucket, Key="big.bin")["UploadId"]- part = s3.upload_part(Bucket=bucket, Key="big.bin", UploadId=upload, PartNumber=1, Body=b"x" * 16)- done = s3.complete_multipart_upload(- Bucket=bucket,- Key="big.bin",- UploadId=upload,- MultipartUpload={"Parts": [{"PartNumber": 1, "ETag": part["ETag"]}]},- )- [record] = _records(sqs, url, 1)- assert record["s3"]["object"]["versionId"] == done["VersionId"]---def test_delete_objects_sends_an_event_per_object(s3, sqs, notified):- bucket, url, configure = notified- configure("s3:ObjectRemoved:*", versioned=True)- for k in ("a.txt", "b.txt"):- s3.put_object(Bucket=bucket, Key=k, Body=b"x")- version = s3.put_object(Bucket=bucket, Key="c.txt", Body=b"x")["VersionId"]- s3.delete_objects(- Bucket=bucket,- Delete={"Objects": [{"Key": "a.txt"}, {"Key": "b.txt"}, {"Key": "c.txt", "VersionId": version}]},- )- records = _records(sqs, url, 3)- got = sorted((r["s3"]["object"]["key"], r["eventName"]) for r in records)- assert got == [- ("a.txt", "ObjectRemoved:DeleteMarkerCreated"),- ("b.txt", "ObjectRemoved:DeleteMarkerCreated"),- ("c.txt", "ObjectRemoved:Delete"),- ]---def test_delete_objects_in_an_unversioned_bucket(s3, sqs, notified):- bucket, url, configure = notified- configure("s3:ObjectRemoved:*")- s3.put_object(Bucket=bucket, Key="u.txt", Body=b"x")- s3.delete_objects(Bucket=bucket, Delete={"Objects": [{"Key": "u.txt"}, {"Key": "never-there.txt"}]})- [record] = _records(sqs, url, 1)- assert (record["s3"]["object"]["key"], record["eventName"]) == ("u.txt", "ObjectRemoved:Delete")diff --git a/tests/test_sqs.py b/tests/test_sqs.pyindex 712855c7..0624d149 100644--- a/tests/test_sqs.py+++ b/tests/test_sqs.py@@ -1,7 +1,9 @@import ioimport jsonimport os+import reimport time+import uuidimport uuid as _uuid_modimport zipfilefrom urllib.parse import urlparse@@ -10,6 +12,13 @@ import boto3import pytestfrom botocore.config import Configfrom botocore.exceptions import ClientError+from conftest import REGION, sqs_policy_allow_s3, sqs_policy_allow_sns++from ministack.core.iam_evaluator import EvalContext, evaluate_resource_policy+from ministack.core.responses import request_scope+from ministack.services import s3 as s3_svc+from ministack.services import sns as sns_svc+from ministack.services import sqs as sqs_svcdef _make_zip(code: str) -> bytes:@@ -1627,3 +1636,621 @@ def test_unicode_sqs_message(sqs):sqs.send_message(QueueUrl=url, MessageBody=msg)resp = sqs.receive_message(QueueUrl=url, MaxNumberOfMessages=1)assert resp["Messages"][0]["Body"] == msg+++# --- Queue and topic resource policies (in-process, AUTH toggled) ---++ACCT_A = "111111111111"+ACCT_B = "222222222222"+++def _reset_messaging_state():+ from ministack.core.iam_evaluator import _request_caller_arn++ sqs_svc.reset()+ sns_svc.reset()+ s3_svc.reset()+ _request_caller_arn.set("")+++def auth(monkeypatch):+ """AUTH=true with clean in-process SQS/SNS/S3 state."""+ import ministack.app as app_mod++ monkeypatch.setattr(app_mod, "AUTH", True)+ _reset_messaging_state()+ yield+ _reset_messaging_state()+++def noauth(monkeypatch):+ """AUTH=false with clean in-process SQS/SNS/S3 state."""+ import ministack.app as app_mod++ monkeypatch.setattr(app_mod, "AUTH", False)+ _reset_messaging_state()+ yield+ _reset_messaging_state()+++def _uniq(prefix):+ return f"{prefix}-{uuid.uuid4().hex[:8]}"+++def _as(account, fn, *args, **kwargs):+ """Call an in-process service function as *account* in the test region."""+ with request_scope(account, REGION):+ return fn(*args, **kwargs)+++def _resp_detail(resp):+ """``(status, code, message)`` of an in-process SNS/S3 response tuple."""+ status, _headers, body = resp+ text = body.decode("utf-8", errors="replace") if isinstance(body, bytes) else str(body)+ code = re.search(r"<Code>([^<]*)</Code>", text)+ msg = re.search(r"<Message>([^<]*)</Message>", text)+ return (status, code.group(1) if code else "",+ msg.group(1) if msg else "")+++def _resp_code(resp):+ """``(status, code)`` of an in-process SNS/S3 response tuple."""+ status, code, _ = _resp_detail(resp)+ return status, code+++def _qarn(account, name):+ return f"arn:aws:sqs:{REGION}:{account}:{name}"+++def _tarn(account, name):+ return f"arn:aws:sns:{REGION}:{account}:{name}"+++def _make_queue(account, name, policy=None):+ url = _as(account, sqs_svc._act_create_queue,+ {"QueueName": name}, "")["QueueUrl"]+ if policy is not None:+ _as(account, sqs_svc._act_set_queue_attributes,+ {"QueueUrl": url,+ "Attributes": {"Policy": json.dumps(policy)}}, url)+ return url+++def _queue_bodies(arn):+ q = sqs_svc._queue_by_arn(arn)+ assert q is not None+ return [m["body"] for m in q["messages"]]+++def _deny_all(arn, action):+ return {"Version": "2012-10-17", "Statement": [{+ "Sid": "deny", "Effect": "Deny", "Principal": {"AWS": "*"},+ "Action": action, "Resource": arn}]}+++def _allow_account(arn, account, action):+ return {"Version": "2012-10-17", "Statement": [{+ "Sid": "x", "Effect": "Allow", "Principal": {"AWS": account},+ "Action": action, "Resource": arn}]}+++def _make_topic(account, name):+ assert _resp_code(_as(account, sns_svc._create_topic,+ {"Name": name})) == (200, "")+ return _tarn(account, name)+++def _set_topic_policy(account, arn, policy):+ assert _resp_code(_as(account, sns_svc._set_topic_attributes, {+ "TopicArn": arn, "AttributeName": "Policy",+ "AttributeValue": json.dumps(policy)})) == (200, "")+++def _topic_policy(arn):+ topic = sns_svc._topic_by_arn_any_scope(arn)+ assert topic is not None+ return json.loads(topic["attributes"]["Policy"])+++def _make_bucket(account, name):+ assert _resp_code(_as(account, s3_svc._create_bucket,+ name, b"", {})) == (200, "")+++def _notif_xml(kind, arn):+ tag = "Queue" if kind == "sqs" else "Topic"+ return (+ "<NotificationConfiguration>"+ f"<{tag}Configuration><{tag}>{arn}</{tag}>"+ "<Event>s3:ObjectCreated:*</Event>"+ f"</{tag}Configuration>"+ "</NotificationConfiguration>"+ ).encode()+++def _put_notif(account, bucket, kind, arn, skip=False):+ headers = {"x-amz-skip-destination-validation": "true"} if skip else {}+ return _as(account, s3_svc._put_bucket_notification,+ bucket, _notif_xml(kind, arn), headers)+++# ---------------------------------------------------------------------------+# Unit: evaluate_resource_policy Service principals+# ---------------------------------------------------------------------------++def _ctx(action="sqs:SendMessage", resource="arn:aws:sqs:us-east-1:1:q",+ source_arn="", source_account="", principal="arn:aws:iam::1:root"):+ return EvalContext(+ principal_arn=principal,+ principal_type="Service",+ principal_account="1",+ action=action,+ resource_arn=resource,+ region="us-east-1",+ service_context={"aws:sourcearn": source_arn,+ "aws:sourceaccount": source_account,+ "aws:sourceowner": source_account},+ )+++def _svc_policy():+ return {+ "Version": "2012-10-17",+ "Statement": [{+ "Effect": "Allow",+ "Principal": {"Service": "s3.amazonaws.com"},+ "Action": "sqs:SendMessage",+ "Resource": "arn:aws:sqs:us-east-1:1:q",+ "Condition": {"ArnLike": {"aws:SourceArn": "arn:aws:s3:*:*:bkt"}},+ }],+ }+++class TestEvaluateResourcePolicyService:+ def test_service_principal_allows_matching_service(self):+ assert evaluate_resource_policy(+ _svc_policy(), _ctx(source_arn="arn:aws:s3:::bkt"),+ service="s3.amazonaws.com").decision == "Allow"++ def test_service_principal_ignores_other_service(self):+ assert evaluate_resource_policy(+ _svc_policy(), _ctx(source_arn="arn:aws:s3:::bkt"),+ service="sns.amazonaws.com").decision == "ImplicitDeny"++ def test_service_principal_ignored_without_service_arg(self):+ # Legacy callers (Lambda layer policies, S3 bucket policies) keep the+ # old behaviour: Service entries never match.+ assert evaluate_resource_policy(+ _svc_policy(), _ctx(source_arn="arn:aws:s3:::bkt")).decision == "ImplicitDeny"++ def test_wrong_source_arn_denies(self):+ assert evaluate_resource_policy(+ _svc_policy(), _ctx(source_arn="arn:aws:s3:::other"),+ service="s3.amazonaws.com").decision == "ImplicitDeny"++ def test_aws_principal_still_matches(self):+ doc = {"Statement": [{+ "Effect": "Allow", "Principal": {"AWS": "222222222222"},+ "Action": "sqs:SendMessage", "Resource": "*"}]}+ ctx = _ctx(principal="arn:aws:iam::222222222222:root")+ assert evaluate_resource_policy(doc, ctx).decision == "Allow"++ def test_star_principal_matches(self):+ doc = {"Statement": [{+ "Effect": "Allow", "Principal": "*",+ "Action": "sqs:SendMessage", "Resource": "*"}]}+ assert evaluate_resource_policy(doc, _ctx()).decision == "Allow"+++# ---------------------------------------------------------------------------+# Cross-account SQS+# ---------------------------------------------------------------------------++class TestCrossAccountSqs:+ def test_send_denied_without_policy(self, auth):+ url = _make_queue(ACCT_A, _uniq("xpol-q"))+ with pytest.raises(sqs_svc._Err) as exc:+ _as(ACCT_B, sqs_svc._act_send_message,+ {"QueueUrl": url, "MessageBody": "no-policy"}, url)+ assert exc.value.code == "AccessDenied"+ assert exc.value.status == 403++ def test_send_allowed_with_policy(self, auth):+ name = _uniq("xpol-q")+ url = _make_queue(ACCT_A, name, _allow_account(+ _qarn(ACCT_A, name), ACCT_B, "sqs:SendMessage"))+ _as(ACCT_B, sqs_svc._act_send_message,+ {"QueueUrl": url, "MessageBody": "with-policy"}, url)+ assert _queue_bodies(_qarn(ACCT_A, name)) == ["with-policy"]++ def test_send_allowed_via_add_permission(self, auth):+ name = _uniq("xpol-q")+ url = _make_queue(ACCT_A, name)+ _as(ACCT_A, sqs_svc._act_add_permission,+ {"QueueUrl": url, "Label": "xacct",+ "AWSAccountIds": [ACCT_B], "Actions": ["SendMessage"]}, url)+ _as(ACCT_B, sqs_svc._act_send_message,+ {"QueueUrl": url, "MessageBody": "via-add-permission"}, url)+ assert _queue_bodies(_qarn(ACCT_A, name)) == ["via-add-permission"]++ def test_explicit_deny_blocks_same_account(self, auth):+ name = _uniq("xpol-q")+ url = _make_queue(ACCT_A, name, _deny_all(+ _qarn(ACCT_A, name), "sqs:SendMessage"))+ with pytest.raises(sqs_svc._Err) as exc:+ _as(ACCT_A, sqs_svc._act_send_message,+ {"QueueUrl": url, "MessageBody": "deny-me"}, url)+ assert exc.value.code == "AccessDenied"++ def test_get_queue_url_with_owner_account(self, auth):+ name = _uniq("xpol-q")+ url = _make_queue(ACCT_A, name, _allow_account(+ _qarn(ACCT_A, name), ACCT_B, "sqs:GetQueueUrl"))+ found = _as(ACCT_B, sqs_svc._act_get_queue_url,+ {"QueueName": name,+ "QueueOwnerAWSAccountId": ACCT_A}, "")["QueueUrl"]+ assert found == url++ def test_get_queue_url_cross_account_denied_without_policy(self, auth):+ name = _uniq("xpol-q")+ _make_queue(ACCT_A, name)+ with pytest.raises(sqs_svc._Err) as exc:+ _as(ACCT_B, sqs_svc._act_get_queue_url,+ {"QueueName": name,+ "QueueOwnerAWSAccountId": ACCT_A}, "")+ assert exc.value.code == "AccessDenied"+ assert exc.value.status == 403++ def test_get_queue_url_same_account_denied_by_explicit_deny(self, auth):+ name = _uniq("xpol-q")+ _make_queue(ACCT_A, name, _deny_all(+ _qarn(ACCT_A, name), "sqs:GetQueueUrl"))+ with pytest.raises(sqs_svc._Err) as exc:+ _as(ACCT_A, sqs_svc._act_get_queue_url,+ {"QueueName": name}, "")+ assert exc.value.code == "AccessDenied"+ assert exc.value.status == 403++ def test_get_queue_url_same_account_passes_without_policy(self, auth):+ name = _uniq("xpol-q")+ url = _make_queue(ACCT_A, name)+ assert _as(ACCT_A, sqs_svc._act_get_queue_url,+ {"QueueName": name}, "") == {"QueueUrl": url}+++# ---------------------------------------------------------------------------+# Cross-account SNS (+ AddPermission/RemovePermission)+# ---------------------------------------------------------------------------++class TestCrossAccountSns:+ def test_publish_denied_by_default(self, auth):+ arn = _make_topic(ACCT_A, _uniq("xpol-t"))+ assert _resp_code(_as(ACCT_B, sns_svc._publish, {+ "TopicArn": arn, "Message": "no-grant"})) == (403, "AuthorizationError")++ def test_publish_allowed_with_policy(self, auth):+ name = _uniq("xpol-t")+ arn = _make_topic(ACCT_A, name)+ _set_topic_policy(ACCT_A, arn, _allow_account(+ arn, ACCT_B, "sns:Publish"))+ assert _resp_code(_as(ACCT_B, sns_svc._publish, {+ "TopicArn": arn, "Message": "granted"})) == (200, "")+ topic = sns_svc._topic_by_arn_any_scope(arn)+ assert [m["message"] for m in topic["messages"]] == ["granted"]++ def test_explicit_deny_blocks_same_account(self, auth):+ name = _uniq("xpol-t")+ arn = _make_topic(ACCT_A, name)+ _set_topic_policy(ACCT_A, arn, _deny_all(arn, "sns:Publish"))+ assert _resp_code(_as(ACCT_A, sns_svc._publish, {+ "TopicArn": arn, "Message": "deny-me"})) == (403, "AuthorizationError")++ def test_add_permission_grants_cross_account_publish(self, auth):+ arn = _make_topic(ACCT_A, _uniq("xpol-t"))+ assert _resp_code(_as(ACCT_A, sns_svc._add_permission, {+ "TopicArn": arn, "Label": "perm-1",+ "AWSAccountId.member.1": ACCT_B,+ "ActionName.member.1": "Publish"})) == (200, "")+ stmt = next(s for s in _topic_policy(arn)["Statement"]+ if s.get("Sid") == "perm-1")+ assert ACCT_B in stmt["Principal"]["AWS"]+ assert "sns:Publish" in stmt["Action"]+ assert _resp_code(_as(ACCT_B, sns_svc._publish, {+ "TopicArn": arn,+ "Message": "via-add-permission"})) == (200, "")++ def test_add_permission_rejects_duplicate_label(self, auth):+ arn = _make_topic(ACCT_A, _uniq("xpol-t"))+ params = {"TopicArn": arn, "Label": "dup",+ "AWSAccountId.member.1": ACCT_B,+ "ActionName.member.1": "Publish"}+ assert _resp_code(_as(ACCT_A, sns_svc._add_permission, params)) == (200, "")+ assert _resp_code(+ _as(ACCT_A, sns_svc._add_permission, params)) == (+ 400, "InvalidParameter")++ def test_cross_account_tag_access_needs_grant(self, auth):+ name = _uniq("xpol-t")+ arn = _make_topic(ACCT_A, name)+ assert _resp_code(_as(ACCT_B, sns_svc._list_tags_for_resource, {+ "ResourceArn": arn})) == (403, "AuthorizationError")+ _set_topic_policy(ACCT_A, arn, _allow_account(+ arn, ACCT_B, ["sns:TagResource", "sns:ListTagsForResource"]))+ assert _resp_code(_as(ACCT_B, sns_svc._tag_resource, {+ "ResourceArn": arn, "Tags.member.1.Key": "k",+ "Tags.member.1.Value": "v"})) == (200, "")+ assert _resp_code(_as(ACCT_B, sns_svc._list_tags_for_resource, {+ "ResourceArn": arn})) == (200, "")+ assert sns_svc._topic_by_arn_any_scope(arn)["tags"] == {"k": "v"}++ def test_remove_permission_drops_statement(self, auth):+ arn = _make_topic(ACCT_A, _uniq("xpol-t"))+ _as(ACCT_A, sns_svc._add_permission, {+ "TopicArn": arn, "Label": "drop",+ "AWSAccountId.member.1": ACCT_B,+ "ActionName.member.1": "Publish"})+ assert _resp_code(_as(ACCT_B, sns_svc._publish, {+ "TopicArn": arn, "Message": "before"})) == (200, "")+ assert _resp_code(_as(ACCT_A, sns_svc._remove_permission, {+ "TopicArn": arn, "Label": "drop"})) == (200, "")+ assert all(s.get("Sid") != "drop"+ for s in _topic_policy(arn)["Statement"])+ assert _resp_code(_as(ACCT_B, sns_svc._publish, {+ "TopicArn": arn, "Message": "after"})) == (403, "AuthorizationError")+++# ---------------------------------------------------------------------------+# S3 notifications gated on the destination policy+# ---------------------------------------------------------------------------++def _bucket_and_queue(tag):+ bucket, qname = _uniq(f"{tag}-bkt"), _uniq(f"{tag}-q")+ _make_bucket(ACCT_A, bucket)+ url = _make_queue(ACCT_A, qname)+ return bucket, url, _qarn(ACCT_A, qname)+++def _assert_put_rejected(bucket, kind, arn):+ status, code, msg = _resp_detail(_put_notif(ACCT_A, bucket, kind, arn))+ assert status == 400+ assert code == "InvalidArgument"+ assert "Unable to validate the following destination configurations" in msg+++def _bucket_topic_queue(tag):+ bucket = _uniq(f"{tag}-bkt")+ _make_bucket(ACCT_A, bucket)+ topic = _make_topic(ACCT_A, _uniq(f"{tag}-t"))+ qname = _uniq(f"{tag}-q")+ qurl = _make_queue(ACCT_A, qname,+ sqs_policy_allow_sns(_qarn(ACCT_A, qname), topic))+ assert _resp_code(_as(ACCT_A, sns_svc._subscribe, {+ "TopicArn": topic, "Protocol": "sqs",+ "Endpoint": _qarn(ACCT_A, qname)})) == (200, "")+ return bucket, topic, qurl, _qarn(ACCT_A, qname)+++class TestS3NotificationPolicies:+ def test_s3_put_rejected_without_queue_policy(self, auth):+ bucket, _url, arn = _bucket_and_queue("xpol")+ _assert_put_rejected(bucket, "sqs", arn)++ def test_s3_put_rejected_by_unrelated_policy(self, auth):+ bucket, url, arn = _bucket_and_queue("xpol")+ _as(ACCT_A, sqs_svc._act_set_queue_attributes,+ {"QueueUrl": url, "Attributes": {"Policy": json.dumps(+ _allow_account(arn, ACCT_B, "sqs:SendMessage"))}}, url)+ _assert_put_rejected(bucket, "sqs", arn)++ def test_s3_put_rejected_by_wrong_source_arn(self, auth):+ bucket, url, arn = _bucket_and_queue("xpol")+ _as(ACCT_A, sqs_svc._act_set_queue_attributes,+ {"QueueUrl": url, "Attributes": {"Policy": json.dumps(+ sqs_policy_allow_s3(arn, "no-such-bucket", ACCT_A))}}, url)+ _assert_put_rejected(bucket, "sqs", arn)++ def test_s3_to_sqs_delivered_with_policy(self, auth):+ bucket, url, arn = _bucket_and_queue("xpol")+ _as(ACCT_A, sqs_svc._act_set_queue_attributes,+ {"QueueUrl": url, "Attributes": {"Policy": json.dumps(+ sqs_policy_allow_s3(arn, bucket, ACCT_A))}}, url)+ assert _resp_code(_put_notif(ACCT_A, bucket, "sqs", arn)) == (200, "")+ bodies = _queue_bodies(arn)+ assert len(bodies) == 1+ assert "s3:TestEvent" in bodies[0]++ def test_s3_skip_validation_stores_but_delivery_blocked(self, auth):+ bucket, _url, arn = _bucket_and_queue("xpol")+ assert _resp_code(+ _put_notif(ACCT_A, bucket, "sqs", arn, skip=True)) == (200, "")+ assert _queue_bodies(arn) == []+ _as(ACCT_A, s3_svc._deliver_event_to_sqs,+ arn, {"Records": []}, REGION, bucket_name=bucket)+ assert _queue_bodies(arn) == []++ def test_s3_to_sns_default_policy_delivers(self, auth):+ bucket, topic, _qurl, qarn = _bucket_topic_queue("xpol")+ assert _resp_code(_put_notif(ACCT_A, bucket, "sns", topic)) == (200, "")+ bodies = _queue_bodies(qarn)+ assert len(bodies) == 1+ assert "s3:TestEvent" in bodies[0]++ def test_s3_put_rejected_by_topic_policy_without_s3(self, auth):+ bucket, topic, _qurl, _qarn = _bucket_topic_queue("xpol")+ _set_topic_policy(ACCT_A, topic, _allow_account(+ topic, ACCT_A, "sns:Publish"))+ _assert_put_rejected(bucket, "sns", topic)++ def test_s3_to_sns_skip_validation_delivery_blocked(self, auth):+ bucket, topic, _qurl, qarn = _bucket_topic_queue("xpol")+ _set_topic_policy(ACCT_A, topic, _allow_account(+ topic, ACCT_A, "sns:Publish"))+ assert _resp_code(+ _put_notif(ACCT_A, bucket, "sns", topic, skip=True)) == (200, "")+ _as(ACCT_A, s3_svc._deliver_event_to_sns,+ topic, {"Records": []}, REGION, bucket_name=bucket)+ assert _queue_bodies(qarn) == []++ def test_s3_to_cross_account_sns_topic_delivered_with_policy(self, auth):+ bucket = _uniq("xpol-xbkt")+ _make_bucket(ACCT_A, bucket)+ topic = _make_topic(ACCT_B, _uniq("xpol-xt"))+ _set_topic_policy(ACCT_B, topic, {+ "Version": "2012-10-17", "Statement": [{+ "Sid": "s3-xacct", "Effect": "Allow",+ "Principal": {"Service": "s3.amazonaws.com"},+ "Action": "sns:Publish", "Resource": topic,+ "Condition": {+ "ArnLike": {"aws:SourceArn": f"arn:aws:s3:::{bucket}"},+ "StringEquals": {"aws:SourceAccount": ACCT_A}}}]})+ qname = _uniq("xpol-xq")+ _make_queue(ACCT_B, qname, sqs_policy_allow_sns(+ _qarn(ACCT_B, qname), topic))+ assert _resp_code(_as(ACCT_B, sns_svc._subscribe, {+ "TopicArn": topic, "Protocol": "sqs",+ "Endpoint": _qarn(ACCT_B, qname)})) == (200, "")+ # The topic lives in another account, but its policy allows this+ # bucket's deliveries, so the PUT validates instead of failing.+ assert _resp_code(_put_notif(ACCT_A, bucket, "sns", topic)) == (200, "")+ bodies = _queue_bodies(_qarn(ACCT_B, qname))+ assert len(bodies) == 1+ assert "s3:TestEvent" in bodies[0]++ def test_s3_to_cross_account_sqs_queue_delivered_with_policy(self, auth):+ bucket = _uniq("xpol-xbkt")+ _make_bucket(ACCT_A, bucket)+ qname = _uniq("xpol-xq")+ _make_queue(ACCT_B, qname, sqs_policy_allow_s3(+ _qarn(ACCT_B, qname), bucket, ACCT_A))+ assert _resp_code(+ _put_notif(ACCT_A, bucket, "sqs", _qarn(ACCT_B, qname))) == (200, "")+ bodies = _queue_bodies(_qarn(ACCT_B, qname))+ assert len(bodies) == 1+ assert "s3:TestEvent" in bodies[0]++ def test_s3_to_cross_account_topic_rejected_without_policy(self, auth):+ bucket = _uniq("xpol-xbkt")+ _make_bucket(ACCT_A, bucket)+ topic = _make_topic(ACCT_B, _uniq("xpol-xt"))+ _assert_put_rejected(bucket, "sns", topic)+++# ---------------------------------------------------------------------------+# SNS→SQS fanout gated on the queue policy+# ---------------------------------------------------------------------------++def _topic_and_queue(tag):+ topic = _make_topic(ACCT_A, _uniq(f"{tag}-t"))+ qname = _uniq(f"{tag}-q")+ qurl = _make_queue(ACCT_A, qname)+ assert _resp_code(_as(ACCT_A, sns_svc._subscribe, {+ "TopicArn": topic, "Protocol": "sqs",+ "Endpoint": _qarn(ACCT_A, qname)})) == (200, "")+ return topic, qurl, _qarn(ACCT_A, qname)+++class TestSnsFanoutPolicies:+ def test_fanout_blocked_without_policy(self, auth):+ topic, _qurl, qarn = _topic_and_queue("xpol-fan")+ assert _resp_code(_as(ACCT_A, sns_svc._publish, {+ "TopicArn": topic, "Message": "no-policy"})) == (200, "")+ assert _queue_bodies(qarn) == []++ def test_fanout_blocked_by_wrong_source_arn(self, auth):+ topic, qurl, qarn = _topic_and_queue("xpol-fan")+ _as(ACCT_A, sqs_svc._act_set_queue_attributes,+ {"QueueUrl": qurl, "Attributes": {"Policy": json.dumps(+ sqs_policy_allow_sns(+ qarn, f"arn:aws:sns:{REGION}:{ACCT_A}:wrong-topic"))}},+ qurl)+ assert _resp_code(_as(ACCT_A, sns_svc._publish, {+ "TopicArn": topic, "Message": "wrong-source"})) == (200, "")+ assert _queue_bodies(qarn) == []++ def test_fanout_delivered_with_policy(self, auth):+ topic, qurl, qarn = _topic_and_queue("xpol-fan")+ _as(ACCT_A, sqs_svc._act_set_queue_attributes,+ {"QueueUrl": qurl, "Attributes": {"Policy": json.dumps(+ sqs_policy_allow_sns(qarn, topic))}}, qurl)+ assert _resp_code(_as(ACCT_A, sns_svc._publish, {+ "TopicArn": topic, "Message": "with-policy"})) == (200, "")+ bodies = _queue_bodies(qarn)+ assert len(bodies) == 1+ assert "with-policy" in bodies[0]++ def test_cross_account_fanout_needs_queue_policy(self, auth):+ topic = _make_topic(ACCT_A, _uniq("xpol-xfan"))+ _set_topic_policy(ACCT_A, topic, _allow_account(+ topic, ACCT_B, "sns:Subscribe"))+ qname = _uniq("xpol-xfanq")+ qurl = _make_queue(ACCT_B, qname)+ assert _resp_code(_as(ACCT_B, sns_svc._subscribe, {+ "TopicArn": topic, "Protocol": "sqs",+ "Endpoint": _qarn(ACCT_B, qname)})) == (200, "")+ assert _resp_code(_as(ACCT_A, sns_svc._publish, {+ "TopicArn": topic, "Message": "no-queue-policy"})) == (200, "")+ assert _queue_bodies(_qarn(ACCT_B, qname)) == []++ _as(ACCT_B, sqs_svc._act_set_queue_attributes,+ {"QueueUrl": qurl, "Attributes": {"Policy": json.dumps(+ sqs_policy_allow_sns(_qarn(ACCT_B, qname), topic))}}, qurl)+ assert _resp_code(_as(ACCT_A, sns_svc._publish, {+ "TopicArn": topic, "Message": "with-queue-policy"})) == (200, "")+ bodies = _queue_bodies(_qarn(ACCT_B, qname))+ assert len(bodies) == 1+ assert "with-queue-policy" in bodies[0]++ def test_cross_account_subscribe_denied_without_policy(self, auth):+ topic = _make_topic(ACCT_A, _uniq("xpol-xfan"))+ assert _resp_code(_as(ACCT_B, sns_svc._subscribe, {+ "TopicArn": topic, "Protocol": "email",+ "Endpoint": "[email protected]"})) == (403, "AuthorizationError")+++# ---------------------------------------------------------------------------+# AUTH=false stays permissive+# ---------------------------------------------------------------------------++class TestAuthDisabledIsPermissive:+ def test_cross_account_send_passes_without_policy(self, noauth):+ name = _uniq("xpol-q")+ url = _make_queue(ACCT_A, name)+ _as(ACCT_B, sqs_svc._act_send_message,+ {"QueueUrl": url, "MessageBody": "open"}, url)+ assert _queue_bodies(_qarn(ACCT_A, name)) == ["open"]++ def test_cross_account_publish_passes_without_policy(self, noauth):+ arn = _make_topic(ACCT_A, _uniq("xpol-t"))+ assert _resp_code(_as(ACCT_B, sns_svc._publish, {+ "TopicArn": arn, "Message": "open"})) == (200, "")++ def test_explicit_deny_ignored(self, noauth):+ name = _uniq("xpol-q")+ url = _make_queue(ACCT_A, name, _deny_all(+ _qarn(ACCT_A, name), "sqs:SendMessage"))+ _as(ACCT_A, sqs_svc._act_send_message,+ {"QueueUrl": url, "MessageBody": "deny-ignored"}, url)+ assert _queue_bodies(_qarn(ACCT_A, name)) == ["deny-ignored"]++ def test_get_queue_url_ignores_deny(self, noauth):+ name = _uniq("xpol-q")+ url = _make_queue(ACCT_A, name, _deny_all(+ _qarn(ACCT_A, name), "sqs:GetQueueUrl"))+ assert _as(ACCT_A, sqs_svc._act_get_queue_url,+ {"QueueName": name}, "") == {"QueueUrl": url}++ def test_s3_put_and_delivery_without_policy(self, noauth):+ bucket, _url, arn = _bucket_and_queue("xpol")+ assert _resp_code(_put_notif(ACCT_A, bucket, "sqs", arn)) == (200, "")+ bodies = _queue_bodies(arn)+ assert len(bodies) == 1+ assert "s3:TestEvent" in bodies[0]++ def test_fanout_without_policy(self, noauth):+ topic, _qurl, qarn = _topic_and_queue("xpol-fan")+ assert _resp_code(_as(ACCT_A, sns_svc._publish, {+ "TopicArn": topic, "Message": "open"})) == (200, "")+ assert len(_queue_bodies(qarn)) == 1diff --git a/tests/test_sts.py b/tests/test_sts.pyindex 77ee1e3e..d1212943 100644--- a/tests/test_sts.py+++ b/tests/test_sts.py@@ -190,7 +190,7 @@ def test_sts_get_session_token_rejects_unknown_caller_when_auth_enabled(monkeypa)assert status == 403- assert b"UnrecognizedClientException" in payload+ assert b"InvalidClientTokenId" in payloadassert issued_key not in sts_mod._sessions@@ -708,12 +708,7 @@ def test_sts_get_caller_identity_credential_rejections_require_auth(if not auth_enabled and credential_kind == "unknown":assert b":root</Arn>" in payloadif auth_enabled:- expected = (- b"UnrecognizedClientException"- if credential_kind == "unknown"- else b"InvalidClientTokenId"- )- assert expected in payload+ assert b"InvalidClientTokenId" in payloadfinally:iam_svc._access_keys.pop_scoped(account, None, key, None)diff --git a/tests/test_tls.py b/tests/test_tls.pyindex 236ff7ff..e0738c7e 100644--- a/tests/test_tls.py+++ b/tests/test_tls.py@@ -167,13 +167,31 @@ def test_generated_cert_names_the_cognito_issuer_host(tmp_path, monkeypatch):monkeypatch.delenv("MINISTACK_SSL_CERT", raising=False)monkeypatch.delenv("MINISTACK_SSL_KEY", raising=False)cert_path, _key = tls.resolve_tls_material()- assert "DNS:cognito-idp.eu-west-2.amazonaws.com" in _san_of(cert_path)+ san = _san_of(cert_path)+ assert "DNS:cognito-idp.eu-west-2.amazonaws.com" in san+ assert "DNS:cognito-idp.ap-southeast-2.amazonaws.com" in sanassert tls.cognito_idp_host() == "cognito-idp.eu-west-2.amazonaws.com"-def test_cached_cert_without_the_issuer_host_is_regenerated(tmp_path, monkeypatch):- """A cert cached by an older build, or for another region, cannot serve the- issuer host, so reusing it would fail the handshake."""+def test_cached_cert_without_the_issuer_hosts_is_regenerated(tmp_path, monkeypatch):+ """A cert cached by an older build lacks the issuer hosts and is replaced."""+ from ministack.core import tls++ monkeypatch.setenv("TMPDIR", str(tmp_path))+ monkeypatch.delenv("MINISTACK_SSL_CERT", raising=False)+ monkeypatch.delenv("MINISTACK_SSL_KEY", raising=False)+ tls_dir = tmp_path / "ministack-tls"+ tls_dir.mkdir()+ subprocess.run([+ "openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes",+ "-keyout", str(tls_dir / "server.key"), "-out", str(tls_dir / "server.crt"),+ "-days", "1", "-subj", "/CN=old", "-addext", "subjectAltName=DNS:localhost",+ ], check=True, capture_output=True)+ cert, _ = tls.resolve_tls_material()+ assert "DNS:cognito-idp.us-east-1.amazonaws.com" in _san_of(cert)+++def test_cached_cert_is_reused_across_regions(tmp_path, monkeypatch):from ministack.core import tlsmonkeypatch.setenv("TMPDIR", str(tmp_path))@@ -182,11 +200,9 @@ def test_cached_cert_without_the_issuer_host_is_regenerated(tmp_path, monkeypatcmonkeypatch.setenv("MINISTACK_REGION", "eu-west-2")first, _ = tls.resolve_tls_material()first_bytes = open(first, "rb").read()-monkeypatch.setenv("MINISTACK_REGION", "us-east-1")second, _ = tls.resolve_tls_material()- assert "DNS:cognito-idp.us-east-1.amazonaws.com" in _san_of(second)- assert open(second, "rb").read() != first_bytes+ assert open(second, "rb").read() == first_bytesdef test_byo_certificate_is_never_regenerated(tmp_path, monkeypatch):