Light, fluffy, and always free - The AWS Local Emulator alternative
chore(tooling): gate partition literals in src/main against a per-file baseline (#4270)
Hector Ventura0a601d54ebde
diff --git a/.github/workflows/partition-literals.yml b/.github/workflows/partition-literals.ymlnew file mode 100644index 0000000000..39aa463019--- /dev/null+++ b/.github/workflows/partition-literals.yml@@ -0,0 +1,58 @@+name: Partition Literals++# Keeps the commercial-partition assumption from growing back while floci moves to+# serving every AWS partition. tools/partition/partition_literals.py inventories the+# hardcoded arn:aws: prefixes, amazonaws.com hosts, hosted-zone ids and hand-rolled+# partition regexes in src/main/java against the checked-in per-file baseline: a+# literal in a new file fails, growth in an existing file fails, and a drop fails until+# the baseline is regenerated so the remaining count stays exact. Python-only, so it is+# fast and independent of the Java build job.++on:+ pull_request:+ paths:+ - 'src/main/java/**'+ - 'tools/partition/**'+ - 'Makefile'+ - '.github/workflows/partition-literals.yml'+ push:+ branches:+ - main+ paths:+ - 'src/main/java/**'+ - 'tools/partition/**'+ - 'Makefile'+ - '.github/workflows/partition-literals.yml'++permissions:+ contents: read++concurrency:+ group: ${{ github.workflow }}-${{ github.ref }}+ cancel-in-progress: true++jobs:+ partition-literals:+ name: Partition literals match baseline+ runs-on: ubuntu-latest+ timeout-minutes: 5+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1++ - name: Set up Python+ uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0+ with:+ python-version: "3.x"++ - name: Install tooling deps+ run: pip install -r tools/partition/requirements.txt++ - name: Run partition tooling tests+ run: make partition-test++ - name: Check partition literals against the baseline+ run: make partition-check++ - name: Print the remaining-literal audit+ if: always()+ run: make partition-auditdiff --git a/AGENTS.md b/AGENTS.mdindex f88bf47175..3bbeb64fb4 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -111,6 +111,25 @@ Floci must implement real AWS wire protocols.---+### Partition literals++Floci serves every AWS partition, not only the commercial one, so a literal that bakes+`aws` into a code path is a bug in a GovCloud, China, ISO or EUSC deployment even when it+prints correctly in `us-east-1`. `make partition-check` (CI: Partition Literals) inventories+`src/main/java` for `arn:aws:` prefixes, `amazonaws.com` hosts, Route 53 hosted-zone ids+and hand-rolled `arn:aws[a-z-]*:` regexes against `tools/partition/baseline.tsv`, per file.+A literal in a new file fails, growth in an existing file fails, and a drop fails until you+run `make partition-baseline` and commit the smaller baseline.++- Mint ARNs through `AwsArnUtils.Arn.of(...)` / `RegionResolver.buildArn(...)`, recognise+ them with `AwsArnUtils.isArn` / `PARTITION_REGEX`, and derive hosts from the region's+ DNS suffix (`AwsRegions.dnsSuffixFor`).+- A literal that is genuinely partition-invariant (an XML namespace URI, an S3 canned-ACL+ grantee URI) goes in `tools/partition/allowlist.yaml` with a reason, or ends its line+ with `// partition-literal: <reason>`. Both are printed by `make partition-audit`.+- Tests are not scanned; only assert a partition-dependent value when the test is about+ partitions.+## XML / JSON Rules- Use `XmlBuilder` for XML responsesdiff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex df772fcf91..f686420ff3 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -317,6 +317,19 @@ hand-written and preserved across regeneration, keyed by action name.Registering a new service's action table is one entry in `tools/docs/services.yaml`.+### Partition literals++Floci serves every AWS partition, so `src/main/java` must not bake the commercial one in.+`make partition-check` (run in CI) inventories `arn:aws:` prefixes, `amazonaws.com` hosts,+Route 53 hosted-zone ids and hand-rolled partition regexes per file against+`tools/partition/baseline.tsv`: a literal in a new file fails, growth fails, and a drop+fails until you run `make partition-baseline` and commit the smaller baseline.+`make partition-audit` prints what is left per package; `make partition-test` runs the+tooling's unit tests. Mint ARNs through `AwsArnUtils` and derive hosts from the region's+DNS suffix; a literal that really is partition-invariant goes in+`tools/partition/allowlist.yaml` with a reason, or ends its line with+`// partition-literal: <reason>`.+## Reporting Security IssuesPlease do **not** open public issues for security vulnerabilities. Report them privately by emailing the maintainer or using [GitHub private vulnerability reporting](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing/privately-reporting-a-security-vulnerability).diff --git a/Makefile b/Makefileindex 4c192df2d5..6945e9cb5a 100644--- a/Makefile+++ b/Makefile@@ -5,6 +5,10 @@## Service-matrix docs: docs/services/index.md's Service Matrix table is checked# against ResolvedServiceCatalog.java so a registered service can't ship undocumented.+#+# Partition literals: hardcoded arn:aws: prefixes, amazonaws.com hosts and hosted-zone+# ids in src/main are inventoried against tools/partition/baseline.tsv so the+# commercial-partition assumption can only shrink. See tools/partition/.PYTHON ?= python3@@ -34,7 +38,7 @@ PREFIX ?= $(HOME)/.local.DEFAULT_GOAL := help.PHONY: help dev build test native native-host run-native native-install native-image native-up native-down native-logs clean-sidecars clean-volumes compat docker-tests \- docs-sync docs-check docs-test+ docs-sync docs-check docs-test partition-check partition-baseline partition-audit partition-testhelp: ## List the targets below@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf " %-14s %s\n", $$1, $$2}'@@ -145,3 +149,20 @@ docs-check: ## CI gate: regenerate and fail if anything is stale, unregistered,docs-test: ## Run the docs tooling's unit tests$(PYTHON) -m pytest tools/docs -q++partition-check: ## CI gate: partition literals in src/main must match tools/partition/baseline.tsv+ @$(PYTHON) tools/partition/partition_literals.py --check || { \+ echo ""; \+ echo "error: partition literals drifted from tools/partition/baseline.tsv (see above)."; \+ echo " Fix the new literal, or run 'make partition-baseline' after removing some."; \+ exit 1; \+ }++partition-baseline: ## Regenerate tools/partition/baseline.tsv from the current tree (commit the result)+ $(PYTHON) tools/partition/partition_literals.py --write-baseline++partition-audit: ## Print the per-package table of remaining partition literals+ @$(PYTHON) tools/partition/partition_literals.py --audit++partition-test: ## Run the partition tooling's unit tests+ $(PYTHON) -m pytest tools/partition -qdiff --git a/src/test/java/io/github/hectorvent/floci/services/resourceexplorer2/ResourceExplorer2IntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/resourceexplorer2/ResourceExplorer2IntegrationTest.javaindex c52388a193..620c3b3720 100644--- a/src/test/java/io/github/hectorvent/floci/services/resourceexplorer2/ResourceExplorer2IntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/resourceexplorer2/ResourceExplorer2IntegrationTest.java@@ -1,5 +1,8 @@package io.github.hectorvent.floci.services.resourceexplorer2;+import io.github.hectorvent.floci.core.common.AwsArnUtils;+import io.github.hectorvent.floci.core.common.AwsArnUtils.Arn;+import io.github.hectorvent.floci.core.common.AwsRegions;import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;import io.quarkus.test.junit.QuarkusTest;import org.junit.jupiter.api.AfterEach;@@ -1503,8 +1506,16 @@ class ResourceExplorer2IntegrationTest {for (var resource : resources) {String arn = (String) resource.get("Arn");String service = (String) resource.get("Service");- assertTrue(arn.startsWith("arn:aws:"),- "ARN must start with arn:aws:, got: " + arn);+ Arn parsed = AwsArnUtils.parse(arn);+ String expectedPartition = AwsRegions.partitionFor(parsed.region());+ // Only commercial regions are held to their partition here: resources another test+ // leaves in a China or GovCloud region are still minted arn:aws: by build sites that+ // have not moved to Arn.of yet, which the partition plan's build-site phase fixes and+ // then tightens this assertion to every region.+ if ("aws".equals(expectedPartition)) {+ assertEquals(expectedPartition, parsed.partition(),+ "ARN partition must be the one its region belongs to, got: " + arn);+ }assertTrue(arn.contains(service),"ARN should contain service '" + service + "', got: " + arn);}diff --git a/tools/partition/allowlist.yaml b/tools/partition/allowlist.yamlnew file mode 100644index 0000000000..548fafeb54--- /dev/null+++ b/tools/partition/allowlist.yaml@@ -0,0 +1,22 @@+# Literals partition_literals.py must not count. Every entry needs a reason a reviewer can+# check; the audit prints how many occurrences each entry excuses.+#+# files: the whole file is exempt (path relative to the repo root)+# patterns: a Python regex matched against the full source line of the literal++files:+ - path: src/main/java/io/github/hectorvent/floci/core/common/AwsNamespaces.java+ reason: XML namespace URIs (http://s3.amazonaws.com/doc/2006-03-01/ and the Query+ namespaces) are identifiers, not hosts, and are byte-identical in every partition;+ minio and the China/GovCloud SDKs emit the same strings.++patterns:+ - regex: amazonaws\.com/doc/\d{4}-\d{2}-\d{2}/+ reason: an XML namespace URI outside AwsNamespaces (CloudFormation keeps its own);+ the same partition-invariant identifier class as that file.+ - regex: acs\.amazonaws\.com/groups/+ reason: S3 canned-ACL grantee URIs (AllUsers, AuthenticatedUsers, LogDelivery) are+ partition-invariant identifiers in the S3 model.+ - regex: \bPARTITION_REGEX\b+ reason: the line builds its regex from AwsArnUtils.PARTITION_REGEX, which is the+ sanctioned way to recognise an ARN in any partition.diff --git a/tools/partition/baseline.tsv b/tools/partition/baseline.tsvnew file mode 100644index 0000000000..1e23f4d3ef--- /dev/null+++ b/tools/partition/baseline.tsv@@ -0,0 +1,136 @@+# category path count+# Generated by tools/partition/partition_literals.py --write-baseline; do not hand-edit.+arn-literal src/main/java/io/github/hectorvent/floci/core/common/IamEnforcementFilter.java 8+arn-literal src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayController.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/apigateway/ExecuteApiSigV4Authorizer.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/applicationautoscaling/ApplicationAutoScalingService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/appsync/graphql/auth/IamAuthValidator.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreCredentialProviderService.java 4+arn-literal src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreToolsService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessor.java 3+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/ApiGatewayDomainCfnProvisioner.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/CfnResourceDispatcher.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/CodeBuildCfnProvisioner.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/RedshiftClusterCfnProvisioner.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudhsmv2/CloudHsmV2JsonHandler.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudtrail/CloudTrailService.java 7+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsCrossAccountService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactController.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/connect/ConnectService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/controlcatalog/ControlCatalogService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/controltower/ControlTowerControlService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/controltower/ControlTowerService.java 4+arn-literal src/main/java/io/github/hectorvent/floci/services/controltower/SeededLandingZoneFactory.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/datasync/DataSyncService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/detective/DetectiveService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/ec2/Ec2IpamService.java 3+arn-literal src/main/java/io/github/hectorvent/floci/services/ec2/Ec2Service.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/ecs/EcsService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/eks/EksClusterManager.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/eks/EksWorkerAuthentication.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/elasticache/ElastiCacheQueryHandler.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/emrserverless/EmrServerlessService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/guardduty/GuardDutyService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/iam/AwsManagedPolicies.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/iam/IamPolicyEvaluator.java 4+arn-literal src/main/java/io/github/hectorvent/floci/services/iam/SAMLAssertionVerifier.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/iam/SAMLProviderService.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/kinesisanalytics/KinesisAnalyticsV2JsonHandler.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/kinesisanalytics/container/FlinkContainerManager.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/kms/KmsService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/lambda/LambdaController.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/lambda/LambdaLayerService.java 3+arn-literal src/main/java/io/github/hectorvent/floci/services/lambdamicrovms/LambdaMicrovmsController.java 3+arn-literal src/main/java/io/github/hectorvent/floci/services/lambdamicrovms/LambdaMicrovmsService.java 3+arn-literal src/main/java/io/github/hectorvent/floci/services/macie2/MacieController.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/macie2/MacieService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/marketplace/MarketplaceDeploymentService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/marketplace/MarketplaceMeteringService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/marketplace/MarketplaceReportingService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/marketplace/MarketplaceService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/networkfirewall/NetworkFirewallService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/oam/OamService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/ram/RamService.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/rds/RdsService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/rds/proxy/RdsSigV4Validator.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/redshift/RedshiftService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/redshift/proxy/S3CopySimulator.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/redshiftdata/RedshiftDataResourceResolver.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/resourceexplorer2/ResourceExplorer2Service.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/route53resolver/Route53ResolverService.java 5+arn-literal src/main/java/io/github/hectorvent/floci/services/s3/S3ControlController.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/s3/S3PublicAccessEvaluator.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/s3/S3ResourcePolicyProvider.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/s3/S3Service.java 4+arn-literal src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/securityhub/SecurityHubService.java 4+arn-literal src/main/java/io/github/hectorvent/floci/services/servicecatalog/ServiceCatalogService.java 7+arn-literal src/main/java/io/github/hectorvent/floci/services/servicequotas/ServiceQuotasService.java 2+arn-literal src/main/java/io/github/hectorvent/floci/services/ses/SesService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/ses/SesTenantService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/signin/SigninService.java 3+arn-literal src/main/java/io/github/hectorvent/floci/services/ssm/SsmService.java 3+arn-literal src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java 32+arn-literal src/main/java/io/github/hectorvent/floci/services/stepfunctions/StepFunctionsService.java 3+arn-literal src/main/java/io/github/hectorvent/floci/services/swf/SwfService.java 1+arn-literal src/main/java/io/github/hectorvent/floci/services/verifiedpermissions/VerifiedPermissionsService.java 2+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayExecuteController.java 1+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreGatewayRuleService.java 3+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreToolsService.java 1+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/budgets/BudgetsService.java 2+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/controltower/ControlTowerControlService.java 1+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/controltower/ControlTowerService.java 1+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/firehose/ProcessingConfigurationValidator.java 1+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/kms/KmsService.java 2+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/lambda/LambdaService.java 1+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/ses/SesReceiptRuleService.java 2+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/ssoadmin/SsoAdminService.java 9+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/ssooidc/SsoOidcService.java 1+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/timestreaminfluxdb/TimestreamInfluxDbValidation.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java 2+dns-suffix src/main/java/io/github/hectorvent/floci/core/common/AwsRegions.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/account/AccountService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayExecuteApiHostFilter.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayExecuteController.java 4+dns-suffix src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/apigatewayv2/ApiGatewayV2Service.java 2+dns-suffix src/main/java/io/github/hectorvent/floci/services/apigatewayv2/websocket/WebSocketProxyEventBuilder.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/applicationautoscaling/ApplicationAutoScalingService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/appsync/AppSyncService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/aps/ApsService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreCredentialProviderService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreGatewayService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationTemplateEngine.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessor.java 4+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/StackSetService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/CognitoCfnProvisioner.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/S3CfnProvisioner.java 4+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudtrail/CloudTrailLogWriter.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudtrail/CloudTrailService.java 5+dns-suffix src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/connect/ConnectService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/ec2/Ec2QueryHandler.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/eks/EksOidcService.java 2+dns-suffix src/main/java/io/github/hectorvent/floci/services/eks/EksPodIdentityCredentialsController.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/eks/EksPodIdentityWebhook.java 2+dns-suffix src/main/java/io/github/hectorvent/floci/services/emr/EmrHandler.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/guardduty/GuardDutyService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/iam/IamService.java 6+dns-suffix src/main/java/io/github/hectorvent/floci/services/iam/SAMLAssertionVerifier.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/iam/StsQueryHandler.java 2+dns-suffix src/main/java/io/github/hectorvent/floci/services/macie2/MacieController.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/ram/RamController.java 2+dns-suffix src/main/java/io/github/hectorvent/floci/services/redshift/proxy/S3CopySimulator.java 2+dns-suffix src/main/java/io/github/hectorvent/floci/services/s3/S3Service.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/s3/S3VirtualHostFilter.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/signin/SigninService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/ssoportal/SsoPortalService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/services/verifiedpermissions/VerifiedPermissionsService.java 1+dns-suffix src/main/java/io/github/hectorvent/floci/tools/ami/AmiImageTool.java 1+hosted-zone-id src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java 2+hosted-zone-id src/main/java/io/github/hectorvent/floci/services/elb/ElbClassicService.java 1+hosted-zone-id src/main/java/io/github/hectorvent/floci/services/elbv2/ElbV2Service.java 1diff --git a/tools/partition/partition_literals.py b/tools/partition/partition_literals.pynew file mode 100644index 0000000000..f3f13d3aaa--- /dev/null+++ b/tools/partition/partition_literals.py@@ -0,0 +1,409 @@+#!/usr/bin/env python3+"""Keep hardcoded AWS-partition assumptions in src/main from growing back.++Floci is moving from a commercial-only (`aws`) emulator to one that serves every AWS+partition (`aws-cn`, `aws-us-gov`, the ISO partitions, `aws-eusc`). The blockers are+string literals that bake the commercial partition into a code path: an `arn:aws:`+prefix, an `amazonaws.com` host, a Route 53 hosted zone that only exists in one+partition, a hand-rolled `arn:aws[a-z-]*:` regex that recognises some partitions but+not others. Each one is a silent bug in a non-commercial deployment, and each one is+trivial to add by accident, because the literal is exactly what an `aws` deployment+prints.++This script inventories those literals in `src/main/java` by category and gates them+against a checked-in per-file baseline (`baseline.tsv`):++- a literal in a file the baseline does not know fails (new files start clean);+- a file whose count grew fails;+- a file whose count dropped also fails, until the baseline is regenerated, so the+ "N remaining" number stays exact and the drop is visible in the PR diff.++Comments never count. A literal that is genuinely partition-invariant is excused in one+of two ways, both of which must state a reason: an entry in `allowlist.yaml` (a whole+file, or a regex matched against the source line), or a trailing+`// partition-literal: <reason>` comment on the line. The audit always prints the+escapes so they are reviewed, not forgotten.++Run from anywhere in the repo:+ python3 tools/partition/partition_literals.py # per-package audit+ python3 tools/partition/partition_literals.py --check # CI gate against baseline.tsv+ python3 tools/partition/partition_literals.py --write-baseline # regenerate baseline.tsv+"""+from __future__ import annotations++import argparse+import re+import sys+from collections import Counter, defaultdict+from dataclasses import dataclass+from pathlib import Path++import yaml++REPO_ROOT = Path(__file__).resolve().parents[2]+SOURCE_ROOT = REPO_ROOT / "src/main/java"+PACKAGE_ROOT = SOURCE_ROOT / "io/github/hectorvent/floci"+TOOL_DIR = Path(__file__).resolve().parent+ALLOWLIST = TOOL_DIR / "allowlist.yaml"+BASELINE = TOOL_DIR / "baseline.tsv"++ESCAPE_RE = re.compile(r"//\s*partition-literal:\s*(?P<reason>\S.*?)\s*$")++REGION_PREFIXES = "us|eu|ap|ca|sa|me|af|il|mx|cn|us-gov|us-iso|us-isob|eu-isoe|us-isof|eusc-de"+++@dataclass(frozen=True)+class Category:+ name: str+ regex: re.Pattern+ gated: bool+ description: str+++# Regexes run against the content of one string literal (one line of a text block), never+# against comments. Order is the audit column order.+CATEGORIES: tuple[Category, ...] = (+ Category(+ "arn-literal",+ re.compile(r"\barn:aws(?:-[a-z]+)*:"),+ True,+ "an ARN prefix naming one partition (arn:aws:, arn:aws-cn:, ...); mint through AwsArnUtils",+ ),+ Category(+ "arn-regex-dialect",+ re.compile(r"arn:aws(?=[\[(\\])"),+ True,+ "a hand-rolled partition alternation in a regex; use AwsArnUtils.PARTITION_REGEX",+ ),+ Category(+ "dns-suffix",+ re.compile(r"amazonaws\.com(?!\.cn)|cloudfront\.net|signin\.aws|public\.ecr\.aws"),+ True,+ "a commercial DNS suffix or host; derive from the partition's dnsSuffix",+ ),+ Category(+ "hosted-zone-id",+ re.compile(r"^Z[0-9A-Z]{12,14}$"),+ True,+ "a Route 53 hosted zone id, which differs per region and partition",+ ),+ Category(+ "region-literal",+ re.compile(+ r"(?<![a-z0-9-])(?:" + REGION_PREFIXES + r")-[a-z]+-\d[a-z]?(?![a-z0-9-])"+ ),+ False,+ "a region or availability-zone id; report-only until the us-east-1 assumptions are swept",+ ),+)++CATEGORY_BY_NAME = {category.name: category for category in CATEGORIES}+GATED = tuple(category for category in CATEGORIES if category.gated)+++@dataclass(frozen=True)+class Literal:+ line: int+ text: str+++@dataclass(frozen=True)+class Finding:+ category: str+ path: str+ line: int+ source: str+ excuse: str | None = None++ @property+ def counted(self) -> bool:+ return self.excuse is None+++def scan_java(text: str) -> tuple[list[Literal], dict[int, str]]:+ """Extracts string literals (one per line for text blocks) and per-line escape comments.++ A tiny lexer rather than a regex: `//` inside a string is a URL, not a comment, and a+ `"` inside a comment or char literal is not a string. Block comments and text blocks+ span lines, so line numbers are tracked by hand.+ """+ literals: list[Literal] = []+ escapes: dict[int, str] = {}+ i = 0+ n = len(text)+ line = 1+ while i < n:+ char = text[i]+ if char == "\n":+ line += 1+ i += 1+ elif text.startswith("//", i):+ end = text.find("\n", i)+ end = n if end < 0 else end+ match = ESCAPE_RE.match(text[i:end])+ if match:+ escapes[line] = match.group("reason")+ i = end+ elif text.startswith("/*", i):+ end = text.find("*/", i + 2)+ end = n if end < 0 else end + 2+ line += text.count("\n", i, end)+ i = end+ elif text.startswith('"""', i):+ j = i + 3+ while j < n and not text.startswith('"""', j):+ j += 2 if text[j] == "\\" else 1+ block = text[i + 3:j]+ for offset, part in enumerate(block.split("\n")):+ if part.strip():+ literals.append(Literal(line + offset, part))+ line += block.count("\n")+ i = min(j + 3, n)+ elif char == '"':+ j = i + 1+ buffer: list[str] = []+ while j < n and text[j] not in '"\n':+ if text[j] == "\\":+ buffer.append(text[j:j + 2])+ j += 2+ else:+ buffer.append(text[j])+ j += 1+ literals.append(Literal(line, "".join(buffer)))+ i = j + 1 if j < n and text[j] == '"' else j+ elif char == "'":+ j = i + 1+ while j < n and text[j] not in "'\n":+ j += 2 if text[j] == "\\" else 1+ i = j + 1 if j < n and text[j] == "'" else j+ else:+ i += 1+ return literals, escapes+++@dataclass(frozen=True)+class AllowRule:+ kind: str+ value: str+ reason: str+++def load_allowlist(path: Path) -> list[AllowRule]:+ if not path.exists():+ return []+ data = yaml.safe_load(path.read_text(encoding="utf-8")) or {}+ rules: list[AllowRule] = []+ for entry in data.get("files") or []:+ rules.append(AllowRule("file", str(entry["path"]), str(entry["reason"])))+ for entry in data.get("patterns") or []:+ rules.append(AllowRule("pattern", str(entry["regex"]), str(entry["reason"])))+ for rule in rules:+ if not rule.reason.strip():+ raise ValueError(f"allowlist entry {rule.value!r} has no reason")+ return rules+++def excuse_for(rel_path: str, source_line: str, rules: list[AllowRule]) -> str | None:+ for rule in rules:+ if rule.kind == "file" and rule.value == rel_path:+ return f"allowlist file: {rule.reason}"+ if rule.kind == "pattern" and re.search(rule.value, source_line):+ return f"allowlist pattern {rule.value!r}: {rule.reason}"+ return None+++def collect_findings(source_root: Path, rules: list[AllowRule]) -> list[Finding]:+ findings: list[Finding] = []+ for java in sorted(source_root.rglob("*.java")):+ rel_path = java.relative_to(REPO_ROOT).as_posix() if java.is_relative_to(REPO_ROOT) \+ else java.relative_to(source_root).as_posix()+ text = java.read_text(encoding="utf-8")+ lines = text.split("\n")+ literals, escapes = scan_java(text)+ for literal in literals:+ source_line = lines[literal.line - 1] if literal.line - 1 < len(lines) else ""+ for category in CATEGORIES:+ hits = len(category.regex.findall(literal.text))+ if hits == 0:+ continue+ excuse = excuse_for(rel_path, source_line, rules)+ if excuse is None and literal.line in escapes:+ excuse = f"escape: {escapes[literal.line]}"+ findings.extend(+ Finding(category.name, rel_path, literal.line, source_line.strip(), excuse)+ for _ in range(hits)+ )+ return findings+++def count(findings: list[Finding], gated_only: bool = False) -> Counter:+ counter: Counter = Counter()+ for finding in findings:+ if not finding.counted:+ continue+ if gated_only and not CATEGORY_BY_NAME[finding.category].gated:+ continue+ counter[(finding.category, finding.path)] += 1+ return counter+++def read_baseline(path: Path) -> Counter:+ counter: Counter = Counter()+ if not path.exists():+ return counter+ for raw in path.read_text(encoding="utf-8").splitlines():+ if not raw.strip() or raw.startswith("#"):+ continue+ parts = raw.split("\t")+ if len(parts) != 3:+ raise ValueError(f"{path}: malformed baseline row {raw!r}")+ category, rel_path, value = parts+ if category not in CATEGORY_BY_NAME:+ raise ValueError(f"{path}: unknown category {category!r}")+ counter[(category, rel_path)] = int(value)+ return counter+++def format_baseline(counter: Counter) -> str:+ rows = ["# category\tpath\tcount", "# Generated by tools/partition/partition_literals.py --write-baseline; do not hand-edit."]+ for (category, rel_path), value in sorted(counter.items()):+ rows.append(f"{category}\t{rel_path}\t{value}")+ return "\n".join(rows) + "\n"+++def write_baseline(path: Path, findings: list[Finding]) -> Counter:+ counter = count(findings, gated_only=True)+ path.write_text(format_baseline(counter), encoding="utf-8")+ return counter+++def check(findings: list[Finding], baseline: Counter) -> list[str]:+ """Returns one problem per (category, file) that drifted from the baseline; empty when green."""+ current = count(findings, gated_only=True)+ by_key: dict[tuple[str, str], list[Finding]] = defaultdict(list)+ for finding in findings:+ if finding.counted:+ by_key[(finding.category, finding.path)].append(finding)+ problems: list[str] = []+ for key in sorted(set(current) | set(baseline)):+ category, rel_path = key+ if not CATEGORY_BY_NAME[category].gated:+ continue+ now = current.get(key, 0)+ before = baseline.get(key, 0)+ if now == before:+ continue+ if key not in baseline:+ head = f"{rel_path}: {now} {category} literal(s) with no baseline entry for this file"+ elif now > before:+ head = f"{rel_path}: {category} grew from {before} to {now}"+ else:+ head = f"{rel_path}: {category} dropped from {before} to {now}; run 'make partition-baseline' and commit baseline.tsv"+ lines = [head]+ if now > before:+ lines.extend(f" {f.path}:{f.line}: {f.source}" for f in by_key[key])+ lines.append(f" {CATEGORY_BY_NAME[category].description}")+ problems.append("\n".join(lines))+ return problems+++def package_of(rel_path: str) -> str:+ try:+ relative = Path(rel_path).relative_to(PACKAGE_ROOT.relative_to(REPO_ROOT))+ except ValueError:+ return Path(rel_path).parent.as_posix()+ parts = relative.parts[:-1]+ return "/".join(parts[:2]) if parts else "(root)"+++def format_audit(findings: list[Finding], baseline: Counter) -> str:+ names = [category.name for category in CATEGORIES]+ per_package: dict[str, Counter] = defaultdict(Counter)+ totals: Counter = Counter()+ for finding in findings:+ if finding.counted:+ per_package[package_of(finding.path)][finding.category] += 1+ totals[finding.category] += 1+ width = max([len("package"), len("total")] + [len(p) for p in per_package]) + 2+ header = f"{'package':<{width}}" + "".join(f"{name:>19}" for name in names) + f"{'total':>8}"+ out = [header, "-" * len(header)]+ for package, counter in sorted(per_package.items(), key=lambda item: (-sum(item[1].values()), item[0])):+ out.append(+ f"{package:<{width}}"+ + "".join(f"{counter.get(name, 0):>19}" for name in names)+ + f"{sum(counter.values()):>8}"+ )+ out.append("-" * len(header))+ out.append(+ f"{'total':<{width}}"+ + "".join(f"{totals.get(name, 0):>19}" for name in names)+ + f"{sum(totals.values()):>8}"+ )+ gate = "".join(+ f"{'gated' if category.gated else 'report-only':>19}" for category in CATEGORIES+ )+ out.append(f"{'':<{width}}{gate}")+ if baseline:+ baseline_totals: Counter = Counter()+ for (category, _), value in baseline.items():+ baseline_totals[category] += value+ out.append("")+ out.append("baseline.tsv: " + ", ".join(+ f"{category.name} {baseline_totals.get(category.name, 0)} -> {totals.get(category.name, 0)}"+ for category in GATED+ ))+ excused = [finding for finding in findings if not finding.counted]+ escapes = sorted({(f.path, f.line, f.excuse) for f in excused if f.excuse and f.excuse.startswith("escape:")})+ allowed: Counter = Counter(+ f.excuse for f in excused if f.excuse and f.excuse.startswith("allowlist")+ )+ out.append("")+ out.append(f"escapes (// partition-literal: <reason>): {len(escapes)}")+ out.extend(f" {path}:{line} {excuse[len('escape: '):]}" for path, line, excuse in escapes)+ out.append(f"allow-listed occurrences: {sum(allowed.values())}")+ out.extend(f" {value:>4} {excuse}" for excuse, value in sorted(allowed.items()))+ return "\n".join(out)+++def main(argv: list[str] | None = None) -> int:+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)+ mode = parser.add_mutually_exclusive_group()+ mode.add_argument("--check", action="store_true", help="exit 1 when a gated count drifted from the baseline")+ mode.add_argument("--write-baseline", action="store_true", help="regenerate the baseline from the current tree")+ mode.add_argument("--audit", action="store_true", help="print the per-package table (the default)")+ parser.add_argument("--source", type=Path, default=SOURCE_ROOT, help=argparse.SUPPRESS)+ parser.add_argument("--allowlist", type=Path, default=ALLOWLIST, help=argparse.SUPPRESS)+ parser.add_argument("--baseline", type=Path, default=BASELINE, help=argparse.SUPPRESS)+ args = parser.parse_args(argv)++ rules = load_allowlist(args.allowlist)+ findings = collect_findings(args.source, rules)++ if args.write_baseline:+ counter = write_baseline(args.baseline, findings)+ print(f"wrote {args.baseline.relative_to(REPO_ROOT) if args.baseline.is_relative_to(REPO_ROOT) else args.baseline}: "+ f"{sum(counter.values())} gated literal(s) across {len({p for _, p in counter})} file(s)")+ return 0++ baseline = read_baseline(args.baseline)+ if args.check:+ problems = check(findings, baseline)+ for problem in problems:+ print(problem)+ if problems:+ print("")+ print(f"error: {len(problems)} file(s) drifted from tools/partition/baseline.tsv. "+ "Route new ARNs through AwsArnUtils and hosts through the partition's dnsSuffix; "+ "when a literal is genuinely partition-invariant, add it to tools/partition/allowlist.yaml "+ "or end the line with '// partition-literal: <reason>'.")+ return 1+ gated_total = sum(count(findings, gated_only=True).values())+ print(f"partition literals match baseline.tsv ({gated_total} gated occurrence(s) remaining)")+ return 0++ print(format_audit(findings, baseline))+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/partition/requirements.txt b/tools/partition/requirements.txtnew file mode 100644index 0000000000..26efb36e3a--- /dev/null+++ b/tools/partition/requirements.txt@@ -0,0 +1,2 @@+PyYAML>=6.0+pytest>=7.0diff --git a/tools/partition/test_partition_literals.py b/tools/partition/test_partition_literals.pynew file mode 100644index 0000000000..899bd4b7e4--- /dev/null+++ b/tools/partition/test_partition_literals.py@@ -0,0 +1,368 @@+"""Tests for partition_literals.++Run with: pytest tools/partition -q (or: make partition-test)+"""+from __future__ import annotations++from collections import Counter+from pathlib import Path++import pytest++import partition_literals as p+++# --------------------------------------------------------------------------- #+# Lexer+# --------------------------------------------------------------------------- #+def literal_texts(src: str) -> list[str]:+ return [literal.text for literal in p.scan_java(src)[0]]+++def test_scan_java_extracts_plain_string_literals_with_line_numbers():+ src = 'String a = "arn:aws:s3:::b";\nString b = "x";\n'+ literals, escapes = p.scan_java(src)+ assert [(l.line, l.text) for l in literals] == [(1, "arn:aws:s3:::b"), (2, "x")]+ assert escapes == {}+++def test_scan_java_ignores_line_and_block_comments():+ src = (+ '// "arn:aws:iam::123:root" in a line comment\n'+ '/* "arn:aws:iam::123:root"\n spanning lines */\n'+ '/** {@code arn:aws-us-gov:iam::...} */\n'+ 'String a = "kept";\n'+ )+ literals, _ = p.scan_java(src)+ assert [(l.line, l.text) for l in literals] == [(5, "kept")]+++def test_scan_java_keeps_double_slash_inside_a_string():+ # A URL is not a comment start, so the host after it must still be seen.+ assert literal_texts('String u = "https://sts.amazonaws.com/x";') == ["https://sts.amazonaws.com/x"]+++def test_scan_java_handles_escaped_quotes_and_char_literals():+ src = "char q = '\"'; String s = \"say \\\"hi\\\" arn:aws:sqs\"; char b = '\\\\';"+ assert literal_texts(src) == ['say \\"hi\\" arn:aws:sqs']+++def test_scan_java_splits_text_blocks_per_line():+ src = (+ 'String policy = """\n'+ ' {"Principal": {"Service": "ec2.amazonaws.com"},\n'+ ' "Resource": "arn:aws:s3:::b/*"}\n'+ ' """;\n'+ 'String after = "tail";\n'+ )+ literals, _ = p.scan_java(src)+ assert [l.line for l in literals] == [2, 3, 5]+ assert "ec2.amazonaws.com" in literals[0].text+ assert "arn:aws:s3:::b/*" in literals[1].text+ assert literals[2].text == "tail"+++def test_scan_java_records_the_escape_comment_by_line():+ src = (+ 'String a = "arn:aws:iam::aws:policy/X"; // partition-literal: managed-policy owner is always aws\n'+ 'String b = "arn:aws:iam::aws:policy/Y"; // an ordinary comment\n'+ )+ _, escapes = p.scan_java(src)+ assert escapes == {1: "managed-policy owner is always aws"}+++def test_scan_java_escape_needs_a_reason():+ _, escapes = p.scan_java('String a = "arn:aws:x"; // partition-literal: \n')+ assert escapes == {}+++# --------------------------------------------------------------------------- #+# Categories+# --------------------------------------------------------------------------- #+def categories_of(text: str) -> list[str]:+ return [c.name for c in p.CATEGORIES if c.regex.search(text)]++[email protected]("text", [+ "arn:aws:iam::123456789012:role/x",+ "arn:aws:states:::lambda:invoke",+ "arn:aws-cn:greengrass:::runtime/function/executable",+ "arn:aws-us-gov:iam::aws:policy/AdministratorAccess",+ "^arn:aws:iam::(\\d{12}):role/.+$",+])+def test_arn_literal_matches_any_single_partition_prefix(text):+ assert "arn-literal" in categories_of(text)++[email protected]("text", [+ "^arn:aws[a-zA-Z-]*:sns:[a-z0-9-]+:\\d{12}:.+$",+ "^arn:aws(?:-eusc|-cn|-us-gov|-iso|-iso-[a-z])?:iam::",+ "arn:aws(-[^:]+)?:iam::([0-9]{12})?:role/.+",+ "^arn:aws[^:]*:elasticloadbalancing:",+ "arn:aws(?:-[a-z]{1,5}){0,3}:sso:::instance/",+])+def test_arn_regex_dialect_matches_hand_rolled_partition_alternations(text):+ assert categories_of(text) == ["arn-regex-dialect"]+++def test_arn_regex_dialect_does_not_match_a_plain_literal():+ assert "arn-regex-dialect" not in categories_of("arn:aws:iam::123456789012:role/x")++[email protected]("text,expected", [+ ("ec2.us-east-1.amazonaws.com", True),+ ("https://signin.aws.amazon.com/saml", True),+ ("d111111abcdef8.cloudfront.net", True),+ ("public.ecr.aws/lambda/java", True),+ ("ec2.cn-north-1.amazonaws.com.cn", False),+ ("c2s.ic.gov", False),+])+def test_dns_suffix_matches_commercial_hosts_only(text, expected):+ assert ("dns-suffix" in categories_of(text)) is expected++[email protected]("text,expected", [+ ("Z2FDTNDATAQYW2", True),+ ("Z35SXDOTRQ7X7K", True),+ ("Z2FDTNDATAQYW2/extra", False),+ ("ZONE", False),+])+def test_hosted_zone_id_matches_whole_literals_only(text, expected):+ assert ("hosted-zone-id" in categories_of(text)) is expected++[email protected]("text,expected", [+ ("us-east-1", True),+ ("us-gov-west-1", True),+ ("eusc-de-east-1", True),+ ("us-isob-east-1", True),+ ("ec2.eu-central-1.amazonaws.com", True),+ ("us-east-1a", True),+ ("i-us-east-1", False),+ ("us-east-10x", False),+ ("aws-global", False),+])+def test_region_literal_matches_region_and_zone_ids(text, expected):+ assert ("region-literal" in categories_of(text)) is expected+++def test_region_literal_is_report_only_and_the_rest_are_gated():+ gated = {c.name for c in p.CATEGORIES if c.gated}+ assert gated == {"arn-literal", "arn-regex-dialect", "dns-suffix", "hosted-zone-id"}+ assert not p.CATEGORY_BY_NAME["region-literal"].gated+++# --------------------------------------------------------------------------- #+# Findings, allow-list, escapes+# --------------------------------------------------------------------------- #+def write_java(root: Path, rel: str, body: str) -> Path:+ path = root / rel+ path.parent.mkdir(parents=True, exist_ok=True)+ path.write_text(body, encoding="utf-8")+ return path+++def test_collect_findings_counts_each_occurrence_and_skips_comments(tmp_path):+ write_java(tmp_path, "a/A.java", (+ 'class A {\n'+ ' // "arn:aws:iam::1:root"\n'+ ' String two = "arn:aws:s3:::x arn:aws:s3:::y";\n'+ ' String host = "sqs.us-east-1.amazonaws.com";\n'+ '}\n'+ ))+ findings = p.collect_findings(tmp_path, [])+ counted = p.count(findings)+ assert counted == Counter({+ ("arn-literal", "a/A.java"): 2,+ ("dns-suffix", "a/A.java"): 1,+ ("region-literal", "a/A.java"): 1,+ })+ assert {f.line for f in findings} == {3, 4}+++def test_allowlist_file_and_pattern_excuse_but_still_report(tmp_path):+ write_java(tmp_path, "ns/AwsNamespaces.java", 'class N { String s3 = "http://s3.amazonaws.com/doc/2006-03-01/"; }\n')+ # A pattern excuses the whole source line, so the counted host sits on its own line.+ write_java(tmp_path, "s3/Acl.java", (+ 'class A {\n'+ ' String g = "http://acs.amazonaws.com/groups/global/AllUsers";\n'+ ' String h = "s3.amazonaws.com";\n'+ '}\n'+ ))+ rules = [+ p.AllowRule("file", "ns/AwsNamespaces.java", "namespaces are invariant"),+ p.AllowRule("pattern", r"acs\.amazonaws\.com/groups/", "canned ACL URIs"),+ ]+ findings = p.collect_findings(tmp_path, rules)+ assert p.count(findings) == Counter({("dns-suffix", "s3/Acl.java"): 1})+ excuses = sorted(f.excuse for f in findings if f.excuse)+ assert excuses == [+ "allowlist file: namespaces are invariant",+ "allowlist pattern 'acs\\\\.amazonaws\\\\.com/groups/': canned ACL URIs",+ ]+++def test_escape_comment_excuses_only_its_own_line(tmp_path):+ write_java(tmp_path, "x/X.java", (+ 'class X {\n'+ ' String a = "arn:aws:iam::aws:policy/A"; // partition-literal: owner slot is literal aws\n'+ ' String b = "arn:aws:iam::aws:policy/B";\n'+ '}\n'+ ))+ findings = p.collect_findings(tmp_path, [])+ assert p.count(findings) == Counter({("arn-literal", "x/X.java"): 1})+ escaped = [f for f in findings if f.excuse]+ assert [(f.line, f.excuse) for f in escaped] == [(2, "escape: owner slot is literal aws")]+++def test_load_allowlist_rejects_an_entry_without_a_reason(tmp_path):+ allowlist = tmp_path / "allowlist.yaml"+ allowlist.write_text("patterns:\n - regex: foo\n reason: ' '\n")+ with pytest.raises(ValueError, match="no reason"):+ p.load_allowlist(allowlist)+++def test_load_allowlist_reads_files_and_patterns(tmp_path):+ allowlist = tmp_path / "allowlist.yaml"+ allowlist.write_text(+ "files:\n - path: a/B.java\n reason: why\npatterns:\n - regex: x\\.y\n reason: because\n"+ )+ assert p.load_allowlist(allowlist) == [+ p.AllowRule("file", "a/B.java", "why"),+ p.AllowRule("pattern", "x\\.y", "because"),+ ]+++def test_repo_allowlist_loads_and_every_pattern_compiles():+ import re+ for rule in p.load_allowlist(p.ALLOWLIST):+ if rule.kind == "pattern":+ re.compile(rule.value)+++# --------------------------------------------------------------------------- #+# Baseline and the gate+# --------------------------------------------------------------------------- #+def findings_for(tmp_path: Path, files: dict[str, str]) -> list[p.Finding]:+ for rel, body in files.items():+ write_java(tmp_path, rel, body)+ return p.collect_findings(tmp_path, [])+++def test_baseline_round_trips_only_gated_categories(tmp_path):+ findings = findings_for(tmp_path, {+ "a/A.java": 'class A { String r = "us-east-1"; String a = "arn:aws:s3:::b"; }\n',+ })+ baseline = tmp_path / "baseline.tsv"+ written = p.write_baseline(baseline, findings)+ assert written == Counter({("arn-literal", "a/A.java"): 1})+ assert p.read_baseline(baseline) == written+ text = baseline.read_text()+ assert text.startswith("# category\tpath\tcount\n")+ assert "region-literal" not in text+++def test_read_baseline_rejects_malformed_rows(tmp_path):+ baseline = tmp_path / "baseline.tsv"+ baseline.write_text("arn-literal\ta/A.java\n")+ with pytest.raises(ValueError, match="malformed"):+ p.read_baseline(baseline)+ baseline.write_text("nope\ta/A.java\t1\n")+ with pytest.raises(ValueError, match="unknown category"):+ p.read_baseline(baseline)+++def test_check_is_green_when_counts_match(tmp_path):+ findings = findings_for(tmp_path, {"a/A.java": 'class A { String a = "arn:aws:s3:::b"; }\n'})+ baseline = Counter({("arn-literal", "a/A.java"): 1})+ assert p.check(findings, baseline) == []+++def test_check_fails_a_literal_in_a_file_the_baseline_does_not_know(tmp_path):+ findings = findings_for(tmp_path, {"n/New.java": 'class N { String a = "arn:aws:s3:::b"; }\n'})+ problems = p.check(findings, Counter())+ assert len(problems) == 1+ assert "1 arn-literal literal(s) with no baseline entry for this file" in problems[0]+ assert "n/New.java:1:" in problems[0]+ assert "AwsArnUtils" in problems[0]+++def test_check_fails_growth_and_lists_every_occurrence(tmp_path):+ findings = findings_for(tmp_path, {+ "a/A.java": 'class A {\n String a = "arn:aws:s3:::b";\n String c = "arn:aws:s3:::d";\n}\n',+ })+ baseline = Counter({("arn-literal", "a/A.java"): 1})+ problems = p.check(findings, baseline)+ assert len(problems) == 1+ assert "arn-literal grew from 1 to 2" in problems[0]+ assert "a/A.java:2:" in problems[0] and "a/A.java:3:" in problems[0]+++def test_check_fails_a_drop_until_the_baseline_is_regenerated(tmp_path):+ findings = findings_for(tmp_path, {"a/A.java": 'class A { String a = "clean"; }\n'})+ baseline = Counter({("arn-literal", "a/A.java"): 3})+ problems = p.check(findings, baseline)+ assert len(problems) == 1+ assert "dropped from 3 to 0" in problems[0]+ assert "make partition-baseline" in problems[0]+++def test_check_ignores_report_only_drift(tmp_path):+ findings = findings_for(tmp_path, {"a/A.java": 'class A { String r = "us-east-1"; String z = "us-west-2"; }\n'})+ assert p.check(findings, Counter()) == []+++def test_check_treats_an_escaped_literal_as_absent(tmp_path):+ findings = findings_for(tmp_path, {+ "a/A.java": 'class A { String a = "arn:aws:s3:::b"; // partition-literal: fixture\n}\n',+ })+ assert p.check(findings, Counter()) == []+++# --------------------------------------------------------------------------- #+# Audit and CLI+# --------------------------------------------------------------------------- #+def test_package_of_groups_by_service_and_falls_back_to_the_parent():+ assert p.package_of("src/main/java/io/github/hectorvent/floci/services/s3/S3Service.java") == "services/s3"+ assert p.package_of("src/main/java/io/github/hectorvent/floci/core/common/AwsArnUtils.java") == "core/common"+ assert p.package_of("src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java") == "config"+ assert p.package_of("src/main/java/io/github/hectorvent/floci/services/s3/model/Bucket.java") == "services/s3"+ assert p.package_of("elsewhere/x/Y.java") == "elsewhere/x"+++def test_format_audit_lists_totals_escapes_and_allowlist_hits(tmp_path):+ write_java(tmp_path, "a/A.java", (+ 'class A {\n'+ ' String a = "arn:aws:s3:::b";\n'+ ' String e = "arn:aws:s3:::c"; // partition-literal: fixture value\n'+ ' String g = "http://acs.amazonaws.com/groups/global/AllUsers";\n'+ '}\n'+ ))+ rules = [p.AllowRule("pattern", r"acs\.amazonaws\.com/groups/", "canned ACL")]+ findings = p.collect_findings(tmp_path, rules)+ report = p.format_audit(findings, Counter({("arn-literal", "a/A.java"): 4}))+ assert "escapes (// partition-literal: <reason>): 1" in report+ assert "fixture value" in report+ assert "allow-listed occurrences: 1" in report+ assert "arn-literal 4 -> 1" in report+++def test_cli_check_and_write_baseline(tmp_path, capsys):+ write_java(tmp_path / "src", "a/A.java", 'class A { String a = "arn:aws:s3:::b"; }\n')+ allowlist = tmp_path / "allowlist.yaml"+ allowlist.write_text("files: []\npatterns: []\n")+ baseline = tmp_path / "baseline.tsv"+ common = ["--source", str(tmp_path / "src"), "--allowlist", str(allowlist), "--baseline", str(baseline)]++ assert p.main(["--check"] + common) == 1+ assert "no baseline entry" in capsys.readouterr().out++ assert p.main(["--write-baseline"] + common) == 0+ assert "1 gated literal(s) across 1 file(s)" in capsys.readouterr().out++ assert p.main(["--check"] + common) == 0+ assert "match baseline.tsv (1 gated occurrence(s) remaining)" in capsys.readouterr().out++ assert p.main(["--audit"] + common) == 0+ out = capsys.readouterr().out+ assert "arn-literal" in out and "total" in out