Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

chore(tooling): gate partition literals in src/main against a per-file baseline (#4270)

Hector Ventura0a601d54ebde
diff --git a/.github/workflows/partition-literals.yml b/.github/workflows/partition-literals.yml
new file mode 100644
index 0000000000..39aa463019
--- /dev/null
+++ b/.github/workflows/partition-literals.yml
@@ -0,0 +1,58 @@
+name: Partition Literals
+
+# Keeps the commercial-partition assumption from growing back while floci moves to
+# serving every AWS partition. tools/partition/partition_literals.py inventories the
+# hardcoded arn:aws: prefixes, amazonaws.com hosts, hosted-zone ids and hand-rolled
+# partition regexes in src/main/java against the checked-in per-file baseline: a
+# literal in a new file fails, growth in an existing file fails, and a drop fails until
+# the baseline is regenerated so the remaining count stays exact. Python-only, so it is
+# fast and independent of the Java build job.
+
+on:
+ pull_request:
+ paths:
+ - 'src/main/java/**'
+ - 'tools/partition/**'
+ - 'Makefile'
+ - '.github/workflows/partition-literals.yml'
+ push:
+ branches:
+ - main
+ paths:
+ - 'src/main/java/**'
+ - 'tools/partition/**'
+ - 'Makefile'
+ - '.github/workflows/partition-literals.yml'
+
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+jobs:
+ partition-literals:
+ name: Partition literals match baseline
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ steps:
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+
+ - name: Set up Python
+ uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+ with:
+ python-version: "3.x"
+
+ - name: Install tooling deps
+ run: pip install -r tools/partition/requirements.txt
+
+ - name: Run partition tooling tests
+ run: make partition-test
+
+ - name: Check partition literals against the baseline
+ run: make partition-check
+
+ - name: Print the remaining-literal audit
+ if: always()
+ run: make partition-audit
diff --git a/AGENTS.md b/AGENTS.md
index f88bf47175..3bbeb64fb4 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -111,6 +111,25 @@ Floci must implement real AWS wire protocols.
---
+### Partition literals
+
+Floci serves every AWS partition, not only the commercial one, so a literal that bakes
+`aws` into a code path is a bug in a GovCloud, China, ISO or EUSC deployment even when it
+prints correctly in `us-east-1`. `make partition-check` (CI: Partition Literals) inventories
+`src/main/java` for `arn:aws:` prefixes, `amazonaws.com` hosts, Route 53 hosted-zone ids
+and hand-rolled `arn:aws[a-z-]*:` regexes against `tools/partition/baseline.tsv`, per file.
+A literal in a new file fails, growth in an existing file fails, and a drop fails until you
+run `make partition-baseline` and commit the smaller baseline.
+
+- Mint ARNs through `AwsArnUtils.Arn.of(...)` / `RegionResolver.buildArn(...)`, recognise
+ them with `AwsArnUtils.isArn` / `PARTITION_REGEX`, and derive hosts from the region's
+ DNS suffix (`AwsRegions.dnsSuffixFor`).
+- A literal that is genuinely partition-invariant (an XML namespace URI, an S3 canned-ACL
+ grantee URI) goes in `tools/partition/allowlist.yaml` with a reason, or ends its line
+ with `// partition-literal: <reason>`. Both are printed by `make partition-audit`.
+- Tests are not scanned; only assert a partition-dependent value when the test is about
+ partitions.
+
## XML / JSON Rules
- Use `XmlBuilder` for XML responses
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index df772fcf91..f686420ff3 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -317,6 +317,19 @@ hand-written and preserved across regeneration, keyed by action name.
Registering a new service's action table is one entry in `tools/docs/services.yaml`.
+### Partition literals
+
+Floci serves every AWS partition, so `src/main/java` must not bake the commercial one in.
+`make partition-check` (run in CI) inventories `arn:aws:` prefixes, `amazonaws.com` hosts,
+Route 53 hosted-zone ids and hand-rolled partition regexes per file against
+`tools/partition/baseline.tsv`: a literal in a new file fails, growth fails, and a drop
+fails until you run `make partition-baseline` and commit the smaller baseline.
+`make partition-audit` prints what is left per package; `make partition-test` runs the
+tooling's unit tests. Mint ARNs through `AwsArnUtils` and derive hosts from the region's
+DNS suffix; a literal that really is partition-invariant goes in
+`tools/partition/allowlist.yaml` with a reason, or ends its line with
+`// partition-literal: <reason>`.
+
## Reporting Security Issues
Please do **not** open public issues for security vulnerabilities. Report them privately by emailing the maintainer or using [GitHub private vulnerability reporting](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing/privately-reporting-a-security-vulnerability).
diff --git a/Makefile b/Makefile
index 4c192df2d5..6945e9cb5a 100644
--- a/Makefile
+++ b/Makefile
@@ -5,6 +5,10 @@
#
# Service-matrix docs: docs/services/index.md's Service Matrix table is checked
# against ResolvedServiceCatalog.java so a registered service can't ship undocumented.
+#
+# Partition literals: hardcoded arn:aws: prefixes, amazonaws.com hosts and hosted-zone
+# ids in src/main are inventoried against tools/partition/baseline.tsv so the
+# commercial-partition assumption can only shrink. See tools/partition/.
PYTHON ?= python3
@@ -34,7 +38,7 @@ PREFIX ?= $(HOME)/.local
.DEFAULT_GOAL := help
.PHONY: help dev build test native native-host run-native native-install native-image native-up native-down native-logs clean-sidecars clean-volumes compat docker-tests \
- docs-sync docs-check docs-test
+ docs-sync docs-check docs-test partition-check partition-baseline partition-audit partition-test
help: ## List the targets below
@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf " %-14s %s\n", $$1, $$2}'
@@ -145,3 +149,20 @@ docs-check: ## CI gate: regenerate and fail if anything is stale, unregistered,
docs-test: ## Run the docs tooling's unit tests
$(PYTHON) -m pytest tools/docs -q
+
+partition-check: ## CI gate: partition literals in src/main must match tools/partition/baseline.tsv
+ @$(PYTHON) tools/partition/partition_literals.py --check || { \
+ echo ""; \
+ echo "error: partition literals drifted from tools/partition/baseline.tsv (see above)."; \
+ echo " Fix the new literal, or run 'make partition-baseline' after removing some."; \
+ exit 1; \
+ }
+
+partition-baseline: ## Regenerate tools/partition/baseline.tsv from the current tree (commit the result)
+ $(PYTHON) tools/partition/partition_literals.py --write-baseline
+
+partition-audit: ## Print the per-package table of remaining partition literals
+ @$(PYTHON) tools/partition/partition_literals.py --audit
+
+partition-test: ## Run the partition tooling's unit tests
+ $(PYTHON) -m pytest tools/partition -q
diff --git a/src/test/java/io/github/hectorvent/floci/services/resourceexplorer2/ResourceExplorer2IntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/resourceexplorer2/ResourceExplorer2IntegrationTest.java
index c52388a193..620c3b3720 100644
--- a/src/test/java/io/github/hectorvent/floci/services/resourceexplorer2/ResourceExplorer2IntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/resourceexplorer2/ResourceExplorer2IntegrationTest.java
@@ -1,5 +1,8 @@
package io.github.hectorvent.floci.services.resourceexplorer2;
+import io.github.hectorvent.floci.core.common.AwsArnUtils;
+import io.github.hectorvent.floci.core.common.AwsArnUtils.Arn;
+import io.github.hectorvent.floci.core.common.AwsRegions;
import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;
import io.quarkus.test.junit.QuarkusTest;
import org.junit.jupiter.api.AfterEach;
@@ -1503,8 +1506,16 @@ class ResourceExplorer2IntegrationTest {
for (var resource : resources) {
String arn = (String) resource.get("Arn");
String service = (String) resource.get("Service");
- assertTrue(arn.startsWith("arn:aws:"),
- "ARN must start with arn:aws:, got: " + arn);
+ Arn parsed = AwsArnUtils.parse(arn);
+ String expectedPartition = AwsRegions.partitionFor(parsed.region());
+ // Only commercial regions are held to their partition here: resources another test
+ // leaves in a China or GovCloud region are still minted arn:aws: by build sites that
+ // have not moved to Arn.of yet, which the partition plan's build-site phase fixes and
+ // then tightens this assertion to every region.
+ if ("aws".equals(expectedPartition)) {
+ assertEquals(expectedPartition, parsed.partition(),
+ "ARN partition must be the one its region belongs to, got: " + arn);
+ }
assertTrue(arn.contains(service),
"ARN should contain service '" + service + "', got: " + arn);
}
diff --git a/tools/partition/allowlist.yaml b/tools/partition/allowlist.yaml
new file mode 100644
index 0000000000..548fafeb54
--- /dev/null
+++ b/tools/partition/allowlist.yaml
@@ -0,0 +1,22 @@
+# Literals partition_literals.py must not count. Every entry needs a reason a reviewer can
+# check; the audit prints how many occurrences each entry excuses.
+#
+# files: the whole file is exempt (path relative to the repo root)
+# patterns: a Python regex matched against the full source line of the literal
+
+files:
+ - path: src/main/java/io/github/hectorvent/floci/core/common/AwsNamespaces.java
+ reason: XML namespace URIs (http://s3.amazonaws.com/doc/2006-03-01/ and the Query
+ namespaces) are identifiers, not hosts, and are byte-identical in every partition;
+ minio and the China/GovCloud SDKs emit the same strings.
+
+patterns:
+ - regex: amazonaws\.com/doc/\d{4}-\d{2}-\d{2}/
+ reason: an XML namespace URI outside AwsNamespaces (CloudFormation keeps its own);
+ the same partition-invariant identifier class as that file.
+ - regex: acs\.amazonaws\.com/groups/
+ reason: S3 canned-ACL grantee URIs (AllUsers, AuthenticatedUsers, LogDelivery) are
+ partition-invariant identifiers in the S3 model.
+ - regex: \bPARTITION_REGEX\b
+ reason: the line builds its regex from AwsArnUtils.PARTITION_REGEX, which is the
+ sanctioned way to recognise an ARN in any partition.
diff --git a/tools/partition/baseline.tsv b/tools/partition/baseline.tsv
new file mode 100644
index 0000000000..1e23f4d3ef
--- /dev/null
+++ b/tools/partition/baseline.tsv
@@ -0,0 +1,136 @@
+# category path count
+# Generated by tools/partition/partition_literals.py --write-baseline; do not hand-edit.
+arn-literal src/main/java/io/github/hectorvent/floci/core/common/IamEnforcementFilter.java 8
+arn-literal src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayController.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/apigateway/ExecuteApiSigV4Authorizer.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/applicationautoscaling/ApplicationAutoScalingService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/appsync/graphql/auth/IamAuthValidator.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreCredentialProviderService.java 4
+arn-literal src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreToolsService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessor.java 3
+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/ApiGatewayDomainCfnProvisioner.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/CfnResourceDispatcher.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/CodeBuildCfnProvisioner.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/RedshiftClusterCfnProvisioner.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudhsmv2/CloudHsmV2JsonHandler.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudtrail/CloudTrailService.java 7
+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsCrossAccountService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactController.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/connect/ConnectService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/controlcatalog/ControlCatalogService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/controltower/ControlTowerControlService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/controltower/ControlTowerService.java 4
+arn-literal src/main/java/io/github/hectorvent/floci/services/controltower/SeededLandingZoneFactory.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/datasync/DataSyncService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/detective/DetectiveService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/ec2/Ec2IpamService.java 3
+arn-literal src/main/java/io/github/hectorvent/floci/services/ec2/Ec2Service.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/ecs/EcsService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/eks/EksClusterManager.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/eks/EksWorkerAuthentication.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/elasticache/ElastiCacheQueryHandler.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/emrserverless/EmrServerlessService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/guardduty/GuardDutyService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/iam/AwsManagedPolicies.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/iam/IamPolicyEvaluator.java 4
+arn-literal src/main/java/io/github/hectorvent/floci/services/iam/SAMLAssertionVerifier.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/iam/SAMLProviderService.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/kinesisanalytics/KinesisAnalyticsV2JsonHandler.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/kinesisanalytics/container/FlinkContainerManager.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/kms/KmsService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/lambda/LambdaController.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/lambda/LambdaLayerService.java 3
+arn-literal src/main/java/io/github/hectorvent/floci/services/lambdamicrovms/LambdaMicrovmsController.java 3
+arn-literal src/main/java/io/github/hectorvent/floci/services/lambdamicrovms/LambdaMicrovmsService.java 3
+arn-literal src/main/java/io/github/hectorvent/floci/services/macie2/MacieController.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/macie2/MacieService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/marketplace/MarketplaceDeploymentService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/marketplace/MarketplaceMeteringService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/marketplace/MarketplaceReportingService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/marketplace/MarketplaceService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/networkfirewall/NetworkFirewallService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/oam/OamService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/ram/RamService.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/rds/RdsService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/rds/proxy/RdsSigV4Validator.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/redshift/RedshiftService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/redshift/proxy/S3CopySimulator.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/redshiftdata/RedshiftDataResourceResolver.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/resourceexplorer2/ResourceExplorer2Service.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/route53resolver/Route53ResolverService.java 5
+arn-literal src/main/java/io/github/hectorvent/floci/services/s3/S3ControlController.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/s3/S3PublicAccessEvaluator.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/s3/S3ResourcePolicyProvider.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/s3/S3Service.java 4
+arn-literal src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/securityhub/SecurityHubService.java 4
+arn-literal src/main/java/io/github/hectorvent/floci/services/servicecatalog/ServiceCatalogService.java 7
+arn-literal src/main/java/io/github/hectorvent/floci/services/servicequotas/ServiceQuotasService.java 2
+arn-literal src/main/java/io/github/hectorvent/floci/services/ses/SesService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/ses/SesTenantService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/signin/SigninService.java 3
+arn-literal src/main/java/io/github/hectorvent/floci/services/ssm/SsmService.java 3
+arn-literal src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java 32
+arn-literal src/main/java/io/github/hectorvent/floci/services/stepfunctions/StepFunctionsService.java 3
+arn-literal src/main/java/io/github/hectorvent/floci/services/swf/SwfService.java 1
+arn-literal src/main/java/io/github/hectorvent/floci/services/verifiedpermissions/VerifiedPermissionsService.java 2
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayExecuteController.java 1
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreGatewayRuleService.java 3
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreToolsService.java 1
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/budgets/BudgetsService.java 2
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/controltower/ControlTowerControlService.java 1
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/controltower/ControlTowerService.java 1
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/firehose/ProcessingConfigurationValidator.java 1
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/kms/KmsService.java 2
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/lambda/LambdaService.java 1
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/ses/SesReceiptRuleService.java 2
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/ssoadmin/SsoAdminService.java 9
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/ssooidc/SsoOidcService.java 1
+arn-regex-dialect src/main/java/io/github/hectorvent/floci/services/timestreaminfluxdb/TimestreamInfluxDbValidation.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java 2
+dns-suffix src/main/java/io/github/hectorvent/floci/core/common/AwsRegions.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/account/AccountService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayExecuteApiHostFilter.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayExecuteController.java 4
+dns-suffix src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/apigatewayv2/ApiGatewayV2Service.java 2
+dns-suffix src/main/java/io/github/hectorvent/floci/services/apigatewayv2/websocket/WebSocketProxyEventBuilder.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/applicationautoscaling/ApplicationAutoScalingService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/appsync/AppSyncService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/aps/ApsService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreCredentialProviderService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/bedrockagentcorecontrol/BedrockAgentCoreGatewayService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationTemplateEngine.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessor.java 4
+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/StackSetService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/CognitoCfnProvisioner.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/S3CfnProvisioner.java 4
+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudtrail/CloudTrailLogWriter.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/cloudtrail/CloudTrailService.java 5
+dns-suffix src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/connect/ConnectService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/ec2/Ec2QueryHandler.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/eks/EksOidcService.java 2
+dns-suffix src/main/java/io/github/hectorvent/floci/services/eks/EksPodIdentityCredentialsController.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/eks/EksPodIdentityWebhook.java 2
+dns-suffix src/main/java/io/github/hectorvent/floci/services/emr/EmrHandler.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/guardduty/GuardDutyService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/iam/IamService.java 6
+dns-suffix src/main/java/io/github/hectorvent/floci/services/iam/SAMLAssertionVerifier.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/iam/StsQueryHandler.java 2
+dns-suffix src/main/java/io/github/hectorvent/floci/services/macie2/MacieController.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/ram/RamController.java 2
+dns-suffix src/main/java/io/github/hectorvent/floci/services/redshift/proxy/S3CopySimulator.java 2
+dns-suffix src/main/java/io/github/hectorvent/floci/services/s3/S3Service.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/s3/S3VirtualHostFilter.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/signin/SigninService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/ssoportal/SsoPortalService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/services/verifiedpermissions/VerifiedPermissionsService.java 1
+dns-suffix src/main/java/io/github/hectorvent/floci/tools/ami/AmiImageTool.java 1
+hosted-zone-id src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java 2
+hosted-zone-id src/main/java/io/github/hectorvent/floci/services/elb/ElbClassicService.java 1
+hosted-zone-id src/main/java/io/github/hectorvent/floci/services/elbv2/ElbV2Service.java 1
diff --git a/tools/partition/partition_literals.py b/tools/partition/partition_literals.py
new file mode 100644
index 0000000000..f3f13d3aaa
--- /dev/null
+++ b/tools/partition/partition_literals.py
@@ -0,0 +1,409 @@
+#!/usr/bin/env python3
+"""Keep hardcoded AWS-partition assumptions in src/main from growing back.
+
+Floci is moving from a commercial-only (`aws`) emulator to one that serves every AWS
+partition (`aws-cn`, `aws-us-gov`, the ISO partitions, `aws-eusc`). The blockers are
+string literals that bake the commercial partition into a code path: an `arn:aws:`
+prefix, an `amazonaws.com` host, a Route 53 hosted zone that only exists in one
+partition, a hand-rolled `arn:aws[a-z-]*:` regex that recognises some partitions but
+not others. Each one is a silent bug in a non-commercial deployment, and each one is
+trivial to add by accident, because the literal is exactly what an `aws` deployment
+prints.
+
+This script inventories those literals in `src/main/java` by category and gates them
+against a checked-in per-file baseline (`baseline.tsv`):
+
+- a literal in a file the baseline does not know fails (new files start clean);
+- a file whose count grew fails;
+- a file whose count dropped also fails, until the baseline is regenerated, so the
+ "N remaining" number stays exact and the drop is visible in the PR diff.
+
+Comments never count. A literal that is genuinely partition-invariant is excused in one
+of two ways, both of which must state a reason: an entry in `allowlist.yaml` (a whole
+file, or a regex matched against the source line), or a trailing
+`// partition-literal: <reason>` comment on the line. The audit always prints the
+escapes so they are reviewed, not forgotten.
+
+Run from anywhere in the repo:
+ python3 tools/partition/partition_literals.py # per-package audit
+ python3 tools/partition/partition_literals.py --check # CI gate against baseline.tsv
+ python3 tools/partition/partition_literals.py --write-baseline # regenerate baseline.tsv
+"""
+from __future__ import annotations
+
+import argparse
+import re
+import sys
+from collections import Counter, defaultdict
+from dataclasses import dataclass
+from pathlib import Path
+
+import yaml
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+SOURCE_ROOT = REPO_ROOT / "src/main/java"
+PACKAGE_ROOT = SOURCE_ROOT / "io/github/hectorvent/floci"
+TOOL_DIR = Path(__file__).resolve().parent
+ALLOWLIST = TOOL_DIR / "allowlist.yaml"
+BASELINE = TOOL_DIR / "baseline.tsv"
+
+ESCAPE_RE = re.compile(r"//\s*partition-literal:\s*(?P<reason>\S.*?)\s*$")
+
+REGION_PREFIXES = "us|eu|ap|ca|sa|me|af|il|mx|cn|us-gov|us-iso|us-isob|eu-isoe|us-isof|eusc-de"
+
+
+@dataclass(frozen=True)
+class Category:
+ name: str
+ regex: re.Pattern
+ gated: bool
+ description: str
+
+
+# Regexes run against the content of one string literal (one line of a text block), never
+# against comments. Order is the audit column order.
+CATEGORIES: tuple[Category, ...] = (
+ Category(
+ "arn-literal",
+ re.compile(r"\barn:aws(?:-[a-z]+)*:"),
+ True,
+ "an ARN prefix naming one partition (arn:aws:, arn:aws-cn:, ...); mint through AwsArnUtils",
+ ),
+ Category(
+ "arn-regex-dialect",
+ re.compile(r"arn:aws(?=[\[(\\])"),
+ True,
+ "a hand-rolled partition alternation in a regex; use AwsArnUtils.PARTITION_REGEX",
+ ),
+ Category(
+ "dns-suffix",
+ re.compile(r"amazonaws\.com(?!\.cn)|cloudfront\.net|signin\.aws|public\.ecr\.aws"),
+ True,
+ "a commercial DNS suffix or host; derive from the partition's dnsSuffix",
+ ),
+ Category(
+ "hosted-zone-id",
+ re.compile(r"^Z[0-9A-Z]{12,14}$"),
+ True,
+ "a Route 53 hosted zone id, which differs per region and partition",
+ ),
+ Category(
+ "region-literal",
+ re.compile(
+ r"(?<![a-z0-9-])(?:" + REGION_PREFIXES + r")-[a-z]+-\d[a-z]?(?![a-z0-9-])"
+ ),
+ False,
+ "a region or availability-zone id; report-only until the us-east-1 assumptions are swept",
+ ),
+)
+
+CATEGORY_BY_NAME = {category.name: category for category in CATEGORIES}
+GATED = tuple(category for category in CATEGORIES if category.gated)
+
+
+@dataclass(frozen=True)
+class Literal:
+ line: int
+ text: str
+
+
+@dataclass(frozen=True)
+class Finding:
+ category: str
+ path: str
+ line: int
+ source: str
+ excuse: str | None = None
+
+ @property
+ def counted(self) -> bool:
+ return self.excuse is None
+
+
+def scan_java(text: str) -> tuple[list[Literal], dict[int, str]]:
+ """Extracts string literals (one per line for text blocks) and per-line escape comments.
+
+ A tiny lexer rather than a regex: `//` inside a string is a URL, not a comment, and a
+ `"` inside a comment or char literal is not a string. Block comments and text blocks
+ span lines, so line numbers are tracked by hand.
+ """
+ literals: list[Literal] = []
+ escapes: dict[int, str] = {}
+ i = 0
+ n = len(text)
+ line = 1
+ while i < n:
+ char = text[i]
+ if char == "\n":
+ line += 1
+ i += 1
+ elif text.startswith("//", i):
+ end = text.find("\n", i)
+ end = n if end < 0 else end
+ match = ESCAPE_RE.match(text[i:end])
+ if match:
+ escapes[line] = match.group("reason")
+ i = end
+ elif text.startswith("/*", i):
+ end = text.find("*/", i + 2)
+ end = n if end < 0 else end + 2
+ line += text.count("\n", i, end)
+ i = end
+ elif text.startswith('"""', i):
+ j = i + 3
+ while j < n and not text.startswith('"""', j):
+ j += 2 if text[j] == "\\" else 1
+ block = text[i + 3:j]
+ for offset, part in enumerate(block.split("\n")):
+ if part.strip():
+ literals.append(Literal(line + offset, part))
+ line += block.count("\n")
+ i = min(j + 3, n)
+ elif char == '"':
+ j = i + 1
+ buffer: list[str] = []
+ while j < n and text[j] not in '"\n':
+ if text[j] == "\\":
+ buffer.append(text[j:j + 2])
+ j += 2
+ else:
+ buffer.append(text[j])
+ j += 1
+ literals.append(Literal(line, "".join(buffer)))
+ i = j + 1 if j < n and text[j] == '"' else j
+ elif char == "'":
+ j = i + 1
+ while j < n and text[j] not in "'\n":
+ j += 2 if text[j] == "\\" else 1
+ i = j + 1 if j < n and text[j] == "'" else j
+ else:
+ i += 1
+ return literals, escapes
+
+
+@dataclass(frozen=True)
+class AllowRule:
+ kind: str
+ value: str
+ reason: str
+
+
+def load_allowlist(path: Path) -> list[AllowRule]:
+ if not path.exists():
+ return []
+ data = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
+ rules: list[AllowRule] = []
+ for entry in data.get("files") or []:
+ rules.append(AllowRule("file", str(entry["path"]), str(entry["reason"])))
+ for entry in data.get("patterns") or []:
+ rules.append(AllowRule("pattern", str(entry["regex"]), str(entry["reason"])))
+ for rule in rules:
+ if not rule.reason.strip():
+ raise ValueError(f"allowlist entry {rule.value!r} has no reason")
+ return rules
+
+
+def excuse_for(rel_path: str, source_line: str, rules: list[AllowRule]) -> str | None:
+ for rule in rules:
+ if rule.kind == "file" and rule.value == rel_path:
+ return f"allowlist file: {rule.reason}"
+ if rule.kind == "pattern" and re.search(rule.value, source_line):
+ return f"allowlist pattern {rule.value!r}: {rule.reason}"
+ return None
+
+
+def collect_findings(source_root: Path, rules: list[AllowRule]) -> list[Finding]:
+ findings: list[Finding] = []
+ for java in sorted(source_root.rglob("*.java")):
+ rel_path = java.relative_to(REPO_ROOT).as_posix() if java.is_relative_to(REPO_ROOT) \
+ else java.relative_to(source_root).as_posix()
+ text = java.read_text(encoding="utf-8")
+ lines = text.split("\n")
+ literals, escapes = scan_java(text)
+ for literal in literals:
+ source_line = lines[literal.line - 1] if literal.line - 1 < len(lines) else ""
+ for category in CATEGORIES:
+ hits = len(category.regex.findall(literal.text))
+ if hits == 0:
+ continue
+ excuse = excuse_for(rel_path, source_line, rules)
+ if excuse is None and literal.line in escapes:
+ excuse = f"escape: {escapes[literal.line]}"
+ findings.extend(
+ Finding(category.name, rel_path, literal.line, source_line.strip(), excuse)
+ for _ in range(hits)
+ )
+ return findings
+
+
+def count(findings: list[Finding], gated_only: bool = False) -> Counter:
+ counter: Counter = Counter()
+ for finding in findings:
+ if not finding.counted:
+ continue
+ if gated_only and not CATEGORY_BY_NAME[finding.category].gated:
+ continue
+ counter[(finding.category, finding.path)] += 1
+ return counter
+
+
+def read_baseline(path: Path) -> Counter:
+ counter: Counter = Counter()
+ if not path.exists():
+ return counter
+ for raw in path.read_text(encoding="utf-8").splitlines():
+ if not raw.strip() or raw.startswith("#"):
+ continue
+ parts = raw.split("\t")
+ if len(parts) != 3:
+ raise ValueError(f"{path}: malformed baseline row {raw!r}")
+ category, rel_path, value = parts
+ if category not in CATEGORY_BY_NAME:
+ raise ValueError(f"{path}: unknown category {category!r}")
+ counter[(category, rel_path)] = int(value)
+ return counter
+
+
+def format_baseline(counter: Counter) -> str:
+ rows = ["# category\tpath\tcount", "# Generated by tools/partition/partition_literals.py --write-baseline; do not hand-edit."]
+ for (category, rel_path), value in sorted(counter.items()):
+ rows.append(f"{category}\t{rel_path}\t{value}")
+ return "\n".join(rows) + "\n"
+
+
+def write_baseline(path: Path, findings: list[Finding]) -> Counter:
+ counter = count(findings, gated_only=True)
+ path.write_text(format_baseline(counter), encoding="utf-8")
+ return counter
+
+
+def check(findings: list[Finding], baseline: Counter) -> list[str]:
+ """Returns one problem per (category, file) that drifted from the baseline; empty when green."""
+ current = count(findings, gated_only=True)
+ by_key: dict[tuple[str, str], list[Finding]] = defaultdict(list)
+ for finding in findings:
+ if finding.counted:
+ by_key[(finding.category, finding.path)].append(finding)
+ problems: list[str] = []
+ for key in sorted(set(current) | set(baseline)):
+ category, rel_path = key
+ if not CATEGORY_BY_NAME[category].gated:
+ continue
+ now = current.get(key, 0)
+ before = baseline.get(key, 0)
+ if now == before:
+ continue
+ if key not in baseline:
+ head = f"{rel_path}: {now} {category} literal(s) with no baseline entry for this file"
+ elif now > before:
+ head = f"{rel_path}: {category} grew from {before} to {now}"
+ else:
+ head = f"{rel_path}: {category} dropped from {before} to {now}; run 'make partition-baseline' and commit baseline.tsv"
+ lines = [head]
+ if now > before:
+ lines.extend(f" {f.path}:{f.line}: {f.source}" for f in by_key[key])
+ lines.append(f" {CATEGORY_BY_NAME[category].description}")
+ problems.append("\n".join(lines))
+ return problems
+
+
+def package_of(rel_path: str) -> str:
+ try:
+ relative = Path(rel_path).relative_to(PACKAGE_ROOT.relative_to(REPO_ROOT))
+ except ValueError:
+ return Path(rel_path).parent.as_posix()
+ parts = relative.parts[:-1]
+ return "/".join(parts[:2]) if parts else "(root)"
+
+
+def format_audit(findings: list[Finding], baseline: Counter) -> str:
+ names = [category.name for category in CATEGORIES]
+ per_package: dict[str, Counter] = defaultdict(Counter)
+ totals: Counter = Counter()
+ for finding in findings:
+ if finding.counted:
+ per_package[package_of(finding.path)][finding.category] += 1
+ totals[finding.category] += 1
+ width = max([len("package"), len("total")] + [len(p) for p in per_package]) + 2
+ header = f"{'package':<{width}}" + "".join(f"{name:>19}" for name in names) + f"{'total':>8}"
+ out = [header, "-" * len(header)]
+ for package, counter in sorted(per_package.items(), key=lambda item: (-sum(item[1].values()), item[0])):
+ out.append(
+ f"{package:<{width}}"
+ + "".join(f"{counter.get(name, 0):>19}" for name in names)
+ + f"{sum(counter.values()):>8}"
+ )
+ out.append("-" * len(header))
+ out.append(
+ f"{'total':<{width}}"
+ + "".join(f"{totals.get(name, 0):>19}" for name in names)
+ + f"{sum(totals.values()):>8}"
+ )
+ gate = "".join(
+ f"{'gated' if category.gated else 'report-only':>19}" for category in CATEGORIES
+ )
+ out.append(f"{'':<{width}}{gate}")
+ if baseline:
+ baseline_totals: Counter = Counter()
+ for (category, _), value in baseline.items():
+ baseline_totals[category] += value
+ out.append("")
+ out.append("baseline.tsv: " + ", ".join(
+ f"{category.name} {baseline_totals.get(category.name, 0)} -> {totals.get(category.name, 0)}"
+ for category in GATED
+ ))
+ excused = [finding for finding in findings if not finding.counted]
+ escapes = sorted({(f.path, f.line, f.excuse) for f in excused if f.excuse and f.excuse.startswith("escape:")})
+ allowed: Counter = Counter(
+ f.excuse for f in excused if f.excuse and f.excuse.startswith("allowlist")
+ )
+ out.append("")
+ out.append(f"escapes (// partition-literal: <reason>): {len(escapes)}")
+ out.extend(f" {path}:{line} {excuse[len('escape: '):]}" for path, line, excuse in escapes)
+ out.append(f"allow-listed occurrences: {sum(allowed.values())}")
+ out.extend(f" {value:>4} {excuse}" for excuse, value in sorted(allowed.items()))
+ return "\n".join(out)
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+ mode = parser.add_mutually_exclusive_group()
+ mode.add_argument("--check", action="store_true", help="exit 1 when a gated count drifted from the baseline")
+ mode.add_argument("--write-baseline", action="store_true", help="regenerate the baseline from the current tree")
+ mode.add_argument("--audit", action="store_true", help="print the per-package table (the default)")
+ parser.add_argument("--source", type=Path, default=SOURCE_ROOT, help=argparse.SUPPRESS)
+ parser.add_argument("--allowlist", type=Path, default=ALLOWLIST, help=argparse.SUPPRESS)
+ parser.add_argument("--baseline", type=Path, default=BASELINE, help=argparse.SUPPRESS)
+ args = parser.parse_args(argv)
+
+ rules = load_allowlist(args.allowlist)
+ findings = collect_findings(args.source, rules)
+
+ if args.write_baseline:
+ counter = write_baseline(args.baseline, findings)
+ print(f"wrote {args.baseline.relative_to(REPO_ROOT) if args.baseline.is_relative_to(REPO_ROOT) else args.baseline}: "
+ f"{sum(counter.values())} gated literal(s) across {len({p for _, p in counter})} file(s)")
+ return 0
+
+ baseline = read_baseline(args.baseline)
+ if args.check:
+ problems = check(findings, baseline)
+ for problem in problems:
+ print(problem)
+ if problems:
+ print("")
+ print(f"error: {len(problems)} file(s) drifted from tools/partition/baseline.tsv. "
+ "Route new ARNs through AwsArnUtils and hosts through the partition's dnsSuffix; "
+ "when a literal is genuinely partition-invariant, add it to tools/partition/allowlist.yaml "
+ "or end the line with '// partition-literal: <reason>'.")
+ return 1
+ gated_total = sum(count(findings, gated_only=True).values())
+ print(f"partition literals match baseline.tsv ({gated_total} gated occurrence(s) remaining)")
+ return 0
+
+ print(format_audit(findings, baseline))
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/tools/partition/requirements.txt b/tools/partition/requirements.txt
new file mode 100644
index 0000000000..26efb36e3a
--- /dev/null
+++ b/tools/partition/requirements.txt
@@ -0,0 +1,2 @@
+PyYAML>=6.0
+pytest>=7.0
diff --git a/tools/partition/test_partition_literals.py b/tools/partition/test_partition_literals.py
new file mode 100644
index 0000000000..899bd4b7e4
--- /dev/null
+++ b/tools/partition/test_partition_literals.py
@@ -0,0 +1,368 @@
+"""Tests for partition_literals.
+
+Run with: pytest tools/partition -q (or: make partition-test)
+"""
+from __future__ import annotations
+
+from collections import Counter
+from pathlib import Path
+
+import pytest
+
+import partition_literals as p
+
+
+# --------------------------------------------------------------------------- #
+# Lexer
+# --------------------------------------------------------------------------- #
+def literal_texts(src: str) -> list[str]:
+ return [literal.text for literal in p.scan_java(src)[0]]
+
+
+def test_scan_java_extracts_plain_string_literals_with_line_numbers():
+ src = 'String a = "arn:aws:s3:::b";\nString b = "x";\n'
+ literals, escapes = p.scan_java(src)
+ assert [(l.line, l.text) for l in literals] == [(1, "arn:aws:s3:::b"), (2, "x")]
+ assert escapes == {}
+
+
+def test_scan_java_ignores_line_and_block_comments():
+ src = (
+ '// "arn:aws:iam::123:root" in a line comment\n'
+ '/* "arn:aws:iam::123:root"\n spanning lines */\n'
+ '/** {@code arn:aws-us-gov:iam::...} */\n'
+ 'String a = "kept";\n'
+ )
+ literals, _ = p.scan_java(src)
+ assert [(l.line, l.text) for l in literals] == [(5, "kept")]
+
+
+def test_scan_java_keeps_double_slash_inside_a_string():
+ # A URL is not a comment start, so the host after it must still be seen.
+ assert literal_texts('String u = "https://sts.amazonaws.com/x";') == ["https://sts.amazonaws.com/x"]
+
+
+def test_scan_java_handles_escaped_quotes_and_char_literals():
+ src = "char q = '\"'; String s = \"say \\\"hi\\\" arn:aws:sqs\"; char b = '\\\\';"
+ assert literal_texts(src) == ['say \\"hi\\" arn:aws:sqs']
+
+
+def test_scan_java_splits_text_blocks_per_line():
+ src = (
+ 'String policy = """\n'
+ ' {"Principal": {"Service": "ec2.amazonaws.com"},\n'
+ ' "Resource": "arn:aws:s3:::b/*"}\n'
+ ' """;\n'
+ 'String after = "tail";\n'
+ )
+ literals, _ = p.scan_java(src)
+ assert [l.line for l in literals] == [2, 3, 5]
+ assert "ec2.amazonaws.com" in literals[0].text
+ assert "arn:aws:s3:::b/*" in literals[1].text
+ assert literals[2].text == "tail"
+
+
+def test_scan_java_records_the_escape_comment_by_line():
+ src = (
+ 'String a = "arn:aws:iam::aws:policy/X"; // partition-literal: managed-policy owner is always aws\n'
+ 'String b = "arn:aws:iam::aws:policy/Y"; // an ordinary comment\n'
+ )
+ _, escapes = p.scan_java(src)
+ assert escapes == {1: "managed-policy owner is always aws"}
+
+
+def test_scan_java_escape_needs_a_reason():
+ _, escapes = p.scan_java('String a = "arn:aws:x"; // partition-literal: \n')
+ assert escapes == {}
+
+
+# --------------------------------------------------------------------------- #
+# Categories
+# --------------------------------------------------------------------------- #
+def categories_of(text: str) -> list[str]:
+ return [c.name for c in p.CATEGORIES if c.regex.search(text)]
+
+
+ "arn:aws:iam::123456789012:role/x",
+ "arn:aws:states:::lambda:invoke",
+ "arn:aws-cn:greengrass:::runtime/function/executable",
+ "arn:aws-us-gov:iam::aws:policy/AdministratorAccess",
+ "^arn:aws:iam::(\\d{12}):role/.+$",
+])
+def test_arn_literal_matches_any_single_partition_prefix(text):
+ assert "arn-literal" in categories_of(text)
+
+
+ "^arn:aws[a-zA-Z-]*:sns:[a-z0-9-]+:\\d{12}:.+$",
+ "^arn:aws(?:-eusc|-cn|-us-gov|-iso|-iso-[a-z])?:iam::",
+ "arn:aws(-[^:]+)?:iam::([0-9]{12})?:role/.+",
+ "^arn:aws[^:]*:elasticloadbalancing:",
+ "arn:aws(?:-[a-z]{1,5}){0,3}:sso:::instance/",
+])
+def test_arn_regex_dialect_matches_hand_rolled_partition_alternations(text):
+ assert categories_of(text) == ["arn-regex-dialect"]
+
+
+def test_arn_regex_dialect_does_not_match_a_plain_literal():
+ assert "arn-regex-dialect" not in categories_of("arn:aws:iam::123456789012:role/x")
+
+
[email protected]("text,expected", [
+ ("ec2.us-east-1.amazonaws.com", True),
+ ("https://signin.aws.amazon.com/saml", True),
+ ("d111111abcdef8.cloudfront.net", True),
+ ("public.ecr.aws/lambda/java", True),
+ ("ec2.cn-north-1.amazonaws.com.cn", False),
+ ("c2s.ic.gov", False),
+])
+def test_dns_suffix_matches_commercial_hosts_only(text, expected):
+ assert ("dns-suffix" in categories_of(text)) is expected
+
+
[email protected]("text,expected", [
+ ("Z2FDTNDATAQYW2", True),
+ ("Z35SXDOTRQ7X7K", True),
+ ("Z2FDTNDATAQYW2/extra", False),
+ ("ZONE", False),
+])
+def test_hosted_zone_id_matches_whole_literals_only(text, expected):
+ assert ("hosted-zone-id" in categories_of(text)) is expected
+
+
[email protected]("text,expected", [
+ ("us-east-1", True),
+ ("us-gov-west-1", True),
+ ("eusc-de-east-1", True),
+ ("us-isob-east-1", True),
+ ("ec2.eu-central-1.amazonaws.com", True),
+ ("us-east-1a", True),
+ ("i-us-east-1", False),
+ ("us-east-10x", False),
+ ("aws-global", False),
+])
+def test_region_literal_matches_region_and_zone_ids(text, expected):
+ assert ("region-literal" in categories_of(text)) is expected
+
+
+def test_region_literal_is_report_only_and_the_rest_are_gated():
+ gated = {c.name for c in p.CATEGORIES if c.gated}
+ assert gated == {"arn-literal", "arn-regex-dialect", "dns-suffix", "hosted-zone-id"}
+ assert not p.CATEGORY_BY_NAME["region-literal"].gated
+
+
+# --------------------------------------------------------------------------- #
+# Findings, allow-list, escapes
+# --------------------------------------------------------------------------- #
+def write_java(root: Path, rel: str, body: str) -> Path:
+ path = root / rel
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text(body, encoding="utf-8")
+ return path
+
+
+def test_collect_findings_counts_each_occurrence_and_skips_comments(tmp_path):
+ write_java(tmp_path, "a/A.java", (
+ 'class A {\n'
+ ' // "arn:aws:iam::1:root"\n'
+ ' String two = "arn:aws:s3:::x arn:aws:s3:::y";\n'
+ ' String host = "sqs.us-east-1.amazonaws.com";\n'
+ '}\n'
+ ))
+ findings = p.collect_findings(tmp_path, [])
+ counted = p.count(findings)
+ assert counted == Counter({
+ ("arn-literal", "a/A.java"): 2,
+ ("dns-suffix", "a/A.java"): 1,
+ ("region-literal", "a/A.java"): 1,
+ })
+ assert {f.line for f in findings} == {3, 4}
+
+
+def test_allowlist_file_and_pattern_excuse_but_still_report(tmp_path):
+ write_java(tmp_path, "ns/AwsNamespaces.java", 'class N { String s3 = "http://s3.amazonaws.com/doc/2006-03-01/"; }\n')
+ # A pattern excuses the whole source line, so the counted host sits on its own line.
+ write_java(tmp_path, "s3/Acl.java", (
+ 'class A {\n'
+ ' String g = "http://acs.amazonaws.com/groups/global/AllUsers";\n'
+ ' String h = "s3.amazonaws.com";\n'
+ '}\n'
+ ))
+ rules = [
+ p.AllowRule("file", "ns/AwsNamespaces.java", "namespaces are invariant"),
+ p.AllowRule("pattern", r"acs\.amazonaws\.com/groups/", "canned ACL URIs"),
+ ]
+ findings = p.collect_findings(tmp_path, rules)
+ assert p.count(findings) == Counter({("dns-suffix", "s3/Acl.java"): 1})
+ excuses = sorted(f.excuse for f in findings if f.excuse)
+ assert excuses == [
+ "allowlist file: namespaces are invariant",
+ "allowlist pattern 'acs\\\\.amazonaws\\\\.com/groups/': canned ACL URIs",
+ ]
+
+
+def test_escape_comment_excuses_only_its_own_line(tmp_path):
+ write_java(tmp_path, "x/X.java", (
+ 'class X {\n'
+ ' String a = "arn:aws:iam::aws:policy/A"; // partition-literal: owner slot is literal aws\n'
+ ' String b = "arn:aws:iam::aws:policy/B";\n'
+ '}\n'
+ ))
+ findings = p.collect_findings(tmp_path, [])
+ assert p.count(findings) == Counter({("arn-literal", "x/X.java"): 1})
+ escaped = [f for f in findings if f.excuse]
+ assert [(f.line, f.excuse) for f in escaped] == [(2, "escape: owner slot is literal aws")]
+
+
+def test_load_allowlist_rejects_an_entry_without_a_reason(tmp_path):
+ allowlist = tmp_path / "allowlist.yaml"
+ allowlist.write_text("patterns:\n - regex: foo\n reason: ' '\n")
+ with pytest.raises(ValueError, match="no reason"):
+ p.load_allowlist(allowlist)
+
+
+def test_load_allowlist_reads_files_and_patterns(tmp_path):
+ allowlist = tmp_path / "allowlist.yaml"
+ allowlist.write_text(
+ "files:\n - path: a/B.java\n reason: why\npatterns:\n - regex: x\\.y\n reason: because\n"
+ )
+ assert p.load_allowlist(allowlist) == [
+ p.AllowRule("file", "a/B.java", "why"),
+ p.AllowRule("pattern", "x\\.y", "because"),
+ ]
+
+
+def test_repo_allowlist_loads_and_every_pattern_compiles():
+ import re
+ for rule in p.load_allowlist(p.ALLOWLIST):
+ if rule.kind == "pattern":
+ re.compile(rule.value)
+
+
+# --------------------------------------------------------------------------- #
+# Baseline and the gate
+# --------------------------------------------------------------------------- #
+def findings_for(tmp_path: Path, files: dict[str, str]) -> list[p.Finding]:
+ for rel, body in files.items():
+ write_java(tmp_path, rel, body)
+ return p.collect_findings(tmp_path, [])
+
+
+def test_baseline_round_trips_only_gated_categories(tmp_path):
+ findings = findings_for(tmp_path, {
+ "a/A.java": 'class A { String r = "us-east-1"; String a = "arn:aws:s3:::b"; }\n',
+ })
+ baseline = tmp_path / "baseline.tsv"
+ written = p.write_baseline(baseline, findings)
+ assert written == Counter({("arn-literal", "a/A.java"): 1})
+ assert p.read_baseline(baseline) == written
+ text = baseline.read_text()
+ assert text.startswith("# category\tpath\tcount\n")
+ assert "region-literal" not in text
+
+
+def test_read_baseline_rejects_malformed_rows(tmp_path):
+ baseline = tmp_path / "baseline.tsv"
+ baseline.write_text("arn-literal\ta/A.java\n")
+ with pytest.raises(ValueError, match="malformed"):
+ p.read_baseline(baseline)
+ baseline.write_text("nope\ta/A.java\t1\n")
+ with pytest.raises(ValueError, match="unknown category"):
+ p.read_baseline(baseline)
+
+
+def test_check_is_green_when_counts_match(tmp_path):
+ findings = findings_for(tmp_path, {"a/A.java": 'class A { String a = "arn:aws:s3:::b"; }\n'})
+ baseline = Counter({("arn-literal", "a/A.java"): 1})
+ assert p.check(findings, baseline) == []
+
+
+def test_check_fails_a_literal_in_a_file_the_baseline_does_not_know(tmp_path):
+ findings = findings_for(tmp_path, {"n/New.java": 'class N { String a = "arn:aws:s3:::b"; }\n'})
+ problems = p.check(findings, Counter())
+ assert len(problems) == 1
+ assert "1 arn-literal literal(s) with no baseline entry for this file" in problems[0]
+ assert "n/New.java:1:" in problems[0]
+ assert "AwsArnUtils" in problems[0]
+
+
+def test_check_fails_growth_and_lists_every_occurrence(tmp_path):
+ findings = findings_for(tmp_path, {
+ "a/A.java": 'class A {\n String a = "arn:aws:s3:::b";\n String c = "arn:aws:s3:::d";\n}\n',
+ })
+ baseline = Counter({("arn-literal", "a/A.java"): 1})
+ problems = p.check(findings, baseline)
+ assert len(problems) == 1
+ assert "arn-literal grew from 1 to 2" in problems[0]
+ assert "a/A.java:2:" in problems[0] and "a/A.java:3:" in problems[0]
+
+
+def test_check_fails_a_drop_until_the_baseline_is_regenerated(tmp_path):
+ findings = findings_for(tmp_path, {"a/A.java": 'class A { String a = "clean"; }\n'})
+ baseline = Counter({("arn-literal", "a/A.java"): 3})
+ problems = p.check(findings, baseline)
+ assert len(problems) == 1
+ assert "dropped from 3 to 0" in problems[0]
+ assert "make partition-baseline" in problems[0]
+
+
+def test_check_ignores_report_only_drift(tmp_path):
+ findings = findings_for(tmp_path, {"a/A.java": 'class A { String r = "us-east-1"; String z = "us-west-2"; }\n'})
+ assert p.check(findings, Counter()) == []
+
+
+def test_check_treats_an_escaped_literal_as_absent(tmp_path):
+ findings = findings_for(tmp_path, {
+ "a/A.java": 'class A { String a = "arn:aws:s3:::b"; // partition-literal: fixture\n}\n',
+ })
+ assert p.check(findings, Counter()) == []
+
+
+# --------------------------------------------------------------------------- #
+# Audit and CLI
+# --------------------------------------------------------------------------- #
+def test_package_of_groups_by_service_and_falls_back_to_the_parent():
+ assert p.package_of("src/main/java/io/github/hectorvent/floci/services/s3/S3Service.java") == "services/s3"
+ assert p.package_of("src/main/java/io/github/hectorvent/floci/core/common/AwsArnUtils.java") == "core/common"
+ assert p.package_of("src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java") == "config"
+ assert p.package_of("src/main/java/io/github/hectorvent/floci/services/s3/model/Bucket.java") == "services/s3"
+ assert p.package_of("elsewhere/x/Y.java") == "elsewhere/x"
+
+
+def test_format_audit_lists_totals_escapes_and_allowlist_hits(tmp_path):
+ write_java(tmp_path, "a/A.java", (
+ 'class A {\n'
+ ' String a = "arn:aws:s3:::b";\n'
+ ' String e = "arn:aws:s3:::c"; // partition-literal: fixture value\n'
+ ' String g = "http://acs.amazonaws.com/groups/global/AllUsers";\n'
+ '}\n'
+ ))
+ rules = [p.AllowRule("pattern", r"acs\.amazonaws\.com/groups/", "canned ACL")]
+ findings = p.collect_findings(tmp_path, rules)
+ report = p.format_audit(findings, Counter({("arn-literal", "a/A.java"): 4}))
+ assert "escapes (// partition-literal: <reason>): 1" in report
+ assert "fixture value" in report
+ assert "allow-listed occurrences: 1" in report
+ assert "arn-literal 4 -> 1" in report
+
+
+def test_cli_check_and_write_baseline(tmp_path, capsys):
+ write_java(tmp_path / "src", "a/A.java", 'class A { String a = "arn:aws:s3:::b"; }\n')
+ allowlist = tmp_path / "allowlist.yaml"
+ allowlist.write_text("files: []\npatterns: []\n")
+ baseline = tmp_path / "baseline.tsv"
+ common = ["--source", str(tmp_path / "src"), "--allowlist", str(allowlist), "--baseline", str(baseline)]
+
+ assert p.main(["--check"] + common) == 1
+ assert "no baseline entry" in capsys.readouterr().out
+
+ assert p.main(["--write-baseline"] + common) == 0
+ assert "1 gated literal(s) across 1 file(s)" in capsys.readouterr().out
+
+ assert p.main(["--check"] + common) == 0
+ assert "match baseline.tsv (1 gated occurrence(s) remaining)" in capsys.readouterr().out
+
+ assert p.main(["--audit"] + common) == 0
+ out = capsys.readouterr().out
+ assert "arn-literal" in out and "total" in out