Light, fluffy, and always free - The AWS Local Emulator alternative
feat(codeartifact): add PyPI format support via pypiserver sidecar (#4998)
Priamo Germosenfca7d87f2d58
diff --git a/docs/services/codeartifact.md b/docs/services/codeartifact.mdindex 58128c4ab4..92e265f8aa 100644--- a/docs/services/codeartifact.md+++ b/docs/services/codeartifact.md@@ -7,10 +7,10 @@Floci supports the CodeArtifact control plane: domains, repositories, resource policies, tags,and public upstream (external) connections. Package publish/fetch through the CodeArtifact APIitself is implemented for the `generic` format only, matching AWS's own restriction that-`PublishPackageVersion` accepts only `generic`. The `maven` and `npm` formats are each served-through their own real package-manager-protocol proxy (`mvn`/Gradle and `npm`/`yarn`/`pnpm`-publish and resolve all work against the URL `GetRepositoryEndpoint` returns); the remaining-formats (PyPI, NuGet, etc.) have no real proxy behind them yet.+`PublishPackageVersion` accepts only `generic`. The `maven`, `npm`, and `pypi` formats are each+served through their own real package-manager-protocol proxy (`mvn`/Gradle, `npm`/`yarn`/`pnpm`,+and `pip`/`twine` publish and resolve all work against the URL `GetRepositoryEndpoint` returns);+the remaining formats (NuGet, etc.) have no real proxy behind them yet.## Supported Actions@@ -141,6 +141,45 @@ client-unreachable address; without this, `npm install` would try to fetch the tfrom an address it cannot reach. One config knob, `FLOCI_SERVICES_CODEARTIFACT_NPM_IMAGE`, pinsthe image version.+## The pypi repository endpoint++`GetRepositoryEndpoint` for `format=pypi` returns `http://localhost:4566/codeartifact/pypi/<domain>/<repository>/`.+Real pip and twine can install from and publish to that URL directly; it speaks the real PyPI+simple-repository protocol (simple index, package download, twine's multipart upload), proxied+straight through to a real [pypiserver](https://pypi.org/project/pypiserver/) instance rather than+reimplemented. Every request needs a token from `GetAuthorizationToken` scoped to the domain being+accessed, sent as HTTP Basic with the token as the password (any username) - AWS documents `pip`'s+index URL embedding `aws:$CODEARTIFACT_AUTH_TOKEN@` and twine's `.pypirc`/environment variables as+`username=aws, password=<token>`+([configure pip](https://docs.aws.amazon.com/codeartifact/latest/ug/python-configure-pip.html),+[configure twine](https://docs.aws.amazon.com/codeartifact/latest/ug/python-configure-twine.html)),+the same Basic convention Maven's HTTP wagon uses, not npm's Bearer-only `_authToken`; a Bearer+token is also accepted, matching npm, since accepting both costs nothing. A missing, invalid,+expired, or wrong-domain token gets a 401 challenging `Basic`.++Like Verdaccio, pypiserver has no native concept of multiple named indexes inside one instance, so+each CodeArtifact repository gets its own pypiserver container instead of sharing one, started+lazily on first use and identified internally by a fresh id generated at `CreateRepository` time,+so a repository deleted and recreated under the same name never inherits the previous one's+packages. `DeleteRepository` stops and removes that repository's container immediately, the same+as npm. Unlike Verdaccio, no public-URL environment variable is needed: pypiserver's simple-index+responses link to package files with a root-relative path (`/packages/<file>`), which pip and+twine resolve against whatever host they actually connected to, not an address the container+returns itself.++Unlike Reposilite (`redeployment: false`) and Verdaccio (which rejects a duplicate publish+natively, matching real npm), pypiserver has no overwrite protection of its own: publishing the+same package name, version, and filename twice with different content silently replaces the file+(confirmed by hand against a live instance with both twine and a raw multipart upload). Floci's+proxy closes this gap itself, matching AWS's own documented behavior for every format+("Overwriting package assets" in CodeArtifact's packages overview): before forwarding an upload, it+checks the target repository's own simple index for the exact filename being uploaded; if the+filename doesn't exist yet, the upload proceeds, if it exists with byte-identical content the+upload succeeds without touching pypiserver again (idempotent, so a client's retry after a dropped+response is never a spurious conflict), and only a name collision with genuinely different content+gets rejected with 409. One config knob, `FLOCI_SERVICES_CODEARTIFACT_PYPI_IMAGE`, pins the image+version.+## AWS-compatible failuresDomain and repository names, tags, pagination, duplicate names, missing upstreams, policy-revision@@ -175,34 +214,66 @@ state.`DisposePackageVersions`, and `UpdatePackageVersionsStatus` don't exist yet; the only way tomove a version from `Unfinished` to `Published` today is a follow-up `PublishPackageVersion`call that omits the `unfinished` flag.-- **The 5 GB asset size quota is nominal.** `PublishPackageVersion` and the Maven repository- endpoint both receive the request body as a single byte array before Floci ever checks its- length, so a request already large enough to exhaust available heap fails before the quota check- runs. The rejection (`ServiceQuotaExceededException` from `PublishPackageVersion`, HTTP 413 from- the Maven endpoint) is correct for anything that does fit in memory; it is not itself a streaming- size limit.-- **Maven artifacts and npm packages do not survive a Floci restart, even under persistent- storage.** The Reposilite instance and every per-repository Verdaccio container have no volume- attached and are removed on shutdown along with everything published to them. CodeArtifact- repository/domain metadata (including each format's stored sidecar container id) survives a- restart the same way any other Floci state does under persistent storage mode; the artifacts and- packages themselves do not, so the first request after a restart re-provisions empty storage and- returns 404 for anything published before the restart.+- **The 5 GB asset size quota is nominal.** `PublishPackageVersion` and the Maven and pypi+ repository endpoints all receive the request body as a single byte array before Floci ever+ checks its length, so a request already large enough to exhaust available heap fails before the+ quota check runs. The rejection (`ServiceQuotaExceededException` from `PublishPackageVersion`,+ HTTP 413 from the Maven and pypi endpoints) is correct for anything that does fit in memory; it+ is not itself a streaming size limit.+- **Maven artifacts, npm packages, and pypi packages do not survive a Floci restart, even under+ persistent storage.** The Reposilite instance and every per-repository Verdaccio and pypiserver+ container have no volume attached and are removed on shutdown along with everything published to+ them. CodeArtifact repository/domain metadata (including each format's stored sidecar container+ id) survives a restart the same way any other Floci state does under persistent storage mode; the+ artifacts and packages themselves do not, so the first request after a restart re-provisions+ empty storage and returns 404 for anything published before the restart.- **`GetAuthorizationToken` tokens are not revocable and are not tied to any IAM identity.** RealCodeArtifact tokens are scoped to the calling principal's permissions; Floci's are scoped only tothe domain named in the request; anyone who obtains one keeps the same domain-scoped access forits full lifetime.-- **Neither the Maven nor the npm repository endpoint resolves through upstream repositories or+- **The Maven, npm, and pypi sidecar containers have no authentication of their own, and anything+ else sharing their Docker network can reach them directly on their container port.** Real+ CodeArtifact's token check happens once, at Floci's own proxy layer (`CodeArtifactMavenController`/+ the npm data plane/`CodeArtifactPypiController`), before a request ever reaches Reposilite,+ Verdaccio, or pypiserver; those containers' own auth is deliberately disabled since the proxy is+ meant to be the only path in. Binding the published host port to loopback only keeps an outside+ host off that port; it does not stop another container already on the same configured Docker+ network (for example a Lambda, Batch, or ECS workload container) from reaching the sidecar on its+ container-internal address, bypassing the proxy's token check entirely. Closing this for real+ would mean isolating each sidecar on its own Docker network reachable only from Floci's own+ process, a change that would need to apply uniformly across all three sidecar-backed formats, not+ a one-off fix to any single proxy.+- **Neither the Maven, npm, nor pypi repository endpoint resolves through upstream repositories orexternal connections.** On real CodeArtifact, a repository with another repository configured asan upstream (`UpdateRepository`'s `upstreams`) or with an `AssociateExternalConnection` to a- public repository (`public:maven-central`, `public:npmjs`, etc.) serves packages from those- sources too, not just its own. Floci's proxies only ever look up the repository's own backing- storage: a package that exists solely in an upstream, or only through an external connection,- returns 404 through a repository that has it configured as one.+ public repository (`public:maven-central`, `public:npmjs`, `public:pypi`, etc.) serves packages+ from those sources too, not just its own. Floci's proxies only ever look up the repository's own+ backing storage: a package that exists solely in an upstream, or only through an external+ connection, returns 404 through a repository that has it configured as one; the pypi proxy's+ pypiserver container is also started with `--disable-fallback` specifically so it never silently+ redirects a missing package to the real, public PyPI.- **The npm repository endpoint does not enforce the 5 GB asset size quota.** Unlike`PublishPackageVersion` and the Maven endpoint, it streams the request straight through to thebacking Verdaccio container rather than buffering it first, so there is nowhere in the requestpath to check a byte count against the quota before forwarding it.+- **The pypi repository endpoint only serves the HTML Simple Repository API, not the PEP 691/700+ JSON variant.** AWS documents both: pip/uv can request+ `application/vnd.pypi.simple.v1+json` from `/simple/<project>/` and get a JSON index with+ per-file hashes, `requires-python`, and `upload-time`. Floci's proxy only ever forwards+ pypiserver's native HTML response regardless of the client's `Accept` header, since pypiserver+ itself has no JSON index support to pass through. Low practical risk in itself, since PEP 691+ requires clients to accept HTML as a fallback and pip/uv both do, but it is a real gap against+ AWS's documented surface, not just a quirk of the sidecar.+- **`GetPackageVersionAsset` and `DescribePackageVersion` can't see Maven-, npm-, or pypi-uploaded+ artifacts.** Those three formats' repository endpoints are deliberately metadata-free passthroughs+ straight to their sidecar container (Reposilite/Verdaccio/pypiserver) and never create the+ generic-format package-version record these two actions depend on+ (`CodeArtifactService.getPackageVersionAsset`/`describePackageVersion` only ever look up+ `packageVersions.getForAccount`, the record only the `generic` format's `PublishPackageVersion`+ flow creates). Real CodeArtifact's `GetPackageVersionAsset` works uniformly across every format.+ The artifact is really there and downloadable through that format's own wire endpoint+ (`mvn dependency:get`, `npm install`, `pip install`, or a direct `curl`/`twine`); only the+ CodeArtifact-native asset-get/describe calls 404 for it.See the [CodeArtifact API Reference](https://docs.aws.amazon.com/codeartifact/latest/APIReference/Welcome.html).@@ -215,3 +286,4 @@ See the [CodeArtifact API Reference](https://docs.aws.amazon.com/codeartifact/la| `FLOCI_SERVICES_CODEARTIFACT_MAVEN_URL` | unset | When set, use this URL and skip Reposilite container management || `FLOCI_SERVICES_CODEARTIFACT_MAVEN_TOKEN` | unset | `name:secret` access token for a pre-configured `MAVEN_URL` || `FLOCI_SERVICES_CODEARTIFACT_NPM_IMAGE` | `verdaccio/verdaccio:6.10.4` | Verdaccio image used to serve the `npm` format |+| `FLOCI_SERVICES_CODEARTIFACT_PYPI_IMAGE` | `pypiserver/pypiserver:v2.4.2` | pypiserver image used to serve the `pypi` format |diff --git a/docs/services/index.md b/docs/services/index.mdindex d49be6ea62..c3e8a875fa 100644--- a/docs/services/index.md+++ b/docs/services/index.md@@ -121,7 +121,7 @@ Operation counts are exact. For dispatch-table services (Query and JSON 1.1) eac| [AWS Backup](backup.md) | `/backup-vaults/*`, `/backup/plans/*`, `/backup-jobs/*`, `/supported-resource-types` | REST JSON | 20 || [AWS FIS](fis.md) | `/experimentTemplates/*`, `/experiments/*`, `/actions/*`, `/targetResourceTypes/*`, `/safetyLevers/*`, `/tags/*` | REST JSON | 26 || [CodeGuru Reviewer](codegurureviewer.md) | `/associations`, `/associations/{associationArn}`, `/tags/*` | REST JSON | 7 |-| [CodeArtifact](codeartifact.md) | `/v1/domain*`, `/v1/repository*`, `/v1/package/version*`, `/v1/tag*`, `/v1/authorization-token`, `/codeartifact/maven/*`, `/codeartifact/npm/*` | REST JSON | 26 |+| [CodeArtifact](codeartifact.md) | `/v1/domain*`, `/v1/repository*`, `/v1/package/version*`, `/v1/tag*`, `/v1/authorization-token`, `/codeartifact/maven/*`, `/codeartifact/npm/*`, `/codeartifact/pypi/*` | REST JSON | 26 || [CloudFront](cloudfront.md) | `/2020-05-31/distribution/*`, `/2020-05-31/cache-policy/*`, `/2020-05-31/function/*` | REST XML | 50 || [Route53](route53.md) | `/2013-04-01/hostedzone/*`, `/2013-04-01/healthcheck/*`, `/2013-04-01/change/*` | REST XML | 25 || [Route 53 Resolver](route53resolver.md) | `POST /` + `X-Amz-Target: Route53Resolver.*` | JSON 1.1 | 18 |diff --git a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.javaindex 20e1b9acb2..7b749edb00 100644--- a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java+++ b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java@@ -876,6 +876,15 @@ public interface EmulatorConfig {*/@WithDefault("verdaccio/verdaccio:6.10.4")String npmImage();++ /**+ * Image used for the per-repository pypiserver container backing the {@code pypi} format.+ * Same reasoning as {@link #npmImage()}: pypiserver has no native concept of multiple+ * named indexes inside one instance either, so this gets one container per CodeArtifact+ * repository too, and there is no single external instance a URL override could name.+ */+ @WithDefault("pypiserver/pypiserver:v2.4.2")+ String pypiImage();}interface ConnectServiceConfig {diff --git a/src/main/java/io/github/hectorvent/floci/core/common/MultipartFormParser.java b/src/main/java/io/github/hectorvent/floci/core/common/MultipartFormParser.javanew file mode 100644index 0000000000..2c5fb1ee04--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/core/common/MultipartFormParser.java@@ -0,0 +1,187 @@+package io.github.hectorvent.floci.core.common;++import java.nio.charset.StandardCharsets;+import java.util.ArrayList;+import java.util.Arrays;+import java.util.LinkedHashMap;+import java.util.List;+import java.util.Locale;+import java.util.Map;+import java.util.Optional;++/**+ * Parses a {@code multipart/form-data} body into its plain text fields and, at most, one file+ * part, without RESTEasy's {@code MultipartFormDataInput}: that reads reflectively in a way that+ * needs its own native-image registration per caller, where this is a handful of byte-array+ * scans. Pulled out of {@code S3Controller}'s presigned-POST handling once+ * {@code CodeArtifactPypiController} needed the identical parsing for a twine upload's own+ * multipart body.+ */+public final class MultipartFormParser {++ private MultipartFormParser() {+ }++ /** One named field's value, or the one file part carrying a {@code filename} parameter. */+ public record FilePart(String fieldName, String filename, String contentType, byte[] content) {}++ /** {@code fields} excludes the file part; {@code file} is empty when the body carried none. */+ public record ParsedForm(Map<String, String> fields, Optional<FilePart> file) {}++ /** The {@code boundary} parameter of a {@code multipart/form-data} Content-Type, if present. */+ public static Optional<String> extractBoundary(String contentType) {+ if (contentType == null) {+ return Optional.empty();+ }+ for (String part : contentType.split(";")) {+ String trimmed = part.trim();+ if (trimmed.toLowerCase(Locale.ROOT).startsWith("boundary=")) {+ String boundary = trimmed.substring("boundary=".length()).trim();+ if (boundary.startsWith("\"") && boundary.endsWith("\"")) {+ boundary = boundary.substring(1, boundary.length() - 1);+ }+ return Optional.of(boundary);+ }+ }+ return Optional.empty();+ }++ public static ParsedForm parse(byte[] body, String boundary) {+ Map<String, String> fields = new LinkedHashMap<>();+ FilePart file = null;++ byte[] boundaryBytes = ("--" + boundary).getBytes(StandardCharsets.UTF_8);+ for (byte[] part : splitParts(body, boundaryBytes)) {+ int headerEnd = indexOfDoubleNewline(part);+ if (headerEnd < 0) {+ continue;+ }+ String headers = new String(part, 0, headerEnd, StandardCharsets.UTF_8);+ int bodyStart = headerEnd + 4; // skip \r\n\r\n+ byte[] partBody = Arrays.copyOfRange(part, bodyStart, part.length);++ // Trim the trailing CRLF the boundary split leaves on every part body.+ if (partBody.length >= 2+ && partBody[partBody.length - 2] == '\r'+ && partBody[partBody.length - 1] == '\n') {+ partBody = Arrays.copyOf(partBody, partBody.length - 2);+ }++ String disposition = extractHeaderValue(headers, "Content-Disposition");+ if (disposition == null) {+ continue;+ }+ String fieldName = extractDispositionParam(disposition, "name");+ if (fieldName == null) {+ continue;+ }++ String filename = extractDispositionParam(disposition, "filename");+ if (filename != null) {+ String partContentType = extractHeaderValue(headers, "Content-Type");+ file = new FilePart(fieldName, filename,+ partContentType == null ? null : partContentType.trim(), partBody);+ } else {+ fields.put(fieldName, new String(partBody, StandardCharsets.UTF_8));+ }+ }+ return new ParsedForm(fields, Optional.ofNullable(file));+ }++ private static List<byte[]> splitParts(byte[] body, byte[] boundary) {+ List<byte[]> parts = new ArrayList<>();+ int pos = indexOf(body, boundary, 0);+ if (pos < 0) {+ return parts;+ }+ pos += boundary.length;+ if (pos < body.length - 1 && body[pos] == '-' && body[pos + 1] == '-') {+ return parts; // closing boundary immediately, no parts+ }+ if (pos < body.length - 1 && body[pos] == '\r' && body[pos + 1] == '\n') {+ pos += 2;+ }++ while (pos < body.length) {+ int nextBoundary = indexOf(body, boundary, pos);+ if (nextBoundary < 0) {+ break;+ }+ parts.add(Arrays.copyOfRange(body, pos, nextBoundary));+ pos = nextBoundary + boundary.length;+ if (pos < body.length - 1 && body[pos] == '-' && body[pos + 1] == '-') {+ break;+ }+ if (pos < body.length - 1 && body[pos] == '\r' && body[pos + 1] == '\n') {+ pos += 2;+ }+ }+ return parts;+ }++ private static int indexOf(byte[] data, byte[] pattern, int fromIndex) {+ outer:+ for (int i = fromIndex; i <= data.length - pattern.length; i++) {+ for (int j = 0; j < pattern.length; j++) {+ if (data[i + j] != pattern[j]) {+ continue outer;+ }+ }+ return i;+ }+ return -1;+ }++ private static int indexOfDoubleNewline(byte[] data) {+ for (int i = 0; i < data.length - 3; i++) {+ if (data[i] == '\r' && data[i + 1] == '\n' && data[i + 2] == '\r' && data[i + 3] == '\n') {+ return i;+ }+ }+ return -1;+ }++ private static String extractHeaderValue(String headers, String headerName) {+ String lowerHeaders = headers.toLowerCase(Locale.ROOT);+ String lowerName = headerName.toLowerCase(Locale.ROOT) + ":";+ int idx = lowerHeaders.indexOf(lowerName);+ if (idx < 0) {+ return null;+ }+ int valueStart = idx + lowerName.length();+ int lineEnd = headers.indexOf('\r', valueStart);+ if (lineEnd < 0) {+ lineEnd = headers.indexOf('\n', valueStart);+ }+ if (lineEnd < 0) {+ lineEnd = headers.length();+ }+ return headers.substring(valueStart, lineEnd).trim();+ }++ private static String extractDispositionParam(String disposition, String paramName) {+ String search = paramName + "=";+ int idx = disposition.indexOf(search);+ if (idx < 0) {+ return null;+ }+ int valueStart = idx + search.length();+ if (valueStart >= disposition.length()) {+ return null;+ }+ if (disposition.charAt(valueStart) == '"') {+ valueStart++;+ int valueEnd = disposition.indexOf('"', valueStart);+ if (valueEnd < 0) {+ return disposition.substring(valueStart);+ }+ return disposition.substring(valueStart, valueEnd);+ } else {+ int valueEnd = disposition.indexOf(';', valueStart);+ if (valueEnd < 0) {+ valueEnd = disposition.length();+ }+ return disposition.substring(valueStart, valueEnd).trim();+ }+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/core/common/ResolvedServiceCatalog.java b/src/main/java/io/github/hectorvent/floci/core/common/ResolvedServiceCatalog.javaindex 41892d388e..48e5bbf4b1 100644--- a/src/main/java/io/github/hectorvent/floci/core/common/ResolvedServiceCatalog.java+++ b/src/main/java/io/github/hectorvent/floci/core/common/ResolvedServiceCatalog.java@@ -9,6 +9,8 @@ import io.github.hectorvent.floci.services.appconfig.AppConfigDataController;import io.github.hectorvent.floci.services.batch.BatchController;import io.github.hectorvent.floci.services.bedrock.BedrockController;import io.github.hectorvent.floci.services.bedrockruntime.BedrockRuntimeController;+import io.github.hectorvent.floci.services.codeartifact.CodeArtifactMavenController;+import io.github.hectorvent.floci.services.codeartifact.CodeArtifactPypiController;import io.github.hectorvent.floci.services.cognito.CognitoOAuthController;import io.github.hectorvent.floci.services.cognito.CognitoWellKnownController;import io.github.hectorvent.floci.services.eks.EksController;@@ -743,7 +745,9 @@ public class ResolvedServiceCatalog {"codeartifact", config.storage().mode(), 5000L, null, ServiceProtocol.REST_JSON,protocols(ServiceProtocol.REST_JSON),Set.of(), Set.of("codeartifact"), Set.of(),- Set.of(io.github.hectorvent.floci.services.codeartifact.CodeArtifactController.class)),+ Set.of(io.github.hectorvent.floci.services.codeartifact.CodeArtifactController.class,+ CodeArtifactMavenController.class,+ CodeArtifactPypiController.class)),descriptor("verifiedpermissions", "verifiedpermissions", config.services().verifiedpermissions().enabled(), true,"verifiedpermissions", config.storage().mode(), 5000L, null, ServiceProtocol.JSON,protocols(ServiceProtocol.JSON),diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenController.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenController.javaindex 635f5d6926..bb73185e8a 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenController.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenController.java@@ -18,10 +18,6 @@ import jakarta.ws.rs.core.HttpHeaders;import jakarta.ws.rs.core.MediaType;import jakarta.ws.rs.core.Response;-import java.nio.charset.StandardCharsets;-import java.util.Base64;-import java.util.Optional;-/*** Serves the real Maven repository wire protocol behind the URL {@code GetRepositoryEndpoint}* returns for the {@code maven} format: plain GET/PUT/HEAD over a GAV path, not a CodeArtifact@@ -43,9 +39,6 @@ import java.util.Optional;@Path("/codeartifact/maven/{domain}/{repository}/{gav:.+}")public class CodeArtifactMavenController {- private static final String BEARER_PREFIX = "Bearer ";- private static final String BASIC_PREFIX = "Basic ";-private final CodeArtifactService service;private final ReposiliteSidecarClient reposilite;@@ -96,7 +89,7 @@ public class CodeArtifactMavenController {}private String requireRepositoryId(HttpHeaders headers, String domain, String repository) {- String token = extractToken(headers);+ String token = CodeArtifactTokenExtractor.extractToken(headers);AuthorizationTokenScope scope = service.resolveAuthorizationToken(token, domain).orElseThrow(() -> new NotAuthorizedException("Basic realm=\"floci-codeartifact\""));String repoId;@@ -108,33 +101,4 @@ public class CodeArtifactMavenController {reposilite.ensureRepository(repoId);return repoId;}-- /**- * {@code null} when neither scheme is present, malformed, or the Basic credentials have no- * password field; {@link CodeArtifactService#resolveAuthorizationToken} treats a null token- * the same as any other invalid one.- */- private static String extractToken(HttpHeaders headers) {- String authorization = headers.getHeaderString(HttpHeaders.AUTHORIZATION);- if (authorization == null) {- return null;- }- if (authorization.startsWith(BEARER_PREFIX)) {- return authorization.substring(BEARER_PREFIX.length());- }- if (authorization.startsWith(BASIC_PREFIX)) {- return passwordFromBasicCredentials(authorization.substring(BASIC_PREFIX.length()));- }- return null;- }-- private static String passwordFromBasicCredentials(String base64Credentials) {- try {- String decoded = new String(Base64.getDecoder().decode(base64Credentials), StandardCharsets.UTF_8);- int separator = decoded.indexOf(':');- return separator >= 0 ? decoded.substring(separator + 1) : null;- } catch (IllegalArgumentException e) {- return null;- }- }}diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiController.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiController.javanew file mode 100644index 0000000000..3f129ca385--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiController.java@@ -0,0 +1,284 @@+package io.github.hectorvent.floci.services.codeartifact;++import io.github.hectorvent.floci.core.common.AwsException;+import io.github.hectorvent.floci.core.common.MultipartFormParser;+import io.github.hectorvent.floci.services.codeartifact.CodeArtifactService.AuthorizationTokenScope;+import jakarta.inject.Inject;+import jakarta.ws.rs.Consumes;+import jakarta.ws.rs.GET;+import jakarta.ws.rs.NotAuthorizedException;+import jakarta.ws.rs.NotFoundException;+import jakarta.ws.rs.POST;+import jakarta.ws.rs.Path;+import jakarta.ws.rs.PathParam;+import jakarta.ws.rs.Produces;+import jakarta.ws.rs.core.Context;+import jakarta.ws.rs.core.HttpHeaders;+import jakarta.ws.rs.core.MediaType;+import jakarta.ws.rs.core.Response;++import java.net.URI;+import java.net.http.HttpClient;+import java.net.http.HttpRequest;+import java.net.http.HttpRequest.BodyPublishers;+import java.net.http.HttpResponse;+import java.net.http.HttpResponse.BodyHandlers;+import java.nio.charset.StandardCharsets;+import java.time.Duration;+import java.util.Arrays;+import java.util.Locale;+import java.util.Optional;+import java.util.concurrent.ConcurrentHashMap;++/**+ * Serves the real PyPI simple-repository wire protocol behind the URL+ * {@code GetRepositoryEndpoint} returns for the {@code pypi} format: the same plain GET/POST+ * pip and twine speak against real PyPI, not a CodeArtifact API action. Backed by one pypiserver+ * container per repository, provisioned on first use.+ *+ * <p>Requires the bearer token real CodeArtifact requires here, obtained from+ * {@code GetAuthorizationToken} and scoped to the domain being accessed. AWS documents pip and+ * twine both authenticating with HTTP Basic ({@code username=aws, password=<token>}, verified+ * against AWS's own pip and twine configuration docs), the same convention Maven's HTTP wagon+ * uses, not npm's Bearer-only {@code _authToken}; {@link CodeArtifactTokenExtractor} accepts+ * either scheme.+ *+ * <p>A pip/twine request carries no SigV4 {@code Authorization} header, so there is no account or+ * Region to resolve from it the way the CodeArtifact JSON API does; the token itself supplies+ * both, since it was issued under a specific account and Region.+ *+ * <p>Unlike Maven and npm, this rejects a re-upload of a file pypiserver has already published+ * itself: pypiserver has no immutability check of its own (confirmed by uploading the same+ * name/version/filename twice against a real instance; the second upload silently replaces the+ * first), unlike Reposilite ({@code redeployment: false}) and Verdaccio (rejects natively,+ * matching real npm). A pre-upload check against pypiserver's own simple index and the existing+ * file's own bytes is what gives PyPI the same immutability guarantee the other two formats+ * already get for free from their sidecars, including AWS's own documented exception to it:+ * republishing an asset whose content is byte-identical to what is already there succeeds+ * idempotently rather than conflicting (verified against AWS's CodeArtifact packages-overview+ * docs, "Overwriting package assets": only a same-name upload with <i>different</i> content+ * returns 409).+ */+@Path("/codeartifact/pypi/{domain}/{repository}")+public class CodeArtifactPypiController {++ private static final String SIMPLE_PATH = "/simple/";++ private final CodeArtifactService service;+ private final PypiserverSidecarManager pypiserver;+ private final HttpClient httpClient;+ private final ConcurrentHashMap<String, Object> uploadLocks = new ConcurrentHashMap<>();++ @Inject+ public CodeArtifactPypiController(CodeArtifactService service, PypiserverSidecarManager pypiserver) {+ this(service, pypiserver, HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(5)).build());+ }++ CodeArtifactPypiController(CodeArtifactService service, PypiserverSidecarManager pypiserver,+ HttpClient httpClient) {+ this.service = service;+ this.pypiserver = pypiserver;+ this.httpClient = httpClient;+ }++ private record RepositoryHandle(String repoId, String baseUrl, String pathPrefix) {+ }++ // No route for the bare "/simple/" project-list endpoint: AWS's CodeArtifact User Guide+ // documents it as unsupported ("Unsupported PyPI APIs", only /simple/<project>/ is served).++ /**+ * pypiserver's own simple-index HTML links to package files with a root-relative+ * {@code href="/packages/<file>"}, resolved by pip against whatever host it actually+ * connected to: this proxy's own host, not pypiserver's. Left unrewritten, that resolves to+ * {@code /packages/<file>} at the proxy's root, not under this repository's+ * {@code /codeartifact/pypi/<domain>/<repository>/} prefix where {@link #downloadFile} is+ * actually routed, so pip would 404 trying to install anything the index lists. Verified+ * against a real pypiserver response before writing this, not assumed.+ */+ @GET+ @Path("simple/{package}/")+ @Produces(MediaType.TEXT_HTML)+ public Response packageIndex(@Context HttpHeaders headers, @PathParam("domain") String domain,+ @PathParam("repository") String repository,+ @PathParam("package") String packageName) {+ RepositoryHandle handle = requireRepository(headers, domain, repository);+ Response response = proxyGet(handle.baseUrl(), SIMPLE_PATH + packageName + "/");+ if (response.getStatus() != 200) {+ return response;+ }+ String html = new String((byte[]) response.getEntity(), StandardCharsets.UTF_8);+ String rewritten = html.replace("href=\"/packages/", "href=\"" + handle.pathPrefix() + "/packages/");+ return Response.status(200).entity(rewritten.getBytes(StandardCharsets.UTF_8)).type(MediaType.TEXT_HTML)+ .build();+ }++ @GET+ @Path("packages/{filename}")+ @Produces(MediaType.WILDCARD)+ public Response downloadFile(@Context HttpHeaders headers, @PathParam("domain") String domain,+ @PathParam("repository") String repository,+ @PathParam("filename") String filename) {+ String baseUrl = requireRepository(headers, domain, repository).baseUrl();+ return proxyGet(baseUrl, "/packages/" + filename);+ }++ /**+ * Handles a twine upload: {@code POST} to the repository root with a+ * {@code multipart/form-data} body whose {@code content} part is the package file and whose+ * {@code name}/{@code version} fields identify it. When the exact filename being uploaded is+ * already published, compares its existing bytes to the new upload: byte-identical content+ * succeeds without forwarding anything (idempotent, matching AWS's own documented behavior for+ * a retried publish), while different content is rejected with 409 before ever reaching+ * pypiserver, since pypiserver has no immutability check of its own.+ *+ * <p>The existence check and the forwarded publish run inside a lock keyed by this+ * repository's id: unlike Reposilite ({@code redeployment: false}), pypiserver has no atomic+ * backstop of its own, so without this lock two uploads racing the same new filename could+ * both observe "not published yet" and both succeed, with pypiserver silently keeping whichever+ * write landed last instead of either one being rejected.+ */+ @POST+ @Consumes(MediaType.WILDCARD)+ public Response upload(@Context HttpHeaders headers, @PathParam("domain") String domain,+ @PathParam("repository") String repository, byte[] body) {+ byte[] content = body != null ? body : new byte[0];+ // Same 5 GB AWS quota PublishPackageVersion enforces, and just as nominal: RESTEasy+ // Reactive has already buffered the full body into this byte[] before this method runs,+ // so this rejects anything that made it this far rather than bounding memory use upfront.+ if (content.length > CodeArtifactService.MAX_ASSET_FILE_SIZE_BYTES) {+ return Response.status(413).build();+ }+ RepositoryHandle handle = requireRepository(headers, domain, repository);++ String contentType = headers.getHeaderString(HttpHeaders.CONTENT_TYPE);+ String boundary = MultipartFormParser.extractBoundary(contentType).orElse(null);+ if (boundary == null) {+ return Response.status(400).entity("Could not determine multipart boundary from Content-Type.").build();+ }+ MultipartFormParser.ParsedForm form = MultipartFormParser.parse(content, boundary);+ String packageName = form.fields().get("name");+ Optional<MultipartFormParser.FilePart> file = form.file();+ if (packageName == null || packageName.isBlank() || file.isEmpty()) {+ return Response.status(400).entity("Upload must include name and content fields.").build();+ }++ Object lock = uploadLocks.computeIfAbsent(handle.repoId(), k -> new Object());+ synchronized (lock) {+ Optional<byte[]> existing = existingFileContent(handle.baseUrl(), packageName, file.get().filename());+ if (existing.isPresent()) {+ if (Arrays.equals(existing.get(), file.get().content())) {+ return Response.status(200).build();+ }+ return Response.status(409)+ .entity("File already exists. See "+ + "https://docs.aws.amazon.com/codeartifact/latest/ug/PyPI-format.html")+ .build();+ }++ return forwardUpload(handle.baseUrl(), contentType, content);+ }+ }++ /**+ * {@code empty()} when {@code filename} isn't listed in the package's simple index yet, its+ * actual bytes otherwise. pypiserver normalizes a package name for its own simple-index path+ * the same way real PyPI does (PEP 503: lowercase, runs of {@code -_.} collapsed to one+ * {@code -}); querying with the raw, as-uploaded name for a package whose normalized form+ * differs would 404 even though the package exists.+ *+ * <p>A connectivity failure here throws rather than returning {@code empty()}: treating "the+ * check couldn't complete" the same as "nothing published yet" would let an upload with+ * different content through to pypiserver (which has no conflict check of its own) during+ * exactly the moment this proxy's own protection is unavailable, the opposite of fail-safe.+ */+ private Optional<byte[]> existingFileContent(String baseUrl, String packageName, String filename) {+ String normalized = normalizePackageName(packageName);+ HttpRequest indexRequest = HttpRequest.newBuilder(URI.create(baseUrl + SIMPLE_PATH + normalized + "/"))+ .timeout(Duration.ofSeconds(5)).GET().build();+ HttpResponse<String> indexResponse;+ try {+ indexResponse = httpClient.send(indexRequest, BodyHandlers.ofString());+ } catch (Exception e) {+ throw new IllegalStateException(+ "Could not check pypi package " + packageName + " for an existing " + filename+ + " before publishing", e);+ }+ if (indexResponse.statusCode() == 404) {+ return Optional.empty();+ }+ if (indexResponse.statusCode() != 200) {+ throw new IllegalStateException(+ "Could not check pypi package " + packageName + " for an existing " + filename+ + ": upstream returned " + indexResponse.statusCode());+ }+ if (!indexResponse.body().contains(filename)) {+ return Optional.empty();+ }++ HttpRequest fileRequest = HttpRequest.newBuilder(URI.create(baseUrl + "/packages/" + filename))+ .timeout(Duration.ofSeconds(10)).GET().build();+ HttpResponse<byte[]> fileResponse;+ try {+ fileResponse = httpClient.send(fileRequest, BodyHandlers.ofByteArray());+ } catch (Exception e) {+ throw new IllegalStateException(+ "Could not fetch the existing pypi file " + filename + " to check for a content match", e);+ }+ if (fileResponse.statusCode() != 200) {+ throw new IllegalStateException(+ "Could not fetch the existing pypi file " + filename + " to check for a content match: "+ + "upstream returned " + fileResponse.statusCode());+ }+ return Optional.of(fileResponse.body());+ }++ private static String normalizePackageName(String name) {+ return name.toLowerCase(Locale.ROOT).replaceAll("[-_.]+", "-");+ }++ private Response forwardUpload(String baseUrl, String contentType, byte[] content) {+ HttpRequest request = HttpRequest.newBuilder(URI.create(baseUrl + "/"))+ .timeout(Duration.ofSeconds(30))+ .header("Content-Type", contentType)+ .POST(BodyPublishers.ofByteArray(content))+ .build();+ HttpResponse<Void> response;+ try {+ response = httpClient.send(request, BodyHandlers.discarding());+ } catch (Exception e) {+ throw new IllegalStateException("Could not reach the pypi sidecar to publish this file", e);+ }+ return Response.status(response.statusCode()).build();+ }++ private Response proxyGet(String baseUrl, String path) {+ HttpRequest request = HttpRequest.newBuilder(URI.create(baseUrl + path))+ .timeout(Duration.ofSeconds(10)).GET().build();+ HttpResponse<byte[]> response;+ try {+ response = httpClient.send(request, BodyHandlers.ofByteArray());+ } catch (Exception e) {+ throw new IllegalStateException("Could not reach the pypi sidecar at " + path, e);+ }+ if (response.statusCode() == 404) {+ return Response.status(404).build();+ }+ String contentType = response.headers().firstValue("Content-Type").orElse("application/octet-stream");+ return Response.status(response.statusCode()).entity(response.body()).type(contentType).build();+ }++ private RepositoryHandle requireRepository(HttpHeaders headers, String domain, String repository) {+ String token = CodeArtifactTokenExtractor.extractToken(headers);+ AuthorizationTokenScope scope = service.resolveAuthorizationToken(token, domain)+ .orElseThrow(() -> new NotAuthorizedException("Basic realm=\"floci-codeartifact\""));+ String repoId;+ try {+ repoId = service.ensureFormatContainerId("pypi", scope.region(), domain, scope.owner(), repository);+ } catch (AwsException e) {+ throw new NotFoundException();+ }+ String baseUrl = pypiserver.ensureReady(repoId, null);+ return new RepositoryHandle(repoId, baseUrl, "/codeartifact/pypi/" + domain + "/" + repository);+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.javaindex 9ad97fa76e..11c8b490a9 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.java@@ -73,7 +73,7 @@ public class CodeArtifactService implements Resettable {* and the backfill in {@link #ensureFormatContainerId} already handle it, including for* repositories created before that format's proxy existed.*/- private static final List<String> CONTAINER_BACKED_FORMATS = List.of("maven", "npm");+ private static final List<String> CONTAINER_BACKED_FORMATS = List.of("maven", "npm", "pypi");private static final Logger LOG = Logger.getLogger(CodeArtifactService.class);diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactTokenExtractor.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactTokenExtractor.javanew file mode 100644index 0000000000..28048a316d--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactTokenExtractor.java@@ -0,0 +1,57 @@+package io.github.hectorvent.floci.services.codeartifact;++import jakarta.ws.rs.core.HttpHeaders;++import java.nio.charset.StandardCharsets;+import java.util.Base64;++/**+ * Extracts a CodeArtifact authorization token from an incoming request's {@code Authorization}+ * header, accepting either scheme real package-manager clients use: {@code Bearer <token>}+ * directly, or HTTP Basic with the token as the password field and any username. AWS documents+ * only Basic for Maven's {@code settings.xml} and pip/twine's {@code username=aws} convention;+ * accepting Bearer too costs nothing and matches what a Bearer-only format like npm sends.+ *+ * <p>Pulled out of {@code CodeArtifactMavenController} once {@code CodeArtifactPypiController}+ * needed the identical extraction: pip and twine authenticate exactly the way Maven's HTTP wagon+ * does (verified against AWS's own pip and twine configuration docs, both showing+ * {@code username=aws, password=<token>}), not the Bearer-only convention npm's {@code _authToken}+ * uses.+ */+final class CodeArtifactTokenExtractor {++ private static final String BEARER_PREFIX = "Bearer ";+ private static final String BASIC_PREFIX = "Basic ";++ private CodeArtifactTokenExtractor() {+ }++ /**+ * {@code null} when neither scheme is present, malformed, or the Basic credentials have no+ * password field; {@link CodeArtifactService#resolveAuthorizationToken} treats a null token+ * the same as any other invalid one.+ */+ static String extractToken(HttpHeaders headers) {+ String authorization = headers.getHeaderString(HttpHeaders.AUTHORIZATION);+ if (authorization == null) {+ return null;+ }+ if (authorization.startsWith(BEARER_PREFIX)) {+ return authorization.substring(BEARER_PREFIX.length());+ }+ if (authorization.startsWith(BASIC_PREFIX)) {+ return passwordFromBasicCredentials(authorization.substring(BASIC_PREFIX.length()));+ }+ return null;+ }++ private static String passwordFromBasicCredentials(String base64Credentials) {+ try {+ String decoded = new String(Base64.getDecoder().decode(base64Credentials), StandardCharsets.UTF_8);+ int separator = decoded.indexOf(':');+ return separator >= 0 ? decoded.substring(separator + 1) : null;+ } catch (IllegalArgumentException e) {+ return null;+ }+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.javanew file mode 100644index 0000000000..deadd17d08--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.java@@ -0,0 +1,131 @@+package io.github.hectorvent.floci.services.codeartifact;++import io.github.hectorvent.floci.config.EmulatorConfig;+import io.github.hectorvent.floci.core.common.ContainerTeardown;+import io.github.hectorvent.floci.core.common.docker.ContainerBuilder;+import io.github.hectorvent.floci.core.common.docker.ContainerLifecycleManager;+import io.github.hectorvent.floci.core.common.docker.ContainerLifecycleManager.ContainerInfo;+import io.github.hectorvent.floci.core.common.docker.ContainerLifecycleManager.EndpointInfo;+import io.github.hectorvent.floci.core.common.docker.ContainerSpec;+import io.github.hectorvent.floci.core.common.docker.ContainerStorageHelper;+import io.github.hectorvent.floci.core.common.docker.PerKeyContainerPool;+import jakarta.enterprise.context.ApplicationScoped;+import jakarta.inject.Inject;+import org.jboss.logging.Logger;++import java.util.List;++/**+ * Lazily starts and manages one pypiserver container per CodeArtifact repository backing the+ * {@code pypi} format. Like Verdaccio (npm), pypiserver has no native concept of multiple named+ * indexes inside one instance, so pypi repositories each get their own container instead of+ * sharing one the way Reposilite's Maven repositories do. The per-key container lifecycle (map of+ * running containers, per-key start lock, health poll, restart-on-unhealthy, stop-all on+ * shutdown/reset) is generic and lives in {@link PerKeyContainerPool}; this class only knows how+ * to build and configure a pypiserver container specifically.+ *+ * <p>Each container is configured with authentication disabled (real CodeArtifact authorization+ * happens once at Floci's proxy layer, {@code CodeArtifactPypiController}, before a request ever+ * reaches this container) and {@code --disable-fallback}, matching the Maven and npm proxies' own+ * deliberate choice not to resolve upstream repositories or external connections: pypiserver+ * otherwise redirects a package missing from its local index to the real, public PyPI.+ *+ * <p>Unlike Verdaccio, this needs no public-URL environment variable: pypiserver's simple-index+ * responses link to package files with a root-relative path ({@code /packages/<file>}), which pip+ * and twine resolve against whatever host they actually connected to, not an address the+ * container returns itself. There is nothing here for an internal, client-unreachable address to+ * leak into. That root-relative path still needs rewriting to this repository's own+ * {@code /codeartifact/pypi/<domain>/<repository>} prefix before a client sees it, since the+ * download route lives there, not at the proxy's bare root; {@code CodeArtifactPypiController}+ * does that rewrite, not this class.+ */+@ApplicationScoped+public class PypiserverSidecarManager implements RepositorySidecarManager, ContainerTeardown {++ private static final Logger LOG = Logger.getLogger(PypiserverSidecarManager.class);+ private static final String FORMAT = "pypi";+ private static final int PYPISERVER_PORT = 8080;+ private static final String HEALTH_PATH = "/health";+ private static final String PACKAGES_DIR = "/data/packages";++ private final ContainerBuilder containerBuilder;+ private final ContainerLifecycleManager lifecycleManager;+ private final EmulatorConfig config;+ private final PerKeyContainerPool pool;++ @Inject+ public PypiserverSidecarManager(ContainerBuilder containerBuilder, ContainerLifecycleManager lifecycleManager,+ EmulatorConfig config) {+ this.containerBuilder = containerBuilder;+ this.lifecycleManager = lifecycleManager;+ this.config = config;+ this.pool = new PerKeyContainerPool(lifecycleManager, HEALTH_PATH);+ }++ @Override+ public String format() {+ return FORMAT;+ }++ /**+ * Base URL of a ready pypiserver instance for this pypi repository, starting its container if+ * this is the first use. {@code publicUrl} is unused: see the class javadoc for why pypiserver+ * needs no self-referential URL rewritten.+ */+ @Override+ public String ensureReady(String pypiRepositoryId, String publicUrl) {+ return pool.ensureReady(pypiRepositoryId, () -> startContainer(pypiRepositoryId));+ }++ /** Stops and removes the container for one pypi repository, if one was ever started. */+ @Override+ public void release(String pypiRepositoryId) {+ pool.stopContainer(pypiRepositoryId);+ }++ private PerKeyContainerPool.StartedContainer startContainer(String pypiRepositoryId) {+ String image = config.services().codeartifact().pypiImage();+ String containerName = ContainerStorageHelper.dockerName(config, "floci-pypiserver-" + pypiRepositoryId);+ lifecycleManager.removeIfExists(containerName);++ // Loopback-only: this container authenticates nothing on its own (auth disabled below), so+ // the Bearer check in CodeArtifactPypiController is the only thing standing between a+ // client and the backing storage. Publishing this to every interface would let anyone who+ // can reach the host bypass that check entirely, the same reasoning as Verdaccio's binding.+ ContainerSpec spec = containerBuilder.newContainer(image)+ .withName(containerName)+ .withLoopbackPortBinding(PYPISERVER_PORT, 0)+ .withDockerNetwork(config.services().dockerNetwork())+ .withEmbeddedDns()+ .withLogRotation()+ .withCmd(List.of(+ "run",+ "-p", String.valueOf(PYPISERVER_PORT),+ // Disables pypiserver's own auth entirely (real CodeArtifact authorization+ // is already enforced before a request reaches this container) and its+ // default redirect-to-real-PyPI for a package missing from the local index,+ // matching the Maven and npm proxies' own choice not to resolve upstreams.+ "-a", ".",+ "-P", ".",+ "--disable-fallback",+ "--health-endpoint", HEALTH_PATH,+ PACKAGES_DIR))+ .build();+ String containerId = lifecycleManager.create(spec);+ try {+ ContainerInfo info = lifecycleManager.startCreated(containerId, spec);+ EndpointInfo endpoint = info.getEndpoint(PYPISERVER_PORT);+ String url = "http://" + endpoint;+ LOG.infov("pypiserver sidecar for pypi repository {0} is ready at {1}", pypiRepositoryId, url);+ return new PerKeyContainerPool.StartedContainer(containerId, url);+ } catch (RuntimeException e) {+ lifecycleManager.stopAndRemove(containerId, null);+ throw e;+ }+ }++ @Override+ public void stopManagedContainers() {+ pool.stopAll();+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/s3/S3Controller.java b/src/main/java/io/github/hectorvent/floci/services/s3/S3Controller.javaindex d61b07bca5..d6f7506ce5 100644--- a/src/main/java/io/github/hectorvent/floci/services/s3/S3Controller.java+++ b/src/main/java/io/github/hectorvent/floci/services/s3/S3Controller.java@@ -9,6 +9,7 @@ import io.github.hectorvent.floci.core.common.AwsException;import io.github.hectorvent.floci.core.common.AwsNamespaces;import io.github.hectorvent.floci.core.common.AwsRegions;import io.github.hectorvent.floci.core.common.IamEnforcementFilter;+import io.github.hectorvent.floci.core.common.MultipartFormParser;import io.github.hectorvent.floci.core.common.XmlBuilder;import io.github.hectorvent.floci.core.common.XmlParser;import io.github.hectorvent.floci.core.common.RegionResolver;@@ -3444,55 +3445,14 @@ public class S3Controller {}private Response doHandlePresignedPost(String bucket, String contentType, byte[] body) {- String boundary = extractBoundary(contentType);- if (boundary == null) {- throw new AwsException("InvalidArgument",- "Could not determine multipart boundary from Content-Type.", 400);- }-- Map<String, String> fields = new LinkedHashMap<>();- byte[] fileData = null;- String fileContentType = null;+ String boundary = MultipartFormParser.extractBoundary(contentType).orElseThrow(() ->+ new AwsException("InvalidArgument",+ "Could not determine multipart boundary from Content-Type.", 400));- byte[] boundaryBytes = ("--" + boundary).getBytes(StandardCharsets.UTF_8);- List<byte[]> parts = splitMultipartParts(body, boundaryBytes);-- for (byte[] part : parts) {- int headerEnd = indexOfDoubleNewline(part);- if (headerEnd < 0) {- continue;- }- String headers = new String(part, 0, headerEnd, StandardCharsets.UTF_8);- int bodyStart = headerEnd + 4; // skip \r\n\r\n- byte[] partBody = Arrays.copyOfRange(part, bodyStart, part.length);-- // Trim trailing \r\n from part body- if (partBody.length >= 2- && partBody[partBody.length - 2] == '\r'- && partBody[partBody.length - 1] == '\n') {- partBody = Arrays.copyOf(partBody, partBody.length - 2);- }-- String disposition = extractHeaderValue(headers, "Content-Disposition");- if (disposition == null) {- continue;- }- String fieldName = extractDispositionParam(disposition, "name");- if (fieldName == null) {- continue;- }-- String filename = extractDispositionParam(disposition, "filename");- if (filename != null) {- fileData = partBody;- String partContentType = extractHeaderValue(headers, "Content-Type");- if (partContentType != null) {- fileContentType = partContentType.trim();- }- } else {- fields.put(fieldName, new String(partBody, StandardCharsets.UTF_8));- }- }+ MultipartFormParser.ParsedForm form = MultipartFormParser.parse(body, boundary);+ Map<String, String> fields = form.fields();+ byte[] fileData = form.file().map(MultipartFormParser.FilePart::content).orElse(null);+ String fileContentType = form.file().map(MultipartFormParser.FilePart::contentType).orElse(null);String key = fields.get("key");if (key == null || key.isEmpty()) {@@ -3768,123 +3728,6 @@ public class S3Controller {return fields.get(fieldName);}- private static String extractBoundary(String contentType) {- if (contentType == null) {- return null;- }- for (String part : contentType.split(";")) {- String trimmed = part.trim();- if (trimmed.toLowerCase(Locale.ROOT).startsWith("boundary=")) {- String boundary = trimmed.substring("boundary=".length()).trim();- if (boundary.startsWith("\"") && boundary.endsWith("\"")) {- boundary = boundary.substring(1, boundary.length() - 1);- }- return boundary;- }- }- return null;- }-- private static List<byte[]> splitMultipartParts(byte[] body, byte[] boundary) {- java.util.ArrayList<byte[]> parts = new java.util.ArrayList<>();- int pos = indexOf(body, boundary, 0);- if (pos < 0) {- return parts;- }- // Skip past the first boundary line- pos += boundary.length;- // Skip the CRLF or -- after boundary- if (pos < body.length - 1 && body[pos] == '-' && body[pos + 1] == '-') {- return parts; // closing boundary immediately- }- if (pos < body.length - 1 && body[pos] == '\r' && body[pos + 1] == '\n') {- pos += 2;- }-- while (pos < body.length) {- int nextBoundary = indexOf(body, boundary, pos);- if (nextBoundary < 0) {- break;- }- parts.add(Arrays.copyOfRange(body, pos, nextBoundary));- pos = nextBoundary + boundary.length;- // Check for closing boundary --- if (pos < body.length - 1 && body[pos] == '-' && body[pos + 1] == '-') {- break;- }- // Skip CRLF after boundary- if (pos < body.length - 1 && body[pos] == '\r' && body[pos + 1] == '\n') {- pos += 2;- }- }- return parts;- }-- private static int indexOf(byte[] data, byte[] pattern, int fromIndex) {- outer:- for (int i = fromIndex; i <= data.length - pattern.length; i++) {- for (int j = 0; j < pattern.length; j++) {- if (data[i + j] != pattern[j]) {- continue outer;- }- }- return i;- }- return -1;- }-- private static int indexOfDoubleNewline(byte[] data) {- for (int i = 0; i < data.length - 3; i++) {- if (data[i] == '\r' && data[i + 1] == '\n' && data[i + 2] == '\r' && data[i + 3] == '\n') {- return i;- }- }- return -1;- }-- private static String extractHeaderValue(String headers, String headerName) {- String lowerHeaders = headers.toLowerCase(Locale.ROOT);- String lowerName = headerName.toLowerCase(Locale.ROOT) + ":";- int idx = lowerHeaders.indexOf(lowerName);- if (idx < 0) {- return null;- }- int valueStart = idx + lowerName.length();- int lineEnd = headers.indexOf('\r', valueStart);- if (lineEnd < 0) {- lineEnd = headers.indexOf('\n', valueStart);- }- if (lineEnd < 0) {- lineEnd = headers.length();- }- return headers.substring(valueStart, lineEnd).trim();- }-- private static String extractDispositionParam(String disposition, String paramName) {- String search = paramName + "=";- int idx = disposition.indexOf(search);- if (idx < 0) {- return null;- }- int valueStart = idx + search.length();- if (valueStart >= disposition.length()) {- return null;- }- if (disposition.charAt(valueStart) == '"') {- valueStart++;- int valueEnd = disposition.indexOf('"', valueStart);- if (valueEnd < 0) {- return disposition.substring(valueStart);- }- return disposition.substring(valueStart, valueEnd);- } else {- int valueEnd = disposition.indexOf(';', valueStart);- if (valueEnd < 0) {- valueEnd = disposition.length();- }- return disposition.substring(valueStart, valueEnd).trim();- }- }private static final int MAX_INLINE_TAGS = 10;private static final int MAX_INLINE_TAGGING_HEADER_BYTES = 8 * 1024;diff --git a/src/main/resources/application.yml b/src/main/resources/application.ymlindex 7b6e7f65c5..f2e43b0114 100644--- a/src/main/resources/application.yml+++ b/src/main/resources/application.yml@@ -928,6 +928,7 @@ floci:# maven-url: http://floci-reposilite:8080 # set to skip Reposilite sidecar container managementmaven-image: "dzikoysk/reposilite:3.6.3"npm-image: "verdaccio/verdaccio:6.10.4"+ pypi-image: "pypiserver/pypiserver:v2.4.2"marketplace:enabled: truedms:diff --git a/src/test/java/io/github/hectorvent/floci/core/common/MultipartFormParserTest.java b/src/test/java/io/github/hectorvent/floci/core/common/MultipartFormParserTest.javanew file mode 100644index 0000000000..036810fbc2--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/core/common/MultipartFormParserTest.java@@ -0,0 +1,118 @@+package io.github.hectorvent.floci.core.common;++import org.junit.jupiter.api.Test;++import java.io.ByteArrayOutputStream;+import java.io.IOException;+import java.nio.charset.StandardCharsets;+import java.util.Optional;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertFalse;+import static org.junit.jupiter.api.Assertions.assertTrue;++class MultipartFormParserTest {++ private static final String BOUNDARY = "boundary123";++ @Test+ void extractBoundaryReadsTheParameterRegardlessOfQuoting() {+ assertEquals(Optional.of("abc"), MultipartFormParser.extractBoundary("multipart/form-data; boundary=abc"));+ assertEquals(Optional.of("abc"),+ MultipartFormParser.extractBoundary("multipart/form-data; boundary=\"abc\""));+ assertEquals(Optional.empty(), MultipartFormParser.extractBoundary("application/json"));+ assertEquals(Optional.empty(), MultipartFormParser.extractBoundary(null));+ }++ @Test+ void parseSeparatesTextFieldsFromTheFilePart() {+ byte[] body = multipart(+ field("name", "requests"),+ field("version", "2.34.2"),+ file("content", "requests-2.34.2-py3-none-any.whl", "application/octet-stream",+ "fake wheel bytes"));++ MultipartFormParser.ParsedForm form = MultipartFormParser.parse(body, BOUNDARY);++ assertEquals("requests", form.fields().get("name"));+ assertEquals("2.34.2", form.fields().get("version"));+ assertFalse(form.fields().containsKey("content"));+ assertTrue(form.file().isPresent());+ MultipartFormParser.FilePart file = form.file().get();+ assertEquals("content", file.fieldName());+ assertEquals("requests-2.34.2-py3-none-any.whl", file.filename());+ assertEquals("application/octet-stream", file.contentType());+ assertEquals("fake wheel bytes", new String(file.content(), StandardCharsets.UTF_8));+ }++ @Test+ void parseReturnsAnEmptyFileWhenNoPartCarriesAFilename() {+ byte[] body = multipart(field("key", "value"));++ MultipartFormParser.ParsedForm form = MultipartFormParser.parse(body, BOUNDARY);++ assertEquals("value", form.fields().get("key"));+ assertTrue(form.file().isEmpty());+ }++ @Test+ void parsePreservesBinaryContentThatContainsCrlfBytes() {+ byte[] binary = {0x50, 0x4B, 0x03, 0x04, '\r', '\n', 0x00, (byte) 0xFF};+ byte[] body = multipart(filePart("content", "binary.bin", "application/octet-stream", binary));++ MultipartFormParser.ParsedForm form = MultipartFormParser.parse(body, BOUNDARY);++ assertTrue(form.file().isPresent());+ assertArrayEqualsExactly(binary, form.file().get().content());+ }++ @Test+ void parseOnAnEmptyOrMalformedBodyReturnsNoFieldsAndNoFile() {+ MultipartFormParser.ParsedForm form = MultipartFormParser.parse(new byte[0], BOUNDARY);++ assertTrue(form.fields().isEmpty());+ assertTrue(form.file().isEmpty());+ }++ private static void assertArrayEqualsExactly(byte[] expected, byte[] actual) {+ assertEquals(expected.length, actual.length);+ for (int i = 0; i < expected.length; i++) {+ assertEquals(expected[i], actual[i], "byte at index " + i);+ }+ }++ private static byte[] multipart(byte[]... parts) {+ StringBuilder header = new StringBuilder();+ ByteArrayOutputStream out = new ByteArrayOutputStream();+ try {+ for (byte[] part : parts) {+ out.write(("--" + BOUNDARY + "\r\n").getBytes(StandardCharsets.UTF_8));+ out.write(part);+ out.write("\r\n".getBytes(StandardCharsets.UTF_8));+ }+ out.write(("--" + BOUNDARY + "--\r\n").getBytes(StandardCharsets.UTF_8));+ } catch (IOException e) {+ throw new IllegalStateException(e);+ }+ return out.toByteArray();+ }++ private static byte[] field(String name, String value) {+ String part = "Content-Disposition: form-data; name=\"" + name + "\"\r\n\r\n" + value;+ return part.getBytes(StandardCharsets.UTF_8);+ }++ private static byte[] file(String fieldName, String filename, String contentType, String content) {+ return filePart(fieldName, filename, contentType, content.getBytes(StandardCharsets.UTF_8));+ }++ private static byte[] filePart(String fieldName, String filename, String contentType, byte[] content) {+ String header = "Content-Disposition: form-data; name=\"" + fieldName + "\"; filename=\"" + filename + "\"\r\n"+ + "Content-Type: " + contentType + "\r\n\r\n";+ byte[] headerBytes = header.getBytes(StandardCharsets.UTF_8);+ byte[] result = new byte[headerBytes.length + content.length];+ System.arraycopy(headerBytes, 0, result, 0, headerBytes.length);+ System.arraycopy(content, 0, result, headerBytes.length, content.length);+ return result;+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/core/common/ServiceEnablementIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/core/common/ServiceEnablementIntegrationTest.javaindex 24de93b4ce..19e5df4aa8 100644--- a/src/test/java/io/github/hectorvent/floci/core/common/ServiceEnablementIntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/core/common/ServiceEnablementIntegrationTest.java@@ -170,6 +170,49 @@ class ServiceEnablementIntegrationTest {.body("message", equalTo("Service fis is not enabled."));}+ /**+ * {@code CodeArtifactMavenController} and {@code CodeArtifactPypiController} serve a real+ * package-manager wire protocol, not a CodeArtifact JSON API action, so nothing about a+ * request to either carries a SigV4 signature or an {@code X-Amz-Target} for+ * {@code ServiceEnabledFilter} to resolve a service from the normal ways. Both must still be+ * registered in the {@code codeartifact} descriptor's own resource-class set, matched by JAX-RS+ * routing alone: without that registration a disabled codeartifact service does not stop+ * either from reaching its own token check and responding with a 401 instead of ever saying+ * the service itself is unavailable. Caught after CodeArtifactPypiController needed the exact+ * same registration CodeArtifactMavenController had been missing.+ */+ @Test+ void mavenWireProtocolRequestsAreRejectedWhenServiceDisabled() {+ given()+ .header("Authorization", "Bearer whatever")+ .when()+ .get("/codeartifact/maven/some-domain/some-repo/some/path/1.0/file.jar")+ .then()+ .statusCode(400)+ .contentType("application/json")+ .header("X-Amzn-Errortype", "ServiceNotAvailableException")+ .body("__type", equalTo("ServiceNotAvailableException"))+ .body("message", equalTo("Service codeartifact is not enabled."));+ }++ @Test+ void pypiWireProtocolRequestsAreRejectedWhenServiceDisabled() {+ // Not the bare "/simple/" root: AWS doesn't support that path either (only+ // /simple/<project>/ is), so Floci has no route for it at all, and a request to a+ // genuinely unrouted path 404s before this filter ever runs. "/simple/<project>/" is a+ // real, routed endpoint, so it still demonstrates the service-disabled rejection.+ given()+ .header("Authorization", "Bearer whatever")+ .when()+ .get("/codeartifact/pypi/some-domain/some-repo/simple/some-pkg/")+ .then()+ .statusCode(400)+ .contentType("application/json")+ .header("X-Amzn-Errortype", "ServiceNotAvailableException")+ .body("__type", equalTo("ServiceNotAvailableException"))+ .body("message", equalTo("Service codeartifact is not enabled."));+ }+@Testvoid signedRdsDataExecuteRequestsReturnJsonWhenServiceDisabled() {given()@@ -203,6 +246,7 @@ class ServiceEnablementIntegrationTest {public Map<String, String> getConfigOverrides() {return Map.of("floci.services.acm.enabled", "false",+ "floci.services.codeartifact.enabled", "false","floci.services.dynamodb.enabled", "false","floci.services.ecs.enabled", "false","floci.services.fis.enabled", "false",diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiControllerTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiControllerTest.javanew file mode 100644index 0000000000..866d144a91--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiControllerTest.java@@ -0,0 +1,326 @@+package io.github.hectorvent.floci.services.codeartifact;++import com.sun.net.httpserver.HttpServer;+import io.github.hectorvent.floci.core.common.AwsException;+import io.github.hectorvent.floci.services.codeartifact.CodeArtifactService.AuthorizationTokenScope;+import jakarta.ws.rs.NotAuthorizedException;+import jakarta.ws.rs.NotFoundException;+import jakarta.ws.rs.core.HttpHeaders;+import jakarta.ws.rs.core.Response;+import org.junit.jupiter.api.AfterEach;+import org.junit.jupiter.api.BeforeEach;+import org.junit.jupiter.api.Test;++import java.net.InetSocketAddress;+import java.net.http.HttpClient;+import java.nio.charset.StandardCharsets;+import java.time.Duration;+import java.util.Optional;+import java.util.concurrent.CountDownLatch;+import java.util.concurrent.ExecutorService;+import java.util.concurrent.Executors;+import java.util.concurrent.Future;+import java.util.concurrent.TimeUnit;+import java.util.concurrent.TimeoutException;+import java.util.concurrent.atomic.AtomicBoolean;+import java.util.concurrent.atomic.AtomicInteger;+import java.util.concurrent.atomic.AtomicReference;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertThrows;+import static org.junit.jupiter.api.Assertions.assertTrue;+import static org.mockito.ArgumentMatchers.any;+import static org.mockito.ArgumentMatchers.anyString;+import static org.mockito.ArgumentMatchers.eq;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.when;++/**+ * Exercises {@link CodeArtifactPypiController} against a fake pypiserver backend (a real+ * {@link HttpServer}, not a mock of the JDK {@link HttpClient}, since that client's request/response+ * types are effectively unmockable) so the proxy-passthrough, auth-resolution, and pre-upload+ * overwrite-rejection logic all run for real. {@link CodeArtifactService} and+ * {@link PypiserverSidecarManager} are mocked: what they do is already covered by+ * {@code CodeArtifactServiceTest} and {@code PypiserverSidecarManagerTest}, what matters here is how+ * this controller uses their results.+ */+class CodeArtifactPypiControllerTest {++ private static final String DOMAIN = "dom";+ private static final String REPOSITORY = "repo";+ private static final String REGION = "us-east-1";+ private static final String OWNER = "000000000000";++ private HttpServer backend;+ private String backendUrl;+ private CodeArtifactService service;+ private PypiserverSidecarManager pypiserver;+ private CodeArtifactPypiController controller;++ @BeforeEach+ void setUp() throws Exception {+ backend = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);+ backend.start();+ backendUrl = "http://127.0.0.1:" + backend.getAddress().getPort();++ service = mock(CodeArtifactService.class);+ pypiserver = mock(PypiserverSidecarManager.class);+ when(service.resolveAuthorizationToken(anyString(), eq(DOMAIN)))+ .thenReturn(Optional.of(new AuthorizationTokenScope(OWNER, REGION)));+ when(service.ensureFormatContainerId(eq("pypi"), eq(REGION), eq(DOMAIN), eq(OWNER), eq(REPOSITORY)))+ .thenReturn("pypi-repo-id");+ when(pypiserver.ensureReady(eq("pypi-repo-id"), any())).thenReturn(backendUrl);++ HttpClient httpClient = HttpClient.newBuilder().connectTimeout(Duration.ofSeconds(5)).build();+ controller = new CodeArtifactPypiController(service, pypiserver, httpClient);+ }++ @AfterEach+ void tearDown() {+ backend.stop(0);+ }++ @Test+ void requireBaseUrlRejectsAMissingOrUnresolvableToken() {+ when(service.resolveAuthorizationToken(anyString(), eq(DOMAIN))).thenReturn(Optional.empty());++ assertThrows(NotAuthorizedException.class,+ () -> controller.packageIndex(headers("Bearer bogus"), DOMAIN, REPOSITORY, "my-pkg"));+ }++ @Test+ void requireBaseUrlTranslatesAMissingRepositoryIntoNotFound() {+ when(service.ensureFormatContainerId(eq("pypi"), eq(REGION), eq(DOMAIN), eq(OWNER), eq(REPOSITORY)))+ .thenThrow(new AwsException("ResourceNotFoundException", "no such repository", 404));++ assertThrows(NotFoundException.class,+ () -> controller.packageIndex(headers("Bearer good"), DOMAIN, REPOSITORY, "my-pkg"));+ }++ @Test+ void packageIndexRewritesRootRelativePackageLinksToThisRepositorysPathPrefix() {+ // Real pypiserver response shape, confirmed against a live container: a root-relative+ // href, not the bare filename a naive fixture might use.+ backend.createContext("/simple/my-pkg/", exchange -> respond(exchange, 200, "text/html",+ "<a href=\"/packages/my-pkg-1.0.tar.gz#sha256=abc\">my-pkg-1.0.tar.gz</a>"));++ Response response = controller.packageIndex(headers("Bearer good"), DOMAIN, REPOSITORY, "my-pkg");++ assertEquals(200, response.getStatus());+ String body = new String((byte[]) response.getEntity(), StandardCharsets.UTF_8);+ assertTrue(body.contains("href=\"/codeartifact/pypi/" + DOMAIN + "/" + REPOSITORY+ + "/packages/my-pkg-1.0.tar.gz#sha256=abc\""), body);+ }++ @Test+ void downloadFileProxiesRawBytesFromTheBackend() {+ byte[] content = "package bytes".getBytes(StandardCharsets.UTF_8);+ backend.createContext("/packages/my-pkg-1.0.tar.gz",+ exchange -> respond(exchange, 200, "application/octet-stream", content));++ Response response = controller.downloadFile(headers("Bearer good"), DOMAIN, REPOSITORY, "my-pkg-1.0.tar.gz");++ assertEquals(200, response.getStatus());+ assertEquals("package bytes", new String((byte[]) response.getEntity(), StandardCharsets.UTF_8));+ }++ @Test+ void downloadFileReturns404WhenTheBackendHasNoSuchFile() {+ backend.createContext("/packages/missing.tar.gz", exchange -> respond(exchange, 404, "text/plain", ""));++ Response response = controller.downloadFile(headers("Bearer good"), DOMAIN, REPOSITORY, "missing.tar.gz");++ assertEquals(404, response.getStatus());+ }++ @Test+ void uploadRejectsABodyWithNoDeterminableBoundary() {+ Response response = controller.upload(headers("Bearer good", "application/octet-stream"), DOMAIN, REPOSITORY,+ "not multipart".getBytes(StandardCharsets.UTF_8));++ assertEquals(400, response.getStatus());+ }++ @Test+ void uploadRejectsAMultipartBodyMissingNameOrContent() {+ byte[] body = multipart("boundary123", "version", "1.0.0");++ Response response = controller.upload(+ headers("Bearer good", "multipart/form-data; boundary=boundary123"), DOMAIN, REPOSITORY, body);++ assertEquals(400, response.getStatus());+ }++ @Test+ void uploadRejectsAFileAlreadyListedInTheBackendsSimpleIndex() {+ backend.createContext("/simple/my-pkg/",+ exchange -> respond(exchange, 200, "text/html", "<a href=\"my_pkg-1.0.tar.gz\">my_pkg-1.0.tar.gz</a>"));+ AtomicReference<Boolean> uploadReceived = new AtomicReference<>(false);+ backend.createContext("/", exchange -> {+ if ("POST".equals(exchange.getRequestMethod())) {+ uploadReceived.set(true);+ }+ respond(exchange, 200, "text/plain", "");+ });+ byte[] body = multipartWithFile("boundary123", "my-pkg", "my_pkg-1.0.tar.gz", "new content");++ Response response = controller.upload(+ headers("Bearer good", "multipart/form-data; boundary=boundary123"), DOMAIN, REPOSITORY, body);++ assertEquals(409, response.getStatus());+ assertEquals(false, uploadReceived.get());+ }++ @Test+ void uploadOfByteIdenticalContentSucceedsIdempotentlyWithoutForwardingToTheBackend() {+ String sameContent = "same content";+ backend.createContext("/simple/my-pkg/",+ exchange -> respond(exchange, 200, "text/html", "<a href=\"my_pkg-1.0.tar.gz\">my_pkg-1.0.tar.gz</a>"));+ backend.createContext("/packages/my_pkg-1.0.tar.gz",+ exchange -> respond(exchange, 200, "application/octet-stream", sameContent));+ AtomicReference<Boolean> uploadReceived = new AtomicReference<>(false);+ backend.createContext("/", exchange -> {+ if ("POST".equals(exchange.getRequestMethod())) {+ uploadReceived.set(true);+ }+ respond(exchange, 200, "text/plain", "");+ });+ byte[] body = multipartWithFile("boundary123", "my-pkg", "my_pkg-1.0.tar.gz", sameContent);++ Response response = controller.upload(+ headers("Bearer good", "multipart/form-data; boundary=boundary123"), DOMAIN, REPOSITORY, body);++ assertEquals(200, response.getStatus());+ assertEquals(false, uploadReceived.get());+ }++ @Test+ void uploadForwardsANewFileToTheBackendWhenNothingConflicts() {+ backend.createContext("/simple/my-pkg/", exchange -> respond(exchange, 404, "text/plain", ""));+ AtomicReference<String> receivedBody = new AtomicReference<>();+ backend.createContext("/", exchange -> {+ if ("POST".equals(exchange.getRequestMethod())) {+ receivedBody.set(new String(exchange.getRequestBody().readAllBytes(), StandardCharsets.UTF_8));+ }+ respond(exchange, 200, "text/plain", "");+ });+ byte[] body = multipartWithFile("boundary123", "my-pkg", "my_pkg-1.0.tar.gz", "new content");++ Response response = controller.upload(+ headers("Bearer good", "multipart/form-data; boundary=boundary123"), DOMAIN, REPOSITORY, body);++ assertEquals(200, response.getStatus());+ assertTrue(receivedBody.get().contains("new content"));+ }++ @Test+ void uploadFailsClosedWhenTheExistenceCheckCannotReachTheBackend() {+ // Nothing listens on this port: the existence check's own connect attempt fails, which+ // must reject the upload rather than silently treating the failure as "not published yet."+ when(pypiserver.ensureReady(eq("pypi-repo-id"), any())).thenReturn("http://127.0.0.1:1");+ byte[] body = multipartWithFile("boundary123", "my-pkg", "my_pkg-1.0.tar.gz", "content");++ assertThrows(IllegalStateException.class, () -> controller.upload(+ headers("Bearer good", "multipart/form-data; boundary=boundary123"), DOMAIN, REPOSITORY, body));+ }++ @Test+ void concurrentUploadsOfDifferentContentForTheSameNewFilenameAreSerialized() throws Exception {+ CountDownLatch holdFirstCheck = new CountDownLatch(1);+ CountDownLatch firstCheckEntered = new CountDownLatch(1);+ AtomicInteger indexHits = new AtomicInteger(0);+ AtomicBoolean published = new AtomicBoolean(false);++ backend.createContext("/simple/my-pkg/", exchange -> {+ if (indexHits.incrementAndGet() == 1) {+ firstCheckEntered.countDown();+ try {+ assertTrue(holdFirstCheck.await(5, TimeUnit.SECONDS), "test never released the first check");+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ }+ }+ if (published.get()) {+ respond(exchange, 200, "text/html", "<a href=\"/packages/my_pkg-1.0.tar.gz\">my_pkg-1.0.tar.gz</a>");+ } else {+ respond(exchange, 404, "text/plain", "");+ }+ });+ backend.createContext("/packages/my_pkg-1.0.tar.gz",+ exchange -> respond(exchange, 200, "application/octet-stream", "content-A"));+ backend.createContext("/", exchange -> {+ if ("POST".equals(exchange.getRequestMethod())) {+ published.set(true);+ }+ respond(exchange, 200, "text/plain", "");+ });++ ExecutorService pool = Executors.newFixedThreadPool(2);+ try {+ Future<Response> first = pool.submit(() -> controller.upload(+ headers("Bearer good", "multipart/form-data; boundary=boundary123"), DOMAIN, REPOSITORY,+ multipartWithFile("boundary123", "my-pkg", "my_pkg-1.0.tar.gz", "content-A")));+ assertTrue(firstCheckEntered.await(5, TimeUnit.SECONDS), "first upload never started its check");++ Future<Response> second = pool.submit(() -> controller.upload(+ headers("Bearer good", "multipart/form-data; boundary=boundary123"), DOMAIN, REPOSITORY,+ multipartWithFile("boundary123", "my-pkg", "my_pkg-1.0.tar.gz", "content-B")));++ // The second upload must block on the same repository's lock, held by the first, and+ // so cannot even reach the backend yet: without the lock this would race ahead and+ // both uploads could observe "not published yet" before either one's POST lands.+ assertThrows(TimeoutException.class, () -> second.get(500, TimeUnit.MILLISECONDS));+ assertEquals(1, indexHits.get(), "second upload must not have reached the backend yet");++ holdFirstCheck.countDown();+ assertEquals(200, first.get(5, TimeUnit.SECONDS).getStatus());+ assertEquals(409, second.get(5, TimeUnit.SECONDS).getStatus());+ } finally {+ pool.shutdownNow();+ }+ }++ private static HttpHeaders headers(String authorization) {+ return headers(authorization, "text/plain");+ }++ private static HttpHeaders headers(String authorization, String contentType) {+ HttpHeaders headers = mock(HttpHeaders.class);+ when(headers.getHeaderString("Authorization")).thenReturn(authorization);+ when(headers.getHeaderString(HttpHeaders.CONTENT_TYPE)).thenReturn(contentType);+ return headers;+ }++ private static byte[] multipart(String boundary, String fieldName, String value) {+ String body = "--" + boundary + "\r\n"+ + "Content-Disposition: form-data; name=\"" + fieldName + "\"\r\n\r\n"+ + value + "\r\n"+ + "--" + boundary + "--\r\n";+ return body.getBytes(StandardCharsets.UTF_8);+ }++ private static byte[] multipartWithFile(String boundary, String packageName, String filename, String content) {+ String body = "--" + boundary + "\r\n"+ + "Content-Disposition: form-data; name=\"name\"\r\n\r\n"+ + packageName + "\r\n"+ + "--" + boundary + "\r\n"+ + "Content-Disposition: form-data; name=\"content\"; filename=\"" + filename + "\"\r\n"+ + "Content-Type: application/octet-stream\r\n\r\n"+ + content + "\r\n"+ + "--" + boundary + "--\r\n";+ return body.getBytes(StandardCharsets.UTF_8);+ }++ private static void respond(com.sun.net.httpserver.HttpExchange exchange, int status, String contentType,+ String body) throws java.io.IOException {+ respond(exchange, status, contentType, body.getBytes(StandardCharsets.UTF_8));+ }++ private static void respond(com.sun.net.httpserver.HttpExchange exchange, int status, String contentType,+ byte[] body) throws java.io.IOException {+ exchange.getResponseHeaders().add("Content-Type", contentType);+ exchange.sendResponseHeaders(status, body.length);+ exchange.getResponseBody().write(body);+ exchange.close();+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.javanew file mode 100644index 0000000000..adc1925ee5--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.java@@ -0,0 +1,402 @@+package io.github.hectorvent.floci.services.codeartifact;++import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;+import io.quarkus.test.junit.QuarkusTest;+import io.quarkus.test.junit.TestProfile;+import org.junit.jupiter.api.BeforeAll;+import org.junit.jupiter.api.MethodOrderer;+import org.junit.jupiter.api.Order;+import org.junit.jupiter.api.Test;+import org.junit.jupiter.api.TestMethodOrder;++import java.io.IOException;+import java.nio.charset.StandardCharsets;+import java.util.Base64;+import java.util.concurrent.CountDownLatch;+import java.util.concurrent.ExecutorService;+import java.util.concurrent.Executors;+import java.util.concurrent.TimeUnit;+import java.util.concurrent.atomic.AtomicInteger;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.containsString;+import static org.hamcrest.Matchers.startsWith;+import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertTrue;++/**+ * Runs a real PyPI-shaped upload/download round trip against a live, per-repository pypiserver+ * container, started by {@link PypiserverSidecarManager} exactly as in production. The upload body+ * is a hand-built multipart form matching the fields a real {@code twine upload} sends (verified+ * against a live pypiserver instance and real twine by hand during design), not a captured fixture:+ * unlike npm's envelope, the parsing itself ({@link io.github.hectorvent.floci.core.common.MultipartFormParser})+ * is already covered by its own unit tests and by {@code CodeArtifactPypiControllerTest}, so what+ * this class needs to prove is the real container's behavior, not the parser's fidelity to one+ * exact client's byte-for-byte output.+ */+@QuarkusTest+@TestProfile(CodeArtifactPypiSidecarProfile.class)+@TestMethodOrder(MethodOrderer.OrderAnnotation.class)+class CodeArtifactPypiDockerIntegrationTest {++ private static final String AUTH = "AWS4-HMAC-SHA256 Credential=AKID/20260904/us-east-1/codeartifact/aws4_request";+ private static final String DOMAIN = "pypi-sidecar-domain";+ private static final String REPO = "pypi-sidecar-repo";+ private static final String PACKAGE_NAME = "floci-pypiserver-spike";+ // Real sdist filenames use underscores where the package name uses hyphens; exercising this+ // exact mismatch is what proves the PEP 503 normalization in the overwrite-check actually runs,+ // not just a name that happens to already match its own normalized form.+ private static final String FILENAME = "floci_pypiserver_spike-1.0.0.tar.gz";++ private static String bearerToken;++ @BeforeAll+ static void setUp() {+ CodeArtifactPypiSidecarProfile.requireDockerAndTheSidecarImage();+ RestAssuredJsonUtils.configureAwsContentTypes();+ }++ @Test+ @Order(0)+ void createDomainRepositoryAndAuthorizationToken() {+ given().contentType("application/json").header("Authorization", AUTH).body("{}")+ .post("/v1/domain?domain=" + DOMAIN)+ .then().statusCode(200);+ given().contentType("application/json").header("Authorization", AUTH).body("{}")+ .post("/v1/repository?domain=" + DOMAIN + "&repository=" + REPO)+ .then().statusCode(200);++ bearerToken = given().header("Authorization", AUTH)+ .post("/v1/authorization-token?domain=" + DOMAIN)+ .then().statusCode(200)+ .extract().jsonPath().getString("authorizationToken");+ }++ @Test+ @Order(1)+ void uploadThenDownloadRoundTripsTheExactBytesThroughTheSimpleIndex() {+ byte[] content = "real-sdist-bytes".getBytes(StandardCharsets.UTF_8);++ given().header("Authorization", "Bearer " + bearerToken)+ .contentType("multipart/form-data; boundary=boundary123")+ .body(twineUploadBody("boundary123", PACKAGE_NAME, FILENAME, content))+ .post("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/")+ .then().statusCode(200);++ String indexHtml = given().header("Authorization", "Bearer " + bearerToken)+ .get("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/simple/" + PACKAGE_NAME + "/")+ .then().statusCode(200)+ .extract().asString();+ assertTrue(indexHtml.contains(FILENAME), "simple index must list the uploaded file: " + indexHtml);++ byte[] fetched = given().header("Authorization", "Bearer " + bearerToken)+ .get("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/packages/" + FILENAME)+ .then().statusCode(200)+ .extract().asByteArray();+ assertEquals(new String(content, StandardCharsets.UTF_8), new String(fetched, StandardCharsets.UTF_8));+ }++ /**+ * AWS's own CodeArtifact docs ("Overwriting package assets" in packages-overview.html) say a+ * republish of an asset whose content is byte-identical to what is already published succeeds+ * because the operation is idempotent; only different content gets the 409 the next test+ * covers. A naive "is this filename already listed" check would reject both cases alike, which+ * would make twine's own retry-after-a-dropped-response behavior fail spuriously.+ */+ @Test+ @Order(2)+ void reuploadingTheExactSameFileSucceedsIdempotently() {+ given().header("Authorization", "Bearer " + bearerToken)+ .contentType("multipart/form-data; boundary=boundary123")+ .body(twineUploadBody("boundary123", PACKAGE_NAME, FILENAME, "real-sdist-bytes".getBytes(StandardCharsets.UTF_8)))+ .post("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/")+ .then().statusCode(200);++ byte[] fetched = given().header("Authorization", "Bearer " + bearerToken)+ .get("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/packages/" + FILENAME)+ .then().statusCode(200)+ .extract().asByteArray();+ assertEquals("real-sdist-bytes", new String(fetched, StandardCharsets.UTF_8));+ }++ @Test+ @Order(3)+ void reuploadingTheSameFileIsRejectedAndTheOriginalContentSurvives() {+ byte[] differentContent = "different-sdist-bytes".getBytes(StandardCharsets.UTF_8);++ given().header("Authorization", "Bearer " + bearerToken)+ .contentType("multipart/form-data; boundary=boundary123")+ .body(twineUploadBody("boundary123", PACKAGE_NAME, FILENAME, differentContent))+ .post("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/")+ .then().statusCode(409);++ byte[] fetched = given().header("Authorization", "Bearer " + bearerToken)+ .get("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/packages/" + FILENAME)+ .then().statusCode(200)+ .extract().asByteArray();+ assertEquals("real-sdist-bytes", new String(fetched, StandardCharsets.UTF_8));+ }++ @Test+ @Order(4)+ void missingPackageAndMissingRepositoryAreNotFound() {+ given().header("Authorization", "Bearer " + bearerToken)+ .get("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/simple/does-not-exist/")+ .then().statusCode(404);++ given().header("Authorization", "Bearer " + bearerToken)+ .get("/codeartifact/pypi/" + DOMAIN + "/no-such-repo/simple/does-not-exist/")+ .then().statusCode(404);+ }++ @Test+ @Order(5)+ void repositoriesAreIsolatedFromEachOther() {+ given().contentType("application/json").header("Authorization", AUTH).body("{}")+ .post("/v1/repository?domain=" + DOMAIN + "&repository=other-repo")+ .then().statusCode(200);++ given().header("Authorization", "Bearer " + bearerToken)+ .get("/codeartifact/pypi/" + DOMAIN + "/other-repo/simple/" + PACKAGE_NAME + "/")+ .then().statusCode(404);+ }++ @Test+ @Order(6)+ void missingOrWrongDomainTokensAreUnauthorized() {+ // /simple/<package>/, not the bare /simple/ root: AWS doesn't support that path either,+ // so there's no route for it at all, and an unrouted path 404s before auth ever runs.+ // Challenges as Basic, not Bearer: real pip and twine both authenticate with HTTP Basic+ // (username=aws, password=<token>), so this is the scheme a retry would actually honor.+ given().get("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/simple/" + PACKAGE_NAME + "/")+ .then().statusCode(401)+ .header("WWW-Authenticate", startsWith("Basic"));++ given().header("Authorization", "Bearer not-a-real-token")+ .get("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/simple/" + PACKAGE_NAME + "/")+ .then().statusCode(401);++ given().contentType("application/json").header("Authorization", AUTH).body("{}")+ .post("/v1/domain?domain=other-pypi-token-domain")+ .then().statusCode(200);+ String otherDomainToken = given().header("Authorization", AUTH)+ .post("/v1/authorization-token?domain=other-pypi-token-domain")+ .then().statusCode(200)+ .extract().jsonPath().getString("authorizationToken");++ given().header("Authorization", "Bearer " + otherDomainToken)+ .get("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/simple/" + PACKAGE_NAME + "/")+ .then().statusCode(401);+ }++ @Test+ @Order(7)+ void aRealPipClientsBasicAuthCredentialsAreAcceptedWithTheTokenAsThePassword() {+ String basic = "Basic " + Base64.getEncoder().encodeToString(("aws:" + bearerToken).getBytes(StandardCharsets.UTF_8));++ given().header("Authorization", basic)+ .get("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/simple/" + PACKAGE_NAME + "/")+ .then().statusCode(200)+ .body(containsString(FILENAME));+ }++ @Test+ @Order(8)+ void concurrentFirstUseOfANewRepositoryOnlyStartsOneContainer() throws InterruptedException {+ given().contentType("application/json").header("Authorization", AUTH).body("{}")+ .post("/v1/repository?domain=" + DOMAIN + "&repository=concurrent-repo")+ .then().statusCode(200);++ int attempts = 8;+ ExecutorService pool = Executors.newFixedThreadPool(attempts);+ CountDownLatch ready = new CountDownLatch(attempts);+ CountDownLatch go = new CountDownLatch(1);+ AtomicInteger notFoundResponses = new AtomicInteger();+ try {+ for (int i = 0; i < attempts; i++) {+ pool.submit(() -> {+ ready.countDown();+ try {+ go.await();+ given().header("Authorization", "Bearer " + bearerToken)+ .get("/codeartifact/pypi/" + DOMAIN + "/concurrent-repo/simple/does-not-exist/")+ .then().statusCode(404);+ notFoundResponses.incrementAndGet();+ } catch (InterruptedException ignored) {+ Thread.currentThread().interrupt();+ }+ });+ }+ ready.await(5, TimeUnit.SECONDS);+ go.countDown();+ } finally {+ pool.shutdown();+ assertTrue(pool.awaitTermination(30, TimeUnit.SECONDS));+ }+ assertEquals(attempts, notFoundResponses.get());+ }++ @Test+ @Order(9)+ void recreatingASameNamedRepositoryDoesNotInheritThePreviousOnesPackages() {+ given().contentType("application/json").header("Authorization", AUTH).body("{}")+ .post("/v1/repository?domain=" + DOMAIN + "&repository=reused-name")+ .then().statusCode(200);+ given().header("Authorization", "Bearer " + bearerToken)+ .contentType("multipart/form-data; boundary=boundary123")+ .body(twineUploadBody("boundary123", PACKAGE_NAME, FILENAME, "first-generation".getBytes(StandardCharsets.UTF_8)))+ .post("/codeartifact/pypi/" + DOMAIN + "/reused-name/")+ .then().statusCode(200);++ given().header("Authorization", AUTH)+ .delete("/v1/repository?domain=" + DOMAIN + "&repository=reused-name")+ .then().statusCode(200);+ given().contentType("application/json").header("Authorization", AUTH).body("{}")+ .post("/v1/repository?domain=" + DOMAIN + "&repository=reused-name")+ .then().statusCode(200);++ given().header("Authorization", "Bearer " + bearerToken)+ .get("/codeartifact/pypi/" + DOMAIN + "/reused-name/simple/" + PACKAGE_NAME + "/")+ .then().statusCode(404);+ }++ @Test+ @Order(10)+ void aDomainCreatedInANonDefaultRegionIsServedThroughTheTokensOwnRegion() {+ String nonDefaultRegionAuth =+ "AWS4-HMAC-SHA256 Credential=AKID/20260904/us-west-2/codeartifact/aws4_request";+ String domain = "pypi-sidecar-non-default-region-domain";++ given().contentType("application/json").header("Authorization", nonDefaultRegionAuth).body("{}")+ .post("/v1/domain?domain=" + domain)+ .then().statusCode(200);+ given().contentType("application/json").header("Authorization", nonDefaultRegionAuth).body("{}")+ .post("/v1/repository?domain=" + domain + "&repository=" + REPO)+ .then().statusCode(200);+ String token = given().header("Authorization", nonDefaultRegionAuth)+ .post("/v1/authorization-token?domain=" + domain)+ .then().statusCode(200)+ .extract().jsonPath().getString("authorizationToken");++ given().header("Authorization", "Bearer " + token)+ .contentType("multipart/form-data; boundary=boundary123")+ .body(twineUploadBody("boundary123", PACKAGE_NAME, FILENAME, "non-default-region-bytes".getBytes(StandardCharsets.UTF_8)))+ .post("/codeartifact/pypi/" + domain + "/" + REPO + "/")+ .then().statusCode(200);++ given().header("Authorization", "Bearer " + token)+ .get("/codeartifact/pypi/" + domain + "/" + REPO + "/simple/" + PACKAGE_NAME + "/")+ .then().statusCode(200);+ }++ @Test+ @Order(11)+ void aDomainCreatedUnderANonDefaultAccountIsServedThroughTheTokensOwnAccount() {+ String otherAccountAuth = "AWS4-HMAC-SHA256 Credential=111122223333/20260904/us-east-1/codeartifact/aws4_request";+ String domain = "pypi-sidecar-cross-account-domain";++ given().contentType("application/json").header("Authorization", otherAccountAuth).body("{}")+ .post("/v1/domain?domain=" + domain)+ .then().statusCode(200);+ given().contentType("application/json").header("Authorization", otherAccountAuth).body("{}")+ .post("/v1/repository?domain=" + domain + "&repository=" + REPO)+ .then().statusCode(200);+ String token = given().header("Authorization", otherAccountAuth)+ .post("/v1/authorization-token?domain=" + domain)+ .then().statusCode(200)+ .extract().jsonPath().getString("authorizationToken");++ given().header("Authorization", "Bearer " + token)+ .contentType("multipart/form-data; boundary=boundary123")+ .body(twineUploadBody("boundary123", PACKAGE_NAME, FILENAME, "cross-account-bytes".getBytes(StandardCharsets.UTF_8)))+ .post("/codeartifact/pypi/" + domain + "/" + REPO + "/")+ .then().statusCode(200);++ given().header("Authorization", "Bearer " + token)+ .get("/codeartifact/pypi/" + domain + "/" + REPO + "/simple/" + PACKAGE_NAME + "/")+ .then().statusCode(200);+ }++ /**+ * Proves the actual mechanism, not just that {@code PypiserverSidecarManager} has a method+ * named right: {@code ContainerTeardowns.stopAll} runs every {@code ContainerTeardown} on+ * {@code /state/reset}, and this is what stops a live container.+ *+ * <p>Ordered before every other test, for the same reason {@code CodeArtifactNpmDockerIntegrationTest}'s+ * equivalent reset test runs first: a reset stops every container this process's pool is+ * tracking, not just the one this test starts, so running it after the other tests would sweep+ * up their containers too and break the baseline comparison for a reason unrelated to whether+ * this test's own container actually stopped.+ */+ @Test+ @Order(-1)+ void stateResetStopsTheRepositorysPypiserverContainer() throws Exception {+ int baseline = runningPypiserverTestContainerCount();++ String domain = "pypi-sidecar-reset-domain";+ given().contentType("application/json").header("Authorization", AUTH).body("{}")+ .post("/v1/domain?domain=" + domain)+ .then().statusCode(200);+ given().contentType("application/json").header("Authorization", AUTH).body("{}")+ .post("/v1/repository?domain=" + domain + "&repository=reset-repo")+ .then().statusCode(200);+ String token = given().header("Authorization", AUTH)+ .post("/v1/authorization-token?domain=" + domain)+ .then().statusCode(200)+ .extract().jsonPath().getString("authorizationToken");+ // A 404 read still starts the container; only a missing token skips it.+ given().header("Authorization", "Bearer " + token)+ .get("/codeartifact/pypi/" + domain + "/reset-repo/simple/does-not-exist/")+ .then().statusCode(404);+ assertEquals(baseline + 1, runningPypiserverTestContainerCount(),+ "expected reset-repo's pypiserver container to be running before reset");++ given().post("/_floci/state/reset").then().statusCode(200);++ assertEquals(baseline, runningPypiserverTestContainerCount(),+ "state reset must stop reset-repo's pypiserver container");+ }++ private static int runningPypiserverTestContainerCount() throws IOException, InterruptedException {+ Process process = new ProcessBuilder("docker", "ps", "-q",+ "--filter", "name=floci-aws-codeartifact-pypi-test-pypiserver-")+ .redirectErrorStream(true).start();+ String output = new String(process.getInputStream().readAllBytes(), StandardCharsets.UTF_8);+ process.waitFor();+ return output.isBlank() ? 0 : (int) output.lines().count();+ }++ /**+ * The fields a real {@code twine upload} sends (verified by hand against a live pypiserver+ * instance during design); pypiserver, and this controller's own parsing, only look at+ * {@code name} and the {@code content} file part, so the rest are included for realism only.+ */+ private static byte[] twineUploadBody(String boundary, String packageName, String filename, byte[] content) {+ String preamble = "--" + boundary + "\r\n"+ + "Content-Disposition: form-data; name=\":action\"\r\n\r\n"+ + "file_upload\r\n"+ + "--" + boundary + "\r\n"+ + "Content-Disposition: form-data; name=\"protocol_version\"\r\n\r\n"+ + "1\r\n"+ + "--" + boundary + "\r\n"+ + "Content-Disposition: form-data; name=\"name\"\r\n\r\n"+ + packageName + "\r\n"+ + "--" + boundary + "\r\n"+ + "Content-Disposition: form-data; name=\"version\"\r\n\r\n"+ + "1.0.0\r\n"+ + "--" + boundary + "\r\n"+ + "Content-Disposition: form-data; name=\"filetype\"\r\n\r\n"+ + "sdist\r\n"+ + "--" + boundary + "\r\n"+ + "Content-Disposition: form-data; name=\"content\"; filename=\"" + filename + "\"\r\n"+ + "Content-Type: application/octet-stream\r\n\r\n";+ String closing = "\r\n--" + boundary + "--\r\n";+ byte[] preambleBytes = preamble.getBytes(StandardCharsets.UTF_8);+ byte[] closingBytes = closing.getBytes(StandardCharsets.UTF_8);+ byte[] body = new byte[preambleBytes.length + content.length + closingBytes.length];+ System.arraycopy(preambleBytes, 0, body, 0, preambleBytes.length);+ System.arraycopy(content, 0, body, preambleBytes.length, content.length);+ System.arraycopy(closingBytes, 0, body, preambleBytes.length + content.length, closingBytes.length);+ return body;+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiSidecarProfile.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiSidecarProfile.javanew file mode 100644index 0000000000..d247a98e41--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiSidecarProfile.java@@ -0,0 +1,52 @@+package io.github.hectorvent.floci.services.codeartifact;++import io.quarkus.test.junit.QuarkusTestProfile;+import org.eclipse.microprofile.config.ConfigProvider;+import org.junit.jupiter.api.Assumptions;++import java.util.Map;++/**+ * Shared by every CodeArtifact integration test that needs a real pypiserver sidecar container,+ * started by {@link PypiserverSidecarManager} exactly as in production. Namespaces containers as+ * {@code floci-codeartifact-pypi-test-*} so the manager's remove-before-start never touches a+ * developer's real running Floci. Needs Docker; skipped without it. Locally it is also skipped+ * when the pinned image is not present, so a developer without registry access is not stuck; in+ * CI a missing image is a failure, since a vanished tag must not pass silently.+ */+public class CodeArtifactPypiSidecarProfile implements QuarkusTestProfile {++ @Override+ public Map<String, String> getConfigOverrides() {+ return Map.of("floci.docker.resource-namespace", "codeartifact-pypi-test");+ }++ public static void requireDockerAndTheSidecarImage() {+ Assumptions.assumeTrue(isDockerAvailable(), "Docker daemon must be available for CodeArtifact pypi "+ + "sidecar integration tests");+ String image = ConfigProvider.getConfig().getValue("floci.services.codeartifact.pypi-image", String.class);+ Assumptions.assumeTrue(imageUsable(image), "pypiserver sidecar image " + image + " is not present locally");+ }++ private static boolean isDockerAvailable() {+ return run("docker", "version", "--format", "{{.Server.Version}}");+ }++ private static boolean imageUsable(String image) {+ if ("true".equals(System.getenv("CI"))) {+ return true;+ }+ return run("docker", "image", "inspect", image);+ }++ private static boolean run(String... command) {+ try {+ Process process = new ProcessBuilder(command).redirectErrorStream(true).start();+ return process.waitFor() == 0;+ } catch (Exception ignored) {+ // Docker (or the image) isn't available; the caller turns this into a skipped test,+ // not a failure, so there is nothing more useful to log here.+ return false;+ }+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.javaindex 1270056b76..9cede4acc8 100644--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.java@@ -300,7 +300,7 @@ class CodeArtifactServiceTest {service.createDomain(REGION, "dom", null, Map.of());CodeArtifactRepository created = service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());- assertEquals(Set.of("maven", "npm"), created.getSidecarContainerIds().keySet());+ assertEquals(Set.of("maven", "npm", "pypi"), created.getSidecarContainerIds().keySet());created.getSidecarContainerIds().values().forEach(id -> assertTrue(id != null && !id.isBlank()));}@@ -313,7 +313,7 @@ class CodeArtifactServiceTest {service.deleteRepository(REGION, "dom", null, "repo");CodeArtifactRepository recreated = service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());- for (String format : Set.of("maven", "npm")) {+ for (String format : Set.of("maven", "npm", "pypi")) {assertTrue(!first.getSidecarContainerIds().get(format).equals(recreated.getSidecarContainerIds().get(format)),"a recreated repository must never reuse the previous one's container id for a format, "@@ -506,6 +506,9 @@ class CodeArtifactServiceTest {String endpoint = service.getRepositoryEndpoint(REGION, "dom", null, "repo", "npm", null);assertEquals("http://localhost:4566/codeartifact/npm/dom/repo/", endpoint);+ String pypiEndpoint = service.getRepositoryEndpoint(REGION, "dom", null, "repo", "pypi", null);+ assertEquals("http://localhost:4566/codeartifact/pypi/dom/repo/", pypiEndpoint);+AwsException e = assertThrows(AwsException.class,() -> service.getRepositoryEndpoint(REGION, "dom", null, "repo", "not-a-format", null));assertEquals("ValidationException", e.getErrorCode());diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManagerTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManagerTest.javanew file mode 100644index 0000000000..24406cf5c8--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManagerTest.java@@ -0,0 +1,73 @@+package io.github.hectorvent.floci.services.codeartifact;++import io.github.hectorvent.floci.config.EmulatorConfig;+import io.github.hectorvent.floci.core.common.ContainerTeardown;+import io.github.hectorvent.floci.core.common.docker.ContainerBuilder;+import io.github.hectorvent.floci.core.common.docker.ContainerLifecycleManager;+import io.github.hectorvent.floci.core.common.docker.PerKeyContainerPool;+import org.junit.jupiter.api.Test;++import java.lang.reflect.Field;+import java.util.concurrent.ConcurrentHashMap;++import static org.junit.jupiter.api.Assertions.assertInstanceOf;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.verify;+import static org.mockito.Mockito.verifyNoInteractions;++/**+ * Same reasoning as {@code VerdaccioSidecarManagerTest}: {@link PypiserverSidecarManager}'s own+ * {@link PerKeyContainerPool#stopAll()} behavior is already covered by {@code PerKeyContainerPoolTest},+ * so what matters here is the wiring, that the manager genuinely implements {@link ContainerTeardown}+ * and hands the pool container ids it actually started.+ */+class PypiserverSidecarManagerTest {++ private final ContainerBuilder containerBuilder = mock(ContainerBuilder.class);+ private final ContainerLifecycleManager lifecycleManager = mock(ContainerLifecycleManager.class);+ private final EmulatorConfig config = mock(EmulatorConfig.class);++ @Test+ void implementsContainerTeardownSoStateResetAndNukeCanFindIt() {+ assertInstanceOf(ContainerTeardown.class, manager());+ }++ @Test+ void stopManagedContainersStopsEveryContainerThePoolIsTracking() throws Exception {+ PypiserverSidecarManager manager = manager();+ seedPooledContainer(manager, "pypi-repo-1", "tracked-pypiserver-1");+ seedPooledContainer(manager, "pypi-repo-2", "tracked-pypiserver-2");++ manager.stopManagedContainers();++ verify(lifecycleManager).stopAndRemove("tracked-pypiserver-1", null);+ verify(lifecycleManager).stopAndRemove("tracked-pypiserver-2", null);+ }++ @Test+ void stopManagedContainersIsANoOpWhenNothingWasEverStarted() {+ PypiserverSidecarManager manager = manager();++ manager.stopManagedContainers();++ verifyNoInteractions(lifecycleManager);+ }++ private PypiserverSidecarManager manager() {+ return new PypiserverSidecarManager(containerBuilder, lifecycleManager, config);+ }++ @SuppressWarnings("unchecked")+ private static void seedPooledContainer(PypiserverSidecarManager manager, String key, String containerId)+ throws Exception {+ Field poolField = PypiserverSidecarManager.class.getDeclaredField("pool");+ poolField.setAccessible(true);+ PerKeyContainerPool pool = (PerKeyContainerPool) poolField.get(manager);++ Field containersField = PerKeyContainerPool.class.getDeclaredField("containers");+ containersField.setAccessible(true);+ ConcurrentHashMap<String, PerKeyContainerPool.StartedContainer> containers =+ (ConcurrentHashMap<String, PerKeyContainerPool.StartedContainer>) containersField.get(pool);+ containers.put(key, new PerKeyContainerPool.StartedContainer(containerId, "http://127.0.0.1:1"));+ }+}