Light, fluffy, and always free - The AWS Local Emulator alternative
feat(codeartifact): add ListPackages for every format (#6622)
Priamo Germosen454d7b6ebda4
diff --git a/docs/services/codeartifact.md b/docs/services/codeartifact.mdindex b3bfa9eb00..eeb75438f8 100644--- a/docs/services/codeartifact.md+++ b/docs/services/codeartifact.md@@ -46,6 +46,7 @@ behind them yet.| `GetPackageVersionAsset` | Downloads one asset from a package version by name, optionally pinned to a specific revision. || `ListPackageVersionAssets` | Lists the assets of a package version, with size and hashes. || `ListPackageVersions` | Lists a package's versions with status, revision, and origin, paginated and sorted most recently published first; `defaultDisplayVersion` is `dist-tags.latest` for npm, the most recently published version otherwise. |+| `ListPackages` | Lists package names across every format in a repository, with each package's origin controls; paginated, optionally filtered by format, namespace prefix, package-name prefix, or publish/upstream restriction. || `GetPackageVersionReadme` | Returns the readme text for a package version; `npm`, `pypi`, and `nuget` only, matching AWS's own restriction for this action. || `CopyPackageVersions` | Copies one or more versions of a package from one repository to another in the same domain. || `UpdatePackageVersionsStatus` | Moves one or more versions of a package to `Published`, `Unlisted`, or `Archived`; all-or-nothing per call, with `versionRevisions`/`expectedStatus` as optional preconditions. |@@ -257,12 +258,11 @@ state.that is not the caller's own account looks up that account's domain/repository with notrust-policy or permissions-policy enforcement, consistent with Floci's IAM enforcement beingopt-in elsewhere, but worth knowing if you rely on domain-sharing semantics.-- **`ListPackages` isn't implemented.** Every package-version lifecycle action now is: a version- can be moved to `Unlisted`/`Archived`/`Published` with `UpdatePackageVersionsStatus`, to- `Disposed` (assets permanently deleted) with `DisposePackageVersions`, or removed outright,- from every status including `Disposed`, with `DeletePackageVersions`. The only way to move a- version out of `Unfinished` today is still either a follow-up `PublishPackageVersion` call that- omits the `unfinished` flag, or `UpdatePackageVersionsStatus` itself.+- **The native per-format download routes don't consult a container-backed version's status+ override.** `GetPackageVersionAsset` correctly blocks an Archived/Disposed version, but a plain+ `mvn`/`npm install`/`pip install`/`nuget` fetch against the Maven GAV, npm tarball, pypi+ filename, or nuget content route proxies straight to the sidecar and never checks it, so that+ same version's bytes are still fetchable through those routes.- **A container-backed (`maven`/`npm`/`pypi`/`nuget`) version's status override does not survive aFloci restart.** It's kept in memory only, for the same reason the byte-count and readmebookkeeping those formats already need are: the sidecar content it describes doesn't survive adiff --git a/docs/services/index.md b/docs/services/index.mdindex ecaed49288..76156af35f 100644--- a/docs/services/index.md+++ b/docs/services/index.md@@ -124,7 +124,7 @@ Operation counts are exact. For dispatch-table services (Query and JSON 1.1) eac| [AWS Backup](backup.md) | `/backup-vaults/*`, `/logically-air-gapped-backup-vaults/*`, `/backup/plans/*`, `/backup-jobs/*`, `/supported-resource-types` | REST JSON | 35 || [AWS FIS](fis.md) | `/experimentTemplates/*`, `/experiments/*`, `/actions/*`, `/targetResourceTypes/*`, `/safetyLevers/*`, `/tags/*` | REST JSON | 26 || [CodeGuru Reviewer](codegurureviewer.md) | `/associations`, `/associations/{associationArn}`, `/tags/*` | REST JSON | 7 |-| [CodeArtifact](codeartifact.md) | `/v1/domain*`, `/v1/repository*`, `/v1/package`, `/v1/package/version*`, `/v1/tag*`, `/v1/authorization-token`, `/codeartifact/maven/*`, `/codeartifact/npm/*`, `/codeartifact/pypi/*` | REST JSON | 36 |+| [CodeArtifact](codeartifact.md) | `/v1/domain*`, `/v1/repository*`, `/v1/package`, `/v1/package/version*`, `/v1/tag*`, `/v1/authorization-token`, `/codeartifact/maven/*`, `/codeartifact/npm/*`, `/codeartifact/pypi/*` | REST JSON | 37 || [CloudFront](cloudfront.md) | `/2020-05-31/distribution/*`, `/2020-05-31/cache-policy/*`, `/2020-05-31/function/*` | REST XML | 50 || [Route53](route53.md) | `/2013-04-01/hostedzone/*`, `/2013-04-01/healthcheck/*`, `/2013-04-01/change/*` | REST XML | 27 || [Route 53 Resolver](route53resolver.md) | `POST /` + `X-Amz-Target: Route53Resolver.*` | JSON 1.1 | 22 |diff --git a/src/main/java/io/github/hectorvent/floci/core/common/docker/PerKeyContainerPool.java b/src/main/java/io/github/hectorvent/floci/core/common/docker/PerKeyContainerPool.javaindex 0897eb3c01..15800b8cc0 100644--- a/src/main/java/io/github/hectorvent/floci/core/common/docker/PerKeyContainerPool.java+++ b/src/main/java/io/github/hectorvent/floci/core/common/docker/PerKeyContainerPool.java@@ -102,6 +102,17 @@ public class PerKeyContainerPool {}}+ /**+ * Whether a container for {@code key} is currently tracked by this pool, without starting one+ * for a key that has none: a plain map read, never {@link #ensureReady}'s own health probe or+ * {@code Starter} call. For a caller that only wants to ask an already-running container+ * something (whether it holds any content worth enumerating, say) and must not provision a+ * brand-new one just to find out it has nothing.+ */+ public boolean isTracked(String key) {+ return containers.containsKey(key);+ }+/*** Stops and removes the container for one key, if one was ever started. No-op otherwise.* Shares {@link #ensureReady}'s per-key lock so a stop racing a concurrent first-use of thediff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClient.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClient.javaindex f9f7421587..958903a945 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClient.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClient.java@@ -198,6 +198,55 @@ public class BagetterSidecarClient implements RepositorySidecarManager {return result;}+ /**+ * BaGetter's NuGet V3 Search resource ({@code /v3/search}), pressed into service purely as an+ * admin-side enumeration for this management API, a different purpose from the native NuGet+ * client protocol this class's own class-level documentation explains it deliberately never+ * stands up Search to serve. An empty {@code q} matches every package (confirmed live); {@code+ * data} empties out once every page has been read, the loop's own termination signal rather+ * than {@code totalHits}, which was confirmed live to report results *remaining from the+ * current skip*, not a stable grand total, so trusting it as a loop bound would stop one page+ * early. {@code id} is returned real-cased by BaGetter (confirmed live), lowercased here to+ * match what {@code CodeArtifactService}'s own {@code accountingPackageName} already+ * normalizes every NuGet package name to (AWS's own documented NuGet normalization). NuGet+ * packages have no namespace.+ *+ * <p>Raised by Greptile's review: the real NuGet V3 Search resource excludes prerelease+ * versions whenever {@code prerelease} is omitted (the official API's own documented default),+ * so a package whose only published version is a prerelease (e.g. {@code 1.0.0-beta}) would+ * never appear here without it. {@code prerelease=true} is passed explicitly since this call+ * enumerates every package a repository holds, regardless of what any one version's own+ * stability looks like, not a client-facing search that should default to stable results.+ *+ * <p>Raised by hectorvent's review: every CodeArtifact repository gets a nuget sidecar id at+ * creation regardless of whether it is ever actually used through nuget, so without checking+ * {@link BagetterSidecarManager#isRunning} first, a {@code ListPackages} call with no format+ * filter would start this repository's own BaGetter container just to find it empty, every+ * time, for a repository that has never published anything through nuget.+ */+ @Override+ public List<PackageIdentifier> listPackages(String repositoryContainerId, String domain, String repository) {+ if (!manager.isRunning(repositoryContainerId)) {+ return List.of();+ }+ List<PackageIdentifier> result = new ArrayList<>();+ int skip = 0;+ int take = 100;+ while (true) {+ JsonNode page = getJson(repositoryContainerId, "/v3/search?q=&prerelease=true&skip=" + skip + "&take="+ + take).orElseThrow(() -> new IllegalStateException("Could not list packages on the BaGetter "+ + "sidecar"));+ JsonNode data = page.path("data");+ if (!data.isArray() || data.isEmpty()) {+ return result;+ }+ for (JsonNode entry : data) {+ result.add(new PackageIdentifier(null, entry.path("id").asText("").toLowerCase(Locale.ROOT)));+ }+ skip += data.size();+ }+ }+private Optional<JsonNode> getJson(String repositoryContainerId, String encodedPath) {String baseUrl = ensureReady(repositoryContainerId, null);return getJsonAbsolute(baseUrl + encodedPath);diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarManager.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarManager.javaindex 8ef75e45d2..5818a8aff8 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarManager.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarManager.java@@ -73,6 +73,18 @@ public class BagetterSidecarManager implements ContainerTeardown {pool.stopContainer(nugetRepositoryId);}+ /**+ * Whether this nuget repository's own BaGetter container is already running, without starting+ * one for a repository that has never used nuget at all. Every CodeArtifact repository gets a+ * nuget sidecar id at creation regardless of whether it is ever actually used through nuget+ * (see {@code CodeArtifactService#ensureFormatContainerId}'s own callers), so a caller that+ * only wants to ask an already-running container what it holds must check this first rather+ * than provision a brand-new container just to find it has nothing.+ */+ public boolean isRunning(String nugetRepositoryId) {+ return pool.isTracked(nugetRepositoryId);+ }+private PerKeyContainerPool.StartedContainer startContainer(String nugetRepositoryId) {String image = config.services().codeartifact().nugetImage();String containerName = ContainerStorageHelper.dockerName(config, "floci-bagetter-" + nugetRepositoryId);diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactController.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactController.javaindex f5e5b61669..3dd434d258 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactController.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactController.java@@ -495,6 +495,43 @@ public class CodeArtifactController {return ok(response);}+ @POST+ @Path("/v1/packages")+ @Consumes(MediaType.WILDCARD)+ public Response listPackages(@Context HttpHeaders headers, @QueryParam("domain") String domain,+ @QueryParam("domain-owner") String domainOwner,+ @QueryParam("repository") String repository,+ @QueryParam("format") String format,+ @QueryParam("namespace") String namespace,+ @QueryParam("package-prefix") String packagePrefix,+ @QueryParam("max-results") String maxResults,+ @QueryParam("next-token") String nextToken,+ @QueryParam("publish") String publish,+ @QueryParam("upstream") String upstream) {+ String region = regionResolver.resolveRegion(headers);+ CodeArtifactService.ListPackagesResult result = service.listPackages(region, domain, domainOwner,+ repository, format, namespace, packagePrefix,+ Pagination.parseMaxResults(maxResults, "ValidationException"), nextToken, publish, upstream);+ ObjectNode response = objectMapper.createObjectNode();+ ArrayNode packages = response.putArray("packages");+ result.packages().forEach(summary -> {+ ObjectNode node = packages.addObject();+ node.put("format", summary.format());+ if (summary.namespace() != null) {+ node.put("namespace", summary.namespace());+ }+ node.put("package", summary.packageName());+ ObjectNode origin = node.putObject("originConfiguration");+ ObjectNode restrictions = origin.putObject("restrictions");+ restrictions.put("publish", summary.publishRestriction());+ restrictions.put("upstream", summary.upstreamRestriction());+ });+ if (result.nextToken() != null) {+ response.put("nextToken", result.nextToken());+ }+ return ok(response);+ }+@GET@Path("/v1/package/version/readme")public Response getPackageVersionReadme(@Context HttpHeaders headers, @QueryParam("domain") String domain,diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.javaindex c8a6aa0f2f..c2ba109e39 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.java@@ -163,6 +163,22 @@ public class CodeArtifactService implements Resettable {public record PackageVersionSummary(String version, String revision, String status) {}public record ListPackageVersionsResult(String format, String namespace, String packageName,String defaultDisplayVersion, List<PackageVersionSummary> versions, String nextToken) {}+ public record PackageSummaryEntry(String format, String namespace, String packageName,+ String publishRestriction, String upstreamRestriction) {}+ public record ListPackagesResult(List<PackageSummaryEntry> packages, String nextToken) {}+ /** Every package a repository holds, regardless of format, before origin-restriction filtering+ * and pagination are applied in {@link #listPackages}. */+ private record PackageCoordinate(String format, String namespace, String packageName) {}++ /**+ * {@link #allPackagesInRepository}'s own two outputs, kept together so {@link #listPackages}+ * never has to re-derive one from the other: {@code coordinates} is already deduplicated and+ * normalized, and {@code originConfigsByCoordinate} is keyed by that exact same normalized+ * coordinate, resolved from the one scan of {@code packageOriginConfigs} that method already+ * does for its own union.+ */+ private record PackagesWithOriginConfigs(List<PackageCoordinate> coordinates,+ Map<PackageCoordinate, PackageOriginConfig> originConfigsByCoordinate) {}/** Internal working shape shared by the generic and container-backed branches of {@link* #listPackageVersions} before status/origin filtering and pagination are applied.* {@code publishedEpochSeconds} is {@code null} when nothing is known to order by. */@@ -615,9 +631,9 @@ public class CodeArtifactService implements Resettable {* {@link #ensureFormatContainerId} plus {@link RepositorySidecarManager#beginUse}, both under* this same {@code synchronized} monitor {@link #deleteRepository} also holds for its own* sidecar cleanup, so a delete can never run its cleanup in the gap between resolving- * {@code repoId} and registering use of it. Only {@link #listPackageVersions} needs this: it- * is the one caller of {@link #ensureFormatContainerId} whose sidecar call is bracketed by- * {@code beginUse}/{@code endUse} at all.+ * {@code repoId} and registering use of it. Used by every caller of {@link+ * #ensureFormatContainerId} whose sidecar call is bracketed by {@code beginUse}/{@code+ * endUse} at all: {@link #listPackageVersions} and {@link #listPackages}.*/private synchronized String resolveContainerIdAndBeginUse(String format, String region, String domain,String domainOwner, String repository, RepositorySidecarManager manager) {@@ -2353,6 +2369,211 @@ public class CodeArtifactService implements Resettable {return pv;}+ /**+ * Every package a repository holds, for {@code ListPackages}: every format unless+ * {@code requestedFormat} narrows it to one, in which case only that one format's own source+ * is ever touched, container-backed sidecar included: without this, an unfiltered-by-format+ * caller's own lazily-provisioned sidecar containers would stay the real cost of answering+ * "all formats", but a caller who already named one format would otherwise still pay to start+ * every other format's container for nothing, just to filter its packages back out a moment+ * later.+ *+ * <p>{@code generic} is read straight from the stored {@link CodeArtifactPackageVersion}+ * records' own {@code format}/{@code namespace}/{@code packageName} fields (the same technique+ * {@link #genericPackageVersions} already uses for one known package, here scanned+ * repository-wide instead of by one package's own prefix), plus any origin-config-only+ * package's own matching fields on {@link PackageOriginConfig}, never by parsing a scanned key+ * back into parts, which {@link #escapeKeySegment}'s own javadoc explains is lossy. Every+ * container-backed format's own new {@link RepositorySidecarManager#listPackages} answers the+ * same for real sidecar content, plus the same origin-config-only union. {@code domainOwner}+ * is threaded through (rather than just the already-resolved {@code owner}) only because+ * {@link #resolveContainerIdAndBeginUse} re-resolves it itself, the same double-resolution+ * {@link #listPackageVersions} already does at its own call site.+ *+ * <p>Raised by Greptile's review: every coordinate here, from every source, is now normalized+ * through {@link #accountingPackageName} before it is ever added to the result set or used as+ * an origin-config lookup key. Without it, a pypi/nuget package whose {@link+ * PackageOriginConfig} was set under its as-typed spelling (case, or pypi's {@code -}/{@code _}/+ * {@code .} variants) would union into a second, spelling-distinct entry alongside the same+ * package's real, sidecar-normalized one, and the config lookup keyed by the normalized+ * spelling would never find a config stored under the raw one, silently falling back to the+ * ALLOW/BLOCK defaults instead of what was actually configured. Resolving every origin config+ * once here, into a map keyed by this same normalized coordinate, is what lets {@link+ * #listPackages} read it correctly regardless of which spelling the config was originally set+ * under.+ */+ private PackagesWithOriginConfigs allPackagesInRepository(String region, String domain, String domainOwner,+ String owner, String repository, String requestedFormat) {+ Set<PackageCoordinate> coordinates = new LinkedHashSet<>();+ Map<PackageCoordinate, PackageOriginConfig> originConfigsByCoordinate = new LinkedHashMap<>();+ String originPrefix = region + "::" + domain + "::" + repository + "::";+ if (requestedFormat == null || "generic".equals(requestedFormat)) {+ for (CodeArtifactPackageVersion pv : packageVersions.scanForAccount(owner,+ k -> k.startsWith(originPrefix))) {+ coordinates.add(new PackageCoordinate(pv.getFormat(), pv.getNamespace(),+ accountingPackageName(pv.getFormat(), pv.getPackageName())));+ }+ }+ for (PackageOriginConfig config : packageOriginConfigs.scanForAccount(owner,+ k -> k.startsWith(originPrefix))) {+ if (config.getFormat() != null+ && (requestedFormat == null || requestedFormat.equals(config.getFormat()))) {+ PackageCoordinate coordinate = new PackageCoordinate(config.getFormat(), config.getNamespace(),+ accountingPackageName(config.getFormat(), config.getPackageName()));+ coordinates.add(coordinate);+ originConfigsByCoordinate.put(coordinate, config);+ }+ }+ for (String format : CONTAINER_BACKED_FORMATS) {+ if (requestedFormat != null && !requestedFormat.equals(format)) {+ continue;+ }+ RepositorySidecarManager manager = sidecarRegistry.forFormat(format).orElse(null);+ if (manager == null) {+ continue;+ }+ String repoId = resolveContainerIdAndBeginUse(format, region, domain, domainOwner, repository, manager);+ try {+ for (RepositorySidecarManager.PackageIdentifier id : manager.listPackages(repoId, domain,+ repository)) {+ coordinates.add(new PackageCoordinate(format, id.namespace(),+ accountingPackageName(format, id.packageName())));+ }+ } catch (UnsupportedOperationException e) {+ throw new AwsException("InternalServerException", "Listing " + format + " packages is not yet "+ + "supported by this emulator.", 500);+ } finally {+ manager.endUse(repoId);+ }+ // Raised by Greptile's review: disposing a container-backed package's last version+ // deletes its real sidecar content, so the sidecar's own listing above no longer names+ // it at all, but updatePackageVersionsStatus/disposePackageVersions leave a status+ // override behind specifically so ListPackageVersions/DescribePackageVersion can keep+ // answering Disposed/Archived for it. Without this, a package with no origin+ // configuration either (nothing else to keep it in the union) would silently disappear+ // from ListPackages the moment its last version was disposed. Only ever adds something+ // for npm/pypi/nuget (see containerBackedOverriddenVersionStatusesByRepository's own+ // javadoc for why its index is namespace-less only); the same gap for maven or a scoped+ // npm package, both of which always carry a namespace, is not closed here.+ for (String packageName : containerBackedOverriddenVersionStatusesByRepository(region, domain,+ domainOwner, repository, format).keySet()) {+ coordinates.add(new PackageCoordinate(format, null, packageName));+ }+ }+ return new PackagesWithOriginConfigs(new ArrayList<>(coordinates), originConfigsByCoordinate);+ }++ /**+ * Sorts the same way {@link #packageVersionSortCursor} does for its own action, as an opaque+ * cursor: {@code ListPackages} has no {@code sortBy} parameter at all in the wire model (unlike+ * {@code ListPackageVersions}' {@code PUBLISHED_TIME}), so alphabetical by format, then+ * namespace, then package name is the best-justified deterministic order available, not an+ * AWS-confirmed one.+ *+ * <p>Raised by Greptile's review: joining the three fields with a plain {@code ::} the way an+ * earlier version of this method did collides for two different coordinates whose namespace+ * and package name happen to straddle that same separator differently, e.g. namespace+ * {@code a::b} with package {@code c} against namespace {@code a} with package {@code b::c};+ * {@code Pagination}'s own resume logic (strictly-greater-cursor) would then skip whichever of+ * the two sorts second. Each field is now escaped through {@link #escapeKeySegment}, the same+ * helper this service's own storage keys already use for exactly this collision, before being+ * joined.+ */+ private static String packageSortCursor(PackageSummaryEntry entry) {+ return escapeKeySegment(entry.format()) + "::" + escapeKeySegment(entry.namespace()) + "::"+ + escapeKeySegment(entry.packageName());+ }++ /**+ * Lists every package a repository holds, across every format at once unless {@code format}+ * filters to one. {@code namespace} is a prefix filter exactly like {@code packagePrefix} (its+ * own doc: "although this option is called --namespace and not --namespace-prefix, it has+ * prefix-matching behavior"), and real AWS rejects it when {@code format} is omitted ("Using+ * --namespace by itself results in an error," the User Guide's own documented search-option+ * combination rule): {@code format} combined with a namespace-less format (pypi, nuget) is+ * still accepted, since nothing in that same rule singles it out, it just can never match+ * anything, the same way a {@code packagePrefix} nothing matches returns empty rather than an+ * error.+ *+ * <p>{@code publish}/{@code upstream} filter against each package's own resolved restriction+ * (its {@link PackageOriginConfig} if one was ever set, else CodeArtifact's documented default+ * of publish {@code ALLOW}, upstream {@code BLOCK}), the same resolution {@link+ * #describePackage} already does for one known package.+ *+ * <p>Default page size is 100 (the User Guide's own stated default), not the 50 {@code+ * ListPackageVersions} uses: a real, AWS-documented difference between the two actions, not+ * an oversight copying one convention over the other.+ */+ public ListPackagesResult listPackages(String region, String domain, String domainOwner, String repository,+ String format, String namespace, String packagePrefix, Integer maxResults, String nextToken,+ String publish, String upstream) {+ requireNonBlank(domain, "domain");+ requireNonBlank(repository, "repository");+ if (format != null && !PACKAGE_FORMATS.contains(format)) {+ throw validation("format must be one of " + PACKAGE_FORMATS + ".");+ }+ if (namespace != null) {+ if (format == null) {+ throw validation("format must be specified when namespace is provided.");+ }+ validatePackageToken("namespace", namespace);+ }+ if (packagePrefix != null) {+ validatePackageToken("packagePrefix", packagePrefix);+ }+ if (publish != null && !ORIGIN_RESTRICTION_VALUES.contains(publish)) {+ throw validation("publish must be one of " + ORIGIN_RESTRICTION_VALUES + ".");+ }+ if (upstream != null && !ORIGIN_RESTRICTION_VALUES.contains(upstream)) {+ throw validation("upstream must be one of " + ORIGIN_RESTRICTION_VALUES + ".");+ }+ String owner = effectiveOwner(domainOwner);+ requireRepository(owner, repositoryKey(region, domain, repository), repository);++ List<PackageSummaryEntry> entries = new ArrayList<>();+ PackagesWithOriginConfigs packages = allPackagesInRepository(region, domain, domainOwner, owner, repository,+ format);+ for (PackageCoordinate coordinate : packages.coordinates()) {+ if (namespace != null+ && (coordinate.namespace() == null || !coordinate.namespace().startsWith(namespace))) {+ continue;+ }+ if (packagePrefix != null && !coordinate.packageName().startsWith(packagePrefix)) {+ continue;+ }+ // Resolved from the same normalized map allPackagesInRepository already built, rather+ // than a fresh key lookup here: see that method's own javadoc for why a direct lookup+ // keyed by this (already normalized) coordinate could otherwise miss a config stored+ // under its caller's original, un-normalized spelling.+ Optional<PackageOriginConfig> config = Optional.ofNullable(+ packages.originConfigsByCoordinate().get(coordinate));+ // Raised by Greptile's review: a PackageOriginConfig persisted before format/namespace/+ // packageName existed on it (see that model's own javadoc) has none of the three, so+ // allPackagesInRepository's own scan can never resolve it into this map at all, even+ // though the package it configures is still found here through its real version record+ // or sidecar listing. Falling back to the original key-based lookup, exactly like+ // describePackage's own, is what keeps that package's real restriction (not the ALLOW/+ // BLOCK default) resolving for it too.+ if (config.isEmpty()) {+ config = packageOriginConfigs.getForAccount(owner, packageKey(region, domain, repository,+ coordinate.format(), coordinate.namespace(), coordinate.packageName()));+ }+ String publishRestriction = config.map(PackageOriginConfig::getPublishRestriction).orElse("ALLOW");+ String upstreamRestriction = config.map(PackageOriginConfig::getUpstreamRestriction).orElse("BLOCK");+ if (publish != null && !publish.equals(publishRestriction)) {+ continue;+ }+ if (upstream != null && !upstream.equals(upstreamRestriction)) {+ continue;+ }+ entries.add(new PackageSummaryEntry(coordinate.format(), coordinate.namespace(),+ coordinate.packageName(), publishRestriction, upstreamRestriction));+ }+ PaginatedResult<PackageSummaryEntry> page = Pagination.paginate(entries,+ CodeArtifactService::packageSortCursor, maxResults, nextToken, 100, 1000, "ValidationException");+ return new ListPackagesResult(page.items(), page.nextToken());+ }+/*** Floci never ingests packages from an upstream source, so every package it holds was first* published directly. That is the one case where CodeArtifact's default origin controls are@@ -2421,7 +2642,7 @@ public class CodeArtifactService implements Resettable {requireRepository(owner, repositoryKey(region, domain, repository), repository);String key = packageKey(region, domain, repository, format, namespace, packageName);packageOriginConfigs.putForAccount(owner, key, new PackageOriginConfig(publishRestriction,- upstreamRestriction));+ upstreamRestriction, format, namespace, packageName));return new PackageDescription(format, namespace, packageName, publishRestriction, upstreamRestriction);}diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClient.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClient.javaindex 740de7d2e4..a2412bad37 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClient.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClient.java@@ -470,6 +470,51 @@ public class PypiserverSidecarClient implements RepositorySidecarManager {}}+ /**+ * pypiserver's own root simple index (confirmed live: {@code GET /simple/} with no project+ * name lists every registered project, each as an {@code <a href="project-name/">} entry,+ * the same href shape {@link #SIMPLE_INDEX_HREF} already parses for one project's own files).+ * Each href is already the PEP 503-normalized name real pypiserver registers it under+ * (confirmed: it never shows the originally-published casing/separators), matching what+ * {@code CodeArtifactService}'s own {@code accountingPackageName} already normalizes pypi+ * names to, so no further normalization is needed here. PyPI packages have no namespace.+ *+ * <p>Raised by hectorvent's review: every CodeArtifact repository gets a pypi sidecar id at+ * creation regardless of whether it is ever actually used through pypi, so without checking+ * {@link PypiserverSidecarManager#isRunning} first, a {@code ListPackages} call with no format+ * filter would start this repository's own pypiserver container just to find it empty, every+ * time, for a repository that has never published anything through pypi.+ */+ @Override+ public List<PackageIdentifier> listPackages(String repositoryContainerId, String domain, String repository) {+ if (!manager.isRunning(repositoryContainerId)) {+ return List.of();+ }+ String baseUrl = ensureReady(repositoryContainerId, null);+ URI uri = SidecarUriUtils.combine(URI.create(baseUrl), "/simple/");+ HttpRequest request = HttpRequest.newBuilder(uri).timeout(Duration.ofSeconds(10)).GET().build();+ HttpResponse<String> response;+ try {+ response = httpClient.send(request, BodyHandlers.ofString());+ } catch (Exception e) {+ throw new IllegalStateException("Could not reach the pypiserver sidecar to list packages", e);+ }+ if (response.statusCode() == 404) {+ return List.of();+ }+ if (response.statusCode() != 200) {+ throw new IllegalStateException("Could not list packages on the pypiserver sidecar: upstream returned "+ + response.statusCode());+ }+ List<PackageIdentifier> result = new ArrayList<>();+ SIMPLE_INDEX_HREF.matcher(response.body()).results().forEach(match -> {+ String href = match.group(1);+ String name = href.endsWith("/") ? href.substring(0, href.length() - 1) : href;+ result.add(new PackageIdentifier(null, name));+ });+ return result;+ }+private Optional<String> simpleIndex(String baseUrl, String packageName) {String normalized = normalizePackageName(packageName);URI uri = SidecarUriUtils.combine(URI.create(baseUrl), "/simple/" + SidecarUriUtils.encodeSegment(normalized)diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.javaindex 617253639b..b0b43f4063 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.java@@ -65,6 +65,18 @@ public class PypiserverSidecarManager implements ContainerTeardown {pool.stopContainer(pypiRepositoryId);}+ /**+ * Whether this pypi repository's own pypiserver container is already running, without starting+ * one for a repository that has never used pypi at all. Every CodeArtifact repository gets a+ * pypi sidecar id at creation regardless of whether it is ever actually used through pypi (see+ * {@code CodeArtifactService#ensureFormatContainerId}'s own callers), so a caller that only+ * wants to ask an already-running container what it holds must check this first rather than+ * provision a brand-new container just to find it has nothing.+ */+ public boolean isRunning(String pypiRepositoryId) {+ return pool.isTracked(pypiRepositoryId);+ }+private PerKeyContainerPool.StartedContainer startContainer(String pypiRepositoryId) {String image = config.services().codeartifact().pypiImage();String containerName = ContainerStorageHelper.dockerName(config, "floci-pypiserver-" + pypiRepositoryId);diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClient.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClient.javaindex ceaf912e9b..a26ba6ec9b 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClient.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClient.java@@ -328,6 +328,152 @@ public class ReposiliteSidecarClient implements RepositorySidecarManager {packageName, version))));}+ /**+ * One {@code /api/maven/details} response, split into what {@link #walkMavenDirectory} needs:+ * which file names this directory holds directly, and which entries are subdirectories to+ * possibly recurse into.+ */+ private record MavenDirectoryListing(List<String> fileNames, List<String> subdirectories) {}++ /**+ * Reposilite has no single list-everything endpoint (confirmed live: only the same one-level+ * {@code /api/maven/details} directory listing {@link #listPackageVersionAssetNames}/{@link+ * #listPackageVersions} already read one known directory at a time), so this walks the+ * repository's directory tree from its root, recursing into {@code DIRECTORY} entries.+ *+ * <p>A plain groupId segment can never be told apart from the group/artifact directory itself+ * by its own shape alone: both hold nothing but further {@code DIRECTORY} entries (confirmed+ * live: {@code spike}'s own listing, one {@code DIRECTORY} entry named {@code 1.0.0}, looks+ * identical in shape to {@code com}'s own listing, one {@code DIRECTORY} entry named {@code+ * example}). What tells them apart is one level deeper: a version directory's own children are+ * never further directories, only real files (or, after a {@code DisposePackageVersions}, none+ * at all); a plain groupId segment's children always are.+ *+ * <p>Raised by Greptile's review: a real Maven coordinate layout can have both at the very same+ * directory, since nothing stops one groupId from being a literal prefix of another's own+ * segments (AWS's own worked example shape: {@code com:foo:1.0.0} and {@code com.foo:bar:1.0.0}+ * share the real directory {@code com/foo}, one holding both a version directory {@code 1.0.0}+ * and a further groupId segment {@code bar}). Peeking only one representative child, as an+ * earlier version of this method did, answered only "is the current directory package-shaped+ * or group-shaped", a false either/or that depended on which child happened to be peeked (or on+ * {@code maven-metadata.xml}'s presence short-circuiting before any sibling was ever looked at+ * either), silently dropping whichever of the two coordinates it didn't settle on. {@link+ * #walkMavenDirectory} now checks every child of the current directory, not just one: a+ * version-shaped child (no further directories) proves the current directory is itself a+ * package, recorded at most once regardless of how many such children it has; a group-shaped+ * child is recursed into as a further namespace segment, every time, independent of whether the+ * current directory was already recorded. {@code maven-metadata.xml} still short-circuits+ * straight to a record (covering a package {@link #recordVersionAvailable} was called for that+ * happens to have zero version directories of its own, e.g. a {@code CopyPackageVersions}+ * source whose every version failed), but no longer skips the sibling-child walk below it.+ *+ * <p>Raised by hectorvent's review: every CodeArtifact repository gets a maven sidecar id at+ * creation regardless of whether it is ever actually used through maven, so without checking+ * {@link ReposiliteSidecarManager#isStarted} first, exactly like {@link #releaseRepository}+ * already does for the same reason, a {@code ListPackages} call with no format filter would+ * start the one shared Reposilite container just to walk it for a repository that has never+ * published anything through maven.+ */+ @Override+ public List<PackageIdentifier> listPackages(String repositoryContainerId, String domain, String repository) {+ if (!manager.isStarted()) {+ return List.of();+ }+ String baseUrl = manager.ensureReady();+ List<PackageIdentifier> result = new ArrayList<>();+ walkMavenDirectory(baseUrl, "/api/maven/details/" + SidecarUriUtils.encodeSegment(repositoryContainerId)+ + "/", List.of(), result);+ return result;+ }++ private void walkMavenDirectory(String baseUrl, String detailsPath, List<String> pathSegments,+ List<PackageIdentifier> result) {+ MavenDirectoryListing listing = fetchMavenDirectoryListing(baseUrl, detailsPath);+ if (listing == null) {+ return;+ }+ boolean recordedAsPackage = false;+ if (listing.fileNames().contains("maven-metadata.xml")) {+ recordMavenPackage(pathSegments, result);+ recordedAsPackage = true;+ }+ for (String subdirectory : listing.subdirectories()) {+ String childPath = detailsPath + SidecarUriUtils.encodeSegment(subdirectory) + "/";+ MavenDirectoryListing childListing = fetchMavenDirectoryListing(baseUrl, childPath);+ if (childListing == null) {+ continue;+ }+ // Raised by Greptile's review: a real Maven snapshot version directory also holds its+ // own maven-metadata.xml (listing that version's timestamped builds), alongside its+ // real artifact files (jar, pom, checksums), so "has maven-metadata.xml" alone cannot+ // tell that apart from the artifact-level-metadata-only directory the check below+ // exists for (one with no version directories of its own and, this time, no other real+ // files alongside its metadata.xml either). Only the latter shape, a directory whose+ // files are entirely maven-metadata.xml and its checksum sidecars and nothing else, is+ // excluded from being treated as version-shaped; a snapshot version directory's own+ // real artifact files keep it version-shaped exactly like any other.+ boolean isArtifactMetadataOnly = childListing.fileNames().contains("maven-metadata.xml")+ && childListing.fileNames().stream().allMatch(name -> name.equals("maven-metadata.xml")+ || name.startsWith("maven-metadata.xml."));+ if (childListing.subdirectories().isEmpty() && !isArtifactMetadataOnly) {+ // This child holds no further directories of its own (real asset files, or none at+ // all post-dispose) and is not artifact-metadata-only either: it is version-shaped,+ // proving the current directory is (also) the group/artifact level itself,+ // regardless of what its other children look like. An artifact-metadata-only child+ // (an artifact-level directory with zero version directories of its own, see+ // recordVersionAvailable's own javadoc, the exact shape a native PUT straight to+ // that file produces) is never version-shaped, no matter how empty it otherwise+ // looks; falling into the recurse below instead is what lets that child record+ // itself through this same method's own metadata.xml short-circuit above, rather+ // than this directory being wrongly recorded as the package in its place.+ if (!recordedAsPackage) {+ recordMavenPackage(pathSegments, result);+ recordedAsPackage = true;+ }+ continue;+ }+ List<String> nextSegments = new ArrayList<>(pathSegments);+ nextSegments.add(subdirectory);+ walkMavenDirectory(baseUrl, childPath, nextSegments, result);+ }+ }++ private void recordMavenPackage(List<String> pathSegments, List<PackageIdentifier> result) {+ if (pathSegments.size() >= 2) {+ String packageName = pathSegments.get(pathSegments.size() - 1);+ String namespace = String.join(".", pathSegments.subList(0, pathSegments.size() - 1));+ result.add(new PackageIdentifier(namespace, packageName));+ }+ }++ private MavenDirectoryListing fetchMavenDirectoryListing(String baseUrl, String detailsPath) {+ HttpRequest request = HttpRequest.newBuilder(SidecarUriUtils.combine(URI.create(baseUrl), detailsPath))+ .timeout(Duration.ofSeconds(30))+ .header("Authorization", manager.basicAuthHeader())+ .GET()+ .build();+ HttpResponse<String> response = send(request, BodyHandlers.ofString());+ if (response.statusCode() == 404) {+ return null;+ }+ if (response.statusCode() != 200) {+ throw new IllegalStateException("Could not list Maven packages on the Reposilite sidecar: upstream "+ + "returned " + response.statusCode());+ }+ List<String> fileNames = new ArrayList<>();+ List<String> subdirectories = new ArrayList<>();+ for (JsonNode entry : readTree(response.body()).path("files")) {+ String type = entry.path("type").asText();+ String name = entry.path("name").asText();+ if ("FILE".equals(type)) {+ fileNames.add(name);+ } else if ("DIRECTORY".equals(type)) {+ subdirectories.add(name);+ }+ }+ return new MavenDirectoryListing(fileNames, subdirectories);+ }+private Instant latestFileTimestamp(String baseUrl, String versionDetailsPath) {HttpRequest request = HttpRequest.newBuilder(SidecarUriUtils.combine(URI.create(baseUrl), versionDetailsPath)).timeout(Duration.ofSeconds(30))diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/RepositorySidecarManager.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/RepositorySidecarManager.javaindex dec946a69b..6c2f0ba2d5 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/RepositorySidecarManager.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/RepositorySidecarManager.java@@ -217,6 +217,29 @@ public interface RepositorySidecarManager {+ "sidecar");}+ /**+ * One package this sidecar holds, identified the same way {@link #packageExists} is:+ * {@code namespace} is {@code null} for a format or package that has none.+ */+ record PackageIdentifier(String namespace, String packageName) {}++ /**+ * Every package this sidecar holds at least one real version of, for bridging {@code+ * ListPackages} when there is no generic-format record to list packages from. Unlike {@link+ * #listPackageVersions}, which already knows which one package to ask about, this has no+ * known starting point at all: each implementation needs its own new, format-specific+ * repository-wide enumeration, confirmed live against each real sidecar protocol rather than+ * assumed (Verdaccio's {@code /-/all}, pypiserver's root simple index, BaGetter's Search+ * resource pressed into service for this admin-only purpose, Reposilite's own directory+ * listing walked recursively since it has no single list-everything endpoint). The default+ * throws {@link UnsupportedOperationException}, the same convention {@link #deletePackage}+ * already uses for a capability not every sidecar backs; every container-backed format's own+ * client overrides this.+ */+ default List<PackageIdentifier> listPackages(String repositoryContainerId, String domain, String repository) {+ throw new UnsupportedOperationException(format() + " packages cannot be listed through this sidecar");+ }+/*** Marks that {@code repositoryContainerId} is about to be used for {@link* #listPackageVersions}, called by {@code CodeArtifactService} right after it resolves thatdiff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClient.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClient.javaindex 5939563930..755e73623b 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClient.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClient.java@@ -153,6 +153,58 @@ public class VerdaccioSidecarClient implements RepositorySidecarManager {return result;}+ /**+ * Verdaccio's own {@code /-/all} endpoint (confirmed live), scoped to this repository's own+ * backend the same way every other call here is (one {@code baseUrl} per repository): its+ * response object's keys are every package's own real path exactly as {@link #packagePath}+ * already builds it ({@code @scope/name} or a bare {@code name}), plus one non-package key,+ * {@code _updated}, filtered out here rather than treated as a package named "_updated".+ *+ * <p>Raised by hectorvent's review: every CodeArtifact repository gets an npm sidecar id at+ * creation regardless of whether it is ever actually used through npm, so without checking+ * {@link VerdaccioSidecarManager#isRunning} first, a {@code ListPackages} call with no format+ * filter would start this repository's own Verdaccio container just to find it empty, every+ * time, for a repository that has never published anything through npm.+ */+ @Override+ public List<PackageIdentifier> listPackages(String repositoryContainerId, String domain, String repository) {+ if (!manager.isRunning(repositoryContainerId)) {+ return List.of();+ }+ String baseUrl = ensureReady(repositoryContainerId, publicUrl(domain, repository));+ HttpRequest request = HttpRequest.newBuilder(SidecarUriUtils.combine(URI.create(baseUrl), "/-/all"))+ .timeout(Duration.ofSeconds(30)).GET().build();+ HttpResponse<String> response;+ try {+ response = httpClient.send(request, BodyHandlers.ofString());+ } catch (Exception e) {+ throw new IllegalStateException("Could not reach the Verdaccio sidecar to list packages", e);+ }+ if (response.statusCode() != 200) {+ throw new IllegalStateException("Could not list packages on the Verdaccio sidecar: upstream returned "+ + response.statusCode());+ }+ JsonNode all;+ try {+ all = mapper.readTree(response.body());+ } catch (Exception e) {+ throw new IllegalStateException("Could not parse Verdaccio's package listing", e);+ }+ List<PackageIdentifier> result = new ArrayList<>();+ all.fieldNames().forEachRemaining(key -> {+ if ("_updated".equals(key)) {+ return;+ }+ if (key.startsWith("@")) {+ int slash = key.indexOf('/');+ result.add(new PackageIdentifier(key.substring(1, slash), key.substring(slash + 1)));+ } else {+ result.add(new PackageIdentifier(null, key));+ }+ });+ return result;+ }+/*** npm's own {@code dist-tags.latest}, AWS's documented preference for npm's {@code* defaultDisplayVersion} ahead of the most-recently-published fallback every other formatdiff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarManager.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarManager.javaindex 4471ce3604..8ba2382fc0 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarManager.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarManager.java@@ -82,6 +82,18 @@ public class VerdaccioSidecarManager implements ContainerTeardown {pool.stopContainer(npmRepositoryId);}+ /**+ * Whether this npm repository's own Verdaccio container is already running, without starting+ * one for a repository that has never used npm at all. Every CodeArtifact repository gets an+ * npm sidecar id at creation regardless of whether it is ever actually used through npm (see+ * {@code CodeArtifactService#ensureFormatContainerId}'s own callers), so a caller that only+ * wants to ask an already-running container what it holds must check this first rather than+ * provision a brand-new container just to find it has nothing.+ */+ public boolean isRunning(String npmRepositoryId) {+ return pool.isTracked(npmRepositoryId);+ }+private PerKeyContainerPool.StartedContainer startContainer(String npmRepositoryId, String publicUrl) {String image = config.services().codeartifact().npmImage();String containerName = ContainerStorageHelper.dockerName(config, "floci-verdaccio-" + npmRepositoryId);diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/model/PackageOriginConfig.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/model/PackageOriginConfig.javaindex f41ee1a37c..8542872056 100644--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/model/PackageOriginConfig.java+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/model/PackageOriginConfig.java@@ -9,19 +9,38 @@ import io.quarkus.runtime.annotations.RegisterForReflection;* configuration, it is just CodeArtifact's documented default (publish ALLOW, upstream BLOCK);* see {@code CodeArtifactService.PackageDescription}'s own Javadoc for why that default holds* for every package Floci serves until a caller overrides it.+ *+ * <p>{@code format}/{@code namespace}/{@code packageName} are carried on the value here, not only+ * encoded into the storage key that names it, for the same reason {@code+ * CodeArtifactPackageVersion} already carries its own: {@code ListPackages} needs to enumerate+ * every package a whole repository holds, including one that only ever got an origin+ * configuration and no real version, and the key itself is escaped specifically to make that+ * lossy to recover (see {@code CodeArtifactService#escapeKeySegment}'s own javadoc). Reading+ * these fields back off the scanned value instead, the same way {@code+ * CodeArtifactService#genericPackageVersions} already reads a version record's own fields rather+ * than parsing its key, avoids that entirely. {@code null} for a record written before this field+ * existed; a package that old is simply left out of that one enumeration, the same accepted gap+ * {@code GetPackageVersionReadme}'s own pre-existing-field limitation already documents elsewhere.*/@RegisterForReflection@JsonIgnoreProperties(ignoreUnknown = true)public class PackageOriginConfig {private String publishRestriction;private String upstreamRestriction;+ private String format;+ private String namespace;+ private String packageName;public PackageOriginConfig() {}- public PackageOriginConfig(String publishRestriction, String upstreamRestriction) {+ public PackageOriginConfig(String publishRestriction, String upstreamRestriction, String format,+ String namespace, String packageName) {this.publishRestriction = publishRestriction;this.upstreamRestriction = upstreamRestriction;+ this.format = format;+ this.namespace = namespace;+ this.packageName = packageName;}public String getPublishRestriction() {@@ -39,4 +58,28 @@ public class PackageOriginConfig {public void setUpstreamRestriction(String upstreamRestriction) {this.upstreamRestriction = upstreamRestriction;}++ public String getFormat() {+ return format;+ }++ public void setFormat(String format) {+ this.format = format;+ }++ public String getNamespace() {+ return namespace;+ }++ public void setNamespace(String namespace) {+ this.namespace = namespace;+ }++ public String getPackageName() {+ return packageName;+ }++ public void setPackageName(String packageName) {+ this.packageName = packageName;+ }}diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClientTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClientTest.javaindex 3ba8da7cf2..636af91416 100644--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClientTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClientTest.java@@ -19,6 +19,7 @@ import java.util.List;import java.util.Optional;import java.util.concurrent.ConcurrentHashMap;import java.util.concurrent.CopyOnWriteArrayList;+import java.util.concurrent.atomic.AtomicReference;import static org.junit.jupiter.api.Assertions.assertEquals;import static org.junit.jupiter.api.Assertions.assertFalse;@@ -341,6 +342,53 @@ class BagetterSidecarClientTest {});}+ /**+ * Raised by Greptile's review: the real NuGet V3 Search resource excludes prerelease versions+ * whenever {@code prerelease} is omitted, so a package whose only published version is a+ * prerelease would never come back from {@link BagetterSidecarClient#listPackages} without it.+ */+ @Test+ void listPackagesIncludesAPackageWhoseOnlyVersionIsAPrerelease() throws Exception {+ BagetterSidecarManager manager = manager();+ seedPooledContainer(manager, "nuget-repo-1", "tracked-bagetter-1", backendUrl);+ AtomicReference<String> requestedQuery = new AtomicReference<>();+ backend.createContext("/v3/search", exchange -> {+ String query = exchange.getRequestURI().getQuery();+ requestedQuery.set(query);+ boolean firstPage = query.contains("skip=0");+ byte[] body = (firstPage ? "{\"data\":[{\"id\":\"demo-pkg\"}]}" : "{\"data\":[]}")+ .getBytes(StandardCharsets.UTF_8);+ exchange.getResponseHeaders().set("Content-Type", "application/json");+ exchange.sendResponseHeaders(200, body.length);+ exchange.getResponseBody().write(body);+ exchange.close();+ });++ List<RepositorySidecarManager.PackageIdentifier> packages = client(manager).listPackages("nuget-repo-1",+ "dom", "repo");++ assertEquals(1, packages.size());+ assertEquals("demo-pkg", packages.get(0).packageName());+ assertTrue(requestedQuery.get().contains("prerelease=true"));+ }++ /**+ * Raised by hectorvent's review: every CodeArtifact repository gets a nuget sidecar id at+ * creation regardless of whether it is ever actually used through nuget, so {@code+ * ListPackages} with no format filter must not start this repository's own BaGetter+ * container just to find out it was never used. No context is registered on {@code backend}+ * at all here: a real container start would have nothing to connect to and throw.+ */+ @Test+ void listPackagesReturnsEmptyWithoutStartingAContainerThatWasNeverUsed() {+ BagetterSidecarManager manager = manager();++ List<RepositorySidecarManager.PackageIdentifier> packages = client(manager).listPackages("nuget-repo-1",+ "dom", "repo");++ assertTrue(packages.isEmpty());+ }+private BagetterSidecarManager manager() {return new BagetterSidecarManager(containerBuilder, lifecycleManager, config);}diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenDockerIntegrationTest.javaindex 7ab3b714ed..d5e40ce136 100644--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenDockerIntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenDockerIntegrationTest.java@@ -669,6 +669,32 @@ class CodeArtifactMavenDockerIntegrationTest {.body("versions[0].status", equalTo("Disposed"));}+ /**+ * A throwaway groupId/artifactId, not {@link #GAV}: {@code ListPackages} walks Reposilite's+ * whole directory tree repository-wide, so publishing its own package here (rather than+ * reusing one another test already created) keeps this test's own expected set independent of+ * whatever else this class's shared repository happens to hold by the time it runs.+ */+ @Test+ @Order(24)+ void listPackagesWalksTheRealReposiliteDirectoryTreeToFindTheGroupAndArtifact() {+ byte[] content = "list-packages-bytes".getBytes(StandardCharsets.UTF_8);+ given().header("Authorization", "Bearer " + bearerToken).body(content)+ .put("/codeartifact/maven/" + DOMAIN + "/" + REPO+ + "/com/example/nested/listpkgsprobe/1.0.0/listpkgsprobe-1.0.0.jar")+ .then().statusCode(200);++ JsonPath body = given().header("Authorization", AUTH)+ .post("/v1/packages?domain=" + DOMAIN + "&repository=" + REPO + "&format=maven"+ + "&namespace=com.example.nested")+ .then().statusCode(200)+ .extract().jsonPath();+ List<String> names = body.getList("packages.package", String.class);+ List<String> namespaces = body.getList("packages.namespace", String.class);+ assertTrue(names.contains("listpkgsprobe"), "expected listpkgsprobe among " + names);+ assertEquals("com.example.nested", namespaces.get(names.indexOf("listpkgsprobe")));+ }+/*** A throwaway groupId/artifactId, not {@link #GAV}: deleting it exercises Reposilite's real* directory delete (confirmed live before this was built: a single {@code DELETE} on thediff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNpmDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNpmDockerIntegrationTest.javaindex 0a19f9dae6..33699d3964 100644--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNpmDockerIntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNpmDockerIntegrationTest.java@@ -22,6 +22,7 @@ import java.io.IOException;import java.io.InputStream;import java.nio.charset.StandardCharsets;import java.util.Base64;+import java.util.List;import java.util.Map;import java.util.concurrent.CountDownLatch;import java.util.concurrent.ExecutorService;@@ -789,6 +790,40 @@ class CodeArtifactNpmDockerIntegrationTest {.body("versions[0].origin.originType", equalTo("INTERNAL"));}+ /**+ * Bridges to Verdaccio's own {@code /-/all} endpoint, confirmed live: besides {@link+ * #PACKAGE_NAME}, already published and confirmed present by {@link+ * #listPackageVersionsBridgesToTheRealVerdaccioSidecar} above, this also publishes a fresh+ * scoped package of its own to confirm the {@code @scope/name} key shape {@code+ * VerdaccioSidecarClient#listPackages} parses it into a namespace/package pair really is what+ * a live Verdaccio container returns, not just what its own unit test assumes.+ */+ @Test+ @Order(22)+ void listPackagesBridgesToTheRealVerdaccioAllEndpoint() {+ String scopedPackage = "listpkgsprobe";+ byte[] tarball = "list-packages-tarball".getBytes(StandardCharsets.UTF_8);+ String envelope = "{\"name\":\"@myscope/" + scopedPackage + "\",\"versions\":{\"1.0.0\":{\"name\":"+ + "\"@myscope/" + scopedPackage + "\",\"version\":\"1.0.0\",\"dist\":{\"tarball\":"+ + "\"http://ignored/" + scopedPackage + "-1.0.0.tgz\"}}},\"_attachments\":{\"" + scopedPackage+ + "-1.0.0.tgz\":{\"content_type\":\"application/octet-stream\",\"data\":\""+ + Base64.getEncoder().encodeToString(tarball) + "\",\"length\":" + tarball.length + "}}}";+ given().urlEncodingEnabled(false).header("Authorization", "Bearer " + bearerToken)+ .contentType("application/json").body(envelope)+ .put("/codeartifact/npm/" + DOMAIN + "/" + REPO + "/@myscope%2F" + scopedPackage)+ .then().statusCode(anyOf201Or200());++ JsonPath body = given().header("Authorization", AUTH)+ .post("/v1/packages?domain=" + DOMAIN + "&repository=" + REPO + "&format=npm")+ .then().statusCode(200)+ .extract().jsonPath();+ List<String> names = body.getList("packages.package", String.class);+ List<String> namespaces = body.getList("packages.namespace", String.class);+ assertTrue(names.contains(PACKAGE_NAME), "expected " + PACKAGE_NAME + " among " + names);+ assertTrue(names.contains(scopedPackage), "expected " + scopedPackage + " among " + names);+ assertEquals("myscope", namespaces.get(names.indexOf(scopedPackage)));+ }+/*** Proves the actual mechanism, not just that {@code VerdaccioSidecarManager} has a method* named right: {@code ContainerTeardowns.stopAll} runs every {@code ContainerTeardown} ondiff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNuGetDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNuGetDockerIntegrationTest.javaindex 0fbfa3f147..1b352ebd3f 100644--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNuGetDockerIntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNuGetDockerIntegrationTest.java@@ -14,6 +14,7 @@ import org.junit.jupiter.api.TestMethodOrder;import java.io.ByteArrayOutputStream;import java.nio.charset.StandardCharsets;+import java.util.List;import java.util.Locale;import java.util.Map;import java.util.zip.ZipEntry;@@ -25,6 +26,7 @@ import static org.hamcrest.Matchers.equalTo;import static org.hamcrest.Matchers.startsWith;import static org.junit.jupiter.api.Assertions.assertArrayEquals;import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertTrue;/*** Runs a real NuGet V3 push/install round trip against a live, per-repository BaGetter container,@@ -482,6 +484,35 @@ class CodeArtifactNuGetDockerIntegrationTest {.body("versions[0].origin.originType", equalTo("INTERNAL"));}+ /**+ * Bridges to BaGetter's own NuGet V3 Search resource, confirmed live: {@code+ * BagetterSidecarClient#listPackages} presses it into service purely for this admin listing+ * (this class's own class-level documentation explains why the native NuGet client protocol+ * never gets a Search endpoint of its own). A mixed-case id, confirmed back lowercased, since+ * a live search response returns the id real-cased exactly as pushed, and {@code+ * CodeArtifactService}'s own {@code accountingPackageName} normalizes every NuGet name to+ * lowercase (AWS's own documented NuGet normalization).+ */+ @Test+ @Order(17)+ void listPackagesBridgesToTheRealBagetterSearchResource() {+ String packageId = "ListPackages.Probe";+ byte[] nupkg = buildNupkg(packageId, VERSION, "list packages description", null, null);+ given().header("Authorization", "Bearer " + bearerToken)+ .contentType("multipart/form-data; boundary=boundary123")+ .body(pushBody("boundary123", nupkg))+ .put("/codeartifact/nuget/" + DOMAIN + "/" + REPO + "/api/v2/package")+ .then().statusCode(201);++ JsonPath body = given().header("Authorization", AUTH)+ .post("/v1/packages?domain=" + DOMAIN + "&repository=" + REPO + "&format=nuget")+ .then().statusCode(200)+ .extract().jsonPath();+ List<String> names = body.getList("packages.package", String.class);+ assertTrue(names.contains(packageId.toLowerCase(Locale.ROOT)), "expected "+ + packageId.toLowerCase(Locale.ROOT) + " among " + names);+ }+private static long describeDomainAssetSizeBytes() {return given().header("Authorization", AUTH).get("/v1/domain?domain=" + DOMAIN)diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.javaindex 2b787b9936..bf1ed139f6 100644--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.java@@ -771,6 +771,33 @@ class CodeArtifactPypiDockerIntegrationTest {+ defaultDisplayVersion);}+ /**+ * Bridges to pypiserver's own root {@code /simple/} index, confirmed live: uploads a package+ * with a mixed-case, dotted name to confirm {@code PypiserverSidecarClient#listPackages}'s own+ * href parsing really gets back the PEP 503-normalized project name a live pypiserver root+ * index reports, not the originally-uploaded spelling, matching what {@code+ * CodeArtifactService}'s own {@code accountingPackageName} already normalizes pypi names to.+ */+ @Test+ @Order(22)+ void listPackagesBridgesToTheRealPypiserverRootSimpleIndex() {+ String packageName = "List.Packages_Probe";+ String filename = "list_packages_probe-1.0.0.tar.gz";+ byte[] content = "list-packages-probe-bytes".getBytes(StandardCharsets.UTF_8);+ given().header("Authorization", "Bearer " + bearerToken)+ .contentType("multipart/form-data; boundary=boundary123")+ .body(twineUploadBody("boundary123", packageName, filename, content))+ .post("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/")+ .then().statusCode(200);++ JsonPath body = given().header("Authorization", AUTH)+ .post("/v1/packages?domain=" + DOMAIN + "&repository=" + REPO + "&format=pypi")+ .then().statusCode(200)+ .extract().jsonPath();+ List<String> names = body.getList("packages.package", String.class);+ assertTrue(names.contains("list-packages-probe"), "expected list-packages-probe among " + names);+ }+/*** Proves the actual mechanism, not just that {@code PypiserverSidecarManager} has a method* named right: {@code ContainerTeardowns.stopAll} runs every {@code ContainerTeardown} ondiff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.javaindex d05d19c4f8..e3a22e1e27 100644--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.java@@ -1864,7 +1864,8 @@ class CodeArtifactServiceTest {service.createDomain(REGION, "dom", null, Map.of());service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());String legacyRawKey = REGION + "::dom::repo::generic::a\\cb::x";- packageOriginConfigStore.putForAccount(ACCOUNT_ID, legacyRawKey, new PackageOriginConfig("BLOCK", "BLOCK"));+ packageOriginConfigStore.putForAccount(ACCOUNT_ID, legacyRawKey,+ new PackageOriginConfig("BLOCK", "BLOCK", null, null, null));service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "generic", "a:b", "x", "ALLOW", "ALLOW");@@ -5003,6 +5004,283 @@ class CodeArtifactServiceTest {assertEquals(content.length, pinned.asset().getSize());}+ // ----------------------------------------------------------------- ListPackages++ @Test+ void listPackagesReturnsEmptyForARepositoryWithNothingPublished() {+ service.createDomain(REGION, "dom", null, Map.of());+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());++ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null,+ null, null, null, null, null, null);++ assertTrue(result.packages().isEmpty());+ assertNull(result.nextToken());+ }++ @Test+ void listPackagesReturnsEveryGenericPackageInTheRepositoryWithDefaultOriginRestrictions() {+ service.createDomain(REGION, "dom", null, Map.of());+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ publishGeneric("repo", "my-pkg", "1.0.0", "a".getBytes(StandardCharsets.UTF_8));+ publishGeneric("repo", "other-pkg", "1.0.0", "b".getBytes(StandardCharsets.UTF_8));++ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null,+ null, null, null, null, null, null);++ Set<String> names = result.packages().stream().map(CodeArtifactService.PackageSummaryEntry::packageName)+ .collect(Collectors.toSet());+ assertEquals(Set.of("my-pkg", "other-pkg"), names);+ result.packages().forEach(p -> {+ assertEquals("generic", p.format());+ assertEquals("ns", p.namespace());+ assertEquals("ALLOW", p.publishRestriction());+ assertEquals("BLOCK", p.upstreamRestriction());+ });+ }++ @Test+ void listPackagesIncludesAnOriginConfigOnlyPackageWithNoRealVersion() {+ service.createDomain(REGION, "dom", null, Map.of());+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "generic", "ns", "never-published",+ "BLOCK", "ALLOW");++ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null,+ null, null, null, null, null, null);++ assertEquals(1, result.packages().size());+ CodeArtifactService.PackageSummaryEntry entry = result.packages().get(0);+ assertEquals("never-published", entry.packageName());+ assertEquals("BLOCK", entry.publishRestriction());+ assertEquals("ALLOW", entry.upstreamRestriction());+ }++ @Test+ void listPackagesRoutesEveryContainerBackedFormatToTheSidecarBridge() {+ service.createDomain(REGION, "dom", null, Map.of());+ CodeArtifactRepository created = service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ when(reposiliteClient.listPackages(eq(created.getSidecarContainerIds().get("maven")), eq("dom"),+ eq("repo"))).thenReturn(List.of(+ new RepositorySidecarManager.PackageIdentifier("com.example", "a")));+ when(verdaccioClient.listPackages(eq(created.getSidecarContainerIds().get("npm")), eq("dom"), eq("repo")))+ .thenReturn(List.of(new RepositorySidecarManager.PackageIdentifier(null, "my-pkg")));+ when(pypiserverClient.listPackages(eq(created.getSidecarContainerIds().get("pypi")), eq("dom"), eq("repo")))+ .thenReturn(List.of(new RepositorySidecarManager.PackageIdentifier(null, "my-pkg")));+ when(bagetterClient.listPackages(eq(created.getSidecarContainerIds().get("nuget")), eq("dom"), eq("repo")))+ .thenReturn(List.of(new RepositorySidecarManager.PackageIdentifier(null, "my-pkg")));++ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null,+ null, null, null, null, null, null);++ assertEquals(4, result.packages().size());+ assertTrue(result.packages().stream().anyMatch(p -> "maven".equals(p.format()) && "a".equals(p.packageName())+ && "com.example".equals(p.namespace())));+ assertEquals(3, result.packages().stream().filter(p -> "my-pkg".equals(p.packageName())).count());+ }++ @Test+ void listPackagesFormatFilterOnlyTouchesThatOneFormatsSidecar() {+ service.createDomain(REGION, "dom", null, Map.of());+ CodeArtifactRepository created = service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ when(pypiserverClient.listPackages(eq(created.getSidecarContainerIds().get("pypi")), eq("dom"), eq("repo")))+ .thenReturn(List.of(new RepositorySidecarManager.PackageIdentifier(null, "my-pkg")));++ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", "pypi",+ null, null, null, null, null, null);++ assertEquals(1, result.packages().size());+ assertEquals("pypi", result.packages().get(0).format());+ verify(verdaccioClient, never()).listPackages(anyString(), anyString(), anyString());+ verify(bagetterClient, never()).listPackages(anyString(), anyString(), anyString());+ verify(reposiliteClient, never()).listPackages(anyString(), anyString(), anyString());+ }++ @Test+ void listPackagesNamespacePrefixFiltersAndRequiresFormat() {+ service.createDomain(REGION, "dom", null, Map.of());+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ publishGeneric("repo", "my-pkg", "1.0.0", "a".getBytes(StandardCharsets.UTF_8));++ AwsException e = assertThrows(AwsException.class, () -> service.listPackages(REGION, "dom", null, "repo",+ null, "n", null, null, null, null, null));+ assertEquals("ValidationException", e.getErrorCode());++ CodeArtifactService.ListPackagesResult matches = service.listPackages(REGION, "dom", null, "repo", "generic",+ "n", null, null, null, null, null);+ assertEquals(1, matches.packages().size());++ CodeArtifactService.ListPackagesResult noMatches = service.listPackages(REGION, "dom", null, "repo",+ "generic", "other", null, null, null, null, null);+ assertTrue(noMatches.packages().isEmpty());+ }++ @Test+ void listPackagesPackagePrefixFiltersByName() {+ service.createDomain(REGION, "dom", null, Map.of());+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ publishGeneric("repo", "pat-test", "1.0.0", "a".getBytes(StandardCharsets.UTF_8));+ publishGeneric("repo", "other", "1.0.0", "b".getBytes(StandardCharsets.UTF_8));++ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null, null,+ "pat", null, null, null, null);++ assertEquals(1, result.packages().size());+ assertEquals("pat-test", result.packages().get(0).packageName());+ }++ @Test+ void listPackagesPublishAndUpstreamFilterByResolvedRestriction() {+ service.createDomain(REGION, "dom", null, Map.of());+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ publishGeneric("repo", "default-pkg", "1.0.0", "a".getBytes(StandardCharsets.UTF_8));+ publishGeneric("repo", "blocked-pkg", "1.0.0", "b".getBytes(StandardCharsets.UTF_8));+ service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "generic", "ns", "blocked-pkg", "BLOCK",+ "ALLOW");++ CodeArtifactService.ListPackagesResult allowPublish = service.listPackages(REGION, "dom", null, "repo", null,+ null, null, null, null, "ALLOW", null);+ assertEquals(1, allowPublish.packages().size());+ assertEquals("default-pkg", allowPublish.packages().get(0).packageName());++ CodeArtifactService.ListPackagesResult allowUpstream = service.listPackages(REGION, "dom", null, "repo",+ null, null, null, null, null, null, "ALLOW");+ assertEquals(1, allowUpstream.packages().size());+ assertEquals("blocked-pkg", allowUpstream.packages().get(0).packageName());+ }++ /**+ * Raised by Greptile's review: a pypi package's origin configuration is stored under whatever+ * spelling the caller set it with, but the real sidecar always lists the package under its own+ * PEP 503-normalized spelling. Without normalizing both sides to the same form before union-ing+ * them, the two would appear as two distinct packages, and the normalized one's own+ * restrictions would never resolve to what was actually configured.+ */+ @Test+ void listPackagesNormalizesAPypiPackageSoAnOriginConfigSetUnderADifferentSpellingIsNotDuplicatedOrLost() {+ service.createDomain(REGION, "dom", null, Map.of());+ CodeArtifactRepository created = service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ when(pypiserverClient.listPackages(eq(created.getSidecarContainerIds().get("pypi")), eq("dom"), eq("repo")))+ .thenReturn(List.of(new RepositorySidecarManager.PackageIdentifier(null, "my-proj")));+ service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "pypi", null, "My_Proj", "BLOCK", "ALLOW");++ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", "pypi",+ null, null, null, null, null, null);++ assertEquals(1, result.packages().size());+ assertEquals("my-proj", result.packages().get(0).packageName());+ assertEquals("BLOCK", result.packages().get(0).publishRestriction());+ assertEquals("ALLOW", result.packages().get(0).upstreamRestriction());+ }++ /**+ * Raised by Greptile's review: {@code packageSortCursor} used to join its three fields with a+ * plain {@code ::}, which namespace/package names are allowed to contain themselves, so two+ * different coordinates could collide on the exact same cursor and {@code Pagination}'s+ * strictly-greater-cursor resume would then skip whichever one sorted second.+ */+ @Test+ void listPackagesPagesPastACursorCollisionWithoutSkippingEitherPackage() {+ service.createDomain(REGION, "dom", null, Map.of());+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "generic", "a::b", "c", "ALLOW", "BLOCK");+ service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "generic", "a", "b::c", "ALLOW", "BLOCK");++ CodeArtifactService.ListPackagesResult firstPage = service.listPackages(REGION, "dom", null, "repo", null,+ null, null, 1, null, null, null);+ assertEquals(1, firstPage.packages().size());+ assertNotNull(firstPage.nextToken());++ CodeArtifactService.ListPackagesResult secondPage = service.listPackages(REGION, "dom", null, "repo", null,+ null, null, 1, firstPage.nextToken(), null, null);+ assertEquals(1, secondPage.packages().size());++ assertNotEquals(firstPage.packages().get(0).namespace() + "/" + firstPage.packages().get(0).packageName(),+ secondPage.packages().get(0).namespace() + "/" + secondPage.packages().get(0).packageName());+ }++ /**+ * Raised by Greptile's review: a {@link PackageOriginConfig} persisted before {@code format}/+ * {@code namespace}/{@code packageName} existed on it has none of the three, so {@code+ * allPackagesInRepository}'s own normalized map can never resolve it. The package it configures+ * is still found here through its real {@code generic} version record, so its restriction must+ * still resolve to what was actually saved, not silently fall back to the ALLOW/BLOCK default.+ */+ @Test+ void listPackagesFallsBackToTheRawKeyLookupForAnOriginConfigPersistedBeforeItCarriedItsOwnCoordinate() {+ service.createDomain(REGION, "dom", null, Map.of());+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ publishGeneric("repo", "legacy-pkg", "1.0.0", "a".getBytes(StandardCharsets.UTF_8));+ String legacyRawKey = REGION + "::dom::repo::generic::ns::legacy-pkg";+ packageOriginConfigStore.putForAccount(ACCOUNT_ID, legacyRawKey,+ new PackageOriginConfig("BLOCK", "ALLOW", null, null, null));++ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null,+ null, null, null, null, null, null);++ assertEquals(1, result.packages().size());+ assertEquals("BLOCK", result.packages().get(0).publishRestriction());+ assertEquals("ALLOW", result.packages().get(0).upstreamRestriction());++ CodeArtifactService.ListPackagesResult blockedOnly = service.listPackages(REGION, "dom", null, "repo", null,+ null, null, null, null, "BLOCK", null);+ assertEquals(1, blockedOnly.packages().size());+ }++ /**+ * Raised by Greptile's review: disposing a pypi package's only version deletes its real+ * sidecar content, so the sidecar's own listing no longer names it, but {@code+ * disposePackageVersions} leaves a status override behind precisely so {@code+ * ListPackageVersions} can keep reporting it as {@code Disposed}. With no origin configuration+ * either, nothing else in the union would otherwise keep this package in {@code ListPackages}.+ */+ @Test+ void listPackagesIncludesAPypiPackageWhoseOnlyVersionWasDisposed() {+ service.createDomain(REGION, "dom", null, Map.of());+ CodeArtifactRepository created = service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ String repoId = created.getSidecarContainerIds().get("pypi");+ when(pypiserverClient.listPackageVersions(eq(repoId), eq("dom"), eq("repo"), isNull(), eq("my-pkg")))+ .thenReturn(List.of(+ new RepositorySidecarManager.PackageVersionTimestamp("1.0.0", Instant.ofEpochSecond(1000))));+ when(pypiserverClient.listPackageVersionAssetNames(eq(repoId), eq("dom"), eq("repo"), isNull(),+ eq("my-pkg"), eq("1.0.0"))).thenReturn(List.of("my_pkg-1.0.0.tar.gz"));+ service.disposePackageVersions(REGION, "dom", null, "repo", "pypi", null, "my-pkg", List.of("1.0.0"), null,+ null);+ when(pypiserverClient.listPackageVersions(eq(repoId), eq("dom"), eq("repo"), isNull(), eq("my-pkg")))+ .thenReturn(List.of());++ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", "pypi",+ null, null, null, null, null, null);++ assertEquals(1, result.packages().size());+ assertEquals("my-pkg", result.packages().get(0).packageName());+ }++ @Test+ void listPackagesDefaultPageSizeIsOneHundredNotFifty() {+ service.createDomain(REGION, "dom", null, Map.of());+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());+ for (int i = 0; i < 120; i++) {+ publishGeneric("repo", String.format("pkg-%03d", i), "1.0.0", "a".getBytes(StandardCharsets.UTF_8));+ }++ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null, null,+ null, null, null, null, null);++ assertEquals(100, result.packages().size());+ assertNotNull(result.nextToken());+ }++ @Test+ void listPackagesRejectsAnInvalidFormat() {+ service.createDomain(REGION, "dom", null, Map.of());+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());++ AwsException e = assertThrows(AwsException.class, () -> service.listPackages(REGION, "dom", null, "repo",+ "not-a-format", null, null, null, null, null, null));+ assertEquals("ValidationException", e.getErrorCode());+ }+// ---------------------------------------------------------- GetPackageVersionReadme@Testdiff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClientTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClientTest.javaindex 2fef9e662b..334a4aa94f 100644--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClientTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClientTest.java@@ -511,6 +511,23 @@ class PypiserverSidecarClientTest {});}+ /**+ * Raised by hectorvent's review: every CodeArtifact repository gets a pypi sidecar id at+ * creation regardless of whether it is ever actually used through pypi, so {@code+ * ListPackages} with no format filter must not start this repository's own pypiserver+ * container just to find out it was never used. No context is registered on {@code backend}+ * at all here: a real container start would have nothing to connect to and throw.+ */+ @Test+ void listPackagesReturnsEmptyWithoutStartingAContainerThatWasNeverUsed() {+ PypiserverSidecarManager manager = manager();++ List<RepositorySidecarManager.PackageIdentifier> packages = client(manager).listPackages("pypi-repo-1",+ "dom", "repo");++ assertTrue(packages.isEmpty());+ }+private PypiserverSidecarManager manager() {return new PypiserverSidecarManager(containerBuilder, lifecycleManager, config);}diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClientTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClientTest.javaindex 7de469bbc3..5c2874e9af 100644--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClientTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClientTest.java@@ -1221,6 +1221,125 @@ class ReposiliteSidecarClientTest {new String(md5Body.get(), StandardCharsets.UTF_8));}+ /**+ * Raised by hectorvent's review: every CodeArtifact repository gets a maven sidecar id at+ * creation regardless of whether it is ever actually used through maven, so {@code+ * ListPackages} with no format filter must not start the one shared Reposilite container just+ * to walk a repository that has never used maven. No context is registered for {@code+ * /api/maven/details/dom--repo/} at all here: a real walk would have nothing to connect to and+ * throw.+ */+ @Test+ void listPackagesReturnsEmptyWithoutStartingTheSharedContainerThatWasNeverStarted() {+ when(manager.isStarted()).thenReturn(false);++ List<RepositorySidecarManager.PackageIdentifier> packages = client.listPackages("dom--repo", "dom", "repo");++ assertThat(packages, hasSize(0));+ }++ /**+ * Raised by Greptile's review: {@code com/foo} is both {@code com:foo}'s own version directory+ * ({@code 1.0.0}) and a further groupId segment for {@code com.foo:bar}'s own package+ * ({@code bar}). {@code bar} is listed before {@code 1.0.0} here deliberately, since an earlier+ * version of {@link ReposiliteSidecarClient#listPackages} peeked only the first child, so with+ * {@code bar} first it fell through to treating every child (including the real version+ * directory {@code 1.0.0}) as a further groupId segment, recursed into {@code 1.0.0} where it+ * found no subdirectories and silently returned, and never recorded {@code com:foo} at all.+ */+ @Test+ void listPackagesFindsBothAPackageAndASiblingGroupSharingItsDirectory() {+ server.createContext("/api/maven/details/dom--repo/",+ exchange -> respond(exchange, 200, "{\"name\":\"dom--repo\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"com\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/",+ exchange -> respond(exchange, 200, "{\"name\":\"com\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"foo\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/foo/",+ exchange -> respond(exchange, 200, "{\"name\":\"foo\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"bar\",\"type\":\"DIRECTORY\"},"+ + "{\"name\":\"1.0.0\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/foo/1.0.0/",+ exchange -> respond(exchange, 200, "{\"name\":\"1.0.0\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"foo-1.0.0.jar\",\"type\":\"FILE\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/foo/bar/",+ exchange -> respond(exchange, 200, "{\"name\":\"bar\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"1.0.0\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/foo/bar/1.0.0/",+ exchange -> respond(exchange, 200, "{\"name\":\"1.0.0\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"bar-1.0.0.jar\",\"type\":\"FILE\"}]}"));++ List<RepositorySidecarManager.PackageIdentifier> packages = client.listPackages("dom--repo", "dom", "repo");++ assertThat(packages, containsInAnyOrder(+ new RepositorySidecarManager.PackageIdentifier("com", "foo"),+ new RepositorySidecarManager.PackageIdentifier("com.foo", "bar")));+ }++ /**+ * Raised by Greptile's review: {@code com/example/tool} has zero version directories of its+ * own (a native PUT straight to its {@code maven-metadata.xml}, with no real version content+ * ever written under it, the same artifact-level-only shape {@code recordVersionAvailable}'s+ * own javadoc already describes), so peeking it from its parent sees no subdirectories at all+ * and, before this fix, treated that as proof the parent ({@code com:example}) was the real+ * package, recording a package that does not exist and skipping the real one,+ * {@code com.example:tool}, entirely.+ */+ @Test+ void listPackagesRecognizesAnArtifactDirectoryWithNoVersionsOfItsOwnThroughItsOwnMetadata() {+ server.createContext("/api/maven/details/dom--repo/",+ exchange -> respond(exchange, 200, "{\"name\":\"dom--repo\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"com\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/",+ exchange -> respond(exchange, 200, "{\"name\":\"com\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"example\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/example/",+ exchange -> respond(exchange, 200, "{\"name\":\"example\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"tool\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/example/tool/",+ exchange -> respond(exchange, 200, "{\"name\":\"tool\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"maven-metadata.xml\",\"type\":\"FILE\"}]}"));++ List<RepositorySidecarManager.PackageIdentifier> packages = client.listPackages("dom--repo", "dom", "repo");++ assertThat(packages, containsInAnyOrder(new RepositorySidecarManager.PackageIdentifier("com.example",+ "tool")));+ }++ /**+ * Raised by Greptile's review of the fix above: a real Maven snapshot version directory also+ * holds its own {@code maven-metadata.xml} (listing that version's timestamped builds),+ * alongside its real artifact files, so excluding every {@code maven-metadata.xml}-holding+ * child from being version-shaped, as that fix did, wrongly recursed into {@code+ * 1.0-SNAPSHOT} as though it were a further groupId segment, adding the nonexistent package+ * {@code com.example.tool:1.0-SNAPSHOT} instead of the real one, {@code com.example:tool}.+ */+ @Test+ void listPackagesTreatsASnapshotVersionDirectoryAsVersionShapedDespiteItsOwnMetadata() {+ server.createContext("/api/maven/details/dom--repo/",+ exchange -> respond(exchange, 200, "{\"name\":\"dom--repo\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"com\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/",+ exchange -> respond(exchange, 200, "{\"name\":\"com\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"example\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/example/",+ exchange -> respond(exchange, 200, "{\"name\":\"example\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"tool\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/example/tool/",+ exchange -> respond(exchange, 200, "{\"name\":\"tool\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"1.0-SNAPSHOT\",\"type\":\"DIRECTORY\"}]}"));+ server.createContext("/api/maven/details/dom--repo/com/example/tool/1.0-SNAPSHOT/",+ exchange -> respond(exchange, 200, "{\"name\":\"1.0-SNAPSHOT\",\"type\":\"DIRECTORY\",\"files\":["+ + "{\"name\":\"maven-metadata.xml\",\"type\":\"FILE\"},"+ + "{\"name\":\"maven-metadata.xml.sha1\",\"type\":\"FILE\"},"+ + "{\"name\":\"tool-1.0-SNAPSHOT.jar\",\"type\":\"FILE\"}]}"));++ List<RepositorySidecarManager.PackageIdentifier> packages = client.listPackages("dom--repo", "dom", "repo");++ assertThat(packages, containsInAnyOrder(new RepositorySidecarManager.PackageIdentifier("com.example",+ "tool")));+ }+private static void respond(HttpExchange exchange, int status, String body) throws IOException {byte[] bytes = body.getBytes(StandardCharsets.UTF_8);exchange.sendResponseHeaders(status, bytes.length);diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClientTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClientTest.javaindex b03cb4bdb4..a1f2660609 100644--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClientTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClientTest.java@@ -14,6 +14,7 @@ import java.lang.reflect.Field;import java.net.InetSocketAddress;import java.net.http.HttpClient;import java.nio.charset.StandardCharsets;+import java.util.List;import java.util.Optional;import java.util.concurrent.ConcurrentHashMap;import java.util.concurrent.atomic.AtomicReference;@@ -257,6 +258,23 @@ class VerdaccioSidecarClientTest {});}+ /**+ * Raised by hectorvent's review: every CodeArtifact repository gets an npm sidecar id at+ * creation regardless of whether it is ever actually used through npm, so {@code+ * ListPackages} with no format filter must not start this repository's own Verdaccio+ * container just to find out it was never used. No context is registered on {@code backend}+ * at all here: a real container start would have nothing to connect to and throw.+ */+ @Test+ void listPackagesReturnsEmptyWithoutStartingAContainerThatWasNeverUsed() {+ VerdaccioSidecarManager manager = manager();++ List<RepositorySidecarManager.PackageIdentifier> packages = client(manager).listPackages("npm-repo-1",+ "dom", "repo");++ assertTrue(packages.isEmpty());+ }+private VerdaccioSidecarManager manager() {return new VerdaccioSidecarManager(containerBuilder, lifecycleManager, config);}