Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

feat(codeartifact): add ListPackages for every format (#6622)

Priamo Germosen454d7b6ebda4
diff --git a/docs/services/codeartifact.md b/docs/services/codeartifact.md
index b3bfa9eb00..eeb75438f8 100644
--- a/docs/services/codeartifact.md
+++ b/docs/services/codeartifact.md
@@ -46,6 +46,7 @@ behind them yet.
| `GetPackageVersionAsset` | Downloads one asset from a package version by name, optionally pinned to a specific revision. |
| `ListPackageVersionAssets` | Lists the assets of a package version, with size and hashes. |
| `ListPackageVersions` | Lists a package's versions with status, revision, and origin, paginated and sorted most recently published first; `defaultDisplayVersion` is `dist-tags.latest` for npm, the most recently published version otherwise. |
+| `ListPackages` | Lists package names across every format in a repository, with each package's origin controls; paginated, optionally filtered by format, namespace prefix, package-name prefix, or publish/upstream restriction. |
| `GetPackageVersionReadme` | Returns the readme text for a package version; `npm`, `pypi`, and `nuget` only, matching AWS's own restriction for this action. |
| `CopyPackageVersions` | Copies one or more versions of a package from one repository to another in the same domain. |
| `UpdatePackageVersionsStatus` | Moves one or more versions of a package to `Published`, `Unlisted`, or `Archived`; all-or-nothing per call, with `versionRevisions`/`expectedStatus` as optional preconditions. |
@@ -257,12 +258,11 @@ state.
that is not the caller's own account looks up that account's domain/repository with no
trust-policy or permissions-policy enforcement, consistent with Floci's IAM enforcement being
opt-in elsewhere, but worth knowing if you rely on domain-sharing semantics.
-- **`ListPackages` isn't implemented.** Every package-version lifecycle action now is: a version
- can be moved to `Unlisted`/`Archived`/`Published` with `UpdatePackageVersionsStatus`, to
- `Disposed` (assets permanently deleted) with `DisposePackageVersions`, or removed outright,
- from every status including `Disposed`, with `DeletePackageVersions`. The only way to move a
- version out of `Unfinished` today is still either a follow-up `PublishPackageVersion` call that
- omits the `unfinished` flag, or `UpdatePackageVersionsStatus` itself.
+- **The native per-format download routes don't consult a container-backed version's status
+ override.** `GetPackageVersionAsset` correctly blocks an Archived/Disposed version, but a plain
+ `mvn`/`npm install`/`pip install`/`nuget` fetch against the Maven GAV, npm tarball, pypi
+ filename, or nuget content route proxies straight to the sidecar and never checks it, so that
+ same version's bytes are still fetchable through those routes.
- **A container-backed (`maven`/`npm`/`pypi`/`nuget`) version's status override does not survive a
Floci restart.** It's kept in memory only, for the same reason the byte-count and readme
bookkeeping those formats already need are: the sidecar content it describes doesn't survive a
diff --git a/docs/services/index.md b/docs/services/index.md
index ecaed49288..76156af35f 100644
--- a/docs/services/index.md
+++ b/docs/services/index.md
@@ -124,7 +124,7 @@ Operation counts are exact. For dispatch-table services (Query and JSON 1.1) eac
| [AWS Backup](backup.md) | `/backup-vaults/*`, `/logically-air-gapped-backup-vaults/*`, `/backup/plans/*`, `/backup-jobs/*`, `/supported-resource-types` | REST JSON | 35 |
| [AWS FIS](fis.md) | `/experimentTemplates/*`, `/experiments/*`, `/actions/*`, `/targetResourceTypes/*`, `/safetyLevers/*`, `/tags/*` | REST JSON | 26 |
| [CodeGuru Reviewer](codegurureviewer.md) | `/associations`, `/associations/{associationArn}`, `/tags/*` | REST JSON | 7 |
-| [CodeArtifact](codeartifact.md) | `/v1/domain*`, `/v1/repository*`, `/v1/package`, `/v1/package/version*`, `/v1/tag*`, `/v1/authorization-token`, `/codeartifact/maven/*`, `/codeartifact/npm/*`, `/codeartifact/pypi/*` | REST JSON | 36 |
+| [CodeArtifact](codeartifact.md) | `/v1/domain*`, `/v1/repository*`, `/v1/package`, `/v1/package/version*`, `/v1/tag*`, `/v1/authorization-token`, `/codeartifact/maven/*`, `/codeartifact/npm/*`, `/codeartifact/pypi/*` | REST JSON | 37 |
| [CloudFront](cloudfront.md) | `/2020-05-31/distribution/*`, `/2020-05-31/cache-policy/*`, `/2020-05-31/function/*` | REST XML | 50 |
| [Route53](route53.md) | `/2013-04-01/hostedzone/*`, `/2013-04-01/healthcheck/*`, `/2013-04-01/change/*` | REST XML | 27 |
| [Route 53 Resolver](route53resolver.md) | `POST /` + `X-Amz-Target: Route53Resolver.*` | JSON 1.1 | 22 |
diff --git a/src/main/java/io/github/hectorvent/floci/core/common/docker/PerKeyContainerPool.java b/src/main/java/io/github/hectorvent/floci/core/common/docker/PerKeyContainerPool.java
index 0897eb3c01..15800b8cc0 100644
--- a/src/main/java/io/github/hectorvent/floci/core/common/docker/PerKeyContainerPool.java
+++ b/src/main/java/io/github/hectorvent/floci/core/common/docker/PerKeyContainerPool.java
@@ -102,6 +102,17 @@ public class PerKeyContainerPool {
}
}
+ /**
+ * Whether a container for {@code key} is currently tracked by this pool, without starting one
+ * for a key that has none: a plain map read, never {@link #ensureReady}'s own health probe or
+ * {@code Starter} call. For a caller that only wants to ask an already-running container
+ * something (whether it holds any content worth enumerating, say) and must not provision a
+ * brand-new one just to find out it has nothing.
+ */
+ public boolean isTracked(String key) {
+ return containers.containsKey(key);
+ }
+
/**
* Stops and removes the container for one key, if one was ever started. No-op otherwise.
* Shares {@link #ensureReady}'s per-key lock so a stop racing a concurrent first-use of the
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClient.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClient.java
index f9f7421587..958903a945 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClient.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClient.java
@@ -198,6 +198,55 @@ public class BagetterSidecarClient implements RepositorySidecarManager {
return result;
}
+ /**
+ * BaGetter's NuGet V3 Search resource ({@code /v3/search}), pressed into service purely as an
+ * admin-side enumeration for this management API, a different purpose from the native NuGet
+ * client protocol this class's own class-level documentation explains it deliberately never
+ * stands up Search to serve. An empty {@code q} matches every package (confirmed live); {@code
+ * data} empties out once every page has been read, the loop's own termination signal rather
+ * than {@code totalHits}, which was confirmed live to report results *remaining from the
+ * current skip*, not a stable grand total, so trusting it as a loop bound would stop one page
+ * early. {@code id} is returned real-cased by BaGetter (confirmed live), lowercased here to
+ * match what {@code CodeArtifactService}'s own {@code accountingPackageName} already
+ * normalizes every NuGet package name to (AWS's own documented NuGet normalization). NuGet
+ * packages have no namespace.
+ *
+ * <p>Raised by Greptile's review: the real NuGet V3 Search resource excludes prerelease
+ * versions whenever {@code prerelease} is omitted (the official API's own documented default),
+ * so a package whose only published version is a prerelease (e.g. {@code 1.0.0-beta}) would
+ * never appear here without it. {@code prerelease=true} is passed explicitly since this call
+ * enumerates every package a repository holds, regardless of what any one version's own
+ * stability looks like, not a client-facing search that should default to stable results.
+ *
+ * <p>Raised by hectorvent's review: every CodeArtifact repository gets a nuget sidecar id at
+ * creation regardless of whether it is ever actually used through nuget, so without checking
+ * {@link BagetterSidecarManager#isRunning} first, a {@code ListPackages} call with no format
+ * filter would start this repository's own BaGetter container just to find it empty, every
+ * time, for a repository that has never published anything through nuget.
+ */
+ @Override
+ public List<PackageIdentifier> listPackages(String repositoryContainerId, String domain, String repository) {
+ if (!manager.isRunning(repositoryContainerId)) {
+ return List.of();
+ }
+ List<PackageIdentifier> result = new ArrayList<>();
+ int skip = 0;
+ int take = 100;
+ while (true) {
+ JsonNode page = getJson(repositoryContainerId, "/v3/search?q=&prerelease=true&skip=" + skip + "&take="
+ + take).orElseThrow(() -> new IllegalStateException("Could not list packages on the BaGetter "
+ + "sidecar"));
+ JsonNode data = page.path("data");
+ if (!data.isArray() || data.isEmpty()) {
+ return result;
+ }
+ for (JsonNode entry : data) {
+ result.add(new PackageIdentifier(null, entry.path("id").asText("").toLowerCase(Locale.ROOT)));
+ }
+ skip += data.size();
+ }
+ }
+
private Optional<JsonNode> getJson(String repositoryContainerId, String encodedPath) {
String baseUrl = ensureReady(repositoryContainerId, null);
return getJsonAbsolute(baseUrl + encodedPath);
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarManager.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarManager.java
index 8ef75e45d2..5818a8aff8 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarManager.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarManager.java
@@ -73,6 +73,18 @@ public class BagetterSidecarManager implements ContainerTeardown {
pool.stopContainer(nugetRepositoryId);
}
+ /**
+ * Whether this nuget repository's own BaGetter container is already running, without starting
+ * one for a repository that has never used nuget at all. Every CodeArtifact repository gets a
+ * nuget sidecar id at creation regardless of whether it is ever actually used through nuget
+ * (see {@code CodeArtifactService#ensureFormatContainerId}'s own callers), so a caller that
+ * only wants to ask an already-running container what it holds must check this first rather
+ * than provision a brand-new container just to find it has nothing.
+ */
+ public boolean isRunning(String nugetRepositoryId) {
+ return pool.isTracked(nugetRepositoryId);
+ }
+
private PerKeyContainerPool.StartedContainer startContainer(String nugetRepositoryId) {
String image = config.services().codeartifact().nugetImage();
String containerName = ContainerStorageHelper.dockerName(config, "floci-bagetter-" + nugetRepositoryId);
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactController.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactController.java
index f5e5b61669..3dd434d258 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactController.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactController.java
@@ -495,6 +495,43 @@ public class CodeArtifactController {
return ok(response);
}
+ @POST
+ @Path("/v1/packages")
+ @Consumes(MediaType.WILDCARD)
+ public Response listPackages(@Context HttpHeaders headers, @QueryParam("domain") String domain,
+ @QueryParam("domain-owner") String domainOwner,
+ @QueryParam("repository") String repository,
+ @QueryParam("format") String format,
+ @QueryParam("namespace") String namespace,
+ @QueryParam("package-prefix") String packagePrefix,
+ @QueryParam("max-results") String maxResults,
+ @QueryParam("next-token") String nextToken,
+ @QueryParam("publish") String publish,
+ @QueryParam("upstream") String upstream) {
+ String region = regionResolver.resolveRegion(headers);
+ CodeArtifactService.ListPackagesResult result = service.listPackages(region, domain, domainOwner,
+ repository, format, namespace, packagePrefix,
+ Pagination.parseMaxResults(maxResults, "ValidationException"), nextToken, publish, upstream);
+ ObjectNode response = objectMapper.createObjectNode();
+ ArrayNode packages = response.putArray("packages");
+ result.packages().forEach(summary -> {
+ ObjectNode node = packages.addObject();
+ node.put("format", summary.format());
+ if (summary.namespace() != null) {
+ node.put("namespace", summary.namespace());
+ }
+ node.put("package", summary.packageName());
+ ObjectNode origin = node.putObject("originConfiguration");
+ ObjectNode restrictions = origin.putObject("restrictions");
+ restrictions.put("publish", summary.publishRestriction());
+ restrictions.put("upstream", summary.upstreamRestriction());
+ });
+ if (result.nextToken() != null) {
+ response.put("nextToken", result.nextToken());
+ }
+ return ok(response);
+ }
+
@GET
@Path("/v1/package/version/readme")
public Response getPackageVersionReadme(@Context HttpHeaders headers, @QueryParam("domain") String domain,
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.java
index c8a6aa0f2f..c2ba109e39 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactService.java
@@ -163,6 +163,22 @@ public class CodeArtifactService implements Resettable {
public record PackageVersionSummary(String version, String revision, String status) {}
public record ListPackageVersionsResult(String format, String namespace, String packageName,
String defaultDisplayVersion, List<PackageVersionSummary> versions, String nextToken) {}
+ public record PackageSummaryEntry(String format, String namespace, String packageName,
+ String publishRestriction, String upstreamRestriction) {}
+ public record ListPackagesResult(List<PackageSummaryEntry> packages, String nextToken) {}
+ /** Every package a repository holds, regardless of format, before origin-restriction filtering
+ * and pagination are applied in {@link #listPackages}. */
+ private record PackageCoordinate(String format, String namespace, String packageName) {}
+
+ /**
+ * {@link #allPackagesInRepository}'s own two outputs, kept together so {@link #listPackages}
+ * never has to re-derive one from the other: {@code coordinates} is already deduplicated and
+ * normalized, and {@code originConfigsByCoordinate} is keyed by that exact same normalized
+ * coordinate, resolved from the one scan of {@code packageOriginConfigs} that method already
+ * does for its own union.
+ */
+ private record PackagesWithOriginConfigs(List<PackageCoordinate> coordinates,
+ Map<PackageCoordinate, PackageOriginConfig> originConfigsByCoordinate) {}
/** Internal working shape shared by the generic and container-backed branches of {@link
* #listPackageVersions} before status/origin filtering and pagination are applied.
* {@code publishedEpochSeconds} is {@code null} when nothing is known to order by. */
@@ -615,9 +631,9 @@ public class CodeArtifactService implements Resettable {
* {@link #ensureFormatContainerId} plus {@link RepositorySidecarManager#beginUse}, both under
* this same {@code synchronized} monitor {@link #deleteRepository} also holds for its own
* sidecar cleanup, so a delete can never run its cleanup in the gap between resolving
- * {@code repoId} and registering use of it. Only {@link #listPackageVersions} needs this: it
- * is the one caller of {@link #ensureFormatContainerId} whose sidecar call is bracketed by
- * {@code beginUse}/{@code endUse} at all.
+ * {@code repoId} and registering use of it. Used by every caller of {@link
+ * #ensureFormatContainerId} whose sidecar call is bracketed by {@code beginUse}/{@code
+ * endUse} at all: {@link #listPackageVersions} and {@link #listPackages}.
*/
private synchronized String resolveContainerIdAndBeginUse(String format, String region, String domain,
String domainOwner, String repository, RepositorySidecarManager manager) {
@@ -2353,6 +2369,211 @@ public class CodeArtifactService implements Resettable {
return pv;
}
+ /**
+ * Every package a repository holds, for {@code ListPackages}: every format unless
+ * {@code requestedFormat} narrows it to one, in which case only that one format's own source
+ * is ever touched, container-backed sidecar included: without this, an unfiltered-by-format
+ * caller's own lazily-provisioned sidecar containers would stay the real cost of answering
+ * "all formats", but a caller who already named one format would otherwise still pay to start
+ * every other format's container for nothing, just to filter its packages back out a moment
+ * later.
+ *
+ * <p>{@code generic} is read straight from the stored {@link CodeArtifactPackageVersion}
+ * records' own {@code format}/{@code namespace}/{@code packageName} fields (the same technique
+ * {@link #genericPackageVersions} already uses for one known package, here scanned
+ * repository-wide instead of by one package's own prefix), plus any origin-config-only
+ * package's own matching fields on {@link PackageOriginConfig}, never by parsing a scanned key
+ * back into parts, which {@link #escapeKeySegment}'s own javadoc explains is lossy. Every
+ * container-backed format's own new {@link RepositorySidecarManager#listPackages} answers the
+ * same for real sidecar content, plus the same origin-config-only union. {@code domainOwner}
+ * is threaded through (rather than just the already-resolved {@code owner}) only because
+ * {@link #resolveContainerIdAndBeginUse} re-resolves it itself, the same double-resolution
+ * {@link #listPackageVersions} already does at its own call site.
+ *
+ * <p>Raised by Greptile's review: every coordinate here, from every source, is now normalized
+ * through {@link #accountingPackageName} before it is ever added to the result set or used as
+ * an origin-config lookup key. Without it, a pypi/nuget package whose {@link
+ * PackageOriginConfig} was set under its as-typed spelling (case, or pypi's {@code -}/{@code _}/
+ * {@code .} variants) would union into a second, spelling-distinct entry alongside the same
+ * package's real, sidecar-normalized one, and the config lookup keyed by the normalized
+ * spelling would never find a config stored under the raw one, silently falling back to the
+ * ALLOW/BLOCK defaults instead of what was actually configured. Resolving every origin config
+ * once here, into a map keyed by this same normalized coordinate, is what lets {@link
+ * #listPackages} read it correctly regardless of which spelling the config was originally set
+ * under.
+ */
+ private PackagesWithOriginConfigs allPackagesInRepository(String region, String domain, String domainOwner,
+ String owner, String repository, String requestedFormat) {
+ Set<PackageCoordinate> coordinates = new LinkedHashSet<>();
+ Map<PackageCoordinate, PackageOriginConfig> originConfigsByCoordinate = new LinkedHashMap<>();
+ String originPrefix = region + "::" + domain + "::" + repository + "::";
+ if (requestedFormat == null || "generic".equals(requestedFormat)) {
+ for (CodeArtifactPackageVersion pv : packageVersions.scanForAccount(owner,
+ k -> k.startsWith(originPrefix))) {
+ coordinates.add(new PackageCoordinate(pv.getFormat(), pv.getNamespace(),
+ accountingPackageName(pv.getFormat(), pv.getPackageName())));
+ }
+ }
+ for (PackageOriginConfig config : packageOriginConfigs.scanForAccount(owner,
+ k -> k.startsWith(originPrefix))) {
+ if (config.getFormat() != null
+ && (requestedFormat == null || requestedFormat.equals(config.getFormat()))) {
+ PackageCoordinate coordinate = new PackageCoordinate(config.getFormat(), config.getNamespace(),
+ accountingPackageName(config.getFormat(), config.getPackageName()));
+ coordinates.add(coordinate);
+ originConfigsByCoordinate.put(coordinate, config);
+ }
+ }
+ for (String format : CONTAINER_BACKED_FORMATS) {
+ if (requestedFormat != null && !requestedFormat.equals(format)) {
+ continue;
+ }
+ RepositorySidecarManager manager = sidecarRegistry.forFormat(format).orElse(null);
+ if (manager == null) {
+ continue;
+ }
+ String repoId = resolveContainerIdAndBeginUse(format, region, domain, domainOwner, repository, manager);
+ try {
+ for (RepositorySidecarManager.PackageIdentifier id : manager.listPackages(repoId, domain,
+ repository)) {
+ coordinates.add(new PackageCoordinate(format, id.namespace(),
+ accountingPackageName(format, id.packageName())));
+ }
+ } catch (UnsupportedOperationException e) {
+ throw new AwsException("InternalServerException", "Listing " + format + " packages is not yet "
+ + "supported by this emulator.", 500);
+ } finally {
+ manager.endUse(repoId);
+ }
+ // Raised by Greptile's review: disposing a container-backed package's last version
+ // deletes its real sidecar content, so the sidecar's own listing above no longer names
+ // it at all, but updatePackageVersionsStatus/disposePackageVersions leave a status
+ // override behind specifically so ListPackageVersions/DescribePackageVersion can keep
+ // answering Disposed/Archived for it. Without this, a package with no origin
+ // configuration either (nothing else to keep it in the union) would silently disappear
+ // from ListPackages the moment its last version was disposed. Only ever adds something
+ // for npm/pypi/nuget (see containerBackedOverriddenVersionStatusesByRepository's own
+ // javadoc for why its index is namespace-less only); the same gap for maven or a scoped
+ // npm package, both of which always carry a namespace, is not closed here.
+ for (String packageName : containerBackedOverriddenVersionStatusesByRepository(region, domain,
+ domainOwner, repository, format).keySet()) {
+ coordinates.add(new PackageCoordinate(format, null, packageName));
+ }
+ }
+ return new PackagesWithOriginConfigs(new ArrayList<>(coordinates), originConfigsByCoordinate);
+ }
+
+ /**
+ * Sorts the same way {@link #packageVersionSortCursor} does for its own action, as an opaque
+ * cursor: {@code ListPackages} has no {@code sortBy} parameter at all in the wire model (unlike
+ * {@code ListPackageVersions}' {@code PUBLISHED_TIME}), so alphabetical by format, then
+ * namespace, then package name is the best-justified deterministic order available, not an
+ * AWS-confirmed one.
+ *
+ * <p>Raised by Greptile's review: joining the three fields with a plain {@code ::} the way an
+ * earlier version of this method did collides for two different coordinates whose namespace
+ * and package name happen to straddle that same separator differently, e.g. namespace
+ * {@code a::b} with package {@code c} against namespace {@code a} with package {@code b::c};
+ * {@code Pagination}'s own resume logic (strictly-greater-cursor) would then skip whichever of
+ * the two sorts second. Each field is now escaped through {@link #escapeKeySegment}, the same
+ * helper this service's own storage keys already use for exactly this collision, before being
+ * joined.
+ */
+ private static String packageSortCursor(PackageSummaryEntry entry) {
+ return escapeKeySegment(entry.format()) + "::" + escapeKeySegment(entry.namespace()) + "::"
+ + escapeKeySegment(entry.packageName());
+ }
+
+ /**
+ * Lists every package a repository holds, across every format at once unless {@code format}
+ * filters to one. {@code namespace} is a prefix filter exactly like {@code packagePrefix} (its
+ * own doc: "although this option is called --namespace and not --namespace-prefix, it has
+ * prefix-matching behavior"), and real AWS rejects it when {@code format} is omitted ("Using
+ * --namespace by itself results in an error," the User Guide's own documented search-option
+ * combination rule): {@code format} combined with a namespace-less format (pypi, nuget) is
+ * still accepted, since nothing in that same rule singles it out, it just can never match
+ * anything, the same way a {@code packagePrefix} nothing matches returns empty rather than an
+ * error.
+ *
+ * <p>{@code publish}/{@code upstream} filter against each package's own resolved restriction
+ * (its {@link PackageOriginConfig} if one was ever set, else CodeArtifact's documented default
+ * of publish {@code ALLOW}, upstream {@code BLOCK}), the same resolution {@link
+ * #describePackage} already does for one known package.
+ *
+ * <p>Default page size is 100 (the User Guide's own stated default), not the 50 {@code
+ * ListPackageVersions} uses: a real, AWS-documented difference between the two actions, not
+ * an oversight copying one convention over the other.
+ */
+ public ListPackagesResult listPackages(String region, String domain, String domainOwner, String repository,
+ String format, String namespace, String packagePrefix, Integer maxResults, String nextToken,
+ String publish, String upstream) {
+ requireNonBlank(domain, "domain");
+ requireNonBlank(repository, "repository");
+ if (format != null && !PACKAGE_FORMATS.contains(format)) {
+ throw validation("format must be one of " + PACKAGE_FORMATS + ".");
+ }
+ if (namespace != null) {
+ if (format == null) {
+ throw validation("format must be specified when namespace is provided.");
+ }
+ validatePackageToken("namespace", namespace);
+ }
+ if (packagePrefix != null) {
+ validatePackageToken("packagePrefix", packagePrefix);
+ }
+ if (publish != null && !ORIGIN_RESTRICTION_VALUES.contains(publish)) {
+ throw validation("publish must be one of " + ORIGIN_RESTRICTION_VALUES + ".");
+ }
+ if (upstream != null && !ORIGIN_RESTRICTION_VALUES.contains(upstream)) {
+ throw validation("upstream must be one of " + ORIGIN_RESTRICTION_VALUES + ".");
+ }
+ String owner = effectiveOwner(domainOwner);
+ requireRepository(owner, repositoryKey(region, domain, repository), repository);
+
+ List<PackageSummaryEntry> entries = new ArrayList<>();
+ PackagesWithOriginConfigs packages = allPackagesInRepository(region, domain, domainOwner, owner, repository,
+ format);
+ for (PackageCoordinate coordinate : packages.coordinates()) {
+ if (namespace != null
+ && (coordinate.namespace() == null || !coordinate.namespace().startsWith(namespace))) {
+ continue;
+ }
+ if (packagePrefix != null && !coordinate.packageName().startsWith(packagePrefix)) {
+ continue;
+ }
+ // Resolved from the same normalized map allPackagesInRepository already built, rather
+ // than a fresh key lookup here: see that method's own javadoc for why a direct lookup
+ // keyed by this (already normalized) coordinate could otherwise miss a config stored
+ // under its caller's original, un-normalized spelling.
+ Optional<PackageOriginConfig> config = Optional.ofNullable(
+ packages.originConfigsByCoordinate().get(coordinate));
+ // Raised by Greptile's review: a PackageOriginConfig persisted before format/namespace/
+ // packageName existed on it (see that model's own javadoc) has none of the three, so
+ // allPackagesInRepository's own scan can never resolve it into this map at all, even
+ // though the package it configures is still found here through its real version record
+ // or sidecar listing. Falling back to the original key-based lookup, exactly like
+ // describePackage's own, is what keeps that package's real restriction (not the ALLOW/
+ // BLOCK default) resolving for it too.
+ if (config.isEmpty()) {
+ config = packageOriginConfigs.getForAccount(owner, packageKey(region, domain, repository,
+ coordinate.format(), coordinate.namespace(), coordinate.packageName()));
+ }
+ String publishRestriction = config.map(PackageOriginConfig::getPublishRestriction).orElse("ALLOW");
+ String upstreamRestriction = config.map(PackageOriginConfig::getUpstreamRestriction).orElse("BLOCK");
+ if (publish != null && !publish.equals(publishRestriction)) {
+ continue;
+ }
+ if (upstream != null && !upstream.equals(upstreamRestriction)) {
+ continue;
+ }
+ entries.add(new PackageSummaryEntry(coordinate.format(), coordinate.namespace(),
+ coordinate.packageName(), publishRestriction, upstreamRestriction));
+ }
+ PaginatedResult<PackageSummaryEntry> page = Pagination.paginate(entries,
+ CodeArtifactService::packageSortCursor, maxResults, nextToken, 100, 1000, "ValidationException");
+ return new ListPackagesResult(page.items(), page.nextToken());
+ }
+
/**
* Floci never ingests packages from an upstream source, so every package it holds was first
* published directly. That is the one case where CodeArtifact's default origin controls are
@@ -2421,7 +2642,7 @@ public class CodeArtifactService implements Resettable {
requireRepository(owner, repositoryKey(region, domain, repository), repository);
String key = packageKey(region, domain, repository, format, namespace, packageName);
packageOriginConfigs.putForAccount(owner, key, new PackageOriginConfig(publishRestriction,
- upstreamRestriction));
+ upstreamRestriction, format, namespace, packageName));
return new PackageDescription(format, namespace, packageName, publishRestriction, upstreamRestriction);
}
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClient.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClient.java
index 740de7d2e4..a2412bad37 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClient.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClient.java
@@ -470,6 +470,51 @@ public class PypiserverSidecarClient implements RepositorySidecarManager {
}
}
+ /**
+ * pypiserver's own root simple index (confirmed live: {@code GET /simple/} with no project
+ * name lists every registered project, each as an {@code <a href="project-name/">} entry,
+ * the same href shape {@link #SIMPLE_INDEX_HREF} already parses for one project's own files).
+ * Each href is already the PEP 503-normalized name real pypiserver registers it under
+ * (confirmed: it never shows the originally-published casing/separators), matching what
+ * {@code CodeArtifactService}'s own {@code accountingPackageName} already normalizes pypi
+ * names to, so no further normalization is needed here. PyPI packages have no namespace.
+ *
+ * <p>Raised by hectorvent's review: every CodeArtifact repository gets a pypi sidecar id at
+ * creation regardless of whether it is ever actually used through pypi, so without checking
+ * {@link PypiserverSidecarManager#isRunning} first, a {@code ListPackages} call with no format
+ * filter would start this repository's own pypiserver container just to find it empty, every
+ * time, for a repository that has never published anything through pypi.
+ */
+ @Override
+ public List<PackageIdentifier> listPackages(String repositoryContainerId, String domain, String repository) {
+ if (!manager.isRunning(repositoryContainerId)) {
+ return List.of();
+ }
+ String baseUrl = ensureReady(repositoryContainerId, null);
+ URI uri = SidecarUriUtils.combine(URI.create(baseUrl), "/simple/");
+ HttpRequest request = HttpRequest.newBuilder(uri).timeout(Duration.ofSeconds(10)).GET().build();
+ HttpResponse<String> response;
+ try {
+ response = httpClient.send(request, BodyHandlers.ofString());
+ } catch (Exception e) {
+ throw new IllegalStateException("Could not reach the pypiserver sidecar to list packages", e);
+ }
+ if (response.statusCode() == 404) {
+ return List.of();
+ }
+ if (response.statusCode() != 200) {
+ throw new IllegalStateException("Could not list packages on the pypiserver sidecar: upstream returned "
+ + response.statusCode());
+ }
+ List<PackageIdentifier> result = new ArrayList<>();
+ SIMPLE_INDEX_HREF.matcher(response.body()).results().forEach(match -> {
+ String href = match.group(1);
+ String name = href.endsWith("/") ? href.substring(0, href.length() - 1) : href;
+ result.add(new PackageIdentifier(null, name));
+ });
+ return result;
+ }
+
private Optional<String> simpleIndex(String baseUrl, String packageName) {
String normalized = normalizePackageName(packageName);
URI uri = SidecarUriUtils.combine(URI.create(baseUrl), "/simple/" + SidecarUriUtils.encodeSegment(normalized)
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.java
index 617253639b..b0b43f4063 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarManager.java
@@ -65,6 +65,18 @@ public class PypiserverSidecarManager implements ContainerTeardown {
pool.stopContainer(pypiRepositoryId);
}
+ /**
+ * Whether this pypi repository's own pypiserver container is already running, without starting
+ * one for a repository that has never used pypi at all. Every CodeArtifact repository gets a
+ * pypi sidecar id at creation regardless of whether it is ever actually used through pypi (see
+ * {@code CodeArtifactService#ensureFormatContainerId}'s own callers), so a caller that only
+ * wants to ask an already-running container what it holds must check this first rather than
+ * provision a brand-new container just to find it has nothing.
+ */
+ public boolean isRunning(String pypiRepositoryId) {
+ return pool.isTracked(pypiRepositoryId);
+ }
+
private PerKeyContainerPool.StartedContainer startContainer(String pypiRepositoryId) {
String image = config.services().codeartifact().pypiImage();
String containerName = ContainerStorageHelper.dockerName(config, "floci-pypiserver-" + pypiRepositoryId);
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClient.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClient.java
index ceaf912e9b..a26ba6ec9b 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClient.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClient.java
@@ -328,6 +328,152 @@ public class ReposiliteSidecarClient implements RepositorySidecarManager {
packageName, version))));
}
+ /**
+ * One {@code /api/maven/details} response, split into what {@link #walkMavenDirectory} needs:
+ * which file names this directory holds directly, and which entries are subdirectories to
+ * possibly recurse into.
+ */
+ private record MavenDirectoryListing(List<String> fileNames, List<String> subdirectories) {}
+
+ /**
+ * Reposilite has no single list-everything endpoint (confirmed live: only the same one-level
+ * {@code /api/maven/details} directory listing {@link #listPackageVersionAssetNames}/{@link
+ * #listPackageVersions} already read one known directory at a time), so this walks the
+ * repository's directory tree from its root, recursing into {@code DIRECTORY} entries.
+ *
+ * <p>A plain groupId segment can never be told apart from the group/artifact directory itself
+ * by its own shape alone: both hold nothing but further {@code DIRECTORY} entries (confirmed
+ * live: {@code spike}'s own listing, one {@code DIRECTORY} entry named {@code 1.0.0}, looks
+ * identical in shape to {@code com}'s own listing, one {@code DIRECTORY} entry named {@code
+ * example}). What tells them apart is one level deeper: a version directory's own children are
+ * never further directories, only real files (or, after a {@code DisposePackageVersions}, none
+ * at all); a plain groupId segment's children always are.
+ *
+ * <p>Raised by Greptile's review: a real Maven coordinate layout can have both at the very same
+ * directory, since nothing stops one groupId from being a literal prefix of another's own
+ * segments (AWS's own worked example shape: {@code com:foo:1.0.0} and {@code com.foo:bar:1.0.0}
+ * share the real directory {@code com/foo}, one holding both a version directory {@code 1.0.0}
+ * and a further groupId segment {@code bar}). Peeking only one representative child, as an
+ * earlier version of this method did, answered only "is the current directory package-shaped
+ * or group-shaped", a false either/or that depended on which child happened to be peeked (or on
+ * {@code maven-metadata.xml}'s presence short-circuiting before any sibling was ever looked at
+ * either), silently dropping whichever of the two coordinates it didn't settle on. {@link
+ * #walkMavenDirectory} now checks every child of the current directory, not just one: a
+ * version-shaped child (no further directories) proves the current directory is itself a
+ * package, recorded at most once regardless of how many such children it has; a group-shaped
+ * child is recursed into as a further namespace segment, every time, independent of whether the
+ * current directory was already recorded. {@code maven-metadata.xml} still short-circuits
+ * straight to a record (covering a package {@link #recordVersionAvailable} was called for that
+ * happens to have zero version directories of its own, e.g. a {@code CopyPackageVersions}
+ * source whose every version failed), but no longer skips the sibling-child walk below it.
+ *
+ * <p>Raised by hectorvent's review: every CodeArtifact repository gets a maven sidecar id at
+ * creation regardless of whether it is ever actually used through maven, so without checking
+ * {@link ReposiliteSidecarManager#isStarted} first, exactly like {@link #releaseRepository}
+ * already does for the same reason, a {@code ListPackages} call with no format filter would
+ * start the one shared Reposilite container just to walk it for a repository that has never
+ * published anything through maven.
+ */
+ @Override
+ public List<PackageIdentifier> listPackages(String repositoryContainerId, String domain, String repository) {
+ if (!manager.isStarted()) {
+ return List.of();
+ }
+ String baseUrl = manager.ensureReady();
+ List<PackageIdentifier> result = new ArrayList<>();
+ walkMavenDirectory(baseUrl, "/api/maven/details/" + SidecarUriUtils.encodeSegment(repositoryContainerId)
+ + "/", List.of(), result);
+ return result;
+ }
+
+ private void walkMavenDirectory(String baseUrl, String detailsPath, List<String> pathSegments,
+ List<PackageIdentifier> result) {
+ MavenDirectoryListing listing = fetchMavenDirectoryListing(baseUrl, detailsPath);
+ if (listing == null) {
+ return;
+ }
+ boolean recordedAsPackage = false;
+ if (listing.fileNames().contains("maven-metadata.xml")) {
+ recordMavenPackage(pathSegments, result);
+ recordedAsPackage = true;
+ }
+ for (String subdirectory : listing.subdirectories()) {
+ String childPath = detailsPath + SidecarUriUtils.encodeSegment(subdirectory) + "/";
+ MavenDirectoryListing childListing = fetchMavenDirectoryListing(baseUrl, childPath);
+ if (childListing == null) {
+ continue;
+ }
+ // Raised by Greptile's review: a real Maven snapshot version directory also holds its
+ // own maven-metadata.xml (listing that version's timestamped builds), alongside its
+ // real artifact files (jar, pom, checksums), so "has maven-metadata.xml" alone cannot
+ // tell that apart from the artifact-level-metadata-only directory the check below
+ // exists for (one with no version directories of its own and, this time, no other real
+ // files alongside its metadata.xml either). Only the latter shape, a directory whose
+ // files are entirely maven-metadata.xml and its checksum sidecars and nothing else, is
+ // excluded from being treated as version-shaped; a snapshot version directory's own
+ // real artifact files keep it version-shaped exactly like any other.
+ boolean isArtifactMetadataOnly = childListing.fileNames().contains("maven-metadata.xml")
+ && childListing.fileNames().stream().allMatch(name -> name.equals("maven-metadata.xml")
+ || name.startsWith("maven-metadata.xml."));
+ if (childListing.subdirectories().isEmpty() && !isArtifactMetadataOnly) {
+ // This child holds no further directories of its own (real asset files, or none at
+ // all post-dispose) and is not artifact-metadata-only either: it is version-shaped,
+ // proving the current directory is (also) the group/artifact level itself,
+ // regardless of what its other children look like. An artifact-metadata-only child
+ // (an artifact-level directory with zero version directories of its own, see
+ // recordVersionAvailable's own javadoc, the exact shape a native PUT straight to
+ // that file produces) is never version-shaped, no matter how empty it otherwise
+ // looks; falling into the recurse below instead is what lets that child record
+ // itself through this same method's own metadata.xml short-circuit above, rather
+ // than this directory being wrongly recorded as the package in its place.
+ if (!recordedAsPackage) {
+ recordMavenPackage(pathSegments, result);
+ recordedAsPackage = true;
+ }
+ continue;
+ }
+ List<String> nextSegments = new ArrayList<>(pathSegments);
+ nextSegments.add(subdirectory);
+ walkMavenDirectory(baseUrl, childPath, nextSegments, result);
+ }
+ }
+
+ private void recordMavenPackage(List<String> pathSegments, List<PackageIdentifier> result) {
+ if (pathSegments.size() >= 2) {
+ String packageName = pathSegments.get(pathSegments.size() - 1);
+ String namespace = String.join(".", pathSegments.subList(0, pathSegments.size() - 1));
+ result.add(new PackageIdentifier(namespace, packageName));
+ }
+ }
+
+ private MavenDirectoryListing fetchMavenDirectoryListing(String baseUrl, String detailsPath) {
+ HttpRequest request = HttpRequest.newBuilder(SidecarUriUtils.combine(URI.create(baseUrl), detailsPath))
+ .timeout(Duration.ofSeconds(30))
+ .header("Authorization", manager.basicAuthHeader())
+ .GET()
+ .build();
+ HttpResponse<String> response = send(request, BodyHandlers.ofString());
+ if (response.statusCode() == 404) {
+ return null;
+ }
+ if (response.statusCode() != 200) {
+ throw new IllegalStateException("Could not list Maven packages on the Reposilite sidecar: upstream "
+ + "returned " + response.statusCode());
+ }
+ List<String> fileNames = new ArrayList<>();
+ List<String> subdirectories = new ArrayList<>();
+ for (JsonNode entry : readTree(response.body()).path("files")) {
+ String type = entry.path("type").asText();
+ String name = entry.path("name").asText();
+ if ("FILE".equals(type)) {
+ fileNames.add(name);
+ } else if ("DIRECTORY".equals(type)) {
+ subdirectories.add(name);
+ }
+ }
+ return new MavenDirectoryListing(fileNames, subdirectories);
+ }
+
private Instant latestFileTimestamp(String baseUrl, String versionDetailsPath) {
HttpRequest request = HttpRequest.newBuilder(SidecarUriUtils.combine(URI.create(baseUrl), versionDetailsPath))
.timeout(Duration.ofSeconds(30))
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/RepositorySidecarManager.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/RepositorySidecarManager.java
index dec946a69b..6c2f0ba2d5 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/RepositorySidecarManager.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/RepositorySidecarManager.java
@@ -217,6 +217,29 @@ public interface RepositorySidecarManager {
+ "sidecar");
}
+ /**
+ * One package this sidecar holds, identified the same way {@link #packageExists} is:
+ * {@code namespace} is {@code null} for a format or package that has none.
+ */
+ record PackageIdentifier(String namespace, String packageName) {}
+
+ /**
+ * Every package this sidecar holds at least one real version of, for bridging {@code
+ * ListPackages} when there is no generic-format record to list packages from. Unlike {@link
+ * #listPackageVersions}, which already knows which one package to ask about, this has no
+ * known starting point at all: each implementation needs its own new, format-specific
+ * repository-wide enumeration, confirmed live against each real sidecar protocol rather than
+ * assumed (Verdaccio's {@code /-/all}, pypiserver's root simple index, BaGetter's Search
+ * resource pressed into service for this admin-only purpose, Reposilite's own directory
+ * listing walked recursively since it has no single list-everything endpoint). The default
+ * throws {@link UnsupportedOperationException}, the same convention {@link #deletePackage}
+ * already uses for a capability not every sidecar backs; every container-backed format's own
+ * client overrides this.
+ */
+ default List<PackageIdentifier> listPackages(String repositoryContainerId, String domain, String repository) {
+ throw new UnsupportedOperationException(format() + " packages cannot be listed through this sidecar");
+ }
+
/**
* Marks that {@code repositoryContainerId} is about to be used for {@link
* #listPackageVersions}, called by {@code CodeArtifactService} right after it resolves that
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClient.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClient.java
index 5939563930..755e73623b 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClient.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClient.java
@@ -153,6 +153,58 @@ public class VerdaccioSidecarClient implements RepositorySidecarManager {
return result;
}
+ /**
+ * Verdaccio's own {@code /-/all} endpoint (confirmed live), scoped to this repository's own
+ * backend the same way every other call here is (one {@code baseUrl} per repository): its
+ * response object's keys are every package's own real path exactly as {@link #packagePath}
+ * already builds it ({@code @scope/name} or a bare {@code name}), plus one non-package key,
+ * {@code _updated}, filtered out here rather than treated as a package named "_updated".
+ *
+ * <p>Raised by hectorvent's review: every CodeArtifact repository gets an npm sidecar id at
+ * creation regardless of whether it is ever actually used through npm, so without checking
+ * {@link VerdaccioSidecarManager#isRunning} first, a {@code ListPackages} call with no format
+ * filter would start this repository's own Verdaccio container just to find it empty, every
+ * time, for a repository that has never published anything through npm.
+ */
+ @Override
+ public List<PackageIdentifier> listPackages(String repositoryContainerId, String domain, String repository) {
+ if (!manager.isRunning(repositoryContainerId)) {
+ return List.of();
+ }
+ String baseUrl = ensureReady(repositoryContainerId, publicUrl(domain, repository));
+ HttpRequest request = HttpRequest.newBuilder(SidecarUriUtils.combine(URI.create(baseUrl), "/-/all"))
+ .timeout(Duration.ofSeconds(30)).GET().build();
+ HttpResponse<String> response;
+ try {
+ response = httpClient.send(request, BodyHandlers.ofString());
+ } catch (Exception e) {
+ throw new IllegalStateException("Could not reach the Verdaccio sidecar to list packages", e);
+ }
+ if (response.statusCode() != 200) {
+ throw new IllegalStateException("Could not list packages on the Verdaccio sidecar: upstream returned "
+ + response.statusCode());
+ }
+ JsonNode all;
+ try {
+ all = mapper.readTree(response.body());
+ } catch (Exception e) {
+ throw new IllegalStateException("Could not parse Verdaccio's package listing", e);
+ }
+ List<PackageIdentifier> result = new ArrayList<>();
+ all.fieldNames().forEachRemaining(key -> {
+ if ("_updated".equals(key)) {
+ return;
+ }
+ if (key.startsWith("@")) {
+ int slash = key.indexOf('/');
+ result.add(new PackageIdentifier(key.substring(1, slash), key.substring(slash + 1)));
+ } else {
+ result.add(new PackageIdentifier(null, key));
+ }
+ });
+ return result;
+ }
+
/**
* npm's own {@code dist-tags.latest}, AWS's documented preference for npm's {@code
* defaultDisplayVersion} ahead of the most-recently-published fallback every other format
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarManager.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarManager.java
index 4471ce3604..8ba2382fc0 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarManager.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarManager.java
@@ -82,6 +82,18 @@ public class VerdaccioSidecarManager implements ContainerTeardown {
pool.stopContainer(npmRepositoryId);
}
+ /**
+ * Whether this npm repository's own Verdaccio container is already running, without starting
+ * one for a repository that has never used npm at all. Every CodeArtifact repository gets an
+ * npm sidecar id at creation regardless of whether it is ever actually used through npm (see
+ * {@code CodeArtifactService#ensureFormatContainerId}'s own callers), so a caller that only
+ * wants to ask an already-running container what it holds must check this first rather than
+ * provision a brand-new container just to find it has nothing.
+ */
+ public boolean isRunning(String npmRepositoryId) {
+ return pool.isTracked(npmRepositoryId);
+ }
+
private PerKeyContainerPool.StartedContainer startContainer(String npmRepositoryId, String publicUrl) {
String image = config.services().codeartifact().npmImage();
String containerName = ContainerStorageHelper.dockerName(config, "floci-verdaccio-" + npmRepositoryId);
diff --git a/src/main/java/io/github/hectorvent/floci/services/codeartifact/model/PackageOriginConfig.java b/src/main/java/io/github/hectorvent/floci/services/codeartifact/model/PackageOriginConfig.java
index f41ee1a37c..8542872056 100644
--- a/src/main/java/io/github/hectorvent/floci/services/codeartifact/model/PackageOriginConfig.java
+++ b/src/main/java/io/github/hectorvent/floci/services/codeartifact/model/PackageOriginConfig.java
@@ -9,19 +9,38 @@ import io.quarkus.runtime.annotations.RegisterForReflection;
* configuration, it is just CodeArtifact's documented default (publish ALLOW, upstream BLOCK);
* see {@code CodeArtifactService.PackageDescription}'s own Javadoc for why that default holds
* for every package Floci serves until a caller overrides it.
+ *
+ * <p>{@code format}/{@code namespace}/{@code packageName} are carried on the value here, not only
+ * encoded into the storage key that names it, for the same reason {@code
+ * CodeArtifactPackageVersion} already carries its own: {@code ListPackages} needs to enumerate
+ * every package a whole repository holds, including one that only ever got an origin
+ * configuration and no real version, and the key itself is escaped specifically to make that
+ * lossy to recover (see {@code CodeArtifactService#escapeKeySegment}'s own javadoc). Reading
+ * these fields back off the scanned value instead, the same way {@code
+ * CodeArtifactService#genericPackageVersions} already reads a version record's own fields rather
+ * than parsing its key, avoids that entirely. {@code null} for a record written before this field
+ * existed; a package that old is simply left out of that one enumeration, the same accepted gap
+ * {@code GetPackageVersionReadme}'s own pre-existing-field limitation already documents elsewhere.
*/
@RegisterForReflection
@JsonIgnoreProperties(ignoreUnknown = true)
public class PackageOriginConfig {
private String publishRestriction;
private String upstreamRestriction;
+ private String format;
+ private String namespace;
+ private String packageName;
public PackageOriginConfig() {
}
- public PackageOriginConfig(String publishRestriction, String upstreamRestriction) {
+ public PackageOriginConfig(String publishRestriction, String upstreamRestriction, String format,
+ String namespace, String packageName) {
this.publishRestriction = publishRestriction;
this.upstreamRestriction = upstreamRestriction;
+ this.format = format;
+ this.namespace = namespace;
+ this.packageName = packageName;
}
public String getPublishRestriction() {
@@ -39,4 +58,28 @@ public class PackageOriginConfig {
public void setUpstreamRestriction(String upstreamRestriction) {
this.upstreamRestriction = upstreamRestriction;
}
+
+ public String getFormat() {
+ return format;
+ }
+
+ public void setFormat(String format) {
+ this.format = format;
+ }
+
+ public String getNamespace() {
+ return namespace;
+ }
+
+ public void setNamespace(String namespace) {
+ this.namespace = namespace;
+ }
+
+ public String getPackageName() {
+ return packageName;
+ }
+
+ public void setPackageName(String packageName) {
+ this.packageName = packageName;
+ }
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClientTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClientTest.java
index 3ba8da7cf2..636af91416 100644
--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClientTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/BagetterSidecarClientTest.java
@@ -19,6 +19,7 @@ import java.util.List;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.concurrent.atomic.AtomicReference;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
@@ -341,6 +342,53 @@ class BagetterSidecarClientTest {
});
}
+ /**
+ * Raised by Greptile's review: the real NuGet V3 Search resource excludes prerelease versions
+ * whenever {@code prerelease} is omitted, so a package whose only published version is a
+ * prerelease would never come back from {@link BagetterSidecarClient#listPackages} without it.
+ */
+ @Test
+ void listPackagesIncludesAPackageWhoseOnlyVersionIsAPrerelease() throws Exception {
+ BagetterSidecarManager manager = manager();
+ seedPooledContainer(manager, "nuget-repo-1", "tracked-bagetter-1", backendUrl);
+ AtomicReference<String> requestedQuery = new AtomicReference<>();
+ backend.createContext("/v3/search", exchange -> {
+ String query = exchange.getRequestURI().getQuery();
+ requestedQuery.set(query);
+ boolean firstPage = query.contains("skip=0");
+ byte[] body = (firstPage ? "{\"data\":[{\"id\":\"demo-pkg\"}]}" : "{\"data\":[]}")
+ .getBytes(StandardCharsets.UTF_8);
+ exchange.getResponseHeaders().set("Content-Type", "application/json");
+ exchange.sendResponseHeaders(200, body.length);
+ exchange.getResponseBody().write(body);
+ exchange.close();
+ });
+
+ List<RepositorySidecarManager.PackageIdentifier> packages = client(manager).listPackages("nuget-repo-1",
+ "dom", "repo");
+
+ assertEquals(1, packages.size());
+ assertEquals("demo-pkg", packages.get(0).packageName());
+ assertTrue(requestedQuery.get().contains("prerelease=true"));
+ }
+
+ /**
+ * Raised by hectorvent's review: every CodeArtifact repository gets a nuget sidecar id at
+ * creation regardless of whether it is ever actually used through nuget, so {@code
+ * ListPackages} with no format filter must not start this repository's own BaGetter
+ * container just to find out it was never used. No context is registered on {@code backend}
+ * at all here: a real container start would have nothing to connect to and throw.
+ */
+ @Test
+ void listPackagesReturnsEmptyWithoutStartingAContainerThatWasNeverUsed() {
+ BagetterSidecarManager manager = manager();
+
+ List<RepositorySidecarManager.PackageIdentifier> packages = client(manager).listPackages("nuget-repo-1",
+ "dom", "repo");
+
+ assertTrue(packages.isEmpty());
+ }
+
private BagetterSidecarManager manager() {
return new BagetterSidecarManager(containerBuilder, lifecycleManager, config);
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenDockerIntegrationTest.java
index 7ab3b714ed..d5e40ce136 100644
--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenDockerIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactMavenDockerIntegrationTest.java
@@ -669,6 +669,32 @@ class CodeArtifactMavenDockerIntegrationTest {
.body("versions[0].status", equalTo("Disposed"));
}
+ /**
+ * A throwaway groupId/artifactId, not {@link #GAV}: {@code ListPackages} walks Reposilite's
+ * whole directory tree repository-wide, so publishing its own package here (rather than
+ * reusing one another test already created) keeps this test's own expected set independent of
+ * whatever else this class's shared repository happens to hold by the time it runs.
+ */
+ @Test
+ @Order(24)
+ void listPackagesWalksTheRealReposiliteDirectoryTreeToFindTheGroupAndArtifact() {
+ byte[] content = "list-packages-bytes".getBytes(StandardCharsets.UTF_8);
+ given().header("Authorization", "Bearer " + bearerToken).body(content)
+ .put("/codeartifact/maven/" + DOMAIN + "/" + REPO
+ + "/com/example/nested/listpkgsprobe/1.0.0/listpkgsprobe-1.0.0.jar")
+ .then().statusCode(200);
+
+ JsonPath body = given().header("Authorization", AUTH)
+ .post("/v1/packages?domain=" + DOMAIN + "&repository=" + REPO + "&format=maven"
+ + "&namespace=com.example.nested")
+ .then().statusCode(200)
+ .extract().jsonPath();
+ List<String> names = body.getList("packages.package", String.class);
+ List<String> namespaces = body.getList("packages.namespace", String.class);
+ assertTrue(names.contains("listpkgsprobe"), "expected listpkgsprobe among " + names);
+ assertEquals("com.example.nested", namespaces.get(names.indexOf("listpkgsprobe")));
+ }
+
/**
* A throwaway groupId/artifactId, not {@link #GAV}: deleting it exercises Reposilite's real
* directory delete (confirmed live before this was built: a single {@code DELETE} on the
diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNpmDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNpmDockerIntegrationTest.java
index 0a19f9dae6..33699d3964 100644
--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNpmDockerIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNpmDockerIntegrationTest.java
@@ -22,6 +22,7 @@ import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
+import java.util.List;
import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
@@ -789,6 +790,40 @@ class CodeArtifactNpmDockerIntegrationTest {
.body("versions[0].origin.originType", equalTo("INTERNAL"));
}
+ /**
+ * Bridges to Verdaccio's own {@code /-/all} endpoint, confirmed live: besides {@link
+ * #PACKAGE_NAME}, already published and confirmed present by {@link
+ * #listPackageVersionsBridgesToTheRealVerdaccioSidecar} above, this also publishes a fresh
+ * scoped package of its own to confirm the {@code @scope/name} key shape {@code
+ * VerdaccioSidecarClient#listPackages} parses it into a namespace/package pair really is what
+ * a live Verdaccio container returns, not just what its own unit test assumes.
+ */
+ @Test
+ @Order(22)
+ void listPackagesBridgesToTheRealVerdaccioAllEndpoint() {
+ String scopedPackage = "listpkgsprobe";
+ byte[] tarball = "list-packages-tarball".getBytes(StandardCharsets.UTF_8);
+ String envelope = "{\"name\":\"@myscope/" + scopedPackage + "\",\"versions\":{\"1.0.0\":{\"name\":"
+ + "\"@myscope/" + scopedPackage + "\",\"version\":\"1.0.0\",\"dist\":{\"tarball\":"
+ + "\"http://ignored/" + scopedPackage + "-1.0.0.tgz\"}}},\"_attachments\":{\"" + scopedPackage
+ + "-1.0.0.tgz\":{\"content_type\":\"application/octet-stream\",\"data\":\""
+ + Base64.getEncoder().encodeToString(tarball) + "\",\"length\":" + tarball.length + "}}}";
+ given().urlEncodingEnabled(false).header("Authorization", "Bearer " + bearerToken)
+ .contentType("application/json").body(envelope)
+ .put("/codeartifact/npm/" + DOMAIN + "/" + REPO + "/@myscope%2F" + scopedPackage)
+ .then().statusCode(anyOf201Or200());
+
+ JsonPath body = given().header("Authorization", AUTH)
+ .post("/v1/packages?domain=" + DOMAIN + "&repository=" + REPO + "&format=npm")
+ .then().statusCode(200)
+ .extract().jsonPath();
+ List<String> names = body.getList("packages.package", String.class);
+ List<String> namespaces = body.getList("packages.namespace", String.class);
+ assertTrue(names.contains(PACKAGE_NAME), "expected " + PACKAGE_NAME + " among " + names);
+ assertTrue(names.contains(scopedPackage), "expected " + scopedPackage + " among " + names);
+ assertEquals("myscope", namespaces.get(names.indexOf(scopedPackage)));
+ }
+
/**
* Proves the actual mechanism, not just that {@code VerdaccioSidecarManager} has a method
* named right: {@code ContainerTeardowns.stopAll} runs every {@code ContainerTeardown} on
diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNuGetDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNuGetDockerIntegrationTest.java
index 0fbfa3f147..1b352ebd3f 100644
--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNuGetDockerIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactNuGetDockerIntegrationTest.java
@@ -14,6 +14,7 @@ import org.junit.jupiter.api.TestMethodOrder;
import java.io.ByteArrayOutputStream;
import java.nio.charset.StandardCharsets;
+import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.zip.ZipEntry;
@@ -25,6 +26,7 @@ import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.startsWith;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* Runs a real NuGet V3 push/install round trip against a live, per-repository BaGetter container,
@@ -482,6 +484,35 @@ class CodeArtifactNuGetDockerIntegrationTest {
.body("versions[0].origin.originType", equalTo("INTERNAL"));
}
+ /**
+ * Bridges to BaGetter's own NuGet V3 Search resource, confirmed live: {@code
+ * BagetterSidecarClient#listPackages} presses it into service purely for this admin listing
+ * (this class's own class-level documentation explains why the native NuGet client protocol
+ * never gets a Search endpoint of its own). A mixed-case id, confirmed back lowercased, since
+ * a live search response returns the id real-cased exactly as pushed, and {@code
+ * CodeArtifactService}'s own {@code accountingPackageName} normalizes every NuGet name to
+ * lowercase (AWS's own documented NuGet normalization).
+ */
+ @Test
+ @Order(17)
+ void listPackagesBridgesToTheRealBagetterSearchResource() {
+ String packageId = "ListPackages.Probe";
+ byte[] nupkg = buildNupkg(packageId, VERSION, "list packages description", null, null);
+ given().header("Authorization", "Bearer " + bearerToken)
+ .contentType("multipart/form-data; boundary=boundary123")
+ .body(pushBody("boundary123", nupkg))
+ .put("/codeartifact/nuget/" + DOMAIN + "/" + REPO + "/api/v2/package")
+ .then().statusCode(201);
+
+ JsonPath body = given().header("Authorization", AUTH)
+ .post("/v1/packages?domain=" + DOMAIN + "&repository=" + REPO + "&format=nuget")
+ .then().statusCode(200)
+ .extract().jsonPath();
+ List<String> names = body.getList("packages.package", String.class);
+ assertTrue(names.contains(packageId.toLowerCase(Locale.ROOT)), "expected "
+ + packageId.toLowerCase(Locale.ROOT) + " among " + names);
+ }
+
private static long describeDomainAssetSizeBytes() {
return given().header("Authorization", AUTH)
.get("/v1/domain?domain=" + DOMAIN)
diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.java
index 2b787b9936..bf1ed139f6 100644
--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactPypiDockerIntegrationTest.java
@@ -771,6 +771,33 @@ class CodeArtifactPypiDockerIntegrationTest {
+ defaultDisplayVersion);
}
+ /**
+ * Bridges to pypiserver's own root {@code /simple/} index, confirmed live: uploads a package
+ * with a mixed-case, dotted name to confirm {@code PypiserverSidecarClient#listPackages}'s own
+ * href parsing really gets back the PEP 503-normalized project name a live pypiserver root
+ * index reports, not the originally-uploaded spelling, matching what {@code
+ * CodeArtifactService}'s own {@code accountingPackageName} already normalizes pypi names to.
+ */
+ @Test
+ @Order(22)
+ void listPackagesBridgesToTheRealPypiserverRootSimpleIndex() {
+ String packageName = "List.Packages_Probe";
+ String filename = "list_packages_probe-1.0.0.tar.gz";
+ byte[] content = "list-packages-probe-bytes".getBytes(StandardCharsets.UTF_8);
+ given().header("Authorization", "Bearer " + bearerToken)
+ .contentType("multipart/form-data; boundary=boundary123")
+ .body(twineUploadBody("boundary123", packageName, filename, content))
+ .post("/codeartifact/pypi/" + DOMAIN + "/" + REPO + "/")
+ .then().statusCode(200);
+
+ JsonPath body = given().header("Authorization", AUTH)
+ .post("/v1/packages?domain=" + DOMAIN + "&repository=" + REPO + "&format=pypi")
+ .then().statusCode(200)
+ .extract().jsonPath();
+ List<String> names = body.getList("packages.package", String.class);
+ assertTrue(names.contains("list-packages-probe"), "expected list-packages-probe among " + names);
+ }
+
/**
* Proves the actual mechanism, not just that {@code PypiserverSidecarManager} has a method
* named right: {@code ContainerTeardowns.stopAll} runs every {@code ContainerTeardown} on
diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.java
index d05d19c4f8..e3a22e1e27 100644
--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/CodeArtifactServiceTest.java
@@ -1864,7 +1864,8 @@ class CodeArtifactServiceTest {
service.createDomain(REGION, "dom", null, Map.of());
service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
String legacyRawKey = REGION + "::dom::repo::generic::a\\cb::x";
- packageOriginConfigStore.putForAccount(ACCOUNT_ID, legacyRawKey, new PackageOriginConfig("BLOCK", "BLOCK"));
+ packageOriginConfigStore.putForAccount(ACCOUNT_ID, legacyRawKey,
+ new PackageOriginConfig("BLOCK", "BLOCK", null, null, null));
service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "generic", "a:b", "x", "ALLOW", "ALLOW");
@@ -5003,6 +5004,283 @@ class CodeArtifactServiceTest {
assertEquals(content.length, pinned.asset().getSize());
}
+ // ----------------------------------------------------------------- ListPackages
+
+ @Test
+ void listPackagesReturnsEmptyForARepositoryWithNothingPublished() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+
+ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null,
+ null, null, null, null, null, null);
+
+ assertTrue(result.packages().isEmpty());
+ assertNull(result.nextToken());
+ }
+
+ @Test
+ void listPackagesReturnsEveryGenericPackageInTheRepositoryWithDefaultOriginRestrictions() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ publishGeneric("repo", "my-pkg", "1.0.0", "a".getBytes(StandardCharsets.UTF_8));
+ publishGeneric("repo", "other-pkg", "1.0.0", "b".getBytes(StandardCharsets.UTF_8));
+
+ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null,
+ null, null, null, null, null, null);
+
+ Set<String> names = result.packages().stream().map(CodeArtifactService.PackageSummaryEntry::packageName)
+ .collect(Collectors.toSet());
+ assertEquals(Set.of("my-pkg", "other-pkg"), names);
+ result.packages().forEach(p -> {
+ assertEquals("generic", p.format());
+ assertEquals("ns", p.namespace());
+ assertEquals("ALLOW", p.publishRestriction());
+ assertEquals("BLOCK", p.upstreamRestriction());
+ });
+ }
+
+ @Test
+ void listPackagesIncludesAnOriginConfigOnlyPackageWithNoRealVersion() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "generic", "ns", "never-published",
+ "BLOCK", "ALLOW");
+
+ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null,
+ null, null, null, null, null, null);
+
+ assertEquals(1, result.packages().size());
+ CodeArtifactService.PackageSummaryEntry entry = result.packages().get(0);
+ assertEquals("never-published", entry.packageName());
+ assertEquals("BLOCK", entry.publishRestriction());
+ assertEquals("ALLOW", entry.upstreamRestriction());
+ }
+
+ @Test
+ void listPackagesRoutesEveryContainerBackedFormatToTheSidecarBridge() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ CodeArtifactRepository created = service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ when(reposiliteClient.listPackages(eq(created.getSidecarContainerIds().get("maven")), eq("dom"),
+ eq("repo"))).thenReturn(List.of(
+ new RepositorySidecarManager.PackageIdentifier("com.example", "a")));
+ when(verdaccioClient.listPackages(eq(created.getSidecarContainerIds().get("npm")), eq("dom"), eq("repo")))
+ .thenReturn(List.of(new RepositorySidecarManager.PackageIdentifier(null, "my-pkg")));
+ when(pypiserverClient.listPackages(eq(created.getSidecarContainerIds().get("pypi")), eq("dom"), eq("repo")))
+ .thenReturn(List.of(new RepositorySidecarManager.PackageIdentifier(null, "my-pkg")));
+ when(bagetterClient.listPackages(eq(created.getSidecarContainerIds().get("nuget")), eq("dom"), eq("repo")))
+ .thenReturn(List.of(new RepositorySidecarManager.PackageIdentifier(null, "my-pkg")));
+
+ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null,
+ null, null, null, null, null, null);
+
+ assertEquals(4, result.packages().size());
+ assertTrue(result.packages().stream().anyMatch(p -> "maven".equals(p.format()) && "a".equals(p.packageName())
+ && "com.example".equals(p.namespace())));
+ assertEquals(3, result.packages().stream().filter(p -> "my-pkg".equals(p.packageName())).count());
+ }
+
+ @Test
+ void listPackagesFormatFilterOnlyTouchesThatOneFormatsSidecar() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ CodeArtifactRepository created = service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ when(pypiserverClient.listPackages(eq(created.getSidecarContainerIds().get("pypi")), eq("dom"), eq("repo")))
+ .thenReturn(List.of(new RepositorySidecarManager.PackageIdentifier(null, "my-pkg")));
+
+ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", "pypi",
+ null, null, null, null, null, null);
+
+ assertEquals(1, result.packages().size());
+ assertEquals("pypi", result.packages().get(0).format());
+ verify(verdaccioClient, never()).listPackages(anyString(), anyString(), anyString());
+ verify(bagetterClient, never()).listPackages(anyString(), anyString(), anyString());
+ verify(reposiliteClient, never()).listPackages(anyString(), anyString(), anyString());
+ }
+
+ @Test
+ void listPackagesNamespacePrefixFiltersAndRequiresFormat() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ publishGeneric("repo", "my-pkg", "1.0.0", "a".getBytes(StandardCharsets.UTF_8));
+
+ AwsException e = assertThrows(AwsException.class, () -> service.listPackages(REGION, "dom", null, "repo",
+ null, "n", null, null, null, null, null));
+ assertEquals("ValidationException", e.getErrorCode());
+
+ CodeArtifactService.ListPackagesResult matches = service.listPackages(REGION, "dom", null, "repo", "generic",
+ "n", null, null, null, null, null);
+ assertEquals(1, matches.packages().size());
+
+ CodeArtifactService.ListPackagesResult noMatches = service.listPackages(REGION, "dom", null, "repo",
+ "generic", "other", null, null, null, null, null);
+ assertTrue(noMatches.packages().isEmpty());
+ }
+
+ @Test
+ void listPackagesPackagePrefixFiltersByName() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ publishGeneric("repo", "pat-test", "1.0.0", "a".getBytes(StandardCharsets.UTF_8));
+ publishGeneric("repo", "other", "1.0.0", "b".getBytes(StandardCharsets.UTF_8));
+
+ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null, null,
+ "pat", null, null, null, null);
+
+ assertEquals(1, result.packages().size());
+ assertEquals("pat-test", result.packages().get(0).packageName());
+ }
+
+ @Test
+ void listPackagesPublishAndUpstreamFilterByResolvedRestriction() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ publishGeneric("repo", "default-pkg", "1.0.0", "a".getBytes(StandardCharsets.UTF_8));
+ publishGeneric("repo", "blocked-pkg", "1.0.0", "b".getBytes(StandardCharsets.UTF_8));
+ service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "generic", "ns", "blocked-pkg", "BLOCK",
+ "ALLOW");
+
+ CodeArtifactService.ListPackagesResult allowPublish = service.listPackages(REGION, "dom", null, "repo", null,
+ null, null, null, null, "ALLOW", null);
+ assertEquals(1, allowPublish.packages().size());
+ assertEquals("default-pkg", allowPublish.packages().get(0).packageName());
+
+ CodeArtifactService.ListPackagesResult allowUpstream = service.listPackages(REGION, "dom", null, "repo",
+ null, null, null, null, null, null, "ALLOW");
+ assertEquals(1, allowUpstream.packages().size());
+ assertEquals("blocked-pkg", allowUpstream.packages().get(0).packageName());
+ }
+
+ /**
+ * Raised by Greptile's review: a pypi package's origin configuration is stored under whatever
+ * spelling the caller set it with, but the real sidecar always lists the package under its own
+ * PEP 503-normalized spelling. Without normalizing both sides to the same form before union-ing
+ * them, the two would appear as two distinct packages, and the normalized one's own
+ * restrictions would never resolve to what was actually configured.
+ */
+ @Test
+ void listPackagesNormalizesAPypiPackageSoAnOriginConfigSetUnderADifferentSpellingIsNotDuplicatedOrLost() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ CodeArtifactRepository created = service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ when(pypiserverClient.listPackages(eq(created.getSidecarContainerIds().get("pypi")), eq("dom"), eq("repo")))
+ .thenReturn(List.of(new RepositorySidecarManager.PackageIdentifier(null, "my-proj")));
+ service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "pypi", null, "My_Proj", "BLOCK", "ALLOW");
+
+ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", "pypi",
+ null, null, null, null, null, null);
+
+ assertEquals(1, result.packages().size());
+ assertEquals("my-proj", result.packages().get(0).packageName());
+ assertEquals("BLOCK", result.packages().get(0).publishRestriction());
+ assertEquals("ALLOW", result.packages().get(0).upstreamRestriction());
+ }
+
+ /**
+ * Raised by Greptile's review: {@code packageSortCursor} used to join its three fields with a
+ * plain {@code ::}, which namespace/package names are allowed to contain themselves, so two
+ * different coordinates could collide on the exact same cursor and {@code Pagination}'s
+ * strictly-greater-cursor resume would then skip whichever one sorted second.
+ */
+ @Test
+ void listPackagesPagesPastACursorCollisionWithoutSkippingEitherPackage() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "generic", "a::b", "c", "ALLOW", "BLOCK");
+ service.putPackageOriginConfiguration(REGION, "dom", null, "repo", "generic", "a", "b::c", "ALLOW", "BLOCK");
+
+ CodeArtifactService.ListPackagesResult firstPage = service.listPackages(REGION, "dom", null, "repo", null,
+ null, null, 1, null, null, null);
+ assertEquals(1, firstPage.packages().size());
+ assertNotNull(firstPage.nextToken());
+
+ CodeArtifactService.ListPackagesResult secondPage = service.listPackages(REGION, "dom", null, "repo", null,
+ null, null, 1, firstPage.nextToken(), null, null);
+ assertEquals(1, secondPage.packages().size());
+
+ assertNotEquals(firstPage.packages().get(0).namespace() + "/" + firstPage.packages().get(0).packageName(),
+ secondPage.packages().get(0).namespace() + "/" + secondPage.packages().get(0).packageName());
+ }
+
+ /**
+ * Raised by Greptile's review: a {@link PackageOriginConfig} persisted before {@code format}/
+ * {@code namespace}/{@code packageName} existed on it has none of the three, so {@code
+ * allPackagesInRepository}'s own normalized map can never resolve it. The package it configures
+ * is still found here through its real {@code generic} version record, so its restriction must
+ * still resolve to what was actually saved, not silently fall back to the ALLOW/BLOCK default.
+ */
+ @Test
+ void listPackagesFallsBackToTheRawKeyLookupForAnOriginConfigPersistedBeforeItCarriedItsOwnCoordinate() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ publishGeneric("repo", "legacy-pkg", "1.0.0", "a".getBytes(StandardCharsets.UTF_8));
+ String legacyRawKey = REGION + "::dom::repo::generic::ns::legacy-pkg";
+ packageOriginConfigStore.putForAccount(ACCOUNT_ID, legacyRawKey,
+ new PackageOriginConfig("BLOCK", "ALLOW", null, null, null));
+
+ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null,
+ null, null, null, null, null, null);
+
+ assertEquals(1, result.packages().size());
+ assertEquals("BLOCK", result.packages().get(0).publishRestriction());
+ assertEquals("ALLOW", result.packages().get(0).upstreamRestriction());
+
+ CodeArtifactService.ListPackagesResult blockedOnly = service.listPackages(REGION, "dom", null, "repo", null,
+ null, null, null, null, "BLOCK", null);
+ assertEquals(1, blockedOnly.packages().size());
+ }
+
+ /**
+ * Raised by Greptile's review: disposing a pypi package's only version deletes its real
+ * sidecar content, so the sidecar's own listing no longer names it, but {@code
+ * disposePackageVersions} leaves a status override behind precisely so {@code
+ * ListPackageVersions} can keep reporting it as {@code Disposed}. With no origin configuration
+ * either, nothing else in the union would otherwise keep this package in {@code ListPackages}.
+ */
+ @Test
+ void listPackagesIncludesAPypiPackageWhoseOnlyVersionWasDisposed() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ CodeArtifactRepository created = service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ String repoId = created.getSidecarContainerIds().get("pypi");
+ when(pypiserverClient.listPackageVersions(eq(repoId), eq("dom"), eq("repo"), isNull(), eq("my-pkg")))
+ .thenReturn(List.of(
+ new RepositorySidecarManager.PackageVersionTimestamp("1.0.0", Instant.ofEpochSecond(1000))));
+ when(pypiserverClient.listPackageVersionAssetNames(eq(repoId), eq("dom"), eq("repo"), isNull(),
+ eq("my-pkg"), eq("1.0.0"))).thenReturn(List.of("my_pkg-1.0.0.tar.gz"));
+ service.disposePackageVersions(REGION, "dom", null, "repo", "pypi", null, "my-pkg", List.of("1.0.0"), null,
+ null);
+ when(pypiserverClient.listPackageVersions(eq(repoId), eq("dom"), eq("repo"), isNull(), eq("my-pkg")))
+ .thenReturn(List.of());
+
+ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", "pypi",
+ null, null, null, null, null, null);
+
+ assertEquals(1, result.packages().size());
+ assertEquals("my-pkg", result.packages().get(0).packageName());
+ }
+
+ @Test
+ void listPackagesDefaultPageSizeIsOneHundredNotFifty() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+ for (int i = 0; i < 120; i++) {
+ publishGeneric("repo", String.format("pkg-%03d", i), "1.0.0", "a".getBytes(StandardCharsets.UTF_8));
+ }
+
+ CodeArtifactService.ListPackagesResult result = service.listPackages(REGION, "dom", null, "repo", null, null,
+ null, null, null, null, null);
+
+ assertEquals(100, result.packages().size());
+ assertNotNull(result.nextToken());
+ }
+
+ @Test
+ void listPackagesRejectsAnInvalidFormat() {
+ service.createDomain(REGION, "dom", null, Map.of());
+ service.createRepository(REGION, "dom", null, "repo", null, null, Map.of());
+
+ AwsException e = assertThrows(AwsException.class, () -> service.listPackages(REGION, "dom", null, "repo",
+ "not-a-format", null, null, null, null, null, null));
+ assertEquals("ValidationException", e.getErrorCode());
+ }
+
// ---------------------------------------------------------- GetPackageVersionReadme
@Test
diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClientTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClientTest.java
index 2fef9e662b..334a4aa94f 100644
--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClientTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/PypiserverSidecarClientTest.java
@@ -511,6 +511,23 @@ class PypiserverSidecarClientTest {
});
}
+ /**
+ * Raised by hectorvent's review: every CodeArtifact repository gets a pypi sidecar id at
+ * creation regardless of whether it is ever actually used through pypi, so {@code
+ * ListPackages} with no format filter must not start this repository's own pypiserver
+ * container just to find out it was never used. No context is registered on {@code backend}
+ * at all here: a real container start would have nothing to connect to and throw.
+ */
+ @Test
+ void listPackagesReturnsEmptyWithoutStartingAContainerThatWasNeverUsed() {
+ PypiserverSidecarManager manager = manager();
+
+ List<RepositorySidecarManager.PackageIdentifier> packages = client(manager).listPackages("pypi-repo-1",
+ "dom", "repo");
+
+ assertTrue(packages.isEmpty());
+ }
+
private PypiserverSidecarManager manager() {
return new PypiserverSidecarManager(containerBuilder, lifecycleManager, config);
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClientTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClientTest.java
index 7de469bbc3..5c2874e9af 100644
--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClientTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/ReposiliteSidecarClientTest.java
@@ -1221,6 +1221,125 @@ class ReposiliteSidecarClientTest {
new String(md5Body.get(), StandardCharsets.UTF_8));
}
+ /**
+ * Raised by hectorvent's review: every CodeArtifact repository gets a maven sidecar id at
+ * creation regardless of whether it is ever actually used through maven, so {@code
+ * ListPackages} with no format filter must not start the one shared Reposilite container just
+ * to walk a repository that has never used maven. No context is registered for {@code
+ * /api/maven/details/dom--repo/} at all here: a real walk would have nothing to connect to and
+ * throw.
+ */
+ @Test
+ void listPackagesReturnsEmptyWithoutStartingTheSharedContainerThatWasNeverStarted() {
+ when(manager.isStarted()).thenReturn(false);
+
+ List<RepositorySidecarManager.PackageIdentifier> packages = client.listPackages("dom--repo", "dom", "repo");
+
+ assertThat(packages, hasSize(0));
+ }
+
+ /**
+ * Raised by Greptile's review: {@code com/foo} is both {@code com:foo}'s own version directory
+ * ({@code 1.0.0}) and a further groupId segment for {@code com.foo:bar}'s own package
+ * ({@code bar}). {@code bar} is listed before {@code 1.0.0} here deliberately, since an earlier
+ * version of {@link ReposiliteSidecarClient#listPackages} peeked only the first child, so with
+ * {@code bar} first it fell through to treating every child (including the real version
+ * directory {@code 1.0.0}) as a further groupId segment, recursed into {@code 1.0.0} where it
+ * found no subdirectories and silently returned, and never recorded {@code com:foo} at all.
+ */
+ @Test
+ void listPackagesFindsBothAPackageAndASiblingGroupSharingItsDirectory() {
+ server.createContext("/api/maven/details/dom--repo/",
+ exchange -> respond(exchange, 200, "{\"name\":\"dom--repo\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"com\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/",
+ exchange -> respond(exchange, 200, "{\"name\":\"com\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"foo\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/foo/",
+ exchange -> respond(exchange, 200, "{\"name\":\"foo\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"bar\",\"type\":\"DIRECTORY\"},"
+ + "{\"name\":\"1.0.0\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/foo/1.0.0/",
+ exchange -> respond(exchange, 200, "{\"name\":\"1.0.0\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"foo-1.0.0.jar\",\"type\":\"FILE\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/foo/bar/",
+ exchange -> respond(exchange, 200, "{\"name\":\"bar\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"1.0.0\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/foo/bar/1.0.0/",
+ exchange -> respond(exchange, 200, "{\"name\":\"1.0.0\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"bar-1.0.0.jar\",\"type\":\"FILE\"}]}"));
+
+ List<RepositorySidecarManager.PackageIdentifier> packages = client.listPackages("dom--repo", "dom", "repo");
+
+ assertThat(packages, containsInAnyOrder(
+ new RepositorySidecarManager.PackageIdentifier("com", "foo"),
+ new RepositorySidecarManager.PackageIdentifier("com.foo", "bar")));
+ }
+
+ /**
+ * Raised by Greptile's review: {@code com/example/tool} has zero version directories of its
+ * own (a native PUT straight to its {@code maven-metadata.xml}, with no real version content
+ * ever written under it, the same artifact-level-only shape {@code recordVersionAvailable}'s
+ * own javadoc already describes), so peeking it from its parent sees no subdirectories at all
+ * and, before this fix, treated that as proof the parent ({@code com:example}) was the real
+ * package, recording a package that does not exist and skipping the real one,
+ * {@code com.example:tool}, entirely.
+ */
+ @Test
+ void listPackagesRecognizesAnArtifactDirectoryWithNoVersionsOfItsOwnThroughItsOwnMetadata() {
+ server.createContext("/api/maven/details/dom--repo/",
+ exchange -> respond(exchange, 200, "{\"name\":\"dom--repo\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"com\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/",
+ exchange -> respond(exchange, 200, "{\"name\":\"com\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"example\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/example/",
+ exchange -> respond(exchange, 200, "{\"name\":\"example\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"tool\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/example/tool/",
+ exchange -> respond(exchange, 200, "{\"name\":\"tool\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"maven-metadata.xml\",\"type\":\"FILE\"}]}"));
+
+ List<RepositorySidecarManager.PackageIdentifier> packages = client.listPackages("dom--repo", "dom", "repo");
+
+ assertThat(packages, containsInAnyOrder(new RepositorySidecarManager.PackageIdentifier("com.example",
+ "tool")));
+ }
+
+ /**
+ * Raised by Greptile's review of the fix above: a real Maven snapshot version directory also
+ * holds its own {@code maven-metadata.xml} (listing that version's timestamped builds),
+ * alongside its real artifact files, so excluding every {@code maven-metadata.xml}-holding
+ * child from being version-shaped, as that fix did, wrongly recursed into {@code
+ * 1.0-SNAPSHOT} as though it were a further groupId segment, adding the nonexistent package
+ * {@code com.example.tool:1.0-SNAPSHOT} instead of the real one, {@code com.example:tool}.
+ */
+ @Test
+ void listPackagesTreatsASnapshotVersionDirectoryAsVersionShapedDespiteItsOwnMetadata() {
+ server.createContext("/api/maven/details/dom--repo/",
+ exchange -> respond(exchange, 200, "{\"name\":\"dom--repo\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"com\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/",
+ exchange -> respond(exchange, 200, "{\"name\":\"com\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"example\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/example/",
+ exchange -> respond(exchange, 200, "{\"name\":\"example\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"tool\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/example/tool/",
+ exchange -> respond(exchange, 200, "{\"name\":\"tool\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"1.0-SNAPSHOT\",\"type\":\"DIRECTORY\"}]}"));
+ server.createContext("/api/maven/details/dom--repo/com/example/tool/1.0-SNAPSHOT/",
+ exchange -> respond(exchange, 200, "{\"name\":\"1.0-SNAPSHOT\",\"type\":\"DIRECTORY\",\"files\":["
+ + "{\"name\":\"maven-metadata.xml\",\"type\":\"FILE\"},"
+ + "{\"name\":\"maven-metadata.xml.sha1\",\"type\":\"FILE\"},"
+ + "{\"name\":\"tool-1.0-SNAPSHOT.jar\",\"type\":\"FILE\"}]}"));
+
+ List<RepositorySidecarManager.PackageIdentifier> packages = client.listPackages("dom--repo", "dom", "repo");
+
+ assertThat(packages, containsInAnyOrder(new RepositorySidecarManager.PackageIdentifier("com.example",
+ "tool")));
+ }
+
private static void respond(HttpExchange exchange, int status, String body) throws IOException {
byte[] bytes = body.getBytes(StandardCharsets.UTF_8);
exchange.sendResponseHeaders(status, bytes.length);
diff --git a/src/test/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClientTest.java b/src/test/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClientTest.java
index b03cb4bdb4..a1f2660609 100644
--- a/src/test/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClientTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/codeartifact/VerdaccioSidecarClientTest.java
@@ -14,6 +14,7 @@ import java.lang.reflect.Field;
import java.net.InetSocketAddress;
import java.net.http.HttpClient;
import java.nio.charset.StandardCharsets;
+import java.util.List;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicReference;
@@ -257,6 +258,23 @@ class VerdaccioSidecarClientTest {
});
}
+ /**
+ * Raised by hectorvent's review: every CodeArtifact repository gets an npm sidecar id at
+ * creation regardless of whether it is ever actually used through npm, so {@code
+ * ListPackages} with no format filter must not start this repository's own Verdaccio
+ * container just to find out it was never used. No context is registered on {@code backend}
+ * at all here: a real container start would have nothing to connect to and throw.
+ */
+ @Test
+ void listPackagesReturnsEmptyWithoutStartingAContainerThatWasNeverUsed() {
+ VerdaccioSidecarManager manager = manager();
+
+ List<RepositorySidecarManager.PackageIdentifier> packages = client(manager).listPackages("npm-repo-1",
+ "dom", "repo");
+
+ assertTrue(packages.isEmpty());
+ }
+
private VerdaccioSidecarManager manager() {
return new VerdaccioSidecarManager(containerBuilder, lifecycleManager, config);
}