Apache Solr open-source search software
= APIs in Solr
:toc: left
Solr's codebase currently has a handful of ways of defining APIs.
This complexity stems largely from two ongoing transitions:
1. Away from our "v1" APIs and towards the new "v2" API.
2. Away from a legacy API framework towards the off-the-shelf JAX-RS library for implementing our v2 APIs
As we finish these transitions, this complexity should simplify considerably.
But in the interim, this document can help guide developers who need to understand or modify APIs in Solr.
== API Types
APIs in Solr (regardless of whether v1 or v2) can typically be classified as either "per-core" or "cluster" APIs.
Per-core APIs, as the name suggests, typically affect only a single core or collection, usually used to search or analyze that core's contents in some way.
Implementation-wise, they're registered on the `SolrCore` object itself.
They are configured in solrconfig.xml, which also means they can differ from one core to another based on the configset being used.
Alternatively "cluster" APIs potentially affect the entire Solr instance or cluster.
They're registered on the `CoreContainer` object itself.
It's much less common to provide configuration for these APIs, but it is possible to do so using `solr.xml`.
== V1 APIs
v1 APIs are the primary way that users consume Solr, as our v2 APIs remain "experimental".
Many new APIs are added as "v2 only", however updates to existing v1 APIs still happen frequently.
v1 APIs exist in Solr as implementations of the `SolrRequestHandler` interface, usually making use of the `RequestHandlerBase` base class.
RequestHandlers have two methods of primary interest:
1. `init`, often used to parse per-API configuration or otherwise setup the class
2. `handleRequest` (or `handleRequestBody` if using `RequestHandlerBase`), the main entrypoint for the API, containing the business logic and constructing the response.
While they do define many aspects of the endpoint's interface (e.g. query parameters, request body format, response format, etc.), RequestHandler's don't actually specify the URL path that they're located at.
These paths are instead either hardcoded at registration time (see `CoreContainer.load` and `ImplicitPlugins.json`), or specified by users in configuration files (typically `solrconfig.xml`).
== V2 APIs
v2 APIs are currently still "experimental", and not necessarily recommended yet for users.
But they're approaching parity with v1 and will eventually replace Solr's "RequestHandler"-based APIs.
=== New "JAX-RS" APIs
New v2 APIs in Solr are written in compliance with "JAX-RS", a library and specification that uses annotations to define APIs.
Many libraries implement the JAX-RS spec: Solr currently uses the implementation provided by the "Jersey" project.
These v2 API definitions consist of two parts: a JAX-RS annotated interface in the `api` module "defining" the API, and a class in `core` "implementing" the interface.
Separating the API "definition" and "implementation" in this way allows us to only define each API in a single place, and use code generators to produce other API-related bits such as SolrJ code and ref-guide documentation.
=== Writing JAX-RS APIs
Writing a new v2 API may appear daunting, but additions in reality are actually pretty simple:
. *Agree on Endpoint Syntax*: Before implementing, developers should generate consensus around what the new API will look like. General guidelines and conventions for the v2 API syntax are described <<apis-v2-conventions.adoc,here>>.
. *Create POJO ("Plain Old Java Object") classes as needed to represent the API's request and response*:
** POJOs are used to represent both the body of the API request (for some `POST` and `PUT` endpoints), as well as the response from the API.
** Re-use of existing classes here is preferred where possible. A library of available POJOs can be found in the `org.apache.solr.client.api.model` package of the `api` gradle project.
** POJO class fields are typically "public" and annotated with the Jackson `@JsonProperty` annotations to allow serialization/deserialization.
** POJO class fields should also have a Swagger `@Schema` annotation where possible, describing the purpose of the field. These descriptions are technically non-functional, but add lots of value to our OpenAPI spec and any artifacts generated downstream.
. *Find or create an interface to hold the v2 API definition*:
** API interfaces live in the `org.apache.solr.client.api.endpoint` package of the `api` gradle project. Interfaces are usually given an "-Api" suffix to indicate their role.
** If a new API is similar enough to existing APIs, it may make sense to add the new API definition into an existing interface instead of creating a wholly new one. Use your best judgement.
. *Add a method to the chosen interface representing the API*:
** The method should take an argument representing each path and query parameter (annotated with `@PathParam` or `@QueryParam` as appropriate). If the API is a `PUT` or `POST` that expects a request body, the method should take the request body POJO as its final argument, annotated with `@RequestBody`.
** Each method parameter should also be annotated with the Swagger `@Parameter` annotation. Like the `@Schema` annotation mentioned above, `@Parameter` isn't strictly required for correct operation, but they add lots of value to our OpenAPI spec and generated artifacts.
** As a return value, the method should return the response-body POJO.
. *Futher JAX-RS Annotation*: The interface method in step (3) has specified its inputs and outputs, but several additional annotations are needed to define how users access the API, and to make it play nice with the code-generation done by Solr's build.
** Each interface must have a `@Path` annotation describing the path that the API is accessed from. Specific interface methods can also be given `@Path` annotations, making the "effective path" a concatenation of the interface and method-level values. `@Path` supports a limited regex syntax, and curly-brackets can be used to create named placeholders for path-parameters.
** Each interface method should be given an HTTP-method annotation (e.g. `@GET`, `@POST`, etc.)
** Each interface method must be marked with a Swagger `@Operation` annotation. This annotation is used to provide metadata about the API that appears in the OpenAPI specification and in any artifacts generated from that downstream. At a minimum, `summary` and `tags` values should be specified on the annotation. (`tags` is used by our SolrJ code generation to group similar APIs together. Typically APIs are only given a single tag representing the plural name of the most relevant "resource" (e.g. `tags = {"aliases"}`, `tags = {"replica-properties"}`)
. *Create a class implementing the API interface*: Implementation classes live in the `core` gradle project, typically in the `org.apache.solr.handler` package or one of its descendants.
** Implementing classes must extent `JerseyResource`, and are typically named similarly to the API interface created in (2) above without the "-Api" suffix. e.g. `class AddReplicaProperty extends JerseyResource implements AddReplicaPropertyApi`)
** Solr's use of Jersey offers us some limited dependency-injection ("DI") capabilities. Class constructors annotated with `@Inject` can depend on a selection of types made available through DI, such as `CoreContainer`, `SolrQueryRequest`, `SolrCore`, etc. See the factory-bindings in `JerseyApplications` (or other API classes) for a sense of which types are available for constructor injection.
** Add a body to your classes method(s). For the most part this is "normal" Java development.
. *Register your API*: APIs must be registered to be available at runtime. If the v2 API is associated with an existing v1 RequestHandler, the API class name can be added to the handler's `getJerseyResources` method. If there is no associated RequestHandler, the API should be registered similar to other APIs in `CoreContainer.load`.
A good example for each of these steps can be seen in Solr's v2 "add-replica-property" API, which has a defining interface https://github.com/apache/solr/blob/9426902acb7081a2e9a1fa29699c5286459e1365/solr/api/src/java/org/apache/solr/client/api/endpoint/AddReplicaPropertyApi.java[AddReplicaPropertyApi], an implementing class https://github.com/apache/solr/blob/9426902acb7081a2e9a1fa29699c5286459e1365/solr/core/src/java/org/apache/solr/handler/admin/api/AddReplicaProperty.java[AddReplicaProperty], and the two POJOs https://github.com/apache/solr/blob/main/solr/api/src/java/org/apache/solr/client/api/model/AddReplicaPropertyRequestBody.java[AddReplicaPropertyRequestBody] and https://github.com/apache/solr/blob/main/solr/api/src/java/org/apache/solr/client/api/model/SolrJerseyResponse.java[SolrJerseyResponse].
=== Relationship Between V1 and V2 Implementations
Most v2 APIs have a corresponding legacy v1 API (e.g. `/admin/cores?action=RELOAD` backs `POST /api/cores/coreName/reload`).
Where both exist, **the actual business logic should live in the v2 implementation class, and the v1 handler should delegate to it** - not the other way around.
This convention exists for a few reasons:
1. It makes it easier to delete the v1 code down the road.
2. It prevents the v2 code from "falling behind" or being forgotten when query params or API functionality changes, since v1 traffic continuously exercises the same v2 code path.
3. It gives the v2 endpoint test coverage "for free" - most of Solr's existing test suite targets v1 APIs, so having v1 call v2 gives confidence that the v2 endpoint works correctly even in the absence of direct v2-specific tests.
How this delegation is implemented differs depending on whether the API executes synchronously or is processed by the Overseer.
==== Synchronous (CoreAdminAPIBase) APIs
Note: this is distinct from "per-core" APIs generally (see the "API Types" section above), which are registered on individual `SolrCore` objects (e.g. `/select`, `/update`, `/admin/ping`). `CoreAdminAPIBase`-derived APIs are instead a "cluster" API registered on the `CoreContainer`, corresponding to the legacy `/admin/cores` handler.
For `CoreAdminAPIBase`-derived APIs (i.e. `/api/cores/...`), the v1 `CoreAdminHandler.CoreAdminOp` implementation is reduced to a thin adapter: parse the v1 `SolrParams` into the v2 request-body POJO, construct the v2 API class, call it, and squash the response back into the v1 `NamedList` via `V2ApiUtils.squashIntoSolrResponseWithoutHeader`.
```java
class MergeIndexesOp implements CoreAdminHandler.CoreAdminOp {
@Override
public void execute(CoreAdminHandler.CallInfo it) throws Exception {
SolrParams params = it.req.getParams();
final var requestBody = new MergeIndexesRequestBody();
// ... populate requestBody from params ...
final var mergeIndexesApi =
new MergeIndexes(it.handler.coreContainer, it.handler.coreAdminAsyncTracker, it.req, it.rsp);
final var response = mergeIndexesApi.mergeIndexes(cname, requestBody);
V2ApiUtils.squashIntoSolrResponseWithoutHeader(it.rsp, response);
}
}
```
See `RenameCore`/`CoreAdminOperation.RENAME_OP`, `MergeIndexes`/`MergeIndexesOp`, and `SplitCoreAPI`/`SplitOp` for examples of this pattern.
==== Overseer-driven (AdminAPIBase) APIs
Note: not all collection-level APIs are Overseer-driven, and that's expected to become even less true over time. This section covers only the subset of `AdminAPIBase`-derived APIs that are processed asynchronously by the Overseer via a ZK-based command queue.
For these APIs (i.e. `/api/collections/...`), the underlying `cloud/api/collections/*Cmd.java` executor (e.g. `SplitShardCmd`, `CreateShardCmd`) is *not* moved into v2 - it stays registered in `CollApiCmds.java` and continues to execute on the Overseer thread exactly as before.
"Logic in v2" here means the v2 class is responsible for:
1. Validating parameters (including any cross-parameter conflict checks previously done by the v1 handler, so the check applies uniformly to v1 and native v2 requests)
2. Building the `ZkNodeProps` "remote message" the Overseer command expects
3. Submitting it via `AdminAPIBase.submitRemoteMessageAndHandleResponse`/`submitRemoteMessageAndHandleAsync` - the same ZK-queue submission machinery v1 always used
The v1 `CollectionOperation` enum entry is then reduced to a one-line delegation:
```java
SPLITSHARD_OP(
SPLITSHARD,
(req, rsp, h) -> {
SplitShardAPI.invokeWithV1Params(h.coreContainer, req, rsp);
return null;
}),
```
Returning `null` from a `CollectionOperation` lambda tells `CollectionsHandler.invokeAction()` that the request has already been fully handled.
Returning a non-null `Map<String, Object>` instead falls back to the older pattern where `CollectionsHandler` itself builds and submits the `ZkNodeProps` message generically - this is still the state of many not-yet-migrated v1 actions, and is *not* an example to follow for new migrations.
See `CreateShard`/`CREATESHARD_OP`, `DeleteShard`/`DELETESHARD_OP`, and `SplitShardAPI`/`SPLITSHARD_OP` for examples of the target pattern.
==== Standard v1-delegation entry points
By convention, a migrated v2 API class exposes two `public static` methods for v1 to call:
* `createRequestBodyFromV1Params(SolrParams params)` - parses legacy params into the v2 request-body POJO.
* `invokeWithV1Params(CoreContainer, SolrQueryRequest, SolrQueryResponse)` (sometimes named `invokeFromV1Params` - naming isn't fully consistent yet) - constructs the v2 API class, calls it, and squashes the response into the v1 `SolrQueryResponse`.
==== Don't forward `async` from v1
`CoreAdminHandler.handleRequestBody()` and `CollectionsHandler.invokeAction()` already wrap the *entire* v1 dispatch in their own async-task submission, keyed by the request's `async` parameter, before the `CoreAdminOp`/`CollectionOperation` even runs.
If a v1 adapter also copies that same `async` value into the v2 request body, the v2 class's own async handling (`handlePotentiallyAsynchronousTask`, `submitRemoteMessageAndHandleAsync`) will try to register the identical task id a second time, and fail with `Duplicate request with the same requestid found.`
*Leave the v2 request body's `async` field unset when delegating from v1.*
`MergeIndexesRequestBody` has no `async` field at all, and `SplitOp`'s v1 adapter deliberately omits setting one - both are correct examples of this. This only applies to genuine v1-delegation adapters; native v2 JSON requests should set `async` normally, and the v2 class's own handling of it is unaffected.
Watch for this same class of bug in *internal* sub-requests too: some Overseer commands (e.g. `SplitShardCmd`'s `splitByPrefix` sub-request, sent via `ShardRequestTracker.sendShardRequest`) piggyback an internal correlation id on the `async` param purely for response tracking, not because the sub-request is meant to be tracked as a real async task. If a migrated v2 method routes such a sub-request through its own async wrapper, the same "duplicate requestid" failure results. Keep any such synchronous-by-design branches (e.g. a `getRanges=true` calculation) entirely outside the async wrapper, matching the original v1 code's control flow rather than funneling every branch through the same async-handling path.
=== Legacy v2 API Framework
While we've settled on JAX-RS as our framework for defining v2 APIs going forward, Solr still retains many v2 APIs that were written using an older homegrown framework.
This framework defines APIs using annotations (e.g. `@EndPoint`) similar to those used by JAX-RS, but lacks the full range of features and 3rd-party tooling.
We're in the process of migrating these API definitions to JAX-RS and hope to remove all support for this legacy framework in a future release.