Apache Solr open-source search software
= Working with HTML Templates
:toc:
The Solr Ref Guide uses Antora to build the HTML version of the site.
== What is Antora?
Antora is a static site generator, meaning that it takes some set of documents and produces HTML pages.
It allows for templating of the pages, so each page has the same look and feel without having to code headers, footers, logos, etc., into every page.
Antora is an open source project, online at https://antora.org/.
== How We Use Antora
The following sections describe the main features of Antora that you will encounter while working with the Solr Ref Guide.
=== Asciidoctor
Antora uses Asciidoctor.js by default as its content markup language.
Read more on the Solr xref:asciidoc-syntax.adoc[AsciiDoc Syntax] page, or on Antora's guide: https://docs.antora.org/antora/latest/asciidoc/asciidoc/
=== playbook.yml
The `playbook.yml` is a global configuration file that drives many of the options used when building the site.
We have template-ized `playbook.template.yml`, instead of a hardcoded `playbook.yml`.
Each gradle task that requires a `playbook.yml`, such as `gradlew buildLocalSite` or `gradlew buildOfficialSite`, will generate the `playbook.yml` that is needed based off of this template.
It is unlikely that you will manually need to edit this very often.
=== antora.yml
The `antora.yml` is a configuration file for a specific version of a component in the Antora build.
Therefore every version we release for the Solr ref-guide, or the Solr Operator ref-guide will have a separate `antora.yml`.
This is also where version-specific variables are configured, such as dependency versions (lucene, zookeeper, etc.), Solr version, and javadocs location.
The `antora.yml` that is used when building a local version of the site is not the version checked-into the repository.
Instead a local `antora.yml` is built in the `build/` directory.
The checked-in `antora.yml` is used when the official ref-guide is built.
Antora will checkout all specified branches/tags and use the antora.yml to build the ref-guide for that branch's version.
In order to test changes to the `antora.yml`, do not change this file directly.
Instead, update `antora.template.yaml`, and `gradlew buildLocalSite` will build a site with the changes.
The only reason you will likely need to change the `antora.template.yml` is if you are introducing new variables for dependency versions.
== Auto-generated CLI Reference Pages
The `bin/solr` command-line tool documentation is partially auto-generated from the picocli `@Command`, `@Option`, and `@Parameters` annotations in the Java source.
=== How it works
For each command registered in `org.apache.solr.cli.SolrCLI` that uses picocli annotations, a dedicated AsciiDoc page is generated by `picocli.codegen.docgen.manpage.ManPageGenerator` and committed to:
----
solr/solr-ref-guide/modules/deployment-guide/pages/cli/
----
The generated pages are post-processed to be Antora-compatible (correct page title, `:page-toclevels:` attribute, ASF license header, DO NOT EDIT notice, and the experimental warning).
Two things about that post-processing are worth knowing before changing it:
* Whether an option is required is worked out by re-parsing the synopsis that picocli generated, because `ManPageGenerator` emits no per-option marker (https://github.com/remkop/picocli/issues/2519[picocli#2519]).
A change to picocli's synopsis layout will therefore change the markers; the build fails loudly rather than silently publishing pages without them.
* `runManPageGenerator` resolves `project.objects` and the `cliClasspath` configuration at execution time, which is not compatible with Gradle's configuration cache should it ever be enabled for this build.
The navigation section in `deployment-nav.adoc` between the `// CLI-DOCS-START` and `// CLI-DOCS-END` markers is also updated automatically.
The main landing page `solr-control-script-reference.adoc` provides an overview of the available tools with links to each man-page.
=== Gradle targets
`./gradlew :solr:solr-ref-guide:generateCliDocs`::
Regenerates all pages under `pages/cli/` from the current picocli annotations and updates the nav.
Run this after modifying any `@Command`, `@Option`, or `@Parameters` annotation in `solr/core/src/java/org/apache/solr/cli/`.
`./gradlew :solr:solr-ref-guide:checkCliDocsUpToDate`::
Verifies that the committed pages in `pages/cli/` are in sync with the current annotations.
It is wired into `gradlew check` alongside the other ref-guide checks, so it is skipped when the ref guide is excluded with `-Prefguide.include=false`.
Note that upgrading picocli can change the generated wording without any annotation having been touched; if the check fails after a version bump, re-run `generateCliDocs` and commit the result.
=== Adding docs for a new tool
The new command will automatically appear in the generated pages, in the navigation and in the command table on `pages/cli/index.adoc` when running `gradlew generateCliDocs`.
`pages/cli/index.adoc` is the only hand-written page in that directory; everything matching `solr-*.adoc` is generated and must not be edited by hand.
`solr-control-script-reference.adoc` documents the default (commons-cli) implementation and should not gain per-command links to the generated pages while the picocli implementation is experimental.
=== What a tool migration should include
When migrating a tool to picocli:
* A `*PicocliTest` subclass of the tool's existing test that overrides only the invocation strategy, so every case runs on both implementations. See `ZkSubcommandsPicocliTest` for the pattern.
* The tool's `.bats` file added to `integrationTestsPicocli` in `solr/packaging/build.gradle`, but only once *every* command that suite invokes has been migrated, not just the one under test.
`test_auth.bats`, for example, uses `solr assert` to wait for startup, so it cannot be added until the assert command is migrated.
Be aware that a failure part-way through that suite is especially awkward: `solr auth enable` appends `SOLR_AUTH_TYPE` and `SOLR_AUTHENTICATION_OPTS` to the `solr.in.sh` of the *built distribution* under `solr/packaging/build/`, so an aborted run leaves every later `bin/solr` invocation trying to authenticate.
If BATS runs start hanging for no apparent reason, check that file, and delete the distribution directory to force `installFullDist` to recreate it.
* A `gradlew :solr:solr-ref-guide:generateCliDocs` run, with the result committed.
* The tool registered in `SolrCLI`'s `subcommands` only once `callTool()` actually works; `SolrCLIPicocliTest` enforces the rules that apply to every registered command.
== Building the HTML Site
A Gradle target `gradlew buildLocalSite` will build the full HTML site (found in `solr/solr-ref-guide/build/site`).
This target builds the navigation for the left-hand menu, and converts all `.adoc` files to `.html`, including navigation and inter-document links.
The `checkSiteLinks` target also checks that all inter-doc and javadocs references are correct and resolvable.