Skip to content

armgit / solr

Public Imported from GitHub
0

Apache Solr open-source search software

SOLR-14790: Migrate docker usage documentation to the ref-guide (#101)

Houston Putman2d2b6a297220
diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt
index 0054194c483..93901f46daa 100644
--- a/solr/CHANGES.txt
+++ b/solr/CHANGES.txt
@@ -125,6 +125,8 @@ when told to. The admin UI now tells it to. (Nazerke Seidan, David Smiley)
* SOLR-15340: Rename shardsWhitelist and extract AllowListUrlChecker to use it more broadly. (Bruno Roustant)
+* SOLR-14790: Move Solr Docker image documentation to the ref guide. (Houston Putman)
+
Other Changes
----------------------
* SOLR-14656: Autoscaling framework removed (Ishan Chattopadhyaya, noble, Ilan Ginzburg)
diff --git a/solr/docker/README.md b/solr/docker/README.md
deleted file mode 100644
index d6d865dd151..00000000000
--- a/solr/docker/README.md
+++ /dev/null
@@ -1,275 +0,0 @@
-# Supported tags and respective `Dockerfile` links
-
-See [Docker Hub](https://hub.docker.com/apache/solr?tab=tags) for a list of image tags available to pull.
-
-# Getting started with the Docker image
-
-Instructions below apply to `apache/solr:8.0.0` and above.
-
-## Running Solr with host-mounted directories
-
-Typically users first want to run a single standalone Solr server in a container, with a single core for data, while storing data in a local directory.
-This is a convenient mechanism for developers, and could be used for single-server production hosts too.
-
-```console
-$ mkdir solrdata
-$ docker run -d -v "$PWD/solrdata:/var/solr" -p 8983:8983 --name my_solr apache/solr solr-precreate gettingstarted
-```
-
-Then with a web browser go to `http://localhost:8983/` to see the Admin Console (adjust the hostname for your docker host).
-In the web UI if you click on "Core Admin" you should now see the "gettingstarted" core.
-
-Next load some of the example data that is included in the container:
-
-```console
-$ docker exec -it my_solr post -c gettingstarted example/exampledocs/manufacturers.xml
-```
-
-In the UI, find the "Core selector" popup menu and select the "gettingstarted" core, then select the "Query" menu item. This gives you a default search for `*:*` which returns all docs. Hit the "Execute Query" button, and you should see a few docs with data. Congratulations!
-
-
-# Single server with Docker-compose
-
-You can use Docker Compose to run a single standalone server.
-And you could use Docker Volumes instead of host-mounted directories.
-For example, with a `docker-compose.yml` containing the following:
-
-```yaml
-version: '3'
-services:
- solr:
- image: apache/solr
- ports:
- - "8983:8983"
- volumes:
- - data:/var/solr
- command:
- - solr-precreate
- - gettingstarted
-volumes:
- data:
-```
-
-you can simply run:
-
-```console
-docker-compose up -d
-```
-
-## Single-command demo
-
-For quick demos of Solr docker, there is a single command that starts Solr, creates a collection called "demo", and loads sample data into it:
-
-```console
-$ docker run --name solr_demo -d -p 8983:8983 apache/solr solr-demo
-```
-## Distributed Solr
-
-See [this example](docs/docker-compose.yml)
-for an example Docker Compose file that starts up Solr in a simple cluster configuration.
-
-# How the image works
-
-The container contains an installation of Solr, as installed by the [service installation script](https://lucene.apache.org/solr/guide/7_7/taking-solr-to-production.html#service-installation-script).
-This stores the Solr distribution in `/opt/solr`, and configures Solr to use `/var/solr` to store data and logs, using the `/etc/default/solr` file for configuration.
-If you want to persist the data, mount a volume or directory on `/var/solr`.
-Solr expects some files and directories in `/var/solr`; if you use your own directory or volume you can either pre-populate them, or let Solr docker copy them for you. See [init-var-solr](scripts/init-var-solr).
-If you want to use custom configuration, mount it in the appropriate place. See below for examples.
-
-The Solr docker distribution adds [scripts](scripts) in `/opt/docker-solr/scripts` to make it easier to use under Docker, for example to create cores on container startup.
-
-## Creating cores
-
-When Solr runs in standalone mode, you create "cores" to store data. On a non-Docker Solr, you would run the server in the background, then use the [Solr control script](https://lucene.apache.org/solr/guide/7_7/solr-control-script-reference.html) to create cores and load data. With Solr docker you have various options.
-
-The first is exactly the same: start Solr running in a container, then execute the control script manually in the same container:
-
-```console
-$ docker run -d -p 8983:8983 --name my_solr apache/solr
-$ docker exec -it my_solr solr create_core -c gettingstarted
-```
-
-This is not very convenient for users, and makes it harder to turn it into configuration for Docker Compose and orchestration tools like Kubernetes.
-So, typically you will use the `solr-precreate` command which prepares the specified core and then runs Solr:
-
-```console
-$ docker run -d -p 8983:8983 --name my_solr apache/solr solr-precreate gettingstarted
-```
-
-The `solr-precreate` command takes an optional extra argument to specify a configset directory below `/opt/solr/server/solr/configsets/`.
-This allows you to specify your own config. See [this example](https://github.com/docker-solr/docker-solr-examples/tree/master/custom-configset).
-
-The third option is to use the `solr-create` command. This runs a Solr in the background in the container, then uses the Solr control script to create the core, then stops the Solr server and restarts it in the foreground. This method is less popular because the double Solr run can be confusing.
-
-```console
-$ docker run -d -p 8983:8983 --name my_solr apache/solr solr-create -c gettingstarted
-```
-
-Finally, you can run your own command-line and specify what to do, and even invoke mounted scripts. For example:
-
-```console
-$ docker run -p 8983:8983 -v $PWD/mysetup.sh:/mysetup.sh --name my_solr apache/solr bash -c "precreate-core gettingstarted && source /mysetup.sh && solr-foreground"
-```
-
-## Creating collections
-
-In a "SolrCloud" cluster you create "collections" to store data; and again you have several options for creating a core.
-
-These examples assume you're running [this example cluster](docs/docker-compose.yml)
-
-The first way to create a collection is to go to the [Solr Admin UI](http://localhost:8983/), select "Collections" from the left-hand side navigation menu, then press the "Add Collection" button, give it a name, select the `_default` config set, then press the "Add Collection" button.
-
-The second way is through the Solr control script on one of the containers:
-
-```console
-$ docker exec solr1 solr create -c gettingstarted2
-```
-
-The third way is to use a separate container:
-
-```console
-$ docker run -e SOLR_HOST=solr1 --network docs_solr apache/solr solr create_collection -c gettingstarted3 -p 8983
-```
-
-The fourth way is to use the remote API, from the host or from one of the containers, or some new container on the same network (adjust the hostname accordingly):
-
-```console
-curl 'http://localhost:8983/solr/admin/collections?action=CREATE&name=gettingstarted3&numShards=1&collection.configName=_default'
-```
-
-If you want to use a custom config for your collection, you first need to upload it, and then refer to it by name when you create the collection.
-See the Ref guide on how to use the [ZooKeeper upload](https://lucene.apache.org/solr/guide/7_7/solr-control-script-reference.html) or the [configset API](https://lucene.apache.org/solr/guide/7_0/configsets-api.html).
-
-
-## Loading your own data
-
-There are several ways to load data; let's look at the most common ones.
-
-The most common first deployment is to run Solr standalone (not in a cluster), on a workstation or server, where you have local data you wish to load.
-One way of doing that is using a separate container, with a mounted volume containing the data, using the host network so you can connect to the mapped port:
-
-```console
-# start Solr. Listens on localhost:8983
-$ docker run --name my_solr -p 8983:8983 apache/solr solr-precreate books
-
-# get data
-$ mkdir mydata
-$ wget -O mydata/books.csv https://raw.githubusercontent.com/apache/lucene-solr/master/solr/example/exampledocs/books.csv
-$ docker run --rm -v "$PWD/mydata:/mydata" --network=host apache/solr post -c books /mydata/books.csv
-```
-
-If you use the [this example cluster](docs/docker-compose.yml) the same works, or you can just start your loading container in the same network:
-
-```console
-$ docker run -e SOLR_HOST=solr1 --network=mycluster_solr apache/solr solr create_collection -c books -p 8983
-$ docker run --rm -v "$PWD/mydata:/mydata" --network=mycluster_solr apache/solr post -c books /mydata/books.csv -host solr1
-```
-
-Alternatively, you can make the data available on a volume at Solr start time, and then load it from `docker exec` or a custom start script.
-
-
-## solr.in.sh configuration
-
-In Solr it is common to configure settings in [solr.in.sh](https://github.com/apache/lucene-solr/blob/master/solr/bin/solr.in.sh),
-as documented in the [Solr Reference Guide](https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production#TakingSolrtoProduction-Environmentoverridesincludefile).
-
-The `solr.in.sh` file can be found in `/etc/default`:
-
-```console
-$ docker run apache/solr cat /etc/default/solr.in.sh
-```
-
-It has various commented-out values, which you can override when running the container, like:
-
-```
-$ docker run -d -p 8983:8983 -e SOLR_HEAP=800m apache/solr
-```
-
-You can also mount your own config file. Do no modify the values that are set at the end of the file.
-
-## Extending the image
-
-The Solr docker image has an extension mechanism. At run time, before starting Solr, the container will execute scripts
-in the `/docker-entrypoint-initdb.d/` directory. You can add your own scripts there either by using mounted volumes
-or by using a custom Dockerfile. These scripts can for example copy a core directory with pre-loaded data for continuous
-integration testing, or modify the Solr configuration.
-
-Here is a simple example. With a `custom.sh` script like:
-
-```console
-#!/bin/bash
-set -e
-echo "this is running inside the container before Solr starts"
-```
-
-you can run:
-
-```console
-$ docker run --name solr_custom1 -d -v $PWD/custom.sh:/docker-entrypoint-initdb.d/custom.sh apache/solr
-$ sleep 5
-$ docker logs solr_custom1 | head
-/opt/docker-solr/scripts/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/set-heap.sh
-this is running inside the container before Solr starts
-
-Starting Solr on port 8983 from /opt/solr/server
-```
-
-With this extension mechanism it can be useful to see the shell commands that are being executed by the `docker-entrypoint.sh`
-script in the docker log. To do that, set an environment variable using Docker's `-e VERBOSE=yes`.
-
-Instead of using this mechanism, you can of course create your own script that does setup and then call `solr-foreground`, mount that script into the container, and execute it as a command when running the container.
-
-Other ways of extending the image are to create custom Docker images that inherit from this one.
-
-## Debugging with jattach
-
-The `jcmd`, `jmap` `jstack` tools can be useful for debugging Solr inside the container. These tools are not included with the JRE, but this image includes the [jattach](https://github.com/apangin/jattach) utility which lets you do much of the same.
-
- Usage: jattach <pid> <cmd> [args ...]
-
- Commands:
- load : load agent library
- properties : print system properties
- agentProperties : print agent properties
- datadump : show heap and thread summary
- threaddump : dump all stack traces (like jstack)
- dumpheap : dump heap (like jmap)
- inspectheap : heap histogram (like jmap -histo)
- setflag : modify manageable VM flag
- printflag : print VM flag
- jcmd : execute jcmd command
-
-Example commands to do a thread dump and get heap info for PID 10:
-
- jattach 10 threaddump
- jattach 10 jcmd GC.heap_info
-
-# Updating from Docker-solr5-7 to 8
-
-For Solr 8, the docker-solr distribution switched from just extracting the Solr tar, to using the [service installation script](https://lucene.apache.org/solr/guide/7_7/taking-solr-to-production.html#service-installation-script). This was done for various reasons: to bring it in line with the recommendations by the Solr Ref Guide, to make it easier to mount volumes, and because we were [asked to](https://github.com/docker-solr/docker-solr/issues/173).
-
-This is a backwards incompatible change, and means that if you're upgrading from an older version, you will most likely need to make some changes. If you don't want to upgrade at this time, specify `apache/solr:7` as your container image. If you use `solr:8` you will use the new style. If you use just `solr` then you risk being tripped up by backwards incompatible changes; always specify at least a major version.
-
-Changes:
-
-- The Solr data is now stored in `/var/solr/data` rather than `/opt/solr/server/solr`. The `/opt/solr/server/solr/mycores` no longer exists
-- The custom `SOLR_HOME` can no longer be used, because various scripts depend on the new locations. Consequently, `INIT_SOLR_HOME` is also no longer supported.
-
-# Running under tini
-
-The Solr docker image runs Solr under [tini](https://github.com/krallin/tini), to make signal handling work better; in particular, this allows you to `kill -9` the JVM. If you run `docker run --init`, or use `init: true` in `docker-compose.yml`, or have added `--init` to `dockerd`, docker will start its `tini` and docker-solr will notice it is not PID 1, and just `exec` Solr. If you do not run with `--init`, then the docker entrypoint script detects that it is running as PID 1, and will start the `tini` present in the docker-solr image, and run Solr under that. If you really do not want to run `tini`, and just run Solr as PID 1 instead, then you can set the `TINI=no` environment variable.
-
-# Out of memory handling
-
-You can use the `OOM` environment variable to control the behaviour of the Solr JVM when an out-of-memory error occurs.
-If you specify `OOM=exit`, Solr docker will add `-XX:+ExitOnOutOfMemoryError` to the JVM arguments, so that the JVM will exit.
-If you specify `OOM=crash`, Solr docker will add `-XX:+CrashOnOutOfMemoryError` to the JVM arguments, so the JVM will crash and produces text and binary crash files (if core files are enabled).
-If you specify `OOM=script`, Solr docker will add `-XX:OnOutOfMemoryError=/opt/docker-solr/scripts/oom_solr.sh`, so the JVM will run that script (and if you want to you can mount your own in its place).
-
-# About this repository
-
-This repository is available on , and the official build is on the [Docker Hub](https://hub.docker.com/apache/solr/).
-
-# History
-
-This project was started in 2015 by [Martijn Koster](https://github.com/makuk66) in the [docker-solr](https://github.com/docker-solr/docker-solr) repository. In 2019 maintainership and copyright was transferred to the Apache Lucene/Solr project, and in 2020 the project was migrated to live within the Solr project. Many thanks to Martijn for all your contributions over the years!
diff --git a/solr/docker/docs/docker-compose.yml b/solr/docker/example/docker-compose.yml
similarity index 100%
rename from solr/docker/docs/docker-compose.yml
rename to solr/docker/example/docker-compose.yml
diff --git a/solr/docker/docs/precreate-collection.sh b/solr/docker/example/precreate-collection.sh
similarity index 100%
rename from solr/docker/docs/precreate-collection.sh
rename to solr/docker/example/precreate-collection.sh
diff --git a/solr/docker/docs/set-heap.sh b/solr/docker/example/set-heap.sh
similarity index 100%
rename from solr/docker/docs/set-heap.sh
rename to solr/docker/example/set-heap.sh
diff --git a/solr/solr-ref-guide/src/deployment-and-operations.adoc b/solr/solr-ref-guide/src/deployment-and-operations.adoc
index ffb11fed53f..df96c0554e8 100644
--- a/solr/solr-ref-guide/src/deployment-and-operations.adoc
+++ b/solr/solr-ref-guide/src/deployment-and-operations.adoc
@@ -1,5 +1,5 @@
= Deployment and Operations
-:page-children: solr-control-script-reference, solr-configuration-files, taking-solr-to-production, making-and-restoring-backups, running-solr-on-hdfs, aws-solrcloud-tutorial, upgrading-a-solr-cluster, solr-upgrade-notes
+:page-children: solr-control-script-reference, solr-configuration-files, taking-solr-to-production, making-and-restoring-backups, running-solr-in-docker, running-solr-on-hdfs, aws-solrcloud-tutorial, upgrading-a-solr-cluster, solr-upgrade-notes
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
@@ -29,6 +29,8 @@ Common administrative tasks include:
<<making-and-restoring-backups.adoc#,Making and Restoring Backups>>: Describes backup strategies for your Solr indexes.
+<<running-solr-in-docker.adoc#,Running Solr in Docker>>: How to use the official Solr Docker image.
+
<<running-solr-on-hdfs.adoc#,Running Solr on HDFS>>: How to use HDFS to store your Solr indexes and transaction logs.
<<aws-solrcloud-tutorial.adoc#,SolrCloud on AWS EC2>>: A tutorial on deploying Solr in Amazon Web Services (AWS) using EC2 instances.
diff --git a/solr/docker/Docker-FAQ.md b/solr/solr-ref-guide/src/docker-faq.adoc
similarity index 66%
rename from solr/docker/Docker-FAQ.md
rename to solr/solr-ref-guide/src/docker-faq.adoc
index b7904d0dc00..dbb84c237d6 100644
--- a/solr/docker/Docker-FAQ.md
+++ b/solr/solr-ref-guide/src/docker-faq.adoc
@@ -1,10 +1,22 @@
-
-Docker Solr FAQ
-===============
-
-
-How do I persist Solr data and config?
---------------------------------------
+= Solr Docker FAQ
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+== How do I persist Solr data and config?
Your data is persisted already, in your container's filesystem.
If you `docker run`, add data to Solr, then `docker stop` and later
@@ -17,22 +29,21 @@ container from that image, and that will have your data in it.
For some use-cases it is convenient to provide a modified `solr.in.sh` file to Solr.
For example to point Solr to a ZooKeeper host:
-```
+[source,bash]
+----
docker create --name my_solr -P solr
docker cp my_solr:/opt/solr/bin/solr.in.sh .
sed -i -e 's/#ZK_HOST=.*/ZK_HOST=cylon.lan:2181/' solr.in.sh
docker cp solr.in.sh my_solr:/opt/solr/bin/solr.in.sh
docker start my_solr
# With a browser go to http://cylon.lan:32873/solr/#/ and confirm "-DzkHost=cylon.lan:2181" in the JVM Args section.
-```
+----
But usually when people ask this question, what they are after is a way
-to store Solr data and config in a separate [Docker Volume](https://docs.docker.com/userguide/dockervolumes/).
+to store Solr data and config in a separate https://docs.docker.com/userguide/dockervolumes/[Docker Volume].
That is explained in the next two questions.
-
-How can I mount a host directory as a data volume?
---------------------------------------------------
+== How can I mount a host directory as a data volume?
This is useful if you want to inspect or modify the data in the Docker host
when the container is not running, and later easily run new containers against that data.
@@ -47,32 +58,31 @@ the whole directory a volume, then we need to provide those configuration files
in our volume, which we can do by copying them from a temporary container.
For example:
-```
+[source,bash]
+----
# create a directory to store the server/solr directory
-$ mkdir /home/docker-volumes/mysolr1
+mkdir /home/docker-volumes/mysolr1
# make sure its host owner matches the container's solr user
-$ sudo chown 8983:8983 /home/docker-volumes/mysolr1
+sudo chown 8983:8983 /home/docker-volumes/mysolr1
# copy the solr directory from a temporary container to the volume
-$ docker run -it --rm -v /home/docker-volumes/mysolr1:/target apache/solr cp -r server/solr /target/
+docker run -it --rm -v /home/docker-volumes/mysolr1:/target solr cp -r server/solr /target/
# pass the solr directory to a new container running solr
-$ SOLR_CONTAINER=$(docker run -d -P -v /home/docker-volumes/mysolr1/solr:/opt/solr/server/solr apache/solr)
+SOLR_CONTAINER=$(docker run -d -P -v /home/docker-volumes/mysolr1/solr:/opt/solr/server/solr solr)
# create a new core
-$ docker exec -it --user=solr $SOLR_CONTAINER solr create_core -c gettingstarted
+docker exec -it --user=solr $SOLR_CONTAINER solr create_core -c gettingstarted
# check the volume on the host:
-$ ls /home/docker-volumes/mysolr1/solr/
-configsets gettingstarted README.txt solr.xml zoo.cfg
-```
+ls /home/docker-volumes/mysolr1/solr/
+# configsets gettingstarted README.txt solr.xml zoo.cfg
+----
Note that if you add or modify files in that directory from the host, you must `chown 8983:8983` them.
-
-How can I use a Data Volume Container?
---------------------------------------
+== How can I use a Data Volume Container?
You can avoid the concerns about UID mismatches above, by using data volumes only from containers.
You can create a container with a volume, then point future containers at that same volume.
@@ -81,24 +91,25 @@ By separating the data and the code, you can change the code and re-use the data
But there are pitfalls:
-- if you remove the container that owns the volume, then you lose your data.
- Docker does not even warn you that a running container is dependent on it.
-- if you point multiple solr containers at the same volume, you will have multiple instances
- write to the same files, which will undoubtedly lead to corruption
-- if you do want to remove that volume, you must do `docker rm -v containername`;
- if you forget the `-v` there will be a dangling volume which you can not easily clean up.
+* if you remove the container that owns the volume, then you lose your data.
+Docker does not even warn you that a running container is dependent on it.
+* if you point multiple solr containers at the same volume, you will have multiple instances
+write to the same files, which will undoubtedly lead to corruption
+* if you do want to remove that volume, you must do `docker rm -v containername`;
+if you forget the `-v` there will be a dangling volume which you can not easily clean up.
Here is an example:
-```
+[source,bash]
+----
# create a container with a volume on the path that solr uses to store data.
docker create -v /opt/solr/server/solr --name mysolr1data solr /bin/true
# pass the volume to a new container running solr
-SOLR_CONTAINER=$(docker run -d -P --volumes-from=mysolr1data apache/solr)
+SOLR_CONTAINER=$(docker run -d -P --volumes-from=mysolr1data solr)
# create a new core
-$ docker exec -it --user=solr $SOLR_CONTAINER solr create_core -c gettingstarted
+docker exec -it --user=solr $SOLR_CONTAINER solr create_core -c gettingstarted
# make a change to the config, using the config API
docker exec -it --user=solr $SOLR_CONTAINER curl http://localhost:8983/solr/gettingstarted/config -H 'Content-type:application/json' -d'{
@@ -112,43 +123,41 @@ docker exec -it --user=solr $SOLR_CONTAINER curl http://localhost:8983/solr/gett
docker exec -it --user=solr $SOLR_CONTAINER bash -c 'cd server; java -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -jar start.jar --stop'
# create a new container
-SOLR_CONTAINER=$(docker run -d -P --volumes-from=mysolr1data apache/solr)
+SOLR_CONTAINER=$(docker run -d -P --volumes-from=mysolr1data solr)
# check our core is still there:
docker exec -it --user=solr $SOLR_CONTAINER ls server/solr/gettingstarted
# check the config modification is still there:
docker exec -it --user=solr $SOLR_CONTAINER curl http://localhost:8983/solr/gettingstarted/config/overlay?omitHeader=true
-```
-
+----
-Can I use volumes with SOLR_HOME?
----------------------------------
+== Can I use volumes with SOLR_HOME?
Solr supports a SOLR_HOME environment variable to point to a non-standard location of the Solr home directory.
You can use this in Solr docker, in combination with volumes:
-```
-docker run -it -v $PWD/mysolrhome:/mysolrhome -e SOLR_HOME=/mysolrhome apache/solr
-```
+[source,bash]
+----
+docker run -it -v $PWD/mysolrhome:/mysolrhome -e SOLR_HOME=/mysolrhome solr
+----
This does need a pre-configured directory at that location.
To make this easier, Solr docker supports a INIT_SOLR_HOME setting, which copies the contents
from the default directory in the image to the SOLR_HOME (if it is empty).
-```
+[source,bash]
+----
mkdir mysolrhome
sudo chown 8983:8983 mysolrhome
-docker run -it -v $PWD/mysolrhome:/mysolrhome -e SOLR_HOME=/mysolrhome -e INIT_SOLR_HOME=yes apache/solr
-```
+docker run -it -v $PWD/mysolrhome:/mysolrhome -e SOLR_HOME=/mysolrhome -e INIT_SOLR_HOME=yes solr
+----
-Note: If SOLR_HOME is set, the "solr-precreate" command will put the created core in the SOLR_HOME directory
+NOTE: If SOLR_HOME is set, the "solr-precreate" command will put the created core in the SOLR_HOME directory
rather than the "mycores" directory.
-
-Can I run ZooKeeper and Solr clusters under Docker?
----------------------------------------------------
+== Can I run ZooKeeper and Solr clusters under Docker?
At the network level the ZooKeeper nodes need to be able to talk to each other,
and the Solr nodes need to be able to talk to the ZooKeeper nodes and to each other.
@@ -157,28 +166,26 @@ In ZooKeeper that is done with a configuration file that lists hostnames or IP a
In Solr that is done with a parameter that specifies a host or IP address, which is then stored in ZooKeeper.
In typical clusters, those hostnames/IP addresses are pre-defined and remain static through the lifetime of the cluster.
-In Docker, inter-container communication and multi-host networking can be facilitated by [Docker Networks](https://docs.docker.com/engine/userguide/networking/).
+In Docker, inter-container communication and multi-host networking can be facilitated by https://docs.docker.com/engine/userguide/networking/[Docker Networks].
But, crucially, Docker does not normally guarantee that IP addresses of containers remain static during the lifetime of a container.
In non-networked Docker, the IP address seems to change every time you stop/start.
In a networked Docker, containers can lose their IP address in certain sequences of starting/stopping, unless you take steps to prevent that.
IP changes cause problems:
-- If you use hardcoded IP addresses in configuration, and the addresses of your containers change after a stops/start, then your cluster will stop working and may corrupt itself.
-- If you use hostnames in configuration, and the addresses of your containers change, then you might run into problems with cached hostname lookups.
-- And if you use hostnames there is another problem: the names are not defined until the respective container is running,
+* If you use hardcoded IP addresses in configuration, and the addresses of your containers change after a stops/start, then your cluster will stop working and may corrupt itself.
+* If you use hostnames in configuration, and the addresses of your containers change, then you might run into problems with cached hostname lookups.
+* And if you use hostnames there is another problem: the names are not defined until the respective container is running,
So when for example the first ZooKeeper node starts up, it will attempt a hostname lookup for the other nodes, and that will fail.
This is especially a problem for ZooKeeper 3.4.6; future versions are better at recovering.
Docker 1.10 has a new `--ip` configuration option that allows you to specify an IP address for a container.
It also has a `--ip-range` option that allows you to specify the range that other containers get addresses from.
-Used together, you can implement static addresses. See [this example](docs/docker-networking.md).
+Used together, you can implement static addresses. See the <<docker-networking.adoc#,Solr Docker networking guide>> for more information.
+== Can I run ZooKeeper and Solr with Docker Links?
-Can I run ZooKeeper and Solr with Docker Links?
------------------------------------------------
-
-Docker's [Legacy container links](https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/) provide a way to
+Docker's https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/[Legacy container links] provide a way to
pass connection configuration between containers. It only works on a single machine, on the default bridge.
It provides no facilities for static IPs.
Note: this feature is expected to be deprecated and removed in a future release.
@@ -188,110 +195,112 @@ But for some use-cases, such as quick demos or one-shot automated testing, it ca
Run ZooKeeper, and define a name so we can link to it:
-```console
-$ docker run --name zookeeper -d -p 2181:2181 -p 2888:2888 -p 3888:3888 jplock/zookeeper
-```
+[source,bash]
+----
+docker run --name zookeeper -d -p 2181:2181 -p 2888:2888 -p 3888:3888 jplock/zookeeper
+----
Run two Solr nodes, linked to the zookeeper container:
-```console
-$ docker run --name solr1 --link zookeeper:ZK -d -p 8983:8983 \
- apache/solr \
+[source,bash]
+----
+docker run --name solr1 --link zookeeper:ZK -d -p 8983:8983 \
+ solr \
bash -c 'solr start -f -z $ZK_PORT_2181_TCP_ADDR:$ZK_PORT_2181_TCP_PORT'
-$ docker run --name solr2 --link zookeeper:ZK -d -p 8984:8983 \
- apache/solr \
+docker run --name solr2 --link zookeeper:ZK -d -p 8984:8983 \
+ solr \
bash -c 'solr start -f -z $ZK_PORT_2181_TCP_ADDR:$ZK_PORT_2181_TCP_PORT'
-```
+----
Create a collection:
-```console
-$ docker exec -i -t solr1 solr create_collection \
+[source,bash]
+----
+docker exec -i -t solr1 solr create_collection \
-c gettingstarted -shards 2 -p 8983
-```
-
-Then go to `http://localhost:8983/solr/#/~cloud` (adjust the hostname for your docker host) to see the two shards and Solr nodes.
+----
+Then go to `+http://localhost:8983/solr/#/~cloud+` (adjust the hostname for your docker host) to see the two shards and Solr nodes.
-How can I run ZooKeeper and Solr with Docker Compose?
------------------------------------------------------
+== How can I run ZooKeeper and Solr with Docker Compose?
-See the [docker compose example](docs/docker-compose.yml).
+See the <<running-solr-in-docker.adoc#docker-compose,docker compose example>>.
-How can I get rid of "shared memory" warnings on Solr startup?
------------------------------------------------------
+== How can I get rid of "shared memory" warnings on Solr startup?
When starting the docker image you typically see these log lines:
-```
+[source,text]
+----
OpenJDK 64-Bit Server VM warning: Failed to reserve shared memory. (error = 1)
-```
+----
If your set up can run without huge pages or you do not require it, the least-friction way to
remove this warning is to disable large paging in the JVM via the environment variable:
-```
+[source,bash]
+----
SOLR_OPTS=-XX:-UseLargePages
-```
+----
In your Solr Admin UI, you will see listed under the JVM args both the original `-XX:+UseLargePages`
set by the `GC_TUNE` environment variable and further down the list the overriding `-XX:-UseLargePages` argument.
-
-I'm confused about the different invocations of solr -- help?
--------------------------------------------------------------
+== I'm confused about the different invocations of solr -- help?
The different invocations of the Solr docker image can look confusing, because the name of the
-image is "apache/solr" and the Solr command is also "solr", and the image interprets various arguments in
+image is "solr" and the Solr command is also "solr", and the image interprets various arguments in
special ways. I'll illustrate the various invocations:
-
To run an arbitrary command in the image:
-```
-docker run -it apache/solr date
-```
+[source,bash]
+----
+docker run -it solr date
+----
-Here "apache/solr" is the name of the image, and "date" is the command.
+Here "solr" is the name of the image, and "date" is the command.
This does not invoke any Solr functionality.
-
To run the Solr server:
-```
-docker run -it apache/solr
-```
+[source,bash]
+----
+docker run -it solr
+----
-Here "apache/solr" is the name of the image, and there is no specific command,
+Here "solr" is the name of the image, and there is no specific command,
so the image defaults to run the "solr" command with "-f" to run it in the foreground.
-
To run the Solr server with extra arguments:
-```
-docker run -it apache/solr -h myhostname
-```
+[source,bash]
+----
+docker run -it solr -h myhostname
+----
This is the same as the previous one, but an additional argument is passed.
The image will run the "solr" command with "-f -h myhostname".
To run solr as an arbitrary command:
-```
-docker run -it apache/solr solr zk --help
-```
+[source,bash]
+----
+docker run -it solr solr zk --help
+----
-Here the first "apache/solr" is the image name, and the second "solr"
+Here the first "solr" is the image name, and the second "solr"
is the "solr" command. The image runs the command exactly as specified;
no "-f" is implicitly added. The container will print help text, and exit.
If you find this visually confusing, it might be helpful to use more specific image tags,
and specific command paths. For example:
-```
-docker run -it apache/solr:6 bin/solr -f -h myhostname
-```
+[source,bash]
+----
+docker run -it solr bin/solr -f -h myhostname
+----
Finally, the Solr docker image offers several commands that do some work before
then invoking the Solr server, like "solr-precreate" and "solr-demo".
@@ -299,24 +308,27 @@ See the README.md for usage.
These are implemented by the `docker-entrypoint.sh` script, and must be passed
as the first argument to the image. For example:
-```
-docker run -it apache/solr:6 solr-demo
-```
+[source,bash]
+----
+docker run -it solr solr-demo
+----
It's important to understand an implementation detail here. The Dockerfile uses
`solr-foreground` as the `CMD`, and the `docker-entrypoint.sh` implements
that by by running "solr -f". So these two are equivalent:
-```
-docker run -it apache/solr:6
-docker run -it apache/solr:6 solr-foreground
-```
+[source,bash]
+----
+docker run -it solr
+docker run -it solr solr-foreground
+----
whereas:
-```
-docker run -it apache/solr:6 solr -f
-```
+[source,bash]
+----
+docker run -it solr solr -f
+----
is slightly different: the "solr" there is a generic command, not treated in any
special way by `docker-entrypoint.sh`. In particular, this means that the
@@ -326,14 +338,16 @@ of the other two invocations.
You also need to keep that in mind when you want to invoke solr from the bash
command. For example, this does NOT run `docker-entrypoint-initdb.d` scripts:
-```
+[source,bash]
+----
docker run -it -v $PWD/set-heap.sh:/docker-entrypoint-initdb.d/set-heap.sh \
- apache/solr:6 bash -c "echo hello; solr -f"
-```
+ solr bash -c "echo hello; solr -f"
+----
but this does:
-```
+[source,bash]
+----
docker run -it $PWD/set-heap.sh:/docker-entrypoint-initdb.d/set-heap.sh \
- apache/solr:6 bash -c "echo hello; /opt/docker-solr/scripts/docker-entrypoint.sh solr-foreground"
-```
+ solr bash -c "echo hello; /opt/docker-solr/scripts/docker-entrypoint.sh solr-foreground"
+----
diff --git a/solr/docker/docs/docker-networking.md b/solr/solr-ref-guide/src/docker-networking.adoc
similarity index 75%
rename from solr/docker/docs/docker-networking.md
rename to solr/solr-ref-guide/src/docker-networking.adoc
index ff5e4d4c46d..43cb8615488 100644
--- a/solr/docker/docs/docker-networking.md
+++ b/solr/solr-ref-guide/src/docker-networking.adoc
@@ -1,17 +1,32 @@
-Example of Zookeeper and Solr cluster with Docker networking
-------------------------------------------------------------
-
-_Note: this article dates from Jan 2016. While this approach would still work, in Jan 2019 this would typically done with Docker cluster and orchestration tools like Kubernetes. See for example [this blog post](https://lucidworks.com/2019/02/07/running-solr-on-kubernetes-part-1/)._
+= Solr & Zookeeper with Docker Networking
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+_Note: this article dates from Jan 2016. While this approach would still work, in Jan 2019 this would typically done with Docker cluster and orchestration tools like Kubernetes. See for example https://lucidworks.com/2019/02/07/running-solr-on-kubernetes-part-1/[this blog post]._
In this example I'll create a cluster with 3 ZooKeeper nodes and 3 Solr nodes, distributed over 3 machines (trinity10, trinity20, trinity30).
I'll use an overlay network, specify fixed IP addresses when creating containers, and I'll pass in explicit `/etc/hosts` entries to make sure they are available even when nodes are down.
-I won't show the configuration of the key-value store to configuration to enable networking, see [the docs](https://docs.docker.com/engine/userguide/networking/get-started-overlay/) for that.
+I won't show the configuration of the key-value store to configuration to enable networking, see https://docs.docker.com/engine/userguide/networking/get-started-overlay/[the docs] for that.
I'll not use Docker Swarm in this example, but specifically place and configure containers where I want them by ssh'ing into the appropriate Docker host.
To make this example easier to understand I'll just use shell commands.
-For actual use you may want to use a fancier deployment tool like [Fabric](http://www.fabfile.org).
+For actual use you may want to use a fancier deployment tool like http://www.fabfile.org[Fabric].
-Note: this example requires Docker 1.10.
+NOTE: this example requires Docker 1.10.
I'll run these commands from the first machine, trinity10.
@@ -20,23 +35,26 @@ addresses to use for containers, whereas the `--subnet` specifies all possible a
network. So effectively, addresses in the subnet but outside the range are reserved for containers
that specifically use the `--ip` option.
-```
+[source,bash]
+----
docker network create --driver=overlay --subnet 192.168.22.0/24 --ip-range=192.168.22.128/25 netzksolr
-```
+----
As a simple test, check the automatic assignment and specific assignment work:
-```
-$ docker run -i --rm --net=netzksolr busybox ip -4 addr show eth0 | grep inet
- inet 192.168.23.129/24 scope global eth0
-$ docker run -i --rm --net=netzksolr --ip=192.168.22.5 busybox ip -4 addr show eth0 | grep inet
- inet 192.168.22.5/24 scope global eth0
-```
+[source,bash]
+----
+docker run -i --rm --net=netzksolr busybox ip -4 addr show eth0 | grep inet
+# inet 192.168.23.129/24 scope global eth0
+docker run -i --rm --net=netzksolr --ip=192.168.22.5 busybox ip -4 addr show eth0 | grep inet
+# inet 192.168.22.5/24 scope global eth0
+----
So next create containers for ZooKeeper nodes.
First define some environment variables for convenience:
-```
+[source,bash]
+----
# the machine to run the container on
ZK1_HOST=trinity10.lan
ZK2_HOST=trinity20.lan
@@ -49,19 +67,21 @@ ZK3_IP=192.168.22.12
# the Docker image
ZK_IMAGE=jplock/zookeeper
-```
+----
Then create the containers:
-```
+[source,bash]
+----
ssh -n $ZK1_HOST "docker pull jplock/zookeeper && docker create --ip=$ZK1_IP --net netzksolr --name zk1 --hostname=zk1 --add-host zk2:$ZK2_IP --add-host zk3:$ZK3_IP -it $ZK_IMAGE"
ssh -n $ZK2_HOST "docker pull jplock/zookeeper && docker create --ip=$ZK2_IP --net netzksolr --name zk2 --hostname=zk2 --add-host zk1:$ZK1_IP --add-host zk3:$ZK3_IP -it $ZK_IMAGE"
ssh -n $ZK3_HOST "docker pull jplock/zookeeper && docker create --ip=$ZK3_IP --net netzksolr --name zk3 --hostname=zk3 --add-host zk1:$ZK1_IP --add-host zk2:$ZK2_IP -it $ZK_IMAGE"
-```
+----
Next configure those containers by creating ZooKeeper's `zoo.cfg` and `myid` files:
-```
+[source,bash]
+----
# Add ZooKeeper nodes to the ZooKeeper config.
# If you use hostnames here, ZK will complain with UnknownHostException about the other nodes.
# In ZooKeeper 3.4.6 that stays broken forever; in 3.4.7 that does recover.
@@ -82,11 +102,12 @@ rm zoo.cfg
echo 1 | ssh $ZK1_HOST 'dd of=myid && docker cp myid zk1:/tmp/zookeeper/myid && rm myid'
echo 2 | ssh $ZK2_HOST 'dd of=myid && docker cp myid zk2:/tmp/zookeeper/myid && rm myid'
echo 3 | ssh $ZK3_HOST 'dd of=myid && docker cp myid zk3:/tmp/zookeeper/myid && rm myid'
-```
+----
Now start the containers:
-```
+[source,bash]
+----
ssh -n $ZK1_HOST 'docker start zk1'
ssh -n $ZK2_HOST 'docker start zk2'
ssh -n $ZK3_HOST 'docker start zk3'
@@ -113,12 +134,14 @@ ssh -n $ZK3_HOST "docker exec -i zk3 bash -c 'echo stat | nc localhost 2181'"
# Optional: verify we can connect a zookeeper client. This should show the `[zookeeper]` znode.
printf "ls /\nquit\n" | ssh $ZK1_HOST docker exec -i zk1 /opt/zookeeper/bin/zkCli.sh
-```
+----
That's the ZooKeeper cluster running.
Next, we create Solr containers in much the same way:
-```
+
+[source,bash]
+----
ZKSOLR1_HOST=trinity10.lan
ZKSOLR2_HOST=trinity20.lan
ZKSOLR3_HOST=trinity30.lan
@@ -134,11 +157,12 @@ HOST_OPTIONS="--add-host zk1:$ZK1_IP --add-host zk2:$ZK2_IP --add-host zk3:$ZK3_
ssh -n $ZKSOLR1_HOST "docker pull $SOLR_IMAGE && docker create --ip=$ZKSOLR1_IP --net netzksolr --name zksolr1 --hostname=zksolr1 -it $HOST_OPTIONS $SOLR_IMAGE"
ssh -n $ZKSOLR2_HOST "docker pull $SOLR_IMAGE && docker create --ip=$ZKSOLR2_IP --net netzksolr --name zksolr2 --hostname=zksolr2 -it $HOST_OPTIONS $SOLR_IMAGE"
ssh -n $ZKSOLR3_HOST "docker pull $SOLR_IMAGE && docker create --ip=$ZKSOLR3_IP --net netzksolr --name zksolr3 --hostname=zksolr3 -it $HOST_OPTIONS $SOLR_IMAGE"
-```
+----
Now configure Solr to know where its ZooKeeper cluster is, and start the containers:
-```
+[source,bash]
+----
for h in zksolr1 zksolr2 zksolr3; do
docker cp zksolr1:/opt/solr/bin/solr.in.sh .
sed -i -e 's/#ZK_HOST=""/ZK_HOST="zk1:2181,zk2:2181,zk3:2181"/' solr.in.sh
@@ -168,52 +192,56 @@ ssh -n $ZKSOLR3_HOST docker logs zksolr3
ssh -n $ZKSOLR1_HOST "docker exec -i zksolr1 /bin/bash -c 'wget -O - http://zksolr1:8983/'"
ssh -n $ZKSOLR2_HOST "docker exec -i zksolr2 /bin/bash -c 'wget -O - http://zksolr2:8983/'"
ssh -n $ZKSOLR3_HOST "docker exec -i zksolr3 /bin/bash -c 'wget -O - http://zksolr3:8983/'"
-```
+----
Next let's create a collection:
-```
+[source,bash]
+----
ssh -n $ZKSOLR1_HOST docker exec -i zksolr1 /opt/solr/bin/solr create_collection -c my_collection1 -shards 2 -p 8983
-```
+----
To load data, and see it was split over shards:
-```
-mak@trinity10:~$ docker exec -it --user=solr zksolr1 bin/post -c my_collection1 example/exampledocs/manufacturers.xml
-/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -classpath /opt/solr/dist/solr-core-5.4.0.jar -Dauto=yes -Dc=my_collection1 -Ddata=files org.apache.solr.util.SimplePostTool example/exampledocs/manufacturers.xml
-SimplePostTool version 5.0.0
-Posting files to [base] url http://localhost:8983/solr/my_collection1/update...
-Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
-POSTing file manufacturers.xml (application/xml) to [base]
-1 files indexed.
-COMMITting Solr index changes to http://localhost:8983/solr/my_collection1/update...
-Time spent: 0:00:01.093
-mak@trinity10:~$ docker exec -it --user=solr zksolr1 bash -c "wget -q -O - 'http://zksolr1:8983/solr/my_collection1/select?q=*:*&indent=true&rows=100&fl=id' | egrep '<str name=.id.>' | wc -l"
+[source,bash]
+----
+docker exec -it --user=solr zksolr1 bin/post -c my_collection1 example/exampledocs/manufacturers.xml
+# /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -classpath /opt/solr/dist/solr-core-9.0.0.jar -Dauto=yes -Dc=my_collection1 -Ddata=files org.apache.solr.util.SimplePostTool example/exampledocs/manufacturers.xml
+# SimplePostTool version 9.0.0
+# Posting files to [base] url http://localhost:8983/solr/my_collection1/update...
+# Entering auto mode. File endings considered are xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
+# POSTing file manufacturers.xml (application/xml) to [base]
+# 1 files indexed.
+# COMMITting Solr index changes to http://localhost:8983/solr/my_collection1/update...
+# Time spent: 0:00:01.093
+docker exec -it --user=solr zksolr1 bash -c "wget -q -O - 'http://zksolr1:8983/solr/my_collection1/select?q=*:*&indent=true&rows=100&fl=id' | egrep '<str name=.id.>' | wc -l"
11
-mak@trinity10:~$ docker exec -it --user=solr zksolr1 bash -c "wget -q -O - 'http://zksolr1:8983/solr/my_collection1/select?q=*:*&shards=shard1&rows=100&indent=true&fl=id' | grep '<str name=.id.>' | wc -l"
+docker exec -it --user=solr zksolr1 bash -c "wget -q -O - 'http://zksolr1:8983/solr/my_collection1/select?q=*:*&shards=shard1&rows=100&indent=true&fl=id' | grep '<str name=.id.>' | wc -l"
4
-mak@trinity10:~$ docker exec -it --user=solr zksolr1 bash -c "wget -q -O - 'http://zksolr1:8983/solr/my_collection1/select?q=*:*&shards=shard2&rows=100&indent=true&fl=id' | grep '<str name=.id.>' | wc -l"
+docker exec -it --user=solr zksolr1 bash -c "wget -q -O - 'http://zksolr1:8983/solr/my_collection1/select?q=*:*&shards=shard2&rows=100&indent=true&fl=id' | grep '<str name=.id.>' | wc -l"
7
-```
+----
Now to get external access to this overlay network from outside we can use a container to proxy the connections.
-For a simple TCP proxy container with an exposed port on the Docker host, proxying to a single Solr node, you can use [brandnetworks/tcpproxy](https://github.com/brandnetworks/tcpproxy):
+For a simple TCP proxy container with an exposed port on the Docker host, proxying to a single Solr node, you can use https://github.com/brandnetworks/tcpproxy[brandnetworks/tcpproxy]:
-```
+[source,bash]
+----
ssh -n trinity10.lan "docker pull brandnetworks/tcpproxy && docker run -p 8001 -p 8002 --net netzksolr --name zksolrproxy --hostname=zksolrproxy.netzksolr -tid brandnetworks/tcpproxy --connections 8002:zksolr1:8983"
docker port zksolrproxy 8002
-```
+----
-Or use a suitably configured HAProxy to round-robin between all Solr nodes. Or, instead of the overlay network, use [Project Calico](http://www.projectcalico.org) and configure L3 routing so you do not need to mess with proxies.
+Or use a suitably configured HAProxy to round-robin between all Solr nodes. Or, instead of the overlay network, use http://www.projectcalico.org[Project Calico] and configure L3 routing so you do not need to mess with proxies.
-Now I can get to Solr on http://trinity10:32774/solr/#/. In the Cloud -> Tree -> /live_nodes view I see the Solr nodes.
+Now I can get to Solr on `http://trinity10:32774/solr/#/`. In the Cloud -> Tree -> /live_nodes view I see the Solr nodes.
From the Solr UI select the collection1 core, and click on Cloud -> Graph to see how it has created
two shards across our Solr nodes.
Now, by way of test, we'll stop the Solr containers, and start them out-of-order, and verify the IP addresses are unchanged, and check the same results come back:
-```
+[source,bash]
+----
ssh -n $ZKSOLR1_HOST docker kill zksolr1
ssh -n $ZKSOLR2_HOST docker kill zksolr2
ssh -n $ZKSOLR3_HOST docker kill zksolr3
@@ -231,13 +259,14 @@ ssh -n $ZKSOLR3_HOST 'docker inspect --format "{{ .NetworkSettings.Networks.netz
docker exec -it --user=solr zksolr1 bash -c "wget -q -O - 'http://zksolr1:8983/solr/my_collection1/select?q=*:*&indent=true&rows=100&fl=id' | egrep '<str name=.id.>' | wc -l"
docker exec -it --user=solr zksolr1 bash -c "wget -q -O - 'http://zksolr1:8983/solr/my_collection1/select?q=*:*&shards=shard1&rows=100&indent=true&fl=id' | grep '<str name=.id.>' | wc -l"
docker exec -it --user=solr zksolr1 bash -c "wget -q -O - 'http://zksolr1:8983/solr/my_collection1/select?q=*:*&shards=shard2&rows=100&indent=true&fl=id' | grep '<str name=.id.>' | wc -l"
-```
+----
Good, that works.
-
Finally To clean up this example:
-```
+
+[source,bash]
+----
ssh -n $ZK1_HOST "docker kill zk1; docker rm zk1"
ssh -n $ZK2_HOST "docker kill zk2; docker rm zk2"
ssh -n $ZK3_HOST "docker kill zk3; docker rm zk3"
@@ -246,4 +275,4 @@ ssh -n $ZKSOLR2_HOST "docker kill zksolr2; docker rm zksolr2"
ssh -n $ZKSOLR3_HOST "docker kill zksolr3; docker rm zksolr3"
ssh -n trinity10.lan "docker kill zksolrproxy; docker rm zksolrproxy"
docker network rm netzksolr
-```
+----
diff --git a/solr/solr-ref-guide/src/running-solr-in-docker.adoc b/solr/solr-ref-guide/src/running-solr-in-docker.adoc
new file mode 100644
index 00000000000..96d8ef0aac7
--- /dev/null
+++ b/solr/solr-ref-guide/src/running-solr-in-docker.adoc
@@ -0,0 +1,306 @@
+= Running Solr in Docker
+:page-children: docker-faq, docker-networking
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+== Getting started with the Docker image
+
+Instructions below apply to `solr:8.0.0` and above.
+
+
+See the https://hub.docker.com/_/solr?tab=tags[Docker Hub page] for a full list of tags and architectures available.
+
+=== Running Solr with host-mounted directories
+
+Typically users first want to run a single standalone Solr server in a container, with a single core for data, while storing data in a local directory.
+This is a convenient mechanism for developers, and could be used for single-server production hosts too.
+
+[source,bash]
+----
+mkdir solrdata
+docker run -d -v "$PWD/solrdata:/var/solr" -p 8983:8983 --name my_solr solr solr-precreate gettingstarted
+----
+
+Then with a web browser go to `+http://localhost:8983/+` to see the Admin Console (adjust the hostname for your docker host).
+In the web UI if you click on "Core Admin" you should now see the "gettingstarted" core.
+
+Next load some of the example data that is included in the container:
+
+[source,bash]
+----
+docker exec -it my_solr post -c gettingstarted example/exampledocs/manufacturers.xml
+----
+
+In the UI, find the "Core selector" popup menu and select the "gettingstarted" core, then select the "Query" menu item. This gives you a default search for `*:*` which returns all docs. Hit the "Execute Query" button, and you should see a few docs with data. Congratulations!
+
+=== Docker-Compose
+
+You can use Docker Compose to run a single standalone server or a multi-node cluster.
+And you could use Docker Volumes instead of host-mounted directories.
+For example, with a `docker-compose.yml` containing the following:
+
+[source,yaml]
+----
+version: '3'
+services:
+ solr:
+ image: solr
+ ports:
+ - "8983:8983"
+ volumes:
+ - data:/var/solr
+ command:
+ - solr-precreate
+ - gettingstarted
+volumes:
+ data:
+----
+
+you can simply run:
+
+[source,bash]
+----
+docker-compose up -d
+----
+
+=== Single-command demo
+
+For quick demos of Solr docker, there is a single command that starts Solr, creates a collection called "demo", and loads sample data into it:
+
+[source,bash]
+----
+docker run --name solr_demo -d -p 8983:8983 solr solr-demo
+----
+
+== How the image works
+
+The container contains an installation of Solr, as installed by the <<taking-solr-to-production.adoc#service-installation-script,service installation script>>.
+This stores the Solr distribution in `/opt/solr`, and configures Solr to use `/var/solr` to store data and logs, using the `/etc/default/solr` file for configuration.
+If you want to persist the data, mount a volume or directory on `/var/solr`.
+Solr expects some files and directories in `/var/solr`; if you use your own directory or volume you can either pre-populate them, or let Solr docker copy them for you.
+If you want to use custom configuration, mount it in the appropriate place. See below for examples.
+
+The Solr docker distribution adds scripts in `/opt/solr/docker/scripts` to make it easier to use under Docker, for example to create cores on container startup.
+
+=== Creating cores
+
+When Solr runs in standalone mode, you create "cores" to store data. On a non-Docker Solr, you would run the server in the background, then use the <<solr-control-script-reference.adoc#,Solr control script>> to create cores and load data. With Solr docker you have various options.
+
+The first is exactly the same: start Solr running in a container, then execute the control script manually in the same container:
+
+[source,bash]
+----
+docker run -d -p 8983:8983 --name my_solr solr
+docker exec -it my_solr solr create_core -c gettingstarted
+----
+
+This is not very convenient for users, and makes it harder to turn it into configuration for Docker Compose and orchestration tools like Kubernetes.
+So, typically you will use the `solr-precreate` command which prepares the specified core and then runs Solr:
+
+[source,bash]
+----
+docker run -d -p 8983:8983 --name my_solr solr solr-precreate gettingstarted
+----
+
+The `solr-precreate` command takes an optional extra argument to specify a configset directory below `/opt/solr/server/solr/configsets/`.
+This allows you to specify your own config. See https://github.com/docker-solr/docker-solr-examples/tree/master/custom-configset[this example].
+
+The third option is to use the `solr-create` command. This runs a Solr in the background in the container, then uses the Solr control script to create the core, then stops the Solr server and restarts it in the foreground. This method is less popular because the double Solr run can be confusing.
+
+[source,bash]
+----
+docker run -d -p 8983:8983 --name my_solr solr solr-create -c gettingstarted
+----
+
+Finally, you can run your own command-line and specify what to do, and even invoke mounted scripts. For example:
+
+[source,bash]
+----
+docker run -p 8983:8983 -v $PWD/mysetup.sh:/mysetup.sh --name my_solr solr bash -c "precreate-core gettingstarted && source /mysetup.sh && solr-foreground"
+----
+
+=== Creating collections
+
+In a "SolrCloud" cluster you create "collections" to store data; and again you have several options for creating a core.
+
+These examples assume you're running a <<docker-compose,docker compose cluster>>.
+
+The first way to create a collection is to go to the http://localhost:8983/[Solr Admin UI], select "Collections" from the left-hand side navigation menu, then press the "Add Collection" button, give it a name, select the `_default` config set, then press the "Add Collection" button.
+
+The second way is through the Solr control script on one of the containers:
+
+[source,bash]
+----
+docker exec solr1 solr create -c gettingstarted2
+----
+
+The third way is to use a separate container:
+
+[source,bash]
+----
+docker run -e SOLR_HOST=solr1 --network docs_solr solr solr create_collection -c gettingstarted3 -p 8983
+----
+
+The fourth way is to use the remote API, from the host or from one of the containers, or some new container on the same network (adjust the hostname accordingly):
+
+[source,bash]
+----
+curl 'http://localhost:8983/solr/admin/collections?action=CREATE&name=gettingstarted3&numShards=1&collection.configName=_default'
+----
+
+If you want to use a custom config for your collection, you first need to upload it, and then refer to it by name when you create the collection.
+See the Ref guide on how to use the <<solr-control-script-reference.adoc#upload-a-configuration-set,ZooKeeper upload>> or the <<configsets-api.adoc#configsets-upload,ConfigSet API>>.
+
+=== Loading your own data
+
+There are several ways to load data; let's look at the most common ones.
+
+The most common first deployment is to run Solr standalone (not in a cluster), on a workstation or server, where you have local data you wish to load.
+One way of doing that is using a separate container, with a mounted volume containing the data, using the host network so you can connect to the mapped port:
+
+[source,bash]
+----
+# start Solr. Listens on localhost:8983
+docker run --name my_solr -p 8983:8983 solr solr-precreate books
+
+# get data
+mkdir mydata
+wget -O mydata/books.csv https://raw.githubusercontent.com/apache/solr/main/solr/example/exampledocs/books.csv
+docker run --rm -v "$PWD/mydata:/mydata" --network=host solr post -c books /mydata/books.csv
+----
+
+The same works if you use the <<docker-compose,example docker compose cluster>>, or you can just start your loading container in the same network:
+
+[source,bash]
+----
+docker run -e SOLR_HOST=solr1 --network=mycluster_solr solr solr create_collection -c books -p 8983
+docker run --rm -v "$PWD/mydata:/mydata" --network=mycluster_solr solr post -c books /mydata/books.csv -host solr1
+----
+
+Alternatively, you can make the data available on a volume at Solr start time, and then load it from `docker exec` or a custom start script.
+
+=== solr.in.sh configuration
+
+In Solr it is common to configure settings in https://github.com/apache/solr/blob/main/solr/bin/solr.in.sh[solr.in.sh],
+as documented in the <<taking-solr-to-production.adoc#environment-overrides-include-file,Taking Solr to Production page>>.
+
+The `solr.in.sh` file can be found in `/etc/default`:
+
+[source,bash]
+----
+docker run solr cat /etc/default/solr.in.sh
+----
+
+It has various commented-out values, which you can override when running the container, like:
+
+[source,bash]
+----
+docker run -d -p 8983:8983 -e SOLR_HEAP=800m solr
+----
+
+You can also mount your own config file. Do no modify the values that are set at the end of the file.
+
+=== Extending the image
+
+The Solr docker image has an extension mechanism. At run time, before starting Solr, the container will execute scripts
+in the `/docker-entrypoint-initdb.d/` directory. You can add your own scripts there either by using mounted volumes
+or by using a custom Dockerfile. These scripts can for example copy a core directory with pre-loaded data for continuous
+integration testing, or modify the Solr configuration.
+
+Here is a simple example. With a `custom.sh` script like:
+
+[source,bash]
+----
+#!/bin/bash
+set -e
+echo "this is running inside the container before Solr starts"
+----
+
+you can run:
+
+[source,console]
+----
+$ docker run --name solr_custom1 -d -v $PWD/custom.sh:/docker-entrypoint-initdb.d/custom.sh solr
+$ sleep 5
+$ docker logs solr_custom1 | head
+/opt/docker-solr/scripts/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/set-heap.sh
+this is running inside the container before Solr starts
+
+Starting Solr on port 8983 from /opt/solr/server
+----
+
+With this extension mechanism it can be useful to see the shell commands that are being executed by the `docker-entrypoint.sh`
+script in the docker log. To do that, set an environment variable using Docker's `-e VERBOSE=yes`.
+
+Instead of using this mechanism, you can of course create your own script that does setup and then call `solr-foreground`, mount that script into the container, and execute it as a command when running the container.
+
+Other ways of extending the image are to create custom Docker images that inherit from this one.
+
+=== Debugging with jattach
+
+The `jcmd`, `jmap` `jstack` tools can be useful for debugging Solr inside the container. These tools are not included with the JRE, but this image includes the https://github.com/apangin/jattach[jattach] utility which lets you do much of the same.
+
+....
+Usage: jattach <pid> <cmd> [args ...]
+
+ Commands:
+ load : load agent library
+ properties : print system properties
+ agentProperties : print agent properties
+ datadump : show heap and thread summary
+ threaddump : dump all stack traces (like jstack)
+ dumpheap : dump heap (like jmap)
+ inspectheap : heap histogram (like jmap -histo)
+ setflag : modify manageable VM flag
+ printflag : print VM flag
+ jcmd : execute jcmd command
+....
+
+Example commands to do a thread dump and get heap info for `PID 10`:
+
+
+[source,bash]
+----
+jattach 10 threaddump
+jattach 10 jcmd GC.heap_info
+----
+
+== Updating from Solr 5-7 to 8+
+
+In Solr 8, the Solr Docker image switched from just extracting the Solr tar, to using the <<taking-solr-to-production.adoc#service-installation-script,service installation script>>. This was done for various reasons: to bring it in line with the recommendations by the Solr Ref Guide and to make it easier to mount volumes.
+
+This is a backwards incompatible change, and means that if you're upgrading from an older version, you will most likely need to make some changes. If you don't want to upgrade at this time, specify `solr:7` as your container image. If you use `solr:8` you will use the new style. If you use just `solr` then you risk being tripped up by backwards incompatible changes; always specify at least a major version.
+
+Changes:
+
+* The Solr data is now stored in `/var/solr/data` rather than `/opt/solr/server/solr`. The `/opt/solr/server/solr/mycores` no longer exists
+* The custom `SOLR_HOME` can no longer be used, because various scripts depend on the new locations. Consequently, `INIT_SOLR_HOME` is also no longer supported.
+
+== Running under tini
+
+The Solr docker image runs Solr under https://github.com/krallin/tini[tini], to make signal handling work better; in particular, this allows you to `kill -9` the JVM. If you run `docker run --init`, or use `init: true` in `docker-compose.yml`, or have added `--init` to `dockerd`, docker will start its `tini` and docker-solr will notice it is not PID 1, and just `exec` Solr. If you do not run with `--init`, then the docker entrypoint script detects that it is running as PID 1, and will start the `tini` present in the docker-solr image, and run Solr under that. If you really do not want to run `tini`, and just run Solr as PID 1 instead, then you can set the `TINI=no` environment variable.
+
+== Out of memory handling
+
+You can use the `OOM` environment variable to control the behaviour of the Solr JVM when an out-of-memory error occurs.
+If you specify `OOM=exit`, Solr docker will add `-XX:+ExitOnOutOfMemoryError` to the JVM arguments, so that the JVM will exit.
+If you specify `OOM=crash`, Solr docker will add `-XX:+CrashOnOutOfMemoryError` to the JVM arguments, so the JVM will crash and produces text and binary crash files (if core files are enabled).
+If you specify `OOM=script`, Solr docker will add `-XX:OnOutOfMemoryError=/opt/docker-solr/scripts/oom_solr.sh`, so the JVM will run that script (and if you want to you can mount your own in its place).
+
+== History
+
+The Docker-Solr project was started in 2015 by https://github.com/makuk66[Martijn Koster] in the https://github.com/docker-solr/docker-solr[docker-solr] repository. In 2019 maintainership and copyright was transferred to the Apache Lucene/Solr project, and in 2020 the project was migrated to live within the Solr project. Many thanks to Martijn for all your contributions over the years!