Skip to content

armgit / ente

Public Imported from GitHub
0

πŸ’š End-to-end encrypted cloud for everything.

Tighten server operational documentation

Manav Rathidfde22d5265b
diff --git a/server/RUNNING.md b/server/RUNNING.md
index ecc05260e3..884ba7f6c7 100644
--- a/server/RUNNING.md
+++ b/server/RUNNING.md
@@ -1,14 +1,14 @@
# Running Museum
-You can run a Docker compose cluster containing museum, the web app, and the essential auxiliary services it requires (database and object storage). This is the easiest and simplest way to get started, and also provides an isolated environment that doesn't clutter your machine.
+The Docker Compose setup runs Museum, Postgres, and MinIO in an isolated cluster. This is the easiest way to get started.
-You can also run museum directly on your machine if you wish - it is a single static go binary.
+You can also run the static Go binary directly on your machine.
-This document describes these different approaches (you can choose any one), and also outlines configuration.
+This document covers both approaches and Museum's configuration.
- [Run using pre-built Docker images](docs/quickstart.md)
- [Run using Docker, building image from source](#build-and-run-using-docker)
-- [Run with Docker, Γ  la carte](#pre-built-images)
+- [Use individual pre-built images](#pre-built-images)
- [Run without Docker](#running-without-docker)
- [Configuration](#configuration)
@@ -22,44 +22,44 @@ For more details, see [docs/quickstart.md](docs/quickstart.md).
## Build and run using Docker
-Start the cluster (in the `ente/server` directory)
+From `ente/server`, start the cluster:
```sh
docker compose up --build
```
-Once the cluster has started, you should be able to do call museum
+Once the cluster has started, call Museum:
```sh
curl http://localhost:8080/ping
```
-Or connect from the [web app](../web)
+Or connect from the [web app](../web):
```sh
NEXT_PUBLIC_ENTE_ENDPOINT=http://localhost:8080 npm run dev
```
-Or connect from the [mobile app](../mobile)
+Or connect from the [mobile app](../mobile):
```sh
flutter run --dart-define=endpoint=http://localhost:8080
```
-Or interact with the other services in the cluster, e.g. connect to the DB
+Connect to Postgres:
```sh
docker compose exec postgres env PGPASSWORD=pgpass psql -U pguser -d ente_db
```
-Or interact with the MinIO S3 API
+Use the MinIO S3 API:
```sh
AWS_ACCESS_KEY_ID=changeme AWS_SECRET_ACCESS_KEY=changeme1234 \
aws s3 --endpoint-url http://localhost:3200 ls s3://b2-eu-cen
```
-Or open the MinIO dashboard at http://localhost:3201
+Or open the MinIO dashboard at http://localhost:3201.
> [!NOTE]
>
@@ -67,15 +67,15 @@ Or open the MinIO dashboard at http://localhost:3201
> [!WARNING]
>
-> The default credentials are user changeme / password changeme1234. Goes without saying, but remember to change them!
+> The sample credentials are user `changeme` and password `changeme1234`. Change them before any non-local use.
> [!NOTE]
>
-> While we've provided a MinIO based Docker compose file to make it easy for people to get started, if you're running it in production we recommend using an external S3.
+> For production, we recommend using external S3-compatible storage instead of the bundled MinIO service.
### Cleanup
-Persistent data is stored in Docker volumes and will persist across container restarts. The volume can be saved / inspected using the `docker volumes` command.
+Persistent data is stored in Docker volumes and survives container restarts. Inspect the volumes with `docker volume ls`.
To remove stopped containers, use `docker compose rm`. To also remove volumes, use `docker compose down -v`.
@@ -85,21 +85,21 @@ You can spin up independent clusters, each with its own volumes, by using the `-
### Pruning images
-Each time museum gets rebuilt from source, a new image gets created but the old one is retained as a dangling image. You can use `docker image prune --force`, or `docker system prune` if that's fine with you, to remove these.
+Each source build creates a new image and leaves the old one dangling. Remove dangling images with `docker image prune --force`, or use `docker system prune` for a broader cleanup.
## Pre-built images
-## server
+### Server
-If you have setup the database and object storage externally and only want to run Ente's server, you can just pull and run the image from **`ghcr.io/ente/server`**.
+If you have provisioned Postgres and object storage separately, pull the server image from `ghcr.io/ente/server`.
```sh
docker pull ghcr.io/ente/server
```
-## web
+### Web
-Similarly, there is a pre-built Docker image containing all the web apps which you can just pull and run the from **`ghcr.io/ente/web`**.
+The image at `ghcr.io/ente/web` contains all the web apps.
```sh
docker pull ghcr.io/ente/web
@@ -109,7 +109,7 @@ For details about configuring the web image, see [web/docs/docker.md](../web/doc
## Running without Docker
-The museum binary can be run by using `go run cmd/museum/main.go`. But first, you'll need to prepare your machine for development. Here we give the steps, with examples that work for macOS (please adapt to your OS).
+The following development setup runs Museum directly on macOS. Adapt the package installation commands for other operating systems.
### Install [Go](https://golang.org/dl/)
@@ -125,7 +125,7 @@ brew install postgresql@15
> [!NOTE]
>
-> Here we install same major version of Postgres as our production database to avoid surprises, but if you're using a newer Postgres that should work fine too.
+> This installs the same Postgres major version used in production. Newer versions should also work.
Run the service:
@@ -147,7 +147,7 @@ brew services run postgresql@15
> brew services start postgresql@15
> ```
-Create the database and user (one time)
+Create the database and user once:
```sh
psql postgres -c "CREATE USER pguser WITH PASSWORD 'pgpass';"
@@ -165,9 +165,6 @@ psql postgres -c "CREATE DATABASE ente_db OWNER pguser;"
> ```sh
> psql ente_db
> ```
->
-> Data is stored in `/opt/homebrew/var/postgresql@15`
-
### Install Local S3
If you don't have a test S3 bucket, you can run a S3 compatible API locally. This section outlines using minio. Garage is also a newer alternative.
@@ -177,7 +174,7 @@ brew install minio minio-mc
brew services run minio
```
-One time bucket creation.
+Create the bucket once:
```sh
mc alias set local http://127.0.0.1:9000 minioadmin minioadmin
@@ -238,9 +235,7 @@ The script creates a temporary `ente_test_*` database on the selected Postgres i
## Configuration
-Now that you have museum running (either inside Docker or standalone), we can talk about configuring it.
-
-By default, museum runs in the "local" configuration using values specified in `local.yaml`.
+By default, Museum uses the values in `configurations/local.yaml`.
To override these values, you can create a file named `museum.yaml` in the current directory. This path is git-ignored for convenience. Note that if you run the Docker compose cluster without creating this file, Docker will create an empty directory named `museum.yaml` which you can `rmdir` if you need to provide a config file later on.
@@ -248,4 +243,4 @@ The keys and values supported by this configuration file are documented in [conf
> [!TIP]
>
-> If your mobile app is able to connect to your self hosted instance but is not able to view or upload images, see [ente.com/help/self-hosting/administration/object-storage](https://ente.com/help/self-hosting/administration/object-storage).
+> If your mobile app can connect to your self-hosted instance but cannot view or upload images, see [ente.com/help/self-hosting/administration/object-storage](https://ente.com/help/self-hosting/administration/object-storage).
diff --git a/server/config/example.env b/server/config/example.env
index cce51cc3c5..37aae48998 100644
--- a/server/config/example.env
+++ b/server/config/example.env
@@ -1,27 +1,14 @@
-# Copy this file to .env in the same directory in which this file is present
-# This file contains the environment variables needed for Ente's cluster to run properly.
-# This file is split based on services declared in Docker Compose file
-#
-# Service: Postgres
-# This is used for storing data in database pertaining to collections, files, users, subscriptions, etc.
-# These credentials are needed for accessing the database via Museum.
-# Please set a strong password for accessing the database.
-# Enter these values in museum.yaml file under `db`.
-# This need not be defined if using external DB (i. e. no Compose service for PostgreSQL is used)
+# Copy to .env beside config/compose.yaml.
+
+# These must match db.user, db.password, and db.name in museum.yaml.
POSTGRES_USER=pguser
POSTGRES_PASSWORD=<password>
POSTGRES_DB=ente_db
-# Service: MinIO
-# This is used for MinIO object storage service that's shipped by default with the compose file
-# to reduce need for an external S3-compatible bucket for quick testing.
-# It is recommended to use an external bucket for long-term usage.
-# The credentials required for accessing the object storage is documented below.
-# It is not needed to configure these variables if you are using an external bucket
-# Enter the user value into key and password in secret for buckets in museum.yaml under `s3` section.
+# These must match the key and secret of each MinIO-backed s3 bucket in museum.yaml.
+# Omit them when using external buckets instead of the bundled MinIO service.
MINIO_ROOT_USER=-
MINIO_ROOT_PASSWORD=<secret>
-# Service: Web
-# Replace this with the API endpoint for Museum.
+# Museum endpoint used by the web apps.
ENTE_API_ORIGIN=http://localhost:8080
diff --git a/server/config/example.yaml b/server/config/example.yaml
index a5a162521e..4be0c73fed 100644
--- a/server/config/example.yaml
+++ b/server/config/example.yaml
@@ -1,13 +1,6 @@
-# Copy this file to museum.yaml in the same directory in which this file is present
+# Copy to museum.yaml beside config/compose.yaml.
-# This section is meant for configuration of database.
-# Museum uses these values and credentials for connecting
-# to the database.
-# Set a strong password and if using PostgreSQL Docker container
-# provided in Docker Compose file, ensure db.password is same
-# as POSTGRES_PASSWORD
-# Similarly ensure db.user and db.name are same as POSTGRES_USER and
-# POSTGRES_DB respectively
+# These must match the POSTGRES_* values in .env when using the bundled Postgres service.
db:
host: postgres
port: 5432
@@ -15,12 +8,11 @@ db:
user: pguser
password: <password>
-# This section is for configuring storage buckets. Omit this section if
-# you only intend to use Ente Auth
+# Omit s3 when using only Ente Auth.
s3:
- # Change this to false if enabling SSL
+ # Set false for HTTPS endpoints.
are_local_buckets: true
- # Only path-style URL works if disabling are_local_buckets with MinIO
+ # MinIO requires path-style URLs when are_local_buckets is false.
use_path_style_urls: true
b2-eu-cen:
# Uncomment the below configuration to override the top-level configuration
@@ -49,7 +41,7 @@ s3:
region: eu-central-2
bucket: scw-eu-fr-v3
-# Specify the base endpoints for various web apps you're running.
+# Base endpoints for the web apps you run.
apps:
public-albums: http://localhost:3002
embed-albums: http://localhost:3006
@@ -73,16 +65,13 @@ space:
# Bare email address. webpush-go adds the mailto: prefix.
subscriber:
-# Key used for encrypting customer emails before storing them in DB
+# Replace these fixed development secrets before storing real data.
#
# To make it easy to get started, some randomly generated (but fixed) values are
# provided here. But if you're really going to be using museum, please generate
-# new keys. You can use `go run tools/gen-random-keys/main.go` for that.
-#
-# Replace values in key and JWT for security
+# new keys using `go run tools/gen-random-keys/main.go`.
key:
encryption: yvmG/RnzKrbCb9L3mgsmoxXr9H7i2Z4qlbT0mL3ln4w=
hash: KXYiG07wC7GIgvCSdg+WmyWdXDAn6XKYJtp/wkEU7x573+byBRAYtpTP0wwvi8i/4l37uicX1dVTUzwH3sLZyw==
-# JWT secrets
jwt:
secret: i2DecQmfGreG6q1vBj5tCokhlN41gcfS2cjOs9Po-u8=
diff --git a/server/configurations/local.yaml b/server/configurations/local.yaml
index 4457831af6..a3fd19649c 100644
--- a/server/configurations/local.yaml
+++ b/server/configurations/local.yaml
@@ -1,66 +1,33 @@
-# Configuring museum
-# ------------------
+# Museum loads configuration in this order:
#
-# 1. If the environment variable `ENVIRONMENT` is specified, then it is used to
-# load one of the files from the `configurations/` directory. If not present,
-# then by default `local.yaml` (this file) will get loaded.
+# 1. configurations/$ENVIRONMENT.yaml, or this file when ENVIRONMENT is unset.
+# 2. credentials-file, when present.
+# 3. museum.yaml in the current working directory, when present.
+# 4. ENTE_* environment variables.
#
-# 2. If the "credentials-file" config option is set then museum will look for
-# that file. If found, that file will also be loaded, and entries specified
-# therein will override the defaults specified here.
+# Later sources override earlier ones.
#
-# 3. Then, museum will look for a file named `museum.yaml` (gitignored) in the
-# current working directory. If found, this file will also be loaded and
-# merged.
+# Environment variables replace nesting and hyphens with underscores:
#
-# 4. Config can be overridden with via environment variables (details below).
+# db.user -> ENTE_DB_USER
+# s3.b2-eu-cen -> ENTE_S3_B2_EU_CEN
#
-# Environment variables
-# ---------------------
-#
-# All configuration options can be overridden via environment variables. The
-# environment variable should have the prefix "ENTE_", and any nesting should be
-# replaced by underscores.
-#
-# For example, the nested string "db.user" in the config file can alternatively
-# be specified (or be overridden) by setting an environment variable named
-# ENTE_DB_USER.
-#
-# Hyphens "-" will also be be replaced with underscores, so a string like "s3.b2-eu-cen"
-# can be specified by the environment variable named "S3_B2_EU_CEN".
-#
-#
-# Empty strings
-# -------------
-#
-# The empty string indicates missing values (to match go convention).
-#
-# This also means that to override a value that is specified in local.yaml in a
-# subsequently loaded config file, you should specify the key as an empty string
-# (`key: ""`) instead of leaving it unset.
+# Empty strings represent missing values.
+# In a later file, use `key: ""` to clear an earlier value; omitting the key leaves the earlier value unchanged.
#
# ---
-# If this option is specified, then it is loaded and gets merged-in over the
-# defaults present in default.yaml. This provides a way to inject credentials
-# and other overrides.
-#
-# The default is to look for a file named credentials.yaml in the CWD.
+# Optional configuration loaded before museum.yaml.
+# Defaults to credentials.yaml in the current working directory.
#credentials-file: credentials.yaml
-# Some credentials (e.g. the TLS cert) are cumbersome to provide inline in the
-# YAML configuration file, thus these are loaded at runtime from separate files.
-#
-# This is the directory where museum should look for them.
-#
-# Currently, the following files are loaded (if needed)
+# Directory containing credentials kept outside YAML:
#
# - credentials/{tls.cert,tls.key}
# - credentials/pst-service-account.json
# - credentials/fcm-service-account.json
#
-# The default is to look for a these files in a directory named credentials
-# under the CWD.
+# Defaults to credentials in the current working directory.
#credentials-dir: credentials
# By default, museum logs to stdout when running locally. Specify this path to
diff --git a/server/scripts/deploy/README.md b/server/scripts/deploy/README.md
index 1e25337533..d2fabf1ccd 100644
--- a/server/scripts/deploy/README.md
+++ b/server/scripts/deploy/README.md
@@ -1,38 +1,28 @@
# Production Deployments
-This document outlines how we ourselves deploy museum. Note that this is very specific to our use case, and while this might be useful as an example, this is likely overkill for simple self hosted deployments.
+This describes Ente's production deployment. It is specific to our infrastructure and is generally unnecessary for self-hosted instances.
## Overview
-We use museum's Dockerfile to build images which we then run on vanilla Ubuntu servers (+ Docker installed). For ease of administration, we wrap Docker commands to start/stop it in a systemd service.
+We run Museum's Docker image on Ubuntu hosts with Docker and manage it with systemd, following the [service pattern](../../../infra/services/README.md) used by the rest of our infrastructure.
-- The production machines are vanilla Ubuntu instances, with Docker and Promtail installed.
+- [server-release.yml](../../../.github/workflows/server-release.yml) builds and publishes the image.
-- There is a [GitHub action](../../../.github/workflows/server-release.yml) to build museum Docker images using its Dockerfile.
+- [museum.service](museum.service) runs the container directly; [museum.nginx.service](museum.nginx.service) runs it behind Nginx.
-- We wrap the commands to start and stop containers using these images in a systemd service.
+- `systemctl start|stop|status museum` manages the running image.
-- We call this general concept of standalone Docker images that are managed using systemd as "services". More examples and details [here](../../../infra/services/README.md).
-
-- So museum is a "service". You can see its systemd unit definition in [museum.service](museum.service)
-
-- On the running instance, we use `systemctl start|stop|status museum` to manage the current image.
-
-- To update museum, use [update-and-restart-museum.sh](update-and-restart-museum.sh). It pulls the latest image before restarting the service.
-
-- Optionally and alternatively, museum can also be run behind an Nginx. This option has a separate service definition.
+- [update-and-restart-museum.sh](update-and-restart-museum.sh) pulls the latest image and restarts the service.
## Installation
-To bring up an additional museum node:
-
-Prepare the instance to run our services.
+To bring up another Museum node, prepare the instance to run our services.
-Setup [promtail](../../../infra/services/promtail/README.md), [prometheus and node-exporter](../../../infra/services/prometheus/README.md) services.
+Set up [Promtail](../../../infra/services/promtail/README.md), [Prometheus and node-exporter](../../../infra/services/prometheus/README.md).
If running behind Nginx, install the [nginx](../../../infra/services/nginx/README.md) service.
-Add credentials
+Add credentials:
```sh
sudo mkdir -p /root/museum/credentials
@@ -41,7 +31,7 @@ sudo tee /root/museum/credentials/fcm-service-account.json
sudo tee /root/museum/credentials.yaml
```
-Add billing data from the pricing-data repo
+Add billing data from the pricing-data repository:
```sh
scp /path/to/pricing-data/{us,in,black-friday}.json <instance>:
@@ -50,7 +40,7 @@ sudo mkdir -p /root/museum/data/billing
sudo mv *.json /root/museum/data/billing/
```
-Add the TLS credentials (optional if running behind Nginx)
+Add TLS credentials unless running behind Nginx:
```sh
sudo tee /root/museum/credentials/tls.cert
@@ -71,7 +61,7 @@ sudo mv museum.service /etc/systemd/system
sudo systemctl daemon-reload
```
-If running behind Nginx, tell it about museum config (modified with suitable rate limits)
+If running behind Nginx, install Museum's Nginx configuration with suitable rate limits:
```sh
scp scripts/deploy/museum.nginx.conf <instance>:
@@ -82,13 +72,13 @@ sudo systemctl reload nginx
## Starting
-SSH into the instance, and run
+SSH into the instance and run:
```sh
./update-and-restart-museum.sh
```
-This'll ask for sudo credentials, pull the latest Docker image, restart the museum service and start tailing the logs (as a sanity check).
+The script pulls the latest image, restarts Museum, and tails its logs.
## Rollback