Skip to content

Take control over your live stream video by running it yourself. Streaming + chat out of the box.

ActivityPub integration test (#4761)

Gabe Kangase9cc14ac7277
diff --git a/.github/workflows/activitypub-federation-tests.yaml b/.github/workflows/activitypub-federation-tests.yaml
new file mode 100644
index 0000000000..11b0109621
--- /dev/null
+++ b/.github/workflows/activitypub-federation-tests.yaml
@@ -0,0 +1,57 @@
+name: ActivityPub Federation Tests
+
+on:
+ push:
+ paths:
+ - "activitypub/**"
+ - "webserver/**"
+ - "test/automated/activitypub/**"
+ - ".github/workflows/activitypub-federation-tests.yaml"
+ pull_request:
+ paths:
+ - "activitypub/**"
+ - "webserver/**"
+ - "test/automated/activitypub/**"
+ - ".github/workflows/activitypub-federation-tests.yaml"
+
+jobs:
+ federation-test:
+ runs-on: ubuntu-latest
+
+ steps:
+ - id: skip_check
+ uses: fkirc/skip-duplicate-actions@v5
+ with:
+ concurrent_skipping: "same_content_newer"
+
+ - name: Check out repository code
+ uses: actions/checkout@v6
+
+ - name: Set up Go
+ uses: actions/setup-go@v6
+ with:
+ go-version: "^1.23"
+ cache: true
+
+ - name: Install Caddy
+ run: |
+ sudo apt-get update
+ sudo apt-get install -y debian-keyring debian-archive-keyring apt-transport-https curl
+ curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
+ curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
+ sudo apt-get update
+ sudo apt-get install -y caddy
+
+ - name: Install dependencies
+ run: |
+ sudo apt-get install -y snac2 ffmpeg
+
+ - name: Run setup
+ run: |
+ cd test/automated/activitypub
+ sudo ./setup.sh
+
+ - name: Run federation test
+ run: |
+ cd test/automated/activitypub
+ CI=true USER_COUNT=20 ./run.sh
diff --git a/test/automated/activitypub/.gitignore b/test/automated/activitypub/.gitignore
new file mode 100644
index 0000000000..df9128702e
--- /dev/null
+++ b/test/automated/activitypub/.gitignore
@@ -0,0 +1 @@
+certs/
diff --git a/test/automated/activitypub/Caddyfile b/test/automated/activitypub/Caddyfile
new file mode 100644
index 0000000000..6655096df8
--- /dev/null
+++ b/test/automated/activitypub/Caddyfile
@@ -0,0 +1,35 @@
+# Caddyfile for ActivityPub federation testing
+# Routes requests to owncast.local and snac.local to their respective backends
+
+{
+ # Disable automatic HTTPS since we use mkcert certificates
+ auto_https off
+ # Disable admin API
+ admin off
+ # Suppress logging
+ log {
+ level error
+ }
+}
+
+# HTTPS listener using mkcert certificates
+https://:{$PROXY_PORT} {
+ tls {$CERT_FILE} {$KEY_FILE}
+
+ # Route based on Host header
+ @owncast host owncast.local
+ @snac host snac.local
+
+ handle @owncast {
+ reverse_proxy localhost:{$OWNCAST_PORT}
+ }
+
+ handle @snac {
+ reverse_proxy localhost:{$SNAC_PORT}
+ }
+
+ # Default: return error for unknown hosts
+ handle {
+ respond "Unknown host" 404
+ }
+}
diff --git a/test/automated/activitypub/README.md b/test/automated/activitypub/README.md
new file mode 100644
index 0000000000..19882437c5
--- /dev/null
+++ b/test/automated/activitypub/README.md
@@ -0,0 +1,133 @@
+# ActivityPub Federation Test
+
+This test verifies Owncast's ActivityPub federation by having snac2 users follow the Owncast instance and confirming message delivery.
+
+## One-Time Setup
+
+### 1. Install mkcert
+
+```bash
+# Ubuntu/Debian
+sudo apt install mkcert
+
+# Or download directly
+curl -sLO https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64
+chmod +x mkcert-v1.4.4-linux-amd64
+sudo mv mkcert-v1.4.4-linux-amd64 /usr/local/bin/mkcert
+```
+
+### 2. Install the local CA
+
+This installs a Certificate Authority into your system's trust store. All certificates generated by mkcert will be trusted.
+
+```bash
+mkcert -install
+```
+
+### 3. Generate certificates for the test domains
+
+```bash
+cd test/automated/activitypub
+mkdir -p certs
+mkcert -cert-file certs/cert.pem -key-file certs/key.pem owncast.local snac.local localhost 127.0.0.1
+```
+
+### 4. Add hosts entries
+
+```bash
+sudo sh -c 'echo "127.0.0.1 owncast.local snac.local" >> /etc/hosts'
+```
+
+### 5. Install snac2
+
+```bash
+# Ubuntu/Debian
+sudo apt install snac2
+
+# Or build from source
+git clone https://codeberg.org/grunfink/snac2.git
+cd snac2
+make
+sudo make install
+```
+
+### 6. Install Caddy
+
+Caddy is used as the HTTPS reverse proxy for TLS termination. It will be installed automatically by `setup.sh`, or you can install it manually:
+
+```bash
+# Download binary directly
+curl -sL "https://github.com/caddyserver/caddy/releases/download/v2.8.4/caddy_2.8.4_linux_amd64.tar.gz" | sudo tar -xz -C /usr/local/bin caddy
+
+# Or see: https://caddyserver.com/docs/install
+```
+
+## Running the Test
+
+```bash
+# Run with default 100 users
+./run.sh
+
+# Run with fewer users for quick testing
+USER_COUNT=10 ./run.sh
+
+# Keep servers running after test for debugging
+KEEP_RUNNING=true ./run.sh
+
+# Adjust follow request throttling (default 0.1s)
+FOLLOW_DELAY=0.2 ./run.sh
+
+# Run in CI mode (skip interactive prompts)
+CI=true ./run.sh
+```
+
+## Configuration Options
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `USER_COUNT` | 100 | Number of test users to create |
+| `FOLLOW_DELAY` | 0.1 | Delay in seconds between follow requests |
+| `KEEP_RUNNING` | false | Keep servers running after test for debugging |
+| `CI` | false | Skip interactive prompts for CI environments |
+| `PROXY_PORT` | 8443 | HTTPS proxy port |
+| `SNAC_PORT` | 9080 | snac2 HTTP port |
+| `OWNCAST_PORT` | 8080 | Owncast HTTP port |
+
+## What the Test Does
+
+1. Creates a temporary snac2 instance with test users
+2. Starts an HTTPS reverse proxy for TLS termination
+3. Starts Owncast configured for federation
+4. Has all snac2 users follow Owncast
+5. Sends a message from Owncast
+6. Verifies all followers received the message
+
+## Test Results
+
+The test reports:
+- **Followers Registered**: Number of successful follow requests
+- **Messages Delivered**: Number of users who received the message
+- **Delivery Time**: Time to deliver to all followers
+- **Follow Success Rate**: Percentage of follow requests that succeeded
+- **Delivery Rate**: Percentage of registered followers who received the message
+
+## Troubleshooting
+
+### Certificate errors from snac2
+
+Make sure you ran `mkcert -install` and generated the certificates. The CA must be in the system trust store for snac2 to trust the certificates.
+
+### Port already in use
+
+Kill any leftover processes:
+```bash
+pkill -f "snac httpd /tmp"
+pkill -f "caddy run"
+```
+
+### Hosts file not configured
+
+Verify the entries exist:
+```bash
+grep -E 'owncast.local|snac.local' /etc/hosts
+```
diff --git a/test/automated/activitypub/run.sh b/test/automated/activitypub/run.sh
new file mode 100755
index 0000000000..bb5ad65f7b
--- /dev/null
+++ b/test/automated/activitypub/run.sh
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+# Run the ActivityPub federation test
+#
+# Usage:
+# ./run.sh # Run with 100 users
+# USER_COUNT=50 ./run.sh # Run with 50 users
+# KEEP_RUNNING=true ./run.sh # Keep servers running after test
+#
+# Prerequisites:
+# Add to /etc/hosts: 127.0.0.1 owncast.local snac.local
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+exec "${SCRIPT_DIR}/test-federation.sh" "$@"
diff --git a/test/automated/activitypub/setup.sh b/test/automated/activitypub/setup.sh
new file mode 100755
index 0000000000..8d1def6071
--- /dev/null
+++ b/test/automated/activitypub/setup.sh
@@ -0,0 +1,83 @@
+#!/bin/bash
+
+# One-time setup script for ActivityPub federation tests
+# Run with: sudo ./setup.sh
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+NC='\033[0m'
+
+log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
+log_error() { echo -e "${RED}[ERROR]${NC} $1"; }
+
+if [[ ${EUID} -ne 0 ]]; then
+ log_error "This script must be run as root (sudo ./setup.sh)"
+ exit 1
+fi
+
+# Get the actual user who ran sudo
+ACTUAL_USER="${SUDO_USER:-${USER}}"
+ACTUAL_HOME=$(getent passwd "${ACTUAL_USER}" | cut -d: -f6)
+
+log_info "Setting up ActivityPub federation test environment..."
+
+# 1. Install mkcert
+if ! command -v mkcert &> /dev/null; then
+ log_info "Downloading mkcert..."
+ curl -sL "https://github.com/FiloSottile/mkcert/releases/download/v1.4.4/mkcert-v1.4.4-linux-amd64" -o /usr/local/bin/mkcert
+ chmod +x /usr/local/bin/mkcert
+else
+ log_info "mkcert already installed"
+fi
+
+# 2. Install CA into system trust store
+log_info "Installing mkcert CA into system trust store..."
+sudo -u "${ACTUAL_USER}" CAROOT="${ACTUAL_HOME}/.local/share/mkcert" mkcert -install
+
+# 3. Generate certificates
+log_info "Generating certificates..."
+mkdir -p "${SCRIPT_DIR}/certs"
+chown "${ACTUAL_USER}:${ACTUAL_USER}" "${SCRIPT_DIR}/certs"
+sudo -u "${ACTUAL_USER}" CAROOT="${ACTUAL_HOME}/.local/share/mkcert" mkcert \
+ -cert-file "${SCRIPT_DIR}/certs/cert.pem" \
+ -key-file "${SCRIPT_DIR}/certs/key.pem" \
+ owncast.local snac.local localhost 127.0.0.1
+
+# 4. Add hosts entries
+if ! grep -q "owncast.local" /etc/hosts; then
+ log_info "Adding hosts entries..."
+ echo "127.0.0.1 owncast.local snac.local" >> /etc/hosts
+else
+ log_info "Hosts entries already exist"
+fi
+
+# 5. Check for snac2
+if ! command -v snac &> /dev/null; then
+ log_error "snac2 is not installed. Please install it manually."
+ log_error "See: https://codeberg.org/grunfink/snac2"
+ exit 1
+else
+ log_info "snac2 found: $(command -v snac)"
+fi
+
+# 6. Install Caddy
+if ! command -v caddy &> /dev/null; then
+ log_info "Installing Caddy..."
+ CADDY_VERSION="2.8.4"
+ curl -sL "https://github.com/caddyserver/caddy/releases/download/v${CADDY_VERSION}/caddy_${CADDY_VERSION}_linux_amd64.tar.gz" | tar -xz -C /usr/local/bin caddy
+ chmod +x /usr/local/bin/caddy
+ log_info "Caddy installed"
+else
+ log_info "Caddy already installed: $(command -v caddy)"
+fi
+
+echo ""
+log_info "Setup complete!"
+echo ""
+echo "You can now run the test:"
+echo " cd ${SCRIPT_DIR}"
+echo " ./run.sh"
diff --git a/test/automated/activitypub/test-federation.sh b/test/automated/activitypub/test-federation.sh
new file mode 100755
index 0000000000..8da2f2d862
--- /dev/null
+++ b/test/automated/activitypub/test-federation.sh
@@ -0,0 +1,806 @@
+#!/bin/bash
+# shellcheck disable=SC2317 # cleanup() is invoked via trap, not direct call
+
+# ActivityPub Federation Test using snac2
+#
+# This test:
+# 1. Builds/installs snac2
+# 2. Creates 100 test users in snac2
+# 3. Starts a local HTTPS reverse proxy
+# 4. Starts snac2 server (HTTP internally, HTTPS via proxy)
+# 5. Starts Owncast (HTTP internally, HTTPS via proxy)
+# 6. Has all 100 snac2 users follow Owncast
+# 7. Sends a message from Owncast
+# 8. Verifies message delivery to snac2 inboxes
+#
+# Requirements:
+# - openssl (for generating certs)
+# - curl
+# - node (for the HTTPS proxy)
+# - Go (for building Owncast)
+# - C compiler (for building snac2)
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(git rev-parse --show-toplevel)"
+
+# Configuration
+USER_COUNT="${USER_COUNT:-100}"
+FOLLOW_DELAY="${FOLLOW_DELAY:-0.1}" # Delay between follow requests (seconds)
+CI="${CI:-false}" # Skip interactive prompts for CI environments
+PROXY_PORT="${PROXY_PORT:-8443}"
+SNAC_PORT="${SNAC_PORT:-9080}"
+SNAC_HOSTNAME="snac.local"
+OWNCAST_PORT="${OWNCAST_PORT:-8080}"
+OWNCAST_HOSTNAME="owncast.local"
+ADMIN_USER="admin"
+ADMIN_PASS="abc123"
+FEDERATION_USERNAME="streamer"
+
+# URLs (HTTPS via proxy)
+SNAC_URL="https://${SNAC_HOSTNAME}:${PROXY_PORT}"
+OWNCAST_URL="https://${OWNCAST_HOSTNAME}:${PROXY_PORT}"
+
+# Directories
+TEMP_DIR=""
+SNAC_DATA_DIR=""
+SNAC_BIN=""
+OWNCAST_DB=""
+
+# PIDs and state
+SNAC_PID=""
+OWNCAST_PID=""
+PROXY_PID=""
+TEST_STREAM_PID=""
+SNAC_USERNAMES=()
+
+# Colors
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+NC='\033[0m'
+
+log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
+log_warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }
+log_error() { echo -e "${RED}[ERROR]${NC} $1"; }
+log_test() { echo -e "${BLUE}[TEST]${NC} $1"; }
+
+kill_leftover_processes() {
+ # Kill any leftover test processes from previous runs
+ local killed=false
+
+ # Kill snac2 instances running from /tmp (test instances only)
+ if pkill -f "snac httpd /tmp" 2>/dev/null; then
+ killed=true
+ fi
+
+ # Kill any local-proxy.js instances
+ if pkill -f "local-proxy.js" 2>/dev/null; then
+ killed=true
+ fi
+
+ # Kill anything on our test ports
+ local proxy_pid
+ proxy_pid=$(lsof -ti :"${PROXY_PORT}" 2>/dev/null) || true
+ if [[ -n "${proxy_pid}" ]]; then
+ kill "${proxy_pid}" 2>/dev/null || true
+ killed=true
+ fi
+
+ local snac_pid
+ snac_pid=$(lsof -ti :"${SNAC_PORT}" 2>/dev/null) || true
+ if [[ -n "${snac_pid}" ]]; then
+ kill "${snac_pid}" 2>/dev/null || true
+ killed=true
+ fi
+
+ if [[ "${killed}" == "true" ]]; then
+ log_info "Killed leftover processes from previous run"
+ sleep 1
+ fi
+}
+
+cleanup() {
+ log_info "Cleaning up..."
+
+ if [[ -n "${PROXY_PID}" ]] && kill -0 "${PROXY_PID}" 2>/dev/null; then
+ kill "${PROXY_PID}" 2>/dev/null || true
+ wait "${PROXY_PID}" 2>/dev/null || true
+ fi
+
+ if [[ -n "${OWNCAST_PID}" ]] && kill -0 "${OWNCAST_PID}" 2>/dev/null; then
+ kill "${OWNCAST_PID}" 2>/dev/null || true
+ wait "${OWNCAST_PID}" 2>/dev/null || true
+ fi
+
+ if [[ -n "${SNAC_PID}" ]] && kill -0 "${SNAC_PID}" 2>/dev/null; then
+ kill "${SNAC_PID}" 2>/dev/null || true
+ wait "${SNAC_PID}" 2>/dev/null || true
+ fi
+
+ if [[ -n "${TEST_STREAM_PID}" ]] && kill -0 "${TEST_STREAM_PID}" 2>/dev/null; then
+ kill "${TEST_STREAM_PID}" 2>/dev/null || true
+ wait "${TEST_STREAM_PID}" 2>/dev/null || true
+ fi
+
+ if [[ -n "${TEMP_DIR}" ]] && [[ -d "${TEMP_DIR}" ]]; then
+ rm -rf "${TEMP_DIR}"
+ fi
+
+ log_info "Cleanup complete."
+}
+
+trap cleanup EXIT
+
+setup_temp_dir() {
+ TEMP_DIR=$(mktemp -d)
+ SNAC_DATA_DIR="${TEMP_DIR}/snac-data"
+ OWNCAST_DB="${TEMP_DIR}/owncast.db"
+
+ log_info "Temp directory: ${TEMP_DIR}"
+}
+
+check_hosts_entry() {
+ # Check if hosts entries exist
+ if ! grep -q "${OWNCAST_HOSTNAME}" /etc/hosts 2>/dev/null || ! grep -q "${SNAC_HOSTNAME}" /etc/hosts 2>/dev/null; then
+ log_warn "Required /etc/hosts entries not found."
+ log_warn "Please add the following to /etc/hosts:"
+ echo ""
+ echo " 127.0.0.1 ${OWNCAST_HOSTNAME} ${SNAC_HOSTNAME}"
+ echo ""
+ log_warn "You may need to run: sudo sh -c 'echo \"127.0.0.1 ${OWNCAST_HOSTNAME} ${SNAC_HOSTNAME}\" >> /etc/hosts'"
+ exit 1
+ fi
+ log_info "Hosts entries verified"
+}
+
+install_snac2() {
+ log_info "Setting up snac2..."
+
+ # Check if snac2 is already installed
+ if command -v snac &> /dev/null; then
+ SNAC_BIN=$(command -v snac)
+ log_info "Using system snac2: ${SNAC_BIN}"
+ return
+ fi
+
+ # Clone and build snac2
+ local snac_src="${TEMP_DIR}/snac2-src"
+ log_info "Cloning snac2..."
+ git clone --depth 1 https://codeberg.org/grunfink/snac2.git "${snac_src}" 2>/dev/null
+
+ log_info "Building snac2..."
+ pushd "${snac_src}" > /dev/null
+ make
+ SNAC_BIN="${snac_src}/snac"
+ popd > /dev/null
+
+ log_info "snac2 built: ${SNAC_BIN}"
+}
+
+check_certs() {
+ # Use pre-generated mkcert certificates from the script directory
+ CERT_DIR="${SCRIPT_DIR}/certs"
+
+ if [[ ! -f "${CERT_DIR}/cert.pem" ]] || [[ ! -f "${CERT_DIR}/key.pem" ]]; then
+ log_error "Certificates not found in ${CERT_DIR}"
+ log_error "Please run the one-time setup. See README.md for instructions:"
+ log_error " mkcert -install"
+ log_error " mkcert -cert-file certs/cert.pem -key-file certs/key.pem owncast.local snac.local localhost 127.0.0.1"
+ exit 1
+ fi
+
+ log_info "Using certificates from ${CERT_DIR}"
+}
+
+init_snac2() {
+ log_info "Initializing snac2..."
+
+ # snac2 advertises URLs via the proxy (HTTPS on PROXY_PORT)
+ local snac_host_port="${SNAC_HOSTNAME}:${PROXY_PORT}"
+
+ # Use snac init with piped input: address, port, hostname, prefix, admin email
+ printf "127.0.0.1\n%s\n%s\n\[email protected]\n" "${SNAC_PORT}" "${snac_host_port}" | \
+ "${SNAC_BIN}" init "${SNAC_DATA_DIR}" > /dev/null 2>&1
+
+ if [[ ! -f "${SNAC_DATA_DIR}/server.json" ]]; then
+ log_error "snac2 init failed - server.json not created"
+ return 1
+ fi
+
+ log_info "snac2 initialized"
+}
+
+create_snac_users() {
+ log_info "Creating ${USER_COUNT} users in snac2 using snac adduser..."
+
+ # Generate a unique prefix for this test run to avoid conflicts
+ local run_id
+ run_id=$(date +%s%N | sha256sum | head -c 8)
+ log_info "Test run ID: ${run_id}"
+
+ # Store usernames for later use in follow requests
+ SNAC_USERNAMES=()
+
+ local created=0
+ local failed=0
+
+ for i in $(seq 1 "${USER_COUNT}"); do
+ local username="test${run_id}u${i}"
+ local displayname="Test User ${i}"
+
+ # Use snac adduser with piped input (username, display name)
+ if printf "%s\n%s\n" "${username}" "${displayname}" | "${SNAC_BIN}" adduser "${SNAC_DATA_DIR}" > /dev/null 2>&1; then
+ SNAC_USERNAMES+=("${username}")
+ created=$((created + 1))
+ else
+ failed=$((failed + 1))
+ fi
+
+ if [[ $((created + failed)) -gt 0 ]] && [[ $(((created + failed) % 20)) -eq 0 ]]; then
+ log_info "Created $((created + failed))/${USER_COUNT} users (successful: ${created})..."
+ fi
+ done
+
+ log_info "Created ${created} users in snac2 (${failed} failed)"
+}
+
+start_proxy() {
+ log_info "Starting HTTPS reverse proxy (Caddy)..."
+
+ # Check if caddy is installed
+ if ! command -v caddy &> /dev/null; then
+ log_error "Caddy is not installed. Please install it: https://caddyserver.com/docs/install"
+ return 1
+ fi
+
+ # Set environment variables for Caddyfile
+ export PROXY_PORT="${PROXY_PORT}"
+ export OWNCAST_PORT="${OWNCAST_PORT}"
+ export SNAC_PORT="${SNAC_PORT}"
+ export CERT_FILE="${CERT_DIR}/cert.pem"
+ export KEY_FILE="${CERT_DIR}/key.pem"
+
+ caddy run --config "${SCRIPT_DIR}/Caddyfile" --adapter caddyfile &
+ PROXY_PID=$!
+
+ log_info "Caddy started with PID ${PROXY_PID}"
+ sleep 2
+
+ # Verify proxy is running
+ if ! kill -0 "${PROXY_PID}" 2>/dev/null; then
+ log_error "Caddy failed to start"
+ return 1
+ fi
+
+ # Verify proxy is accepting connections
+ local max_attempts=10
+ local attempt=0
+ while [[ ${attempt} -lt ${max_attempts} ]]; do
+ if curl -sk "https://127.0.0.1:${PROXY_PORT}/" > /dev/null 2>&1; then
+ log_info "Caddy proxy is ready"
+ return 0
+ fi
+ attempt=$((attempt + 1))
+ sleep 1
+ done
+
+ log_error "Caddy proxy did not become ready"
+ return 1
+}
+
+start_snac2() {
+ log_info "Starting snac2 server..."
+
+ # mkcert certificates are trusted system-wide, no special env vars needed
+ "${SNAC_BIN}" httpd "${SNAC_DATA_DIR}" &
+ SNAC_PID=$!
+
+ log_info "snac2 started with PID ${SNAC_PID}"
+
+ # Wait for snac2 to be ready
+ local max_attempts=30
+ local attempt=0
+
+ while [[ ${attempt} -lt ${max_attempts} ]]; do
+ if curl -s "http://127.0.0.1:${SNAC_PORT}/" > /dev/null 2>&1; then
+ log_info "snac2 is ready"
+ return 0
+ fi
+ attempt=$((attempt + 1))
+ sleep 1
+ done
+
+ log_error "snac2 did not become ready"
+ return 1
+}
+
+build_owncast() {
+ log_info "Building Owncast..."
+
+ pushd "${REPO_ROOT}" > /dev/null
+ CGO_ENABLED=1 go build -o owncast main.go
+ popd > /dev/null
+
+ log_info "Owncast built"
+}
+
+start_owncast() {
+ log_info "Starting Owncast..."
+
+ # Start Owncast with test environment variables and debug flags
+ OWNCAST_ALLOW_INTERNAL_FEDERATION=true \
+ OWNCAST_INSECURE_SKIP_VERIFY=true \
+ "${REPO_ROOT}/owncast" -database "${OWNCAST_DB}" &
+ OWNCAST_PID=$!
+
+ log_info "Owncast started with PID ${OWNCAST_PID}"
+
+ # Wait for Owncast to be ready
+ local max_attempts=30
+ local attempt=0
+
+ while [[ ${attempt} -lt ${max_attempts} ]]; do
+ if curl -s "http://localhost:${OWNCAST_PORT}/api/status" > /dev/null 2>&1; then
+ log_info "Owncast is ready"
+ return 0
+ fi
+ attempt=$((attempt + 1))
+ sleep 1
+ done
+
+ log_error "Owncast did not become ready"
+ return 1
+}
+
+configure_owncast() {
+ log_info "Configuring Owncast with URL: ${OWNCAST_URL}"
+
+ local base_url="http://localhost:${OWNCAST_PORT}"
+ local auth
+ auth=$(echo -n "${ADMIN_USER}:${ADMIN_PASS}" | base64)
+
+ # Set server URL
+ curl -s -X POST "${base_url}/api/admin/config/serverurl" \
+ -H "Authorization: Basic ${auth}" \
+ -H "Content-Type: application/json" \
+ -d "{\"value\": \"${OWNCAST_URL}\"}" > /dev/null
+
+ # Set federation username
+ curl -s -X POST "${base_url}/api/admin/config/federation/username" \
+ -H "Authorization: Basic ${auth}" \
+ -H "Content-Type: application/json" \
+ -d "{\"value\": \"${FEDERATION_USERNAME}\"}" > /dev/null
+
+ # Enable federation
+ curl -s -X POST "${base_url}/api/admin/config/federation/enable" \
+ -H "Authorization: Basic ${auth}" \
+ -H "Content-Type: application/json" \
+ -d '{"value": true}' > /dev/null
+
+ # Disable private mode (auto-accept follows)
+ curl -s -X POST "${base_url}/api/admin/config/federation/private" \
+ -H "Authorization: Basic ${auth}" \
+ -H "Content-Type: application/json" \
+ -d '{"value": false}' > /dev/null
+
+ log_info "Owncast configured"
+}
+
+prompt_for_test_stream() {
+ # Skip interactive prompts in CI mode
+ if [[ "${CI}" == "true" ]]; then
+ log_info "CI mode - skipping test stream prompt"
+ return
+ fi
+
+ echo ""
+ echo "----------------------------------------"
+ read -p "Would you like to start a test live stream? [y/N] " -n 1 -r
+ echo ""
+
+ if [[ ${REPLY} =~ ^[Yy]$ ]]; then
+ log_info "Starting test stream..."
+ "${REPO_ROOT}/test/ocTestStream.sh" &
+ TEST_STREAM_PID=$!
+
+ echo ""
+ log_info "Test stream started with PID ${TEST_STREAM_PID}"
+ log_info "You can view the stream at: http://localhost:${OWNCAST_PORT}"
+ echo ""
+ read -p "Press Enter when you are watching the stream and ready to continue..." -r
+ echo ""
+ fi
+}
+
+send_follow_requests() {
+ log_info "Sending follow requests from snac2 users..."
+
+ local owncast_actor="${OWNCAST_URL}/federation/user/${FEDERATION_USERNAME}"
+ log_info "Following actor: ${owncast_actor}"
+
+ # First verify the actor URL is accessible via curl
+ log_info "Verifying actor URL is accessible..."
+ local curl_test
+ curl_test=$(curl -s --max-time 5 -H "Accept: application/activity+json" "${owncast_actor}" 2>&1)
+ if echo "${curl_test}" | grep -q '"type"'; then
+ log_info "Actor URL is accessible via curl"
+ else
+ log_error "Actor URL not accessible: ${curl_test}"
+ fi
+
+ local successful=0
+ local failed=0
+ local total=${#SNAC_USERNAMES[@]}
+
+ for username in "${SNAC_USERNAMES[@]}"; do
+ # Use snac follow command and capture output
+ # mkcert certificates are trusted system-wide, no special env vars needed
+ local follow_output
+ follow_output=$("${SNAC_BIN}" follow "${SNAC_DATA_DIR}" "${username}" "${owncast_actor}" 2>&1)
+ local follow_exit=$?
+
+ if [[ ${follow_exit} -eq 0 ]] && [[ ! "${follow_output}" =~ "cannot" ]]; then
+ successful=$((successful + 1))
+ else
+ failed=$((failed + 1))
+ # Show the first error for debugging
+ if [[ ${failed} -eq 1 ]]; then
+ log_warn "snac follow error: ${follow_output}"
+ fi
+ fi
+
+ if [[ $((successful + failed)) -gt 0 ]] && [[ $(((successful + failed) % 20)) -eq 0 ]]; then
+ log_info "Follows sent: $((successful + failed))/${total} (successful: ${successful})"
+ fi
+
+ # Throttle follow requests to prevent overwhelming snac2
+ sleep "${FOLLOW_DELAY}"
+ done
+
+ log_test "Follow requests: ${successful} successful, ${failed} failed"
+
+ # Check snac2 queue directory for pending activities
+ log_info "Checking snac2 queue..."
+ local queue_count
+ queue_count=$(find "${SNAC_DATA_DIR}" -name "*.json" -path "*/queue/*" 2>/dev/null | wc -l)
+ log_info "snac2 queue has ${queue_count} pending items"
+
+ # Give snac2 background thread time to process all pending follows
+ # Wait longer for more users
+ local wait_time=$((10 + USER_COUNT / 10))
+ log_info "Waiting ${wait_time}s for snac2 to process follow requests..."
+ sleep "${wait_time}"
+}
+
+verify_followers() {
+ log_info "Verifying followers..." >&2
+
+ local auth
+ auth=$(echo -n "${ADMIN_USER}:${ADMIN_PASS}" | base64)
+ local response
+
+ response=$(curl -s "http://localhost:${OWNCAST_PORT}/api/admin/followers?limit=200" \
+ -H "Authorization: Basic ${auth}")
+
+ local count
+ count=$(echo "${response}" | grep -o '"total":[0-9]*' | grep -o '[0-9]*' || echo "0")
+
+ log_test "Owncast reports ${count} followers" >&2
+ echo "${count}"
+}
+
+send_test_message() {
+ log_info "Sending test message from Owncast..."
+
+ local auth
+ auth=$(echo -n "${ADMIN_USER}:${ADMIN_PASS}" | base64)
+ local message
+ message="Test message sent at $(date -u +%Y-%m-%dT%H:%M:%SZ) to ${USER_COUNT} followers"
+
+ curl -s -X POST "http://localhost:${OWNCAST_PORT}/api/admin/federation/send" \
+ -H "Authorization: Basic ${auth}" \
+ -H "Content-Type: application/json" \
+ -d "{\"value\": \"${message}\"}" > /dev/null
+
+ log_info "Message sent: ${message}"
+}
+
+check_snac_inboxes_count() {
+ local users_with_messages=0
+
+ for username in "${SNAC_USERNAMES[@]}"; do
+ if user_has_message "${username}"; then
+ users_with_messages=$((users_with_messages + 1))
+ fi
+ done
+
+ echo "${users_with_messages}"
+}
+
+user_has_message() {
+ local username="$1"
+ local user_dir="${SNAC_DATA_DIR}/user/${username}"
+
+ # snac2 stores incoming posts in various places depending on type
+ # Check timeline, public, private directories for any .json files
+ local count
+ for subdir in public private timeline; do
+ if [[ -d "${user_dir}/${subdir}" ]]; then
+ count=$(find "${user_dir}/${subdir}" -name "*.json" -type f 2>/dev/null | wc -l)
+ if [[ "${count}" -gt 0 ]]; then
+ return 0
+ fi
+ fi
+ done
+
+ # Also check the global object store for objects addressed to this user
+ # snac2 might store activities in the object directory
+ return 1
+}
+
+check_snac_inboxes() {
+ log_info "Checking snac2 user inboxes for delivered messages..." >&2
+
+ local users_with_messages=0
+ local users_without_messages=()
+ local total=${#SNAC_USERNAMES[@]}
+
+ for username in "${SNAC_USERNAMES[@]}"; do
+ if user_has_message "${username}"; then
+ users_with_messages=$((users_with_messages + 1))
+ else
+ users_without_messages+=("${username}")
+ fi
+ done
+
+ log_test "${users_with_messages}/${total} users received the message" >&2
+
+ if [[ ${#users_without_messages[@]} -gt 0 ]] && [[ ${#users_without_messages[@]} -le 10 ]]; then
+ log_warn "Users missing messages: ${users_without_messages[*]}" >&2
+ elif [[ ${#users_without_messages[@]} -gt 10 ]]; then
+ log_warn "Users missing messages: ${users_without_messages[*]:0:10} ... and $((${#users_without_messages[@]} - 10)) more" >&2
+ fi
+
+ echo "${users_with_messages}"
+}
+
+verify_all_followers_received_message() {
+ local followers=$1
+ local max_wait=${2:-60}
+
+ log_info "Verifying all ${followers} followers receive the message (max ${max_wait}s)..." >&2
+
+ local start_time
+ start_time=$(date +%s)
+ local last_count=0
+ local delivery_time=0
+ local current_time
+ local count
+
+ while true; do
+ current_time=$(date +%s)
+ local elapsed=$((current_time - start_time))
+
+ if [[ ${elapsed} -ge ${max_wait} ]]; then
+ log_warn "Timeout after ${max_wait}s" >&2
+ delivery_time=${elapsed}
+ break
+ fi
+
+ count=$(check_snac_inboxes_count)
+
+ if [[ ${count} -ne ${last_count} ]]; then
+ local pct=$((count * 100 / followers))
+ log_info "Delivery progress: ${count}/${followers} (${pct}%)" >&2
+ last_count=${count}
+ fi
+
+ if [[ ${count} -ge ${followers} ]]; then
+ delivery_time=${elapsed}
+ log_test "All ${followers} followers received the message in ${delivery_time}s" >&2
+ echo "${delivery_time}"
+ return 0
+ fi
+
+ sleep 2
+ done
+
+ local final_count
+ final_count=$(check_snac_inboxes_count)
+ echo "${delivery_time}"
+ if [[ ${final_count} -lt ${followers} ]]; then
+ log_error "Only ${final_count}/${followers} followers received the message" >&2
+ return 1
+ fi
+
+ return 0
+}
+
+print_results() {
+ local followers=$1
+ local delivered=$2
+ local delivery_time=$3
+
+ echo ""
+ echo "========================================"
+ echo "ActivityPub Federation Test Results"
+ echo "========================================"
+ echo "Test Users Created: ${USER_COUNT}"
+ echo "Followers Registered: ${followers}"
+ echo "Messages Delivered: ${delivered}"
+ if [[ -n "${delivery_time}" ]] && [[ "${delivery_time}" -gt 0 ]]; then
+ echo "Delivery Time: ${delivery_time}s"
+ fi
+ echo ""
+
+ if [[ "${followers}" -gt 0 ]]; then
+ local follow_rate=$((followers * 100 / USER_COUNT))
+ echo "Follow Success Rate: ${follow_rate}%"
+ fi
+
+ if [[ "${followers}" -gt 0 ]]; then
+ local delivery_rate=$((delivered * 100 / followers))
+ echo "Delivery Rate: ${delivery_rate}%"
+ fi
+
+ echo ""
+ echo "----------------------------------------"
+ if [[ "${delivered}" -eq "${followers}" ]] && [[ "${followers}" -gt 0 ]]; then
+ echo -e "${GREEN}TEST PASSED${NC}"
+ echo "All ${followers} followers received the message."
+ elif [[ "${delivered}" -gt 0 ]]; then
+ echo -e "${RED}TEST FAILED${NC}"
+ echo "Only ${delivered} of ${followers} followers received the message."
+ echo "Missing: $((followers - delivered)) followers"
+ elif [[ "${followers}" -eq 0 ]]; then
+ echo -e "${RED}TEST FAILED${NC}"
+ echo "No followers were registered."
+ else
+ echo -e "${RED}TEST FAILED${NC}"
+ echo "No messages were delivered to any followers."
+ fi
+ echo "========================================"
+}
+
+main() {
+ # Kill any leftover processes from previous runs
+ kill_leftover_processes
+
+ echo ""
+ echo "========================================"
+ echo "ActivityPub Federation Test"
+ echo "========================================"
+ log_info "Configuration: ${USER_COUNT} test users"
+ log_info "Owncast URL: ${OWNCAST_URL}"
+ log_info "snac2 URL: ${SNAC_URL}"
+ echo ""
+
+ # ==========================================
+ # STEP 1: Setup snac2 server with test users
+ # ==========================================
+ echo "----------------------------------------"
+ echo "STEP 1: Setup snac2 server with test users"
+ echo "----------------------------------------"
+ setup_temp_dir
+ check_hosts_entry
+ install_snac2
+ check_certs
+ init_snac2
+ create_snac_users
+ start_proxy
+ start_snac2
+ echo ""
+
+ # ==========================================
+ # STEP 2: Setup Owncast with federation
+ # ==========================================
+ echo "----------------------------------------"
+ echo "STEP 2: Setup Owncast with federation"
+ echo "----------------------------------------"
+ build_owncast
+ start_owncast
+ configure_owncast
+ sleep 2
+
+ # Optional: prompt for test stream
+ prompt_for_test_stream
+ echo ""
+
+ # ==========================================
+ # STEP 3: snac2 users follow Owncast
+ # ==========================================
+ echo "----------------------------------------"
+ echo "STEP 3: snac2 users follow Owncast"
+ echo "----------------------------------------"
+ send_follow_requests
+
+ # Wait for all followers to be registered (with timeout)
+ local followers=0
+ local max_wait=60 # Maximum seconds to wait for followers
+ local waited=0
+ local check_interval=2
+
+ log_info "Waiting for ${USER_COUNT} followers to be registered..."
+ while [[ "${followers}" -lt "${USER_COUNT}" ]] && [[ "${waited}" -lt "${max_wait}" ]]; do
+ sleep "${check_interval}"
+ waited=$((waited + check_interval))
+ followers=$(verify_followers)
+ if [[ "${followers}" -lt "${USER_COUNT}" ]]; then
+ log_info "Followers registered: ${followers}/${USER_COUNT} (waited ${waited}s)"
+ fi
+ done
+
+ if [[ "${followers}" -eq 0 ]]; then
+ log_error "No followers registered - cannot proceed with message delivery test"
+ print_results "${followers}" 0
+ exit 1
+ fi
+
+ if [[ "${followers}" -lt "${USER_COUNT}" ]]; then
+ log_warn "Only ${followers}/${USER_COUNT} followers registered after ${max_wait}s timeout"
+ else
+ log_info "All ${followers} followers registered"
+ fi
+ echo ""
+
+ # ==========================================
+ # STEP 4: Owncast sends message to followers
+ # ==========================================
+ echo "----------------------------------------"
+ echo "STEP 4: Owncast sends message to followers"
+ echo "----------------------------------------"
+ send_test_message
+ echo ""
+
+ # ==========================================
+ # STEP 5: Verify all followers received message
+ # ==========================================
+ echo "----------------------------------------"
+ echo "STEP 5: Verify all followers received message"
+ echo "----------------------------------------"
+ local delivery_success=0
+ local delivery_time
+ if delivery_time=$(verify_all_followers_received_message "${followers}" 60); then
+ delivery_success=1
+ fi
+
+ local delivered
+ delivered=$(check_snac_inboxes)
+
+ print_results "${followers}" "${delivered}" "${delivery_time}"
+
+ if [[ "${KEEP_RUNNING:-}" == "true" ]]; then
+ log_info "Keeping servers running (Ctrl+C to stop)..."
+ log_info " - Owncast: http://localhost:${OWNCAST_PORT} (${OWNCAST_URL})"
+ log_info " - snac2: http://localhost:${SNAC_PORT} (${SNAC_URL})"
+ wait
+ fi
+
+ # Exit with appropriate code
+ # All conditions must be met:
+ # 1. All follow requests succeeded (followers == USER_COUNT)
+ # 2. Message delivery completed successfully
+ # 3. All followers received the message (delivered == followers)
+ if [[ "${followers}" -ne "${USER_COUNT}" ]]; then
+ log_error "Follow count mismatch: expected ${USER_COUNT}, got ${followers}"
+ exit 1
+ fi
+
+ if [[ "${delivery_success}" -ne 1 ]]; then
+ log_error "Message delivery failed"
+ exit 1
+ fi
+
+ if [[ "${delivered}" -ne "${followers}" ]]; then
+ log_error "Delivery count mismatch: expected ${followers}, got ${delivered}"
+ exit 1
+ fi
+
+ log_info "All tests passed"
+ exit 0
+}
+
+main "$@"