Database governance built for humans and agents — controlling changes and access across every major database.
Go cicddata-governancedata-maskingdata-securitydatabase-accessdatabase-governancedevopsdevsecopsflywaygitopsliquibasemongodbmysqloraclepostgresqlschema-migrationssecuritysql-clientsqlservertidb
feat: launch HA readiness (#19646)
p0nyb649ba9a6a40
diff --git a/docs/operations/high-availability.md b/docs/operations/high-availability.mdnew file mode 100644index 0000000000..7407a01582--- /dev/null+++ b/docs/operations/high-availability.md@@ -0,0 +1,89 @@+# High availability for Bytebase++This runbook describes the current operator-facing behavior for running Bytebase in a high-availability (HA) topology.++## Current support boundary++- Bytebase can detect and operate with multiple active replicas that share the same metadata database.+- HA is license-gated. The subscription API exposes this as the `ha` field on `Subscription`.+- The bundled Helm chart in `helm-charts/bytebase` currently renders a single-replica StatefulSet (`replicas: 1`). It does not provide a multi-replica deployment switch.++In other words: the runtime has HA awareness, but operators must still supply the multi-replica deployment mechanics outside the current Helm chart.++## Prerequisites++Before you keep more than one Bytebase server active at the same time, make sure all of the following are true:++1. **HA is enabled in the license.**+ - Check `GET /v1/subscription` and confirm `ha: true`.+2. **Every HA replica starts with `--ha` and the same shared metadata database.**+ - Active replicas are counted as Bytebase servers sharing the same database.+ - HA mode requires external PostgreSQL, so each replica must use the same `PG_URL` value instead of the embedded database.+3. **All replicas use the same external URL.**+ - Start replicas with the same `--external-url` value so Bytebase reports one `externalUrl` for user access and callbacks.+4. **Your platform provides traffic management and rollout control.**+ - For example, a load balancer plus a rolling update strategy managed by your orchestrator.++## How replica detection works++Bytebase tracks live replicas with heartbeats:++- Each replica writes a heartbeat immediately on startup and then every 10 seconds.+- A replica is considered active when it has sent a heartbeat within the last 30 seconds.+- The actuator API exposes the current active replica count as `replicaCount` on `GET /v1/actuator/info`.+- Bytebase always reports at least one active replica for the current server, even if heartbeat counting fails.++## What happens when HA is not licensed++If more than one active replica is detected and the license does not enable HA, Bytebase does not permit the HA topology.++The current runtime behavior is to log warnings such as:++```text+multiple replicas detected (<count>) but HA is not enabled in license+```++When that condition is present, background runners that check the replica limit skip work instead of continuing in an unsupported topology. This includes scheduler and cleaner paths used for task execution, plan checks, schema sync, approvals, and stale-run cleanup.++## Operator validation checklist++Use this checklist when enabling or validating HA:++1. Call `GET /v1/subscription` and verify `ha` is `true`.+2. Call `GET /v1/actuator/info` and record:+ - `version`+ - `externalUrl`+ - `replicaCount`+3. Confirm every replica starts with `--ha`, points to the same `PG_URL`, and uses the same external URL.+4. After adding or restarting replicas, allow at least 30 seconds for the active replica count to settle.+5. Review logs for HA-license warnings before declaring the topology healthy.++## Troubleshooting++### `replicaCount` stays at `1`++Check the following:++- The additional Bytebase server is actually running.+- The replica can reach the shared metadata database and write heartbeats.+- You waited long enough for the new replica to start and publish heartbeats.++### `replicaCount` is lower than expected during a restart++A replica falls out of the active set after roughly 30 seconds without a heartbeat. A brief drop during restarts or node moves can therefore be expected.++### Logs show `multiple replicas detected ... but HA is not enabled in license`++This means the deployment topology and license do not match. Resolve it by doing one of the following:++- reduce the deployment back to a single active replica, or+- install a license where `GET /v1/subscription` returns `ha: true`.++### Old heartbeat rows exist in the database++Stale heartbeat rows are cleaned up separately and do not define the active replica count. Active counting only considers heartbeats from the last 30 seconds.++## Related docs++- [Upgrade guidance](./upgrade.md)+- [Helm chart README](../../helm-charts/bytebase/README.md)diff --git a/docs/operations/upgrade.md b/docs/operations/upgrade.mdnew file mode 100644index 0000000000..e7749e1f2b--- /dev/null+++ b/docs/operations/upgrade.md@@ -0,0 +1,76 @@+# Upgrade guidance for HA-aware Bytebase deployments++This runbook focuses on upgrade-time checks for environments that may run Bytebase with more than one active replica.++## Scope++- Covers operator checks before, during, and after an upgrade.+- Applies to the current runtime behavior in this repository.+- Does **not** introduce a new deployment mechanism.++The bundled Helm chart still deploys a single replica by default, so HA rollouts remain operator-managed outside the chart.++## Before you upgrade++1. **Confirm the current topology.**+ - Call `GET /v1/actuator/info` and capture `version`, `externalUrl`, and `replicaCount`.+2. **Confirm whether HA is licensed.**+ - Call `GET /v1/subscription` and verify the `ha` field.+3. **Protect the metadata database.**+ - Take the backup or snapshot your standard operating procedure requires before upgrading Bytebase.+4. **Confirm shared configuration is consistent.**+ - Every HA replica should start with `--ha`, use the same `PG_URL`, and keep the same external URL during the upgrade.++## Helm chart behavior during upgrades++The chart README uses `helm upgrade` for version and configuration changes, and that is still the correct command.++However, the chart currently renders a single-replica StatefulSet. Upgrading the chart does not enable an HA topology and does not add a chart value for running multiple Bytebase application replicas.++If you operate multiple Bytebase servers today, treat that topology as operator-managed and keep the upgrade procedure for those extra replicas in your own platform tooling.++## Multi-replica upgrade guidance++If you operate multiple Bytebase replicas outside the bundled chart:++1. **Only keep multiple active replicas if HA is licensed.**+ - Without `ha: true`, Bytebase logs HA restriction warnings and background runners skip work when more than one replica is active.+2. **Keep HA startup arguments consistent across replicas.**+ - Start each replica with `--ha` and the same shared external PostgreSQL `PG_URL` before beginning the rollout.+3. **Prefer a rolling replacement instead of a full stop/start.**+ - Replace replicas gradually with your orchestrator so at least one healthy replica remains available.+4. **Wait for replica health to settle between steps.**+ - After each rollout step, check `GET /v1/actuator/info` and confirm `replicaCount` matches your expectation.+5. **Watch logs during the rollout.**+ - Investigate any HA-license warnings before continuing to the next replica.++## Single-replica upgrade guidance++If your environment uses only the bundled Helm chart with its default topology:++1. Run `helm upgrade` with the new Bytebase version or configuration.+2. Wait for the single StatefulSet replica to become healthy.+3. Verify the updated version via `GET /v1/actuator/info`.++## After the upgrade++Validate the following before closing the change:++1. `GET /v1/actuator/info` returns the expected `version`.+2. `GET /v1/actuator/info` returns the expected `replicaCount`.+3. `GET /v1/subscription` still shows the expected `ha` value.+4. Logs do not show `multiple replicas detected ... but HA is not enabled in license`.+5. Background operations resume normally after the rollout window.++## When to stop the rollout++Pause the upgrade and investigate if any of the following occur:++- `replicaCount` does not recover to the expected value.+- Replicas disagree on the external URL or metadata database configuration.+- HA-license warnings appear in a topology that is supposed to stay multi-replica.++## Related docs++- [High availability runbook](./high-availability.md)+- [Helm chart README](../../helm-charts/bytebase/README.md)diff --git a/frontend/src/components/BannersWrapper.vue b/frontend/src/components/BannersWrapper.vueindex dc469faf40..dd588dc9b5 100644--- a/frontend/src/components/BannersWrapper.vue+++ b/frontend/src/components/BannersWrapper.vue@@ -3,8 +3,7 @@<BannerDemo v-if="shouldShowDemoBanner" /><BannerSubscription v-if="shouldShowSubscriptionBanner" /><BannerExternalUrl v-if="shouldShowExternalUrlBanner" />- <!-- TODO(ed): remove the isDev when HA is ready -->- <BannerHA v-if="isDev() && shouldShowHABanner" />+ <BannerHA v-if="shouldShowHABanner" /><BannerAnnouncement /></template>diff --git a/helm-charts/bytebase/README.md b/helm-charts/bytebase/README.mdindex 564ca4f3fb..bcdc13fe58 100644--- a/helm-charts/bytebase/README.md+++ b/helm-charts/bytebase/README.md@@ -19,6 +19,15 @@ install <RELEASE_NAME> bytebase-repo/bytebase- Kubernetes 1.24+- Helm 3.9.0++## High availability note++The bundled Helm chart currently deploys Bytebase as a single-replica StatefulSet. It does not expose a Helm value for running multiple Bytebase application replicas.++If you are preparing an HA deployment, treat the multi-replica topology as operator-managed outside this chart. Each HA replica must still start with `--ha` and the same shared external PostgreSQL `PG_URL`; review the operator runbook before proceeding:++- [High availability runbook](../../docs/operations/high-availability.md)+- [Upgrade guidance](../../docs/operations/upgrade.md)+## Installing the Chart```bash@@ -57,6 +66,11 @@ helm delete --namespace <YOUR_NAMESPACE> <RELEASE_NAME>Use `helm upgrade` command to upgrade the bytebase version or configuration.+For HA-aware operations, note that this chart still renders `replicas: 1`. A `helm upgrade` does not enable a multi-replica topology by itself. If you operate multiple Bytebase servers outside this chart, keep `--ha` and the shared external PostgreSQL `PG_URL` consistent on every replica and follow the operator runbook and upgrade guidance here:++- [High availability runbook](../../docs/operations/high-availability.md)+- [Upgrade guidance](../../docs/operations/upgrade.md)+```bashhelm -n <YOUR_NAMESPACE> \--set "bytebase.option.port"={NEW_PORT} \@@ -102,4 +116,6 @@ upgrade bytebase-release bytebase-repo/bytebase- Contact <[email protected]>- [Bytebase Docs](https://docs.bytebase.com)+- [High availability runbook](../../docs/operations/high-availability.md)+- [Upgrade guidance](../../docs/operations/upgrade.md)- [Bytebase GitHub Issue Page](https://github.com/bytebase/bytebase/issues/new/choose)