Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

docs: describe the kept late statistics and the in-place overlay write

Yudhi Armyndharis232b93330b35
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 300119d61..35fd089b8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -229,7 +229,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Fix creating a channel with a null description answering a null description on whatsapp-web.js.
- Fix a group add or create charging a bare number and its `@c.us` form as two new contacts against the daily cold allowance.
- Keep the SQLite database file path when the Infrastructure page saves SQLite.
-- Write `data/.env.generated` through a staged file and a rename, so a failed save keeps the previous configuration.
+- Write `data/.env.generated` through a staged file and a rename, so a failed save keeps the previous configuration; where a rename cannot replace the file, such as a single-file mount, it is written in place, and a symlink is written through to its target.
- Refuse a configuration save that the next boot would reject, with the boot validation message, instead of persisting it.
- Reset `REDIS_TLS` and clear `REDIS_USERNAME` when the Infrastructure page selects the built-in Redis.
- Stop a bundled container on restart once its datastore is switched off, even when a pinned host still names it.
@@ -238,7 +238,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Restore a data backup onto a PostgreSQL schema that lacks an optional table instead of failing with a 500.
- Start the selected built-in containers once Docker answers after a failed first connection, and report an unreachable Docker inventory as unknown.
- Bound the Docker image and container inspections with the deadline of the other short calls, and pull an image only when the daemon says it is absent.
-- Wait at most 2 s for the database statistics in a metrics scrape and omit their series when the read is late.
+- Wait at most 2 s for the database statistics in a metrics scrape; a late read is kept and served for up to 5 minutes, and the series are omitted only when a read failed or none is recent.
- Record a `session_stopped` audit log entry for each session an import with `stopOrphans` stops, including when the import does not commit.
- Fix the Python SDK raising `ValueError` instead of the API error when a response carries an oversized numeric `Retry-After`, from SDK 0.7.0.
- Fix the Python SDK API and timeout errors failing to unpickle or copy, so they survive a worker process boundary, from SDK 0.7.0.
@@ -313,6 +313,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Drop a null optional field from plugin catalog entries instead of answering it.
- Fix the JavaScript SDK raw requests with a fragment in the path dropping their query parameters.
- Cap the JavaScript SDK `retryAfterSeconds` at 2147483 seconds, the longest delay `setTimeout` accepts once multiplied by 1000.
+- Report the database name to the Infrastructure form only under PostgreSQL, and drop a stored SQLite file path when a PostgreSQL save sends no name.
+- The message list no longer answers an empty page when its `after` row is deleted while the page is being read.
+- The message list reads its page rows by primary key on SQLite, which cuts a 100 row read of a 300k row session from about 70 ms to 5 ms.
### Dependencies
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.md
index 7ceb7e534..b1ee5f9cf 100644
--- a/docs/06-api-specification.md
+++ b/docs/06-api-specification.md
@@ -5284,7 +5284,8 @@ When the data database cannot be read the database-derived series (`openwa_sessi
`openwa_messages*`) are OMITTED rather than reported as zero — a zero would fire an alert
claiming every session had dropped. `openwa_stats_available` is what tells the two cases apart,
so alert on it rather than reading a missing series as zero. A scrape waits at most 2 seconds for
-that read and omits the series when it is late, so a database that stalls costs those series and not the
+that read and serves the last result it kept, up to 5 minutes old, when it is late; the series are omitted when
+the read failed or no result is that recent, so a database that stalls costs those series and not the
rest of the scrape. `docs/10` lists every series.
```
diff --git a/docs/10-devops-infrastructure.md b/docs/10-devops-infrastructure.md
index 179fca743..1f3176d75 100644
--- a/docs/10-devops-infrastructure.md
+++ b/docs/10-devops-infrastructure.md
@@ -930,9 +930,11 @@ aggregate overview from `StatsService` plus `process.memoryUsage()`, memoizes th
short TTL (~5s, so back-to-back scrapes don't repeat the DB scan), and exposes it at
`GET /api/metrics`.
-A scrape waits at most 2 seconds for that overview. When the read fails or is still pending after that,
-the database-derived series are left out and `openwa_stats_available` reads 0, while the process, HTTP,
-webhook and queue series are served as usual. A read still pending is shared by the scrapes that arrive
+A scrape waits at most 2 seconds for that overview. When the read fails, the database-derived series are
+left out and `openwa_stats_available` reads 0. When it is still pending after that, the scrape serves the last
+result it kept, up to 5 minutes old, and `openwa_stats_available` stays 1; the late result fills the memo for the
+next scrape. With no result that recent, such as the first scrape after start, the series are left out and the
+gauge reads 0. The process, HTTP, webhook and queue series are served as usual. A read still pending is shared by the scrapes that arrive
meanwhile, so they answer without waiting again and add no query of their own, whatever
`STATS_CACHE_TTL_MS` is. The 2 seconds bound the wait, not the query: nothing cancels it in the database.
@@ -983,29 +985,29 @@ export class MetricsService {
**Exported metric names** (the complete set — nothing else is emitted):
-| Metric | Type | Labels | Meaning |
-| -------------------------------------------- | --------- | ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
-| `openwa_up` | gauge | — | Always `1` when scraped |
-| `openwa_process_uptime_seconds` | gauge | — | Process uptime |
-| `openwa_process_resident_memory_bytes` | gauge | — | RSS |
-| `openwa_process_heap_used_bytes` | gauge | — | V8 heap used |
-| `openwa_event_loop_delay_p99_seconds` | gauge | none | p99 event-loop delay since the previous uncached scrape |
-| `openwa_event_loop_delay_max_seconds` | gauge | none | Maximum event-loop delay since the previous uncached scrape |
-| `openwa_unhandled_rejections_total` | counter | `kind` | Unhandled promise rejections since process start (`other` or `page_context_lost`) |
-| `openwa_queue_jobs` | gauge | `queue`, `state` | BullMQ jobs per queue in `wait`/`active`/`delayed`/`failed` (cluster-wide, from Redis) |
-| `openwa_stats_available` | gauge | — | 1 when the last overview read of the database-derived series below succeeded, 0 if it failed or took over 2 s |
-| `openwa_sessions_total` | gauge | — | Configured sessions |
-| `openwa_sessions_active` | gauge | — | READY (active) sessions |
-| `openwa_sessions` | gauge | `status` | Session count per status |
-| `openwa_messages_total` | gauge | `direction` (`incoming`/`outgoing`) | Current stored messages by direction |
-| `openwa_messages_failed_total` | gauge | — | Current messages in FAILED state |
-| `openwa_webhook_delivery_failures_total` | counter | — | Webhook delivery failures since process start: retries exhausted, never sent, or stopped by shutdown between direct retries |
-| `openwa_session_reconnect_attempts_total` | counter | — | Reconnect attempts scheduled across all sessions since process start |
-| `openwa_session_reconnect_loop_alerts_total` | counter | — | Reconnect-loop alerts emitted since process start |
-| `openwa_sessions_restricted` | gauge | — | Sessions whose account WhatsApp is currently restricting |
-| `openwa_send_pacing_refusals_total` | counter | `reason` | Sends refused by the pacing governor since process start |
-| `http_requests_total` | counter | `method`, `route`, `status` | HTTP requests served, by method, route and status |
-| `http_request_duration_seconds` | histogram | `method`, `route` | HTTP request duration (`_bucket` / `_sum` / `_count`) |
+| Metric | Type | Labels | Meaning |
+| -------------------------------------------- | --------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
+| `openwa_up` | gauge | — | Always `1` when scraped |
+| `openwa_process_uptime_seconds` | gauge | — | Process uptime |
+| `openwa_process_resident_memory_bytes` | gauge | — | RSS |
+| `openwa_process_heap_used_bytes` | gauge | — | V8 heap used |
+| `openwa_event_loop_delay_p99_seconds` | gauge | none | p99 event-loop delay since the previous uncached scrape |
+| `openwa_event_loop_delay_max_seconds` | gauge | none | Maximum event-loop delay since the previous uncached scrape |
+| `openwa_unhandled_rejections_total` | counter | `kind` | Unhandled promise rejections since process start (`other` or `page_context_lost`) |
+| `openwa_queue_jobs` | gauge | `queue`, `state` | BullMQ jobs per queue in `wait`/`active`/`delayed`/`failed` (cluster-wide, from Redis) |
+| `openwa_stats_available` | gauge | — | 1 when the last overview read of the database-derived series below succeeded, or is served from a late read (5 min), 0 if it failed |
+| `openwa_sessions_total` | gauge | — | Configured sessions |
+| `openwa_sessions_active` | gauge | — | READY (active) sessions |
+| `openwa_sessions` | gauge | `status` | Session count per status |
+| `openwa_messages_total` | gauge | `direction` (`incoming`/`outgoing`) | Current stored messages by direction |
+| `openwa_messages_failed_total` | gauge | — | Current messages in FAILED state |
+| `openwa_webhook_delivery_failures_total` | counter | — | Webhook delivery failures since process start: retries exhausted, never sent, or stopped by shutdown between direct retries |
+| `openwa_session_reconnect_attempts_total` | counter | — | Reconnect attempts scheduled across all sessions since process start |
+| `openwa_session_reconnect_loop_alerts_total` | counter | — | Reconnect-loop alerts emitted since process start |
+| `openwa_sessions_restricted` | gauge | — | Sessions whose account WhatsApp is currently restricting |
+| `openwa_send_pacing_refusals_total` | counter | `reason` | Sends refused by the pacing governor since process start |
+| `http_requests_total` | counter | `method`, `route`, `status` | HTTP requests served, by method, route and status |
+| `http_request_duration_seconds` | histogram | `method`, `route` | HTTP request duration (`_bucket` / `_sum` / `_count`) |
The last two are deliberately **unprefixed** so a generic RED dashboard or alert rule matches them
without knowing anything about OpenWA. They come from `src/common/metrics/request-metrics.ts`, which
@@ -1013,7 +1015,7 @@ without knowing anything about OpenWA. They come from `src/common/metrics/reques
Not every row appears on every scrape, and the difference matters when you write alerts. The
database-derived series (`openwa_sessions*`, `openwa_messages*`) are **omitted entirely** when the
-overview cannot be read within 2 seconds — `openwa_stats_available` is what tells the two cases apart, so alert on it
+overview read failed, or is late with no result from the last 5 minutes — `openwa_stats_available` is what tells the two cases apart, so alert on it
rather than reading a missing series as zero. `openwa_send_pacing_refusals_total` appears only once
the governor has refused something. `openwa_queue_jobs` appears only with `QUEUE_ENABLED=true`, and a
queue whose counts cannot be read within 2 s is left out of that scrape. For these, `absent()` is the
@@ -1033,7 +1035,7 @@ seen, so keep new renderers on that composition.
> **The database-derived series can be absent.** `openwa_sessions_*`, `openwa_messages_*` and the per-status
> breakdown come from `StatsService.getOverview()`, which is memoized for `STATS_CACHE_TTL_MS` (default 30 s,
-> shared with `GET /api/stats/overview`) behind the 5 s render cache. If that read fails or takes over 2 s — an
+> shared with `GET /api/stats/overview`) behind the 5 s render cache. If that read fails, or takes over 2 s with no result from the last 5 minutes to serve — an
> outage, a statement timeout, pool exhaustion, a `SQLITE_BUSY` under load — they are OMITTED rather than reported
> as zero, and `openwa_stats_available` goes to 0. The process, HTTP and webhook series keep being served, so `up` stays 1
> and still means "the process is alive". Alert on `openwa_stats_available == 0` for the degradation itself;