Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
Python asraudiochineseemotion-recognitionfunasrmcp-servermultilingual-asropenai-compatible-apiparaformerpunctuationpytorchreal-time-asrspeaker-diarizationspeech-recognitionspeech-to-textstreaming-asrtranscriptionvllmvoice-activity-detectionwhisper-alternative
docs(ops): make site monitoring aware of log rotation
LauraGPTdce73ccb98a3
diff --git a/docs/operations/funasr-com-site-release.md b/docs/operations/funasr-com-site-release.mdindex d1b8650c3..54a17cb3a 100644--- a/docs/operations/funasr-com-site-release.md+++ b/docs/operations/funasr-com-site-release.md@@ -84,21 +84,62 @@ the direct GitHub URL so attribution does not change search metadata. Redirecttargets are defined only in `web-pages/nginx/conversion-map.conf`; never accept atarget from a query parameter.-Check after one hour and again after 24 hours:+For each new release, check after one hour and again after 24 hours, recording+completion once. For release `20260930T211757Z`, both checks are complete; the+[follow-up record](https://github.com/modelscope/FunASR/pull/3745) reports passing+static checks with legacy-upstream and log-rotation warnings. Do not repeat these+completed windows as pending work.++Before reading logs, discover the serving master and verify its PID, command and+start identity. A failed systemd unit or an empty PID file does not mean that the+manually started master has stopped. Set `NGINX_MASTER_PID` to the freshly verified+master, then inspect log descriptors and file metadata without sending signals:```bash-tail -200 /var/log/nginx/error.log-tail -200 /var/log/nginx/funasr-conversions.log-curl -fsSI https://www.funasr.com/-curl -fsSI https://www.funasr.com/deploy/vllm.html-curl -fsSI https://www.funasr.com/blog/+set -eu+master_pid=${NGINX_MASTER_PID:?set the verified Nginx master PID}+case "$master_pid" in+ ''|*[!0-9]*) printf '%s\n' 'NGINX_MASTER_PID must be numeric' >&2; exit 2 ;;+esac+ps -p "$master_pid" -o pid=,ppid=,lstart=,args=+find "/proc/$master_pid/fd" -maxdepth 1 -type l -lname '/var/log/nginx/*' \+ -printf '%f -> %l\n' -exec stat -Lc '%d:%i %s %n' {} \;+stat -Lc '%d:%i %s %n' /var/log/nginx/error.log /var/log/nginx/funasr-conversions.log```-Count non-smoke conversion requests by route:+Compare device/inode identities, not just filenames. After rotation, live+descriptors may still reference `.log.1` or deleted files while the current+`.log` paths are empty. Empty files are not evidence of no requests or errors.+Read the actual open logs and relevant rotated and compressed history for the+chosen time interval; do not draw a health conclusion if that coverage is missing.+Use equal-duration before/after windows and retain aggregate status/upstream+counts only, not visitor addresses or raw requests. Filter known validation+traffic when possible, but remaining requests are not organic traffic by default+and do not establish star-growth attribution.++The 2026-10-02 read-only inspection found an empty configured PID file while the+manual master was alive. The packaged rotation selector could not find that+master in test mode, and its init-script helper is written to return success unconditionally.+The event that emptied the PID file was not established. Repairing process+ownership or log reopening is a separate operational action: do not rewrite the+PID file, restart the working master, or invoke service actions during acceptance.++Check legacy upstream errors separately. Existing WebSocket error-page routing+converts upstream `502/503/504` failures into `302` redirects, so zero visible 5xx+is not proof of backend health. Confirm the upstream/listener against the current+configuration; do not attribute pre-existing backend timeouts to a static release.++Header probes supplement, but do not replace, the page-body hashes, indexed-route,+asset, mobile-layout and fixed-redirect checks described above:```bash-grep -vE '"(FunASR release smoke|curl/)' /var/log/nginx/funasr-conversions.log \- | awk '{print $7}' | sort | uniq -c | sort -nr+curl --max-time 15 -fsSI https://www.funasr.com/+curl --max-time 15 -fsSI https://www.funasr.com/deploy/vllm.html+curl --max-time 15 -fsSI https://www.funasr.com/blog/```-Roll back on elevated 5xx responses, missing indexed routes, mobile overflow, missing assets, invalid conversion redirects, or a failed static validation.+Investigate elevated 5xx responses, missing indexed routes, mobile overflow,+missing assets, invalid conversion redirects or failed static validation. Roll+back when the static release is implicated, after checking the current release+and retained backup; do not roll back healthy static content solely because a+legacy backend or log-rotation path is unhealthy.diff --git a/web-pages/product-site/tests/test_release_scripts.py b/web-pages/product-site/tests/test_release_scripts.pyindex 01fc03cdd..3d339e438 100644--- a/web-pages/product-site/tests/test_release_scripts.py+++ b/web-pages/product-site/tests/test_release_scripts.py@@ -146,3 +146,52 @@ def test_nginx_contract_uses_current_release_and_hardening():assert 'funasr-conversions.log' in configassert r'\.[0-9a-f]{12}' in configassert 'max-age=31536000, immutable' in config+++def test_monitoring_docs_use_actual_open_log_inodes():+ document = (REPO_ROOT / 'docs/operations/funasr-com-site-release.md').read_text()+ monitoring = document.split('## Monitoring\n', 1)[1]++ assert '/proc/$master_pid/fd' in monitoring+ assert "stat -Lc '%d:%i %s %n'" in monitoring+ assert '.log.1' in monitoring+ assert 'rotated and compressed history' in monitoring+ assert 'tail -200 /var/log/nginx/' not in monitoring+ assert "grep -vE" not in monitoring+++def test_monitoring_metadata_snippet_requires_explicit_numeric_master_pid():+ document = (REPO_ROOT / 'docs/operations/funasr-com-site-release.md').read_text()+ monitoring = document.split('## Monitoring\n', 1)[1]+ blocks = re.findall(r'\x60\x60\x60bash\n(.*?)\x60\x60\x60', monitoring, re.S)+ assert blocks and '{NGINX_MASTER_PID:?' in blocks[0]+ for block in blocks:+ subprocess.run(['bash', '-n'], input=block, text=True, check=True)+ assert not re.search(r'(?m)^\s*(kill|invoke-rc.d|start-stop-daemon)\b', block)++ environment = os.environ.copy()+ environment.pop('NGINX_MASTER_PID', None)+ missing = subprocess.run(+ ['bash', '-c', blocks[0]], env=environment, capture_output=True, text=True,+ )+ assert missing.returncode != 0+ assert 'verified Nginx master PID' in missing.stderr++ environment['NGINX_MASTER_PID'] = 'not-a-pid'+ invalid = subprocess.run(+ ['bash', '-c', blocks[0]], env=environment, capture_output=True, text=True,+ )+ assert invalid.returncode == 2+ assert 'NGINX_MASTER_PID must be numeric' in invalid.stderr+++def test_monitoring_docs_distinguish_static_checks_from_backend_health():+ document = (REPO_ROOT / 'docs/operations/funasr-com-site-release.md').read_text()+ monitoring = document.split('## Monitoring\n', 1)[1]++ assert '502/503/504' in monitoring+ assert '302' in monitoring+ assert 'not proof of backend health' in monitoring+ assert 'equal-duration' in monitoring+ assert 'not organic traffic' in monitoring+ assert 'static release is implicated' in monitoring