ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.
Python astracloudflarecontrol-panelcyberpaneldnsfablehosting-control-panelhosting-panelkimi-k3server-managementserverkitubuntu-web-panelwebserver
feat(extensions): installed plugins survive panel updates (#48)
Juan Denis1c911b50dfee
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex ab9057cf..f1788388 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -20,6 +20,15 @@ awaiting a stable release:### Added+- **Extensions platform (Phase 7 — installed extensions survive panel updates)** —+ previously a panel update deployed a fresh source tree preserving only `.env`+ and the database, silently wiping any URL/registry/upload-installed extension's+ files (the install row then flipped to `error` on the next boot). Two-layer fix:+ the updater now carries user-installed plugin directories forward into the new+ tree (before the frontend build, so their UI recompiles), and the backend gained+ a boot-time repair pass that restores builtin installs from `builtin-extensions/`+ and re-downloads URL installs from their recorded source — upload-only installs+ it can't restore get a clear "re-upload" error instead of a cryptic import failure.- **Extensions platform (Phase 7 — extensions can now contribute tabs to coretab groups)** — a new `tabs` contribution kind lets an installed extension adda real tab to a core-owned tab group (Files, Servers, Observability): the tabdiff --git a/backend/app/services/plugin_service.py b/backend/app/services/plugin_service.pyindex c343d348..e67b9332 100644--- a/backend/app/services/plugin_service.py+++ b/backend/app/services/plugin_service.py@@ -297,7 +297,8 @@ def _update_plugin_metadata(plugin, manifest):plugin.category = manifest.get('category', 'utility')-def install_from_url(url, user_id=None, expected_sha256=None, force=False):+def install_from_url(url, user_id=None, expected_sha256=None, force=False,+ hot_load=True):"""Download and install a plugin from a URL.Args:@@ -306,6 +307,9 @@ def install_from_url(url, user_id=None, expected_sha256=None, force=False):expected_sha256: if given, the downloaded zip must match this digestbefore extraction — a mismatch is a hard failure (no partial install)force: allow reinstalling over an already-active plugin (used by updates)+ hot_load: register the blueprint into the running app immediately.+ The boot-time repair pass (#48) passes False — load_all_plugins+ registers right after, and a double registration would error.Returns:InstalledPlugin instance@@ -327,10 +331,11 @@ def install_from_url(url, user_id=None, expected_sha256=None, force=False):return _install_from_buffer(buf, source_url=url, source_type='url', user_id=user_id, force=force,+ hot_load=hot_load,)-def install_from_path(path, user_id=None):+def install_from_path(path, user_id=None, force=False, hot_load=True):"""Install a plugin from a local directory on the panel host.Useful during plugin development: point at the working tree, install,@@ -395,6 +400,7 @@ def install_from_path(path, user_id=None):return _install_from_buffer(buf, source_url=path, source_type='local', user_id=user_id,+ force=force, hot_load=hot_load,)@@ -411,14 +417,16 @@ def install_from_zip(zip_bytes, user_id=None, source_name=None):)-def _install_from_buffer(buf, source_url, source_type, user_id=None, force=False):+def _install_from_buffer(buf, source_url, source_type, user_id=None, force=False,+ hot_load=True):"""Shared install pipeline: takes a seekable BytesIO containing a zip,extracts it into the panel's plugin dirs, and registers / hot-loadsthe resulting blueprint. All public install_* helpers funnel here sobehavior matches across URL / local / upload sources.`force=True` allows reinstalling over an active plugin (the update path);- normal installs still refuse to clobber an active install.+ normal installs still refuse to clobber an active install. `hot_load=False`+ skips the immediate blueprint registration (the boot repair pass, #48)."""_ensure_dirs()@@ -584,7 +592,7 @@ def _install_from_buffer(buf, source_url, source_type, user_id=None, force=Falsedb.session.commit()# Try to register the blueprint immediately (hot-load)- if has_backend and entry_point:+ if has_backend and entry_point and hot_load:try:_register_plugin_blueprint(plugin)except Exception as e:@@ -821,6 +829,87 @@ def _regenerate_frontend_manifest():logger.info(f'Frontend plugin manifest regenerated with {len(entries)} plugin(s)')+def repair_missing_plugins():+ """Self-heal installed plugins whose files are gone (#48).++ A panel update deploys a fresh source tree preserving only `.env` and the+ database, so a URL/upload-installed plugin's extracted files (and a+ copy-installed builtin's backend under ``app/plugins/``) can vanish while+ the InstalledPlugin row stays active — the boot loader would then flip it+ to `error`. Before the loader runs, restore what we can:++ - builtin slug → reinstall from ``builtin-extensions/`` (in-repo,+ always present in the fresh tree)+ - source_type 'url' → re-download from ``source_url`` (no checksum —+ the original install verified it; best effort)+ - anything else → mark `error` with a re-upload hint++ Reinstalls run with ``hot_load=False`` — load_all_plugins registers the+ blueprints right after this pass. Never raises; each repair is+ best-effort and logged. The update.sh carry-forward is the first line of+ defense; this is the backstop (e.g. Docker image updates).+ """+ builtin_by_slug = {}+ try:+ for entry in list_builtin_extensions():+ builtin_by_slug[entry['slug']] = entry+ except Exception:+ pass++ plugins = InstalledPlugin.query.filter(+ InstalledPlugin.status.in_([+ InstalledPlugin.STATUS_ACTIVE,+ InstalledPlugin.STATUS_ERROR,+ ]),+ ).all()++ for plugin in plugins:+ slug = plugin.slug+ # Flagships load in-place from builtin-extensions/ (repo-tracked, never+ # copy-installed) — nothing on disk to go missing.+ if slug in FLAGSHIP_SLUGS:+ continue++ backend_missing = bool(plugin.has_backend) and not os.path.isdir(+ os.path.join(BACKEND_PLUGINS_DIR, slug))+ frontend_missing = bool(plugin.has_frontend) and not os.path.isdir(+ os.path.join(FRONTEND_PLUGINS_DIR, slug))+ if not backend_missing and not frontend_missing:+ continue++ halves = ' + '.join(h for h, m in (('backend', backend_missing),+ ('frontend', frontend_missing)) if m)+ logger.warning(+ f'Plugin {slug}: {halves} files missing (panel update?) — attempting repair')++ try:+ if slug in builtin_by_slug:+ install_from_path(builtin_by_slug[slug]['path'],+ user_id=plugin.installed_by,+ force=True, hot_load=False)+ logger.info(f'Plugin {slug}: restored from builtin-extensions/')+ elif plugin.source_type == 'url' and plugin.source_url:+ install_from_url(plugin.source_url, user_id=plugin.installed_by,+ force=True, hot_load=False)+ logger.info(f'Plugin {slug}: reinstalled from {plugin.source_url}')+ else:+ plugin.status = InstalledPlugin.STATUS_ERROR+ plugin.error_message = (+ 'Plugin files are missing (likely removed by a panel update) '+ 'and there is no source URL to reinstall from. Re-upload the '+ 'plugin zip from the Marketplace to restore it.'+ )+ db.session.commit()+ except Exception as e:+ logger.error(f'Plugin {slug}: repair failed: {e}')+ try:+ plugin.status = InstalledPlugin.STATUS_ERROR+ plugin.error_message = f'Repair after panel update failed: {e}'+ db.session.commit()+ except Exception:+ db.session.rollback()++def load_all_plugins(app):"""Load all active plugin blueprints at app startup.@@ -829,6 +918,13 @@ def load_all_plugins(app):_ensure_dirs()with app.app_context():+ # Restore any install whose files a panel update wiped (#48) BEFORE+ # querying/loading, so a repaired plugin loads normally this boot.+ try:+ repair_missing_plugins()+ except Exception as e:+ logger.warning(f'Plugin repair pass failed: {e}')+plugins = InstalledPlugin.query.filter_by(status=InstalledPlugin.STATUS_ACTIVE,has_backend=True,diff --git a/backend/tests/test_plugins_pipeline.py b/backend/tests/test_plugins_pipeline.pyindex 046ac197..68071a55 100644--- a/backend/tests/test_plugins_pipeline.py+++ b/backend/tests/test_plugins_pipeline.py@@ -223,3 +223,86 @@ def test_disable_guard_returns_503(app):r = c.get('/api/v1/guard-test/ping')assert r.status_code == 503assert r.get_json()['status'] == 'disabled'+++# --------------------------------------------------------------------------- #+# Boot repair pass (#48): plugins whose files a panel update wiped+# --------------------------------------------------------------------------- #++def test_repair_restores_builtin_files(app, plugin_dirs):+ """A builtin install whose extracted files vanished (fresh update tree) is+ restored from builtin-extensions/ and stays active."""+ import shutil+ _write_builtin(plugin_dirs['builtin'])+ plugin = plugin_service.install_builtin_extension('serverkit-demo')+ assert (plugin_dirs['frontend'] / 'serverkit-demo' / 'index.jsx').exists()++ # Simulate the update wiping the extracted plugin dirs.+ shutil.rmtree(plugin_dirs['frontend'] / 'serverkit-demo')++ plugin_service.repair_missing_plugins()++ assert (plugin_dirs['frontend'] / 'serverkit-demo' / 'index.jsx').exists()+ db.session.refresh(plugin)+ assert plugin.status == InstalledPlugin.STATUS_ACTIVE+++def test_repair_reinstalls_url_plugin_from_source(app, plugin_dirs, monkeypatch):+ """A URL-installed plugin with missing files is re-downloaded from its+ source_url, without hot-loading (the boot loader registers right after)."""+ p = InstalledPlugin(+ name='remote-thing', display_name='Remote Thing', slug='remote-thing',+ version='1.0.0', status=InstalledPlugin.STATUS_ACTIVE,+ has_backend=True, source_type='url',+ source_url='https://example.com/remote-thing.zip',+ )+ p.manifest = {}+ db.session.add(p)+ db.session.commit()++ calls = []+ monkeypatch.setattr(+ plugin_service, 'install_from_url',+ lambda url, **kw: calls.append((url, kw)))++ plugin_service.repair_missing_plugins()++ assert calls, 'repair should reinstall from the recorded source_url'+ url, kw = calls[0]+ assert url == 'https://example.com/remote-thing.zip'+ assert kw.get('force') is True+ assert kw.get('hot_load') is False+++def test_repair_marks_sourceless_install_as_error(app, plugin_dirs):+ """An upload-installed plugin (no source URL) with missing files can't be+ auto-restored — it flips to error with a re-upload hint instead."""+ p = InstalledPlugin(+ name='uploaded-thing', display_name='Uploaded Thing',+ slug='uploaded-thing', version='1.0.0',+ status=InstalledPlugin.STATUS_ACTIVE,+ has_frontend=True, source_type='upload', source_url='uploaded.zip',+ )+ p.manifest = {}+ db.session.add(p)+ db.session.commit()++ plugin_service.repair_missing_plugins()++ db.session.refresh(p)+ assert p.status == InstalledPlugin.STATUS_ERROR+ assert 'Re-upload' in p.error_message+++def test_repair_leaves_intact_installs_alone(app, plugin_dirs, monkeypatch):+ """A healthy install (files present) is not touched by the repair pass."""+ _write_builtin(plugin_dirs['builtin'])+ plugin = plugin_service.install_builtin_extension('serverkit-demo')++ monkeypatch.setattr(+ plugin_service, 'install_from_path',+ lambda *a, **kw: pytest.fail('repair must not reinstall a healthy plugin'))++ plugin_service.repair_missing_plugins()+ db.session.refresh(plugin)+ assert plugin.status == InstalledPlugin.STATUS_ACTIVEdiff --git a/scripts/test/test_update.sh b/scripts/test/test_update.shindex 60b42853..58051cee 100644--- a/scripts/test/test_update.sh+++ b/scripts/test/test_update.sh@@ -411,6 +411,36 @@ elsebad "report_app_uptime_regressions should pass when nothing regressed"fi+# --------------------------------------------------------------------------+# T18 — preserve_installed_plugins (#48): user-installed plugin dirs are+# carried into the new tree; dirs the new tree already ships are NOT+# overwritten; __pycache__ is skipped; and the function never dies under+# set -e (it runs inside deploy_source/deploy_release).+# --------------------------------------------------------------------------+t="$WORK/t18"+mkdir -p "$t/old/backend/app/plugins/third-party" \+ "$t/old/frontend/src/plugins/third-party" \+ "$t/old/backend/app/plugins/shipped-plugin" \+ "$t/old/backend/app/plugins/__pycache__" \+ "$t/new/backend/app/plugins/shipped-plugin"+printf 'user-code\n' > "$t/old/backend/app/plugins/third-party/blueprint.py"+printf 'user-ui\n' > "$t/old/frontend/src/plugins/third-party/index.jsx"+printf 'old-copy\n' > "$t/old/backend/app/plugins/shipped-plugin/__init__.py"+printf 'stale\n' > "$t/old/backend/app/plugins/__pycache__/x.pyc"+printf 'repo-copy\n' > "$t/new/backend/app/plugins/shipped-plugin/__init__.py"+if (+ set -Eeuo pipefail+ preserve_installed_plugins "$t/old" "$t/new"+ [ -f "$t/new/backend/app/plugins/third-party/blueprint.py" ]+ [ -f "$t/new/frontend/src/plugins/third-party/index.jsx" ]+ [ ! -d "$t/new/backend/app/plugins/__pycache__" ]+ grep -q repo-copy "$t/new/backend/app/plugins/shipped-plugin/__init__.py"+) >/dev/null 2>&1; then+ ok "preserve_installed_plugins carries user plugins forward without clobbering repo-shipped ones"+else+ bad "preserve_installed_plugins lost a user plugin, clobbered a shipped one, or died under set -e"+fi+# --------------------------------------------------------------------------printf '\n%d passed, %d failed, %d skipped\n\n' "$PASS" "$FAIL" "$SKIP"[ "$FAIL" -eq 0 ]diff --git a/scripts/update.sh b/scripts/update.shindex 529d2335..2778c745 100644--- a/scripts/update.sh+++ b/scripts/update.sh@@ -747,6 +747,39 @@ backup_current() {fi}+# ---------------------------------------------------------------------------+# Carry user-installed plugins across a redeploy (#48).+#+# URL/registry/upload-installed extensions live as extracted dirs under+# backend/app/plugins/<slug> and frontend/src/plugins/<slug>. A fresh+# clone/tarball only contains the repo-tracked ones, so anything the user+# installed would silently vanish — copy forward every plugin dir the new+# tree doesn't already ship. Runs BEFORE the frontend build so carried+# frontends compile into the new bundle. The backend also has a boot-time+# repair pass as a backstop (plugin_service.repair_missing_plugins).+# ---------------------------------------------------------------------------+preserve_installed_plugins() {+ local src="$1" target="$2"+ local sub dir name+ for sub in backend/app/plugins frontend/src/plugins; do+ [ -d "$src/$sub" ] || continue+ for dir in "$src/$sub"/*/; do+ [ -d "$dir" ] || continue+ name="$(basename "$dir")"+ [ "$name" = "__pycache__" ] && continue+ if [ ! -e "$target/$sub/$name" ]; then+ mkdir -p "$target/$sub" 2>/dev/null || true+ if cp -a "$dir" "$target/$sub/$name" 2>/dev/null; then+ info "Preserved installed plugin: $sub/$name"+ else+ warn "Could not preserve plugin $sub/$name"+ fi+ fi+ done+ done+ return 0+}+# ---------------------------------------------------------------------------# Deploy source into the next directory# ---------------------------------------------------------------------------@@ -777,6 +810,8 @@ deploy_source() {cp "$tmp_db" "$target/backend/instance/serverkit.db" 2>/dev/null || truerm -f "$tmp_env" "$tmp_db"+ preserve_installed_plugins "$INSTALL_DIR" "$target"+chmod +x "$target/serverkit"chmod +x "$target/scripts/"*.sh 2>/dev/null || true@@ -814,6 +849,7 @@ deploy_release() {cp "$INSTALL_DIR/.env" "$unpacked/.env" 2>/dev/null || truemkdir -p "$unpacked/backend/instance"cp "$INSTALL_DIR/backend/instance/serverkit.db" "$unpacked/backend/instance/serverkit.db" 2>/dev/null || true+ preserve_installed_plugins "$INSTALL_DIR" "$unpacked"rm -rf "$target"cp -a "$unpacked" "$target"