ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.
Python astracloudflarecontrol-panelcyberpaneldnsfablehosting-control-panelhosting-panelkimi-k3server-managementserverkitubuntu-web-panelwebserver
feat(manifest): network identity — fromServer, SERVER_PUBLIC_IP, shared net (plan 35 ph5)
Juan Denis0b21ef93f5d4
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 175d9cad..92cfc9b1 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -56,6 +56,13 @@ awaiting a stable release:are listed in plain words in the plan and written to a`manifest.host_requirements` audit line on apply — never applied silently;`kernelModules` are an advisory `/proc/modules` check.+- **Appliance tier — network identity** — an env var can bind the service's own+ advertised address with `fromServer: { property: publicIp|hostname }` (the+ WebRTC/NAT need; a missing IP is a `fromserver_no_ip` blocker), and templates+ gain a `${SERVER_PUBLIC_IP}` magic variable. Manifest-generated app projects+ now attach to a shared external `serverkit` docker network (created+ idempotently), so a `fromService` host reference resolves cross-app by+ container name at runtime. Static egress IPs remain out of scope (documented).- **New Service wizard clarity** — each source card now shows a short explainerstrip when selected (what it does, what to have ready, the next steps), and a"Docs" link to the matching serverkit.ai guide (hidden under White Label). Thediff --git a/backend/app/services/buildpack_service.py b/backend/app/services/buildpack_service.pyindex c4650c3f..1bdb26f1 100644--- a/backend/app/services/buildpack_service.py+++ b/backend/app/services/buildpack_service.py@@ -662,7 +662,8 @@ class BuildpackService:@classmethoddef generate_compose(cls, plan: Dict, app_name: str, extra_ports: Optional[list] = None,volumes: Optional[list] = None,- named_volumes: Optional[list] = None) -> str:+ named_volumes: Optional[list] = None,+ networks: Optional[list] = None) -> str:"""Return a docker-compose snippet wrapping the generated image.Appliance tier (plan 35): ``extra_ports`` are raw compose publish@@ -690,10 +691,19 @@ class BuildpackService:lines.append(' volumes:')for spec in volumes:lines.append(f' - "{spec}"')+ if networks:+ lines.append(' networks:')+ for net in networks:+ lines.append(f' - {net}')if named_volumes:lines.append('volumes:')for name in named_volumes:lines.append(f' {name}: {{}}')+ if networks:+ lines.append('networks:')+ for net in networks:+ lines.append(f' {net}:')+ lines.append(' external: true')return '\n'.join(lines) + '\n'# ------------------------------------------------------------------ #diff --git a/backend/app/services/docker_service.py b/backend/app/services/docker_service.pyindex 65fa2f85..dc31de51 100644--- a/backend/app/services/docker_service.py+++ b/backend/app/services/docker_service.py@@ -781,6 +781,24 @@ class DockerService:except Exception as e:return {'success': False, 'error': str(e)}+ @staticmethod+ def ensure_network(name, driver='bridge'):+ """Idempotently ensure a Docker network exists (appliance tier, plan 35).++ The shared external ``serverkit`` network lets manifest-generated app+ projects resolve each other by container name (fromService.host)."""+ try:+ inspect = subprocess.run(+ ['docker', 'network', 'inspect', name],+ capture_output=True, text=True)+ if inspect.returncode == 0:+ return {'success': True, 'existed': True}+ except Exception as e:+ return {'success': False, 'error': str(e)}+ created = DockerService.create_network(name, driver=driver)+ created['existed'] = False+ return created+@staticmethoddef remove_network(network_id):"""Remove a Docker network."""@@ -1221,7 +1239,7 @@ class DockerService:@staticmethoddef create_docker_app(app_path, app_name, image, ports=None, volumes=None, env=None,named_volumes=None, cpu_limit=None, memory_limit=None,- host_requirements=None):+ host_requirements=None, networks=None):"""Create a Docker-based application with docker-compose.``named_volumes`` is a list of managed Docker volume names to declare as@@ -1283,6 +1301,14 @@ class DockerService:if hr.get('devices'):svc['devices'] = list(hr['devices'])+ if networks:+ # Attach to shared external network(s) so manifest apps resolve+ # each other by container name (fromService.host, plan 35).+ for net in networks:+ DockerService.ensure_network(net)+ compose['services'][app_name]['networks'] = list(networks)+ compose['networks'] = {net: {'external': True} for net in networks}+compose_path = os.path.join(app_path, 'docker-compose.yml')with open(compose_path, 'w') as f:yaml.dump(compose, f, default_flow_style=False)diff --git a/backend/app/services/env_reference_service.py b/backend/app/services/env_reference_service.pyindex f1604cc0..41f2781b 100644--- a/backend/app/services/env_reference_service.py+++ b/backend/app/services/env_reference_service.py@@ -28,8 +28,39 @@ class EnvReferenceResolver:return cls._resolve_secret(ref.get('secret'))if kind == 'service':return cls._resolve_service(app, ref.get('service'), ref.get('property'))+ if kind == 'server':+ return cls._resolve_server(app, ref.get('property'))return None, f'unknown reference kind: {kind}'+ # -- fromServer (appliance tier, plan 35) -------------------------------++ @classmethod+ def _resolve_server(cls, app, prop: Optional[str]) -> Tuple[Optional[str], Optional[str]]:+ """The service's own advertised identity — the WebRTC/NAT need. Resolves+ against the app's target server, or the panel host when unassigned."""+ prop = prop or 'publicIp'+ server_id = getattr(app, 'server_id', None)+ if server_id:+ from app.models.server import Server+ srv = Server.query.get(server_id)+ if not srv:+ return None, 'target server not found'+ if prop == 'publicIp':+ return (srv.ip_address, None) if srv.ip_address \+ else (None, 'target server has no recorded IP')+ if prop == 'hostname':+ return (getattr(srv, 'hostname', None) or srv.name or None), None+ return None, f'unknown server property `{prop}`'+ # panel host+ if prop == 'publicIp':+ from app.services.site_domain_service import SiteDomainService+ ip = SiteDomainService.server_ip()+ return (ip, None) if ip else (None, 'panel host public IP is not configured')+ if prop == 'hostname':+ import socket+ return socket.gethostname(), None+ return None, f'unknown server property `{prop}`'+# -- fromSecret ---------------------------------------------------------@classmethoddiff --git a/backend/app/services/manifest_apply_service.py b/backend/app/services/manifest_apply_service.pyindex 97a16f84..e1b9b036 100644--- a/backend/app/services/manifest_apply_service.py+++ b/backend/app/services/manifest_apply_service.py@@ -378,6 +378,11 @@ class ManifestApplyService:desired = {'kind': 'service', 'service': sr['name'], 'property': sr['property']}if cls._ref_changed(live_vars.get(key), desired):steps.append(cls._set_env_ref_step(name, app, key, desired))+ elif source == 'server':+ sv = ref['server_ref']+ desired = {'kind': 'server', 'property': sv['property']}+ if cls._ref_changed(live_vars.get(key), desired):+ steps.append(cls._set_env_ref_step(name, app, key, desired))# disksfor disk in resolved.get('disks', []):@@ -498,11 +503,25 @@ class ManifestApplyService:if resolved.get('kind') != 'app':continuename = resolved['name']+ server_ref = resolved.get('server')++ # fromServer publicIp must resolve (plan 35) — independent of the+ # other appliance features; a service may bind only its own IP.+ needs_ip = any(+ r.get('source') == 'server'+ and (r.get('server_ref') or {}).get('property') == 'publicIp'+ for r in resolved.get('env_refs') or [])+ if needs_ip and not cls._server_public_ip(server_ref):+ where = f'server {server_ref}' if server_ref else 'the panel host'+ blockers.append({+ 'kind': 'fromserver_no_ip', 'service': name,+ 'message': f'service {name} binds its public IP via fromServer, but '+ f'{where} has no recorded public IP — set it, then re-apply.'})+feats = cls._service_features(resolved)if not feats:continueneed = ' and '.join(feats)- server_ref = resolved.get('server')# Remote target: appliance apply runs on the panel host only (plan 17's# remote-dispatch deferral). Say so instead of half-deploying.@@ -644,6 +663,18 @@ class ManifestApplyService:if str(name).isdigit() else None) \or ContainerRegistry.query.filter_by(name=name).first()+ @classmethod+ def _server_public_ip(cls, server_ref: Optional[str]) -> Optional[str]:+ """Advertised public IP of the manifest target (panel host by default)."""+ if server_ref:+ srv = cls._resolve_server_obj(server_ref)+ return getattr(srv, 'ip_address', None) if srv else None+ try:+ from app.services.site_domain_service import SiteDomainService+ return SiteDomainService.server_ip()+ except Exception:+ return None+@staticmethoddef _registry_has_credential(reg) -> bool:# ECR authenticates via a key-pair exchange, so no stored secret needed.diff --git a/backend/app/services/manifest_spec_service.py b/backend/app/services/manifest_spec_service.pyindex 2f9facf0..969cd172 100644--- a/backend/app/services/manifest_spec_service.py+++ b/backend/app/services/manifest_spec_service.py@@ -57,6 +57,9 @@ FROM_SERVICE_PROPERTIES = ('connectionString', 'host', 'port', 'database', 'username', 'password', 'url',)+# Appliance tier (plan 35): a service's own advertised identity.+FROM_SERVER_PROPERTIES = ('publicIp', 'hostname')+BACKUP_SCHEDULES = ('hourly', 'daily', 'weekly', 'monthly')# Appliance tier (plan 35): typed L4 port publishes.@@ -216,6 +219,14 @@ MANIFEST_SCHEMA: Dict[str, Any] = {"property": {"enum": list(FROM_SERVICE_PROPERTIES)},},},+ "fromServer": {+ "type": "object",+ "required": ["property"],+ "additionalProperties": False,+ "properties": {+ "property": {"enum": list(FROM_SERVER_PROPERTIES)},+ },+ },"generate": {"type": "boolean"},},},@@ -672,18 +683,21 @@ class ManifestSpecService:from_secret = cls._alias(var, 'fromSecret', 'from_secret')from_service = cls._alias(var, 'fromService', 'from_service')+ from_server = cls._alias(var, 'fromServer', 'from_server')has_value = 'value' in var and var.get('value') is not Nonegenerate = bool(var.get('generate'))- sources = [s for s in (has_value, bool(from_secret), bool(from_service), generate) if s]+ sources = [s for s in (has_value, bool(from_secret), bool(from_service),+ generate, bool(from_server)) if s]if len(sources) > 1:errors.append(f'{where}: env `{key}` must declare exactly one of '- 'value/fromSecret/fromService/generate'+ 'value/fromSecret/fromService/fromServer/generate')source = 'placeholder'service_ref = None+ server_ref = Noneif has_value:source = 'value'elif from_secret:@@ -696,6 +710,9 @@ class ManifestSpecService:'name': from_service.get('name'),'property': from_service.get('property'),}+ elif isinstance(from_server, dict):+ source = 'server'+ server_ref = {'property': from_server.get('property') or 'publicIp'}return {'key': key,@@ -703,6 +720,7 @@ class ManifestSpecService:'value': var.get('value') if has_value else None,'secret_name': from_secret or None,'service_ref': service_ref,+ 'server_ref': server_ref,'secret': source in ('secret', 'generate') or bool(var.get('secret')),}diff --git a/backend/app/services/template_service.py b/backend/app/services/template_service.pyindex 470dd03f..02ad9447 100644--- a/backend/app/services/template_service.py+++ b/backend/app/services/template_service.py@@ -205,7 +205,9 @@ class TemplateService:# Matches ``${SERVICE_...}`` magic tokens specifically (a subset of the# generic ``${VAR}`` substitution pattern). ``<NAME>`` may be empty-safe:# we require at least one trailing char after the prefix.- MAGIC_TOKEN_PATTERN = r'\$\{(SERVICE_(?:PASSWORD|USER|FQDN|URL|BASE64)_[A-Z0-9_]+)\}'+ MAGIC_TOKEN_PATTERN = (+ r'\$\{(SERVICE_(?:PASSWORD|USER|FQDN|URL|BASE64)_[A-Z0-9_]+|SERVER_PUBLIC_IP)\}'+ )# Default password length for magic SERVICE_PASSWORD_* tokens.MAGIC_PASSWORD_LENGTH = 32@@ -214,6 +216,8 @@ class TemplateService:def _classify_magic_token(cls, token: str):"""Return ``(kind, name)`` for a bare magic token (no ``${}``), or``(None, None)`` if it is not a recognized magic variable."""+ if token == 'SERVER_PUBLIC_IP': # appliance tier (plan 35), no name suffix+ return 'server_public_ip', ''for prefix, kind in cls.MAGIC_PREFIXES:if token.startswith(prefix):return kind, token[len(prefix):]@@ -259,6 +263,15 @@ class TemplateService:scheme = str(context.get('scheme') or 'http')host = context.get('fqdn') or context.get('app_name') or 'localhost'return f'{scheme}://{host}'+ if kind == 'server_public_ip':+ ip = context.get('server_public_ip')+ if not ip:+ try:+ from app.services.site_domain_service import SiteDomainService+ ip = SiteDomainService.server_ip()+ except Exception:+ ip = None+ return str(ip or '')return ''@classmethoddiff --git a/backend/tests/test_manifest_network_identity.py b/backend/tests/test_manifest_network_identity.pynew file mode 100644index 00000000..65bf64bd--- /dev/null+++ b/backend/tests/test_manifest_network_identity.py@@ -0,0 +1,150 @@+"""Proving tests for Phase 5 of the Appliance tier (plan 35):+the fromServer binder, the ${SERVER_PUBLIC_IP} magic var, and cross-app+name resolution over the shared serverkit network."""++import pytest+import yaml++import app.models.application_manifest # noqa: F401+from app.services.manifest_spec_service import ManifestSpecService, ManifestError+from app.services.manifest_apply_service import ManifestApplyService+from app.services.env_reference_service import EnvReferenceResolver+from app.services.template_service import TemplateService+from app.services.docker_service import DockerService+++FROMSERVER_MANIFEST = {+ 'version': 1,+ 'services': [{+ 'name': 'bridge', 'type': 'docker',+ 'envVars': [{'key': 'ADVERTISE_IP', 'fromServer': {'property': 'publicIp'}}],+ }],+}+++def project(app):+ from app import db+ from app.models import Project, Environment+ from app.services.workspace_service import WorkspaceService+ ws = WorkspaceService.ensure_default_workspace()+ proj = Project(workspace_id=ws.id, name='Net', slug='net')+ db.session.add(proj)+ db.session.commit()+ env = Environment(project_id=proj.id, name='Production', slug='production', is_default=True)+ db.session.add(env)+ db.session.commit()+ return proj+++def owner(app):+ from app import db+ from app.models import User+ user = User.query.filter_by(username='testadmin').first()+ if not user:+ user = User(username='testadmin', email='[email protected]', role='admin')+ if hasattr(user, 'set_password'):+ user.set_password('admin')+ db.session.add(user)+ db.session.commit()+ return user+++# -- normalizer -------------------------------------------------------------++def test_fromserver_normalizes():+ n = ManifestSpecService.normalize(FROMSERVER_MANIFEST)+ var = n['services'][0]['env_vars'][0]+ assert var['source'] == 'server'+ assert var['server_ref'] == {'property': 'publicIp'}+++def test_fromserver_conflicts_with_value():+ with pytest.raises(ManifestError):+ ManifestSpecService.normalize({+ 'version': 1,+ 'services': [{'name': 'x', 'type': 'docker', 'envVars': [+ {'key': 'IP', 'value': '1.2.3.4', 'fromServer': {'property': 'publicIp'}}]}],+ })+++# -- resolver ---------------------------------------------------------------++def test_resolver_binds_panel_public_ip(monkeypatch, project, owner):+ from app import db+ from app.models import Application+ from app.services.site_domain_service import SiteDomainService+ monkeypatch.setattr(SiteDomainService, 'server_ip', classmethod(lambda cls: '203.0.113.9'))+ app_row = Application(name='bridge', app_type='docker', user_id=owner.id,+ project_id=project.id, status='stopped')+ db.session.add(app_row)+ db.session.commit()+ value, err = EnvReferenceResolver.resolve(app_row, {'kind': 'server', 'property': 'publicIp'})+ assert err is None+ assert value == '203.0.113.9'+++def test_fromservice_host_resolves_to_sibling_name(project, owner):+ from app import db+ from app.models import Application+ for nm in ('api', 'web'):+ db.session.add(Application(name=nm, app_type='docker', user_id=owner.id,+ project_id=project.id, status='stopped'))+ db.session.commit()+ web = Application.query.filter_by(project_id=project.id, name='web').first()+ value, err = EnvReferenceResolver.resolve(+ web, {'kind': 'service', 'service': 'api', 'property': 'host'})+ assert err is None+ assert value == 'api' # resolves by container name on the shared network+++# -- apply ------------------------------------------------------------------++def test_apply_binds_fromserver(monkeypatch, project, owner):+ from app.models import Application+ from app.services.env_service import EnvService+ from app.services.site_domain_service import SiteDomainService+ monkeypatch.setattr(SiteDomainService, 'server_ip', classmethod(lambda cls: '198.51.100.4'))++ n = ManifestSpecService.normalize(FROMSERVER_MANIFEST)+ result = ManifestApplyService.apply(project, n, user_id=owner.id)+ assert result['success'] is True, result++ bridge = Application.query.filter_by(project_id=project.id, name='bridge').first()+ env = EnvService.get_effective_env(bridge.id)+ assert env.get('ADVERTISE_IP') == '198.51.100.4'++ plan2 = ManifestApplyService.plan(project, n)+ assert plan2['step_count'] == 0, plan2['summary']+++def test_fromserver_no_ip_blocks(monkeypatch, project):+ from app.services.site_domain_service import SiteDomainService+ monkeypatch.setattr(SiteDomainService, 'server_ip', classmethod(lambda cls: None))+ n = ManifestSpecService.normalize(FROMSERVER_MANIFEST)+ plan = ManifestApplyService.plan(project, n)+ assert 'fromserver_no_ip' in {b['kind'] for b in plan['blockers']}+++# -- magic var --------------------------------------------------------------++def test_server_public_ip_magic_var():+ substituted, generated = TemplateService.resolve_magic_variables(+ 'ip=${SERVER_PUBLIC_IP}', {'server_public_ip': '192.0.2.55'})+ assert substituted == 'ip=192.0.2.55'+ assert generated['SERVER_PUBLIC_IP'] == '192.0.2.55'+++# -- shared network ---------------------------------------------------------++def test_create_docker_app_attaches_shared_network(tmp_path, monkeypatch):+ monkeypatch.setattr(DockerService, 'ensure_network',+ staticmethod(lambda name, driver='bridge': {'success': True}))+ res = DockerService.create_docker_app(+ str(tmp_path), 'web', 'img:latest', networks=['serverkit'])+ assert res['success'], res+ with open(res['compose_file']) as fh:+ compose = yaml.safe_load(fh)+ assert compose['services']['web']['networks'] == ['serverkit']+ assert compose['networks'] == {'serverkit': {'external': True}}diff --git a/docs/SERVERKIT_YAML.md b/docs/SERVERKIT_YAML.mdindex f19302da..fdab7818 100644--- a/docs/SERVERKIT_YAML.md+++ b/docs/SERVERKIT_YAML.md@@ -166,8 +166,25 @@ Each entry has a `key` plus **exactly one** value source:| `value: <literal>` | A non-secret literal. Secrets never live in the manifest. || `fromSecret: <name>` | Reference to a vault secret by name. Resolved at injection time; the value never lands in the env row and masking stays intact. A missing secret is a **plan-time** error. || `fromService: { name, property }` | Reference to a sibling service's connection property. For db services: `connectionString`, `host`, `port`, `database`, `username`, `password`. For app services: `host`, `port`, `url`. |+| `fromServer: { property }` | The service's own advertised identity — `publicIp` or `hostname`. `publicIp` is what a NAT'd media/WebRTC bridge must advertise to clients. Resolves against the target server (or the panel host); a missing IP is a plan-time blocker (`fromserver_no_ip`). || `generate: true` | On first apply ServerKit generates a random value, stores it as a vault secret, and rewires the entry as a `fromSecret` reference — a committable manifest with zero secrets in git. |+## Addressing++- **In a unit** — containers reach each other by their short name on the unit's+ private (compose default) network.+- **Across apps** — a manifest app resolves a sibling app by its service name+ (`fromService: { name, property: host }`). ServerKit attaches manifest-generated+ projects to a shared external `serverkit` docker network (created idempotently+ via `DockerService.ensure_network`), so that name resolves at runtime.+- **Public IP** — `fromServer: { property: publicIp }` (or the+ `${SERVER_PUBLIC_IP}` template magic var) binds the host's advertised address —+ the WebRTC/NAT need.++**Static egress IPs are out of scope.** Advertising an inbound IP is container+config; pinning the *source* IP of outbound traffic is network infrastructure+(secondary IPs, SNAT) that lives below the manifest.+## Disks & backups```yamldiff --git a/docs/serverkit-yaml.schema.json b/docs/serverkit-yaml.schema.jsonindex 3eabf150..eebf0f4a 100644--- a/docs/serverkit-yaml.schema.json+++ b/docs/serverkit-yaml.schema.json@@ -271,6 +271,15 @@}}},+ "fromServer": {+ "description": "Reference to the service's own target server identity (appliance tier). publicIp is what a NAT'd media/WebRTC bridge must advertise to clients.",+ "type": "object",+ "required": ["property"],+ "additionalProperties": false,+ "properties": {+ "property": { "type": "string", "enum": ["publicIp", "hostname"] }+ }+ },"generate": {"description": "When true, ServerKit generates a random value on first apply, stores it as a vault secret, and rewires this entry as a fromSecret reference.","type": "boolean"