ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.
Python astracloudflarecontrol-panelcyberpaneldnsfablehosting-control-panelhosting-panelkimi-k3server-managementserverkitubuntu-web-panelwebserver
feat(manifest): BYO image + host requirements (plan 35 ph4)
Juan Denis619b2215da2a
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6bbda9fc..175d9cad 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -48,6 +48,14 @@ awaiting a stable release:named volumes (`{unit}-{container}-{disk}`) let two containers mount the samepath without colliding. `dependsOn` cycles and buildpack-key mixing areplan-time errors.+- **Appliance tier — BYO image + host requirements** — a service (or unit+ container) can declare a ready-made `image:` with an optional private+ `registry:` (an unknown/uncredentialed registry is a plan-time blocker; ECR is+ accepted via its key-pair exchange), and `hostRequirements:`+ (`privileged`/`capAdd`/`sysctls`/`devices`/`kernelModules`). Host requirements+ are listed in plain words in the plan and written to a+ `manifest.host_requirements` audit line on apply — never applied silently;+ `kernelModules` are an advisory `/proc/modules` check.- **New Service wizard clarity** — each source card now shows a short explainerstrip when selected (what it does, what to have ready, the next steps), and a"Docs" link to the matching serverkit.ai guide (hidden under White Label). Thediff --git a/backend/app/services/docker_service.py b/backend/app/services/docker_service.pyindex 2743451e..65fa2f85 100644--- a/backend/app/services/docker_service.py+++ b/backend/app/services/docker_service.py@@ -1220,7 +1220,8 @@ class DockerService:@staticmethoddef create_docker_app(app_path, app_name, image, ports=None, volumes=None, env=None,- named_volumes=None, cpu_limit=None, memory_limit=None):+ named_volumes=None, cpu_limit=None, memory_limit=None,+ host_requirements=None):"""Create a Docker-based application with docker-compose.``named_volumes`` is a list of managed Docker volume names to declare as@@ -1230,6 +1231,10 @@ class DockerService:``cpu_limit`` / ``memory_limit`` (task #23) emit ``cpus`` / ``mem_limit``on the app's service block so Docker enforces the caps.++ ``host_requirements`` (appliance tier, plan 35) emits+ ``privileged`` / ``cap_add`` / ``sysctls`` / ``devices`` — exactly the+ fields catalog templates already pass through."""try:os.makedirs(app_path, exist_ok=True)@@ -1266,6 +1271,18 @@ class DockerService:# instead of a fresh anonymous one.compose['volumes'] = {name: {} for name in named_volumes}+ if host_requirements:+ hr = host_requirements+ svc = compose['services'][app_name]+ if hr.get('privileged'):+ svc['privileged'] = True+ if hr.get('cap_add'):+ svc['cap_add'] = list(hr['cap_add'])+ if hr.get('sysctls'):+ svc['sysctls'] = dict(hr['sysctls'])+ if hr.get('devices'):+ svc['devices'] = list(hr['devices'])+compose_path = os.path.join(app_path, 'docker-compose.yml')with open(compose_path, 'w') as f:yaml.dump(compose, f, default_flow_style=False)diff --git a/backend/app/services/manifest_apply_service.py b/backend/app/services/manifest_apply_service.pyindex 3d97c313..97a16f84 100644--- a/backend/app/services/manifest_apply_service.py+++ b/backend/app/services/manifest_apply_service.py@@ -298,6 +298,12 @@ class ManifestApplyService:('app_type', 'port', 'healthcheck_path', 'runtime','build_command', 'start_command', 'cpu', 'memory')}payload['name'] = name+ # BYO image (plan 35): stamp the declared image + private registry+ if resolved.get('image'):+ payload['image'] = resolved['image']+ if resolved.get('registry'):+ reg = cls._resolve_registry(resolved['registry'])+ payload['registry_id'] = reg.id if reg else None# fleet targeting (#21): route the app to a server when declaredserver_ref = resolved.get('server')if server_ref:@@ -541,6 +547,32 @@ class ManifestApplyService:'service': name, 'kind': 'firewall_none','message': f'no manageable firewall detected — public ports for {name} 'f'will be published but not firewall-managed.'})++ # BYO image (plan 35): a private registry must be known + credentialed.+ for ref in cls._image_registry_refs(resolved):+ reg_name = ref.get('registry')+ if not reg_name:+ continue # anonymous pull is fine+ reg = cls._resolve_registry(reg_name)+ if reg is None or not cls._registry_has_credential(reg):+ blockers.append({+ 'kind': 'registry_credential', 'service': name, 'registry': reg_name,+ 'message': f'service {name} pulls `{ref["image"]}` from registry '+ f'`{reg_name}`, which is '+ + ('unknown' if reg is None else 'missing a stored credential')+ + f' — add the registry credential, then re-apply.'})++ # Host requirements (plan 35): listed in plain words, never silent.+ for hr in resolved.get('host_requirements') or []:+ for phrase in cls._hostreq_phrases(hr):+ issues.append({'service': name, 'kind': 'host_requirement',+ 'message': f'{name} requests {phrase}'})+ for mod in hr.get('kernel_modules') or []:+ if not cls._kernel_module_present(mod):+ issues.append({+ 'service': name, 'kind': 'kernel_module',+ 'message': f'{name} needs kernel module `{mod}` — not confirmed '+ f'loaded on the host (advisory; verify before relying on it).'})return blockers, issues# -- blocker seams (patched in tests) -----------------------------------@@ -589,6 +621,60 @@ class ManifestApplyService:return (Server.query.filter_by(id=server_ref).first()or Server.query.filter_by(name=server_ref).first())+ @staticmethod+ def _image_registry_refs(resolved: Dict[str, Any]) -> List[Dict[str, Any]]:+ """Every (image, registry) pair a service pulls — top-level + containers."""+ refs: List[Dict[str, Any]] = []+ if resolved.get('image'):+ refs.append({'image': resolved['image'], 'registry': resolved.get('registry')})+ for c in resolved.get('containers') or []:+ if c.get('image'):+ refs.append({'image': c['image'], 'registry': c.get('registry')})+ return refs++ @staticmethod+ def _resolve_registry(name: Optional[str]):+ if not name:+ return None+ try:+ from app.models.container_registry import ContainerRegistry+ except Exception:+ return None+ return (ContainerRegistry.query.filter_by(id=name).first()+ if str(name).isdigit() else None) \+ or ContainerRegistry.query.filter_by(name=name).first()++ @staticmethod+ def _registry_has_credential(reg) -> bool:+ # ECR authenticates via a key-pair exchange, so no stored secret needed.+ if getattr(reg, 'provider', None) == 'ecr':+ return True+ return bool(getattr(reg, 'secret_encrypted', None))++ @staticmethod+ def _hostreq_phrases(hr: Dict[str, Any]) -> List[str]:+ phrases: List[str] = []+ if hr.get('privileged'):+ phrases.append('a PRIVILEGED container')+ for cap in hr.get('cap_add') or []:+ phrases.append(f'capability {cap}')+ for k, v in (hr.get('sysctls') or {}).items():+ phrases.append(f'sysctl {k}={v}')+ for dev in hr.get('devices') or []:+ phrases.append(f'device {dev}')+ return phrases++ @staticmethod+ def _kernel_module_present(module: str) -> bool:+ """Advisory /proc/modules check. Unverifiable (non-Linux / unreadable)+ returns False so the plan warns rather than silently passing."""+ try:+ with open('/proc/modules') as fh:+ loaded = {line.split()[0] for line in fh if line.strip()}+ return module in loaded+ except Exception:+ return False+@staticmethoddef _ports_differ(current: List[Dict[str, Any]], desired: List[Dict[str, Any]]) -> bool:def key(p):@@ -642,6 +728,18 @@ class ManifestApplyService:cls._snapshot_existing(project, normalized, tag='after')cls._finish_job(job, failed, results)++ # Host requirements never apply silently (plan 35): audit every one.+ hr_issues = [i for i in plan.get('issues', []) if i.get('kind') == 'host_requirement']+ if hr_issues:+ try:+ from app.services.audit_service import AuditService+ AuditService.log('manifest.host_requirements', user_id=user_id,+ target_type='project', target_id=project.id,+ details={'requirements': [i['message'] for i in hr_issues]})+ except Exception:+ pass+if manifest_row is not None:manifest_row.status = STATUS_ERROR if failed else STATUS_APPLIEDif not failed:@@ -717,6 +815,8 @@ class ManifestApplyService:healthcheck_path=payload.get('healthcheck_path'),cpu_limit=payload.get('cpu'),memory_limit=payload.get('memory'),+ docker_image=payload.get('image'),+ registry_id=payload.get('registry_id'),buildpack_overrides=json.dumps(overrides) if overrides else None,source='manifest',user_id=user_id,@@ -743,6 +843,8 @@ class ManifestApplyService:app.cpu_limit = changes['cpu']if 'memory' in changes:app.memory_limit = changes['memory']+ if 'image' in changes:+ app.docker_image = changes['image']if 'build_command' in changes or 'start_command' in changes:overrides = {}if app.buildpack_overrides:@@ -944,6 +1046,8 @@ class ManifestApplyService:changes['cpu'] = resolved['cpu']if resolved.get('memory') and app.memory_limit != resolved['memory']:changes['memory'] = resolved['memory']+ if resolved.get('image') and app.docker_image != resolved['image']:+ changes['image'] = resolved['image']overrides = {}if app.buildpack_overrides:try:diff --git a/backend/app/services/manifest_spec_service.py b/backend/app/services/manifest_spec_service.pyindex 1625b6fe..2f9facf0 100644--- a/backend/app/services/manifest_spec_service.py+++ b/backend/app/services/manifest_spec_service.py@@ -107,6 +107,9 @@ MANIFEST_SCHEMA: Dict[str, Any] = {"startCommand": {"type": "string"},"port": {"type": "integer", "minimum": 1, "maximum": 65535},"ports": {"type": "array", "items": {"$ref": "#/definitions/portDecl"}},+ "image": {"type": "string"},+ "registry": {"type": "string"},+ "hostRequirements": {"$ref": "#/definitions/hostRequirements"},"bootstrap": {"$ref": "#/definitions/bootstrap"},"containers": {"type": "object",diff --git a/backend/tests/test_manifest_image_hostreq.py b/backend/tests/test_manifest_image_hostreq.pynew file mode 100644index 00000000..325dde2a--- /dev/null+++ b/backend/tests/test_manifest_image_hostreq.py@@ -0,0 +1,166 @@+"""Proving tests for Phase 4 of the Appliance tier (plan 35):+BYO `image`/`registry` + `hostRequirements` (privileged/capAdd/sysctls/devices/+kernelModules)."""++import pytest+import yaml++import app.models.application_manifest # noqa: F401+from app.services.manifest_spec_service import ManifestSpecService+from app.services.manifest_apply_service import ManifestApplyService+from app.services.unit_compose_service import UnitComposeService+from app.services.docker_service import DockerService+++IMAGE_MANIFEST = {+ 'version': 1,+ 'services': [{+ 'name': 'vpn', 'type': 'docker',+ 'image': 'ghcr.io/acme/vpn:1.2',+ 'registry': 'acme-ghcr',+ 'hostRequirements': {+ 'capAdd': ['NET_ADMIN'],+ 'sysctls': {'net.ipv4.ip_forward': '1'},+ 'devices': ['/dev/net/tun'],+ 'kernelModules': ['wireguard'],+ },+ }],+}+++def project(app):+ from app import db+ from app.models import Project, Environment+ from app.services.workspace_service import WorkspaceService+ ws = WorkspaceService.ensure_default_workspace()+ proj = Project(workspace_id=ws.id, name='VPN', slug='vpn')+ db.session.add(proj)+ db.session.commit()+ env = Environment(project_id=proj.id, name='Production', slug='production', is_default=True)+ db.session.add(env)+ db.session.commit()+ return proj+++def owner(app):+ from app import db+ from app.models import User+ user = User.query.filter_by(username='testadmin').first()+ if not user:+ user = User(username='testadmin', email='[email protected]', role='admin')+ if hasattr(user, 'set_password'):+ user.set_password('admin')+ db.session.add(user)+ db.session.commit()+ return user+++def _registry(name, provider='generic', secret='enc'):+ from app import db+ from app.models.container_registry import ContainerRegistry+ reg = ContainerRegistry(name=name, provider=provider, secret_encrypted=secret)+ db.session.add(reg)+ db.session.commit()+ return reg+++# -- normalizer -------------------------------------------------------------++def test_image_and_hostreq_normalize():+ n = ManifestSpecService.normalize(IMAGE_MANIFEST)+ svc = n['services'][0]+ assert svc['image'] == 'ghcr.io/acme/vpn:1.2'+ assert svc['registry'] == 'acme-ghcr'+ hr = svc['host_requirements']+ assert hr['cap_add'] == ['NET_ADMIN']+ assert hr['sysctls'] == {'net.ipv4.ip_forward': '1'}+ assert hr['devices'] == ['/dev/net/tun']+ assert hr['kernel_modules'] == ['wireguard']+++# -- generators -------------------------------------------------------------++def test_create_docker_app_emits_host_requirements(tmp_path):+ res = DockerService.create_docker_app(+ str(tmp_path), 'vpn', 'img:latest',+ host_requirements={'privileged': True, 'cap_add': ['NET_ADMIN'],+ 'sysctls': {'net.ipv4.ip_forward': '1'},+ 'devices': ['/dev/net/tun']})+ assert res['success'], res+ with open(res['compose_file']) as fh:+ compose = yaml.safe_load(fh)+ svc = compose['services']['vpn']+ assert svc['privileged'] is True+ assert svc['cap_add'] == ['NET_ADMIN']+ assert svc['sysctls'] == {'net.ipv4.ip_forward': '1'}+ assert svc['devices'] == ['/dev/net/tun']+++def test_unit_compose_emits_host_requirements():+ containers = [{+ 'name': 'bridge', 'image': 'img', 'ports': [], 'disks': [], 'env_vars': [],+ 'bootstrap': None, 'health_check': None, 'depends_on': [],+ 'host_requirements': {'privileged': True, 'cap_add': ['NET_ADMIN'],+ 'sysctls': {'k': 'v'}, 'devices': ['/dev/net/tun'],+ 'kernel_modules': []},+ }]+ compose = UnitComposeService.render('u', containers)+ svc = compose['services']['bridge']+ assert svc['privileged'] is True+ assert svc['cap_add'] == ['NET_ADMIN']+ assert svc['devices'] == ['/dev/net/tun']+++# -- apply / stamping -------------------------------------------------------++def test_apply_stamps_image_and_registry(project, owner):+ from app.models import Application+ reg = _registry('acme-ghcr')+ n = ManifestSpecService.normalize(IMAGE_MANIFEST)+ result = ManifestApplyService.apply(project, n, user_id=owner.id)+ assert result['success'] is True, result+ app_row = Application.query.filter_by(project_id=project.id, name='vpn').first()+ assert app_row.docker_image == 'ghcr.io/acme/vpn:1.2'+ assert app_row.registry_id == reg.id+++# -- blockers ---------------------------------------------------------------++def test_unknown_registry_blocks(project):+ n = ManifestSpecService.normalize(IMAGE_MANIFEST) # 'acme-ghcr' not created+ plan = ManifestApplyService.plan(project, n)+ kinds = {b['kind'] for b in plan['blockers']}+ assert 'registry_credential' in kinds+++def test_registry_without_credential_blocks(project):+ _registry('acme-ghcr', secret=None)+ n = ManifestSpecService.normalize(IMAGE_MANIFEST)+ plan = ManifestApplyService.plan(project, n)+ assert 'registry_credential' in {b['kind'] for b in plan['blockers']}+++def test_ecr_registry_accepted_without_secret(project):+ _registry('acme-ghcr', provider='ecr', secret=None)+ n = ManifestSpecService.normalize(IMAGE_MANIFEST)+ plan = ManifestApplyService.plan(project, n)+ assert 'registry_credential' not in {b['kind'] for b in plan['blockers']}+++# -- advisory issues --------------------------------------------------------++def test_host_requirements_listed_in_plain_words(project, monkeypatch):+ monkeypatch.setattr(ManifestApplyService, '_kernel_module_present', lambda mod: False)+ _registry('acme-ghcr')+ n = ManifestSpecService.normalize(IMAGE_MANIFEST)+ plan = ManifestApplyService.plan(project, n)+ hr_msgs = [i['message'] for i in plan['issues'] if i.get('kind') == 'host_requirement']+ assert any('capability NET_ADMIN' in m for m in hr_msgs)+ assert any('sysctl net.ipv4.ip_forward=1' in m for m in hr_msgs)+ assert any('device /dev/net/tun' in m for m in hr_msgs)+ # kernel module advisory (not a blocker)+ km = [i['message'] for i in plan['issues'] if i.get('kind') == 'kernel_module']+ assert any('wireguard' in m for m in km)+ assert plan['blockers'] == []diff --git a/docs/SERVERKIT_YAML.md b/docs/SERVERKIT_YAML.mdindex e447f676..f19302da 100644--- a/docs/SERVERKIT_YAML.md+++ b/docs/SERVERKIT_YAML.md@@ -186,6 +186,34 @@ not the empty in-container path. `schedule` is one of`hourly` / `daily` / `weekly` / `monthly`; `retain` is the number of backups tokeep.+## BYO image & host requirements++`image:` is a first-class peer of the buildpack path — declare a ready-made+image instead of building one. `registry:` names a private registry to+authenticate against before pulling; an unknown or uncredentialed registry is a+plan-time **blocker** (`registry_credential`). ECR registries authenticate via a+key-pair exchange, so they need no stored secret.++`hostRequirements:` exposes the elevated compose fields real appliances need:++```yaml+services:+ - name: vpn+ type: docker+ image: ghcr.io/acme/vpn:1.2+ registry: acme-ghcr+ hostRequirements:+ capAdd: [NET_ADMIN]+ sysctls: { net.ipv4.ip_forward: "1" }+ devices: [/dev/net/tun]+ kernelModules: [wireguard] # advisory /proc/modules check only+```++Every host requirement is **listed in plain words in the plan** and written to a+`manifest.host_requirements` audit line on apply — `privileged` is a big hammer+and it never applies silently. `kernelModules` are advisory: an unconfirmed+module is a warning, never a silent pass, and is unverifiable off-Linux.+## Multi-container unitsSome services are not one image — a web frontend, a signaling process, a mediadiff --git a/docs/serverkit-yaml.schema.json b/docs/serverkit-yaml.schema.jsonindex 27cb4496..3eabf150 100644--- a/docs/serverkit-yaml.schema.json+++ b/docs/serverkit-yaml.schema.json@@ -78,6 +78,18 @@"type": "array","items": { "$ref": "#/definitions/portDecl" }},+ "image": {+ "description": "Container image reference for a BYO-image app (appliance tier) — a first-class peer of the buildpack path.",+ "type": "string"+ },+ "registry": {+ "description": "Named private registry to authenticate against before pulling `image`. A registry that is unknown or has no stored credential is a plan-time blocker.",+ "type": "string"+ },+ "hostRequirements": {+ "description": "Elevated host requirements (appliance tier) — privileged/capAdd/sysctls/devices/kernelModules. Always listed in plain words in the plan; never applied silently.",+ "$ref": "#/definitions/hostRequirements"+ },"bootstrap": {"description": "First-boot bootstrap (appliance tier). Runs once via `docker compose run --rm` after volumes exist and before the first up — for appliances that generate a config tree once.","$ref": "#/definitions/bootstrap"