Record a sky camera around the clock, keep the footage on your own disk, and find what flew past.
TypeScript astronomycloudflare-workerscomputer-visionmcpmeteor-detectionobject-trackingrtspself-hostedsky-camerayolo
{
"nbformat": 4,
"nbformat_minor": 5,
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU",
"colab": {
"provenance": []
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Train a detector on your Workshop data\n",
"\n",
"This notebook trains a YOLO detector on images you reviewed in openskycam and publishes it to your model registry, where workflows can use it.\n",
"\n",
"1. **Freeze a dataset version** of each bucket you want to train on: open the bucket in Workshop and choose **Freeze version**, or call the API ([Buckets and datasets](https://docs.openskycam.org/workshop/datasets/)). Turn on rejected images if you want them to teach the model what *not* to flag.\n",
"2. **Create two keys** under **API access** on your website: one that can read footage for the cameras in those versions (Agent or Footage), and one with **Publish trained models**.\n",
"3. **Run it on a GPU.** In Colab, choose *Runtime → Change runtime type → GPU*. Training at 1920 px needs a large GPU; 1280 or 640 runs on smaller ones.\n",
"\n",
"To try a published model before training your own, see the [night detector](https://github.com/Deibler/openskycam/tree/main/models/night-detector)."
],
"id": "cell-0"
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"# Your deployment\n",
"API = \"https://api.example.com\" # your API_HOST\n",
"VERSIONS = [\"planes-2026-10-25\"] # dataset version IDs to train on; their class names are merged\n",
"\n",
"# The model\n",
"BASE_MODEL = \"yolo26s.pt\" # Ultralytics weights to start from, or a path to your own .pt\n",
"IMGSZ = 1280 # input size: a multiple of 32, at most 2048. Workflows run the model at this size.\n",
"EPOCHS = 50\n",
"REJECTED_AS_EMPTY = True # train rejected images as empty sky, so the model learns what not to flag\n",
"\n",
"# Publishing\n",
"FAMILY = \"my-detector\" # lowercase letters, digits, dots, underscores or hyphens\n",
"VERSION = \"v1\"\n",
"NAME = \"My detector\"\n",
"PUBLISH = True\n",
"\n",
"LIMIT = None # a number of images per split, for a quick end-to-end test; never publishes"
],
"id": "cell-1"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup\n",
"\n",
"Keys come from Colab Secrets or environment variables named `OPENSKYCAM_READ_KEY` and `OPENSKYCAM_PUBLISH_KEY`, so they never appear in the notebook. Ultralytics is pinned to the version the processor runs, so the processor can load what you train."
],
"id": "cell-2"
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"import hashlib, importlib.util, json, os, shutil, subprocess, sys, time, urllib.error, urllib.request\n",
"from pathlib import Path\n",
"\n",
"ULTRALYTICS = \"8.4.58\" # the processor's version (processor/requirements.txt)\n",
"if importlib.util.find_spec(\"ultralytics\") is None:\n",
" subprocess.run([sys.executable, \"-m\", \"pip\", \"install\", \"-q\", f\"ultralytics=={ULTRALYTICS}\"], check=True)\n",
"if importlib.util.find_spec(\"skyvision_processor\") is None: # only for publishing; it needs no other packages\n",
" subprocess.run([sys.executable, \"-m\", \"pip\", \"install\", \"-q\", \"--no-deps\",\n",
" \"git+https://github.com/Deibler/openskycam#subdirectory=processor\"], check=True)\n",
"import cv2, numpy as np, torch, ultralytics\n",
"from ultralytics import YOLO\n",
"\n",
"try:\n",
" from google.colab import userdata\n",
"except ImportError:\n",
" userdata = None\n",
"\n",
"\n",
"def secret(name):\n",
" value = \"\"\n",
" if userdata:\n",
" try:\n",
" value = userdata.get(name) or \"\"\n",
" except Exception: # not set, or this notebook has no access to it\n",
" pass\n",
" return (value or os.environ.get(name, \"\")).strip()\n",
"\n",
"\n",
"READ_KEY, PUBLISH_KEY = secret(\"OPENSKYCAM_READ_KEY\"), secret(\"OPENSKYCAM_PUBLISH_KEY\")\n",
"assert READ_KEY, \"Set OPENSKYCAM_READ_KEY: a key that can read the cameras in your dataset versions\"\n",
"assert not PUBLISH or LIMIT or PUBLISH_KEY, \"Set OPENSKYCAM_PUBLISH_KEY (Publish trained models), or PUBLISH = False\"\n",
"assert IMGSZ % 32 == 0 and 32 <= IMGSZ <= 2048, \"IMGSZ must be a multiple of 32, at most 2048\"\n",
"if ultralytics.__version__ != ULTRALYTICS:\n",
" print(f\"Ultralytics {ultralytics.__version__} is installed; the processor runs {ULTRALYTICS}. Test the model before relying on it.\")\n",
"DEVICE = 0 if torch.cuda.is_available() else \"mps\" if torch.backends.mps.is_available() else \"cpu\"\n",
"# Absolute: Ultralytics puts a relative project folder under its own runs directory instead.\n",
"WORK = (Path(\"openskycam-training\") / f\"{FAMILY}-{VERSION}\").resolve()\n",
"WORK.mkdir(parents=True, exist_ok=True)\n",
"print(\"training on\", DEVICE, \"in\", WORK)\n",
"\n",
"\n",
"def get(path, attempts=4):\n",
" for attempt in range(attempts):\n",
" try:\n",
" request = urllib.request.Request(API + path, headers={\"Authorization\": f\"Bearer {READ_KEY}\",\n",
" \"User-Agent\": \"openskycam-training-notebook\"})\n",
" with urllib.request.urlopen(request, timeout=120) as response:\n",
" return response.read()\n",
" except urllib.error.HTTPError as error:\n",
" if error.code < 500 or attempt == attempts - 1:\n",
" raise RuntimeError(f\"GET {path}: HTTP {error.code} {error.read()[:200]!r}\") from None\n",
" except OSError:\n",
" if attempt == attempts - 1:\n",
" raise\n",
" time.sleep(2 ** attempt)"
],
"id": "cell-3"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Download the dataset\n",
"\n",
"Each version's manifest lists its images, boxes, reviews and a split. A split covers whole camera-days, so near-identical frames of one night never land on both sides. Images are checked against their SHA-256 and shrunk to `IMGSZ` on the long side with the same resize the processor uses, so the model trains on what it will see."
],
"id": "cell-4"
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"from concurrent.futures import ThreadPoolExecutor\n",
"\n",
"manifests = [json.loads(get(f\"/v1/workshop/versions/{v}\"))[\"manifest\"] for v in VERSIONS]\n",
"CLASSES = sorted({name for m in manifests for name in m[\"class_names\"]})\n",
"DATA = WORK / f\"data-{IMGSZ}\"\n",
"items = []\n",
"for manifest in manifests:\n",
" for item in manifest[\"items\"]:\n",
" if item[\"review\"] == \"rejected\" and not REJECTED_AS_EMPTY:\n",
" continue\n",
" boxes = [] if item[\"review\"] == \"rejected\" else [\n",
" (CLASSES.index(a[\"class_name\"]), a[\"box\"]) for a in item[\"annotations\"] if \"box\" in a]\n",
" items.append({**item, \"boxes\": boxes})\n",
"if LIMIT:\n",
" items = [i for split in (\"train\", \"validation\", \"test\") for i in [i for i in items if i[\"split\"] == split][:LIMIT]]\n",
"\n",
"\n",
"def shrink(image):\n",
" # The processor's own resize: INTER_AREA down to IMGSZ on the long side, never up.\n",
" scale = IMGSZ / max(image.shape[:2])\n",
" if scale >= 1:\n",
" return image\n",
" return cv2.resize(image, (max(1, int(image.shape[1] * scale)), max(1, int(image.shape[0] * scale))),\n",
" interpolation=cv2.INTER_AREA)\n",
"\n",
"\n",
"def prepare(item):\n",
" folder = \"val\" if item[\"split\"] == \"validation\" else item[\"split\"]\n",
" image = DATA / \"images\" / folder / f\"{item['sha256']}.jpg\"\n",
" if not image.exists():\n",
" data = get(item[\"media_path\"])\n",
" assert hashlib.sha256(data).hexdigest() == item[\"sha256\"], f\"{item['item_id']}: checksum mismatch\"\n",
" frame = cv2.imdecode(np.frombuffer(data, np.uint8), cv2.IMREAD_COLOR)\n",
" part = image.with_suffix(\".part.jpg\")\n",
" cv2.imwrite(str(part), shrink(frame), [cv2.IMWRITE_JPEG_QUALITY, 95])\n",
" part.replace(image)\n",
" lines = []\n",
" for k, (x1, y1, x2, y2) in item[\"boxes\"]: # YOLO: class, centre x, centre y, width, height, as fractions\n",
" x1, y1, x2, y2 = (min(max(v, 0.0), 1.0) for v in (x1, y1, x2, y2))\n",
" lines.append(f\"{k} {(x1 + x2) / 2:.6f} {(y1 + y2) / 2:.6f} {x2 - x1:.6f} {y2 - y1:.6f}\")\n",
" (DATA / \"labels\" / folder / f\"{item['sha256']}.txt\").write_text(\"\\n\".join(lines))\n",
"\n",
"\n",
"for kind in (\"images\", \"labels\"):\n",
" for folder in (\"train\", \"val\", \"test\"):\n",
" (DATA / kind / folder).mkdir(parents=True, exist_ok=True)\n",
"with ThreadPoolExecutor(8) as pool:\n",
" list(pool.map(prepare, items))\n",
"(DATA / \"data.yaml\").write_text(json.dumps({\"path\": str(DATA.resolve()), \"train\": \"images/train\", \"val\": \"images/val\",\n",
" \"test\": \"images/test\", \"names\": dict(enumerate(CLASSES))}))\n",
"counts = {s: sum(i[\"split\"] == s for i in items) for s in (\"train\", \"validation\", \"test\")}\n",
"print(\"classes:\", CLASSES)\n",
"print(\"images:\", counts)\n",
"assert counts[\"train\"] and counts[\"validation\"], \"Training needs train and validation images; freeze a version with more days\""
],
"id": "cell-5"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Train\n",
"\n",
"Training resumes from the last finished epoch if the notebook was interrupted."
],
"id": "cell-6"
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"RUN = WORK / \"train\"\n",
"last, best = RUN / \"weights\" / \"last.pt\", RUN / \"weights\" / \"best.pt\"\n",
"if last.exists() and not best.exists():\n",
" YOLO(str(last)).train(resume=True)\n",
"elif not best.exists():\n",
" YOLO(BASE_MODEL).train(data=str(DATA / \"data.yaml\"), imgsz=IMGSZ, epochs=1 if LIMIT else EPOCHS, patience=15,\n",
" batch=-1 if DEVICE == 0 else 2, device=DEVICE, seed=0, fliplr=0.5, flipud=0.0,\n",
" project=str(WORK), name=\"train\", exist_ok=True, plots=not LIMIT, workers=0 if LIMIT else 8)\n",
"assert best.exists(), \"training did not produce best.pt\""
],
"id": "cell-7"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Score on the test split\n",
"\n",
"The test images were never used for training or for choosing the best epoch, so these numbers are the honest ones to compare models by."
],
"id": "cell-8"
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"model = YOLO(str(best))\n",
"evaluations = []\n",
"if counts[\"test\"]:\n",
" metrics = model.val(data=str(DATA / \"data.yaml\"), split=\"test\", imgsz=IMGSZ, device=DEVICE, plots=False).results_dict\n",
" evaluations.append({\"label\": \"Test split\", \"split\": \"test\", \"input_size\": IMGSZ,\n",
" \"dataset\": \", \".join(VERSIONS) + f\": {counts['test']} images\",\n",
" \"metrics\": {k: round(float(v), 5) for k, v in metrics.items()}, \"independently_reproduced\": False})\n",
" print(json.dumps(evaluations[-1][\"metrics\"], indent=1))\n",
"else:\n",
" print(\"This dataset has no test images; freeze a version that covers more days to get a test score.\")"
],
"id": "cell-9"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Publish\n",
"\n",
"Publishing uploads the checkpoint in checksum-verified chunks and creates an immutable model version. Afterwards, choose `FAMILY/VERSION` in a workflow's model step and start with a test run."
],
"id": "cell-10"
},
{
"cell_type": "code",
"metadata": {},
"execution_count": null,
"outputs": [],
"source": [
"from skyvision_processor.service.modelInferenceService import publish\n",
"from skyvision_processor.utils.httpSharedUtil import Client\n",
"\n",
"card = {\n",
" \"family\": FAMILY, \"version\": VERSION, \"name\": NAME,\n",
" \"description\": f\"{BASE_MODEL} fine-tuned at {IMGSZ} px on dataset versions {', '.join(VERSIONS)}.\",\n",
" \"task\": \"object-detection\", \"adapter\": \"ultralytics\",\n",
" \"classes\": [{\"id\": int(k), \"name\": v} for k, v in sorted(model.names.items())],\n",
" \"input\": {\"kind\": \"image\", \"channels\": 3, \"color_space\": \"RGB\", \"training_size\": IMGSZ,\n",
" \"preprocessing\": \"cv2.INTER_AREA to the long side, as the processor does, then Ultralytics letterbox\"},\n",
" \"evaluations\": evaluations,\n",
" \"caveats\": [f\"Trained with Ultralytics {ultralytics.__version__}.\", \"Metrics come from this notebook, not a Workshop run.\"],\n",
" \"dependencies\": [],\n",
"}\n",
"(WORK / \"model.json\").write_text(json.dumps(card, indent=2))\n",
"if LIMIT or not PUBLISH:\n",
" print(\"Not published:\", \"a LIMIT test run\" if LIMIT else \"PUBLISH = False\", f\"\\nWeights: {best}\\nCard: {WORK / 'model.json'}\")\n",
"else:\n",
" published = publish(Client(API, PUBLISH_KEY), card, [best])\n",
" print(\"Published\", f\"{published['family']}/{published['version']}\")"
],
"id": "cell-11"
}
]
}