Image Pipeline Playground
Merge branch 'main' of github.com:mrajaeim/image-pipes
Mohammad Rajaei4713d4e96d7c
diff --cc README.mdindex 4bae3aa,8360832..414fc4e--- a/README.md+++ b/README.md@@@ -1,90 -1,282 +1,307 @@@# Image Pipes- [](LICENSE)- [](https://github.com/mrajaeim/image-pipes)-- **A visual IDE for computer vision pipelines.**-- Build, debug, and export OpenCV workflows without writing preprocessing code by hand.-- ## Why Image Pipes?-- Computer vision preprocessing is usually hidden inside long Python scripts. Changing the pipeline often means editing code, re-running everything, and debugging intermediate images manually.-- Image Pipes makes that process visual: compose an OpenCV DAG in the browser, execute it live, inspect every step, and export runnable Python when you’re ready.-- It is useful for:-- - experimenting with preprocessing techniques- - debugging computer vision pipelines- - teaching OpenCV concepts- - rapidly prototyping image workflows- - exporting pipelines into production-ready Python code-- ## Features-- - Drag-and-drop OpenCV nodes (color, filters, morphology, contours, histograms, clustering, and more)- - **Albumentations augment** nodes (catalog-driven) with multi-target ports: image, optional mask, bboxes, keypoints- - Live WebSocket execution with per-node previews, timings, and an execution log- - Result caching and seeded stochastic nodes for reproducible experiments- - **Run to selected** — execute only ancestors of a node while tuning parameters- - **Starters** palette group for source nodes (Load Images, Blank Image, Annotations, …)- - **Export** / **Load** workflow JSON; **Templates** gallery includes Blur & Canny, Hist Equalize, Morphology, Contours, and Albu Augment- - **Export Python** to a standalone script in the Monaco panel- - Catalog-driven UI — the frontend does not hard-code nodes; `/api/nodes` metadata builds the palette automatically+ A visual workflow editor for computer vision pipelines powered by **OpenCV** and **Albumentations**.++ Instead of writing hundreds of lines of preprocessing code, build your image processing pipeline visually, inspect every intermediate transformation in real time, and export clean, production-ready Python code.++ ---++ ## 📚 Table of Contents++ - [Overview](#overview)+ - [Demo](#demo)+ - [Why Image Pipes?](#why-image-pipes)+ - [Key Features](#key-features)+ - [How It Works](#how-it-works)+ - [Architecture](#architecture)+ - [Tech Stack](#tech-stack)+ - [Quick Start](#quick-start)+ - [Prerequisites](#prerequisites)+ - [Backend Setup](#backend-setup)+ - [Frontend Setup](#frontend-setup)+ - [Docker Setup](#docker-setup)+ - [Quality Checks](#quality-checks)+ - [Adding a New Node](#adding-a-new-node)+ - [Exporting Code](#exporting-code)+ - [Roadmap](#roadmap)+ - [Contributing](#contributing)+ - [FAQ](#faq)+ - [License](#license)++ ---++ # Overview++ Image Pipes is a visual workflow editor for computer vision powered by **OpenCV** and **Albumentations**.++ Instead of writing hundreds of lines of boilerplate preprocessing scripts, you compose your workflow visually, inspect intermediate frame transformations in real time, and export deterministic, production-ready Python code.++ ```text+ ┌──────────────┐ ┌──────────────┐ ┌───────────────┐ ┌──────────────┐+ │ Load Image │ ──► │ Grayscale │ ──► │ Gaussian Blur │ ──► │ Canny │+ └──────────────┘ └──────────────┘ └───────────────┘ └──────────────┘+ │+ ▼+ ┌──────────────┐+ │ Find Contours│+ └──────────────┘+ ```- ## Architecture highlights+ ---- Capabilities that matter beyond “React + OpenCV”:+ # Demo- - DAG-based execution engine with topological validation- - Metadata-driven UI (dynamic node registration)- - Live WebSocket progress, previews, and per-node duration- - Deterministic execution with seeded randomness- - Result caching across runs- - Code generation (`emit_python` → standalone script)- - Full-stack architecture: React Flow editor + FastAPI + OpenCV+ > [!NOTE]+ > **Demo Coming Soon**- ## Stack+ A short GIF/video will showcase:-- Drag-and-drop workflow editing-- Live parameter tuning-- Real-time execution previews-- Python code generation-- Interactive debugging+- **Frontend:** React 19, Vite, TypeScript, MUI, React Flow, Zustand, TanStack Query, React Hook Form, Zod, Monaco+- **Backend:** FastAPI, OpenCV, NumPy, Pillow, Albumentations 2.0.8 (MIT), Pydantic v2, WebSockets+- **Desktop:** Electron + PyInstaller backend sidecar (optional installer)+- **Tooling:** `uv`, `npm`, optional Docker- ## Demo+ ---- > **Tip:** A short GIF of canvas editing → Run → previews → Export Python converts more visitors than any feature list. Record from the local app and drop files under `docs/demo/` (e.g. `edit.gif`, `run.gif`, `export.gif`), then link them here.+ # Why Image Pipes?-Traditional computer vision experimentation is repetitive:+## Desktop app (recommended for end users)++Image Pipes can run as a single **Electron** desktop app. The window starts the Python backend for you and opens the UI—no separate terminals.++### Build an installer++Requires [Node.js](https://nodejs.org/) and [uv](https://docs.astral.sh/uv/):++```bash+npm run install:all+npm run build:desktop+```++Installers are written to `desktop/release/` (NSIS on Windows, DMG on macOS, AppImage on Linux). See [`desktop/README.md`](desktop/README.md).++CI builds installers on every push/PR to `main` (and on `workflow_dispatch`) via [`.github/workflows/desktop.yml`](.github/workflows/desktop.yml). Push a `v*` tag to publish a GitHub Release with the artifacts.++### Run the desktop shell in development++```bash+cd frontend && npm run build && cd ..+cd backend && uv sync && cd ..+npm --prefix desktop install+npm run desktop+```++## Local development (browser)- ### Backend+ 1. Edit Python script+ 2. Run the program+ 3. Save temporary images+ 4. Compare outputs+ 5. Tune parameters+ 6. Repeat dozens of times++ ## Traditional Workflow vs Image Pipes++ | Traditional | Image Pipes |+ |------------|-------------|+ | Hundreds of lines of boilerplate | Visual node graph |+ | `cv2.imshow()` debugging | Live previews |+ | Manual parameter tuning | Interactive controls |+ | Hidden intermediate states | Inspect every node |+ | Script-only workflow | Export clean Python code |++ ---++ # Key Features++ ## 🎨 Visual Workflow Editor++ - React Flow powered canvas+ - Smooth zoom & pan+ - Dynamic grid snapping+ - Automatic connection validation+ - Rich node controls+ - sliders+ - dropdowns+ - switches+ - numeric inputs++ ---++ ## 👁️ Real-Time Preview++ - Low-latency WebSocket execution+ - Live intermediate images+ - Bounding box overlays+ - Mask visualization+ - Keypoint visualization+ - Execution timing+ - Runtime logs++ ---++ ## 🧪 Rich Processing Library++ ### OpenCV++ - Color conversions+ - Blur & filtering+ - Thresholding+ - Morphological operations+ - Histograms+ - Edge detection+ - Contours+ - Geometric transforms+ - K-Means clustering++ ### Albumentations++ Supports:++ - Image transforms+ - Bounding boxes+ - Segmentation masks+ - Keypoints+ - Multi-target augmentation++ Deterministic seeded execution enables reproducible experiments.++ ---++ ## ⚡ Smart Execution Engine++ - DAG validation+ - Topological sorting+ - Partial execution+ - Run-to-selected-node+ - In-memory caching+ - Incremental recomputation++ ---++ ## 📦 Metadata-Driven Design++ New processing nodes are automatically discovered.++ Adding a backend node requires **zero frontend changes**.++ Features include:++ - Automatic node discovery+ - Dynamic UI generation+ - JSON workspace import/export++ ---++ # How It Works++ ```text+ Image+ │+ ▼+ Visual Graph+ │+ ▼+ Validate DAG+ │+ ▼+ Execute Nodes+ │+ ▼+ Preview Every Step+ │+ ▼+ Export Native Python+ ```++ ---++ # Architecture++ ```text+ ┌─────────────────────────┐+ │ React Flow Editor │+ └────────────┬────────────┘+ │+ WebSocket / REST+ │+ ▼+ ┌─────────────────────────┐+ │ DAG Execution Engine │+ └────────────┬────────────┘+ │+ ┌─────────────────────┴─────────────────────┐+ ▼ ▼+ ┌────────────────────┐ ┌────────────────────┐+ │ OpenCV Nodes │ │ Albumentations │+ └──────────┬─────────┘ └──────────┬─────────┘+ │ │+ └─────────────────────┬─────────────────────┘+ │+ ▼+ ┌─────────────────────────┐+ │ Previews & Export │+ └────────────┬────────────┘+ ```++ ---++ # Tech Stack++ ## Frontend++ | Technology | Purpose |+ |------------|---------|+ | React 19 | UI |+ | TypeScript | Type safety |+ | Vite | Build tool |+ | React Flow | Graph editor |+ | Zustand | State management |+ | TanStack Query | Server state |+ | Material UI | Components |+ | Monaco Editor | Code editor |+ | React Hook Form | Forms |+ | Zod | Validation |++ ---++ ## Backend++ | Technology | Purpose |+ |------------|---------|+ | FastAPI | API |+ | WebSockets | Live execution |+ | Pydantic v2 | Data validation |+ | OpenCV | Image processing |+ | Albumentations | Data augmentation |+ | Pillow | Image loading |+ | NumPy | Numerical operations |+ | Custom DAG Engine | Workflow execution |++ ---++ ## Tooling++ - Python 3.12++ - uv+ - Node.js 20++ - npm+ - Docker+ - Docker Compose++ ---++ # Quick Start++ ## Prerequisites++ - Python **3.12+**+ - Node.js **20+**+ - **uv** (recommended)++ ---++ ## Backend Setup```bashcd backend@@@ -102,77 -306,191 +331,191 @@@ npm instalnpm run dev```- Open [http://127.0.0.1:5173](http://127.0.0.1:5173). Vite proxies `/api` and `/ws` to the FastAPI server.+ Open:- ### Quality checks+ ```+ http://127.0.0.1:5173+ ```++ The Vite development server automatically proxies:++ - `/api`+ - `/ws`++ to the backend.+------## Docker Setup++Or with Docker (single container serves UI + API on port 8000):++ ```bash+ docker compose up --build+ ```+----++Open [http://127.0.0.1:8000](http://127.0.0.1:8000). Persistent cache/uploads/outputs use named volumes. Image builds are verified in CI by [`.github/workflows/docker.yml`](.github/workflows/docker.yml).++++## Usage++ # Quality Checks++ ## Backend```bash- # backendcd backend+uv run ruff check+uv run pytest+ ```++ ## Frontend- # frontend+ ```bashcd frontend- npm run typecheck+npm run lint++ npm run typecheck```- ## Production (single process)+ ---- Build the frontend, then serve it from FastAPI:+ # Adding a New Node- ```bash- cd frontend && npm run build && cd ../backend- uv run uvicorn app.main:app --host 0.0.0.0 --port 8000+ Image Pipes uses a metadata-driven architecture.++ Adding a custom node requires **zero frontend modifications**.++ ## 1. Create a Node++ ```python+ from app.nodes.base import BaseNode+ import cv2++ class GaussianBlurNode(BaseNode):+ category = "Filters"+ display_name = "Gaussian Blur"++ def execute(self, inputs: dict, params: dict):+ kernel_size = params.get("kernel_size", 5)+ k = kernel_size if kernel_size % 2 else kernel_size + 1+ output = cv2.GaussianBlur(inputs["image"], (k, k), 0)+ return {"image": output}++ def emit_python(self, inputs, params):+ k = params.get("kernel_size", 5)+ return (+ f"blurred = cv2.GaussianBlur("+ f"{inputs['image']}, ({k}, {k}), 0)"+ )```- Or with Docker (single container serves UI + API on port 8000):+ ---++ ## 2. Register the Node++ Register it in the node catalog.++ The frontend automatically:++ - discovers the node+ - renders its controls+ - exposes it in the palette++ No frontend code changes required.++ > [!TIP]+ > See `docs/architecture.md` for the complete node schema.++ ---++ # Exporting Code++ Image Pipes generates standalone Python without proprietary runtime dependencies.++ ```python+ import cv2++ image = cv2.imread("input.jpg")++ gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)+ blur = cv2.GaussianBlur(gray, (5, 5), 0)+ edges = cv2.Canny(blur, 50, 150)++ cv2.imwrite("output_edges.png", edges)+ ```++ ---++ # Roadmap++ - [ ] Video pipeline support+ - [ ] Camera streaming+ - [ ] ONNX Runtime nodes+ - [ ] PyTorch inference nodes+ - [ ] Custom Python nodes+ - [ ] CUDA acceleration+ - [ ] Batch processing+ - [ ] Plugin SDK+ - [ ] Cloud workspace synchronization++ ---++ # Contributing++ Contributions are welcome!```bash- docker compose up --build+ # Fork the repository++ # Create a feature branch+ git checkout -b feature/amazing-node++ # Commit your changes+ git commit -m "Add custom threshold node"++ # Push+ git push origin feature/amazing-node```- Open [http://127.0.0.1:8000](http://127.0.0.1:8000). Persistent cache/uploads/outputs use named volumes. Image builds are verified in CI by [`.github/workflows/docker.yml`](.github/workflows/docker.yml).+ Then open a Pull Request.- ## Usage+ For larger changes, please open an issue first to discuss the proposal.++ ---++ # FAQ++ ### Is Image Pipes only a visual wrapper?++ No.++ The exported pipeline consists entirely of standard Python and OpenCV code and runs independently without Image Pipes installed.++ ---- 1. Drag nodes from the **Starters** / palette onto the canvas (or click **Example**).- 2. Connect matching port types (image / mask / bboxes / keypoints), select a node, and edit parameters in the inspector.- 3. For **Load Images**, pick files or a folder (uploaded to the backend).- 4. For augmentation with labels, add an **Annotations** starter (Pascal VOC bboxes + xy keypoints) and optionally wire a mask image into the `mask` port.- 5. Click **Run** for the full graph, or **Run to selected** / node menu **Run to here** for a partial DAG.- 6. Watch live previews (bboxes/keypoints are overlaid on the image) and the execution log.- 7. Use **Export** / **Load** for workflow JSON; **Export Python** for a standalone script.- 8. Open **About** (info button next to the brand) for license and GitHub.+ ### Does it support bounding boxes and segmentation masks?- Example workflows live under [`backend/examples/`](backend/examples/) (e.g. Blur & Canny, Albu Augment, HSV Color Mask, CLAHE & Sharpen, K-Means Palette; image: [`lena.png`](backend/examples/lena.png)).+ Yes.- ## Albumentations augments+ Albumentations dual transforms automatically keep:- The **augment** palette category exposes Albumentations transforms via a backend catalog (color, blur/noise, weather, normalize/channels, and dual geometry). Every augment node shares the same ports:+ - bounding boxes+ - segmentation masks+ - keypoints- | Port | Required | Notes |- |------|----------|--------|- | `image` | yes | OpenCV BGR; converted to RGB for Albumentations |- | `mask` | no | Same H×W; stays aligned under Dual transforms |- | `bboxes` | no | Pascal VOC `[x_min, y_min, x_max, y_max, label]` |- | `keypoints` | no | `[x, y]` lists |+ synchronized throughout the pipeline.- Image-only transforms pass annotations through unchanged. Dual geometry transforms (flips, rotates, affine, crops, …) update mask/bboxes/keypoints together. Runs are seeded from the global seed / sample index.+ ---- **Dependency note:** Image Pipes pins the MIT-licensed [`albumentations==2.0.8`](https://pypi.org/project/albumentations/2.0.8/) package so this project stays MIT-clean. The newer maintained `albumentationsx` line is AGPL and is intentionally not used here.+ ### Can I use it offline?- ## Adding a node+ Yes.- See [docs/architecture.md](docs/architecture.md) for the registration guide. In short:+ Everything runs locally on your machine.- 1. Subclass `BaseNode` in `backend/app/nodes/`.- 2. Implement `execute()` and `emit_python()`.- 3. Register the instance in `backend/app/nodes/__init__.py`.+ ---- No frontend changes are required beyond what the `/api/nodes` metadata already drives.+ # License- ## License+ Distributed under the **MIT License**.- This project is licensed under the [MIT License](LICENSE).+ See the **LICENSE** file for details.