Skip to content

Free Motion Capture for Everyone πŸ’€βœ¨

TypeScript
6.8 KB
Raw Blame

FreeMoCap UI

React/Electron desktop interface for FreeMoCap β€” live multi-camera streaming, recording, and synchronized playback.

Built with React 19, TypeScript, Material UI, Redux Toolkit, and packaged as an Electron desktop app via electron-vite.


Quick Start

npm install
npm run dev        # Vite dev server + Electron

The UI expects the FreeMoCap Python server running on localhost:53117 (default). Start the server first:

# From the repo root
python -m freemocap

Pages

Cameras (/cameras)

Live multi-camera view with real-time WebSocket streaming. Configure camera resolution, framerate, exposure, and rotation. Start/stop recordings from here.

Playback (/playback)

Browse and play back recorded sessions with frame-perfect multi-camera synchronization.

Recording Browser β€” lists all recordings from ~/freemocap_data/recordings/, sorted newest-first. Each entry shows:

  • Number of camera streams
  • Total recording size on disk
  • Recording duration and frame count (read from timestamp CSVs)
  • Effective FPS
  • Relative time ("2h ago", "3d ago")

You can also type or paste a custom folder path to load any recording.

Synced Video Player β€” opens all camera streams in a responsive grid. Key features:

  • Frame-locked synchronization β€” every camera displays the exact same frame number at all times, driven by a single authoritative counter
  • Per-camera overlays β€” frame number (top-right, green), camera ID (bottom-left), SMPTE timecode (bottom-right)
  • Smooth native playback β€” uses HTMLVideoElement.play() for hardware-decoded rendering, with a rAF drift-correction loop for sync
  • Frame stepping β€” ←/β†’ keys step 1 frame, Shift+←/β†’ step 10 frames
  • Variable speed β€” 0.1Γ— through 8Γ— with frame-lock maintained at all rates
  • Keyboard shortcuts β€” Space (play/pause), Home/End (jump to start/end)

Project Structure

src/
β”œβ”€β”€ pages/
β”‚   β”œβ”€β”€ CamerasPage.tsx          # Live camera view
β”‚   β”œβ”€β”€ PlaybackPage.tsx         # Recording browser β†’ synced player
β”‚   └── WelcomePage.tsx          # Landing / home
β”‚
β”œβ”€β”€ components/
β”‚   β”œβ”€β”€ playback/
β”‚   β”‚   β”œβ”€β”€ SyncedVideoPlayer.tsx   # Core frame-locked multi-video player
β”‚   β”‚   β”œβ”€β”€ PlaybackControls.tsx    # Transport bar (play, seek, step, speed)
β”‚   β”‚   β”œβ”€β”€ RecordingBrowser.tsx    # Recording list with metadata + manual path
β”‚   β”‚   └── index.ts               # Barrel exports
β”‚   β”‚
β”‚   β”œβ”€β”€ camera-views/            # CameraView, CameraViewsGrid
β”‚   β”œβ”€β”€ camera-config-panel/     # Resolution, framerate, exposure controls
β”‚   β”œβ”€β”€ camera-config-tree-view/ # Sidebar camera tree
β”‚   β”œβ”€β”€ camera-view-settings-overlay/  # Image scale slider
β”‚   β”œβ”€β”€ framerate-viewer/        # Real-time FPS D3 charts
β”‚   β”œβ”€β”€ recording-info-panel/    # Recording path, start/stop, settings
β”‚   β”œβ”€β”€ video-folder-panel/      # Legacy folder picker (Electron IPC)
β”‚   β”œβ”€β”€ ui-components/           # Header, Footer, LeftPanel, ThemeToggle
β”‚   └── common/                  # ErrorBoundary
β”‚
β”œβ”€β”€ services/
β”‚   β”œβ”€β”€ server/
β”‚   β”‚   β”œβ”€β”€ ServerContextProvider.tsx   # WebSocket lifecycle
β”‚   β”‚   └── server-helpers/
β”‚   β”‚       β”œβ”€β”€ server-urls.ts          # API endpoint URLs
β”‚   β”‚       β”œβ”€β”€ websocket-connection.ts # WebSocket manager
β”‚   β”‚       β”œβ”€β”€ canvas-manager.ts       # Canvas rendering
β”‚   β”‚       └── frame-processor/        # Binary frame parsing
β”‚   └── electron-ipc/                   # Electron main ↔ renderer IPC
β”‚
β”œβ”€β”€ store/
β”‚   β”œβ”€β”€ store.ts                 # Redux store configuration
β”‚   β”œβ”€β”€ hooks.ts                 # useAppDispatch, useAppSelector
β”‚   └── slices/
β”‚       β”œβ”€β”€ cameras/             # Camera state, detection, config
β”‚       β”œβ”€β”€ recording/           # Recording state, start/stop thunks
β”‚       β”œβ”€β”€ videos/              # Video file state, folder selection
β”‚       β”œβ”€β”€ framerate/           # FPS tracking data
β”‚       β”œβ”€β”€ log-records/         # Server log entries
β”‚       └── theme/               # Light/dark mode
β”‚
└── layout/
    β”œβ”€β”€ AppContent.tsx           # Top-level app wrapper
    β”œβ”€β”€ BaseContentRouter.tsx    # React Router routes
    └── BasePanelLayout.tsx      # Resizable panel layout (react-resizable-panels)

Key Dependencies

Package Purpose
react / react-dom UI framework
@mui/material Component library
@reduxjs/toolkit / react-redux State management
react-router-dom Client-side routing
react-resizable-panels Draggable panel layout
d3 Framerate charts
electron Desktop packaging
vite Build tool and dev server

Build

npm run build          # Production build (renderer + electron)
npm run build:unpack   # Build unpacked app for testing

See electron-builder.json for packaging configuration.


Server Communication

The UI talks to the Python server over two channels:

HTTP REST β€” camera control, recording management, and playback. Base URL defaults to http://localhost:53117. Key playback endpoints:

  • GET /freemocap/playback/recordings β€” list all recordings with metadata (size, frames, duration, fps)
  • POST /freemocap/playback/load β€” load a recording folder
  • GET /freemocap/playback/video/{video_id} β€” stream a video file (supports HTTP range requests for seeking)
  • GET /freemocap/playback/timestamps/{video_id} β€” get timestamp CSV info

WebSocket β€” real-time binary frame streaming at ws://localhost:53117/freemocap/websocket/connect. Delivers synchronized multi-frame payloads (one JPEG per camera per frame event) plus JSON log messages.


Playback Sync Architecture

The SyncedVideoPlayer component guarantees frame-perfect synchronization across all camera views:

  1. Native playback β€” all <video> elements use .play() for smooth hardware decoding
  2. Authoritative counter β€” a requestAnimationFrame loop computes the current frame from wall-clock elapsed Γ— fps Γ— playbackRate
  3. Drift correction β€” every ~3 ticks, each video's .currentTime is compared against the target; videos drifting >0.5 frames are force-seeked back
  4. Single source of truth β€” the frame number overlay on every camera comes from the authoritative counter, never from individual video elements
  5. Pause/step β€” sets .currentTime directly on all elements (no .play() involved)

This hybrid approach delivers smooth rendering while maintaining the hard sync guarantee.