Skip to content

armgit / ALIVE

Public Imported from GitHub
0

ALIVE: Making Streaming Video Models Interactive

Python
README

ALIVE

ALIVE: Making Streaming Video Models Interactive

Paper — arXiv:2611.21366 Model — ALIVE-8B on Hugging Face Hugging Face Dataset — coming soon FSI-Bench on Hugging Face — coming soon Project page Watch the demo

🤗 Hugging Face Daily Papers · #2 Paper of the day

Jinzhao Li1,3,* · Yuan Xu2,3,* · Jinxuan Wu4 · Yukun Wang1 · Yijia Lei1 · Hanyu Ji2 · Yinuo Chen1 · Zining Huang1
Chi Zhang1 · Hang Ye2 · Yiwu Zhong2 · Honglei Yan3 · Panwang Pan3 · Yizhou Wang2 · Yin Li5 · Miao Liu1,†

1 Tsinghua University   2 Peking University   3 ByteDance
4 Beihang University   5 University of Wisconsin–Madison
* Equal contribution   † Corresponding author

An interactive streaming video model for proactive monitoring,
streaming task coordination, and feedback-seeking interaction.

Demo

ALIVE demo cover with the paper title, authors, and affiliations. Click to watch the complete 4:38 demo with sound.

▶ Watch the full demo · 4:38 · English + 中文   ·   Download MP4

More demos: Real-world live conversations ↗   ·   Web-video demos ↗

Overview

Streaming video LLMs can continuously observe the world, yet often handle user requests in isolation. ALIVE turns them into interactive assistants with three capabilities:

Capability What ALIVE does
🔵 Proactive Monitoring Monitor unfolding events and respond when the requested condition is met.
🟣 Streaming Task Coordination Maintain multiple requests and update active tasks as instructions change.
🟠 Feedback-Seeking Interaction Ask for clarification, request a concrete user action, or propose continued monitoring when more evidence is needed.

ALIVE overview: proactive monitoring, streaming task coordination, and feedback-seeking interaction along a continuous stream.

Quick Start

Run the ALIVE browser demo locally with streaming speech recognition and speech synthesis on two GPUs: one for ALIVE and one shared by ASR/TTS (Linux, Python 3.12).

Component Hugging Face checkpoint
ALIVE ALIVE-Team/ALIVE (ALIVE-8B)
ASR Qwen/Qwen3-ASR-1.7B
TTS Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice

setup.py installs the isolated environments and downloads the ASR/TTS checkpoints; download ALIVE-8B separately:

git clone https://github.com/lijinzhao30/ALIVE.git
cd ALIVE/Web_Inference

python3.12 scripts/setup.py
huggingface-cli download ALIVE-Team/ALIVE --include "ALIVE-8B/*" --local-dir models/ALIVE

cp config.example.json config.json         # set "model_path": "models/ALIVE/ALIVE-8B"
python3.12 start.py                        # open https://localhost:8090

See the web inference guide for details.

Data

We build ALIVE-SFT-288K for supervised fine-tuning and ALIVE-RL-3K for reinforcement learning.

Construction and composition of ALIVE-SFT-288K and ALIVE-RL-3K.

Method

ALIVE is trained in two stages:

  • Contrastive supervised fine-tuning teaches when and how to respond.
  • Reinforcement learning with Behavior-Aligned Credit Assignment (BACA) refines decisions, responses, and silence across multi-turn interactions.

The ALIVE training framework: contrastive supervised fine-tuning followed by reinforcement learning with Behavior-Aligned Credit Assignment.

Benchmark

  • We introduce FSI-Bench to evaluate whether streaming video LLMs, when the current context is insufficient to resolve a request, can identify the missing prerequisite, actively seek the required feedback, and complete the task.
  • ALIVE achieves the highest overall scores among the compared models on four public benchmarks and FSI-Bench, demonstrating strong streaming understanding and interactive capabilities.

General streaming video understanding results on OVO-Bench, StreamingBench, and ProactiveVideoQA.

Streaming interaction results on IPIBench and FSI-Bench.

TODO

  • Paper
  • ALIVE-8B checkpoint
  • Local web inference
  • Benchmark and evaluation code
  • Training data

Citation

@article{li2026alive,
  title={ALIVE: Making Streaming Video Models Interactive},
  author={Li, Jinzhao and Xu, Yuan and Wu, Jinxuan and Wang, Yukun and Lei, Yijia and Ji, Hanyu and Chen, Yinuo and Huang, Zining and Zhang, Chi and Ye, Hang and others},
  journal={arXiv preprint arXiv:2611.21366},
  year={2026}
}

License

Code is released under the Apache License 2.0. See third-party notices and asset credits.