Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

feat(integrations): add the pipecat-threews avatar processor, fix a cross-turn transcript leak

nirholas0cee019fb3bc
diff --git a/integrations/pipecat/pipecat-threews/CHANGELOG.md b/integrations/pipecat/pipecat-threews/CHANGELOG.md
new file mode 100644
index 000000000..d04996d5f
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/CHANGELOG.md
@@ -0,0 +1,19 @@
+# Changelog
+
+All notable changes to `pipecat-threews` are documented in this file.
+
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/).
+
+## [0.1.0] - 2026-10-13
+
+### Added
+
+- `ThreeWsAvatarProcessor`, a Pipecat frame processor that republishes a
+ pipeline's TTS transcript, best-effort gesture cues, and user speech
+ boundaries as JSON transport messages for three.ws's `<agent-3d>` web
+ component to render client-side.
+- `gesture_for_text`, the text heuristic used to decide a gesture cue.
+- Foundational example (`examples/avatar_bot.py`) joining a local LiveKit
+ room and speaking a scripted demo through a real local `espeak-ng` TTS
+ service.
diff --git a/integrations/pipecat/pipecat-threews/README.md b/integrations/pipecat/pipecat-threews/README.md
new file mode 100644
index 000000000..36f519b1f
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/README.md
@@ -0,0 +1,140 @@
+# pipecat-threews
+
+A [Pipecat](https://github.com/pipecat-ai/pipecat) frame processor that publishes
+[three.ws](https://three.ws) 3D avatar motion data from any Pipecat pipeline.
+
+**Maintainer:** three.ws ([github.com/nirholas/three.ws](https://github.com/nirholas/three.ws)).
+This is a community integration maintained by the three.ws team, not by Daily
+(the maintainers of Pipecat).
+
+## What it does
+
+three.ws avatars (the `<agent-3d>` web component) render entirely on the
+visitor's own device: the 3D model, its rig and its animation clip library
+already live in the browser. `ThreeWsAvatarProcessor` never synthesizes video
+or audio of its own. Instead it taps the TTS and user-turn frames already
+flowing through your pipeline and republishes them as small JSON messages
+over the active transport's data channel:
+
+```json
+{"type": "transcript", "text": "Hi, how can I help today?"}
+{"type": "gesture", "name": "wave"}
+{"type": "speech_started"}
+{"type": "speech_ended", "transcript": "what can you do"}
+```
+
+Point your frontend at
+[`<agent-3d voice="livekit">`](https://github.com/nirholas/three.ws/blob/main/src/element.js)
+(or any embed that loads
+[`src/runtime/livekit-voice.js`](https://github.com/nirholas/three.ws/blob/main/src/runtime/livekit-voice.js)),
+which already parses this message shape over LiveKit's data channel. Any
+transport whose Pipecat implementation routes `OutputTransportMessageFrame`/
+`OutputTransportMessageUrgentFrame` to a real data channel (LiveKit, Daily)
+works the same way.
+
+## Installation
+
+```bash
+pip install pipecat-threews
+```
+
+## Usage with a Pipecat Pipeline
+
+Add `ThreeWsAvatarProcessor()` after your TTS service and before
+`transport.output()`, so it observes both the assistant's `TTSTextFrame`s and
+the user's `UserStartedSpeakingFrame` / `UserStoppedSpeakingFrame` /
+`TranscriptionFrame`s on their way out:
+
+```python
+from pipecat.pipeline.pipeline import Pipeline
+from pipecat_threews import ThreeWsAvatarProcessor
+
+pipeline = Pipeline(
+ [
+ transport.input(),
+ stt,
+ context_aggregator.user(),
+ llm,
+ tts,
+ ThreeWsAvatarProcessor(),
+ transport.output(),
+ context_aggregator.assistant(),
+ ]
+)
+```
+
+No API key, account or environment variable is required: the processor reads
+state from frames already in your pipeline and republishes it as motion data
+on whatever room or call the transport is already connected to.
+
+**Your TTS service must be constructed with `push_start_frame=True` and
+`push_stop_frames=True`.** `ThreeWsAvatarProcessor` buffers `TTSTextFrame`
+text between the `TTSStartedFrame`/`TTSStoppedFrame` pair and publishes on
+`TTSStoppedFrame`. Pipecat's `TTSService` base class defaults both flags to
+`False`, so a TTS service built without them never emits a `TTSStoppedFrame`
+and the processor silently never publishes. See `examples/avatar_bot.py`'s
+`EspeakTTSService` for the correct construction.
+
+### How it decides what to show
+
+- **Lip sync** comes from every finished assistant turn: the full text is
+ sent once (buffered between `TTSStartedFrame` and `TTSStoppedFrame`), and
+ the browser times it into visemes the same way it already does for every
+ other voice path in three.ws (`src/runtime/lipsync.js`), so there is
+ nothing provider specific for the browser to learn.
+- **Gestures** are a best-effort read of that same text
+ (`pipecat_threews.gesture_for_text`): a greeting or farewell triggers
+ `wave`, an agreement triggers `nod`, a disagreement triggers `shrug`. A
+ miss just means no gesture fires, never a wrong one.
+- **Listening pose** comes from `UserStartedSpeakingFrame` /
+ `UserStoppedSpeakingFrame`, with the last finalized `TranscriptionFrame`
+ attached to the `speech_ended` message.
+
+## How to run the example
+
+[`examples/avatar_bot.py`](examples/avatar_bot.py) is a complete,
+runnable foundational example: a bot that joins a LiveKit room and speaks a
+scripted demo sequence through a real local `espeak-ng` text-to-speech
+service (no cloud TTS, no API keys), with `ThreeWsAvatarProcessor` wired in
+to publish the matching transcript and gesture data.
+
+1. Install `espeak-ng` (`apt install espeak-ng` or equivalent).
+2. Run a local, open-source LiveKit server:
+
+ ```bash
+ docker run --rm -p 7880:7880 -p 7881:7881 -p 7882:7882/udp \
+ -e LIVEKIT_KEYS="devkey: devsecret" \
+ livekit/livekit-server --dev --bind 0.0.0.0
+ ```
+
+3. Install this package and its example dependencies:
+
+ ```bash
+ pip install -e ".[dev]"
+ ```
+
+4. Run the bot:
+
+ ```bash
+ LIVEKIT_URL=ws://localhost:7880 \
+ LIVEKIT_API_KEY=devkey \
+ LIVEKIT_API_SECRET=devsecret \
+ LIVEKIT_ROOM_NAME=pipecat-threews-demo \
+ python examples/avatar_bot.py
+ ```
+
+5. Join `pipecat-threews-demo` with any LiveKit client (or three.ws's own
+ `<agent-3d voice="livekit">` pointed at the same server) to hear the bot
+ speak and watch the avatar data arrive on the room's data channel.
+
+## Pipecat version compatibility
+
+Tested with Pipecat v1.12.0.
+
+## License
+
+Apache-2.0, matching the three.ws monorepo this package ships inside. See
+the repository [LICENSE](../../../LICENSE). Apache-2.0 is a permissive
+OSI-approved license and satisfies Pipecat's community-integration
+requirement of "a permissive license (BSD-2 like Pipecat, or equivalent
+open source terms)".
diff --git a/integrations/pipecat/pipecat-threews/docs-submission/docs-json-navigation.md b/integrations/pipecat/pipecat-threews/docs-submission/docs-json-navigation.md
new file mode 100644
index 000000000..ebfc4078d
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/docs-submission/docs-json-navigation.md
@@ -0,0 +1,23 @@
+# Docs PR: docs.json navigation entry
+
+In `docs.json`, the `Extensions` group lives at
+`navigation.tabs[4].groups[1].pages[3]` and is a flat, alphabetically sorted
+list of page paths. Add `"api-reference/server/extensions/pipecat-threews"`
+after `pipecat-duplexjev` and before `voicemail`:
+
+```json
+{
+ "group": "Extensions",
+ "pages": [
+ "api-reference/server/extensions/ivr",
+ "api-reference/server/extensions/pipecat-backchannel",
+ "api-reference/server/extensions/pipecat-duplexjev",
+ "api-reference/server/extensions/pipecat-threews",
+ "api-reference/server/extensions/voicemail"
+ ]
+}
+```
+
+No `redirects` entry is needed: this is a brand-new page with no prior URL,
+and the other community extension pages (`pipecat-backchannel`,
+`pipecat-duplexjev`) have none either.
diff --git a/integrations/pipecat/pipecat-threews/docs-submission/pipecat-threews.mdx b/integrations/pipecat/pipecat-threews/docs-submission/pipecat-threews.mdx
new file mode 100644
index 000000000..677ec7b26
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/docs-submission/pipecat-threews.mdx
@@ -0,0 +1,115 @@
+---
+title: "three.ws Avatar"
+description: "Publishes three.ws 3D avatar motion data (lip-sync transcript and gesture cues) from any Pipecat pipeline"
+---
+
+import { CommunityMaintained } from "/snippets/community-maintained.mdx";
+
+<CommunityMaintained
+ maintainer="nirholas"
+ maintainerUrl="https://github.com/nirholas"
+ repo="https://github.com/nirholas/three.ws"
+/>
+
+## Overview
+
+`ThreeWsAvatarProcessor` drives a [three.ws](https://three.ws) 3D avatar
+(the `<agent-3d>` web component) from a Pipecat pipeline. three.ws avatars
+render entirely on the visitor's own device: the 3D model, its rig, and its
+animation clip library already live in the browser. This processor never
+synthesizes video or audio of its own. Instead it taps the TTS and
+user-turn frames already flowing through your pipeline and republishes them
+as small JSON messages over the active transport's data channel (lip-sync
+transcript, best-effort gesture cues, and user speech boundaries), which the
+three.ws client already knows how to render.
+
+<CardGroup cols={2}>
+ <Card
+ title="Source Repository"
+ icon="github"
+ href="https://github.com/nirholas/three.ws/tree/main/integrations/pipecat/pipecat-threews"
+ >
+ Source code, the foundational example, and the three.ws platform this
+ integration drives
+ </Card>
+ <Card
+ title="PyPI Package"
+ icon="cube"
+ href="https://pypi.org/project/pipecat-threews/"
+ >
+ The `pipecat-threews` package on PyPI
+ </Card>
+</CardGroup>
+
+## Installation
+
+This is a community-maintained package distributed separately from
+`pipecat-ai`:
+
+```bash
+pip install pipecat-threews
+```
+
+## Prerequisites
+
+No account or API key of its own. The processor only reads frames already
+flowing through your pipeline.
+
+- **Pipecat 1.0.0 or later**: `pipecat-ai>=1.0.0`
+- **A TTS service in the pipeline, constructed with `push_start_frame=True`
+ and `push_stop_frames=True`**: the processor buffers `TTSTextFrame` text
+ between the `TTSStartedFrame`/`TTSStoppedFrame` pair and publishes on
+ `TTSStoppedFrame`. Pipecat's `TTSService` base class defaults both flags
+ to `False`, so a TTS service built without them never emits a
+ `TTSStoppedFrame` and the processor never publishes anything, silently.
+ See `examples/avatar_bot.py`'s `EspeakTTSService` for the correct
+ construction.
+- **A transport with a data channel** (LiveKit, Daily): the processor
+ publishes via `OutputTransportMessageUrgentFrame`, which only reaches
+ clients when the transport routes transport-message frames to a real
+ data channel.
+- **A three.ws `<agent-3d>` embed** (or any client reading the same JSON
+ shape) to render the avatar. See
+ [`src/runtime/livekit-voice.js`](https://github.com/nirholas/three.ws/blob/main/src/runtime/livekit-voice.js)
+ in the three.ws repository.
+
+## Configuration
+
+Constructor arguments for `ThreeWsAvatarProcessor`.
+
+<ParamField path="topic" type="str" default="threews.avatar">
+ The transport data-channel topic to publish on. Only meaningful for
+ transports (like LiveKit) whose data messages carry a topic string;
+ transports without one ignore it.
+</ParamField>
+
+## Usage
+
+Place one instance after your TTS service and before `transport.output()`,
+so it observes both the assistant's `TTSTextFrame`s and the user's
+`UserStartedSpeakingFrame` / `UserStoppedSpeakingFrame` /
+`TranscriptionFrame`s on their way out:
+
+```python
+from pipecat.pipeline.pipeline import Pipeline
+from pipecat_threews import ThreeWsAvatarProcessor
+
+pipeline = Pipeline(
+ [
+ transport.input(),
+ stt,
+ context_aggregator.user(),
+ llm,
+ tts,
+ ThreeWsAvatarProcessor(),
+ transport.output(),
+ context_aggregator.assistant(),
+ ]
+)
+```
+
+## Compatibility
+
+Tested with Pipecat v1.12.0. Check the [source
+repository](https://github.com/nirholas/three.ws/tree/main/integrations/pipecat/pipecat-threews)
+for the latest tested version and changelog.
diff --git a/integrations/pipecat/pipecat-threews/docs-submission/supported-services-row.md b/integrations/pipecat/pipecat-threews/docs-submission/supported-services-row.md
new file mode 100644
index 000000000..9e68d185b
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/docs-submission/supported-services-row.md
@@ -0,0 +1,21 @@
+# Docs PR: supported-services.mdx row
+
+Add this row to the `## Extensions` table in
+`api-reference/server/services/supported-services.mdx` (alphabetical by
+link text, after "Pipecat Backchannel"):
+
+```
+| [three.ws Avatar](/api-reference/server/extensions/pipecat-threews) | `uv add pipecat-threews` | Community |
+```
+
+Full table after the addition:
+
+```
+| Extension | Setup | Maintainer |
+| --------------------------------------------------------------------------- | ---------------------------- | ---------- |
+| [DuplexJev Turn Analyzer](/api-reference/server/extensions/pipecat-duplexjev) | `uv add pipecat-duplexjev` | Community |
+| [IVRNavigator](/api-reference/server/extensions/ivr) | Built in | Pipecat |
+| [Pipecat Backchannel](/api-reference/server/extensions/pipecat-backchannel) | `uv add pipecat-backchannel` | Community |
+| [three.ws Avatar](/api-reference/server/extensions/pipecat-threews) | `uv add pipecat-threews` | Community |
+| [VoicemailDetector](/api-reference/server/extensions/voicemail) | Built in | Pipecat |
+```
diff --git a/integrations/pipecat/pipecat-threews/examples/avatar_bot.py b/integrations/pipecat/pipecat-threews/examples/avatar_bot.py
new file mode 100644
index 000000000..2f7b5d52b
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/examples/avatar_bot.py
@@ -0,0 +1,166 @@
+#
+# Copyright (c) 2026, three.ws
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+
+"""Foundational example: a Pipecat bot publishing three.ws avatar data.
+
+Speech-to-text and the LLM are skipped on purpose: this example proves the
+``ThreeWsAvatarProcessor`` end to end (a real local TTS speaks, the
+processor republishes synced transcript and gesture data over the room's
+data channel) without needing a cloud account for any leg of the pipeline.
+
+Text-to-speech is a thin wrapper around the ``espeak-ng`` command line
+(``EspeakTTSService`` below), a real open-source offline synthesizer, not a
+mock: every audio frame the room receives came out of an actual
+``espeak-ng`` subprocess.
+
+Run it against a local open-source ``livekit-server``
+(see ``../../../livekit/livekit-plugins-threews/README.md`` for the docker
+command) with ``LIVEKIT_URL``, ``LIVEKIT_API_KEY`` and ``LIVEKIT_API_SECRET``
+set in the environment:
+
+ python avatar_bot.py
+"""
+
+import asyncio
+import io
+import shutil
+import wave
+from collections.abc import AsyncGenerator
+
+from loguru import logger
+from pipecat.frames.frames import EndFrame, ErrorFrame, Frame, TTSAudioRawFrame, TTSSpeakFrame
+from pipecat.pipeline.pipeline import Pipeline
+from pipecat.pipeline.runner import PipelineRunner
+from pipecat.pipeline.task import PipelineParams, PipelineTask
+from pipecat.runner.livekit import configure
+from pipecat.services.tts_service import TTSService
+from pipecat.transports.livekit.transport import LiveKitParams, LiveKitTransport
+
+from pipecat_threews import ThreeWsAvatarProcessor
+
+ESPEAK_SAMPLE_RATE = 22050
+DEMO_LINES = [
+ "Hello! I'm the three.ws avatar, speaking through Pipecat.",
+ "Yes, this processor publishes a gesture cue with that line.",
+ "Goodbye, and thanks for watching the demo!",
+]
+
+
+class EspeakTTSService(TTSService):
+ """Offline TTS backed by the ``espeak-ng`` binary.
+
+ No network call, no API key: every utterance is synthesized locally by
+ shelling out to ``espeak-ng --stdout``, which writes a WAV file this
+ service decodes into the PCM frames Pipecat's audio pipeline expects.
+ """
+
+ def __init__(self, *, voice: str = "en-us", rate_wpm: int = 165, **kwargs) -> None:
+ """Initialize the service.
+
+ Args:
+ voice: The ``espeak-ng`` voice name, forwarded to its ``-v`` flag.
+ rate_wpm: Speech rate in words per minute, forwarded to ``-s``.
+ **kwargs: Additional arguments passed to the parent ``TTSService``.
+ """
+ super().__init__(
+ sample_rate=ESPEAK_SAMPLE_RATE,
+ push_start_frame=True,
+ push_stop_frames=True,
+ **kwargs,
+ )
+ if shutil.which("espeak-ng") is None:
+ raise RuntimeError(
+ "espeak-ng is not installed; install it (e.g. `apt install espeak-ng`) "
+ "to run this example"
+ )
+ self._voice = voice
+ self._rate_wpm = rate_wpm
+
+ def can_generate_metrics(self) -> bool:
+ """Check if this service can generate processing metrics.
+
+ Returns:
+ True, as this service supports metrics.
+ """
+ return True
+
+ async def run_tts(self, text: str, context_id: str) -> AsyncGenerator[Frame, None]:
+ """Synthesize ``text`` with ``espeak-ng`` and yield PCM audio frames.
+
+ Args:
+ text: The text to synthesize into speech.
+ context_id: The context ID for tracking audio frames.
+
+ Yields:
+ Frame: Audio frames containing the synthesized speech.
+ """
+ try:
+ proc = await asyncio.create_subprocess_exec(
+ "espeak-ng",
+ "-v",
+ self._voice,
+ "-s",
+ str(self._rate_wpm),
+ "--stdout",
+ text,
+ stdout=asyncio.subprocess.PIPE,
+ stderr=asyncio.subprocess.PIPE,
+ )
+ wav_bytes, stderr = await proc.communicate()
+ if proc.returncode != 0:
+ raise RuntimeError(f"espeak-ng failed: {stderr.decode(errors='replace')}")
+
+ with wave.open(io.BytesIO(wav_bytes), "rb") as wav_file:
+ pcm = wav_file.readframes(wav_file.getnframes())
+
+ await self.start_tts_usage_metrics(text)
+
+ chunk_size = self.sample_rate // 5 * 2 # ~200ms of 16-bit mono PCM per chunk
+ for offset in range(0, len(pcm), chunk_size):
+ chunk = pcm[offset : offset + chunk_size]
+ if chunk:
+ await self.stop_ttfb_metrics()
+ yield TTSAudioRawFrame(chunk, self.sample_rate, 1, context_id=context_id)
+ except Exception as error:
+ logger.exception("espeak-ng synthesis failed")
+ yield ErrorFrame(error=f"espeak-ng TTS error: {error}")
+
+
+async def main() -> None:
+ """Join a local LiveKit room and speak a scripted demo sequence."""
+ url, token, room_name = await configure()
+
+ transport = LiveKitTransport(
+ url,
+ token,
+ room_name,
+ params=LiveKitParams(audio_out_enabled=True),
+ )
+
+ tts = EspeakTTSService()
+ avatar = ThreeWsAvatarProcessor()
+
+ pipeline = Pipeline([tts, avatar, transport.output()])
+ task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
+
+ async def speak_demo_lines() -> None:
+ # Queuing every line at once would let the TTS service (near-instant
+ # for espeak-ng) race through all three turns before any audio
+ # actually plays, publishing avatar data for the whole script in a
+ # single burst. A real conversational bot is never this instant: a
+ # turn takes as long as it takes to actually speak before the next
+ # one starts. Pace it the same way so the demo matches real usage.
+ for line in DEMO_LINES:
+ await task.queue_frames([TTSSpeakFrame(line)])
+ await asyncio.sleep(2)
+ await task.queue_frames([EndFrame()])
+
+ runner = PipelineRunner()
+ await asyncio.gather(runner.run(task), speak_demo_lines())
+
+
+if __name__ == "__main__":
+ asyncio.run(main())
diff --git a/integrations/pipecat/pipecat-threews/pyproject.toml b/integrations/pipecat/pipecat-threews/pyproject.toml
new file mode 100644
index 000000000..3b415c79a
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/pyproject.toml
@@ -0,0 +1,68 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "pipecat-threews"
+version = "0.1.0"
+description = "Pipecat frame processor publishing three.ws 3D avatar motion data"
+readme = "README.md"
+license = { text = "Apache-2.0" }
+requires-python = ">=3.10"
+authors = [{ name = "three.ws", email = "[email protected]" }]
+keywords = ["pipecat", "voice", "ai", "realtime", "avatar", "3d", "threews"]
+classifiers = [
+ "Intended Audience :: Developers",
+ "License :: OSI Approved :: Apache Software License",
+ "Topic :: Multimedia :: Sound/Audio",
+ "Topic :: Scientific/Engineering :: Artificial Intelligence",
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.10",
+ "Programming Language :: Python :: 3 :: Only",
+]
+dependencies = [
+ "pipecat-ai>=1.0.0",
+]
+
+[project.urls]
+Documentation = "https://three.ws/docs"
+Website = "https://three.ws/"
+Source = "https://github.com/nirholas/three.ws"
+Changelog = "https://github.com/nirholas/three.ws/blob/main/integrations/pipecat/pipecat-threews/CHANGELOG.md"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+# Mirrors pipecat-ai/pipecat's own [tool.ruff] block (their pyproject.toml), the
+# config their CONTRIBUTING.md asks every community integration to pass.
+[tool.ruff]
+exclude = [".git"]
+line-length = 100
+
+[tool.ruff.lint]
+select = [
+ "ASYNC", # Blocking calls and other misuse inside async functions
+ "B", # Likely bugs and design problems
+ "D", # Docstring rules
+ "F", # Pyflakes
+ "I", # Import rules
+ "RUF006", # asyncio tasks nothing holds a reference to
+ "UP", # Modern Python
+]
+ignore = [
+ "ASYNC109",
+ "ASYNC240",
+ "ASYNC110",
+ "B024",
+ "B027",
+ "B007",
+ "B904",
+ "B905",
+ "D105",
+]
+
+[tool.ruff.lint.per-file-ignores]
+"examples/*" = ["D"]
+
+[tool.ruff.lint.pydocstyle]
+convention = "google"
diff --git a/integrations/pipecat/pipecat-threews/src/pipecat_threews/__init__.py b/integrations/pipecat/pipecat-threews/src/pipecat_threews/__init__.py
new file mode 100644
index 000000000..4840dc924
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/src/pipecat_threews/__init__.py
@@ -0,0 +1,13 @@
+#
+# Copyright (c) 2026, three.ws
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+
+"""Pipecat frame processor publishing three.ws 3D avatar motion data."""
+
+from .avatar import ThreeWsAvatarProcessor, gesture_for_text
+
+__all__ = ["ThreeWsAvatarProcessor", "gesture_for_text"]
+
+__version__ = "0.1.0"
diff --git a/integrations/pipecat/pipecat-threews/src/pipecat_threews/avatar.py b/integrations/pipecat/pipecat-threews/src/pipecat_threews/avatar.py
new file mode 100644
index 000000000..2bb7d8033
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/src/pipecat_threews/avatar.py
@@ -0,0 +1,166 @@
+#
+# Copyright (c) 2026, three.ws
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+
+"""Frame processor publishing three.ws avatar motion data from a pipeline."""
+
+import re
+
+from pipecat.frames.frames import (
+ Frame,
+ OutputTransportMessageUrgentFrame,
+ TranscriptionFrame,
+ TTSStartedFrame,
+ TTSStoppedFrame,
+ TTSTextFrame,
+ UserStartedSpeakingFrame,
+ UserStoppedSpeakingFrame,
+)
+from pipecat.processors.frame_processor import FrameDirection, FrameProcessor
+
+DEFAULT_TOPIC = "threews.avatar"
+
+# Gesture cues are a best-effort read of the assistant's own final text, the
+# same heuristic family three.ws's browser widget already uses for lip sync
+# (src/runtime/lipsync.js) and the one its LiveKit plugin uses
+# (livekit-plugins-threews): no NLU model, just the opening/closing words a
+# turn is likely to use. A miss just means no gesture fires, never a wrong
+# one, so the pattern favors precision over recall.
+_GREETING_RE = re.compile(
+ r"^\s*(hi|hello|hey there|hey|greetings|good (morning|afternoon|evening))\b",
+ re.IGNORECASE,
+)
+_FAREWELL_RE = re.compile(
+ r"\b(goodbye|bye for now|take care|farewell)[.!]?\s*$",
+ re.IGNORECASE,
+)
+_AGREEMENT_RE = re.compile(
+ r"^\s*(yes|yeah|yep|absolutely|definitely|of course)\b",
+ re.IGNORECASE,
+)
+_DISAGREEMENT_RE = re.compile(
+ r"^\s*(no|nope|not quite|not really|unfortunately)\b",
+ re.IGNORECASE,
+)
+
+
+def gesture_for_text(text: str) -> str | None:
+ """Best-effort gesture slot name for a line of assistant speech.
+
+ Args:
+ text: The assistant's finished turn, as spoken by the TTS service.
+
+ Returns:
+ One of the animation slot names three.ws avatars already know
+ (``wave``, ``nod``, ``shrug``), matched against
+ ``src/runtime/animation-slots.js`` in the three.ws client, or
+ ``None`` if the text does not read like a greeting, farewell,
+ agreement or disagreement.
+ """
+ if _GREETING_RE.search(text) or _FAREWELL_RE.search(text):
+ return "wave"
+ if _AGREEMENT_RE.search(text):
+ return "nod"
+ if _DISAGREEMENT_RE.search(text):
+ return "shrug"
+ return None
+
+
+class ThreeWsAvatarProcessor(FrameProcessor):
+ """Publishes three.ws avatar motion data alongside a Pipecat pipeline.
+
+ three.ws avatars (the ``<agent-3d>`` web component) render entirely on
+ the visitor's own device: the 3D model, its rig and its animation clip
+ library already live in the browser. This processor never synthesizes
+ video or audio of its own. Instead it taps the TTS and user-turn frames
+ already flowing through the pipeline and republishes them as small JSON
+ messages via ``OutputTransportMessageUrgentFrame``, which any transport
+ with a data channel (LiveKit, Daily, a WebSocket transport) delivers to
+ connected clients:
+
+ - ``{"type": "transcript", "text": "..."}`` for every finished assistant
+ turn, which the browser times into visemes locally
+ (``src/runtime/lipsync.js``).
+ - ``{"type": "gesture", "name": "wave" | "nod" | "shrug"}`` when the
+ turn's text reads like a greeting, farewell, agreement or
+ disagreement.
+ - ``{"type": "speech_started"}`` / ``{"type": "speech_ended", "transcript": "..."}``
+ around the user's own turns, so the avatar can switch to a listening
+ pose.
+
+ Place one instance late in the pipeline, after the TTS service and
+ before ``transport.output()``, so it observes both the assistant's
+ ``TTSTextFrame`` frames and the user's ``UserStartedSpeakingFrame``,
+ ``UserStoppedSpeakingFrame`` and ``TranscriptionFrame`` frames on their
+ way out.
+
+ Example::
+
+ pipeline = Pipeline(
+ [
+ transport.input(),
+ stt,
+ context_aggregator.user(),
+ llm,
+ tts,
+ ThreeWsAvatarProcessor(),
+ transport.output(),
+ context_aggregator.assistant(),
+ ]
+ )
+ """
+
+ def __init__(self, *, topic: str = DEFAULT_TOPIC) -> None:
+ """Initialize the processor.
+
+ Args:
+ topic: The transport data-channel topic to publish on. Defaults
+ to ``"threews.avatar"``, the topic three.ws's LiveKit client
+ (``src/runtime/livekit-voice.js``) already subscribes to.
+ """
+ super().__init__()
+ self._topic = topic
+ self._tts_text_buffer = ""
+ self._speaking_tts = False
+ self._last_user_transcript = ""
+
+ async def process_frame(self, frame: Frame, direction: FrameDirection) -> None:
+ """Observe TTS and user-turn frames, then forward every frame unchanged.
+
+ Args:
+ frame: The frame being processed.
+ direction: The direction the frame is flowing in the pipeline.
+ """
+ await super().process_frame(frame, direction)
+
+ if isinstance(frame, TTSStartedFrame):
+ self._speaking_tts = True
+ self._tts_text_buffer = ""
+ elif isinstance(frame, TTSTextFrame) and self._speaking_tts:
+ self._tts_text_buffer += frame.text
+ elif isinstance(frame, TTSStoppedFrame) and self._speaking_tts:
+ self._speaking_tts = False
+ text = self._tts_text_buffer.strip()
+ self._tts_text_buffer = ""
+ if text:
+ await self._publish({"type": "transcript", "text": text})
+ gesture = gesture_for_text(text)
+ if gesture:
+ await self._publish({"type": "gesture", "name": gesture})
+ elif isinstance(frame, TranscriptionFrame) and frame.finalized:
+ self._last_user_transcript = frame.text
+ elif isinstance(frame, UserStartedSpeakingFrame):
+ self._last_user_transcript = ""
+ await self._publish({"type": "speech_started"})
+ elif isinstance(frame, UserStoppedSpeakingFrame):
+ await self._publish({"type": "speech_ended", "transcript": self._last_user_transcript})
+
+ await self.push_frame(frame, direction)
+
+ async def _publish(self, payload: dict) -> None:
+ await self.push_frame(
+ OutputTransportMessageUrgentFrame(message=payload),
+ FrameDirection.DOWNSTREAM,
+ )
diff --git a/integrations/pipecat/pipecat-threews/tests/__init__.py b/integrations/pipecat/pipecat-threews/tests/__init__.py
new file mode 100644
index 000000000..fc16b7f6a
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/tests/__init__.py
@@ -0,0 +1 @@
+"""Tests for pipecat-threews."""
diff --git a/integrations/pipecat/pipecat-threews/tests/test_avatar.py b/integrations/pipecat/pipecat-threews/tests/test_avatar.py
new file mode 100644
index 000000000..582979616
--- /dev/null
+++ b/integrations/pipecat/pipecat-threews/tests/test_avatar.py
@@ -0,0 +1,153 @@
+# Copyright 2026 three.ws
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Regression test for ``ThreeWsAvatarProcessor``'s TTS-turn publish contract.
+
+A real deployment ships a demo bot (``examples/avatar_bot.py``) that was
+found, via a real local LiveKit run, to publish an assistant turn's gesture
+cue only if the TTS service is constructed with ``push_stop_frames=True``
+(Pipecat's ``TTSService`` defaults it to ``False``, so ``TTSStoppedFrame``
+never fires and the processor never publishes, silently). This test drives
+the processor directly with a real ``TTSStartedFrame`` /
+``TTSTextFrame`` / ``TTSStoppedFrame`` sequence through Pipecat's own test
+pipeline (``pipecat.tests.utils.run_test``), so a regression in the
+processor's own frame handling, independent of any particular TTS service's
+construction, fails this test rather than silently publishing nothing.
+"""
+
+from __future__ import annotations
+
+import pytest
+from pipecat.frames.frames import (
+ AggregationType,
+ OutputTransportMessageUrgentFrame,
+ TranscriptionFrame,
+ TTSStartedFrame,
+ TTSStoppedFrame,
+ TTSTextFrame,
+ UserStartedSpeakingFrame,
+ UserStoppedSpeakingFrame,
+)
+from pipecat.tests.utils import SleepFrame, run_test
+
+from pipecat_threews.avatar import ThreeWsAvatarProcessor
+
+
+async def test_finished_tts_turn_publishes_transcript_and_gesture():
+ """A full TTSStarted/TTSText/TTSStopped turn must publish both messages."""
+ processor = ThreeWsAvatarProcessor()
+
+ down_frames, _ = await run_test(
+ processor,
+ frames_to_send=[
+ TTSStartedFrame(),
+ TTSTextFrame("Hello! ", aggregated_by=AggregationType.SENTENCE),
+ TTSTextFrame("I'm the three.ws avatar.", aggregated_by=AggregationType.SENTENCE),
+ TTSStoppedFrame(),
+ ],
+ )
+
+ published = [f.message for f in down_frames if isinstance(f, OutputTransportMessageUrgentFrame)]
+ assert published == [
+ {"type": "transcript", "text": "Hello! I'm the three.ws avatar."},
+ {"type": "gesture", "name": "wave"},
+ ]
+
+
+async def test_tts_text_outside_a_turn_is_not_published():
+ """Without a TTSStoppedFrame, no buffered text is ever published."""
+ processor = ThreeWsAvatarProcessor()
+
+ down_frames, _ = await run_test(
+ processor,
+ frames_to_send=[
+ TTSStartedFrame(),
+ TTSTextFrame("Hello there", aggregated_by=AggregationType.SENTENCE),
+ ],
+ )
+
+ published = [f for f in down_frames if isinstance(f, OutputTransportMessageUrgentFrame)]
+ assert published == []
+
+
+async def test_user_speech_boundaries_publish_listen_events():
+ """User VAD boundaries publish speech_started/speech_ended with the transcript.
+
+ ``UserStartedSpeakingFrame``/``UserStoppedSpeakingFrame`` are Pipecat
+ ``SystemFrame``s, processed immediately on a fast path that bypasses the
+ processor's regular frame queue; ``TranscriptionFrame`` is not, so it
+ queues behind other work on a separately-scheduled task. Sent back to
+ back with no gap, ``UserStoppedSpeakingFrame`` can finish processing
+ before the ``TranscriptionFrame`` just ahead of it does, so the
+ ``SleepFrame`` here (Pipecat's own test-only frame for this exact
+ purpose) gives the regular queue a chance to drain first, matching how a
+ real STT service's finalized transcript has normally already landed by
+ the time VAD reports the user stopped talking.
+ """
+ processor = ThreeWsAvatarProcessor()
+
+ down_frames, _ = await run_test(
+ processor,
+ frames_to_send=[
+ UserStartedSpeakingFrame(),
+ TranscriptionFrame(
+ text="what can you do", user_id="user-1", timestamp="", finalized=True
+ ),
+ SleepFrame(sleep=0.05),
+ UserStoppedSpeakingFrame(),
+ ],
+ )
+
+ published = [f.message for f in down_frames if isinstance(f, OutputTransportMessageUrgentFrame)]
+ assert published == [
+ {"type": "speech_started"},
+ {"type": "speech_ended", "transcript": "what can you do"},
+ ]
+
+
+async def test_user_stopped_speaking_does_not_leak_prior_turn_transcript():
+ """A turn with no finalized transcript yet must not report the previous turn's.
+
+ ``_last_user_transcript`` is set whenever a finalized ``TranscriptionFrame``
+ arrives and, before this test's fix, was never cleared. If a second
+ turn's own transcript is slow to finalize (the STT race described in
+ ``test_user_speech_boundaries_publish_listen_events``), the processor
+ must not attribute the first turn's leftover text to the second turn's
+ ``speech_ended``.
+ """
+ processor = ThreeWsAvatarProcessor()
+
+ down_frames, _ = await run_test(
+ processor,
+ frames_to_send=[
+ UserStartedSpeakingFrame(),
+ TranscriptionFrame(text="first turn", user_id="user-1", timestamp="", finalized=True),
+ SleepFrame(sleep=0.05),
+ UserStoppedSpeakingFrame(),
+ UserStartedSpeakingFrame(),
+ UserStoppedSpeakingFrame(),
+ ],
+ )
+
+ published = [f.message for f in down_frames if isinstance(f, OutputTransportMessageUrgentFrame)]
+ assert published == [
+ {"type": "speech_started"},
+ {"type": "speech_ended", "transcript": "first turn"},
+ {"type": "speech_started"},
+ {"type": "speech_ended", "transcript": ""},
+ ]
diff --git a/tests/e2e/agent-3d-pipecat-avatar.spec.js b/tests/e2e/agent-3d-pipecat-avatar.spec.js
new file mode 100644
index 000000000..3b0baa92a
--- /dev/null
+++ b/tests/e2e/agent-3d-pipecat-avatar.spec.js
@@ -0,0 +1,152 @@
+import { test, expect } from '@playwright/test';
+import { spawn } from 'node:child_process';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { SignJWT } from 'jose';
+
+// Work order 087: proves the Pipecat `pipecat-threews` package end to end,
+// browser side, the same way agent-3d-livekit-avatar.spec.js proves the
+// LiveKit plugin. A real Pipecat pipeline (integrations/pipecat/
+// pipecat-threews/examples/avatar_bot.py) joins the same kind of real local
+// open-source livekit-server, speaks three lines through a real espeak-ng
+// TTS service, and `ThreeWsAvatarProcessor` republishes them as
+// `transcript`/`gesture` data-channel messages that a real Chromium loading
+// `<agent-3d voice="livekit">` renders through the exact same browser code
+// path (LiveKitVoice never knows which backend published the message).
+//
+// `video: 'on'` is set for this spec only so the run also produces the demo
+// video pipecat-ai/pipecat's COMMUNITY_INTEGRATIONS.md asks every community
+// integration PR to link (saved under the gitignored test-results/, not
+// committed; the submission doc records the local path for the owner to
+// attach when the PR is actually opened).
+
+const ROOM = `e2e-pipecat-${Date.now()}`;
+const LIVEKIT_URL = process.env.LIVEKIT_URL || 'ws://localhost:7880';
+const LIVEKIT_API_KEY = process.env.LIVEKIT_API_KEY || 'devkey';
+const LIVEKIT_API_SECRET = process.env.LIVEKIT_API_SECRET || 'devsecret';
+
+const HERE = path.dirname(fileURLToPath(import.meta.url));
+const EXAMPLES_DIR = path.join(
+ HERE,
+ '..',
+ '..',
+ 'integrations',
+ 'pipecat',
+ 'pipecat-threews',
+ 'examples',
+);
+
+async function mintBrowserToken(room) {
+ const secret = new TextEncoder().encode(LIVEKIT_API_SECRET);
+ const now = Math.floor(Date.now() / 1000);
+ return new SignJWT({
+ video: { roomJoin: true, room, canPublish: true, canSubscribe: true },
+ sub: 'e2e-browser',
+ iss: LIVEKIT_API_KEY,
+ nbf: now,
+ exp: now + 3600,
+ })
+ .setProtectedHeader({ alg: 'HS256' })
+ .sign(secret);
+}
+
+function spawnBot(room) {
+ const child = spawn('python3', ['-u', 'avatar_bot.py', '--room', room], {
+ cwd: EXAMPLES_DIR,
+ env: {
+ ...process.env,
+ LIVEKIT_URL,
+ LIVEKIT_API_KEY,
+ LIVEKIT_API_SECRET,
+ },
+ stdio: ['ignore', 'pipe', 'pipe'],
+ });
+ let log = '';
+ child.stdout.on('data', (d) => (log += d.toString()));
+ child.stderr.on('data', (d) => (log += d.toString()));
+ return { child, getLog: () => log };
+}
+
+test.use({ video: 'on' });
+
+test.describe('agent-3d Pipecat avatar processor (real local livekit-server)', () => {
+ test.setTimeout(120_000);
+
+ test('a real Pipecat bot speaking in the room drives the browser avatar lipsync + gesture', async ({
+ page,
+ }) => {
+ // The bot speaks its scripted lines immediately on connecting, with no
+ // wait for a participant to join (unlike the LiveKit AgentSession demo,
+ // whose session startup latency happens to give the browser time to
+ // join first). So the browser must be fully joined to the room, not
+ // just past protocol-object creation, before the bot is spawned at all,
+ // or its first lines race the browser's own room.connect() and are lost.
+ let bot;
+ try {
+ const token = await mintBrowserToken(ROOM);
+ await page.route('**/api/agents/*/livekit-token', (route) =>
+ route.fulfill({
+ status: 200,
+ contentType: 'application/json',
+ body: JSON.stringify({ token, serverUrl: LIVEKIT_URL }),
+ }),
+ );
+
+ const errors = [];
+ page.on('pageerror', (err) => errors.push(String(err?.message || err)));
+
+ await page.goto('/agent-3d-livekit-harness.html', { waitUntil: 'domcontentloaded' });
+
+ await expect
+ .poll(() => page.evaluate(() => document.getElementById('a3d')?._livekitVoice?.connected === true), {
+ timeout: 60_000,
+ })
+ .toBe(true);
+
+ // Attach the collectors with an awaited evaluate (not a long-lived
+ // in-page Promise) so this call only resolves once the listeners are
+ // actually registered, guaranteeing they're in place before the bot
+ // (spawned right after) can publish anything.
+ await page.evaluate(() => {
+ window.__threewsEvents = [];
+ const protocol = document.getElementById('a3d')._avatar.protocol;
+ protocol.on('speak', (action) =>
+ window.__threewsEvents.push({ type: 'speak', text: action.payload?.text }),
+ );
+ protocol.on('gesture', (action) =>
+ window.__threewsEvents.push({ type: 'gesture', name: action.payload?.name }),
+ );
+ });
+
+ bot = spawnBot(ROOM);
+
+ await expect
+ .poll(
+ () =>
+ page.evaluate(
+ () => window.__threewsEvents.filter((e) => e.type === 'speak').length,
+ ),
+ { timeout: 60_000 },
+ )
+ .toBeGreaterThanOrEqual(3);
+
+ // Hold on the final frame for a moment so the saved video has a
+ // visible tail after the last gesture instead of cutting off mid-clip.
+ await page.waitForTimeout(2000);
+
+ const events = await page.evaluate(() => window.__threewsEvents);
+ const speaks = events.filter((e) => e.type === 'speak').map((e) => e.text);
+ const gestures = events.filter((e) => e.type === 'gesture').map((e) => e.name);
+
+ expect(speaks, `bot log:\n${bot.getLog()}`).toEqual([
+ "Hello! I'm the three.ws avatar, speaking through Pipecat.",
+ 'Yes, this processor publishes a gesture cue with that line.',
+ 'Goodbye, and thanks for watching the demo!',
+ ]);
+ expect(gestures).toEqual(['wave', 'nod']);
+ expect(errors, `page errors: ${errors.join(' | ')}`).toEqual([]);
+ } finally {
+ bot?.child.kill('SIGTERM');
+ }
+ });
+});