;
These native nodes are pre-installed with ComfyStream when you install the
package. You do not need to install them manually. See the [ComfyStream
Installation Guide](/technical/get-started/install) for details.
The ComfyStream package provides native nodes essential for real-time AI workflows in ComfyUI, handling tasks like converting audio and images into tensors, processing data, and converting tensors back into usable formats. Additionally, the **ComfyStream Toolkit** includes a **StreamPack**, which extends functionality with advanced custom nodes for video, audio, and data workflows. Learn more in the [StreamPack Overview](/technical/custom-nodes/stream-pack/overview) page.
## Input Nodes
ComfyStream provides input nodes to convert media inputs into tensors, enabling AI processing in real-time workflows.
Converts audio inputs into tensors, enabling real-time audio processing in ComfyStream workflows.
Converts image inputs into tensor format for computation in real-time AI workflows.
A passthrough node that specifies a reference image for real-time AI workflows, such as LivePortrait.
## Output Nodes
ComfyStream provides output nodes to convert tensors back into media formats for real-time streaming.
Converts audio tensors into audio frames.
Converts image tensors into image frames.
## Task-Specific Nodes
Most task-specific nodes are part of the StreamPack, but the ComfyStream package includes a few for example purposes.
Shifts the pitch of an input audio stream, demonstrating audio
transformations in real-time workflows.
Check out the [Example Workflow](https://github.com/livepeer/comfystream/blob/main/workflows/comfystream/audio-tensor-utils-example-api.json) to see the `PitchShifter` in action.
# Feature Bank
Source: https://docs.comfystream.org/technical/custom-nodes/stream-pack/feature-bank
Improve temporal consistency of real-time using feature banks.
This node is implemented based on the research paper ["Looking Backward:
Streaming Video-to-Video Translation with Feature
Banks"](https://arxiv.org/pdf/2405.15757). See the
[Acknowledgments](#acknowledgments) section for details.
## Introduction
The **Feature Bank** node improves **temporal consistency**—the visual coherence between consecutive frames—in generative video models by **caching and reusing features** from earlier frames. This reduces flickering and enables smoother transitions, resulting in a more stable visual flow. By controlling how much these cached features influence each frame, it also enables fine-grained **stylistic consistency**—especially valuable in **real-time contexts** where visual stability is essential.
## Visual Comparison
The video below compares a [baseline workflow](https://github.com/livepeer/ComfyUI-Stream-Pack/blob/main/examples/workflows/sd15-tensorrt-api.json) with a [real-time workflow using Feature Bank](https://github.com/livepeer/ComfyUI-Stream-Pack/blob/main/examples/workflows/sd15-tensorrt-feature-bank-api.json). Compared to the baseline, the Feature Bank provides:
* **Reduced Flickering** – Backgrounds and recurring objects appear more stable.
* **Improved Temporal Consistency** – Objects retain consistent form, texture, and color across frames.
* **Smoother Motion** – Transitions feel more natural, with fewer visual jumps or jitters.
## How It Works
The Feature Bank node enhances temporal consistency by integrating with the [self-attention layers](https://huggingface.co/docs/transformers/en/attention) of the diffusion model. It caches features from previous frames and re-injects them into future ones—stabilizing transitions without altering the underlying model.
It performs three core steps:
1. **Caching** – Stores attention features at regular intervals.
2. **Filtering** – Selects relevant cached features using cosine similarity.
3. **Injection** – Blends selected features into the current frame based on the configured strength.
The node supports any self-attention-based model (e.g., Stable Diffusion) and works with both **text-to-image (T2I)** and **image-to-image (I2I)** workflows.
## Adding the Node
An example T2I workflow using the Feature Bank node is available
[here](https://github.com/livepeer/ComfyUI-Stream-Pack/blob/main/examples/workflows/sd15-tensorrt-feature-bank-api.json).
Ensure the [StreamPack](https://github.com/livepeer/ComfyUI-Stream-Pack)
custom nodes are installed in your ComfyUI setup. Follow the [installation
instructions](/technical/custom-nodes/stream-pack/overview#installation) for
a step-by-step guide.
Right-click on an empty area of the canvas and choose **Add Node**.
Search for `Feature Bank` under the `StreamPack/model_patches/unet`
category.
Insert the node between your **Model Loader** and **Sampling Node**.
Wire up the inputs and outputs. The node runs automatically using the
parameters you set.
## Parameters
The following parameters can be configured in the Feature Bank node:
* **Start with a moderate cache interval.** Lower values improve consistency but increase memory usage.
* **Set injection strength gradually.** Start around 0.5 to balance coherence and novelty.
* **Match similarity threshold to your content.** Higher values are best for static backgrounds or subjects.
* **Use fewer cached frames for fast-changing scenes,** and more for looping or stable content.
* **Monitor memory use,** especially with low cache intervals or high resolutions.
* **Experiment creatively.** Small tweaks to injection or similarity can unlock unique visual styles.
**Feature Cache Interval**: Determines how often features from the attention
layers are cached (e.g., every 4 frames). Lower values improve consistency but
increase memory usage.
**Use Feature Injection**: Enables or disables the influence of cached
features. Set to `true` to activate the Feature Bank.
**Feature Injection Strength**: Controls how much influence cached features
have on the current frame (0 = none, 1 = full). A blended approach is possible
with values between 0 and 1.
**Feature Similarity Threshold**: Sets the minimum cosine similarity for
reusing cached features. Higher values ensure only relevant features are
reused.
**Feature Bank Max Frames**: Maximum number of past frames to cache. Older
frames are automatically discarded.
## Strengths and Limitations
While the Feature Bank offers major stability and stylistic benefits, it also introduces a few trade-offs depending on your use case.
### Strengths
* **Improved Temporal Consistency** – Reduces flickering and stabilizes frame-to-frame transitions.
* **Stylistic Control** – Adjust the influence of past frames for consistent or evolving styles.
* **Creative Exploration** – Enables novel stylistic variations by selectively injecting past features.
### Limitations
* **Reduced Novelty** – High injection strength may overly rely on past frames, limiting new details.
* **Memory Usage** – Storing features increases memory consumption, especially at high resolution.
* **Parameter Tuning** – May require iteration to balance consistency and variation effectively.
## Acknowledgments
This node is based on the research paper ["Looking Backward: Streaming Video-to-Video Translation with Feature Banks"](https://arxiv.org/pdf/2405.15757) by **Liang et al. 2023**. We thank the original authors for their contributions and collaboration in adapting this concept for **ComfyUI**.
If you use the Feature Bank node in your research or refer to the paper’s results, please cite the following:
```bibtex theme={null}
@article{liang2024looking,
title={Looking Backward: Streaming Video-to-Video Translation with Feature Banks},
author={Liang, Feng and Kodaira, Akio and Xu, Chenfeng and Tomizuka, Masayoshi and Keutzer, Kurt and Marculescu, Diana},
journal={arXiv preprint arXiv:2405.15757},
year={2024}
}
@article{kodaira2023streamdiffusion,
title={StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation},
author={Kodaira, Akio and Xu, Chenfeng and Hazama, Toshiki and Yoshimoto, Takanori and Ohno, Kohei and Mitsuhori, Shogo and Sugano, Soichi and Cho, Hanying and Liu, Zhijian and Keutzer, Kurt},
journal={arXiv preprint arXiv:2312.12491},
year={2023}
}
```
# Overview
Source: https://docs.comfystream.org/technical/custom-nodes/stream-pack/overview
Discover StreamPack's custom nodes for enhancing video, audio, and data workflows with real-time performance.
The ComfyUI-Stream-Pack is in active development and not yet stable. We
welcome [your
feedback](https://github.com/livepeer/ComfyUI-Stream-Pack/issues/new) as we
continue to improve it.
The ComfyUI-Stream-Pack provides a [custom node pack](https://github.com/livepeer/ComfyUI-Stream-Pack) for ComfyUI, designed specifically for building real-time video, audio, and data pipelines. These tools enhance output quality, optimize performance, and include benchmarking and profiling features to help users create and refine efficient AI-driven workflows.
## Installation
We are actively working on publishing the StreamPack as a ComfyUI registry
node. Once available, you will be able to install it directly via the ComfyUI
Manager for a more streamlined setup process.
To use the StreamPack nodes, you need to install the StreamPack repository in your ComfyUI setup.
Open a terminal and navigate to the `custom_nodes` directory in your ComfyUI workspace:
```bash theme={null}
cd /workspace/ComfyUI/custom_nodes
```
Clone the StreamPack repository from GitHub:
```bash theme={null}
git clone https://github.com/livepeer/ComfyUI-Stream-Pack.git
cd ComfyUI-Stream-Pack
```
Activate your ComfyUI environment and install the required dependencies:
```bash theme={null}
conda activate comfyui
pip install -r requirements.txt
```
Restart your ComfyUI server to load the new nodes.
## StreamPack Nodes Overview
We also recommend **third-party nodes** from other teams that work well with
real-time workflows. While not maintained by the ComfyStream team, these nodes
are highly compatible and can enhance your pipelines. Learn more on the
[Third-Party Nodes](/technical/custom-nodes/third-party-nodes) page.
StreamPack includes a variety of custom nodes grouped into four main categories: **Video Quality Nodes**, which enhance video output; **Performance Nodes**, which optimize real-time workflows; **Benchmarking and Profiling Tools**, which help analyze and refine workflow efficiency; and **Task-Specific Nodes**, which provide specialized functionality for real-time video workflows.
* [Feature Bank](/technical/custom-nodes/stream-pack/feature-bank): Improves video quality by using information from previous frames to reduce flickering and ensure smoother transitions.
* [Super Resolution](/technical/custom-nodes/stream-pack/super-resolution): A high-performance upscaling node that uses neural network models like FSRCNN and EDSR to enhance video resolution in real-time, balancing speed and quality.
The nodes in this category are still under development and will be released
soon.
The nodes in this category are still under development and will be released
soon.
* **Facial Mesh**: A MediaPipe Face Mesh node that detects and tracks facial landmarks. It can be used as an additional ControlNet input to enhance facial tracking in real-time workflows.
# Super Resolution
Source: https://docs.comfystream.org/technical/custom-nodes/stream-pack/super-resolution
Upscale video frames in real-time using GPU acceleration.
## Introduction
The **Super Resolution** node **upscales** video frames with pre-trained models. The process leverages GPU acceleration through [OpenCV](https://opencv.org/) compiled with CUDA support, ensuring exceptional speed ideal for real-time applications. When using workflows with base Stable Diffusion models optimized for **512x512**, higher resolutions can compromise efficiency and FPS. In these scenarios, a dedicated super-resolution node allows upscaling without significantly affecting performance or quality.
## Setup Method
The node is optimized for high-speed upscaling and requires OpenCV with CUDA support. By running ComfyStream with the **[Docker setup](/technical/get-started/install#install-with-docker)**, you get a precompiled package from [ComfyUI-Stream-Pack/releases](https://github.com/JJassonn69/ComfyUI-Stream-Pack/releases/tag/v1.0) included in the Docker image. This eliminates the need for manual compilation.
If you prefer to install OpenCV with CUDA support manually, follow the guide in article [Installing OpenCV with CUDA Support](https://medium.com/@juancrrn/installing-opencv-4-with-cuda-in-ubuntu-20-04-fde6d6a0a367).This guides you through the multiple steps and prerequisites needed to successfully compile OpenCV package.
You can use the following code block as a reference, modify the paths and variables to match your system.
```bash [expandable] theme={null}
# Install system libraries required for compiling opencv
apt update && apt install -yqq \
git \
wget \
nano \
socat \
libsndfile1 \
build-essential \
llvm \
tk-dev \
cmake \
libgflags-dev \
libgoogle-glog-dev \
libjpeg-dev \
libavcodec-dev \
libavformat-dev \
libavutil-dev \
libswscale-dev
# Clone OpenCV repositories
git clone --depth 1 --branch 4.11.0 https://github.com/opencv/opencv.git
git clone --depth 1 --branch 4.11.0 https://github.com/opencv/opencv_contrib.git
# Create build directory
mkdir -p opencv/build
# Create a toolchain file with absolute path
cat > custom_toolchain.cmake << EOF
# Custom toolchain file to exclude Conda paths
# Set system compilers
set(CMAKE_C_COMPILER "/usr/bin/gcc")
set(CMAKE_CXX_COMPILER "/usr/bin/g++")
# Set system root directories
set(CMAKE_FIND_ROOT_PATH "/usr")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
# Explicitly exclude Conda paths
list(APPEND CMAKE_IGNORE_PATH
"/workspace/miniconda3"
"/workspace/miniconda3/envs"
"/workspace/miniconda3/envs/comfystream"
"/workspace/miniconda3/envs/comfystream/lib"
)
# Set RPATH settings
set(CMAKE_SKIP_BUILD_RPATH FALSE)
set(CMAKE_BUILD_WITH_INSTALL_RPATH FALSE)
set(CMAKE_INSTALL_RPATH "/usr/local/lib:/usr/lib/x86_64-linux-gnu")
set(PYTHON_LIBRARY "/workspace/miniconda3/envs/comfystream/lib/")
set(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
EOF
# Set environment variables for OpenCV
echo 'export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc
# Build and install OpenCV with CUDA support
cd opencv/build
cmake \
-D CMAKE_TOOLCHAIN_FILE=/workspace/custom_toolchain.cmake \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D WITH_CUDA=ON \
-D WITH_CUDNN=ON \
-D WITH_CUBLAS=ON \
-D WITH_TBB=ON \
-D CUDA_ARCH_LIST="8.0+PTX" \
-D OPENCV_DNN_CUDA=ON \
-D OPENCV_ENABLE_NONFREE=ON \
-D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda \
-D OPENCV_EXTRA_MODULES_PATH=/workspace/opencv_contrib/modules \
-D PYTHON3_EXECUTABLE=/workspace/miniconda3/envs/comfystream/bin/python3.11 \
-D PYTHON_INCLUDE_DIR=/workspace/miniconda3/envs/comfystream/include/python3.11 \
-D PYTHON_LIBRARY=/workspace/miniconda3/envs/comfystream/lib/libpython3.11.so \
-D HAVE_opencv_python3=ON \
-D WITH_NVCUVID=OFF \
-D WITH_NVCUVENC=OFF \
..
make -j$(nproc)
make install
ldconfig
```
This will install the opencv package **cv2** in `/usr/local/lib/python3.11/site-packages`.
Since the opencv compiles with a specific version of numpy make sure to have the same numpy version in your environment.
After the compilation, you can verify that OpenCV is installed correctly by running:
```bash theme={null}
python3 -c "import cv2; print(cv2.cuda.getCudaEnabledDeviceCount())"
```
This should output `1` or higher depending on the number of CUDA-enabled devices in your machine if installation was successful.
## Adding the Node
An example upscaling workflow using the Super Resolution node is available
[here](https://github.com/livepeer/ComfyUI-Stream-Pack/blob/main/examples/workflows/super-res-fscnn-upscale-2x-gpu-api.json).
Ensure the [StreamPack](https://github.com/livepeer/ComfyUI-Stream-Pack) custom nodes are installed in your ComfyUI setup. Follow the [installation instructions](/technical/custom-nodes/stream-pack/overview#installation) for a step-by-step guide.
Right-click on an empty area of the canvas and choose **Add Node**.
Search for `SuperResolution` under the `StreamPack/` category.
There are two nodes under SuperResolution: `SR Model Loader` and `SR Upscale`. You need both to upscale, so place them on the canvas.
* In `SR Model Loader`, you'll see widgets to select the upscaling model and scale factor.
* Connect the output of `SR Model Loader` to the input of `SR Upscale`.
* In the `SR Upscale` node, toggle the `use CUDA` widget to `true` to enable CUDA acceleration.
* Connect an image to the input of `SR Upscale` and a preview node to the output.
## Parameters
The following parameters control how the Super Resolution models behave:
* **Select the Model for Upscaling** – Choose the model based on desired upscaling quality and performance; refer to the comparison table above.
* **Scaling Factor Selection** – Consider the required output resolution when choosing the scale factor. A higher scaling factor increases processing time, potentially reducing real-time performance.
* **Using CUDA Acceleration** – While CPU can be used for upscaling, it is significantly slower. Use CUDA acceleration for optimal performance.
### SR Model Loader Node
**Model Type**: Select the upscaling model to use. The quality and performance
vary by model. Models are downloaded and loaded automatically when selected in
the canvas. Below is a basic comparison of the models.
| Model | Architecture | Features | Best For | Speed | Quality |
| ---------------- | ------------------------ | ----------------------------- | ---------------------------- | ----- | ------- |
| **FSRCNN-small** | Lightweight CNN | Fast, minimal memory use | Real-time processing, mobile | ★★★★★ | ★★ |
| **FSRCNN** | CNN with larger features | Good balance of speed/quality | General purpose | ★★★★ | ★★★ |
| **ESPCN** | Sub-pixel convolutions | Efficient upscaling at end | Text/line drawings | ★★★★ | ★★★ |
| **VDSR** | Very deep CNN | Better edge reconstruction | Detailed images with edges | ★★★ | ★★★★ |
| **LapSRN** | Laplacian pyramid | Progressive upscaling | Sharp edges, details | ★★★ | ★★★★ |
| **EDSR** | Deep residual network | Most parameters, best quality | Maximum detail | ★★ | ★★★★★ |
**Scale Factor**: Select the upscaling factor. The image will be upscaled by
this factor. For example, if the input image is `512x512`, the output will be
`1024x1024` with a scale factor of `2`.
### SR Upscale Node
**Use CUDA**: Toggle to `true` to enable CUDA acceleration for upscaling.
## Strengths and Limitations
While the Super Resolution node offers significant performance benefits compared to other upscaling methods, it also introduces a few trade-offs depending on your use case.
### Strengths
* **Wide suite of supported models** – Option to choose from multiple models (FSRCNN, ESPCN, LapSRN, EDSR) with different quality/speed tradeoffs.
* **Modular Design** – Once a model is loaded, it can be reused across multiple upscaling operations.
* **Fast performance via CUDA** – Configurable CUDA acceleration for optimal performance.
* **Multiple Scale Factors** – Support for 2x, 3x, and 4x upscaling.
### Limitations
* **Setup Complexity** – The node requires a version of OpenCV with CUDA support, which may require additional setup, since the CUDA is only available for NVIDIA GPUs, it will not work on other GPUs.
## Acknowledgments
This nodepack implements models originally created by:
* [FSRCNN (Dong et al., 2016)](https://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html) – [GitHub Repository](https://github.com/ryanontheinside/FSRCNN_Tensorflow)
* [EDSR (Lim et al., 2017)](https://arxiv.org/abs/1707.02921) – [GitHub Repository](https://github.com/ryanontheinside/EDSR_Tensorflow)
* [ESPCN (Shi et al., 2016)](https://arxiv.org/abs/1609.05158) – [GitHub Repository](https://github.com/ryanontheinside/TF-ESPCN)
* [LapSRN (Lai et al., 2017)](https://arxiv.org/abs/1710.01992) – [GitHub Repository](https://github.com/ryanontheinside/TF-LapSRN)
# Third-Party Nodes
Source: https://docs.comfystream.org/technical/custom-nodes/third-party-nodes
Discover recommended third-party ComfyUI nodes to enhance your real-time AI workflows.
export const GitHubFooter = ({url, float = "right"}) =>
;
The ComfyStream team has identified several third-party nodes that are highly compatible with ComfyStream and can enhance your **real-time AI workflows**. While these nodes are **not maintained by the ComfyStream team**, they are recommended for their **functionality and compatibility**.
## Optimization Nodes
Nodes in this category are designed to improve performance and reduce latency in real-time workflows.
Optimizes stable diffusion performance by generating a static TensorRT engine based on Stable Diffusion parameters. This reduces inference time, making it ideal for low-latency workflows.
Explore the [Example Workflow](https://gist.github.com/yondonfu/592a04e075e790981cd401124e0d84e4#file-api-workflow-json).
## Node Packs
Node packs include collections of nodes designed for specific use cases, such as real-time video processing and control.
A suite of nodes for real-time video processing, including tools for parameter adjustments, motion detection, and FPS monitoring. Ideal for interactive workflows with live input and output.
Explore the [Example Workflows](https://github.com/ryanontheinside/ComfyUI_RealTimeNodes/tree/main/examples).
## Task-Specific Nodes
Task-specific nodes are designed for specialized workflows, such as depth estimation or object tracking.
Generates depth maps from RGB frames with low latency and efficient VRAM usage. Useful for workflows requiring depth-based effects or object tracking.
Explore the [Example Workflow](https://github.com/livepeer/comfystream/blob/main/workflows/comfystream/depth-anything-v2-trt-example-api.json).
Liveportrait implementation with TensorRT engines for faster inference. Follow the documentation in the github repository to first build the docker image and engines.
These engines will be saved locally and will be reused when running a FasterLivePortrait workflow.
Explore the [Example Workflow](https://github.com/livepeer/comfystream/blob/main/workflows/comfystream/live-portrait-daydream.json).
# Install ComfyStream
Source: https://docs.comfystream.org/technical/get-started/install
Learn how to install ComfyStream using Docker or as a custom node for ComfyUI.
ComfyStream is available as a custom node through the [ComfyUI Manager](https://registry.comfy.org/nodes/comfystream) or as a Docker image. Follow the method that best suits your setup.
You can also install Comfystream by cloning the repository into your ComfyUI/custom\_nodes folder — see [Manual Installation](/technical/get-started/install#manual-installation-cloning-repository) for details.
Install ComfyStream directly into an existing ComfyUI setup.
Run ComfyStream and ComfyUI together in a prebuilt Docker container.
***
## Install with ComfyUI Manager
If you already have ComfyUI installed, the easiest way to install ComfyStream is via the built-in **ComfyUI Manager**.
Download and install [ComfyUI](https://github.com/hiddenswitch/ComfyUI?tab=readme-ov-file#installing) if you haven't already.
Currently **comfystream** works only with the [hiddenswitch
fork](https://github.com/hiddenswitch/ComfyUI) of ComfyUI, the team is
actively working add full support for [official
ComfyUI](https://github.com/comfyanonymous/ComfyUI).
ComfyUI must be run with frontend version `v1.24.2` or older. You can do this by launching ComfyUI with the flag `--front-end-version Comfy-Org/ComfyUI_frontend@v1.24.2`
Follow the [ComfyUI Manager installation guide](https://docs.comfy.org/essentials/core-concepts/nodes#installing-the-manager) if you haven't already.
1. Launch **ComfyUI**.
2. Open the **Manager** tab.
3. Search for **ComfyStream** and click **Install**.
Restart your ComfyUI server to load the new custom node.
## Manual Installation
If you prefer more control over the installation process, you can install ComfyStream manually using one of the following methods:
Python 3.12 or greater is required
The following commands install the latest version of ComfyUI and ComfyStream
```bash theme={null}
pip install comfy-cli
comfy install
comfy node registry-install comfystream
```
After completing the installation, start ComfyUI with the following command
```bash theme={null}
comfy launch -- --listen 0.0.0.0 --port 8188 --front-end-version Comfy-Org/ComfyUI_frontend@v1.24.2
```
This will start the ComfyUI server with ComfyStream installed.
If you prefer to install ComfyStream manually by cloning the repository instead of using the Manager, follow these steps:
```bash theme={null}
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI
pip -r requirements.txt
cd custom_nodes
git clone https://github.com/livepeer/comfystream.git
cd comfystream
python install.py
```
After completing the installation, navigate to the root of the ComfyUI directory.
```bash theme={null}
cd ComfyUI
python main.py --listen --front-end-version Comfy-Org/ComfyUI_frontend@v1.24.2
```
This will start the ComfyUI server with ComfyStream installed.
***
## Install with Docker
Run ComfyStream in a prebuilt Docker environment, either on your own GPU or a cloud server.
Run ComfyStream locally with your own GPU.
Deploy ComfyStream on a remote GPU using RunPod or Ansible.
### Local GPU
If you have a compatible GPU on Windows or Linux, you can run ComfyStream locally via Docker.
#### Prerequisites
First, install the required system software:
* [Docker
Engine](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository)
* [NVIDIA CUDA
Toolkit](https://developer.nvidia.com/cuda-12-8-1-download-archive?target_os=Linux\&target_arch=x86_64\&Distribution=Ubuntu\&target_version=24.04\&target_type=runfile_local)
* [NVIDIA Container
Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
* From a new command prompt, run: `wsl --install` - Update WSL if
needed: `wsl.exe --update`
Open a new **Ubuntu (WSL)** tab from Terminal.
Inside of WSL, install [NVIDIA CUDA
Toolkit](https://developer.nvidia.com/cuda-12-8-1-download-archive?target_os=Linux\&target_arch=x86_64\&Distribution=WSL-Ubuntu\&target_version=2.0\&target_type=runfile_local)
Install [Docker
Desktop](https://docs.docker.com/get-started/introduction/get-docker-desktop/).
Ensure Docker Desktop is set to use the WSL 2 Engine in settings.
Inside WSL: `docker ps` should return a result.
Proceed with the next steps in your WSL terminal.
#### Run the Docker Container
These folders store your models and generated outputs. Docker mounts them into the container.
```bash Linux theme={null}
mkdir -p ~/models/ComfyUI--models ~/models/ComfyUI--output
```
```batch Command Line theme={null}
mkdir %USERPROFILE%\models\ComfyUI--models %USERPROFILE%\models\ComfyUI--output
```
```powershell PowerShell theme={null}
New-Item -ItemType Directory -Path "$env:USERPROFILE\models\ComfyUI--models", "$env:USERPROFILE\models\ComfyUI--output"
```
```bash theme={null}
docker pull livepeer/comfystream:stable
```
If using Windows, ensure Docker Desktop is running first.
```bash theme={null}
docker run -it --gpus all \
-p 8188:8188 \
-p 8889:8889 \
-p 5678:5678 \
-p 3000:3000 \
-v ~/models/ComfyUI--models:/workspace/ComfyUI/models \
-v ~/models/ComfyUI--output:/workspace/ComfyUI/output \
livepeer/comfystream:stable --download-models --build-engines --server
```
Available flags:
* `--download-models` downloads some default models
* `--build-engines` optimizes the runtime for your GPU
* `--server` starts ComfyUI server (accessible on port 8188)
* `--api` enables the API server
* `--ui` starts the ComfyStream UI (accessible on port 3000)
* `--use-volume` should be used with a mount point at /app/storage. It is used during startup to save/load models and compiled engines to a host volume mount for persistence
The `--download-models` and `--build-engines` flags are only needed the first time (or when adding new models).
Open your browser and go to [http://localhost:8188](http://localhost:8188) to start using ComfyUI with ComfyStream.
The ComfyStream UI is available at [http://localhost:3000](http://localhost:3000) where you can start live streams directly by keeping the stream URL as `http://localhost:8889` and selecting a workflow.
### Remote GPU
If you don't have a local GPU, you can run ComfyStream on a cloud server. Choose between a managed deployment with **RunPod** or manual setup using **Ansible**.
#### Run with RunPod
RunPod provides a simple one-click deployment of ComfyStream in a managed container environment — perfect for testing or avoiding manual setup.
Use this template to get started:
[livepeer-comfystream](https://console.runpod.io/deploy?template=w01m180vxx\&ref=u8tlskew)
RunPod does not persist pod data by default. To ensure models and engines persist across pod restarts, use the RunPod template [livepeer-comfystream-volume](https://console.runpod.io/deploy?template=j4p1g7t5vs\&ref=u8tlskew)
This template uses the `--use-volume` flag to save all models and engines to the mount path `/app/storage`. A network volume is required.
When using the persistent template, create and attach a network volume.
Select a suitable GPU (e.g. RTX 4090), then click **Deploy On-Demand**.
Use the **Logs** button to watch the container spin up.
Once running, click **Connect** from the pod dashboard.
Click the **HTTP Service → :8188** link. It will open in a new browser tab.
To access the ComfyStream UI, use the TCP ports to form a URL beginning with `https://` followed by the IP:port for port 3000.
You can start a live stream directly from the ComfyStream UI by keeping the stream URL as `http://localhost:8889` and selecting a workflow.
#### Deploy with Ansible
Use [Ansible](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_intro.html) to deploy ComfyStream to your own cloud VM (like AWS, GCP, Azure, or TensorDock). Great if you want more control or a repeatable, fully automated setup.
Provision a VM with at least:
* **GPU**: 20GB VRAM
* **RAM**: 16GB
* **CPU**: 4 vCPUs
* **Disk**: 100GB
Allow incoming traffic on:
* **22** (SSH – internal port)
* **8189** (ComfyStream HTTPS – internal port)
Ensure your cloud provider maps these to **public ports**. Internal port **8189** is proxied to **8188**, which serves the ComfyUI interface.
Install Ansible on your **local machine** (not the VM). See the [official installation guide](https://docs.ansible.com/ansible/latest/installation_guide/index.html).
```bash theme={null}
git clone https://github.com/livepeer/comfystream.git
cd comfystream/scripts/ansible
```
No need to install anything on the VM — Ansible sets it all up over SSH.
Open `inventory.yml` and add your VM's public IP address.
Edit the `comfyui_password` value in `plays/setup_comfystream.yaml`.
This sets the password used to access ComfyUI in your browser.
```bash theme={null}
ansible-playbook -i inventory.yaml plays/setup_comfystream.yaml
```
If you're not using a root user, add `--ask-become-pass` to enter your sudo password.
The playbook pulls a \~20GB Docker image on the VM. This may take a while on first run.
If needed, SSH into the VM and run:
```bash theme={null}
docker pull livepeer/comfystream:stable
```
Helpful if you're unsure whether the download is progressing during first-time setup.
Visit `https://:` in your browser.
***
## Accessing ComfyStream
1. Click the **ComfyStream** menu button in ComfyUI:
2. Open **Server Settings** to verify that ComfyStream is configured to bind to the correct interface and port.
For remote environments like RunPod, set the Host to `0.0.0.0`.
3. Click **Save** to apply the settings.
4. Open the **ComfyStream** menu again and click **Start ComfyStream Server**. Wait for the server status indicator to turn green.
You can monitor ComfyStream server logs in the ComfyUI log terminal tab.
5. Click **Open ComfyStream UI** to launch it in a new tab. Alternatively, double-click the node graph and search for **ComfyStream UI Preview**.
## Troubleshooting
If you encounter a `toomanyrequests` error while pulling the Docker image, either
wait a few minutes or provide your Docker credentials when running the playbook:
```bash theme={null}
ansible-playbook -i inventory.yaml plays/setup_comfystream.yaml -e "docker_hub_username=your_dockerhub_username docker_hub_password=your_dockerhub_pat"
```
# Introduction
Source: https://docs.comfystream.org/technical/get-started/introduction
Build real-time AI video and audio workflows using the ComfyStream toolkit for ComfyUI.
## Overview
**ComfyStream is a custom node** that adds powerful real-time video and audio capabilities to [ComfyUI](https://www.comfy.org/), making it easy to build interactive, AI-powered media workflows. It extends ComfyUI with specialized tools for streaming, live processing, and on-the-fly workflow updates, including:
* **[ComfyStream](https://github.com/livepeer/comfystream)** – A custom node that streams audio and video from your webcam and microphone into ComfyUI for real-time AI processing, then returns the processed output.
* **[ComfyUI-Stream-Pack](https://github.com/livepeer/ComfyUI-Stream-Pack)** – A collection of custom nodes designed to support advanced real-time audio and video workflows.
## Getting Started
To get started, follow the installation guide below or explore the stream pack for additional nodes.
Step-by-step instructions to install ComfyStream and start creating
real-time workflows.
A collection of custom nodes for building real-time audio and video
workflows.
## How It Works
ComfyStream enables real-time processing of audio and video streams by integrating a [WebRTC](https://webrtc.org/) server for low-latency, bidirectional communication, a custom tensor-based pipeline for converting media frames to and from tensors, and ComfyUI's [EmbeddedComfyClient](https://github.com/hiddenswitch/ComfyUI/blob/ffc1912eff10cf57aec4fee358a5b6946e74a9e5/comfy/client/embedded_comfy_client.py#L123) for AI inference.
### Data Flow Overview
Here's how the system processes live audio and video end-to-end:
1. **Input:** WebRTC receives video, audio, and control data from the client.
2. **Workflow Injection:** The pipeline dynamically modifies the ComfyUI workflow by replacing standard input/output nodes with custom tensor nodes.
3. **Inference:** The EmbeddedComfyClient processes incoming tensors in real-time using the updated workflow.
4. **Output Conversion:** Processed tensors are converted back to video and audio, and streamed back to the client via WebRTC.
5. **Live Control:** A control channel allows the client to update the workflow or modify parameters on the fly, without restarting the session.
This high-level overview is visualized below:
```mermaid theme={null}
flowchart TD
A[Client Input] --> B[WebRTC Receiver]
B --> C[Media to Tensor Conversion]
C --> D[Embedded ComfyUI Client]
D --> E[Tensor to Media Conversion]
E --> F[WebRTC Sender]
subgraph Control Channel
A2[Live Workflow Updates] --> D
end
```
# Run First Workflow
Source: https://docs.comfystream.org/technical/get-started/quickstart
Run your first real-time AI workflow with ComfyStream and see it live in Comfystream UI.
After you have successfully installed ComfyStream by following the [installation guide](/technical/get-started/install), you can run your first AI workflow.
## Prerequisites
Download [example workflows](https://github.com/livepeer/comfystream/tree/main/workflows/comfystream) from the ComfyStream GitHub repository to test your setup.
## Accessing ComfyStream UI
When you click the ComfyStream node and select **ComfyStream UI Preview** in ComfyUI, it will open a window that looks similar to the image below:
If you prefer to run workflows in a new tab rather than inside ComfyUI, click **Open ComfyStream UI** from the [ComfyStream Menu](/technical/get-started/install#accessing-comfystream).
This will open a similar dialog box in a new browser tab.
Opening the UI will ask for permission to access your video and audio so it
can stream your webcam and microphone. You need to provide access to use
ComfyStream.
## Running Your First Workflow
ComfyStream currently supports only the API workflow format. To export a
workflow in this format, use **Workflow > Export (API)** in ComfyUI. Support
for the native ComfyUI format is under development and will be available in a
future release.
In the ComfyStream UI, click **Choose files** and select one of the
previously downloaded workflows.
Click **Start Stream** to start the workflow. Once the stream starts, you
will see **Starting Stream..** at the bottom of the page. Depending on the
workflow complexity, it may take some time to start. Please wait patiently.
Once the stream starts, you will see the AI effects being applied in
real-time.
To see the similar preview, copy the
[example-workflow.json](https://github.com/livepeer/comfystream/raw/main/workflows/comfystream/sd15-multi-cnet-depthanything-face-api.json)
to local storage and launch it from comfystream UI as shown above.
## Troubleshooting
For advanced users, you can check the logs in the ComfyStream server to see if
everything is loading properly without any major errors.
The ComfyStream server logs are in the ComfyUI log window. Open the bottom panel using the **Toggle Bottom Panel** button at the top-right of the interface.
If you encounter the following error when starting a stream on RunPod:
```bash theme={null}
Error proxying offer: Cannot connect to host localhost:8889 ssl:default [Connect call failed ('127.0.0.1', 8889)]
```
Please check the [ComfyStream Server Settings](/technical/get-started/install#accessing-comfystream) to ensure ComfyStream is running on host `0.0.0.0` when using RunPod. The default **Stream URL** in ComfyStream UI (`http://127.0.0.1:8889`) is correct when using the [RunPod template](/technical/get-started/install#run-with-runpod).
# Stream using OBS
Source: https://docs.comfystream.org/technical/get-started/streaming
Stream your real-time AI workflows using OBS and browser source from ComfyStream UI.
After setting up ComfyStream and running your first workflow, you can use OBS Studio to capture and broadcast your AI-enhanced streams to your favourite platforms — whether it's YouTube, Twitch, or distributed infrastructure providers like Livepeer.
## Prerequisites
Download and install [OBS Studio](https://obsproject.com/) for your
operating system. OBS (Open Broadcaster Software) is a free and open-source
software for video recording and live streaming.
Before streaming with OBS, you need to have a running ComfyStream workflow.
If you haven't done this yet, follow the [run your first
workflow](/technical/get-started/quickstart#running-your-first-workflow)
guide first.
## Setting Up OBS with ComfyStream
Click the icon in the bottom right corner of the ComfyStream video output to open the stream in a new fullscreen browser tab.
This dedicated view provides a clean video source without any UI elements, perfect for capturing in OBS.
Copy the URL from the new browser tab. You'll need this to add the stream as a source in OBS.
1. Open **OBS Studio**.
2. In the **Sources** panel, click the **+** button.
3. Select **Browser** from the list of source types.
4. Name your source (e.g., "ComfyStream Output") and click **OK**.
5. Paste the URL you copied from the ComfyStream fullscreen tab into the **URL** field.
6. Set the **Width** and **Height** fields to **1024** to match the resolution of the ComfyStream video.
7. Click **OK** to confirm and add the browser source.
You may need to adjust other browser source settings depending on your specific requirements and system performance.
Once added, you can resize and position the ComfyStream browser source within your OBS scene as needed.
You can also add additional sources like webcams, images, or text overlays to enhance your stream.
## Streaming to Platforms
Use the same OBS setup to stream to platforms like:
* YouTube Live
* Twitch
* Facebook Live
* Custom RTMP servers.
Each platform provides a unique **stream key** and **RTMP URL**. Configure these in **OBS Settings →
Stream** to start broadcasting.
Livepeer is a distributed video streaming network offering cost-effective
streaming infrastructure. Follow the [Livepeer
Documentation](https://docs.livepeer.org/developers/guides/stream-via-obs)
for detailed instructions on how to:
1. Create a stream on **Livepeer
Studio**.
2. Configure OBS with your Livepeer **Stream Key** and **RTMP
ingest** URL.
3. Start broadcasting your **ComfyStream AI-enhanced video**.
## Troubleshooting
If you experience performance issues, try reducing the resolution or frame
rate in both ComfyStream and OBS settings. Browser sources can be
resource-intensive, especially with AI-generated content.
# Changelog
Source: https://docs.comfystream.org/technical/references/changelog
Track the latest updates, features, improvements, and bug fixes for the ComfyStream toolkit.
## BYOC Workflows, ComfyUI-Manager, and Performance Enhancements
View the full detailed changelog on [GitHub](https://github.com/livepeer/comfystream/releases/tag/v0.1.6).
This release introduces significant enhancements for production deployments with pytrickle support for Bring Your Own Container (BYOC) workflows, ComfyUI-Manager integration with performance-optimized blacklist loading, and improved model efficiency using TAESD safetensors and all documentation links now points correct site.
### New Features
* **BYOC Workflows with Pytrickle:** Integrated pytrickle support enabling ComfyStream workflows as Bring Your Own Containers
* **ComfyUI-Manager Integration:** Added ComfyUI-Manager with blacklist loading for improved performance and easier custom node management
* **FasterLivePortrait Support:** Enhanced opencv-cuda for compatilibity with Python 3.12 and resolved issues with FasterLivePortrait integration
* **Model Efficiency and Security:** Switched to TAESD safetensors format (replacing .pth files) for better model loading efficiency and security from pickled files.
### UI & Documentation
* **Refactored UI Popup:** Removed inline HTML/JS code for a cleaner, more maintainable codebase
* **Updated Documentation Links:** All references now point to the official documentation site
### Performance & Stability
* **Input Timeout Protection:** Added timeouts to LoadTensor and LoadAudioTensor nodes to prevent hanging
* **Enhanced Error Logging:** Improved exception logging for input-related issues
* **OpenCV-CUDA Compilation:** GitHub workflow for automated opencv-cuda compilation
* **Dependency Updates:**
* Upgraded pytrickle to v0.1.4
* Replaced deprecated pynvml with nvidia-ml-py
* Updated Next.js from 15.2.4 to 15.5.3
### Bug Fixes
* Fixed pyproject.toml metadata for ComfyUI registry compatibility
* Corrected sample rate in audio transcription workflows
* Added missing Cairo and Pango development libraries in Docker images
* Updated BYOC capability name in launch configurations
* Removed unnecessary GitHub workflow triggers
## Audio Driven Workflows and Text Outputs
View the full detailed changelog on [GitHub](https://github.com/livepeer/comfystream/releases/tag/v0.1.5).
This release adds support for audio driven workflows and text outputs. An example real-time text transcription demo workflow using Whisper has been added.
The docker ComfyUI workspace has been upgraded to v0.3.56. ComfyStream is updated to support newer versions of ComfyUI.
### New Features
* Support for audio driven workflows
* Text outputs via RTC data channel
* Real-time text transcription demo with Whisper
### Improvements
* Upgraded ComfyStream to support the latest ComfyUI version
* Python version bumped to 3.12 for compatibility
* Fixes for custom node imports, prompt conversion, and launch configurations
### New Contributors
* @doctorpangloss
## RTX 5090 Support and Infrastructure Improvements
View the full detailed changelog on [GitHub](https://github.com/livepeer/comfystream/releases/tag/v0.1.4).
This release adds support for NVIDIA's latest RTX 5090 GPUs and includes significant infrastructure improvements for better development experience and deployment flexibility. The base image has been updated to support the latest CUDA and TensorRT versions.
### New Hardware Support
* **RTX 5090 GPU Support:** Full compatibility with NVIDIA's latest Blackwell architecture
* **Updated CUDA Requirements:** Minimum supported driver version `NVIDIA 570.124.06` with `CUDA 12.8`
* **PyTorch Blackwell Support:** Updated PyTorch for optimal performance on RTX 5090
### Infrastructure & Developer Experience
* **Entrypoint Flexibility:** Added `--api` and `--ui` flags to entrypoint.sh for selective service startup
* **FasterLivePortrait Workflow:** New workflow documentation for FasterLivePortrait integration
* **Improved Container Stability:** Fixed devcontainer loading issues and conda environment installation
* **Better Volume Management:** Enhanced volume mount for persistent storage across container restarts
### Base System Updates
* **CUDA 12.8 and TensorRT 10.12:** Updated base to support newer compute capability GPUs.
* **Dependency Stability:** Pinned `numpy<2.0` for consistent behavior across environments
* **Network Configuration:** Fixed server manager host binding for proper network access
## MPEG Stream Egress and OBS Integration
View the full detailed changelog on [GitHub](https://github.com/livepeer/comfystream/releases/tag/v0.1.2).
This release introduces token-gated MPEG stream egress capabilities, enabling direct OBS integration for live streaming workflows. Along with significant stability improvements through dependency pinning and bug fixes.
### New Features
* **Token-Gated MPEG Stream Egress:** External MPEG stream endpoint with authentication for secure access
* **OBS Video Source Support:** Demo page and direct OBS integration for capturing ComfyStream output in live streaming setups
* **Separate OBS Tab:** Dedicated interface for OBS-specific functionality and configuration
### Stability & Performance
* **Dependency Pinning:** Pinned critical packages for improved stability:
* xformers==0.0.30
* pytorch==2.7.0
* onnxmltools==1.13.0
* **Client API Fixes:** Resolved `comfystream.client.update_prompts` lock issues and error handling
* **Build Process Optimization:** Streamlined Docker workflows by removing separate build processes
### Developer Experience
* **Contributing Guide:** Added comprehensive documentation for project contributors
* **Volume Mount Improvements:** Enhanced volume handling with proper --server flag qualification
* **CI/CD Enhancements:** Improved workflow triggers and build automation for ai-runner integration
## Multi Resolution, Upscaling and Pipeline Integration
Requires ComfyUI version [0.3.27](https://github.com/comfyanonymous/ComfyUI/releases/tag/v0.3.27) or older due to dependency constraints.View the full detailed changelog on [GitHub](https://github.com/livepeer/comfystream/compare/v0.0.5...v0.1.0).
This release introduces significant enhancements for visual quality and developer flexibility, including **multi-resolution support**, an **improved depth mapping model**, and a new **exportable pipeline class** for seamless integration into custom applications. Docker images are available at `livepeer/comfyui-base:v0.1.0` and `livepeer/comfystream:v0.1.0`.
### New Features
* **Multi-Resolution Support:** Dynamic resolution handling with precompiled TensorRT engine examples supporting combinations of width/height from 448px to 704px
* **Super Resolution Node:** Enhanced image quality through OpenCV-based super resolution
* **Exportable Pipeline Class:** Access the `comfystream.pipeline` class to integrate ComfyStream directly into your applications
### New Models
* **Depth Anything V2:** Integrated the large model variant with TensorRT compilation for improved depth mapping
### Developer Experience
* **Simplified Integration:** Refactored package structure to make ComfyStream easier to import and use in custom applications
## Enhanced Branding, Metrics & Developer Experience
View the full ComfyStream changelog on [GitHub](https://github.com/livepeer/comfystream/releases).
This release introduces the first official **ComfyStream brand**, real-time **metrics support**, and an **improved developer experience**, with continued enhancements to **quality and reliability**. Early adopters can now monitor workflow performance via Prometheus or API endpoints, while enjoying a smoother and more stable development experience.
### Documentation and Branding
We launched the first version of the **ComfyStream documentation site**, complete with onboarding guides, node usage examples, and a dedicated Media Kit with official logos and branding assets.
### New Features
* **Metrics and Stream Stats:** Monitor performance in real time via Prometheus or API, with detailed FPS metrics
* **Combo Panel (Experimental):** Early UI for grouped node control
* **Log Level Configuration:** Adjust ComfyUI logging verbosity via environment variables
* **Search Query Support:** Quickly access preset queries to streamline stream startup
### Improvements
* **Performance Enhancements:** Improved FPS and reduced latency
* **Docker Build Optimization:** Speeds up builds and ensures consistent environments
* **Developer Tooling:** Smoother setup with devcontainers, Ansible provisioning, and HTTPS support in VSCode
### Bug Fixes
* Resolved Safari compatibility issues
* Fixed camera and microphone selection bugs
* Addressed audio input stability issues
* Fixed preview node display and rendering problems
## ComfyStream Enters Public Alpha
ComfyStream is now in **public alpha**, ready for early adopters to explore real-time AI workflows within ComfyUI. This release focuses on improved stability, a better developer experience, and two major features: a **custom ComfyUI node** and **real-time audio input support**.
### Custom Node
ComfyStream is available as a custom node via the [ComfyUI Node Registry](https://registry.comfy.org/nodes/comfystream), making it easy to integrate into your workflows and unlock real-time streaming capabilities.
### Audio Support
ComfyStream now supports real-time audio input, enabling features like voice modulation and sound effects. Audio can run alongside video in your workflows for multi-modal processing.
# Contributing
Source: https://docs.comfystream.org/technical/references/contributing
Guide and instructions for contributing to ComfyStream.
## Overview
ComfyStream is an open-source project maintained by the [Livepeer Community](https://www.livepeer.org/). We welcome contributions that improve the project's functionality,
performance, and user experience. We encourage developers to report any issues encountered during installation or usage.
Sharing these findings helps prevent others from encountering the same problems.
The Livepeer community values its developers and contributors. To recognize these efforts, we feature contributors through the
[Contributors Spotlight](https://spotlight.livepeer.dev/). Additionally, our [Bounty Program](https://livepeer.notion.site/Livepeer-Bounty-Board-13b0a34856878041b9ddecaa1380b2e6)
offers rewards for contributions that enhance the project.
## How to Contribute
For detailed instructions on the contribution process, please consult the official [Contributing Guide](https://github.com/livepeer/comfystream/blob/main/CONTRIBUTING.md) on GitHub.
# Media Kit
Source: https://docs.comfystream.org/technical/references/media-kit
Download ComfyStream's official logos and brand assets.
ComfyStream is an open-source project built by and for the community. This media kit includes official logos and icons for use in documentation, tutorials, social posts, integrations, or anything that helps share or build on the project.
[Download Media Kit (ZIP)](https://github.com/livepeer/comfystream/releases/download/v0.0.5/comfystream-brand-kit.zip)
## Typography
ComfyStream uses [Inter](https://fonts.google.com/specimen/Inter?preview.text=Comfystream) for its docs and [Montserrat Alternates](https://fonts.google.com/specimen/Montserrat+Alternates?preview.text=Comfystream) in the logo. Both are open source and available on [Google Fonts](https://fonts.google.com).
## Logos
Official logo variants for use on light or dark backgrounds.
## Icons
Standalone icons for compact spaces like avatars, overlays, or badges.