Skip to content

Add ComfyStream live-runner for go-livepeer orchestrators - #594

Draft
eliteprox wants to merge 19 commits into
mainfrom
feat/comfypeer-liverunner
Draft

eliteprox wants to merge 19 commits into
mainfrom
feat/comfypeer-liverunner

Conversation

@eliteprox

Copy link
Copy Markdown
Collaborator

Summary

  • Add a Livepeer live-runner path that registers ComfyStream as app comfystream (capacity 1) against a go-livepeer orchestrator with -useLiveRunners, driving comfystream.Pipeline in-process (no BYOC/pytrickle subprocess).
  • Expose an agent-shaped HTTP surface for video-in→text-out and live trickle sessions: POST /analyze, POST /start_stream, POST /update_stream, GET /text, GET /healthz.
  • Ship Docker packaging (Dockerfile.live-runner, docker-compose.live-runner.yml, --live-runner entrypoint), optional .[live-runner] deps, a smoke client, and a stub analyze workflow.
  • Also includes supporting workspace/ComfyUI cwd cleanup (--workspace / --cwd), constraint/override-based node installs, and CI workflow fork-restriction removals needed to operate from this branch.

Motivation

ComfyPeer / live-runner work wants ComfyStream sellable as a session wall-clock / latency runner on an existing orchestrator—same registration + trickle path as the transcode live-runner—rather than a batch $/image race or legacy BYOC.

What's included

Live-runner

Piece Role
server/live_runner.py Register runner, create trickle channels, run Pipeline sessions
server/live_runner_client.py Smoke client: reserve → analyze/stream → update → stop
docker/Dockerfile.live-runner Overlay image on livepeer/comfystream + livepeer-gateway
docker-compose.live-runner.yml Host-network attach to an already-running orchestrator
docker/entrypoint.sh --live-runner → python server/live_runner.py …
workflows/comfystream/analyze-stub-api.json Minimal video→text stub for bring-up
pyproject.toml Optional live-runner extra

Supporting (earlier on branch)

  • Normalize ComfyUI workspace handling (--workspace, config passthrough, client init)
  • Script/dependency hygiene (constraints.txt, overrides.txt, setup_nodes / build_trt)
  • Remove GitHub Actions repo-owner guards so fork CI can run

Legacy BYOC (server/byoc.py) is unchanged.

Quick test

# Attach to an orchestrator that already has -useLiveRunners
docker compose -f docker-compose.live-runner.yml up -d --build
curl -sk "$LIVEPEER_ORCH_URL/discovery" | jq '.[].runners[].app'

pip install '.[live-runner]'
python server/live_runner_client.py sample.mp4 \
  --workflow workflows/comfystream/analyze-stub-api.json \
  --discovery "$LIVEPEER_ORCH_URL/discovery"

Test plan

  • Compose build succeeds; runner appears in orchestrator discovery as comfystream
  • Smoke client analyze against stub workflow returns text
  • start_stream + frame publish produces trickle video (and optional text)
  • update_stream applies mid-session workflow/prompt change
  • Orchestrator session release cleans up active session (capacity returns to 1)
  • /healthz healthy while idle and during a session
  • Confirm secrets/host defaults in docker-compose.live-runner.yml are overridden via env (no committed orch secrets in final merge)
  • Existing WebRTC / BYOC paths still work; workspace --config still preserves --workspace

Notes / follow-ups

  • Depends on livepeer-gateway branch ja/live-runner until that lands on a tagged release.
  • Compose file currently carries host-specific GPU UUID / orch URL defaults—should be env-only before merge.
  • Protobuf: live-runner image installs protobuf>=6.31.1 for gateway pb2; call out any ComfyUI pin tension in review.

Copilot AI and others added 14 commits January 10, 2026 06:42
Co-authored-by: eliteprox <16746274+eliteprox@users.noreply.github.com>
Remove repository restrictions from GitHub workflows to enable fork operation
- Standardize on --workspace (with --cwd alias) so ComfyStream can launch from any directory while mapping to the correct ComfyUI cwd.
- Pass the workspace/CWD and logging settings cleanly through server args → pipeline → ComfyStreamClient so ComfyUI’s configuration is honored.
- Add config passthrough support: when a ComfyUI config file is provided, forward it directly and skip overriding flags to match ComfyUI’s precedence rules.
- Ensure ComfyUI packages load correctly via the client initialization (__init__.py/ComfyStreamClient), removing legacy env handling and relying on proper cwd/config inputs.
- Update Docker, supervisord, launch configs, and docs to use the unified workspace flag and defaults.
This reverts commit 6853675.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: eliteprox <16746274+eliteprox@users.noreply.github.com>
Preserve workspace cwd when using --config in frame_processor
- Cleaned up __init__.py by removing unused imports and defining __all__.
- Introduced overrides.txt for managing package dependencies with specific versions and extra index URLs.
- Updated setup_models.py and setup_nodes.py to improve configuration handling and error reporting during node installations.
- Enhanced utils.py with new functions for workspace management and improved config path resolution.
…ng and dependency management

- Refactored build_trt.py to include a setup_comfy function for better workspace initialization and module imports.
- Added a workspace argument to build_trt_engine for flexibility in specifying the ComfyUI workspace.
- Updated constraints.txt to pin specific versions of onnxruntime and huggingface-hub, and added a new dependency for transformers.
- Modified setup_nodes.py to utilize a centralized constraints file for pip installations, ensuring consistent dependency management across custom nodes.
… dependency management and module imports

- Added import for comfy.sd in build_trt.py to streamline module access.
- Updated constraints.txt to include a new dependency on nvidia-modelopt with a minimum version requirement.
- Modified setup_nodes.py to handle nvidia-modelopt version conflicts by stripping unnecessary extras from the package line.
- Introduced docker-compose.live-runner.yml to facilitate running ComfyStream as a live-runner connected to an existing Livepeer orchestrator.
- Created Dockerfile.live-runner to build the necessary environment for the live-runner, including dependencies for livepeer-gateway.
- Updated entrypoint.sh to handle the new --live-runner flag for starting the service.
- Added server/live_runner.py to implement the live-runner functionality, enabling video analysis and live streaming.
- Developed server/live_runner_client.py as a smoke client for testing the live-runner features.
- Enhanced README.md with instructions for using the live-runner and added a new workflow JSON for analysis.
- Updated pyproject.toml to include live-runner dependencies.
- Ensured proper integration with Livepeer's API for session management and media handling.
Comment thread server/live_runner.py Fixed
Comment thread server/live_runner.py Fixed
Comment thread server/live_runner.py Fixed
The room can attach to the orchestrator, drive ws_stream, and load the passthrough workflows without going through the fal batch routes.
Commit the route index, schema bytes, and operator overlay so schema hashes stay stable without copying OpenAPI documents into the image.
Batch jobs run FalSubmit and FalCollect in an isolated process pool, and SaveFalResult gives ComfyUI the output node those graphs were missing.
Install stream-pack and fal-api only, keep a GCC 14 libstdc++ so av can load, and fail the image if node setup does not finish.
Comment thread server/webrtc_trickle_bridge.py Fixed
Comment thread server/webrtc_trickle_bridge.py Fixed
Log the traceback server-side, accept payment-ticket headers up to 256KB, and pin setuptools below 82 so ComfyUI tests can import pkg_resources.
@codecov

codecov Bot commented Sep 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.26667% with 74 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.79070%. Comparing base (06b2a27) to head (5443744).
⚠️ Report is 10 commits behind head on main.

Files with missing lines Patch % Lines
server/live_runner_batch_client.py 27.47253% 66 Missing ⚠️
test/conftest.py 50.00000% 7 Missing ⚠️
test/test_batch_http.py 99.30556% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@                  Coverage Diff                  @@
##                 main        #594          +/-   ##
=====================================================
- Coverage   100.00000%   82.79070%   -17.20930%     
=====================================================
  Files               1           6           +5     
  Lines              55         430         +375     
=====================================================
+ Hits               55         356         +301     
- Misses              0          74          +74     
Files with missing lines Coverage Δ
test/test_fal_catalog.py 100.00000% <100.00000%> (ø)
test/test_job_pool.py 100.00000% <100.00000%> (ø)
test/test_batch_http.py 99.30556% <99.30556%> (ø)
test/conftest.py 50.00000% <50.00000%> (ø)
server/live_runner_batch_client.py 27.47253% <27.47253%> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e9dace...5443744. Read the comment docs.

Files with missing lines Coverage Δ
test/test_fal_catalog.py 100.00000% <100.00000%> (ø)
test/test_job_pool.py 100.00000% <100.00000%> (ø)
test/test_batch_http.py 99.30556% <99.30556%> (ø)
test/conftest.py 50.00000% <50.00000%> (ø)
server/live_runner_batch_client.py 27.47253% <27.47253%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants