feat(lingbot-video): add native Dense/MoE inference and four-GPU runtime

CLAUDE.md

@@ -6,7 +6,7 @@ TeleFuser is a high-performance framework for efficient multimodal generation mo
 
 **Tech Stack:** Python 3.10-3.13, PyTorch 2.6+, CUDA 12.8+, FastAPI, Ray
 
-**Supported Models:** WanVideo (Wan2.1/2.2), Qwen-Image, Z-Image, FlashVSR, HunyuanVideo, Flux2 Klein, LTX Video, LiveAct, LongCat-Video, LingBot-World
+**Supported Models:** WanVideo (Wan2.1/2.2), Qwen-Image, Z-Image, FlashVSR, HunyuanVideo, Flux2 Klein, LTX Video, LiveAct, LongCat-Video, LingBot-World, LingBot-Video
 
 ## Commands
 
@@ -43,6 +43,7 @@ telefuser/
 │   ├── longcat_video/ # LongCat-Video: T2V, I2V
 │   ├── lingbot_world_fast/  # LingBot shared causal-fast engine
 │   ├── lingbot_world_v2/    # LingBot-World v2 causal-fast facade
+│   ├── lingbot_video/      # LingBot-Video Dense/MoE/refiner runtime
 │   └── common/       # Shared pipeline utilities
 ├── models/           # Model architectures: DiT, VAE, text encoders
 ├── ops/              # Custom operations: attention, FFN, normalization
@@ -82,6 +83,14 @@ telefuser/
 - Interpret chunk period as output cadence: real-time operation requires p95 to stay
   below the media duration represented by one chunk, with margin for transport and encoding.
 
+### LingBot-Video Single-Process Runtime
+
+- Dense and MoE LingBot-Video requests use structured JSON captions. Spatial height and width must be divisible by 16: the Wan VAE downsamples by 8 and the DiT uses a spatial patch size of 2.
+- TI2V conditions are independent Qwen3-VL visual tokens and a VAE clean frame-zero latent. Preserve both paths and reapply the latent condition after every denoising step.
+- The MoE refiner is a separately loaded stage. For a shared GPU, release/offload base stages before loading it, and retain the native RGB handoff rather than introducing an MP4 round trip.
+- The sorted eager MoE path is the validated single-GPU correctness implementation. Four-GPU MoE uses native grouped GEMM when `torch._grouped_mm` is available and retains sorted eager as an explicit fallback. FP8 and expert parallelism require separate parity and benchmark evidence before being enabled.
+- Distributed LingBot base sampling keeps the complete scheduler loop inside each stage worker. Do not move per-step latents through the parent process. Spawned distributed workers use one PyTorch intra-op CPU thread per rank, matching `torchrun` and avoiding host launch-thread oversubscription.
+
 ### Layer Architecture Principles For Models
 
 TeleFuser's model follows a strict layered architecture for operations:
@@ -145,6 +154,7 @@ When adding or porting a pipeline, preserve upstream behavior and reuse TeleFuse
 - Before editing, select the closest maintained pipeline, public example, and tests as structural baselines. Read the relevant adding-new-example, adding-new-model, adding-new-stage, model-loading, configuration, and service documentation.
 - Inventory required model-specific classes and configuration fields and map them to upstream behavior and the selected baseline.
 - List every proposed framework-level or cross-pipeline interface, general-purpose configuration field, environment variable, loader, registry, CLI option, or service schema deviation. The expected list is empty.
+- Reusing a framework API does not authorize changing that API. During model or pipeline integration, do not add or broaden shared loading, registry, configuration, orchestration, or service behavior merely for convenience. First use the existing API exactly as-is, including its supported file-list and wildcard inputs. If it cannot express the requirement, report the precise gap, alternatives, affected callers, and compatibility impact, then obtain explicit user approval before editing shared framework code.
 - Reuse `BasePipeline`, `BaseStage`, `ModuleManager`, existing configuration dataclasses, example contracts, and service schemas. Do not create parallel interfaces or attach ad-hoc configuration attributes for convenience.
 - If existing extension points cannot express a requirement, report the exact gap and obtain user approval before introducing a new public interface or configuration mechanism.
 - Do not add an environment variable during pipeline integration unless the user explicitly requests it or an existing documented variable already has the exact required semantics. Prefer function parameters for request-scoped inputs, dataclass fields for runtime configuration, CLI options for command-line inputs, and service schemas for API inputs.

README.md

@@ -17,6 +17,7 @@ TeleFuser is a high-performance runtime for world model inference and multimodal
 
 ## News 📰
 
+- ✨ **2026-07-22**: **NEW** Added [**LingBot-Video**](examples/lingbot_video/README.md) support for Dense and MoE T2I/T2V/TI2V generation, native four-GPU CFG/SP execution, and in-memory MoE refinement.
 - ✨ **2026-07-15**: Added [**LingBot-World v2**](https://github.com/Robbyant/lingbot-world-v2) support for offline generation, interactive WebRTC streaming, and multi-GPU inference.
 
 - ✨ **2026-07-06**: Added external **CacheSeek** latent cache integration for service-mode cross-request reuse. Cache hits can skip the first N denoising steps; the Wan2.2 cache-enabled service example snapshots `[5, 10, 15, 20, 25]` by default. See [docs/en/latent_cache.md](docs/en/latent_cache.md).
@@ -195,6 +196,7 @@ telefuser/
 | `HunyuanVideo` | T2V, I2V | Supported via [examples/hunyuan_video/README.md](examples/hunyuan_video/README.md) |
 | `LTX Video` | I2V + Audio | Unified audio-video generation via [examples/ltx_video/README.md](examples/ltx_video/README.md) |
 | `LongCat-Video` | T2V, I2V, VC | Long-form generation and continuation via [examples/longcat_video/README.md](examples/longcat_video/README.md) |
+| **NEW** `LingBot-Video` | T2I, T2V, TI2V, MoE refiner | Dense/MoE generation with native CFG/SP and an in-memory base-to-refiner path; see [examples/lingbot_video/README.md](examples/lingbot_video/README.md) |
 
 ### Image Generation and Other Multimodal Pipelines
 

README_zh.md

@@ -17,6 +17,7 @@ TeleFuser 是一个面向世界模型推理与多模态生成的高性能运行
 
 ## News 📰
 
+- ✨ **2026-07-22**：**NEW** 新增 [**LingBot-Video**](examples/lingbot_video/README.md) 支持，覆盖 Dense/MoE T2I、T2V、TI2V、原生四卡 CFG/SP 推理与内存直传 MoE refiner。
 - ✨ **2026-07-15**：新增 [**LingBot-World v2**](https://github.com/Robbyant/lingbot-world-v2) 支持，支持离线生成、交互式 WebRTC 流和多卡推理。
 
 - ✨ **2026-07-06**：新增外部 **CacheSeek** latent cache 集成，支持服务模式下跨请求复用；命中后可跳过前 N 步去噪。Wan2.2 服务示例默认快照 `[5, 10, 15, 20, 25]`。配置和安装方式见 [docs/zh/latent_cache.md](docs/zh/latent_cache.md)。
@@ -194,6 +195,7 @@ telefuser/
 | `HunyuanVideo` | T2V, I2V | 见 [examples/hunyuan_video/README.md](examples/hunyuan_video/README.md) |
 | `LTX Video` | I2V + Audio | 统一音视频生成，见 [examples/ltx_video/README.md](examples/ltx_video/README.md) |
 | `LongCat-Video` | T2V, I2V, VC | 长视频生成与续写，见 [examples/longcat_video/README.md](examples/longcat_video/README.md) |
+| **NEW** `LingBot-Video` | T2I, T2V, TI2V, MoE refiner | 支持原生 CFG/SP 的 Dense/MoE 生成与内存直传 base-to-refiner，见 [examples/lingbot_video/README.md](examples/lingbot_video/README.md) |
 
 ### 图像与其他多模态生成
 

docs/en/configuration.md

@@ -251,6 +251,7 @@ class ParallelConfig:
     pp_degree: int = 1           # Pipeline parallelism
     tp_degree: int = 1           # Tensor parallelism
     enable_fsdp: bool = False
+    worker_intra_op_threads: int = 1  # CPU intra-op threads per worker
 
     def validate(self) -> None:
         """Validate that device count matches parallelism degrees."""

docs/en/index.md

@@ -56,7 +56,8 @@ Reusable stages, model configs, schedulers, and pipeline orchestration.
 
 | Model | Tasks | Description |
 |-------|-------|-------------|
-| LingBot-World-Fast | Bidirectional streaming | Interactive world model via WebRTC DataChannel |
+| LingBot-World v2 | Bidirectional streaming | Camera-controlled interactive world model via WebRTC |
+| LingBot-World-Fast | Bidirectional streaming | Legacy/causal-fast interactive world model via WebRTC DataChannel |
 
 ### Video Generation
 
@@ -68,6 +69,7 @@ Reusable stages, model configs, schedulers, and pipeline orchestration.
 | FlashVSR | VSR | Video super-resolution |
 | LiveAct | S2V | Speech-to-video |
 | LongCat-Video | T2V, I2V | Long video generation |
+| LingBot-Video | T2I, T2V, TI2V, MoE refiner | Precision-first Dense and MoE video generation |
 
 ### Image Generation
 

docs/en/parallel.md

@@ -385,8 +385,14 @@ class ParallelConfig:
     enable_fsdp: bool = False             # Enable FSDP
     timeout: int = 600                    # Timeout in seconds
     queue_with_cpu: bool = False          # Use CPU queue
+    worker_intra_op_threads: int = 1      # CPU intra-op threads per worker
 ```
 
+The default of one thread matches `torchrun` and prevents CPU oversubscription
+across GPU worker processes. Increase it only for distributed stages with
+substantial CPU-side computation; it does not change the parent process thread
+pool.
+
 ### Validation Rules
 
 ```python
@@ -611,4 +617,4 @@ RuntimeError: ParallelWorker timeout
 - [Ulysses: Sequence Parallelism](https://arxiv.org/abs/2309.14509)
 - [Ring Attention](https://arxiv.org/abs/2310.01889)
 - [GPipe: Pipeline Parallelism](https://arxiv.org/abs/1811.06965)
-- [PyTorch Distributed](https://pytorch.org/docs/stable/distributed.html)
+- [PyTorch Distributed](https://pytorch.org/docs/stable/distributed.html)

docs/en/service.md

@@ -100,6 +100,7 @@ See the [Stream Server Guide](stream_server.md) for full streaming documentation
 | `LiveAct` | S2V (speech-to-video) | Speech-driven talking head generation |
 | `FlashVSR` | VSR | Streaming video super-resolution |
 | `LongCat-Video` | T2V, I2V, VC | Long-form generation and continuation |
+| `LingBot-World v2` | Bidirectional world-model streaming | Camera-controlled WebRTC loop — see [Stream Server Guide](stream_server.md) |
 
 ### Video Generation
 
@@ -108,6 +109,7 @@ See the [Stream Server Guide](stream_server.md) for full streaming documentation
 | `WanVideo` (Wan2.1 / Wan2.2) | T2V, I2V, FL2V | Main video generation family |
 | `HunyuanVideo` | T2V, I2V | Supported via service examples |
 | `LTX Video` | I2V + Audio | Unified audio-video generation |
+| `LingBot-Video` | T2I, T2V, TI2V, MoE refiner | Dense/MoE generation with native CFG/SP and in-memory refinement; see `examples/lingbot_video/README.md` |
 
 ### Image Generation
 

docs/zh/configuration.md

@@ -251,6 +251,7 @@ class ParallelConfig:
     pp_degree: int = 1           # 流水线并行
     tp_degree: int = 1           # 张量并行
     enable_fsdp: bool = False
+    worker_intra_op_threads: int = 1  # 每个 worker 的 CPU intra-op 线程数
 
     def validate(self) -> None:
         """验证设备数量与并行度匹配。"""

docs/zh/index.md

@@ -55,7 +55,8 @@ AdaTaylorCache 和运行时缓存控制，面向重复生成工作负载。
 
 | 模型 | 任务 | 描述 |
 |------|------|------|
-| LingBot-World-Fast | 双向流式推理 | 通过 WebRTC DataChannel 的交互式世界模型 |
+| LingBot-World v2 | 双向流式推理 | 通过 WebRTC 的相机控制交互式世界模型 |
+| LingBot-World-Fast | 双向流式推理 | Legacy/causal-fast WebRTC 交互式世界模型 |
 
 ### 视频生成
 
@@ -67,6 +68,7 @@ AdaTaylorCache 和运行时缓存控制，面向重复生成工作负载。
 | FlashVSR | VSR | 视频超分辨率 |
 | LiveAct | S2V | 语音转视频 |
 | LongCat-Video | T2V, I2V | 长视频生成 |
+| LingBot-Video | T2I, T2V, TI2V, MoE refiner | 精度优先的 Dense/MoE 视频生成 |
 
 ### 图像生成
 

docs/zh/parallel.md

@@ -385,8 +385,12 @@ class ParallelConfig:
     enable_fsdp: bool = False             # 启用 FSDP
     timeout: int = 600                    # 超时时间（秒）
     queue_with_cpu: bool = False          # 使用 CPU 队列
+    worker_intra_op_threads: int = 1      # 每个 worker 的 CPU intra-op 线程数
 ```
 
+默认值 1 与 `torchrun` 一致，可避免多个 GPU worker 的 CPU 线程过度竞争。只有分布式
+stage 包含大量 CPU 计算时才需要提高该值；它不会修改父进程的线程池。
+
 ### 验证规则
 
 ```python
@@ -611,4 +615,4 @@ RuntimeError: ParallelWorker timeout
 - [Ulysses: Sequence Parallelism](https://arxiv.org/abs/2309.14509)
 - [Ring Attention](https://arxiv.org/abs/2310.01889)
 - [GPipe: Pipeline Parallelism](https://arxiv.org/abs/1811.06965)
-- [PyTorch Distributed](https://pytorch.org/docs/stable/distributed.html)
+- [PyTorch Distributed](https://pytorch.org/docs/stable/distributed.html)

docs/zh/service.md

@@ -100,6 +100,7 @@ TeleFuser 提供两种服务命令，针对不同工作负载类型优化：
 | `LiveAct` | S2V（语音转视频） | 语音驱动说话人头部生成 |
 | `FlashVSR` | VSR | 流式视频超分辨率 |
 | `LongCat-Video` | T2V, I2V, VC | 长视频生成和续写 |
+| `LingBot-World v2` | 双向世界模型流式推理 | 相机控制的 WebRTC 闭环 — 参见[流式服务指南](stream_server.md) |
 
 ### 视频生成
 
@@ -108,6 +109,7 @@ TeleFuser 提供两种服务命令，针对不同工作负载类型优化：
 | `WanVideo`（Wan2.1 / Wan2.2） | T2V, I2V, FL2V | 主要视频生成系列 |
 | `HunyuanVideo` | T2V, I2V | 通过服务示例支持 |
 | `LTX Video` | I2V + Audio | 音视频统一生成 |
+| `LingBot-Video` | T2I, T2V, TI2V, MoE refiner | 支持原生 CFG/SP 的 Dense/MoE 生成与内存直传 refinement；参见 `examples/lingbot_video/README.md` |
 
 ### 图像生成
 

examples/example_config.yaml

@@ -116,6 +116,24 @@ pipelines:
       attn_impl: TORCH_SDPA
       frame_num: 13
 
+  # ============================================================
+  # lingbot_video — Fixed Dense and MoE examples
+  # ============================================================
+
+  lingbot_video_dense_1_3b_t2v:
+    script: lingbot_video/lingbot_video_dense_1_3b.py
+    gpu_count: 1
+    output_type: video
+    timeout_seconds: 1800
+    resolution: "480p"
+
+  lingbot_video_moe_30b_refiner_t2v:
+    script: lingbot_video/lingbot_video_moe_30b.py
+    gpu_count: 4
+    output_type: video
+    timeout_seconds: 3600
+    resolution: "480p"
+
   # ============================================================
   # hunyuan_video
   # ============================================================

examples/lingbot_video/README.md

@@ -0,0 +1,84 @@
+# LingBot-Video
+
+Dense 1.3B and MoE 30B are separate examples. Each module exposes
+`PPL_CONFIG`, `CONTRACT`, `get_pipeline`, `run`, and `run_with_file` for the
+shared CLI runner and TeleFuser service.
+Both examples default to the official five-second structured caption in `assets/t2v_5s.json.example` and the validated 832x480 LingBot landscape geometry.
+Their default checkpoints are resolved from `TF_MODEL_ZOO_PATH`, which defaults
+to `/hhb-data/aigc/model_zoo` in the current environment.
+
+The model-specific files also contain their checkpoint loading, stage assembly, request
+handling, refiner lifecycle, and output encoding. Shared behavior uses TeleFuser contract
+templates and video utilities directly.
+
+Use a structured JSON caption produced by the LingBot rewriter. Dense 1.3B T2V:
+
+```bash
+python examples/lingbot_video/lingbot_video_dense_1_3b.py \
+  --model_root /path/to/lingbot-video-dense-1.3b \
+  --prompt "$(cat /path/to/caption.json)" --output_path result.mp4
+```
+
+Pass `--task i2v --first_image_path first_frame.png` for TI2V.
+
+For the MoE checkpoint refiner, use the in-memory base-to-refiner path:
+
+```bash
+python examples/lingbot_video/lingbot_video_moe_30b.py \
+  --model_root /path/to/lingbot-video-moe-30b-a3b --refine \
+  --prompt "$(cat /path/to/caption.json)" --output_path result.mp4
+```
+
+On four GPUs the base and refiner DiTs remain resident together during refiner
+denoising by default. Both worker groups are released before the high-resolution
+VAE decode. Use `--no-refiner_co_resident` for the lower-memory sequential
+lifecycle, or `--no-refine` for MoE base-only generation. `expert_backend=auto` keeps
+the validated sorted eager path on one GPU and selects native grouped GEMM for
+four-GPU inference. The grouped path requires a CUDA PyTorch build that exposes
+`torch._grouped_mm`; use `--expert_backend sorted` as the explicit fallback.
+`--expert_backend fp8` quantizes routed expert weights per output channel and
+uses native dynamic W8A8 scaled GEMMs. It reduces expert residency but is an
+explicit memory-oriented backend until a grouped FP8 kernel is available.
+
+Four-GPU base and refiner stages can split the devices as CFG2 x SP2:
+
+```bash
+python examples/lingbot_video/lingbot_video_moe_30b.py \
+  --gpu_num 4 --cfg_parallel_degree 2 \
+  --refiner_gpu_num 4 --refiner_cfg_parallel_degree 2 \
+  --refiner_co_resident --expert_backend fp8 \
+  --model_root /path/to/lingbot-video-moe-30b-a3b \
+  --output_path result.mp4
+```
+
+CFG parallel and batch CFG are mutually exclusive. Use
+`--cfg_parallel_degree 2` for Dense or MoE base generation and
+`--refiner_cfg_parallel_degree 2` for the refiner. A degree of one retains SP4.
+
+Set `PPL_CONFIG["model_root"]` in the selected example, then serve structured-caption T2I/T2V/TI2V requests with:
+
+```bash
+telefuser serve examples/lingbot_video/lingbot_video_dense_1_3b.py --port 8000
+```
+
+Serve MoE independently with `lingbot_video_moe_30b.py`. Refiner requests are
+enabled by default and can override the `refine` contract parameter. Set
+`PPL_CONFIG["refiner_parallelism"] = 4` to select its distributed FSDP stage
+explicitly; otherwise it inherits service parallelism. The CFG degrees determine
+whether four workers use SP4 or CFG2 x SP2.
+
+```bash
+telefuser serve examples/lingbot_video/lingbot_video_moe_30b.py --port 8000
+```
+
+Dense and MoE base checkpoints support TeleFuser-native four-GPU FSDP plus Ulysses sequence parallelism:
+
+```bash
+telefuser serve examples/lingbot_video/lingbot_video_dense_1_3b.py --gpu-num 4 --port 8000
+```
+
+For MoE, set `PPL_CONFIG["model_root"]` in the MoE example and serve
+`lingbot_video_moe_30b.py`. Do not use `torchrun` for either service: TeleFuser
+creates and manages the workers. Configure `refiner_co_resident=False` when the
+selected dtype, CFG layout, or GPU memory cannot hold both DiTs during refiner
+denoising.