Infrastructure

SGLang

Open-source serving engine that streams realtime/causal video over WebSocket — incremental frames with reusable KV-cache state.

Visit site Back to the directory

docs.sglang.io · listing ≠ endorsement · re-checked 2026-09-03

Operator: Open source · SGLang project
Format: WebSocket realtime-video endpoint
Model: Realtime pipelines: SANA-WM · LingBot World
Access: Self-host (open source)
Role: Serving layer for your own live-gen endpoint

What is SGLang?

SGLang — best known as an LLM serving framework — has been extended to diffusion with a realtime path: a /v1/realtime_video/generate WebSocket endpoint streams video frames incrementally while keeping a live session whose causal KV-cache is reused across chunks until disconnect. It ships realtime pipelines for world models (LingBot World, NVIDIA SANA-WM) and is the self-hosted serving layer if you want to run your own live-gen endpoint rather than rent one.

What SGLang does

A /v1/realtime_video/generate WebSocket endpoint that streams frames incrementally
Reuses a session’s causal KV-cache state across chunks until disconnect — no regeneration from scratch
Ships realtime pipelines for world models (LingBot World, NVIDIA SANA-WM)
MessagePack framing plus delta-gzip frame compression for low-bandwidth streaming
Open source — self-host the serving layer rather than renting a live-gen endpoint

Who is SGLang for?

Serving your own realtime/causal video models behind a WebSocket API
Building world-model streaming (camera-controlled exploration) pipelines
Self-hosting live generation to control cost and data flow

Pros and cons of SGLang

Pros

Open source with real engineering — an LLM serving framework extended to diffusion
Reusable KV-cache state is the right primitive for continuous streaming
Avoids per-second vendor markup on always-on live generation

Cons

A serving engine — you still supply and operate the model
Realtime diffusion support is newer and narrower than its LLM serving core
World-model pipelines are research/niche as of Sep 2026

What people are saying

Verbatim quotes from public coverage — collected 2026-09-03.

“Realtime and causal video pipelines generate video incrementally and reuse state across chunks.”
SGLang docs

Frequently asked questions

What is SGLang’s realtime video support?

A WebSocket endpoint (/v1/realtime_video/generate) that generates video incrementally and reuses a live session’s KV-cache state across chunks.

Is SGLang open source?

Yes — SGLang is an open-source serving framework, and the diffusion/realtime extensions are part of the same project.

What can I serve with it?

It ships realtime pipelines for world models like LingBot World and NVIDIA SANA-WM, plus your own causal diffusion models.

Alternatives in Infrastructure