Deep Dive into SGLang, Volume I: Runtime, Scheduling, Memory, and Decoding

Deep Dive into SGLang, Volume I explains SGLang's runtime request path as a set of algorithms, data structures, and systems tradeoffs. This volume focuses on the core inference-serving loop: tokenization, admission, prefill and decode scheduling, prefix lookup, KV-cache allocation, forward execution, sampling, streaming, and cleanup. Instead of treating SGLang as a catalog of commands, it builds a working model of the runtime state that makes modern LLM serving possible. Volume I covers: - The inference serving problem and request lifecycle- Transformer inference cost models- SGLang's runtime as a distributed state machine- Continuous batching and chunked prefill- KV-cache memory management- RadixAttention and prefix reuse- Hierarchical caching- Attention backends and forward batches- Model architecture registration- Sampling and logits processing- Structured outputs, reasoning parsers, and tool-call parsing- Speculative decoding- Diffusion language models and blockwise decoding >This is Volume I of Deep Dive into SGLang. Volume II continues into quantization, distributed serving, kernels, hardware backends, benchmarking, correctness, multimodal serving, diffusion inference, and technical extension. Independent explanatory guide. Not affiliated with or endorsed by the SGLang project.