SGLang

[Test] Add unit tests for utils/field_validators

## What 为 `sglang.srt.utils.field_validators` 添加单元测试。 ## Test ```bash pytest test/registered/unit/utils/test_field_validators.py -v ``` 测试输出： ``` =========================================================================================== platform linux -- Python 3.10.13, pytest-9.1.1, pluggy-1.6.0 collected 12 items test/registered/unit/utils/test_field_vali

Main transport1

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation <!-- Describe the purpose and goals of this pull request. --> ## Modifications <!-- Det

Add perf and acc test for CPU models for nightly

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation <!-- Describe the purpose and goals of this pull request. --> This PR is to add perform

Run the shared-expert MLP on the alternate stream while keeping atten…

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation Improve K3 inference performance by overlapping the shared-expert MLP with the routed-e

[Bugfix] Bound SGLANG_PORT search to valid TCP range

## Motivation `SGLANG_PORT` is a base-port hint: when that port is occupied, `get_open_port()` searches upward to preserve a predictable range for deployments behind firewalls. The previous unbounded loop continued past TCP port 65535. `is_port_available()` treats the resulting `OverflowError` as unavailable, so `SGLANG_PORT=65535` with that port occupied ca

[Fix] Spawn, don't fork, the benchmark server process

`sglang.benchmark.*` launched the server with the platform-default start method (fork on Linux), but the parent already initializes the accelerator in ServerArgs.__post_init__ -- get_xpu_memory_capacity() calls torch.xpu.mem_get_info(), get_device_sm() calls torch.cuda.get_device_capability(). The forked child inherits a dead context and dies on its first de

[diffusion] Decouple encoder parallelism from the DiT parallel layout

## Motivation Encoder layout and DiT parallelism are independent concerns: an encoder is never sharded by the DiT's tp/sp/cfg — only by its own folding mechanism — and the unit that shares a request batch is the **pipeline replica**, not the world. Yet both encoder-parallel mechanisms were gated on the DiT's layout: - `encoder_parallel=dp` was rejected whene

[Metrics] Add kv_cache_usage_perc Prometheus gauge (#5979)

## Motivation Fixes #5979 — users migrating from vLLM expect a Prometheus gauge for KV cache utilization (`vllm:gpu_cache_usage_perc`); SGLang currently has no KV-cache-only gauge. Three earlier attempts (#20158, #22612, #27701) all exposed `stats.token_usage` under a new name and stalled for the same reason: - `token_usage` is `max(full, swa, mamba)` — the

[bugfix] [NPU] fix K*B dim for transpose batch matmul ops exceed 65536, and change envs to server args.

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation <img width="1572" height="204" alt="image" src="https://github.com/user-attachments/ass

[SM120] dspark: widen SWA index width to an instantiated sparse-MLA shape

## Purpose DSPARK does not start on consumer Blackwell (SM120). CUDA graph capture aborts: ``` Check failed: (ok) is false: Unsupported sparse-MLA prefill configuration: model=DSV4 num_heads=16 topk=192 page_block_size=64 topk_extra=0 extra_page_block_size=0 ``` `build_dspark_swa_page_indices` derives its index width from `swa_window + block_size`, which lan

[rust] Unify MM feature transport on POSIX shm (drop inline mode)

## Motivation The Rust MM worker had two feature transports: inline (rank 1: features ride the drain by value) and shm (TP>1: segment + `ShmPointerMMData` stub). This unifies on shm as the single transport for every topology and deletes the inline path, then removes the copies that made shm slower than inline. ## Modifications - **One transport**: the worker

[diffusion] [NPU] Optimize LTX-2/2.3 inference performance for NPU

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation NPU compatibility and performance optimizations for LTX-2/2.3 inference. <!-- Describe

[CI] Fix argument quoting in AMD install_with_retry

## Motivation Every multimodal-gen AMD lane (`PR Test ROCm 7.2 (AMD)`) currently fails in **Install dependencies**, on every PR (reproduced on this branch, `kan/unify_shmem`, and #34713): ``` Attempt 3/3: docker exec ci_sglang pip install ... sgl-eval @ git+https://github.com/sgl-project/sgl-eval.git@6690895... ERROR: Invalid requirement: '@': Expected packa

[PD] Batch final DSV4 SWA transfer

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation Reduce Mooncake PD transfer overhead for DeepSeek-V4 DSpark by batching the final targe

fix(dsa): use int64 page offsets in index buffer accessors

## Motivation DSA index buffer accessors compute source addresses using: `page_index * buf_numel_per_page` `page_indices` are commonly stored as int32. For sufficiently large index buffers, this multiplication overflows before pointer arithmetic is widened, causing K/scale data to be read from the wrong page and potentially producing corrupted model output.

Feiyue/mori pp dcp

# [mori] PD: pair state/KV transfer entries by layer id (support PP prefill, fix cross-TP KDA reshard) ## Summary Make the `mori` (moriio) PD transfer backend pair prefill/decode transfer entries by **global layer id** instead of positionally, so **pipeline-parallel (PP) prefill** works, and fix the KDA/mamba recurrent-state reshard under `prefill_tp != deco

[kernel] One rmsnorm kernel for every hidden size, tuned from Python

> Generated by Claude. ## Motivation Three rmsnorm kernels (`rmsnorm.cuh`, `fused_add_rmsnorm.cuh`, `rmsnorm_hf.cuh`) with three hand-written dispatches covered a fraction of the hidden sizes SGLang serves. `_is_supported_rmsnorm_hidden_size` admitted only multiples of 256 (512 above 8192); fused-add only multiples of 16 up to 8192. **1536, 2304, 2560 — Qwen

[XPU][test] Add cache_salt=None to _make_req in test_lmcache_radix_cache.py

## Summary - Adds `cache_salt=None` to the `SimpleNamespace` request stand-in in `_make_req` at `test/registered/xpu/test_lmcache_radix_cache.py:64`. - Fixes `AttributeError: 'types.SimpleNamespace' object has no attribute 'cache_salt'` on `stage-b-test-1-gpu-xpu`, introduced when #30827 added `cache_salt=req.cache_salt` to `RadixCache.cache_finished_req` (`

[Core] Organize environment variable registry

## Summary - reorganize all 560 `Envs` fields into 64 focused subsystem sections with consistent three-line headers - split the former catch-all blocks and move misplaced fields next to their owning runtime, backend, model, or test workflow - document organization rules for future fields, including the under-30-fields section limit and AST verification requi

K3 pp

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation <!-- Describe the purpose and goals of this pull request. --> ## Modifications <!-- Det

[UT][NPU] Re-register NPU attention unit tests in base-a CI suites

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation <!-- Describe the purpose and goals of this pull request. --> NPU unit tests under test

[Feature][DSpark] Support Speculators-format checkpoints and ragged multimodal CUDA graphs

<!-- Suggested title: [Feature][DSpark] Support native Speculators checkpoints and ragged multimodal CUDA graphs --> > **Draft:** the implementation and public-checkpoint smoke tests are complete. > Before marking this PR ready, I plan to add a standard downstream accuracy run > and a larger multi-concurrency performance run. ## Motivation This PR extends SG

K3 cpp

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation <!-- Describe the purpose and goals of this pull request. --> ## Modifications <!-- Det

[PD] Clean up stale staging watermark subscribers

## Motivation Fixes #34737. In heterogeneous-TP disaggregated serving, the decode-side staging watermark subscriber registry outlives individual prefill connections. When a prefill node fails or restarts, `CommonKVManager._handle_node_failure()` removes its connection-pool entries and cached sockets but leaves its subscriber in `DecodeStagingHandler._wm_subs

Fix DSpark draft logits when the target lm_head is quantized

The DSpark draft borrows the target's `lm_head` and projected through it with a dense matmul on `.weight`. When the target checkpoint quantizes `lm_head` (e.g. a ModelOpt NVFP4 export), that weight is packed `uint8` of shape `[vocab, hidden / 2]`, so the matmul gets the wrong K and the `hidden.to(weight.dtype)` cast reinterprets bf16 activations as `uint8`.

[Benchmark] Fix per-run speculative accept length errorr in `Serving Benchmark Result` when dp_size>1

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation <!-- Describe the purpose and goals of this pull request. --> `bench_serving` currently

fix: support Qwen3_5MoeForCausalLM in fused MoE tuning utils

## Problem `get_model_config` in `benchmark/kernels/fused_moe_triton/common_utils.py` does not recognize the `Qwen3_5MoeForCausalLM` architecture (used by Qwen3.8). It falls through to the default Mixtral branch, which reads `num_local_experts` from a config object that has no such attribute: ``` AttributeError: 'Qwen3_5MoeTextConfig' object has no attribute

[NPU] [DOC] Add Kimi-K3 deployment tutorial on Ascend NPU

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.io to discuss further. --> ## Motivation <!-- Describe the purpose and goals of this pull request. --> Add Kimi-K3 deployment tu

[Diffusion] Add @triple-mu as a code owner

## Summary - add `@triple-mu` as a code owner for the SGLang diffusion runtime and models - add `@triple-mu` as a code owner for diffusion kernels ## Why This expands reviewer coverage for diffusion changes across both the main `multimodal_gen` tree and the diffusion-specific kernel tree. ## Validation - `git diff --check` - confirmed that the GitHub account

feat(rust-server): add model extension hooks

## Motivation The native Rust API server currently owns a Qwen-specific multimodal path and built-in chat rendering. Model packages need stable extension boundaries to reuse the native HTTP, request lifecycle, tokenizer, and egress implementation without forking the server. ## Modifications - Add injectable native multimodal and chat preprocessing interfaces

perf(moe): add RTX 5080 Granite FP8 Triton config

## Motivation SGLang does not currently ship a Triton FP8 MoE configuration for the NVIDIA GeForce RTX 5080 and the `E=40,N=512` Granite MoE shape. The fallback heuristic leaves measurable kernel and serving performance on the table. ## Modifications Add one Triton 3.6.0 FP8 W8A8 configuration file for: - GPU: NVIDIA GeForce RTX 5080 (SM120) - model: `ibm-gr

fix(xpu): use index_copy_ instead of masked_scatter_ in mm-embed merge

## Summary `embed_mm_inputs._scatter` in `python/sglang/srt/managers/mm_utils.py` calls `dest.masked_scatter_(mask.expand_as(dest), src)` on the multimodal embed merge path (e.g. DeepSeek-OCR). On Intel XPU (torch 2.13+xpu, oneAPI DPC++ 2026.1) that reaches `torch-xpu-ops/src/ATen/native/xpu/sycl/Indexing.cpp:436`, where the size check `totalElements <= srcS

[Doc] Add SGLang ecosystem section to README

The main README currently has no link to SGLang-Omni or the other sibling projects. This adds a short ecosystem section after About so visitors can discover them from the main repo. <!-- pr-states:start --> --- ### CI States Latest PR Test (Base): <!-- slot:pr-test:start -->:white_check_mark: [Run #31725689345](https://github.com/sgl-project/sglang/actions/r

[DSA] Optimize TRTLLM prefill RoPE/FP8 quantization

## Motivation On GLM-5.2 NVFP4 with TRTLLM DSA prefill, the fused `RopeQuantize` path also processes the large 512-wide no-RoPE Q component. At the DP8 prefill shape this is bandwidth-expensive; splitting no-RoPE conversion from the 64-wide RoPE work reduces kernel time once the local token count is large enough. ## Modifications - Add a Triton kernel that c

Partially supports using MHA kernels in MLA forward when page-size > 1.

## Motivation add Partially support to use MHA kernels when page-size > 1. related to #6411 ## Modifications add AttnForwardMethod.MHA_PAGED_PREFILL, and when any(forward_batch.extend_prefix_lens_cpu) == False, we use MHA to run MLA. ## Checklist - [ ] Format your code according to the [Code Formatting with Pre-Commit](https://docs.sglang.ai/references/contr

token length check relaxed

<!-- Thank you for your contribution! We appreciate it. The following guidelines will help improve your pull request and facilitate feedback. If anything is unclear, don't hesitate to submit your pull request and ask the maintainers for assistance. --> ## Motivation <!-- Explain the purpose of this PR and the goals it aims to achieve. --> i am using lighteva

Update bench_one_batch_server.py

fix bug for key error <!-- Thank you for your contribution! We appreciate it. The following guidelines will help improve your pull request and facilitate feedback. If anything is unclear, don't hesitate to submit your pull request and ask the maintainers for assistance. --> ## Motivation <!-- Explain the purpose of this PR and the goals it aims to achieve. -

[bug] add a length check in sample_sharegpt_requests

## Motivation When running ``` python3 -m sglang.bench_serving --backend sglang --dataset-name sharegpt --num-prompts 1 --sharegpt-output-len xxx --sharegpt-context-len yyy ``` if `xxx >= yyy`, the sampled dataset becomes empty, and the following strange error occurs at runtime: ``` Token indices sequence length is longer than the specified maximum sequence

[Doc] Update support_new_models.md

<!-- Thank you for your contribution! We appreciate it. The following guidelines will help improve your pull request and facilitate feedback. If anything is unclear, don't hesitate to submit your pull request and ask the maintainers for assistance. --> ## Motivation `SGLang` now uses `skip_if` and the `CI environment` to control which methods are executed, a

[Docker] Use cache mount for pip cache to speed up builds

## Motivation Currently docker build disables pip cache, probably in an attempt to reduce image size. Docker has a feature designed specifically for this use case - `--mount-type=cache`. This allows docker to transparently reuse any previous pip downloads without increasing image size. ## Modifications Add `--mount=type=cache,target=/root/.cache/pip` to the

CUDA implementation supporting the ep_scatter and ep_gather operator

<!-- Thank you for your contribution! We appreciate it. The following guidelines will help improve your pull request and facilitate feedback. If anything is unclear, don't hesitate to submit your pull request and ask the maintainers for assistance. --> ## Motivation CUDA implementation supporting the ep_scatter and ep_gather operator <!-- Explain the purpose

[8254]: Fix setting up of completion_template name

[8440]: Fix rid mismatch with batch size

Fixes test_mixed_none_and_images_with_parallel_samples for rid mismatches

[Feature] Add support for TNG-R1T2-Chimera tool call parsing.

<!-- Thank you for your contribution! We appreciate it. The following guidelines will help improve your pull request and facilitate feedback. If anything is unclear, don't hesitate to submit your pull request and ask the maintainers for assistance. --> ## Motivation This PR adds full tool calling support for models like tngtech/DeepSeek-TNG-R1T2-Chimera. The

[Bug] Fix to support compressed tensor 2 4 sparse mm serving in SGLang

<!-- Thank you for your contribution! We appreciate it. The following guidelines will help improve your pull request and facilitate feedback. If anything is unclear, don't hesitate to submit your pull request and ask the maintainers for assistance. --> ## Motivation <!-- Explain the purpose of this PR and the goals it aims to achieve. --> After fix, we can a

Ability to use custom-all-reduce on systems with more than 2 PCIe GPUs via env var

<!-- Thank you for your contribution! We appreciate it. The following guidelines will help improve your pull request and facilitate feedback. If anything is unclear, don't hesitate to submit your pull request and ask the maintainers for assistance. --> ## Motivation <!-- Explain the purpose of this PR and the goals it aims to achieve. --> Allow custom all-re

Support GLM4.5 fused moe triton tuning

## Motivation Support GLM4.5 and GLM4.5-Air fused moe triton tuning. ## Modifications - Change `benchmark/kernels/fused_moe_triton/tuning_fused_moe_triton.py` - Change `glm4_moe.py` restoring to a specific model config `n_routed_experts` value limited ## Accuracy Test ``` Accuracy: 0.970 Invalid: 0.000 Latency: 19.836 s Output throughput: 1087.133 token/s ``

replace role literals with str for more flexible chat templating

<!-- Thank you for your contribution! We appreciate it. The following guidelines will help improve your pull request and facilitate feedback. If anything is unclear, don't hesitate to submit your pull request and ask the maintainers for assistance. --> ## Motivation The role literals make experimenting with unique roles awkward, requiring code changes. This

[feat] add grammar sessions

<!-- Thank you for your contribution! Please follow these guidelines to enhance your pull request. If anything is unclear, submit your PR and reach out to maintainers for assistance. Join our Slack community at https://slack.sglang.ai to discuss further. --> ## Motivation Enables persistent grammar for multi-turn conversations where the same grammar needs to

support torch.compile(dynamic=True) to avoid recompile

Currently, when both torch compile and CUDA graph are enabled, torch compile incurs a non-negligible overhead (~270s on deepseek r1). This patch supports using an environment variable to enable dynamic mode in torch.compile(), avoiding recompilation caused by inconsistent tensor sizes(~130s on deepseek r1). <!-- Thank you for your contribution! Please follow