The signatures were verified against the specified public key
What's Changed
chore(release): backport proxy request-handling maintenance and refresh runtime deps for 1.96.1 by @yuneng-berri in https://github.com/BerriAI/litellm/pull/36494
## TLDR Problem this solves: - Per-deployment `model_info` capability overrides are ignored when `base_model` is set - Azure gpt-5.6 deployments 400 on any temperature other than 1, with no workaround - Affects the temperature, logprobs/top_p, xhigh, and minimal/low effort gates How it solves it: - Threads the deployment name into the gpt-5 capability gates
## TLDR Fixes #26618 Problem this solves: - Azure Fireworks (`FW-*`) models lacked cost-map entries, so spend tracking was wrong or zero - Public Azure pricing only lists a subset of Fireworks catalog models How it solves it: - Adds `azure_ai/FW-*` pricing for the requested Fireworks-on-Foundry models - Uses Azure Data Zone retail meters where published; Mic
## TLDR Problem this solves: - DeepSeek non `none` reasoning effort is silently dropped. - The `/v1/messages` endpoint converts the `efforts` parameter in the request into a reasoning budget that DeepSeek ignores. How it solves it: - Pass-through `reasoning_effort` to upstream when `reasoning_effort` is not `none` - Added the correct capabilities in `model_p
## TLDR Problem this solves: - Datadog Tools panel stays empty for every tool call - Cache hit ratio and cache write tokens never populate How it solves it: - Map tool calls into DD's ToolCall shape on the output message - Forward prompt-cache counts into top-level span metrics ## Relevant issues Fixes #35786 ## Linear ticket ## Pre-Submission checklist - [x
## Summary Fixes #36402 Anthropic's `/v1/messages` API accepts `system` as either a **string** or a **list of content blocks**. Only the string form was being logged — the list form was silently dropped because of the `isinstance(system, str)` guard in `append_system_prompt_messages`. This means Claude Code, any caller using `cache_control` blocks, and any o
## TLDR Problem this solves: - Proxies run multi-worker without Redis and nobody notices - Rate limits, budgets, cache invalidation then silently work per worker - No single doc says what Redis is required for How it solves it: - Default-on red banner across the Admin UI when no Redis - Links to a new doc listing everything degraded without Redis - `LITELLM_
## TLDR Problem this solves: - TypedDict payloads stay silently mutable after construction - Nothing steers new fields toward PEP 705 `ReadOnly` How it solves it: - New LIT012 rule: every TypedDict field must be `ReadOnly[...]` - All 4519 existing fields grandfathered in type-discipline-budget.json - `# writable-ok: <reason>` escape hatch for deliberately wr
## TLDR Problem this solves: - One cleanup run could delete 500k rows per table, unbounded in time - A single delete batch could hold row locks indefinitely - Batch size and run length were env-only, not reachable from the dashboard - Nothing reported what the cleanup job cost the database How it solves it: - One wall-clock budget per run, shared across ever
## TLDR Problem this solves: - Database work shows up in APM as `litellm-server -> localhost`. Prisma's Python client talks to a local Rust query engine over loopback HTTP, so anything instrumenting the transport (ddtrace's httpx patching, an httpx OTEL instrumentor) names the peer `localhost`, and litellm's own `postgres` span carried nothing better: `servi
## TLDR Problem this solves: - Optional MCP outages can prevent proxy workers becoming healthy - Rolling replacements can cascade into gateway-wide user outages - Each remote MCP multiplies startup delay and failure exposure How it solves it: - Warm metadata in the background without delaying readiness - Share discovery work and failed-attempt cooldowns acro
## TLDR Problem this solves: - Complexity router default model could only come from the tiers - No way to point the fallback at another model - A pin equal to the derived value read back as unpinned - Partly filled routers saved, then failed at startup How it solves it: - Adds a Default Model select to the auto router form - Records the pin in the router con
## TLDR Problem this solves: - A request rejected for an unparsable body left no log row - Admin UI Request Logs showed nothing for that client-visible 400 - The Failure filter could not surface it either How it solves it: - Auth now records that rejection as a failed request - The 400 the caller receives is unchanged - Row carries status failure, code 400,
## TLDR Problem this solves: - An upstream that drops its response leaves requests pending forever - Tool discovery only ended when our cancel scope killed it - Clients saw an authenticated server with zero tools - Per-server `timeout` was ignored, prompts and resources hung unbounded How it solves it: - Give the MCP client session a read timeout - Report an
## TLDR Problem this solves: - A sole tagged pre-routing strategy captured every request, tags ignored - Untagged requests were semantically routed to tiers never opted into - Marker pseudo-deployments sat in the selection pool, causing unmapped-provider 400s How it solves it: - Match request tags against every registered strategy before any fallback - Tagge
## TLDR Problem this solves: - Tagged requests through a tagged auto-router 401 on the routed tier - The tag that picked the router is re-applied to the tier's deployments - The same 401 hit /v1/messages requests tagged via the x-litellm-tags header - Admins had to repeat the marker's tag on every tier deployment How it solves it: - Router stamps the rewritt
## TLDR Problem this solves: - Nothing recorded which background job ran on a pod, when, for how long, whether it succeeded, or how much work it moved, so during an incident job activity could only be inferred from database load - A job that overran its interval and started being skipped left no trace at all - The single-owner cron lock decided which pod doe
## TLDR Problem this solves: - Operators could see how many requests were in flight on a pod but not how many the proxy was shedding, so there was no way to tell "throttle upstream" apart from "add pods" - The configured concurrency ceiling was invisible, and worse, the setting most operators reach for is not enforced on a default install How it solves it: -
## TLDR Problem this solves: - Prompt Management's form-free editor surfaces still used legacy dashboard UI primitives How it solves it: - Moves the route-owned editor, dialog, history, message, tool, and variable surfaces to installed shadcn primitives - Preserves non-modal history interaction, short-viewport reachability, textarea growth, keyboard behavior
## TLDR Problem this solves: - A batch row that can never be routed is retried every poll cycle, forever - A batch whose provider record 404s is retried every poll cycle, forever - Either one permanently occupies the oldest-N poll page and starves every batch behind it How it solves it: - Retire (mark processed, without costing) only the two failures that ar
## TLDR Problem this solves: - Redacted tool arguments are parsed as JSON - Every affected tool call emits a false warning How it solves it: - Share one sentinel across producers and the parser - Preserve tool identity while returning empty arguments ## User Flow Before: a proxy admin with message logging disabled sees a warning for every streamed tool call
## Summary When calling `/customer/update` with `budget_duration` and `max_budget`, the `budget_duration` was not being propagated to the newly created budget. This meant the budget reset schedule was never set. This PR fixes the issue by: 1. Adding `budget_duration` field to `UpdateCustomerRequest` in `_types.py` 2. Computing `budget_reset_at` from `budget_
## TLDR Problem this solves: - Groq registry missed five models Groq currently serves - gpt-oss and Llama context/completion limits disagreed with Groq docs - Groq's announced shutdown dates were absent from the registry How it solves it: - Adds Qwen 3.6 27B, both Prompt Guard 2 sizes, both Orpheus voices - Corrects gpt-oss max completion tokens and Llama wi
## TLDR Problem this solves: - Contributors can't find why layers are shaped as they are - So they hand-roll plumbing next to existing generic machinery - CONTRIBUTING.md never states the live-proxy proof bar - Reasoning lived only in `CLAUDE.md`, which partners don't read How it solves it: - Adds `adr/` with a process, template, and index - First ADR: how p
## Title fix(http_handler): dispose aiohttp session when `AsyncHTTPHandler` is finalized without a running loop ## Relevant issues Follow-up to the recycle-time disposal fix (#33428 / #32003). That fix covers sessions replaced by `_get_valid_client_session()`; this PR covers the clients that are **never recycled** and still leak. ## Pre-Submission checklist
## TLDR Problem this solves: - An upstream that stops answering wedges MCP tool discovery - Clients see "no tools" and operators only see "was cancelled" - The configured MCP client timeout was ignored entirely How it solves it: - Give the MCP client session a read timeout - Report it as a timeout, never as the gateway's own failure - Log the server and the
## TLDR Problem this solves: - Only admins could add skills, users could not submit - Registering a skill published it instantly, with no review - Nothing tied an approval to the reviewed content How it solves it: - Non-admin submissions land pending and unpublished - Admins approve or reject, with notes - Approval carries a fingerprint of the reviewed manif
## Problem Every model write endpoint (`/model/new`, `/model/update`, `/model/delete`, block/unblock, `clear_cache`) reconciles the in-process `llm_router` against the DB by reading a snapshot of all models and then rewriting router state. That read-modify-write is not serialized, so two concurrent writes interleave: 1. Request A reads the DB snapshot. 2. Re
## TLDR Problem this solves: - Configured defaults overwrite a deliberate "no budget reset" on team and key create - Explicitly sending `budget_duration: null` was indistinguishable from omitting it - Team UI forms had no way to pick or restore "never resets" How it solves it: - `/team/new` and `/key/generate` skip the configured default for an explicitly-nu
## TLDR Problem this solves: - Every trace exported by the `langfuse_otel` callback reports `version` and `release` as null in Langfuse, because litellm writes them to attribute keys Langfuse v4 does not recognize. The values do reach Langfuse, they just land in the generic span attribute bag where nothing resolves them - Team-scoped and key-scoped OTel v2 e
`is_error_str_rate_limit` treats any standalone `429` in the stringified exception as a rate limit, and for openai-compatible providers that check runs before the status-code branch in `_map_openai_exception`. Providers echo the request back in validation errors, so a 400 whose body happens to contain a `429` comes out as `RateLimitError`. Tokenised prompts
## TLDR Problem this solves: - otel v2 sends every Phoenix trace to one env-configured project - v1's per-key/team `phoenix_project_name` routing silently stopped working - clients sending `phoenix_project_name` in request metadata got a 400 How it solves it: - key/team metadata project rides Phoenix's `x-project-name` OTLP header - routed spans root their o
## TLDR Problem this solves: - Muse Spark 1.2 is missing from the cost map - Its calls are billed at $0 - `reasoning_effort` is rejected with a 400 - The cheaper contributor SKU is unpriced too - Web search grounding ($2.50 per 1,000 queries) is billed at $0 How it solves it: - Adds `meta/muse-spark-1.2` at $1.25/$4.25 per M tokens - Adds `meta/muse-spark-1.
## TLDR Problem this solves: - Token counting routes are free but reserve budget - Nothing ever reconciles that reservation, so it leaks - One call can brick a budgeted key at $0 spend - Google `:countTokens` leaks even when the call fails How it solves it: - Skip budget reservation on all token counting routes - Covers `/v1/messages/count_tokens` and both G
## TLDR Problem this solves: - `/mcp` tool calls logged only `content-type` as headers - Custom headers were invisible to callbacks, hooks, guardrails - `/mcp-rest/tools/call` already exposed the full set - Responses API MCP calls logged an empty header dict How it solves it: - Rebuild the synthetic request from the connection's headers - Share one builder w
## TLDR Problem this solves: - Log drawer collapse rows still render on antd - Two shared components duplicate one hand-rolled collapse - Neither row can be opened from the keyboard How it solves it: - Moves both onto the shadcn `ui/collapsible` primitive - Swaps the antd arrow glyphs for lucide chevrons - Each row becomes a real focusable button ## User Flo
## TLDR Problem this solves: - A `langfuse_mask_input: false` request header redacted the prompt it was asked to keep. Header values arrive as strings and the trace path reads `mask_input` / `mask_output` with a bare truthiness check, so the string `"false"` is truthy and the payload is replaced with `redacted-by-litellm` - A `langfuse_update_trace_keys: tra
## TLDR Problem this solves: - A team's own Langfuse keys arrive inside that team's own Langfuse traces - Request metadata carries the whole auth object, unfiltered - `debug_langfuse` ships the same credentials a second time How it solves it: - Emitted blob now comes from StandardLoggingPayload, an allowlist - Steering keys keep reading raw metadata, so beha
## TLDR Problem this solves: - Bedrock Converse chat responses report a 5m/1h cache-write split (`cacheDetails`), but LiteLLM never reads it - Cost calc bills the entire cache write at the 5m rate whenever that split is missing - So every 1h-TTL cache write on the standard Bedrock chat path is undercounted How it solves it: - `AmazonConverseConfig._transform
## TLDR Problem this solves: - Configured `default_priority` caused requests to return HTTP 500 - Scheduled calls never received the computed default priority How it solves it: - Forward the computed priority into the scheduler - Prevent scheduler re-entry while executing queued requests ## User Flow Before: a developer omits priority and the configured defa
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### The Feature Would be grate to allow toolset configuration via config for deployments, api and database is a bit manual, config would be the easiest way to allow for clean deployments that can change stuff without breaking state. ### Motiv
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? ## Description Anthropic Messages passthrough does not honor `use_bearer_for_custom_base` when generating auth headers. For custom Anthropic-compatible `/v1/messages` gateways such as Cloudflare AI Gateway, the upstream exp
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? Spend rows are removed from the in-memory queue before the database write is awaited, so any cancellation of a flush loses them permanently. There is no requeue and no shielding. `update_spend_logs_job` pops the batch under
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? `litellm_deployment_state` is a Prometheus gauge keyed by four labels: litellm_model_name, model_id, api_base, api_provider (litellm/types/integrations/prometheus.py:536). Three different code paths write to it, and they de
## Summary A virtual key's `object_permission.mcp_servers` grants that are added via `POST /key/update` **after** the key was originally created via `POST /key/generate` are not honored by the MCP `tools/list` path — the server ID shows up correctly in `GET /key/info`, and a fresh out-of-process reproduction of the actual permission-resolution functions retu
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? when using the docker image, headroom is answering with a 404 To solve the issue heardoom docker image must be launched with ```txt HEADROOM_COMPRESS_ALLOW_REMOTE=1 ``` ### Steps to Reproduce launch without the variable =>
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate ### The Feature Add an MCP Server filter to the Request Logs UI and the underlying `/spend/logs/ui` and `/spend/logs/v2` endpoints. After selecting a server, provide an optional MCP Namespaced Tool filter populated with tools from that server
### What happened Setting a deployment's `weight` (or `rpm`/`tpm`) via `os.environ/` makes **every request to that model group return HTTP 500**: ``` TypeError: unsupported operand type(s) for +: 'int' and 'str' total_weight = sum(weights) ``` `os.environ/` substitution is string-only, so the numeric router params arrive as `str`. The config loads fine, the
## Relation to #28735 — this is the mirror image, not a duplicate [#28735](https://github.com/BerriAI/litellm/issues/28735) (and the earlier [#8450](https://github.com/BerriAI/litellm/issues/8450), closed as not planned) report the **egress** side: LiteLLM *emits* its synthetic usage chunk with a non-empty `choices`, violating the OpenAI spec and breaking do