The signatures were verified against the specified public key
What's Changed
chore(release): backport proxy request-handling maintenance and refresh runtime deps for 1.96.1 by @yuneng-berri in https://github.com/BerriAI/litellm/pull/36494
## Relevant issues Fixes #33987 ## Linear ticket ## Pre-Submission checklist - [x] I have added meaningful tests - [x] My PR passes all CI/CD checks (e.g., lint, format, unit tests) - [x] My PR's scope is as isolated as possible; it only solves 1 specific problem - [x] I have received a Greptile **Confidence Score of at least 4/5** before requesting a mainta
## Summary - Repeated managed batch retrieve could nest a unified `output_file_id` / `error_file_id` inside another unified ID. - `model_mappings` then pointed at a managed ID instead of the raw provider file/gs URI, so content download failed. - Add `_unwrap_managed_output_file_id` to peel nested managed output IDs, keep the outermost response ID, and alway
## TLDR Problem this solves: - Teams granted a config model access group saw empty Models + Endpoints - A model sharing a group's name was hidden from listings yet callable How it solves it: - Resolve team.models through the same resolver runtime auth uses - Listings keep a literal model whose name collides with a group ## User Flow Before: a team member who
## TLDR Problem this solves: - guardrails-monitor, projects and logs still render antd and Tremor - an antd-only spinner primitive sat inside the shadcn `ui/` folder - three of the migrated components had no test at all How it solves it: - migrates the files those three routes exclusively own onto shadcn - deletes `AntDLoadingSpinner`, moving its one call si
## TLDR Problem this solves: - Concurrent logging crashes sync success/failure handlers - `RuntimeError: dictionary changed size during iteration` silently drops logs How it solves it: - Snapshot `model_call_details.items()` with `list()` before iterating ## Relevant issues Fixes #34719 ## Linear ticket ## Pre-Submission checklist - [x] I have added meaningf
## Summary Fixes #34906 `mcp_read_resource()` rejected any call when more than one MCP server was allowed in the session, even though the requested URI is unambiguous and `read_resource` is a read-only operation. This broke MCP Apps UI resources (`_meta.ui.resourceUri`) as soon as a second server was aggregated. ## Fix Instead of rejecting based on session s
## TLDR Problem this solves: - Default Team Settings was a hand-rolled useState form: no validation, no tests - Its permission list had drifted; 4 selectable routes were missing - Third consumer needed to keep qualifying the shadcn + RHF + zod form pattern How it solves it: - Rebuilds the form on the default-user-settings template with zod validation and ful
## TLDR Problem this solves: - `POST /v1/batches` (or the SDK equivalent) against a Vertex AI model with `vertex_location: 'global'` is silently accepted at request time, then forwards a URL containing `/locations/global/batchPredictionJobs` to Vertex, which responds with HTTP 400 downstream - The failure surfaces only after the request has already been acce
## Title fix(proxy): guard optional `prisma` import in DB exception classifiers ## Relevant issues Fixes #35457 ## Pre-Submission checklist - [x] I have Added testing in the `tests/litellm/` directory, Adding at least 1 test is a hard requirement - see details - [x] I have added a screenshot of my new test passing locally - [x] My PR is based on the `main` b
## TLDR Problem this solves: - Same deployment logs blank `model_id` on some rows, hiding real spend - `model` also drops its `openai/` prefix on those rows How it solves it: - Carry the router's deployment identity into `litellm_metadata` - So spend logs are identical whether or not a request sends `litellm_metadata` ## Relevant issues Fixes #35472 ## Linea
## Problem On a proxy started **without** a `DATABASE_URL` (a master-key-only deployment — a documented, supported configuration), **every** auth failure returns an unrelated HTTP **500** instead of a **401**. This includes a plain unauthenticated request: ```yaml model_list: [] general_settings: master_key: os.environ/LITELLM_KEY ``` ``` $ curl -s -o /dev/n
## Problem fixes https://github.com/BerriAI/litellm/issues/35860 Six proxy routes returned an opaque **500** when a client omitted a required body field, instead of a deliberate **4xx**. The e2e release gate caught this — 7 `test_missing_*_returns_error` / `test_search_missing_query_returns_error` tests in `tests/e2e/llm_translation/` require a 4xx (both `as
## TLDR Problem this solves: - AWS raised gpt-5.6 sol/terra/luna on Bedrock to 1M context - our cost map still caps them at 272k input tokens - router pre-call checks reject long-context requests as too big How it solves it: - bump `max_input_tokens` to 1000000 for the three bedrock_mantle entries - same bump in the bundled backup cost map ## Relevant issues
## TLDR Problem this solves: - AnthropicConfig.transform_request built the request dict as {model, messages, **optional_params}, so system and tools (added via optional_params) landed after messages in the serialized JSON body - On the Vertex AI global endpoint, Anthropic's prompt cache keys off the raw request bytes, so a stable system/tools prefix only hit
Fixes #35937 ## Problem `get_form_data()` in `litellm/proxy/common_utils/http_parsing_utils.py` (used by `/v1/audio/transcriptions`) called `dict(form)` on Starlette's `FormData` object before iterating it. `FormData` is a multidict that can hold multiple values under the same key, but `dict()` collapses repeated keys and silently keeps only the last one. Th
## TLDR ### What does this PR do? Fix async image generation dropping an explicitly supplied `custom_llm_provider` before provider resolution ### Problem `aimage_generation()` resolved the provider using only `model` and `api_base` For a custom OpenAI-compatible image model with a bare model name, the explicit provider was ignored and LiteLLM raised `LLM Pro
## TLDR Problem this solves: - Nimble's Search API is not reachable through LiteLLM - No cost tracking or dashboard entry for Nimble searches How it solves it: - Adds `NimbleSearchConfig`, the 18th native search provider - Registers it in the enum, config map, pricing map, and dashboard Disclosure: I work at Nimble. The pricing entry is our public pay-as-you
## TLDR Problem this solves: - The daily compat-matrix publisher only exists on an unmerged PR checkout - It runs the stale `tests/claude_code` suite, not `tests/e2e/claude_code` - The published matrix misses the GPT columns the e2e suite already tests How it solves it: - Ports the cron publisher to `tests/e2e/claude_code/cron_vm/` - The runner shims the e2e
## TLDR Problem this solves: - Successful passthrough requests leak budget reservations into Redis - The shared spend counter drifts above real spend on every request - Keys hit false BudgetExceededError while Postgres spend stays tiny How it solves it: - Passthrough metadata now carries the pre-call budget reservation - The success cost callback can reconci
## TLDR Problem this solves: - Module always created its own VPC, subnets, NAT - No way to reuse an existing Postgres - No way to reuse an existing Redis - Customers cannot deploy in locked-down accounts How it solves it: - `vpc_id` + subnet id inputs skip all networking - `create_database` false takes a `database_url` - `create_redis` false takes a `redis_u
## TLDR Problem this solves: - Vertex and Anthropic each copy the whole SSE-fragment buffer on every chunk, O(n^2) in payload size - Anthropic also has no parse-deferral heuristic and can wedge forever on two concatenated JSON envelopes How it solves it: - Add a shared JSONFragmentAccumulator in litellm_core_utils/, an O(1)-append buffer with a peel-one-valu
## TLDR Problem this solves: - Key fallbacks saved at creation vanish on reopen - No way to review or change them afterwards How it solves it: - Key info panel now lists saved router settings - Key edit form now embeds the router settings editor - Cleared settings are sent, unrelated edits leave them alone ## User Flow Before: an admin who created a key with
## TLDR Problem this solves: - Alias param forwarding read the first same-name deployment, not the marker - A plain sibling's api_base and api_key leaked onto routed tier calls - Whether routing worked depended on model_list order How it solves it: - Forward params from the `auto_router/` marker entry, matched by the selected strategy's tags - Never forward
## TLDR Problem this solves: - Admin Settings still used legacy dashboard primitives How it solves it: - Moves eight form-free views onto installed shadcn primitives ## User Flow Before: a proxy admin can manage Admin Settings, but the SSO, Vault, and UI Settings views use the legacy visual system 1. They open `http://localhost:4000/ui/?page=admin-panel` 2.
## TLDR Problem this solves: - Redaction deepcopies binary response bodies it cannot redact - Those hold a thread lock, so the copy raises - The raise aborts success logging for that request How it solves it: - Decide redactability before copying, not after - Unrecognized shapes already discard the copy anyway ## User Flow Before: an admin who has message re
## TLDR Problem this solves: - Default Team Settings still used legacy dashboard UI primitives How it solves it: - Moves the route-owned settings surface to installed shadcn primitives - Preserves loading, view, edit, permission, save, and cancel behavior ## User Flow Before: a proxy admin can manage default team settings, but the settings surface follows th
## TLDR Problem this solves: - Policies blast-radius details still used legacy UI primitives How it solves it: - Moves the route-owned impact popover onto shadcn components - Preserves loading, empty, global, key, and team states ## User Flow Before: a proxy admin can inspect policy impact, but the blast-radius popover follows the legacy dashboard visual sys
## TLDR Problem this solves: - Anthropic `image` blocks crash token counting - `/v1/messages/count_tokens` returns 500 for image messages - Context-window check silently skipped when a message carries an image How it solves it: - Count `image` blocks through the existing image path - Handle all three sources: base64, url, file - Nested `tool_result` image bl
## TLDR Problem this solves: - A 400 for an unparsable body leaves no Request Logs row - Failure filter returns "No logs found" for a client-visible failure - The rejection happens in auth, before any endpoint logs it How it solves it: - Auth reports that rejection through the endpoints' failure hook - The row carries failure status, code 400, and the client
## TLDR Problem this solves: - Key router settings vanish when a key is reopened - Saved fallbacks cannot be verified or edited - A fixed-field editor would wipe unlisted routing fields How it solves it: - Key info panel summarises the stored router settings - Edit view embeds the router settings accordion - Editor value merges over the stored object, never
## TLDR Problem this solves: - Saving retry settings wipes all configured model group aliases - Aliases and retry settings share one stored settings row How it solves it: - Stop the alias field defaulting to an empty map - An unset alias is now omitted, so the merge keeps it ## User Flow Before: an admin who set up model group aliases loses every one of them
## TLDR Problem this solves: - Deleting a parameter in the model editor doesn't remove it - The saved config keeps sending the old value How it solves it: - The editor now sends a deleted param as an explicit null - The update endpoint clears any param explicitly set to null ## User Flow Before: an admin who deletes a parameter from a model's LiteLLM Params
## TLDR Problem this solves: - A staggered elected job runs once per replica, not once - Its lease dedupes for the body's runtime, not the TTL - A 10-pod fleet can run a daily job ~10 times a day - Staggering these bought nothing: one replica does the work How it solves it: - An elected job offsets by job id alone, not by pod - Every replica shares one insta
## TLDR Problem this solves: - A batch created through a passthrough with a tag containing a NUL code point is never cost tracked. PostgreSQL rejects NUL in jsonb with `22P05`, and because the tags go into the managed object's CREATE payload, the error aborts the entire row insert rather than just that column - With no `LiteLLM_ManagedObjectTable` row, `Chec
## TLDR **Problem this solves:** - `/spend/logs/v2` only filters by `startTime`, making long-running requests invisible for billing when scanned by completion time windows - Requests that start at 14:30 and end at 14:34 are lost because they appear with `startTime=14:30` but are scanned at 14:34-14:36 **How it solves it:** - Adds `filter_time_by` parameter w
## TLDR Problem this solves: - One deployment's `model_info` overwrote its siblings' metadata - Cached backend model metadata got rewritten per request - Nested metadata was shared, so merges mutated the cache How it solves it: - Return a fresh merged mapping instead of updating the cache - Deep-copy cached backend info before merging custom info ## User Flo
## TLDR Problem this solves: - Mixed Cohere inputs become invalid `texts` objects - Structured responses can tokenize non-string input objects How it solves it: - Preserve mixed content in Cohere's `inputs` field - Use Cohere billing metadata for multimodal usage ## User Flow Before: a developer embedding a product caption and image receives a provider valid
## TLDR Problem this solves: - Nemotron 3.5 Lightning is unmapped in the cost map - OpenRouter and DeepInfra calls log at $0 spend - Context window and tool/reasoning support resolve as unknown How it solves it: - Adds the OpenRouter and DeepInfra ids at $0.05/$0.20 per M - Keeps the packaged backup cost map in sync ## User Flow Before: a developer adding NV
## TLDR Problem this solves: - Bedrock GPT-5.6 now allows 1M input tokens - Our map still capped Sol/Terra/Luna at 272K - Prompts past 272K were billed at short-context rates How it solves it: - `max_input_tokens` raised to 1000000 for all three - Added `*_above_272k_tokens` input, cache and output rates - Backup price map kept in sync ## User Flow Before: a
## Summary Enable `reasoning_effort` for Mistral chat models that Mistral documents as supporting it, not just the models whose name contains `magistral`. ## Problem `POST /v1/chat/completions` with `model=mistral-medium-3-5` and `reasoning_effort` fails with: ``` litellm.UnsupportedParamsError: mistral does not support parameters: ['reasoning_effort'], for
## TLDR Problem this solves: - Cohere Embed v4 rejects multimodal inputs sent as texts How it solves it: - Preserves mixed content under Cohere's inputs field ## User Flow Before: a developer requesting a multimodal embedding receives a provider validation error 1. They call `litellm.embedding` with text and image content 2. Cohere receives the object under
## TLDR Problem this solves: - Pricing schema rejected `cache_creation_input_token_cost` inside tiers - Cache-creation tokens were never billed at tier rates - Dashscope tiers were billed graduated, not all-or-nothing - Tiered-only models (volcengine) were billed at $0 How it solves it: - Schema generator and committed schema accept the tier field - Tier sel
## What does this PR do? `ComplexityRouter` (and `QualityRouter`, which delegates to the same shared scorer) scores the **full text** — system prompt + user message concatenated — when computing four of its five keyword-match complexity signals: `codePresence`, `technicalTerms`, `simpleIndicators`, and `multiStepPatterns`. Only `reasoningMarkers` was already
## TLDR Problem this solves: - Backend Any counts keep drifting toward their basedpyright ceilings - 16 hotspot files carried 1,328 reportAny/reportExplicitAny errors How it solves it: - Real types at each Any seam: TypedDicts, Protocols, precise annotations - Zero casts, zero ignores, zero new Anys; the only suppression comments are five `writable-ok` reaso
## TLDR Problem this solves: - Google replaced the role-tagged turn with typed steps - We still publish the turn list Google now rejects - Replayed history reaches other providers as all-user - Compliance tests fetched the spec live, so staging went red How it solves it: - Vendor the spec, read it from disk - Publish the step shapes, keep the old turn workin
## TLDR Problem this solves: - Users with multiple Entra app roles get an arbitrary one - Entra does not guarantee `roles` claim ordering - `proxy_admin` can silently lose to `internal_user` - Only Microsoft SSO is affected; generic/Okta already ranks correctly How it solves it: - Reuse the existing privilege hierarchy for app roles - Highest privilege wins,
## TLDR Problem this solves: - Bitdeer AI's OpenAI-compatible inference API has no named provider in LiteLLM - Callers must manually pass a custom api_base/api_key today How it solves it: - Registers bitdeer-ai in the JSON-configured provider system (providers.json) - Adds pricing for 3 models to model_prices_and_context_window.json ## User Flow Before: a de
## TLDR Problem this solves: - Ollama tool schemas disappear from hidden log parameters - The dashboard can misdiagnose valid tool requests How it solves it: - Preserve parameters before provider request transformation - Cover request and log schema retention together ## User Flow Before: a developer sees an empty tool schema in the dashboard even though Oll