The signatures were verified against the specified public key
What's Changed
chore(release): backport proxy request-handling maintenance and refresh runtime deps for 1.96.1 by @yuneng-berri in https://github.com/BerriAI/litellm/pull/36494
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### The Feature LiteLLM should support adding Claude skills (and similar AI skill plugin integrations) to private repositories. There should be a supported way to authenticate to a private repo, either via SSH key or a GitHub access token, so
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? While trying to add new Custom MCP Server in UI got the error message: Error creating MCP Server: {"detail":{"error":"Error creating mcp server: Could not find field at `createOneLiteLLM_MCPServerTable.data.approval_status`
## Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is
### What happened? I noticed that new proxy configs were not getting picked up, specifically general_settings, but I suspect it will apply to most of what inside table LiteLLM_Config. In litellm/proxy/proxy_server.py, the function_update_config_fields is pulling in values from config and overwriting the newly deployed values. That or the initial update, at d
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? Please remove this line it getting flooded in logs ### Steps to Reproduce 1. Use and non openai openrouter model in litellm ### Relevant log output ```shell rovider List: https://docs.litellm.ai/docs/providers Provider List
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? When streaming Azure Responses API requests through the LiteLLM proxy, it sends `response.output_text.delta` events but **omits the required setup events** (`response.created`, `response.in_progress`, `response.output_item.
## TLDR Problem this solves: - Anthropic streaming with a Bedrock guardrail returns a 500 - The stream arrives as SSE frames the assembler cannot read - Output scanning never runs, so nothing is guarded either How it solves it: - Assemble the SSE frames into a response, then scan as usual - Convert back to SSE frames only when the guardrail rewrote the text
Anthropic's API now supports automatic prompt caching via a top-level `cache_control` field in the request body. This removes the need to manually annotate individual content blocks — the API automatically places the cache breakpoint at the last cacheable block. Changes: - Add `cache_control` to `AnthropicMessagesRequestOptionalParams` TypedDict - Add `cache
## What Adds an opt-in cloud-signing path to the asqav callback (`litellm/integrations/asqav/asqav.py`). When `ASQAV_API_KEY` and `ASQAV_AGENT_ID` are both set, each record's digests are POSTed to the asqav agent sign endpoint (`POST /api/v1/agents/{agent_id}/sign`) and the returned signature id and verification url are bound into the local JSONL line. With
## Relevant issues This is a copy of #28054 by @ririnto, rebased onto the current `litellm_internal_staging` so it can run through our CI. All credit for the implementation goes to @ririnto; this branch only resolves the merge conflict against the moved base and normalizes formatting to the current ruff - Closes #28053 - Addresses #25666; restores Claude `th
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### The Feature **Description:** Currently, when LiteLLM invokes an A2A (Agent-to-Agent) agent, it has limited support for custom headers. While some static headers can be configured via `config.yaml`, there is no flexible way to provide head
## TLDR Problem this solves: - Provider sunset dates ship in the registry but stay invisible - Operators find out a model died when calls fail - No lead time to test and migrate replacements How it solves it: - New `GET /model/deprecations` buckets configured models by urgency - Daily Slack alert lists deprecated and imminent models - Per deployment `model_i
_Reopened from #26491 after renaming the branch from `sandcastle/compat-matrix-stack` to `litellm_compat_matrix_stack` so it matches the `/litellm_.*/` CircleCI filter. GitHub's branch-rename API failed to update the original PR's head ref, so the PR was detached on the old SHA. Conversation history lives on #26491._ --- ## Relevant issues Implements the v0
## Relevant issues <!-- e.g. "Fixes #000" --> ## Linear ticket <!-- if you are an internal contributor (e.g., your username is postfixed with -berri or -berriai), add "Resolves " followed by the Linear ticket e.g. "Resolves LIT-1234" to magically link the Linear ticket to the GitHub PR --> ## Pre-Submission checklist **Please complete all items before asking
## TLDR Problem this solves: - Some providers discount per-token rates during a daily window - The cost map only modeled one static rate per model - Logged spend therefore overstates cost inside that window How it solves it: - A model entry can carry an `off_peak_pricing` block - `hours_utc` takes one `"HH:MM-HH:MM"` window or a list - Windows may wrap past
## TLDR Problem this solves: - Admins cannot answer "is it safe to turn the auto-router on?" before routing production traffic through it - Prior attempts (#36250, #36571) accumulated counters, a status column, and lifecycle loops whose cross-pod consistency drew review round after review round How it solves it: - A shadow eval job samples a slice of one key
### What happened? During Tool call (Steam and also in nonStream) there is an exception when using Gemini models with OCI provider. ### Relevant log output ```shell ... litellm-1 | 14:45:17 - LiteLLM Router:INFO: router.py:1622 - litellm.acompletion(model=oci/google.gemini-2.5-pro) Exception litellm.APIConnectionError: 4 validation errors for OCICompletionRe
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? When routing requests through LiteLLM with model `chatgpt/gpt-5.2-codex`, requests that include a system message fail with: `litellm.BadRequestError: ChatgptException - {"detail":"System messages are not allowed"}` I’m tryi
Fixes #31206 ## Problem When `REDIS_CLUSTER_NODES` is set, `get_redis_connection_pool()` returns `None` because the cluster-mode path exits early: ```python # litellm/_redis.py if "startup_nodes" in redis_kwargs: return None # cluster mode skips the connection pool ``` This causes `RedisCache.__init__` to store `self.async_redis_conn_pool = None`. At shutdow
## Summary Fixes #25308 ### Root cause `claude-haiku-4-5-20251001` and `claude-haiku-4-5` (the direct Anthropic API model IDs) were missing `supports_native_structured_output: true` in `model_prices_and_context_window.json`, even though the model supports it (all the Bedrock and regional cross-region variants already had the flag set correctly). When a calle
## Relevant issues Fixes #32242 ## Linear ticket N/A ## Pre-Submission checklist - [x] I have added meaningful tests - [x] My PR passes all CI/CD checks (e.g., lint, format, unit tests) - [x] My PR's scope is as isolated as possible; it only solves 1 specific problem - [x] I have requested a Greptile review by commenting `@greptileai` and received a **Confid
## Relevant issues Fixes #33974 ## Linear ticket ## Pre-Submission checklist **Please complete all items before asking a LiteLLM maintainer to review your PR** - [x] I have added meaningful tests - [x] My PR passes all CI/CD checks (e.g., lint, format, unit tests) - [x] My PR's scope is as isolated as possible; it only solves 1 specific problem - [x] I have
## TLDR Problem this solves: - Destinations can only be managed by hand through the credentials API How it solves it: - Adds destinations to Logging & Alerts, with their backend and scope - Lets an admin create one and edit which teams or orgs it reaches - Keeps the existing proxy-wide OTEL callbacks as separate dropdown entries ## Relevant issues ## Linear
## Summary When running the LiteLLM proxy with multiple uvicorn workers (`--num_workers > 1`) and Prometheus callbacks enabled, Prometheus metrics are silently lost because each worker process maintains its own metrics registry. This PR auto-detects this scenario in `proxy_cli.py` and creates a temporary shared directory for `PROMETHEUS_MULTIPROC_DIR`, enabl
## Description This PR enables correct routing for GET requests to the \/vllm/*\ passthrough endpoints (e.g., \/vllm/metrics\). ## Problem Previously, \ llm_proxy_route\ relied on \is_passthrough_request_using_router_model\ which only checked the request body for a \model\ field. Since LiteLLM's \get_request_body\ ignores bodies for GET requests, these reque
## Relevant issues Fixes #27197 ## Linear ticket N/A ## What changed - Wired the existing Bulk Invite Users CSV template generator to the `Download CSV Template` button. - Added a regression test that opens the modal and verifies the CSV template download flow creates a Blob URL, clicks the generated anchor, and revokes the URL. ## Why The component already
## Relevant issues ## Linear ticket Resolves LIT-2877 ## Pre-Submission checklist - [x] I have added testing in `tests/test_litellm/proxy/agent_session_endpoints/` (39 tests, all passing) - [x] PR passes the new module's unit tests (`pytest tests/test_litellm/proxy/agent_session_endpoints/`) - [x] PR scope is one specific thing: ship the agent/session/run wi
Fixes #31030 ### Description This PR fixes a bug in the Anthropic pass-through endpoint where the global `drop_params=True` and `additional_drop_params` settings were completely ignored regardless of the downstream provider. This caused unsupported parameters like `thinking` and `context_management` to be blindly forwarded to Vertex AI and Bedrock for models
## TL;DR Fixes non-admin key updates that returned 403 when the dashboard submitted a safe access preset, while preserving administrator control over management and custom route allowlists. Fixes #34975. ## What changed - add `key_type` to key update requests and generated UI types - normalize dashboard route edits through a dedicated helper - submit exact p
## TLDR Problem this solves: - Classifier rubric is calibrated to consumer chat - Its top tier reads "non-trivial code, multi-step technical work" - That is the median request in developer and agent traffic - So routine engineering routes to the most expensive tier - Prose rules did not move the boundary; only examples did How it solves it: - Adds calibratio
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? When using Xiaomi MiMo models (`MiMo-V2-Pro` and `MiMo-V2-Omni`) with the `output_config` parameter, requests fail with: ```javascript Xiaomi_mimoException - AsyncCompletions.create() got an unexpected keyword argument 'out
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### The Feature DeepSeek charge different rates for the same model depending on the time window: off-peak hours: lower input/output token cost peak hours: standard or higher input/output token cost Currently, LiteLLM's model_info pricing fiel
### What happened? A model group configured as LiteLLM's native adaptive router returns **HTTP 500 on every request, permanently**, with: ``` gammavariate: alpha and beta must be > 0.0 ``` surfacing from `litellm.router.py::async_function_with_fallbacks()`. Once it starts it never recovers, including across proxy restarts. Deleting one row from `LiteLLM_Adap
## Summary Fixes LIT-2969 Google's v1beta Interactions API introduces breaking changes on May 26 (default flip) / June 8 (sunset): - `outputs` array replaced by `steps` array in all responses - `response_mime_type` removed; `response_format` becomes polymorphic; `image_config` moves out of `generation_config` This PR migrates LiteLLM to the new schema while
## Relevant issues Closes #30311 ## Linear ticket n/a (community contribution) ## Pre-Submission checklist - [x] I have added meaningful tests - [x] My PR's scope is as isolated as possible; it only solves 1 specific problem - [x] Greptile review complete: **5/5 Confidence Score** ([review link](https://github.com/BerriAI/litellm/pull/30325#issuecomment-4696
## Summary `get_optional_params_embeddings` did not apply `drop_params` to `dimensions` for Azure and OpenAI-compatible providers. Requests for legacy embedding models could retain an unsupported `dimensions` field even when the caller or global configuration asked LiteLLM to drop unsupported parameters. Related context: #26787. That issue is now closed as s
## Summary Adds `aws_session_tags` support to Bedrock's STS role assumption flow, enabling per-user/per-team cost attribution in CUR 2.0 billing data. **Problem:** When `aws_session_tags` is passed in `optional_params`, it is not recognized as an AWS authentication parameter. It falls through to `additional_request_params` and gets forwarded in the Bedrock r
## Relevant issues <!-- e.g., "Fixes #000" --> ## Linear ticket <!-- if you are an internal contributor, add "Resolves " followed by the Linear ticket e.g., "Resolves LIT-1234" to link the Linear ticket to the GitHub PR. If you don't have one, leave the section blank rather than guessing --> ## Pre-Submission checklist **Please complete all items before aski
## TLDR Problem this solves: - Hanging-request Slack alert always shows empty Key Alias / Team Alias - Metadata is read from the wrong location at proxy pre-call time How it solves it: - Fall back to the top-level request metadata when the resolved metadata is empty - A populated `litellm_params.metadata` is always preferred (guard on emptiness) ## Relevant
## TLDR Problem this solves: - Streaming responses emit zero bytes during a long time-to-first-token - Idle-timeout hops then abort a connection that is perfectly healthy - Existing keepalive settings act on sockets, not on the response How it solves it: - Optional interval emits an SSE comment whenever the stream is silent - Races the timer alongside the ex
*Re-lands #30273 (reverted during staging)*. No functional changes to the original; rebased onto current main and refactored so the diff is purely additive. ## The problem Claude Code 2.1.126+ added gateway model discovery: when ANTHROPIC_BASE_URL points at a gateway, it queries {base_url}/v1/models at startup and populates the /model picker with the discove
## TLDR Problem this solves: - OTEL v2 could only export to one proxy-wide destination set, so different teams could not send traces to their own backends - an admin-owned destination decided which logger owned a backend proxy-wide, so registering one destination scoped to a single team rewrote every other tenant's telemetry pipeline for the life of the proc
## TLDR Problem this solves: - A team cannot tell which destinations receive its traces How it solves it: - Adds `resolved_logging_exporters` to `/team/info` and `/organization/info` - Names only; endpoints, headers and the access map stay admin information - Lists a destination only when it would actually fire for that identity ## Relevant issues ## Linear
## Relevant issues Closes #10177 Supersedes / unblocks [#18293](https://github.com/BerriAI/litellm/pull/18293) if that branch stays stale — **credit to [@CryptoCanuck](https://github.com/CryptoCanuck)** for the implementation. ## Why this PR [#18293](https://github.com/BerriAI/litellm/pull/18293) already has Greptile 5/5 and the right design (`useDarkMode`,
## TLDR Problem this solves: - CustomLogger has no per-attempt deployment failure signal, only a request-level one gated to fire once - Fallback chain attempts past the first are invisible to callbacks that need a per-deployment count How it solves it: - Adds async_post_call_failure_deployment_hook, completing the existing pre-call/post-success hook pair - F
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### The Feature Recently, with [#21560](https://github.com/BerriAI/litellm/issues/21560), priority/flex paygo pricing was added for Vertex AI. While this is correctly configured for newer models (like Gemini 3 and 3.1), it is missing for gemi
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? What I'm tryin to achieve? Following the guide here to setup the ephemeral token minting for setting up the webrtc connection to azure openai. https://docs.litellm.ai/blog/realtime_webrtc_http_endpoints My litellm version:
### Check for existing issues - [x] I have searched the existing issues and checked that my issue is not a duplicate. ### What happened? When using azure_ai/model_router/..., LiteLLM correctly returns the actual selected model in the API response, but /spend/logs stores the router deployment model instead. Example: - actual response model: - azure_ai/gpt-5.4
## Summary GPT 5.5 on Amazon Bedrock runs on a new platform (Bedrock Mantle) that **only supports the Response API** (`/v1/responses`). It does NOT support the Chat Completions API (`/v1/chat/completions`). Our enterprise customers use LiteLLM as a unified proxy and call all models via `/chat/completions`. Currently GPT 5.5 on Bedrock cannot be used through