Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
43.5K stars7.9K forksPythonApache-2.0data-sciencedeep-learningdeploymentdistributedhyperparameter-optimizationhyperparameter-searchlarge-language-modelsllmkeyword: Ray distributed computingkeyword: Ray framework
Ray Data: In this Ray release, we've enabled DataSourceV2 by default (#64821), so readparquet and friends use the new scan/listing infrastructure with row-group-aware chunking and predicate splitting. Hash Shuffle V2 eliminates the aggregat
HashShuffleV2 supports join (#63598, #64538, #64687). This lets shuffles reuse standard map/reduce scheduling, backpressure, and resource accounting.
Ray Serve: The HAProxy ingress is now distributed as the ray-haproxy PyPI package instead of being compiled into images, and it is the default HAProxy binary (#64141, #64163, #64164). We've also added gRPC support to the HAProxy direct-ingr
Ray Core: We've added an embedded RocksDB storage backend for GCS fault tolerance (REP-64), selectable with RAYgcsstorage=rocksdb and RAYgcsstoragepath (#63657). GCS fault tolerance no longer requires an external Redis instance. We've also
## Description A documentation style and grammar pass over the JAX TPU profiling user guide added in #64735, plus the TPU profiling section that PR added to `profiling.md`. #64735 merged before a docs-team review landed (@rueian had asked for one on the thread). This follow-up applies [the Ray documentation style guide](https://docs.ray.io/en/master/ray-cont
## Description When the async connection to Redis drops, hiredis frees the raw `redisAsyncContext` and nothing puts it back. Every subsequent GCS command fails with `Status::Disconnected` until `num_redis_request_retries` runs out (~3.5s), and gcs_server aborts with `Failed to run redis cmds`. A Redis restart, an idle timeout, a proxy dropping its backend, o
## Summary Adds a backward-compatible `multiplexed_model_ids` field to `ReplicaDetails` and keeps it synchronized when the controller receives updated multiplexed model IDs. Extends the existing multiplexing test to verify the values through `ServeInstanceDetails` returned by `serve_instance.get_serve_details()`. Closes #65347 ## Duplicate check I checked op
## Description Fixes the **server-side** half of #65365. Ray's Python gRPC servers previously read TLS cert/key/CA files once at startup via `grpc.ssl_server_credentials()`. When cert-manager rotates files on disk (e.g. KubeRay mTLS), running servers kept presenting the old certificate until process restart. This PR switches `add_port_to_grpc_server()` to `g
## Summary - return a structured `503 Service Unavailable` when a submission job still exists but its driver agent is no longer registered or reachable - preserve successful log retrieval, unknown-job `404` behavior, and the public SDK's existing `RuntimeError` contract - document the JSON error response and bump the Jobs API version The response does not im
## Summary This PR adds multi-host TPU support to Ray Data LLM batch inference (`vLLMEngineProcessorConfig` / `build_processor`). With a TPU `accelerator_type` and a topology in `accelerator_config`, `build_processor` reserves one slice placement group for the processor, places the engine actor on bundle 0 (CPU only, `capture_child_tasks=True`), and releases
## Description Ray Data currently captures UDF `fn_args` in the serialized UDF wrapper. As a result, direct `ObjectRef` values aren't passed as top-level Ray task arguments: - UDFs must manually call `ray.get()`. - Ray's scheduler can't wait for these dependencies before scheduling map work. - Map workers may reserve CPU or GPU resources and then block insid
## Summary - Implements all stubbed methods in `TaskiqTaskProcessorAdapter`, completing the adapter that bridges Ray Serve's sync `TaskProcessorAdapter` ABC to taskiq's async broker API - Adds 6 E2E tests covering both standalone adapter usage and full Ray Serve deployment integration ## Why is this change needed? PR #61008 (`[taskiq - 2/n]`) added the confi
## Why are these changes needed? The YAML under `doc/redirects/` is the source of truth for the docs.ray.io Read the Docs redirect configuration, managed with [rtd-redirects](https://github.com/anyscale/rtd-redirects). Today a PR that edits only redirect YAML runs no check at all, so a malformed or unreachable redirect can merge unnoticed. Nothing routes red
The `text_embedding_{fixed_size,autoscaling,fixed_size_chaos}` release tests were jailed with `ModuleNotFoundError: No module named 'tf_keras'` / `ValueError: ... Keras 3 ... is not yet supported in Transformers`. Root cause: the imperative `byod_install_text_embedding.sh` pip-installed transformers/sentence-transformers/torch onto the raw cu123 base image,
## Why are these changes needed? Marks Ray Compiled Graphs (cgraph / aDAG) as deprecated, pointing users to **Ray Direct Transport (RDT)** as the successor for direct GPU-to-GPU tensor transfer between actors. The wind-down has already informally started (the GPU cgraph test target is disabled with a "will be deprecated/removed soon" note, and all `compiled_
## Changes - Remove deploy_mode from ServeInstanceDetails, plus the single member ServeDeployMode enum and doc example - Remove the RAY_AGENT_ADDRESS deprecation warning and its three CLI call sites - Remove use_new_handle_api from DeploymentHandle.options - Raise ValueError instead of warning on '#' in deployment names, since '#' is the replica ID delimiter
## Description As mentioned in the issue 64992. `log_to_driver=True` (the default) prepends a `(name pid=...)` prefix to every worker/actor stdout/stderr line forwarded to the driver. There was previously no way to disable *only* that prefix: `RAY_COLOR_PREFIX=0` only strips ANSI color, `log_to_driver=False` drops forwarding entirely (logs never reach the dr
## Description - Adds a `local://` runtime_env URI: `working_dir: local:///app` - Used in place with no uploading or unpacking ## Changes - `Protocol.LOCAL`: `parse_uri` returns the path itself, rejects non absolute paths. - `working_dir`/`py_modules`: no upload or download, 0 bytes cached, never deleted. - new section in the docs.
## Description Running Ray on Kubernetes involves two schedulers. Kubernetes places pods on machines, and Ray places tasks and actors on pods. The docs cover each layer well on its own, but nothing explains how they relate or which one to investigate when a workload doesn't start. The closest existing page is `k8s-autoscaler.md`, but it's scoped to autoscali
## Description `doc/source/ray-core/scheduling/index.rst` carried a standing note that the page lacked an overview of the features and configs that affect scheduling, along with their default values. Most of those defaults are already documented, but they're spread across the resources, labels, and placement group pages, so a reader has no single place to se
## Summary When a node is drained (spot reclaim, compaction, scale-down), its displaced (soft-stopped) HAProxy processes keep serving established connections with a frozen config in which torn-down replicas remain nominally UP. Requests those processes accept are black-holed until `hard-stop-after` instead of failing over. This PR adds an env-gated `observe
## Why are these changes needed? After merging #64175, benchmark testing showed that scaling to 3000 nodes (15000 tasks × 0.2 CPU + 30MB memory) still takes ~51 minutes. Profiling with py-spy revealed two independent hotspots in the scheduling loop: 1. **Pre-filter logic flaw** — fixed in #65171 (OR-across-dimensions → AND-within-shape/OR-across-shapes) 2. *
## Description `WorkerContext::GetGeneratorReturnId` documents that the caller supplies both `task_id` and `put_index` or neither, because deducing only one of them keys the ObjectID to one task while drawing the index from another. The `RAY_CHECK` meant to enforce that is a tautology: ```cpp RAY_CHECK((task_id.IsNil() && !put_index.has_value()) || (!task_id
## Description documents is five, the code is 2 the number is incorrect ## Related issues > Link related issues: "Fixes #65364" ## Additional information update the number
## Description Allow combining the `image_uri` runtime environment field with pip. The runtime env agent prepares one virtual environment per cache key (image digest, Python ABI, Ray version, normalized pip config, and pip install environment) using the image's own interpreter, publishes it atomically under a per-key file lock, and mounts it read-only into e
## Summary Removes the `banned_words` lint check and its Buildkite matrix entry. `ci/lint/check-banned-words.sh` is a `git grep` for three fixed strings across `*.py`, `*.rst`, `*.md`, and `*.ipynb`. Vale already flags all three, at `error` severity, with better messages. This PR deletes the redundant check. ## The three terms, and what catches them today Th
## Why Ray announced the `rayproject/ray-ml` deprecation in #46378 and, in a follow-up comment on that issue, stopped publishing the images entirely as of Ray 2.50 — "not even with `-deprecated` tags." Nothing a reader can see says so. That issue went stale and was closed, and the deprecation never reached the docs. The gap costs the reader real time, becaus
## Description `WebDatasetDatasource._read_stream` emits one single-row DataFrame per sample, which means the output buffer runs `PandasBlockAccessor.size_bytes` once per sample. On large shards this dominates read-side CPU (~63% of the `ReadWebDataset` operator's CPU). This changes the behavior of readwebdataset and let it accumulates 512 samples per emitte
## What Currently, a unary request returns its single response through an interposed async generator instead of returning directly. Instead we split the handler into three pieces to maintain the speed of the unary-unary hot path. 1. `_prepare_direct_ingress_grpc_request` does the setup every request needs regardless of shape. 2. `_direct_ingress_grpc_unary_r
Add deployment_scheduler.py, router.py, and request_router.py to the mypy + pyrefly serve allowlists and fix their type errors. - request_router: declare the members MultiplexMixin/FIFOMixin use from the composed RequestRouter (bare annotations, Callable for methods, so they don't shadow the real methods via MRO), fix the ReplicaID/str key confusion at its s
## Description Some third-party NVML-compatible libraries export 'nvmlDeviceGetProcessesUtilizationInfo' but terminate the calling process when the API is invoked. Because this happens in native code, Python exception handling cannot prevent the Reporter Agent from exiting. ## What changed - Detect incompatible PPU devices before calling the process utilizat
## Why Fixes #64712. The `pool_eligible` check in `NixlTensorTransport` rejected pools whose device differed from the source tensors. `MemoryPoolManager.allocate_for_tensors` already handles cross-device copies via `.copy_()`, so the constraint was unnecessary. Removing it lets a CPU memory pool back GPU tensor sends, which is useful when GPU memory is scarc
Following up on #56195 and making CancelWorkerLease truly idempotent by tombstoning lease ids. There's currently a race with RequestWorkerLease where if the RequestWorkerLease RPC arrives AFTER CancelWorkerLease RPC then we just repeatedly trigger additional CancelWorkerLease RPC requests. We've seen a couple cases where there's a large amount of CancelWorke
## Why this change Ray on Kubernetes docs use "Ray Pod" 89 times, and this PR removes the term. It's the concrete counterpart to the discussion in #65239, where a reviewer asked whether the Kubernetes casing rule should capitalize Pod in "Ray pod". Investigating that question produced a better answer than either casing. **A Pod isn't a Ray node, it's a host
## Why Stacked on #65375, and should merge after it. #65375 fixes a pre-commit `exclude` bug that had kept the `vale` hook from matching any files. Narrowing that exclude so Vale can see `doc/source/data` prose also exposed those files to `trailing-whitespace` and `end-of-file-fixer` for the first time. Rather than make a config-only change rewrite 17 unrela
## Why The embedded RocksDB GCS backend page told readers the feature wasn't in a Ray release yet and pointed its manifest at `rayproject/ray:nightly`. The Ray half of that is no longer true: the backend shipped in **2.57.0** (`_is_rocksdb_gcs` is present in `ray-2.57.0`, absent in `ray-2.56.1`), and `rayproject/ray:2.57.0` resolves on Docker Hub. Beyond the
## Why Three claims in this paragraph are stale or wrong: 1. **The default image is misstated.** The doc says `rayproject/ray:nightly`. The code says `DEFAULT_DOCKER_IMAGE = "rayproject/ray:nightly-py{major}{minor}-cpu"` (`python/ray/autoscaler/_private/fake_multi_node/test_utils.py:26`), formatted with the interpreter's own version at `test_utils.py:195`. 2
## Why this change is needed `lint: doc_readme` carries the `always` tag, so it runs on every pull request, including documentation-prose-only ones. Its inputs are the repo-root `README.rst` and `python/setup.py`. Nothing under `doc/` feeds it, so a prose change cannot change its result. What it actually checks, from `ci/lint/lint.sh`: ```bash doc_readme() {
Two independent cleanups to `python/ray/serve/tests/test_cli.py`. **1. Explicit wait timeouts.** 17 of the file's 41 `wait_for_condition` calls had an effective timeout of 10s, and 12 of those were bare calls inheriting the default rather than stating one. Every call now carries an explicit timeout, and existing values are doubled, matching the convention al
Follow-up to #63273 ([1/2]) completing the TracingConfig feature. - Added a `GLOBAL_TRACING_CONFIG` long-poll namespace. The controller broadcasts the global tracing config which is subscribed by the proxies, which runs `setup_tracing` on delivery. This wires proxy tracing without the init-time controller callback that cannot work (proxies can start before t
## Description This PR modernizes the `setup.py` build configuration script by refactoring legacy `.format()` string formatting to modern Python f-strings. Additionally, it explicitly enforces `encoding="utf-8"` in `find_version()` to ensure cross-platform consistency and prevent encoding errors on non-UTF-8 default environments (e.g., Windows). ## Related i
## Description vLLM's **engine error handler** reads `app.state.server` to flip `should_exit` and stop the uvicorn server its launcher created. `VLLMEngine.build_asgi_app` calls only `build_app` and `init_app_state`, neither of which sets it. So on the direct-streaming path, every `EngineGenerateError` / `EngineDeadError` raises `AttributeError` inside the h
## Description https://github.com/ray-project/kuberay/pull/5121 updated the env var from `RAY_ROOT_DIR` to `STORAGE_ROOT_DIR`. The default storage root dir is now the bucket root itself, so updated example and file structure in the documentation. ## Related issues Related to https://github.com/ray-project/kuberay/pull/5121 ## Additional information
# Description We chose a terse name to be less redundant originally. However, when imported like `from ray.util.tpu import dispatch`, the function is taking a commonly used word away as an identifier and is also ambiguously named at call sites. For readability, rename dispatch to run_on_slice, and keep the former name as an alias for potential existing uses.
## Why are these changes needed? PyArrow HDFS loads `libhdfs`, which embeds a HotSpot JVM in the Python process. HotSpot uses some `SIGSEGV` events as recoverable VM control flow, while Ray's CoreWorker installs an Abseil failure-signal handler that can interpret the same signal as a fatal native crash. The conflict can be established during handler installa
# Issue #65237 — autoscaler requests a bundle larger than the node it was copied from ## 1. Root cause `DefaultClusterAutoscalerV2` builds its autoscaler request out of node *shapes*: `_get_node_resource_spec_and_count()` turns every live worker node's resource dict into a `_NodeResourceSpec`, and `try_trigger_scaling()` sends `spec.to_bundle()` back to the
## Description When multiple raylets share a network namespace and use the same worker port range, they currently allocate ports in the same ascending order. This causes deterministic collisions near the beginning of the range, even when most ports remain available. This PR builds and validates the worker port pool once during raylet startup, then shuffles i
## Description The RuntimeEnv agent caches one logger per job (`_per_job_logger_cache`) and never closes it, and `logging.getLogger` holds a second reference in the global registry. Each logger owns `1 + len(log_files)` file descriptors, so open FDs grow with the cumulative number of jobs the node has handled until every setup fails with `OSError: [Errno 24]
## Description Ray Data can read ORC files, but it can't write them. This adds `Dataset.write_orc()`, backed by a new `ORCDatasink` that writes each output block with `pyarrow.orc.write_table`. The API forwards PyArrow ORC writer options and supports `min_rows_per_file` through the existing block-based file datasink infrastructure. ## Related issues Follow-u
## Description `_setup_logging()` in the KubeRay autoscaler entry point attached a single `logging._StderrHandler` to the root logger, so every record — INFO included — went to the container's **stderr**. Kubernetes' CRI log format tags each line with the stream it arrived on (`TIMESTAMP STREAM TAG CONTENT`), and log collectors derive severity from that fiel
## Why this change is needed `doc/source/conf.py` points `redoc_uri` at the ReDoc CDN's unpinned `latest` tag: ```python redoc_uri = "https://cdn.redoc.ly/redoc/latest/bundles/redoc.standalone.js" ``` `sphinxcontrib-redoc` injects that script tag alongside the spec; it doesn't bundle a renderer. So the published Jobs API page at `cluster/running-applications
## Why this change is needed `summary` isn't a valid field on an OpenAPI Response Object, in either 3.0.x or 3.1. The valid fields are `description`, `headers`, `content`, and `links`. `doc/source/cluster/running-applications/job-submission/openapi.yml` carries 21 of them, and they make the document fail validation against the `3.1.0` version it declares: ``
> **Stacked PR.** This targets `doc-1513-remove-invalid-response-summary` (#65457), not `master`, because that PR makes the spec valid and this one shouldn't inherit a failing validator run. Review #65457 first; GitHub retargets this to `master` automatically when it merges. The two changes are independent in substance and can be split if you'd rather take t