Fixed connection-state handling in squashed pipelines. Commands following AUTH, SELECT, HELLO, CLIENT, or RESET now observe the updated connection state instead of stale authentication, database, or RESP protocol state (#8016).
Fixed compressed QList node memory accounting and defragmentation (#8011, #8014).
fixes: #7994 Summary: This PR replaces the fixed per-shard replication backlog entry count with time- and byte-based retention. Changes: - Adds a five-second default age target via --shard_repl_backlog_time_ms. - Adds --shard_repl_backlog_max_bytes, defaulting to 0.5% of maxmemory. - Deprecates --shard_repl_backlog_len and updates stale-partial-sync guidance
A client blocked with `XREADGROUP ... BLOCK 0 STREAMS <key> >` remains blocked when the watched stream becomes invalid. Operations such as `DEL`, expiration, `FLUSHDB`/`FLUSHALL`, or replacing the stream with another data type do not trigger reevaluation of the blocked command. With `BLOCK 0`, the connection can remain blocked indefinitely. Reproduction: Cre
## Summary `FT.CREATE` accepts an index definition that omits the required `SCHEMA` keyword. It returns `OK` and exposes the index in `FT._LIST`, but the resulting index is unusable: `FT.INFO` reports that the index does not exist. The command should be rejected during parsing/validation rather than creating a listed-but-non-operational index. ## Reproductio
## Unconditional skips (broken / flaky / WIP) - [ ] `tests/dragonfly/shutdown_test.py:15` — class `TestDflyAutoLoadSnapshot` (covers `test_gracefull_shutdown`) — "Currently we can not guarantee that on shutdown if command is executed and value is written we response before breaking the connection" @vyavdoshenko -> check if the invariants/what's written is tr
**Describe the bug** On a RESP3 connection, a blocking `XREAD` that is woken by a new entry replies in the RESP2 array shape instead of the RESP3 map shape. Real Redis replies with a map on the same path. This breaks clients that trust the negotiated protocol. `node-redis` selects its reply parser from the `HELLO` version, so it applies the RESP3 transform t
#### 1. P1 — A heterogeneous blocking queue can hide `XREADGROUP` forever `BlockingController` keeps one FIFO queue per `(db, key)` for all blocking commands. `NotifyWatchQueue()` checks only the front waiter; when its checker returns `kKeyNotFound` (BLPOP / BZPOP\* / BLMPOP on a missing key), the whole scan aborts (`blocking_controller.cc:259-265`) and the
`NotifyPending()` iterates the live `awakened_indices_` / `awakened_keys` sets with no reentrancy guard and no snapshot (`blocking_controller.cc:150-182`). The stream readiness checker calls `FindReadOnly()` (`stream_family.cc:3032`), which can enter `ExpireIfNeeded()` (`db_slice.cc:1451-1493`); that path can suspend **before** deleting the entry — in `Recor
Heartbeat expiry and eviction scan only the default namespace (`engine_shard.cc:784-785`, `862-864`), and the post-expiry wake dispatch uses only the default namespace's `BlockingController` (`engine_shard.cc:959-964`). Keys in ACL namespaces are never actively expired at all — no deletion, no keyspace notification, no wake. Only a later tenant command that
`HasEntries2()` creates or updates the consumer while checking readiness (`stream_family.cc:4042-4055`), and the single-shard path performs `OpRead()` and journals group side effects (`stream_family.cc:3262-3272`) before the accumulated error is checked (`stream_family.cc:3285-3287`). Pre-existing bug, not introduced by the wake fixes. Scenario: ```text XGRO
The wake path resolves only `Transaction::GetWakeKey()` (`stream_family.cc:3139-3142`) and builds a reply with cardinality exactly one (`stream_family.cc:3221-3229`). The root cause is in the transaction layer: `NotifySuspended` claims a once-only blocking barrier and records a single wake key (`transaction.cc:1578`, `1598`), so the woken command structurall
For a woken blocked read, per-shard validation statuses are scanned in numeric shard-ID order (`stream_family.cc:3119-3135`), so the reported error is decided by key-to-shard hashing. Within a single shard, argument order is respected — the defect is strictly cross-shard. The initial non-blocked path is worse: the aggregate error (`AggregateValue`, `executio
`cluster_family.cc:519` takes the DbSlice from `namespaces->GetDefaultNamespace()` instead of the calling transaction's namespace. A namespace-bound ACL user with `+@admin` / `+dflycluster` can invoke the command; it then flushes default-namespace data while the tenant's keys in those slots survive indefinitely (including past slot migration), and blocked te
The woken blocked read runs a cross-shard validation hop and then an action hop that revalidates only the wake-key shard (`stream_family.cc:3117-3137`, `3156-3164`); `FlushSlotsFb()` deletes slot keys in a detached fiber without transaction locks and yields between chunks (`db_slice.cc:955-985`). A flush landing between the two hops can delete a sibling stre
Under RESP3, XREAD and XREADGROUP return *-1 when no entries are available or a BLOCK timeout occurs. Redis returns the RESP3 null value (_\r\n) instead. Update these paths to emit RESP3 null replies while preserving *-1 for RESP2.
<!-- **Commits Must Be Signed and Your PR title must conform to the conventional commit spec** * See: https://github.com/dragonflydb/dragonfly/blob/main/CONTRIBUTING.md * Please follow the section on `pre-commit hooks`, a linter will validate before you push Example PR Title: <type>(<scope>)!: <description> * `type` = bug, chore, feat, fix, docs, build, styl
Preserve the logical type of offloaded values when serializing snapshots, full-sync streams, and DUMP payloads. Previously, offloaded listpack hashes were read as strings. Debug builds aborted on a DCHECK, while release builds serialized the raw listpack bytes as a STRING. Root cause: `SerializerBase` routed every external value through `ReadTieredString` an
## Summary - `Renamer::FinalizeRename()` ran `DelSrc` and `DeserializeDest` in the same transaction hop with no ordering guarantee between shards. If `DeserializeDest` failed (e.g. OOM), the source key could already be deleted, silently losing its data. - Now deserializes into the destination first, and only deletes the source once that succeeds. `COPY` is u
The changes that I merged into 1.40 branch Only functional change: 1. Fixes small bins defrag by using bounded TraverseBySegmentOrder to avoid blowing up iteration costs when the table is largely empty Metrics & config: 1. Exports missing tiering fields from info to metrics 2. Adds new configurable values to allow limiting CPU limit consumption of offloading
## Summary - `FT.CREATE` accepted an index definition missing the required `SCHEMA` keyword, returning `OK` and listing the index in `FT._LIST`, but the index was unusable (`FT.INFO` reported it as not found). - Root cause: without `SCHEMA`, tokens that look like field definitions fall through the "unsupported parameters are ignored" branch in `CreateDocInde
## Summary `OutgoingMigration::FinalizeMigration` currently pauses **every** client command on the node — via `dfly::Pause(..., ClientPause::ALL, ...)` — for the entire duration of each finalize attempt, even though the only reason for the pause is to stop *new transactions on the slots being migrated* from initializing with stale cluster-slot-ownership info
This PR fixes a known issue that was already fixed for V1 at 5ef35182. Defer V2 pipeline execution when queued control work predates the next parsed command. Let IoLoopV2 drain that work so UNSUBSCRIBE cannot discard earlier Pub/Sub messages. Tests: Extend the unsubscribe regression across V1 and V2 with standalone and deferred-GET commands. Bound the starva
1. Prefetching bucket data It requires really large pipelines to reach large differences <img width="1650" height="660" alt="image" src="https://github.com/user-attachments/assets/5b675ef9-35f4-4a82-b157-941f9a6a22ef" /> 2. Caching last command id Helps avoiding absl::AsciiStrToUpper + hashtable lookup + ifs for simple cases like all GET pipeline. Adds addit
Two coupled TLS defects: - The `tls` CONFIG callback reconfigured listeners one by one; a bad cert/key failed midway and left listeners split between old and new TLS state (a failed callback only rolls back the flag value). - `tls_bytes` summed per-thread counters, but OpenSSL objects are allocated on one thread and freed on another (startup contexts are bui
## What Adds a scheduled GitHub Actions workflow, `.github/workflows/ioredis-tests.yml`, that runs the existing [ioredis](https://github.com/luin/ioredis) end-to-end test suite against a freshly built Dragonfly binary every day, and points to it from `tests/README.md`. ## Why (closes part of #383) `#383` asked for e2e coverage of the `ioredis` client. Lookin
XReadBlock gated the RESP3 map shape on opts->read_group, so a plain blocking XREAD always fell back to the RESP2 array shape even on a RESP3 connection. The non-blocking path already applies the map shape to both XREAD and XREADGROUP, so drop the read_group condition. Fixes #8056 <!-- **Commits Must Be Signed and Your PR title must conform to the convention
Deleting a key because its newly set expiration is already in the past emitted no `__keyevent@<db>__:expired` notification, so a subscriber waiting for the event was blocked forever (found via the upstream Valkey pubsub TCL suite: "expired event (Expiration time is already expired)" hangs). This covers the `DbSlice::UpdateExpire` funnel: EXPIRE/PEXPIRE/(P)EX
It has been seen in certain cases that the stream size accounting breaks due to a negative delta. In that case, the metrics depending on this type go completely wrong and start showing object size in terrabytes instead of a few GiB. There are a couple of fixes and safeguards: * original check that size>=delta if delta<0 was not doing anythingi it was always