101 signals · 152 observation events
Temporal service
v1.31.2Prereleases and milestones indicate public plans; they are not delivery commitments.
**Is your feature request related to a problem? Please describe.** My team is working on a new deployment of Temporal and we recently spent an entire day debugging an issue related to a missing key in configuration due to lack of error messaging and validation by the bootstrap code. Example: ``` metrics: hostPort: "127.0.0.1:8125" prefix: "temporal" ``` need
## Expected Behavior - When a scheduled action succeeds, subsequent scheduled actions should not receive a `ContinuedFailure` failure payload. ## Actual Behavior - If a scheduled action fails, setting `ContinuedFailure`'s payload, and then a subsequent scheduled action succeeds without returning a payload (a `null` payload), `ContinuedFailure` will continue
Today is _June 19th, 2024_. If you create a schedule and specify "every 365 days" the result will be that the first run is on `2024-12-18 00:00:00`. **That's in 6 months**. I've tried passing `StartAt: 2025-06-19` but I get `2025-12-18 00:00:00` as a result. I understand this may be known and as-designed behavior but it struck me as unexpected to ask for a 3
**Is your feature request related to a problem? Please describe.** We'd like to have a metric for whenever schedules skip a scheduled run for any reason (either by overlap policy, missing the catchup window, or from a buffer overrun). **Describe the solution you'd like** - A new metric is introduced to keep track of skipped scheduled runs. **Describe alterna
## Expected Behavior `SignalWithStartWorkflowExecution` for a workflow ID makes progress: it either signals the running execution or starts a new run, and returns within the client deadline. I suppose, but haven't checked, that this behavior could occur in other APIs which interacts with current executions in a similar fashion (Continue As New?, Updates?, ..
> 🏕️ This pull request was created by [camper](https://github.com/temporalio/camper), an automated security campaign tool. ## Finding <table> <tr><td><strong>Rule</strong></td><td><code>claude-code-action-unhardened</code></td></tr> <tr><td><strong>Severity</strong></td><td>MEDIUM</td></tr> <tr><td><strong>Repository</strong></td><td><code>temporalio/tempora
## Temporal version - Temporal Server: 1.31.0 - Persistence: PostgreSQL - Visibility: Elasticsearch ## Summary A Schedule's internal scheduler Workflow can become permanently blocked in `refreshWorkflows` when: 1. A Schedule-started Workflow has a Workflow-level Retry Policy. 2. The scheduled execution exhausts retries, producing a multi-run chain ending in
**Description** The Cassandra schema file [/cassandra/temporal/schema.cql](https://github.com/temporalio/temporal/blob/main/schema/cassandra/temporal/schema.cql) currently configures nearly all tables with LeveledCompactionStrategy (LCS). LCS is not a good general-purpose default — it's optimized for read-heavy workloads with low write amplification toleranc
# AWS Inspector Vulnerability Scan Report (CRITICAL & HIGH Only) Scan performed on the latest Temporal images using Amazon ECR Enhanced Scanning with AWS Inspector. ## Test Metadata - **Test Date:** August 12, 2026 - **Tested Versions:** | Image | Version | Image Digest | |---|---|---| | `temporalio/ui:latest` | **v2.53.2** | `sha256:f3a23c7133762f9bbdd7b340
## Expected Behavior The partial counter configuration documented in `tools/fairsim/README.md` should override only the supplied values. For example: ```bash ./fairsim \ -counter-params=<(echo '{"CMS":{"W":100}}') \ -- -tasks=1000 -keys=500 ``` should set the CMS width to `100` while retaining all other values from `counter.DefaultCounterParams`. ## Actual B
### What are you really trying to do? Keep the number of Worker Deployment Versions per deployment under `matching.maxVersionsInDeployment` by periodically deleting drained versions, so that new versions can always register. ### Describe the bug A Worker Deployment can end up listing a version in `DescribeWorkerDeployment`'s `versionSummaries` **after that v
### Summary When a History cluster is running at its system persistence-QPS limit, the fleet stays stable only because `ResourceExhausted` errors put queue readers/task executables onto their long reschedule backoff, which spreads persistence load out over time. If persistence briefly returns a *different* transient error (`Unavailable`, or context deadline
I’m trying to self-host Temporal v1.29 using Helm. For both default and visibility databases I use MySQL 8.0.45. I get the following error in the manage-schema-visibility-store job: ERROR Unable to update SQL schema. {“error”: “error executing statement: Error 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL
⚠️ This is part of a stacked PR set, to be merged into `chrsmith/wc-add-worker-cb-variant`. This will not go directly into `main`. --- ## What changed? Add a new field to the CHASM Callback component to persist the terminal failure of a CHASM Callback. The [CallbackState](https://github.com/temporalio/temporal/blob/main/chasm/lib/callback/proto/v1/message.pr
Makes test execution return an exit code and error instead of terminating inside the orchestration loop. Final artifact failures, retry-validation failures, execution errors, timeouts, and normal pass/fail outcomes are tested directly while Main remains the process-exit seam. Stack: depends on #11518 and completes the canonical Go test JSON reporting pipelin
## What changed? - `WithRequestID` now applies to `UpdateComponent`, enabling execution-level idempotency guarding via request ID. - When a request ID is passed as part of an `UpdateComponent` call (via API handler), it is persisted upon successful updateFn call. If it is already present, instead, `UpdateComponent` fails with a `FailedPrecondition`. - When a
Moves retry selection, argument rewriting, and missing-rerun validation into a focused policy over canonical attempt results. The current runner still owns orchestration, so this PR changes the decision module without switching the reporting pipeline. Stack: depends on #11515. Canonical JUnit rendering follows in #11517.
Adds the pure canonical attempt-history renderer, including parent-failure suppression, diagnostics, timeouts, aborts, build errors, retry suffixes, crash reports, exact counters, and numeric durations. Integration coverage verifies the persisted JUnit shape and the downstream sharding consumer. Stack: depends on #11516. The production cutover follows in #11
Switches orchestration to canonical attempt results and JUnit rendering, then removes the legacy JUnit merger and temporary diagnostic adapter in the same PR. Intermediate and final artifacts come from one attempt history, and report writes reject inconsistent counters. Stack: depends on #11517. Explicit orchestration outcomes follow in #11033.
## What changed? - Build one index from parsed test attempts keyed by normalized test name and GitHub Actions run commit. - Reuse that index when selecting and bisecting tests instead of rescanning every parsed attempt for each candidate. - Use `githubActions...` names in the bisect path to distinguish GitHub Actions runs from Temporal workflows. ## Why? A r
## What changed? Tie process-global gRPC resolver registrations to their Fx and test owners. Resolver URLs use monotonic opaque registration IDs, test callers register cleanup directly, and matching test clients close their RPC factories during cluster shutdown. The leak test drops the corresponding host expectation. ## Why? The process-global resolver map r
## Summary Fixes #10991 — `CountWorkflowExecutions` with `GROUP BY ExecutionStatus` was 8-16x slower than necessary on MySQL because `mysqlQueryConverter.buildCountStmt` unconditionally `LEFT JOIN`s `custom_search_attributes` and `chasm_search_attributes`, even when the query never references a column from either table. ### Root cause `GROUP BY` is only ever
> 🏕️ This pull request was created by [camper](https://github.com/temporalio/camper), an automated security campaign tool. ## Finding <table> <tr><td><strong>Rule</strong></td><td><code>checkout-below-v7</code></td></tr> <tr><td><strong>Severity</strong></td><td>HIGH</td></tr> <tr><td><strong>Repository</strong></td><td><code>temporalio/temporal</code></td><
⚠️ This is part of a stacked PR set, to be merged into `chrsmith/wc-persist-callback-terminal-failures`. This will not go directly into `main`, until the overall feature is code complete. --- ## What changed? The API surface area for the worker callbacks feature adds a new `completion_callbacks` field to SANOs, allowing users to register callbacks to be exec
## What changed? Fixed bookkeeping of mutable state approximate size on the two paths that mutate an `ActivityInfo` without adding its bytes to the counter: `AddActivityTaskStartedEvent` (start), and the heartbeat handler, where `RetryLastWorkerIdentity` now moves into `UpdateActivityProgress`. ## Why? Both sites mutate the pointer `GetActivityInfo` returned
## What changed? - Shows Bayesian commit suspects directly below the overall stats. - Separates synthetic test-runner timeouts from test failures and reports affected artifacts. - Shows final-retry test failures as affected-artifact counts rather than workflow-run rates. ## Why? The synthetic `testrunner.TotalTimeout` event was reported both as a flaky test
## What changed? Clear removed tests from the shared cluster's backing slice. ## Why? `sharedClusterT` shortened `activeTests` without clearing the removed interface slot. The backing array could therefore retain completed tests and their functional test state.
## What changed? Use the `jobID` parameter in the `deterministicRequestID` function to allow multiple batch operations to signal the same workflow. ## Why? Two batch operations that send the same signal to the same workflow/run ID will only send one signal, the second will be de-duped in the signal logic because the request ID was hashed from workflow id / r
## What changed? Only use 4-core ARM runners instead of 8-core; and increase shard size for test sharding. ## Why? 8-core runners have - at least during peak hours - very long provisioning time (several minutes). By using 4-core we reduce that time, but risk OOM kills as they have less memory. To counter that, we increase the number of test shards so that fe
## What changed? Opt-in validator that exhaustively validate all protobuf fields in 1 place. ## Why? Adds semantic validation for protobufs as a thin layer before the proto request reaches actual business logic. ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [x] added new unit test(s) - [ ] added n
## What changed? _Describe what has changed in this PR._ ## Why? _Tell your future self why have you made these changes._ ## How did you test it? - [ ] built - [ ] run locally and tested manually - [ ] covered by existing tests - [ ] added new unit test(s) - [ ] added new functional test(s) ## Potential risks _Any change is risky. Identify all risks you are
## What changed? Adds `testcontext.EnsureRemaining` and has `await` use it so long await calls can request additional test-scoped context time while still respecting the test context cap. ## Why? Await calls can need more time than the default test context has left (esp after the environment setup). Extending the test timeout in this way allows for (1) stuck
## What changed? Only use 4-core ARM runners instead of 8-core; and increase shard size for test sharding. ## Why? 8-core runners have - at least during peak hours - very long provisioning time (several minutes). By using 4-core we reduce that time, but risk OOM kills as they have less memory. To counter that, we increase the number of test shards so that fe
## What changed? Stacks on #10417. Adds clearer timeout diagnostics for await failures and test context cleanup after the core context-extension behavior is in place. The report now includes aligned detail rows, attempt duration summaries, attempt timeout counts, deadline-limit causes, and context extension summaries when available. The await reporting tests
## Summary - roll four mixed-version server instances while OMES workloads run - retain each instance's port allocation across replacements and verify membership convergence - route through a mutable frontend proxy and add proxy lifecycle coverage - temporarily resolve OMES from the exact commit in temporalio/omes#426; replace this with the upstream pseudo-v
## What changed? Add data race summary to CI report ## Why? Notify when CI detects data race issues in main. ## How did you test it? - [X] built - [X] run locally and tested manually - [X] covered by existing tests - [X] added new unit test(s) - [ ] added new functional test(s)
> **Part 2 of a 5-PR series** building to replication stream namespace isolation (a restructuring of #10147): read buffer → reader group → lane protocol → isolation manager → sender isolation. > #11263 (read buffer) has merged, so this PR's diff is now standalone against `main`. · **Next in series: #11303** (lane wire protocol + receiver routing). ## What ch
> **Part 3 of a 5-PR series** building to replication stream namespace isolation (a restructuring of #10147): read buffer → reader group → lane protocol → isolation manager → sender isolation. > #11302 (reader group) has merged, so this PR's diff is now standalone against `main`. · **Next in series: #11304** (isolation manager). ## What changed? The wire-lev
> **Part 4 of a 5-PR series** building to replication stream namespace isolation (a restructuring of #10147): read buffer → reader group → lane protocol → isolation manager → sender isolation. > **Stacked on #11303** (lane wire protocol). This PR targets `main`, so until the earlier PRs in the series merge its diff also includes their commits — review after
## What changed? The `objectleak` package has no support for establishing a baseline; therefore it erroneously reports false positives. ## Why? Ensure `objectleak` report is actionable and insightful.
## What changed? `zapLogger.Skip()` now carries the logger's accumulated `tags` into the returned clone, matching the `baseZl` field it already forwarded. Also updated `TestThrottleLogger` to apply a `service` tag *before* the throttled logger wraps it, mirroring what `ThrottledLoggerProvider` does. ## Why? Tags applied before a `Skip()` were silently droppe
⚠️ This is part of a stacked PR set, to be merged into `feature/worker-callbacks`. This will not go directly into `main`, until the overall feature is code complete. --- ## What changed? The worker callbacks feature introduces a new variant of `commonpb.Callback` to describe callbacks for invoking a Nexus operation on a waiting worker. This PR adds the suppo
## What changed? `addJitter` in `common/backoff/retrypolicy.go` never applied jitter. With a 2s base and `WithJitter(0.1)`, every delay came back as exactly 2.000s instead of spread across `[2.0s, 2.2s)`. ```go // before return duration * time.Duration(1+jitterPct*rand.Float64()) // after return time.Duration(float64(duration) * (1 + jitterPct*rand.Float64()
## What changed? Adds `NamespaceLifecycle` wide events on the two paths that decide when a namespace handover can complete. Also threads `ShardID` and an `EventLogger` into `HandoverTrackerParams` so the tracker can attribute an event to its shard. **Shard handover tracker** (`service/history/shard/`) — each shard holds its own replication watermark for a na
## What changed? - Add a rate-limited GitHub API client for efficiently retrieving workflow, artifact, and commit data. - Aggregate test runs once and reuse the index for report generation and bisect analysis. - Add parallel processing and coverage for API handling, aggregation, parsing, and report/bisect behavior. ## Why? The flaky-test report needs to proc
## What changed? Skip unbuildable replication tasks on stream sender instead of blocking the stream. The skip is logged and new metric ReplicationTaskSendSkipped added. ## Why? When the replication stream sender cannot build ("convert") a task, it retries and, once the retry budget is exhausted, returns an error that tears the stream down. On reconnect the s
## What changed? - Give the current and previous-release mixed-brain servers separate Prometheus endpoints. - Scrape both endpoints after the OMES workload and fail on inconsistent label sets within either server or incompatible label sets across versions. - Require `task_requests` coverage and narrowly allowlist label additions that already exist between th
## What changed? - Reorganize `chasm/lib/activity` ## Why? - Improve navigability and codebase comprehensibility ## How did you test it? - [x] covered by existing tests
## What changed? Flip the default of `history.enableHostLevelEventsCache` from `false` to `true`, so history shards share a single host-level events cache instead of each allocating a shard-level one. ## Why? We have been using host level history cache for a while. We can now enable it in code by default. ## How did you test it? - [x] built - [x] covered by
## What changed? This PR updates the CHASM `Callback` component to support the new `Worker`-variant. Unlike `Nexus` callbacks, which just issue an HTTP `POST` request to deliver a Nexus completion result (and potentially in another namespace), these new `Worker` callbacks are for invoking a Nexus operation within the same namespace. This PR does _not_ enable