> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/temporalio/temporal/llms.txt
> Use this file to discover all available pages before exploring further.
# History Service
> Deep dive into the History Service architecture and internal implementation
# History Service
The History Service is the core of Temporal's durable execution engine. It manages individual Workflow Executions, maintains their state, and drives them to completion by coordinating with the Matching Service and Worker processes.
## Overview
The History Service handles two main types of gRPC requests:
1. **Requests from User Applications**: Start, Cancel, Query, Update, Signal, Reset, etc.
2. **Requests from Workers**: Completion of Workflow Tasks and Activity Tasks
For system architecture context, see the [Architecture Overview](/architecture/overview)
## Core Responsibilities
When handling requests, the History Service:
1. **Determines new History Events** that the request implies and appends them to Workflow History
2. **Creates Transfer Tasks** if Workflow or Activity Tasks should be added to Matching Service
3. **Creates Timer Tasks** when the workflow is blocked on a timer or timeout
4. **Updates Mutable State** to reflect the current state of the workflow execution
## History Shards
### Sharding Architecture
A single Temporal Cluster manages millions of concurrent Workflow Executions by partitioning them into **History Shards**:
* The total number of shards is **fixed at cluster creation** and cannot be changed
* Each workflow execution is assigned to exactly one shard (based on workflow ID hash)
* Multiple History Service instances run in the cluster, each owning a subset of shards
* Shard ownership is coordinated via the Ringpop membership protocol
### Shard Ownership
"Owning" a History Shard means:
* **Synchronously handling** all incoming RPC requests for workflows in that shard
* **Asynchronously processing** background tasks (timers, transfers, visibility, replication)
* Maintaining in-memory metadata including:
* `RangeID`: Monotonically increasing generation number for fencing
* Queue states: Acknowledged/processed positions of each internal queue
### Persistence Mapping
With Cassandra, History Shards map 1:1 to partitions of the `executions` table. Similar mappings exist for MySQL, PostgreSQL, and SQLite.
```go theme={null}
// Code entrypoint: service/history/history_engine.go
// When History service starts, Start() is called for each shard owned by that host
```
## Workflow Execution Mutable State
### What is Mutable State?
For every workflow execution, Temporal maintains a collection of data structures called **Mutable State** that summarizes:
* Identities of in-progress activities
* Active timers and their expiration times
* Child workflow executions
* Pending signals and updates
* Current workflow task information
* Execution status and result
### Why Cache Mutable State?
While this data could be recomputed from Workflow History Events, doing so would be slow. Therefore:
* Mutable State is **persisted** alongside history events
* Recently accessed executions are **cached in memory**
* Cache improves latency for active workflows
### Persistence Details
Due to Cassandra's limited RDBMS support (the most important persistence backend), Mutable State is persisted in a **single row** with a layout similar to its in-memory representation.
```go theme={null}
// Code entrypoint: service/history/workflow/mutable_state_impl.go
// MutableStateImpl implements the MutableState interface
// Loaded from persistence: service/history/workflow/context.go:248
```
## Workflow Execution History
### Event Structure
Workflow Execution History is a **linear sequence of History Events** (or a branching topology for reset workflows):
* Each event has an **EventID** (monotonically increasing)
* Some events carry **payloads** (e.g., activity results, signal data)
* Events are **immutable** once written
* Events are **publicly exposed** via API
### Event Sourcing Property
The sequence of History Events alone is sufficient to recover:
* All Mutable State information
* All tasks relating to the workflow execution
This is the foundation of Temporal's event sourcing implementation.
### Event Types
Events represent state transitions in workflow execution:
* `WorkflowExecutionStarted`
* `WorkflowTaskScheduled`, `WorkflowTaskStarted`, `WorkflowTaskCompleted`
* `ActivityTaskScheduled`, `ActivityTaskStarted`, `ActivityTaskCompleted`
* `TimerStarted`, `TimerFired`
* `WorkflowExecutionCompleted`, `WorkflowExecutionFailed`
* And many more...
See [`temporal/api/enums/v1/event_type.proto`](https://github.com/temporalio/api/blob/master/temporal/api/enums/v1/event_type.proto) for the complete list.
In Temporal, "event" refers to Workflow History Event, not event-driven architecture events.
## RPC Handling
### User Application Requests
Handlers for user application requests are defined in `service/history/api/`:
* `StartWorkflow`, `SignalWorkflow`, `UpdateWorkflow`
* `CancelWorkflow`, `TerminateWorkflow`
* `QueryWorkflow`, `DescribeWorkflow`
* `ResetWorkflow`
Many of these use a shared code path for state transitions.
```go theme={null}
// Code entrypoint: service/history/api/signalworkflow/api.go
// Example: SignalWorkflow request handling
```
### Worker Completion Requests
Handlers for worker completion requests:
**`RespondWorkflowTaskCompleted`**
* Contains a sequence of Commands: `StartTimer`, `ScheduleActivityTask`, `StartChildWorkflowExecution`, etc.
* See [`temporal/api/enums/v1/command_type.proto`](https://github.com/temporalio/api/blob/master/temporal/api/enums/v1/command_type.proto)
**`RespondActivityTaskCompleted`**
* Contains the result of the activity
**Failure Responses**
* `RespondWorkflowTaskFailed`, `RespondActivityTaskFailed`, etc.
```go theme={null}
// Code entrypoint: service/history/workflow_task_handler_callbacks.go:371
// WorkflowTaskCompleted handler accumulates updates from commands
```
## Queue Processing
The "task queues" discussed here are internal History Service implementation details, distinct from the user-facing Task Queues in Matching Service.
### Queue Types
Each History Shard manages several internal task queues:
1. **Transfer Queue**: Immediate execution tasks
2. **Timer Queue**: Scheduled execution tasks
3. **Visibility Queue**: Update visibility store
4. **Replication Queue**: Cross-cluster replication
5. **Archival Queue**: Archive history to long-term storage
### Queue Processor Architecture
When the History Service starts, for each owned shard:
1. A `QueueProcessor` starts for each queue type
2. Multiple `Reader` goroutines read tasks from persistence
3. Tasks are submitted to a shared task execution framework
4. Goroutine pool executes tasks across all shards
5. Queue "ack levels" are periodically checkpointed
```mermaid theme={null}
flowchart LR
subgraph Shard["History Shard"]
direction TB
RPC[RPC Handler]
Transfer[Transfer Queue]
Timer[Timer Queue]
Visibility[Visibility Queue]
end
subgraph Processor["Queue Processor"]
Readers[Multiple Readers]
Scheduler[Task Scheduler]
end
subgraph Execution["Task Execution"]
Pool[Goroutine Pool]
Executor[Task Executor]
end
RPC -->|Write Tasks| Transfer
RPC -->|Write Tasks| Timer
RPC -->|Write Tasks| Visibility
Transfer --> Readers
Timer --> Readers
Visibility --> Readers
Readers --> Scheduler
Scheduler --> Pool
Pool --> Executor
```
```go theme={null}
// Code entrypoint: service/history/history_engine.go:288
// Start() initiates queue processors for each shard
// Readers: service/history/queues/reader.go:51
// Task framework: common/tasks/
```
### Transfer Queue
Transfer Tasks are available for **immediate execution**:
* Enqueue Workflow Task in Matching Service
* Enqueue Activity Task in Matching Service
* Start Child Workflow Execution
* Cancel External Workflow Execution
* Signal External Workflow Execution
* Delete Workflow Execution
When executed, a Transfer Task typically makes an RPC to the Matching Service.
```go theme={null}
// Code entrypoint: service/history/transfer_queue_active_task_executor.go:114
// Execute() method dispatches by task type
// processActivityTask: transfer_queue_active_task_executor.go:169
// processWorkflowTask: transfer_queue_active_task_executor.go:217
```
### Timer Queue
Timer Tasks become available at their **trigger time**:
* **User Timer Timeout**: `workflow.sleep()` timer expires
* **Activity Timeout**: Schedule-to-start, start-to-close, heartbeat timeouts
* **Workflow Task Timeout**: Workflow task not completed in time
* **Workflow Timeout**: Workflow execution timeout
* **Delete History Event**: Retention period expired
When a user timer fires, a new Workflow Task is scheduled to advance the workflow.
```go theme={null}
// Code entrypoint: service/history/timer_queue_active_task_executor.go:90
// Execute() handles tasks differently by timer type
// executeUserTimerTimeoutTask: timer_queue_active_task_executor.go:136
// Activity retries: timer_queue_active_task_executor.go:426
```
### Visibility Queue
Visibility Tasks update workflow metadata in the visibility storage:
* `StartWorkflowExecution`: Record new workflow
* `CloseWorkflowExecution`: Record workflow completion
* `UpsertWorkflowSearchAttributes`: Update search attributes
Visibility storage may be a different database than the primary persistence store.
## State Transitions
### State Transition Pattern
A workflow execution state transition follows this pattern:
```mermaid theme={null}
flowchart LR
StateA["State A
(Mutable State)"] --> Input{Input}
Input -->|"RPC from App
RPC from Worker
Timer Fired
External Event"| Transaction
subgraph Transaction["Atomic Transaction"]
direction TB
StateB["State B
(Updated Mutable State)"]
Tasks["New History Tasks
(Transfer/Timer)"]
end
Transaction --> Events["New History Events
(Appended)"]
```
All state transitions use the same implementation pathway:
1. **In-memory**: Create new History Event(s), update Mutable State, create History Tasks
2. **Persistence**: Append History Events to storage
3. **Atomically**: Update Mutable State and add History Tasks in a database transaction
```go theme={null}
// Code entrypoint: service/history/api/update_workflow_util.go:37
// GetAndUpdateWorkflowWithNew is the generic utility for state transitions
// Calls: service/history/workflow/context.go:459 (UpdateWorkflowExecutionAsActive)
// SQL persistence: common/persistence/sql/execution.go:339
// Cassandra persistence: common/persistence/cassandra/execution_store.go:142
```
### State Transition Call Sites
**RPC from User Application:**
```go theme={null}
// Example: SignalWorkflow
// service/history/api/signalworkflow/api.go:54
```
**RPC from Worker:**
```go theme={null}
// Example: WorkflowTaskCompleted
// service/history/workflow_task_handler_callbacks.go:690
```
**Timer Task Processor:**
```go theme={null}
// Example: User timer timeout
// service/history/timer_queue_active_task_executor.go:136
```
### Consistency Guarantees
Temporal maintains strong consistency through:
Mutable State and History Tasks are persisted atomically using database transactions, ensuring they are always consistent with each other.
Mutable State stores the identity of the latest History Event. An event is only "valid" if it's reflected in Mutable State. On failure to persist, we reload from persistence.
Transfer Tasks written by a History Shard will eventually result in the required Workflow or Activity Task being created in Matching Service. This is the [Transactional Outbox](https://microservices.io/patterns/data/transactional-outbox.html) pattern.
## Example: Simple Workflow Execution
Here's how a simple workflow that calls an activity is processed:
```javascript theme={null}
function myWorkflow() {
result = callActivity(myActivity);
workflowSleep(THIRTY_DAYS);
return result;
}
```
```mermaid theme={null}
sequenceDiagram
participant App as User Application
participant Worker
participant Handler as History Shard
RPC Handler
participant QP as History Shard
Queue Processors
participant Matching as Matching Service
App->>Handler: StartWorkflowExecution
note right of Handler: State transition:
initialize Workflow History
update Mutable State
create Transfer Task
QP->>Matching: enqueue Workflow Task
note over Worker: dequeue Workflow Task
Worker->>Handler: RespondWorkflowTaskCompleted
(ScheduleActivity)
note right of Handler: State transition:
append to History
update Mutable State
create Transfer Task
QP->>Matching: enqueue Activity Task
note over Worker: dequeue Activity Task
Worker->>Handler: RespondActivityTaskCompleted
note right of Handler: State transition:
append to History
update Mutable State
create Transfer Task
QP->>Matching: enqueue Workflow Task
note over Worker: dequeue Workflow Task
Worker->>Handler: RespondWorkflowTaskCompleted
(StartTimer)
note right of Handler: State transition:
append to History
update Mutable State
create Timer Task
QP->>QP: thirty days elapses
timer fired
note right of Handler: State transition:
append to History
update Mutable State
create Transfer Task
QP->>Matching: enqueue Workflow Task
```
For detailed sequence diagrams, see [Workflow Lifecycle](/architecture/workflow-lifecycle).
## Additional Responsibilities
The History Service has many other responsibilities not covered in detail here:
* **Shard Membership and Ownership**: Managing which service instance owns which shards
* **Replication**: Cross-cluster replication and conflict resolution
* **Visibility**: Updating visibility storage for workflow search
* **Archival**: Archiving histories to long-term storage
* **Workflow Reset**: Resetting workflow execution to a previous point
## Further Reading
Detailed sequence diagrams of workflow execution
How tasks are distributed to workers
How Temporal implements event sourcing
High-level system architecture