Running Tests

Learn how to run and write tests for Temporal Server

Temporal Server has a comprehensive test suite covering unit, integration, and functional tests.

Test Categories

Temporal defines three categories of tests:

Unit Tests

Tests with no external dependencies beyond the test target and Go mocks. Run fast and should have maximum coverage.

Location: Throughout the codebase in *_test.go files

Integration Tests

Tests covering integration between the server and its dependencies (Cassandra, PostgreSQL, MySQL, Elasticsearch, etc.).

Locations:

• ./common/persistence/tests - Database integration tests
• ./tools/tests - Database tool integration tests
• ./temporaltest - Embedded server tests

Functional Tests

End-to-end tests covering the complete functionality of Temporal Server.

Locations:

• ./tests - Main functional test suite
• ./tests/xdc - Cross-DC replication tests
• ./tests/ndc - Named datacenter tests

Running Tests

Run Unit Tests

make unit-test

Unit tests do not require runtime dependencies.

Run Integration Tests

make integration-test

Integration tests require runtime dependencies. Start them first:

make start-dependencies

Run Functional Tests

make functional-test

Optionally specify persistence type and driver:

PERSISTENCE_TYPE=nosql PERSISTENCE_DRIVER=cassandra make functional-test
PERSISTENCE_TYPE=sql PERSISTENCE_DRIVER=mysql8 make functional-test
PERSISTENCE_TYPE=sql PERSISTENCE_DRIVER=postgres12 make functional-test

Run All Tests

make test

This runs unit, integration, and functional tests sequentially.

Test Options

Build Tags

Tests require specific build tags:

• test_dep - Enables test hooks (required for most tests)
• integration - Includes integration test dependencies
• disable_grpc_modules - Reduces binary size during tests

The Makefile automatically applies: -tags disable_grpc_modules,test_dep

Test Timeout

Default timeout is 35 minutes (configurable via TEST_TIMEOUT):

TEST_TIMEOUT=60m make test

Race Detection

Race detection is enabled by default. To disable:

TEST_RACE_FLAG=off make unit-test

Test Shuffling

Test order is randomized by default. To disable:

TEST_SHUFFLE_FLAG=off make unit-test

Running Individual Tests

Run a specific test or test suite:

go test -v -tags test_dep <package_path> -run <TestName>

Examples:

# Run a specific test
go test -v -tags test_dep ./common/persistence -run TestPersistenceStartWorkflow

# Run a test suite
go test -v -tags test_dep github.com/temporalio/temporal/common/persistence -run TestCassandraPersistenceSuite

# Run with testify method filter
go test -v -tags test_dep ./common/persistence -run TestCassandraPersistenceSuite -testify.m TestPersistenceStartWorkflow

Code Coverage

Generate coverage reports:

Unit Test Coverage

make unit-test-coverage

Integration Test Coverage

make integration-test-coverage

Functional Test Coverage

make functional-test-coverage

Coverage reports are written to .testoutput/ and available on Codecov.

Install the Codecov Browser Extension to see code coverage directly in GitHub PRs.

Test Configuration

Build Tags

• test_dep - Enables test hooks implementation for production code paths that are difficult to test
• TEMPORAL_DEBUG - Extends functional test timeouts for debugging sessions
• disable_grpc_modules - Disables gRPC modules for faster compilation

Environment Variables

Logging:

• TEMPORAL_TEST_LOG_FORMAT - Output format (json or console)
• TEMPORAL_TEST_LOG_LEVEL - Log verbosity (debug, info, warn, error, fatal)

OpenTelemetry:

• TEMPORAL_TEST_OTEL_OUTPUT - File path for OTEL trace output on test failures

Cluster Configuration:

• TEMPORAL_TEST_SHARED_CLUSTERS - Number of shared test clusters (default varies)
• TEMPORAL_TEST_DEDICATED_CLUSTERS - Number of dedicated test clusters (default varies)

Data Encoding:

• TEMPORAL_TEST_DATA_ENCODING - Persistence encoding (proto3 or json, default: proto3)

CGO:

• CGO_ENABLED - Set to 0 to disable CGO for faster builds (default: 0)

Writing Tests

See the development guidelines for best practices on writing tests.

Key test utilities:

Test Cluster

Use testcore.NewEnv(t) for functional tests:

func TestMyFeature(t *testing.T) {
    s := testcore.NewEnv(t)
    // Each test gets its own isolated namespace
    namespace := s.Namespace()
    // ...
}

Test Variables

Use testvars package for generating test identifiers:

func TestWorkflow(t *testing.T) {
    tv := testvars.New(t)
    req := &workflowservice.StartWorkflowExecutionRequest{
        RequestId:    tv.Any().String(),
        Namespace:    tv.NamespaceName().String(),
        WorkflowId:   tv.WorkflowID(),
        WorkflowType: tv.WorkflowType(),
        TaskQueue:    tv.TaskQueue(),
    }
}

Task Poller

For end-to-end workflow testing:

s := testcore.NewEnv(t)
poller := s.TaskPoller
// Use poller to handle workflow and activity tasks

Debugging Tests

IDE Debugging (GoLand)

Add build tags to “Go tool arguments” in Run/Debug configuration:

-tags disable_grpc_modules,test_dep

For integration tests, add the integration tag:

-tags disable_grpc_modules,test_dep,integration

See GoLand Debugging documentation for details.

OpenTelemetry Tracing

Debug tests using OTEL traces:

export OTEL_BSP_SCHEDULE_DELAY=100
export OTEL_EXPORTER_OTLP_TRACES_INSECURE=true
export OTEL_TRACES_EXPORTER=otlp
export TEMPORAL_OTEL_DEBUG=true

# Start OTEL collector (e.g., Grafana Tempo)
make start-dependencies

# Run your test
go test -v -tags test_dep ./tests -run TestMyFeature

Traces are available in Grafana at localhost:3000.

See tracing.md for more details.

Fault Injection Tests

Run functional tests with fault injection enabled:

make functional-with-fault-injection-test

Or manually:

go test ./tests -tags test_dep -enableFaultInjection=true

Test Helpers

softassert Package

softassert.That logs errors without stopping test execution:

softassert.That(t, condition, "error message")
// Test continues even if condition is false
// Test ultimately fails if any soft assertions failed

testhooks Package

Injects test-specific behavior into production code (last resort - prefer mocking):

// Only available with -tags=test_dep
testhooks.RegisterHook("UpdateWithStartInBetweenLockAndStart", func() {
    // Inject behavior at specific execution point
})

Test Parallelization

All tests should use t.Parallel() unless there’s a specific reason not to:

func TestMyFeature(t *testing.T) {
    t.Parallel()
    // test code
}

Automate parallelization:

make parallelize-tests

Opt out with //parallelize:ignore comment.

Mixed Brain Tests

Tests that require CGO:

make mixed-brain-test

Troubleshooting

Tests Timeout

Increase timeout:

TEST_TIMEOUT=60m make test

Tests Fail with “image not found”

Start dependencies:

make start-dependencies

Tests Pass Locally but Fail in CI

Ensure you’re using the correct build tags:

go test -tags disable_grpc_modules,test_dep ./...

Clean Test Cache

make clean-test-output