Create a benchmark golden test with default configuration.

This is the simplest way to add a benchmark test:

describe Sorting $ do
  benchGolden "quicksort 1000 elements" $
    return $ quicksort [1000, 999..1]

Default configuration:

• 100 iterations
• 5 warm-up iterations
• 15% tolerance
• Variance warnings enabled
• Standard statistics (not robust mode)

Create a benchmark golden test with custom configuration.

Examples:

-- Tighter tolerance for critical code
benchGoldenWith defaultBenchConfig
  { iterations = 500
  , tolerancePercent = 5.0
  , warmupIterations = 20
  }
  "hot loop" $
  return $ criticalFunction input

-- Robust statistics mode for noisy environments
benchGoldenWith defaultBenchConfig
  { useRobustStatistics = True
  , trimPercent = 10.0
  , outlierThreshold = 3.0
  }
  "benchmark with outliers" $
  return $ computation input

Create a benchmark golden test for an IO action.

This is an alias for benchGolden that makes it clear the action involves IO (e.g., file operations, network calls).

benchGoldenIO "file read" $ do
  contents <- readFile "large-file.txt"
  evaluate (length contents)

Note: For IO actions in noisy environments (CI, shared systems), consider using benchGoldenIOWith with useRobustStatistics = True.

Create an IO benchmark golden test with custom configuration.

Configurable parameters for benchmark execution and comparison.

Default benchmark configuration with sensible defaults.

• 100 iterations
• 5 warm-up iterations
• 15% tolerance on mean time
• 0.01 ms (10 microseconds) absolute tolerance - prevents false failures for fast operations
• Variance warnings enabled at 50% tolerance
• Output to .golden/ directory
• Success on first run (creates baseline)

Hybrid Tolerance Strategy

The default configuration uses BOTH percentage and absolute tolerance:

• Benchmarks pass if mean time is within ±15% OR within ±0.01ms
• This prevents measurement noise from failing fast operations (< 1ms)
• For slower operations (> 1ms), percentage tolerance dominates

Set absoluteToleranceMs = Nothing for percentage-only comparison.

Configuration for a single benchmark golden test.

Statistics stored in golden files.

These represent the baseline performance characteristics of a benchmark on a specific architecture.

Result of running a benchmark and comparing against golden.

Warnings that may be emitted during benchmark comparison.

Machine architecture configuration.

Used to generate unique identifiers for golden file directories, ensuring benchmarks are only compared against equivalent hardware.

Run a benchmark golden test.

This function:

1. Runs warm-up iterations (discarded)
2. Runs the actual benchmark
3. Writes actual results to .actual file
4. If no golden exists, creates it (first run)
5. Otherwise, compares against golden with tolerance

The result includes any warnings (e.g., variance changes).