| Copyright | (c) 2026 |
|---|---|
| License | MIT |
| Maintainer | your.email@example.com |
| Safe Haskell | None |
| Language | Haskell2010 |
Test.Hspec.BenchGolden.Types
Description
This module defines the core data types used by the golds-gym framework:
BenchGolden- Configuration for a benchmark golden testBenchConfig- Configurable benchmark parametersGoldenStats- Statistics stored in golden filesArchConfig- Machine architecture identificationBenchResult- Result of comparing benchmark against golden
Synopsis
- data BenchGolden = BenchGolden {
- benchName :: !String
- benchAction :: !(IO ())
- benchConfig :: !BenchConfig
- data BenchConfig = BenchConfig {}
- defaultBenchConfig :: BenchConfig
- data GoldenStats = GoldenStats {
- statsMean :: !Double
- statsStddev :: !Double
- statsMedian :: !Double
- statsMin :: !Double
- statsMax :: !Double
- statsPercentiles :: ![(Int, Double)]
- statsArch :: !Text
- statsTimestamp :: !UTCTime
- statsTrimmedMean :: !Double
- statsMAD :: !Double
- statsIQR :: !Double
- statsOutliers :: ![Double]
- data ArchConfig = ArchConfig {}
- data BenchResult
- = FirstRun !GoldenStats
- | Pass !GoldenStats !GoldenStats ![Warning]
- | Regression !GoldenStats !GoldenStats !Double !Double !(Maybe Double)
- | Improvement !GoldenStats !GoldenStats !Double !Double !(Maybe Double)
- data Warning
- = VarianceIncreased !Double !Double !Double !Double
- | VarianceDecreased !Double !Double !Double !Double
- | HighVariance !Double
- | OutliersDetected !Int ![Double]
Benchmark Configuration
data BenchGolden Source #
Configuration for a single benchmark golden test.
Constructors
| BenchGolden | |
Fields
| |
Instances
| Example BenchGolden Source # | Instance for BenchGolden without arguments. | ||||
Defined in Test.Hspec.BenchGolden Associated Types
Methods evaluateExample :: BenchGolden -> Params -> (ActionWith (Arg BenchGolden) -> IO ()) -> ProgressCallback -> IO Result # | |||||
| Example (arg -> BenchGolden) Source # | Instance for BenchGolden with an argument. This allows benchmarks to receive setup data from | ||||
Defined in Test.Hspec.BenchGolden Associated Types
Methods evaluateExample :: (arg -> BenchGolden) -> Params -> (ActionWith (Arg (arg -> BenchGolden)) -> IO ()) -> ProgressCallback -> IO Result # | |||||
| type Arg BenchGolden Source # | |||||
Defined in Test.Hspec.BenchGolden | |||||
| type Arg (arg -> BenchGolden) Source # | |||||
Defined in Test.Hspec.BenchGolden | |||||
data BenchConfig Source #
Configurable parameters for benchmark execution and comparison.
Constructors
| BenchConfig | |
Fields
| |
Instances
defaultBenchConfig :: BenchConfig Source #
Default benchmark configuration with sensible defaults.
- 100 iterations
- 5 warm-up iterations
- 15% tolerance on mean time
- 0.01 ms (10 microseconds) absolute tolerance - prevents false failures for fast operations
- Variance warnings enabled at 50% tolerance
- Output to
.golden/directory - Success on first run (creates baseline)
Hybrid Tolerance Strategy
The default configuration uses BOTH percentage and absolute tolerance:
- Benchmarks pass if mean time is within ±15% OR within ±0.01ms
- This prevents measurement noise from failing fast operations (< 1ms)
- For slower operations (> 1ms), percentage tolerance dominates
Set absoluteToleranceMs = Nothing for percentage-only comparison.
Golden File Statistics
data GoldenStats Source #
Statistics stored in golden files.
These represent the baseline performance characteristics of a benchmark on a specific architecture.
Constructors
| GoldenStats | |
Fields
| |
Instances
| FromJSON GoldenStats Source # | |||||
Defined in Test.Hspec.BenchGolden.Types | |||||
| ToJSON GoldenStats Source # | |||||
Defined in Test.Hspec.BenchGolden.Types Methods toJSON :: GoldenStats -> Value # toEncoding :: GoldenStats -> Encoding # toJSONList :: [GoldenStats] -> Value # toEncodingList :: [GoldenStats] -> Encoding # omitField :: GoldenStats -> Bool # | |||||
| Generic GoldenStats Source # | |||||
Defined in Test.Hspec.BenchGolden.Types Associated Types
| |||||
| Show GoldenStats Source # | |||||
Defined in Test.Hspec.BenchGolden.Types Methods showsPrec :: Int -> GoldenStats -> ShowS # show :: GoldenStats -> String # showList :: [GoldenStats] -> ShowS # | |||||
| Eq GoldenStats Source # | |||||
Defined in Test.Hspec.BenchGolden.Types | |||||
| type Rep GoldenStats Source # | |||||
Defined in Test.Hspec.BenchGolden.Types type Rep GoldenStats = D1 ('MetaData "GoldenStats" "Test.Hspec.BenchGolden.Types" "golds-gym-0.2.0.0-7NJIEaTIpAGIPUbJaP5I3x" 'False) (C1 ('MetaCons "GoldenStats" 'PrefixI 'True) (((S1 ('MetaSel ('Just "statsMean") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: (S1 ('MetaSel ('Just "statsStddev") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: S1 ('MetaSel ('Just "statsMedian") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double))) :*: (S1 ('MetaSel ('Just "statsMin") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: (S1 ('MetaSel ('Just "statsMax") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: S1 ('MetaSel ('Just "statsPercentiles") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 [(Int, Double)])))) :*: ((S1 ('MetaSel ('Just "statsArch") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Text) :*: (S1 ('MetaSel ('Just "statsTimestamp") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 UTCTime) :*: S1 ('MetaSel ('Just "statsTrimmedMean") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double))) :*: (S1 ('MetaSel ('Just "statsMAD") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: (S1 ('MetaSel ('Just "statsIQR") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Double) :*: S1 ('MetaSel ('Just "statsOutliers") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 [Double])))))) | |||||
Architecture Configuration
data ArchConfig Source #
Machine architecture configuration.
Used to generate unique identifiers for golden file directories, ensuring benchmarks are only compared against equivalent hardware.
Constructors
| ArchConfig | |
Instances
| FromJSON ArchConfig Source # | |||||
Defined in Test.Hspec.BenchGolden.Types | |||||
| ToJSON ArchConfig Source # | |||||
Defined in Test.Hspec.BenchGolden.Types Methods toJSON :: ArchConfig -> Value # toEncoding :: ArchConfig -> Encoding # toJSONList :: [ArchConfig] -> Value # toEncodingList :: [ArchConfig] -> Encoding # omitField :: ArchConfig -> Bool # | |||||
| Generic ArchConfig Source # | |||||
Defined in Test.Hspec.BenchGolden.Types Associated Types
| |||||
| Show ArchConfig Source # | |||||
Defined in Test.Hspec.BenchGolden.Types Methods showsPrec :: Int -> ArchConfig -> ShowS # show :: ArchConfig -> String # showList :: [ArchConfig] -> ShowS # | |||||
| Eq ArchConfig Source # | |||||
Defined in Test.Hspec.BenchGolden.Types | |||||
| type Rep ArchConfig Source # | |||||
Defined in Test.Hspec.BenchGolden.Types type Rep ArchConfig = D1 ('MetaData "ArchConfig" "Test.Hspec.BenchGolden.Types" "golds-gym-0.2.0.0-7NJIEaTIpAGIPUbJaP5I3x" 'False) (C1 ('MetaCons "ArchConfig" 'PrefixI 'True) ((S1 ('MetaSel ('Just "archId") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Text) :*: S1 ('MetaSel ('Just "archOS") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Text)) :*: (S1 ('MetaSel ('Just "archCPU") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 Text) :*: S1 ('MetaSel ('Just "archModel") 'NoSourceUnpackedness 'SourceStrict 'DecidedStrict) (Rec0 (Maybe Text))))) | |||||
Benchmark Results
data BenchResult Source #
Result of running a benchmark and comparing against golden.
Constructors
| FirstRun !GoldenStats | No golden file existed; baseline created |
| Pass !GoldenStats !GoldenStats ![Warning] | Benchmark passed (golden stats, actual stats, warnings) |
| Regression !GoldenStats !GoldenStats !Double !Double !(Maybe Double) | Performance regression (golden, actual, percent change, tolerance, absolute tolerance) |
| Improvement !GoldenStats !GoldenStats !Double !Double !(Maybe Double) | Performance improvement (golden, actual, percent change, tolerance, absolute tolerance) |
Instances
| Show BenchResult Source # | |
Defined in Test.Hspec.BenchGolden.Types Methods showsPrec :: Int -> BenchResult -> ShowS # show :: BenchResult -> String # showList :: [BenchResult] -> ShowS # | |
| Eq BenchResult Source # | |
Defined in Test.Hspec.BenchGolden.Types | |
Warnings that may be emitted during benchmark comparison.
Constructors
| VarianceIncreased !Double !Double !Double !Double | Stddev increased (golden, actual, percent change, tolerance) |
| VarianceDecreased !Double !Double !Double !Double | Stddev decreased significantly (golden, actual, percent change, tolerance) |
| HighVariance !Double | Current run has unusually high variance |
| OutliersDetected !Int ![Double] | Outliers detected (count, list of outlier timings) |