Property

A Property' is a generator that can fail and keeps a track of some information about the test run.

In most cases, you will probably want to use Property instead, which fixes e at String.

Property that uses strings as errors

Most of falsify's internal functions work with Property', but most user-facing functions use Property instead.

Generator of a random value

Generators can be combined through their Functor, Applicative and Monad interfaces. The primitive generator is prim, but most users will probably want to construct their generators using the predefined from Test.Falsify.Generator as building blocks.

Generators support "internal integrated shrinking". Shrinking is integrated in the sense of Hedgehog, meaning that we don't write a separate shrinker at all, but the shrink behaviour is implied by the generator. For example, if you have a generator genList for a list of numbers, then

filter even <$> genList

will only generate even numbers, and that property is automatically preserved during shrinking. Shrinking is internal in the sense of Hypothesis, meaning that unlike in Hedgehog, shrinking works correctly even in the context of monadic bind. For example, if you do

do n <- genListLength
   replicateM n someOtherGen

then we can shrink n and the results from someOtherGen in any order (that said, users may prefer to use the dedicated list generator for this purpose, which improves on this in a few ways).

NOTE: Gen is NOT an instance of Alternative; this would not be compatible with the generation of infinite data structures. For the same reason, we do not have a monad transformer version of Gen either.

Range of values

Generate value and add it to the log

Generalization of gen that doesn't depend on a Show instance

No log entry is added if Nothing.

N-ary predicate

A predicate of type

Predicate '[Int, Bool, Char, ..]

is essentially a function Int -> Bool -> Char -> .. -> Bool, along with some metadata about that function that allows us to render it in a human readable way. In particular, we construct an Expr for the values that the predicate has been applied to.

Infix version of at

Typical usage example:

assert $
     P.relatedBy ("equiv", equiv)
  .$ ("x", x)
  .$ ("y", y)

Fail the test if the predicate does not hold

Test failure

Discard this test

Variation on collect that does not rely on Show

See collect for detailed discussion.

Label this test

See also label, which does not rely on Show.

Motivation

Labelling is instrumental in understanding the distribution of test data. For example, consider testing a binary tree type, and we want to test some properties of an insert operation (example from "How to specify it!" by John Hughes):

prop_insert_insert :: Property ()
prop_insert_insert = do
  tree     <- gen $ ..
  (k1, v1) <- gen $ ..
  (k2, v2) <- gen $ ..
  assert $ .. (insert k1 v1 $ insert k2 v2 $ tree) ..

We might want to know in what percentage of tests k1 == k2:

collect "sameKey" [k1 == k2]

When we do, falsify will report in which percentage of tests the key are the same, and in which percentage of tests they are not.

Labels with multiple values

In general, a particular label can have multiple values in any given test run. Given a test of n test runs, for each value v reported, falsify will report what percentage of the n runs are labelled with v. That means that these percentages may not add up to 100%; indeed, if we had

collect "sameKey" [True]
..
collect "sameKey" [False]

or, equivalently,

collect "sameKey" [True, False]

then every test would have been reported as labelled with True (100%) as well as with False@ (also 100%). Of course, if we do (like above)

collect "sameKey" [k1 == k2]

each test will be labelled with either True or False, and the percentages will add up to 100%.

Difference from QuickCheck

Since you can call collect anywhere in a property, it is natural that the same label can have multiple values in any given test run. In this regard, collect is closer to QuickCheck's tabulate. However, the statistics of tabulate can be difficult to interpret; QuickCheck reports the frequency of a value as a percentage of the total number of values collected; the frequency reported by falsify here is always in terms of number of test runs, like collect does in QuickCheck. We therefore opted to use the name collect rather than tabulate.

Log some additional information about the test

This will be shown in verbose mode.

Get the context for the current test run

Generate value depending on the test iteration

Rough analogue to QuickCheck's sized function: for test iteration i out of a total of n tests, the callback is passed i / n. This can be used for example to define a Range that starts small and gets larger in successive tests.

Note that this is just a convenience function around getContext; if you want to define ranges that depend on the test iteration i in a different way, use getContext instead.

Test the minimum error thrown by the property

If the given property passes, we will discard this test (in that case, there is nothing to test); this test is also discarded if the given property discards.

NOTE: When testing a particular generator, you might still want to test with some particular property in mind. Otherwise, the minimum value will always simply be the value that the generator produces when given the Minimal sample tree.

See also testMinimumForIteration.

Generalization of testMinimum

See testShrinkingForIteration for detailed discussion of the context.

Test shrinking of a property

A property is normally only shrunk when it fails. We do the same here: if the property succeeds, we discard the test and try again.

If the given property itself discards immediately, then this generator will discard also; otherwise, only shrink steps are considered that do not lead to a discard.

See also testShrinkingForIteration.

Generalization of testShrinking for an arbitrary Iteration

Some properties may behave quite differently given a different iteration context, in which case it is important to be explicit about this.

The shrinking context is constructed so that it accurately reflects the path: Initial for the root of the tree, and then Shrinking as we follow edges downwards. There is no repeated Final step: testShrinkingForIteration and testShrinking are typically used to verify that successive shrink steps result in values that are closer to the generator's origin, which is trivially violated by that repeated final step.

The static context is inherited from the parent property.

Test shrinking of a generator

We check any shrink step that the generator can make (independent of any property).

See also testShrinkingOfGenForIteration.

Generalization of testShrinkingOfGen

See testShrinkingForIteration for detailed discussion of the context.

Test output of the generator

Generalization of testGen

Function a -> b which can be shown, generated, and shrunk

Apply function to argument

See also the Fn, Fn2, and Fn3 patter synonyms.

Like applyFun, but for binary functions

Like applyFun, but for ternary functions

Pattern synonym useful when generating functions of one argument

Pattern synonym useful when generating functions of two arguments

Pattern synonym useful when generating functions of three arguments

Marked element in a container

Marking elements can be a useful technique for dropping elements from a container.

• Locally marking elements to Drop makes it possible to enforce some global constraints about minimum number of required elements before actually dropping them (see selectAllKept). Example: list.
• For containers where we cannot remove random elements, the marks can be used for "outwards propagation": if this element is dropped, then those elements must also be dropped. Example: tree.

Should an element in a container be kept or dropped?

See also Marked.

Hedgehog-style shrink tree

Default generators

GenDefault is similar to QuickCheck's Arbitrary class along with some deriving via helpers. Unlike Arbitrary, GenDefault allows one to choose between sets of default generators with user-defined tags. See Test.Falsify.GenDefault.Std for the standard tag with a few useful instances.

Type tag for these "standard" default generators.

You can use this tag directly or choose type-by-type with ViaTag.