QuickStart

A Simple Example

A simple example of a property definition is

let revRevIsOrig (xs:list<int>) = List.rev(List.rev xs) = xs

This property asserts that the reverse of the reverse of a list is the list itself. To check the property, we load this definition in F# interactive and then invoke

> Check.Quick revRevIsOrig;;
Ok, passed 100 tests.

When a property fails, FsCheck displays a counter-example. For example, if we define

let revIsOrig (xs:list<int>) = List.rev xs = xs

then checking it results in

> Check.Quick revIsOrig;;
Falsifiable, after 2 tests (2 shrinks) (StdGen (1809913113,295281725)):
[0; 1]

FsCheck also shrinks the counter example, so that it is the minimal counter example that still fails the test case. In the example above, we see that the counter example is indeed minimal: the list must have at least two different elements. FsCheck also displays how many times it found a smaller (in some way) counter example and so proceeded to shrink further.

Using FsCheck

To use FsCheck, you download the latest FsCheck source or binary. Build and reference the assembly in any projects containing specifications or test data generators. You can then test properties by loading the module they are defined in into F# interactive, and calling

Check.Quick <propertyName>

or by running and writing a small console application that calls the Check function. Integration with unit test runners such as xUnit and NUnit is possible as well - see Usage Tips for an example.

Grouping properties

Usually, you'll write more than one property to test. FsCheck allows you to group together properties as static members of a class:

type ListProperties =
    static member ``reverse of reverse is original`` xs = RevRevIsOrig xs
    static member ``reverse is original`` xs = RevIsOrig xs

These can be checked at once using the Check.QuickAll function:

> Check.QuickAll<ListProperties>();;
--- Checking ListProperties ---
ListProperties.reverse of reverse is original-Ok, passed 100 tests.
ListProperties.reverse is original-Falsifiable, after 2 tests (1 shrink) (StdGen (1810663113,295281725)):
[1; 0]

FsCheck now also prints the name of each test.
Since all top level functions of a a module are also compiled as static member of a class with the name of the module, you can also use Check.QuickAll to test all the top level functions in a certain module. However, the type of a module is not directly accessible via F#, so you can use the following trick:

> Check.QuickAll typeof<ListProperties>.DeclaringType;;
--- Checking QuickStart ---
QuickStart.revRevIsOrig-Ok, passed 100 tests.
QuickStart.revIsOrig-Falsifiable, after 8 tests (6 shrinks) (StdGen (1810793113,295281725)):
[1; 0]
QuickStart.revRevIsOrigFloat-Falsifiable, after 7 tests (6 shrinks) (StdGen (1810843113,295281725)):
[nan]


What do I do if a test loops or encounters an error?

In this case we know that the property does not hold, but Check.Quick does not display the counter-example. There is another testing function provided for this situation. Repeat the test using

Check.Verbose <property_name>

which displays each test case before running the test: the last test case displayed is thus the one in which the loop or error arises. Check.VerboseAll can be used with types and modules to check groups of properties verbosely.

Caveat

The property above (the reverse of the reverse of a list is the list itself) is not always correct. Consider a list of floats that contains infinity, or nan (not a number). Since infinity <> infinity, and nan <> nan, the reverse of the reverse of [nan,nan] is not actually equal to [nan,nan] if you use straightforward element by element comparison. FsCheck has a knack for finding this kind of specification problem. However, since this behavior is seldom what you want, FsCheck only generates values that are 'neatly' comparable when you leave the type polymorphic (currently, unit, bool, char and string values). To see this error in action, force FsCheck to generate lists of floats:

let revRevIsOrigFloat (xs:list<float>) = List.rev(List.rev xs) = xs


> Check.Quick revRevIsOrigFloat;;
Falsifiable, after 14 tests (2 shrinks) (StdGen (1811583113,295281725)):
[nan]

Last edited Jun 4, 2010 at 7:08 PM by kurt2001, version 11

Comments

No comments yet.