This project is read-only.
2
Vote

Parallel execution of tests

description

FsCheck test runs should be fairly easily parallelizable. Add a configuration parameter that indicates the desired number of tasks t to split in, and then just split the required number of test cases in t, running each in a separate thread. FsCheck uses at this point no mutable values except in the generator registration (afaik), and some basic groundwork has been done by refactoring the test runner into a function 'tests' which generates an infinite sequence of test runs lazily. Use of the async computation expression would be nice to try here.

comments

mausch wrote Jun 20, 2012 at 6:00 AM

I attempted to run FsCheck tests in parallel through Fuchu ( https://github.com/mausch/Fuchu ) and got a race condition in the memoization function:

{{
Exception: System.ArgumentException: An item with the same key has already been added.
at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
at System.Collections.Generic.Dictionary2.Insert(TKey key, TValue value, Boolean add)
at System.Collections.Generic.Dictionary
2.Add(TKey key, TValue value)
at FsCheck.Common.memoizeWith[a,b](IDictionary2 memo, FSharpFunc2 f, a n)
at FsCheck.Runner.check[a](Config config, a p)
}}

I'll try to fix it when I get some time.

mausch wrote Jun 20, 2012 at 9:30 PM

I removed the memoization from TypeClass.GetInstance and the error went away.

mausch wrote Jun 20, 2012 at 10:47 PM

A more detailed stack trace:


Exception: System.ArgumentException: An item with the same key has already been added.
at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource)
at System.Collections.Generic.Dictionary2.Insert(TKey key, TValue value, Boolean add)
at System.Collections.Generic.Dictionary
2.Add(TKey key, TValue value)
G:\prg\fscheck\FsCheck\Common.fs(21,1): FsCheck.Common.memoizeWith[a,b](IDictionary2 memo, FSharpFunc2 f, a n)
G:\prg\fscheck\FsCheck\TypeClass.fs(154,1): FsCheck.TypeClass.TypeClass1.GetInstance(Type instance, FSharpOption1 arguments)
G:\prg\fscheck\FsCheck\TypeClass.fs(182,1): FsCheck.TypeClass.TypeClass1.InstanceFor[T,TypeClassT](FSharpOption1 arguments)
G:\prg\fscheck\FsCheck\Property.fs(135,1): FsCheck.Testable.property[a](a p)
G:\prg\fscheck\FsCheck\Runner.fs(256,1): FsCheck.Runner.check[a](Config config, a p)
G:\prg\Fuchu\Fuchu.FsCheck\FsCheck.fs(33,1): Fuchu.FuchuFsCheckModule.testProperty@31.Invoke(Unit _arg1)
G:\prg\Fuchu\Fuchu\Fuchu.fs(228,1): Fuchu.Impl.execOne@225.Invoke(Tuple`2 tupledArg)

kurt2001 wrote Jun 21, 2012 at 2:22 AM

Thanks - I've added your memoization change with the TimeSpan and int64 changes.