fscheck Discussions Rss Feed

New Post: Unable to run C# NUnit examples

brianhv — Sat, 06 Apr 2013 01:42:48 GMT

The answer was buried in the NUnit documentation and the Resharper options screen. I had to copy FsCheck.NUnit.Addin.dll to the appropriate bin/addins directory. On my machine, that was at C:\Program Files (x86)\JetBrains\ReSharper\v7.1\Bin\addins. I had to create the addins directory there myself. It seems like a similar procedure should work for the standalone test runner but I couldn't get it to work.

I also changed a Resharper setting; not sure if it's necessary or not:
Resharper -> Options -> Tools -> Unit Testing -> NUnit -> Load NUnit Addins -> Always

New Post: Unable to run C# NUnit examples

brianhv — Sat, 06 Apr 2013 01:04:26 GMT

I downloaded the zipfile with the current source tree and successfully built it in VS 2012. However, the tests in the FsCheck.NUnit.CSharpExamples project fail in both the ReSharper test runner and in the NUnit GUI. In both cases, all three tests fail with the message "Test method has non-void return type, but no result is expected."

Is there something else I need to do to get this to work?

New Post: Infinite loop on impossible gen constraints

jackfoxy — Tue, 18 Dec 2012 20:51:34 GMT

It may be useful to have a live example of a custom gen that can generate an infinite loop.

https://github.com/jackfoxy/RandomBitsSolution/blob/master/RandomBitsTest/FsCheckTest.fs

let rndByteSeqGen = 
    gen {
//        let! inclLower = Arb.generate  //http://fscheck.codeplex.com/workitem/16796
        let! inclLower = Gen.suchThat (fun x -> x < System.Byte.MaxValue) Arb.generate
        let! exlUpper = Gen.suchThat (fun x -> x > inclLower) Arb.generate
        let! count = Gen.suchThat (fun x -> x >= 1) Arb.generate
        let x = new RandomBits("7816001122100133")
        return (x.RndByteSeq (inclLower, exlUpper, count) |> List.ofSeq), (inclLower, exlUpper, count) 
    }

inclLower is the inclusive lower bound and exlUpper is the exclusive upper bound for generation of a random sequence of bytes. If inclLower generates System.Byte.MaxValue, then exlUpper can never be generated. This will happen on nearly half of the generation attempts.

New Post: Typeclasses

kurt2001 — Fri, 07 Dec 2012 12:55:32 GMT

Hi,

Yes, in fact I have! There are two main uses of typeclasses in FsCheck. For one of them, I have considered replacing the current (runtime) implementation with an implementation at compile time. For the other one, I don't think it makes sense.

Use 1: map from type to a default generator for the type. I think using a static overloading mechanism here is not idiomatic, as it takes useful functionality away - the ability to change default generators for types at runtime. Two important use cases are - (1) specifying a number of generators scoped to a bunch of test methods, and (2) overriding a particular generator while keeping the default generator for the aggregate type. Say you have a record type (int, string, string, double), and you are fine with the default int and double generator but don't want empty strings. You can then just override the string generator, and keep using the default (reflection-based) generator for the record type. With static overloading you'd have to rewrite the record type generator, which is pretty tedious. Also you end up wrapping primitive types a lot to take care of special cases (NonEmptyString, NonZeroDouble, etc).

To sum up, I think making the whole thing dynamic has some benefits that outweigh the cost (i.e. no type safety so occasionally annoying errors) . I have tried to remedy the latter a bit by having lots of debug output when you override generators (it says what generators it adds, which ones it overrides, etc).

Use 2: map from result type of a test to a Testable (e.g. for unit the test fails it doesn't thrown an exception, for boolean if it returns true, etc). At one point I started refactoring that to the approach you mentioned, but as I remember I ran into ordering issues, it became unwieldy, and compilation became pretty slow pretty quickly. I am unsure if the latter would carry over to FsCheck users (I suspect not). In the end I gave up somewhere halfway through because since this use case is effectively completely contained inside FsCheck, it doesn't bring much to the table for users. It would however get rid of some initialization problems that can potentially still happen.

It's an interesting technique.

Hope that makes sense!

Best, Kurt

New Post: Typeclasses

mausch — Fri, 07 Dec 2012 02:56:23 GMT

Hi, I wonder if you have considered using a technique like https://code.google.com/p/fsharp-typeclasses/ to implement typeclasses in FsCheck.

I know this would be a pretty big change, perhaps not worth the effort. But it's definitely a closer approximation to Haskell's typeclasses (e.g. type-safe, no reflection).

So basically, just trying to pick your brain about the potential pros and cons of this hypothetical change :)

For example, it would seem to make it harder to infer generators via reflection for discriminated unions.

Cheers,
Mauricio

New Post: Generating data for use outside of FsCheck

jackfoxy — Sat, 20 Oct 2012 00:15:21 GMT

I'll take a look at that, thanks. I still think sample is very cool. I wanted to use FsCheck as a data generator on another project, but could not figure out how to do it. Now I can use FsCheck to generate data for any purpose.

New Post: Generating data for use outside of FsCheck

mausch — Sat, 20 Oct 2012 00:11:04 GMT

> Unfortunately this is not enough for me to properly debug.

Try overriding ToString() in the implementation of PhysicistQueue, or switch to a record type.

> I would like to step into one of the failing cases with the debugger, but how do I break into a failing case and not a successful case?

You could use System.Diagnostics.Debugger.Break() when the equality test returns false.

New Post: Generating data for use outside of FsCheck

jackfoxy — Fri, 19 Oct 2012 23:51:20 GMT

I've only started using FsCheck, so likely I am missing something. Here is my use case for sample. Some of my problem stemmed from not understanding FsCheck well enough. My original fsCheck code was failing:

fsCheck "PhysicistQueue" (Prop.forAll (Arb.fromGen physicistQueueIntGen)

...and showing me my generated tuple

FSharpx.DataStructures.Tests.IQueueTest.reverse . reverse = id:PhysicistQueue-Falsifiable, after 2 tests (0 shrinks) (StdGen (2139253415,295630954)):(seq [0; 0; 1; 2; ...], [0; 0; 1; 2; -1])

Unfortunately this is not enough for me to properly debug. I would like to step into one of the failing cases with the debugger, but how do I break into a failing case and not a successful case?

Anyway, sample lets me do this:

let y = sample 100 physicistQueueIntGen

//so I can set a breakpoint after a failure

let rec loop l =

match l with

| [] -> ()

| ((q : IQueue<int>), (l' : int list))::tl ->

let qList = q :?> PhysicistQueue<int> |> PhysicistQueue.rev |> PhysicistQueue.rev |> Seq.toList

if (qList = l')

then loop tl

else loop tl
loop y

New Post: Generating data for use outside of FsCheck

jackfoxy — Fri, 19 Oct 2012 17:14:46 GMT

Oustanding! See my stackoverflow comment. Tweeted this solution as well.

New Post: Generating data for use outside of FsCheck

kurt2001 — Fri, 19 Oct 2012 12:02:28 GMT

Yep, replied with a sample function. I'll move it to the Gen module at some point (it's actually copy pasted from FsCheck.Test, where I use it to test FsCheck's generators....)

New Post: Generating data for use outside of FsCheck

jackfoxy — Fri, 19 Oct 2012 04:44:38 GMT

Sure would be useful if I had access to actual generated data structures outside of FsCheck from the generators I construct. http://stackoverflow.com/questions/12900470/how-do-i-just-generate-data-with-fscheck

New Post: DateTime generator and leap days

mausch — Tue, 09 Oct 2012 23:30:02 GMT

Yeah, good point, intuitively it seemed like a good idea but the ocurrences of leap day bugs I've seen involved parsing dates from external sources, e.g. web forms, functions like string -> DateTime or int -> int -> DateTime, but not functions DateTime -> a

Of course, it's always possible to screw up, e.g.:

let sameDayIn2011 (t: DateTime) = DateTime(2011, t.Month, t.Day)

let sameDayInApril (t: DateTime) = DateTime(t.Year, 4, t.Day)

FsCheck currently finds the bug in sameDayInApril with the default MaxTest = 100 in most runs, but not likely in sameDayIn2011. If I bump MaxTest it seems to take between 260 and 4500 tests to find the failing case.

But I agree that it's probably not common enough to be worth the change, and with a domain like DateTime you shouldn't expect to find all bugs in 100 tests anyway.

Cheers

New Post: DateTime generator and leap days

kurt2001 — Tue, 09 Oct 2012 00:52:48 GMT

Hmm, don't know. Wouldn't that matter mostly to implementors of DateTime - the rest of us hopefully deal with addition subtraction and day counting through DateTime and TImeSpan -right? I think probably more mistakes happen when converting between different type of dates, like DateTime, an Excel date or a SQL date. Not sure if FsCheck matters much there.

I can of course be swayed by a convincing example ;)

New Post: DateTime generator and leap days

mausch — Mon, 08 Oct 2012 00:48:20 GMT

I was just thinking that it might be a good idea to give a higher probability to leap days (and possibly also +/- 1 day around them) to catch the frequent mistakes we programmers tend to make around them.

What do you think?

New Post: No instances of class FsCheck.Arbitrary`1[a] for type System.Object with arguments set []

kurt2001 — Thu, 27 Sep 2012 18:40:43 GMT

Oh, I think that's some initialization order/side effecting weirdness with Arbitrary registration hitting you. The Arbitrary instances are kept in a dictionary somewhere, and the dictionary is initialized with the default instances as part of the initialization of the module in which it is defined.

As per F# rules, this initialization is only run when a function is called in that module. I'm guessing it's in the Arb or Gen module. Anyway, somehow your earlier code didn't trigger that initialization so FsCheck didn't find any of its default generator instances (such as obj and int32, but I overlooked it in the case of obj and thought there just wasn't any). I'm thinking the other generators you wrote somehow did trigger it, so they worked fine subsequently.

This is a gnarly problem I can't seem to find a really good solution for - in QuickCheck typeclasses take care of this, but on the other hand doing it dynamically gives you some interesting flexibility too.

New Post: No instances of class FsCheck.Arbitrary`1[a] for type System.Object with arguments set []

mausch — Thu, 27 Sep 2012 03:34:25 GMT

I tried defining all type arguments to int in the final Arbs when used in Prop.forAll and got "No instances of class FsCheck.Arbitrary`1[a] for type System.Int32 with arguments set []" which is even weirder.

Then I noticed that the exception was thrown from keyValuePairGen... I switched to a monadic gen expression and it fixed it... still not sure why, might be some unexpected strict evaluation. https://github.com/mausch/fsharpx/commit/ffd9a13a77f519c789b5c118704878c25c283436

Cheers

New Post: No instances of class FsCheck.Arbitrary`1[a] for type System.Object with arguments set []

kurt2001 — Thu, 27 Sep 2012 01:36:20 GMT

I would guess one of the generic type arguments gets unexpectedly constrained to object somehow. Try adding them explicitly to the Arb.generate calls, then the compiler will warn about them.

The error basically says that it needs to generate an obj at some point but doesn't have an Arbitrary<obj> instance.

New Post: No instances of class FsCheck.Arbitrary`1[a] for type System.Object with arguments set []

mausch — Wed, 26 Sep 2012 20:13:27 GMT

Hi, I'm getting an exception when trying to use a KeyValuePair generator through forAll, and I can't for the life of me figure out what I'm doing wrong.
The exception is thrown in https://github.com/mausch/fsharpx/commit/ad5adde2ee736129cad58f1c832b2a92a7343323#L1R196
and it's this:

Test 'FSharpx.LensTests.LensKeyValuePairKey' failed: System.Exception : No instances of class FsCheck.Arbitrary`1[a] for type System.Object with arguments set []
	at FsCheck.TypeClass.mi@176.Invoke(String message)
	at Microsoft.FSharp.Core.PrintfImpl.go@512-3[b,c,d](String fmt, Int32 len, FSharpFunc`2 outputChar, FSharpFunc`2 outa, b os, FSharpFunc`2 finalize, FSharpList`1 args, Int32 i)
	at Microsoft.FSharp.Core.PrintfImpl.run@510[b,c,d](FSharpFunc`2 initialize, String fmt, Int32 len, FSharpList`1 args)
	at Microsoft.FSharp.Core.PrintfImpl.capture@529[b,c,d](FSharpFunc`2 initialize, String fmt, Int32 len, FSharpList`1 args, Type ty, Int32 i)
	at .$Reflect.Invoke@617-4.Invoke(T1 inp)
	at FsCheck.TypeClass.GetInstance@165.Invoke(Tuple`2 tupledArg)
	at FsCheck.Common.f@1[a,b](IDictionary`2 memo, FSharpFunc`2 f, a n, Unit _arg1)
	at FsCheck.Common.memoizeWith[a,b](IDictionary`2 memo, FSharpFunc`2 f, a n)
	at FsCheck.TypeClass.TypeClass`1.GetInstance(Type instance, FSharpOption`1 arguments)
	at FsCheck.TypeClass.TypeClass`1.InstanceFor[T,TypeClassT](FSharpOption`1 arguments)
	at FsCheck.Arb.generate[Value]()
	G:\prg\FSharpx\tests\FSharpx.Tests\LensTests.fs(174,0): at FSharpx.LensTests.keyValuePairGen[a,b]()
	G:\prg\FSharpx\tests\FSharpx.Tests\LensTests.fs(179,0): at FSharpx.LensTests.keyValuePairArb[a,b]()
	G:\prg\FSharpx\tests\FSharpx.Tests\LensTests.fs(196,0): at FSharpx.LensTests.checkLensKV[a,b,c](String name, Lens`2 lens)
	G:\prg\FSharpx\tests\FSharpx.Tests\LensTests.fs(201,0): at FSharpx.LensTests.LensKeyValuePairKey()

The "fsCheck" function is just a trivial wrapper over Check.One, and LensProperties.GetSet is Lens<'a,'b> -> 'a -> bool (the first argument is provided there, giving a "'Value -> 'Testable" of 'a -> bool).
I'm doing something very similar a few lines below that: https://github.com/mausch/fsharpx/blob/ad5adde2ee736129cad58f1c832b2a92a7343323/tests/FSharpx.Tests/LensTests.fs#L242 which works fine, so I'm not sure what's wrong here, or what the exception message means.

Thanks for any hints.

Mauricio

New Post: not working with VS2012

kurt2001 — Sat, 11 Aug 2012 19:47:56 GMT

I've now been able to look into this more.

The bindingredirect does work, because all popular test runners including xunit pick up any app.config files for dlls that they load. So that's good.

I've checked Daniel Mohl's additions to FsUnit, and he calls Add-BindingRedirect after install. According to the NuGet docs, NuGet does that automatically on install, but I haven't been able to get that to work. This is not foolproof though, because the NuGet doc states that for Add-BindingRedirect to work reliably, the project should have been built (it looks at all the assemblies in the output folder). In my tests, it usually works though.

But, I have the same problem with XUnit - they released a new version, which means it needs a bindingredirect for xunit.dll too. Updating to a new version of xunit might happen at any time after install of FsCheck, so in the presence of strongly named assemblies, NuGet users simply have to know about Add-BindingRedirect.

For these reasons, I don't think I'll be adding a call to Add-BindingRedirect to the FsCheck package now, but I'll start a discussion on the NuGet forum to see what they have to say about it. Happy to change my mind.

Given all that, here's my plan:

- If you want to get FsCheck 0.8.2 to work with VS2012, building your test and running Add-BindingRedirect in the NuGet console should fix any issues you have. If you don't have NuGet, you'll have to do it manually.

- I'll move the source code to VS2012, targeting .NET 4.0 and FSharp.Core 4.0 for now, but compiling with fsc 3.0. I haven't found any backwards compatibility issues with that so far. I'll check that in shortly.

- Some time after VS2012 is officially released (Aug 15), I'll release a new NuGet package that has two builds of FsCheck:

- for .NET 4.0 and FSharp.Core 4.0

- for .NET 4.5 and FSharp.Core 4.3

That should then work with VS2010 and VS2012, if for the latter .NET4.5 is targeted. I don't want more than one FsCheck package on NuGet, really. That's just confusing, especially since Add-BindingRedirect helps with all other cases. At some point I'll probably drop the first package and just release .NET4.0 with FSharp.Core 4.3, when F#3.0 has become widespread enough (there's enough new features in there to make it worth it!)

The last point is tentative as I haven't tested whether it works yet (and need to do some MSBuild magic to make it happen).

Hope that makes sense - let me know if you have any concerns.

New Post: not working with VS2012

kurt2001 — Wed, 25 Jul 2012 01:12:17 GMT

Thanks guys, sorry for not getting to it - new recent addition to the family is taking up my time at the moment! Hopefully I'll get to it this weekend or so. Haven't forgotten in any case.