2D Head with a clock as an eyeball.
 Wednesday, May 14, 2008

Recently I discovered the xUnit.NET framework for unit testing, and it is a very nice minimalist framework for TDD. I especially like writing tests for F# code using it. It allows me to write very short test suites against any CLR library, in most .NET languages. For example:

#light

open Xunit
open NT.Model

let sut = new Person("Jim")

[<Fact>]
let jim_is_a_geek() = Assert.True(sut.IsAGeek)

The Person class could be written in C# or VB and used here. Also note that the ultimate class name of this test is inferred by the filename. I could add explicit module and namespace information if necessary.

In something like MbUnit / NUnit / Gallio I have to use the following OO style syntax...

#light
namespace NT.Model.Specs
module PersonSpecs

open MbUnit 
open NT.Model

[<TestFixture>]
type jims_behaviours() =
  let sut = new Person("Jim")
  [<Test>]
  member j.is_a_geek() = Assert.True(sut.IsAGeek)

Note that the namespace and module must be declared in order for the gallio runner to discover the tests. I think this might be a bug...

However, regardless of the framework you use, F# also lends itself well to setting up for customized approaches to testing. I feel this has some potential for BDD testing in F#. Take the following example:

#light open Xunit let MustEqual expected actual = Assert.Equal(expected, actual) [<Fact>]let a_fluent_assertion() = 0 |> MustEqual 0

 

Taking this one step further we can take advantage of operator overloading to provide extremely terse tests. Note that I don't actually advocate doing this, only pointing out that its possible:

#light open Xunit let (&=) expected actual = Assert.Equal(expected, actual) let (&!=) notExpected actual = Assert.NotEqual(notExpected, actual) [<Fact>] let an_obscure_assertion() = 0 &!= 1

Based on these samples, I think it is clear that without much effort, a fluent specification interface could be arrived at, perhaps even a domain specific one at that.

F# record types are readily testable also:

#light

open Xunit

type Person =
  { name : string;
    age  : int;
  }

let Jim = { name="Jim"; age=28 }
let Fred = { name="Fred"; age=52 }

let IsNot expected actual = Assert.NotEqual(expected, actual)

[<Fact>]
let jim_is_not_fred() = Jim |> IsNot Fred

Sometimes, the assertion framework isn't needed, but having a test runner is still obviously handy. For the next example I setup a discriminated union and then use pattern matching to determine the outcome of the test simply using the built in failwith keyword:

#light

open Xunit

type Animal =
  | Dog
  | Cat

type Person =
  { name : string;
    age : int;
    pet : Animal; }

let Jim = { name="Jim"; age=28; pet=Cat; }

[<Fact>]
let Jim_cannot_own_cats() = 
  match Jim.pet with
    | Cat -> failwith("Jim owns a cat!")
    | _ -> ()

A snippet from the xUnit console runner output for this failing test:

Tests failed:

1) Jims_specifications.jim_cannot_own_cats : Microsoft.FSharp.Core.FailureException : Jim owns a cat!

On that...hopefully soon tools like Gallio will provide us with some F# support in Visual Studio for the Resharper test runner or even MSTest. At current support for xUnit testing with Gallio in F# projects appears to be a non starter. Alternatively, latest SharpDevelop betas have inbuilt F# and NUnit support which I find to be quite good.

In the meantime we can go all 'old school' on VS 2008 to provide some ease of use.

In the following screen shot I've added the xUnit.NET console runner to external tools by selecting from the menu, Tools, External Tools...

externalTools

Next I like to map this to a keyboard chord, say Ctrl+R,Ctrl+X:

MappingToKeyBoard 

So that now after building, I can run my entire test suite using my keyboard shortcut to yield a testing experience that is 'good enough' for TDD/BDD development in F#, well, that's my opinion of course :)

 testResult


Filed under:  |  |  | 
Wednesday, May 14, 2008 10:24:41 PM (Cen. Australia Standard Time, UTC+09:30)
Ahh corner cases!
Looks like Gallio doesn't handle the global (unnamed) namespace properly. I'll see what I can do about it.
Thursday, May 15, 2008 12:37:34 AM (Cen. Australia Standard Time, UTC+09:30)
Silly question, but which version of xUnit.Net are you using?

At least in v1.0 RTM, Xunit.Sdk.TypeUtility.IsTestClass contains code that checks whether a type is abstract. If so, then it does not consider the type to be a valid test class. However, in your first example, F# is creating a static type to contain the test. Static types are really just abstract types so it doesn't work. At least not when I tried it.

As for Gallio, I have fixed the bug with empty namespaces. I've also modified MbUnit to automatically infer that a type is a test fixture even if no [TestFixture] is present as long as it has a test method within it.

MbUnit (unlike the version of xUnit.Net I'm using, apparently) already allows abstract test classes. So the following now sample works as expected...

#light
open MbUnit.Framework

[<Test>]
let def() = Assert.True(false)

Anyways I'll send you a fresh build to play with tomorrow.
Thursday, May 15, 2008 5:23:40 AM (Cen. Australia Standard Time, UTC+09:30)
Wow - thanks for the extremely prompt replies Jeff,

First off, congrats on such a great platform that Gallio is, I cant wait to really get running with it. Im also very keen on Icarus as a test runner. Also, awesome to see MbUnit.Framework allowing for this style of test, very exciting.

AFAIK the patch described here made it to the RTM release of xUnit 1.0, however you're completely correct, it will ignore abstract types. I can confirm I'm using the RTM dll.

...I feel like Im about to say "Works on my machine!" :D

When I have a source file called Specs.fs that contains:

#light

open Xunit

let should_fail() = Assert.True(false)

Using reflector to disassmble the resultant class in the global namespace: (in C#)

[CompilationMapping(7)]
public class Specs
{
// Methods
[Fact]
public static void should_fail()
{
Assert.True(false);
}
}

Maybe I've missed something, but the type doesnt appear to be static, although the value for CompilationMappingAttribute would indicate that it is really a module as far as F# is concerned, I'm not sure what goes on after this point, does this render the class abstract?

Anyhow, I hope I've helped clarify.









Friday, May 16, 2008 2:00:12 PM (Cen. Australia Standard Time, UTC+09:30)
When I compile the following with F# 1.9.4.15:

#light
open Xunit

[<Fact>]
let should_fail() = Assert.True(false)

I get:

[CompilationMapping(7)]
public static class Test
{
[Fact]
public static void should_fail();
}

Now ironically this will work for MbUnit v3 (in my latest build) but not xUnit.Net. I'm sure Brad and Jim can change things to allow static classes to be used too.
Friday, May 16, 2008 2:42:26 PM (Cen. Australia Standard Time, UTC+09:30)
Oop! Looks like I missed the blog on the latest version of the F# compiler coming out 2 weeks ago. Thanks for pointing this out Jeff, and especially for acting so fast on this.

For those interested the 1.9.4.15 compiler is available here

Comments are closed.
© Copyright 2008 Jim Burger