cLabs

Update: see AllPairs for a cool tool to reduce large combinatorial case sets to only those that really need to be tested.

This came up on the DUnit mailing list and turned into a good example of some options for dealing with testing large case sets in an xUnit framework.

If you have a piece of production code that you need to test with a few different cases, here's a starter approach I use:

procedure TMyTest.Setup;
begin
  // nothing here
end;

procedure TMyTest.TestA;
begin
  DoTest(1, 'X');
end;

procedure TMyTest.TestB;
begin
  DoTest(2, 'Y');
end;

procedure TMyTest.DoTest(Initial: integer; Expected: string);
var
  MyObject: TMyObject;
begin
  MyObject := TMyObject.Create(Initial);
  try
    CheckEquals(Expected, MyObject.Method);
  finally
    MyObject.Free;
  end;
end;

This works pretty well for a few cases, but isn't very scalable. Let's say this jumps up to about 15 cases. If you want, you can still keep the case data hardcoded and do something like this:

procedure TMyTest.Setup;
begin
  // nothing here
end;

procedure TMyTest.TestAll;
const
  Cases: array[0..14, 0..1] of string = (
    ('1',  'A'),
    ('2',  'B'),
    ('3',  'C'),
    ('4',  'D'),
    ('5',  'E'),
    ('6',  'F'),
    ('7',  'G'),
    ('8',  'H'),
    ('9',  'I'),
    ('10', 'J'),
    ('11', 'K'),
    ('12', 'L'),
    ('13', 'M'),
    ('14', 'N'),
    ('15', 'O')
  );
var
  i: integer;
begin
  for i := 0 to 14 do
    DoTest(StrToInt(Cases[i, 0]), Cases[i, 1]);
  end;
end;

procedure TMyTest.DoTest(Initial: integer; Expected: string);
var
  MyObject: TMyObject;
begin
  MyObject := TMyObject.Create(Initial);
  try
    CheckEquals(Expected, MyObject.Method, 'initial value: <' + IntToStr(Initial) + '>');
  finally
    MyObject.Free;
  end;
end;

This has a couple of problems, though. It isn't easily scalable either (though you may not need it to be), and it breaks the whole ‘one test per test method’ -- in other words AllTests now fails and stops executing if only one of the cases it tries fails, and you don't get to execute all of the other cases. If you have 10 out of 15 fail, you'll have to execute the test, find the first failure, fix that case, re-run the test, find the second failure, fix that case, re-run the test ... madness. But sometimes, the code under test is stable enough that going to extra step is not needed. Remember, DTSTTCPW.

But, to make this even better, move your case data out into a database or Excel spreadsheet or text file or XML file, etc. Then change the structure of your test class and build your suite yourself.

First, change your TMyTest to this:

  TMyTest = class(TTestCase)
  private
    FInitialValue: integer;
    FExpected: string;
  public
    property InitialValue: integer write FInitialValue;
    property Expected: string write FExpected;
  published
    procedure DoTest;
  end;

Instead of using convenience Suite building methods, build it yourself:

function BuildSuite: ISuite;
var
  ds: TDataSet;
  MyTest: TMyTest;
  i: integer;
begin
  Result := TTestSuite.Create('MyTest Suite');
  // ...
  // code here to get the ds
  // ...
  i := 1;
  while not ds.Eof do
  begin
    // must pass in published method name here
    MyTest := TMyTest('DoTest');

    // append an integer (or something) to the test name
    // for clarity in referencing the test
    MyTest.FTestName := MyTest.FTestName + IntToStr(i);
    Inc(i);

    MyTest.InitialValue := ds.FieldByName('InitialValue').AsInteger;
    MyTest.Expected := ds.FieldByName('Expected').AsString;
    Result.AddTest(MyTest);

    // don't free MyTest, of course, because the Suite will
    // take care of that later.
  end;
end;

Add the appropriate code to the unit initialization:

initialization
  RegisterTest('AllMyTests', BuildSuite);

Now back to TMyTest.DoTest, to use its internal fields instead of method parameters:

procedure TMyTest.DoTest;
var
  MyObject: TMyObject;
begin
  MyObject := TMyObject.Create(FInitialValue);
  try
    CheckEquals(FExpected, MyObject.Method, 'initial value: <' + IntToStr(FInitialValue) + '>');
  finally
    MyObject.Free;
  end;
end;

And now you will have a separate test instance for every combination. All you have to do to add new tests now is add data to your db table (or Excel, whatever...), no code changes necessary.

tags: ComputersAndTechnology AgileDevelopment