Thursday, 5 April 2012

DUnit and TCustomAttributes

In this article I am playing with Unit tests and TCustomAttributes as I am working on different ideas to build up a lightweight testing framework.  Basically I liked the idea of TestCase attribute from NUnit and I wanted to do something similar using TCustomAttributes and accessing those attributes using the Delphi RTTI library. To understand better my purposes have a look at the following example extracted from NUnit webpage:

[TestCase(12,3,4)]
[TestCase(12,2,6)]
[TestCase(12,4,3)]
public void DivideTest(int n, int d, int q)
{
  Assert.AreEqual( q, n / d );
}

This simple example will execute three times the method or test using the parameters defined on the test case attributes. What if we could perform something similar? Would not be cool?

Here is what I have done so far:


Creation of the Custom Attributes:
type
  TUserPasswordAttribute = class(TCustomAttribute)
  private
    FPassword: string;
    FUserName: string;
    Fresponse: Boolean;
    procedure SetPassword(const Value: string);
    procedure SetUserName(const Value: string);
    procedure Setresponse(const Value: Boolean);
  public
    constructor Create(aUserName: string; aPassword: string; aResponse : Boolean);
    property UserName: string read FUserName write SetUserName;
    property Password: string read FPassword write SetPassword;
    property response : Boolean read Fresponse write Setresponse;
  end;

  TUserAgeAttribute = class(TCustomAttribute)
  private
    FAge: integer;
    FUserName: String;
    Fresponse: Boolean;
    procedure SetAge(const Value: integer);
    procedure SetUserName(const Value: String);
    procedure Setresponse(const Value: Boolean);
  public
    property UserName : String read FUserName write SetUserName;
    property Age : integer read FAge write SetAge;
    property response : Boolean read Fresponse write Setresponse;
    constructor Create(aUserName : string; aAge : Integer; aResponse : Boolean);
  end;

{ TUserPasswordAttribute }

constructor TUserPasswordAttribute.Create(aUserName, aPassword: string; aResponse : Boolean);
begin
  SetUserName(aUserName);
  SetPassword(aPassword);
  Setresponse(aResponse);
end;

procedure TUserPasswordAttribute.SetPassword(const Value: string);
begin
  FPassword := Value;
end;

procedure TUserPasswordAttribute.Setresponse(const Value: Boolean);
begin
  Fresponse := Value;
end;

procedure TUserPasswordAttribute.SetUserName(const Value: string);
begin
  FUserName := Value;
end;

{ TUserAgeAttribute }

constructor TUserAgeAttribute.Create(aUserName: string; aAge: Integer; aResponse : Boolean);
begin
  SetUserName(aUserName);
  SetAge(aAge);
  Setresponse(aResponse);
end;

procedure TUserAgeAttribute.SetAge(const Value: integer);
begin
  FAge := Value;
end;

procedure TUserAgeAttribute.Setresponse(const Value: Boolean);
begin
  Fresponse := Value;
end;

procedure TUserAgeAttribute.SetUserName(const Value: String);
begin
  FUserName := Value;
end;

Those two custom attributes will serve as an example for what I intend to do. I need to test a login and some data from a current user and those bespoke attributes would be used by the test case.

The Framework:
type
  TAttributeProc = reference to procedure(CustomAttr: TCustomAttribute);

  TFrameworkTestCase = class(TTestCase)
  public
    procedure TestAttributesMethod(CustomProc: TAttributeProc);
  end;

implementation

{ TFrameworkTestCase }

procedure TFrameworkTestCase.TestAttributesMethod(CustomProc: TAttributeProc);
var
  ContextRtti: TRttiContext;
  RttiType: TRttiType;
  RttiMethod: TRttiMethod;
  CustomAttr: TCustomAttribute;
begin
  ContextRtti := TRttiContext.Create;
  try
    RttiType := ContextRtti.GetType(Self.ClassType);
    for RttiMethod in RttiType.GetMethods do
      for CustomAttr in RttiMethod.GetAttributes do
        CustomProc(CustomAttr);
  finally
    ContextRtti.Free;
  end;
end;

As you can see TFrameworkTestCase inherits from TTestCase and it adds the magic of reading the custom attributes and invoking the delegate as many times needed.

Using the small framework:
unit TestUnit1;
{

  Delphi DUnit Test Case
  ----------------------
  This unit contains a skeleton test case class generated by the Test Case Wizard.
  Modify the generated code to correctly setup and call the methods from the unit
  being tested.

}

interface

uses
  TestFramework, Windows, Forms, Dialogs, Controls, Classes, RTTI, SysUtils, Variants,
  Graphics, Messages, Unit1, StdCtrls;

type
  TestTLoginCase = class(TFrameworkTestCase)
  strict private
    FLogin: TLogin;
  public
    procedure SetUp; override;
    procedure TearDown; override;
  published
    [TUserPasswordAttribute('User1', 'Password1', True)]
    [TUserPasswordAttribute('User2', 'Password2', True)]
    [TUserPasswordAttribute('User3', 'Password3', True)]
    [TUserPasswordAttribute('User3', '', False)]
    procedure TestUserLogin;
    [TUserAgeAttribute('User1', 26, True)]
    [TUserAgeAttribute('User2', 27, True)]
    [TUserAgeAttribute('User3', 28, False)]
    procedure TestUserAge;
  end;

implementation

procedure TestTLoginCase.SetUp;
begin
  FLogin := TLogin.Create;
end;

procedure TestTLoginCase.TearDown;
begin
  FLogin.Free;
  FLogin := nil;
end;

procedure TestTLoginCase.TestUserAge;
var
  aAge: integer;
  aUserName: string;
  aResponse : boolean;
begin
  TestAttributesMethod(procedure (CustomAttr : TCustomAttribute)
    begin
        if CustomAttr is TUserAgeAttribute then
        begin
          aUserName := TUserAgeAttribute(CustomAttr).UserName;
          aAge := TUserAgeAttribute(CustomAttr).Age;
          aResponse := TUserAgeAttribute(CustomAttr).Response;
          Assert(aResponse=(FLogin.fetchDatauser(aUserName)=aAge), 'Incorrect value ' + aUserName);
        end;
    end);
end;

procedure TestTLoginCase.TestUserLogin;
var
  aPassword: string;
  aUserName: string;
  aResponse : boolean;
begin
  TestAttributesMethod(procedure (CustomAttr : TCustomAttribute)
    begin
        if CustomAttr is TUserPasswordAttribute then
        begin
          aUserName := TUserPasswordAttribute(CustomAttr).UserName;
          aPassword := TUserPasswordAttribute(CustomAttr).Password;
          aResponse := TUserPasswordAttribute(CustomAttr).Response;
          Assert(FLogin.UserLogin(aUserName, aPassword)=aResponse, 'Incorrect user ' + aUserName);
        end;
    end);
end;

initialization
RegisterTest(TestTLoginCase.Suite);

end.

Notice that every test case contains a set of Custom attributes and they will be executed by using a delegate. I have included the Result parameter in the attribute so the test can know the result straight away and inform about it. 


Related links:

6 comments:

  1. The problem with your approach is that it requires putting way to much stuff inside the test method itself like getting out the passed values itself.

    Also it does not improve DUnit itself which is only able to execute unparameterized methods.

    I did this while ago when I saw a question for exactly that on SO (http://stackoverflow.com/questions/8999945/can-i-write-parameterized-tests-in-dunit)

    The only disadvantage of my solution is because attributes in delphi have very restricted kinds of parameters. Like you cannot have arrays as in .Net. So my solution uses a semicolon separated string to pass the arguments. But you can just extend the attribute class with more constructor overloads if you like.

    That way you can easily write test methods that take parameters and specify the parameters in the attributes as in .Net as you showed at the beginning. And actually it makes the test code itself also a one liner and not a bunch of setup code that you have to write every time.

    ReplyDelete
    Replies
    1. Thanks for your comment Stefan, it is always appreciated. What I can say?, I take my hat off to you. I have reviewed your solution for DSharp.Testing.DUnit and I think it is brilliant.
      I'm glad that at least we think alike as I wanted to achieve the same behaviour as C Sharp and you were able to mimic it better.

      I agree with you on the only drawback of using your solution as it relies on the user correctly inputting the semicolon separated values. As per my solution I rely on the TCustomAttribute so there is no misunderstanding as to what values are acceptable within the attribute.

      Jordi

      Delete
    2. Thanks, glad you like it ;)

      With my solution you can easily inherit your attribute classes from TestCaseAttribute and override the constructor. Fill FValues with the passed arguments and you are done. Everything else is handled by the framework.

      That reduces the actual implementation to 4 lines for each attribute (1 SetLength and 3 Assignments) and zero lines inside the test method itself which then just gets some parameters.

      P.S. Be careful with the types you use in the arguments. Turned out variants are kinda broken: http://qc.embarcadero.com/wc/qcmain.aspx?d=104778

      Delete
    3. Thanks Stefan!.

      I'm sure that in the end we can get something really interesting. I'll give a try to your solution and see how can I use it. I'm still thinking of building a lightweight test framework to run test cases using TCustomAttributes as from my point of view I think it is better using the attribute rather than coding a new test. I'm thinking about the login case; imagine we have the Login(user pass) method which it's a simple method but we want to test it against different users.
      So instead of writing a loop inside the test through all the users, I like the idea of leaving the responsibility to the attribute.
      Anyway, I'm still thinking on this as I'm trying to make my unit tests as more readable as possible.

      Jordi

      Delete
    4. I just commited an example showing what I meant. As you can see it has exactly the same functionality as your example but it requires much less writing.

      Delete
    5. Brilliant Stefan. I've just seen the example and this is what I was trying to achieve!.

      Thanks!.

      Delete