Unit-Tests & Delphi Prism: Code Coverage

Anton Kasyanov -

Overview

According to Wikipedia, code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that inspects the code directly and is therefore a form of white box testing.

Sometimes it is needed to measure code coverage for given test suite to see the untested code paths, as they may contain undiscovered bugs. This article shows how to measure code coverage and investigate code paths, regardless of .NET language used. Approach described is applicable to Oxygene, as well as for C# and VB.NET.

Instruments

Libraries used are:- xUnit as test framework ( [http://xunit.codeplex.com| http://xunit.codeplex.com] )

Note that there is much software with similar functionality, either commercial or free.

Sample Project

Let’s create a simple ClassLibrary and cover it with tests.

Create new ClassLibrary project.

Add a class that is intended to convert Int32, Double and GUID values to String and back. Sample results are ’42’ for an Int32 value, ‘2.7182’ for a Double value and {AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56} for a GUID value.

Of course, the following code is primitive, inefficient and should never be used in a real-life project, but it provides needed functionality and is good enough for our needs.

namespace ConverterLibrary;   interface   uses System.Globalization;   type Converter =publicclasspublicmethod ConvertToString(value: Object): String;method ConvertFromString(expectedType: &;Type; value: String): Object;end;   implementation   method Converter.ConvertToString(value: Object): String;begincasetypeOf(value)oftypeOf(Int32):exit(Int32(value).ToString(CultureInfo.InvariantCulture));typeOf(Double):exit(Double(value).ToString(CultureInfo.InvariantCulture));typeOf(Guid):exit(Guid(value).ToString().ToUpperInvariant());end;// case   exit(value.ToString());end;   method Converter.ConvertFromString(expectedType: &;Type; value: String): Object;begincase expectedType oftypeOf(Int32):exit(Int32.Parse(value));typeOf(Double):exit(Double.Parse(value));typeOf(Guid):exit(new Guid(value));end;// case   exit(nil);end;   end.
Now let’s create tests for this library. Add a new ClassLibrary project, reference `xunit.dll` assembly and add the following code:
namespace TestLibrary;   interface   uses Xunit, ConverterLibrary;   type ConverterTests =publicclassprivatevar fTestConverter: Converter;   publicconstructor;   [Fact]method ConvertInt32ToStringTest();   [Fact]method ConvertDoubleToStringTest();   [Fact]method ConvertStringToInt32Test();   [Fact]method ConvertStringToDoubleTest();   [Fact]method ConvertStringToGuidTest();   [Fact]method ConvertStringToUndefinedType();end;   implementation   constructor ConverterTests;beginself.fTestConverter:=new Converter();end;   method ConverterTests.ConvertInt32ToStringTest();beginAssert.Equal('42',self.fTestConverter.ConvertToString(42));end;   method ConverterTests.ConvertDoubleToStringTest();beginAssert.Equal('2.7182',self.fTestConverter.ConvertToString(2.7182));end;   method ConverterTests.ConvertStringToInt32Test();beginAssert.Equal(42,self.fTestConverter.ConvertFromString(typeOf(Int32),'42')as Int32);end;   method ConverterTests.ConvertStringToDoubleTest();beginAssert.Equal(2.7182,self.fTestConverter.ConvertFromString(typeOf(Double),'2.7182')as Double);end;   method ConverterTests.ConvertStringToGuidTest();beginAssert.Equal(new Guid('{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}'),self.fTestConverter.ConvertFromString(typeOf(Guid),'{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}')as Guid);end;   method ConverterTests.ConvertStringToUndefinedType();beginAssert.Null(self.fTestConverter.ConvertFromString(typeOf(Decimal),'3.1415'));end;   end.
All tests will pass:

All tests pass but there is a serious bug in the library. It is easy to notice this bug here, but imagine a big solution with multiple assemblies and thousands of code lines, talking to a remote server. There finding such a bug would cost at least several human-hours.

Now let’s check how the sample library’s code is covered with the tests:

  1. Start PartCover Browser and open File -> Run Target… dialog.

  2. Set proper values and start the tests

As you see you have to specify test runner .exe, name of the assembly containing tests and other parameters for the test runner and list of entities whose code coverage will be logged. For more details please refer to the PartCover manual.

The ‘/noshadow‘ parameter of the test runner parameter is needed in case when you want to see not only overall coverage percent but also the source sode that was left uncovered.

Note: PartCover will not work out-of-the-box on x64 systems. Please follow steps described in this article to get it running.

  1. After test run completes issue the **Views **-> View Coverage Details command.

  2. Now you can investigate code that is still not covered by tests.

  1. As you can see some code in the ConvertToString method is not covered by tests. Let’s add the missed tests to the TestLibrary:
namespace TestLibrary;   ...   ConverterTests=publicclass...[Fact]method ConvertGuidToStringTest();   [Fact]method ConvertUndefinedToStringTest();end;...   implementation   ...   method ConverterTests.ConvertGuidToStringTest();beginAssert.Equal('{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}',self.fTestConverter.ConvertToString(new Guid('{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}')));end;   method ConverterTests.ConvertUndefinedToStringTest();beginAssert.Equal('3.1415',self.fTestConverter.ConvertToString(Decimal(3.1415)));end;   end.
Obviously test `ConvertGuidToStringTest` will fail.

There is a bug in method Converter.ConvertToString that sould be fixed:

method Converter.ConvertToString(value: Object): String;begincasetypeOf(value)oftypeOf(Int32):exit(Int32(value).ToString(CultureInfo.InvariantCulture));typeOf(Double):exit(Double(value).ToString(CultureInfo.InvariantCulture));typeOf(Guid):exit(Guid(value).ToString('B').ToUpperInvariant());end;// case   exit(value.ToString());end;
After this fix all tests will pass and most of the library’s code will be covered with tests.

Summary

This post shows how to use open-source tools with Oxygene to check quality of unit-tests, improve test code coverage and find and fix hidden bugs not detected by previously existing tests.