Unit-Tests & Delphi Prism: Code Coverage


According to Wikipedia, code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that inspects the code directly and is therefore a form of white box testing.

Sometimes it is needed to measure code coverage for given test suite to see the untested code paths, as they may contain undiscovered bugs. This article shows how to measure code coverage and investigate code paths, regardless of .NET language used. Approach described is applicable to Oxygene, as well as for C# and VB.NET.


Libraries used are:- xUnit as test framework ( [http://xunit.codeplex.com| http://xunit.codeplex.com] )
- PartCover (an Open Source code coverage tool. See more details at https://github.com/sawilde/partcover.net4/wiki ). For x64 systems also see blogpost http://www.planetgeek.ch/2009/10/15/get-partcover-running-on-x64-windows to get PartCover running.

Note that there is much software with similar functionality, either commercial or free.

Sample Project

Let’s create a simple ClassLibrary and cover it with tests.

Create new ClassLibrary project.

Add a class that is intended to convert Int32, Double and GUID values to String and back. Sample results are ’42’ for an Int32 value, ‘2.7182’ for a Double value and {AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56} for a GUID value.

Of course, the following code is primitive, inefficient and should never be used in a real-life project, but it provides needed functionality and is good enough for our needs.

namespace ConverterLibrary;   interface   uses System.Globalization;   type Converter =publicclasspublicmethod ConvertToString(value: Object): String;method ConvertFromString(expectedType: &Type; value: String): Object;end;   implementation   method Converter.ConvertToString(value: Object): String;begincasetypeOf(value)oftypeOf(Int32):exit(Int32(value).ToString(CultureInfo.InvariantCulture));typeOf(Double):exit(Double(value).ToString(CultureInfo.InvariantCulture));typeOf(Guid):exit(Guid(value).ToString().ToUpperInvariant());end;// case   exit(value.ToString());end;   method Converter.ConvertFromString(expectedType: &Type; value: String): Object;begincase expectedType oftypeOf(Int32):exit(Int32.Parse(value));typeOf(Double):exit(Double.Parse(value));typeOf(Guid):exit(new Guid(value));end;// case   exit(nil);end;   end.

Now let’s create tests for this library. Add a new ClassLibrary project, reference xunit.dll assembly and add the following code:

namespace TestLibrary;   interface   uses Xunit, ConverterLibrary;   type ConverterTests =publicclassprivatevar fTestConverter: Converter;   publicconstructor;   [Fact]method ConvertInt32ToStringTest();   [Fact]method ConvertDoubleToStringTest();   [Fact]method ConvertStringToInt32Test();   [Fact]method ConvertStringToDoubleTest();   [Fact]method ConvertStringToGuidTest();   [Fact]method ConvertStringToUndefinedType();end;   implementation   constructor ConverterTests;beginself.fTestConverter:=new Converter();end;   method ConverterTests.ConvertInt32ToStringTest();beginAssert.Equal('42',self.fTestConverter.ConvertToString(42));end;   method ConverterTests.ConvertDoubleToStringTest();beginAssert.Equal('2.7182',self.fTestConverter.ConvertToString(2.7182));end;   method ConverterTests.ConvertStringToInt32Test();beginAssert.Equal(42,self.fTestConverter.ConvertFromString(typeOf(Int32),'42')as Int32);end;   method ConverterTests.ConvertStringToDoubleTest();beginAssert.Equal(2.7182,self.fTestConverter.ConvertFromString(typeOf(Double),'2.7182')as Double);end;   method ConverterTests.ConvertStringToGuidTest();beginAssert.Equal(new Guid('{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}'),self.fTestConverter.ConvertFromString(typeOf(Guid),'{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}')as Guid);end;   method ConverterTests.ConvertStringToUndefinedType();beginAssert.Null(self.fTestConverter.ConvertFromString(typeOf(Decimal),'3.1415'));end;   end.

All tests will pass:

Test results with partial code coverage

All tests pass but there is a serious bug in the library. It is easy to notice this bug here, but imagine a big solution with multiple assemblies and thousands of code lines, talking to a remote server. There finding such a bug would cost at least several human-hours.

Now let’s check how the sample library’s code is covered with the tests:

  1. Start PartCover Browser and open File -> Run Target… dialog.

  2. Set proper values and start the tests

Target settings

As you see you have to specify test runner .exe, name of the assembly containing tests and other parameters for the test runner and list of entities whose code coverage will be logged. For more details please refer to the PartCover manual.

The ‘/noshadow‘ parameter of the test runner parameter is needed in case when you want to see not only overall coverage percent but also the source sode that was left uncovered.

Note: PartCover will not work out-of-the-box on x64 systems. Please follow steps described in this article to get it running.

  1. After test run completes issue the Views *-> *View Coverage Details command.

  2. Now you can investigate code that is still not covered by tests.

PartCover Coverage Browser with non-covered code

  1. As you can see some code in the ConvertToString method is not covered by tests. Let’s add the missed tests to the TestLibrary:

namespace TestLibrary;   ...   ConverterTests=publicclass...[Fact]method ConvertGuidToStringTest();   [Fact]method ConvertUndefinedToStringTest();end;...   implementation   ...   method ConverterTests.ConvertGuidToStringTest();beginAssert.Equal('{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}',self.fTestConverter.ConvertToString(new Guid('{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}')));end;   method ConverterTests.ConvertUndefinedToStringTest();beginAssert.Equal('3.1415',self.fTestConverter.ConvertToString(Decimal(3.1415)));end;   end.

Obviously test ConvertGuidToStringTest will fail.

There is a bug in method Converter.ConvertToString that sould be fixed:

method Converter.ConvertToString(value: Object): String;begincasetypeOf(value)oftypeOf(Int32):exit(Int32(value).ToString(CultureInfo.InvariantCulture));typeOf(Double):exit(Double(value).ToString(CultureInfo.InvariantCulture));typeOf(Guid):exit(Guid(value).ToString('B').ToUpperInvariant());end;// case   exit(value.ToString());end;

After this fix all tests will pass and most of the library’s code will be covered with tests.

PartCover Coverage Browser with mostly-covered code


This post shows how to use open-source tools with Oxygene to check quality of unit-tests, improve test code coverage and find and fix hidden bugs not detected by previously existing tests.