Unit-Tests & Delphi Prism: Code Coverage

Overview

According to Wikipedia, code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that inspects the code directly and is therefore a form of white box testing.

Sometimes it is needed to measure code coverage for given test suite to see the untested code paths, as they may contain undiscovered bugs. This article shows how to measure code coverage and investigate code paths, regardless of .NET language used. Approach described is applicable to Oxygene, as well as for C# and VB.NET.

Instruments

Libraries used are:- xUnit as test framework ( [http://xunit.codeplex.com| http://xunit.codeplex.com] )

PartCover (an Open Source code coverage tool. See more details at https://github.com/sawilde/partcover.net4/wiki ). For x64 systems also see blogpost http://www.planetgeek.ch/2009/10/15/get-partcover-running-on-x64-windows to get PartCover running.

Note that there is much software with similar functionality, either commercial or free.

Sample Project

Let’s create a simple ClassLibrary and cover it with tests.

Create new ClassLibrary project.

Add a class that is intended to convert Int32, Double and GUID values to String and back. Sample results are ’42’ for an Int32 value, ‘2.7182’ for a Double value and {AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56} for a GUID value.

Of course, the following code is primitive, inefficient and should never be used in a real-life project, but it provides needed functionality and is good enough for our needs.

namespace ConverterLibrary; interface uses System.Globalization; type Converter =publicclasspublicmethod ConvertToString(value: Object): String;method ConvertFromString(expectedType: &;Type; value: String): Object;end; implementation method Converter.ConvertToString(value: Object): String;begincasetypeOf(value)oftypeOf(Int32):exit(Int32(value).ToString(CultureInfo.InvariantCulture));typeOf(Double):exit(Double(value).ToString(CultureInfo.InvariantCulture));typeOf(Guid):exit(Guid(value).ToString().ToUpperInvariant());end;// case exit(value.ToString());end; method Converter.ConvertFromString(expectedType: &;Type; value: String): Object;begincase expectedType oftypeOf(Int32):exit(Int32.Parse(value));typeOf(Double):exit(Double.Parse(value));typeOf(Guid):exit(new Guid(value));end;// case exit(nil);end; end.

Now let’s create tests for this library. Add a new ClassLibrary project, reference `xunit.dll` assembly and add the following code:

namespace TestLibrary; interface uses Xunit, ConverterLibrary; type ConverterTests =publicclassprivatevar fTestConverter: Converter; publicconstructor; [Fact]method ConvertInt32ToStringTest(); [Fact]method ConvertDoubleToStringTest(); [Fact]method ConvertStringToInt32Test(); [Fact]method ConvertStringToDoubleTest(); [Fact]method ConvertStringToGuidTest(); [Fact]method ConvertStringToUndefinedType();end; implementation constructor ConverterTests;beginself.fTestConverter:=new Converter();end; method ConverterTests.ConvertInt32ToStringTest();beginAssert.Equal('42',self.fTestConverter.ConvertToString(42));end; method ConverterTests.ConvertDoubleToStringTest();beginAssert.Equal('2.7182',self.fTestConverter.ConvertToString(2.7182));end; method ConverterTests.ConvertStringToInt32Test();beginAssert.Equal(42,self.fTestConverter.ConvertFromString(typeOf(Int32),'42')as Int32);end; method ConverterTests.ConvertStringToDoubleTest();beginAssert.Equal(2.7182,self.fTestConverter.ConvertFromString(typeOf(Double),'2.7182')as Double);end; method ConverterTests.ConvertStringToGuidTest();beginAssert.Equal(new Guid('{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}'),self.fTestConverter.ConvertFromString(typeOf(Guid),'{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}')as Guid);end; method ConverterTests.ConvertStringToUndefinedType();beginAssert.Null(self.fTestConverter.ConvertFromString(typeOf(Decimal),'3.1415'));end; end.

All tests will pass:

All tests pass but there is a serious bug in the library. It is easy to notice this bug here, but imagine a big solution with multiple assemblies and thousands of code lines, talking to a remote server. There finding such a bug would cost at least several human-hours.

Now let’s check how the sample library’s code is covered with the tests:

Start PartCover Browser and open File -> Run Target… dialog.
Set proper values and start the tests

As you see you have to specify test runner .exe, name of the assembly containing tests and other parameters for the test runner and list of entities whose code coverage will be logged. For more details please refer to the PartCover manual.

The ‘/noshadow‘ parameter of the test runner parameter is needed in case when you want to see not only overall coverage percent but also the source sode that was left uncovered.

Note: PartCover will not work out-of-the-box on x64 systems. Please follow steps described in this article to get it running.

After test run completes issue the **Views **-> View Coverage Details command.
Now you can investigate code that is still not covered by tests.

As you can see some code in the ConvertToString method is not covered by tests. Let’s add the missed tests to the TestLibrary:

namespace TestLibrary; ... ConverterTests=publicclass...[Fact]method ConvertGuidToStringTest(); [Fact]method ConvertUndefinedToStringTest();end;... implementation ... method ConverterTests.ConvertGuidToStringTest();beginAssert.Equal('{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}',self.fTestConverter.ConvertToString(new Guid('{AE3DE671-AF08-469D-AFF4-7EB2E0AE1F56}')));end; method ConverterTests.ConvertUndefinedToStringTest();beginAssert.Equal('3.1415',self.fTestConverter.ConvertToString(Decimal(3.1415)));end; end.

Obviously test `ConvertGuidToStringTest` will fail.

There is a bug in method Converter.ConvertToString that sould be fixed:

method Converter.ConvertToString(value: Object): String;begincasetypeOf(value)oftypeOf(Int32):exit(Int32(value).ToString(CultureInfo.InvariantCulture));typeOf(Double):exit(Double(value).ToString(CultureInfo.InvariantCulture));typeOf(Guid):exit(Guid(value).ToString('B').ToUpperInvariant());end;// case exit(value.ToString());end;

After this fix all tests will pass and most of the library’s code will be covered with tests.

Summary

This post shows how to use open-source tools with Oxygene to check quality of unit-tests, improve test code coverage and find and fix hidden bugs not detected by previously existing tests.