JUnit Test Suite Support

JT Harness Binary Distribution
Version 4.1.4

June 2008

 

Introduction
About JUnit
Prerequisites
Retrofit Process
Technical Details
Areas for Improvement
References

Top

Introduction

This document explains how to retrofit existing JUnit 3.x or 4.x test suites to run with the test harness. This information can also help you author new JUnit tests that will run under the harness.

Top

About JUnit and JT Harness

JUnit is a simple framework for writing and running automated tests. Written by Erich Gamma and Kent Beck in 1997, JUnit exposed test driven development practices from the Smalltalk world into the Java programming community. JUnit is now an open-source project on SourceForge.net.

The JUnit framework provides a fast, simple interface for creating a set of tests and executing them by a common method (ant, shell script, etcetera). The framework places very few restrictions on what the tester must do to write a test class. The core JUnit distribution has little or no facilities for GUI interaction or reporting, and it has no robust interface for configuration.

The procedure described here allows JUnit tests to be run under the JT Harness, which provides a robust GUI interface, many reporting options, and an opportunity to build a robust configuration system for the tests. The harness can be configured to allow customization of the GUI, report types, result presentation, and more. These services might be useful for users looking to wrap more powerful facilities around their existing test infrastructure.

Top

Prerequisites

To undertake a conversion process, you should be familiar with some of the inner workings of the JUnit test suite you are converting. Specifically, you need to know:

JUnit tests written to work with JUnit 3.x are typically identified as being a subclass of junit.framework.TestCase. To find JUnit 3.x tests, use the com.sun.javatest.junit.JUnitSuperTestFinder class (located in the jt-junit.jar archive) to scan the test classes. Each class that is a subclass of junit.framework.TestCase is designated as a recognized test.

JUnit 4.x style tests do not use a specific superclass, rather, they tag classes with the org.junit.Test annotation. The harness library jt-junit.jar provides the class com.sun.javatest.junit.JUnitAnnotationTestFinder to find 4.x style tests. It operates much like the JUnitSuperTestFinder class, but looks for different criteria.

See the sections on JUnitSuperTestFinder and JUnitAnnotationTestFinder for more details.

Top

Retrofit Process

This procedure describes how to set up files, property settings, and configuration settings before running a JUnit test.

  1. Create a testsuite.jtt file in root of the product directory. If the product unpacks into directory foo/, the testsuite.jtt file should probably be in that directory. It does not necessarily need to be co-located with the tests.

    The .jtt file is a properties formatted file, with key=value on each line. In your .jtt file, the settings for name and id are mandatory. The first is a short descriptive name for your test suite, the second is a internal key used identify this suite.

  2. Select your method for scanning for tests by specifying the finder. The finder line looks like this:

    finder = com.sun.javatest.junit.JUnitSuperTestFinder -superclass junit.framework.TestCase

    See JUnitSuperTestFinder and JUnitAnnotationTestFinder for further information.

  3. Select your TestSuite class, using com.sun.javatest.junit.JUnitTestSuite if you do not subclass it. Use a fully qualified class name. This class should be available on the system classpath or more preferably on the classpath defined in your .jtt file.

    testsuite = com.sun.javatest.junit.JUnitTestSuite

  4. Specify the interview. If you don't have your own interview, use the line below as the default . Again, this class should be available on the system classpath or more preferably on the classpath setting in your .jtt file.

    interview = com.sun.javatest.junit.JUnitBaseInterview

  5. Provide a tests setting. The tests location is important because it is forwarded to the TestFinder you specified in step 2, which determines what you are scanning. This location is often relative to the location of the testsuite.jtt file from step 1. The path should be platform independent (so forward slashes are recommended) . Do not use absolute paths or relative paths to places above testsuite.jtt. One of the following lines might be a good example.

    See JUnitSuperTestFinder and JUnitAnnotationTestFinder for further information.

  6. The classpath for the classes referenced in this file should generally be available on the classpath specified in this file. They do not need to be on the classpath. For example, it might be set to:

    classpath = lib/jt-junit.jar lib/jt-myts.jar

  7. Try running the harness to see if it finds your tests. Moving around Java™ archive (JAR) files and resolving paths will be left as an exercise for the reader. The general form is:

    > cd mytestsuite/
    > java -jar lib/javatest.jar -ts .


    This tells the harness to start and forces it to load the test suite located "here" (the current directory, represented with "."). The testsuite.jtt should be located in the . directory.

    What you should see. The GUI start,s beginning with a splash screen. When the main window comes up, you should see a tree populated with the tests you intended. Check the counters on the main right panel to get a summary of how many tests were found. You can check View|Properties to verify that it loaded the plug-in classes as expected.
Top

Technical Details

This section describes the two main sets of classes that provide JUnit support. First is the JUnitTestFinder (subclass of com.sun.javatest.TestFinder). The variations of JUnitTestFinder, JUnitSuperTestFinder and JUnitAnnotationTestFinder, roughly correspond to JUnit 3.x and 4.x support. The difference is explained below.

The second supporting component is the JUnitMultiTest class that is responsible for executing the tests.

Support Classes

Additional "glue" classes are provided to connect everything together: JUnitTestSuite, JUnitBaseInterview, and JUnitTestRunner. Each supporting class is explained below.

JUnitSuperTestFinder

This class looks for a superclass that identifies the class as a JUnit test. By default it searches the ancestors of each class for junit.framework.TestCase. A test suite might require further derivations of junit.framework.TestCase to support its particular needs , so you can use the -superclass option to specify a more specific class.

For example, consider the following class structure:

java.lang.Object
     junit.framework.TestCase
          foo.BetterTestCase
               product.Test0002a

Test0002a is a subclass of BetterTestCase, and so forth.

To change the superclass you are scanning for, supply the -superclass argument and specify a class name. You can supply this argument multiple times to scan for multiple superclasses. For example, in testsuite.jtt you might specify:

finder = com.sun.javatest.junit.JUnitSuperTestFinder -superclass foo.BetterTestCase -superclass foo.CustomTestCase

Although it does not execute tests, the finder attempts to pick out test methods by looking for public methods that begin with the string "test". It then lists these in a space-separated list, without the parameters (just the method name). The list might contain duplicates because the full signature is not evaluated. Semantics for this TestDescription value are loosely defined at this point. Public comment is welcome (submit your comments to the JT Harness interest form).

This superclass finder generates the following TestDescription (com.sun.javatest.TestDescription) values:

Key Value(s)
keywords junit, junit3
junit.finderscantype superclass
junit.testmethods (list of identified test methods)

JUnitAnnotationTestFinder

This annotation finder scans classes for the org.junit.Test annotation. It uses the same scanning strategy as JUnitSuperTestFinder.

This annotation finder generates the following TestDescription (com.sun.javatest.TestDescription) values:

Key Value(s)
keywords junit, junit4
junit.finderscantype annotation
junit.testmethods (list of identified test methods)

JUnitBareMultiTest

This is the execution class for JUnit 3.x style tests. Execution is accomplished using the class name supplied by the finder (through the TestDescription), which is used to execute that class's TestCase.runBare() method. This might not be sufficient for all test suites. Output sfrom stdout and stderr are captured. The result is pass if no exceptions are thrown, and fail if there are any Throwable results.

JUnitAnnotationMultiTest

This is the execution class that runs tests written in the JUnit 4.x style. It takes the class that was identified by the test finder and executes it using the JUnit library TestResult.Section pieces. Also, because execution is turned over to JUnit, it does not report any of its own debugging output during execution. (For the future, it would be useful to take more advantage of the Result API and any observer APIs that are available.)

Top

Implementation Notes

The use of the junit3 and junit4 keywords might be a generalization, since it really represents how the class was found. A test suite might mix use of version 3 and 4 features, so it does not necessarily mean that the test is completely 4.x compliant. Nonetheless, perhaps being able to mix 3.x style tests out of a mixed set (see com.sun.javatest.finder.ChameleonTestFinder) can be useful. Do not forget that the junit keyword is also added, so JUnit tests can be selected from among non-JUnit tests.

Two of the most likely changes you will make are modifying the test finder or modifying how to execute the test. To change the test finder, simply subclass JUnitTestFinder, provide it on the classpath in testsuite.jtt and change the finder setting in testsuite.jtt .

To change the method for executing a test, you would need to change how it is dispatched in JUnitTestRunner. To change that, you should subclass JUnitTestRunner, provide it on the testsuite.jtt classpath. You will also need to subclass JUnitTestSuite and change that setting in the testsuite.jtt (see Areas for Improvement).

Top

Areas for Improvement

This section lists implementation features that might benefit from user feedback and further development.

Top

References


Copyright © 2008 Sun Microsystems, Inc. All rights reserved.