|
JUnit Test Suite Support
JT Harness Binary Distribution Version 4.1.4
June 2008
|
|
[Skip TOC]
- Introduction
- About JUnit
- Prerequisites
- Retrofit Process
- Technical Details
- Areas for Improvement
- References
Top
Introduction
This document explains how to retrofit existing JUnit 3.x or 4.x test suites to run with the test harness. This information can also help you author new JUnit tests that will run under the harness.
Top
About JUnit and JT Harness
JUnit is a simple framework for writing and running automated tests. Written by Erich Gamma and Kent Beck in 1997, JUnit exposed test driven development practices from the Smalltalk world into the Java programming community. JUnit is now an open-source project on SourceForge.net.
The JUnit framework provides a fast, simple interface for creating a set of tests and executing them by a common method (ant, shell script, etcetera). The framework places very few restrictions on what the tester must do to write a test class. The core JUnit distribution has little or no facilities for GUI interaction or reporting, and it has no robust interface for configuration.
The procedure described here allows JUnit tests to be run under the JT Harness, which provides a robust GUI interface, many reporting options, and an opportunity to build a robust configuration system for the tests. The harness can be configured to allow customization of the GUI, report types, result presentation, and more. These services might be useful for users looking to wrap more powerful facilities around their existing test infrastructure.
Top
Prerequisites
To undertake a conversion process, you should be familiar with some of the inner workings of the JUnit test suite you are converting. Specifically, you need to know:
- How can JUnit tests be distinguished from other tests?
- Which version of JUnit works with the test suite? JUnit 3.x or 4.x?
- Where are the tests stored? Are they in a single directory tree?
- What libraries or supporting processes are required to run the tests?
- What configuration settings or files are necessary to run the tests?
JUnit tests written to work with JUnit 3.x are typically identified as being a subclass of junit.framework.TestCase. To find JUnit 3.x tests, use the com.sun.javatest.junit.JUnitSuperTestFinder class (located in the jt-junit.jar archive) to scan the test classes. Each class that is a subclass of junit.framework.TestCase is designated as a recognized test.
JUnit 4.x style tests do not use a specific superclass, rather, they tag classes with the org.junit.Test annotation. The harness library jt-junit.jar provides the class com.sun.javatest.junit.JUnitAnnotationTestFinder to find 4.x style tests. It operates much like the JUnitSuperTestFinder class, but looks for different criteria.
See the sections on JUnitSuperTestFinder and JUnitAnnotationTestFinder for more details.
Top
Retrofit Process
This procedure describes how to set up files, property settings, and configuration settings before running a JUnit test.
- Create a testsuite.jtt file in root of the product directory. If the product unpacks into directory foo/, the testsuite.jtt file should probably be in that directory. It does not necessarily need to be co-located with the tests.
The .jtt file is a properties formatted file, with key=value on each line. In your .jtt file, the settings for name and id are mandatory. The first is a short descriptive name for your test suite, the second is a internal key used identify this suite.
- Select your method for scanning for tests by specifying the finder. The finder line looks like this:
finder = com.sun.javatest.junit.JUnitSuperTestFinder -superclass junit.framework.TestCase
See JUnitSuperTestFinder and JUnitAnnotationTestFinder for further information.
- Select your TestSuite class, using com.sun.javatest.junit.JUnitTestSuite if you do not subclass it. Use a fully qualified class name. This class should be available on the system classpath or more preferably on the classpath defined in your .jtt file.
testsuite = com.sun.javatest.junit.JUnitTestSuite
- Specify the interview. If you don't have your own interview, use the line below as the default . Again, this class should be available on the system classpath or more preferably on the classpath setting in your .jtt file.
interview = com.sun.javatest.junit.JUnitBaseInterview
- Provide a tests setting. The tests location is important because it is forwarded to the TestFinder you specified in step 2, which determines what you are scanning. This location is often relative to the location of the testsuite.jtt file from step 1. The path should be platform independent (so forward slashes are recommended) . Do not use absolute paths or relative paths to places above testsuite.jtt. One of the following lines might be a good example.
- If you are scanning .java files, they might be located below the tests/ directory.
tests = tests
- If you are scanning class files:
tests = classes
See JUnitSuperTestFinder and JUnitAnnotationTestFinder for further information.
- The classpath for the classes referenced in this file should generally be available on the classpath specified in this file. They do not need to be on the classpath. For example, it might be set to:
classpath = lib/jt-junit.jar lib/jt-myts.jar
- Try running the harness to see if it finds your tests. Moving around Java archive (JAR) files and resolving paths will be left as an exercise for the reader. The general form is:
> cd mytestsuite/
> java -jar lib/javatest.jar -ts .
This tells the harness to start and forces it to load the test suite located "here" (the current directory, represented with "."). The testsuite.jtt should be located in the . directory.
What you should see. The GUI start,s beginning with a splash screen. When the main window comes up, you should see a tree populated with the tests you intended. Check the counters on the main right panel to get a summary of how many tests were found. You can check View|Properties to verify that it loaded the plug-in classes as expected.
Top
Technical Details
This section describes the two main sets of classes that provide JUnit support. First is the JUnitTestFinder (subclass of com.sun.javatest.TestFinder). The variations of JUnitTestFinder, JUnitSuperTestFinder and JUnitAnnotationTestFinder, roughly correspond to JUnit 3.x and 4.x support. The difference is explained below.
The second supporting component is the JUnitMultiTest class that is responsible for executing the tests.
Support Classes
Additional "glue" classes are provided to connect everything together: JUnitTestSuite, JUnitBaseInterview, and JUnitTestRunner. Each supporting class is explained below.
- JUnitTestSuite is a very simple class that instructs the harness to use the JUnitTestRunner to execute tests. If this method was not present, the DefaultTestRunner would be used, which is the traditional way to execute tests, requiring a Script class. Because we have supplied our own TestRunner, we have full control over how the tests are executed. That is, - how many are run simultaneously, how they are launched (exec, etcetera). By extending this class, you will have access to override other aspects of the harness. See the TestRunner API for more information. Note that many of the settings which this document places in the testsuite.jtt file could be hardcoded into the TestSuite subclass. The TestSuite base class provides the functionality to turn the settings places in the testsuite.jtt file into reality.
- JUnitBaseInterview this is a skeleton interview class which does not require any input from the user. If your JUnit tests do not need any setting from the user, you can leave this the way it is. To get values from the user try one of these methods:
- Read a configuration file from a pre-determined location, perhaps a location relative to the test suite root (TestSuite.getRootDir()).
- Ask the user for the values directly, using the com.sun.interview API - this is the primary way in which the harness is designed to get vales from the user. In either case, the value(s) must end up in the Map provided in Interview.export(Map); this is the set of values that the other classes here will have access to, namely the JUnitTestRunner and classes it creates (JUnitMultiTest). Read the JavaTest Architect's Guide for more information.
- JUnitTestRunner is responsible for dispatching tests. It has access to the entire list of tests that will be run during a test run, via an Iterator. A test is represented by a TestDescription at this point, which is why you should customize your test finder to add any settings which you will want later (in this class). The default implementation will execute the test using JUnitBareMultiTest if the TestDescription property "junit.finderscantype" is set to superclass. Otherwise it will use the JUnitAnnotationMultiTest. It is somewhat likely that you will want to change this behavior, perhaps to use your own JUnitMultiTest class, or a subclass of one of these. You can do whatever you want at this point because you know the test class to execute.
JUnitSuperTestFinder
This class looks for a superclass that identifies the class as a JUnit test. By default it searches the ancestors of each class for junit.framework.TestCase. A test suite might require further derivations of junit.framework.TestCase to support its particular needs , so you can use the -superclass option to specify a more specific class.
For example, consider the following class structure:
java.lang.Object
junit.framework.TestCase
foo.BetterTestCase
product.Test0002a
Test0002a is a subclass of BetterTestCase, and so forth.
- If given Test0002a, JUnitSuperFinder ascends the inheritance chain until it reaches either a matching superclass or java.lang.Object. In this case it looks for the TestCase class by default, so when given Test0002a, it ascends two levels, finds java.lang.Object, and returns Test0002a to the harness as a test.
- If this finder is given java.util.ArrayList, it ascends until it reaches java.lang.Object, at which point it decides that the class is not a test and moves on.
To change the superclass you are scanning for, supply the -superclass argument and specify a class name. You can supply this argument multiple times to scan for multiple superclasses. For example, in testsuite.jtt you might specify:
finder = com.sun.javatest.junit.JUnitSuperTestFinder -superclass foo.BetterTestCase -superclass foo.CustomTestCase
Although it does not execute tests, the finder attempts to pick out test methods by looking for public methods that begin with the string "test". It then lists these in a space-separated list, without the parameters (just the method name). The list might contain duplicates because the full signature is not evaluated. Semantics for this TestDescription value are loosely defined at this point. Public comment is welcome (submit your comments to the JT Harness interest form).
This superclass finder generates the following TestDescription (com.sun.javatest.TestDescription) values:
Key |
Value(s) |
keywords |
junit, junit3 |
junit.finderscantype |
superclass |
junit.testmethods |
(list of identified test methods) |
JUnitAnnotationTestFinder
This annotation finder scans classes for the org.junit.Test annotation. It uses the same scanning strategy as JUnitSuperTestFinder.
This annotation finder generates the following TestDescription (com.sun.javatest.TestDescription) values:
Key |
Value(s) |
keywords |
junit, junit4 |
junit.finderscantype |
annotation |
junit.testmethods |
(list of identified test methods) |
JUnitBareMultiTest
This is the execution class for JUnit 3.x style tests. Execution is accomplished using the class name supplied by the finder (through the TestDescription), which is used to execute that class's TestCase.runBare() method. This might not be sufficient for all test suites. Output sfrom stdout and stderr are captured. The result is pass if no exceptions are thrown, and fail if there are any Throwable results.
JUnitAnnotationMultiTest
This is the execution class that runs tests written in the JUnit 4.x style. It takes the class that was identified by the test finder and executes it using the JUnit library TestResult.Section pieces. Also, because execution is turned over to JUnit, it does not report any of its own debugging output during execution. (For the future, it would be useful to take more advantage of the Result API and any observer APIs that are available.)
Top
Implementation Notes
The use of the junit3 and junit4 keywords might be a generalization, since it really represents how the class was found. A test suite might mix use of version 3 and 4 features, so it does not necessarily mean that the test is completely 4.x compliant. Nonetheless, perhaps being able to mix 3.x style tests out of a mixed set (see com.sun.javatest.finder.ChameleonTestFinder) can be useful. Do not forget that the junit keyword is also added, so JUnit tests can be selected from among non-JUnit tests.
Two of the most likely changes you will make are modifying the test finder or modifying how to execute the test. To change the test finder, simply subclass JUnitTestFinder, provide it on the classpath in testsuite.jtt and change the finder setting in testsuite.jtt .
To change the method for executing a test, you would need to change how it is dispatched in JUnitTestRunner. To change that, you should subclass JUnitTestRunner, provide it on the testsuite.jtt classpath. You will also need to subclass JUnitTestSuite and change that setting in the testsuite.jtt (see Areas for Improvement).
Top
Areas for Improvement
This section lists implementation features that might benefit from user feedback and further development.
- The classpath is currently not convenient. The general design of the harness is that the setting in testsuite.jtt affects the tests, rather than the system classpath which the harness is using. This area could use refinement in the current implementation.
- Some additional base implementations of the interview class would be useful. In particular, providing one that reads a properties file and dumps it directly into the Map of Interview.export(Map) would provide a very "quick and dirty" way for people to configure their test suites. Perhaps the location of the file is written as a setting in testsuite.jtt.
Note to architects: Users should not be instructed to alter testsuite.jtt as a regular activity. The design intent is that the settings there are static, and information the user provides should enter through the Interview system. As an architect, you should be configuring the testsuite.jtt for general use during the retrofit process. Once the conversion is completed, the file should remain relatively untouched.
- It might be useful to hardcode the interview class and accept an override in the testsuite.jtt file, rather than forcing the developer to specify it in the file as documented above. Same for the JUnitTestRunner (or just the TestRunner class) in the implementation of JUnitTestSuite.
Top
References
[Top]
Copyright © 2008 Sun Microsystems, Inc. All rights reserved.