AspectJ tests

Testing and Tests

Testing on the AspectJ project is fairly straightforward. For compiler issues/bugs, we write harness tests. For API issues/bugs, we write JUnit tests. We also have other testware for system testing.

This module contains test data and scripts, while the testing-drivers and testing modules contain the harness. Tests differ in the kind of harness they use and their target. (Many tests are unused and date back to earlier times in the project.)

Test Description Driver Sources

Harness tests compile and run test sources. Harness in testing-drivers module ajcTests.xml, and ajcTests10.xml, referring to tests in new/, base/, errors/, design/, pureJava/, etc.

Product tests use a variety of Ant and Bash scripts to drive the product distributions. bash, Ant product/..

AjcTest is a large taskdef which compiles and runs large Java systems. AjcTest Ant Task in testing module other-systems/.., aspectj-external-lib.jar?/other-systems/

Jacks is the Jikes compiler test suite, run for us using a bash script. bash See the Jacks project page

JUnit tests normally are run directly before checking in, but there is an Ant script to run and report on them all. JUnit Every module has a testsrc/ directory with JUnit tests.

Test Description	Driver	Sources
Harness tests compile and run test sources.	Harness in testing-drivers module	ajcTests.xml, and ajcTests10.xml, referring to tests in new/, base/, errors/, design/, pureJava/, etc.
Product tests use a variety of Ant and Bash scripts to drive the product distributions.	bash, Ant	product/..
AjcTest is a large taskdef which compiles and runs large Java systems.	AjcTest Ant Task in testing module	other-systems/.., aspectj-external-lib.jar?/other-systems/
Jacks is the Jikes compiler test suite, run for us using a bash script.	bash	See the Jacks project page
JUnit tests normally are run directly before checking in, but there is an Ant script to run and report on them all.	JUnit	Every module has a testsrc/ directory with JUnit tests.

Harness

These are compiler test cases, the preferred form for any compiler tests. The XML files are test definitions, and the subdirectories contain the test sources. Many of the test source files are no longer used. The test definitions are segregated for convenience as follows:

ajcTests.xml: main test suite. After a release, these tests should always run and pass.
ajcTests10.xml: Tests for the 1.0 compiler. In some cases, error tests differ between the 1.0 and 1.1 compiler only in the errors expected to be generated.
ajcTestsBroken.xml: These tests fail, and the tests are broken. The tests should be fixed and rolled into ajcTests.xml
ajcTestsFailing.xml: These tests fail, and the compiler is broken. When fixed, these tests should be rolled into ajcTests.xml.
ajcTestsAttic.xml: These are broken or failing tests not slated to be fixed.
ajcHarnessTests.xml: These are tests to exercise the test harness itself.

To run the harness, use org.aspectj.testing.drivers.Harness, in the testing-drivers module. For help on harness options, use -help. For for help on component options, see the package docs. Note in particular how to select tests using keywords or supply compiler arguments on the command line.

To write a new test, evolve an existing example. See the dtd and/or the javadoc for the testing harness implementation in the testing module for more details.

To write a test case for a bug, see XXX todo.

Product

Product tests target a built distribution or do compiler (argument) testing not supported by the harness. Many have been superceded by ant tests.

AjcTest

ajctest.xml is an Ant script that compiles and runs large systems. The scripts tell the AjcTest task from the testing module to drive the sources in other-systems through compile/doc/run cycles. To set up the tests, the script combines the aspects checked in to other-systems/ with the Java source files from various open-source projects, contained in aspectj-external-lib.zip.

Jacks

Jacks is the Jikes compiler test suite available at the Jacks project page. We have a bash script to run it using ajc and javac and compare the results.