123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190 |
- <html>
- <title>AspectJ tests</title>
- <body>
- <h1>AspectJ tests</h1>
-
- <h3>Testing and Tests</h3>
- Testing on the AspectJ project is fairly straightforward.
- For compiler issues/bugs, we write harness tests.
- For API issues/bugs, we write JUnit tests.
- We also have other testware for system testing.
- <p>
- This module contains test data and scripts, while the testing-drivers
- and testing modules contain the harness.
-
- Tests differ in the kind of harness they use and their target.
- (Many tests are unused and date back to earlier times in the project.)
-
- <p>
- <table border="1" cellpadding="1">
- <tr><th align="left"><u>Test</u> Description</th>
- <th align="left">Driver</th>
- <th align="left">Sources</th>
- </tr>
-
- <tr><td><a href="#harness">Harness</a> tests
- compile and run test sources.
- </td>
- <td>Harness in testing-drivers module</td>
- <td><a href="ajcTests.xml">ajcTests.xml</a>,
- and <a href="ajcTests.xml">ajcTests10.xml</a>,
- referring to tests in
- <a href="new">new/</a>,
- <a href="base">base/</a>,
- <a href="errors">errors/</a>,
- <a href="design">design/</a>,
- <a href="pureJava">pureJava/</a>,
- etc.
- </td>
-
- </tr>
-
- <tr><td><a href="#product">Product</a> tests use a variety of
- Ant and Bash scripts to drive the product distributions.
- <td>bash,
- <a href="http://jakarta.apache.org/Ant">Ant</a>
- </td>
- <td><a href="product">product/..</a></td>
- </tr>
-
- <tr><td><a href="#ajctest">AjcTest</a> is a large taskdef which
- compiles and runs large Java systems.
- </td>
- <td>AjcTest
- <a href="http://jakarta.apache.org/Ant">Ant</a>
- Task in testing module </td>
- <td><a href="other-systems/">other-systems/..</a>,
- <a href="XXX">aspectj-external-lib.jar?/other-systems/</a></td>
- </tr>
-
- <tr><td><a href="#jacks">Jacks</a> is the Jikes compiler test suite,
- run for us using a bash script.
- <td>bash</td>
- <td>See the
- <a href="http://www-124.ibm.com/developerworks/oss/cvs/jikes/~checkout~/jacks/jacks.html">
- Jacks project page</a>
- </td>
- </tr>
-
-
- <tr><td><a href="#junit">JUnit</a> tests normally are run directly
- before checking in,
- but there is an Ant script to run and report on them all.
- </td>
- <td><a href="junit.org">JUnit</a></td>
- <td>Every module has a testsrc/ directory with JUnit tests.</td>
- </tr>
- </table>
-
- <h3><a name="harness"></a>Harness</h3>
-
- <h4><a name="harness-tests"></a>Harness tests</h4>
- These are compiler test cases, the preferred form for
- any compiler tests. The XML files are test definitions,
- and the subdirectories contain the test sources (many of which
- are unused at present). The test definitions are
- segregated for convenience as follows:
- <ul>
- <li><a href="ajcTests.xml">ajcTests.xml</a>: main test suite.
- After a release, these tests should always run and pass.
- </li>
- <li><a href="ajcTests10.xml">ajcTests10.xml</a>:
- Tests for the 1.0 compiler. In some cases, error tests
- differ between the 1.0 and 1.1 compiler only in the
- errors expected to be generated.
- </li>
- <li><a href="ajcTestsBroken.xml">ajcTestsBroken.xml</a>:
- These tests fail, and the tests are broken.
- The tests should be fixed and rolled into ajcTests.xml
- </li>
- <li><a href="ajcTestsFailing.xml">ajcTestsFailing.xml</a>:
- These tests fail, and the compiler is broken.
- When fixed, these tests should be rolled into
- <a href="ajcTests.xml">ajcTests.xml</a>.
- </li>
- <li><a href="ajcTestsAttic.xml">ajcTestsAttic.xml</a>:
- These are broken or failing tests not slated to be fixed.
- </li>
- <li><a href="ajcHarnessTests.xml">ajcHarnessTests.xml</a>:
- These are tests to exercise the test harness itself.
- </li>
- </ul>
- The tests specify most anything one could want to do with
- compile/run variants. The best summary of the test specification
- format is the <a href="ajcTestSuite.dtd">ajcTestSuite.dtd</a>.
-
- <p>
- To write a new test, evolve an existing example. See the dtd and/or
- the javadoc for the testing harness implementation in the
- <code>testing</code> module for more details.
-
- <p>To write a test case for a bug, see
- <a href="readme-writing-compiler-tests.html">
- readme-writing-compiler-tests.html</a>.
- Many bugs that first present in an IDE or the Ant taskdef
- are compiler bugs that can be written in this form.
- We <em>much</em> prefer this automated form over
- manual steps specified in a bug report.
-
- <h4><a name="harness-runs"></a>Running the harness</h4>
- <p>
- To run the harness, use <code>org.aspectj.testing.drivers.Harness</code>,
- in the <code>testing-drivers</code> module. Since that is the main class
- for the <code>testing-drivers</code> module, you can build and
- run the harness as follows:
- <pre>
- cd build/
- ../lib/ant/bin/ant build-testing-drivers
- cd ../tests
- java -jar ../aj-build/jars/testing-drivers-all.jar ajcTests.xml
- </pre>
-
- The harness enable you to
- <ul>
- <li>control output, hiding streams and logging messages</li>
- <li>select tests using keywords, bug ID's, or titles</li>
- <li>pass compiler arguments on the harness command line</li>
- <li>use different compiler wrappers - the command-line compiler,
- the AJDE IDE support (mostly sans GUI), or
- the Ant taskdef support.</li>
- <li>fork java run steps</li>
- </ul>
- For a listing of harness options, use <code>-help</code>.
- For a fuller description of harness options and components, see
- <a href="../testing-drivers/src/org/aspectj/testing/drivers/package.html">
- the package documentation for the harness</a>.
-
- <h3><a name="product"></a>Product</h3>
- Product tests target a built distribution or do compiler (argument)
- testing not supported by the harness.
- Many have been superceded by ant tests.
- To build aspectj with itself, see
- <a href="product/build-aspectj/build.xml">
- product/build-aspectj/build.xml</a>.
-
- <h3><a name="ajctest"></a>AjcTest</h3>
- <a href="ajctest.xml">ajctest.xml</a>
- is an Ant script that compiles and runs large systems.
- The scripts tell the AjcTest task from the testing module to drive
- the sources in other-systems through compile/doc/run cycles.
- To set up the tests, the script combines the aspects checked in to
- other-systems/ with the Java source files from various open-source
- projects, contained in aspectj-external-lib.zip.
- <!-- XXX install and correct -->
-
-
- <h3><a name="jacks"></a>Jacks</h3>
- Jacks is the Jikes compiler test suite available at the
- <a href="http://www-124.ibm.com/developerworks/oss/cvs/jikes/~checkout~/jacks/jacks.html">
- Jacks project page</a>.
-
- We have a bash script to run it using ajc and javac and compare
- the results.
- <h3><a name="junit"></a>JUnit</h3>
- The JUnit tests should be self-documenting.
- For more information on running them, see
- <a href="../build/readme-build-and-test-aspectj.html">
- ../build/readme-build-and-test-aspectj.html</a>.
-
- </body>
- </html>
|