<html>
+ <head><title>Harness Package Documentation</title></head>
<body>
-This document you how to run the harness,
-specifying options on the command-line for the
-various components that form the harness.
+<p>
+The AspectJ compiler test harness can compile and run AspectJ programs
+as specified by the test definitions.
+This document tells you how to run the harness.
+It describes the options you can specify on the command-line to
+control various components that form the harness, either to
+specify options that augment the test definitions or to
+change how the harness works, e.g., selecting particular tests
+or logging more information.
+For information on how to write a test definition, see
+<code>readme-writing-compiler-tests.html</code> in the testing module.
+</p>
<p>
The harness drives compiler tests, using
a chain of responsibility to map elements
-in the schema of a test definition to implementing classes:
-<p>
+in the schema of a test definition to implementing classes.
+
+</p>
<table border="1" cellpadding="1">
<tr><th align="left">Test feature</th>
<th align="left">Description</th>
<td>subclass feature harness</td>
<td>FeatureHarness</td>
</tr>
-<tr><td><code><suite><code></td>
+<tr><td><code><suite></code></td>
<td>Test suite</td>
<td>AjcTest.Suite</td>
</tr>
-<tr><td> <code><ajc-test><code></td>
+<tr><td> <code><ajc-test></code></td>
<td>Test case</td>
<td>AjcTest</td>
</tr>
-<tr><td> <code><compile><code></td>
+<tr><td> <code><compile></code></td>
<td>Initial (batch) compile run</td>
<td>CompilerRun</td>
</tr>
-<tr><td> <code><inc-compile><code></td>
+<tr><td> <code><inc-compile></code></td>
<td>Incremental re-compile</td>
<td>IncCompilerRun</td>
</tr>
-<tr><td> <code><run><code></td>
+<tr><td> <code><run></code></td>
<td>Run class</td>
<td>JavaRun</td>
</tr>
<run> {sub-} run component (JavaRun)
...
-->
-<p>
+<p/>
The compiler used is the AspectJ compiler <code>ajc</code>
(optionally as wrapped by the Ant task or AJDE API's), but
in principle any compiler accepting similar options can be
used.
-<p>
+<p/>
To run from the command-line, use
<code>Harness.main(String[])</code>.
To run programmatically, use <code>Harness.getHarness()</code>.
each component in the chain may accept and interpret, so
you can modify how the tests run by specifying the following
arguments on the harness command line:
-<p>
+<p/>
<table cellpadding="1" border="1">
<tr><th>Component</th><th>Options</th></tr>
<tr><td rowspan="6" valign="top">Harness
<p>suite files, harness verbosity, temp files, option variants
- </td></tr>
+ </p></td></tr>
<tr><td><u>suite files</u>: ajcTest-compliant .txt or .xml files are accepted.
<!-- XXX link to ajcTestSuite.dtd and .txt definitions -->
</td></tr>
<tr><td rowspan="5" valign="top">FeatureHarness
<p>output and logging options
- </td></tr>
+ </p></td></tr>
<tr><td><u>tracing</u>:
<code>-progressDots</code> will print "." for every passed
test completed and "!" for every test completed but not passed.
<tr><td rowspan="5" valign="top">AjcTest
<p>selection options for keywords, bugID (PR), or title (description)
- </td></tr>
+ </p></td></tr>
<tr><td><u>keywords</u>: <code>-ajctest[Require|Skip]Keywords=one{,two}</code>
will either require or skip a test that has one of the
specified keywords.
will require that the title (description) of a test contain
one of the specified substrings, here either "one" or "two".
Use this to select a few tests you know generally.
- <br>
+ <br/>
<code>"-ajctestTitleList=first title\, in theory, second title"</code>
will require that the title (description) of a test be
exactly "first title, in theory" or "second title".
The entire option must be one argument on the command line.
Use this when working with just a few specific tests.
- <br>
+ <br/>
<code>"-ajctestTitleList=../tests/ajcTestResults.txt"</code>
will require that the title (description) of a test be
equal to one listed in <code>../tests/ajcTestResults.txt</code>
This option only differs from the prior in that the parameter
is a valid file to read.
Use this to re-run a large set of tests.
- <br>
+ <br/>
<code>"-ajctestTitleFailList=../tests/ajcTestResults.txt"</code>
is the same as the <code>-ajctestTitleList={file}</code> variant,
except that only results prefixed "FAIL" are included.
<tr><td rowspan="6" valign="top">CompilerRun
<p>compiler options and side-effects
- </td></tr>
+ </p></td></tr>
<tr><td><u>supported options</u>:
The command line passed to the compiler by <code>CompilerRun</code>
is composed of entries derived from the <code><compile></code>
if they are in the same family. For example, <code>-ajc</code> and
<code>eclipse</code> are both compiler, and <code>-source 1.4</code>
and <code>-source 1.3</code> are both source.
+ <br/>
</li>
- <ul><p>
+ </ul>
</td></tr>
<tr><td><u>auto-skip</u>: After combining global and local options, there may be
conflicting or impossible options, which cause the test to be skipped:
- e.g., <code>-lenient</code> and <code>-strict</code></li>
<li><u>impossible option</u>: It may not be possible in the current configuration to
implement an option - e.g., <code>-usejavac</code> or <code>-eclipse</code>
- when javac or the eclipse implementation is not on the classpath</li>
- <ul><p>
+ when javac or the eclipse implementation is not on the classpath
+ <br/></li>
+ </ul>
</td></tr>
<tr><td><u>source searching</u>: Given <code>-seek:{literal}</code>,
<tr><td rowspan="2" valign="top">JavaRun
- <p>Options and forking</td>
+ <p>Options and forking</p></td>
<td><u>options</u>: options specified in the test are passed
to the main method as they would be on the command-line.
No options passed to the harness are passed through to
than can be supported by the harness (i.e., some 1.1 flavor);
it's very time-consuming otherwise.
Currently forking is only controllable through system properties
- of the invoking vm (defined in JavaRun):
+ of the invoking vm (defined in JavaRun.java):
<ul>
<li><u>javarun.fork</u>: anything to run in a new vm.
</li>
run classpath. Multiple entries must be separated by
the system-dependent path separator.
</li>
+ <li><u>javarun.vmargs</u>: this is added to the fork command-line
+ right after java. Multiple entries must be separated by a comma
+ (and the whole thing should be one parameter), e.g.,
+ <code>-Djavarun.vmargs=-Dname=value,-Dname2="value 2"</code>
+ </li>
</ul>
</td></tr>
</table>
-<p>
+<br/>
Following are some sample configurations:
<ul>
<li><code>java {harness} -hideStreams {suiteFile}</code>
<p>Use this to output only a 1-line summary of the test results
- (tests skipped, incomplete, failed, passed).<p>
+ (tests skipped, incomplete, failed, passed).<br/></p>
</li>
<li><code>java {harness} -hideStreams -traceTestsMin {suiteFile} > results.txt</code>
<p>This writes to result.txt one line [PASS|FAIL] per test, plus a
- 1-line summary of the test results.<p>
+ 1-line summary of the test results.<br/></p>
</li>
<li><code>java {harness} -logFail {suiteFile} -ajctestTitleFailList=results.txt</code>
<p>This re-runs any test that failed from the "results.txt" run,
- verbosely logging any fails.<p>
+ verbosely logging any fails.<br/></p>
</li>
<li><code>java {harness} -hideStreams -logMinFail {suiteFile}</code>
<p>Use this when running tests mainly to see if they pass or
if the failure messages are typically enough information
to indicate why the test is failing. It produces only minimal
- output for failed tests.<p>
+ output for failed tests.<br/></p>
</li>
<li><code>java {harness} -hideStreams -verboseHarness -logFail {suiteFile}</code>
<p>When it's not clear at first glance why a test is failing, before
looking at the test code you can run it and print any harness or test
- setup failures and all the associated messages from the test components.<p>
+ setup failures and all the associated messages from the test components.<br/></p>
</li>
<li><code>java {harness} -hideStreams -usejavac- -ajc -Xlint- {suiteFile}</code>
<p>Because of the trailing '-' on two of the options,
this would do four complete runs with the old (Ajc 1.0) compiler: one with
- no options, one with -lenient, one with -Xlint, and one with both.<p>
+ no options, one with -lenient, one with -Xlint, and one with both.<br/></p>
</li>
<li><code>java {harness} --ajctestPR=101,102 -Xlint- ^usejavac !eclipse {suiteFile}</code>
<p>Run any tests associated with bugs 101 and 102, with and without -Xlint,
- forcing off -usejavac and forcing the use of the new eclipse-based compiler.<p>
+ forcing off -usejavac and forcing the use of the new eclipse-based compiler.<br/></p>
</li>
</ul>
If you have a set of options you use often, you can define a single-word
option alias for it; see <code>Harness.optionAliases</code>.
-<p><u>Configuration</u>: Most tests use the library jars in
+<br/><u>Configuration</u>: Most tests use the library jars in
<code>modules/lib/test</code>, defined in
<code>org.aspectj.testing.harness.bridge.Globals</code>.
Normally the harness finds these by relative path
$ java -Dharness.libdir=../modules/lib/test ...
</pre>
-<p><u>Forking:</u>:
+<br/><u>Forking:</u>:
The harness must be run in a compiler-compatible VM, and the
compiler steps run in-process.
However, the java steps can be run in forked mode, which is useful
projects / aspectj.git / commitdiff