spegling av
https://github.com/eclipse-aspectj/aspectj.git
synced 2024-08-13 14:13:07 +02:00
143 rader
4.9 KiB
HTML
143 rader
4.9 KiB
HTML
<html>
|
|
<title>AspectJ release testing</title>
|
|
<body>
|
|
<h1>AspectJ release testing</h1>
|
|
This describes tests to validate AspectJ before a release.
|
|
|
|
<u>WARNING<u>: this file is being superceded by
|
|
<a href="../build/release-checklist.txt">
|
|
../build/release-checklist.txt</a>.
|
|
|
|
Some tests may be omitted for pre-release distributions, but all
|
|
should be run for final releases.
|
|
For information on how to build and run tests, see
|
|
<a href="../build/readme-build-and-test-aspectj.html">
|
|
../build/readme-build-and-test-aspectj.html</a>
|
|
and
|
|
<a href="readme-tests-module.html">
|
|
readme-tests-module.html</a>.
|
|
|
|
<p>(Most of these can and should be rolled into an Ant script
|
|
for automation; until that script is written, this describes
|
|
tests and could and/or should be run.)
|
|
|
|
<p>Before a release, run the following tests using a Java 1.3 VM
|
|
(the minimum required by the AspectJ tools):
|
|
<ol>
|
|
<li>JUnit tests. Run these before the final build
|
|
using <code>tests/junitModules.xml</code>
|
|
(which runs all the
|
|
<code>modules/{module}/testsrc/{module}ModuleTests.java</code>).
|
|
<p>
|
|
</li>
|
|
|
|
<li>All compiler tests in <code>tests/ajcTests.xml</code>
|
|
and <code>tests/ajcTestsFailing.xml</code>.
|
|
Run these before the final build using the test harness,
|
|
as follows:
|
|
<pre>
|
|
cd tests/
|
|
java -jar {..}/testing-drivers-all.jar ajcTests.xml -release
|
|
</pre>
|
|
The <code>-release</code> option skips tests
|
|
marked <code>purejava</code> or <code>knownLimitation</code>
|
|
and only emits one-line results for each test:
|
|
<pre>
|
|
PASS one test identifier()
|
|
FAIL another test identifier()
|
|
...
|
|
</pre>
|
|
(Use <code>-help</code> on the harness to see the full
|
|
definition of the <code>-release</code> alias.)
|
|
Save the output
|
|
to the appropriate subdirectory of the
|
|
<a href="../../releases">org.aspectj/releases</a> directory
|
|
with the name
|
|
<code>ajcTests-result-{version}.txt</code>.
|
|
or <code>ajcTests-result-{version}-emacssym.txt</code>.
|
|
<p>
|
|
Compare results with a prior release or test run using TestDiffs:
|
|
<pre>
|
|
set CLASSPATH="{..}/testing-drivers-all.jar"
|
|
java org.aspectj.testing.util.TestDiffs {first-run}.txt {second-run}.txt
|
|
</pre>
|
|
This will generate lists of tests fixed, broken, missing, or
|
|
added. For a release, no tests from <code>ajcTests.xml</code>
|
|
should be broken, and missing tests (those in the first run absent from the
|
|
second run) should be justified (usually the test was
|
|
removed as invalid). A test will show up as
|
|
missing from the first run and added to the second run if
|
|
it is renamed between runs.
|
|
For a final release, no test should be <code>FAIL</code> unless it is
|
|
deferred (including those from <code>ajcTestsFailing.xml</code>).
|
|
<p>
|
|
For (at least final) releases, run all relevant variants:
|
|
<ul>
|
|
<li>With and without <code>-emacssym</code></li>
|
|
<li>1.3 and 1.4 VM's</li>
|
|
<li>without -release and with purejava tests. While
|
|
these may fail, they should not crash the compiler.</li>
|
|
</ul>
|
|
|
|
<li>Examples. Run these on the built/installed release.
|
|
The following should build and run the spacewar example:
|
|
<pre>
|
|
cd {aspectj-install}/doc/examples
|
|
{ant 1.5.1} -f build.xml
|
|
</pre>
|
|
Also run 2-3 other targets from Ant and some examples directly:
|
|
<ul>
|
|
<li>
|
|
The target <code>all</code>
|
|
runs all examples, and <code>nonGui</code> runs those
|
|
that do not require manual operation;
|
|
</li>
|
|
|
|
<li>The target<code>tracing-bc</code> does bytecode weaving
|
|
</li>
|
|
<li>The target<code>tracing-adapter-ajc</code> compiles the
|
|
tracing example using the <javac> task and the
|
|
Ajc compiler adapter.
|
|
</li>
|
|
<li>
|
|
Run the compiled examples in a 1.1 VM
|
|
(the minimum required for running AspectJ code).
|
|
</li>
|
|
</ul>
|
|
<p>
|
|
This quick test verifies that the
|
|
<code>aspectjtools.jar</code> and <code>aspectjrt.jar</code>
|
|
are installed and have a matching version, that the examples actually
|
|
compile and run, etc.
|
|
<p>
|
|
</li>
|
|
<li>If there are any bugs which are closed with this release
|
|
but which do not have an automated test case run using
|
|
JUnit or the harness, those bugs should be manually tested
|
|
against the release.
|
|
<p>
|
|
</li>
|
|
<li>Any other tests warranted by release objectives.
|
|
E.g., make sure it works as expected for any demos or
|
|
for any clients (e.g., Emacs, AJDT).
|
|
</li>
|
|
<li>Compile large open-source Java projects using AspectJ.
|
|
</li>
|
|
<li>Run the AspectJ compiler against the Jacks test suite.
|
|
</li>
|
|
<li>Manually step through the
|
|
<a href="ajde/script.html">ajde/script.html</a>
|
|
on the AspectJ browser.
|
|
</ol>
|
|
<p>Some failures with past releases to consider checking:
|
|
<ul>
|
|
<li>debug info not included in .class files</li>
|
|
<li>API not included in documentation</li>
|
|
<li>Ant variables not filtered into doc files (e.g., ant-tasks.html)</li>
|
|
<li>Manual dates not changed in docs or readme</li>
|
|
<li>Inconsistent version information between runtime and tools classes</li>
|
|
</ul>
|
|
|
|
</body>
|
|
</html>
|