<code>tests/ajcTests.xml</code>.
-<p>Before a release, run the following tests using a Java 1.3 VM
-(the minimum required by the AspectJ tools):
-<ol>
- <li>JUnit tests. Run these before the final build
- using <code>tests/junitModules.xml</code>
- (which runs all the
- <code>modules/{module}/testsrc/{module}ModuleTests.java</code>).
- <p>
- </li>
-
- <li>All compiler tests in <code>tests/ajcTests.xml</code>
- and <code>tests/ajcTestsFailing.xml</code>.
- Run these before the final build using the test harness,
- as follows:
-<pre>
- cd tests/
- java -jar {..}/testing-drivers-all.jar ajcTests.xml -release
-</pre>
- The <code>-release</code> option skips tests
- marked <code>purejava</code> or <code>knownLimitation</code>
- and only emits one-line results for each test:
-<pre>
-PASS one test identifier()
-FAIL another test identifier()
-...
-</pre>
- (Use <code>-help</code> on the harness to see the full
- definition of the <code>-release</code> alias.)
- Run with and without the <code>-emacssym</code> option,
- saving the output
- to the appropriate subdirectory of the
- <a href="../../releases">org.aspectj/releases</a> directory
- with the name
- <code>ajcTests-result-{version}.txt</code>
- or <code>ajcTests-result-{version}-emacssym.txt</code>.
- <p>
- Compare results with a prior release or test run using TestDiffs:
-<pre>
- set CLASSPATH="{..}/testing-drivers-all.jar"
- java org.aspectj.testing.util.TestDiffs {first-run}.txt {second-run}.txt
-</pre>
- This will generate lists of tests fixed, broken, missing, or
- added. For a release, no tests from <code>ajcTests.xml</code>
- should be broken, and missing tests (those in the first run absent from the
- second run) should be justified (usually the test was
- removed as invalid). A test will show up as
- missing from the first run and added to the second run if
- it is renamed between runs.
- For a final release, no test should be <code>FAIL</code> unless it is
- deferred (including those from <code>ajcTestsFailing.xml</code>).
- <p>
- For more information, see the instructions for building and running
- the test harness and the tests
- <a href="../tests/readme-tests-module.html">
- ../tests/readme-tests-module.html</a>.
- <p>
- </li>
- <li>Examples. Run these on the built/installed release.
-The following should build and run the spacewar example:
-<pre>
- cd {aspectj-install}/doc/examples
- {ant 1.5.1} -f build.xml
-</pre>
-Also run 2-3 other targets from Ant and some examples directly:
-<ul>
-<li>
-The target <code>all</code>
-runs all examples, and <code>nonGui</code> runs those
-that do not require manual operation;
-</li>
-
-<li>The target<code>tracing-bc</code> does bytecode weaving;
-</li>
-<li>
-Run some of the compiled examples in a 1.1 VM
- (the minimum required for running AspectJ code).
-</li>
-</ul>
-<p>
-This quick test verifies that the
-<code>aspectjtools.jar</code> and <code>aspectjrt.jar</code>
-are installed and have a matching version, that the examples actually
-compile and run, etc.
- <p>
- </li>
- <li>If there are any bugs which are closed with this release
- but which do not have an automated test case run using
- JUnit or the harness, those bugs should be manually tested
- against the release.
- <p>
- </li>
- <li>Any other tests warranted by release objectives.
- E.g., make sure it works as expected for any demos or
- for any clients (e.g., Emacs, AJDT).
- </li>
-</ol>
-<p>Some failures with past releases to consider checking:
-<ul>
- <li>debug info not included in .class files</li>
- <li>API not included in documentation</li>
- <li>Ant variables not filtered into doc files (e.g., ant-tasks.html)</li>
- <li>Manual dates not changed in docs or readme</li>
- <li>Inconsistent version information between runtime and tools classes</li>
-</ul>
-
+<p>Before a release, run the release tests as described in
+ <a href="../tests/readme-release-tests.html">
+ ../tests/readme-release-tests.html</a>.
+
<h4>Release completion</h4>
When the release build is accepted,
tag the tree with the release version
--- /dev/null
+<html>
+<title>AspectJ release testing</title>
+<body>
+<h1>AspectJ release testing</h1>
+This describes tests to validate AspectJ before a release.
+
+Some may be omitted for pre-release distributions, but all
+should be run for final releases.
+For information on how to build and run tests, see
+ <a href="../build/readme-build-and-test-aspectj.html">
+ ../build/readme-build-and-test-aspectj.html</a>
+and
+ <a href="readme-tests-module.html">
+ readme-tests-module.html</a>.
+
+<p>(Most of these can and should be rolled into an Ant script
+for automation; until that script is written, this describes
+tests and could and/or should be run.)
+
+<p>Before a release, run the following tests using a Java 1.3 VM
+(the minimum required by the AspectJ tools):
+<ol>
+ <li>JUnit tests. Run these before the final build
+ using <code>tests/junitModules.xml</code>
+ (which runs all the
+ <code>modules/{module}/testsrc/{module}ModuleTests.java</code>).
+ <p>
+ </li>
+
+ <li>All compiler tests in <code>tests/ajcTests.xml</code>
+ and <code>tests/ajcTestsFailing.xml</code>.
+ Run these before the final build using the test harness,
+ as follows:
+<pre>
+ cd tests/
+ java -jar {..}/testing-drivers-all.jar ajcTests.xml -release
+</pre>
+ The <code>-release</code> option skips tests
+ marked <code>purejava</code> or <code>knownLimitation</code>
+ and only emits one-line results for each test:
+<pre>
+PASS one test identifier()
+FAIL another test identifier()
+...
+</pre>
+ (Use <code>-help</code> on the harness to see the full
+ definition of the <code>-release</code> alias.)
+ Save the output
+ to the appropriate subdirectory of the
+ <a href="../../releases">org.aspectj/releases</a> directory
+ with the name
+ <code>ajcTests-result-{version}.txt</code>.
+ or <code>ajcTests-result-{version}-emacssym.txt</code>.
+ <p>
+ Compare results with a prior release or test run using TestDiffs:
+<pre>
+ set CLASSPATH="{..}/testing-drivers-all.jar"
+ java org.aspectj.testing.util.TestDiffs {first-run}.txt {second-run}.txt
+</pre>
+ This will generate lists of tests fixed, broken, missing, or
+ added. For a release, no tests from <code>ajcTests.xml</code>
+ should be broken, and missing tests (those in the first run absent from the
+ second run) should be justified (usually the test was
+ removed as invalid). A test will show up as
+ missing from the first run and added to the second run if
+ it is renamed between runs.
+ For a final release, no test should be <code>FAIL</code> unless it is
+ deferred (including those from <code>ajcTestsFailing.xml</code>).
+ <p>
+ For (at least final) releases, run all relevant variants:
+ <ul>
+ <li>With and without <code>-emacssym</code></li>
+ <li>1.3 and 1.4 VM's</li>
+ <li>without -release and with purejava tests. While
+ these may fail, they should not crash the compiler.</li>
+ </ul>
+
+ <li>Examples. Run these on the built/installed release.
+The following should build and run the spacewar example:
+<pre>
+ cd {aspectj-install}/doc/examples
+ {ant 1.5.1} -f build.xml
+</pre>
+Also run 2-3 other targets from Ant and some examples directly:
+<ul>
+<li>
+The target <code>all</code>
+runs all examples, and <code>nonGui</code> runs those
+that do not require manual operation;
+</li>
+
+<li>The target<code>tracing-bc</code> does bytecode weaving
+</li>
+<li>The target<code>tracing-adapter-ajc</code> compiles the
+ tracing example using the <javac> task and the
+ Ajc compiler adapter.
+</li>
+<li>
+Run the compiled examples in a 1.1 VM
+ (the minimum required for running AspectJ code).
+</li>
+</ul>
+<p>
+This quick test verifies that the
+<code>aspectjtools.jar</code> and <code>aspectjrt.jar</code>
+are installed and have a matching version, that the examples actually
+compile and run, etc.
+ <p>
+ </li>
+ <li>If there are any bugs which are closed with this release
+ but which do not have an automated test case run using
+ JUnit or the harness, those bugs should be manually tested
+ against the release.
+ <p>
+ </li>
+ <li>Any other tests warranted by release objectives.
+ E.g., make sure it works as expected for any demos or
+ for any clients (e.g., Emacs, AJDT).
+ </li>
+ <li>Compile large open-source Java projects using AspectJ.
+ </li>
+ <li>Run the AspectJ compiler against the Jacks test suite.
+ </li>
+ <li>Manually step through the
+ <a href="ajde/script.html">ajde/script.html</a>
+ on the AspectJ browser.
+</ol>
+<p>Some failures with past releases to consider checking:
+<ul>
+ <li>debug info not included in .class files</li>
+ <li>API not included in documentation</li>
+ <li>Ant variables not filtered into doc files (e.g., ant-tasks.html)</li>
+ <li>Manual dates not changed in docs or readme</li>
+ <li>Inconsistent version information between runtime and tools classes</li>
+</ul>
+
+</body>
+</html>
projects / aspectj.git / commitdiff