diff options
| -rw-r--r-- | build/readme-build-and-test-aspectj.html | 109 | ||||
| -rw-r--r-- | tests/readme-release-tests.html | 138 |
2 files changed, 142 insertions, 105 deletions
diff --git a/build/readme-build-and-test-aspectj.html b/build/readme-build-and-test-aspectj.html index 8c1b36bd2..e7829c658 100644 --- a/build/readme-build-and-test-aspectj.html +++ b/build/readme-build-and-test-aspectj.html @@ -305,111 +305,10 @@ For bug fixes, associated tests in <code>tests/ajcTests.xml</code>. -<p>Before a release, run the following tests using a Java 1.3 VM -(the minimum required by the AspectJ tools): -<ol> - <li>JUnit tests. Run these before the final build - using <code>tests/junitModules.xml</code> - (which runs all the - <code>modules/{module}/testsrc/{module}ModuleTests.java</code>). - <p> - </li> - - <li>All compiler tests in <code>tests/ajcTests.xml</code> - and <code>tests/ajcTestsFailing.xml</code>. - Run these before the final build using the test harness, - as follows: -<pre> - cd tests/ - java -jar {..}/testing-drivers-all.jar ajcTests.xml -release -</pre> - The <code>-release</code> option skips tests - marked <code>purejava</code> or <code>knownLimitation</code> - and only emits one-line results for each test: -<pre> -PASS one test identifier() -FAIL another test identifier() -... -</pre> - (Use <code>-help</code> on the harness to see the full - definition of the <code>-release</code> alias.) - Run with and without the <code>-emacssym</code> option, - saving the output - to the appropriate subdirectory of the - <a href="../../releases">org.aspectj/releases</a> directory - with the name - <code>ajcTests-result-{version}.txt</code> - or <code>ajcTests-result-{version}-emacssym.txt</code>. - <p> - Compare results with a prior release or test run using TestDiffs: -<pre> - set CLASSPATH="{..}/testing-drivers-all.jar" - java org.aspectj.testing.util.TestDiffs {first-run}.txt {second-run}.txt -</pre> - This will generate lists of tests fixed, broken, missing, or - added. For a release, no tests from <code>ajcTests.xml</code> - should be broken, and missing tests (those in the first run absent from the - second run) should be justified (usually the test was - removed as invalid). A test will show up as - missing from the first run and added to the second run if - it is renamed between runs. - For a final release, no test should be <code>FAIL</code> unless it is - deferred (including those from <code>ajcTestsFailing.xml</code>). - <p> - For more information, see the instructions for building and running - the test harness and the tests - <a href="../tests/readme-tests-module.html"> - ../tests/readme-tests-module.html</a>. - <p> - </li> - <li>Examples. Run these on the built/installed release. -The following should build and run the spacewar example: -<pre> - cd {aspectj-install}/doc/examples - {ant 1.5.1} -f build.xml -</pre> -Also run 2-3 other targets from Ant and some examples directly: -<ul> -<li> -The target <code>all</code> -runs all examples, and <code>nonGui</code> runs those -that do not require manual operation; -</li> - -<li>The target<code>tracing-bc</code> does bytecode weaving; -</li> -<li> -Run some of the compiled examples in a 1.1 VM - (the minimum required for running AspectJ code). -</li> -</ul> -<p> -This quick test verifies that the -<code>aspectjtools.jar</code> and <code>aspectjrt.jar</code> -are installed and have a matching version, that the examples actually -compile and run, etc. - <p> - </li> - <li>If there are any bugs which are closed with this release - but which do not have an automated test case run using - JUnit or the harness, those bugs should be manually tested - against the release. - <p> - </li> - <li>Any other tests warranted by release objectives. - E.g., make sure it works as expected for any demos or - for any clients (e.g., Emacs, AJDT). - </li> -</ol> -<p>Some failures with past releases to consider checking: -<ul> - <li>debug info not included in .class files</li> - <li>API not included in documentation</li> - <li>Ant variables not filtered into doc files (e.g., ant-tasks.html)</li> - <li>Manual dates not changed in docs or readme</li> - <li>Inconsistent version information between runtime and tools classes</li> -</ul> - +<p>Before a release, run the release tests as described in + <a href="../tests/readme-release-tests.html"> + ../tests/readme-release-tests.html</a>. + <h4>Release completion</h4> When the release build is accepted, tag the tree with the release version diff --git a/tests/readme-release-tests.html b/tests/readme-release-tests.html new file mode 100644 index 000000000..7ba01adb7 --- /dev/null +++ b/tests/readme-release-tests.html @@ -0,0 +1,138 @@ +<html> +<title>AspectJ release testing</title> +<body> +<h1>AspectJ release testing</h1> +This describes tests to validate AspectJ before a release. + +Some may be omitted for pre-release distributions, but all +should be run for final releases. +For information on how to build and run tests, see + <a href="../build/readme-build-and-test-aspectj.html"> + ../build/readme-build-and-test-aspectj.html</a> +and + <a href="readme-tests-module.html"> + readme-tests-module.html</a>. + +<p>(Most of these can and should be rolled into an Ant script +for automation; until that script is written, this describes +tests and could and/or should be run.) + +<p>Before a release, run the following tests using a Java 1.3 VM +(the minimum required by the AspectJ tools): +<ol> + <li>JUnit tests. Run these before the final build + using <code>tests/junitModules.xml</code> + (which runs all the + <code>modules/{module}/testsrc/{module}ModuleTests.java</code>). + <p> + </li> + + <li>All compiler tests in <code>tests/ajcTests.xml</code> + and <code>tests/ajcTestsFailing.xml</code>. + Run these before the final build using the test harness, + as follows: +<pre> + cd tests/ + java -jar {..}/testing-drivers-all.jar ajcTests.xml -release +</pre> + The <code>-release</code> option skips tests + marked <code>purejava</code> or <code>knownLimitation</code> + and only emits one-line results for each test: +<pre> +PASS one test identifier() +FAIL another test identifier() +... +</pre> + (Use <code>-help</code> on the harness to see the full + definition of the <code>-release</code> alias.) + Save the output + to the appropriate subdirectory of the + <a href="../../releases">org.aspectj/releases</a> directory + with the name + <code>ajcTests-result-{version}.txt</code>. + or <code>ajcTests-result-{version}-emacssym.txt</code>. + <p> + Compare results with a prior release or test run using TestDiffs: +<pre> + set CLASSPATH="{..}/testing-drivers-all.jar" + java org.aspectj.testing.util.TestDiffs {first-run}.txt {second-run}.txt +</pre> + This will generate lists of tests fixed, broken, missing, or + added. For a release, no tests from <code>ajcTests.xml</code> + should be broken, and missing tests (those in the first run absent from the + second run) should be justified (usually the test was + removed as invalid). A test will show up as + missing from the first run and added to the second run if + it is renamed between runs. + For a final release, no test should be <code>FAIL</code> unless it is + deferred (including those from <code>ajcTestsFailing.xml</code>). + <p> + For (at least final) releases, run all relevant variants: + <ul> + <li>With and without <code>-emacssym</code></li> + <li>1.3 and 1.4 VM's</li> + <li>without -release and with purejava tests. While + these may fail, they should not crash the compiler.</li> + </ul> + + <li>Examples. Run these on the built/installed release. +The following should build and run the spacewar example: +<pre> + cd {aspectj-install}/doc/examples + {ant 1.5.1} -f build.xml +</pre> +Also run 2-3 other targets from Ant and some examples directly: +<ul> +<li> +The target <code>all</code> +runs all examples, and <code>nonGui</code> runs those +that do not require manual operation; +</li> + +<li>The target<code>tracing-bc</code> does bytecode weaving +</li> +<li>The target<code>tracing-adapter-ajc</code> compiles the + tracing example using the <javac> task and the + Ajc compiler adapter. +</li> +<li> +Run the compiled examples in a 1.1 VM + (the minimum required for running AspectJ code). +</li> +</ul> +<p> +This quick test verifies that the +<code>aspectjtools.jar</code> and <code>aspectjrt.jar</code> +are installed and have a matching version, that the examples actually +compile and run, etc. + <p> + </li> + <li>If there are any bugs which are closed with this release + but which do not have an automated test case run using + JUnit or the harness, those bugs should be manually tested + against the release. + <p> + </li> + <li>Any other tests warranted by release objectives. + E.g., make sure it works as expected for any demos or + for any clients (e.g., Emacs, AJDT). + </li> + <li>Compile large open-source Java projects using AspectJ. + </li> + <li>Run the AspectJ compiler against the Jacks test suite. + </li> + <li>Manually step through the + <a href="ajde/script.html">ajde/script.html</a> + on the AspectJ browser. +</ol> +<p>Some failures with past releases to consider checking: +<ul> + <li>debug info not included in .class files</li> + <li>API not included in documentation</li> + <li>Ant variables not filtered into doc files (e.g., ant-tasks.html)</li> + <li>Manual dates not changed in docs or readme</li> + <li>Inconsistent version information between runtime and tools classes</li> +</ul> + +</body> +</html> |