diff options
| author | wisberg <wisberg> | 2003-04-26 21:15:23 +0000 |
|---|---|---|
| committer | wisberg <wisberg> | 2003-04-26 21:15:23 +0000 |
| commit | 0557b906e87ef2179694dc919ad67b9fbf0fa077 (patch) | |
| tree | f69b331242f88eeac72282932693842ef617c0ce /tests/readme-release-tests.html | |
| parent | 1e502f504f2ad333ad5627d8e9d106315c38ecc0 (diff) | |
| download | aspectj-0557b906e87ef2179694dc919ad67b9fbf0fa077.tar.gz aspectj-0557b906e87ef2179694dc919ad67b9fbf0fa077.zip | |
moving release testing list to tests/, added some old stuff.
TODO: write test-release.xml ant script
Diffstat (limited to 'tests/readme-release-tests.html')
| -rw-r--r-- | tests/readme-release-tests.html | 138 |
1 files changed, 138 insertions, 0 deletions
diff --git a/tests/readme-release-tests.html b/tests/readme-release-tests.html new file mode 100644 index 000000000..7ba01adb7 --- /dev/null +++ b/tests/readme-release-tests.html @@ -0,0 +1,138 @@ +<html> +<title>AspectJ release testing</title> +<body> +<h1>AspectJ release testing</h1> +This describes tests to validate AspectJ before a release. + +Some may be omitted for pre-release distributions, but all +should be run for final releases. +For information on how to build and run tests, see + <a href="../build/readme-build-and-test-aspectj.html"> + ../build/readme-build-and-test-aspectj.html</a> +and + <a href="readme-tests-module.html"> + readme-tests-module.html</a>. + +<p>(Most of these can and should be rolled into an Ant script +for automation; until that script is written, this describes +tests and could and/or should be run.) + +<p>Before a release, run the following tests using a Java 1.3 VM +(the minimum required by the AspectJ tools): +<ol> + <li>JUnit tests. Run these before the final build + using <code>tests/junitModules.xml</code> + (which runs all the + <code>modules/{module}/testsrc/{module}ModuleTests.java</code>). + <p> + </li> + + <li>All compiler tests in <code>tests/ajcTests.xml</code> + and <code>tests/ajcTestsFailing.xml</code>. + Run these before the final build using the test harness, + as follows: +<pre> + cd tests/ + java -jar {..}/testing-drivers-all.jar ajcTests.xml -release +</pre> + The <code>-release</code> option skips tests + marked <code>purejava</code> or <code>knownLimitation</code> + and only emits one-line results for each test: +<pre> +PASS one test identifier() +FAIL another test identifier() +... +</pre> + (Use <code>-help</code> on the harness to see the full + definition of the <code>-release</code> alias.) + Save the output + to the appropriate subdirectory of the + <a href="../../releases">org.aspectj/releases</a> directory + with the name + <code>ajcTests-result-{version}.txt</code>. + or <code>ajcTests-result-{version}-emacssym.txt</code>. + <p> + Compare results with a prior release or test run using TestDiffs: +<pre> + set CLASSPATH="{..}/testing-drivers-all.jar" + java org.aspectj.testing.util.TestDiffs {first-run}.txt {second-run}.txt +</pre> + This will generate lists of tests fixed, broken, missing, or + added. For a release, no tests from <code>ajcTests.xml</code> + should be broken, and missing tests (those in the first run absent from the + second run) should be justified (usually the test was + removed as invalid). A test will show up as + missing from the first run and added to the second run if + it is renamed between runs. + For a final release, no test should be <code>FAIL</code> unless it is + deferred (including those from <code>ajcTestsFailing.xml</code>). + <p> + For (at least final) releases, run all relevant variants: + <ul> + <li>With and without <code>-emacssym</code></li> + <li>1.3 and 1.4 VM's</li> + <li>without -release and with purejava tests. While + these may fail, they should not crash the compiler.</li> + </ul> + + <li>Examples. Run these on the built/installed release. +The following should build and run the spacewar example: +<pre> + cd {aspectj-install}/doc/examples + {ant 1.5.1} -f build.xml +</pre> +Also run 2-3 other targets from Ant and some examples directly: +<ul> +<li> +The target <code>all</code> +runs all examples, and <code>nonGui</code> runs those +that do not require manual operation; +</li> + +<li>The target<code>tracing-bc</code> does bytecode weaving +</li> +<li>The target<code>tracing-adapter-ajc</code> compiles the + tracing example using the <javac> task and the + Ajc compiler adapter. +</li> +<li> +Run the compiled examples in a 1.1 VM + (the minimum required for running AspectJ code). +</li> +</ul> +<p> +This quick test verifies that the +<code>aspectjtools.jar</code> and <code>aspectjrt.jar</code> +are installed and have a matching version, that the examples actually +compile and run, etc. + <p> + </li> + <li>If there are any bugs which are closed with this release + but which do not have an automated test case run using + JUnit or the harness, those bugs should be manually tested + against the release. + <p> + </li> + <li>Any other tests warranted by release objectives. + E.g., make sure it works as expected for any demos or + for any clients (e.g., Emacs, AJDT). + </li> + <li>Compile large open-source Java projects using AspectJ. + </li> + <li>Run the AspectJ compiler against the Jacks test suite. + </li> + <li>Manually step through the + <a href="ajde/script.html">ajde/script.html</a> + on the AspectJ browser. +</ol> +<p>Some failures with past releases to consider checking: +<ul> + <li>debug info not included in .class files</li> + <li>API not included in documentation</li> + <li>Ant variables not filtered into doc files (e.g., ant-tasks.html)</li> + <li>Manual dates not changed in docs or readme</li> + <li>Inconsistent version information between runtime and tools classes</li> +</ul> + +</body> +</html> |