From: wisberg <wisberg>
Date: Sat, 26 Apr 2003 21:15:23 +0000 (+0000)
Subject: moving release testing list to tests/, added some old stuff.
X-Git-Tag: V1_1_0_RC2~150
X-Git-Url: https://source.dussan.org/?a=commitdiff_plain;h=0557b906e87ef2179694dc919ad67b9fbf0fa077;p=aspectj.git

moving release testing list to tests/, added some old stuff.
TODO: write test-release.xml ant script
---

diff --git a/build/readme-build-and-test-aspectj.html b/build/readme-build-and-test-aspectj.html
index 8c1b36bd2..e7829c658 100644
--- a/build/readme-build-and-test-aspectj.html
+++ b/build/readme-build-and-test-aspectj.html
@@ -305,111 +305,10 @@ For bug fixes, associated tests in
   <code>tests/ajcTests.xml</code>.
   
 
-<p>Before a release, run the following tests using a Java 1.3 VM 
-(the minimum required by the AspectJ tools):
-<ol>
-  <li>JUnit tests.  Run these before the final build
-  using <code>tests/junitModules.xml</code>
-  (which runs all the 
-  <code>modules/{module}/testsrc/{module}ModuleTests.java</code>).
-  <p>
-  </li>
-  
-  <li>All compiler tests in <code>tests/ajcTests.xml</code>
-    and  <code>tests/ajcTestsFailing.xml</code>.
-    Run these before the final build using the test harness,
-    as follows:
-<pre>
-  cd tests/
-  java -jar {..}/testing-drivers-all.jar ajcTests.xml -release 
-</pre>        
-    The <code>-release</code> option skips tests 
-    marked <code>purejava</code> or <code>knownLimitation</code>
-    and only emits one-line results for each test:
-<pre>
-PASS one test identifier()
-FAIL another test identifier()
-...
-</pre>
-    (Use <code>-help</code> on the harness to see the full 
-    definition of the <code>-release</code> alias.)
-    Run with and without the <code>-emacssym</code> option, 
-    saving the output
-    to the appropriate subdirectory of the
-      <a href="../../releases">org.aspectj/releases</a> directory
-    with the name 
-       <code>ajcTests-result-{version}.txt</code> 
-    or <code>ajcTests-result-{version}-emacssym.txt</code>.
-    <p>
-    Compare results with a prior release or test run using TestDiffs:
-<pre>
-  set CLASSPATH="{..}/testing-drivers-all.jar"
-  java org.aspectj.testing.util.TestDiffs {first-run}.txt {second-run}.txt
-</pre>        
-   This will generate lists of tests fixed, broken, missing, or
-   added.  For a release, no tests from <code>ajcTests.xml</code>
-   should be broken, and missing tests (those in the first run absent from the
-   second run) should be justified (usually the test was
-   removed as invalid). A test will show up as
-   missing from the first run and added to the second run if
-   it is renamed between runs.
-   For a final release, no test should be <code>FAIL</code> unless it is 
-   deferred (including those from <code>ajcTestsFailing.xml</code>).
-   <p>
-    For more information, see the instructions for building and running
-    the test harness and the tests
-       <a href="../tests/readme-tests-module.html">
-                ../tests/readme-tests-module.html</a>.
-  <p>
-  </li>  
-  <li>Examples.  Run these on the built/installed release.
-The following should build and run the spacewar example:
-<pre>
-    cd {aspectj-install}/doc/examples
-    {ant 1.5.1} -f build.xml
-</pre>
-Also run 2-3 other targets from Ant and some examples directly:
-<ul>
-<li>
-The target <code>all</code>
-runs all examples, and <code>nonGui</code> runs those
-that do not require manual operation; 
-</li>
-
-<li>The target<code>tracing-bc</code> does bytecode weaving;
-</li>
-<li>
-Run some of the compiled examples in a 1.1 VM
-    (the minimum required for running AspectJ code).
-</li>
-</ul>
-<p>
-This quick test verifies that the 
-<code>aspectjtools.jar</code> and <code>aspectjrt.jar</code>
-are installed and have a matching version, that the examples actually
-compile and run, etc.
-   <p>
-   </li> 
-   <li>If there are any bugs which are closed with this release
-   but which do not have an automated test case run using
-   JUnit or the harness, those bugs should be manually tested 
-   against the release.
-   <p>
-   </li> 
-   <li>Any other tests warranted by release objectives.
-   E.g., make sure it works as expected for any demos or 
-   for any clients (e.g., Emacs, AJDT).
-   </li> 
-</ol>
-<p>Some failures with past releases to consider checking:
-<ul>
-  <li>debug info not included in .class files</li>
-  <li>API not included in documentation</li>
-  <li>Ant variables not filtered into doc files (e.g., ant-tasks.html)</li>
-  <li>Manual dates not changed in docs or readme</li>
-  <li>Inconsistent version information between runtime and tools classes</li>
-</ul>
-   
+<p>Before a release, run the release tests as described in
+  <a href="../tests/readme-release-tests.html">
+           ../tests/readme-release-tests.html</a>.
+                      
 <h4>Release completion</h4>
 When the release build is accepted,
 tag the tree with the release version
diff --git a/tests/readme-release-tests.html b/tests/readme-release-tests.html
new file mode 100644
index 000000000..7ba01adb7
--- /dev/null
+++ b/tests/readme-release-tests.html
@@ -0,0 +1,138 @@
+<html>
+<title>AspectJ release testing</title>
+<body>
+<h1>AspectJ release testing</h1>
+This describes tests to validate AspectJ before a release.
+
+Some may be omitted for pre-release distributions, but all
+should be run for final releases.
+For information on how to build and run tests, see
+  <a href="../build/readme-build-and-test-aspectj.html">
+           ../build/readme-build-and-test-aspectj.html</a>
+and
+  <a href="readme-tests-module.html">
+           readme-tests-module.html</a>.
+
+<p>(Most of these can and should be rolled into an Ant script
+for automation; until that script is written, this describes
+tests and could and/or should be run.)
+
+<p>Before a release, run the following tests using a Java 1.3 VM 
+(the minimum required by the AspectJ tools):
+<ol>
+  <li>JUnit tests.  Run these before the final build
+  using <code>tests/junitModules.xml</code>
+  (which runs all the 
+  <code>modules/{module}/testsrc/{module}ModuleTests.java</code>).
+  <p>
+  </li>
+  
+  <li>All compiler tests in <code>tests/ajcTests.xml</code>
+    and  <code>tests/ajcTestsFailing.xml</code>.
+    Run these before the final build using the test harness,
+    as follows:
+<pre>
+  cd tests/
+  java -jar {..}/testing-drivers-all.jar ajcTests.xml -release 
+</pre>
+    The <code>-release</code> option skips tests 
+    marked <code>purejava</code> or <code>knownLimitation</code>
+    and only emits one-line results for each test:
+<pre>
+PASS one test identifier()
+FAIL another test identifier()
+...
+</pre>
+    (Use <code>-help</code> on the harness to see the full 
+    definition of the <code>-release</code> alias.)
+    Save the output
+    to the appropriate subdirectory of the
+      <a href="../../releases">org.aspectj/releases</a> directory
+    with the name 
+       <code>ajcTests-result-{version}.txt</code>.
+    or <code>ajcTests-result-{version}-emacssym.txt</code>.
+    <p>
+    Compare results with a prior release or test run using TestDiffs:
+<pre>
+  set CLASSPATH="{..}/testing-drivers-all.jar"
+  java org.aspectj.testing.util.TestDiffs {first-run}.txt {second-run}.txt
+</pre>        
+   This will generate lists of tests fixed, broken, missing, or
+   added.  For a release, no tests from <code>ajcTests.xml</code>
+   should be broken, and missing tests (those in the first run absent from the
+   second run) should be justified (usually the test was
+   removed as invalid). A test will show up as
+   missing from the first run and added to the second run if
+   it is renamed between runs.
+   For a final release, no test should be <code>FAIL</code> unless it is 
+   deferred (including those from <code>ajcTestsFailing.xml</code>).
+   <p>
+   For (at least final) releases, run all relevant variants:
+   <ul>
+     <li>With and without <code>-emacssym</code></li>
+     <li>1.3 and 1.4 VM's</li>
+     <li>without -release and with purejava tests.  While 
+         these may fail, they should not crash the compiler.</li>
+   </ul>
+
+  <li>Examples.  Run these on the built/installed release.
+The following should build and run the spacewar example:
+<pre>
+    cd {aspectj-install}/doc/examples
+    {ant 1.5.1} -f build.xml
+</pre>
+Also run 2-3 other targets from Ant and some examples directly:
+<ul>
+<li>
+The target <code>all</code>
+runs all examples, and <code>nonGui</code> runs those
+that do not require manual operation; 
+</li>
+
+<li>The target<code>tracing-bc</code> does bytecode weaving
+</li>
+<li>The target<code>tracing-adapter-ajc</code> compiles the
+    tracing example using the <javac> task and the 
+    Ajc compiler adapter.
+</li>
+<li>
+Run the compiled examples  in a 1.1 VM
+    (the minimum required for running AspectJ code).
+</li>
+</ul>
+<p>
+This quick test verifies that the 
+<code>aspectjtools.jar</code> and <code>aspectjrt.jar</code>
+are installed and have a matching version, that the examples actually
+compile and run, etc.
+   <p>
+   </li> 
+   <li>If there are any bugs which are closed with this release
+   but which do not have an automated test case run using
+   JUnit or the harness, those bugs should be manually tested 
+   against the release.
+   <p>
+   </li> 
+   <li>Any other tests warranted by release objectives.
+   E.g., make sure it works as expected for any demos or 
+   for any clients (e.g., Emacs, AJDT).
+   </li> 
+   <li>Compile large open-source Java projects using AspectJ.       
+   </li>
+   <li>Run the AspectJ compiler against the Jacks test suite.
+   </li>
+   <li>Manually step through the 
+       <a href="ajde/script.html">ajde/script.html</a>
+       on the AspectJ browser.
+</ol>
+<p>Some failures with past releases to consider checking:
+<ul>
+  <li>debug info not included in .class files</li>
+  <li>API not included in documentation</li>
+  <li>Ant variables not filtered into doc files (e.g., ant-tasks.html)</li>
+  <li>Manual dates not changed in docs or readme</li>
+  <li>Inconsistent version information between runtime and tools classes</li>
+</ul>
+ 
+</body>
+</html>