From 5ca7486fb05c879a1741928f8fd5466545232bd4 Mon Sep 17 00:00:00 2001 From: wisberg Date: Sun, 25 May 2003 20:11:11 +0000 Subject: [PATCH] documenting options --- .../org/aspectj/testing/drivers/package.html | 176 +++++++++++++----- 1 file changed, 127 insertions(+), 49 deletions(-) diff --git a/testing-drivers/src/org/aspectj/testing/drivers/package.html b/testing-drivers/src/org/aspectj/testing/drivers/package.html index 6bd1738db..6e8427bc6 100644 --- a/testing-drivers/src/org/aspectj/testing/drivers/package.html +++ b/testing-drivers/src/org/aspectj/testing/drivers/package.html @@ -1,26 +1,69 @@ -This harness drives ajc compiler tests. -You can run it from the command-line or programmatically. -It works through a chain of responsibility roughly mapped to -the schematic of a test definition: -
-   general harness
-      subclass feature harness
-        ajc-test run component
-          compile {sub-} run component
-          inc-compile {sub-} run component
-          run {sub-} run component
+This document you how to run the harness,
+specifying options on the command-line for the
+various components that form the harness.
+

+The harness drives compiler tests, using  
+a chain of responsibility to map elements
+in the schema of a test definition to implementing classes:
+

+
+
+    
+    
+
+
+    
+    
+
+
+    
+    
+
+
+    
+    
+
+
+    
+    
+
+
+    
+    
+
+
+    
+    
+
+
+    
+    
+
+
Test featureDescriptionImplementing class
(loading suites...)general harnessHarness
(logging...)subclass feature harnessFeatureHarness
<suite>Test suiteAjcTest.Suite
  <ajc-test>Test caseAjcTest
    <compile>Initial (batch) compile runCompilerRun
    <inc-compile>Incremental re-compileIncCompilerRun
    <run>Run classJavaRun

+
 

-
-Use Harness.main(String[]) from the command-line,
-or Harness.getHarness() programmatically.
+The compiler used is the AspectJ compiler ajc
+(optionally as wrapped by the Ant task or AJDE API's), but
+in principle any compiler accepting similar options can be
+used.
+

+To run from the command-line, use 
+Harness.main(String[]).
+To run programmatically, use Harness.getHarness().
 Harness.runMain(String[]) takes arguments that
-each component in the chain may accept and interpret:
+each component in the chain may accept and interpret, so
+you can modify how the tests run by specifying the following 
+arguments on the harness command line:
 

 
@@ -31,7 +74,8 @@ each component in the chain may accept and interpret:
   
-  
@@ -39,7 +83,8 @@ each component in the chain may accept and interpret:
      the end of the run, and deletes them.  If you abort the run or specify
      -keepTemp, then temporary (sandbox) directories will remain for analysis.
      In either case, the file system accumulates all temporary directories
-     and files used for a give harness run.
+     and files used for a give harness run; for the ajcTests.xml
+     suite, this runs into thousands of files.
      
     
-
+     
+  
-  

 
ComponentOptions
suite files: ajcTest-compliant .txt or .xml files are accepted.
      
      
-verboseHarness, -quietHarness: 
+  
-verboseHarness, 
+  	  -quietHarness: 
       Log accepted options and skipped tests,
       or do not print even info messages.
      

   
-killTemp: The opposite of -keepTemp,
      this causes the harness to delete temporary (sandbox) directories at 
@@ -70,11 +115,8 @@ each component in the chain may accept and interpret:
      -hideStreams and 
      !eclipse, used to emit tests results in a form
      comparable by org.aspectj.testing.util.TestDiffs. 
-     Note that output in the form emitted by -traceTestsMin
-     is input for the options to select tests by title
-     (e.g., -ajctestTitleList)
-     and for the tool which compare results from different test runs
-     (org.aspectj.testing.util.TestDiffs).    
+     or usable to select tests by title for options like 
+     -ajctestTitleList.
      

      
   
output: -hide{Compiler|Run}Streams will prevent output and
@@ -87,60 +129,86 @@ each component in the chain may accept and interpret:
      The infix {Min} means to log with minimal information, typically only any
      fail messages.
      The infix {Xml} means to log the XML form of the test definition, so that
-     you can inspect the input or re-run arbitrary tests.  (For the latter, consider
-     also using keywords, under -ajctestsRequireKeywords=... below.)
+     you can inspect the input or re-run arbitrary tests.  
+     (You can also re-run a set of tests using keywords
+     (e.g., "-ajctestsRequireKeywords=..." or using titles
+     (e.g., "-ajctestsTitleFailList=ajcTestResults.txt".)
      Finally, the experimental option -XlogPublicType will 
       log the XML test definition for 
   	  any test run that emits any ERROR messages containing the text "public type".
      

   
interaction of output streams and logging: 
-     Streams will be emitted before the test is logged, unless streams are hidden.
+     Streams will be emitted in real-time,
+     before the test is logged, unless streams are hidden.
      When logging in normal (non-Min or -XML) form, the log will emit the streams
      with the test report, so e.g., you can use -hideStreams -logFail to 
-     hide streams for passing tests but emit them for failing tests.
+     hide streams for passing tests but emit them for failing tests
+     in the context of the log.
      
AjcTest
+
AjcTest
     
selection options for keywords, bugID (PR), or title (description)
     

   
keywords: -ajctest[Require|Skip]Keywords=one{,two}
-     will either require or skip tests that have the specified keywords.
+     will either require or skip a test that has one of the 
+     specified keywords.
      

   
Bugs: -ajctestPR=101{,102}
-     will run only tests that are associated with one of the bug id's listed.
+     will require that a test have one of the listed bug id's.
      

   
title: 
      "-ajctestTitleContains=one,two"
-     will require that the title (description) of the test contain 
-     either "one" or "two".
+     will require that the title (description) of a test contain 
+     one of the specified substrings, here either "one" or "two".
      Use this to select a few tests you know generally.
      

-     "-ajctestTitleList=../tests/ajcTestResults.txt"
-     will require that the title (description) of the test be
-     equal to one listed in ../tests/ajcTestResults.txt
-     as a line of the form "[PASS|FAIL] {title}(.."
-     (emitted by the -traceTestsMin option).
-     Use this to re-run a set of tests.
-     

      "-ajctestTitleList=first title\, in theory, second title"
-     will require that the title (description) of the test be
+     will require that the title (description) of a test be
      exactly "first title, in theory" or "second title".  
      The entire option must be one argument on the command line.
-     This option only differs from the prior in not specifying
-     a valid file to read.
      Use this when working with just a few specific tests.
      

+     "-ajctestTitleList=../tests/ajcTestResults.txt"
+     will require that the title (description) of a test be
+     equal to one listed in ../tests/ajcTestResults.txt
+     as a line of the form "[PASS|FAIL] {title}(.."
+     (This form is emitted by the -traceTestsMin option).
+     This option only differs from the prior in that the parameter
+     is a valid file to read.
+     Use this to re-run a large set of tests.
+     

      "-ajctestTitleFailList=../tests/ajcTestResults.txt"
-     is the same as the -ajctestTitleList=.. variant,
+     is the same as the -ajctestTitleList={file} variant,
      except that only results prefixed "FAIL" are included.
      Use this to re-run only the tests that failed from a large set.
      
Combinations: all selectors are applied to each test,
+     so all tests selected will comply with all constraints.
+     Specifying lists within a particular constraints will match
+     a union of tests for that constraint 
+     (e.g., all tests with bug id's 101 or 102),     
+     but there is no way to get a union of constraints 
+     (e.g., all tests with bug id's 101 or 102 or 
+     with keywords pure-java or knownLimitation).
+     However, -ajctestSkipKeywords=... can return all
+     tests without the specified keywords, so it can form unions like
+     "all tests without the knownLimitation keyword, but with 
+     bug id's 101 or 102".
+     Title lists can work similarly.  E.g., to run the failed
+     incremental tests from ajcTestResults.txt, specify
+     -ajctestTitleFailList=../tests/ajcTestResults.txt
+     -ajctestRequireKeywords=incremental-test.
+     

     
 
CompilerRun
 
compiler options and side-effects
     

   
supported options: Options given on the command-line have
-     the same meaning as the options in the test specification.
+     the same meaning they would have to the compiler
+     (i.e., the same meaning as they would have as a value in the
+     options attribute in the <compile> entity
+     in the <ajc-test> test specification).
      Only one-word options are supported; for this reason, -source 1.4 is
      specified as -source14 and converted by CompilerRun back 
      to -source 1.4. Unsupported options include
@@ -149,6 +217,13 @@ each component in the chain may accept and interpret:
      -classpath,
      -outjar, and
      -sourceroot.
+     (But argfile, 
+      classpath and       
+     sourceroot 
+     are attributes accepted in the <compile> entity,
+     and in any case the test calculates and uses
+     -d and 
+     -classpath.)
      

   
compiler selectors: 
   	 Use -ajc or -eclipse to select the old
@@ -160,10 +235,13 @@ each component in the chain may accept and interpret:
       and -ajctaskCompiler to run a wrapper around the
       AjcTask (Ant task) interface.
      
option dominance -!^: 
+  
option dominance [-|!|^]: 
      Some tests require or prohibit certain options; 
-     likewise, sometime the person running the tests wants to require that all tests
-     run with or without an option specified on the command-line.  CompilerRun supports encodings and
+     likewise, sometime you want to force all tests
+     run with or without an option specified on the command-line,
+     regardless of its setting in the <compile options=".." ...> 
+     attribute. 
+     CompilerRun supports encodings and
      conflict resolution for these, so an option may be specified as
       -option,
       !option, or 
-- 
2.39.5