From 9bf93f948ce3413eddcb4fffad6daed5a416f628 Mon Sep 17 00:00:00 2001 From: wisberg Date: Sun, 25 May 2003 01:37:49 +0000 Subject: updating harness documentation for new features --- .../src/org/aspectj/testing/drivers/package.html | 59 +++++++++++++++++++--- 1 file changed, 52 insertions(+), 7 deletions(-) (limited to 'testing-drivers/src') diff --git a/testing-drivers/src/org/aspectj/testing/drivers/package.html b/testing-drivers/src/org/aspectj/testing/drivers/package.html index 1bdc35634..6bd1738db 100644 --- a/testing-drivers/src/org/aspectj/testing/drivers/package.html +++ b/testing-drivers/src/org/aspectj/testing/drivers/package.html @@ -69,7 +69,12 @@ each component in the chain may accept and interpret: -traceTestsMin -hideStreams and !eclipse, used to emit tests results in a form - comparable by org.aspectj.testing.util.TestDiffs. + comparable by org.aspectj.testing.util.TestDiffs. + Note that output in the form emitted by -traceTestsMin + is input for the options to select tests by title + (e.g., -ajctestTitleList) + and for the tool which compare results from different test runs + (org.aspectj.testing.util.TestDiffs). output: -hide{Compiler|Run}Streams will prevent output and @@ -95,8 +100,8 @@ each component in the chain may accept and interpret: hide streams for passing tests but emit them for failing tests. -AjcTest -

selection options for keywords, bugID (PR) +AjcTest +

selection options for keywords, bugID (PR), or title (description) keywords: -ajctest[Require|Skip]Keywords=one{,two} will either require or skip tests that have the specified keywords. @@ -104,6 +109,32 @@ each component in the chain may accept and interpret: Bugs: -ajctestPR=101{,102} will run only tests that are associated with one of the bug id's listed. + title: + "-ajctestTitleContains=one,two" + will require that the title (description) of the test contain + either "one" or "two". + Use this to select a few tests you know generally. +
+ "-ajctestTitleList=../tests/ajcTestResults.txt" + will require that the title (description) of the test be + equal to one listed in ../tests/ajcTestResults.txt + as a line of the form "[PASS|FAIL] {title}(.." + (emitted by the -traceTestsMin option). + Use this to re-run a set of tests. +
+ "-ajctestTitleList=first title\, in theory, second title" + will require that the title (description) of the test be + exactly "first title, in theory" or "second title". + The entire option must be one argument on the command line. + This option only differs from the prior in not specifying + a valid file to read. + Use this when working with just a few specific tests. +
+ "-ajctestTitleFailList=../tests/ajcTestResults.txt" + is the same as the -ajctestTitleList=.. variant, + except that only results prefixed "FAIL" are included. + Use this to re-run only the tests that failed from a large set. + CompilerRun

compiler options and side-effects @@ -208,6 +239,16 @@ Following are some sample configurations: (tests skipped, incomplete, failed, passed).

+

  • java {harness} -hideStreams -traceTestsMin {suiteFile} > results.txt +

    This writes to result.txt one line [PASS|FAIL] per test, plus a + 1-line summary of the test results.

    +

    • + +
  • java {harness} -logFail {suiteFile} -ajctestTitleFailList=results.txt +

    This re-runs any test that failed from the "results.txt" run, + verbosely logging any fails.

    +

    • +
  • java {harness} -hideStreams -logMinFail {suiteFile}

    Use this when running tests mainly to see if they pass or if the failure messages are typically enough information @@ -222,7 +263,8 @@ Following are some sample configurations:

  • java {harness} -hideStreams -usejavac- -ajc -Xlint- {suiteFile} -

    This would do four complete runs with the old (Ajc 1.0) compiler: one with +

    Because of the trailing '-' on two of the options, + this would do four complete runs with the old (Ajc 1.0) compiler: one with no options, one with -lenient, one with -Xlint, and one with both.

    • @@ -238,7 +280,7 @@ If you have a set of options you use often, you can define a single-word option alias for it; see Harness.optionAliases.

    Configuration: Most tests use the library jars in -aspectj/modules/lib/test, defined in +modules/lib/test, defined in org.aspectj.testing.harness.bridge.Globals. Normally the harness finds these by relative path ../lib/tests/*.jar, which works whenever the tests are @@ -254,8 +296,10 @@ The harness must be run in a compiler-compatible VM, and the compiler steps run in-process. However, the java steps can be run in forked mode, which is useful when compiling for a VM which can't run the compiler. -Here's how you would run the harness to compile against a -specific version of 1.1: +To compile for a different target VM could require +setting the options for bootclasspath, target, and source. +To run the harness so that any <run.. tasks run in a +separate vm, do something like this: 

        java -Djavarun.java=d:\jdk1.1.8\bin\java.exe \
         -Djavarun.bootclasspath=d:\jdk1.1.8\lib\classes.zip \
@@ -279,5 +323,6 @@ to the classpath, with an effect like these commands
      {mainClass} {option..}
    + -- cgit v1.2.3