diff options
Diffstat (limited to '3rdparty/simpletest/docs/en/reporter_documentation.html')
| -rw-r--r-- | 3rdparty/simpletest/docs/en/reporter_documentation.html | 616 |
1 files changed, 0 insertions, 616 deletions
diff --git a/3rdparty/simpletest/docs/en/reporter_documentation.html b/3rdparty/simpletest/docs/en/reporter_documentation.html deleted file mode 100644 index 8924b7bb67a..00000000000 --- a/3rdparty/simpletest/docs/en/reporter_documentation.html +++ /dev/null @@ -1,616 +0,0 @@ -<html> -<head> -<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> -<title>SimpleTest for PHP test runner and display documentation</title> -<link rel="stylesheet" type="text/css" href="docs.css" title="Styles"> -</head> -<body> -<div class="menu_back"><div class="menu"> -<a href="index.html">SimpleTest</a> - | - <a href="overview.html">Overview</a> - | - <a href="unit_test_documentation.html">Unit tester</a> - | - <a href="group_test_documentation.html">Group tests</a> - | - <a href="mock_objects_documentation.html">Mock objects</a> - | - <a href="partial_mocks_documentation.html">Partial mocks</a> - | - <span class="chosen">Reporting</span> - | - <a href="expectation_documentation.html">Expectations</a> - | - <a href="web_tester_documentation.html">Web tester</a> - | - <a href="form_testing_documentation.html">Testing forms</a> - | - <a href="authentication_documentation.html">Authentication</a> - | - <a href="browser_documentation.html">Scriptable browser</a> -</div></div> -<h1>Test reporter documentation</h1> - This page... - <ul> -<li> - Displaying <a href="#html">results in HTML</a> - </li> -<li> - Displaying and <a href="#other">reporting results</a> - in other formats - </li> -<li> - Using <a href="#cli">SimpleTest from the command line</a> - </li> -<li> - <a href="#xml">Using XML</a> for remote testing - </li> -</ul> -<div class="content"> - - <p> - SimpleTest pretty much follows the MVC-ish pattern - (Model-View-Controller). - The reporter classes are the view and the model is your - test cases and their hiearchy. - The controller is mostly hidden from the user of - SimpleTest unless you want to change how the test cases - are actually run, in which case it is possible to - override the runner objects from within the test case. - As usual with MVC, the controller is mostly undefined - and there are other places to control the test run. - </p> - - <h2> -<a class="target" name="html"></a>Reporting results in HTML</h2> - <p> - The default HTML display is minimal in the extreme. - It reports success and failure with the conventional red and - green bars and shows a breadcrumb trail of test groups - for every failed assertion. - Here's a fail... - <div class="demo"> - <h1>File test</h1> - <span class="fail">Fail</span>: createnewfile->True assertion failed.<br> - <div style="padding: 8px; margin-top: 1em; background-color: red; color: white;">1/1 test cases complete. - <strong>0</strong> passes, <strong>1</strong> fails and <strong>0</strong> exceptions.</div> - </div> - And here all tests passed... - <div class="demo"> - <h1>File test</h1> - <div style="padding: 8px; margin-top: 1em; background-color: green; color: white;">1/1 test cases complete. - <strong>1</strong> passes, <strong>0</strong> fails and <strong>0</strong> exceptions.</div> - </div> - The good news is that there are several points in the display - hiearchy for subclassing. - </p> - <p> - For web page based displays there is the - <span class="new_code">HtmlReporter</span> class with the following - signature... -<pre> -class HtmlReporter extends SimpleReporter { - public __construct($encoding) { ... } - public makeDry(boolean $is_dry) { ... } - public void paintHeader(string $test_name) { ... } - public void sendNoCacheHeaders() { ... } - public void paintFooter(string $test_name) { ... } - public void paintGroupStart(string $test_name, integer $size) { ... } - public void paintGroupEnd(string $test_name) { ... } - public void paintCaseStart(string $test_name) { ... } - public void paintCaseEnd(string $test_name) { ... } - public void paintMethodStart(string $test_name) { ... } - public void paintMethodEnd(string $test_name) { ... } - public void paintFail(string $message) { ... } - public void paintPass(string $message) { ... } - public void paintError(string $message) { ... } - public void paintException(string $message) { ... } - public void paintMessage(string $message) { ... } - public void paintFormattedMessage(string $message) { ... } - protected string getCss() { ... } - public array getTestList() { ... } - public integer getPassCount() { ... } - public integer getFailCount() { ... } - public integer getExceptionCount() { ... } - public integer getTestCaseCount() { ... } - public integer getTestCaseProgress() { ... } -} -</pre> - Here is what some of these methods mean. First the display methods - that you will probably want to override... - <ul class="api"> - <li> - <span class="new_code">HtmlReporter(string $encoding)</span><br> - is the constructor. - Note that the unit test sets up the link to the display - rather than the other way around. - The display is a mostly passive receiver of test events. - This allows easy adaption of the display for other test - systems beside unit tests, such as monitoring servers. - The encoding is the character encoding you wish to - display the test output in. - In order to correctly render debug output when - using the web tester, this should match the encoding - of the site you are trying to test. - The available character set strings are described in - the PHP <a href="http://www.php.net/manual/en/function.htmlentities.php">html_entities()</a> - function. - </li> - <li> - <span class="new_code">void paintHeader(string $test_name)</span><br> - is called once at the very start of the test when the first - start event arrives. - The first start event is usually delivered by the top level group - test and so this is where <span class="new_code">$test_name</span> - comes from. - It paints the page title, CSS, body tag, etc. - It returns nothing (<span class="new_code">void</span>). - </li> - <li> - <span class="new_code">void paintFooter(string $test_name)</span><br> - Called at the very end of the test to close any tags opened - by the page header. - By default it also displays the red/green bar and the final - count of results. - Actually the end of the test happens when a test end event - comes in with the same name as the one that started it all - at the same level. - The tests nest you see. - Closing the last test finishes the display. - </li> - <li> - <span class="new_code">void paintMethodStart(string $test_name)</span><br> - is called at the start of each test method. - The name normally comes from method name. - The other test start events behave the same way except - that the group test one tells the reporter how large - it is in number of held test cases. - This is so that the reporter can display a progress bar - as the runner churns through the test cases. - </li> - <li> - <span class="new_code">void paintMethodEnd(string $test_name)</span><br> - backs out of the test started with the same name. - </li> - <li> - <span class="new_code">void paintFail(string $message)</span><br> - paints a failure. - By default it just displays the word fail, a breadcrumbs trail - showing the current test nesting and the message issued by - the assertion. - </li> - <li> - <span class="new_code">void paintPass(string $message)</span><br> - by default does nothing. - </li> - <li> - <span class="new_code">string getCss()</span><br> - Returns the CSS styles as a string for the page header - method. - Additional styles have to be appended here if you are - not overriding the page header. - You will want to use this method in an overriden page header - if you want to include the original CSS. - </li> - </ul> - There are also some accessors to get information on the current - state of the test suite. - Use these to enrich the display... - <ul class="api"> - <li> - <span class="new_code">array getTestList()</span><br> - is the first convenience method for subclasses. - Lists the current nesting of the tests as a list - of test names. - The first, top level test case, is first in the - list and the current test method will be last. - </li> - <li> - <span class="new_code">integer getPassCount()</span><br> - returns the number of passes chalked up so far. - Needed for the display at the end. - </li> - <li> - <span class="new_code">integer getFailCount()</span><br> - is likewise the number of fails so far. - </li> - <li> - <span class="new_code">integer getExceptionCount()</span><br> - is likewise the number of errors so far. - </li> - <li> - <span class="new_code">integer getTestCaseCount()</span><br> - is the total number of test cases in the test run. - This includes the grouping tests themselves. - </li> - <li> - <span class="new_code">integer getTestCaseProgress()</span><br> - is the number of test cases completed so far. - </li> - </ul> - One simple modification is to get the HtmlReporter to display - the passes as well as the failures and errors... -<pre> -<strong>class ReporterShowingPasses extends HtmlReporter { - - function paintPass($message) { - parent::paintPass($message); - print "<span class=\"pass\">Pass</span>: "; - $breadcrumb = $this->getTestList(); - array_shift($breadcrumb); - print implode("-&gt;", $breadcrumb); - print "-&gt;$message<br />\n"; - } - - protected function getCss() { - return parent::getCss() . ' .pass { color: green; }'; - } -}</strong> -</pre> - </p> - <p> - One method that was glossed over was the <span class="new_code">makeDry()</span> - method. - If you run this method, with no parameters, on the reporter - before the test suite is run no actual test methods - will be called. - You will still get the events of entering and leaving the - test methods and test cases, but no passes or failures etc, - because the test code will not actually be executed. - </p> - <p> - The reason for this is to allow for more sophistcated - GUI displays that allow the selection of individual test - cases. - In order to build a list of possible tests they need a - report on the test structure for drawing, say a tree view - of the test suite. - With a reporter set to dry run that just sends drawing events - this is easily accomplished. - </p> - - <h2> -<a class="target" name="other"></a>Extending the reporter</h2> - <p> - Rather than simply modifying the existing display, you might want to - produce a whole new HTML look, or even generate text or XML. - Rather than override every method in - <span class="new_code">HtmlReporter</span> we can take one - step up the class hiearchy to <span class="new_code">SimpleReporter</span> - in the <em>simple_test.php</em> source file. - </p> - <p> - A do nothing display, a blank canvas for your own creation, would - be... -<pre> -<strong>require_once('simpletest/simpletest.php');</strong> - -class MyDisplay extends SimpleReporter {<strong> - </strong> - function paintHeader($test_name) { } - - function paintFooter($test_name) { } - - function paintStart($test_name, $size) {<strong> - parent::paintStart($test_name, $size);</strong> - } - - function paintEnd($test_name, $size) {<strong> - parent::paintEnd($test_name, $size);</strong> - } - - function paintPass($message) {<strong> - parent::paintPass($message);</strong> - } - - function paintFail($message) {<strong> - parent::paintFail($message);</strong> - } - - function paintError($message) {<strong> - parent::paintError($message);</strong> - } - - function paintException($exception) {<strong> - parent::paintException($exception);</strong> - } -} -</pre> - No output would come from this class until you add it. - </p> - <p> - The catch with using this low level class is that you must - explicitely invoke it in the test script. - The "autorun" facility will not be able to use - its runtime context (whether it's running in a web browser - or the command line) to select the reporter. - </p> - <p> - You explicitely invoke the test runner like so... -<pre> -<?php -require_once('simpletest/autorun.php'); - -$test = new TestSuite('File test'); -$test->addFile('tests/file_test.php'); -$test->run(<strong>new MyReporter()</strong>); -?> -</pre> - ...perhaps like this... -<pre> -<?php -require_once('simpletest/simpletest.php'); -require_once('my_reporter.php'); - -class MyTest extends TestSuite { - function __construct() { - parent::__construct(); - $this->addFile('tests/file_test.php'); - } -} - -$test = new MyTest(); -$test->run(<strong>new MyReporter()</strong>); -?> -</pre> - We'll show how to fit in with "autorun" later. - </p> - - <h2> -<a class="target" name="cli"></a>The command line reporter</h2> - <p> - SimpleTest also ships with a minimal command line reporter. - The interface mimics JUnit to some extent, but paints the - failure messages as they arrive. - To use the command line reporter explicitely, substitute it - for the HTML version... -<pre> -<?php -require_once('simpletest/autorun.php'); - -$test = new TestSuite('File test'); -$test->addFile('tests/file_test.php'); -$test->run(<strong>new TextReporter()</strong>); -?> -</pre> - Then invoke the test suite from the command line... -<pre class="shell"> -php file_test.php -</pre> - You will need the command line version of PHP installed - of course. - A passing test suite looks like this... -<pre class="shell"> -File test -OK -Test cases run: 1/1, Passes: 1, Failures: 0, Exceptions: 0 -</pre> - A failure triggers a display like this... -<pre class="shell"> -File test -1) True assertion failed. - in createNewFile -FAILURES!!! -Test cases run: 1/1, Passes: 0, Failures: 1, Exceptions: 0 -</pre> - </p> - <p> - One of the main reasons for using a command line driven - test suite is of using the tester as part of some automated - process. - To function properly in shell scripts the test script should - return a non-zero exit code on failure. - If a test suite fails the value <span class="new_code">false</span> - is returned from the <span class="new_code">SimpleTest::run()</span> - method. - We can use that result to exit the script with the desired return - code... -<pre> -<?php -require_once('simpletest/autorun.php'); - -$test = new TestSuite('File test'); -$test->addFile('tests/file_test.php'); -<strong>exit ($test->run(new TextReporter()) ? 0 : 1);</strong> -?> -</pre> - Of course we wouldn't really want to create two test scripts, - a command line one and a web browser one, for each test suite. - The command line reporter includes a method to sniff out the - run time environment... -<pre> -<?php -require_once('simpletest/autorun.php'); - -$test = new TestSuite('File test'); -$test->addFile('tests/file_test.php'); -<strong>if (TextReporter::inCli()) {</strong> - exit ($test->run(new TextReporter()) ? 0 : 1); -<strong>}</strong> -$test->run(new HtmlReporter()); -?> -</pre> - This is the form used within SimpleTest itself. - When you use the "autorun.php", and no - test has been run by the end, this is pretty much - the code that SimpleTest will run for you implicitely. - </p> - <p> - In other words, this is gives the same result... -<pre> -<?php -require_once('simpletest/autorun.php'); - -class MyTest extends TestSuite { - function __construct() { - parent::__construct(); - $this->addFile('tests/file_test.php'); - } -} -?> -</pre> - </p> - - <h2> -<a class="target" name="xml"></a>Remote testing</h2> - <p> - SimpleTest ships with an <span class="new_code">XmlReporter</span> class - used for internal communication. - When run the output looks like... -<pre class="shell"> -<?xml version="1.0"?> -<run> - <group size="4"> - <name>Remote tests</name> - <group size="4"> - <name>Visual test with 48 passes, 48 fails and 4 exceptions</name> - <case> - <name>testofunittestcaseoutput</name> - <test> - <name>testofresults</name> - <pass>This assertion passed</pass> - <fail>This assertion failed</fail> - </test> - <test> - ... - </test> - </case> - </group> - </group> -</run> -</pre> - To get your normal test cases to produce this format, on the - command line add the <span class="new_code">--xml</span> flag. -<pre class="shell"> -php my_test.php --xml -</pre> - You can do teh same thing in the web browser by adding the - URL parameter <span class="new_code">xml=1</span>. - Any true value will do. - </p> - <p> - You can consume this format with the parser - supplied as part of SimpleTest itself. - This is called <span class="new_code">SimpleTestXmlParser</span> and - resides in <em>xml.php</em> within the SimpleTest package... -<pre> -<?php -require_once('simpletest/xml.php'); - -... -$parser = new SimpleTestXmlParser(new HtmlReporter()); -$parser->parse($test_output); -?> -</pre> - The <span class="new_code">$test_output</span> should be the XML format - from the XML reporter, and could come from say a command - line run of a test case. - The parser sends events to the reporter just like any - other test run. - There are some odd occasions where this is actually useful. - </p> - <p> - Most likely it's when you want to isolate a problematic crash - prone test. - You can collect the XML output using the backtick operator - from another test. - In that way it runs in its own process... -<pre> -<?php -require_once('simpletest/xml.php'); - -if (TextReporter::inCli()) { - $parser = new SimpleTestXmlParser(new TextReporter()); -} else { - $parser = new SimpleTestXmlParser(new HtmlReporter()); -} -$parser->parse(`php flakey_test.php --xml`); -?> -</pre> - </p> - <p> - Another use is breaking up large test suites. - </p> - <p> - A problem with large test suites is thet they can exhaust - the default 16Mb memory limit on a PHP process. - By having the test groups output in XML and run in - separate processes, the output can be reparsed to - aggregate the results into a much smaller footprint top level - test. - </p> - <p> - Because the XML output can come from anywhere, this opens - up the possibility of aggregating test runs from remote - servers. - A test case already exists to do this within the SimpleTest - framework, but it is currently experimental... -<pre> -<?php -<strong>require_once('../remote.php');</strong> -require_once('simpletest/autorun.php'); - -$test_url = ...; -$dry_url = ...; - -class MyTestOnAnotherServer extends RemoteTestCase { - function __construct() { - $test_url = ... - parent::__construct($test_url, $test_url . ' --dry'); - } -} -?> -</pre> - The <span class="new_code">RemoteTestCase</span> takes the actual location - of the test runner, basically a web page in XML format. - It also takes the URL of a reporter set to do a dry run. - This is so that progress can be reported upward correctly. - The <span class="new_code">RemoteTestCase</span> can be added to test suites - just like any other test suite. - </p> - - </div> - References and related information... - <ul> -<li> - SimpleTest project page on <a href="http://sourceforge.net/projects/simpletest/">SourceForge</a>. - </li> -<li> - SimpleTest download page on <a href="http://www.lastcraft.com/simple_test.php">LastCraft</a>. - </li> -<li> - The <a href="http://simpletest.org/api/">developer's API for SimpleTest</a> - gives full detail on the classes and assertions available. - </li> -</ul> -<div class="menu_back"><div class="menu"> -<a href="index.html">SimpleTest</a> - | - <a href="overview.html">Overview</a> - | - <a href="unit_test_documentation.html">Unit tester</a> - | - <a href="group_test_documentation.html">Group tests</a> - | - <a href="mock_objects_documentation.html">Mock objects</a> - | - <a href="partial_mocks_documentation.html">Partial mocks</a> - | - <span class="chosen">Reporting</span> - | - <a href="expectation_documentation.html">Expectations</a> - | - <a href="web_tester_documentation.html">Web tester</a> - | - <a href="form_testing_documentation.html">Testing forms</a> - | - <a href="authentication_documentation.html">Authentication</a> - | - <a href="browser_documentation.html">Scriptable browser</a> -</div></div> -<div class="copyright"> - Copyright<br>Marcus Baker 2006 - </div> -</body> -</html> |