diff options
| -rwxr-xr-x | README-TESTS.md | 46 |
1 files changed, 25 insertions, 21 deletions
diff --git a/README-TESTS.md b/README-TESTS.md index 9623f7a1e1..2354aa2699 100755 --- a/README-TESTS.md +++ b/README-TESTS.md @@ -2,30 +2,30 @@ ## Project setup The project currently supports running TestBench 3+ (java) tests. Each test consists of a Vaadin UI class and a TestBench script/java test. -All test UI classes go into uitest/src. These files are automatically packaged into a war file which is deployed to a Jetty server during the build process so that tests can open and interact with the UI:s. For development purposes, the Jetty server can be started in Eclipse, see running tests in Eclipse. +All test UI classes go into `uitest/src`. These files are automatically packaged into a war file which is deployed to a Jetty server during the build process so that tests can open and interact with the UI:s. For development purposes, the Jetty server can be started in Eclipse, see running tests in Eclipse. -The project is setup so that /run is mapped to a specialized servlet which allows you to add the UI you want to test to the URL, e.g. http://localhost:8888/run/com.vaadin.tests.component.label.LabelModes or just http://localhost:8888/run/LabelModes if there are no multiple classes named LabelModes. Because of caching, the ?restartApplication parameter is needed after the first run if you want to run multiple test classes. +The project is setup so that `/run` is mapped to a specialized servlet which allows you to add the UI you want to test to the URL, e.g. http://localhost:8888/run/com.vaadin.tests.component.label.LabelModes or just http://localhost:8888/run/LabelModes if there are no multiple classes named LabelModes. Because of caching, the ?restartApplication parameter is needed after the first run if you want to run multiple test classes. ## Creating a new test Creating a new test involves two steps: Creating a test UI (typically this should be provided in the ticket) and creating a TestBench test for the UI. ## Creating a new test UI -Test UIs are created inside the uitest/src folder in the com.vaadin.tests package. For tests for a given component, use the com.vaadin.tests.components.<component> package. For other features, use a suitable existing com.vaadin.tests.<something> package or create a new one if no suitable exists. +Test UIs are created inside the `uitest/src` folder in the `com.vaadin.tests` package. For tests for a given component, use the `com.vaadin.tests.components.<component>` package. For other features, use a suitable existing com.vaadin.tests.<something> package or create a new one if no suitable exists. The test should be named according to what it tests, e.g. EnsureFormTooltipWorks. Names should not refer to a ticket, e.g. Ticket1123 will automatically be rejected during code review as it is non-descriptive. There are a couple of helper UI super classes which you can use for the test. You should never extend UI directly: -* AbstractTestUI +* `AbstractTestUI` * Automatically sets up a VerticalLayout with a description label on the top. Use addComponent() to add components to the layout * Supports ?transport=websocket/streaming/xhr parameter for setting push mode. This is for testing core push functionality, typically you should just add @Push to your test UI -* AbstractTestUIWithLog +* `AbstractTestUIWithLog` * AbstractTestUI but adds a log at the top to which you can add rows using log(String). Handy for printing state information which the TB test can assert reliably. -* ~~AbstractTestCase, TestBase~~ - * Old base classes for tests. Don’t use for any new tests. Extends LegacyApplication. -* AbstractComponentTest, AbstractFieldTest, ... +* `AbstractComponentTest`, `AbstractFieldTest`, ... * Base classes for generic component tests. Generates test which have a menu on the top containing options for configuring the component. Classes follow the same component hierarchy as Vaadin component classes and this way automatically gets menu items for setting features the parent class supports. * Gotcha: If you add a new feature to a menu you need to run and possibly (probably) fix all TB tests which use the class as they will click on the wrong item (fixable by implementing [http://dev.vaadin.com/ticket/11307](http://dev.vaadin.com/ticket/11307)) +[Note] `AbstractTestCase` and `TestBase` are old base classes for tests. Don't use them for any new tests. They extend `LegacyApplication` which is a deprecated class. + ## Creating a TestBench test for a UI All new test for the projects must be created as TestBench3+ tests ### Test class naming @@ -33,10 +33,10 @@ TestBench 3+ tests usually follow a naming convention which automatically maps t ### Super class There are a couple of super classes you can use for a TB3+ test: -* MultiBrowserTest +* `MultiBrowserTest` * Ensures the test is run on the browsers we support automatically * **This is what you typically should use** -* WebsocketTest +* `WebsocketTest` * Run only on browsers which supports websockets ### Creating the test @@ -46,11 +46,11 @@ The actual test is one method in the test class created for the given UI. The me public void testLabelModes() throws Exception { -“@Test” is needed for it to be run at all. +`@Test` is needed for it to be run at all. -“throws Exception” should usually be added to avoid catching exceptions inside the test, as most often an exception means the test has failed. Unhandled exceptions will always fail the test; if you use checked exceptions, you'll need to handle them somehow or decide they're automatically failures on exception. +`throws Exception` should usually be added to avoid catching exceptions inside the test, as most often an exception means the test has failed. Unhandled exceptions will always fail the test; if you use checked exceptions, you'll need to handle them somehow or decide they're automatically failures on exception. -The beginning of the test should request the UI using openTestURL(); +The beginning of the test should request the UI using `openTestURL()`; public void testLabelModes() throws Exception { // Causes the test to be opened with the debug console open. Typically not needed @@ -97,15 +97,15 @@ Use ids in your UI class. Define the IDs as constants in the UI class. Use the c ## Running the DevelopmentServerLauncher -The Jetty included in the project can be started in Eclipse by running the “Development Server (Vaadin)” launch configuration in /eclipse (right click -> Debug as -> Development Server (Vaadin) ). This deploys all the tests to localhost:8888 (runs on port 8888, don’t change that unless you want problems). Use /run/<uiclass> to open a test. +The Jetty included in the project can be started in Eclipse by running the “Development Server (Vaadin)” launch configuration in `/eclipse` (right click -> Debug as -> Development Server (Vaadin) ). This deploys all the tests to `localhost:8888` (runs on port 8888, don’t change that unless you want problems). Use `/run/<uiclass>` to open a test. The DevelopmentServerLauncher has a built-in feature which ensures only one instance can be running at a time. There is therefore no need to first stop the old and and then start a new one. Start the server and the old one will be killed automatically. ## Setup before running any tests in Eclipse Before running any tests in Eclipse you need to -1. copy uitest/eclipse-run-selected-test.properties to work/eclipse-run-selected-test.properties -2. edit work/eclipse-run-selected-test.properties - 1. Define com.vaadin.testbench.screenshot.directory as the directory where you checked out the screenshots repository (this directory contains the “references” subdirectory) +1. copy `uitest/eclipse-run-selected-test.properties` to `work/eclipse-run-selected-test.properties` +2. edit `work/eclipse-run-selected-test.properties` + 1. Define `com.vaadin.testbench.screenshot.directory` as the directory where you checked out the screenshots repository (this directory contains the “references” subdirectory) ## Running TB3+ tests @@ -116,15 +116,19 @@ TB3+ tests are standard JUnit tests which can be run using the “Run as -> JUni #### Debugging remotely Running remotely on a single browser (as described above) can be used to debug issues with a given browser but with the downside that you cannot see what is happening with the browser. In theory this is possible if you figure out on what machine the test is run (no good way for this at the moment) and use VNC to connect to that. #### Debugging locally -A better option is to run the test on a local browser instance. To do this you need to add a `@RunLocally` annotation to the test class. @RunLocally uses Firefox by default but also supports other browsers using `@RunLocally(CHROME)`, `@RunLocally(SAFARI)` and `@RunLocally(PHANTOMJS)`. +A better option is to run the test on a local browser instance. To do this you need to add a `@RunLocally` annotation to the test class. `@RunLocally` uses Firefox by default but also supports other browsers using `@RunLocally(CHROME)`, `@RunLocally(SAFARI)`, `@RunLocally(Browser.IE11)` and `@RunLocally(PHANTOMJS)`. + +By default using `@RunLocally` annotation in Framework tests is not allowed. In order to run a test locally, you need to uncomment the line `com.vaadin.testbench.allowRunLocally=true` in `work/eclipse-run-selected-test.properties`. -Some local configuration is needed for certain browsers, especially if you did not install them in the “standard” locations. +Besides, some local configurations are needed for certain browsers, especially if you did not install them in the “standard” locations. **PhantomJS**: PhantomJS is a headless browser, good especially for fast validation. Download it from the [PhantomJS site](http://phantomjs.org/download.html) and add the binary to your PATH. -**Firefox**: If you have Firefox in your PATH, this is everything you need. If Firefox cannot be started, add a **firefox.path** property to `/work/eclipse-run-selected-test.properties`, pointing to your Firefox binary (e.g.`firefox.path=/Applications/Firefox 17 ESR.app/Contents/MacOS/firefox`) +**Firefox**: If you have Firefox in your PATH, this is everything you need. If Firefox cannot be started, add a **firefox.path** property to `/work/eclipse-run-selected-test.properties`, pointing to your Firefox binary (e.g.`firefox.path=/Applications/Firefox 17 ESR.app/Contents/MacOS/firefox`). + +**Chrome**: You need to [download ChromeDriver](http://chromedriver.storage.googleapis.com/index.html) and add a chrome.driver.path property to `/work/eclipse-run-selected-test.properties`, pointing to the ChromeDriver binary (NOT the Chrome binary). -**Chrome**: You need to [download ChromeDriver](http://chromedriver.storage.googleapis.com/index.html) and add a chrome.driver.path property to `/work/eclipse-run-selected-test.properties`, pointing to the ChromeDriver binary (NOT the Chrome binary) +**IE11**: You need to [download IEDriverServer according to the selenium version](http://selenium-release.storage.googleapis.com/index.html) and add a webdriver.ie.driver property to `/work/eclipse-run-selected-test.properties`, point to the IEDriveServer binary. **Safari**: At least on Mac, no configuration should be needed. |