diff options
author | Erik Lumme <erik@vaadin.com> | 2017-09-12 15:36:15 +0300 |
---|---|---|
committer | Erik Lumme <erik@vaadin.com> | 2017-09-12 15:36:15 +0300 |
commit | 38c2ab8d71052474ae61d6cf3627d99840b7c66f (patch) | |
tree | 0085340b52740dd6eafa407a08b247f244075094 /documentation/articles | |
parent | ddbd2e00f048563047da4303058fb667dae2a4a3 (diff) | |
download | vaadin-framework-38c2ab8d71052474ae61d6cf3627d99840b7c66f.tar.gz vaadin-framework-38c2ab8d71052474ae61d6cf3627d99840b7c66f.zip |
Migrate LoadTestingWithGatling
Diffstat (limited to 'documentation/articles')
-rw-r--r-- | documentation/articles/LoadTestingWithGatling.asciidoc | 267 | ||||
-rw-r--r-- | documentation/articles/contents.asciidoc | 1 |
2 files changed, 268 insertions, 0 deletions
diff --git a/documentation/articles/LoadTestingWithGatling.asciidoc b/documentation/articles/LoadTestingWithGatling.asciidoc new file mode 100644 index 0000000000..ec5bc80525 --- /dev/null +++ b/documentation/articles/LoadTestingWithGatling.asciidoc @@ -0,0 +1,267 @@ +[[loading-testing-with-gatling]] +Load testing with Gatling +------------------------- + +http://gatling.io[Gatling] is a powerful tool for load testing. Compared +to WebDriver/Selenium/TestBench, it doesn't render the actual content, +but just simulates messages clients send to the server. The server +doesn't know if the test tool actually does something with the +responses, so it gives you perfectly valid numbers for your applications +performance - on the server side. It scales very well, so you don' t +need huge army of nodes to bombard your application under test. It can +be used with Vaadin as such, but with these tips you hopefully get +started easier. + +[[vaadin-tips-to-make-tests-more-stable-and-easier-to-create]] +Vaadin tips to make tests more stable and easier to create +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Gatling works with Vaadin "out of the box", but there are some obstacles +you might face if you don't understand a bit how Vaadin communication +works. E.g. plain recordings from standard Vaadin apps will not work +properly. + +The communication that Vaadin's "thin client" in browser does with the +server side has couple of checks that it does to improve robustness and +security. One can simulate these with Gatling as well, by e.g. +https://github.com/mstahv/v-quiz/blob/master/src/test/scala/loadtest/WebSocketVaadinSimulation.scala#L84[reading the XSRF preventation key into a variable] and passing the value +https://github.com/mstahv/v-quiz/blob/master/src/test/scala/loadtest/WebSocketVaadinSimulation.scala#L95[in +each response]. However, these setting can be disabled during load +testing to make it easier to write and maintain your application. The +effect for the scalability, when disabling or configuring these, should +be negligible. Feel free to do these, but also remember to remove these +"hacks" when building your production war file. Consider e.g. using +separate maven profile and inject different parameters with it. + +[[disabling-xsrf-presentation-key]] +Disabling XSRF presentation key +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The XSRF preventation can be disabled with following servlet parameter +(or similar servlet 3 style parameter). NOTE, do not leave this for +public apps in production. + +[source,xml] +.... +<context-param> + <param-name>disable-xsrf-protection</param-name> + <param-value>true</param-value> +</context-param> +.... + +[[disabling-syncid-happens-with-similar-parameter]] +Disabling syncId happens with similar parameter +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +[source,xml] +.... +<context-param> + <param-name>syncIdCheck</param-name> + <param-value>false</param-value> +</context-param> +.... + +If you want to do the above with Java Servlet 3.0 annotations, use the +following: + +[source,java] +.... +initParams = { + @WebInitParam(name = "disable-xsrf-protection", value = "true"), + @WebInitParam(name = "syncIdCheck", value = "false")} +.... + +[[using-debug-ids-in-communication]] +Using debug ids in communication +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you want to minimize the effort needed for maintaining your +scalability tests, you probably want to do a small hack to the Vaadin +communication mechanism. Normally Vaadin uses a incrementing session +wide identifier to connect components to their "client side +counterparts". Thus, if you add a one single component to your login +screen, the whole load test simulation might be broken. + +You can set "id" for each component, but in recent Vaadin versions this +id is no more used in the communication, but only assigned to +client dom. This can still be enforced with a specially crafted +extension to VaadinServlet. An implementation for the "competing tool" JMeter can be +found at <<jmeter-vaadin-servlet-extension>>. This implementation works for Gatling users +as well. Note that, it is suggested to do this only for load testing, and NOT +for the production. + +[[ignoring-obsolete-static-file-requests]] +Ignoring "obsolete" static file requests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +One of the most simplest and cheapest method to improve your apps +scalability is to serve static files form a separate server or from a +CDN provider. Thus it might make sense to leave loading those files away +from your test script. If you do the script manually, just don't add +requests for static files (js/css/images/...). If you recorded you test +case, just remove these form the script. Check out the example project +that only uses the required files. + +[[testing-with-websockets]] +Testing with WebSockets +~~~~~~~~~~~~~~~~~~~~~~~ + +If your want to load test your application with the most advanced +communication channel, WebSockets, you can do that with Gatling as well. +Using the recorder in this case doesn't work, but handcrafting the test +case isn't that hard once you get started. The example app has a branch +with WebSocket test case. With WebSocket communication it might also be +handy to disable xsrf preventation and the so called "syncid". + +First two request are just normal http requests. The first gets the +"host page" and also the initial state request is done with normal XHR. +The difference to normal Vaadin communication is that it is to be sent +to "/PUSH" address. + +After the initial state request you start to use the special WebSocket +API in Gatling. There are lot of things to keep in mind with this +fundamentally totally different kind of communication mechanism. Check +out Gatling's generic websocket help for basic info. + +When you start handcrafting the WebSocket simulation, the easiest tool +is probably Chrome's dev tools. With that you can open normal browser +session and "sniff" the traffic that is sent to the server and also the +messages that are received. An easy option is just to copy paste the +payloads and possibly add some verification to ensure proper answers are +received. The websocket example is built with special variable to work +without disabling xsrf verification. + +If you are using random input in your load tests, something that is +highly suggested for realistic numbers, you might end up in small +problems. The message format, by Atmosphere, has a weird number and "|" +in front of each message. That number tells the message length and it +must really match the real message length. Create a simple helper +function to calculate that if your input data length varies. + +[source,javascript] +.... +import io.gatling.core.session._ +import io.gatling.core.session.el._ + +def atmoMessage(message: Expression[String]) = message.map(m => m.length + '|' + m) + +.sendText(atmoMessage("SomeMessage")) +.... + +If (and when) you probably want to close the websocket connection +cleanly, you need to notify the server with an extra xhr with a +identifier given by the atmosphere framework. The key is the first +message that the server sends when you connect to it. + +Check out this script for +https://github.com/mstahv/v-quiz/blob/master/src/test/scala/loadtest/WebSocketVaadinSimulation.scala[an +example using WebSocket] communication. It also saves XSRF preventation +key to variable, so it don't need it to be disabled from the server. + +[[configuring-gatling-to-the-web-app-build]] +Configuring Gatling to the Web app build +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +It is a good habit to keep your tests in the same projects as your +actual application. Then you can easily verify your application still +scales, after you have for example written a cryptic SQL query. + +Even better if you can make a Gatling script to be executed during +builds to make. Gatling has http://gatling.io/docs/current/extensions/maven_plugin/[a +maven plugin] that can do exactly this thing. +https://github.com/mstahv/gatling-vaadin-example[The example project +setup] executes a test during basic "mvn install". With similar setup in +a real project, your CI server most likely saves results stored under +target directory. This way it is easy to check it out afterwards how the +performance of your application has evolved during its development. + +[[jmeter-vaadin-servlet-extension]] +JMeter Vaadin Servlet extension +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The implementation referred to in <<using-debug-ids-in-communication>> + +[source,java] +.... +package com.example.vaadin7jmeterservlet; + +import com.vaadin.server.ClientConnector; +import com.vaadin.server.DeploymentConfiguration; +import com.vaadin.server.ServiceException; +import com.vaadin.server.VaadinRequest; +import com.vaadin.server.VaadinService; +import com.vaadin.server.VaadinServlet; +import com.vaadin.server.VaadinServletService; +import com.vaadin.server.VaadinSession; +import com.vaadin.ui.Component; + +/** + * @author Marcus Hellberg (marcus@vaadin.com) + * Further modified by Johannes Tuikkala (johannes@vaadin.com) + */ +public class JMeterServlet extends VaadinServlet { + private static final long serialVersionUID = 898354532369443197L; + + public JMeterServlet() { + System.setProperty(getPackageName() + "." + "disable-xsrf-protection", + "true"); + } + + @Override + protected VaadinServletService createServletService( + DeploymentConfiguration deploymentConfiguration) + throws ServiceException { + JMeterService service = new JMeterService(this, deploymentConfiguration); + service.init(); + + return service; + } + + private String getPackageName() { + String pkgName; + final Package pkg = this.getClass().getPackage(); + if (pkg != null) { + pkgName = pkg.getName(); + } else { + final String className = this.getClass().getName(); + pkgName = new String(className.toCharArray(), 0, + className.lastIndexOf('.')); + } + return pkgName; + } + + public static class JMeterService extends VaadinServletService { + private static final long serialVersionUID = -5874716650679865909L; + + public JMeterService(VaadinServlet servlet, + DeploymentConfiguration deploymentConfiguration) + throws ServiceException { + super(servlet, deploymentConfiguration); + } + + @Override + protected VaadinSession createVaadinSession(VaadinRequest request) + throws ServiceException { + return new JMeterSession(this); + } + } + + public static class JMeterSession extends VaadinSession { + private static final long serialVersionUID = 4596901275146146127L; + + public JMeterSession(VaadinService service) { + super(service); + } + + @Override + public String createConnectorId(ClientConnector connector) { + if (connector instanceof Component) { + Component component = (Component) connector; + return component.getId() == null ? super + .createConnectorId(connector) : component.getId(); + } + return super.createConnectorId(connector); + } + } +} +.... diff --git a/documentation/articles/contents.asciidoc b/documentation/articles/contents.asciidoc index f3f9611485..dfbfd7c3be 100644 --- a/documentation/articles/contents.asciidoc +++ b/documentation/articles/contents.asciidoc @@ -57,4 +57,5 @@ are great, too. - link:IIInjectionAndScopes.asciidoc[II - Injection and scopes] - link:CreatingSecureVaadinApplicationsUsingJEE6.asciidoc[Creating secure Vaadin applications using JEE6] - link:UsingVaadinCDIWithJAASAuthentication.asciidoc[Using Vaadin CDI with JAAS authentication] +- link:LoadTestingWithGatling.asciidoc[Load testing with Gatling] - link:CreatingAUIExtension.asciidoc[Creating a UI extension] |