Let’s have a look at how to automate the execution of your loadUI Test scenarios with a Continuous Integration tool; we will use Hudson in this example but any other CI-tool supporting command-line execution (luntbuild, cruise-control, bamboo, etc) should work just as well.

What is Continuous Automated Load Testing?

Usually when one speaks about test automation the execution of functional tests is in focus via tools like JUnit, etc. These tests are run repeatedly (for example every hour) to ensure that no regression errors have been introduced with the latest changes made to the system. Continuous automated load-testing has a similar goal; to detect regression errors in system performance while a system is being developed. Typically you set up your load tests to run a scenario that loads your system as defined by your performance requirements and then add a number of assertions that continuously validate the obtained performance; if your assertions fail so does your test and your continuous build tool will report this accordingly.

One major challenge specifically related to load testing is that a large number of factors will influence the results, especially when testing network based services (web sites, etc). If a backup job starts running on the server you are testing at the same time as your tests that will probably have a negative effect on performance. Likewise, if multiple load tests are being run simultaneously over the same network  you might have contention issues that will slow down the response times (although the server might be responding well within your defined limits).

The Initial loadUI Project

We’ll start by creating a simple loadUI project containing just a Generator and a Web Page Runner:

In the Settings dialog for the Web Page Runner we’ll make sure that server errors are reported as failures:

Also we’ve set a global limit of 60 seconds or 100 failures:

(Now when running the test it will fail if we get 100 assertion failures before the limit of 60 seconds is reached)

Be sure to save the project to a known location (use the Project menu “Save As” option) before moving on.

Automating with Hudson

Next up we’ll automate the execution of this project using the popular Hudson CI server, download and install it on the machine that you will use for running the tests and make sure loadUI is installed on that same machine also. In this example I will have the loadUI project locally on the same machine, but you could just as well have it stored in subversion or any other SCM system supported by Hudson.

In Hudson now create a new Job (“Free style software project”):

and configure it to run the loadUI command-line runner with our loadUI project every hour:

The command we have given is as follows:

"D:\workspace\loadui-project-1.0.2\loadui-installer\target\controller\loadui-cmd.bat" -p "c:\dev\loadui-stuff\test-cmdline-project.xml"

Which first is the path to the command-line runner and then our loadUI project file (specified with the –p argument, see the command-line reference for a complete list of arguments)

After created we directly run the build in Hudson and use its Console-Output functionality to follow the execution of the test; start by pressing the “Build Now” button in the project overview:

You will see an entry being created in the “Build History” at the bottom left; click the entry which will take you to the Build screen where you can click to see the Console Output of the loadUI Runner:

The output to the right will be continuously updated and (hopefully) finish with the following rows:

Great! Now that we have the test up and running lets go back to loadUI and add some assertions that validate the avg response times of our service.

Adding Assertions

As described on the loadUI Website (see Validating Results), assertions can be used to validate the output of most loadUI components. In our test we want to ensure that the average response time is never above 50 ms, for this we need to add both a Statistics Component (that calculates the avg) and an Assertion (that does the validation). After doing this the project looks as follows:

The added parts are:

  • A Statistics component that is connected to the left output of the Web Page Runner which will allow it calculate performance related statistics
  • An Assertion component that is connected to the output of the Statistics component and configured to assert the avg value to be between 0 and 50 ms

Now when running the test we can see that a number of failures are immediately logged both on the Assertion component and in the global Failures counter:

Which eventually results in the test ending before the set time limit is up:

Now save your project and go back to Hudson and add an email notification for test failures (on the bottom of the Job Configuration screen):

(be sure to configure your SMTP server in the global Hudson configuration screen)

Now when we run the build in Hudson we see the following in the console output at the end of the run:

And we get a corresponding email in our inbox due to the failure:

Creating Reports

Finally let’s configure the job to create a PDF for each execution; add the required –r and –F arguments for this to the loadUI-cmd.bat invocation in your build configuration:

Now when running your test you will get a PDF created into the specified folder, open it to see the generated Summary Report:

Wrapping Up

That’s it, your continuous loadUI Load Test is up and running now ready to notify if performance falls below the defined levels. As mentioned before, setting this up with other CI tools should be equally simple, keep an eye on the loadUI website for specific configuration examples for some of the more popular tools out there.

原文地址：http://www.dzone.com/links/r/automating_loadui_tests_with_hudson_eviware_blog.html。