Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.
Data-driven testing is a pretty well-known subject. We’ve all heard about the advantanges of keeping your test data in files that you can design and maintain independently of how you execute the tests. With manual testing, you can choose how far you go down the data-driven path. When it comes to testing APIs, your tests are likely to be data-driven whether you intend them to be or not. The question really centres on how effectively you manage the separation between the elements of test data, configuration data, and implementation logic.
One of the main differences between SmartBear Software’s commercial Ready!API software and their open source product SoapUI is the built-in support in Ready!API for data-driven testing. Ready!API includes a number of features that allow you to store and access data, pass the data to a series of SOAP or REST requests, and write the results of the responses into a report. Typically these tests run iteratively over one or more API calls.
The SoapUI community has recognised the value of this style of testing and several online resources have emerged describing how to implement data-driven frameworks in the open source versions of SoapUI. I want to build on this work by describing a framework with the following features:
The articles in mysoapui.net offer a number of approaches to scripting that can you can use to implement a data-driven framework in SoapUI. You can select from whichever approach meets your needs.
In preparing these articles, I’ve used SoapUI Open-Source 5.3.0 under Linux and Windows.
I’m going to assume in this article that you are conversant with the following features of SoapUI:
I listed as a requirement that the framework will offer support for multiple data formats. Let’s take a look at some of the most common, and identify some of points of comparison between them.
|Text – comma-separated values||Compact, easy to edit if the amount of data is small||Relies on each row using the columns in the same way|
|JSON||Fairly compact, good support for hierarchical data||May not be easy to edit|
|XML||Many tools available for editing||Can be verbose|
|Excel||Can format data in a human-readable way||Requires spreadsheet software to read and write|
These articles will show how to keep the processing of data in these data formats separate from the execution of the test steps in the test cases.
These articles also show how to implement data-driven test cases where the test data is retrieved from a database.
I mentioned that the framework will be maintainable across large numbers of test cases. There are several scripting techniques we can use to achieve this.
|Procedural scripts stored as a Groovy test step in the SoapUI project file||Easily accessible; easy to modify small scripts in single instances.||Difficult to maintain when used often|
|Class-based scripts accessed via the GroovyScriptEngine.||The same scripting logic can be accessed from within large numbers of test cases and across different SoapUI projects.||Each file can expose only one class. Base classes may need to be duplicated.|
|Compiled scripts stored in JAR files||The same scripting logic can be accessed from within large numbers of test cases||Can be used to develop large frameworks in an extensible manner.
Requires knowledge of Java / Groovy compilation techniques.
Requires a re-start of SoapUI when a new JAR file is compiled.
mysoapui.net shows examples of each of these techniques so that you can select the approach that best meets your needs. First, though, we need to create a service definition in order to support our test cases.