My SoapUI Thoughts on Effective Testing in SoapUI

Procedural Scripting

For this example, I’m going to implement a procedural script as a test step in the Posts test case.

Prerequisites

  • You have created a test case that creates, reads, updates and deletes post resources in your SoapUI project

Goals

  • A groovy script that implements data-driven testing via data in comma-separated value (CSV) format
  • An understanding of how to separate test data from the processing logic

Insert a Groovy test step called Datasource Driver as the first test step in the test case, and copy the following code into it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import com.eviware.soapui.impl.wsdl.teststeps.*
import com.eviware.soapui.impl.wsdl.teststeps.registry.PropertiesStepFactory

final def tc = testRunner.testCase,
    propsStep = tc.getTestStepByName( "Props" ) ?:
        tc.addTestStep( PropertiesStepFactory.PROPERTIES_TYPE, "Props" ),
    testSteps = tc.getTestStepsOfType( RestTestRequestStep.class )

try {
	final def fileName = context.expand( '${projectDir}' ) + '/' +
            tc.getPropertyValue( "PropertiesFile" ),
        lines = new File( fileName )?.readLines(),
        names = lines[ 0 ]?.split( ',' )

	( 1 ..< lines.size ).each { rowNum ->
        final def values = lines[ rowNum ].split( ',' )
        names.eachWithIndex { it, i -> 
            propsStep.setPropertyValue( it, values[ i ] ) 
        }
        testSteps.each { 
            it.disabled ? null : testRunner.runTestStep( it ) 
        }
    }
}
catch( Exception e ) { testRunner.cancel( "$e" ) }
testRunner.gotoStepByName( propsStep.name )

Believe it or not, but in these few lines of code we have the fundamentals of data-driven processing. This code snippet meets two of our four original criteria for a successful framework: independence from the test data, and independence from the test steps. Let’s look at a sample of test data that this script can process.

The Test Data

For this example, I’ve stored my test data in a file called posts.csv as comma-separated values as follows.

id,userId,title,body
1,1,Title one,The
1,1,Title two,quick
1,1,Title three,brown
2,1,Title four,fox
3,1,Title five,jumps
5,1,Title six,over
8,1,Title seven,the
9,1,Title eight,lazy
10,1,Title nine,dog.

Note that the file contains no blank lines, and that the dot after the word ‘dog’ is the last character in the file. This file should be in the same directory as the SoapUI project file.

The first row of data contains the field names which will be used when processing each subsequent row. Let’s review the description of the API and describe how these field names relate to it. Looking at Creating a resource we see that the POST method expects the following fields in the request:

data: {
    title: 'foo',
    body: 'bar',
    userId: 1
}

Our CSV data includes a column for each of the title, body and userId fields. The order in which the fields appear is not important. The CSV data also includes as its first field a record Id which in a real-world test would probably need to be unique. For the purposes of this test any value will suffice.

Data-driven tests also often include a field called something like “ExpectedResult”. The values in this field can be used to verify the result of some processing or calculation by the server and aren’t submitted as part of the request. The API we’re testing here merely returns in the response the same values submitted in the request, so we don’t need an “ExpectedResult” column.

Our next task is to run the test case and test steps to call the four HTTP methods.

Running the test case

When you run the test case, you should see the following behaviour:

  • The first time the test case runs with this script as one of the steps, a properties step called “Props” is automatically created.
  • If you open the “Props” step and leave it open and visible while running the test case a second time, you’ll see its properties update with the values in each successive line in the CSV file for each iteration of the test case.
  • You can experiment by adding or removing rows from the props.csv file and observing the test case execute with the new data. Recall that the file should contain no blank lines; while it’s easy to enhance the script to accommodate blank lines, I’ve elected to prioritise keeping the script simple to simplify the description.

Assertions

Before going into the Datasource Driver code in depth, we should take the time now to complete the REST test steps by adding assertions. Without assertions, the steps have no value – but now that the test steps have run and have returned responses, we can examine the POST step to see what elements our tests should attempt to validate.

The POST response

Looking at the raw response from the POST step, we see the following information.

HTTP/1.1 201 Created
Date: Mon, 26 Dec 2016 09:30:49 GMT
.
.
.
Etag: W/"49-Qh4B0oMUV6YIjffCClBJyQ"

{
  "title": "Title nine",
  "body": "dog.",
  "userId": 1,
  "id": 101
}

(I have omitted several lines of the header for brevity.) This raw response includes the fields we submitted in the request, so they form a good basis for JSONPath assertions. If the response contains data other than what we submitted, we should assume something went wrong. We can assert that the title field contains the value specified in the test data and contained in the Props step as follows: TitleAssertion01 The use of the ${Props#title} property as the Expected Result means that the assertion will always compare the response against the current corresponding data value. And having created this assertion, it’s good practice to rename it from the generic “JsonPath Match” to something specific, such as “title”.

Now that the test step contains an assertion for the one of the data fields, we should also consider adding assertions for the HTTP response code and response time. These types of assertions are easy to create and help to ensure that your tests capture any change to your API’s performance. Let’s go ahead and add an HTTP Status Code assertion to check that future calls to the POST request return a 201 Created code, and a Service Level Agreement assertion to check that the calls return in under 1000ms. HTTPAssertion01 SLAAssertion01

After renaming the assertions from SoapUI’s default names, the assertion list on the POST test step should look like this: POSTAssertions01

Assertions in the remaining steps

Now that the assertions are complete for the POST step, I’m going to summarise the assertions I’ve created for the other steps in the test case.

Test Step Assertion Type Expression Expected result
GET JSONPath Match $.id ${Props#id}
PUT JSONPath Match $.title ${Props#title}
DELETE HTTP Status Code n/a 200

You could create more assertions for these test steps, but these are sufficient for our purposes.

At this point, you might want to show the window for the Props test step and execute the test case again, watching as the properties are updated with the values from the props.csv file. If everything has been correctly set up, each test step should run and all the assertions should pass.

Now is a good time to save your SoapUI project file before we (finally!) get into the heart of the processing – the Datasource Driver script.

The script in depth

I’m going to describe the processing of this script by means of a line-by-line walkthrough. Along the way, I’ll mention some techniques in Groovy scripting and provide some links to a very comprehensive blog if you want to explore these concepts in more detail.

Lines 1 - 2

import com.eviware.soapui.impl.wsdl.teststeps.*
import com.eviware.soapui.impl.wsdl.teststeps.registry.PropertiesStepFactory

These import statements allow the script to refer to REST test steps and Property test steps.

Line 4

final def tc = testRunner.testCase,

This declaration allows the script to reference the containing testCase by means of a concise variable name tc. We’ll use this frequently so it’s worth making a constant with a short name for it.

Line 5

propsStep = tc.getTestStepByName( "Props" ) ?:

Access the existing Properties test step in the test case which has the name ‘Props’. Groovy’s Elvis operator ?: conditionally leads on to Line 6 if such a step doesn’t exist.

Line 6

tc.addTestStep( PropertiesStepFactory.PROPERTIES_TYPE, "Props" ),

If the test case does not contain a Properties test step called ‘Props’, then create this step. The step will be appended to any existing steps in the test case.

Line 7

testSteps = tc.getTestStepsOfType( RestTestRequestStep.class )

Access a list of all the test steps in the test case of type REST request. Line 16 will iterate over these steps when the test data has been collated.

Line 9

	try {

As the script is about to read a file, and thereby pass control to the operating system, it’s good practice to enclose the following logic in a try / catch block. This form of exception handling can cater for the case where the user specifies a file name that doesn’t exist, for example.

Lines 10-11

final def fileName = context.expand( '${projectDir}' ) + '/' + 
				tc.getPropertyValue( "PropertiesFile" ),

Assemble the datafile name from the path of the SoapUI project folder together with the PropertiesFile property specified on the test case.

Line 12

lines = new File( fileName )?.readLines(),

Read every line from the datafile into an array of strings called lines.

Line 13

names = lines[ 0 ]?.split( ',' )

Parse the first line of the file into an array of property names, using the comma character as the separator. Line 15 will use these property names to create entries in the ‘Props’ test step.

Line 15

( 1 ..< lines.size ).each { rowNum ->

This syntax is an example of deriving a loop via a Groovy closure based on a numeric range. It creates a loop with a control variable called rowNum, whose values increment from 1 through to one less than the number of items in the lines array.

I could also have written this loop control statement in either of the following ways:

  • for( int rowNum in 1 .. lines.size - 1 ) {
  • for( int rowNum = 1; rowNum < lines.size; rowNum++ ) {

But I’m happy to stick with Groovy’s idioms where possible.

Line 16

final def values = lines[ rowNum ].split( ',' )

Parse the next line of the file into an array of property values, using the comma character as the separator. The following line in the script will use these values to populate the entries in the ‘Props’ test step.

Lines 17-19

names.eachWithIndex { it, i -> propsStep.setPropertyValue( it, values[ i ] ) }

This is an example of Groovy’s eachWithIndex closure. This closure iterates over the list of property names and assigns the values 0, 1, 2 … to the variable i each time it executes, allowing us to index each item in the values array. The setPropertyValue statement creates a property in the ‘Props’ step for each name and with the corresponding value from the array of values created in Line 14.

In the real world, this line should accommodate blank lines in the datafile. I’ll show you how to check for this in a class-based implementation.

Lines 20-22

testSteps.each { it.disabled ? null : testRunner.runTestStep( it ) }

This closure iterates over the list of REST test steps declared in Line 6, running all steps that aren’t disabled.

Line 25

catch( Exception e ) { testRunner.cancel( "$e" ) }

This catch block cancels the test case in response to any errors thrown in the try block.

Line 26

testRunner.gotoStepByName( propsStep.name )

This last line requires a little explanation. When the test case is executed, the script will run all the REST test steps in the test case for each line in the datafile. When the script exits, SoapUI will then move on to the next step in the test case, being the first of the REST test steps. In other words, the REST steps will run twice on the last row of the test data – firstly under the control of the script, and secondly under the control of the test case itself.

This (probably) isn’t what we want. To prevent this, the script sets the execution point to the properties step – after all the REST steps – before exiting. As the current test step is now set to the last step in the test case, the REST steps won’t run twice.

At this point everything should be set up and you should now be able to run the test case, watching as the test steps execute once for each line in the datafile.

Keeping everything separate

You’ll have noticed that the test script does not refer to any of the test steps, test data, or requests by name. In the same way, the test steps are also independent of the scripting logic. Using the ‘Props’ step as an intermediary allows us to preserve the independence of all the elements of our data-driven framework.

Let’s now reap the benefits of this approach by creating a test case for the todos resource.