My SoapUI Thoughts on Effective Testing in SoapUI

Scripting with Classes

We’ve seen that one test case can be easily cloned to run against other resource types, but in doing so we’ve duplicated the Datasource Driver script. This practice has several quite significant limitations.

  • When any enhancements are required to the script (for example, to accommodate blank lines in the datafiles) you’d need to make changes in every copy of the script.
  • As your scripts become more sophisticated, they can become cumbersome to edit in SoapUI’s Groovy script test step editor.
  • Over time, you’re likely to end up with a very large SoapUI project file. This could have implications if you’re working in a team environment where more than one person needs to update the project file.

These limitations aren’t sustainable, so I’d like to introduce a couple of ways that a script can be accessed by more than one test case.

Prerequisites

  • A test case configured to test the todos resource
  • An understanding of object-oriented concepts (but not necessarily via Groovy)

Goals

  • An understanding of how to create and invoke classes defined in external scripts

Other approaches to script re-use in SoapUI

Before continuing, I’d like to mention that some suggested solutions floating around online to the question of script re-use (including, unfortunately, on the SmartBear community forum) have the feel of workarounds and should probably be avoided. One particularly tenacious approach involves the use of a separate test suite containing the scripts you want to re-use. You’re then supposed to access these scripts from the calling test case by wandering up and down the project hierarchy via cumbersome constructions like this:

final def scriptLibrary = testRunner.testCase.testSuite.project.testSuites["ScriptLibrary"],
	  myClass = scriptLibrary.testCases["myFunctionalArea"].testSteps["myClass"]
myClass.run( testRunner, context )

I don’t recommend this approach. Apart from anything else, you’ll still need to duplicate your scripts across project files. For truly generic techniques like data-driven testing, that’s exactly what you want to avoid doing. This approach also locks you in to SoapUI’s existing testcase / testsuite / project hierarchy. Who’s to say that future versions of SoapUI won’t introduce another level to the hierarchy?

For me, though, the main lack of appeal in storing common scripts in a separate test suite is that Groovy itself offers well-understood, scalable techniques that don’t rely on SoapUI at all. They are specifically designed to facilitate code re-use, and absolutely constitute the right tool for this particular job.

External scripts in Groovy

The Pro versions of SoapUI (and Ready!API) include built-in support for script libraries which aren’t available in the open-source version of SoapUI. Fortunately, Groovy itself also provides its own approach to accessing external scripts which allows us to side-step this limitation. My experience has been that using Groovy’s approach can actually be preferable to using the features in the Pro versions of SoapUI. Consider the case where you have one SoapUI licence but want to run your tests on three other machines. If you implement the tests in such a way as to avoid any of the features in the Pro version, you can run them wherever you like.

First though, we need to implement our script as a class in order to invoke it from more than one location.

Getting Classy

Given that we have already implemented data-driven processing via a procedural script, the simplest way to create a class to do the same thing would be to copy the script into the constructor of a class almost without modification. But given that we need to move to a class-based implementation, I’d like to extend the design a little and show the advantanges of object-oriented inheritance in implementing support for different datafile formats. We’ve looked at CSV data, and through inheritance we can address other datafile formats including JSON, XML, and Excel. I want to show how a derived class can be used to process different datafile formats, and the base class can implement the common functionality relating to invoking the test steps and marshalling the results.

The Base class – Datasource

I’ll show you how to include this code in a file shortly. For now, though, let’s walk through the features of our base class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
abstract class Datasource {
	Datasource( log, testRunner, context )
	{
		this.testRunner 	= testRunner
		this.context		= context
		this.log		= log
		testData		= []

		fileName = context.expand( '${projectDir}' ) + '/' + 						testRunner.testCase.getPropertyValue( "PropertiesFile" )			
	}

	abstract boolean initialise()

	public void runTests()
	{
		final def tc = testRunner.testCase,
			propsStep = tc.getTestStepByName( "Props" ) ?:
			tc.addTestStep( PropertiesStepFactory.PROPERTIES_TYPE, "Props" ),
				testSteps = tc.getTestStepsOfType( RestTestRequestStep.class )
		testSteps.removeAll{ it.disabled }

		// Populate the properties step and run the enabled test steps.
		final def names = testData[ 0 ]
		( 1 ..< testData.size ).each { rowNo ->
			testData[ rowNo ].eachWithIndex{ it, i -> propsStep.setPropertyValue( 
				names[ i ], it ) }

			final def logProp = propsStep.getPropertyList()?.getAt( 0 )
			testSteps.each { 
				final def stepName = it.name
				it.name += " (${logProp?.name} = ${logProp?.value})"
				testRunner.runTestStep( it )
				sleep( 20 )
				it.name = stepName
			}
		}

		testRunner.gotoStepByName( propsStep.name )
	}

	public void tearDown() {}

	protected testRunner
	protected context
	protected log 
	protected testData
	protected fileName
}

Let’s break this down.

I’ve defined the Datasource class as abstract because it can only be instantiated indirectly via a derived class in order to process a specific type of datafile. There’s only one constructor which takes the parameters provided by the calling script, including the test case context. Even though I use the context only to locate the full path to the datafile name and nowhere thereafter outside the constructor, I still store it as a member of the class. I’ll discuss the motivation for doing this when I describe the way in which the class is invoked from the test cases.

Lastly in the constructor, you’ll see a member called testData, initialised as an empty list. testData is key to the functioning of this class because it will be populated with a list of lists assembled from the name / value pairs from each line in the datafile. The first list in testData will contain the property names, and each subsequent list will contain the property values from a line in the datafile.

Moving on from the constructor, there’s an abstract method called initialise(). Derived classes must implement this method because this is where they will provide the processing for the datafile format types.

The runTests() method should look very familiar from our procedural script shown here. It creates the ‘Props’ step, populates the values in the step with the values in each row of data, and invokes the REST test steps that aren’t disabled. Line 20 uses Groovy’s removeAll closure to detect and remove disabled steps from the list.

Setting the Test Step name

Lines 30-34 surround the call to run the test step. Several examples online of data-driven scripting in SoapUI have included the ability to update the test step name as the test case executes with some distinguishing feature. In this case, I’ve used the name and value of the first property in the ‘Props’ step to assemble a string which we can append to the test case name. As the steps run, they will temporarily change their name to e.g. PUT (id = 3277). Apart from giving you something to look at while the test case runs, having a unique name for each of the test steps facilitates the review of the log when the test case has finished. You can tell exactly which step ran with which data, and relate any failed assertions back to the data that caused the problem.

You may notice a call to sleep() when the test steps are renamed. I’ve found that the SoapUI GUI doesn’t always update in time for the new name to be applied, so the sleep() call is a workaround. (And yes, I’m aware that a worker thread should communicate with the GUI via a more mature mechanism than simply calling sleep(). But let’s weigh the cost of such an implementation against our overall goal here.)

The last thing I do in the Datasource class is provide a tearDown() stub. Derived classes can override this method if they wish. We’ll look at this when we implement a datasink.

With our base class in place, let’s do some subclassing.

The Derived class - CSVDatasource

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
@InheritConstructors
class CSVDatasource extends Datasource
{
	boolean initialise()
	{
		try {
			final def lines = new File( fileName )?.readLines()
			( 0 ..< lines?.size ).each { lineNum ->
				final def values = lines[ lineNum ]?.split( ',' )
				if( values[ 0 ]?.trim() ) {
					testData.add( values )
				}
			}
		}
		catch( Exception e ) { testRunner.cancel( "$e" ) }
		true
	}
}

Groovy has a nifty annotation called InheritConstructors. This allows derived classes to reuse the constructors of the base class without needing to implement them themselves.

The derived class contains only one method: initialise(). In our case, it’s pretty similar to the procedural implementation we’ve already seen, but with a couple of minor differences. Let me take you through the different code.

The loop processes each line of the file and makes a list from each line. If the first value in the list is not empty, it adds the list to the testData class member, forming a list of lists. The base class will iterate over the testData list, using the first member list as names, and assigning the members of the remaining lists as values in the ‘Props’ test step.

The initialise() method then exits by returning true, idiomatically omitting the return statment. In other implementations, the method could return false if necessary to indicate to the calling class that something went wrong and that processing should not proceed. This gives a bit of extra flexibility if for some reason you don’t want to throw your own exception.

Putting the classes together

Now that we know what the classes do, how should they appear in a file that SoapUI can execute? You need to create a file called CSVDatasourceFull.groovy, laid out as follows, and saved in the same directory as your SoapUI project file. The Groovy script engine will look for the first class in the file, so in this case our derived class must appear before the base class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
/////////////////////////////////////////
//	Datasource driver - an opensource driver implementation.
//	Read the properties provided in CSV format.
//

import groovy.transform.InheritConstructors
import com.eviware.soapui.impl.wsdl.teststeps.*
import com.eviware.soapui.impl.wsdl.teststeps.registry.PropertiesStepFactory

@InheritConstructors
class CSVDatasource extends Datasource
{
	.
	.	// As shown above
	.
}

abstract class Datasource
{
	.
	.	// As shown above
	.
}

You can see that I’ve abbreviated the contents of the classes here, but you should copy them directly from the examples shown earlier.

With the CSVDatasourceFull.groovy file saved in your SoapUI project directory, there’s only one task remaining.

Invoking the CSVDatasource class from a script

Create a clone of the Todos test case and call it e.g. “Todos – external script”. Delete the existing datasource driver test step and replace it with a new Groovy test step with the following contents.

1
2
3
4
5
6
7
8
9
10
11
final def projectDir = context.expand( '${projectDir}' ),
		engine	 = new GroovyScriptEngine( ( String[] ) [ projectDir ] ),
		datasource = engine.loadScriptByName( 'CSVDatasourceFull.groovy' ).
			newInstance( log, testRunner, context )

datasource.with {
	if( initialise() ) {
		runTests()
		tearDown()
	}
}

As you can see, invoking our class requires a fairly brief script. I’ve located the script file via the project directory here for simplicity but for your needs you would probably want to refer to a separate directory so that all your projects can access it. My practice is to create an application-level property (in SoapUI: File → Preferences → Global Properties) so that all your scripts can be centralised.

The instance of the GroovyScriptEngine then takes the name of the script file and makes a class out of it, on which you can then invoke the methods you’ve defined.

You’ll recall that I mentioned that the base class keeps a copy of the context parameter and declares a tearDown() method, even though it doesn’t use either of them here. The rationale is to provide an interface that is as generic as possible, so that the invoking script won’t need to change even though it works with different classes. If you wanted to, you could even store the name of the script file as a property on the test case, and pass it to the script engine via a context property. This would allow you to invoke your classes through the same calling script.

With the new Groovy test step in place, you should be able to run the test case and get the results below, noticing that the log now contains unique names for each test step. The script has renamed each step with the id field from the datafile. TODOsTC01 With our class-based script in place, let’s explore the testing of some of the other HTTP methods included in our service definition.