Execution

As mentioned before units have to be implemented in Java. A unit is a method in a class. For the framework to have access to the unit the class should implement the interface org.ivalidator.framework.test.Unit. But this is not a mandatory requirement. Mandatory requirements are the existance of a default constructor and that the method that represents the unit may not have any parameters. The constructor as well as the method must be visible for the framework.

The framework also supports classes rsp. methods that have been developed for JUnit. When a class extends junit.framework.TestCase the methods setUp() and tearDown() are called before and after the execution of the testmethod. (This may be deactivated via a configuration property)

A unit is executed in the following steps:

1. The framework looks up the class in the classpath for the testclasses (which has to be provided by the descriptor repository). If the class cannot be found the execution of the unit is aborted.
2. If the class is found, an instance is created with the default constructor. If that is not possible the execution of the unit is aborted.
3. If the instance is of the type junit.framework.TestCase the method setUp() of the created instance is called. For that reason JUnit must be found on the classpath for the testclasses. If an exeption is thrown during the execution of the method, execution is continued with step 6.
4. If the class implements the interface org.ivalidator.framework.test.Unit the method setContext(UnitContext) is called. The provided context is only valid during the runtime of the unit method. If an exeption is thrown during the execution of the method, execution is continued with step 6.
5. The framework tries to execute the unit method of the created instance. If an exeption is thrown during the execution of the method, execution is continued with step 6.
6. If the instance is of the type junit.framework.TestCase the method tearDown() of the created instance is called. For that reason JUnit must be found on the classpath for the testclasses.

A unit can communitcate 4 different error types to the framework:

• failure
• severe failure
• error
• severe error

Internally these error types are implemented as specific exceptions. All other exceptions that are thrown by the unit are treated as severe error. Only exceptions of the type junit.framework.AssertionFailedError are treated as severe failure. If one of the errors described in step (1) and (2) occurs they are called execution error.

Context

A context is provided for the unit during execution time. Via the context the unit has access to the framework. Methods for the following areas are provided by the context:

• Error handling
• Access to test parameters
• Access to test data
• Logging
• Adapter handling

Additionally a context can be asked which role the unit has during the current execution. The unit can be executed as Setup, Test, Checkup or Teardown.

All classes and interfaces of the framework that are used in the implementation of a unit can be found in the package org.ivalidator.framework.test.

Error Handling

In addition to the already mentioned error types a unit can also explicitly report a successful test execution to the framework. This can be used to pass information about the success on to the reporting.

For every error type the context provides three options to report the error to the framework. For a severe failure they are:

1. severeFailure(String message)
2. severeFailure(boolean condition, String message)
3. severeFailure(Throwable t)

Option (1) leads to an immediate error and the provided message is passed on to the reporting.
Option (2) only leads to an error if the provided condition evaluates to false.
Option (3) leads to an immediate error and the information for the reporting is taken from the provided exception.
To report a succesful test execution only option (3) is available (success(String message)).

The error types are represented by different RuntimeExceptions that are thrown by the methods. If these exceptions are caught within a unit they are not reported to the framework.

Access to Test Parameters

Parameters are defined in the test description and are therefore static for the complete test run. A parameter consists of a name and a value. The value may be any Java object. In addition to "simple" parameters the framework also provides parameter lists. The context provides the methods

• Parameter getParameter(String name)
• Parameterlist getParameterlist(String name)

to access the parameters and parameter lists. If no parameter rsp. parameter list with the given name is available the methods return null. getParameter() also returns null if the given name denotes a parameterlist and vice versa.

Parameter

The interface Parameter provides the method

• Object getValue()

to get the parameter value. Addionally Parameter offers methods to convert the value into the basic datatypes of Java:

• boolean asBoolean()
• byte asByte()
• char asChar()
• double asDouble()
• float asFloat()
• int asInt()
• long asLong()
• short asShort()
• String asString()

If a conversion fails the method throws a RuntimeException.

Finally there are 3 methods that facilitate the generation of another object from the parameter value:

1. Object asObject(Class clazz)
2. Object asObject(Class factory, String method)
3. Object asObject(Object factory, String method)

Option (1) tries to create an instance of class clazz by calling the constructor with a parameter of type Object with the parameter value as parameter.
Option (2) tries to create an instance by calling the static method method of the class factory with the parameter value as parameter.
Option (3) tries to call the method method of the provided object factory
All three methods throw RuntimeExceptions in case of an error.

Parameterlists

Parameterlists can group a set of parameters to one entry that can be accessed via an index or an iterator. The names of the parameters within the parameterlistentry must be unique.

The interface Parameterlist provides the following methods:

• public boolean isEmpty()
• public Parameters get(int index)
• public int size()
• public Iterator iterator()
• public Parameters[] toArray()

Parameters provides the methods

• Parameter getParameter(String name)
• Parameterlist getParameterlist(String name)

Here the same rules apply as for the methods the context of the unit provides.

Test data

Test data, unlike test parameters, can be created during the execution of a unit and then registered with the framework. From this time onwards the data is available for all subsequently executed units (except units within flows with parallel execution). Test data, like parameters, consist of a name/value pair. The value again can be any Java object.

The context of a unit provides the following methods:

• Object getData(String name)
• Object getData(int index, String name)
• void setData(String name, Object value)
• Object removeData(String name)
• void resetData()

for the maintenance of test data. The method setData(String name, Object value) registers test data with the framework. If test data of the same name already exist they are overwritten. The method Object getData(String name) allows access to already registered test data. null is returned if no data of the provided name exist. The methods Object removeData(String name) and resetData() deregister specific rsp. all test data with the framework. The former method returns the removed object.

During the parallel execution of a flow it may happen that units access the same test data. The order of those accesses cannot be predicted, but in any case the last update of test data wins. In order to prevent units in a parallel flow from influencing each other every test within the flow gets its own copy of the current test data before the flow execution starts. The method setData() affects the original als well as the copy. The method getData() only accesses the copy. The actual state of the test data after the execution of the flow depends on the random order in which the units have changed the test data. Change of test data in this cas only means the registration resp. deregistration of test data. For changes of the objects behind the test data the test developer is responsible. If simultanously executed units call changing methods of the same instance of an object they might influence each other after all.

After the execution of a flow the checkups of the suite are executed. If a unit takes the role of a checkup another option of getData() is available. This option expects a name and an index as parameters. If the unit does not take the role of a checkup the method acts, as if only the name was provided. The provided index refers to the tests of the executed flow in the order of their definition in the test description based on 0. The method returns the test data that was registered with the given name after the execution of the indexed test. Like this a checkup can evaluate test data that have been changed by a certain test even if subsequent tests have overwritten the data afterwards. If the flow was run in parallel mode the method returns the copy of the test data of the indexed test.

In case of an error during the execution of a test changes of the test data are rolled back.

Logging

The call to getLogger() returns an instance of the interface Logger. The framework uses this interface to capsulate the logging with Log4J. Accordingly methods are provided for logging on various levels. The catagory for the logger is derived from the concatenation of the prefix org.ivalidator, the role of the unit and the unit name from the test description.

Adapter Handling

The meaning of adapters is explained in detail in the section Adapter. Here only the access to adapters from a unit is described. Adapter, like test parameter and test data, are registered with the framework under a unique name and can be used via this name.

The UnitContext provides the following methods:

1. String[] getAdapterNames()
2. Adapter getAdapter(String name, Parameters parameterMap, Object initObj, AdapterFactory adapterFactory)
3. Adapter getAdapter(String name, String adapterName)
4. void removeAdapter(String name)
5. void removeAndCloseAdapter(String name)

Method (1) returns the names of all currently registered adapters.

Method (2) checks if an adapter with the given name is registered. If that is the case the adapter is returned. If this is not the case an instance is created via the provided adapterFactory with the given parameters parameterMap and initObj. The instance is registered with the framework under the name name and the adapter is returned to the calling unit. If no adapter could be created the method returns null.

Method (3) checks if an adapter with the given name is registered. If that is the case the adapter is returned. If this is not the case the framework checks the AdapterLib provided by the test description for an adapter template with the provided adapterName. If the template is found it is used to create an adapter, register it under the name name with the framework and return it to the calling unit. This method also returns null if no adapter can be created.

The framework distinguishes between the deregistration and the closing of an adapter. If an adapter is deregistered it is not known to the framework any more. If an adapter is closed it releases its resources. Method (4) deregisters an adapter with the framework without closing it. When a unit deregisters an adapter it is responsible for closing it. Method (5) deregisters an adapter with the framework and closes it at the same time. At the end of a test run all adapters are closed automatically.