Home Repositories XML Repository
XML Repository

Configuration of the XML Repository

The description of the tests, the constitution of test scenarios as well as the supply of parameters for the test units is done separatly from the actual test implementation through a repository. This must implement certain interfaces of the framework to supply the necessary information during the test run.

The default repository of the iValidator is the xml-repository for the test descriptions (Descriptor Repository). Other repositories (e.g. database) can be implemented. Further information will be subject of the developers guide. This guide explains how to describe test scenarios using the XML Repository.

Configuration for the XML Descriptor Repository

To start a test run with an xml repository the following entries must be made in the test configuration file. 

    <descriptor repository="xml">
        <property name="inputpath" value="input"/>
        <property name="input" value="testsuite_simple.xml"/>
        <property name="classpath" value="classes"/>
    </descriptor>

The repository attribute denominates the repository type as xml. The xml descriptor repository expects the following properties:

Property m/o Description
inputpath o Path where the xml repository looks for the test description and, if applicable, imports. The path can be absolute or relative to the start directory of the TestRunner. (NB: Not relative to the configuration directory!)
input m URL or filename of the test description to be read by the xml repository.
classpath m Classpath for the test classes.
schematest
o
true/false. Validate the test description to the xml-schema. Default is false.

The xml repository processes a test description in xml format. This special input file for the xml repository will be refered to XML-Descriptor-Repository according to the name of the root element for the remainder of this document.

Configuration of the XML Report Repository

To store the reports of a test run in xml format the following must be defined in the configuration file:

   <report repository="xml">
        <property name="outputdir" value="results"/>
        <property name="stylesheet.use" value="true"/>
        <property name="stylesheet.xsl" value="my_report.xsl"/>
        <property name="stylesheet.copy" value="my_report.xsl;report.css;plus.gif;minus.gif;title-logo.gif"/>
    </report>

The repository attribute defines the repository type xml. The xml report repository expects the following properties:

Property m/o Description
outputdir o Directory into which the xml repository writes the report files. The path can be absolute or relative to the start directory of the TestRunner. (NB: Not relative to the configuration directory!) If no outputdir is supplied the reports will not be written. This does not result in an error.
stylesheet.use o If you only set the property 'stylesheet.use=true', the reports will be generated with a standard stylesheet which is delivered with the iValidator. To involve your own stylesheet set the property 'stylesheet.xsl'.
stylesheet.xsl o This will generate the line

<?xml-stylesheet type="text/xsl" href="/[stylesheet.xsl]"?>

in your report file.
stylesheet.copy o With the property 'stylesheet.copy' you specify all needed files. These files will be copied to the result directory.
Default: "report.xsl;report.css;plus.gif;minus.gif;title-logo.gif"

The xml repository creates a report file in xml format. This special output format will be refered to as XML-Report-Repository for the remainder of this document.

The XML-Descriptor-Repository <descriptor-repository>

The structure of the xml descriptor repository is defined with a W3C Schema xml-repository.xsd.

The root element is the <descriptor-repository>. Then the following elements may occur optionally and in any order:
<settings/>
<test/>
<parameterlib/>
<unitlib/>
<adapterlib/>

The interfaces of the iValidator framework expect the top-level test descriptor and an adapterlist. The parameter library and unit library can be used in the xml repository for a better structure of the descriptions. The tests' references to the library will be resolved during the import of the repository at the begin of the test run.

Not all elements must be defined in a repository. An import mechanism helps to divide the definitions into several xml files for better readability. Only the "main" descriptor repository (the one that is given as input property in the configuration file) must contain at least the <test> element.

The Test Descriptor: <test>

The xml descriptor repository does not explicitly distinguish test suites and test units. Both are described within the <test> element. The <test> elements may be nested unlimited to build the test hierarchy.

Description of a test unit

Test units contain either an alias attribut as a reference to the unitlib or an instance element.

Attribute m/o Description
name o A test name must be supplied if no alias is supplied. It can be supplied in addition to an alias.
A test name may not contain the symbols '.' and '$' because they are used during the test run to create the fully qualified test names.
alias o The alias references the unit definition in the unit lib of the same name. If no additional name attribute is supplied, the test unit gets the name of the unit in the unit lib, that is the alias.
Element m/o Description
parameters o A test unit may contain a <parameters> element. The parameters defined in the test unit overwrite those defined in the unit lib. A detailed description of the <parameters> element can be found in the test suite section.
instance o As an alternative to referencing the unit lib with the alias attribute, the test unit instance can be defined explicity with class and methodname.

Examples:

    <test name="myTestUnit">
       <instance class="org.ivalidator.testapp.test.LoginTest" method="testLogin"/>
    </test>

The instance of the test unit is defined with the instance element. Better practice is the definition in the unit lib to separate the implementation details from the description of the test hierarchie and thus get a higher reusablity.

    <test alias="loginTest"/>

The test unit is defined in the unit lib of the repository. 

    <test name="myLoginTest" alias="LoginTest"/>

The test unit is defined in the unit lib of the repository. During the test run it will be named differently.

Description of a test suite

Test suites contain at least one flow element. Additionally setups, teardowns and checkups can be specified. The flow element may contain any number of test elements each of which can be a suite or unit. Like that a test hierarchy of unlimited depth (theoretical) can be constructed.

Attribute m/o Description
name o A test name must be supplied if no alias is supplied. It can be supplied in addition to an alias.
A test name may not contain the symbols '.' and '$' because they are used during the test run to create the fully qualified test names.
alias o The alias attribute for a test suite can be used in connection with the import attribute. If an import is specified the alias can be used to reference a specific test description within the imported descriptor repository. The alias contains the fully qualified name of the referenced test, that is the names of all test suites leading to the one in question concatenated with separating dots.
A test suite cannot have an alias that references the unit lib because then it would be a test unit.
import o A test may have an import attribute that references another descriptor repository for the further test description. Like this complex test hierarchies can be structured more concisely. The imported descriptor repository will be looked for relatively to the input path of the "main" descriptor repository as specified in the configuration file. If no alias attribute is specified the top-level test in the imported repository is referenced.

Example for import:

	main-repository.xml:
	<descriptor-repository>
		<test name="main">
			<flow>
				<test name="myTest" alias="test1.test12" import="test1-descriptor.xml"/>
				<test name="myNextTest"/>
			</flow>
			<checkups>
				<checkup alias="was_everything_okay"/>
			</checkups>
		</test>
	</descriptor-repository>
	
	test1-descriptor.xml:
	<descriptor-repository>
		<test name="test1">
			<flow>
				<test name="test11">...</test>
				<test name="test12">...</test>
			</flow>
		</test>
	</descriptor-repository>

Element m/o Description
parameters o A test unit may contain a <parameters> element. The parameters defined in the test unit overwrite those defined in the unit lib. A detailed description of the <parameters> element can be found in the test suite section.
setups o The element contains a list of units whose implementation is either defined with the instance-element or with an alias-attribute reference to the unit lib. The setup units may contain parameters. Basically setup-, checkup- and teardown-units are not different from test units.
flow m The flow element may contain as many test elements as wanted that may again be suites or units.
checkups o See <setups> element
teardowns o >See <setups> element

The <parameters> element

There are 2 alternatives for the specification of parameters, namely plain parameters as name/value pairs and parameterlists. The parameters can be specified explicitly with the unit or test definitions or within the parameter lib as parametergroups that can be referenced by an alias.

Within the <parameters> element the elements <parameter>, <parameterlist> and <parametergroup> may occur in any order.

It's a basic setting of the iValidator framework in which order the parameters in the test hierarchy take precedence. The setting is made in the configuration file.

     <property name="ivalidator.parentparamsfirst" value="true"/>

parentparamsfirst="true" means that the parameters of any parent test suite overwrite those defined in unterlying tests.
With parentparamsfirst="false" (default setting) the parameters of a parent test suite are only relevant if there's no other declaration on lower levels.

Element m/o Description
parameter o An unlimited number of parameters in the form of name / value pairs can be provided.
The parameter value can be checked by the implementing unit(see validation run).
parameterlist o An unlimited number of parameterlists can be provided. A parameterlist is identified by its unique name. It has 0-n list entries that constitute groups of parameters with equal names and different values. The entries are identified implicitly by their index in the parameterlist. The validation run can be used to verify that the contents of the parameter list entries are valid.
parametergroup o The parametergroup within the xml test descriptions is used only to reference the corresponding parametergroups in the parameter lib. The element parametergroup in this context must (and must only) contain an alias attribute. All the other attributes and elements that the schema allows only apply to parametergroups within the parameterlist.
If for a test parametergroups are provided as well as parameters with the same names then the parameters defined with the test take precedence over those defined in the parametergroup in the parameter lib.

Example:

	<parameters>
		<parameter name="paramname" value="paramvalue"/>
		<parameterlist name="paramlist">
			<parameterlistentry>
				<parameter name="param1" value="value11"/>
				<parameter name="param2" value="value12"/>
			</parameterlistentry>
			<parameterlistentry>
				<parameter name="param1" value="value21"/>
				<parameter name="param2" value="value22"/>
			</parameterlistentry>
			<parameterlistentry>
				<parameter name="param1" value="value31"/>
				<parameter name="param2" value="value32"/>
			</parameterlistentry>
		</parameterlist>
		<parametergroup alias="parametergroup1"/>
		<parametergroup alias="parametergroup2"/>
		...
	</parameters>

The <flow> element

The flow element contains all the tests of the suite.

Element m/o Description
test o A flow may contain an unlimited amount of test elements. Every test again corresponds to the complete test description, can be a unit or a suite, may contain parameters, setups, checkups, teardowns, my be imported from another repository etc.
control o Addionally various control elements can be defined within a flow. Control elements have a few functions concerning process control and error handling. A control element has the following 3 attributes: type (mandatory), action (optional), value (mandatory)

The 2 following control types are available at the moment:

1. process control

	<control type="flowexecution" value="[parallel | sequential]"/>

By default tests within a flow are processed sequentially.

2. result control

	<control type="result" 
		value="[success | error | severe_error | failure | severe_failure]" 
		action="[next_test_inits | next_test_noinits | next_flow | stop]"/>

The defaults are described in the unit implementation guide.


The <setups>-, <checkups>- und <teardowns>-elements

For a test suite setups can be defined before the flow and checkups and teardowns can be specified after the flow. Each of these elements contains a list of units whose implementation is either defined by an instance element or by the reference to the unitlib.

Attribute m/o Description
name o The name attribute defines the name of the setup, checkup or teardown. If no alias attribute is defined, name attribute and instance element are mandatory.
alias o The alias attribute reference a unit in the unitlib. The alias corresponds to the unique name of the unit in the unitlib. If the setup, checkup or teardown has no name attribute the name of the unit is used. If no alias attribute is defined, name attribute and instance element are mandatory.
Element m/o Description
parameters o Same as the parameters for the test units.
instance o May be used to define the implementation of the unit wit a class and a method name. Better practice is to use the alias.

Example:

	<setups>
		<setup alias="setup1InUnitlib">
			<parameters>...</parameters>
		</setup>
		<setup name="mySetup2" alias="unit2InUnitlib"/>
		<setup name="mySetup3">
			<instance class="myClass" method="myMethod">
		</setup>
		...
	</setups>

 

The unit library <unitlib>

The unit library of the repository contains the instance definitions of all necessary test units, setups, checkups and teardowns. At unit level there is no distinction between the unit types. This is only done within the test description so that e.g. the same instance may be used as setup in one scenario or as teardown in another.

The unit lib, as the name implies, only contains units. No further nesting into suites is possible here.

 

Attribute m/o Description
import o The unitlib can contain an import attribute to reference further descriptor repositories. Several repositories can be given as a semicolon separated list. The repositories are expected relative to the inputpath of the "main" repository as defined in the configuration file. The reference refers to the unitlib element in the imported repository.
Element m/o Description
unit o The unitlib may contain an unlimited number of elements that define the instance of a test unit, setup, teardown etc. From the test description the units are referenced via an alias attribute that must correspond to the name of the unit in the unitlib.

Example:

	<unitlib import="unitlib1.xml;unitlib2.xml">
		<unit name="unitname">
			<parameters>
				<parameter name="user" value="default_user"/>
				<parameter name="pass" value="default_password"/>
			</parameters>
			<instance class="org.ivalidator.testapp.test.LoginTest" method="testLoginSuccessful"/>
		</unit>
		<unit>...</unit>
	</unitlib>

The <unit> Element

Attribute

 m/o Description
name m The units are indentified by a unique name. For a unit within the unitlib the name is a mandatory attribute. This is not supplied by the schema (because the unittype definition in the schema is meant for the units in the unitlib as well as in the test descriptions where the name is optional), but is checked during the input of the repository. The uniqueness of the name is expected by the xml repository but not verified. In case there are more than one units of the same name, only the first is applied.
alias n/a For a unit within the unitlib the alias attribute is not applicable even if the W3C schema would allow it (because the unittype definition in the schema is meant for the units in the unitlib as well as in the test descriptions)

 

 

Element m/o Description
parameters o Parameters may be supplied for a unit. As with the test definitions references to the parameterlib via parametergroup alias are possible. The parameter definitions in the unit lib can be seen as default values, they may be overwritten by the referencing tests, setups, teardowns, etc.
instance m The main purpose of the unit definition is to supply unit implementation in the form of a class and method. The testrunner checks during the validation run if the given class and method are available on the classpath given in the configuration file for this repository.

 

The Parameter Library <parameterlib>

The parameter library of the repository contains parametergroups that may be referenced by an test or unit within the test descriptions as well as the unit library.

Attribute m/o Description
import o The parameterlib can contain an import attribute to reference further descriptor repositories. Several repositories can be given as a semicolon separated list. The repositories are expected relative to the importing repositories' location or on the inputpath of the "main" repository as defined in the configuration file. The reference refers to the parameterlib element in the imported repository.

 

Element m/o Description
parametergroup o The parameterlib may contain an unlimited number of parametergroup elements that define a group of parameters and parameterlists. From the test description the parametergroups are referenced via an alias attribute that must correspond to the name of the parametergroup in the parameterlib.

Example:

	<parameterlib import="parameterlib1.xml;parameterlib2.xml">
		<parametergroup name="loginGroup">
			<parameterlist name="logins">
				<parameterlistentry>
					<parameter name="user" value="user1"/>
					<parameter name="pass" value="password1"/>
				</parameterlistentry>
				<parameterlistentry>
					<parameter name="user" value="user2"/>
					<parameter name="pass" value="password2"/>
				</parameterlistentry>
			</parameterlist>
			<parameter name="domain" value="xyz"/>
			<parameter name="timeout" value="1000"/>
		</parametergroup>
		<parametergroup>...</parametergroup>
	</parameterlib>

The <parametergroup> element

Attribute m/o Description
name m The parametergroups are indentified by a unique name. For a parametergroup within the parameterlib the name is a mandatory attribute. This is not supplied by the schema (because the parametergrouptype definition in the schema is meant for the parametergroups in the parameterlib as well as in the test descriptions where the name is optional), but is checked during the input of the repository. The uniqueness of the name is expected by the xml repository but not verified. In case there are more than one parametergroups of the same name, only the first is applied.
alias n/a For a parametergroup within the parameterlib the alias attribute is not applicable even if the W3C schema allows it because the parametergrouptype definition in the schema is meant for the parametergroups in the parameterlib as well as in the test descriptions).

 

Element m/o Description
parameter o An unlimited number of parameters in the form of name / value pairs can be provided.
The parameter value can be checked by the implementing unit(see validation run). A parameter from a parametergroup will be overwritten by a parameter of the same name in the referencing test description or unit definition.
parameterlist o An unlimited number of parameterlists can be provided. A parameterlist is identified by its unique name. It has 0-n list entries that constitute groups of parameters with equal names and different values. The entries are identified implicitly by their index in the parameterlist. The validation run can be used to verify that the contents of the parameter list entries are valid. A parameterlist from a parametergroup will be overwritten by a parameterlist of the same name in the referencing test description or unit definition.


The adapter library <adapterlib>

The adapter library provides adapter settings for the framework that are required by the test implementaitons. Other than the parameter lib and the unit lib the adapterlib is not used for the structuring of the test descriptions and resolved during the input of the descriptor repository but the adapterlist is directly requested from the framework (interface DescriptorRepository, method AdapterDescriptor[] getAdapterDescriptors()).

Attribute m/o Description
import o The adapterlib can contain an import attribute to reference further descriptor repositories. Several repositories can be given as a semicolon separated list. The repositories are expected relative to the importing repositories' location or on the inputpath of the "main" repository as defined in the configuration file. The reference refers to the adapterlib element in the imported repository.

Element m/o Description
adapter o The adapterlib may contain an unlimited number of adapter elements.

Example:

	<adapterlib import="adapterlib.xml">
		<adapter name="adaptername" class="org.ivalidator.adapters.MyAdapter">
			<parameters>
				<parameter name="param1" value="value1"/>
			</parameters>
		</adapter>
		<adapter>...</adapter>
	</adapterlib>

The <adapter> element

Attribute m/o Description
name m The adapters are identified by a unique name.
class m The adapter class. Details for the implementation of an adapter can be found in the section adapter.
Element m/o Description
parameters o For an adapter parameters can be defined according to the same rules as for tests and units including references to the parameter lib.

 

The XML Report Repository <report-repository>

The xml repository creates one report for the validation run and one for the test run of every descriptor repository. If the configuration file contains more than one descriptor repository there will be a corresponding number of reports. The report is updated after every step during the test run so that the results are available up to the current test in the hierarchy at any point during the test run.

The filenames for the report repositories are automatically created from a prefix and the timestamp of the run. The reports of the validation run are named "validate-DD.MM.CCYY_hh_mm_ss.xml", the reports of the test run are named "execute-DD.MM.CCYY_hh_mm_ss.xml"

The structure of the xml report repository is very simple. Every entity in the test (suites, test units, setups, teardowns, checkups) have the same result entry. The structure of the report is mainly the same as that of the testhierachy, only the "brackets" <setups>, <checkups>, <teardowns>, as well as <flow> are missing.
A different structure may occur, if for example after an error the rule "next_test_inits" is applied. Then the report will contain teardown and setup entries between two tests that are not defined in the test description.

Example of a report:

<report xmlns="http://www.ivalidator.org/schemas/xml-repository">
    <type>test</type>
    <name>FULL_TEST_SUITE</name>
    <result>success</result>
    <start-time>2005-01-26T16:46:21.015+01:00</start-time>
    <end-time>2005-01-26T16:46:21.676+01:00</end-time>
    <duration>00:00:00.661</duration>
    <setup>
        <type>setup</type>
        <name>startServer</name>
        <result>success</result>
        <note></note>
        <start-time>2005-01-26T16:46:21.015+01:00</start-time>
        <end-time>2005-01-26T16:46:21.035+01:00</end-time>
        <duration>00:00:00.020</duration>
    </setup>
    <test>
        <type>test</type>
        <name>Successful_Login</name>
        <result>success</result>
        <start-time>2005-01-26T16:46:21.246+01:00</start-time>
        <end-time>2005-01-26T16:46:21.476+01:00</end-time>
        <duration>00:00:00.230</duration>
        <test>
            <type>test</type>
            <name>testLoginSuccessful</name>
            <result>success</result>
            <note></note>
            <start-time>2005-01-26T16:46:21.246+01:00</start-time>
            <end-time>2005-01-26T16:46:21.246+01:00</end-time>
            <duration>00:00:00.000</duration>
        </test>
        <checkup>
            <type>checkup</type>
            <name>checkupLogin</name>
            <result>success</result>
            <note></note>
            <start-time>2005-01-26T16:46:21.376+01:00</start-time>
            <end-time>2005-01-26T16:46:21.376+01:00</end-time>
            <duration>00:00:00.000</duration>
        </checkup>
    </test>
    <teardown>
        <type>teardown</type>
        <name>stopServer</name>
        <result>success</result>
        <note></note>
        <start-time>2005-01-26T16:46:21.586+01:00</start-time>
        <end-time>2005-01-26T16:46:21.586+01:00</end-time>
        <duration>00:00:00.000</duration>
    </teardown>
</report>