Extending UTP 2 with cascading arbitration specifications

In testing, the term arbitration describes the process of calculating the verdict after the execution of a test case or test suite. The calculation of the verdict follows a defined rule set that clearly specifies which verdict to produce under which conditions. In many situations, these rules simply follow the scheme that any deviation between the expected and the actual response of the system under test leads to fail. UTP 2 introduces the concept of arbitration specifications on various hierarchy levels to determine the final verdict of an arbitration target. It provides default arbitration specifications that adhere to the above-mentioned straightforward calculation of verdicts, but allows for overriding these default ones with user-defined arbitration specifications. Unfortunately, this override mechanism adversely affects the maintainability of test cases and test actions because of its high degree of intrusion. Arbitration targets, such as test sets, test cases and procedural elements, and arbitration specifications are tightly coupled with each other, losing the ability to reuse these arbitration targets in a different context with different arbitration specifications. In this paper, we suggest to replace this highly intrusive override mechanism with a decoupling binding mechanism. This binding mechanism increases both comprehensibility and maintainability of test specifications on one hand, because arbitration targets remain independent of any potential arbitration specification. On the other hand, it offers a high degree of reusability and flexibility to the user because of a cascading override mechanism inspired by W3C cascading style sheets.