A Yats (Yet Another Test System) test scenario consists of test cases that may be contained in test groups. This creates a test tree structure. Each node (test case or test group) can be configured.
For test cases, the following properties can be set:
- Repetitions – single run, specified number of repetitions, infinite repetition (until test is canceled) or repeating until the test finishes with a specified test result;
- Single run result evaluation method: for each test run the reported result can be evaluated. In the current Yats version it is possible to ignore the test result or apply a result mapping. For example if it is expected for a test to fail, the FAIL result can be changed to a PASS
- Repetition result evaluation method: the results of individual repetitions can be ignored (repetition result is PASS even if some repetitions fail) or can be evaluated to best/worst result;
- A test case can be disabled
For test groups, the properties are slightly different:
It is possible to change the execution order: the test cases may run in the same order as shown in test sequence, run in parallel or random order. These settings only apply to the child steps and are not inherited in deeper levels. A test group may be configured to stop executing when a step fails.