Using Tags in PTL
The contents of this page appear in Guidelines for Writing PTL Tests.PTL test tags let you list or execute a category of similar or related test cases across test suites in test directories. To include a test case in a category, tag it with the “@tags(<tag_name>)” decorator. Tag names are case-insensitive.
See the pbs_benchpress page for how you can use tags to select tests.
Pre-defined PTL Test Tags
Tag Name | Description | |
---|---|---|
1 | smoke | Tests related to basic features of PBS, such as job or reservation submission/execution/tracking, etc. |
2 | server | Test related to server features exclusively. Ex: server requests, receiving & sending job for execution etc. |
3 | sched | Tests related to scheduler exclusively. Ex: tests related to scheduler daemon, placement of jobs, implementation of scheduling policy etc. |
4 | mom | Tests related to mom, i.e. processing of jobs received from server and reporting back etc. Ex: Mom polling etc. |
5 | comm | Tests related to communication between server, scheduler and mom. |
6 | hooks | Tests related to server hooks or mom hooks |
7 | reservations | Tests related to reservations |
8 | configuration | Tests related to any PBS daemon configurations. |
9 | accounting | Tests related to accounting logs |
10 | scheduling_policy | Tests related to job scheduling policy of the scheduler - |
11 | multi_node | Tests involving more than one node complex |
12 | commands | Tests related to PBS commands and its outputs (Client related) |
13 | security | Tests related to authentication, authorisation etc. |
14 | windows | Tests that can run only on windows platform |
15 | cray | Tests that can run only on cray platform |
16 | cpuset | Tests that can run only on cpuset system |
Tagging Test Cases
Examples of tagging test cases:
All the test cases of pbs_smoketest.py are tagged with “smoke”.
>>>>>
@tags('smoke')
class SmokeTest(PBSTestSuite)
>>>>>
Multiple tags can be specified, as shown here:
>>>>>
@tags(‘smoke’, ’mom’, ’configuration’)
class Mom_fail_requeue(TestFunctional)
>>>>>
Using Tags to List Tests
Use the --tags-info option to list the test cases with a specific tag. For example, to find test cases tagged with "smoke":
pbs_benchpress --tags-info --tags=smoke
Finding Existing Tests
To find a test case of a particular feature:
Ex: Find a ASAP reservations test case
- Look for the appropriate directory – which is functional in this case
- Look for the existence of the relevant feature test suite file in this directory or run command to find test suites
ex. pbs_reservations.py
ex. pbs_benchpress -t TestFunctional -i
- Look for the test suite info to get doc string of test cases related to ASAP reservation
pbs_benchpress -t TestReservations -i –verbose
- All reservations tests can be listed as below
pbs_benchpress --tags-info--tags=reservations --verbose
The same command can be used to list tests inside the directories. Ex: All reservations tests inside performance directory
Placing New Tests in Correct Location
To add a new test case of a particular feature or bug fix:
Ex: A test case for a bug fix that updated accounting logs
- Look for the appropriate directory - which is functional in this case
- Look for the existence of the relevant feature test suite file or run command to list test suites of base class
ex. In Functional test directory any test file / test suites associated with “log”. If present, add test case into that test suite
ex. pbs_benchpress -t TestFunctional -i
- If test suite is not present, add a new test suite with name Test<Featurename> in file with name pbs_<featurename>.py
- Tag the new test case if necessary
If the test case seems to belong to any of the features listed in tag list, it can be tagged so.
Ex. @tags(‘accounting’)
Using Tags to Run Desired Tests
Use the --tags option to execute the test cases, including hierarchical tests, tagged with a specific tag. For example, to execute the test cases tagged with "smoke":
pbs_benchpress --tags=smoke
Ex: All scheduling_policy tests
- Look for the appropriate directory - which is functional in this case
- Look for that feature in tag list. If present, run with tag name as below:
pbs_benchpress --tags=scheduling_policy
- If no tags present then look for relevant test suite/s present, run the same
pbs_benchpress -t <suite names>