PTL test tags let you list or execute a category of similar or related test cases across test suites in test directories. To include a test case in a category, tag it with the “@tags(<tag_name>)” decorator. Tag names are case-insensitive. See the pbs_benchpress page for how you can use tags to select tests.
| Tag Name | Description |
---|
1 | smoke | Tests related to basic features of PBS, such as job or reservation submission/execution/tracking, etc. | 2 | server | Test related to server features exclusively. Ex: server requests, receiving & sending job for execution etc. | 3 | sched | Tests related to scheduler exclusively. Ex: tests related to scheduler daemon, placement of jobs, implementation of scheduling policy etc. | 4 | mom | Tests related to mom, i.e. processing of jobs received from server and reporting back etc. Ex: Mom polling etc. | 5 | comm | Tests related to communication between server, scheduler and mom. | 6 | hooks | Tests related to server hooks or mom hooks | 7 | reservations | Tests related to reservations | 8 | configuration | Tests related to any PBS daemon configurations. | 9 | accounting | Tests related to accounting logs | 10 | scheduling_policy | Tests related to job scheduling policy of the scheduler - | 11 | multi_node | Tests involving more than one node complex | 12 | commands | Tests related to PBS commands and its outputs (Client related) | 13 | security | Tests related to authentication, authorisation etc. | 14 | windows | Tests that can run only on windows platform | 15 | cray | Tests that can run only on cray platform | 16 | cpuset | Tests that can run only on cpuset system |
Examples of tagging test cases: All the test cases of pbs_smoketest.py are tagged with “smoke”. >>>>> @tags('smoke') class SmokeTest(PBSTestSuite) >>>>> Multiple tags can be specified, as shown here: >>>>> @tags(‘smoke’, ’mom’, ’configuration’) class Mom_fail_requeue(TestFunctional) >>>>> Use the --tags-info option to list the test cases with a specific tag. For example, to find test cases tagged with "smoke": pbs_benchpress --tags-info --tags=smoke To find a test case of a particular feature: Ex: Find a ASAP reservations test case - Look for the appropriate directory – which is functional in this case
- Look for the existence of the relevant feature test suite file in this directory or run command to find test suites
ex. pbs_reservations.py ex. pbs_benchpress -t TestFunctional -i - Look for the test suite info to get doc string of test cases related to ASAP reservation
pbs_benchpress -t TestReservations -i –verbose - All reservations tests can be listed as below
pbs_benchpress --tags-info--tags=reservations --verbose The same command can be used to list tests inside the directories. Ex: All reservations tests inside performance directory To add a new test case of a particular feature or bug fix: Ex: A test case for a bug fix that updated accounting logs - Look for the appropriate directory - which is functional in this case
- Look for the existence of the relevant feature test suite file or run command to list test suites of base class
ex. In Functional test directory any test file / test suites associated with “log”. If present, add test case into that test suite ex. pbs_benchpress -t TestFunctional -i - If test suite is not present, add a new test suite with name Test<Featurename> in file with name pbs_<featurename>.py
- Tag the new test case if necessary
If the test case seems to belong to any of the features listed in tag list, it can be tagged so. Ex. @tags(‘accounting’) Use the --tags option to execute the test cases, including hierarchical tests, tagged with a specific tag. For example, to execute the test cases tagged with "smoke": pbs_benchpress --tags=smoke Ex: All scheduling_policy tests - Look for the appropriate directory - which is functional in this case
- Look for that feature in tag list. If present, run with tag name as below:
pbs_benchpress --tags=scheduling_policy - If no tags present then look for relevant test suite/s present, run the same
pbs_benchpress -t <suite names> |