Using Tags in PTL

The contents of this page appear in Guidelines for Writing PTL Tests.PTL test tags let you list or execute a category of similar or related test cases across test suites in test directories.  To include a test case in a category, tag it with the “@tags(<tag_name>)” decorator. Tag names are case-insensitive.

See the pbs_benchpress page for how you can use tags to select tests.


Pre-defined PTL Test Tags


Tag Name

Description

1

smoke

Tests related to basic features of PBS, such as job or reservation submission/execution/tracking, etc.

2

server

Test related to server features exclusively. Ex: server requests, receiving & sending job for execution etc.

3

sched

Tests related to scheduler exclusively. Ex: tests related to scheduler daemon, placement of jobs, implementation of scheduling policy etc.

4

mom

Tests related to mom, i.e. processing of jobs received from server and reporting back etc. Ex: Mom polling etc.

5

comm

Tests related to communication between server, scheduler and mom.

6

hooks

Tests related to server hooks or mom hooks

7

reservations

Tests related to reservations

8

configuration

Tests related to any PBS daemon configurations.

9

accounting

Tests related to accounting logs

10

scheduling_policy

Tests related to job scheduling policy of the scheduler -

11

multi_node

Tests involving more than one node complex

12

commands

Tests related to PBS commands and its outputs (Client related)

13

security

Tests related to authentication, authorisation etc.

14

windows

Tests that can run only on windows platform

15

cray

Tests that can run only on cray platform

16

cpuset

Tests that can run only on cpuset system


Tagging Test Cases

Examples of tagging test cases:

All the test cases of pbs_smoketest.py are tagged with “smoke”.

>>>>>
@tags('smoke')
class SmokeTest(PBSTestSuite)
>>>>>

Multiple tags can be specified, as shown here:
>>>>>
@tags(‘smoke’, ’mom’, ’configuration’)
class Mom_fail_requeue(TestFunctional)
>>>>>

Using Tags to List Tests

Use the --tags-info option to list the test cases with a specific tag.  For example, to find test cases tagged with "smoke":

    pbs_benchpress --tags-info --tags=smoke

Finding Existing Tests

To find a test case of a particular feature:

Ex: Find a ASAP reservations test case

  1. Look for the appropriate directory – which is functional in this case
  2. Look for the existence of the relevant feature test suite file in this directory or run command to find test suites

ex. pbs_reservations.py

ex. pbs_benchpress -t  TestFunctional -i

  1. Look for the test suite info to get doc string of test cases related to ASAP reservation

pbs_benchpress -t TestReservations -i –verbose

  1. All reservations tests can be listed as below

pbs_benchpress --tags-info--tags=reservations --verbose

The same command can be used to list tests inside the directories. Ex: All reservations tests inside performance directory

Placing New Tests in Correct Location

To add a new test case of a particular feature or bug fix:

Ex: A test case for a bug fix that updated accounting logs

  1. Look for the appropriate directory - which is functional in this case
  2. Look for the existence of the relevant feature test suite file or run command to list test suites of base class

ex. In Functional test directory any test file / test suites associated with “log”. If present, add test case into that test suite

ex. pbs_benchpress -t  TestFunctional -i

  1. If test suite is not present, add a new test suite with name Test<Featurename> in file with name pbs_<featurename>.py
  2. Tag the new test case if necessary

If the test case seems to belong to any of the features listed in tag list, it can be tagged so.

Ex. @tags(‘accounting’)

Using Tags to Run Desired Tests

Use the --tags option to execute the test cases, including hierarchical tests, tagged with a specific tag.  For example, to execute the test cases tagged with "smoke":

    pbs_benchpress --tags=smoke

Ex: All scheduling_policy tests

  1. Look for the appropriate directory - which is functional in this case
  2. Look for that feature in tag list. If present, run with tag name as below:

pbs_benchpress --tags=scheduling_policy

  1. If no tags present then look for relevant test suite/s present, run the same

pbs_benchpress -t <suite names>  



OSS Site Map

Developer Guide Pages