Reforming Requirements Decorator in PTL

‘requirements()’ decorator in PTL is used to specify the PBS cluster configuration necessary to run a PTL test and during the test run this data is evaluated against the cluster data passed in custom parameters of pbs_benchpress. The current format of PBS cluster specification in the decorator specifies PBS daemon counts a and certain set of flags. This causes ambiguity in actual number of nodes needed by the test to run. Hence, this design is to reform the cluster data specification in order to have clear information of the PBS cluster needed to run the test.

Interface Updates:

Interface 1: ‘--hosts=<hostname>[m-n][:port][@<path to pbs conf>],<hostname>[k-l][:port][@<path to pbs conf>],<hostname>[m-n][:port][@<path to pbs conf>]’
Visibility: Public
Change Control: Public

This new option is introduced to specify the hostnames of the cluster to be passed to PTL instead of the current per daemon format. (moms=<host1:host2>,comms=<host3:host4>)
In case where single hostname or no hostnames are passed, the local host where pbs_benchpress is being run is considered as single node PBS cluster host and all tests needing single daemon of each type will be executed on this host.
Its value should be a comma separated list of hostnames or indexed host names range for greater number of hosts along with the non-default port number or non-default path of pbs.conf file. This non-default port number and non-default pbs.conf file path is same for all the hosts specified in the range. For example:

pbs_benchpress -t TestFeature.test_t1 --hosts=x1,x2,x3
pbs_benchpress -t TestFeature.test_t1 --hosts=x1,x[2-100],x[101-110]
pbs_benchpress -t TestFeature.test_t1 --hosts=x1,x[2-100]:9000@/etc/exec/pbs.conf,x[101-110]


Interface 2: @requirements() decorator format - “host1=<string>, host2=<string>”
Visibility: public
Change Control: public
New form of requirements decorator uses below mentioned strings to specify the type of nodes the test case needs in order to run.

The string specifications are as follows:


The format of specifying the types of hosts is list of hosts and the type of daemon combination the host needs:
“host1=<string>, host2=<string>”

Key changes:
1. The daemon names in the strings are interchangeable i.e. they can be specified in any order. Ex: "server,sched,comm" is same as "comm,sched,server"

2. The default requirements now maps to remote mom setup for all single mom cluster setup i.e. 2 hosts being necessary. (host1="server,sched,comm", host2="mom")

3. PTL would detect the type of installation on each of the host names passed in benchpress and compare with the host’s list expectations to determine whether test is runnable on the given cluster.

4. The tests requiring only one daemon of each type (Ex: host1="server,sched,comm", host2="mom") can run on single host when only one hostname or no hostnames are passed to pbs_benchpress.

5. In the case when large number of hostnames are to be specified, we can index the hostnames like - host2_10="mom"

Example 1:

DEFAULT - 1 mom node setup: 2 hosts
@requirements(host1="server,sched,comm", host2="mom")
pbs_benchpress --hosts=x1,x2
Older version: @requirements(no_mom_on_server=True)

Example 2:

2 mom nodes setup: 3 hosts
@requirements(host1="server,sched,comm", host2="mom", host3="mom")
pbs_benchpress --hosts=x1,x2,x3
Older version: @requirements(num_moms=2)

Example 3:

2 moms and 1 client: 4 hosts
@requirements(host1="server,sched,comm", host2="mom", host3="mom", host4=”client”)
pbs_benchpress --hosts=x1,x2,x3,x4
Older version: @requirements(num_moms=2, num_clients=1)

Example 4:

2 moms, 1 client & comm not running on server: 5 hosts
@requirements(host1=”server, sched”, host2=”mom”, host3=”mom”, host4=”comm”, host5=”client”)
pbs_benchpress --hosts=x1,x2,x3,x4,x5
Older version: @requirements(num_moms=2, num_clients=1, no_comm_on_server=True)

Example 5:

10 moms, 1 client & comm not running on server: 13 hosts
@requirements(host1=”server, sched”, host2_11=”mom”, host12=”client”, host13=”comm”)
pbs_benchpress --hosts=x[1-13] / pbs_benchpress --hosts=x1,x[2-11],x12,x13 / pbs_benchpress --hosts=x1,x2,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13
Older version: @requirements(num_moms=10, num_clients=1, no_comm_on_server=True)


Interface 3: --get-required-hosts
Visibility: Public
Change Control: Public

This option gives out the number of hosts necessary to execute the tests specified in pbs_benchpress. This also mentions the test cases that will run with the given cluster information. When none of the tests are specified PTL will give the information of all the tests in current tests directory. The test suite information option ‘-i' or '--gen-ts-tree’ will also give out the number of hosts needed by the given test suite or test case along with the test information.

pbs_benchpress -p <cluster_info> -t <TestSuite1>,<TestSuite2> --get-required-hosts
pbs_benchpress -t <TestSuite1>,<TestSuite2> --get-required-hosts
pbs_benchpress --tags "sched" --get-required-hosts
pbs_benchpress --get-required-hosts


Interface 4: run_test_as in requirements() decorator
Visibility: Public
Change Control: Public

We now can specify with which user a PTL test needs to be specifically run as. For example, when a test suite should be run as ‘root’, below can be specified. By default all tests run as the test runner who triggers the pbs_benchpress. The test user must be users as specified in PTL like - ROOT_USER, ADMIN_USER, TEST_USER1 etc.


@requirements(host1=SERVER_SCHED_COMM, host2=MOM, host3=MOM, run_test_as=ROOT_USER)


Interfaces removal:

With the above design of requirements decorator and the pbs_benchpress’ cluster information input through custom parameters; below mentioned options in custom parameters are no more required:

-p mom=
-p moms=
-p server=
-p servers=
-p nomom=
-p clients=

Below listed flags in requirements decorator will be removed:


These options will no longer be used.