How to Write a PTL Test

This page appears in Guidelines for Writing PTL Tests.

Main Parts of a PTL Test

You can think of a PTL test as having 3 parts:

1. Setting up your environment
2. Running your test
3. Checking the results

Using Attributes

Many PTL commands take an attribute dictionary. 

• This is of the form of {‘attr_name1’: ‘val1’, ‘attr_name2’: ‘val2’, …, ‘attr_nameN’: ‘valN’}
• To modify resources, use the ‘attr_name.resource_name’ form like ‘resources_available.ncpus’ or ‘Resource_List.ncpus’
• Many of the ‘attr_nameN’ can be replaced with its formal name like ATTR_o or ATTR_queue.  A list of these can be found in the pbs_ifl.h C header file.

Setting Up Your Environment

The idea here is to create an environment which can run the test no matter what machine the test is being run on.  You may need to create queues, nodes, or resources, or set attributes, etc.

First you need to set up your vnode(s).  This is a required step because if you don’t, the natural vnode will be left as is.  This means the vnode will have different resources depending on what machine the test is run on.   This can be done in one of two ways: 

1. If you only need one vnode, you can modify the natural vnode.
   1. This is done by setting resources/attributes with self.server.manager().  You use self.server.manager(MGR_CMD_SET, VNODE, {attribute dictionary} id=self.mom.shortname)
2. If you need more than one vnode, you create them with self.server.create_vnodes(attrib={attribute dictionary}, num=N, mom=self.mom)

After you set up your vnodes, you might need to set attributes on servers or queues or even create new queues or resources.  This is all done via the self.server.manager() call.

Examples of Setting up Environment

• To create a queue named workq2:
  • self.server.manager(MGR_CMD_CREATE, QUEUE, {attribute dictionary}, id=<name>)
• Similarly to create a resource:
  • self.server.manager(MGR_CMD_CREATE, RSC, {‘type’: <type>, ‘flag’: <flags>}, id=<name>)
• To set a server attribute:
  • self.server.manager(MGR_CMD_SET, SERVER, {attribute dictionary})
• To set a queue attribute:
  • self.server.manager(MGR_CMD_SET, QUEUE, {attribute dictionary}, id=<queue name>)

Creating Your Test Workload

Usually to run a test you need to submit jobs or reservations.  These are of the form:

• j = Job(<user>, {attribute dictionary})

OR

• r = Reservation(<user>, {attribute dictionary})
• <user> can be one of the test users that are created for PTL.  A common user is TEST_USER.

The attribute dictionary usually consists of the resources (e.g. Resource_List.ncpus or Resource_List.select) and maybe other attributes like ATTR_o.  To submit a job to another queue, use ATTR_queue.

This just creates a PTL job or reservation object.  By default these jobs will sleep for 100 seconds and exit.  To change the sleep time of a job, you do ‘j.set_sleep_time(N)’

Finally you submit your job/reservation.

• job_id = self.server.submit(j)
• resv_id = self.server.submit(r)

Many tests require more than one reservation.  Follow the above steps multiple times for those.

Once you have submitted your job or reservation, you should check if it is in the correct state.

• self.server.expect(JOB, {ATTR_STATE: ‘R’, id=job_id)
• self.server.expect(RESV, {reserve_state’: (MATCH_RE, ‘RESV_CONFIRMED|2}) (don’t worry about this funny match, just use it).

As you are running your test, you should make sure most steps have correctly completed.  This is mostly done through expect().  The expect() function will query PBS 60 times once every half seconds (total of 30 seconds) to see if the attributes are true.  If after 60 attempts the attribute is still not true, a PtlExpectError exception will be raised.

Running Your Test

Checking Your Results

This is when you check to see if your test has correctly passed.  To do this you will either use self.server.expect() as described above, log_match(), or the series of assert functions provided by unittest.  The most useful asserts are self.assertTrue(), self.assertFalse(), self.assertEquals().  There are asserts for all the normal conditional operators (even the in operator).  For example, self.assertGreater(a, b) tests a > b.  Each of the asserts take a final argument that is a message which is printed if the assertion fails.  The log_match() function is on each of the daemon objects (e.g. self.server, self.mom, self.scheduler, etc). 

Examples of Checking Results

• self.server.expect(NODE, {‘state’: ‘offline’}, id=<name>)
• self.scheduler.log_match(‘Insufficient amount of resource’)
• self.assertTrue(a)
• self.assertEquals(a, b)

OSS Site Map

Developer Guide Pages