Tunable test parameters
- Vishesh Hadimane
- Community Forum: http://community.pbspro.org/t/tunable-test-parameters/1535
- Use Cases:
- Test developers need a user friendly way to specify configurable test parameters at test case level.
- Test executors need a way to configure the test parameters.
- Example:
- Test parameters for a test can be
- No_of_jobs which can be scaled up like 100k jobs on cloud/good configuration machine and scaled down to 1k jobs on a vm.
- qsub_exec_script which can be a path to the job script to be used for job submission.
- Test parameters for a test can be
- Example:
- Interface 1: @testparams(<param1>=<value1>,<param2>=<value2>)
Synopsis : @testparams is a decorator to specify tunable test parameters for a test and their default values
Details:
1. Test Parameters can be specified using testparams decorator in key value pair format. The parameters and its values will be stored in self.conf dictionary of the test case.
@testparams decorator should be only used at test case level not at testsuite level.
Variable names should be kept unique across the test cases to avoid collisions when modifying it at CLI.
Example:
@testparams(x=y,a=b)
Can be accessed inside testcase as self.conf['x']
2. As the decorator stores the key value pair data in self.conf dictionary it can be modified through pbs_benchpress.
The variable name should be mentioned with testsuite name to avoid collisions when running multiple testsuites ot tags.
Example:
pbs_benchpress -p TestsuiteA.x=z,TestsuiteA.a=c
- Interface 2: Test Params info added to pbs_benchpress -i --verbose -t <PBSTestSuite>
Synopsis : The test parameters will be added to test doc string and can be viewed by pbs_benchpress -i --verbose -t <pbstestsuite>
Example:
If test runner wants to know which parameters of the test can be modified he can do that by running
pbs_benchpress -i --verbose -t Teststat
Test Case Doc:
<doc string>
Test Params:
x=y
a=b
Project Documentation Main Page