Page Comparison

• Community Forum: http://community.pbspro.org/t/tunable-test-parameters/1535
• Use Cases:
  • Test developers need a user friendly way to specify configurable test parameters at test case level.
  • Test executors need a way to configure the test parameters.
  Summary : 
  • 
    
  Test Framework should provide tests with option to scale & tune tests based on inputs.
  
  • • Example:
      • Test parameters for a test can be
        • No_of_jobs which can be scaled up like 100k jobs on cloud/good configuration machine and scaled down to 1k jobs on a vm.
        •  qsub_exec_script which can be a path to the job script to be used for job submission.
•  Interface 1: @testparams(<param1>=<value1>,<param2>=<value2>)

      Synopsis : @testparams  @testparams is a decorator to specify tunable test parameters for a test and their default values of test specific parameters

      Details: 

      1. Test Parameters can be specified using testparams decorator in key value pair format. The parameters and its values will be stored in self.conf dictionary of the test case. 

          @testparams decorator should be only used at test case level not at testsuite level.

           Variable names should be kept unique across the test cases to avoid collisions when modifying it at CLI.

          Example: 

          @testparams(x=y,a=b)

...

      2. As the decorator stores the key value pair data in self.conf dictionary it can be modified through pbs_benchpress.

          The variable name should be mentioned with testsuite name to avoid collisions when running multiple testsuites ot tags.

          Example:

          pbs_benchpress -p TestsuiteA.x=z,TestsuiteA.a=c

•  Interface 2: Test Params info added to pbs_benchpress -i --verbose -t <PBSTestSuite>

...