This page appears in the PTL Developer's Guide.

You use pbs_benchpress to run or list test suites and test cases.

How to run pbs_benchpress

Go to pbspro/test/tests in your cloned GitHub directory structure.

Syntax:

   pbs_benchpress <option(s)

Privilege

By default, pbs_benchpress automatically makes its current user a PBS Manager, via "qmgr -c 's s managers += < current user>".

Options to pbs_benchpress

--cumulative-tc-failure-threshold=<count>

Cumulative test case failure threshold. Defaults to 100.

Set count to 0 to disable this threshold.

If not zero, must be greater than or equal to 'tc-failure-threshold'

--db-access=<path to DB parameter keyword file>

Path to a file that defines DB parameters (PostreSQL only); can be absolute or relative.

See https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS.

You probably need to specify the following parameters:

dbname: database name

user: user name used to authenticate

password: password used to authenticate

host: database host address

port: connection port number

--db-name=<database name>

Database where test information and results metrics are stored for later processing.

Defaults to ptl_test_results.db.

--db-type=<database type>

Can be one of "file", "html", "json", "pgsql", "sqlite".

Defaults to "json".

--eval-tags='<Python expression>'

Selects only tests where Python expression evaluates to True. Can be applied multiple times.

Example:

'smoke and (not regression)' - Selects all tests which have "smoke" tag and don't have "regression" tag

'priority>4' - Selects all tests which have "priority" tag and where the value of priority is greater than 4

--exclude=<tests to exclude>

Comma-separated string of tests to exclude

-f <test files to run>

Comma-separated list of test file names

-F

Sets logging format to include timestamp and level

--follow-child

If set, walks the test hierarchy and runs each test. Starts from current directory.

-g <path to test suite file>

Absolute path to file containing comma-separated list of test suites

Used for very long list of tests

--genhtml-bin=<path to genhtml binary>

Defaults to "genhtml": PTL assumes genhtml is in PATH

-h

Displays usage information

-i

Prints test name and docstring to stdout, then exits.

-l <log level>

Specifies log level to use. Log levels are documented in PTL.

-L

Prints the following to stdout, then exits: test suite, file, module, suite docstring, tests, and number of test suites and test cases in current directory and all its subdirectories.

--lcov-baseurl=<url>

Base URL for lcov HTML report. Uses url as base URL.

--lcov-bin=<path to lcov binary>

Defaults to "lcov": PTL assumes lcov is in PATH

--lcov-data=<path to directory>

Path to directory containing .gcno files; can be absolute or relative

--lcov-nosrc

If set, do not include PBS source in coverage analysis.

By default, PBS source is included in coverage analysis

--lcov-out=<path to output directory>

Output path for HTML report.

Default path is TMPDIR/pbscov-YYYYMMDD_HHmmSS

--list-tags

Recursively list all tags in tests in current directory and its subdirectories.

--log-conf=<path to logging config file>

Absolute path to logging configuration file. No default.

File format specified in https://docs.python.org/2/library/logging.config.html#configuration-file-format.

--max-postdata-threshold=<count>

Max number of diagnostic tool runs per testsuite. Diagnostic tool is run on failure. Defaults to 10 sets of output from diagnostic tool.

Set count to 0 to disable this threshold.

--max-pyver=<version>

Maximum Python version

--min-pyver=<version>

Minimum Python version

-o <logfile>

Name of log file.

Logging output is always printed stdout; this option specifies that it should be printed to specified log file as well.

-p <test parameter>

Comma-separated list of <key>=<value> pairs. Note that the comma cannot be used in value. Pre-defined pairs are in 'Custom parameters:' in http://pbspro.org/ptldocs/ptl.html#ptl.utils.pbs_testsuite.PBSTestSuite but you can add your own along with those.

--param-file=<parameter file>

If set, gets test parameters from file

Overrides -p

--post-analysis-data=<post-analysis data directory>

Path to directory where diagnostic tool output is placed.

--stop-on-failure

If set, stop when any test fails

-t <test suites>

Comma-separated list of test suites to run

--tags=<list of tags>

Selects only tests that have tag tag. Can be applied multiple times.

Format: [!]<tag>[,<tag>]

Examples:

smoke - Selects all tests which have "smoke" tag

!smoke - Selects all tests which don't have "smoke" tag

smoke,regression - Selects all tests which have both "smoke" and "regression" tag

--tags-info

Lists all selected test suites

When used with --verbose, lists all test cases

Must be used with --tags or --eval-tags

You can use -t or --exclude to limit selection

--tc-failure-threshold=<count>

Test case failure threshold per testsuite

Defaults to 10

To disable, set count to 0

--timeout=<seconds>

Duration after which no test suites are run

--verbose

Shows verbose output

Must be used with -i, -L or --tags-info

--version

Prints version number to stdout and exits

OSS Site Map

Developer Guide Pages