Objective:
After PTL rpm Installation, the user has to cd to tests dir to execute test cases. We have -f option in pbs_benchpress but it supports only file. But we cannot use it to run a whole bunch of different tests.
when user runs pbs_benchpress -t testcase, it should do a path search – search current directory, next search test case in installed location.
Description
Currently, PTL doesn't support running test case from non-test directory using pbs-_benchpress TestExecution option ('-t') i.e. a directory where all tests exist.
...
This design document is to support pbs_benchpress to execute test cases(or test suite by it's its class name) from any directory on the host apart from tests directory.
...
Forum discussion at : http://community.pbspro.org/t/enhancement-of-pbs-benchpress-to-run-from-non-tests-directory/1660
PR : https://github.com/PBSPro/pbspro/pull/1165
Details:
File : utils/plugins/ptl_test_loader.py
...
- When a test case is run using "pbs_benhcpress Execution option -t" option, PTL will look for the test case in current directory, if it doesn't find then it will search in PTL installed directory (Ex: /opt/ptl/tests)
- User can also specify path to tests directory it is not present in PTL installed directory. If user wish to place tests in some non-default directory, then user can mention using environment variable "PTL_TESTS_DIR". Ex: export PTL_TESTS_DIR=/home/user/temp/tests"
- If PTL_TESTS_DIR is mentioned, then PTL will search in PTL_TESTS_DIR directory only when it doesn't find test case in current and PTL installed directory.
- Currently PTL_TESTS_DIR is limited to single path specification. We can mention PTL_TESTS_DIR to point to single path.
Currently PTL supports getting test info only from tests directory. Now, one can get test info using pbs_benchpress INFO option (pbs_benchpress -i TestSuitName) from any directory apart from test directory.
Summary of order how a test case being searched:
1) current dir
2) user set dir(PTL_TESTS_DIR) (if set)
3) Installed dir (/opt/ptl/)
Example:
Assuming PTL is installed in "/opt/ptl/" path and tests directory is present under "/opt/ptl/tests path".
...
PTL first tries to find test case in current directory, when if it fails to find then it will search in "/opt/ptl/tests/" path. If it fails to find there as well then it will search in user defined path given in environment variable "PTL_TESTS_DIR".
...
In this case when a test case is executed from /home/user directory, PTL throws error saying "Unknown testcasetest case" as it fails to find the test case in current directory as well as in PTL installed directory which is "/opt/ptl/tests/".
Scenario-4:
Assuming tests directory is moved to "/tmp/tests/" path and if user has set multiple paths in PTL_TESTS_DIR, then PTL will throw an error saying "Invalid directory specified in PTL_TESTS_DIR"
Ex: export PTL_TESTS_DIR=/tmp/tests:/home/user/tests