Nose Plugins for PTL
- Anne Urban
This page appears in Improving the PTL Framework.
Nose Plugins for PTL
PTL nose plugins are written for test collection, selection, observation, and reporting. Below are the various PTL plugins: PTLTestLoader: Load test cases from a given set of parameters PTLTestRunner: Run tests PTLTestInfo: Generate test suite and test case information PTLTestDb: Update the PTL execution data into a given data base type PTLTestData: Save post-analysis data on test case failure or error PTLTestTags: Load test cases from a given set of parameters for tag names Nose supports plugins for test collection, selection, observation, and reporting. Plugins need to implement certain mandatory methods. Plugin (hooks) fall into four broad categories: selecting and loading tests, handling errors raised by tests, preparing objects used in the testing process, and watching and reporting on test results. Some of the PTL plugins, with their purposes, are listed below: Load test cases from given parameters. Based on the parameters provided while running. This plugin is necessary to load the required list of tests. It also checks for unknown test suites and test cases if any, and excludes any tests specified for exclusion. PTL plugin to run tests. This plugin defines the following functionalities related to running tests: Start or stop the test Add error, success, or failure Track test timeout If post analysis is required, raise an exception when a test case failure threshold is reached Log handler for capturing logs printed by the test case via logging module Test result handling Load test cases from given parameters to get their information docstrings. This plugin is used to generate test suite info, rather than running tests. This info includes test suite and test case docstrings in hierarchical order. It can be queried using test suite names or tags. Update the PTL execution data into a given data base type such as File, HTML, etc. This plugin is for uploading PTL data into a data base. PTL data includes all the information regarding a PTL execution such as test name, test suite name, PBS version tested on, hostname where the test is run, test start time, end time, duration, and test status (skipped, timed out, errored, failed, succeeded, etc.) PTL data can be uploaded to any type of database, including File, JSON, HTML, and PostgresSQL. The PTLTestDb plugin also defines a method to send analyzed PBS daemon log information either to the screen or to a database file, which is used with the pbs_loganalyzer test command. Save post analysis data on test case failure or error. This plugin is for saving post analysis data on test case failure in PTL, along with tracking failures and handling test execution when failure thresholds are reached. Load test cases from given parameters for tag names. This plugin provides test tagging. It defines a Decorator function that adds tags to classes, functions, or methods, and loads the matching set of tests according to a given set of parameters. http://nose.readthedocs.io/en/latest/testing.htmlPTLTestLoader
PTLTestRunner
PTLTestInfo
PTLTestDb
PTLTestData
PTLTestTags
Nose Documentation
http://pythontesting.net/framework/nose/nose-introduction/
http://nose.readthedocs.io/en/latest/plugins/writing.html