Non-printing characters in job attribute and resource values as well as hook names

Overview

Users can type special non-printing characters (special control characters) in terminal and submit them as a job’s environment variable, or as account name using qsub. Similarly, a user can set a reservation attribute value with an embedded non-printing characters using pbs_rsub. Also with qmgr, one can create hook names with special non-printing characters, or set a node's comment value to have such special characters. These non-printing characters are potentially dangerous as they can alter  the output of qstat, pbs_rstat, pbsnodes, printjob, tracejob, and qmgr.

Forum Discussion: http://community.pbspro.org/t/escape-control-characters-in-job-environment-variables-and-account-name/1369

Technical Details

The first 32 ASCII characters are special characters that are not displayable, but they can be used to modify content on the terminal, for example, creating white spaces, moving the cursor  or changing the text and background color.

Here is an example of an environment variable that can change text color:

export abc=$(tput bold; tput setaf 1)
qsub -v abc -- /bin/sleep 60

It will display a bold red text after 'abc=' in the Variable_List output of qstat -f.

Here is another example:

qsub -A h$(tput bold; tput setaf 1)d  -- /bin/sleep 60

You should see bold red text after 'h' in Account_Name output in qstat -f, qstat -f -Fjson, as well as tracejob (run as root), and printjob of the <job>.JB file under mom_priv/jobs .

And another example shows how node comments can be introduced with special characters in them:

pbsnodes -C h$(tput bold; tput setaf 1)d <node_name>

Running pbsnodes -a, qmgr -c 'list node @default'pbsnodes -a -Fjson would show in bold red all text after 'h' in node comment.


Same with pbs reservations where the authorized hostnames list can be set to have embedded special characters in them:

pbs_rsub -H h$(tput bold; tput setaf 1)d -R<hhmm> -D 30

Running pbs_rstat -f, pbs_rstat -F would show in bold red the text after the 'Authorized_Hosts' output.


Finally, hook names can be created using qmgr with special characters in them:

qmgr -c "c h h$(tput bold; tput setaf 1)d"

Executing a qmgr -c 'print hook' or qmgr -c "list hook" would show bold red text after the 'h' output in hook name.

Most of the linux utility commands use some kind of scheme to escape non-printing characters when printing to stdout For example, cat -v displays non-printing characters using a caret followed by a capital letter or a symbol. This design proposes using the same scheme in PBS for showing non-printing characters in attribute or resource value  output in qstat, pbs_rstat, pbsnodes, printjob, tracejob, and qmgr. 

Below is a table that contains the mapping of non-printing characters to their escaped representation, EXCEPT for the following characters:TAB (\t), LF (\n) which are displayed as is (like in 'cat -v`) or converted in some form by PBS in its output (.e.g JSON format). 


Dec  Char                           Escaped
---  -----------------------------  -------
  0  NUL (null)                     ^@
  1  SOH (start of heading)         ^A
  2  STX (start of text)            ^B
  3  ETX (end of text)              ^C
  4  EOT (end of transmission)      ^D
  5  ENQ (enquiry)                  ^E
  6  ACK (acknowledge)              ^F
  7  BEL (bell)                     ^G
  8  BS  (backspace)                ^H
  9  TAB (horizontal tab)           ^I
 10  LF  (NL line feed, new line)   ^J
 11  VT  (vertical tab)             ^K
 12  FF  (NP form feed, new page)   ^L
 13  CR  (carriage return)          ^M
 14  SO  (shift out)                ^N
 15  SI  (shift in)                 ^O
 16  DLE (data link escape)         ^P
 17  DC1 (device control 1)         ^Q
 18  DC2 (device control 2)         ^R
 19  DC3 (device control 3)         ^S
 20  DC4 (device control 4)         ^T
 21  NAK (negative acknowledge)     ^U
 22  SYN (synchronous idle)         ^V
 23  ETB (end of trans. block)      ^W
 24  CAN (cancel)                   ^X
 25  EM  (end of medium)            ^Y
 26  SUB (substitute)               ^Z
 27  ESC (escape)                   ^[
 28  FS  (file separator)           ^\
 29  GS  (group separator)          ^]
 30  RS  (record separator)         ^^
 31  US  (unit separator)           ^_


The conversion is done by qstat , pbs_rstat, printjob, tracejob, pbsnodes, and qmgr when printing to stdout, the attribute/resource values as well as hook names.

No storing of the converted value is done internally by PBS, as the change is only cosmetic, preventing terminals from altering display when seeing special non-printing characters.

So for example, given:

qsub -A  $(tput bold; tput setaf 1) -- /bin/sleep 60

<jobid>

qstat -f <jobid> would now display Account_Name value with the ^<translated character> notation:

Account_Name = h^[[1m^[[31md


NOTE: The output conversion is only done in Linux/Unix code.


OSS Site Map

Project Documentation Main Page

Developer Guide Pages