Support PTL on Shasta
Follow the PBS Pro Design Document Guidelines.
Links
- Link to discussion on Developer Forum: http://community.pbspro.org/t/supporting-ptl-on-shasta/1710
- Link to issue: <issue link if available>
- Link to pull request: <PR link if available>
Overview
In order for PTL to work on Cray's new Shasta systems, some things have to change.
Glossary
Userhosts - hosts that users can log into and submit jobs from. Root does not have ssh access to these hosts. Each user gets their own userhost.
Technical Details
PTL will run where the server is running.
PTL needs to know if we're running on Shasta.
The Shasta hook should be loaded and enabled before running any test on Shasta.
Non-root commands must be run from userhosts. In order to do this, PTL must support running user commands from multiple hosts.
The users do not exist where the server is running on Shasta, so the user-exists check should be changed to check if the PbsUser instance has a userhost set: if it does, check that userhost instead.
Since remote copy doesn't work, we need to set $usecp to tell the moms to do a local copy to the shared directory.
Changes:
- Load the Shasta hook in Server.revert_to_defaults().
- Change usages of Cray to XC, since tests that used to skip on Cray machines could run on Shasta
- Add is_Shasta() to the MoM class.
PTL will look for a file at path /etc/cray/xname and if found return True. - Change DshUtils.getplatform() to return 'shasta' if on a shasta
PTL will look for a file at path /etc/cray/xname and if found return "shasta". - Add an optional hostname and port to the users lists.
The format of this parameter will be -p=test-users=username[@ip[+port]][:username2[@ip2[+port2]]]
If a command needs to be run as a user, it will use the ip and port if they exist, otherwise it will fall back onto the server's client. - Change usage of ssh to connect to the remote host as the intended user, instead of as root
Instead of ssh userhost sudo -u user <cmd>, it will be ssh user@userhost <cmd> - Change PBSTestSuite.check_users_exist to check the user's userhost, if it exists.
Pass PbsUser.userhost to the check_user_exists() call - Set $usecp in mom_priv/config
New as of 9/4
is_localhost should check the hostnames reported, and if they're the server hostname, return `True`
Since we know pbs-host and pbs-service-nmn are the two hostnames for the server on shasta, if they're used in is_localhost, we should return TrueSet PBS_PUBLIC_HOST_NAME on the server conf.
Since commands need this on the server pod to successfully authenticate, this needs to be set to pbs-hostAdd user-host specific code to run_copy, create_temp_file, and Job.create_script
This is to allow submitting jobs with scripts on Shasta, since otherwise they won't run.
The code should check if we're on shasta and the user has a user host - if so, then create the file as root on the local host, then copy it to the user-host as the user.Allow the shasta platform to use proc_utils
There's a check for the platform, just add 'shasta' to the listImplement @skipOnShasta
If self.mom.is_shasta() is true, then skip the test.Change hard-coded usernames to str(TEST_USER)
There are many hard-coded usernames in PTL, when it should be using the name of the PbsUser.
Project Documentation Main Page