Support PTL on Shasta

Owned by Vincent Stumpf (Deactivated)
Last updated: Sept 04, 2019
2 min read

Follow the PBS Pro Design Document Guidelines.

Overview

In order for PTL to work on Cray's new Shasta systems, some things have to change.

Glossary

Userhosts - hosts that users can log into and submit jobs from. Root does not have ssh access to these hosts. Each user gets their own userhost.

Technical Details

PTL will run where the server is running.

PTL needs to know if we're running on Shasta.

The Shasta hook should be loaded and enabled before running any test on Shasta.

Non-root commands must be run from userhosts. In order to do this, PTL must support running user commands from multiple hosts.

The users do not exist where the server is running on Shasta, so the user-exists check should be changed to check if the PbsUser instance has a userhost set: if it does, check that userhost instead.

Since remote copy doesn't work, we need to set $usecp to tell the moms to do a local copy to the shared directory.

Changes:

  1. Load the Shasta hook in Server.revert_to_defaults().
  2. Change usages of Cray to XC, since tests that used to skip on Cray machines could run on Shasta
  3. Add is_Shasta() to the MoM class.
    PTL will look for a file at path /etc/cray/xname and if found return True.
  4. Change DshUtils.getplatform() to return 'shasta' if on a shasta
    PTL will look for a file at path /etc/cray/xname and if found return "shasta".
  5. Add an optional hostname and port to the users lists.
    The format of this parameter will be -p=test-users=username[@ip[+port]][:username2[@ip2[+port2]]]
    If a command needs to be run as a user, it will use the ip and port if they exist, otherwise it will fall back onto the server's client.
  6. Change usage of ssh to connect to the remote host as the intended user, instead of as root
    Instead of ssh userhost sudo -u user <cmd>, it will be ssh user@userhost <cmd>
  7. Change PBSTestSuite.check_users_exist to check the user's userhost, if it exists.
    Pass PbsUser.userhost to the check_user_exists() call
  8. Set $usecp in mom_priv/config

New as of 9/4

  1. is_localhost should check the hostnames reported, and if they're the server hostname, return `True`

    Since we know pbs-host and pbs-service-nmn are the two hostnames for the server on shasta, if they're used in is_localhost, we should return True

  2. Set PBS_PUBLIC_HOST_NAME on the server conf.

    Since commands need this on the server pod to successfully authenticate, this needs to be set to pbs-host

  3. Add user-host specific code to run_copy, create_temp_file, and Job.create_script

    This is to allow submitting jobs with scripts on Shasta, since otherwise they won't run.
    The code should check if we're on shasta and the user has a user host - if so, then create the file as root on the local host, then copy it to the user-host as the user.

  4. Allow the shasta platform to use proc_utils
    There's a check for the platform, just add 'shasta' to the list

  5. Implement @skipOnShasta
    If self.mom.is_shasta() is true, then skip the test.

  6. Change hard-coded usernames to str(TEST_USER)
    There are many hard-coded usernames in PTL, when it should be using the name of the PbsUser.


OSS Site Map

Project Documentation Main Page

Developer Guide Pages