PP-719: Enhance setUp in PTL specifically for Cray platforms

Community discussion is located here: http://community.pbspro.org/t/pp-719-enhance-setup-in-ptl-specifically-for-cray-platforms/463

Overview:

• PBS is installed and configured on Cray which result in specific settings. These initial settings are referred to as the out-of-box settings below.  These settings should also apply to the Cray ALPS simulator when PBS is properly configured.  These are settings for a freshly installed and configured PBS on Cray or Cray ALPS simulator.

• As of current support for Cray CLE 5.2 and 6.0, below are the out-of-box configuration settings per PBS daemon:

•  

  • These lines should appear in PBS_HOME/mom_priv/config:

  •         $vnodedef_additive 0

  •         $alps_client <path to ALPS API>

  •         $usecp *:<Absolute path to /home directory>  <Absolute path to /home directory>
    
    Notes: $usecp setting above is used to enable local copy of job output of jobs that ran on compute nodes.
    $vnodedef_additive 0 will allow MoM to tell the server that there are vnodes missing, which the server will mark as stale.
    
    
    

•  

  • PBS_HOME/sched_priv/sched_config will have 'vntype' in the resources line.

•  

  • Based on what is returned by the ALPS inventory :

    • vnodes representing the compute nodes will get created

    • the Cray-specific custom resources (e.g. PBScraynid, PBScrayhost, etc) will get created in PBS_HOME/server_priv/resourcedef.

    • the appropriate Cray-specific custom resources will be added to the vnodes.

    • since the mom config $vnode_per_numa_node is unset by default, there will be no PBScrayseg attribute on the vnodes representing the compute nodes.

  • Server settings :

    • flatuid = true

    • scheduling = true

    • the default queue is workq

Interface:  MoM.__init__() 

• Visibility: Public

• Change Control: Stable

• Synopsis: Modifications to include initial MoM config settings for Cray.
  
  

• Standing of the interface : modified interface
  
  

• Details

  • if on real Cray or Cray ALPS simulator, then initialize the PBS_HOME/mom_priv/config to have these:

  • $vnodedef_additive 0

  • $alps_client <path to ALPS API>

  •                $usecp *:<Absolute path to /home directory>  <Absolute path to /home directory>

Interface:  Scheduler.revert_to_defaults() 

• Visibility: Public

• Change Control: Stable

• Synopsis: Modifications to include 'vntype' in sched_config out-of-box settings for Cray.
  
  

• Standing of the interface : modified interface
  
  

• Details

  • After copying the out-of-box scheduler configuration in PBS_EXEC/etc/pbs_sched_config to PBS_HOME/sched_priv/sched_config :

    • If platform is a real Cray or Cray ALPS simulator, then add "vntype" to resources line of PBS_HOME/sched_priv/sched_config.

Interface:  PBSTestSuite.revert_mom()

• Visibility: Public

• Change Control: Stable

• Synopsis: Modifications to have additional steps of deleting and creating nodes back, for retaining Cray specific resources.

• Standing of the interface : modified interface

• Details

  • If on a Cray or Cray ALPS simulator then perform below:

    • Delete all nodes and create them back.

Interface: Server.revert_to_defaults()

• Visibility: Public

• Change Control: Stable

• Synopsis: Modifications to hooks settings for Cray.

• Standing of the interface : modified interface

• Details
  
  

  • If on a Cray or Cray ALPS simulator then perform below:

    • Restore the default 'PBS_translate_mpp' hook by:

      • Copying PBS_EXEC/lib/python/altair/pbs_hooks/PBS_translate_mpp.HK to PBS_HOME/server_priv/hooks/PBS_translate_mpp.HK

        • In this way, the shipped version (i.e. default version) of the hook will be in PBS_HOME for any PTL test to enable if desired.