PP-425 to PP-434 - Server Periodic hooks support.

• • • Interface 1: New server periodic site hook event type "periodic"

    • Visibility: Public
    • Change Control: Stable
    • Details: Admin can create a server periodic hook. which would run in timely fashion as per the frequency set by the admin.
      • There is a new event type "periodic" which is used to create a server periodic hook

      • Usage:

        qmgr -c "create hook h1"

        qmgr -c "set hook h1 event=periodic, enabled=1, freq=120"

        qmgr -c "import hook h1 application/x-python default <script path>"

      • This hook runs as pbsadmin and will have all manager privileges and it will be able to modify attributes of objects as a manager.

      • While setting the hook's event as periodic if the user attribute of the hook is found out to be anything other than pbsadmin, the following error shows up on console

        "qmgr obj=h1 svr=default: user value of a server periodic hook must be pbsadmin
        qmgr: hook error returned from server"

        
        

    • Interface 2: When/where does a server periodic hook run

    • • Visibility: Public
      • Change Control: Stable
      • Details: 
        • • Server periodic hook can start at three different instances provided server is able to find the hook content (i.e. hook is already imported)
                          1st Instance - When a hook is enabled
                          2nd Instance - When a hook is imported
                          3rd Instance - When PBS server starts up
    • • • At all these instances if an already existing server periodic hook is found with the same name then that hook gets reloaded which means at that point in time we will restart the hook frequency.
        • Server periodic hook will run at the frequency set for the hook.
        • Server periodic hook always run in background and does not block pbs_server.
        • In case of failover, Secondary server will start the hook frequency of all registered server periodic hooks as soon as it takes control.
          
          

    • Interface 3: frequency of server periodic hook

      • Visibility: Public

      • Change Control: Stable
      • Details: 
        • For a server periodic hook by default the frequency is considered to be 120 seconds. 
        • Frequency can not be negative or 0, the number set in frequency always represents seconds.
          • If a user tries to set frequency to anything <= 0 then qmgr throws following error on console
            "qmgr obj=h1 svr=default: set_hook_freq: freq value '<value specified>' of a hook must be > 0
             qmgr: hook error returned from server"
        • If server finds the same server periodic hook to be already running when it's hook frequency expires, it will neither kill the already running hook nor it start another one. At this point server will just restart the hook frequency.
          
          
    • Interface 4: What happens when accept()/reject() is called in a server periodic hook
      • Visibility: Public
      • Change Control: Experimental
      • Details: 
        • One can still call pbs.event().accept() or pbs.event().reject() inside server periodic hook. 
        • When accept() is called, all the changes done on objects exposed in the hook are committed to PBS server.
        • Upon a reject() an error message "run_periodic_hook;periodic request rejected by <hook name>" gets logged in server log and 
        • no changes are committed to server.
          • Calling pbs.event().reject() will not affect the next iteration of server periodic hook, It will continue to run as it is supposed to.

• • • Interface 5: Changes to server object
      • Visibility: Public
      • Change Control: Experimental
      • Details:
        
        • Server object exposes an interface "create" to create a new object to server. This interface can only accept one of the following as inputs: node object, queue object.
          • It takes the object that needs to be added to server as the input. 
          • usage :
            n1 = pbs.vnode()
            n1.name = "mom1"
            n1.resources_available[ncpus] = 32
            pbs.server().create(n1)
          • When server object's interfaces like vnode(),queue(),resv(),job() can be used to create an empty node, queue, reservation, job object respectively. To create an empty object user has to make sure that nothing is passed as an argument to these interfaces.
        • Server object exposes an interface "delete" to delete an object from server. This interface can only be one of the following: job object, node object, reservation object, queue object.
          • It takes the object that needs to be deleted as an input.
          • usage:
            n1 = pbs.server().vnode("mom1")
            pbs.server().delete(n1)
        • Server object exposes and interface "submit" to submit a job/reservation to server.
          • It takes the job/reservation object that needs to be submitted as an input.
          • usage:
            j1 = pbs.job()
            j1.Resource_list.["ncpus"] = int(4)
            j1.executable = "<jsdl-hpcpa:Executable>top</jsdl-hpcpa:Executable>"
            pbs.server().submit(j1)
      
      
    • Interface 6: event() object
      
      • Visibility: Public
      • Change Control: Experimental
      • Details:
        
        • Since server periodic hook runs on timely basis, So the event object has no job context or event associated with it.
        • To manage the complex like adding/modifying/deleting a new node the hook needs to make of server object (pbs.server())
          
          

• • • Interface 7: Additions to log/error messages

      • Visibility: Public
      • Change Control: Stable
      • Details: If admin tries to change the "order" attribute of a server periodic hook then an info log message "Setting order for a periodic hook has no effect" is logged in the server logs.
        

               09/15/2016 13:58:14;0004;Server@centos;Hook;h1;mgr_hook_set at request of root@centos
       09/15/2016 13:58:14;0004;Server@centos;Hook;h1;Setting order for a periodic hook has no effect

        


        • If one tries to create a hook with a wrong event name then following error message it displayed while executing qmgr command.

Should be one or more of: queuejob,modifyjob,resvsub,movejob,runjob,provision,periodic,execjob_begin,execjob_prologue,execjob_epilogue,execjob_preterm,execjob_end,exechost_periodic,execjob_launch,exechost_startup,execjob_attach or "" for no event

Example:

# qmgr -c "s h h1 event=pop"

qmgr obj=h1 svr=default: invalid argument (pop) to event. Should be one or more of: queuejob,modifyjob,resvsub,movejob,runjob,provision,periodic,execjob_begin,execjob_prologue,execjob_epilogue,execjob_preterm,execjob_end,exechost_periodic,execjob_launch,exechost_startup,execjob_attach or "" for no event

qmgr: hook error returned from server

• • • • • If server periodic hook runs successfully, following message is logged in server log at log level DEBUG3

          "Server periodic hook ran successfully"

        • If server periodic hook encounters error while running and exits abnormally then following message is logged in server log at log level DEBUG3

"Server periodic hook encountered errors"

• If a periodic hook is deleted while it is running then upon it's completion following error message is logged in server log

"A periodic hook disappeared"

• • • Interface 8: Create and add a new node to a server

    • Visibility: Public
    • Change Control: Experimental
    • Details: Admin can create a server periodic hook. which would run in timely fashion and add a new node when need be.
      • Hook  can create a new node object and add this node object to server object
        • To create a blank vnode object, hook must use "vnode" interface exposed by pbs module.
        • Server object exposes a new interface "create" to add a new object to pbs complex. While adding a node this object can be a vnode object.
        • usage :
          n1 = pbs.vnode()
          n1.name = "mom1"
          n1.resources_available["ncpus"] = 32
          pbs.server().create(n1)
        • Node creation will only be attempted when server periodic hook calls accept. In case server fails to create a node, appropriate log message will be logged in server logs.