Setting the scheduler host name that the server uses
Follow the PBS Pro Design Document Guidelines.
Links
- Link to the new forum discussion: Discussion Forum Topic
- Link to original and now outdated discussion on the Obsolete Discussion Forum
- Link to the original and now outdated pull request: https://github.com/PBSPro/pbspro/pull/1200
Overview
When running the PBS server and scheduler in a Kubernetes pod the scheduler can’t connect to the server because it looks like the connections are coming from a different IP address than the server’s. To get around this issue, there are a couple of options that already exist in PBS.
Possible solutions
Option 1: start the scheduler with -c <clientsfile>
The admin/PBS manager would have to make a file that is a list of known hosts allowed to connect to the scheduler. And then the admin/PBS manager would have to start the scheduler by manually typing pbs_sched -c <clientsfile>. This would mean the PBS start script located at /etc/init.d cannot be used for starting the scheduler (at least not without modifying the script).
Option 1a:
Change the init.d script to make a configuration file of a particular name and location get auto-loaded during startup of the scheduler. The admin would still need to create the specified configuration file.
Option 2:
Change the IP address that the server will use to connect to the scheduler. This can be done with PBS today by using the existing -S <default scheduler port> option of pbs_server. It is possible to also pass in a hostname:port number. However, this means the server will have to be started separately from the script
Although in both options 1 and 2 above, it is possible to have an admin/PBS manager edit the init.d script, it’s probably not the best idea.
Option 3:
Create a new PBS conf variable to be able to provide the hostname portion of the
pbs_server -S option. There already exists a conf variable called PBS_SCHEDULER_SERVICE_PORT that tells the scheduler which port it should use.
- The new conf variable will be called PBS_SCHEDULER_HOST_NAME.
(this will match the server’s variable (PBS_SERVER_HOST_NAME) - The pbs_server -S option will have higher priority than the proposed conf variable.
However, but then there came questions about how this affects multisched, and whether PBS_SCHEDULER_HOST_NAME should be used (if it exists) when creating the multiple schedulers.
And also whether it would be okay to have the admins set the sched_host value for the other non-default schedulers differently from the default scheduler. 2 different methods for default vs multisched seemed like a bad idea.
This lead us to option 4.
Option 4:
Update qmgr to allow the default scheduler's sched_host value to be set.
In this way the multisched scheduler's sched_host and the default scheduler's sched_host would be set in the same way.
The PBS admin will have to call qmgr to set the sched_host for the default scheduler when they want the server to use a different hostname to connect to the scheduler.
Example of the syntax:
qmgr -c "set sched default sched_host=<scheduler host name>"
There will be no change to the default behavior of sched_host for the default scheduler.
There will also be no change to the behavior of sched_port, sched_priv, sched_logs, etc as defined by the multisched feature (PP-337: Multiple schedulers servicing the PBS cluster)
We believe this will be the most practical way to set the scheduler name for the server.
Project Documentation Main Page