Follow the PBS Pro Design Document Guidelines.
The special pbs_mom.cpuset binary is only used in HPE systems MC990X/UV300 and ICE-X to run jobs using cpusets. There's special code in pbs_mom that is enclosed in #ifdef MOM_CPUSET that is compiled to create pbs_mom.cpuset. Now cgroups also supports cpusets, and running the standard pbs_mom with pbs_cgroups hook enabled works just as well.
Instead of running pbs_mom.cpuset, one can just enable the pbs_cgroups hook using the following steps:
# mount | grep mount
← returns empty, and not something like "cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory"
If the mount returns empty, then edit the pbs_cgroups.json file such that 'enabled' values for 'memory' and 'memsw' under cgroup are false:
"cgroup": {
...
"memory": {
"enabled": false,
...
"memsw": {
"enabled": false,
5. Re-import the updated hook config file:
qmgr -c "import hook pbs_cgroups application/x-config default pbs_cgroups.json"
6. Enable the cgroups hook:
qmgr -c "set hook pbs_cgroups enabled=true"
7. Restart pbs_mom so it can get the latest hook information:
# ps -eaf | grep pbs_mom
<mom_pid> ... /opt/pbs/sbin/pbs_mom
# kill <mom_pid>
# /opt/pbs/sbin/pbs_mom
For sites with large NUMA machines and machines with GPUs, it is recommended to set vnode_per_numa_node to true.
Project Documentation Main Page