Follow the PBS Pro Design Document Guidelines.
...
- qmgr -c "export hook pbs_cgroups application/x-config default pbs_cgroups.cfjson"
This will create a file called "pbs_cgroups.cfjson" in the current directory. - Edit the pbs_cgroups.cf json file and set "vnode_per_numa_node" to true
Example: "vnode_per_numa_node" : true - Edit the pbs_cgroups.cf json file and set "use_hyperthreads" to true
Example: "use_hyperthreads" : true - If memory subsystem of cgroups is not mounted on the system, disable 'memory' and 'memsw' from the hook config file. That is,
...
If the mount returns empty, then edit the pbs_cgroups.cf json file such that 'enabled' values for 'memory' and 'memsw' under cgroup are false:
...
5. Re-import the updated hook config file:
qmgr -c "import hook pbs_cgroups application/x-config default pbs_cgroups.cfjson"
6. Enable the cgroups hook:
qmgr -c "set hook pbs_cgroups enabled=true"
7. kill -HUP each Restart pbs_mom so it can get the latest hook information:
# ps -eaf | grep pbs_mom
<mom_pid> ... /opt/pbs/sbin/pbs_mom
# kill <mom_pid>
# /opt/pbs/sbin/pbs_mom
For sites with large NUMA machines and machines with GPUs, it is recommended to set vnode_per_numa_node to true.
...
Project Documentation Main Page
...