Follow the PBS Pro Design Document Guidelines.
...
- qmgr -c "export hook pbs_cgroups application/x-config default pbs_cgroups.cf"
This will create a file called "pbs_cgroups.cf" in the current directory. - edit Edit the pbs_cgroups.cf file and set "vnode_per_numa_node" to true
Example: "vnode_per_numa_node" : true - Edit the pbs_cgroups.cf file and set "use_hyperthreads" to true
Example: "use_hyperthreads" : true - If memory subsystem of cgroups is not mounted on the system, disable 'memory' and 'memsw' from the hook config file. That is,
# mount | grep mount
← returns empty, and not something like "cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory"
If the mount returns empty, then edit the pbs_cgroups.cf file such that 'enabled' values for 'memory' and 'memsw' under cgroup are false:
"cgroup": {
...
"memory": {
"enabled": false,
...
"memsw": {
"enabled": false,
5. Re-import the updated hook config file:
qmgr -c "import hook pbs_cgroups application/x-config default pbs_cgroups.cf"
6. Enable the cgroups hook:
qmgr -c "set hook pbs_cgroups enabled=true"
7. kill -HUP each pbs_mom so it can get the latest hook information
For sites with large NUMA machines and machines with GPUs, it is recommended to set vnode_per_numa_node to true.
...
Project Documentation Main Page
...