PP-1018: [STALLED] As an admin, I would like to be able to prescribe the order in which placement sets are considered, so that I can direct the scheduler always consider first a particular set of resources

Currently the order of placement set used while job placement is hard coded as below and cannot be customized by admin

The scheduler examines the placement sets in the pool and orders them, from smallest to largest, according to the following rules:

The new interface introduced in this EDD, node_group_sort_key will give the admin a free hand in customizing the order of placement sets considered in placement pool during job placement. This new interface provides multiple sorting domains in a placement pool with multiple placement series.

The syntax of node_group_sort_key is similar to node_sort_key sched_config, i.e a multi-word multi-line key, as described below

node_group_sort_key:  “<resource>  HIGH | LOW      total | assigned | unused”      <prime option>

where
resource    could be a custom vnode resources (including string / string array) or built-in vnode resource like ncpu or mem

total           Use the resources_available value
assigned   Use the resources_assigned value
unused      Use the value given by resources_available - resources_assigned

**ignored for non-consumable resources

HIGH    to sort descending order i.e, high first and low last
LOW     to sort ascending order i.e, low first and high last

Unlike node_sort_key, here the resource is not restricted to numerical value. The idea is to allow use of string values such as "string array resource names that define placement series" to order placement sets as per alphabetical order of the string values used in defining each placement set.

node_group_sort_key can be defined more than once in multiple lines with different resources in the sched_config. The scheduler will order the placement sets based on multiple node_group_sort_key in the order it appears in sched_config. This behavior is same as in node_sort_key

Default Value:

The default value of node_group_sort_key shall be defined in the sched_config as below. This value will be equivalent to the current hard coded rules mentioned at the beginning above.

Current behavior: Placement sets are created and partitioned based on the different string values defined at the vnodes under custom string array resources named in node_group_key list, indexed in a single dimention, i.e after flatening the resource names.  For example, if the server’s node_group_key attribute contains “router,switch”, and router can take the values “R1” and “R2” and switch can take the values “S1”, “S2”, and “S3”, then there are five placement sets "R1, R2, S1, S2, S3", in two placement series, in the server’s placement pool.

New behavior introduced with this EDD: Placement sets are created and partitioned based on the combination of different string values defined at the vnodes under custom string array resources named in node_group_key list, indexed multi dimensionally with custom string array resources named in the node_group_key as the index appearing in that order. This is in addition to the placement sets created as in current behavior, i.e flatened index.  For example, if the server’s node_group_key attribute contains “router,switch”, and router can take the values “R1” and “R2” and switch can take the values “S1”, “S2”, and “S3”, then there are elleven placement sets "R1-S1, R1-S2, R1-S3, R2-S1, R2-S2, R2-S3, R1, R2, S1, S2, S3", in two placement series, in the server’s placement pool.

**The implicit unset placement set and the placement set containing all nodes is not shown here.