Ramp Rate Limiting Use Cases and Requirements

Forum discussions for UCR

1. Background:
   Ramp rate limiting: controlling the rate of change in power consumption in the lifecycle of a job. Cray provides a capmc control knob to set C-state limits and sleep state limit. Delivering the delta in power consumption for each CPU is too expensive, cannot be delivered. Cray will provide an optional configuration to record (and retrieve) the maximum "delta"/"ramp" for the whole system. The sysadmin sets the allowed delta/ramp. Ramp limiting can be "simplified": for example setting a constant sleep state limit will keep the system to spike down. Similarly, setting a minimum could be used to avoid ramp up in the power consumption. A core changes C-state every few microseconds: C-0 is the maximum power consumption state, the "idler" the core becomes the deeper the C-state it goes (C-1, C-2, ...C-6).
2. Use Cases:
   
   1.  Prevent steep swings in power usage as jobs start/end
      
      1. Step nodes through power steps at random intervals
      2. It is acceptable to set a minimum C-state above C-6 to prevent spikes (e.g. C-1)
   2. Allow nodes that are going to be idle to drop to C6
      
      1. Allow an admin to set maximum number/percent of nodes that are allowed to drop to C-6
      2. Note that on a busy system nodes should never drop to C-6, they will stay at the minimum C-state (e.g. C-1) and higher
3. Requirements:
   1. There shall be a switch to allow an admin to enable/disable ramp rate limiting
   2. There shall be a minimum C-state of C-1 set. A core's power needs can go up and down but the core shall normally drop only to C-1
   3. If PBS determines that a node/set of nodes will be idle for a time PBS shall step the node(s) down to C-6
   4. When PBS determines that a node/set of nodes will be needed for a job PBS shall step the node(s) back up to C-1
   5. PBS shall step the nodes between the minimum C-state and C-6 in random sleep intervals