BACKGROUND: A QUICK REFRESHER ON IDLE STATES
Here is a quick summary of what C-states are. C-states are idle power saving states, in contrast to P-states, which are execution power saving states. During a P-state, the processor is still executing instructions, whereas during a C-state (other than C0), the processor is idle, meaning that nothing is executing. To make a quick analogy, a processor lying idle is like a house with all the lights on when no one is at home. Consuming all that power is doing nothing other than providing your electric company a little extra income. What is the best option? If no one is at home, meaning the house is idle, why leave the lights on? The same applies to a processor. If no one is using it, why keep the unused circuits powered up and consuming energy? Shut them down and save.
C0 is the “null” idle power state, meaning it is the non-idle state when the core is actually executing and not idle.
THE DIFFERENCE BETWEEN CORE AND PACKAGE IDLE STATES
The coprocessor has up to 60+ cores in one package. Core idle power states (C-states) are per core, meaning that one of those 60+ cores can be in C0, i.e. it is executing and not idle, while the one right next door is in a deep power conservation state of C6. In contrast, PC-states are Package idle states which are idle power conservation states for the entire package, meaning all 60+ cores and supporting circuitry on the silicon. As you can guess, to drop the package into a PC-6 state, all the cores must also be in a C6 state. Why? Since the package has functionality that supports all the cores, to “turn off” some package circuitry impacts all of them.
 |
| Figure 1 Dropping a core into a core-C1 state |
WHAT CORE IDLE STATES ARE THERE?
Each core has 2 idle states, C1 and C6 (and, of course, C0).
C0 to Core-C1 Transition: Look at Figure 1. C1 happens when all 4 hardware (HW) threads supported by a core have executed a HALT instruction. At this point, let us now think of each of the 4 HW threads as the Operating System (OS) perceives them, namely as 4 separate CPUs (CPU 0 through 3). Step 1: the first three CPUs belonging to that core execute their HALT instruction. Step 2: that last CPU, CPU-0 in the illustration, attempts to execute its HALT instruction. Step 3: it interrupts to an idle residency data collection routine. This routine collects, you guessed it, idle residency data and stores that data in a data structure accessible to the OS. CPU 0 then HALTs. Step 4: at this point, all the CPUs are halted and the core enters a core-C1 state. In the core-C1 state, the core (and its “CPUs”) is clock gated1.
 |
| Figure 2 Is it worth entering C6: Is the next interrupted far enough out? |
 |
| Figure 3 Is it worth entering C6: Is the estimated idle time high enough? |
After entering core-C1: Now that the core is in C1, the coprocessor’s Power Management routine comes into play. It needs to figure out whether it is worthwhile to shut the core down further and drop it into a core-C6 state. In a core-C6 state, further parts of the core are shut down and power gated. Remember that the coprocessor’s Power Management SW executes on the OS core, typically core 0, and is not affected by the shutdown of other cores.
What type of decisions does the coprocessor’s Power Management have to make? There are two primary ones as we discussed in the last blog2: Question#1: Will there (probably) be a net power savings? Question #2: Will any restart latency adversely affect the performance of the processor or of applications executing on the processor? Those decisions correspond to two major scenarios and are shown in the two clever illustrations above. (See Figures 2 and 3.) Scenario 1 is where the coprocessor PM looks at how far away is the next scheduled or expected interrupt. If that interrupt is soon enough, it may not be worth shutting down the core further and suffering the added latency caused by bringing the core back up to C0. As is the case in life, the processor can never get anything for free. The price of dropping into a deeper C state is an added latency resulting from bringing the core/package back up to the non-idle state. Scenario 2 is where the coprocessor’s Power Management looks at the history of core activity (meaning its HW threads) and figures out whether the execution (C0) and idle (C1) patterns of the core make core-C6 power savings worthwhile.
If the answers to both of these questions are “yes”, then the core drops down into a core-C6 state.
After entering core-C6: Well dear reader, it looks like I have run out of time. The processor next decides if it can drop into the package idle states. I will cover that discussion in my next blog in this series.
NEXT: PACKAGE C-STATES, THE DETAILS
REFERENCES
For those of you with a passion for power management, check out the Intel® Xeon Phi™ Coprocessor Software Developer’s Guide. It has state diagrams and other goodies. I recommend sections 2.1.13, “Power Management”, and all of section 3.1, “Power Management (PM)” for your late night reading.
Kidd, Taylor, “There's got to be a catch,” Intel® Corporation, April 29th, 2008. http://software.intel.com/en-us/blogs/2008/04/29/theres-got-to-be-a-catch.
Kidd, Taylor, “(update) C-states, C-states and even more C-states,” Intel® Corporation, March 27th, 2008. http://software.intel.com/en-us/blogs/2008/03/27/update-c-states-c-states-and-even-more-c-states.
ENDNOTES
1All CPUs have at least one oscillator (clock) that emits a timing pulse. The circuits of the processor use this timing pulse to coordinate all activities.
Immagine icona:
