Multicore Expert Series: QorIQ Power Management Technologies

June 2012 

TM 

Freescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, 

ColdFire+, C-Ware, the Energy Efficient Solutions logo, Kinetis, mobileGT, PowerQUICC, 

Processor Expert, QorIQ, Qorivva, StarCore, Symphony and VortiQa are trademarks of 

Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. Airfast, BeeKit, BeeStack, 

CoreNet, Flexis, MagniV, MXC, Platform in a Package, QorIQ Qonverge, QUICC Engine, 

Ready Play, SafeAssure, the SafeAssure logo, SMARTMOS, TurboLink, Vybrid and 

Xtrinsic are trademarks of Freescale Semiconductor, Inc. All other product or service 

names are the property of their respective owners. © 2012 Freescale Semiconductor, Inc. 

. 

Li Yang 

Software Engineering Manager

• Saving energy is a world problem 

• Power management features that 

Freescale provides 

• How to use power management features 

TM 

2 

Freescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, ColdFire+, C-Ware, the Energy Efficient Solutions logo, Kinetis, 

mobileGT, PowerQUICC, Processor Expert, QorIQ, Qorivva, StarCore, Symphony and VortiQa are trademarks of Freescale Semiconductor, Inc., 

Reg. U.S. Pat. & Tm. Off. Airfast, BeeKit, BeeStack, CoreNet, Flexis, MagniV, MXC, Platform in a Package, QorIQ Qonverge, QUICC Engine, 

Ready Play, SafeAssure, the SafeAssure logo, SMARTMOS, TurboLink, Vybrid and Xtrinsic are trademarks of Freescale Semiconductor, Inc. All 

other product or service names are the property of their respective owners. © 2012 Freescale Semiconductor, Inc.

• Power Management Basics 

• QorIQ power management features 

• Linux ® power management 

• Design your low-power solution 

TM 

3 






TM 









.

• CMOS Energy Consumption 

− Dynamic Energy Consumption 

− Static Energy Consumption 

TM 

5 






Dynamic Power Static Power 

• Reduce voltage 

• Reduce frequency 

• Clock gating 

• Reduce active-load 

capacitance 

TM 

6 

• Reduce voltage 

• Less leaky transistors 

(better process) 

• Power gating 






• Manual 

− User command (PC-style) 

Standby 

Hibernate 

• Automatic 

− Base on the work load of native tasks (On some PCs) 

− Base on the load of external I/O (QorIQ style) 

TM 

7 






CPU is active CPU is inactive 

• Frequency scaling 

• Shutdown part of the 

cores 

• Multiple power rails 

TM 

8 

• System Sleep 






TM 









.

DDR controller on on on on 

Interrupt controller on on on on 

Devices that generate 

wakeup event 

Voltage for other onchip 

devices 

Clock for other on-chip 

devices 

TM 

Doze Nap Sleep Deep Sleep 

Core voltage on on on off 

Core Timebase on on off off 

Core clock on off off off 

Core L1 Cache on off off off 

Core instruction fetch off off off off 

on on on on 

on on on off 

on on off off 

10 






Off during Deep Sleep On during Deep Sleep 

TM 

11 






DDR controller on on on on 

Interrupt controller on on on on 

Devices that generate 

wakeup event 

Voltage for other onchip 

devices 

Clock for other on-chip 

devices 

TM 

Core Wait Core Doze Core Nap Device Sleep 

Core voltage on on on on 

Core Timebase on on on off 

Core clock on on off off 

Core L1 Cache on on off off 

Core instruction fetch off off off off 

on on on on 

on on on on 

on on on off 

12 






Cluster State PCL00 PCL00 PCL00 PCL00 PCL00 PCL10 

Core State PH00 PH10/PW10 PH15 PW20 PH20 PH20 

Cluster Voltage 

Core Voltage 

Cluster Clock On On On On On Off 

Core Clock On On Off Off Off Off 

L2 Cache SW Flushed 

L1 Cache SW Invalidated HW Invalidated SW Invalidated SW Invalidated 

Wakeup Time Active Immediate < 30 ns < 200 ns < 600 ns < 1us 

Power 

TM 

13 

NEW NEW NEW 






• JOG 

− MPC8536 

− P1022 

− P1023 

TM 

14 






Security 

Accel 

XOR 

LCD 

(DIU) 

TDM 

/I 2 S 

IEEE- 

1588 

2x GE 

MAC 

32kB 

I-Cache 

TM 

New PLL 

Config. 

e500 Core 

STOPPED 

SATA 

PLL 

32kB 

D-Cache 

x2 SerDes 

256KB 

L2 

Coherency Module 

System Bus 

2x DMA 

32kB 

I-Cache 

32kB 

D-Cache 

System Bus 

On-Chip Network 

PCI 

Express ® 

STOPPED 

e500 Core 

PLL 

PCI 

Express 

x4 SerDes 

15 

Reset + 

Clocking Logic 

PLL 

Re-Config 

COP 

STOP 

Advanced 

Power Management 

STOP 

PCI 

Express 

PLL PLL 

Sys PLL PLL 

DDR PLL 

Re-Config 

Fuse 

P1022 

DDR2/DDR3, 

SDRAM 

Controller 

Enhanced 

Local Bus 

Perf Mon, 

DUART, MPIC 

2x I2C, Timers 

SPI 

2x USB2.0 

SD/MMC 






• MPIC Timer 

• External Interrupt 

• Ethernet Magic packet (on some SoCs) 

• Ethernet User defined packet (on some SoCs) 

• USB events (on some SOCs) 

• GPIO lines (on some SoCs) 

• SD card (on some SoCs) 

TM 

16 






• Packets received during sleep buffered in external DDR 

• Possible to sleep longer before wakeup from 

− Timer 

− Magic packet 

− User-defined packet 

− Buffered packets exceed defined threshold 

TM 

17 






CPU 

L2 

Platform 

Enet2 

Enet1 

• Packet-Lossless 

− CPU + L2 Powered Off 

− Path from Enet to DDR powered 

− Power down more SoC logic 

− First packet enables Rx wake-up timer 

TM 

CPU 

Timers 

USB2 

USB1 

DDR 

Ctlr 

DDR 

In self-refresh 

until packet 

arrives 

18 

= Powered down in Deep Sleep 

= Powered in Deep Sleep 






Today: All CPUs in Pool Channel 

dequeue until all FQs empty. 

Broadcast notification when work 

arrives. 

P0 P1 P2 P3 

Shared L2 

Task Queue 

T5 T4 T3 T2 T1 

Threshold 2 

TM 

Threshold 1 

QMan 

19 

AMP Series: DPAA uses task queue 

thresholds to inform CPUs they 

are not needed. CPUs selectively 

awakened as needed. 

P0 P1 P2 P3 

Drowsy 

Shared L2 

Drowsy 

Active CPUs 

12 

11 

10 

9 

8 

7 

6 

5 

4 

3 

Burst 

2 

1 

Day Night 






• e6500 core reintroduces support for AltiVec technology 

• AltiVec technology has very specific use cases that don’t apply to 

every usage or application that may run on the core 

• In order to reduce power for non-AltiVec code streams, the e6500 

core introduces a Drowsy AltiVec unit 

− 10% power savings vs. dynamic clock gating of AltiVec technology alone 

• The drowsy AltiVec mode can be entered through software control 

or through a hardware-triggered event 

− The hardware-triggered mode uses an instruction counter to count the number 

of non-AltiVec instructions. When a certain threshold is exceeded, then the 

Power Management unit will put the AltiVec unit into Drowsy mode. Software 

can enable or disable this mode and also set the count threshold 

TM 

20 






TM 









.

• CPU idle 

• CPU hotplug 

• System suspend 

− standby 

− mem 

• System hibernation 

• Cpufreq 

• DPM 

TM 

22 






• cpu_idle() -- arch/powerpc/kernel/idle.c 

− A never-ending task on each CPU 

− Runs when there is nothing else to run 

− Calls ppc_md.power_save() when possible 

• e500_idle() -- arch/powerpc/kernel/idle_e500.S 

− The ppc_md.power_save() for e500 platforms 

− Calls wait instruction for e500mc platforms 

− Enters Doze state on other e500 platforms 

− Enters Nap instead of Doze when defined powersave_nap kernel 

parameter or set sysfs entry. 

− Exits when there is an interrupt to the core 

TM 

23 






• Removes or plugs a CPU to the Linux system 

• Build-time option: 

− CONFIG_HOTPLUG_CPU 

• Run-time control through sysfs: /sys/devices/system/cpu/* 

− Removing 

#echo 0 > /sys/devices/system/cpu/cpuX/online 

− Plugging 

#echo 1 > /sys/devices/system/cpu/cpuX/online 

TM 

24 






• Send CPU_DOWN_PREPARE notification to all in-kernel 

interested modules 

• Migrate all processes to other CPU(s) 

• Migrate all interrupts to a new CPU 

• Migrate timers/bottom half/tasklets to a new CPU 

• Call an arch specific routine __cpu_disable() 

• Finally send CPU_DEAD notification to all in-kernel modules 

TM 

25 






1. __cpu_disable() calls platform specific cpu_disable 

1. Clear bit the online cpu mask 

2. Migrate IRQ 

2. On the cpu to be unplugged 

1. cpu_idle() will find its offload in cpu_mask 

2. Calls platform specific ppc_md.cpu_die() = 

smp_85xx_mach_cpu_die() 

3. Flush and disable L1 

4. Actually set the cpu to NAP state 

TM 

26 






• Put system into low-power states on user command 

• Suspend states: 

− Standby 

− Suspend to RAM (mem) 

− Suspend to disk/hibernation (disk) 

• Run-time control through sysfs 

#echo state > /sys/power/state 

• Wakeup through interrupts 

− Different among SoCs 

TM 

27 






1. Sync file systems 

2. Prepare 

1. Send PM_SUSPEND_PREPARE notification 

2. Disable user-mode helper 

3. Freeze processes 

3. Suspend console 

4. Call suspend callbacks of device drivers (interrupt enabled) 

5. Platform-specific suspend prepare operations 

6. Call late suspend callbacks of device drivers (interrupt disabled) 

7. Disable all CPUs other than the booting CPU 

8. Disable all interrupts 

9. Call suspend callbacks for system core devices 

10. Call platform-specific suspend_ops->enter() 

TM 

28 






pmc_suspend_enter() -- arch/powerpc/sysdev/fsl_pmc.c 

1. Disable local interrupts 

2. Flush L1 dcache 

3. Setting SLP bit in powmgtcsr register 

4. Hardware enter SLEEP state 

TM 

29 






mpc85xx_enter_deep_sleep() -arch/powerpc/platforms/85xx/sleep.S 

1. Enable SPE/FPU 

2. Disable local interrupt 

3. Backup important SPR registers in memory 

4. Save current BPTR 

5. Flush and disable L2 and L1 cache 

6. Set resume entry point mpc85xx_deep_resume() to 

BPTR(boot page translate register) 

7. Disable decrementer 

8. Set deep sleep bit in power management unit 

TM 

30 






• Hardware wait for POWER_OK signal or voltage ramp up timer 

expire 

• Core reboot from page mapped by BPTR -- 

mpc85xx_deep_resume() 

• Enable L2 cache 

• Restore original BPTR and jump to normal memory address 

• Restore TLB1 

• Restore SPR registers 

• Return to common resume code (opposite to suspend 

sequence) 

TM 

31 






• Jog mode 

• Change the core frequency to 533 MHz for each core 

− # echo 500000 > 

/sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed 

− # echo 500000 > 

/sys/devices/system/cpu/cpu1/cpufreq/scaling_setspeed 

TM 

32 






• PM notifier 

• SCHED_MC 

TM 

33 






• Make it possible for subsystems to register notifiers on 

suspend and hibernation events 

TM 

34 






• Linux scheduling policy 

• /sys/devices/system/cpu/sched_mc_power_savings 

• Performance drawback 

• Not yet supported on Power Architecture ® 

TM 

P0 P1 P2 P3 

Shared L2 

35 

P0 P1 P2 P3 

IDLE 

Shared L2 

IDLE 






• CPU topology 

− SMT 

− Multicore 

− CPU packages 

− NUMA 

TM 

36 






TM 









.

100% 

80% 

60% 

40% 

20% 

0% 

Boot up 

No work, 

enter deep 

sleep 

TM 

Process 

request 

Wakeup by 

pre-defined 

network 

packet 

No work, 

enter deep 

sleep 

38 

Maintenance 

work, enter 

JOG 

Wakeup by 

pre-defined 

network 

packet 

Work 

finished, 

enter deep 

sleep 






• Examine the load profile of user application 

• Define a policy for entering low power states 

− When? 

− Which state? 

• A compromise of power saving and response time 

TM 

39 






• Clock Generators 

− Power only exists while switching. By definition, a clock exerts power 

every half cycle. 

• Gigabit Ethernet PHY 

− 10/100 power is 50% or less than that of 1G 

• Power Supplies 

− Can be extremely inefficient if not designed carefully (85% is 

considered great?!) 

• DDR3 Clock Terminations 

− Don’t let the DDR3 vendors fool you! There is more power hidden than 

you realize. 

• 3.3V Rails 

− 3.3 time X amps adds up fast 

• Thermals 

− Keep it cool! 

TM 

40 






• Consider clock consolidations – less than 166 MHz should not 

be an issue from a signal-integrity perspective 

− DDRCLK tied to SYSCLK 

− PCI CLK tied to SYSCLK 

• Watch those buffers 

− Some multiple input buffers still leak or run even without a need 

− Turn them off in SW or with proper strapping 

TM 

41 






• Can your application use 10/100? 

− 1G is becoming the norm 

− Possibly the biggest source of power on the system in a low-power 

state and 2nd largest in a dynamic state after the processor itself 

• Don’t forget the transmitters 

− In deep sleep the link must stay active 

− Need low power phys to accommodate 

TM 

42 






• Designing a power supply that can perform well, deliver at peak 

power, AND be efficient during low power modes is challenging 

• Efficiency generally goes down when drawing low power 

• Upwards of 95% efficiency can be realized with some research 

and simulation 

TM 

43 






• While DDR3 devices can achieve very low power in self-refresh 

mode, there is no way to disable the clock termination during 

these modes 

• With standard termination, this can consume almost 200 mW 

• Through careful simulation, increasing the value of the 

termination can achieve closer to 80 mW 

• There may be other termination schemes (such as LVDS-like 

termination) that may improve the power slightly 

TM 

44 






• Freescale devices require OVdd at 3.3V 

− Runs much of the I/O on the device 

• Consider the 2.5V rail where appropriate 

• Minimize the power on that rail during deep sleep 

− Delete the I/O that is not necessary to the final product (BOM 

optimize for power, not just cost) 

TM 

45 






• Can you afford a bigger heatsink? 

• Can you afford an active heatsink? 

• Can you increase the airflow? 

• Can you maintain ambient to 25 o C during deep sleep? 

TM 

46 






• Power Management Basics 

• QorIQ power management features 

• Linux ® power management 

• Design your low-power solution 

TM 

Freescale on Kaixin 

Tag yourself in photos 

and upload your own! 

Session materials will be posted @ www.freescale.com/FTF 

47 

Weibo? 

Please use hashtag 

#FTF2012#

Multicore Expert Series: QorIQ Power Management Technologies

Create successful ePaper yourself

Delete template?

Save as template?