Robust System Design - VLSI

Robust System Design
Subhasish Mitra
Robust Systems Group
Dept. of EE & Dept. of CS
Stanford University
www.stanford.edu/~subh
Acknowledgment: Students & Collaborators
1

Radiation-Induced Soft Errors
“It’s ridiculous. I’ve got a $300,000
server that doesn’t work. The thing
should be bullet-proof.”
Causes: α particles (package), Neutrons (cosmic rays)
2

Who Cares About Soft Errors ?
20K processors server farm
1 major flip-flop error every 20 days
Silent data corruption
$ 20K $ 3,616 bank deposit
Downtime: $100K - $10M / hr.
Memory ECC routinely used
System error rates increasing
Comb. logic
Flipflop
Un
protected
memory
Soft error rate
contributions
3

System Outage Causes
Unknown
Power, operator
Software
Hardware
HPC, Los Alamos [Schroeder DSN 06]
(Disk failures excluded)
Hardware failures can be significant
Depends on application
Power
Software Hardware
Storage servers [Shazli ITC 08]
(Disk failures excluded)
4

Robust System Design Challenges
Radiation,
Erratic bits
Technology challenges
Inputs Robust System
Post-Silicon bugs
System complexity
Aging,
Early-life failures
Acceptable results
Constraints:
Power, performance
5

Robust System Design
Meet user expectations
Despite underlying disturbances
Thorough validation & test
Tolerate imperfect hardware
Beyond silicon-CMOS: imperfection-immune logic
6

Outline
Introduction
Thorough validation & test
Tolerate imperfect hardware
Beyond silicon-CMOS: imperfection-immune logic
Conclusion
7

Who Cares About Post-Silicon Validation ?
Design
Pre-Silicon
Verification
35 % development time
25 % design resources
Post-Silicon
Validation
High
Volume
Barcelona shipment delayed 6 months due to
bug in TLB.
Bios fix: 10 -20% performance penalty
“Post-silicon cost & complexity is rising faster
than design cost” – S. Yerramilli, V.P., Intel
8

Post-Silicon Bug Localization Challenge
Pinpoint from system failure (e.g., crash)
Bug location, exposing stimulus
Electrical bugs: days to weeks per bug
Run apps. (OS, games)
Root-cause & fix
Localization
dominates cost
Detect bugs
Localize bugs
9

IFRA Key Message
IFRA: Instruction Footprint Recording and Analysis
Effective: 96% accurate
No system failure reproduction
No system simulation
Inexpensive: 1% area
Practical
Alpha 21264
Intel Nehalem
10

IFRA Principle
Design
Phase
Post-Si
Validation
No
Insert recorders
inside chip design
Record special info. in
recorders / Run tests
Failure
detected?
Yes
Scan out recorder
contents
Post-analyze offline
Localized Bug: (location, stimulus)
1% area cost
60KB for Alpha 21264
Non-intrusive
No failure reproduction
Single test run
No system simulation
Self-consistency
vs. test program binary
11

IFRA Hardware in Superscalar Processor
FETCH
DECODE
DISPATCH
ISSUE
EXECUTE
COMMIT
Branch Predictor I-TLB I-Cache
Fetch Queue ID assignment
Pipeline Registers
Decoders
Pipeline Registers
Reg Map Reg Free
Pipeline Registers
Instruction Window
Pipeline Registers
MUL 2xALU
2xBr FPU 2xLSU
Pipeline Registers
Reg Rename
Phys Regfile
D-Cache
D-TLB
Reorder Buffer Reg Map
Pipeline Registers
Recorders
Recorders
Recorders
Recorders
Recorders
Recorders
Alpha 21264
(open source)
Part of
scan chain
No at-speed
routing
Post-Trigger
Generator
12

Recording Example
FETCH
DECODE
Branch Predictor I-TLB I-Cache
Fetch Queue
Pipeline Reg
Decoder
Pipeline Reg INST2 INST1 ID1 ID2
INST2 INST1 Auxiliary Info: PC2 PC1 ID2 ID1
INST2 INST1 ID2 ID1 ID2 Auxiliary Info: PC2
ID1 Auxiliary Info: PC1
INST2 INST1 ID2 ID1 Auxiliary Info: Decoded bits2 bits1
Special ID
assignment rule
Instruction Footprints
Recorder 1
ID Assignment
Recorder 2
ID2 Auxiliary Info: Decoded bits2
ID1 Auxiliary Info: Decoded bits1
13

Special Rule for Instruction ID
Simplistic schemes inadequate
Speculation + flushes, out-of-order, loops
Multiple clocks, voltage frequency scaling
Special rule [Park TCAD 09]
ID width: log 24n bits
n = max. instructions in flight
8 bits for Alpha (n = 64)
No timestamp or global synchronization
14

What to Record ?
Pipeline
stage
Auxiliary information
Description Bits per recorder
Number of
recorders
Fetch Program Counter 32 4
Decode Decoding results 4 4
Dispatch Register names residue 6 4
Issue Operands residue 6 4
Execution
(ALU, MUL)
Result residue 3 4
Execution None 0 2
(Branch)
Execution
(Load/Store)
Result residue;
Memory address
35 2
Total required storage for all recorders: 60 KBytes
15

Early Warnings MUST
t=0
Error after 5 billion cycles
(e.g., speedpath)
Test Program Execution
Need to capture
in recorder storage
Early failure detection (post-triggers)
Error detection – residue, array parity
Deadlock & segfault
Failure after 6 billion cycles
(e.g., crash)
time
Early failure
suspect
detection
Special early warnings pause recording
16

Post-Analysis Overview
Test program
binary
Link footprints
High-level
analysis
Low-level
analysis
Footprints
from recorders
List of bug
location-stimulus pairs
Micro-architecture independent
Control-flow analysis
Data-dependency analysis
Decoding analysis
Load/Store analysis
Micro-architecture dependent
Residue consistency check
17

Link Footprints
Test program
binary
…
PC0 INST0
PC1 INST1
PC2 INST2
PC3 INST3
PC4 INST4
PC5 INST5
PC6 INST6
…
… …
Fetch-stage
recorder
…
ID: 4
ID: 5
ID: 6
ID: 7
…
ID: 7 PC5
ID: 0 PC0
ID: 5 PC3
ID: 6 PC4
ID: 7 PC5
ID: 0 PC6
PC0
PC1
PC2
PC3
ID: 0 PC4
Special ID rule ensures:
Issue-stage
recorder
ID: 0 AUX0
ID: 7
ID: 6
ID: 5
AUX1
AUX2
AUX3
ID: 0 AUX3
ID: 7 AUX5
ID: 5 AUX6
ID: 4 AUX7
ID: 6 AUX8
ID: 0 AUX9
ID: 7 AUX10
Execution-stage
recorder
…
ID: 7
ID: 5
ID: 7
ID: 6
ID: 5
ID: 4
ID: 7
ID: 6
…
ID: 0 AUX20
AUX21
AUX22
AUX23
AUX24
AUX25
AUX26
AUX27
AUX28
ID: 0 AUX29
Uncommitted instructions uniquely identified
…
…
Committed instructions correctly linked
earlier
time
later
18

Debug Example
Link footprints
HLA1 HLA2 HLA3 HLA4
?
? ?
? ? ?
?
? ?
? ?
?
? ? ? ? ?
?
Bug locations + exposing stimulus
HLA: High-level analysis
Low-level analysis
?
?
19

IFRA Results – Alpha 21264
Correct
localization
(96%)
Complete
miss (4%)
Exciting results for Intel Nehalem
☺
Total candidates: > 200,000
☺
Exact
localization
(78%)
Avg. 6
candidates
(22%)
1 of 200 design blocks, 1 of 1,000 error appearance cycles
20

Outline
Introduction
Thorough validation & test
Tolerate imperfect hardware
Beyond silicon-CMOS: imperfection-immune logic
Conclusion
21

Low-Cost Error Detection Most Important
Why concurrent error detection (CED) ?
Crashes vs. silent errors
Traditional CED expensive
Low-cost – How ?
New failure mode signatures
Optimize across abstraction layers
Configurable & reuse
22

Low-cost Resilience
Failure
rate
Burn-in difficult
Iddq
ineffective
Early-life failures
(infant mortality)
Circuit Failure Prediction
On-line Diagnostics
Built-In Soft Error
Resilience (BISER)
45nm data:
Errors reduced: 1,000X
Transistor aging
Guardbands
expensive
Lifetime Wearout Time
Global optimization software orchestrated
23

BISER Latch Soft Error Correction
IN
Comb.
logic
A B
C-element
(A, B)
OUT
00
Clock
1
11
0
Latch
D
C
D
C
Previous value
retained
Q
Q
A
01
Previous value
retained
Redundant Latch (Scan Test & Debug reuse)
B
10
Weak keeper
OUT
C-element
Key Observation: Latches vulnerable only in Opaque state (Clock = 0)
24

Architecture-Aware BISER Insertion
cumulative error coverage
100%
80%
60%
40%
20%
10X chip-level protection
2X
2.5%
power
penalty
0%
0% 20% 40% 60% 80% 100%
cumulative latch coverage
Alpha 21264
error injection
9% chip-level
power penalty
Ack: Prof. S.J. Patel,
UIUC for error injector
Optimized BISER insertion: verification-guided ?
25

Reconfigurable Correction – Economy Mode
Scan Clock
B = 1
Scan Data
Scan Clock A
Capture = 0
Update
System
Data
System
Clock
1D
C1
Scan / Checking Flip-flop
1D
C1
2D
C2
Q
1D
Q Q
C1
Integrated design quality
+
&
1D
C1
2D
C2
Q
System Flip- flop
Scan
Output
C-element Keeper
System
Output
Soft error correction, scan test, post-silicon debug 26

Single Event Multiple Upsets (SEMU)
SEMUs (aka MBUs) increasing
Single error assumption not sufficient
Measured
Error rate
(arbitrary
units)
10 4
10 3
10 2
10
1
“Basic”
flip-flop
2X
flip-flop
Radiation
experiment results
BISER
flip-flop
New
LEAP
flip-flop
[IRPS 10]
27

Low-cost Resilience
Failure
rate
Burn-in difficult
Iddq
ineffective
Early-life failures
(infant mortality)
Circuit Failure Prediction
On-line Diagnostics
Built-In Soft Error
Resilience (BISER)
45nm data:
Errors reduced: 1,000X
Transistor aging
Guardbands
expensive
Lifetime Wearout Time
Global optimization software orchestrated
28

Circuit Failure Prediction Early Indicator
BEFORE errors appear
Failure Prediction Error Detection
Before errors appear After errors appear
+ No corrupt data & states – Corrupt data & states
+ Low cost
“A little fire is quickly trodden out;
Which, being suffer'd, rivers cannot quench.”
– High cost
+ Self-diagnosis – Limited diagnosis
Both can be efficiently combined
William Shakespeare
King Henry the Sixth
Part III
29

Circuit Failure Prediction Applicability
Degradation delay SHIFT ≠ delay fault
Transistor aging
e.g., Negative Bias Temperature Instability
Adaptive (≠ worst-case) guardbands
Gate-oxide Early-Life Failures (ELF)
Burn-in alternatives
30

Standard Normal Quantile
Ids [µA] after 5340 min stress
New Gate-oxide ELF Signature: 90nm
Delay shifts over time: distinct from NBTI, PBTI, hot-e
4
2
0
-2
-4
-6
90
80
70
60
50
40
30
240 min. stress
5340 min. stress
20 40 60 80 100
Ids[µA]
W = 0.2µm
Fresh
Outliers
20
60 70 80 90 100
Fresh Ids [µA]
10
min.
stress
952 pairs:
I ds outliers
11.6% of entire
population
Outliers
Y
60
50
40
30
20
10
885
pairs
92%
Outlier
locations
random in
0.2µm array
0
0 50 100 150 200 250
X
952 pairs:
Largest I g increase
11.6% of entire
population
Delay Shifts [ps]
1,500
1,000
500
0
Iddq [A] at VDD=1V
10-4 10-4 10-4 10-5 10-5 10-5 10-6 10-6 10-6 10-7 10-7 10-7 Inverter chain
input LOW
Inverter chain
input HIGH
0 20 40 60 80 100
Rising
transition
Stress Time [sec]
100X
Delay jump
-500
0 20 40 60 80 100
Stress Time [sec] Falling
transition
31

Circuit Failure Prediction: How ?
Periodic on-line self-test & diagnostics
Periodic minimal power costs
Clock control reuse: test & debug, DVFS
Concurrent with application execution
Special flip-flops
Self-healing
Optimized lifetime power efficiency [DATE 10]
32

On-line Self-Test & Diagnostics: CASP
Pseudo-random BIST difficult
CASP: Concurrent, Autonomous, Stored Patterns
Multi-core no visible downtime
Software orchestration
Stored test patterns: off-chip FLASH
Test compression: X-Compact
Comparable or better than production tests
Major Technology Trends Favor CASP
33

CASP Online Diagnostics Flow
Scheduling
Pre-processing
Core 4
selected
for test
Core N
normal
operation
Core 4
resume
operation
Core N
normal
operation
Power /
performance
-aware
scheduler
Post-processing
Bring core
from online
diagnostics
to normal
operation
Core 4
temporarily
isolated
Core N
normal
operation
Test Application
Core 4
under test
Core N
normal
operation
Prepare
core for
online
diagnostics
Thorough
scan &
functional
testing
34

CASP for SUN OpenSPARC T1 Cores
Test Coverage
Stuck-at: 99.5 %
Transition: 96 %
True-time: 93.5 %
Storage
48 MBytes
Test time per core
0.3 sec.
0.01% area impact
8
processor
cores
Modified
for
CASP
support
on-chip
buffer
(7.5KB)
FPU
Crossbar
Switch
Modified
for
CASP
support
Jbus
Interface
L2
CASP control
DRAM
Control
OFF-CHIP
Flash
48 MB
compressed
test patterns
~ 8K Verilog LOC modified (out of 100K+)
35

Hardware-Only CASP Inefficient
I/O packet drop, interrupt handling
Visible application performance impact
Solutions
VAST – Virtualization Assisted CASP Self-Test
OS migration
CASP-aware OS scheduling
CPUs OS Virtualization s/w
ARM: MP11 x 4 Linux 2.6.7 NEC in-house
NEC
36

Hardware-Only CASP Inefficient
I/O packet drop, interrupt handling
Test
coverage
Visible application performance impact
Solutions
Minimize
system performance
impact
Hardware-only
VAST +
CASP
CASP-aware
OS scheduler
VAST – Virtualization Assisted CASP Self-Test
OS migration
High coverage &
Low cost
CASP-aware OS scheduling
Logic BIST
CPUs OS Virtualization s/w
ARM: MP11 x 4 Linux 2.6.7 NEC in-house
Efficiency
NEC
37

CASP-Aware OS Scheduling: Interactive App.
Workload: Firefox
Platform: Dual quad-core Xeon, Linux 2.6.25.9 scheduler modified
CASP-aware OS scheduling
Hardware-only CASP
< 200ms > 200ms, 500ms
Response
time
☺ No Effect
UNACCEPTABLE
38

CASP for Uncore in SoCs ?
Uncore CASP challenging
Multiple uncore copies not available
Multiple cores affected during uncore CASP
Uncore significant : OpenSPARC T2 (8-cores, 64 threads)
Memory BIST +
Self-repair
Memory
Cores
Core CASP
Uncore
New Uncore CASP
Utilize self-similarity
[VTS 10]
39

Outline
Introduction
Thorough validation & test
Tolerate imperfect hardware
Beyond silicon-CMOS: imperfection-immune logic
Conclusion
40

Carbon Nanotube (CNT) FETs: Big Promise
BUT, Major barriers
Imperfections: Mis-positioned & metallic CNTs
Imperfection-immune design essential
New solutions: VLSI, practical, elegantly simple
First experimental demo: Complex circuits, Latches
Imperfection-immune Half-adder Sum Imperfection-immune D-latch
V VOUT OUT (V)
3
2
1
0
0
V B = 0V
V B = 3V
V
1 2 3 A (V)
1 2 3 A (V)
CNTs
20 µm
20 µm
Collaborator: Prof. H.-S.P. Wong, Stanford
41

Outline
Introduction
Thorough validation & test
Tolerate imperfect hardware
Beyond silicon-CMOS: imperfection-immune logic
Conclusion
42

Conclusion
Robust system design
Efficient techniques practical
Thorough validation & test
IFRA
Tolerate imperfect hardware
BISER + failure prediction + CASP diagnostics
Software-orchestration a MUST
43