Graph-Based Verification in a UVM Environment – Staffan Berg

Graph-Based Verification 

in a UVM Environment 

Staffan Berg 

European Applications Engineer 

July 2012

Graph-Based 

Intelligent Testbench Automation (iTBA) 

• Welcome DVClub 

— Attendees 

— Organizers 

— Presenters 

• Verification Challenges 

— Time To Market 

— Quality Products 

• Step Function Gains 

— Higher Coverage Faster 

— Testbench Automation 

at SoC Level 

— Testbench Re-Use 

— Verification Productivity 

2 

Questa - April 2011 

© 2010 Mentor Graphics Corp. Company Confidential 

www.mentor.com

AGENDA 

• Graphs, Rules & Intelligent Testbench Automation 

• Why Graph-Based Verification is important 

• Fundamentals of iTBA 

• The Role of Coverage 

• Advanced Concepts 

• Graph based stimulus in the Context of OVM/UVM/*VM 

• Applications and Results – what to expect 

3 

SB , Graph Based Verification June 2012 


www.mentor.com

Intelligent Testbench Automation (iTBA) 

From a Graph-Based Stimulus Description Model 

Gary Smith - 

“ . . .The [automatic] generation of a testbench from a system-level design description. . . “ 

Mentor Graphics - 

“ . . . testbench automation that is aware 

of the valid test space, the engineer’s 

verification targets within that space, 

and the current state of the design - that 

uses automation to efficiently achieve 

the verification goals . . . “ 

iTBA Definitions 

Start 

init 

wait_rdy 

Rw_opts setup_rd setup_wr 

Rw_size rw_4 rw_2 rw_1 

ack 

Stop 

Start = init repeat ( wait_rdy Rw_opts Rw_size ack ) ; 

Rw_opts = setup_rd | setup_wr ; 

The graph defines the ―valid test space‖ 

Rw_size = rw_1 | rw_2 | rw_4 ; 

4 



www.mentor.com

Benefits of Graph-based Verification 

• Predictable Coverage Closure 

— Achieve coverage goals in a fraction of the time 

— Graph techology enables N vs N*ln(N) advantage 

• Facilitate Re-use 

— Stimulus model is independent of testbench environment 

– Same model can be re-used for SystemC, RTL, Emulation… 

– Same model can be re-used between projects 

• Brings iTBA to SoC level 

— Automatically generate stimulus at RTL block level 

— Automatically generate stimulus at SoC level (s/w & h/w) 

• Productivity improvements 

— Generation of testbench code 

— Graphical analysis / debug 

— Visualizing test scenario correlation to design spec 

5 



www.mentor.com

Defining the valid test space - Rules 

• Rules have two sections: 

— Declarations 

— Grammar 

• Declarations include: 

— Graph nodes linked to tasks/functions 

— Variables linked to tasks/functions 

— Rule building blocks 

• Grammar defines abstract behavior 

— Replaces directed test procedural code 

— Replaces CRT constraint code 

— Often declarative decision tree of choices 

— May also use algebraic constraints 

• Rules are implementation-independent 

iTBA Rule 

User-Created 

Rule Text 1 

6 SB , Graph Based Verification June 2012 


www.mentor.com

Visualizing the test space - Graphs 

• Rules compiled into graphs 

• Graphs visually depict stimulus 

• Protocol behavior 

• Packet construction options.. 

• Stimulus sizing 

• Stimulus sizing useful to assess 

• What to target for coverage 

• What to generate randomly 

• Simulation time needed to reach 

coverage goals 

• For this example 

• 864 total test combinations 

• 108 tests in CovParams sub-graph 

Rules 

compile 

2 



www.mentor.com

Defining the Verification Goals - Coverage 

• Coverage Strategy defines Goals & Priorities 

• Specify regions to cover graphically 

• Generate stimulus coverage code 

• Two types of stimulus coverage 

• Path coverage – traversal paths / cross coverage 

• Node coverage – like a system verilog coverpoint 

• Path coverage atomic_tb_bl_combos covers 

• 108 combinations of am, bt, bl 

• Node coverage bsz[] covers 

• 8 values of bsz 

Path 

Coverage 

User Added Stimulus 

Coverage Directives 3 

Node 

Coverage 



www.mentor.com

Graph-based iTBA in Simulation 

• Integration/compilation 

• Plug-in to existing methodologies 

• Runtime code 

• Graphs loaded into simulator 

• Algorithms manage graphs during 

simulation 

• Runtime graph debug 

• Debug TB or DUT problems 

• Set graph breakpoints 

• Single-step or run to breakpoint 

iTBA_comp*.sv 

Compiled 

Graph(s) 

iTBA runtime 

tb.sv 

compile 

top.sv 

Simulator 

Runtime 

Graph 

Debugging 



www.mentor.com

The Role of Coverage 

• The rule graphs define the 

entire stimulus space 

• Without coverage goals, 

traversal will be purely random 

• Graph coverage strategy 

defines goals and priorities 

• iTBA algorithms prioritize 

generation 

— Can still generate random vectors 

outside the coverage space 

10 



www.mentor.com

Graph-based Verification Advanced Concepts 

Reactive / adaptive graphs 

• Some fields in a stimulus item may be a property of the 

testbench or DUT state, not randomly selected 

— And may still be included in a cross cover goal 

• These fields become inputs (imports) to a stimulus graph 

— Enables opportunistic targeting of random-resistant scenarios 

DUT state can be used 

to change constraints 

Graph behaviour 

adapts to TB or 

DUT state 



www.mentor.com

Integrating Graph Based Verification 

Generating Stimulus in an OVM/UVM Environment 

• Stimulus is generated by sequences 

• A sequence produces sequence items 

• Driver applies stimulus to DUT 



www.mentor.com

Generating Graph-Based Stimulus 

• Graph integrates within a sequence 

— Graph execution produces sequence items 

• Maximizes reuse of infrastructure 

— No need to change existing drivers, monitors, etc 

iTBA 

Sequence 



www.mentor.com

iTBA OVM/UVM Integration Process 

Identify the target sequence Item 

Describe stimulus domain with a graph 

• Declare a graph variable for each sequence-item field 

• Define relationships between graph variables 

Define stimulus-coverage goals 

• Often corresponds to existing functional coverage goals 

Run the iTBA sequence via an OVM/UVM test 



www.mentor.com

Sequence Item 

• Verifying a simple bus protocol 

— 32-bit address 

— 64-bit data bus 

— Supports burst lengths up to 16 beats 

• Existing UVM sequence Item 

— Contains fields that describe a transaction 



www.mentor.com

Defining Graph Variables 

• Declare a graph variable for each item field 

• Declare the valid domain of each variable 

Sequence Item 

Rules 



www.mentor.com

Declare Variable Relationship 

• Simple bus protocol constraints 

— Address must be aligned to the transfer size 

— Burst-transfer beats may only be 32 or 64-bit width 

• Graph permits flexible description of relationships 

— Branches 

— Algebraic constraints 

• Reuse existing constraints 

— Can be imported automatically 



www.mentor.com

Simple Bus Protocol Graph 

• Graphical view of the stimulus space 

— Automatically created from rule description 

— Intuitive way to visualize choice tree 

• Graph branches restrict burst_len/size 

— Enables visual approach to code review 



www.mentor.com

Graph Integration into Sequence 

• Graph nodes link to tasks in the sequence class 

• Tasks set the value of sequence-item fields 

• Integration code is automatically-created from graph 



www.mentor.com

Define Stimuli-Coverage Goals 

• Simple protocol coverage goals 

— Cover 64 address ranges 

— Cover all valid combinations of transaction parameters 

• Describe address value binning 

• Define graph-coverage strategy 

— Node Coverage for address 

— Path Coverage for transaction parameters 

• Pre-simulation coverage space analysis 



www.mentor.com

Testbench Environment Integration 

• iTBA sequence is just like any other sequence 

— Use standard approaches to select and execute 

• Select the iTBA sequence via a type override 

• Explicitly create and run iTBA sequence 



www.mentor.com

Example iTBA Applications 

• Architectural exploration 

— Performance characterization 

— Behavioral model verification 

• IP Block level verification 

— Bus (AXI, AHB, PCI*, …) 

— Memory (DDR,SDDR, nand flash,…) 

— Peripheral (i2c, dma,…) 

• SOC integration 

— Firmware 

— Subsystem interactions 

• Processor verification 

— L2 cache 

— Instruction set verification 



www.mentor.com

iTBA Verification Results 

• Reached coverage in a single iTBA sim vs multiple CRT sims/seeds 

• Minimum 10x coverage closure advantage for simple CRT cases 

• Often see >100x coverage closure advantage for common CRT cases 



www.mentor.com

Graph-Based iTBA 

Gaining Acceptance Across The Globe 

Industry Design Verification Current Results Time 

Ultra Results Benefits 

Consumer 

Electronics 

Error Checking and 

Correcting Module 

NC Sim 

Specman e 

>18 hours 

100% coverage 

1 

118 minutes 

 

= 

Day 

100% coverage 

9.5 X faster 

Equal coverage 

Switching 

Subsystems 

Multiple Master 

AXI Bus Fabric 

Questa 

Directed Tests 

10,000 tests 2 

400,000 tests 

40 X more tests 

 

= 

Days 

Wireless 

Networking 

Ethernet 802.11 

Device 

VCS 

NTB 

3175 CPU hours 

95% coverage 

35 

48 CPU hours 

 

= 

Hours 

97% coverage 

66 X faster 

+ 2 % coverage 

Storage & 

Networking 

AXI Bus Bridge 

VCS 

SystemVerilog 

26,315,000 tests 

1 

196,000 tests 

170 X faster 

 

= 

79% coverage Week 

100% coverage 

+ 21% coverage 

Office 

Products 

Printer Image 

Processor 

Questa 

SystemVerilog 

8 weeks on 6 CPUs 

60% coverage 

3 

36 hours on 6 CPUs 

 

= 

Days 

100% coverage 

37 X faster 

+ 40% coverage 

Wireless 

Telecom 

Interrupt Controller 

VCS 

Vera and SV 

3 days 

100% coverage 

6 


 

= 

Days 

100% coverage 

27 X faster 


Processors 

Multi-Core Memory 

Sub-system 

Questa 

SystemVerilog 

5 hours 

100% coverage 

1 


 

= 

Day 

100% coverage 

10 X faster 


Basestation 

Telecom 

Proprietary Interface 

Module for a Router 

NC Sim 

Specman e 

825,000 vectors 

100% coverage 

1 

75,000 vectors 

 

= 

Week 

100% coverage 

10 X faster 


24 

iTBA - Accelerating Time to Coverage Closure 


www.mentor.com

Conclusions…. 

• Graph algorithms enable fast, efficient 

— Coverage closure 

— Test scenario generation 

• Graph-based stimulus description emphasizes reuse 

— Vertically from block to SoC level 

– And across multiple testbench environments / languages 

— Horizontally from project to project 

• Easily integrate into existing environments, e.g. OVM/UVM 

25 



www.mentor.com

26 



www.mentor.com

Graph-Based Verification in a UVM Environment – Staffan Berg

Create successful ePaper yourself

Delete template?

Save as template?