Verification of Digital Systems, Spring 2017 11. Testbench Basics
Verification Testbench February 23rd 2017 Nagesh Loke, ARM Cortex-A class CPU Verification Lead Jacob Abraham, Rajesh Ganesan, Kshitiz Gupta, UT Austin
1
What to expect? This lecture aims to:
provide an idea of what a testbench is help develop an understanding of the various components of a testbench build an appreciation of the complexity in a testbench highlight why it is as much a software problem as is a hardware problem
2
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
1
Verification of Digital Systems, Spring 2017 11. Testbench Basics
2
What is a testbench? A testbench helps build an environment to test and verify a design The key components of a testbench are: Stimulus Checker
Stimulus is used to drive inputs of the design to generate a high-level of confidence should be able to exercise all normal input scenarios and a good portion of critical combinations
with ease
A checker is a parallel & independent implementation of the specification is used verify the design output against the modeled output
3
What are we verifying?
opcode[2:0]
An ALU has
an input clock two 8-bit inputs as operands a 3-bit opcode as an operator a 16-bit output
A[7:0]
OUT [15:0]
Performs the following operations: ADD, SUB, AND, OR, XOR, MUL, XNOR B[7:0]
clk
4
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
Verification of Digital Systems, Spring 2017 11. Testbench Basics
What did they do?
What are some of the issues with this approach?
5
What should the approach be? A bit of planning goes a long way A Verification plan talks about: various design features and scenarios that need to be tested architecture of the testbench reuse in higher level testbenches
Testbench should have the ability to:
test as many input data and opcode combinations as possible test different orders of opcodes stress key features/combinations use more machine time and less human time
6
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
3
Verification of Digital Systems, Spring 2017 11. Testbench Basics
SystemVerilog SystemVerilog as a hardware verification language provides a rich set of features Data Types & Aggregate data types Class, Event, Enum, Cast, Parameterization, Arrays, Associative arrays, Queues and manipulating methods
OOP functionality Classes, Inheritance, Encapsulation, Polymorphism, memory management
Processes fork-join control, wait statements
Clocking blocks Interprocess synchronization & communication Semaphores, Mailboxes, named events
Assertions Functional Coverage Virtual Interfaces Constraints 7
Data Types
SystemVerilog introduces several new data types. Verilog variables are 4 state variables (0, 1, X, Z) – wire, reg. SystemVerilog has both 4 state and 2 state variables (0, 1). Testbench can use 2 state variables for loop iterators etc.
8
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
4
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Integer Data Types 2 state data types bit (user defined size) ◦ bit [7:0] addr;
byte (8b, signed) ◦ byte addr;
int (32b, signed) ◦ int addr;
4 state data types reg (user defined size) ◦ reg [7:0] addr;
logic (user defined size) ◦ logic [7:0] addr;
‘logic’ type can be used instead of ‘reg’ or ‘wire’ types. 9
Non Integer Data Types time - 64b, unsigned real - similar to double in C
10
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
5
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Misc. Data Types Arrays – can have packed and unpacked dimensions
packed dimension is specified before the variable name unpacked dimension is specified after the variable name the following declaration is for a 4 entry register file where each entry is 8b wide bit [7:0] register_file [3:0];
11
Misc. Data Types enum – define enumerated types where variables can take any value from a set of all
possible values enum { ADD=0, SUB=1, MUL=2, AND=4, OR=5, XOR=6, XNOR=7 } OPTYPE;
struct – define structures containing multiple variables of same or different types they can be packed or unpacked like arrays struct { bit [9:0] src_addr; bit [1023:0] data; bit [3:0] crc} packet;
union – define a variable that can be accessed as many different types (byte, int, etc.) union {int a; byte b;} var;
string – define a string type variable that can hold a sequence of characters typedef – add user defined types from the basic types
12
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
6
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Misc. Data Types dynamic arrays – the size of the array varies at run time based on need or behaviour. to allocate storage ‘new’ and ‘delete’ methods can be used dynamic arrays are declared as shown below int dynamic_array[];
13
Misc. Data Types associative arrays – a set of key:value pairs usually used for sparse data or when the size
of the array is not known they can be declared as shown below int associative_array[int];
Methods that can be used on associative arrays are:
first() last() next() prev() exists() num() delete()
14
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
7
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Misc. Data Types queue – a variable-size, ordered collection of homogeneous elements Can be used to model a LIFO or FIFO Elements can be inserted and deleted using an index even if they are not the first or the last one. int collection[$] = {10, 20, 30, 40}
Methods associated with queue data types are:
size() insert() delete() push_front() push_back() pop_front() pop_back()
15
Interfaces Interfaces in SystemVerilog are a neat way to denote a bundle of wires. An interface can be used as a module port instead of individual signals. It improves readability of the
code and makes updates to the code easier. It can contain parameters, constants, variables, functions and tasks etc. in itself. interface dut_in; logic [31:0] A, B; logic clk, rst; endinterface module dut(dut_in _in); //one can use _in.A, _in.B in this module … endmodule 16
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
8
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Interfaces RST
CLK Intf
Slave
Slave Modport
CLK_I RST_I ADR_O DAT_I DAT_O WE_O SEL_O STB_O ACK_I CYC_I
Master Modport
Master
CLK_I RST_I ADR_I DAT_O DAT_I WE_I SEL_I STB_I ACK_O CYC_O
17
Interfaces interface WB (input CLK, input RST); logic [31:0] ADR, RDAT, WDAT; logic [15:0] SEL; logic WE, STB, ACK, CYC; modport Master (output ADR, WDAT, WE, SEL, STB, input RDAT, ACK, CYC); modport Slave (input ADR, WDAT, WE, SEL, STB, output RDAT, ACK, CYC); endinterface
18
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
9
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Virtual Interfaces Design
Testbench
Virtual IF
Driver
IF
DUT
19
Object Oriented Programming Procedural Programming Paradigm A program is broken down into data variables and procedures (functions) that operate on that data. A program itself is a sequence of procedure calls, where the main program hands over data to the
procedures.
Object Oriented Programming Paradigm A program is broken into classes (data and methods) and objects (which are instances of classes). Objects operate on their own data. A program is a set of messages to objects.
Why Object Oriented Programming Paradigm?
Modular Has Reusable Components Easy to maintain and enhance Based on interaction between objects
20
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
10
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Object Oriented Programming Constructs An “object” may contain data (data fields - strings, integers, etc.) and methods (member
functions) that operate on the data. Encapsulation - The concept of keeping the data and the methods/functions that work on that data in one structure. This also allows to keep the data safe, as the only allowed ways that the data can be modified is usually through the methods in that class. Objects are created as instances of a “class”. Class is a user-defined type in OOPS. One of the most important features of classes (and OOPS in general) is inheritance. It allows for new classes to be created by inheriting properties from previously defined classes. The new class is called the “derived class” while the old class is referred to as the “base class”.
21
Object Oriented Programming Constructs Class Shape: Data – area(int), perimeter(int), color(string) Methods – getarea(), getperimeter(), getcolor()
Class Triangle(inherited from Shape): Inherited Data – area(int), perimeter(int), color(string) Added Data – length of the 3 sides Methods – getarea(), getperimeter(), getcolor()
Class Quadilateral(inherited from Shape): Inherited Data – area(int), perimeter(int), color(string) Added Data – length of the 4 sides Methods – getarea(), getperimeter(), getcolor()
Class RightTriangle(inherited from Triangle): … Add methods like – getHypotenuse()
Class Rectangle(inherited from Quadilateral): … Add methods like – isSquare()
22
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
11
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Object Oriented Programming Constructs class packet; bit [9:0] src_addr; bit [1023:0] payload; bit [15:0] dst_addr; bit [3:0] crc; protected int packet_id; virtual function void print_packet(); $display(“packet: %d”, packet_id); endfunction endclass : packet
class eth_packet extends packet; protected int frame_no; function void print_packet(); super.print_packet(); $display(“frame: %d”, frame_no); endfunction endclass : eth_packet
23
Object Oriented Programming Constructs
24
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
12
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Object Oriented Programming Constructs
25
Object Oriented Programming Constructs
26
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
13
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Randomization Constructs Variables within a class can be declared as ‘rand’ or ‘randc’ to be generated randomly. rand bit [7:0] data;
‘randc’ is used when a cyclic random variable is needed. It ensures that all unique
elements are exhausted before any repetition occurs.
randc bit [1:0] state; cycle 1 → {10 → 00 → 01 → 11 } cycle 2 → {10 → 11 → 00 → 01 } cycle 3 → {01 → 10 → 11 → 00 }
These variables will be randomized every time the randomize() function is called on an
object of the class. randomize() is a built-in function used for randomization. To pre-process or post-process the randomized variables, pre_randomize() and post_randomize() functions can be used. 27
Constraints Constraints are necessary to direct the randomization towards interesting corner cases Constraint blocks are defined using the keyword ‘constraint’ Constraint blocks can be specified using any of the below expressions
set membership distribution implication if … else iterative variable ordering
28
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
14
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Constraints – Set Membership class packet; rand bit[9:0] src_addr; rand bit[1023:0] data; rand bit[3:0]crc; constraint src_addr_set_const; endclass : packet constraint packet::src_addr_set_const { src_addr inside {[100:127], [1000:1023]}; }
29
Constraints – Distribution class packet; rand bit[9:0] src_addr; rand bit[1023:0] data; rand bit[3:0]crc; constraint src_addr_dist_const; endclass : packet constraint packet::src_addr_dist_const { src_addr dist { [100:127] :/ 50 [1000:1023] :/ 50 } } 30
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
15
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Constraints – Implication class packet; rand bit[9:0] src_addr; rand bit[1023:0] data; rand bit[3:0]crc; constraint src_addr_impl_const; endclass : packet constraint packet::src_addr_impl_const { (src_addr == 10’d100) → (data !inside {0}) }
31
Constraints – Variable Ordering class packet; rand bit[9:0] src_addr; rand bit[1023:0] data; rand bit[3:0]crc; constraint src_addr_vo_const; endclass : packet constraint packet::src_addr_vo_const { solve src_addr before data; }
32
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
16
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Constraints – Iterative class eth_packet; rand byte data[10]; constraint each_elem; endclass : eth_packet constraint eth_packet::each_elem { foreach(data[i]) data[i] > 32; }
33
Process Control SystemVerilog provides three ‘fork’ constructs for process control fork … join fork … join_any fork … join_none
In each case, the only difference is how long does the parent process wait after the fork
and before continuing execution. In the first case: fork … join – the parent process waits for the completion of all child processes to complete before continuing execution In the second case: fork … join_any – the parent process waits for atleast one process to complete before continuing execution In the last case: fork … join_none – the parent process doesn’t wait for the child processes to complete before continuing execution 34
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
17
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Process Control ‘wait’ and ‘disable’ constructs can be used to control individual threads of execution ‘wait’ is used when the parent process is outside the fork … join_any or join_none
case, but the parent wants to wait for the children to finish before continuing execution. ‘disable’ is used when the parent process is outside the fork … join_any or join_none case, but the parent wants to kill the children before continuing execution.
35
Components of a testbench The top testbench now looks different It creates an interface handle It instantiates the DUT
36
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
18
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Testbench TOP A program block is the main entry point A bfm object and a scoreboard object are
created All the components are started A fork/join process ensures that they all start in parallel We exit the fork statement at 0 time Simulation is stopped when $finish is called Multiple initial blocks execute in parallel
37
Transaction class The ALU transaction class: Uses an enum type for optype Uses “rand” to declare inputs that need
to be driven Has a print utility that can be used with a transaction handle/object
38
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
19
Verification of Digital Systems, Spring 2017 11. Testbench Basics
BFM/driver The BFM/driver class:
Has a handle to a virtual interface Declares a alu_trxn data type Has a constructor drive() task: ◦ ◦ ◦ ◦
Does not end Creates a new transaction Randomizes the transaction Passes the handle to drive_trxn() task
drive_trxn () task ◦ consumes time ◦ drives the input signals based on the values in the trxn class ◦ Uses clocking block and non-blocking assignments ◦ Adheres to pin level timing of signals
39
Scoreboard The Scoreboard: Functionality is to continuously check the output
independent of the input stimulus check() task: ◦ Collects information from the interface and populates the trxn class ◦ Calls a compute_expected_out function
compute_expected_out() task ◦ Implements the model of the deisign ◦ Takes in the inputs and gets an expected output ◦ Compares the actual output against the expected output ◦ Issues an error message if the comparison fails
40
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
20
Verification of Digital Systems, Spring 2017 11. Testbench Basics
How does the testbench look like? alu_tb Process control logic
clock generation logic
clk alu_bfm
A B
ALU ALU
OUT
alu_sb
41
How do we know we are done? With a random testbench it is difficult to know what scenarios have been exercised Two techniques are typically used to get a measure of what’s done Code Coverage No additional instrumentation is needed Toggle, Statement, Expression, Branch coverage
Functional Coverage
Requires planning Requires instrumenting code SystemVerilog provide constructs to support functional coverage Provides detailed reports on how frequently coverage was hit with the test sample
Coverage closure is an important aspect of verification quality
42
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
21
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Coverage Point Identify an integral variable or expression to be covered
bit clk, rst; bit [3:0] counter_out; always @ (posedge clock) … Coverage point is declared using the keyword ‘coverpoint’ Selectively disable coverage with an ‘iff’ statement Don’t collect coverage when reset is active
43
Coverage Point covlabel: coverpoint counter_out iff(!rst) { bins min = {0}; bins low = {[1:3]}; bins med = {[4:7]}; bins high = {[8:14]}; bins max = {[15]}; } Keyword ‘bins’ is used to collect sampled values into various named bins Simulator keeps a count for each bin
44
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
22
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Covergroup Keyword ‘bins’ is used to collect sampled values into various named bins Simulator keeps a count for each bin
covergroup cg_cnt@(posedge clk); option.name = “Counter Cover Group”; covlabel1: coverpoint counter_out iff(!rst) { bins min = {0}; … } covlabel2: coverpoint … { } endgroup : cg_cnt 45
Cross Coverage Specified using the cross construct
covergroup cg_cnt@(posedge clk); covlabel1: coverpoint counter_out iff(!rst) { bins low = {[0:7]}; bins high = {[8:15]}; } covlabel2: coverpoint updn iff(!rst) { bins up = {0}; bins dn = {1}; } covlabel1_x_covlabel2: cross covlabel1, covlabel2 endgroup : cg_cnt 46
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
23
Verification of Digital Systems, Spring 2017 11. Testbench Basics
Cross Coverage Specified using the cross construct
covergroup cg_cnt@(posedge clk); covlabel1: coverpoint counter_out iff(!rst) { … } covlabel2: coverpoint updn iff(!rst) { … } covlabel1_x_covlabel2: cross covlabel1, covlabel2 { bins low_dn = binsof (covlabel1.low) && binsof (covlabel2.dn); bins high_up = binsof (covlabel1.high) && binsof (covlabel2.up); } endgroup : cg_cnt 47
Transition Coverage covlabel: coverpoint counter_out iff(!rst) { bins one = {0 => 15}; bins two = {15 => 0}; bins three = {7 => 8 }; bins rest = default sequence; }
48
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
24
Verification of Digital Systems, Spring 2017 11. Testbench Basics
What did we go over …
Built a directed and random testbench Discussed various components of a testbench Modularized and built in complexity into a testbench … for a reason Demonstrated that verification and testbench development requires good planning and software skills
49
Useful pointers
https://verificationacademy.com/ SV Unit YouTube Video EDA Playground http://testbench.in/ Search for SystemVerilog on YouTube
50
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
25
Verification of Digital Systems, Spring 2017 11. Testbench Basics
ARM Cortex A72 CPU
51
ARM CCN-512 SoC Framework
52
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
26
Verification of Digital Systems, Spring 2017 11. Testbench Basics
What are the challenges of verifying complex systems? Typical processor development from scratch could be about 60 man years Multiple parallel developments, multiple sites and it takes a crowd to verify a processor The challenges are numerous – especially when units are put together as a larger unit
or a whole processor and verified Reuse of code becomes an absolute key to avoid duplication of work Multiple times it is essential to integrate an external IP into your system The IP can come with it’s own verification implementation This requires rigorous planning, code structure, & lockstep development Standardization becomes a key consideration
So how do we solve this? 53
Department of Electrical and Computer Engineering, The University of Texas at Austin Loke, Abraham, Ganesan, Gupta, February 23, 2017
27