Spec-based verification: A new method for functional ... - Cadence

WHITE PAPERSPEC-BASED VERIFICATIONA NEW METHODOLOGY FOR FUNCTIONAL VERIFICATION OF SYSTEMS/ASICs

TABLE OF CONTENTSAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1The verification challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Today’s verification methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Test methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Deterministic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3Pre-run generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3Checking strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4Coverage metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5Spec-based verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5Enabling technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6Functional coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6Constraint-driven generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7On-the-fly temporal checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8The Specman Elite methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11TABLE OF FIGURESFigure 1: The spec-based verification methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6Figure 2: Cross-coverage window showing the registers accessed for each CPU opcode . . . . . . . . . . . . . . . .6Figure 3: The constraint solver is the core technology enabling constraint-driven generation . . . . . . . . . . .7Figure 4: The Specman Elite spec-based verification flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9Figure 5: Struct and spec constraint from the interface specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10Figure 6: Temporal check from the interface specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10Figure 7: Test constraint from the functional test plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

ABSTRACTDue to the increasing complexity of today’s systems and ASICs, functional verification has become a majorbottleneck in the design process. Design teams reportedly spend as much as 50 to 70 percent of their timeand resources on functional verification. This paper presents a new methodology for functionally verifyingsystems and ASICs: spec-based verification — an automated and measurable approach to verification thatenables more effective verification methodologies while cutting the overall resource investment in half.INTRODUCTIONIn the past decade, the electronics industry has successfully focused on automating the process of physicaldesign (place-and-route) and design implementation (logic synthesis). However, the process of verifyingdesign functionality has been relatively neglected. Advances in verification have centered primarily onincreasing the speed of simulation, not on automating the verification methodology as a whole.With design complexities increasing exponentially, functional verification has become the main bottleneckof the design process. Faster simulators are only part of the solution. Enabling the execution of morecycles provides some benefit, but often many of these cycles are wasted because they add no additionaldesign coverage.The first step toward improving the efficiency of functional verification for large, complex designs isto raise the level of abstraction of the verification environment to the specification level. The commonspecifications driving the verification process are the design spec, the interface spec, and the functionaltest plan. The verification team can develop and implement a comprehensive verification strategy onlywhen the rules defined in these specifications can be captured in an executable form. However, raisingthe level of abstraction is insufficient, in and of itself. Once these rules have been captured, the task ofgenerating tests and checking results must be automated for the team to have any hope of attaining fullfunctional coverage within reasonable time and resource budgets.Spec-based verification addresses the functional verification bottleneck in a manner similar to the waythe introduction of logic synthesis tools addressed the design challenges posed by increasing designcomplexity. First came the hardware description languages (HDLs) such as Verilog ® and VHDL, which raisedthe level of abstraction from gate-level designs to register-transfer-level (RTL) designs, thereby makinglarge designs much more manageable. However, not until the introduction of logic synthesis automatingthe translation of RTL designs to gates did the real breakthrough occur in time and resource reduction.Similarly, the real value of spec-based verification is created by a composite of features that deliverautomation: a spec-based verification environment, automatic generation of high-quality tests, data andtemporal checkers, and accurate measurement and analysis of functional coverage. With an automatedfunctional verification approach at the center of the verification methodology, the designer gains thethree essential elements needed to overcome the verification bottleneck: quality, productivity, andpredictability. Verification methodologies that incorporate spec-based verification produce reusableverification environments, shorten the verification cycle, and reduce the risk of costly silicon re-spins.THE VERIFICATION CHALLENGEThe dynamics at the root of most verification bottlenecks are the relationships between designcomplexity, verification complexity, engineering resources, and time constraints. As designs grow morecomplex, the verification problems they pose grow exponentially — that is to say, as designs double insize, the verification effort can easily quadruple. As a result, the verification effort can consume as muchas 50 to 70 percent of the entire engineering budget.Unfortunately, neither the schedule nor the available engineering resources offer much in the way offlexibility. More importantly, the cost of missing a time-to-market schedule can force design teams toterminate the verification effort prematurely. This leads to incomplete and inadequate functionalcoverage, creating the potential for a pattern of debug cycles, re-designs, and re-spins that erode theprofitability and deliverability of the end product. Thus, ASIC design quality becomes a function of theverification schedule, rather than the verification metrics.1

TODAY’S VERIFICATION METHODOLOGIESCOMPONENTSTo understand the impact spec-based verification has on minimizing the functional verificationbottleneck, it’s important to understand the components of the typical verification methodology todayand how those components are developed. (It’s also important to keep in mind that these components,and the methods used to develop them, have changed very little in the past 5 to 10 years, despite majoradvances in design complexity.)Beginning with an initial design specification, the design team partitions the design into functionalblocks, which are then assigned to specific design team members. Then the interface specs and functionaltest plan are written. The functional test plan describes what functionality should be tested and how,including normal operating behavior as well as important corner cases to be tested.The next step in the verification strategy is to assemble the verification environment, comprising piecesof code (often written in "C" ) called "stubs," which model the components surrounding the deviceunder test (DUT) in the real system. These stubs interact with the device, injecting stimuli into the deviceand receiving output according to the protocol defined in the interface specification. Sometimes theverification environment also contains monitors used to check the correctness of the functionality.Depending on the types and quantity of tests required, the designer may choose to write the testsmanually or to create a tool that generates tests according to specific directives or parameters. Whenusing an automated test generation strategy, the designer must decide whether to generate thecomplete test before simulation or to generate the test on-the-fly as simulation progresses, reactingto the state of the device.Designers have numerous strategies at their disposal for checking the results of a test. Initially, designersmust decide whether to use white-, black-, or gray-box testing. White-box testing, most commonly usedfor module and algorithm testing, enables the verification engineer to both drive and sense internalsignals. Gray-box testing allows only sensing of internal signals and is the method most often usedwith highly complex designs. Black-box testing offers no access to internal signals at all.Similar to the test generation strategy, the designer must decide whether to have the tests checkedmanually or automatically, on-the-fly or after simulation.In addition, one of the most difficult and critical challenges facing the designer is to establish adequatemetrics to track the progress of the verification effort and measure the coverage of the functional testplan. Effective coverage metrics are essential to avoid redundant or unnecessary testing, as well as todetermine when the verification effort is complete.PROBLEMSMost of the problems associated with functional verification methodologies today arise from the lackof effective automation to cope with the daunting growth in design size and complexity. This makesdeveloping verification environments, software for test generation, and deterministic tests an intensivemanual effort. Checking and debugging test results is also predominantly a manual process.Another problem related to the size issue is the difficulty engineers experience when attempting totrack assumptions made by fellow engineers working on different pieces of the design. Lacking acentrally accessible and unambiguous means of communicating and tracking design intent at thespecification level, engineers are often left with architectural-level bugs that require enormous effortto locate and remove.Locating bugs is always a problem, especially when they occur in unpredictable places. Even the mostcomprehensive functional test plan can completely miss bugs generated by obscure functionalcombinations or ambiguous spec interpretations. This is why so many bugs are found in emulation,or after first silicon is produced. Without the ability to make the spec itself executable, there’s reallyno way to ensure comprehensive functional coverage for the entire design intent.2

The relative inefficiency with which today’s verification environments accommodate midstream specchanges also poses a serious problem. Since most verification environments are an ad hoc collection ofHDL code, C code, a variety of legacy software, and newly acquired products, a single change in thedesign can cause a ripple of required changes throughout the environment, eating up time and addingsubstantial risk.Perhaps the most frustrating problem facing design and verification engineers is the lack of effectivemetrics to measure the progress of verification. Indirect metrics, such as toggle testing or code coverage,indicate if all flip-flops toggled or all lines of code were executed, but they do not give any indication ofwhat functionality was verified. For example, they do not indicate if a processor executed all possiblecombinations of consecutive instructions. There is simply no correspondence between any of these metricsand coverage of the functional test plan. As a result, the verification engineer is never really sure whethera sufficient amount of verification has been performed.TEST METHODOLOGIESDETERMINISTICThe oldest and most common test methodology, still used today, is deterministic testing. These tests aredeveloped manually and normally correspond directly to the functional test plan. Engineers often usedeterministic tests to exercise corner cases — specific sequences that cause the device to enter extremeoperational modes. Usually these tests are checked manually. However, with some additionalprogramming the designer can create self-checking deterministic tests.Although deterministic testing offers the verification engineer precise control by providing access tohard-to-reach corner cases, it has several drawbacks. Generating deterministic tests is a time-consuming,manual programming effort. Although simple tests can be written in minutes, the more complex onescan take days to write and debug. For example, to test a corner case requiring that two asynchronousdata streams reach a specific point at exactly the same time, the verification engineer might have toresort to trial-and-error methods: running the test, seeing how far off it is, correcting it, and trying again.Moreover, midstream changes to the design’s temporal behavior may force the engineer to go throughthis process repeatedly. And when this test is completed, the corner case is tested through only onepossible path.An average project normally develops several hundreds of deterministic tests, which can easily consumeseveral months to create. Checking deterministic tests also consumes considerable time and resources,whether it is performed manually or written into the test.PRE-RUN GENERATIONPre-run generation is a newer methodology for generating tests. It addresses some of the productivityproblems associated with deterministic testing by automating the test generation process. C or C++programs (and sometimes even VHDL and Verilog, despite the lack of good software constructs) areusually used to create the tests prior to simulation. The programs read in a parameter/directives filethat controls the generation of the test. Often these files contain simple weighting systems to directthe random selection of inputs.The generator normally outputs the test into a file, which is then read by the simulator and stored inmemory. The simulator reads the next entry whenever it’s prepared to inject the next set of inputs.Although pre-run generation provides much higher throughput than deterministic testing, it is difficultto control. The parameters are static and do not provide much flexibility. Also, most generators of thistype don’t allow for interdependencies between data streams — each data stream is generatedindependently. This can cause the generator to generate unlikely or even illegal tests.Reaching corner cases using pre-run generation is nearly impossible. The engineer has very little controlover the sequences generated. This makes it difficult to force the occurrence of specific combinations.As such, pre-run generation makes a suitable complement to deterministic testing, but cannot replace it.3

Another problem with pre-run generation is that it’s hard to maintain. As the verification processprogresses, new parameters are often needed. This normally requires modifying the program, sometimesaffecting delicate interdependencies between different parts of the generator.Maintenance problems can also occur when updating the program after a bug is found in the RTL design.To temporarily avoid generating the "buggy" test sequence again, the engineer must patch the code untilthe bug is fixed in the RTL design. Once the bug is fixed, the code must be "unpatched." Several suchpatches often co-exist in the code. The patching/unpatching process sometimes introduces bugs into thegenerator, which may not be noticed until several hours or days of simulation have transpired.The cost of developing and maintaining such a generator requires a minimum of several months perproject, and the cost increases significantly as the generation becomes more complex.A side effect of this methodology is that the full test is usually very large, since it is generated in advance.It is commonly loaded into a simulation memory at the beginning of the test and run from there. Thissignificantly increases the memory requirements for simulation, often causing the simulator to swapmemory, which slows the simulation down by orders of magnitude.CHECKING STRATEGIESThe two most popular ways to determine if test results are good are to compare them to a referencemodel or to create rule-based checks. Both of these checking methods must include the temporalbehavior or protocols of the device as well as the verification of data.Reference models are most common for processor-like designs for which the correct result can bepredicted relatively easily. Designers usually develop the reference model in C or C++ . Stimuli are injectedinto the reference model as well as the device, and their outputs are compared. In gray- and white-boxmethodologies, the comparisons also include the state of internal registers and nodes.Rule-based approaches are more common in communications devices for networking applications, wherethere can be several legal outputs for the same input, or where it’s not easy to predict the correct result.In this case, the engineer often uses specialized techniques to check data integrity and protocols, such asscore boarding, which tracks information about cells or packets without worrying about the order inwhich they appear on the output ports.Engineers perform these checks either on-the-fly or post-run. Simple checks and protocol checks can beperformed on-the-fly by the stubs and monitors using an HDL. Post-run checks are often performed usinga C/C++ or Perl/awk program. The outputs of the test are either saved in a simulator memory and thendumped into a file, or written into the file directly. The program reads the inputs and outputs and checksthe correctness of the results. Often, these methodologies still require some amount of manual checking,usually achieved by viewing actual waveforms or data dumps.The problem with these checking strategies arises from the way they are implemented today. Post-runchecking wastes cycles. If a test runs for 500,000 cycles, but a bug occurred after cycle 2,000, then498,000 cycles are wasted. In addition, since post-run checking cannot detect a problem in real-time,the designer does not have access to the values of the registers and memories of the device at the timethe problem happened. In general, debugging these problems requires re-running the simulation to theappropriate point.On-the-fly checking is more powerful. However, on-the-fly checks are most often implemented in Verilogor VHDL. These languages do not have a powerful temporal language to simplify protocol checks. Theyare low level and lack features like dynamic memory, which simplifies the process of writing thestubs/monitors and increases performance.In addition, reference model checking is often hard to implement on-the-fly, since intermediate results arenot always available. On-the-fly reference models also require a direct interface to the simulator (throughPLI or FLI), which is not easy to write or maintain.4

COVERAGE METRICSMeasuring progress is one of the most important tasks in verification and is the critical element thatenables the designer to decide when to end the verification effort. Several methods are commonly used:• Toggle testing: verifies over a series of tests that all nodes toggled at least once from 1 to 0 and back• Code coverage: demonstrates over a series of tests that all the source lines were exercised; in manycases, there is also an indication as to whether branches in conditional code were executed; sometimesan indication of state-machine transitions is also available• Tracking how many bugs are found each week: possibly the most common metric used to measureprogress; after a period of a few weeks with very few or zero bugs found, the designer assumes that theverification process has reached a point of diminishing returnsUnfortunately, none of these metrics has any direct relation to the functionality of the device, nor is thereany correlation to common user applications. For example, neither toggle testing nor code coverage canindicate if all the types of cells in a communications chip (with and without CRC errors) have entered onall ports. Neither can these metrics determine if all possible sequences of three instructions in a row weretested in a processor.As a result, coverage is still measured mainly by the gut feeling of the verification manager, and eventuallythe decision to tape out is made by management without the support of concrete qualitative data.Not knowing the real state of the verification progress causes engineers to perform many moresimulations than necessary, trading off CPU cycles for "confidence." This usually results in redundant teststhat provide no additional coverage or assurance that verification is complete. The real risk is that thedesign will be sent to production with bugs in it, resulting in another round of silicon. The cost of respinningsilicon includes non-recoverable engineering (NRE) costs to perform the additional productionprocess, the cost of extending the team’s work on the project, and the major cost of missing a time-tomarketwindow. For most designs, this amounts to many millions of dollars.SPEC-BASED VERIFICATIONSpec-based verification is a proven methodology for functional verification that solves many of theproblems design and verification engineers encounter with today’s methodologies. Spec-based verificationcaptures the rules embodied in the specifications (design/interface/functional test plan) in an executableform. An effective application of this methodology provides four essential capabilities to help breakthrough the verification bottleneck:• Automates the verification process, reducing a significant amount of manual work needed to developthe verification environment and tests• Increases product quality by focusing the verification effort on areas of new functional coverage andby discovering bugs not anticipated by the functional test plan• Provides functional coverage analysis capabilities to help measure the progress and completeness ofthe verification effort• Raises the level of abstraction used to describe the environment and tests from the RTL level to thespecification level, capturing the rules defined in the specs in a declarative form and automaticallyensuring conformance to these rules5

Device spec andfunctional test plan - eLegacy code (C, HDL)• Verification environment• Reference models• TestsConstraint-driventest generationData and temporalcheckingFunctionalcoverage analysisHDL simulatorHDLFigure 1: The spec-based verification methodologyENABLING TECHNOLOGIESFUNCTIONAL COVERAGEFunctional coverage analysis is a key enabling technology for spec-based verification methodology.This technology allows the verification engineer to define exactly what functionality of the device shouldbe monitored and reported. To accomplish this, the functional test plan is translated into executabledirectives fed directly to the functional coverage analyzer. This makes the entire functional test planexecutable, including complex multi-cycle scenarios, state machine transitions, and temporal sequences.Figure 2: Cross-coverage window showing the registers accessed for each CPU opcode6

The coverage information can also be crossed, providing information on interrelated functionality.This "querying" ability answers questions about the simultaneous occurrence of certain functions.For example, in a processor design, the verification engineer can cross information about instructions,addressing modes and machine state to see exactly how many times each instruction was executed usingthe different addressing modes for every machine state.Using functional coverage analysis, the engineer finally has a clear indication of which functionality hasbeen exercised and, more importantly, which functionality has not. This focuses the verification effortand eliminates superfluous or redundant tests, ensuring that every test adds coverage. In so doing, theavailability of functional coverage analysis has opened the door to major advancements in verificationmethodologies.The ability to generate clear reports showing which parts of the functional test plan were tested hasenabled a new approach to verification. Beginning with a base of automated tests, coverage reportsare generated to indicate which parts of the functional test plan were exercised. This allows the engineerto close quickly on significant functional coverage of the device with minimal effort. Afterward, the onlyadditional effort required is to generate tests focused on the functionality not yet covered. The coveragereports after each set of tests direct the verification engineer to the types of tests to focus on next.Deterministic tests are needed only for the corner cases that were not reached over the course ofautomated testing. This minimizes the time spent writing deterministic tests.Functional coverage reports are also an essential tool for verification managers. They help monitor theprogress of the verification effort, provide a clear view of the state of the verification, and accuratelydetermine the moment the device is ready to be fabricated.Functional coverage reports also enable the verification engineer to verify the effectiveness of the tests.Once a certain functionality has been tested enough (according to the metrics set by the engineer), otherfunctionality can be targeted and tests that add no new coverage can be discarded. There is no need tocontinue testing "just in case."The impact on the verification schedule when using this coverage approach is significant. It eliminatesmuch of the manual-intensive work of developing the deterministic tests, saving many of frustratingwork. It also significantly reduces the risk of sending the device to production while part of itsfunctionality has not been properly tested. This coverage approach eliminates silicon re-spins, whichin turn saves millions of dollars.CONSTRAINT-DRIVEN GENERATIONThe one feature spec-based verification offers that most significantly contributes to reducing theverification schedule is automated test generation. Armed with a constraint-driven test generator, aspec-based verification methodology can slash weeks or even months off the verification schedule,while placing enormous power in the hands of the engineer.Current cycle’s HDLsignal and register valuesTest constraintsSteer generation totarget areas• Target areas• Weights/probabilities• Corner-case tests• Structured sequences• Bug bypassesConstraint solverNext cycle’s stimulusSpec constraintsDefine what is legal• Protocols• I/O relationships• Context dependencies• Input definitionHDL simulationFigure 3: The constraint solver is the core technology enabling constraint-driven generation7

Constraint-driven generators offer some key advantages over the more common, parameterizedgenerators. First and foremost, they give the verification engineer full control over the generationprocess. By using simple constraints, engineers can generate tests that are completely random, completelydeterministic, or anywhere in between. Either these constraints can be spec constraints, used to define thelegal parameters that the generator must always hold to, or test constraints, which target the generatorto a specific test from the functional test plan. Instead of randomizing only what the user specificallyinstructs the system to randomize, constraint-driven generators take the infinity-minus approach: theyrandomize everything except what the engineer specifically constrains the generator not to randomize.This allows the generator to approach any given test scenario from multiple paths and find bugs thatwere not identified in the functional test plan.This capability to capture the “same” test scenario from different paths is critical. Most post-siliconfunctional bugs are not due to an inadequacy in the functional test plan, but to unanticipated usage bythe end user or ambiguities in the interface specifications. It’s impossible to think of all the possible bugswhen writing the functional test plan, so it is critical that all tests be run from multiple, different paths.Beyond offering support for the common pre-run generation methodology, constraint-driven generatorscan also support the more advanced "on-the-fly" generation methodology. Using the on-the-fly approach,the generator interacts with the DUT and reacts to the state of the device in real-time. In this manner,even hard-to-reach corner cases can be captured because the generation constraints are state dependent.On-the-fly generation also eliminates the need for big memories to hold the entire test. Instead ofgenerating the test all at one time, the inputs are generated as required. This reduces the requiredmemory and swapping by the simulator, significantly increasing simulation performance.Constraint-driven generators are also generic, or design independent. They receive a description of the dataelements to be generated (instructions, packets, or polygons) and generate them accordingly. If the designerintroduces changes to the architecture, the generator easily adapts, making the tests easy to reuse.One of the main challenges facing a verification engineer is to test corner cases in the functionality ofthe device. These corner cases are usually reached after a long sequence of inputs, and they’re oftenthe result of several independent streams of input reaching a special combination at the same time.The constraint-driven generator’s ability to generate massive amounts of tests on-the-fly allows thegenerator to monitor the state of the device constantly and generate the required inputs at exactly theright time to reach the desired corner case. The result is high throughput of effective tests, which achievehigh functional coverage. Thus, when used in combination with functional coverage analysis, constraintdrivengeneration provides easy confirmation that a corner case was reached, and under what conditions.A powerful feature often missing in homegrown pre-run generators is the interface to the simulator.In the generic constraint-driven generators, this simulator interface does not have to be specified forevery project, but is included as part of the generic generator. Manually created generators requirecreation of a custom interface that is specific not only to the simulator, but also to a particular versionof the simulator — a maintenance nightmare.All of these capabilities supplied by on-the-fly constraint-driven generators result in a major reductionin the verification cycle and significant increases in the verification quality.ON-THE-FLY TEMPORAL CHECKSToday’s highly integrated designs contain many components that are often developed by differentengineers, resulting in different interpretations of the protocols between components. This can causethe component interfaces to become the weakest link in the design. On-the-fly temporal checks verifythe temporal behavior and protocols associated with the design or system. They constantly monitor thedesign, sensing triggers (or events) that signal the beginning of a sequence. They then follow thesequence, verifying that it conforms to the temporal rules specified by the verification engineer.There are several types of rules. In some cases, the protocol can be specified explicitly, defining exactlyon which cycle each part of the protocol should be performed. In other cases, the exact cycle of aresponse can’t be defined; it is known only that it must happen eventually, or that it will occur withina certain interval of time. All of these checks — explicit, eventual, and interval — must be easy to specifyand must run on-the-fly to allow for protocol verification and efficient debugging of interface problems.8

Although temporal checks can be done using the current HDLs, their implementation is not always simple.New temporal constructs allow defining these rules in a simple and straightforward manner and are keyto the spec-based verification methodology.On-the-fly checking, particularly in conjunction with on-the-fly generation, is a memory-efficientapproach to verification. Since the checks are performed on-the-fly, only relevant information is stored,and as soon as it is checked, it can be removed. On-the-fly checking also enhances the debug capability.When a check fails, it is possible to access the full state of the device at that point and generate adetailed report. Moreover, the test can then be aborted, since the subsequent cycles will not be usefuland merely waste simulation cycles.High-level temporal constructs also enable the engineer to define the rules independently. This allowsthe engineer to reuse checks developed for a lower-level block when verifying higher levels of integrationthat incorporate them.In summary, on-the-fly temporal checks provide an effective means to capture the interface specificationand verify the protocol of these interfaces, efficiently debug them, and eliminate the ineffectivesimulation cycles that follow the bugs’ occurrence. The powerful temporal constructs used by thesecheckers can also minimize the size of complex checkers, reducing by a factor of four the time it takesto write these checkers.THE SPECMAN ELITE METHODOLOGYIntegrated with the Cadence ® Incisive ® functional verification platform, Specman Elite ® testbenchautmocation delivers a spec-based verification methodology. The Specman Elite system allows you tocapture the rules defined in your specifications (design/interface/functional test plan) and make themexecutable. It has all of the enabling technologies described above: functional coverage analysis,constraint-driven generation of functional tests, and all the constructs necessary for creating powerfuldata and temporal checkers.Design spec andinterface specFunctional test planTest environmentEnvironment developmentVerificationTestsSimulationCoverage analysisFigure 4: The Specman Elite spec-based verification flow9

The Specman Elite verification flow is very similar to the traditional flow, but is enhanced with theenabling technologies described above. As always, the input to the functional verification process isthe design spec (which defines the device’s architecture, functionality, and protocols) and the interfacespec (which defines the target system and the protocols between the DUT and the rest of the system).These specs are used to develop the functional test plan, which defines the verification strategy, the testenvironment, and the functionality and user scenarios that are to be tested. These tests should includetypical tests, stress tests, error tests, and corner-case tests.Once the test plan is defined, the test environment is developed. Existing code from previous projectswritten in Verilog, VHDL, or C can be used and linked into the system. The test environment includesstructs, which define the data to be generated, and spec constraints (from the interface specification,see Figure 5). Spec constraints constrain the generator to generate only legal values. The verificationenvironment also includes the data and temporal checks (see Figure 6), which define how to checkprotocols and monitor the simulations for correct behavior.Interface spec: An instruction consists of an opcode representingthe instruction set, op1, which can access registers zero throughthree, and op2, which is a byte.type command: [ADD, ADDI, SUB, SUBI, JMP, JMPR, JMPC, CALL, RETURN];type register: [REG0, REG1, REG2, REG3];struct instruction {opcode: command;op1: register;op2: byte;};Interface spec: If the opcode is jump to an address in a register (JMPR),the second operand must be zero.extend instruction {keep (opcode == JMPR) => op2 == 0;};Figure 5: Struct and spec constraint from the interface specificationInterface spec: An acknowledge must be received after 3 to 5 cyclesof when a request is seen.expect rise ('top.req') => {[3..5]; rise('top.ack')};Note: signals that are quoted represent DUT signals from the HDL.Figure 6: Temporal check from the interface specificationAt this point, a few deterministic tests can be written and run to verify that the device is getting outof reset and performing the basic operations correctly. Once this is done, instead of writing manydeterministic tests and checking them, constraint-driven tests can now be generated. Specman Elitetestbench automation can generate many tests, drive them into the device, and check that the deviceresponded according to the spec. But more importantly, the Specman Elite system collects the functionalcoverage information, showing what functionality was actually exercised. With this information, it isnow easy to know what functionality has not been covered and to write more specific test constraints(see Figure 7) that target the generator to focus on the untested functionality. This process ensures anefficient verification cycle and guarantees that each test adds new coverage.10

Functional test plan: Test the jump on carry (JMPC) opcode when thecarry bit is high.extend instruction {keep ('top.cpu.carry'== 1) => opcode == JMPC;};Note: signals that are quoted represent DUTsignals from the HDL.Figure 7: Test constraint from the functional test planCONCLUSIONThe ineffectiveness of current verification methodologies is apparent both in the proportion of timespent verifying a design versus the time spent designing it, and in the number of re-spins most designsrequire to become fully functional. The enormous resource expenditures required to verify today’scomplex designs will more than double in the next generation of complexity. Verification teams alreadyfind themselves backed into a no-win situation with only two options — miss the schedule by anindeterminate amount, or abort verification and risk an almost certain re-spin.Spec-based verification provides an effective composite of enabling technologies that automate resourceintensivemanual processes and establish qualitative metrics to ensure sufficient functional coverage.Using these technologies increases the quality of the design while reducing the resources needed toimplement a powerful verification methodology. Verification engineers report experiencing a two- tofour-fold reduction in the verification schedule, while achieving significantly higher verification quality.11

Cadence Design Systems, Inc.Corporate Headquarters2655 Seely AvenueSan Jose, CA 95134800.746.6223408.943.1234www.cadence.com© 2005 Cadence Design Systems, Inc. All rights reserved. Cadence, the Cadence logo, Incisive,Specman Elite, and Verilog are registered trademarks of Cadence Design Systems, Inc. Allothers are the properties of their respective holders.6372 08/05