Will AI Eliminate Verification?
A recent blog post looked at the impact artificial intelligence (AI) is having on chip development, focusing on register-transfer-level (RTL) design in general, and the hardware-software interface (HSI) in particular. It seems natural to extend this topic to ask how AI is affecting functional verification of chip designs. Again, there’s plenty of hype on this topic, but it’s important to separate “perhaps someday” visions from the current reality of what AI can accomplish.
AI and IP Verification
The recent post noted that the generation of RTL designs is a domain where AI has shown some success. It used the example of asking ChatGPT to “generate SystemVerilog code for an 8-bit priority encoder” and showed its output. In theory, such generated designs should be “correct by construction” since AI tools are trained on huge datasets that include many successful real-world RTL IP designs. Does this mean that they don’t need to be verified?
The answer, of course, is “no!” since training datasets may include designs with bugs in them, resulting in a large language model (LLM) with errors. In addition, the well documented tendency of AI tools to hallucinate means that the output may sometimes not make sense. It is important to verify any RTL code generated by AI, just as code written by humans. This verification may happen in a standalone IP block testbench, in higher levels of the chip hierarchy, or both.
This leads to the question of whether AI can generate the verification testbench and tests. Typing “generate SystemVerilog UVM testbench and tests for an 8-bit priority encoder” into ChatGPT yields a basic solution that includes a top-level testbench, UVM agent, driver, monitor, and scoreboard, plus a sequence (shown below) with a few common tests. It’s a good start, although an experienced verification engineer would want to add coverage and some additional tests.
class priority_encoder_sequence extends uvm_sequence #(priority_encoder_transaction);
// Constructor
function new(string name = "priority_encoder_sequence");
super.new(name);
endfunction
// Generate test cases
virtual task body();
priority_encoder_transaction tx;
// Test case 1: All zeros
tx = priority_encoder_transaction::type_id::create("tx");
tx.in_data = 8'b00000000;
start_item(tx);
finish_item(tx);
// Test case 2: Only one bit set
tx.in_data = 8'b00000100;
start_item(tx);
finish_item(tx);
// Test case 3: Multiple bits set
tx.in_data = 8'b11000000;
start_item(tx);
finish_item(tx);
// Test case 4: All ones
tx.in_data = 8'b11111111;
start_item(tx);
finish_item(tx);
endtask
endclass
AI and SoC Verification
It seems likely that any AI tool capable of generating an RTL IP block is also capable of generating a standalone testbench and some basic tests. These could also be used to verify a hand-written IP block, or one generated by a different AI tool or LLM. Of course, it is important to verify at least some of the functionality of an IP block once it is instantiated into the chip design. Since AI is not today able to generate the RTL design for a complete chip, it is unreasonable to expect that it could verify one.
Fortunately, there are some aspects of a complex chip that can be verified by current AI tools. The previous post focused on the HSI, which is an essential part of any system on chip (SoC), and on the generation of the memory-mapped, addressable control and status registers (CSRs) that form the hardware side of the HSI. Generating a testbench and tests to verify a CSR block certainly seems within the scope of AI as well.
Thorough hardware-software verification of the HSI involves both the CSR hardware and the low-level software (such as microcode and device drivers) that programs the registers. HSI verification requires tests that exercise the registers and check the results. Many of these tests could potentially be generated by AI tools, although custom sequences that access the registers may be needed. Thus, there must be a way to specify these sequences as well as the set of registers.
Agnisys AI Verification Solutions
The Agnisys IDesignSpec™ Suite is a specification automation solution that generates register RTL code, SystemVerilog Open Verification Methodology (OVM) testbenches and tests, C/C++ tests, and combined hardware-software verification environments. As discussed in the earlier post, the Agnisys SmartDatasheet site uses leading-edge LLM and natural language processing (NLP) technology to interpret natural language datasheets with CSR definitions to generate the RTL code.
IDesignSpec uses the same register definitions, whether from datasheets or from standard formats such as SystemRDL and IP-XACT, to generate verification testbenches and tests. The C/C++ tests can be run on UVM testbenches, co-verification environments, emulators, and even actual chips. The tests are fine-tuned for any special registers specified, including indirect, indexed, read-only/write-only, alias, lock, shadow, interrupt, counter, paged, virtual, external, and read/write pairs.
When custom CSR programming sequences are required, Agnisys provides an AI solution as well. Users can write these sequences in natural language, which is converted via NLP into UVM and C/C++ sequences. Assertions, an important part of RTL verification, can also be specified as statements of design intent in natural language. The Agnisys iSpec.ai site converts these statements into SystemVerilog Assertions (SVA), which could be for the CSRs or any other parts of the design.
Summary
AI is a long way from generating a full-chip testbench and a complete set of tests, but it adds value to the verification flow today for assertions, CSR programming sequences, and HSI testbenches. This frees up a lot of time for the verification engineers, enabling them to focus on more complex chip-level tests. Agnisys is a proven industry leader in using AI for SoC and IP development, with additional capabilities in development.
