Architecting AI Chips Using IDesignSpec

As you may have gathered from my recent posts on related topics, artificial intelligence (AI) is very important for both Agnisys and our customers. The exponentially increasing demand for AI is driving a boom in the development of chips to support AI algorithms. As our users focus more and more on AI, we’ve evolved our products to help them be successful. In this post, I’d like to touch on some of the many ways that our IDesignSpec™ (IDS) Suite helps you develop AI chips.

What AI Chip Developers Need

For today’s discussion, let’s focus on two aspects of AI chip design: development of core AI engines and chip assembly and rapid exploration of AI architectures. The core engines are usually connected to a central CPU that controls their operation. This arrangement requires the use of high-performance buses to interface between the CPU and the AI engines. The CPU is responsible for providing the weights and hyperparameters required by AI algorithms.

Initialization, configuration, and management of these values is handled via the hardware-software interface (HSI) between the software running on the CPU and the AI hardware. Within the AI engines themselves, state machines are important design elements. Finally, it must be possible to assemble all the pieces together into a complete AI chip. Making this process automatic enables rapid exploration of architectural options and facilitates interaction with physical chip design and version control systems.

CPU Interface Using High-Performance Buses

Although there are several types of CPUs used in AI systems, most of them rely on one of a small set of standard buses. This makes it easy to reuse existing design IP blocks for peripherals and other elements that connect the buses. Many of the new blocks in your chip, including the core AI engines, connect to the rest of the system using one of these standard buses. This makes it easier to reuse portions of your AI design along with the other IP blocks.

We make it much easier to adopt standard buses for your AI chips by providing automated support for bus interfaces as well as the aggregators that connect them. Our Silicon IP Portfolio, part of the IDesignSpec Suite, includes silicon-proven register-transfer-level (RTL) designs for the following buses: 

• AXI4Full
• AXI4Lite
• AHB3Full
• AHB3Lite
• APB
• Tilelink

Initialization and Configuration of Weights

The behavior of AI models is controlled heavily by weights, numerical values to represent the strength or importance of connections between neurons. Weights are key parameters set as the model is trained, controlling how it processes inputs and how it generates outputs. As such, they feed into the AI engines and have a fundamental effect on how your chip operates. As I noted above, one of the roles of the CPU is to initialize and configure weight values. 

The values are passed via the HSI, written over the standard bus and stored in the set of software-visible programmable registers in the design. As you know, our IDS Suite automatically generates these registers as well as other types of registers and memories in your RTL design. We also generate the software sequences that access the registers. Thus, we do most of the work to connect the CPU to the AI engines via the HSI.

Hyperparameter Creation And Management

Hyperparameters are also control variables for AI models, but they are set externally rather than learned during the training process as weights are. Hyperparameters also have a profound effect on how the model operates, and finding the optimal set is critical for best performance and best results. This is accomplished by hyperparameter tuning, trying different hyperparameters and running them through the model to see what works best.

It must be easy to create, change, and manage hyperparameters without changing your AI engines. The HSI and the set of programmable registers provide the perfect mechanism to do this. Whether weights or hyperparameters, AI software can program new values whenever necessary. You can define the registers for IDesignSpec by using our IDS-NG graphical interface or a text-based standard such as SystemRDL IP-XACT, or YAML.

Chip Assembly and Rapid Exploration of AI Architectures

Ultimately, you must bring together all the pieces of your AI design and assemble them into a complete chip. This is historically a tedious and error-prone process, at best aided by some editing scripts. Making this process automatic is especially valuable for AI chips, which have many architectural options. Making the best choice often involves assembling trial versions of designs multiple times, a daunting process if only manual interconnection is available.

Fortunately, we have a solution to help you. As I discussed in detail in a recent post, our IDS-Integrate automates your chip assembly process. You can use wildcards to tell us how your blocks should be interconnected, and we take care of the details. We generate the RTL design for the top level of your chip and can handle lower levels or hierarchy as well. We also connect standard buses, using aggregators, bridges, converters, and crossbars as required. 

Interaction with Physical Design and Version Control

IDS-Integrate is not an isolated tool; it fits into your overall design flow. For example, we accept the Library Exchange Format (LEF) and Design Exchange Format (DEF) files generated by layout tools and adjust the chip as needed to reflect changes made by the place-and-route process. We also interact with the widely used Git version control system to keep all project files proactively managed and consistent.

Summary

You have a lot to do when designing an AI chip. We are eager to do all we can to help, including enabling:

• CPU interfaces and high-performance buses
• Initialization and configuration of weights
• Hyper-parameter creation and management
• State machine design
• Chip assembly and rapid exploration of AI architectures
• Interaction with physical design and version control

We invite you to contact us to learn more.