Artificial intelligence (AI) accelerators are essential for tackling AI workloads like neural networks. These high-performance parallel computation machines provide the processing efficiency that such high data volumes demand. With AI playing increasingly larger roles in our lives—from consumer devices like smart speakers to industrial applications like automated factories—it’s paramount that we also consider the energy efficiency of these applications. When designing AI accelerators, for example, there’s opportunity to optimize power consumption early in the design cycle.
Indeed, power consumption of AI hardware has become an area of critical concern given the impact on our environment. The computational resources needed to generate a best-in-class AI model has doubled every 3.4 months, on average, according to OpenAI, an AI R&D company. Researchers from the University of Massachusetts, Amherst, estimated that training a single deep-learning model (though an energy-intensive one) can generate up to 626,155 pounds of carbon emissions—that’s an amount that five cars can emit over their entire lifetimes. And once an AI model is deployed in the field, it consumes even more energy. AI hardware typically consists of large arrays with up to thousands of tiles (processing elements), requiring billion-plus gate, power-hungry designs. Reducing power consumption can generate a number of benefits, including lower costs, better battery life, and minimized environmental impact.
One key power-related challenge to be aware of relates to glitch power. In electronics design, glitches happen if the signal timing within the paths of a combinatorial circuit are imbalanced, causing a race condition. This, in turn, generates an unwanted signal transition that causes additional dynamic power. The amount of glitch is proportional to the number of operations executed by the system-on-chip (SoC). When you consider the high volume of operations performed when an AI algorithm is run on hardware, you can understand why glitch power makes such an impact on overall power consumption (in many cases, glitch power can consume as much as 40% of total power in a chip). What’s more, glitches can also trigger electro-mechanical (EM) and IR drop issues (even in the power grid).