EDA
System
IP
SolvNetPlus
Training & Education
AI compute is growing far faster than the energy systems that power it. Electrical grids, data center capacity, and cooling systems can’t scale at the same 4–5× annual pace as AI.
Leaders from Microsoft, Samsung, UCLA, and MatX recently discussed what this mismatch between energy and compute means for the design of AI chips, systems, and tools. Speaking at Synopsys Converge 2026, the panelists described how wringing more useful intelligence out of every watt has become a full‑stack challenge.
“The whole AI stack sits on a bedrock of energy,” said Shankar Krishnamoorthy, chief product development officer at Synopsys and moderator of the panel. “And that’s the layer that’s moving the slowest.”
Multiphysics Fusion Technology for Multi-Die Designs Explained
Learn why multiphysics analysis must move earlier in the design flow.
Co-designing for a single energy budget
Across the industry, companies are increasingly treating energy, not raw speed, as the starting point for design.
To stay within realistic power envelopes, chips, boards, and software can no longer be designed and optimized in isolation. Instead, the entire AI stack — from software and models down through racks and silicon — must operate within a fixed energy budget.
“I am a big believer in extreme co‑design and vertical integration,” said Artour Levin, corporate vice president of AI silicon engineering at Microsoft. “You cannot just optimize it locally. You need to look at this holistically and optimize it starting with silicon all the way to software.”
MatX, which develops specialized hardware for AI workloads, takes a similarly broad approach.
“We decided it wouldn’t be good enough to just build excellent chips,” said Avinash Mani, chief development officer at MatX. “We are building our own chassis, systems, and our own racks.”
From power entering the rack through power delivery, conversion, and distribution to the chips, MatX engineers work to reduce losses at every step.
“We lose 15% to 25% of power supplied to the rack because of all of these transitions,” he said. “It requires co-design across the entire rack to get to a system that is wasting the least amount of power.”
Energy efficiency is also reshaping AI models themselves.
At UCLA, researchers are developing a hierarchical memory transformer architecture to make large language models (LLMs) more power efficient. Inspired by human short-term and long-term memory, the approach achieves a 20× increase in model capacity and a 5–10× reduction in energy.
“If you can innovate on models, architecture, and also the silicon implementation,” said Jason Cong, Volgenau chair for engineering excellence at UCLA, “that’s where you have the most significant return on investment.”
Reducing the cost of data movement
The less data a system has to move, the less energy it uses. Hardware teams are addressing that challenge by packing connections more densely and keeping them as short as possible.
At Samsung Foundry, for example, engineers use hybrid copper bonding and high-bandwidth memory (HBM) to boost bandwidth while lowering capacitance per signal. Together, those advances reduce the energy required to move each bit between logic and memory.
“From an integration point of view, the question is how to increase the bandwidth,” said Hyung‑Ock Kim, corporate vice president of design technology at Samsung Foundry. “Hybrid copper bonding provides a narrow pitch. It’s very helpful to increase signaling speed.”
The same physics‑first discipline applies at the transistor level. Kim described how Samsung’s design technology co-optimization (DTCO) team tunes device size to maximize performance while minimizing wasted power.
“We deliver the smallest device for non‑critical paths and a large device for the critical paths,” he explained.
Selective sizing — combined with denser interconnect and HBM packaging — shrinks the wires and drivers needed to move bits, squeezing more work out of each joule.
Cong’s group at UCLA is also exploring how to change the workload itself. In a standard transformer‑based LLM, most of the energy goes into matrix multiplications and the associated data movement. The UCLA team’s recent Lang‑LM prototype pushes most language model inference into table lookups stored in dense on‑chip memory, sidestepping multipliers.
“This is a large language model basically without computation, by memory lookup only,” Cong explained, noting several‑fold speedups and a 4–6× improvement in energy efficiency on FPGA hardware.
Leveraging AI in the design flow
AI workloads are driving power budgets up, but AI-driven EDA tools are also helping push them back down.
Cong outlined three roles for AI in EDA that directly impact efficiency: fast prediction of power, performance, and area (PPA); search and “inverse design” of better architectures and RTL; and agentic flows where AI orchestrates existing tools using embedded domain knowledge.
Together, these capabilities enable earlier, more systematic exploration of the tradeoffs between energy consumption and performance.
“[AI-driven tools] can provide many options and many parameters to be explored to optimize PPA,” Kim said, noting the combination of Synopsys Fusion Compiler and Synopsys DSO.ai. “AI is very useful to shorten run times because it can shuffle some parameters and find good options.”
With agentic AI solutions on the horizon, Levin said the industry has only scratched the surface of what’s possible.
“We are at a very early phase of AI getting into silicon development,” he said. “And, honestly, we don’t have a choice. We won’t be able to scale without using technology like AI. The traditional approach will not allow us to move fast enough.”
“I remember even a year ago when we literally had to encourage, push, and convince engineers to give it a try,” Krishnamoorthy added. “A year later, they are running so fast with it, we are like, ‘Are you checking everything?’”
Optimizing the full stack
With AI compute scaling faster than energy infrastructure, efficient use of power has become a defining design factor. But squeezing every milliwatt out of individual components won’t be enough.
Companies must build workflows that treat efficiency as a full‑stack design objective — linking models, silicon, systems, and software into a single, integrated engineering problem.
“We don’t have a good platform today that allows us to evaluate AI as a distributed system,” Levin said. “And optimizing software on top of it makes it even more challenging.”
The next leap in AI performance will depend not only on more powerful chips, but on better ways to design, model, and optimize the entire stack within a fixed energy budget.
Continue Reading
