Synopsys and Ansys power the future of innovation—connecting silicon to systems.
EDA
System
IP
SolvNetPlus
Training & Education
Part 1: The Rack Becomes the New Unit of Compute
Part 2: Designing AI Systems at Rack Scale
Part 3: Compute Only Scales When Data Can Move
AI factories are being rebuilt around the rack.
The rack brings together the GPUs, memory, and storage needed to support AI workloads — whether for inference, training, or other tasks. And it determines how efficiently those workloads deliver output.
As the industry pushes for increasingly powerful racks, it’s fair to ask why these advances are needed and what they enable. Next-generation AI factories will be able to run much bigger models, deliver real-time multimodal inference, and support agentic workflows at a practical cost per token.
In other words: more performance, more efficiency, more possibility.
AI Doesn’t Scale by Accident
Explore the IP that enables high-performance, scalable AI systems
Hard problems, solved together
To achieve these gains, a handful of hard problems must be solved. And they have to be solved together.
Inside the silicon, compute is moving into 3D. The main die is consumed by compute cores, which pushes I/O and high-bandwidth memory (HBM) onto separate dies. Stacking those dies keeps them close, gives compute direct access to memory, and saves power. Across the rack, many accelerators are connected together and tied to storage and memory through fast interconnects, both die-to-die and rack-scale.
Distance becomes a problem as data rates rise. With copper reaching its limits, the industry is shifting from pluggable optical modules to co-packaged optics, bringing optical connections closer to the silicon.
And the interfaces themselves are being tailored to the work.
For two decades, interface IP was built to a standard. As customers design the rack as an integrated system, nearly every interface IP has to be customized for the workload and delivered as a silicon-proven subsystem — one that already accounts for the conditions it will face. We used to call these harsh data center conditions. Now they are harsh rack conditions.
None of these challenges can be solved in isolation.
Managing heat within a 3D package, delivering power, choosing where copper gives way to optics — each decision affects the others. That is the work of extreme co-design: designing silicon, IP, memory, cooling, and power as one system, with the rack treated as one machine.
Doing more with less
When those problems are solved, the payoff shows up in familiar terms: token per dollar, performance per watt, and total cost of ownership. As racks march toward one megawatt of power, every one of those measures is set by how well the whole system is designed to work together.
Usable AI is the measure.
When a user submits a workload — a coding task, for example — it lands on a collection of GPUs connected closely to memory and storage to perform that task. How fast that work gets done, and at what cost per token, is what separates an efficient AI factory from an expensive one.
That efficiency is what will make the next generation of AI models practical to run. Bigger models, real-time multimodal inference, and agentic workflows are expensive to serve today. Bringing down the cost and energy of every token is what moves them from possible to affordable at scale.
Designing infrastructure for models that keep changing
AI workloads continue to change quickly. What a factory runs today is not what it will run next month, and the models keep growing.
A rack built only for today's workload is already behind.
Rack designs, therefore, have to leave room for adaptation. Connecting many accelerators into a scale-up fabric — and tying them together with fast interconnects and memory — lets a rack take on larger models and reconfigure around new ones. Adaptability is becoming as important as raw compute, and it must be built into the architecture of the rack and every component inside it.
The AI factory of 2030
Between now and the end of the decade, AI factories will scale in two directions at once: inside the silicon through 3D stacking, and within the rack as more accelerators are connected and tied together with fast interconnects and memory capacity.
That trajectory is set, but it is not easy.
Stacking dies concentrates heat in a smaller space. Tailoring each interface to its workload means more design work, not less. Power delivery, cooling, and signal integrity must be addressed together rather than one at a time. There is no single fix for any of it.
What makes the effort worth it is the resulting machine. A rack built and optimized as one system, drawing close to a megawatt, able to take on models that keep growing and changing. These are the foundations of the next-generation AI factory.
More performance, more efficiency, more possibility.
Part 1: The Rack Becomes the New Unit of Compute
Part 2: Designing AI Systems at Rack Scale
Part 3: Compute Only Scales When Data Can Move
Continue Reading
Architecting AI Factories: Rack-Scale Design Series
