Leveraging In-Memory Compute Technology to Run AI at the Edge: A Conversation with TetraMem

Synopsys’ Sanjay Bali sat down with Glenn Ge, CEO of TetraMem, to explore a problem that’s quickly becoming one of the biggest bottlenecks in AI: how to run meaningful intelligence  efficiently on edge devices such as phones, wearables, sensors, and AR/VR devices.

AI is spreading from cloud to the edge, but edge devices face strict constraints on power, latency, and bandwidth. TetraMem is positioning itself not as a drop-in replacement for today’s digital accelerators, but as an enabler for new edge experiences, such as running “ChatGPT-like” capabilities locally or powering smart glasses without relying on constant network connectivity. In Glenn’s view, the barrier isn’t transistor scaling alone. Even as nodes move to 3nm and beyond, efficiency under limited power budgets remains the practical constraint, one that analog in-memory computing can help address.

While in-memory computing concepts date back decades, Glenn explains that commercialization is only becoming viable now because modern AI workloads require massive matrix operations, and RRAM devices have matured to the point where they can be manufactured and controlled reliably.

TetraMem’s most recent tapeout is a compact 22nm Edge AI Accelerator SoC that combines a RISC-V based CPU with analog in-memory compute NPU macros and 4MB of RRAM crossbar memory array. It’s not designed to run large language models, but it is well-suited for tiny models such as voice and small computer-vision workloads. Because RRAM is non-volatile, it can retain stored neural-network weights for long periods without DRAM-style refresh, enabling always-on edge scenarios.

TetraMem has also recently received silicon for its MLX200 platform, described as the first multi-level RRAM SoC ready for evaluation. Target applications include audio processing (denoising, keyword spotting, voice ID) and smart vision sensors. The TetraMem roadmap includes memory-enhanced and smaller variants, as well as a move to 12nm FinFET aimed at significantly higher performance and efficiency.

Sanjay and Glenn also discussed what it takes to build a hardware startup. The conversation highlights the importance of identifying a differentiated solution and how collaboration can help startups accelerate innovation and bring new technologies to market.

Glenn acknowledged the intense challenges of the journey, and also shares that partners like Synopsys make it more manageable by providing a strong technical foundation for solving real customer problems.

If AI is to scale from serving a small fraction of users to becoming as ubiquitous as smartphones, efficiency must improve dramatically. TetraMem’s mission is to advance computing towards brain-like efficiency, making AI more affordable, energy-efficient, and deployable everywhere, from sensors to data centers.

If you’re tracking the future of edge AI hardware, this conversation offers a clear takeaway: the next leap may not come from “more compute,” but from rethinking where compute happens, starting inside memory itself.