CXL 3.1: How Evolving CXL Standards are Pushing Interconnects to Even Higher Performance

Dana Neustadter, Gary Ruggles, Richard Solomon

Jul 02, 2024 / 4 min read

First introduced back in 2019, the Compute Express Link (CXL) is an open industry-standard interconnect between processors and devices such as AI accelerators, memory buffers, smart network interfaces, persistent memory, and solid-state drives. As an industry-standard interface, CXL promotes interoperability between different hardware components from various manufacturers, reducing compatibility issues and allowing for a more diverse and competitive market for data center hardware.

Although it was launched only 5 years ago, much has changed in our world since then, not the least of which is the expanding workloads of data center and high-performance computing environments that power the ever-expanding machine learning and artificial intelligence solutions being used to enhance our lives.

At its core, CXL offers coherency and memory semantics with bandwidth that scales with PCIe while achieving significantly lower latency than PCIe. Read on to learn how leveraging compliant CXL IP allows designers to stay at the forefront of technological advancements and meet market demands and evolving industry standard. This blog will cover how the latest updates in the CXL 3.1 standard include new security features, as well as how it continues to provide the capabilities for memory pooling for next-generation infrastructures.

cxl 3.0

The Evolution of the CXL Standard

data center cxl standard

The transition from CXL 2.0 to CXL 3.0/3.1. Source: CXL Consortium

The journey from CXL 2.0 to CXL 3.x marks significant progress in addressing industry demands. One of the primary drivers behind the evolution to CXL 3.x is the need for higher speeds, aligning with PCIe 6.x. Additionally, there's been a shift toward a more symmetrical coherence model, allowing devices connected to CXL to generate memory cycles and communicate more effectively with each other. This enhancement facilitates a more versatile and efficient use of memory across different devices, moving beyond the traditional host-centric approach​​.

One thing has remained constant: CXL continues to be a pivotal interconnect technology enabling efficient, high-performance communication between CPUs, accelerators, and memory devices. The latest CXL 3.1 specification addresses the growing needs of data-intensive applications in next-generation computing.

Key Features of CXL 3.1

CXL 3.1 marks a significant step forward in interconnect technology building on CXL 3.0, offering enhanced speed, security, flexibility, and efficiency necessary to meet the rapidly increasing industry demands. More specifically, CXL 3.1 offers the following improved features compared to its predecessors:

  • Extended Meta Data: Supports up to 32-bits of meta data per cacheline and 2-bits of meta-data for cache coherency that describe if a cache line is shared, exclusive, modified, or invalid. 

  • New Use Cases: New information gleaned from meta data allows for use cases such as access control, data type tagging, memory-tiering algorithms, and more.

  • Improved Visibility into CXL Memory Device Errors: Provides new information on correctable error limits, the source of the errors, and the transaction occurring during the error condition.  

  • More Control Over Memory Device RAS: Offers memory sparing, DDR5 error-check scrubbing, media testing, patrol-scrub, and information on capacity/performance degradation.

  • Trusted Execution Environment Security Protocol (TSP): Enables virtualization-based TEEs to host confidential computing workloads.

  • Peer-to-Peer Communication: Facilitates direct peer-to-peer CXL memory access for accelerators, improving resource utilization and reducing latency.

While all these features are true enhancements, CXL 3.1 introduces more complexity and, therefore, new challenges. With the release of any new standard, addressing these challenges is crucial to ensure stability and performance. For instance, CXL 3.x has adopted a 256-byte flit format to enable forward error correction, similar to PCIe 6.x, to mitigate bit errors. This change is pivotal in maintaining data integrity and performance at higher transfer rates​​. 

Emphasizing Security with Trusted Execution Environment Security Protocol (TSP)

IP for CXL standard

Security remains a cornerstone of the CXL 3.1 standard. Specifically, the Trusted Execution Environment Security Protocol (TSP) applies to CXL.cache and CXL.mem, and is designed to secure virtualization-based environments, enabling them to host confidential compute workloads. This feature builds on the existing Integrity and Data Encryption (IDE) protocols introduced in CXL 2.0, further enhancing data protection across the interface. TSP's integration ensures that data remains confidential and intact, safeguarding against potential security threats in virtualized environments which is even more important today as data becomes more sensitive and regulations become more strict​​. virtualization support continues to be governed by the TEE Device Interface Security Protocol (TDISP), released in August 2022, managed by PCI-SIG.

These security protocols collectively enhance protocol-related security. Synopsys’ expertise in these areas enables us to provide comprehensive support and ensure that our implementations are both robust and secure.

Future CXL 3.1 Enhancements and Synopsys’ Commitment

Looking ahead, we foresee continuous enhancements in the CXL standard. The industry is rapidly evolving, with new requirements and technologies emerging at a faster and faster pace. Synopsys remains committed to staying ahead of these changes. Our involvement in the standards development process ensures that we are not only prepared for new features but also actively contributing to shaping the future of these standards.

As CXL continues to grow and adapt, Synopsys will be at the forefront, ready to integrate and support these advancements. With a strong focus on security, performance, and adaptability, we are well-equipped to support our customers in leveraging the full potential of CXL 3.1 and beyond. Synopsys CXL IP provides a complete IP solution for CXL (consisting of controller, PHY, IDE Security Modules, and verification IP, which delivers secure, low-latency and high-bandwidth interconnect for AI, machine learning, and cloud computing applications. To lower risk, Synopsys CXL IP solutions are built on Synopsys' PCI Express IP, which has been silicon-proven across a range of applications. For more information, please visit the Synopsys CXL IP Solutions webpage.

Continue Reading