Reduce Link Bring-Up Time with PCIe 5.0 Modes

Billions of internet-connected devices and data-intensive real-time applications are expected to appear on the market in the near future and 100 Gigabit Ethernet (GE) speeds, common in data centers today, will just not be fast enough to handle the bandwidth. Therefore, we’re already anticipating the need for data center operators to migrate their networks from 100 GE to 400 GE, creating demand for faster memory and faster serial bus communications.

PCIe is a high-speed, differential, serial standard for point-to-point communications. Each new generation of the PCIe standard offers additional features and faster data transfer rates than the previous generation. The latest generation, PCIe 5.0, will double the throughput rate of PCIe 4.0. The transfer rate of PCIe 5.0 is 32 gigatransfers per second (GT/s) versus the 16 GT/s supported by PCIe 4.0. With 64 gigabytes per second (GB/s) of unidirectional transfer bandwidth, PCIe 5.0 provides data throughout at 128 GB/s of bidirectional traffic.

In addition to doubling the bandwidth, PCIe 5.0 delivers other new features such as:
• Equalization Bypass Modes for faster link initialization
• Allows alternate protocols to negotiate through link training
• Precoding support to help avoid burst errors
• Loopback enhancements allowing to mimic crosstalk behavior

In this blog, we will cover the PHY logical changes at 32 GT/s and walk through the major changes that take place in the equalization procedure.
But, before diving into the new equalization modes, let’s talk about why we need new modes in the first place…

An increase in data rate also means signal attenuation increases, which can affect higher frequency components the most. This ultimately results in distortion which needs to be compensated for through an Equalization Procedure. Like PCIe 3.0 and 4.0, Equalization is a recommended process for a device operating at 32GT/s to adjust the transmitter and receiver setup to improve the signal quality. The equalization phases (phase 0,1,2,3) for PCIe 5.0 remain the same as the previous generations.

Let’s look at the steps involved to bring-up link to 32 GT/s. The link must initially train to L0 at 2.5 GT/s followed by equalization at 8.0 GT/s, 16 GT/s and 32 GT/s sequentially. This is known as the conventional ‘Full Equalization’ Mode.

As data rates get higher and higher, this approach becomes more time consuming and increases latency of link bring-up. As per some vendors, single speed transition with equalization takes about ~100ms, meaning it would take ~300ms to bring-up link to 32GT/s. The need to optimize link bring-up led to two new optional modes being introduced:

The graph below shows how link bring-up time to 32GT/s is drastically reduced through new equalization modes: