The PCI-SIG recently introduced a new Engineering Change Notice (ECN) extending the PCI Express (PCIe) Base Specification to include optional support for trusted I/O Virtualization.  Known as TDISP, this ECN has been under development for almost two years, and ties together existing PCI-SIG technologies for Single-Root I/O Virtualization (SR IOV) and hardware security via Integrity and Data Encryption (IDE).  While TDISP applies equally to CXL and PCIe devices, further complementary extensions are in development in the CXL consortium to handle additional CXL-specific conditions.  This article explains the new features and benefits of TDISP for SoCs targeting hyperscale datacenters.

PCIe Security with IDE

In late 2021, PCI-SIG released an ECN titled Integrity and Data Encryption (IDE) which added optional capabilities for PCIe devices to perform hardware encryption and integrity checking on packets transferred across PCIe links.  Fundamentally, the goal of IDE is to protect against hardware-level attacks conducted by skilled attackers with both sophisticated tools and direct access to their victim systems.  In prior years, this scenario would have been viewed as astronomically unlikely as companies owned and operated their own datacenters, staffed them with their own trusted employees, and considered physical security to be of paramount importance.  One result of the shift to “cloud computing” is that fewer and fewer companies fit that direct ownership model, with more and more of them relying on shared resources inside massive shared datacenters – potentially located half the world away.  Suddenly the idea of an attacker with a cart filled with logic analyzers and oscilloscopes pulling the covers off running servers probing their internal busses didn’t seem so far-fetched.  

 

The IDE ECN is specifically geared to defeat such attackers equipped with laboratory equipment (like bus analyzers) and even malicious dedicated hardware (such as interposer cards and co-opted PCIe switches).   With the new ECN, PCIe devices can use defined AES GCM algorithms to cryptographically sign and/or encrypt each packet transferred across the PCIe link.  The two types of IDE Streams are: Link – valid only from one device directly connected to another, and Selective – intended to be carried across PCIe switches.  These two Stream types can be combined as shown in Figure 1, with some links carrying both types, and other links only one type.  One important protection this provides is that devices expecting to be directly connected via a Link IDE Stream will fail to establish a secure connection if a malicious PCIe switch is unexpectedly added in between them.  Similarly, Selective IDE Streams are carried unmodified across PCIe switches – meaning the switch itself can never decrypt the data, so an attacker can at best make a copy of the encrypted data.  Mechanisms within the IDE specification protect against further attacks such as forcing retries and injecting packets in attempts to force repeated transmission of the same data to expose the cryptographic keys being used.

Figure 1: IDE Stream flows for Link Streams and Selective Streams

When IDE was introduced, the new features provided essential building blocks for securing computing environments, but it was apparent that some sort of standardized framework would be needed to manage and provide interoperability amongst different implementations.  This is where TDISP comes in.

Subscribe to the Synopsys IP Technical Bulletin

Includes in-depth technical articles, white papers, videos, upcoming webinars, product announcements and more.

TDISP – A Framework for Managing Secure Environments

As the industry has evolved to support the kinds of shared data centers described above, the concept of Trusted Execution Environments (TEEs) has arisen in virtualization.  Here, the idea is that the owner of a Virtual Machine (VM) may not want to trust the Virtual Machine Manager (VMM) (sometimes called a Hypervisor) in the same way a physical server owner might not want to trust the datacenter personnel.  Within a TEE, the VMM still controls resource management as in a traditional virtual environment, but unlike a legacy VM, a TEE Virtual Machine (TVM) can manage its own security.  For example, the VMM may allocate certain encryptable memory pages to the TVM, but it’s the TVM that controls the encryption keys for those memory pages.  On the CPU and memory side of the server, these controls are specific to the CPU and memory architecture in use, so no new specifications are needed to expose them to the VMM.  When the concept is extended to an I/O interface like PCIe, it’s much more important to have an open framework that doesn’t require the VMM or TVM to have such specific knowledge of the devices being connected.

 

TDISP stands for TEE Device Interface Security Protocol, and it provides for three main functions:

1) Establishing a trust relationship between a TVM and a TDISP-capable device

2) Securing the PCIe path between the host and TDISP-capable device

3) Attaching/detaching a TDISP-capable device’s interface to/from a TVM in a secure manner

While much of this functionality resides in software (or firmware), the TDISP specification requires a number of different behaviors and PCIe features in TDISP-capable devices.  

 

TDISP currently focuses on using Virtual Functions as defined by the PCI SIG Single-Root I/O Virtualization (SR IOV) specification, but notes that future revisions may be extended to other schemes.  Physical Functions (PFs) are defined as PCIe Functions which include the SR-IOV capability structure and can therefore operate as the management entities for associated Virtual Functions (VFs).  VFs were originally envisioned as lightweight functions which could be directly assigned to VMs, but which might rely on the VMM to emulate some aspects of their PCIe configuration space.  

 

 

Figure 2: Mapping PCIe Physical and Virtual Functions to Trusted Environments

TDISP logically requires VFs not to rely on the VMM for any security-related functionality, and it formalizes the idea of each VF as an assignable unit by defining a TEE Device Interface (TDI) which encompasses the additional requirements placed on VFs to ensure security.  Functions in TDISP devices must be able to lock down their TDI configuration and report tampering back to their associated TVM in case the VMM (or an attacker) should modify relevant control registers outside the negotiated TDISP scheme.  Naturally PCIe configuration space registers are part of the TDI, but also device-specific registers for functions such as memory mapping, caching, DMA, etc. must be secure from outside manipulation.  TDISP also explicitly requires that devices’ PFs not be able to tamper with any confidentiality and/or integrity features of their associated VFs – again building on the idea that overall management functionality must not be able to impact the security of resources it manages.  

 

The TDISP ECN defines two logical/software entities as part of the management infrastructure:

  • TEE Security Manager (TSM) – exists in the host to enforce security policies on the host and attached devices
  • Device Security Manager (DSM) – exists in each TDISP device to enforce security policies on that device working in conjunction with the TSM

Here again numerous existing virtualization technologies apply to securing the TSM inside of hosts, but a secured environment for devices to implement their DSMs may be new to many SoC designers.  The functions required of a DSM are well specified in the TDISP ECN, but designers should take a very close look at the hardware and software they use for their DSM implementations to ensure that security policies can be robustly enforced.

 

To establish the TVM and device trust relationship, TDISP uses the Component Measurement and Authentication (CMA) ECN, published around the same time as the IDE ECN, to enable cryptographically secured identification and authentication of both parties.  Once complete, the TVM may configure the security aspects of the device and then lock its TDI from further manipulation.  The TDISP specification also documents different mechanisms by which further cryptographic keys may be exchanged in order to facilitate the next phase of securing the PCIe path between the TVM/host and the device.  Provision is made for directly-connected devices to use Link IDE Streams, but Selective IDE Streams are used in the general case.  Note that there are some changes to the IDE specification which were part of the TDISP ECN, so designers with pre-existing IDE implementations should pay close attention to the relevant sections of the TDISP ECN.

Conclusions

Designers developing SoCs for hyperscale datacenters and other security-conscious applications should review the TDISP specification with their customers to see if it can improve the overall security of their systems.  TEE Device Interface Security Protocol or TDISP offers optional support for trusted I/O virtualization and ties together with existing PCI-SIG technologies for SR-IOV and hardware security via IDE.  Implementers of TDISP need to pay close attention to mechanisms for securing their DSMs and take note of changes which may be necessary to any pre-existing IDE implementations.  Synopsys offers a wide range of PCIe and CXL controller IP including support for encryption with IDE and providing the mechanisms necessary for secure TDISP implementation.  Synopsys further offers an array of secured processors and cryptographic tools which can greatly simplify implementation of DSMs.

Synopsys IP Technical Bulletin

In-depth technical articles, white papers, videos, webinars, product announcements and more.

Continue Reading