MIPI Interfaces in IoT Applications
While there are many forms of IoT SoCs, let’s describe a superset of components or interfaces, including MIPI, typically found in an IoT SoC. CSI-2, potentially DSI, and a processor comprise the vision processing component of the SoC. The memory component consists of LPDDR for low-power DRAM and embedded Multi-Media Card (eMMC) for embedded flash. For wired and wireless communication, specifications like Bluetooth low energy, Secure Digital Input Output (SDIO) and USB are leveraged depending on the target application. To secure the data that travels through the cloud and stored in the device, security becomes an essential component, which mainly includes engines such as true random number generators and cryptographic accelerators.
The component where dozens or more sensors are connected is the sensor and control subsystem with I2C or I3C and serial peripheral interface (SPI). I3C is a new MIPI specification that incorporates and unifies key attributes of I2C and SPI while preserving the two-wire serial interface. System designers can connect a larger number of sensors in a device while minimizing power consumption and reducing component and implementation costs. At the same time, utilizing a single I3C bus enables manufacturers to combine a variety of sensors from different vendors to enable new functionalities while supporting longer battery life and cost-effective systems.
MIPI Interfaces in Multimedia Applications
A new use case for the MIPI camera and display interfaces is in multimedia applications such as virtual/augmented reality devices with high resolution cameras and displays. In such devices, the interfaces transmit and receive multiple images from various sources that are then processed and sent to the user with the utmost quality. Below are three examples of a multimedia application implementations:
- High-end multimedia processor: In this implementation, multiple display and camera inputs – typically coming from another application processor that has already processed and received the image but is not yet ready for transmission – come into the image signal processor via CSI-2 and DSI. The image signal processor then transmits the image via CSI-2 or DSI to either the camera or display.
- Multimedia processor: This implementation is mainly for gesture or movement recognition or human machine interface. Two image sensors, via CSI-2 protocols, interface with the processor where the movement or gesture is recognized and processed for further analysis and manipulation. The processed movement or gesture data is then transmitted to the application process via the CSI-2 protocol.
- Bridge IC: Since there are multiple image inputs and outputs, as explained in the automotive section, there is a need for bridge ICs. The bridge IC allows for the output of one application processor to split into two display streams.
Advantages of the MIPI Interfaces
MIPI CSI-2 leverages the MIPI D-PHY physical layer to communicate to the application processor or SoC. The image sensor or CSI-2 device captures and transmits an image to the CSI-2 host where the SoC resides. Before the image is transmitted, it is placed in the memory in individual frames. Each frame is then transmitted through the CSI-2 interface via virtual channels. A virtual channel is used and required for multiple image sensors, which could support different pixel streams, sometime multiple exposures, and assign virtual channel identifications to each frame. Each virtual channel is divided into lines which are transmitted one at a time, allowing for transmission of a complete image from the same image sensor but with multiple pixel streams.
MIPI CSI-2 uses packets for communication which include data format as well as an error correction code (ECC) functionality to protect the header and CRC for the payload. This implementation applies to every packet transmitted from the image sensor to the SoC. A single packet travels through the CSI-2 device controller via the D-PHY and then split into the number of required data lanes. The D-PHY distributes the data to several data lanes operating in high-speed mode and transmits the packet to the receiver via the channel. The CSI-2 receiver using its D-PHY physical layer, extracts and decodes the packet, which is ultimately delivered to the CSI-2 host controller. This process is then repeated frame by frame from the CSI-2 device to the host in an efficient, low power and low cost implementation.
In a typical system with multiple camera and displays, the same physical layer (D-PHY) by both CSI-2 and DSI protocols is utilized. Depending on the target application, there are many considerations to account for during the discovery phase such as required bandwidth and device type. Knowing such considerations can help designers determine the D-PHY version with the required number of lanes and speed per lane, which can then determine the number of pins required to implement in the system. Ultimately, designers can determine the required interface and memory for their target applications. For example, there are implementations where CSI-2 over D-PHY is operating at 1.5 Gbps per lane and other implementations where the operation is up to 2.5 Gbps per lane. The operation at lower speed has implications on power and area, but most importantly, it is not future proof with newer image sensors and display designs that support the faster speed.
The use of multiple cameras and displays are now in applications beyond mobile such as automotive, IoT and multimedia including augmented/virtual reality devices. All of these applications demand high speed and low power camera and display interface solutions that meet the demands of today's high-resolution image processing. MIPI CSI-2 and DSI are proven interfaces in the mobile market, mainly smartphones, and because of their successful implementation, they are being utilized in new applications. Synopsys' broad portfolio of MIPI IP solutions, consisting of controllers, PHYs, verification IP and IP Prototyping Kits, are compatible with the latest MIPI specifications, allowing designers to incorporate the required functionalities in their mobile, automotive and IoT SoCs while meeting their power, performance and time-to-market requirements.