Developing Software for Proprietary Cameras for Integration with CSI-2 Host IP Prototyping Kits

By Ramiro Oliveira, Embedded Software Engineer, Synopsys

CSI-2-compliant cameras are proliferating through the mobile industry due to the CSI-2 standard’s ability to handle high image resolution over fast links with low power consumption. Each camera has specific configuration requirements and characteristics, which complicates the tasks of providing support and integration, due to limited code reusability and steep learning curves. Systems with embedded Linux can leverage existing libraries, API and driver frameworks to reduce design effort. This article describes how to use the existing host-side V4L2 API and V4L2 sub-device interface to ease integration of a CSI-2 compliant camera with an existing system, using Synopsys DesignWare CSI-2 IP Prototyping Kit as an example. This approach will allow you to change your camera easily without having to make any changes to the CSI-2 host driver.

The CSI-2 specification defines a communication interface between cameras and processors that allows devices from different manufacturers to work together. CSI-2 is a standard, robust, scalable, low-power, high-speed, cost-effective interface that supports a wide range of imaging solutions for mobile devices.

CSI-2 Host & Device Software Development

When starting a system design, designers need to keep their options open as they define the specific system requirements. Committing to a single camera could limit production and delay time-to-market due to camera availability or pricing.

Despite the need to keep the camera selection option open, SoC designers may try to accelerate their design process by integrating the software in one driver to make initial development easier and more directly control boot-up sequences. While it may work initially to write CSI-2 host and device code in the same file, eventually the SoC will need to support another device. The designer then needs to either split the code or add support for another device in the initial code. Adding support for another device in the initial code may seem faster, but after a while the code will become “spaghetti code.”

V4L2 API

The V4L2 Linux API is comprised of a set of routines, protocols, and tools for building host-side software applications and driver frameworks in the Linux environment. The software applications are used for video output and capture and the driver framework is used to integrate video-related devices such as cameras. V4L2 is supported by every Linux video capture application. Designers use the drivers found in the Linux kernel for CSI-2 devices and hosts written using the V4L2 API along with documentation and generic example drivers.

V4L2 Sub-device Interface

Device-side systems interfacing with the V4L2 API can be composed of several ICs which can lead to very complex drivers.

Figure 1 shows an example system with a CSI-2 host and device that need to be configured in sequence and with matching parameters. Also, one V4L2 sub-device can control another sub-device, as shown. The CSI-2 host is controlling the camera sub-device, which in turn controls a RAW converter and an image resizer.

Figure 1: V4L2 setup example.

Using the V4L2 Sub-device to Simplify CSI-2 Host Driver Development

The V4L2 sub-device interface offers a robust solution to executing the configuration. Through software, the V4L2 sub-device interface can support all CSI-2 cameras in the market, with minimal or zero changes to the CSI-2 host driver. The main driver will call the sub-device driver, ask it to power on, start streaming, change video mode, and so on, without needing to know anything about the camera it is talking to.

Designers should use the V4L2 sub-device interface when writing the CSI-2 device driver, and also use the V4L2 API to write the CSI-2 host driver. The Linux kernel includes several examples on how to do this along with documentation and generic example drivers.

The DesignWare IP Prototyping Kit for MIPI CSI-2 Host includes a V4L2 driver for Synopsys DesignWare CSI-2 Host IP and a V4L2 sub-device driver for the Omnivision OV5647. Replacing the camera on the system does not require changes to the host-side device driver. From the software perspective for the device side, designers need to either install the appropriate V4L2 sub-device driver for the camera from the Linux kernel, or write a new V4L2 sub-device driver for the camera, themselves.

Designers using the kit can also take advantage of the existing DesignWare CSI-2 Host IP and modify the source code to control other sub-devices to might add to the design (ex: RAW converter, image resizers, etc).

DesignWare IP Prototyping Kit for MIPI CSI-2

The DesignWare IP Prototyping Kit for MIPI CSI-2 Host Controller (Figure 2) includes a complete reference design that consists of a validated IP configuration and necessary SoC integration logic, implemented on Synopsys’ HAPS®-DX FPGA-based prototyping system. With the proven reference design for the IP, designers can accelerate the integration of IP into their target SoC, optimize the IP configuration, and develop drivers and software applications with real world I/Os and hardware. Out-of-the-box support for Linux OS and Windows ensures that software developers are up and running instantly and can focus on the IP specific software (e.g., drivers, bootcode, firmware). The kits plugs into existing software tool chains and interfaces seamlessly with popular embedded software debuggers, providing system-wide debug and analysis capabilities.

Figure 2: DesignWare IP Prototyping Kit for MIPI CSI-2 Host.

DesignWare MIPI CSI-2 Host IP

The DesignWare MIPI CSI-2 IP Host Controller IP (Figure 3) is fully verified configurable digital IP that provides a high-speed serial interface between an application or image processor and MIPI CSI-2 compliant camera sensor. The Synopsys DesignWare MIPI CSI-2 Host Controller IP is architected to interface with the physical layer through the MIPI recommended PPI, providing easy integration to a D-PHY such as the high-quality Synopsys DesignWare MIPI D-PHY IP.