The solution combines the Hannover-based company’s low-power, scalable v-MP4000HDX processor IP core family with an accelerated OpenCV library, and a suite of Full HD and 4K video encoders and decoders. The licensable solution enables new applications for mobile, gaming and TV platforms: augmented reality, gesture interfacing, depth mapping and computational photography enable new user experiences and usage models. In automotive applications, the company’s low-delay, high dynamic range video codecs combined with computer vision techniques enable both camera-based advanced driver assistance systems and surround-view cameras to prevent collisions.
These applications all require a high-performance, low-power flexible video processing subsystem such as the v-MP4000HDX to run the computationally demanding algorithms that can extract meaning from the images the cameras capture, without draining the battery. This same architecture also supports video compression and decompression, so that the system can simultaneously decompress, analyze and compress video data.
“We’ve been shipping our processor for multi-format HD video coding for some time. Our customers asked if we could add support for computer vision after they experienced our power and area efficiency on video coding applications, and so we did,” said Hans-Joachim Stolberg, videantis’s CEO. “We optimized the widely used OpenCV library for our v-MP4000HDX architecture and combined it with our video codecs, so they’re able to run concurrently. In addition, we took great care in optimizing both the hardware and software for low-power operation, which is of course important for mobile applications, but also a major factor for embedded applications.”
The v-MP4000HDX architecture scales from a three-core v-MP4120HDX subsystem with a single stream processor and two media processors to a ten-core v-MP4280HDX subsystem that includes two stream processors and eight media processors. The v-MP4180HDX subsystem performs 192 16-bit pixel operations per cycle, 24 on each of its eight two-issue VLIW media processors. This subsystem is under 2mm2 of silicon in 28nm technology, including memories. With four media processors, the v-MP4140HDX subsystem performs 96 16-bit pixel operations per cycle. This subsystem is under 1mm2 of silicon in 28nm technology, including memories. Thanks to its flexible architecture, the subsystem can run video codecs, imaging, computer vision, and computational photography algorithms all simultaneously.
The v-MP4000HDX accelerates OpenCV, the de facto standard for open computer vision libraries. OpenCV has been downloaded over 3 million times and is widely used in both academia and industry. Using OpenCV, algorithm development can take place in a standard PC-based environment. Once algorithm development and testing are completed, the application can seamlessly be migrated toward an embedded videantis-enabled platform where the OpenCV function calls are accelerated by the v-MP4000HDX subsystem. This acceleration results in a speedup of more than 100x compared to running on the host CPU, and power consumption that’s 1000x lower. This way the v-MP4000HDX supports full HD processing, allowing for more accurate algorithms and an overall higher-quality user experience.
Besides running computer vision algorithms, the v-MP4000HDX supports the following codecs running at Full HD or 4K resolution for decode and Full HD for encode. Since the subsystem is software programmable, additional coding standards can run on the same silicon. The v-MP4000HDX can be firmware upgraded to run H.265/HEVC, VP9 or high dynamic range still image codecs, for example.
The v-MP4000HDX processor IP core, optimized OpenCV library and suite of video codecs are now available for licensing. A development and evaluation system is available that includes a 40nm implementation of the v-MP4280HDX processor at 400MHz, a 1GHz ARM host CPU, camera input, HDMI output and networking.