ZHCSKP3K September 2021 – April 2024 TDA4VM , TDA4VM-Q1
PRODUCTION DATA
The Depth and Motion Perception Accelerator (DMPAC) is a power efficient hardware accelerator that computes dense stereo depth maps (depth) and dense optical flow vectors (motion) from camera inputs.
The image/video sensor-based environmental perception (also known as scene understanding) is at the core of many emerging applications in automotive, industrial and consumer electronics. Typically, this involves detection of all objects in the scene along with their 3D position and motion with regards to the observer or the car by analyzing one or many related input video streams. Various computer vision algorithms are used to achieve these tasks.
A very robust method of obtaining the 3D depth from images is to use two cameras in a stereo setup - two cameras with known relative positions and camera parameters. The two images of the same scene, captured from two different camera poses/perspectives, are analyzed to find disparities among every pixel positions in the images. This is known as the Stereo Disparity map. The disparity values of every pixel can be used to obtain the 3D positions of the object/space they belong to via triangulation.
On the other hand, by analyzing two images from a single camera, captured at two different time instances (that is, two temporal frames in a video), one can determine where each pixel in a past frame moved to in the future frame. This is known as the Optical Flow vector. The flow vectors for each pixel position can be used to obtain 3D structure of the scene, identify moving objects and determine their relative speed and direction of motion.
The DMPAC is dedicated to the aforesaid image processing tasks. The stereo and optical flow processing is partitioned into two top level sub-blocks: the Dense Optical Flow (DOF) engine and the Stereo Disparity Engine (SDE). The DOF and SDE blocks share a common shared local memory, DMA, external messaging and control infrastructure.
For more information, see Depth and Motion Perception Accelerator (DMPAC) section in Processors and Accelerators chapter in the device TRM.