DeepStereoVision

Effiziente und genaue Tiefenwahrnehmung durch Stereovision mit Deep Learning und FPGAs

01/01/2019 to 31/12/2021

The exact 3D surveying of objects or work areas is a basic requirement for many tasks in robotics, automation technology, quality assurance, and inspection. 3D acquisition with stereo vision is then a promising solution, especially when active camera systems that emit light themselves cannot be used because of too strong extraneous light (e.g. outdoors) or too great object distance. Stereo vision uses two cameras to determine the spatial depth via a comparison of the two camera images, similar to human vision. For stereo vision, the computationally intensive image processing is a challenge. If one wants stereo processing in real time, it is therefore necessary to use special hardware systems, as offered by Nerian Vision GmbH. Nerian's systems are based on programmable logic devices (FPGAs), which allow the image processing algorithms to be integrated directly into the hardware. Latest research results have shown that the application of Deep Neural Networks in combination with classical algorithms for stereo vision can significantly improve accuracy. This would make it possible to achieve a comparable or higher accuracy than with active sensors. In this research project, architectures for neural networks will be researched, which are suitable for stereo vision and can be executed on an FPGA. For these architectures suitable training methods must also be researched. At the same time, research will be carried out into how these neural networks can be integrated with a classical algorithm for stereo vision, such as Semi-Global-Matching (SGM), in a FPGA.