Intrinsic 3D vision system
Generate precise 3D point cloud data with a novel deep learning stereo architecture, unlocking new automation possibilities.
Read the paper
Generate precise 3D point cloud data with a novel deep learning stereo architecture, unlocking new automation possibilities.
Read the papera new approach
Traditional stereo camera architectures struggle to produce accurate 3D reconstructions for complex objects such as deep bins, wired bins, or transparent and reflective objects. Our deep plenoptic stereo architecture greatly enhances point cloud accuracy for these objects, delivering richer 3D data. Combined with multi-unit auto camera calibration workflows, the system allows for a modular and scalable light field capture in a robotic workcell without the need for different hardware skus for different workcell sizes.
A first-of-its-kind deep learning algorithm that fuses RGB, IR, and polarization sensor data from multiple viewpoints into a single cost volume, enabling high-resolution, highly accurate 3D reconstructions.
We demonstrate a purely synthetic training framework for our multimodal sensor fusion network to prevent over-fitting. Collecting and passing polarization images (both angle & strength) along with RGB and IR, through a neural network to compute the cost volume, improving depth maps for transparent and reflective objects.
An automatic calibration pipeline that precisely aligns multiple cameras (or sensors) to each other and corrects “drift”, allowing for flexible camera placement and high quality 3D without the need for manual calibration.
the research
This paper has been published to Siggraph Asia
Download Papers
2 files, 20.6 mb
