Multimodal deep learning for robust RGB-D object recognition

doi:https://doi.org/10.1109/iros.2015.7353446

Open AccessPreprint10.1109/iros.2015.7353446

Multimodal deep learning for robust RGB-D object recognition

Andreas Eitel,Jost Tobias Springenberg,Luciano Spinello,Martin Riedmiller,Wolfram Burgard-2015-09-01

132

TL;DRAbstract

Robust object recognition is a crucial ingredient of many, if not all, real-world robotics applications. This paper leverages recent progress on Convolutional Neural Networks (CNNs) and proposes a novel RGB-D architecture for object recognition. Our architecture is composed of two separate CNN processing streams - one for each modality - which are consecutively combined with a late fusion network. We focus on learning with imperfect sensor data, a typical problem in real-world robotics tasks. For accurate learning, we introduce a multi-stage training methodology and two crucial ingredients for handling depth data with CNNs. The first, an effective encoding of depth information for CNNs that enables learning without the need for large depth datasets. The second, a data augmentation scheme for robust learning with depth images by corrupting them with realistic noise patterns. We present state-of-the-art results on the RGB-D object dataset [15] and show recognition in challenging RGB-D re

Chat with Paper

AI Agents for this Paper

Robust object recognition is a crucial ingredient of many, if not all, real-world robotics applications. This paper leverages recent progress on Convolutional Neural Networks (CNNs) and proposes a novel RGB-D architecture for object recognition. Our architecture is composed of two separate CNN processing streams - one for each modality - which are consecutively combined with a late fusion network. We focus on learning with imperfect sensor data, a typical problem in real-world robotics tasks. For accurate learning, we introduce a multi-stage training methodology and two crucial ingredients for handling depth data with CNNs. The first, an effective encoding of depth information for CNNs that enables learning without the need for large depth datasets. The second, a data augmentation scheme for robust learning with depth images by corrupting them with realistic noise patterns. We present state-of-the-art results on the RGB-D object dataset [15] and show recognition in challenging RGB-D re

Keywords

Artificial intelligenceComputer scienceConvolutional neural networkDeep learningRoboticsRGB color modelCognitive neuroscience of visual object recognitionComputer vision

Chat

Click to start Chat