Deep Learning for Robotics and Augmented Reality

We develop Deep Learning methods for 3D pose estimation from simple images. More precisely, our methods can estimate the 3D poses of rigid objects from color images, of articulated objects from depth maps, or the camera location and orientation from images taken in urban environments. These methods have been applied to Robotics and Augmented Reality applications.

Contact: vincent.lepetit@u-bordeaux.fr

3D Object Pose Estimation from Color Images
The green bounding boxes correspond to the ground truth, the blue ones to our pose estimates.
3D Hand Pose Estimation from Depth Maps
Related Publications

Feature Mapping for Learning Fast and Accurate 3D Pose Inference from Synthetic Images. Mahdi Rad, Markus Oberweger, and Vincent Lepetit. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth. Mahdi Rad and Vincent Lepetit. In Proceedings of the International Conference on Computer Vision (ICCV), 2017.

DeepPrior++: Improving Fast and Accurate 3D Hand Pose EstimationMarkus Oberweger and Vincent Lepetit. In Workshop at the International Conference on Computer Vision (ICCV Workshop)2017.

Learning Lightprobes for Mixed Reality IlluminationDavid Mandl, Kwang Moo Yi, Peter Mohr, Peter Roth, Pascal Fua, Vincent Lepetit, Dieter Schmalstieg, and Denis Kalkofen. In Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR)2017.

ALCN: Adaptive Local Contrast Normalization for Robust Object Detection and 3D Pose EstimationMahdi Rad, Peter Roth, and Vincent Lepetit. In Proceedings of the British Machine Vision Conference (BMVC)2017.

Accurate Camera Registration in Urban Environments using High-Level Feature MatchingAnil Armagan, Martin Hirzer, Peter Roth, and Vincent Lepetit. In Proceedings of the British Machine Vision Conference (BMVC)2017.

Learning to Align Semantic Segmentation and 2.5D Maps for GeolocalizationAnil Armagan, Martin Hirzer, Peter Roth, and Vincent LepetitIn Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)2017.

Robust 3D Object Tracking from Monocular Images using Stable PartsAlberto Crivellaro, Mahdi Rad, Yannick Verdie and Kwang Mu Yi, Pascal Fua, and Vincent LepetitIEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)2017.

Detecting Flying Objects using a Single Moving CameraArtem Rozantsev, Vincent Lepetit, and Pascal FuaIEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)2017.

LIFT: Learned Invariant Feature TransformKwang Moo Yi, Eduard Trulls, Vincent Lepetit, and Pascal FuaIn Proceedings of the European Conference on Computer Vision (ECCV)2016.

Structured Prediction of 3D Human Pose with Deep Neural NetworksBugra Tekin, Isinsu Katircioglu, Mathieu Salzmann, Vincent Lepetit, and Pascal FuaIn Proceedings of the British Machine Vision Conference (BMVC)2016.

Automated Age Estimation from Hand MRI Volumes using Deep LearningDarko Stern, Christian Payer, Vincent Lepetit, and Martin UrschlerIn Proceedings of the Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)2016.

Learning Image Descriptors with BoostingTomasz Trzcinski, C. Mario Christoudias, and Vincent LepetitIEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)2015.

Training a Feedback Loop for Hand Pose EstimationMarkus Oberweger, Paul Wohlhart, and Vincent LepetitIn Proceedings of the International Conference on Computer Vision (ICCV)2015.

Projection onto the Manifold of Elongated Structures for Accurate ExtractionAmos Sironi, Vincent Lepetit, and Pascal FuaIn Proceedings of the International Conference on Computer Vision (ICCV)2015.

On Rendering Synthetic Images for Training an Object DetectorArtem Rozantsev, Vincent Lepetit, and Pascal FuaComputer Vision and Image Understanding (CVIU)2015.

Learning Separable FiltersAmos Sironi, Bugra Tekin, Roberto Rigamonti, Vincent Lepetit, and Pascal FuaIEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI)2015

You Should Use Regression to Detect CellsPhilipp Kainz, Martin Urschler, Samuel Schulter, Paul Wohlhart, and Vincent LepetitIn Proceedings of the Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)2015.

Learning Descriptors for Object Recognition and 3D Pose EstimationPaul Wohlhart and Vincent LepetitIn Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)2015.