Publications

2024

  1. telemoma.gif
    TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation
    Shivin Dass, Wensi Ai, Yuqian Jiang, Samik Singh, Jiaheng Hu, Ruohan Zhang, Peter Stone, Ben Abbatematteo, and Roberto Martín-Martín
    arXiv preprint, 2024
  2. droid.gif
    DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
    DROID Dataset Team
    In International Conference on Robotics and Automation (ICRA), 2024
  3. rtx.gif
    Open X-Embodiment: Robotic Learning Datasets and RT-X Models
    Open X-Embodiment Collaboration
    In International Conference on Robotics and Automation (ICRA), 2024
  4. sirius.png
    Model-Based Runtime Monitoring with Interactive Imitation Learning
    Huihan Liu, Shivin Dass, Roberto Martín-Martín, and Yuke Zhu
    In IEEE International Conference on Robotics and Automation (ICRA), 2024

2023

  1. bariflex.gif
    BaRiFlex: A Robotic Gripper with Versatility and Collision Robustness for Robot Learning
    Gu-Cheol Jeong, Arpit Bahety, Gabriel Pedraza, Ashish D. Deshpande, and Roberto Martín-Martín
    arXiv preprint, 2023
  2. mini_behavior.png
    Mini-BEHAVIOR: A Procedurally Generated Benchmark for Long-horizon Decision-Making in Embodied AI
    Emily Jin*, Jiaheng Hu*, Zhuoyi Huang, Ruohan Zhang, Jiajun Wu, Fei-Fei Li, and Roberto Martín-Martín
    arXiv preprint, 2023
  3. elden.png
    ELDEN: Exploration via Local Dependencies
    Zizhao Wang*, Jiaheng Hu*, Peter Stone, and Roberto Martín-Martín
    In Neural Information Processing Systems (NeurIPS), 2023
  4. mutex.gif
    MUTEX: Learning Unified Policies from Multimodal Task Specifications
    Rutav Shah, Roberto Martín-Martín*, and Yuke Zhu*
    In Conference on Robot Learning (CoRL), 2023
  5. principles_social_nav.jpg
    Principles and guidelines for evaluating social robot navigation algorithms
    Anthony Francis, Claudia Pérez-d’Arpino, Chengshu Li, Fei Xia, Alexandre Alahi, Rachid Alami, Aniket Bera, Abhijat Biswas, Joydeep Biswas, Rohan Chandra, and Roberto Martín-Martín
    arXiv preprint, 2023
  6. kurenkov2023modeling.jpg
    Modeling dynamic environments with scene graph memory
    Andrey Kurenkov, Michael Lingelbach, Tanmay Agarwal, Emily Jin, Chengshu Li, Ruohan Zhang, Li Fei-Fei, Jiajun Wu, Silvio Savarese, and Roberto Martín-Martín
    In International Conference on Machine Learning (ICML), 2023
  7. cpg.gif
    Causal Policy Gradient for Whole-Body Mobile Manipulation
    Jiaheng Hu, Peter Stone, and Roberto Martín-Martín
    In Robotics: Science and Systems (RSS), 2023
  8. maskvit.gif
    MaskViT: Masked visual pre-training for video prediction
    Agrim Gupta, Stephen Tian, Yunzhi Zhang, Jiajun Wu, Roberto Martín-Martín, and Li Fei-Fei
    In ICLR, 2023
  9. MAA.png
    Model-Agnostic Hierarchical Attention for 3D Object Detection
    Manli Shu, Le Xue, Ning Yu, Roberto Martín-Martín, Juan Carlos Niebles, Caiming Xiong, and Ran Xu
    arXiv preprint arXiv:2301.02650, 2023

2022

  1. b1k_sm.gif
    BEHAVIOR-1k: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation
    Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín-Martín, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, and  others
    In 6th Annual Conference on Robot Learning, 2022
  2. retrospective.png
    Retrospectives on the embodied ai workshop
    Matt Deitke, Dhruv Batra, Yonatan Bisk, Tommaso Campari, Angel X Chang, Devendra Singh Chaplot, Changan Chen, Claudia Pérez D’Arpino, Kiana Ehsani, Ali Farhadi, and  others
    arXiv preprint arXiv:2210.06849, 2022
  3. ULIP: Learning Unified Representation of Language, Image and Point Cloud for 3D Understanding
    Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese
    arXiv preprint arXiv:2212.05171, 2022
  4. Eye-BEHAVIOR: An Eye-Tracking Dataset for Everyday Household Activities in Virtual, Interactive, and Ecological Environments
    Cem Gokmen, Ruohan Zhang, Sanjana Srivastava, Chengshu Li, Michael Lingelbach, Roberto Martín-Martín, Silvio Savarese, Jiajun Wu, and Li Fei-Fei
    Journal of Vision, 2022
  5. Generating Procedural 3D materials from Images using Neural Networks
    Lyne P Tchapmi, Trishiet Ray, Micael Tchapmi, Bokui Shen, Roberto Martín-Martín, and Silvio Savarese
    In 2022 4th International Conference on Image, Video and Signal Processing, Jan 2022
  6. BEHAVIOR in Habitat 2.0: Simulator-Independent Logical Task Description for Benchmarking Embodied AI Agents
    Ziang Liu, Roberto Martín-Martín, Fei Xia, Jiajun Wu, and Li Fei-Fei
    arXiv preprint arXiv:2206.06489, Jan 2022

2021

  1. ReLMoGen: Leveraging motion generation in reinforcement learning for mobile manipulation
    Fei Xia, Chengshu Li, Roberto Martín-Martín, Or Litany, Alexander Toshev, and Silvio Savarese
    In 2021 International Conference on Robotics and Automation (ICRA), Jan 2021
  2. Robot navigation in constrained pedestrian environments using reinforcement learning
    Claudia Pérez-D’Arpino, Can Liu, Patrick Goebel, Roberto Martín-Martín, and Silvio Savarese
    In 2021 IEEE International Conference on Robotics and Automation (ICRA), Jan 2021
  3. iGibson 1.0, a Simulation Environment for Interactive Tasks in Large Realistic Scenes
    Bokui Shen, Fei Xia, Chengshu Li, Roberto Martín-Martín, Linxi Fan, Guanzhi Wang, Shyamal Buch, Claudia D’Arpino, Sanjana Srivastava, Lyne P Tchapmi, and  others
    In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Jan 2021
  4. Deep affordance foresight: Planning through what can be done in the future
    Danfei Xu, Ajay Mandlekar, Roberto Martín-Martín, Yuke Zhu, Silvio Savarese, and Li Fei-Fei
    In 2021 IEEE International Conference on Robotics and Automation (ICRA), Jan 2021
  5. Semantic and geometric modeling with neural message passing in 3d scene graphs for hierarchical mechanical search
    Andrey Kurenkov, Roberto Martín-Martín, Jeff Ichnowski, Ken Goldberg, and Silvio Savarese
    In 2021 IEEE International Conference on Robotics and Automation (ICRA), Jan 2021
  6. Learning multi-arm manipulation through collaborative teleoperation
    Albert Tung, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martín, Yuke Zhu, Li Fei-Fei, and Silvio Savarese
    In 2021 IEEE International Conference on Robotics and Automation (ICRA), Jan 2021
  7. Greedy hierarchical variational autoencoders for large-scale video prediction
    Bohan Wu, Suraj Nair, Roberto Martín-Martín, Li Fei-Fei, and Chelsea Finn
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jan 2021
  8. Laser: Learning a latent action space for efficient reinforcement learning
    Arthur Allshire, Roberto Martín-Martín, Charles Lin, Shawn Manuel, Silvio Savarese, and Animesh Garg
    In 2021 IEEE International Conference on Robotics and Automation (ICRA), Jan 2021
  9. Jrdb: A dataset and benchmark of egocentric robot visual perception of humans in built environments
    Roberto Martín-Martín, Mihir Patel, Hamid Rezatofighi, Abhijeet Shenoi, JunYoung Gwak, Eric Frankel, Amir Sadeghian, and Silvio Savarese
    IEEE transactions on pattern analysis and machine intelligence, Jan 2021
  10. Differentiable factor graph optimization for learning smoothers
    Brent Yi, Michelle A Lee, Alina Kloss, Roberto Martín-Martín, and Jeannette Bohg
    In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Jan 2021
  11. BEHAVIOR: Benchmark for everyday household activities in virtual, interactive, and ecological environments
    Sanjana Srivastava, Chengshu Li, Michael Lingelbach, Roberto Martín-Martín, Fei Xia, Kent Elliott Vainio, Zheng Lian, Cem Gokmen, Shyamal Buch, Karen Liu, and  others
    In 5th Annual Conference on Robot Learning, Jan 2021
  12. What matters in learning from offline human demonstrations for robot manipulation
    Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martín-Martín
    Jan 2021
  13. iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks
    Chengshu Li, Fei Xia, Roberto Martín-Martín, Michael Lingelbach, Sanjana Srivastava, Bokui Shen, Kent Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, and  others
    In 5th Annual Conference on Robot Learning, Jan 2021
  14. Error-aware imitation learning from teleoperation data for mobile manipulation
    Josiah Wong, Albert Tung, Andrey Kurenkov, Ajay Mandlekar, Li Fei-Fei, Silvio Savarese, and Roberto Martín-Martín
    In 5th Annual Conference on Robot Learning, Jan 2021

2020

  1. 6-pack: Category-level 6d pose tracker with anchor-based keypoints
    Chen Wang, Roberto Martín-Martín, Danfei Xu, Jun Lv, Cewu Lu, Li Fei-Fei, Silvio Savarese, and Yuke Zhu
    In 2020 IEEE International Conference on Robotics and Automation (ICRA), Jan 2020
  2. Hrl4in: Hierarchical reinforcement learning for interactive navigation with mobile manipulators
    Chengshu Li, Fei Xia, Roberto Martín-Martín, and Silvio Savarese
    In Conference on Robot Learning, Jan 2020
  3. Leveraging pretrained image classifiers for language-based segmentation
    David Golub, Roberto Martín-Martín, Ahmed El-Kishky, and Silvio Savarese
    In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Jan 2020
  4. Interactive gibson benchmark: A benchmark for interactive navigation in cluttered environments
    Fei Xia, William B Shen, Chengshu Li, Priya Kasimbeg, Micael Edmond Tchapmi, Alexander Toshev, Roberto Martín-Martín, and Silvio Savarese
    IEEE Robotics and Automation Letters, Jan 2020
  5. JRMOT: A real-time 3d multi-object tracker and a new large-scale dataset
    Abhijeet Shenoi, Mihir Patel, JunYoung Gwak, Patrick Goebel, Amir Sadeghian, Hamid Rezatofighi, Roberto Martín-Martín, and Silvio Savarese
    In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Jan 2020
  6. Learning to generalize across long-horizon tasks from human demonstrations
    Ajay Mandlekar, Danfei Xu, Roberto Martín-Martín, Silvio Savarese, and Li Fei-Fei
    In Robotics: Science and Systems (RSS), Jan 2020
  7. Probabilistic Visual Navigation with Bidirectional Image Prediction
    Noriaki Hirose, Shun Taguchi, Fei Xia, Roberto Martín-Martín, Yasumasa Tahara, Masanori Ishigaki, and Silvio Savarese
    arXiv preprint arXiv:2003.09224, Jan 2020
  8. Visuomotor mechanical search: Learning to retrieve target objects in clutter
    Andrey Kurenkov, Joseph Taglic, Rohun Kulkarni, Marcus Dominguez-Kuhne, Animesh Garg, Roberto Martín-Martín, and Silvio Savarese
    In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Jan 2020
  9. robosuite: A modular simulation framework and benchmark for robot learning
    Yuke Zhu, Josiah Wong, Ajay Mandlekar, and Roberto Martín-Martín
    arXiv preprint arXiv:2009.12293, Jan 2020
  10. Multimodal Sensor Fusion with Differentiable Filters
    Michelle A Lee, Brent Yi, Roberto Martín-Martín, Silvio Savarese, and Jeannette Bohg
    In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Jan 2020
  11. Human-in-the-loop imitation learning using remote teleoperation
    Ajay Mandlekar, Danfei Xu, Roberto Martín-Martín, Yuke Zhu, Li Fei-Fei, and Silvio Savarese
    arXiv preprint arXiv:2012.06733, Jan 2020

2019

  1. The RBO dataset of articulated objects and interactions
    Roberto Martín-Martín, Clemens Eppner, and Oliver Brock
    The International Journal of Robotics Research, Jan 2019
  2. Densefusion: 6d object pose estimation by iterative dense fusion
    Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, and Silvio Savarese
    In IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jan 2019
  3. Mechanical search: Multi-step retrieval of a target object occluded by clutter
    Michael Danielczuk, Andrey Kurenkov, Ashwin Balakrishna, Matthew Matl, David Wang, Roberto Martín-Martín, Animesh Garg, Silvio Savarese, and Ken Goldberg
    In 2019 International Conference on Robotics and Automation (ICRA), Jan 2019
  4. Deep visual mpc-policy learning for navigation
    Noriaki Hirose, Fei Xia, Roberto Martín-Martín, Amir Sadeghian, and Silvio Savarese
    IEEE Robotics and Automation Letters, Jan 2019
  5. Deep local trajectory replanning and control for robot navigation
    Ashwini Pokle, Roberto Martín-Martín, Patrick Goebel, Vincent Chow, Hans M Ewald, Junwei Yang, Zhenkai Wang, Amir Sadeghian, Dorsa Sadigh, Silvio Savarese, and  others
    In 2019 international conference on robotics and automation (ICRA), Jan 2019
  6. Coupled recursive estimation for online interactive perception of articulated objects
    Roberto Martín-Martín, and Oliver Brock
    The International Journal of Robotics Research, Jan 2019
  7. Variable impedance control in end-effector space: An action space for reinforcement learning in contact-rich tasks
    Roberto Martín-Martín, Michelle A Lee, Rachel Gardner, Silvio Savarese, Jeannette Bohg, and Animesh Garg
    In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Jan 2019
  8. DANTE: deep affinity network for clustering conversational interactants
    Mason Swofford, John Charles Peruzzi, Marynel Vázquez, Roberto Martín-Martín, and Silvio Savarese
    In ACM on Human-Computer Interaction (CSCW), Jan 2019
  9. Ac-teach: A bayesian actor-critic method for policy learning with an ensemble of suboptimal teachers
    Andrey Kurenkov, Ajay Mandlekar, Roberto Martín-Martín, Silvio Savarese, and Animesh Garg
    In Conference on Robot Learning, Jan 2019
  10. Social-bigat: Multimodal trajectory forecasting using bicycle-gan and graph attention networks
    Vineet Kosaraju, Amir Sadeghian, Roberto Martín-Martín, Ian Reid, S Hamid Rezatofighi, and Silvio Savarese
    In Advances in Neural Information Processing Systems 2019 (NeurIPS), Jan 2019
  11. Regression planning networks
    Danfei Xu, Roberto Martín-Martín, De-An Huang, Yuke Zhu, Silvio Savarese, and Li Fei-Fei
    In Advances in Neural Information Processing Systems 2019 (NeurIPS), Jan 2019
  12. Vunet: Dynamic scene view synthesis for traversability estimation using an rgb camera
    Noriaki Hirose, Amir Sadeghian, Fei Xia, Roberto Martín-Martín, and Silvio Savarese
    IEEE Robotics and Automation Letters, Jan 2019
  13. Coupled Recursive Estimation for Online Interactive Perception of Articulated Objects–Online IP Data
    Roberto Martín-Martín
    Jan 2019

2018

  1. Physics-based selection of informative actions for interactive perception
    Clemens Eppner, Roberto Martín-Martín, and Oliver Brock
    In 2018 IEEE International Conference on Robotics and Automation (ICRA), Jan 2018
  2. Four aspects of building robotic systems: lessons from the Amazon Picking Challenge 2015
    Clemens Eppner, Sebastian Höfer, Rico Jonschkowski, Roberto Martín-Martín, Arne Sieverling, Vincent Wall, and Oliver Brock
    Autonomous Robots, Jan 2018
  3. Leveraging problem structure in interactive perception for robot manipulation of constrained mechanisms
    Roberto Martín-Martín
    Jan 2018

2017

  1. Building Kinematic and Dynamic Models of Articulated Objects with Multi-Modal Interactive Perception
    Roberto Martín-Martín, and Oliver Brock
    In AAAI Spring Symposium Series - Interactive Multi-Sensory Object Perception for Embodied Agents, Jan 2017
  2. Cross-modal interpretation of multi-modal sensor streams in interactive perception based on coupled recursion
    Roberto Martín-Martín, and Oliver Brock
    In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Jan 2017
  3. Opening a lockbox through physical exploration
    Manuel Baum, Matthew Bernstein, Roberto Martín-Martín, Sebastian Höfer, Johannes Kulick, Marc Toussaint, Alex Kacelnik, and Oliver Brock
    In 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), Jan 2017

2016

  1. An Integrated Approach to Visual Perception of Articulated Objects
    Roberto Martín-Martín, Sebastian Höfer, and Oliver Brock
    In International Conference on Robotics and Automation (ICRA), Jan 2016
  2. Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems
    Clemens Eppner, Sebastian Höfer, Rico Jonschkowski, Roberto Martín-Martín, Arne Sieverling, Vincent Wall, and Oliver Brock
    In Robotics, Science and Systems (RSS), Jan 2016
  3. Probabilistic Multi-Class Segmentation for the Amazon Picking Challenge
    Rico Jonschkowski, Clemens Eppner, Sebastian Höfer, Roberto Martín-Martín, and Oliver Brock
    In International Conference on Intelligent Robots and Systems (IROS), Jan 2016
  4. Estimating the relation of perception and action during interaction
    Roberto Martín-Martín, Arne Sieverling, and Oliver Brock
    In International Workshop on Robotics in the 21st century: Challenges and Promises, Jan 2016
  5. Probabilistic Multi-Class Segmentation for the Amazon Picking Challenge
    Clemens Eppner, Rico Jonschkowski, Sebastian Höfer, Roberto Martín-Martín, and Oliver Brock
    Jan 2016

2015

    2014

    1. Online interactive perception of articulated objects with multi-level recursive estimation based on task-specific priors
      Roberto Martín-Martín, and Oliver Brock
      In IEEE/RSJ International Conference on Intelligent Robots and Systems, Jan 2014
    2. Deterioration of depth measurements due to interference of multiple RGB-D sensors
      Roberto Martín-Martín, Malte Lorbach, and Oliver Brock
      In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Jan 2014