RobIn Robot Interactive Intelligence Lab
2317 Speedway
Austin, Texas 78712
RobIn Lab is directed by Roberto Martín-Martín, and is part of the University of Texas at Austin, Department of Computer Science, UTAI Lab and Texas Robotics.
The research in RobIn explores at the intersection of robotics, computer vision and machine learning, with an emphasis on creating robotic solutions that accomplish interaction-rich tasks in the real world. Our research revolves around the question: what are the mechanisms that enable physically intelligent behavior in embodied agents? Our hypothesis is that intelligence is not a purely computational process, but an interplay between perceiving, reasoning, learning and interacting physically with the world.
Keywords in our research include: learning (reinforcement learning, imitation learning, representation learning), planning (motion planning, task planning, TAMP), control (optimal control, force control), perception (active, interactive, multimodal perception), 2D and 3D reasoning, and Bayesian state estimation, sim2real. Application domains we find exciting involve: mobile manipulation, stationary manipulation, contact-rich manipulation, social and visual navigation.
We are always looking for outstanding new members and collaborators for RobIn. Please, fill in this form and state clearly if you are an UG/MSc/PhD from UT, or a prospective visitor.
news
| Nov 4, 2024 | Roberto is giving a talk at the CoRL24 workshop on Language and Robot Learning. |
|---|---|
| Oct 31, 2024 | Roberto is participating in the Cornell Robotics Seminar. |
| Oct 23, 2024 | Roberto is talking about RobIn’s new line of research on “Zero-shot Ever Improving Robots” at the Amazon Hub Conference. |
| Oct 19, 2024 | Roberto is visiting Brown University for the BigAI seminar series. He will be talking about “What Foundation Models Can and Cannot Do for Bringing Helpful Robotic Assistants into Our Lives”. |
| Oct 14, 2024 | Roberto is giving a talk at the IROS24 workshop on Environment Dynamics Matters: Embodied Navigation to Movable Objects on “Interactive Navigation and Interactive Search: Increasing robots’ capabilities in dynamic, human-centered environments”. |
| Sep 15, 2024 | RobIn as been awarded participation on the DARPA TIAMAT program to explore sim2real alternatives based on foundation models and abstractions. Thanks, DARPA! |
| Sep 7, 2024 | RobIn receives a Catalyst award from UT to study and develop manipulation and tool-usage strategies comparing infants and robots in collaboration with Prof. Jeffrey Lockman. Exciting connections between developmental psychology and robot learning! Thank you UT! |
| Aug 7, 2024 | Our new survey paper, Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes, is released on arxiv! The paper was written in collaboration with Peter Stone’s group and will appear in the 2025 Annual Review of Control, Robotics, and Autonomous Systems. |
| Jul 19, 2024 | ScrewMimic received the Outstanding Student Paper Finalist Award at RSS 2024! |
| Jun 20, 2024 | Two works accepted at RSS 2024: ScrewMimic learns bimanual manipulation behaviors from a single human video and Lang4Sim2Real helps bridge the sim2real gap with language. |
| Jun 10, 2024 | Beginning of the UT Summer Academy offered by the RobIn lab. 25 students will fabricate their own copies of BaRiFlex (design by Gu-Cheol), collect data and train imitation learning policies for manipulation. It is going to be fun! |
| May 17, 2024 | Roberto gives a talk at the ICRA24 workshop Vision-Language Models for Navigation and Manipulation (VLMNM) on “Language as Bridge between Simulation and the Real World” (Albert’s RSS paper). During his talk, he connected live with Austin to teleoperate remotely our Tiago robot using TeleMoMa (Shivin’s paper and code), our teleoperation framework for mobile manipulators. Japan-US connection! |
| Mar 18, 2024 | Roberto was selected as a part of the New Generation Star Project at ICRA sponsored by Nokov! |
| Oct 28, 2023 | Our work miniBEHAVIOR (paper) has been accepted for presentation at two NeurIPS workshops: Agent Learning in Open-Endedness (ALOE) and Generalization in Planning (GenPlan). MiniBEHAVIOR is a novel benchmark that extends the 2D gridworld miniGrid to include semantically meaningful tasks from BEHAVIOR-1K, enabling fast development and prototyping for reasoning in household activities. Congratulations Jiaheng and the rest of the team! |
| Oct 20, 2023 | Roberto gives a keynote speaker talk at the Texas Robotics Symposium on the topic “Technologies for Household Robotics”. |
| Oct 18, 2023 | Roberto participates in an expert panel discussion at the Texas Robotics Symposium on the topic “Household Robotics” together with Dr. Biswas, Dr. Stone and Dr. Fineberg. |
| Oct 1, 2023 | Invited keynote talk by Roberto at the IROS workshop on Integrated Perception, Planning, and Control for Physically and Contextually-Aware Robot Autonomy. His talk was titled “Perceiving to Interact, Interacting to Perceive”. |
| Oct 1, 2023 | First event of Queer in Robotics, a new initiative launched by Roberto and Ruchira with partners from others universities. Queer in Robotics will be organizing events at the upcoming major robotics conferences with the goal of creating a welcoming space to the LGBTQ+ community within the robotics space. |
| Sep 6, 2023 | Invited Lecture by Roberto in the new UT Austin MOOC course Essentials of AI for Life and Society. Roberto introduced the topic “Probabilistic Modeling” to a live online audience of hundreds of participants. |
| Sep 1, 2023 | Dr. Ben Abbatematteo joins the RobIn Lab as a postdoctoral fellow and Assistant Professor of Practice leading the Robot Learning FRI stream. |
| Jul 10, 2023 | Roberto presents RobIn’s work at the RSS’23 workshop Robotics and AI: The Future of Industrial Assembly Tasks. His presentation touches on our research on “Learning Contact-rich Interaction Control”. |
| Jun 6, 2023 | RobIn was awarded with one of the Good Systems grants to explore the effects of AI and robotics in society. Our project will explore the consequences of household and domestic robots in people’s lives, including the life of household workers. |
| Jun 1, 2023 | Two papers presented at ICRA: M-EMBER with Bohan Wu and Fei-Fei Li, and HRON with Michael Lingelbach et al. M-EMBER was nominated to Best Student Paper and Best Manipulation Paper! |
| May 31, 2023 | Andrey Kurenkov presents Scene Graph Memory at AAMAS23 in London, our work for object search in unknown, dynamically changing environments represented as scene graphs. An extension of this work will be also presented at ICML23. |
| Apr 25, 2023 | Roberto presents at Prof. Kristen Grauman’s group on “What can robots learn from videos about long-horizon activities?” comprising the work on Paprika (CVPR23) and MaskViT (ICLR23). |
| Apr 23, 2023 | One paper accepted at RSS23: “Causal Policy Gradient for Whole-Body Mobile Manipulation”, a collaboration with Jiaheng Hu and Peter Stone. Congratulations, Jiaheng! |
| Apr 14, 2023 | Roberto gives a Spotlight talk at TEROS’23 about “Mobile Manipulation: Integration vs. Factorization” including some of the results of our recent RSS paper. |
| Mar 1, 2023 | Final version of our ICLR’23 paper submitted! MaskViT: Masked Visual Pre-Training for Video Prediction, a collaboration with Agrim Gupta et al. |
| Feb 27, 2023 | Two papers accepted at CVPR’23! Procedure-Aware Foundation Model for Instructional Video Understanding with Honglu Zhou and Juan Carlos Niebles and ULIP: Learning Unified Representation of Language, Image and Point Cloud for 3D Understanding with Le Xue et al. |
| Feb 24, 2023 | Roberto presents RobIn at GradFest’23. This presentation summarizes some of the core ideas of our lab and broad future research directions. |
| Feb 8, 2023 | Talk by Roberto at the Texas Innovation Center’s Engineering Education and Research Center. More info |
| Jan 25, 2023 | Talk by Roberto on Datasets in Robotics at the Living and Working with Robots initiative, part of the Good Systems Challenge. |
| Jan 21, 2023 | Two papers accepted at ICRA23: “Task-Driven Graph Attention for Hierarchical Relational Object Navigation”, a collaboration with Michael Lingelbach et al., and “M-EMBER: Tackling Long-Horizon Mobile Manipulation Via Factorized Domain Transfer”, together with Bohan Wu and Dr. Fei-Fei Li. |
| Jan 20, 2023 | MaskVit, Masked Visual Pre-Training for Video Prediction, has been accepted at ICLR23! |
| Jan 5, 2023 | Roberto delivers the William Pierson Field Lecture at Princeton on What do we want robots to do for us (and how do we get there)? |
| Dec 9, 2022 | Talk by Roberto at Nuro on Robots and Humans. |