OopsieRecover

Learning transferable VLA corrections from semantic interventions

Arpit Bahety1,* Skand Peri1,* Sriniket Ambatipudi1 Chen Xu2 Haruki Nishimura2 Masha Itkina2 Roberto Martín-Martín1

1The University of Texas at Austin 2Toyota Research Institute *Equal contribution

π0.5 acting on its own

Real run at 1×, paused once at the first correction. The status follows the steering labels logged during this run; the arcs are drawn to illustrate the choice.

Method

How OopsieRecover works

OopsieRecover on a real-robot example

Interveneπ0.5 (cyan), then the human (orange)
Ground and tracklid, body, holder, pod
Steer, then correct again“Open gripper once pod is over the filter holder”
After 5 roundssteering only where needed

Improving round by round

Load Coffee Pod · one correction per roundround 1 of 3
Correction library
Illustration of the collection loop. In each round π0.5 runs steered by every program learned so far (purple), fails somewhere new (✕), and the human fixes that moment and says why (orange). The three sentences are the coffee-pod correction types shown in the experiment videos.

On CoffeeMug in simulation, OopsieRecover gets one annotated correction per round and HG-DAgger* gets ten. On 25 held-out scenes with new poses, OopsieRecover reaches 84% after 8 corrections; HG-DAgger* is at 28% after 30.

Success rate on CoffeeMug, L1 test scenes.

Simulation

Corrections that hold in new poses and new kitchens

Three RoboCasa tasks, π0.5 fine-tuned on the full RoboCasa dataset, 5–8 corrections per task. L0 repeats the state where corrections were collected, L1 moves the objects, L2 moves them into unseen scenes. Only OopsieRecover keeps working past L0.

CoffeeMugSetup in RoboCasa

Put the mug under the coffee machineCoffeeMugSetup

BlenderLid in RoboCasa

Put the lid on the blenderBlenderLid

StartKettle in RoboCasa

Switch on the kettleStartKettle

  • π0.5
  • HG-DAgger*
  • OLAF
  • VLS
  • OopsieRecover

L0 same state

L1 new poses

L2 new poses and scenes

Zero-shot success, pooled over three tasks, 50 episodes per method. Letters mark groups that are statistically separate (STEP, 95%); a shared letter means no significant difference.

One correction, ten new scenes

From a single episode of avoid colliding with the steam handle, the robot was driven at the handle in 10 unseen scenes. The detector fired every time, and steering prevented the collision over 100 steps.

Collecting the correction
Detection and steering in a new scene

Real robot

Steering a frozen π0.5 on a real robot

DROID hardware with the official π0.5 DROID checkpoint, three to six corrections per task. Pooled over three tasks, OopsieRecover raises task progress from 35% to 61%. HG-DAgger* fine-tunes on the same interventions and does worse than the base policy.

  • π0.5
  • HG-DAgger*
  • OopsieRecover

L0 same state

L1 new poses

L2 new poses and scene

Mean task progress (fraction of stages completed), 20 rollouts per method, task and level. Letters from the NSCORE sequential test (95%).

Put Croffle: put the croissant on the waffle maker

π0.5 reaches for the purple cup instead. HG-DAgger* picks up the croissant and drops it beside the waffle maker.

Corrections active in this runMove towards the croissantDon’t open gripper if croissant is not on top of toasterLet go of croissant when near toaster base
Evaluated at

Shelve Item: put the mug on the shelf

The shelf also holds a wine glass and a bottle. Both baselines fail; the correction keeps the arm away from the fragile objects while placing.

Correction active in this runMove away from fragile objects
Evaluated at

Load Coffee Pod: drop the pod in the filter holder and close the lid

π0.5 never secures the pod. HG-DAgger* lifts it but does not complete the placement.

Corrections active in this runAvoid colliding the pod with the coffee machine lid and bodyOpen the gripper once the pod is over the filter holder
Evaluated at

Citation

@misc{bahety2026oopsierecover,
  title  = {OopsieRecover: Learning Transferable VLA Corrections from Semantic Interventions},
  author = {Bahety, Arpit and Peri, Skand and Ambatipudi, Sriniket and Xu, Chen and Nishimura, Haruki and Itkina, Masha and Mart{\'i}n-Mart{\'i}n, Roberto},
  year   = {2026}
}