Learning to Act under Visual Interruptions with Vision-Language-Action Models

Mingle Jiang, Rui Xu, Yunke Wang, Chang Xu

School of Computer Science, The University of Sydney

Head camera interrupted, real time. Left: the real head camera, hidden from the policy after 6.2 s. Middle: the head view as the policy receives it: real frames until the interruption, then the world-model prediction MINT supplies at each policy query (blue), and no frame where none is accepted. Right: the wrist camera, which stays available. GR00T N1.5 + MINT completes the task: cube into the drawer, drawer closed.

MAIL-Bench. (a) Four availability conditions on RoboCasa365; an interrupted camera delivers no frame and its availability flag is 0. (b) Scenes solved out of 900 with every camera present (healthy) and after one visual role is lost at 30, 45 or 60% of each policy's own completion time. Camera loss removes a large share of both base policies' successes, and MINT recovers part of that loss on every role.

Abstract

Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, but they are typically developed and evaluated with all camera streams available throughout task execution. When a camera stops delivering frames during task execution, the policy must continue acting without access to subsequent observations from the missing view. Despite its practical importance, how such interruptions affect closed-loop manipulation remains insufficiently understood. To investigate this problem, we introduce MAIL-Bench, a benchmark that evaluates visual interruptions with VLA models. By interrupting different cameras at multiple stages of each policy's successful reference trajectory, MAIL-Bench measures how well policies retain their capabilities when visual inputs become unavailable. Building on this benchmark, we propose MINT, which trains VLA policies to handle missing visual inputs and temporarily supplies predicted images when cameras fail. It uses optical-flow extrapolation for missing wrist views or all-vision loss, and an action-conditioned world model for missing third-person views, withdrawing the predictions when they become unreliable. Experiments on π_0.5 and GR00T N1.5 show that MINT improves task success under camera loss over the original models. Experiments on the AgiBot G2 further demonstrate real-robot closed-loop deployment under camera loss.

MAIL-Bench: manipulation under visual interruption

A paired, closed-loop benchmark for evaluating VLA models with missing camera inputs, built on RoboCasa365. Each policy is measured against its own successful rollouts, so the score isolates what camera loss takes away.

Benchmark construction: a healthy rollout defines the reference completion time, then the same scene is replayed with the selected visual role interrupted at 30, 45 or 60 percent of that time and kept unavailable — A successful healthy rollout defines the reference completion time T_healthy. The same scene is then replayed with the selected visual role interrupted at 30%, 45% or 60% of T_healthy and kept unavailable thereafter, while unaffected cameras continue streaming. Shown is an AgiBot G2 drawer example.

Protocol

Environment. The 18 RoboCasa365 Atomic-Seen tasks with 50 fixed scenes each: 900 evaluation scenes.
Visual roles, not devices. The wrist camera forms the wrist group; the base-mounted cameras form the third-person group. Conditions: healthy, wrist missing, third-person missing, all vision missing.
Hard missing. Once interrupted, a camera group provides no further frames and its availability flag is set to 0. No black frames, no frozen images.
Onset relative to the policy. Interruptions start at 30%, 45% and 60% of each policy's own healthy completion time, so fast and slow policies are compared at the same task stage.
Paired and deterministic. Every interruption rollout restarts from the healthy rollout's saved initial state and policy seed; the pre-interruption trajectory is checked to match the reference.

What it measures

For task j, H_j is the healthy success rate over its 50 scenes and M_c,j the fraction of those healthy successes that survive interruption condition c. The task score averages H_j with the nine conditional scores, and the MAIL-Bench score averages the 18 task scores. A policy is therefore rewarded both for solving tasks and for keeping what it can already do when a camera disappears.

How it differs from robustness benchmarks

Benchmark	Visual failure	Onset	Post-loss control
COLOSSEUM	Perturbation	–	–
RoboTwin 2.0	Perturbation	–	–
VLATest	Perturbation	–	–
LIBERO-Plus	Black frame	Start	Closed loop
Missing-modality IL	Camera dropout	Start	Open loop
MAIL-Bench (ours)	Camera loss	Mid-episode	Closed loop

MINT: acting under visual interruption

MINT separates robustness to missing visual inputs from visual prediction. The policy is first trained to act under variable camera availability, then selectively receives predicted views at inference time when a camera is interrupted.

MINT overview: availability-aware training, then at inference optical-flow extrapolation for missing wrist views or all-vision loss and an action-conditioned world model for missing third-person views, with reliability-gated fallback — **MINT overview.** Unreliable predictions are withdrawn, and the policy falls back to acting from the observations that remain.

TrainLearning with missing visual inputs

During fine-tuning, camera groups are dropped as hard missing: their tokens are excluded and the availability flag is set to 0. The policy learns to act from whatever views remain, and healthy performance does not degrade; on GR00T N1.5 it improves on two of the three task categories.

FlowWorldPredicting missing visual inputs

When the wrist view or all vision is lost, short-horizon optical-flow extrapolation of the last frames fills the gap on the CPU. When the third-person view is lost, an action-conditioned world model generates it from the surviving wrist stream and the executed actions.

GateReliability-gated fallback

Each prediction is checked against the views that remain; once a route becomes unreliable, its images are withdrawn and the policy returns to acting under hard missing, which the training stage prepared it for. Predictions are supplied temporarily, never trusted indefinitely.

Cost

MINT does not increase the per-step wall-clock time of healthy execution. Under interruption the optical-flow route runs on the CPU, and the world model adds 9.2 GiB of GPU memory and about 100 TFLOPs per generation, invoked only while a third-person view is missing.

Results on MAIL-Bench

Healthy is the success rate with all cameras available. W, T and B report the scores under wrist, third-person and all-vision interruption at 30 / 45 / 60% of completion time. AVG is the task-category MAIL-Bench score and Δ its change over the corresponding base checkpoint. Bold marks the best result within each backbone and task category.

Method	Tasks	Healthy	Wrist missing			Third-person missing			All vision missing			AVG	Δ
Method	Tasks	Healthy	W30	W45	W60	T30	T45	T60	B30	B45	B60	AVG	Δ
GR00T N1.5
base model	pick-and-place	0.732	0.163	0.276	0.341	0.165	0.224	0.424	0.000	0.000	0.010	0.234	–
	small appliances	0.290	0.153	0.277	0.398	0.245	0.296	0.403	0.110	0.170	0.226	0.257	–
	fixtures & navigation	0.386	0.178	0.253	0.385	0.308	0.319	0.398	0.029	0.084	0.119	0.246	–
+ MINT	pick-and-place	0.724	0.423	0.561	0.673	0.622	0.674	0.647	0.053	0.091	0.166	0.463	+0.229
	small appliances	0.450	0.493	0.536	0.631	0.602	0.628	0.573	0.307	0.419	0.528	0.517	+0.260
	fixtures & navigation	0.497	0.450	0.507	0.649	0.335	0.488	0.566	0.076	0.197	0.378	0.414	+0.168
π_0.5
base model	pick-and-place	0.636	0.059	0.098	0.200	0.239	0.245	0.375	0.000	0.014	0.089	0.196	–
	small appliances	0.257	0.270	0.264	0.342	0.291	0.343	0.413	0.065	0.095	0.156	0.250	–
	fixtures & navigation	0.360	0.151	0.266	0.355	0.453	0.469	0.602	0.043	0.129	0.254	0.308	–
+ MINT	pick-and-place	0.688	0.027	0.111	0.342	0.574	0.735	0.713	0.025	0.090	0.196	0.350	+0.154
	small appliances	0.333	0.291	0.426	0.350	0.513	0.372	0.480	0.218	0.337	0.370	0.369	+0.119
	fixtures & navigation	0.414	0.131	0.261	0.528	0.428	0.498	0.546	0.093	0.128	0.278	0.331	+0.023

Per-task, per-condition results and bootstrap intervals are in the paper's appendix.

What the numbers say

Availability-aware training alone lifts healthy success on GR00T N1.5 (small appliances 0.290 → 0.450): learning under partial observation does not trade off normal execution.
Under interruption MINT outperforms both base models in almost all of the nine conditions, across wrist, third-person and all-vision loss and across early and late onsets.
Third-person loss benefits most from prediction, because the global view can still be inferred from the wrist camera; all-vision loss is the hardest condition for every policy.

Ablation on GR00T N1.5: MAIL-Bench scores at each onset for the base model, missing-view training only, prediction only, and full MINT — **Ablation on GR00T N1.5.** Scores averaged over the three task categories at each onset; H marks the healthy-input score of each variant. Missing-view training governs how far the policy falls when a view is lost early; prediction determines how much of a lost view can be recovered while another view remains (+0.105 at T30).

Real robot: AgiBot G2

The same observation interface transfers to a physical robot without changing the VLA architecture. An availability-aware GR00T N1.5 policy is fine-tuned on 203 successful teleoperated episodes of four tabletop tasks; the G2 head camera provides the third-person view and the right wrist camera the wrist view.

Success rate of GR00T N1.5 with and without MINT on the AgiBot G2 under healthy execution and under sustained third-person, wrist and all-camera interruption, 10 trials per task and condition — **Real robot experiment on the AgiBot G2.** Success rate of GR00T N1.5 and MINT over 10 trials per task and condition; numbers above the bars are successful trials.

A cube-to-drawer episode on the AgiBot G2 with the third-person camera interrupted at step 50: real third-person stream hidden from the policy, the world-model prediction that replaces it, and the wrist stream that stays available — Episode on the cube-to-drawer task with the third-person camera interrupted at step 50. **Top:** real third-person stream, hidden from the policy after interruption. **Middle:** the third-person stream and its world-model prediction. **Bottom:** the wrist stream, which stays available. The policy completes grasping, transport, release and drawer closure.

The four tasks

Healthy episodes; head (third-person) camera on top and right wrist camera below, at five time points.

Healthy cube-into-bowl episode — (b) Cube into bowl

Healthy cup-onto-plate episode — (c) Cup onto plate

Healthy stack-cups episode — (d) Stack cups

BibTeX

@article{jiang2026mint,
  title   = {Learning to Act under Visual Interruptions with Vision-Language-Action Models},
  author  = {Jiang, Mingle and Xu, Rui and Wang, Yunke and Xu, Chang},
  journal = {arXiv preprint},
  year    = {2026}
}