General-purpose robotic manipulation requires robots to perform diverse tasks in open-world environments while improving their skills over time. Despite recent progress, existing systems acquire manipulation skills in a static manner, where capabilities are learned for specific tasks rather than adaptively evolving through physical interaction.
Resembling how repeated practice enables humans to develop muscle memory, we propose HERO, a self-improving hierarchical embodied agent that enables autonomous capability evolution from zero human demonstrations. HERO organizes Heuristic reasoning, Exemplar reuse, and Reflexive execution into a unified Orchestration framework, allowing robots to autonomously bootstrap manipulation experience, rapidly accumulate reusable behaviors, and progressively consolidate recurring interactions into efficient closed-loop visuomotor policies.
Extensive experiments across four real-world tasks demonstrate that HERO substantially reduces human intervention during data collection while achieving robust manipulation, providing a promising path toward self-improving robotic systems.
HERO autonomously collects manipulation data across all four tasks.
From heuristic reasoning (L1), through exemplar reuse (L2), to reflexive visuomotor control (L3).
The distilled L3 policy performs reactive closed-loop control.
A zero-shot module that executes tasks without any human demonstrations. A VLM decomposes instructions into four primitive skills — Grasp, Place, Push, Pull — grounded in a 3D workspace reconstructed via Depth-Anything-3 from multi-view RGB.
A one-shot module that accelerates execution by retrieving and warping similar past trajectories via rigid transformation, avoiding repeated VLM calls.
An efficient closed-loop visuomotor policy distilled from accumulated experiences. Trained on autonomously collected data at 15 Hz, with automatic fallback to L2/L1 for robust deployment.
Pick colored packages into a basket (reverse: return to table).
Stack blocks by color sequence (reverse: unstack to table).
Open drawer, retrieve hidden object (reverse: return & close).
Move cuboids to find hidden item (reverse: re-occlude).
Starting from zero human demonstrations, HERO autonomously collects 664 forward and reverse subtask episodes across all tasks. Successful experiences are curated into L2 exemplars, and frequently recurring behaviors are distilled into L3 visuomotor policies (30 trajectories per task, 10,000 training steps). The orchestrator balances L1 and L2 contributions during data collection and dynamically schedules L3 → L2 → L1 with up to 3 retries during deployment.
@misc{hero2026,
author = {Li, Jialiang and Wang, Yuhan and Li, Haojun and
Zhang, Gaojing and Ye, Yangtian and Liu, Qipeng and
Liang, Haotian and Lian, Wenzhao},
title = {Practice Makes Policies: Bootstrapping and Consolidating
Robotic Capabilities from Zero Human Demonstrations},
year = {2026},
eprint = {2607.26809},
archivePrefix = {arXiv},
}