Ruoqu Chen

Dexterous Manipulation & Contact-Rich Robot Learning

1. 

   Dex-X: Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction

   Learning visual-tactile dexterous manipulation from human videos through simulated interaction.

   Ruoqu Chen, Feixiang Ruan, Liu Cao, Zihao Wang, Botian Xu, Shiqin Tong, Jiajun Liu, Mingzhi Pei, Chenyu Zhang, Wanli Xing, Kaifeng Zhang, and Mengdi Xu^†

   CoRL 2026

   RSS 2026 Workshop on Dexterous Manipulation (Spotlight)

   Abs arXiv Website

   A framework that learns visual-tactile dexterous manipulation skills from human videos by leveraging simulated interaction, bridging the gap between passive human demonstrations and executable robot policies.

1. 

   Uni-VLaT: Whole-Body Tactile Adaptation of VLA Policies for Humanoid Loco-Manipulation

   Adapting pretrained VLA policies with whole-body touch for contact-rich humanoid tasks.

   Zihao Wang^*, Shutong Liu^*, Siqi Zheng, Liu Cao, Ruoqu Chen, Rundong Liu, Yanchao Yang, and Mengdi Xu^†

   arXiv preprint, 2026

   Abs arXiv Website

   Uni-VLaT adds distributed whole-body touch to pretrained vision-language-action policies. Its tactile features support action generation and predict future tactile, proprioceptive, and visual representations, improving contact-rich humanoid control across five real-robot tasks with a 75% average success rate.

1. 

   UltraDP: Generalizable Carotid Ultrasound Scanning with Force-Aware Diffusion Policy

   Generalizing autonomous ultrasound scanning with a force-aware diffusion policy.

   Ruoqu Chen, Xiangjie Yan, Kangchen Lv, Gao Huang, Zheng Li, and Xiang Li^†

   IROS 2025

   Abs arXiv Website

   Present UltraDP, a Diffusion-Policy-based method that receives multi-sensory inputs (ultrasound images, wrist camera image, contact wrench, and probe pose) and generates actions that are fit for multi-modal action distributions in autonomous ultrasound scanning. UltraDP achieves a 95% success rate in transverse scanning on previously unseen subjects, demonstrating its effectiveness.

1. 

   EgoHSI3R: Online 4D Hand-Scene Interaction Reconstruction from Egocentric Video

   Reconstructing camera motion, dense scenes, world-space hands, and hand contact from egocentric video in a streaming manner.

   Mingzhi Pei, Zhangchen Ye, Ruoqu Chen, Liu Cao, Yao Feng, and Mengdi Xu

   CoRL 2026 Workshop Scaling H2R, Oct 2026

   Accepted

   Abs arXiv PDF Website

   Egocentric videos offer an abundant source of hand-scene interactions for embodied learning, which requires recovering hands, camera, and scene consistently in a shared 3D world. Existing methods focus on either camera-frame hands or world-space hand trajectories, without jointly recovering the scene. Building a complete reconstruction from such specialized predictions therefore typically requires per-sequence optimization. We propose EgoHSI3R, a unified feed-forward framework for online 4D hand-scene interaction reconstruction from egocentric monocular video. To jointly recover these components, it maintains a recurrent scene representation that accumulates geometric evidence over time and provides a shared world space and scale. We introduce hand-centric prompts that fuse pretrained hand priors and local scene context, while MANO vertex queries attend to scene features to predict dense per-vertex contact. These designs enable a single network to jointly estimate camera motion, dense scene geometry, world-space bilateral MANO hands, and per-vertex hand contact. EgoHSI3R achieves the lowest world-space hand joint error on ARCTIC, while running faster than prior world-space hand pipelines and retaining competitive camera and scene reconstruction quality.

1. 

   TOOLDEX: An Ego-Exo Human Demonstration Dataset for Creative Physical Tool Use

   Learning creative physical tool use from multi-view human demonstrations with hand and object annotations.

   Anonymous Authors

   CoRL 2026 Workshop, Oct 2026

   Accepted

   Abs Website

   TOOLDEX is a dataset of human tool-use demonstrations captured with eight synchronized RGB-D cameras. It covers diverse combinations of tools, actions, and objects, with annotations including object poses, masks, and hand trajectories to support policy learning, affordance discovery, and tool-use generalization.

Robot Interface

1. 

   Democratizing Robot-Use Agents: A Unified Harness with Open Models

   Jiarui Yuan, Yuxin Zuo, Xiaoyu Huang, Wenbo Zhang, Kwanyoung Park, Junlin Yang, Tianxiang Zhang, Ruoqu Chen, Yutao Ouyang, Bingxiang He, Xingtai Lv, Hongjin Wu, Jiajun Liu, Jiahao Zhao, Tianwei Luo, Haozhan Li, Cheng Qian, Jiaxin Ge, Huan-ang Gao, Kaisen Yang, Jiaqi Deng, Qixin Xu, Siqiao Huang, Bowen Zhou, Mengdi Xu, Zhiyuan Liu, Pieter Abbeel, and Ning Ding

   arXiv preprint, Nov 2026

   Abs arXiv PDF

   Robot-use Agents (RUAs) have shown promising zero-shot manipulation capabilities, but reliance on frontier, proprietary models limits research into further training. To democratize RUA research, we introduce uni-robot-agent, a unified harness for competitive open-weight robot use that supports further training. Without privileged inputs, an out-of-the-box Qwen3.8-Flash-Next achieves a RoboDojo Score of 31.39, surpassing the published GPT-6 Astra-RoboProbe result of 28.97 and many VLA/WAM baselines, and performs competitively on 19 real-world tasks. Our analysis across model generations, families, and sizes links effective robot use to four general capabilities, suggesting that future RUA training should develop them jointly.

1. 

   DexSkill: Learning and Refining Dexterous Manipulation Skills through Interaction

   Grounding hand motion data into reusable dexterous skill banks that LLM agents can retrieve, compose, and extend.

   Ruoqu Chen, Shutong Liu, Feixiang Ruan, Weili Fu, Zihao Wang, and Mengdi Xu^†

   Technical Report, Oct 2026

   Abs Code Website

   Dexterous robots must turn plausible hand poses into stable physical interactions, while language agents need skills they can invoke without generating joint trajectories. RLDexAgents organizes quasi-static hand references in a pose bank and trained, executable policies in a skill bank, connecting motion data to reinforcement-learning teachers, command-conditioned students and an extensible agent toolkit. We analyze approximately 915K human poses, 1.03M Sharpa poses, and 1,400 Wuji2 samples. Representative pose banks improve geometric coverage. We train the final Cluster + phase student with mixed DAgger/PPO and study Joint, Fingertip and Hybrid commands in an interface ablation. On 68 student-held-out tasks, students achieve 50–60% grasp success versus 94% for the teacher; richer commands do not consistently improve retention or target adherence. The toolkit separates high-level skill selection, external wrist control, and 30 Hz tactile hand control. These results establish a working research pipeline while identifying command ambiguity, contact establishment, and teacher–student observation differences as central obstacles to reusable dexterous skills.

1. 

   Robotic Spatiotemporal Editing: A Unified Interface for Controllable Action Generation via Masked Action Reconstruction

   Yanbang Li, Bu He, Yi Tian, Yu-Chiao Huang, Chenyu Zhang, Ruoqu Chen, Yuanzhe Liu, Yuxiang Gao, and Mengdi Xu

   arXiv preprint, Oct 2026

   Abs arXiv PDF Website

   A unified interface for controlling robot manipulation with spatial and temporal references. Masked Action Space (MAP) represents keypoints, flows, orientations, and keyposes as partially specified Cartesian actions. An Inpainting Policy reconstructs the remaining action components using progress-based alignment and training across multiple mask types. A RePaint inference variant further improves constraint satisfaction. Experiments in ManiSkill, LIBERO-10, and real-world manipulation evaluate task success and adherence to user-provided references.

Whole-Body Control & Loco-Manipulation

1. 

   WEAVE: Learning Whole-Body Dexterous Loco-Manipulation from Human–Object Interactions

   Learning coordinated whole-body and dexterous hand control from human-object interactions.

   Liu Cao, Xingze Wu, Jingzhi Cui, Botian Xu, Mingzhi Pei, Ruoqu Chen, and Mengdi Xu

   arXiv preprint, Oct 2026

   Abs arXiv PDF Code Website

   WEAVE learns coordinated humanoid locomotion and dexterous manipulation from captured human-object interactions. Contact-aware retargeting and approach-motion completion produce robot references, while a shared policy uses contact and object geometry to control the body and fingers across multiple objects.

1. 

   HuMiT: Low-Latency Whole-Body Humanoid Teleoperation via Minimal Reference Tracking

   Enabling responsive whole-body humanoid teleoperation from sparse human keypoints.

   Liu Cao, Botian Xu, Ruoqu Chen, Jiajun Liu, Mingzhi Pei, Zihao Wang, Zhongyu Li, and Mengdi Xu^†

   Preprint (Under Review), Oct 2026

   Abs Website

   HuMiT is a whole-body teleoperation system that drives a humanoid from sparse human keypoints alone, achieving  20 ms communication-to-inference latency to enable reactive whole-body skills such as ball catching and loco-manipulation.

1. 

   Whole-Body Model Predictive Control for Mobile Manipulation with Task Priority Transition

   Coordinating mobile manipulation tasks through smooth priority transitions in whole-body control.

   Yushi Wang, Ruoqu Chen, and Mingguo Zhao^†

   ICRA 2025

   Abs Website

   Developed a Whole-Body Model Predictive Control (WBMPC) framework to manage task priorities and scheduling in multi-task mobile manipulation scenarios. Integrated task priorities into a unified weight matrix, enabling smooth transitions across tasks in both spatial and temporal dimensions.