A Reactive Grasping Framework For Multi-DoF Grippers via Task Space Velocity Fields and Joint Space QP

Under review

Yonghyeon Lee1*, Tzu-Yuan Lin1*, Alexander Alexiev1, Sangbae Kim1
* The authors contributed equally to this project
1 MIT Biomimetic Robotics Lab
Paper Code (Coming soon)

TL;DR: We plan globally in task space and track locally in joint space, enabling high-DoF grippers to reactively grasp highly concave objects in real time.

Real-time demos

Abstract

We present a fast and reactive grasping framework for multi-DoF grippers that combines task-space velocity fields with joint-space Quadratic Program (QP) in a hierarchical structure. Reactive, collision-free global motion planning is particularly challenging for high-DoF systems, as simultaneous increases in state dimensionality and planning horizon trigger a combinatorial explosion of the search space, making real-time planning intractable. To address this, we plan globally in a lower-dimensional task space – such as fingertip positions – and track locally in the full joint space while enforcing all constraints. This approach is realized by constructing velocity fields in multiple task-space coordinates (or, in some cases, a subset of joint coordinates) and solving a weighted joint-space QP to compute joint velocities that track these fields with appropriately assigned priorities. Through simulation experiments with privileged knowledge and real-world tests using the recent pose-tracking algorithm, FoundationPose, we verify that our method enables high-DoF arm–hand systems to perform real-time, collision-free reaching motions while adapting to dynamic environments and external disturbances.

1. Simulation Results

1.1. Single Target Object

Videos of our reactive grasping algorithms running on the MuJoCo physics engine demonstrate robust real-time adaptation to dynamic perturbations of both the robot and the objects.

1.1. Wine Glass

Recovery from External Perturbations


Dynamic Adaptation to Moving Objects


Realtime Fingertip Paths Optimization


1.2. Bowl

Recovery from External Perturbations


Dynamic Adaptation to Moving Objects


Realtime Fingertip Paths Optimization


1.3. Mug

Recovery from External Perturbations


Dynamic Adaptation to Moving Objects


Realtime Fingertip Paths Optimization


1.4. Dish

Recovery from External Perturbations


Dynamic Adaptation to Moving Objects


Realtime Fingertip Paths Optimization


▸ Failure Cases

Our method does not guarantee global convergence of the entire robot-hand system. Although the global path is planned in fingertip position space, the low-level QP only provides local tracking and does not account for global feasibility. As a result, complex joint constraints can still cause the system to become trapped -- though this occurs rarely in practice.

Mug


1.2. Cluttered Scenes with Moving Obstacles

Q) What happens if we use linear fingertip velocity fields?

Control with naïve linear fingertip velocity fields causes the robot to become trapped in local minima, failing to converge to the target grasp pose, particularly because of concave geometries.

Box


Wine Glass


Mug


Dish


Bowl


2. Real Robot Results

Videos of our reactive grasping algorithms running with the FoundationPose algorithm -- which enables real-time 6-DoF object pose tracking -- demonstrate robust real-time adaptation to dynamic perturbations of both the robot and the objects.

2.1. Single Target Object

2.2. Obstacles

Obstacles are represented as Superquadrics.

Citation


      @misc{lee2025reactivegraspingframeworkmultidof,
      title={A Reactive Grasping Framework for Multi-DoF Grippers via Task Space Velocity Fields and Joint Space QP}, 
      author={Yonghyeon Lee and Tzu-Yuan Lin and Alexander Alexiev and Sangbae Kim},
      year={2025},
      eprint={2509.01044},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2509.01044},}