Human-centered teleoperation

Research thrust overview

Associated faculty

 

Related research labs

Wearable haptic interface that is wireless, soft, lightweight, and capable of tactile/kinesthetic sensing and feedback for teleoperation, immersive VR/AR training, and assistance for the impaired.

  • Haptics research focused on designing micro-actuation cells that can generate multi-modal cutaneous feedback.
  • Control of actuator arrays to generate haptic sensations that simulate natural human touch.
  • “Superhuman sensing” through optical and electromagnetic sensing of nearby objects.
  • Sensing with acoustics and ultrasound for imaging the internal structure of a contacting object.

Thrust projects described below include:

Soft robotics and sensing

Progress in human-centered teleoperation requires haptic interfaces that are safe and comfortable for contact with the human body. This requires soft robotic systems and materials that match the compliance of natural biological tissue. Using novel material architectures that combine fluids, gels, elastomers, and shape memory matter, we are creating tissue-like sensing skins and artificial muscle actuators for next generation haptics and teleoperation. These soft material technologies can be used for wearable interfaces or be seamlessly incorporated into teleoperated soft robots that interact with their environment in much the same way that humans do.

  • Soft robot gripper
    Soft robot gripper with sensing skin to perform autonomous grasping tasks
  • Soft sensing skin
    Soft sensing skin for tracking hand motion and finger contact
  • Robot augmented with wireless electronic skin
    Robot augmented with wireless electronic skin for pressure sensing, proximity detection, and human vitals monitoring

    • Fingersight

    Tool handling and close-up operation are challenges for the visually impaired that have not been well addressed by assistive devices. The FingerSight device consists of four vibrators (tactors) enclosed in a flexible ring and a small camera mounted at its top. Images captured by the camera are analyzed to recognize targets of reaching and the tip of any tool grasped in the hand, enabling software algorithms to generate tactile signals that guide the user’s hand to the target.

  • later version of fingersight
    A later version of the finger sight ring and camera
  • fingersight version used in experiments
    Full-finger mounted system used in experiments
  • layout of experiment
    Layout of experiment for guidance to target: correction of x, y errors as move in on z

    • Soft haptic devices

    We are developing wearable soft haptic devices using novel magnetorheological materials. Using electromagnets, we can focus and control varying magnetic fields which will transmit forces to the skin of the user in multiple areas.

    In the following video, you can see a user wearing a sheath made of a magnetorehological fluid. Electromagnets are then used to provide variable force feedback to the user’s fingertip.

    Delta grippers for dexterous manipulations

    We are developing compliant robot fingers based on delta kinematics for performing dexterous manipulation tasks. Each delta finger allows the robot to move its fingertip in 3D space, and multiple fingers can be coordinated to perform in-hand manipulation. The links of the fingers are 3D printed of compliant materials to allow for safe interactions for both the robot and its environment. The gripper can be operated through teleoperation or using autonomous manipulation skills.

    Outreach and educational activities using haptic devices

    We are developing inexpensive-open source haptic devices to be used in educational and outreach activities. These devices can be 3-D printed, assembled, and programmed by students and are used to teach mechatronics, robotics, haptics, physics, and mathematics.