User Stories

Nara Institute of Science and Technology Researchers Develop Tactile Object Recognition Algorithms for a Robotic Hand

Challenge

Enable a robotic hand to identify objects based on tactile sensor input

Solution

Use MATLAB to develop machine learning and object recognition algorithms and use Robotics System Toolbox to establish connectivity between the algorithms and the ROS-enabled robot

Results

  • Hundreds of manual steps eliminated
  • Opportunities to try new algorithm ideas increased
  • Expertise shared with other students and researchers

“With Robotics System Toolbox, we seamlessly connected to and controlled our robot directly from the algorithms we had developed in MATLAB, enabling us to minimize development time. We used the time we gained to further our research into new tactile object recognition algorithms.”

Takamitsu Matsubara, Nara Institute of Science and Technology
NAIST’s ROS-enabled Shadow robot performing tactile object recognition.

Robots that can recognize objects via tactile sensations instead of visual cues have a wide range of potential applications, including surgical procedures and rescue operations. Researchers at the Intelligent System Control Laboratory at Nara Institute of Science and Technology (NAIST) in Japan have developed algorithms that enable a dexterous robotic hand equipped with sensors to identify objects via pressure, vibration, and temperature. The object recognition algorithms employ an active learning approach: The hand performs an action such as rubbing, squeezing, or pulling the object, obtaining tactile information that it uses to plan its next action.

NAIST researchers used MATLAB® and Robotics System Toolbox™ to develop two algorithms for tactile object recognition. The first uses machine learning techniques to develop a probabilistic model from observed sensor data. The second uses the learned model to recognize different objects.

“MATLAB let us focus on conducting our research rather than writing code,” says Takamitsu Matsubara, assistant professor at NAIST. “Usually when we are working with a new robot there is a lengthy code writing phase, but MATLAB and Robotics System Toolbox enabled us to minimize this phase and concentrate on improving active tactile object recognition.”

Challenge

Machine learning and object recognition algorithms often require numerous calculations and matrix operations. The NAIST team was already using MATLAB to accelerate the pace of their research. Their challenge was to accelerate testing by connecting their algorithms directly to the Robot Operating System (ROS)-enabled Shadow Dexterous Hand robot that they were using in the lab.

Solution

The NAIST researchers used MATLAB and Robotics System Toolbox to develop tactile object recognition algorithms and fully automate their workflow.

Working in MATLAB the group developed initial versions of the learning and object identification algorithms. Both algorithms incorporated a Gaussian Process Latent Variable Model, also developed in MATLAB.

The team used functions from Global Optimization Toolbox and Optimization Toolbox™ in the learning algorithm to determine what planned motion would be needed to acquire missing information about the object being learned.

For the object recognition algorithm, they used Optimization Toolbox functions to identify the exploratory actions that would provide the most useful new information about the object.

Before testing the algorithms on the actual robot, the researchers ran simulations in MATLAB using a mathematical model of the target object to generate input data.

They accelerated simulations, optimizations, and matrix operations by executing them on several processors of a multicore computer using Parallel Computing Toolbox™.

In order to establish a communication between the algorithm and the ROS-enabled Shadow robot, the team used Robotics System Toolbox to access the robot’s ROS API directly from their MATLAB code.

They invoked Robotics System Toolbox functions in their algorithms for publishing hand and fingertip positions to the robot and for subscribing to BioTac sensor data from the robot.

The NAIST team has completed development of algorithms capable of identifying 10 different objects. They are working on enhanced versions that can identify 100 objects or more.

Results

  • Hundreds of manual steps eliminated. “Before we used Robotics System Toolbox with MATLAB, the learning process took us about 24 hours to complete,” says Daisuke Tanaka, researcher and Ph.D. candidate at NAIST. “Now the process is fully automated, so instead of standing in front of the robot all day, we can do our work while it completes its tasks without us.”

  • Opportunities to try new algorithm ideas increased. “There is no doubt that MATLAB and Robotics System Toolbox have enabled us to concentrate on our research instead of writing C++ code,” says Tanaka. “As a result, we have more time to explore new algorithms.”

  • Expertise shared with other students and researchers. “After our success in using MATLAB and Robotics System Toolbox for our research, junior students at NAIST began using MATLAB with ROS-based robots as well,” says Dr. Matsubara. “Using MATLAB to create an automated environment enabled us to pass on our know-how to other students and researchers.”

Products Used

Learn more about Nara Institute of Science and Technology

Showcase Your Success

Join the Customer Reference Program

Get started