Jason Barnes took a liking for drumming when he was a little boy – banging on whatever he could get his hands on as soon as he started listening to music! However, in 2012, when he was 22, a work accident caused his right hand to be amputated. This, in theory, meant that Jason couldn’t play drums so well anymore. Although a setback, Jason was determined to find a solution that will help him continue his passion for drumming in the future.

Jason did drum initially with what’s called a body-powered prosthetic. It didn’t have any electronics or robotic controls so he had to use a lot of shoulder and elbow to get the hits he wanted. There were two issues with this prosthetic though – it was exhausting and limited how fast he could play. To fix these issues, he had an idea to develop a robotic drumming prosthetic. Since 2013, Jason has been working with Gil Weinberg, a renowned roboticist and founding director of the Georgia Tech for Music Technology to create what’s possibly the world’s most advanced robotic drumming arm using artificial intelligence.

When Jason reached out, Gil immediately knew what he wanted – he wanted to sense more of his muscles and control the bounce of the drumstick. In short, play the drums as humanly natural as possible with a prosthetic arm. As they started working on this project, they realized that there were a lot of complex steps they had to take care of. One of the biggest concern was latency – the delay that occurred between Jason making a hit and the prosthetic actually recognizing it. Even a 50 millisecond lag was noticeable. In addition, it also mattered how the drumstick was gripped and bounced off a drum – these were elements that were hard to calibrate.

The team then decided to use EMG or Electromyography . This method consists of sensors in the prosthetic that pick up electric signals from Jason’s residual limb. Using these signals, the team was able to recognize complex patterns with machine learning and then map them to specific movements. TensorFlow, Google’s open source AI platform, does this in real time – no latency anymore! When Jason flexes his muscle, it tightens his grip on the stick and loosens it when he extends his muscle, just like it would with a normal hand.

This revolutionary robotic drumming arm has opened up many doors for research in accessibility especially in music, enabling many disabled musicians to continue to make music easily without hitting many limitations.

Watch the video below to learn Jason’s story and what all went into making this drumming arm a reality!

Jason is on Instagram as Cybernetx and his music can be found on Spotify, Apple Music and Soundcloud.

Source: Google, Google