Prosthetic Hand with Expanded Gestures Using Sequential AI Models

Project Title

Prosthetic Hand with Expanded Gestures Using Sequential AI Models

Project Area of Specialization Artificial IntelligenceProject Summary

Limb loss is very common among army veterans and it renders the capability of the affected to do any work afterwards and therefore people with such ailments seldom find employment. Limb loss is also very common among people who work in factories without following safety precautions, this is very common in third world countries like Pakistan. In order to rehabilitate the affected people and help them find job many initiatives have been taken by the governments worldwide. One of these initiatives is to develop prosthesis for the people that have been affected.

Prosthetic limbs are artificial implants that are used to replace limbs in a human body that are missing. The use of prothesis can be traced back to 3000 BC to ancient Egypt where passive prosthetic limbs were used to replace missing limbs for aesthetic purposes. However, the modern prosthetic limbs are much more advanced and can be used to carry out complex tasks.

Project Objectives

Our objectives for this project are as follows:

1. Generate a dataset of the hand movements against their respective muscle movement.
2. 3D print a prosthetic arm that will be made with a socket specifically made for the person that will use it.
3. Train a LSTM model that will be trained on the collected dataset. Deploy the model on a Jetson nano and predict hand movement based on EMG sensor values.

Project Implementation Method

The project consists of two parts, simulation and physical test namely.

Software Requirements

The AI model will be a sequential model, specifically an LSTM model. The LSTM model will be trained on a custom dataset that will be collected by us. The dataset will consist of different hand positions and their respective hand gesture sensor data against them. The data will be collected through multiple iterations of every hand gestures from a single individual. The dataset will be collected across a number of individuals so that versatile muscle movements are collected over time for a set number of gestures.

Once the data is collected, the LSTM model will be trained on the data. The trained model will then be used deployed on a jetson nano. The Arduino will collect sensor data from the muscle movement and then the data will be passed to the jetson nano which will send the data to the LSTM model. The prediction made by the LSTM model will then be used to generate servo positions. The servo positions will be sent to the Arduino and the robotic hand will move according to the predicted servo movements.

Hardware Requirements

The robotic hand will be 3D printed and the fingers will have joints that will be controlled using servo motors. The servos will be controlled using an Arduino. The Arduino will also have an Electro Myo-Gramme (EMG)  sensor shield connected to it that will be used to sense the muscle movements.

The sensor readings will be sent to a jetson nano for processing and the raspberry pi will send control signals to the Arduino to control the servo motors. Based on the movement of the servo motors the fingers in the prosthetic arm will move to the desired hand position.

The robotic hand will be made specifically for the person that will be using it. The arm will have a socket built into it where the stump in the hand will fit. Once the arm is place in the right position then the EMG electrodes will be placed on the elbow and the involuntary movement of the muscles when the person attempts to move the prosthetic will be captured through the electrodes. The sensor data will be sent by the Arduino to a jetson nano where the data will be processed. Once the data is processed the final result will be sent to the Arduino again and based on the movement of the muscles the real hand will move

Benefits of the Project

The product will mainly be focused towards the people who cannot afford a prosthetic arm. Since the entire assembly will be 3D printed, and all the electronics will be readily available and off the shelf, the overall cost of the prosthesis will be extremely low compared to the normal commercial prosthesis.

The project will also be developed and manufactured indigenously which will generate employment for engineering, manufacturing and business staff.

The project will also prosper the country with revenue from international purchases.

Technical Details of Final Deliverable

The final deliverable will contain a prosthesis that can be controlled using EMG signals from the wearer’s forearm. The EMG signal will be sensed by the EMG sensors and then amplified. The signal will be passed to a Jetson Nano which will filter the noise from the signal and the signal will be segmented into small windows. The small signal window will be used for feature extraction. The features will be extracted according to Hudgins time domain features. The extracted features will be passed through a neural network that will be running on the jetson nano. The neural network will classify which gesture is being made. Depending on the gesture, another program will control the servo motors inside the prosthesis and the gesture that was predicted by the neural network will be performed.

Final Deliverable of the Project HW/SW integrated systemCore Industry EducationOther Industries Others Core Technology Artificial Intelligence(AI)Other Technologies RoboticsSustainable Development Goals Quality Education, Industry, Innovation and InfrastructureRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	69860
MG995	Equipment	10	3450	34500
Flex Sensors	Equipment	5	2000	10000
Servo controller boards	Equipment	2	1200	2400
Arduino Boards	Equipment	3	1050	3150
Jetson Nano	Equipment	1	14500	14500
EMG sensor	Equipment	2	1500	3000
EMG Electrodes	Miscellaneous	50	20	1000
Screws	Miscellaneous	100	7	700
PCB pillars	Miscellaneous	50	9	450
Wires	Miscellaneous	40	4	160