COTY Humanoid Interactive Therapy Aide Robot
The project at hand is a robot that interacts with the Autistic patients to make them feel better by involving them in activities to make them more socially acceptable and interactive. It is not only responsive but also engages the children in a natural and comfortable way. This robot is not intimid
2025-06-28 16:30:58 - Adil Khan
COTY Humanoid Interactive Therapy Aide Robot
Project Area of Specialization Internet of ThingsProject SummaryThe project at hand is a robot that interacts with the Autistic patients to make them feel better by involving them in activities to make them more socially acceptable and interactive. It is not only responsive but also engages the children in a natural and comfortable way. This robot is not intimidating therefore kids are likely to feel better in its presence. It is helpful for the subjects who are reluctant to see a therapist or psychologist because of the lower cognition or those who just don’t interact or communicate. This robot is likely to be beneficial in therapies.
The motivation for doing this project was primarily an interest in undertaking a challenging project in an interesting area of research. Individuals with mental illnesses like Autism Spectrum Disorder (ASD) need to be effectively treated, even if there is a lack of qualified experts available or there exist compatibility issues with other available cognitive behavioral treatments. Therefore, a robot that is available 24/7 for a specific individual can aid in therapy by implicitly, even when there is no support or therapeutic personnel available.
Project ObjectivesFollowing are the objectives that are supposed to be accomplished at the end of the project.
-
To analyze a subject’s mood.
-
To give ample therapy lessons that will make the subject calm again.
-
To suggest methods and exercises to keep the subject engaged till he gets mentally stable.
-
To keep a log of the subject’s moods in case if needed by the therapist to gauge the mental state of the subject.
We completed our work on the project in the following manner:
Testing on Computer:
We programmed our codes for Facial Detection and Recognition, Mood Recognition and Voice Recognition in Python and then executed them on a Linux-based computer. We made sure that all the codes were perfectly running before we transferred them to Raspberry Pi. It is easy to handle the codes on the computer, as it is much faster due to increased computational power and is fairly easy to use. For this purpose, we used the normal WebCam and microphone that are compatible with Linux Computers and tested our results.
Porting on Raspberry Pi:
Once our codes were fully functional and were completely error-free, we set up our Raspberry Pi with Pi Camera and Microphone. We followed the instructions as were provided to us, by the official website. We transferred the Raspbian image to the SD Card by using the official NOOBS. After the complete initialization and the setup of our RasPi, we ported our codes onto it. We had to install the packages on the RasPi itself. Furthermore, we tested our codes on it. We reduced the data size and code complexity to the minimum, so that we can achieve robust results.
Benefits of the ProjectOur solution recognizes the mood and provides therapy lessons to the subject. It also maintains a log of the therapy sessions. As far as the goals are concerned, we changed them a little bit due to time constraints. The inclusion of emergency measures was subjected to change but that can be used as a future work on the project. For a more robust system, a higher computing platform or use of neural networks can be called into consideration.
Following were the benefits of our project:
-
It effectively recognizes the facial expressions.
-
It has built in voice commands to interact with the subject.
-
It issues the therapy measures for the subject.
-
It assists the professional therapist by providing the log of subject’s history. Thereby, reducing the need for a therapist everytime.
the greetings that appear on the screen when the program starts to run. It shows 3 buttons in the top left corner. “Quit” shuts the program, “Proceed” initiates the program and “Hello” displays the idle screen, which is a gif. The detected face, bounded by a red box, showing the name of the person sitting in front of it. For faces that do not match the database, “Unknown” appears in the bounded box.
As soon as the person appears in front of the camera the rectangular box appears marking the boundary of the face. The person is mapped onto the database and the name is then printed.As the camera focusses on the face, the mood is captured, i.e., any of the six basic emotions; Happy, Sad, Angry, Fear, Disgust, Surprise. These results are of utmost importance as the final product is required to capture the emotions of the subject and then record it in a log, that can aid in carrying out further therapeutic procedures.
We incorporated the stories for the different age categories as our subjec ts were kids. These stories are supposed to engage the subject into a conversation to help them overcoming the anxiety to speak. These stories demand different reactions to them and hence, help in the overcoming the subject’s anxiety thereby assisting in speech therapy.
Due to the computational and resource constraints, the Raspberry was slow to response to facial detection. It took a relatively longer time to run the code. The mood detection was swift, with accuracy up to 90-95% for the pre-trained people. However, for the unknown subjects, the accuracy dropped. Overclocking the Raspberry Pi significantly reduced the time delays; however overclocking can reduce the lifetime of a Raspberry Pi, therefore, is not recommended.
There were time delays for the voice input and the resulting issued commands and greetings. Using a Raspberry Pi compatible USB shield microphone introduced some trials where speech was not detected correctly by the server, upon which it returned the error. The response time directly depends on the speed and bandwidth of the internet connection being provided to the Raspberry Pi module, response time ranging from 1.5 or 2 seconds to a few minutes depending on the Ethernet connection.
Final Deliverable of the Project HW/SW integrated systemType of Industry Medical , Health Technologies Internet of Things (IoT), OthersSustainable Development Goals Good Health and Well-Being for PeopleRequired Resources| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| Total in (Rs) | 9997 | |||
| Micro SD Card 16GB Sandisk C10?? | Equipment | 1 | 550 | 550 |
| Card Reader with Cable?? | Miscellaneous | 1 | 300 | 300 |
| Microphone USB plugged with connector? | Equipment | 1 | 250 | 250 |
| Raspberry Pi3 Model B+?? | Equipment | 1 | 5300 | 5300 |
| Camera Module (Raspberry Pi)?? | Equipment | 1 | 1000 | 1000 |
| LCD MPI3508 3.5” HDMI?? | Equipment | 1 | 2100 | 2100 |
| Speaker with Amplifier? | Equipment | 1 | 497 | 497 |