Multimodal sensing system for classroom

Project Title

Multimodal sensing system for classroom

Project Area of Specialization Electrical/Electronic EngineeringProject Summary

This project aims to build an Artificial Intelligence-powered multimodal sensing system that can monitor the learning performance of students in a classroom environment. The project will consist of three parts: first consists of environmental sensing modules in order to sense data to predict the temperature, illumination, air quality, acoustic noise. We have learned from research papers that these factors affect the concentration levels of human beings. The second part of the project consists of a visual sensing platform where we will use two HD cameras in order to monitor students. Here, video recordings will be fed to Deep Neural Network-based models in order to get information about the student's body posture, students’ movement, how many students are there, even their head orientation, the speech act between student vs instructor, body segmentation. The third component of our project is an Android Application that will be used to get meta-data (such as their concentration level, how much they learned etc.) from students after every class. We plan to collect data for two Semesters but this depends on when on-campus classes resume. At the end of the project, we will undertake the data-driven analysis and report on factors that affect students’ performance in classrooms.

Project Objectives

Hardware part:

Integrate Temperature and Humidity sensor (DHT22)

Integrate Illumination sensor (APDS9960)

Integrate Gas sensor (MQT-135)

Integrate the spw2430 sensor.

Integrate all these sensors with ESP8266 or NODE MCU for internet connection,

Integrate the camera in the classroom.

       

     In the Hardware part, we design a sensing Module to collect the data of sensors

For video data, we integrate only a camera to collect video data of the classroom.

Software part:

  Working on Android studio to develop an android application.

   Data collection and data-driven analysis

Project Implementation Method

https://drive.google.com/file/d/1PpKhIm5UmZKdGeeCpDQwobaxcKGoHsb6/view?usp=sharing

Benefits of the Project

Teaching environments in schools, colleges, and universities are typically sub-optimal. In most cases, the classroom environment is not designed to optimize the learning process. Our project seeks to change this. We aim to build a system that can analyze data from audio, visual, and environmental sensors in order to gauge the performance of students.

Such a system can be useful to design a better classroom environment that can improve the quality of learning, one which engages all students in the learning process.
By creating an atmosphere of understanding and encouraging students to listen with the goal of understanding, we can improve their communication skills.
 

Technical Details of Final Deliverable

Our system will sense the environmental conditions like temperature, humidity, and gas level in the classroom smartphone usage sensing, visual sensing, get the feedback from the smartphone application. We will test such a hypothesis by taking a Telecommunication department as a case study. By this, we can improve the learning efficiency of the classroom. The system will able to monitor and assess the environment to ensure the comfort of the students.

Final Deliverable of the Project HW/SW integrated systemCore Industry TelecommunicationOther Industries Education , IT , Others Core Technology OthersOther Technologies Internet of Things (IoT)Sustainable Development Goals Quality EducationRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	47370
NODE MCU	Equipment	2	700	1400
DHT22	Equipment	2	600	1200
APDS9960	Equipment	2	1200	2400
MQT-135	Equipment	2	200	400
Spw2430	Equipment	2	2080	4160
Sand disk CD card 32GB	Miscellaneous	3	1600	4800
Arduino Nano 33 BLE	Equipment	1	12000	12000
Camera setup (Package)02 CCTV cameras 5 MP	Equipment	1	20500	20500
SD card module	Miscellaneous	3	170	510