Gimbal Based Robotic Eye for Fast Moving Stimulus in Dynamic Social Environment

Project Title

Gimbal Based Robotic Eye for Fast Moving Stimulus in Dynamic Social Environment

Project Area of Specialization Electrical/Electronic EngineeringProject Summary

Robotics deals with design, constructions and use of computer system for controlling the robots. The field of robotics also connects with artificial intelligence and machine learning. Nowadays, humanoid robots are getting popularity which includes full human body design and construction. Swarm of robots are also used for surveillance and for carrying object that require more robots. Working in swarm of robots, fast eye movement is required in order to track different robots for collecting more information. As in human, eyes are the major organ which collects more than 80 percent of information from the environment. So robots use camera sensor as eye to acquire information about its surrounding environment.

In robotic eye, fast movement of eye ball is required to track the fast moving stimulus which is necessary in case of communication between swarm of robots. But in dynamic social environment we have effect of unstructured environment. Camera should be decoupled from the robot body and head for fast moving stimulus tracking. Just like human eye, which is capable of locking the object in the center of retina even when position of head changes. Driven by this motivation, a two axis gimbal system is considered in this project. Two axis gimbal system is used to achieve high decoupling ratio. Mathematical modeling of gimbal system is done by considering the torque disturbances and dynamic unbalance. For efficient tracking and independent eye movement in the bumpy environment, stabilization control system is designed. Non-linear System equations are linearized using Jacobain matrix to achieve linear state space equations. The overall model is simulated on the Simulink with CAD model of the gimbal and proportional integral derivative (PID) controller is used for the control and stability of the system. Simulation results shows the efficacy of our proposed controller.

Figure 1 Gimbal System

Hardware Design
1. Installing of Camera Sensor in Gimbal System to make robotic eye
2. Installing of the Gimbal Based Robotic Eye in a helmet
3. Application of control for proper tracking of stimulus. Control scheme is implemented using microcontroller and proper electronics circuitry is designed.
4. Electronic Gimbal can also be used that have its control systems

Figure 2 Mechanical Sketch of gimbal

Figure 3 Helmet

Figure 4: Our CAD model of mechanical gimbal with two axis YAW and PITCH

Project Objectives

Objective of robotic eye is to track the fast moving stimulus in dynamic environment without head movement as in human eye. Optical system of humans is perfect and highly developed after thousands of years of evolution. Human visual system can lock the object in the center of eye retina even when the position of head changes drastically. Due to that significance, robotic eye algorithm has been developed for the unstructured and bumpy environments. Most of researchers studied the perfect human eye tracking system and tried to implements it on the robots using the visual control, actuators and camera sensors. Researchers have developed many robotic eye tracking systems in last 20 years focusing on the human eye working.

Strong vision control theory and good actuators are useless until the eye ball is not synchronized with control algorithm. It is not possible with high coupling ratio with skeleton head and body. Gimbal Bases system is used which provides strong decoupling of head and body and it is very simple mechanical design. So gimbal system has to modeled with gyro sensors and camera on the inner axis so that we can get the maximum decoupling ratio. For tracking, we are using computer vision techniques as a tool.

Project Implementation Method

Gimbal system is mathematically modelled firstly. Newton law of rotational body is used to design control system of gimbal. According to the newton second law, if more than one torque acts on the body then the net torque is the sum of all the torques due to which body rotates. If the net torque is zero than the body does not rotate.
In dynamic environment, there is always some torque acts on the body, so we have to make net toque to zero to make body static. In gimbal system, disturbance torque is applied on one the gimbal axis and also on the base body due to disturbance in the environment. We insert motors to provide proportional torque which counterbalance the disturbance torque. In this way, line of sight(LOS) is unaffected by the base body’s motion.

From mathematical modelling, we got the two nonlinear and cross coupled equation for two axis yaw and pitch.

Pitch channel

Yaw Channel

So we linearized our model using jacobian and taylor series techniques to get the linearized state space matrices. State Space model is controlled using the proportional integral and derivative (PID) controller.

Figure 1 Stability Loop

High rate camera is used to keep the track of the object. A high processing power is required by the algorithm so we need an embedded system which must be capable of doing so. Open CV is used to detect the different object using python algorithms.

Figure 2 Implementation diagram

Benefits of the Project

Following are the benefits of this design over the existing models :

1. Gimbal system has the property of decoupling anything that is in the inner channel from the support on which it is mounted. Due to this property, high decoupling ratio can be achieved easily. The high decoupling ratio of gimbal system helps the robotic eye in tracking fast stimulus in a dynamic environment using simple camera sensors.
2. The proposed theoretical architecture for feedback response, physiological structure, and mechanical design is more mimic a human vision system than any other existing one.
3. The feedback mechanism or neural system response via fast electrical signal between vision control and camera system is also designed. The proposed vision system is well suitable for combining the existing research on AI based tracking and recognition (vision control) with research on the fast camera sensor/eyeball movement due to the high decoupling ratio, in a dynamic social environment. This makes the proposed design  more mimic an actual human vision system
4. Another contribution is fault dependent mathematical modeling, that is robust against the disturbances of static mass imbalance and friction of the gimbal system. The mathematical modeling provide a base for its stabilization control design.

Some of the applications are:

• Implantation as a Robotic eye ball to keep dynamic object in line of sight by rotating camera in 2D frame.

• A Camera-Based Target Detection and Positioning UAV System for Search and Rescue (SAR) Purposes

• This robotic eye can cover the area of two human eyes. 

• Keep track of dynamic cricket ball without human help.

• Keep track of the tennis ball 

Technical Details of Final Deliverable

Some of the technical details of the final product are:

• This robtic eye can cover +70 to -70 degree in yaw axis and +90 to -70 on pitch axis.
• BLDC motors are used for fast and smooth  tracking of the object
• The speed of human eyeball movement is beyond 800 degree per second that means it can move at max 20 degree per 20 milli seconds. The available gimbal systems in market can rotate at the speed of 90 degree in 20 milli seconds with milli seconds accuracy 12 that is much better and faster than human eye. Our desinged gimbal model can roughly 40 degree per 20 milisecond.
• Size of the gimbal is small which can be fit in an helmet and can be placed on robotic head.
• It can detect face and ball on our case but we can use recognition techniques to detect it for other objects also like birds, vehicles etc.

Final Deliverable of the Project HW/SW integrated systemCore Industry SecurityOther Industries Education , IT , Media , Others , Telecommunication Core Technology RoboticsOther Technologies Artificial Intelligence(AI), NeuroTech, OthersSustainable Development Goals Decent Work and Economic Growth, Industry, Innovation and Infrastructure, Peace and Justice Strong InstitutionsRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	52000
ELP Fish Eye 720P OV9712 Sensor Wide Angle Camera Module with Ir Filte	Equipment	1	8000	8000
FPV gimbal with controller	Equipment	1	8000	8000
Arduino DUE	Equipment	1	2500	2500
Wires and Circuit boards	Miscellaneous	1	1000	1000
Shipping charges	Miscellaneous	1	500	500
Rasberry PI 4	Equipment	1	16000	16000
Lipo Battery	Equipment	1	10000	10000
Electronic Speed Controller	Equipment	2	1500	3000
Helmet	Equipment	1	3000	3000