Salient event Detection in Soccer Video using LSTM with CNN Features

Video analysis, especially sport video analysis is very important. Video analysis include video summarization, action recognition, event detection etc, but event detection in video sequence is the most challenging task due to similar visual contents, capturing event from different view point and the

Project Title

Salient event Detection in Soccer Video using LSTM with CNN Features

Project Area of Specialization

Artificial Intelligence

Project Summary

Video analysis, especially sport video analysis is very important. Video analysis include video summarization, action recognition, event detection etc, but event detection in video sequence is the most challenging task due to similar visual contents, capturing event from different view point and the movement of camera. For example, the legs motion for kicking a football is a simple action, while jumping for a head-shoot is a collective motion of legs, arms, head, and whole body. To detect the most known event in soccer games such as Goal,Penalty,Red card,substitute and Corner, we proposed hybrid deep learning approach which use CNN and LSTM combinedly. We joint LSTM model with the last fully connected layer which find and learn the hidden sequance in the features extracted from the last fully connected layer of CNN. The final output of the proposed method is like a VAR system in Soccer Matches, which detect the predefined event in the given video. Our system can also detect event in live matches through live stream. Our proposed system can be extended for further improvement such as increase number of events and can modify for other sports video analysis by re-training.

Project Objectives

"Salient Event detection in Soccer Video using LSTM with CNN Features" is research based project. The main objective of this project is to build a real time event detector,which detect the salient event in soccer videos. 

The second objective is to built a system which not only detect salient event in soccer videos, but also used for summarization of soccer videos. such as highlight generation of soccer matches on the basis of these key events.

Third objective is to build a system which can use for statistical analysis of complete soccer match, such as count the number of goals,free kicks, corners, red cards and fouls etc.

Project Implementation Method

To implement this idea we used different Deeplearning methods and implementation tools.

For features extraction we use pretrained SqueezNet (CNN Architecture). which extract features from last fully connected layers.

After feature extraction process, the extracted features are then pass to the LSTM model. which find the hidden patterns in sequeneces and classify the events.

For implementation we used Matlab2018a, is an implementation tool. with matconvnet as an deep learning tool box.

Benefits of the Project

The benefits and gains of our project to the AI (Artificial Intelligence) society and multi-media society are in the following terms:

Efficiency: improve the exsiting sport video analysis system, and provide such an efficient system which can be used for multi-task. i.e Salient event detection in soccer videos,Highlight generation of soccer matches and for statistical analysis of sport videos.

Productivity: improving the existing system, and built automatic system for highlight generation instead of manual highlight generation, which very time consuming task. so our system will take less time and produce more output.

Quality: our system will  perform correct detection of soccer event, and will minimize the false detection rate of the existiing systems.

Technical Details of Final Deliverable

The final deliverable version of our software, will accept input soccer video, the input video will be divided into frames, then CNN features are extracted from these frames which are then classify by LSTM. 

Our system can be detect the predifined event in real time environment as well.

Can Generate Highlight on the basis of key events.

Can statistically analyze the soccer matches.

Final Deliverable of the Project

Software System

Type of Industry

IT

Technologies

Artificial Intelligence(AI)

Sustainable Development Goals

Responsible Consumption and Production

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com