Using Twitter and Machine Learning for Disaster Monitoring

Project Title

Using Twitter and Machine Learning for Disaster Monitoring

Project Area of Specialization Artificial IntelligenceProject Summary

In today’s world, disasters, both natural and man-made, are becoming increasingly frequent and new solutions are of a compelling need to provide and disseminate information about these disasters to the public and concerned authorities in an effective and efficient manner. One of the most frequently used ways for information dissemination today is through the use of Social Media, and, when it comes to real-time information, Twitter is often the channel of choice. This project will target to produce a solution that contains both offline Machine Learning training phase and an online prediction phase using twitter data. In this project, we can compare various conventional machine learning and deep learning algorithms for classifying disaster-related tweets into different classes. The models can be tested with various disaster events such as earthquake, flood, hurricane and wildfire to evaluate the efficiency of the models. Moreover, the application should use Kafka or Spark Streaming for hosting both static and real-time Twitter data over the newly trained ML models. Additionally, it should have a visualization component where a live dashboard is updated.

Project Objectives

• A solution for Machine Learning training phase working on real-time predictions using live and batch twitter data.

• We can compare various conventional machine learning algorithms for classifying disaster-related tweets into different classes.

• The models can be tested with various disaster events such as earthquake, flood, hurricane and wildfire.

• Measuring the performance of Machine Learning techniques based on higher accuracy.

• Supporting English language for training and testing our model.

• A live dashboard of updates and visualization will feature this project.

• User typed tweet and an import from Twitter API features will be added to the project for the sentiment analysis.

• Information extraction from tweets and textual description for alert generation/warnings.

Project Implementation Method

In recent years, Twitter has become a major channel for communication during natural disasters. Twitter users have shared news, photos and observations from the midst of hurricanes, blizzards, earthquakes and other emergencies. The responders can use these streams of data generated by social media to find out where the disasters are happening and what specifically has been affected as a result of it. As with most social media conversations, informative signals are often overwhelmed by irrelevant and redundant noise. The responders struggle to glean actionable knowledge from the large volume of tweets and status updates. In order to effectively extract information relevant to disaster relief workers, we propose Twitter for Disaster Response. The goal of the project is to extract information relevant for responders from tweets generated during disasters in real time as well as enable analysis after the disaster has occurred.

It’s clear that we are in need of an effective and efficient mechanism that can provides us the right information at the right time related to the disasters that could let the concerned authorities better handle situations like this. As for the public, a live dashboard will be updated on the events related information.

Therefore, with the help of the existing datasets, the initial approach will be the pre-processing of datasets depending on the datasets. An offline machine learning phase could use different classifiers like, Random Forest (RF), Decision Tree (DT), Support Vector Machine (SVM), etc. The extraction of feature is vital stage in building a machine learning classifier. The target is to reduce raw data into a number of set of features by keeping the accuracy of the data. Techniques like N-Gram, TF-IDF, HashingTF, etc. could be used for feature extraction. For Online streaming, the tweets could be classified on the trained model resulting a sentimental analysis of the positivity, negativity and neutral state of tweets. At last, there should be a live dashboard having charts and maps plots for the final results.

Benefits of the Project

Helping disaster management authorities for timely and effective decisions

 Natural disaster prediction and monitoring

 Crowd sourcing

 Open Information access

 Trained Models with higher accuracy will feature this project for prediction and sentiment analysis of disaster

Technical Details of Final Deliverable

SRS: This contains all the requriemnts for the implementation of the project. This includes the hardware and software needs for the project.

Thesis: A complete guide from the start of the poject to the end of the project. A file that is detailed that explains how each step was performed. 

Implementation Code: This will be a zip file that will include all the codes consisting of different files. Either be of python or web implemenation code.

Final Deliverable of the Project Software SystemCore Industry ITOther IndustriesCore Technology Artificial Intelligence(AI)Other TechnologiesSustainable Development Goals Decent Work and Economic Growth, Industry, Innovation and Infrastructure, Life on LandRequired Resources

Item Name	Type	No. of Units	Per Unit Cost (in Rs)	Total (in Rs)
			Total in (Rs)	55000
Sensor for Flood with IoTs	Equipment	1	15000	15000
Sensor for Fire	Equipment	1	15000	15000
Sensor for Hurricane	Equipment	1	15000	15000
Printing cost and stationary	Miscellaneous	1	10000	10000