How to Do Twitter Sentiment Analysis with Machine Learning

Twitter boasts 330 million monthly active users, which allows businesses to reach a broad audience and connect with customers without intermediaries. On the downside, there’s so much information that it’s hard for brands to quickly detect negative social mentions that could harm their business.

That's why sentiment analysis, which involves monitoring emotions in conversations on social media platforms, has become a key strategy in social media marketing.

Listening to how customers feel on Twitter allows companies to understand their audience, keep on top of what’s being said about their brand, and their competitors, and discover new trends in the industry.

In this guide, learn how you can use sentiment analysis tools to listen to your customers on Twitter, and follow our tutorial on how to perform sentiment analysis in just a few simple steps.

What Is Sentiment Analysis?

Sentiment analysis is the automated process of identifying and classifying subjective information in text data. This might be an opinion, a judgment, or a feeling about a particular topic or product feature.

The most common type of sentiment analysis is ‘polarity detection’ and involves classifying statements as positive, negative or neutral. A polarity sentiment analysis model, for example, automatically tags this tweet as positive:

Twitter comment example showing sentiment: "Love the new security feature".

Test with your own text

Results

TagConfidence
Positive98.9%

Sentiment analysis uses Natural Language Processing (NLP) to make sense of human language, and machine learning to automatically deliver accurate results.

Connect sentiment analysis tools directly to your social platforms , so you can monitor your tweets as and when they come in, 24/7, and get up-to-the-minute insights from your social mentions.

How to Perform Sentiment Analysis on your Twitter Data

Performing sentiment analysis on Twitter data involves five steps:

  1. Gather relevant Twitter data
  2. Clean your data using pre-processing techniques
  3. Create a sentiment analysis machine learning model
  4. Analyze your Twitter data using your sentiment analysis model
  5. Visualize the results of your Twitter sentiment analysis

In this section, we’ll explain each of these stages and provide tools for both coders and non-coders so you can get started with sentiment analysis right away.

1. Gather Twitter Data

It’s important that your Twitter data is representative of what you're trying to find out because you’ll use it to:  

  • Train your sentiment analysis model
  • Test how your model performs on Twitter data

You should also consider the type of tweets you want to analyze: 

  • Current Tweets: useful to track keywords or hashtags in real-time.

  • Historical Tweets: useful to compare sentiments over different periods.

Now, you’re probably wondering how to extract data from Twitter if you don’t already have it saved in your help desk or in an Excel file. There are different ways to do this. Let’s take a closer look at some of the options:

Create a Zap in Zapier

Zapier is a platform that enables different teams (marketing, HR, customer support, product, etc) to connect the apps they use so that they can work together. It’s excellent for non-technical users since you don’t need to write a single line of code to gather tweets.

To create an automated workflow on Zapier (a Zap), just choose one app as the Trigger(this will be the app from where you’ll extract data) and another app (or apps) as the Action (where the data will be sent).

Let’s say you want to extract tweets that mention your brand in real time. You could use Zapier to connect Twitter with Google Sheets and gather tweets as soon as the Zap detects your brand name in tweets:

Steps to set Twitter as the trigger app when creating a Zap.

Go one step further and connect Zapier with MonkeyLearn to automatically perform sentiment analysis on your incoming Twitter data. Learn how to create a Zap for sentiment analysis with MonkeyLearn.

Connect Twitter Data with IFTTT

IFTTT means ‘if this, then that’. Like Zapier, this tool allows you to connect to different apps so that you can set an action when certain criteria is met. Use it to obtain Twitter data with zero lines of code.

Track Twitter Data with Export Tweet

Export Tweet allows you to track a keyword, hashtag or account in real-time, or search for historical data. However, the free version has limitations and we recommend upgrading to take full advantage of the platform.

Download your Data with Tweet Download

Tweet Download enables you to download the tweets from your own account, along with the replies and mentions. This is especially useful for brands that want to track which content works best with users, what are the main things that users claim about their product, etc.

Use The Twitter API

The Twitter API lets you access and interact with public Twitter data.

Use the Twitter Streaming API to connect to Twitter data streams and gather tweets containing keywords, brand mentions, and hashtags, or collect tweets from specific users.

Use the Standard Search API to get historical tweets published up to 7 days ago. Alternatives include historical search APIs (like Historical PowerTrack and Full-Archive Search), that can collect tweets from as early as 2006.

Connect with Tweepy

Tweepyis an easy-to-use Python library for accessing the Twitter API. Get started with Tweepy with this tutorial or dicover other popular libraries you can use with the Twitter API:

2. Prepare Your Data

Once you’ve gathered the tweets you need for your sentiment analysis, you’ll need to prepare your data. Social media data is unstructured and needs to be cleaned before using it to train a sentiment analysis model – good quality data will lead to more accurate results.

Preprocessing a Twitter dataset involves a series of tasks like removing all types of irrelevant information like emojis, special characters, and extra blank spaces. It can also involve making format improvements, delete duplicate tweets, or tweets that are shorter than three characters.

Check out this guide on how to prepare your data.

3. Create a Twitter Sentiment Analysis Model

MonkeyLearn is a machine learning platform that makes it easy to build and implement sentiment analysis. You can get started right away with one of the pre-trained sentiment analysis models or you can train your own using your Twitter data.

Either way, sign up to MonkeyLearn to gain access to the pre-trained models and the model builder.

Then follow this tutorial to perform sentiment analysis on your Twitter data.

Twitter Sentiment Analysis Tutorial

1. Choose a model type

Go to the MonkeyLearn dashboard, then click on the button in the right-hand corner: Create a model, and then choose ‘Classifier’:

MonkeyLearn's sentiment analysis builder: choose a model

2. Decide which type of classification you’d like to do

From the list of classifier type; click on ‘Sentiment Analysis’:

MonkeyLearn's model builder: choose a from sentiment analysis, topic classification, and intent classification

3. Import your Twitter data

The data you import will be your training data, used to train your machine learning model. Upload Twitter data from a CSV or Excel File, then select the columns you want to use:

Model builder: the step to import Twitter data by uploading an Excel or CSV file

Model builder: the step to select the column of Twitter data you want to analyze

5. Tag data to train your classifier

Now, it’s time to train your sentiment analysis model, by manually tagging each of the tweets as Positive, Negative, or Neutral, based on the polarity of the opinion. After tagging the first tweets, the model will start making its own predictions. You can correct them if the answer is not correct:

how to tag and train your sentiment analysis model to detect emotions in tweets

6. Test your classifier

Once you have trained your model with a few examples, you can paste your own texts to see how the sentiment analysis model classifies it:

testing the sentiment analyzer to see if it categorizes tweets correctly

MonkeyLearn provides different stats to measure the performance of your sentiment analysis classifier. These are accuracy, F1 score, precision, and recall. You can also find a Twitter keyword cloud featuring the most frequent terms for each sentiment.

If you are not able to see all the stats, it might mean that you need to tag more data. In this case, for example, the model requires more training data for the category Negative:

sentiment scores and results to show you how your sentiment classifer is performing

Keep in mind that the more training data you tag, the more accurate your classifier becomes. Another way to improve the accuracy of your model is to check all the false positives and false negatives and re-tag the incorrect ones. Here’s how:

a list of tweets and their corresponding polarity tags

4. Analyze Your Twitter Data for Sentiment

Now you’ve got a sentiment analysis model that’s ready to analyze tons of tweets! The next step is to integrate the Twitter data you want to analyze with the sentiment analysis model you just created. There are three ways to do this with MonkeyLearn:

  • Batch Analysis: Go to ‘Batch’ and upload a CSV or an Excel File with new, unseen tweets. The classifier will process all the tweets and provide a new file with the results of the sentiment analysis.
  • Integrations: there are several integrations available you can use to analyze new data with your sentiment analysis model. For example, you could use Google Sheets as an input for your data or Zapier to connect Twitter data  to MonkeyLearn.

available integrations: Zapier, Rapidminer, Google Sheets, Zendesk

  • MonkeyLearn’s API: if you know how to code, you can call MonkeyLearn’s sentiment analysis tools in Python (and other programming languages) to analyze new tweets.

API code snippet

To learn how to analyze your Twitter data in Python using MonkeyLearn’s API, check out this guide on performing sentiment analysis in Python

5. Visualize Your Results

Data visualization tools help explain sentiment analysis results in a simple and effective way.

Take a look at how MonkeyLearn Studio visualizes results from an aspect-based sentiment analysis on Twitter data. MonkeyLearn Studio is an all-in-one text analysis and data visualization suite, featuring ready-made business templates.

Perform sentiment analysis on your Twitter data right away, and filter your results in MonkeyLearn’s dashboard so you can hone in on negative or positive comments and make data-based decisions on the go.

MonkeyLearn Studio's analytics dasboard showing the results of an aspect-based Twitter sentiment analysis.

Other popular data visualization tools include:

You can use this free and simple Google platform to create interactive reports. There are more than 100 sources available to import your data, including CSV, Excel, and Google Sheets. Once you’ve designed your visual report, you can share it with other teams or individuals.

This is a business data analytics platform, created to manage all sorts of data within the different areas of a company. You can connect with different databases and create charts and data tables. Learn how to get started.

Defined as business intelligence and analytics software, Tableau allows you to work with a large number of data sources to create dynamic dashboards and compelling data visualizations. One of the best things about Tableau is that is very easy to use and doesn’t require any coding skills. However, it offers different types of products and some of them are targeted to developers.

Twitter Sentiment Analysis Use Cases

 

Twitter sentiment analysis provides many exciting opportunities. Being able to analyze tweets in real-time, and determine the sentiment that underlies each message, adds a new dimension to social media monitoring.

Here are some of the most common business applications of Twitter sentiment analysis.

Social Media Monitoring

Online reputation is one of the most precious assets for brands. A bad review on social media can be costly to a company if it’s not handled effectively and swiftly. 

Twitter sentiment analysis allows you to keep track of what’s being said about your product or service on social media, and can help you detect angry customers or negative mentions before they turn into a major crisis.

At the same time, Twitter sentiment analysis can provide interesting insights. What do customers love about your brand?  What aspects get the most negative mentions? This tweet, for example, indicates that fast shipping is one of the most valued aspects for this Amazon customer:

Positive tweet about Amazon Prime

Aspect-based sentiment analysis with Twitter can show you which aspects of your business need to be improved and what makes you stand out among your competitors.

Customer Service

Twitter has become an essential channel for customer service. In fact, a growing number of companies have specific teams in charge of delivering customer support via this social media platform. Prompt replies are key since 60% of the customers that complain on social media expect a response within one hour.

But how can you evaluate the performance of your customer support on Twitter? Twitter sentiment analysis allows you to track and analyze all the interactions between your brand and your customers. This can be very useful to analyze customer satisfaction based on the type of feedback you receive.

This tweet, for example, shows a disappointed customer after an interaction with Southwest Airlines’ customer support team:

Negative tweet about SouthWest Airlines

Market Research

Twitter is a major source of consumer insight. In fact, people use it to express all sorts of feelings, observations, beliefs, and opinions about a variety of topics. 

You can use Twitter sentiment analysis to track specific keywords and topics to detect customer trends and interests. Understanding what things potential customers like, what their behaviors are, and how this changes over time is essential if you are planning to launch a new product.

Here’s an example of how Twitter sentiment analysis was used to monitor 4,000 tweets that mentioned halal food. This information allowed researchers to identify different motivations for halal food consumption and segment their market into different types of consumers.

Twitter sentiment analysis can also help you stay one step ahead of your competition.  By identifying competitors’ pain points, you can focus on these areas when promoting your business.

Brand Monitoring

Whether you are launching a new feature on your platform, a site redesign, or a new marketing campaign, you may want to track customer reactions on Twitter. Taking action and making changes or improvements in real-time will help maintain customer loyalty.

Tweet about MailChimp's new branding

Political Campaigns

A huge part of Twitter conversation revolves around news and politics. That makes it an excellent place to measure public opinion, especially during election campaigns. Twitter Sentiment Analysis can provide interesting insights on how people feel about a specific candidate (and you could even track sentiment over time to see how it evolves).

During the US 2016 elections, we performed Twitter sentiment analysis using MonkeyLearn to analyze the polarity of Twitter mentions related to Donald Trump and Hillary Clinton. First, we were able to count the number of positive and negative mentions for each candidate during a period of time. This graph shows Trump’s tweets based on sentiment:

Trump tweet count by sentiment

In contrast, the following graph shows the number of positive, negative, and neutral mentions for Hillary Clinton:

Clinton tweet count by sentiment

Another relevant insight consisted of analyzing the tweets on specific dates, for example on the day of the presidential debate and observing negative or positive reactions, as well as the main keywords mentioned during that day.

Get Started with Twitter Sentiment Analysis

Sentiment analysis helps you monitor your customers emotions on Twitter and understand how they feel. It adds an extra layer to the traditional metrics used to analyze the performance of brands on social media, and provides businesses with powerful opportunities.

Yes, you could sort data by sentiment manually, but what happens when your data starts to grow? Sentiment analysis with machine learning is simple, fast, and scalable, and can provide consistent results with a high level of accuracy.

With a machine learning platform like MonkeyLearn, it’s simple to get started with Twitter sentiment analysis. Contact us today and request a personalized demo from one of our experts

Federico Pascual

June 7th, 2019