Use Cases

Sentiment Analysis of Survey Responses

Federico Pascual
by Federico Pascual

Sentiment Analysis of Survey Responses

A stumble may prevent a fall. We’ve heard this saying more than once but little did we know that it’s a rule we follow without realizing it when it comes to customer experience. When we get the latest batch of customer survey responses, we cringe at the sight of words such as ‘terrible’, ‘problem’ and ‘disappointing’ and try our best to make amends. The thing is that if we make the same mistakes again, we might not be able to prevent the fall — yes, a dramatic metaphor for customer churn — we need to be able to spot potential errors before they occur.

Are we getting too many negative responses? Why? How can we fix issues? What are we doing right and what have we improved over time? What are the most urgent issues to solve?

Assessing responses from customer surveys and creating a report that will give us the answers to these questions is easier said than done. It may take us hours, days or even weeks to go through all responses and find the root of a problem or the reason for a sudden increase in our customer churn. Nonetheless, we know it’s fundamental to delve into customer feedback since a customer that had a poor experience only has a 43% chance of retention for the following year.

So how do we take action on this seemingly Sisyphean task? How can we make sure we don’t teeter off the edge? Sentiment analysis can help us make sense of survey responses to automatically answer questions such as:

  • How many negative responses did we receive?
  • What aspects of our product do customers love?
  • What aspects of our product do customers hate?
  • Has a particular product feature improved?

The next step is learning how to do sentiment analysis on your survey responses. Don’t worry, it isn’t a time-consuming process. Just follow this guide and you’ll learn:

Let’s get to it!

What Is Sentiment Analysis?

Sentiment analysis is the automated process of making sense of a written opinion about any topic. By using machine learning, manually sorting through survey responses can be a thing of the past. A sentiment analysis model can learn to make predictions of new survey responses based on previous ones, and tag them as positive, negative or neutral.

This automated process can also be combined with aspect classification to create an aspect-based sentiment analysis model. An aspect classifier can pinpoint different topics or themes that are mentioned in a survey response, for example, Price, UX, and Customer Support. So, to learn how customers feel about a specific topic or product, you can combine a sentiment classifier and an aspect classifier. This way, you’ll learn if they are complaining about the price, praising customer support or feeling frustrated by bugs when using your product.

Why is it Important for Analyzing Surveys?

You’ve been working day in and day out to improve customer experience. The latest survey response batch just arrived and you start analyzing the qualitative data (aka the open-ended responses). But, this process never seems to end and you’re spending way too much time going through the open-ended data. Eventually, you find some interesting insights and share a report with your team… but a week has already passed, and you need to start doing the same thing all over again for the latest batch. How can you cut down those long hours of inputting information into spreadsheets, tagging it and propping yourself up with vats of coffee?

Sentiment analysis can reduce the number of hours spent on survey analysis to just a few minutes. This means that you can be at the top of your game with the latest customer insights and focus on taking action. Familiar with that feeling of being behind schedule? With sentiment analysis, the results will be ready in real-time and you’ll have a better understanding of what is going on with your customers.

Some of the main benefits of using sentiment analysis for processing survey responses are:

Consistent criteria:

We often get tired, disagree with each other and interpret things differently, so it’s not easy for a person, let alone an entire team, to tag responses consistently without any fluctuations. A sentiment analysis model doesn’t hesitate when tagging; once it has been taught a set of criteria, it will apply it consistently.

Real-time analysis:

Are customers upset with a price change you recently made? Did yesterday’s bug affect customer experience? You know that tackling problems and spotting opportunities is hard when you’re out of step with customer feedback. With sentiment analysis, you can get meaningful answers and proceed in a speedy way.

Scalability:

Thousands of survey responses are processed every week and it’s unrealistic for teams to deal with them manually. Long hours of manual work could be redirected to more human-dependent tasks. Sentiment analysis lets you do this automatically in a cost-efficient way – no coffee breaks needed!

How to do Sentiment Analysis of Survey Responses

Ok, now you get the gist of sentiment analysis of open-ended survey responses. But how do you apply it to your day-to-day work?

Sentiment analysis is no longer exclusive to developers or data scientists thanks to the advent of user-friendly and easy-to-use machine learning tools. Once you learn how to use them, you’ll stop wasting valuable time on tedious work. But first, let’s see how we can get there.

Gathering the Survey Responses

To do sentiment analysis the first thing you’ll need is data (survey responses). Not only is it necessary to train your machine learning model, it’s also what’s going to deliver valuable insights later on. But how can you get the data without manually copying and pasting it?

There are different ways to export responses depending on what survey tool you use. We’ll cover the most popular ones below:

SurveyMonkey

SurveyMonkey is one of the most popular tools for creating online surveys. It offers the possibility of exporting survey responses to a CSV or XLS file for paying clients. In order to do it, you need to go to the ‘Analyze Results’ section of the survey, click on ‘Save As’ and ‘Export file’. There you will have to choose between ‘All Summary Data’ (to organize the survey by question) or ‘All Responses Data’ (to organize the survey by respondent). Don’t forget to select ‘Include Open-ended Responses’, since this is what you’ll be analyzing with a sentiment analysis model:

Typeform

Another popular tool for creating online surveys is Typeform. Unlike SurveyMonkey, Typeform allows all users (free and paid) to download responses on a CSV or XLS format. Just go to your Typeform survey and click on ‘Download all responses’:

Then, Typeform will ask you to choose the format you want to use to download the survey responses. Choose the option you want and then click the ‘Download’ button:

And voilà! You’ll have your survey data on your computer.

Google Forms

You can easily open up your Google Forms on a Google Sheet by going to the ‘Responses’ tab and then clicking on the Google Sheet icon:

Now, download the responses as a CSV or Excel file by going to ‘File’ > ‘Download as’:

And you are good to go.

Creating a Sentiment Analysis Classifier

Now that you have your survey responses on a CSV or Excel file, the next step is getting your hands on your very own sentiment analysis model. Once trained, this model will be able to automatically analyze new survey responses and make predictions whether they are expressing positive, neutral or negative opinions, enabling you to get those insights you’re craving for.

Follow our simple tutorial to train your sentiment analysis model with your data using MonkeyLearn – a platform for analyzing text with machine learning.

Let’s take a look!

1. Create a model:

Once you’ve created a MonkeyLearn account, you can go to the dashboard and click on ‘Create Model’ in the top right-hand corner. These options will appear:

For the purpose of analyzing sentiment within text, you’ll need to select ‘Create Classifier’.

2. Now click on Sentiment Analysis:

3. Import your survey responses:

It’s time to choose the source of your survey responses, which you will use as training examples for your sentiment analysis model. You can upload survey responses from a CSV or Excel file, or use one of the third-party integrations:

4. Start training your model:

It’s time to start manually tagging survey responses as positive, neutral or negative, so that your model can begin to learn from your criteria. Just click on the right tag, select ‘Continue’ and a new response will appear:

Soon enough you’ll see that the model will begin to make predictions. If they aren’t accurate, that means your model needs further training. The more examples that you feed it, the more precise it will become.

5. Testing your model:

How can you know if your model is ready? There’s only one way to be sure: by testing it. Go to the ‘Run’ tab and write a series of mock reviews in the text field to see if your model can accurately analyze them. If your model is making the right predictions, then you know it’s working.

If the predictions aren’t quite there yet, continue training your model by going to the ‘Build’ tab and tagging more examples. Good things come to those who wait, so be patient!

You have other ways of checking the accuracy of your machine learning model. For instance, if you click on ‘Build’ > ‘Stats’, you can see the accuracy and F1 score for the overall model:

And precision and recall for a specific tag:

Don’t worry, all these terms are explained here, along with how you should be using them for improving your model. Take into account that in order to see performance stats, you need to have tagged at least four texts per sentiment.

In the stats section, you can also check out the keyword cloud, which is a good way to check which words are more easily associated with their corresponding tag (the words will appear bigger).

Have you been training your model for a while, without any luck? Make sure you check to see if there are any false positives or false negatives by going to ‘Build’ > ‘Data’. Spotting false positives and negatives and re-tagging them can help you get your model back on track, since incorrect tagging could be disrupting your sentiment analysis predictions and ‘confusing’ your model.

6. Start using your model!

Once your sentiment analysis model is a master of accuracy, you can upload new survey responses to get predictions! In order to do this, you have three options:

The first option is clicking on ‘Run > ‘Batch’ and uploading a CSV or Excel file to run a sentiment analysis on new survey responses. Just click on ‘New batch’, select the file on your computer and the sentiment analysis model will analyze the responses within seconds:

When complete, the survey responses will download to your computer with their corresponding predictions.

Another option for using your model is going to the ‘Integrate’ section and selecting one of the available integrations:

Last but not least, if you know how to code, MonkeyLearn’s API is readily available to analyze your data programmatically:

Creating an Aspect Classifier

Now that you have your sentiment analysis classifier, it’s time to work on the second model for doing an aspect-based sentiment analysis of the survey responses: an aspect classifier! Creating an aspect classifier is pretty similar to creating a sentiment analysis classifier, with some minor differences. Let’s go over them.

First of all, you’ll have to pick ‘topic classification’ instead of ‘sentiment analysis’ and select your data source to train the model. Same thing so far. But here comes the difference: after uploading the data, you’ll have to define your tags. In other words, what aspects or topics you want to detect within the survey responses:

Pick your tags depending on which topics you would like to get insights on. Keep in mind that the more tags you have, the more data you will need to upload to train your model.

After that, it’s time to train your model. Once its predictions are accurate, you can upload new survey responses from a CSV or Excel file, use one of MonkeyLearn’s integrations or the API, and get your aspect predictions in no time.

Are you hoping to combine aspect and sentiment analysis? You can upload the CSV or Excel file with the survey responses and their corresponding sentiment analysis predictions. The tool will run the analysis and give you a table with all the sentiment and aspect tags. The same goes for integrations. First, you run the survey responses through your sentiment analysis model, then through the aspect classifier, and you’re good to go!

Automating the Whole Process with Zapier

Ok, you know how to create some useful models to analyze your survey responses automatically, but you still have to do all the clicking, uploading and downloading yourself… could you also automate this process? Luckily, it’s possible to automate the whole process with a tool like Zapier! But what is it?

Zapier is a tool that automates tasks between apps via a seamless integration (called a zap) that you can easily set up. Basically, you set a trigger that causes an action to pass data back and forth.

It’s all very abstract so far, so let’s illustrate it. Say you get your surveys from SurveyMonkey. In this case, you could do the following:

  • Set a new survey response from SurveyMonkey as the trigger.
  • Set it to do the sentiment analysis of that survey response via MonkeyLearn as the action.
  • Set it to do a survey response aspect classification via MonkeyLearn as the action.
  • Finally instruct the zap to save the survey response and the aspect-based sentiment analysis results on a tool of preference, say Google Sheets.

Great in theory, but how does it come about? The process (or zap, as they call it) is quite straightforward since Zapier has an integration for MonkeyLearn.

1- Log into Zapier or create an account. You can try out the free trial version with multi-step zaps for 14 days.

2- Create a new zap by clicking on the orange ‘Make a Zap!’ button on your dashboard.

3- Now, you’ll have to set your trigger app. In this case, SurveyMonkey, the app that receives our survey responses. But what will be the specific trigger? An incoming survey response. You can select ‘New Response Notification with Answers’ in the ‘Select SurveyMonkey trigger’ section:

Remember to click on ‘Connect to Account’ to get your surveys from SurveyMonkey.

4- Now that we have new responses triggering our zap, we need to set our first action. Go on and add a second step to your zap and select MonkeyLearn as the app. Then, select ‘Classify text’ as the specific action. You’ll also have to click on ‘Connect to Account’ and paste your MonkeyLearn API Key:

Once that’s done, click on ‘Continue’ and select your sentiment analysis model as the classifier you want to use, and the SurveyMonkey response from step one of the zap as the text to analyze:

Keep in mind that to get your surveys analyzed by aspect, you’ll have to follow these steps once more, and select the topic classifier instead.

5- To get the results on Google Sheets, you should add an additional step to your zap after sentiment and aspect analysis. Just click on ‘Add a step’, choose Google Sheets as your desired app and connect your account. Click on ‘Continue’ and select the spreadsheet you want to use. Finally, select ‘Continue’ and your analysis will be automatically added to your Google Sheet!

6- Click ‘Finish’, name your zap, and turn it on!

Data Visualization of the Results

Now, thanks to automatic sentiment analysis on your survey responses, you have the insights about how customers view your products or services.

The thing is, long tables with numbers have a hard time conveying the information. Just picture scrolling through a long list of survey responses with numbers… how would you make sense of that without dozing off? That’s why most companies use data visualization tools to create very visual reports that can help your team grasp the information easily.

Here, we’ll cover the most popular data visualization tools to visualize your results. Read on!

Google Data Studio

With Google’s visualization tool you can easily cross your aspect classification with your sentiment analysis results to create all kinds of graphs and reports. To start off, you can connect Google Data Studio to your source of results, whether it’s an app such as Google Sheets, a CSV or an Excel file. just select it and click on ‘Connect’ and ‘Add to report’.

A blank sheet will open up: this means it’s time to pick the chart you want to use from the toolbar above. Once that’s ready, you can easily share it with your team and choose who can view it and who can edit it.

In case you want in-depth tutorials to get the hang of Google Data Studio, check these out.

Looker

Besides providing easy integrations, Looker comes with an alluring feature: you can zoom in on specific data within a graph. For example, by selecting your filters you can get a more detailed view of the negative and positive responses of one aspect, let’s say, ‘UX’. Moreover, you can interact with data in real time.

To get a closer look at Looker, there are really user-friendly tutorials available on their Youtube channel.

Tableau

Besides offering an easy-to-use drag-and-drop UI to build your graphs and charts, Tableau also comes with an easy connection to your data source. MonkeyLearn’s analysis results are downloaded as CSV or Excel files which you can upload to Tableau to create beautiful charts and visualizations.

If you decide to check this tool out, it may be a good idea to check out this tutorial.

There are more data visualization tools that may suit your needs, like Mode Analytics. This tool provides a cool interaction with the charts and, more importantly, comes with a Slack integration (that can definitely come in handy if your team communicates via Slack). Another tool is Klipfolio, but be warned that it’s easier for cloud integrations. If you want to upload an Excel file, you may need to do a bit of homework.

Use Cases & Applications

Ok, so you’ve managed to reduce working hours and receive valuable insights within minutes. That’s a huge success. But what can we do with the information we gain from sentiment analysis? How would we apply the survey response predictions to our business? There’s more than one answer.

Tackle problems before they grow

To improve your customer experience, you can get a notification when there’s an incoming negative survey. How could this be helpful? Besides making sure the client feels listened to and assisted, you can detect a big problem and tackle it before it turns into a major crisis: Imagine you get a survey response saying ‘The app doesn’t seem to work right, it didn’t let me add my credit card and upgrade my plan’. You can take action right away and turn things around, turning a bad experience into an exceptional one.

Get Better Customer Insights

It’s sometimes hard to spot where there’s room for improvement or where you’re spot-on. With aspect-based sentiment analysis, you can pinpoint where changes need to be made.

Say you have a bunch of negative comments in your survey responses. By running an aspect-based sentiment analysis on them, you may notice that most are tagged ‘Ease of Use’. What’s wrong? By zooming in on those survey responses, you can shed light on the problem and help your UX team take action.

Route Feedback to the Right Teams

A new bunch of survey responses has just arrived. Having all of the teams scroll down to find the ones they can address or are relevant to them can be quite inefficient… and discouraging.

With machine learning, survey responses are automatically tagged and routed to the relevant team. A negative response about Billing arises, for example The relevant support team will receive it automatically and solve the problem without any waiting around. A customer complaining about a major bug? This one will be routed to the dev team so they can take action right away.

Wrap-up

It’s no easy feat staying on top of all the survey data that is created on a daily basis, but it’s also unavoidable if you want to succeed with your product or service. Open-ended survey responses provide valuable insights that can help you understand your customers’ needs and frustrations and prioritize accordingly to provide the best possible experience

With sentiment analysis, you can now measure what used to be immeasurable for businesses. It’s no longer something that requires hours of manually reading responses to extract value. Within a few hours, you can share valuable insights with your team and start making decisions that take your business to the next level.

Interested in learning about how you can use sentiment analysis on survey responses? Request a demo and get a personalized demo from one of our experts.

Federico Pascual

Federico Pascual

COO & Co-Founder @MonkeyLearn. Machine Learning. @500startups B14. @Galvanize SoMa. TEDxDurazno Speaker. Wannabe musician and traveler.

Notification

Have something to say?

Text Analysis with Machine Learning

Turn tweets, emails, documents, webpages and more into actionable data. Automate
business processes and save hours of manual data processing.