Customer Experience (CX) is the key to business success. In fact, 81% of marketers interviewed by Gartner said they expected their companies to compete mostly on the basis of CX in two years’ time, making CX the new marketing battlefront.
Now, more than ever, it’s key for companies to pay close attention to Voice of Customer (VoC) to improve the customer experience. By analyzing and getting insights from customer feedback, companies have better information to make strategic decisions, an accurate understanding of what the customer actually wants and, as a result, a better experience for everyone.
But, what are customers saying about your brand? How can you provide a better experience? With these questions in mind, businesses are using tools that collect public reviews about their products (such as Capterra, G2Crowd, Google Play, and the like). However, they just end up with an overload of puzzling feedback that still doesn’t answer their questions, unless they devote hours of manual labor to analyzing this unstructured data.
Those days are over thanks to sentiment analysis… but what is it? Sentiment analysis is the automated process of understanding the sentiment or opinion of a given text. This machine learning tool can provide insights by automatically analyzing product reviews and separating them into tags: Positive, Neutral, Negative. By using sentiment analysis to structure product reviews, you can:
- Understand what your customers like and dislike about your product.
- Compare your product reviews with those of your competitors.
- Get the latest product insights in real-time, 24/7.
- Save hundreds of hours of manual data processing.
How can you get started with sentiment analysis? Here, we will show you how to run a sentiment analysis on product reviews with MonkeyLearn, in a step-by-step guide. Don’t worry, you don’t need weeks to analyze your data, just a couple hours will do… and then, your sentiment model will run automatically and smoothly in the background.
Here’s what we’ll cover:
Let’s get to it!
1. Gathering Product Reviews
Before you can use a sentiment analysis model, you’ll need to find the product reviews you want to analyze. This section provides a high-level explanation of how you can automatically get these product reviews.
Product reviews are everywhere on the Internet. You might stumble upon your brand’s name on Capterra, G2Crowd, Siftery, Yelp, Amazon, and Google Play, just to name a few, so collecting data manually is probably out of the question. And that’s probably the case if you have new reviews appearing every minute.
Thankfully, the bleak days of copying and pasting are long gone. Web scraping can help to automate and streamline this whole process. Web scraping is a set of tools used to collect information from across the Internet. These tools simulate how people surf the web to gather specific data from different websites. In essence, they automatically find what you would otherwise have to copy and paste manually from any given website.
Generally speaking, web scraping tools can be grouped into two distinct categories: visual scrapers and web scraping frameworks. Let’s take a closer look.
Visual Web Scraping Tools
Visual scrapers are specialized apps for building web scrapers with an easy-to-use, graphic user interface. To use these tools you don’t need to be a programmer or know how to code. Just follow the steps provided by each scraping tool to build your customized web scraper and you’ll be good to go.
Some of the most remarkable visual scraper tools include:
- Dexi.io: you can create a web scraper in just a few minutes with their simple-to-use UI. Check out this step-by-step video that will help you get started with your first scraper. Dexi.io has an integration for MonkeyLearn, with which you’ll be able to seamlessly synchronize the process of gathering and analyzing. Just click on the dashboard section ‘Add-ons’ and you’ll spot it right away.
- ParseHub: according to their team, ParseHub has been able to collect data from 80% of websites that their customers proposed. Similarly to Dexi.io, ParseHub’s user interface is easy to follow. You just enter the website and specify everything you want to scrape. That’s it! After your tool has finished scraping, you can choose to import your data via an API, Excel, Google Sheet or CSV file, and start using sentiment analysis to get insights from the data. Check out this video tutorialto learn how to scrape reviews from a website using Parsehub.
- Import.io: This scraping tool also has a simple user interface. By clicking on the items you’re interested in, the tool learns what information you want to gather from a given website. All that’s left to do is extract the data and download it as an Excel spreadsheet. Check out this tutorial to learn more about building a scraper with Import.io.
Web Scraping Frameworks
Now, if you are a developer or just happen to know how to code, you could use an open-source framework to build your own web scraping tool, and get product reviews from the web tailored to your needs.
These are some of the most used frameworks for web scraping:
2. Running Sentiment Analysis on Product Reviews
Now you have all the product reviews you need, automatically collected with your scraping tool… but how do you make sense of it?
Thankfully, we have the answer! You can automate product review analysis with machine learning. In just a few minutes, you can get the insights your team needs with MonkeyLearn. Our user-friendly platform enables you to build your own text analysis model without needing to know how to code or have experience in machine learning.
So, you have the solution. But how do you put it into practice? The answer is in this brief tutorial. We’ll cover how to build both your own sentiment classifier and aspect classifier.
First of all, we’ll create a sentiment classifier to find out how Positive, Neutral or Negative customers’ views of a product are.
Then, we’ll create an aspect classifier, not only to understand how (sentiment) customers are talking about a brand but what (aspect) they are talking about in their product reviews. Are they praising the UI/UX? Are they complaining about Customer Service?
Once we train these classifiers, you can use them to automatically analyze all of your product reviews with aspect-based sentiment analysis.
But before we do that, we need to know where an opinion starts and where it ends…
Preprocessing Data With Opinion Units
Ok, so you received a product review that reads ‘I think your UX is amazing, though I’m having some issues uploading files’. How would you classify it? Positive because it says ‘amazing’? Negative because it includes the word ‘issues’? Neutral because it has both positive and negative feedback? If we have problems classifying text manually, imagine how complicated it must be for a machine learning model!
Generally, a single review has:
- more than one sentiment,
- more than one aspect.
This is why dividing a long text into smaller units –what we call ‘opinion units’– can be a wise first step. In our previous example, an opinion unit extractor would return two opinion units for that product review:
- ‘I think your app is amazing’ → Which would be tagged as Sentiment: ‘Positive’ and Aspect: ‘UI/UX’
- ‘though I’m having some issues uploading files’ → Which would be tagged as Sentiment: ‘Negative’ and Aspect: ‘Issues & Bugs’
Dividing a full text into opinion units can simplify:
- Tagging data for training a sentiment or aspect model
- Creating a model that’s capable of carrying out an accurate analysis
- Mapping a sentiment to its corresponding aspect or aspects.
That’s why we’ve built an opinion unit extractor to run your product reviews through. Let’s take a look at how it works using a product review:
So, before training your sentiment and aspect models, upload the product reviews to this model to extract its opinion units. After, you can easily tag each opinion unit to train sentiment and aspect classifiers.
Whenever you want to analyze new data with the sentiment and aspect classifiers, remember to partition new reviews into opinion units before analyzing them with a model.
Keep in mind that you can choose to build your own opinion unit extractor for even more accuracy.
Now that we have that out of the way, let’s start with the sentiment classifier!
Create a Sentiment Analysis Classifier
Here you’ll learn how to create and test a sentiment analysis model for analyzing product reviews in six easy steps. Check it out:
1. Create a New Classifier
2. Select the ‘Sentiment Analysis’ option
3. Upload your Product Reviews
Next, you need to select how you want to upload data to train the model. Here, you should upload your product reviews as an Excel or CSV file:
4. Train your Model
Now, it’s time to teach your model which product reviews are positive, neutral or negative:
This may take some time, but it’s necessary for your sentiment classifier to learn the criteria that determines a positive, neutral or negative review:
Over time, your model will start to predict the sentiment behind each review. At first, it may not be 100% correct, but as you train it with more and more product reviews, you’ll see its confidence level increase.
One motto that definitely applies to machine learning is, ‘the more, the merrier’. The more data you tag, the smarter your model will be. It’ll make fewer mistakes and more spot-on tagging by identifying words and expressions that should be associated with positive, negative or neutral sentiments.
5. Test Your Sentiment Classifier
Once you’ve finished training your model, you can test it out to see how accurate your sentiment classifier is. Head over to the ‘Run’ tab, type a review in the text box (or paste it) and click ‘Classify Text’:
Not quite accurate yet? No problem. You can go to the ‘Build’ tab and continue training your model until it’s smart enough.
You can also check out the classifier stats subsection, to quickly understand how well your classifier is at making predictions, and which tags need improvement.
6. Put it to Work
Once your sentiment model is good to go, you can upload new product reviews and analyze them with the same sentiment analysis model to test its predictions!
Just go to the ‘Run’ tab, click ‘Batch’, and follow the steps to upload a CSV or Excel file with your reviews:
The sentiment classifier will analyze the reviews and give you another file with the predictions in return.
Because MonkeyLearn comes with various integrations, you can also analyze your reviews from third-party apps (such as Google Sheets, Zapier and RapidMiner) to get the sentiment predictions in just a couple of clicks:
If you know how to code, another option is to run this model with data from MonkeyLearn’s API. You just need to choose your favorite programming language:
Create an Aspect Classifier
Now that you have your sentiment classifier, you may feel like you still can’t identify what specific features are viewed in a positive or negative light. That’s when the aspect classifier makes its grand entrance. This machine learning model can categorize texts by topic, so, for example, you can divide the product reviews into Price, Product Quality and User Experience. How positively or negatively is each aspect viewed. By combining the results of a sentiment classifier and an aspect classifier, you’ll be able to figure it out!
Let’s see how to train an aspect classifier in this six-step tutorial!
1. Create the Aspect Classifier
2. Choose ‘Topic Classification’
Now, we are building an aspect classifier, so we need to click on Topic Classification. This is the kind of classification that we are interested in running:
3. Upload your Reviews
It’s time to upload a batch of reviews, to train your model and identify different topics or aspects in each piece of text:
4. Define your Tags
What aspects of your product would you like insights on? To illustrate, let’s go for Performance, Updates, and Account:
For your first models, it’s recommended to use a maximum of ten tags (you can always add more later).
5. Train your Model
Same idea as before! Take the time to classify reviews, by manually applying the appropriate tags to train your machine learning model:
In some cases, more than one tag may apply, and that’s ok! Just tag the sample with all the tags that you consider appropriate.
6. Test your Aspect Classifier
Like with the sentiment classifier, you can test your aspect classifier to see how it makes predictions on new product reviews, and understand if it needs to be improved or if it’s ready for showtime!
Remember: you can always go to the ‘Build’ tab and continue training the model to make it more accurate.
7. Put your New Aspect Classifier to Work
Now that your new aspect classifier is up and running, all you need to do is upload new data and let the model do its thing. Use the API, one of our integrations or upload a batch of product reviews that have already been analyzed by your sentiment classifier, and get the results of the aspect classification tool to get a clear analysis of your product.
Now you can discover how clients feel about specific product features!
3. Visualizing the Results
You have the reviews and you have the analysis results, but you want to share your findings with your team. They say a picture is worth a thousand words, but how do you transform the data into something visual? Fear not, for you have tools to aid you in creating awesome graphs and reports with your aspect-based sentiment analysis results!
Google Data Studio
So, imagine you want to create a visual report based upon your product review results. Maybe you’re thinking about including both aspect (Performance, Updates, and Account) and sentiment (Positive, Neutral, and Negative) classification results. Just follow these steps using Google Data Studio, Google’s user-friendly tool for creating data visualizations:
- Create a new report
- Connect to your data source (let’s say, Google Sheets containing the classification results)
- Tailor your visuals (pie charts, graphs, scatter diagrams etc)
- Share the visual insights with your team
To learn more about the ins and outs of Google Data Studio, check out these tutorials.
Like Google Data Studio, Looker allows you to easily connect to databases, such as Amazon Redshift and BigQuery to create beautiful data visualizations.
One compelling function of Looker is its filters: you can create a dashboard tile by aspect… but if you suddenly want to focus on the ‘Performance’ aspect, you can filter by ‘Negative’, ‘Neutral’ or ‘Positive’ sentiments.
You can easily share Looker reports, and customize your dashboard and data deliveries by scheduling to receive via email the latest updates daily, weekly or monthly.
For a step-by-step tutorial on how to use Looker, take a look at their tutorials on YouTube that will teach you everything from viewing and creating dashboards, to creating custom filters and merging results. Check out their YouTube tutorials.
Tableau is a data visualization tool, with a friendly drag-and-drop UI, used to create all the graphs you could possibly want. First, you’ll need to connect Tableau to your data source – a Google Sheet (cloud data) or an Excel file (file data). Here’s a great tutorial that will help you get started with Tableau
Other cool tools for data visualization include Klipfolio, which has dozens of integrations but requires a bit more training, for creating dashboards using Excel files, and Mode, a tool that also lets you interact with the dashboards and provides a cool integration with Slack.
Not sure whether you should invest in visual tools? This may nudge you in the right direction: If you read Sentiment analysis of Slack reviews using R, you’ll come across a rather dull table of numbers…
… which we converted into this guy:
This chart is much easier to understand (and it’s less tempting to scroll past the results). We can actually see them, not just read them. Visual tools can make communication easier and help you understand the results of your product review analysis. This means you can make the most out of your sentiment analysis, and get the insights you’re looking for.
It’s true. We sometimes get caught up in day-to-day tasks and forget to listen to what the client is saying. And it’s also understandable… we don’t want to fall behind on work. However, we do want to stay up to date and competitive, and this is easier said than done if your team has to read a never-ending list of product reviews from various sources. The thing is, customer experience is key to a company’s success, and with sentiment analysis, ‘not having time’ is no longer a valid excuse.
Once you have a trained a machine learning model, sentiment analysis can begin working smoothly in the background – analyzing incoming reviews, 24/7. Just like that, you will be able to view the results of thousands of analyzed reviews from different sources, make visualizations and share them with your team. That way you can see what to boost and what to lose without wasting any time.
You’ll no longer feel like you’re chasing rainbows when it comes to finding out what customers think about your brand!. Machine learning makes it easier to see the bigger picture within seconds, so that you can turn words into numbers, and numbers into actions.
Sign up to our Newsletter
Receive awesome Machine Learning posts and tutorials!