What Are Open-Ended Questions & How Can You Analyze Them?

Gathering information and insights from customers, potential employees, and targeted demographic groups is important to understand their points of view about your company and products.

And what’s one of the most effective ways to do this? Ask questions of course – whether in person, on phone calls, or online surveys.

Open-ended questions are usually the most valuable way to get the data you need because they provide more detailed information.

In this post, you’ll learn everything there is to know about open-ended questions, from how to write them to analyzing them for fine-grained insights.

What Are Open-Ended Questions?

Open-ended questions are questions that can’t simply be answered “Yes/No” or with a fixed or multiple-choice response. Open-ended questions are worded to require an explanatory response in order to find out new, often subjective, information from the responder in their own words.

Responses that require more than one word will help you find out the “Why?”, “How?,” and “What do you think?”, which can offer far more interesting insights and avoid shutting down discourse.

For example, the open-ended question "What do you like best about our product?" elicits a descriptive response in the customers own words, while “Do you like this product” will be answered with a simple “Yes” or “No”, often leaving you in the dark as to why a customer likes or dislikes a product.

Let’s take a closer look at the difference between the two types of question, so that you can gain the insights you need.

Open-Ended vs Close-Ended Questions

Close-ended questions are questions that can only be answered with one of a preset number of responses: “Yes/No,” predefined multiple-choice options, or scaled, e.g., “On a scale of 1 to 10 how happy are you with this product?” Close-ended questions gather quantitative data.

Open-ended questions, on the other hand, gather qualitative data. They lead to more detailed and valuable information because responders are using their own language to explain their ideas and feelings. Open-ended questions can lead to free form answers and actionable insights that the questioner may have never even considered.

Common examples of surveys with both close-ended and open-ended questions are Net Promoter Score (NPS) and customer satisfaction (CSAT) surveys.

NPS, for example, calculates customer loyalty from a simple close-ended question: “How likely from 0 to 10 are you to recommend this product or service to a friend or colleague?”. Then, there’s the option to follow up with an open-ended question, to dig into the details: “Why did you choose the score you did?”

When to Use Open-Ended Over Closed-Ended Questions

Asking the right questions will help you reach your goals, whether you need to improve user experience, generate more leads, or develop your product roadmap.

A closed-ended question will often help you detect if there is a problem in the first place. Text visualization tools can be used to show trends at a glance. For example, one-word answers to the closed-ended question in an NPS survey “How likely are you to recommend our service or product?” can be plotted on a graph to show the number of customers that responded with a positive, moderate, or negative answer. If answers are overwhelmingly negative, you’ll want to find out why.

That’s where open-ended questions come in.

Open-ended questions are ideal for customer surveys to better understand the voice of the customer (VoC) and follow the entire customer journey. They can help businesses find out:

  • What customers love about your business?

What does your business already do successfully that makes customers happy? Finding your consistent strong points can maximize marketing and increase sales.

  • How/what you can improve?

It may be harder to face than the positive, but no one knows the pain points of your business better than your customers. You just have to ask. In fact, leading product managers say that over 50% of their new products and features are motivated by customer feedback.

  • Where have you failed the customer?

Why are customers leaving your company or where specifically have you failed? Maybe you simply promised too much. Improve customer retention by finding out what you can do to meet their expectations – which is especially poignant when it costs 5 times more to acquire new customers than to retain existing ones.

In a nutshell, open-ended questions reinforce close-ended questions because you:

  • Get answers in the responders’ own words. Close-ended questions don’t offer any nuance. Open-ended questions get to the feelings and emotions behind the response.
  • Discover information you may not have even considered. Create new product features or discover new applications or use cases for your product you had never thought of.
  • Deeper, qualitative data. Close-ended questions offer statistics and numbers, which can be helpful to find out “What is happening,” but open-ended questions can help you understand “Why it’s happening.”

Examples of Open-Ended Questions vs Closed-Ended Questions

Think about how you can word your questions to maximize the information you’re likely to receive from the responses.

Below are some examples of open-ended and close-ended questions:

Close-Ended QuestionsOpen-Ended Questions
From 1 to 10, how would you rate your experience?How do you feel about your experience with us?
Did our product/service meet your needs?What can we do to improve our product/service?
What were your favorite features? [A. B. C. D.]How do the features help you to achieve your goals. What features are we missing?
Is there a product/service you prefer to ours?How does our product/service compare to the competition?
Do you think you’ll use our product/service again?What would make you want to use our product/service again?
Would you recommend our products/services?What would you say to someone about our products/services?
Did our customer service meet your needs?What would you say about our customer service?
Were you satisfied with your experience overall?What else do we need to know?
Did our employees meet your needs?What can our employees do better to meet your needs?

Now for some tips on how to ask open-ended customer survey questions:

  • Dig into the negative

While negative feedback may be harder to swallow, it’s usually much more useful to guide you to necessary changes. Ask questions about aspects of your business that customers are unhappy with, so you know how to improve them.

And when you receive negative responses, it’s important to close the customer feedback loop by letting customers know you’ve implemented changes or are simply acknowledging their concerns. Just letting your customers know you’re listening is a huge step toward keeping them happy

  • Don’t ask leading questions

Don’t ask questions that lead the respondent in a particular direction. Don’t assume you already understand your customers’ points of view. That’s the whole purpose of open-ended questions. You want candid, unvarnished opinions.

  • Prompt longer, more detailed responses

Ask questions starting with "why," "how," and "what if?," like you can see in the examples above, to elicit more detailed responses. That way, your customers will feel free to leave as much, or as little, information as they need to.

  • Keep your survey short

Nail down your questions to only the information you need to find out, or it could lead to survey fatigue and respondents will just want to get it over with – leading to bad data.

How to Analyze Open-Ended Questions

It’s clear that open-ended questions offer more information and more in-depth results. But they’re harder to analyze, so a lot of companies stick to close-ended questions that can be calculated easily in spreadsheets. However, text analysis with machine learning has changed the game.

SaaS tools, like MonkeyLearn, can automatically analyze thousands of surveys – and all manner of customer feedback, so you don’t have to spend dozens (or hundreds) of hours on manual survey analysis. Just four steps to getting real, actionable insights from your open-ended questions:

  1. Collect your data: use online survey tools
  2. Structure your data: pre-process responses
  3. Analyze your data: machine learning takes the pain out of data analysis
  4. Visualize your results: data visualization tools help get the most from your data

1. Collect your data

Online survey tools, like SurveyMonkey, Typeform, and Google Forms simplify the process of creating and sending open-ended surveys. They offer easy-to-use templates, so you can customize your surveys with just a few clicks.

2. Structure your data

With online survey tools, you can output your customer feedback data to a CSV or Excel file to set it up for automated analysis. From there, you just need to do a little data cleaning, so that machines can process it.

3. Analyze your data

Here’s where we really get to see machine learning at work. MonkeyLearn can easily be connected to survey tools via the MonkeyLearn API, to help automate the data analysis process. Or, via Monkeylearn’s integrations with Zapier, Zendesk, and Google Sheets. Once you’ve connected MonkeyLearn to your data, you’ll need to choose the type of analysis you want to perform:

  • Sentiment analysis – to automatically classify your opinion units as Positive, Negative, or Neutral.
  • Topic analysis – to classify responses by “aspect” (category, topic, feature, etc.).
  • Aspect-based sentiment analysis – both of the above combined, to know what aspects of your product or service are performing positively or negatively.

Sign up to MonkeyLearn to discover other types of text analysis you can perform and check out our tutorial on how to perform text analysis with machine learning.

4. Visualize your results

Data visualization tools, like MonkeyLearn Studio, show your results in a striking, easy-to-understand visual dashboard, so you can see your fresh insights in broad strokes or minute detail.

The MonkeyLearn Studio dashboard showing multiple text analysis results together.

Check out the MonkeyLearn Studio public dashboard and click around to see how it works. It’s a powerful, all-in-one tool to take you from data collection to analysis to visualization, all in a single dashboard.

Wrap Up

It’s clear that open-ended questions and surveys can help dig into customer feelings and opinions much deeper than simple “Yes/No” or multiple choice questions. The analysis is also much more complex, but machine learning text analysis tools, like MonkeyLearn, can walk you through the process to save time and money.

And once you have your tools set up in MonkeyLearn Studio you can analyze your open-ended responses constantly and in real time for immediately actionable insights.

Take a look at MonkeyLearn’s suite of text analysis tools to see what you can do beyond aspect-based sentiment analysis. Or request a demo and we’ll be happy to walk you through how to analyze your open-ended questions.

Tobias Geisler Mesevage

January 11th, 2021