How Anstice analyzed 14k survey responses to gain super-rich insights

About Anstice

Anstice is a research and marketing communications consultancy that uses behavior science-based approaches to inform and solve business problems. Their services include insights and stakeholder engagement, digital marketing, public relations, issues management, design, customer experience, and marketing and brand strategy.

The story

Anstice regularly carries out quantitative and qualitative research for their customers, helping them to garner opinions about their brand, products, and project proposals.

Anstice recently sent out a survey with SurveyGizmo to gauge public opinion around building a large infrastructure project. Response to the survey was far more successful than they had imagined, and they needed a more scalable solution to manage the unexpected volume of qualitative data and gain rich insights.

The Challenge

Anstice expected to receive about 3,000 responses, but they ended up having to sort 14,000 responses for each open-ended question.

• The main open-ended question asked respondents what the new project would mean to them. This offered some very rich insights, but many of the 12,000 responses were over a paragraph long and Anstice needed a quick and simple way to analyze this data

• They also needed to correctly tag as many responses as possible and reintegrate the results back into SurveyGizmo so they could significance-test the quantitative questions with psychographic data, in addition to the usual demographic information.

The Solution

• Using MonkeyLearn, Anstice trained text analysis models to classify each survey response to the project, using tags developed by using MonkeyLearn’s workflow for a grounded theory approach.

The first text classification model analyzed the 14,000 responses and returned 12,000 that were successfully tagged, and a further refinement of the model captured the remaining responses.

However, that left around 2,000 responses that the model wasn’t able to classify, so the team at Anstice trained a second model using the remaining “untagged” responses.

Once the MonkeyLearn models had tagged all of the responses, Anstice used the findings to inform recommendations to the infrastructure’s planning and design team. They also reintegrated the data back into SurveyGizmo for further analysis, using the coded qualitative themes as psychographic categories for further significance testing.

Testimonial

“What I really appreciate about being able to work with MonkeyLearn is I can ask more open-ended questions than I normally would in a survey, which is really liberating. This is also enabling us to examine the data on a psychographic as well as demographic data, and it's already making a world of difference in terms of getting some super-useful insights.”

Mark Szabo

Director of Insights and Engagement at Anstice