We’ve spoken before about the considerations and challenges of sentiment analysis, but used thoughtfully it can be a powerful tool for understanding your customers. It can be particularly useful for exploring social media and other situations where customers aren’t providing an explicit score or rating to indicate their satisfaction.
In this post we’ll examine how sentiment can be used to more deeply grasp your customers’ thoughts and feelings, especially regarding specific facets of your product. We’ll investigate online reviews of a well-known Australian travel insurance provider to illustrate. We also utilise review data of 5 other major brands for comparison.
To begin with, we’ve selected the key themes to explore by applying our understanding of the domain to the language model automatically generated by Kapiche. This is what we came up with:
Themes by % of reviews
A bird’s-eye view of customer pains
We begin with looking at a high-level visual representation of negative sentiment using the quadrant chart.
This view lays out themes according to both frequency and proportion of negative sentiment. The “Overall” line shows the amount of negative sentiment across all data. Looking to the right of the overall line we can intuitively gauge the most frustrating or problematic themes in the eyes of our customers; we can also assess and compare their frequency by vertical position.
See how the Kapiche sentiment tool works IRL 👆
Very quickly we can see that Missed flight is the most negative theme, but is also comparatively infrequent. Communications on the other hand is both reasonably frequent and significantly more negative than overall. Finally, despite Claims being only modestly more negative than overall, it is very frequent, which likely means it is a significant source of frustration for our customers.
Customer peace of mind and brand perceptions
Next we rank our themes by the level of positive sentiment that comprises them, giving us a view of the most positive themes.
Top 5 themes by % positivity
Peace of mind is overwhelmingly positive and mentioned by almost 9% of reviewers. Perhaps this isn’t a great surprise - a basic assumption would be that peace of mind is a key underlying motivator for purchasing insurance. However, here we can begin to validate our assumptions and quantify the extent to which this theme resonates with our customers. We can also probe much deeper by comparing the theme across different brands using the segmentation chart below.
Brands compared for “Peace of mind”
This chart is explained in detail in our knowledge base but the essential premise is as follows: “Our Brand” makes up about 15% of all reviews analysed, therefore we expect it to also make up the same proportion of reviews that specifically mention the theme Peace of mind. In this case our brand actually comprises almost 30% of reviews mentioning that theme, meaning it is significantly overrepresented. Compared with all other brands in this area we are clear market leaders.
Following on, we can see that a basic customer motivation we may otherwise take for granted actually varies greatly across different brands, and likely across different types of customers. In this vein, further investigation could profile the theme across different customer segments, giving us a detailed picture of brand reputation and perceptions across the market.
Communication issues, a key customer pain
Now we will take a view of themes by how much they impact the overall volume of negative sentiment, which means taking into account frequency and negativity together.
Top 5 themes by negative impact %
We are going to examine the theme with the second highest impact on negative sentiment, Communications. We explain impact in detail here. Practically speaking, it measures how strongly a theme skews the overall level of negativity across all responses. This allows us to prioritize the themes that most strongly drive negative customer perceptions.
To gauge the implications of this theme we can look to the review star ratings, which we were able to acquire with this data.
Star ratings compared for “Communication”
Immediately it is apparent that 1 star ratings are extremely overrepresented and both 4 and 5 star ratings are significantly underrepresented by reviewers that mention this theme.
To get a sense of why customers perceive this theme negatively, we should drill into it. Once we’ve done so, we can examine some of the negative comments left by customers.
Reading through some of the comments seems to reveal a consistent issue with communications not being received as promptly as expected, or in some cases not at all.
“Our flight dates were changed so we needed to adjustment on dates on our policy.Phone company to adjust policy and told that it was done and would be emailed out. Did not recieve email,phone again the date before our fly out date.”
“They didn't respond to my emails (the first they told me they got and didn't bother and the second they claimed they didn't get) and when I called and waited to speak to someone they told me that they would call me back within 2 business days as they were busy and obviously my claim was a low priority to them. They didn't return my call and I had to call back multiple times before I was able to speak to anyone they would deal with my issue.”
At this point we would generally seek to further clarify this lead by considering existing business knowledge and other data sources. For example, with customer survey data, we could investigate how the average number of contacts impacts key customer experience metrics. Or we could examine the comparative performance of different customer service teams.
Determining the biggest points of frustration for our customers is really the crucial first step in delivering a consistently great experience.
Comparing brand perceptions
For our final example, we’ll compare our brand to a top competitor, specifically looking at how different themes impact positive sentiment. In doing this, we’ll use data exports to show how the possibilities for exploring sentiment are extensive and not limited to any of the examples shown so far.
To begin, we add a dashboard filter for a particular brand and then download the corresponding data. We repeat this process for all of the brands (or other segments) to be compared.
Compiling the data in excel, it’s very straightforward to create a chart that highlights the differences between brands (we include the 10 themes with the biggest difference).
Themes compared by % impact on positive sentiment between brands
Themes are ordered by maximum difference in order to highlight the areas where we most diverge from our competition.
For both Prices/value and Easy to setup, we seem to be significantly outperforming our competitor. However, clearly our customers are much less positive about Claims; this theme isn’t positive for our competitor, but it’s only marginally negative. It would be worth taking a closer look into what frustrates our customers with the claims process, and where we might take lessons from other brands in the market.
We also see that our competitor has a modest edge over us in terms of Customer service/support. This would be another interesting theme to compare across all brands.
This kind of detailed comparison between brands is a powerful way to gauge the market landscape based on customer perceptions, allowing you to validate, quantify and galvanize your key differentiators. You can also use it to highlight lagging areas and protect your brand against both pre-existing and emerging threats.
Hopefully we’ve provided you with some ideas on how you can connect with the sentiment of your customers to both better understand their point of view and enrich your overall business intelligence. We’re passionate about using data to deepen understanding and make better decisions. On that note, we’d love to hear from you and explore ways to help you get the most out of your data.