One of Kapiche’s most powerful features is the ability to identify the Impact that specific themes have on overall scores, such as Net Promoter Score (NPS), Sentiment, and CSAT (Customer Satisfaction Score).
Contents:
How Kapiche Calculates Impact Scores
Kapiche calculates Impact by comparing your overall score (NPS, Sentiment, or CSAT) with a hypothetical scenario where all responses related to a specific theme are removed.
Formula:
1. Calculate the overall score
First, we calculate your total NPS, Sentiment, or CSAT Score using all responses in your dataset.
2. Remove responses related to a specific theme
We then recalculate your score as if responses associated with a particular theme did not exist.
3. Determine the difference
The difference between your original and new score is the Impact Score for that theme.
Examples
Example: Impact on NPS
Let’s say your overall NPS is +50, and you analyze a theme called “Delivery Time”. If we remove all responses that mention “Delivery Time”, and your NPS drops to +40, then:
Impact Score for “Delivery Time” = +10
This means Delivery Time is positively impacting your NPS by 10 points.
If the recalculated NPS increased after removing “Delivery Time,” the impact score would be negative, meaning it is dragging NPS down.
Example: Impact on Positive Sentiment
Kapiche allows you to measure the impact of a theme on Positive Sentiment, Negative Sentiment, or Mixed Sentiment separately.
Let’s say your overall Positive Sentiment score is 60%, meaning 60% of responses contain positive sentiment. You analyze a theme called “Product Quality”, and when removing responses mentioning this theme, your Positive Sentiment score drops to 50%.
Impact Score for “Product Quality” = +10 percentage points
This means mentions of Product Quality are driving Positive Sentiment up by 10 percentage points.
If the Positive Sentiment score increased after removing the theme, the impact score would be negative, indicating that the theme was reducing positive sentiment.
Example: Impact on CSAT (Top 2 Box Method)
Many organizations measure CSAT using a Top 2 Box method, which calculates the percentage of respondents who rated their satisfaction in the top two categories (e.g., selecting a 4 or 5 on a 1–5 scale).
Imagine your CSAT Score (Top 2 Box %) is 82%, and you analyze the theme “Checkout Process”. If removing all responses related to this theme changes CSAT to 78%, then:
Impact Score for “Checkout Process” = +4 (or +4 percentage points)
This means that feedback about the Checkout Process is positively contributing to CSAT, with a 4 percentage point increase in Top 2 Box CSAT when this theme is included.
A negative Impact Score would indicate that the theme is bringing the CSAT score down.
Weighting: How Does Frequency Affect Impact?
A common question is: “How does the frequency (# of mentions) of a theme affect the Impact Score?”
More frequent themes tend to have a larger potential impact, but frequency alone does not determine the score.
Kapiche does not simply assign impact based on frequency—instead, we look at how the scores of respondents mentioning a theme differ from the overall score.
Example:
Theme A appears in 500 verbatim responses and has an NPS impact of +5.
Theme B appears in 50 verbatim responses but has an NPS impact of +10.
Even though Theme B appears less frequently, it has a larger impact because the respondents mentioning it have significantly different scores compared to the overall dataset.
Key Takeaways on Weighting:
A theme mentioned frequently but with neutral impact will not shift the score significantly.
A theme with fewer mentions but a strong correlation to a high or low score can have a large impact.
Impact is not just about how often a theme appears—it’s about how much it influences the score compared to the overall dataset.
Is Impact on NPS an Exact Science?
No. They estimate potential influence but don’t guarantee an exact NPS change if a theme is improved. The number is based on patterns in past data, but real-world NPS shifts depend on multiple factors, such as:
Changes in customer expectations - What matters to customers today may not have the same impact in the future.
External influences - Market trends, competitors, and industry changes can affect NPS beyond internal efforts.
How well fixes are implemented - A change might be made, but if it’s not executed well, customer perception may not improve.
Other themes contributing, improving or declining at the same time - NPS is influenced by multiple factors, not just one theme in isolation.
Additionally, NPS changes take time. Customers often need repeated positive experiences before their perception shifts enough to impact the score.
Think of Impact as directional insight, not a prediction—it highlights areas with the most potential to influence NPS rather than guaranteeing a specific movement in the score.es.
Should Impact Be Used in Reporting?
Yes, but with the right expectations to avoid misleading stakeholders.
How to communicate Impact effectively:
DO: Use it to prioritize areas that influence NPS the most.
DON’T: Treat it as an exact forecast (e.g., “Fixing Service will increase NPS by exactly 2.5 points”).
How to set the right expectations:
Impact shows potential influence, not a guaranteed outcome.
Fixing one issue doesn’t guarantee a higher score - other themes also contribute.
Customer perception shifts over time and depends on multiple interactions.
A simple way to explain it to stakeholders:
“Improving this theme has the potential to positively impact NPS, but since multiple factors influence the score, we use this as a guide rather than a prediction.”
Summary:
Impact Scores measure how much a theme influences NPS, Sentiment, or CSAT by comparing overall scores before and after removing that theme.
A positive impact score means the theme is boosting the overall score, while a negative impact score means it is lowering it.
Frequency matters, but the true driver of impact is how much the score differs for responses containing that theme compared to the overall dataset.
Impact isn’t a forecast - it’s a directional indicator to help prioritize areas for improvement.
Use Impact in reporting carefully, ensuring stakeholders understand it as a guide, not a guaranteed result.
This approach helps teams focus on themes that truly shape customer perception and experience, making insights more actionable. Let us know if you have any questions!