When a CS team builds a customer health score from scratch, the first inputs they reach for are almost always product usage metrics. Login frequency, feature activation, seats provisioned versus seats active, API call volume. These are clean numbers. They come out of the product database without ambiguity. They are easy to explain to an executive who wants to know why a score moved.
The problem is not that product usage is a bad signal. It is that health scores built primarily on product usage are measuring something real but incomplete, and the gap shows up at exactly the moment it matters most: late-stage renewal risk.
What product usage actually measures
Product usage tells you whether a customer is doing the thing you built. It does not tell you whether they are happy doing it, whether they think the product is reliable, or whether they are evaluating a replacement. These are all different questions, and they have different data sources.
Consider a scenario: a growing company is using your product daily, logging in with regularity, running the core workflow their ops team depends on. By any usage-based health score, this account looks healthy. What the usage data does not show is that the ops team lead has filed eleven support tickets in the last two months about the same intermittent sync issue, the last three of which received responses she described in the ticket thread as "not helpful." The usage numbers have not changed because the team is too embedded in the workflow to stop using the product while the problem persists. But she is already talking to procurement about alternatives.
This account will score healthy until the day the churn notice arrives. The usage signal is real but it is measuring the wrong thing at this stage of the relationship.
The sentiment dimension that usage data cannot capture
Customer sentiment in B2B is not primarily expressed through star ratings or NPS surveys. It is expressed through the way customers communicate with your team. The progression from collaborative problem-solving to frustrated documentation to silence is a real pattern, and it is recorded in support tickets, email threads, and call notes.
Sentiment data of this kind is structurally different from usage data. Usage tells you what happened. Sentiment tells you how the customer interpreted what happened and what story they are telling themselves about the product. A customer who hits a bug and writes a detailed ticket asking "is this a known issue and is there a workaround?" is in a different place than one who writes "this is broken again." Same event type, different interpretation, different trajectory.
The challenge is that capturing this dimension requires reading the actual text of customer communications, not just counting them. Ticket volume as a number is not the same as ticket language as a signal. A drop in ticket volume can mean either that problems stopped occurring or that the customer stopped expecting solutions. Those two interpretations have completely different implications for renewal risk.
Where standard sentiment analysis falls short for B2B
One approach CS operations teams sometimes try is applying sentiment analysis tooling to support communications. The theory is reasonable: if ticket language can carry signal, then running a sentiment classifier should surface it. In practice, this approach runs into a structural problem with B2B customer language.
Most sentiment analysis models are trained on consumer reviews, social media, and similar data sources where emotional valence is expressed directly. B2B customers communicate in a fundamentally different register. Professional email language is formal even when the sender is deeply frustrated. A customer who has lost confidence in your product entirely will still write in complete sentences with appropriate punctuation. Their frustration is expressed through what they ask versus what they assume, through subject-line compression, through the absence of context they used to provide. A sentiment classifier trained on consumer data will score this as neutral or positive.
B2B communication signals are relational and sequential, not lexical. The signal is in how communication patterns change over time within an account, not in the absolute tone of any individual message. That is a different kind of modeling problem, and it requires training on B2B customer communication specifically.
Building a health model that accounts for both dimensions
The framing that makes this clearer in practice: product usage and communication sentiment are early-warning signals for different problems at different stages of the customer lifecycle.
In the first 90 days of an account, usage data is the right thing to watch. Low adoption in the onboarding window is a product fit or onboarding execution problem, and it needs a product or CS response quickly. Usage signals are appropriate for that phase because the customer has not yet built strong opinions about reliability or trustworthiness. They are still forming them.
In the renewal window, which for most B2B contracts means the 60 to 90 days before the renewal date, the decision has largely shifted from "does this product work?" to "is this vendor a reliable partner?" That question is answered by the quality of support interactions, by how communication has evolved, and by whether the customer feels heard when they report problems. Product usage data does not speak to any of these.
A health model that addresses both dimensions needs two separate signal streams: one reading the product telemetry, and one reading the communication record. They should feed separate indicators, not be averaged together. An account can have healthy usage and deteriorating communication sentiment simultaneously, and averaging those into a single composite score would hide exactly the kind of risk that matters at renewal.
The question health scores should answer but usually do not
Most health score implementations are designed to answer: "How engaged is this customer with the product?" That is a meaningful question for the first half of a customer's lifecycle. It is the wrong question in the renewal window.
The question that renewal risk requires is: "Does this customer believe the product will continue to work for them, and do they trust us to fix it when it does not?" That belief and that trust are expressed in communication data, not in usage data. You cannot infer them from login frequency.
This is not a critique of the teams who built product-usage health scores. Product usage is what was measurable with existing tooling, and it is a real signal. The point is that it is not sufficient for the problem that matters most, which is predicting renewal outcomes in accounts that are already using the product regularly.
What to do with this if you are already running a health model
If your current health score is usage-weighted and you are seeing surprises at renewal in accounts that looked healthy by the score, the diagnostic question is straightforward: look at the support and email communication for the accounts that churned unexpectedly in the last two renewal cycles. Look at the ticket language from 60 to 90 days before the churn. If the language was shifting in the way we describe above and your health score did not reflect it, you are measuring the wrong thing in the renewal window.
Adding a communication quality signal to an existing health model does not require rebuilding everything. It requires treating support sentiment as a separate indicator with its own threshold and its own alert logic, sitting alongside the usage dashboard rather than being averaged into it. The two signals answer different questions. Keeping them separate preserves both answers.
That separation is the core design choice we made when building Sturdy. Usage data lives in your existing tools. The communication signal was not being read at the account level, in a structured way, in time to act on it. That is the gap we are filling.
