In a recent webinar, Jill Rosow, one of Lucidworks’ superstar data scientists, shared how customer support executives can marry quantitative and qualitative data to solve some of the most challenging customer service pain points. Rosow says we should be treating our internal data as a mystery that deserves thoughtful investigation.

We generally have ideas and assumptions about our respective businesses, but once you start delving into the data and peeling away the layers of customers’ interactions with our support assets, you discover a whole new perspective.

How Customer Experiences Can Shape Decisions

In the webinar, Rosow presented a recent example she experienced while making an important decision about pet insurance for her new puppy. She was faced with three distinct scenarios when she interacted with two separate providers.

Examples of customer service scenarios.

These customer service scenarios exemplify the wide berth of what we can experience when we’re contacting a company.


Customer Service Fail

In the first scenario, she tried to find the answer she was looking for by self-servicing in the company’s FAQ—which she quickly discovered did not exist. This led her to an interaction with a chatbot that couldn’t answer her question and she was punted over to an agent. After more than an hour, a couple of transfers and an eventual call drop, she gave up on trying to resolve her inquiry altogether and decided to try a different provider.

Too Slow, But Successful

In the second scenario, with the second provider, she did reach a resolution with two separate interactions:searching in an FAQ, then sending an email and receiving an answer. While the time to resolution was longer than she would have liked, she did find what she was looking for.

Customer Service Win

She was later impressed when she had a follow-up question about her account. When she called, the second provider had all of her previous interaction data on hand, the agent was able to predict the issue she was facing and she got exactly what she was looking for without a transfer or dropped call. It was an easy resolution with one human interaction and the agent was able to address her concerns. She then received a detailed email walking her through the experience once more, so she had a record on file for accountability.

She was so relieved at this third interaction, she actually cancelled her account with the first pet insurance provider and she remains loyal to the second. A slam dunk for the customer service experience on all accounts.

Measuring the Health of Customer Success

What can we learn from these touchpoints? Let’s first think about the measurable and quantitative KPIs that are the standard in the customer service industry so we can create some benchmarks as we explore what was successful – and what wasn’t.

These are the main quantitative KPIs that the customer service industry measures its success against.

These are the main quantitative KPIs that the customer service industry measures its success against.


The metrics above are the gold standard when we talk about support. It’s important to track all of these analytics and ensure that targets are hit. However, because these are quantitative, there’s a lot of room for these numbers to be negatively affected by outliers, usually where the interesting and more challenging cases are surfaced.

These metrics also don’t capture specifics—they don’t show us how a customer interacted with a platform or even deeper insights about the individual customers themselves. This qualitative data is where we can do that investigative analysis to uncover the real health of the business and drive us towards actionable improvements and optimization.

Consumers Want to Be Known

How do we know we’re capturing the right qualitative data to measure optimal customer experiences? There’s some key questions you can ask, which can be modelled with data science, that will give you that full view of your customer.

  1. What questions are your customers asking once they reach your support platform?
    Create a dashboard that tracks these questions, usually captured via an FAQ or knowledgebase search, and make it a habit to view them daily. The top five questions that are consistently asked should always be part of your FAQ. You might see that outliers start to occur more often, in which case, optimize your knowledge base to include these over time. This is a practice that should be ongoing. It will also indicate broader issues that can be brought back to different parts of your organization.
  2. What language are my consumers using?
    This might seem similar to the question above, but this is about identifying trending keywords to create models that will automatically optimize your platform. In the webinar, Rosow walks us through a “stop word” methodology which allows you to prioritize words that are occurring often but also remove words that are non-descriptive prepositions or articles (after, can, the, etc.), or those that are occurring often because they are overly descriptive (like the word “insurance” when you’re an insurance provider). Removing the stop words lets you understand the biggest concerns amongst your customer base and optimize the knowledge you surface to customers trying to self-service; it also helps you train your agents against the most pressing challenges.

    This insurance company removed stop words to discover that customers were trying to educate themselves about all different types of insurance coverage - offering up an opportunity to upsell.

    This insurance company removed stop words to discover that customers were trying to educate themselves about all different types of insurance coverage – offering up an opportunity to upsell.

  3. What are my consumers looking for most? This simple question can help us look at distribution of word counts. Let’s look at that same insurance company from the example above. We know that their customers are looking for all different kinds of coverage, but is there one specific type that is outperforming the rest? Knowing this will offer up all kinds of opportunities to prioritize business actions. Marketing, training, education can all be optimized to align with the consumer’s interests.
Life insurance was top of mind for customer’s on this provider’s site. Now that provider can invest appropriately and not waste budget on offerings that are performing lower than others.

Life insurance was top of mind for customer’s on this provider’s site. Now that provider can invest appropriately and not waste budget on offerings that are performing lower than others.

So far, all of these questions have allowed us to get to know new and current customers alike. What about ensuring that our loyal customer base stays loyal? Attention must be paid to the customer’s already using our products and services.

  • How do I best serve my existing customers? This is where mapping a customer journey is the best tool in your tool box. By protecting the needs of those customers, we retain revenue and save money simply by creating profiles that track the actions those customers are taking. What are our objectives here? We can create a profile that knows what previous questions those customers have asked, and map them to simple identifying information within their account like name, family size and location. This will allow us to create implications that can be used to predict their behavior so that when they contact us again, we’ll know that perhaps their car broke down in a snowstorm or that they might be planning to grow their family and need additional health insurance.

Consumers Want Seamless Conversation

A lot of the above has helped us improve our self-service site experiences, explicitly our FAQs and knowledge bases. What about that next layer? Chatbots, voice automation, etc.? That middle step, before a customer reaches an agent, is also crucial. Here we need to ask more platform-focused questions so that we can augment those support tools with the right data. All of this data will continue to strengthen our call-deflection strategies, which is where we can save the most support dollars.

Let’s look at where our consumers are coming from. The entry point into this layer of call deflection will guide our investments to the right technology. Are they asking for information on mobile? In-app? Desktop? If our customers are mostly inquiring on their desktop, for example, we might want to ensure that chatbots are optimized specifically for that.

We also want to ensure that the experience is the same across all of these entry points. A poor in-app experience that doesn’t align to a great desktop experience could cause us to lose a mobile-native customer and perhaps cause us to lose business in a growing area. A lower number of users on mobile might also inform us that the experience there is in fact subpar. It’s important to look at and analyze all of these interactions, especially against the case load coming from these platforms.

Now let’s cut that with data that answers the question, where do my consumers go for support? Do they try and gather information via chat, email, social media, text, etc.? Knowing where they action before submitting a ticket will also guide our investments. It’s important here to keep an eye on these numbers regularly, as the where is often the piece that will change the most over time.

As an example, social media is a channel that is growing at an exponential rate – it might not be a place where most of your customers are trying to resolve queries right now, but that will likely change in short order so budgets must be reallocated and R&D should be tapped into for this channel. The user interaction data will help you identify when it’s time to make the right decision.

All of this information, married together, is going to help us identify every interaction point so that we can strengthen our channels to best serve our valuable customers. Mapping that customer journey is essential.

A snapshot of what one customer’s journey might look like as they move through our support channels.

A snapshot of what one customer’s journey might look like as they move through our support channels. Having this in-depth view will help us save money and create better experiences over time.

Consumers Want Empowered Problem Solvers

What if our customers still need to access a live agent, even after we’ve done all of this due diligence? Have we failed if we could not deflect every single case? Absolutely not. There will always be complex challenges that our customers will face and having an empathetic support agent at the other end of the phone is still one of the most effective customer service channels we can deploy. The hope is that all of the work mentioned above will both decrease the caseload for our employees and arm them with a treasure trove of data that will help them do an even better job at servicing our customers.

Consumers want agents to have both the knowledge and the authority to answer any question that is posed to them. Because the cases that reach our agents are likely the most difficult to solve, and because our customers might have become frustrated at this point in their journey, supporting the agents themselves will actually drive the most success here. Customer retention, loyalty, delight—it all happens within these interactions. Agent effectiveness is driven by creating the best experiences for our employees at this juncture. Each individual agent should be armed with all of the data we can provide them—a delicate balance that shouldn’t overload them—so that our customers ideally only have to speak to one person during each interaction.

How to Improve on Agent Effectiveness

Do our agents have access to the information they need? Augmenting our service consoles with third-party data from across our organization—ie, Confluence, JIRA, Salesforce, Slack, etc—and presenting it in a way that’s easy to digest helps agents organize their responses. The information should be personalized to the case at hand and in a way that explains who the customer is behind that case so that the agent can understand the customer and their journey. All of that user interaction data we’ve collected from the call-deflection and self-service activity will play a key part in that personalization.

Are our agents able to find what they need? Here, we have to think about how the service console is presented. Are there additional knowledge articles, documents and information that will help the agent answer the customer’s query? Is there a specific insurance policy to address, perhaps something that’s specific to that customer’s location? Again, personalizing the information and making it easy to surface will improve the time-to-resolution and make the experience for the agent more seamless.

Do the agents know who their internal subject matter experts are for a specific case? Being able to access another human in the moment a case needs to be resolved is an often-overlooked touchpoint. Nestling an internal chat tool within the service console, or augmenting a community portal with a search experience could cut that time-to-resolution in half.

Acting on Insights; Solving Additional Pain Points

Rosow also got into some nitty gritty solutioning in the webinar. As a data scientist that is doing the work on the back-end to create these personalized, automated touchpoints for our customers and agents, she surfaced some solutioning best practices against additional, common pain points. Your own teams can adopt these best practices—or we can activate them for you at Lucidworks.

Self-service / Call Deflection

  • Typos and Misspellings: leverages query engagement distribution to drive novel misspelling correction contextual to a search domain; uses an unsupervised ML approach in combination with text matching techniques
  • Long and Specific Queries: enables search engineers to identify the highest vs. lowest performant queries based on distributional head/tail analysis of user interactions
  • Mismatched Vocabulary: identifies like keywords using a synonym approach to surface the same answer even when slightly different language is used

Agent Effectiveness

  • Negative Sentiments: maps customer sentiment analysis from cases with longer time-to-resolution against the customer journey identified above in order to surface agent training challenges
  • Repetitive Cases: discovers similar cases that might be occurring regularly in an agent service console; augment self-service and agent knowledge bases and improve call deflection by moving these case issues into those assets
  • Poor Discoverability: deploys semantic search to uncover the answers for the questions that really can’t seem to get answered by applying intent to a semantic word map

The Best Customer Experience Is Also the Best Employee Experience

We want to be reaching for the right solution for a reason. What data made you reach for a certain feature? What impact will that feature have on the end-user? Will that feature help drive not just my quantitative KPIs, but my qualitative ones as well?

Now that you’ve asked yourself the right questions, you’ve used data to drive the right decisions for your customers and your employees. What can you do to ensure the wheels stay on the bus? How can you keep using your user interaction data? Well, now you’ve got the keys to the kingdom. You’re tracking the actions of your users, you are gathering and investigating that data on a regular basis. And you’re actioning on that data by improving all of the tools you’ve gathered to build your customer service tech stack.

Now that you’ve built these practices, you can breathe a little easier knowing that your users and agents will continuously let you know if what you’re doing is working. Stay observant, stay curious and stay in that loop with an open mind. You can just build better from here.

Check out the full webinar video here or contact us today if you’re ready to put some of these ideas into action.

About Jenny Gomez

Read more from this author


Contact us today to learn how Lucidworks can help your team create powerful search and discovery applications for your customers and employees.