How Personalization Beats Segmentation to Create the Next Best Experience
Customers increasingly expect personalized and engaging experiences that provide value at every step of their journey.
Static and incoherent results are outcomes of using traditional methods of segmentation, grouping individuals in cohorts, and treating everyone in the group similarly. Modern 1-to-1 personalization beats segmentation because it predicts each customer’s intent with AI, based on their own dynamic preferences and behaviors, taking immediate action to deliver relevant next-best-experiences.
Marketers and customer insights professionals must rethink segmentation strategies in light of this new personalization paradigm. AI trained on unstructured data from external sources adds more context and confidence to internal data about customer behavior. Personalization yields far greater conversion, retention, satisfaction, and ultimately, revenue.
Hear from our guest speaker Forrester Analyst, Brandon Purcell, and Lucidworks VP of Product Marketing, Justin Sears, to:
- Discover how AI enables smarter insights about customers
- Explore new approaches to personalization leveraging search and AI
- Understand why marketers and customer insight pros need to rethink segmentation strategies in favor of a more personalized approach
Featuring:
- Brandon Purcell Principal Analyst, Forrester
- Justin Sears VP of Product Marketing, Lucidworks
Transcript:
Justin Sears:
Thanks for joining us today. I’m Justin Sears. I run marketing at Lucidworks. And I’m joined today by Brandon Purcell, principal analyst at Forrester.We’re going to make the case for why personalization beats segmentation to create the next best experience. To kick it off, I’m going to hand it over to Brandon to introduce himself, and share his perspective on personalization.
Brandon Purcell:
Great, thanks Justin. And thanks to you and the Lucidworks team for having me, and to everybody who’s attending today. I’m really excited to talk to you about personalization today. I am a principal analyst at Forrester Research, and I sit on our Customer Insights Team. So I help companies take their massive amounts of data, and distill that data into insights to help them win, serve, and retain customers.
If there’s one thing that is different that has happened in the last 10 years or so from previous history of business, it’s that we are firmly in the age of the customer. And by this, I mean that increasingly empowered customers have been able to disrupt entire industries. If you think about Uber and the transportation industry, or you think about Airbnb and the hospitality industry, or Facebook and democracy as we know it. That’s a little joke.
But the onus is on us as businesses to better understand our customers, and anticipate their needs. The good news is that they’re giving us a lot of breadcrumbs to better understand them. We’ve been accumulating data since the middle of the 20th century – financial data, sales data, product data – In the age of information, in the 90s, we saw the rise of powerhouses like Google and Amazon with globally connected PCs. We had an explosion of data then with transactional data in first and third-party customer data. But today, there’s been an exponential increase in the amount of data we have, whether it be social media data, behavioral data on our customers, mobile data, or even data from the internet of things, connected devices, etc.
The customer insights professional, who I write my research for, is charged with finding these golden nuggets that are buried in data that are useful to win, serve and retain these increasingly empowered customers, especially as industries become increasingly commoditized, and you have to differentiate through offering a more seamless, convenient customer experience. Now, in theory, unearthing these insights should be pretty easy. Customers interact and transact with us, and that creates a data footprint. We apply analytics to that data to unearth these insights, these golden nuggets. And then this isn’t an academic process. We need to take action based on those insights to inform the customer experience so that we can actually win, serve and retain customers. And finally, this insights lifecycle is in fact a cycle, because we need to learn from the efficacy of those actions. So we saw that Justin was likely to churn, and we sent him a retention incentive. Did he or did he not stay with us? That’s important data to continuously optimize this model.
Now, unfortunately, as you can see from the title of this slide, this lifecycle is broken. Companies struggle to turn data into insights, insights into action, and they especially struggle to close the loop on that. So enter artificial intelligence.
AI refers to the theory and capabilities that strive to mimic human intelligence through experience and learning. Now, I realize that many people have their own definition of AI. This is Forrester’s definition, and there are two key facets to this definition. The first piece is some sort of mimicry of the human faculty. And you can see on the right here, there are AI technologies. At this point, everybody knows AI is not a single technology. There are multiple different component technologies. Some of them can sense the world around them. Others can reason upon what’s been sensed or think. And then others actually can take action.
So that’s one piece, mimicking some human faculty. The other piece, and the piece that separates today’s AI from your grandfather’s AI, is machine learning. So AI today, most AI today is using machine learning at its core to learn from data the best way to optimize processes.
Now if this graphic looks a bit familiar, it’s because it actually mirrors that insights lifecycle I just introduced to you, right? Sensing the world around us. I mean, that is data. One of the beauties of AI is that it makes data just data. Unstructured data is now data. Structured data has always been data. We think about that data to create insights. And then of course, we take action. So the promise of AI and the reason that I believe it’s so hyped in the market is because it could very well automate and optimize this insights lifecycle so that you’re continually learning and getting better and better at retaining customers, increasing lifetime value, and selling more to customers.
In the world of customer analytics that I cover at Forester, I like to talk about how customer analytics, in theory, should get the right message to the right customer at the right time. But the truth is that until recently, customer analytics has really focused on that right customer piece, maybe the right time piece, but the right message, the right experience has been lacking. And AI can help us to identify, curate, and deliver that right message or right experience. So what are companies doing today to try to satisfy the demands of these increasingly empowered customers?
Well, most companies are using segmentation. Segmentation has been around forever. I receive more inquiries at Forester about segmentation than anything else, more than AI, more than any sort of predictive analytics. Our clients are interested in segmentation. And it’s too bad, because it’s the best of times for personalization, especially given all of the advances in AI and all of the data we have, and the worst of times for segmentation. And I realize that’s a somewhat provocative statement. So let me tell you why that is.
Look at all of these different millennials. We have 12 different millennials here. Demographic, classic demographic segmentation would treat all of these people the same way. Now, do these people look similar? Would you try to sell Aziz Ansari the same pantsuit as Olivia Pope or Kerry Washington? I hope not. It’s probably not gonna go over very well. No, we can’t treat these people as one static monolithic demographic segment. Instead, we have to use all of the data that we’re collecting on them, the behavioral data, the transactional data, to better understand their wants and needs and respond to them.
So how do I differ segmentation from, or how does segmentation differ from personalization? Well, there are a few different ways to think about this. One is thinking about what your intent is as a business using these methodologies. With segmentation, the intent has traditionally been to drive desirable actions or behaviors. With personalization, you’re thinking more broadly about actually improving the customer experience. You’re thinking about what your customers want and expect from you, and how you meet those needs.
So your outcomes actually change. Instead of looking at traditional metrics like a higher response or conversion rate, like increased retention or sales. Instead, you’re also looking at customer-focused metrics, like increased customer satisfaction or CSAT, or you’re looking at reduced effort scores. And as we’ll see in a second, lifetime value becomes a very key metric in all of this. And finally, the movement from segmentation to personalization also impacts different parts of the experience. So instead of just looking at traditional campaigns, offers and recommendations and messages, you’re actually going to use personalization to impact functionality. And you’re going to surface different content for different types of customers and interact with them in different ways.
Justin, I know Lucidworks has an interesting point of view here. I’d love to hear a little bit from you about the Lucidworks perspective.
Justin Sears:
Yeah, thanks, Brandon. This slide and this grid are a great way to break it down. I was nodding my head here because one of our biggest customers sells a lot of yoga pants online, and they had a big sale in July with record-breaking revenue. And it was because they were able to offer a personalized experience to their shoppers with Fusion and anticipate the next best experience in their shopping experience. And then a couple other examples that also line up really well with this is a prominent discount retailer understands very well the difference between personalization and segmentation. And their team was really focusing on how they could get the most out of personalization. And when they turned on signals in Fusion, which I’ll talk about in a little bit, they saw their conversion rate double over what they were seeing before because they were bringing in that AI piece that you alluded to before. And they didn’t have it before they turned that on. And then the last example that really lines up well in the digital commerce space is a major online bookseller, and we were talking to them before they purchased Fusion.
They split the traffic between their status quo tool and Fusion. And they intended to run a few months before the holiday season. After watching the results for two weeks, about it and shifted all of their traffic to Fusion because they were seeing about a 12-point improvement in conversion just because they were adding that personalization, that level of intelligence using AI spread across all of those experiences of their thousands of shoppers.
Brandon Purcell:
That’s great, Justin. Thanks for those examples.Especially the yoga pants one, I’m not surprised that they experienced a high rate of sales in July. My hope is that after we go back to some semblance of normal, we can continue to wear yoga pants during the workweek. That would be great and very comfortable.
Justin Sears: Hear, hear.
Brandon Purcell:
So from an analytics perspective, segmentation is one way to skin this cat, but there are a number of analytical techniques that you can apply to your customer data to better personalize experiences.
This is kind of the cornerstone of my customer analytics research at Forrester. I’m not gonna go into every single one of these techniques, but I would like to just tell you the way to read this pinwheel diagram is in the outer circle in green, we have the different applications of customer analytics. And so there are analytical techniques that help you identify customer context to improve your marketing to them. Moving clockwise, there are those that help you acquire customers. Every business is interested in growing its customer base. At six o’clock on the diagram, you’ll see techniques that align with retention and building loyalty. Moving along, there are techniques that are fully focused on personalization, what we’re talking about today, although arguably all of these techniques can be used to deliver personalized experiences.
And speaking of experiences, customer experience is another application area for customer analytics. Identifying pain points in the experience, improving the experience, differentiating through experience. So those are the application areas. And then in blue, we have the different techniques that align generally to those application areas. For instance, just if you go right down to six o’clock, you’ll see customer churn and attrition analysis. That’s kind of the canonical customer analytics technique where you have data on customers who’ve historically churned and not churned. You feed them into a supervised learning model. And it can detect patterns amongst customers who have churned that differentiate them from non-churners, so that you can score your entire customer base based on their likelihood to churn.
Now, even in the best of times, not all of these techniques are created equal, meaning some of them are gonna be more valuable for some industries or use cases than others. But we did just see a very rapid shift in the prioritization of these techniques due to the pandemic and how it impacts the customer behavior. What happened was, and you’ve all experienced this firsthand, people stopped leaving their homes, right? And certainly weren’t going into brick and mortar locations. And were increasingly interacting solely digitally. Because of this, companies had to reinvest in descriptive techniques like customer journey analytics, which is there at 11 o’clock, to understand the new volume of customers on digital journeys, what those digital journeys were, the different touch points within them and whether the journeys were actually working or not.
So what were the KPIs at the end of the digital journeys? Did some digital journeys result in higher conversion rates than others or higher drop-off rates than others? So that gives you a sense of what you may need to fix or double down on. At the same time, companies that were analytically advanced and had predictive models in place experienced what’s known in data science as data drift. Because all of a sudden customers’ behavior shifted so drastically, the past no longer reflected the present. And so those predictive models were far less accurate than they were in the past. And so companies have had to scramble to try to retrain or rebuild those predictive models based upon new data.
Now even in the best of times or today, companies are gonna be using many of these models in tandem. And they need a way to orchestrate what experiences should we deliver based upon the insights that we’re deriving from these models? And that orchestration layer is something I like to call the next best experience.
And Justin mentioned the next best experience before. I think of it as the Holy Grail of the customer analytics, a mature customer analytics practice. And it obviously builds on the next best “trilogy” that preceded it, next best product, next best offer, next best action. But it’s a bit different. And I use the term experience intentionally, because it encompasses not just a marketing experience, but we can actually apply the next best experience across, it could be marketing, but also customer experience, customer service, a product experience or some other operational area. So the lens is much broader than just marketing. And also the execution mindset is that personalization execution mindset we talked about before. It’s not inside out, what do we want to get from the customer? But we’re actually striving to identify what the customer is trying to achieve with us. We still want to be profitable. And so we need to look at the long-term profitability of individual customers as opposed to short-term conversions or clicks.
And therefore, as I mentioned before, customer lifetime value being a measure of that future profitability of each customer, is the metric that we’ll want to optimize in this next best experience model. This should be aspirational for you as you’re building out your practice. You can build each of the individual models that will eventually enable you to create this next best experience orchestration. I’m gonna turn it over to Justin to talk a little bit about the Lucidworks approach.
Justin Sears:
Thanks, Brandon. When I read your research and came across your concept of next best experience, I said, that’s what we do at Lucidworks. And I also like the slide you just showed, that it is a journey. So we have customers that are at the first step of that journey and are just thinking about how they might create that experience. And then getting used to using machine learning, and applying that to the experience. And then we have customers that are stage three or stage four of that journey that you just outlined and are doing some really advanced things.
We offer the next best experiences to our clients so they can provide that to their customers or their employees through two types of solutions: the digital commerce solutions for shoppers and digital workplace solutions for employees. And you just referred to the best of times and the worst of times, which of course comes from the Dickens classic, A Tale of Two Cities. My grandfather was a big fan. So that book was oft quoted when I was a kid. And here’s a quote from your paper that summed up really well what we strive to offer with our platform, Lucidworks Fusion.
Personalization, delivering the right experience to the right customer at the right time is not easy, but it’s de rigueur for companies that wish to thrive in the age of the customer. The next best experience that you described in that research is proof to the consumer of how well a company can personalize for them. And of course I mentioned digital workplace. At Lucidworks, we’re on a mission to bring that same level of best-of-breed consumer personalization to the workplace so we can experience that as we work. And some people may be wealthy enough to spend most of their time shopping rather than working. But most of us, for most of us, the opposite’s true. And work can be a lot better, a lot more satisfying, if we get to use the tools that augment our human intelligence and give us more time doing, and less time finding the information we need.
So this statement really sums up the challenge of personalizing for employees to offer them the next best experience at work. Just like you might offer that next best experience to a consumer on an e-commerce website. Organizations possess lots of data, siloed inside disconnected applications and unavailable to employees at their moments of need. So Brandon, we’re really
seeing a theme here. It’s that moment, the moments change from instant to instant. And AI really allows the system to be aware at some level of what type of moment the user finds him or herself in. And at Lucidworks, we know how challenging it can be to personalize that moment of finding, because we began working on it 12 years ago.
We began with a deep understanding of search and Apache Solr, the open-source product for distributed search on big data. And we wrote code to ingest data generated by systems, humans and applications. You see that on the left side of this slide. We’ve developed advanced functionality for indexing, clustering, classification, faceting, filtering, relevancy, ranking, analytics, all of those different steps in the middle. And we’ve bundled it all into the digital commerce and digital workplace solutions available with Lucidworks Fusion. Now because we’ve done the work and become experts at making finding personal for shoppers and workers at the world’s largest organizations, our customers can begin using the Fusion platform without diverting time, attention and money away from their core business so that they can build or maintain their business rather than caring for platforms and applications in house. So what we say is focus on your business, your customers and your employees instead.
Now to abstract it a little bit, I showed the spaghetti chart on the last slide. This is how we want our customers to experience Fusion, because they don’t need to know about the details. So Lucidworks Fusion incorporates AI and machine learning throughout the platform to intelligently ingest, explore and curate the data. When you ingest data, Fusion uses AI to prepare documents, data, records, files for discovery. It clusters those records and classifies and organizes content and identifies entities within the content using natural language processing.
So it’s looking at the data that comes in and identifying people, places, products, or some other entity that you might be interested in. Then once the user interacts with the system, Fusion invokes AI again to predict the user’s intent. Then we can map that intent with the most relevant content and personalized search results, browsing options, or make proactive recommendations that are on point with what the system believes the user might need at that particular moment. Fusion application studio allows teams to quickly create new personalized applications that constantly tune themselves to offer the next best experience. And all of this can either be deployed on premises or it can be self-hosted on public cloud services or managed entirely by Lucidworks in a cloud platform as a service. So there’s lots of flexibility in how our customers can deploy Fusion.
This has been a brief conversation about how personalization can really make life better for your customers, and how Fusion excels at personalization. Hopefully you’re curious to learn more about both topics. So please join Brandon and myself for a Q&A session following in this talk, and we can dive more deeply into these important topics.