Death of the No-Results Query
How one of the world’s top five retailers used Lucidworks Never Null and semantic vector search to increase AOV by 28% and decrease zero results queries by 91%.
Online shoppers expect to interact with technology using a natural cadence, typing or speaking queries to a search engine the way they’d ask questions of a friend. Technology, in turn, should understand not only your words, but your intent, and recommend relevant information.
So when a customer arrives at a website and types in the brand name of an item they’re looking for, they expect the search engine to understand what it is they’re actually after. If you carry the brand the customer typed in, great, an exact match is made, the customer fills their cart, completes the purchase, and leaves satisfied you’ve met their needs. But if that brand isn’t in your inventory, the dreaded “no results found” message fills the screen. They’ve hit a deadend and off they go to another website to fulfill their goals. A product search with no results is a deathtrap for the customer-retailer relationship.
The punchline is, you do have the product they want, just not the brand name they typed into the search box. Many products you carry fulfill the customer’s needs. Maybe you don’t sell the requested Charmin brand but Quilted Northern or store-brand toilet paper, both available in your inventory, are interchangeable alternatives. But this all falls down when your search engine is not smart enough to understand the problem behind the customer’s query and make the connection to suggest solutions in stock.
One of the world’s top five retailers found itself in this null-results dilemma. This was exacerbated by its inventory consisting of a wide selection of different products but limited varieties of each product type. Typical search, driven by simple keyword matching, resulted in copious queries that didn’t return any products. And as we’ve said before, if they can’t find it, they can’t buy it.
Bring in the Experts
The retailer knew that search was not its website’s strongest feature, but investing valuable employee time to become search experts, rather than advancing broader company goals, was not something it saw as advantageous. So when it moved product catalog search from Endeca to Apache Solr a few years ago, a third party contractor was used to make the change.
While open-source Solr reliably scaled to the retailer’s high-traffic demands, the contractor created a like-for-like replacement of Endeca rather than an improvement on its capabilities. Lacking machine learning, search results had to be manually curated with an ever-increasing number of rules. For the retailer’s modest search team, curating a multitude of rules was exhaustingly labor intensive. For its customers, queries with poor result relevancy or no results at all lead to lots of frustration and high bounce rates. And for the business, this all meant high operating costs and lots of lost revenue.
The retailer made another attempt to better its customer and employee experience with search while holding true to its “bring experts in’’ ethos. It went looking for search specialists who could implement and manage highly-scalable online search that would resolve no-results queries, improve relevancy, and reduce its internal team’s rules-curation burden. With its foundations in Solr, track record of improving retailers product discovery, and Fusion platform that employs machine learning to benefit the user, Lucidworks fit the bill.
Solving Null-Results Queries with Semantic Search
Lucidworks identified several query types that didn’t produce results on the retailer’s site:
- Brands/items not carried.
- Items in high demand whose stock availability fluctuated frequently. Products were flagged not to appear in search results when they were out of stock.
- Items available in store only. Only items available in the customer’s local (or selected) store would appear in search results.
- Mismatched vocabulary. Search terms that did not appear in product names or descriptions, more specifically:
- Long-tail queries (“weatherproof fleece-lined pants”),
- Thematic queries (“warm jackets”),
- Symptom queries (“dry cough, acid reflux”),
- Misspellings. All misspelling variations had to be added to a spelling correction rule to produce results.
- Exact queries. Such as product model numbers.
To reduce null-results queries, Lucidworks deployed semantic vector search with a proprietary, deep-learning encoder called Never Null that learns from customer behavior to associate queries with products that have a similar purpose. Never Null uses behavior signals to train search models, continually tuning and improving search results. Using deep learning, a semantic search box is able to yield results to queries based on semantic meaning rather than simply matching products via keywords.
Never Null’s advanced machine learning automatically retrieves relevant results for challenging and previously low performing queries, relieving the retailer’s search team of the repetitive task of analyzing results and manually creating and curating rules. This freed up the search team to spend time on more strategic tactics.
By broadening fields searched, relevant product assortment in search results expanded, driving increased engagement, conversions, and average order value (AOV).
Never Null Put to the Test
The ultimate test of online American retail establishments is Cyber Five, the annual five-day period running from the Thanksgiving holiday on Thursday through the following Monday, called Cyber Monday. As the typical morning average of 250 queries per second (QPS) ramped up to 1250 QPS on our retailer’s website as Cyber Five began, query response time was not affected and 99% of requests were processed in 100-300 milliseconds.
Evidence of search’s essential function for occasional and new shoppers less familiar with a retailer’s site, search engagement jumped from a pre-event daily average of 15% up to 31% over the Cyber Five. And thanks to Never Null’s out-of-the-box models that improve on a cold start—even when signal history is not available—result relevancy for new shoppers made them feel more like familiar friends.
Ultimately, 680M search requests came in over the five-day period with not a single outage. Not a single one. Relevancy improved dramatically, resulting in a 30% increase in search-influenced orders and a 28% increase in search-influenced order value. And those loathed zero-results queries? They were reduced by 91% compared to the previous year.
As Never Null continues to learn from customer behavior signals, the retailer can rest easy knowing that shoppers’ carts will be overflowing with the products they are looking for.