“That’s Not What I Meant!” How Head-Tail Analysis Increases Conversions

Head-tail analysis is a machine learning technique that improves the user experience for search applications. This Lucid Thoughts video explains head-tail analysis and how to optimize the search experience from head to torso to tail. Watch now:

If we plot the frequency of search terms on a scale from most frequent to the least, we get a distribution with:

• A head. The head consists of queries that people search for frequently or occur frequently in a set of data. Most applications and sites see a majority of clicks and successful outcomes – like engagement, views or purchases – related to these terms.
• The torso. Torso queries occur less often than the head and lead to fewer clicks. Sometimes the user gets what they are looking for – sometimes they don’t.
• The tail. Tail queries are queries that just go completely rogue and do not return anything close to what the user wants. So it doesn’t get lots of clicks or conversions giving the user a degraded experience.

Tail searches are missed opportunities.

Sometimes the searches we send to a site or application confuse the machine receiving them. There could be double meanings, misspellings, strange word choices, or other idiosyncrasies that lead to bad search results.

So how can we turn queries with no clicks into ones that do?

By letting the computer analyze user behavior and other signals, it’s able spot patterns to figure out the user’s intent.

With this type of analysis, AI can transform tail queries into head queries by fixing misspellings, weird word orders, or synonyms that need to be added to the synonym list. Simply by getting to know the user and their search patterns, machines can try and reduce the number of times the user says, “Wow, that’s not what I meant.”

Watch more Lucid Thoughts videos