Learning to Rank: Using Machine Learning To Teach Computers to Rank

If you’ve ever searched for something but experienced that eye-rolling moment of, “no, computer, that’s not what I meant,” you’re not alone. But the machine learning technique known as ‘learning to rank’ can teach the machine to recognize how we humans would rank results.

In this episode, Tia explains how our individual search behavior can improve search results for everyone.

You can binge-watch all of season two or jump on back to watch season one.

Subscribe to the Lucid Thoughts channel and be sure to leave your questions and comments on each video.

Transcript:

(looking at her phone, rolling her eyes)

Ok… when I said best Sichuan noodle place I didn’t mean in China…

(scrolling through results on her phone)

There… that’ll work.

At least clicking on this result is going to help others. Really my dinner selection is doing the world a service. I’ll tell you all about it. (looks at phone) Looks so good…

Sometimes when we use descriptive words in our search bars, we have that eye-rolling moment of, “no, computer that’s not what I meant.”

That’s because we know which part of our search is more important, but the computer doesn’t. Like I wanted to know the best Sichuan noodle place, but near me. In this instance, the “where” was the most important part of my search.

So my search results gave me places too far away and some matched my keywords but it still wasn’t what I was looking for.

There’s an easy way to fix this with some good old machine learning.

The technique is called “Learning to Rank” and it teaches the machine to recognize how we as humans would rank results and information.

Let’s go back to our restaurant debacle, and add in another factor. Now I want the best vegetarian Sichuan noodle dinner.

Say we have a restaurant recommendation site. This is how it might rank sites based strictly on keywords.

But here’s how we as humans would rank them. For restaurants, the type and location are the important parts while the name may not matter as much.

So we use this training set to teach the computer how to rank the restaurants.

If we take a bunch of these relevance-ranked results from certain queries, we can use them as training sets to demonstrate the computer how to rank results. This training shows the computer which fields are more important to the user so we get a list in the order we expected.
Now, this is simple if you have a small selection of restaurants and a few characteristics like food type to work with.

But what about online stores with hundreds of thousands of products to search? They could create ranked training sets for every query and product type in their system.

Or they can automate this whole process with signals – remember those?

It starts by having the system monitor signals in user behavior like what they clicked on first or which products they eventually bought. This helps the computer figure out which items users found most useful.

You can use both manual systems and signal-based systems if you’re looking for the best of both worlds.

See? This way your clicks are used for good. You taught the computer a lesson and made life better for everyone.

You’re such a good person.