Using Learning to Rank to Provide Better Search Results
Fusion 4 is built with Apache Solr 7. Solr 7 provides a powerful algorithm for improving search results called Learning to Rank, or LTR. At its essence, LTR uses machine learning to teach the system how to order a set of results based on certain characteristics.
When you send a query to Fusion or Solr, results are returned and ordered by a relevance algorithm called BM25 which is derived from TF-IDF. While this is a simplification, based on this algorithm, the more frequently a rare term occurs in a document, the higher the document will be ranked in the results. This is usually pretty good, but for some types of results it just doesn’t provide the best order possible.
Learning to Rank (LTR) lets you provide a set of results ordered the way you want them to then teach the machine how to rank future sets of results. The default search algorithm is still used to get the initial set of results, but then the system will reorder them based on the ranking model that it trained on.
While a hand-ordered input data is useful for small datasets, using Fusion’s signals capability you can go much further. By using behavioral data (i.e. what users clicked on or actually bought), you can transform your signal data into an automatic ranking set. In this way you can essentially let the users decide which result should be first.
Using Fusion’s signal capture and signal boosting, together with Solr 7’s Learning To Rank capability you can provide better results than using any one method alone.
I’m a Senior Data Engineer at Lucidworks and I’ve put together a technical paper explaining how to do this and how the results compare. Check it out Learning to Rank for Better Search Results. I’ll be hosting a webinar on the same topic, Learning to Rank for Improved Search Results, on April 4th.
Learn more
- Further reading with the ABCs of learning to rank
- Webinar: Learning to Rank for Improved Search Results
- Technical Paper: Learning to Rank for Better Search Results
- Blog: Fusion 4 Ready For Download
- Contact us, we’d love to tell you more about it!