How Reddit built their new faster, scalable search for hundreds of millions of users with Lucidworks
Reddit — “the front page of the internet” — is in the top ten most popular websites in the U.S. with ~10+ million post submissions per month, almost 3 million comments, and ~60 million upvotes and downvotes across hundreds of thousands of communities each day. People use Reddit to gather news and information, share experiences, and find communities and conversations about shared interests.
Search is a key way Reddit’s more than 330M monthly users find content. After trying five different search stacks in 12 years, they needed a solution that could scale and quickly locate relevant content for almost 47M daily searches.
Lucidworks and Reddit partnered to deliver a solution that was fast, scalable, responsive and relevant that laid the foundation for future improvements. Their search cluster was reduced from 200 to 30 nodes with uptime at 99.99% with 400 queries and 1,000 updates per second including life streaming updates.