Solr Distributed Indexing at WalmartLabs

As we countdown to the annual Lucene/Solr Revolution conference in Boston this October, we’re highlighting talks and sessions from past conferences. Today, we’re highlighting Shenghua Wan’s talk, “Solr Distributed Indexing at WalmartLabs”.

As a retail giant, Walmart provides millions of items’ information via its e-commerce websites, and the number grows quickly. This calls for big data technologies to index the documents. Map-Reduce framework is a scalable and high-available base on top of which the distributed indexing can be built. While original Solr has a map-reduce index tool, there exist some barriers which makes it unable to deal with Walmart’s use case easily and efficiently. In this case study, Shenghua demonstrates a way to build your own distributed indexing tool and optimize the performance by making the indexing stage a map-only job before they are merged.

Shenghua Wan is a Senior Software Engineer on the Polaris Search Team at WalmartLabs. His focus is applying big data technologies to deal with large-scale product information to be searched online.

Join us at Lucene/Solr Revolution 2016, the biggest open source conference dedicated to Apache Lucene/Solr on October 11-14, 2016 in Boston, Massachusetts. Come meet and network with the thought leaders building and deploying Lucene/Solr open source search technology. Full details and registration…