Solr Distributed Indexing at WalmartLabs
As we countdown to the annual Lucene/Solr Revolution conference in Boston this October, we’re highlighting talks and sessions from past conferences. Today, we’re highlighting Shenghua Wan’s talk, “Solr Distributed Indexing at WalmartLabs”.
As a retail giant, Walmart provides millions of items’ information via its e-commerce websites, and the number grows quickly. This calls for big data technologies to index the documents. Map-Reduce framework is a scalable and high-available base on top of which the distributed indexing can be built. While original Solr has a map-reduce index tool, there exist some barriers which makes it unable to deal with Walmart’s use case easily and efficiently. In this case study, Shenghua demonstrates a way to build your own distributed indexing tool and optimize the performance by making the indexing stage a map-only job before they are merged.
Shenghua Wan is a Senior Software Engineer on the Polaris Search Team at WalmartLabs. His focus is applying big data technologies to deal with large-scale product information to be searched online.
Join us at Lucene/Solr Revolution 2016, the biggest open source conference dedicated to Apache Lucene/Solr on October 11-14, 2016 in Boston, Massachusetts. Come meet and network with the thought leaders building and deploying Lucene/Solr open source search technology. Full details and registration…
Contact us today to learn how Lucidworks can help your team create powerful search and discovery applications for your customers and employees.