Posting Rich Documents to Apache Solr using SolrJ and Solr Cell (Apache Tika)

Solr Cell, a new feature in the soon to be released Solr 1.4, allows users to send in rich documents such as MS Word and Adobe PDF directly into Solr and have them indexed for search.  All of the examples on the Solr Cell wiki page, however only demonstrate how to send in the documents using the curl command line utility, while many Solr users rely on SolrJ, Solr’s Java-based client.  Thus, I thought I would throw up a quick SolrJ example here (and I’ll update the Wiki) demonstrating how to do this.

For this example, I’m using the standard Solr example and the Solr trunk version from this morning which I got using SVN:

svn co https://svn.apache.org/repos/asf/lucene/solr/trunk apache-solr

Next, after changing into the directory I checked out to, I built the example using Apache Ant:

ant clean example  //slight overkill, but I did it nonetheless

I then changed into the example directory (trunk/example) and ran:

java -jar start.jar

Solr is now up and running.  See the Solr Tutorial for more info on these steps.

On the code side, I used SolrJ by creating a new SolrServer and then constructed the appropriate request, containing a ContentStream (essentially a wrapper around a file) and sent it to Solr:

public class SolrCellRequestDemo {
  public static void main(String[] args) throws IOException, SolrServerException {
    SolrServer server = new CommonsHttpSolrServer("http://localhost:8983/solr");
    ContentStreamUpdateRequest req = new ContentStreamUpdateRequest("/update/extract");
    req.addFile(new File("apache-solr/site/features.pdf"));
    req.setParam(ExtractingParams.EXTRACT_ONLY, "true");
    NamedList<Object> result = server.request(req);
    System.out.println("Result: " + result);
}

In this code for SolrJ, I did an extraction only, but you can easily substitute your parameters based on the Wiki page descriptions on Solr Cell.  The key class to use is the ContentStreamUpdateRequest, which makes sure the ContentStreams are set properly, SolrJ takes care of the rest.

I hope that gives people a quick idea of how they can send files to Solr Cell via SolrJ.  Also, note, that the ContentStreamUpdateRequest is not just Solr Cell specific, you can send CSV to the CSV Update handler and any other Request Handler that works with Content Streams for updates.

For completeness, the output from the code above looks like (some results reformatted for screen width):

Result: {responseHeader={status=0,QTime=1692},null=

,null_metadata={stream_source_info=[null],stream_content_type=[null],stream_size=[13242], producer=[FOP 0.20.5],stream_name=[null],Content-Type=[application/pdf]}}