Actual mileage may vary

A few weeks after the announcement from Microsoft that FAST is no longer to be available on Linux/Unix, interesting stories continue to pop up about use of Lucene and Solr in its place. Most recently, a benchmark from Technology Services Group, an open source content management solutions consultancy and integration shop based out of Chicago. In a blog post earlier this week, they describe a proof of concept for a large pharmaceutical client, benchmarking search on 156,000 documents in an external data source indexed by Lucene. The search application was part of a larger CMS solution centered around EMC documentum.

Lucene/HPI [the TSG Documentum Lucene-based solution] and the external repository was found to be considerably quicker that the existing FAST/Webtop implementation on most queries.

Specific results:

Query	FAST/Webtop	Lucene/HPI
1200 Results	90 seconds	3 seconds
8 Results	5 seconds	3 seconds
10 Results	8 seconds	4 seconds
76 Results	10 seconds	5 seconds
5100 Results	72 seconds	5 seconds
65 Results	6 seconds	3 seconds

Simple configuration of the Lucene index did a better job of returning a more complete search result set than the standard FAST/webtop configuration.  Examples included additional documents that were logical derivatives of the initial search word. For example – a search for “exception report” could return “exceptions report” or “exception reports”. The proof of concept data set also included German documents and Lucene demonstrated multilingual stemming capability.

Better than 10x reduction sure sounds sweet. Now, with any benchmark, the devil is in the details: lies, damned lies, and benchmarks. They’re tougher to construct objectively than a sweet set of outputs might imply. And so for me, the real punchline is in a different set of numbers:

The flexibility of Lucene to index both the metdata and full-text values allowed the client to avoid adding an additional Oracle database to their external cache for attribute storage.

One less check to Oracle — that’s real money.