Using open source isn’t free? Could be search for big data is the win

It’s still not unusual, when you hear talk about open source, that the focus is on the fact that it’s “free”. You can download the software for free, there aren’t any fees to use it, and so on. Depending on the project, you may also get the advantage of an active community for help, support and fixes.

But anybody who’s worked in a shop bigger than a handful of engineers can tell you that software is never “free”, even if it’s open source. There is always a cost involved in using it, even if it’s just to pay your people to manage it, run it, code it, or just watch it and make sure it doesn’t fall over.

In the open source world, those costs are usually for service and support, or for specialized or enhanced versions of an open source project, such as the services we provide here at Lucid Imagination, or Lucidworks Enterprise, which is built on Apache Solr, but with additional features that can cut time or cost out of the effort.

Steve Arnold, noted Enterprise Search blogger behind the Beyond Search blog, spoke to Stephen O’Grady, the co-founder and Principal Analyst of RedMonk, who had an interesting take on why open source is good for users.

Depending on the product, open source can  be even better than proprietary systems that serve the same purpose. But if you’re still going to have to pay for it, where is the value for you, the user? O’Grady explains:

The advantages to [the open source] model from the customer perspective are multiple, but perhaps the most important is what Simon Phipps once observed: users can pay at the point of value, rather than acquisition. Just a few years ago, if you had a project to complete, you’d invite vendors in to do a bake off. They would try to prove to you in an hour or two demo that their software could do the job well enough for you to pay to get it. … [O]pen source software inverts the typical commercial software process. You download the software for free, employ it as you see fit and determine whether it works or not. If it does, you can engage a commercial vendor for support. If it doesn’t, you’re not out the cost of a license. This shift has been transformative in how vendors interact with their customers, whether they’re selling open source software or not.

This shift involves more than just a company’s choice of product for a particular use. Because you can download and experiment with software before even committing to finding a budget, you can explore various uses, and perhaps find new capabilities you didn’t even know you had.

O’Grady delivers a keynote next week, May 26th at Lucene Revolution, “All Data Big and Small“, and chairs a panel discussion, “Search for Tomorrow (RDBMS for Yesterday).” And, it turns out, the interplay of open source, search and data growth are a good example of this virtuous cycle.

We are all–every one of us–generating massive amounts of information. How do you extract, then, the proverbial needle from the haystack? Search is one of the most effective mechanisms for this.

Just as important, however, has been the recognition amongst even conservative IT shops that the database does not need to be the solution to every problem. Search, like a variety of other non-relational tools, is far more of a first class citizen today than it was just a few short years ago.

Read the full post here; you can still register for Lucene Revolution 2011 in San Francisco here.