I like listening to and learning from smart people (especially when they agree with my way of thinking!) and I came across a particularly smart post recently from Matt Aslett of 451 Research. Matt talked about a trend he is describing as a “Data Hub”, where data is accessed and analyzed without having to be moved. This, of course, will get all the Data Warehouse, EII, EAI, and XYZ data bus guys all spun up around the claims they have been making for years. (By the way, do the data bus guys actually drive your data to another location(s) on some kind of electronic vehicle?).
There has to be a reason why The EII and EAI market never really developed into anything big; and why Hadoop has evolved into the giant it is today – after all isn’t storing and managing a boatload of data what Warehouses were designed to do? My guess is that moving data around just does not work – and that the world is comprised of more than just structured data (BTW: please don’t get me started on how inefficient and brittle SQL is for managing the pedabytes of unstructured data that many organizations now want to analyze). Trust me – I do believe in the world where databases, warehouses, Hadoop and more all live together and thrive. After all, no single data storage format is the panacea for all data management problems. So how about we keep all data, both structured and unstructured, in its proper location and use a flexible open platform that layers on top of these technologies to enable search, discovery and analysis of the data they manage? How about we use smart indexing technology to integrate metadata, actual data and text using fuzzy logic algorithms (not key fields) that provide contextual integration for better application development of a new wave of information applications that scale larger – BIG DATA – and make it easier for business professionals to find stuff in real time? TA DA!! Think about it. That’s pretty cool stuff coming soon to an enterprise near you!!