Don’t Build Data Castles Made of Sand

The classic rock fans in the ReadWriteWeb audience will remember the Jimi Hendrix song, “Castles Made of Sand.” As someone who’s been in enterprise search for decades, this song often comes to mind when thinking about data within an organization.

Individual bits of enterprise information can be compared with the grains of sand on a beach. Much like the data in your organization – both structured and unstructured – there can be quintillions of grains of sand on a beach, and no two are alike. (Indeed, The Economist reported last year that 150 exabytes of data were created in 2005. One exabyte is equal to one quintillion bytes and, according to Wikipedia, is the equivalent of about 50,000 years of DVD quality video.)

Sponsor

Just like the waves in the ocean, there won’t be a slowdown in data proliferation anytime soon, if ever. There will be a constant ebb and flow of data and information. Your business generates an incredible amount of data every single day. E-mail, phone calls, memos, reports, spreadsheets, community forums and the like create thousands of data files housed throughout your network.

In an attempt extract value from their data, many organizations build central repositories into which they move all data, so that it can be more easily searched and organized to offer actionable insights. This sounds like a good idea on the surface, but there’s a problem with that strategy: data proliferation. It requires an enormous amount of time and resources to move and organize new data into that central repository. And by the time this is complete, you’ve missed the negative comment in your user community from your most important customer by two days.

Think back to the Hendrix song. As the famous lyric goes, “And so castles made of sand/ fall in the sea/ eventually…” A centralized data repository is a lot like building a sand castle; the castle must be continually rebuilt and organized once a new wave of data comes in and wreaks havoc on the old.

The increased proliferation of data comes at a time when, thanks to Google, employees are accustomed to finding the information they want immediately – and your customers expect answers to resolve their issues just as fast. Searching for information in the enterprise, though, remains a challenge for most organizations. Data is expected to grow by 800% over the next five years, according to Gartner’s 2010 top 10 IT issues list, and 80% of it will be unstructured. I’d be remiss if I didn’t mention how unstructured data can be dispersed across multiple systems, on cloud and on-premise servers and in different geographies, depending on the size and scope of a company.

Unstructured data can be made up of any number of business tools – documents, metadata, audio and video, email, etc. – and can be spread across disparate knowledgebases within an organization. The challenge of putting this all into one place is enormous. This type of data contains extremely valuable information, but if the person who needs it can’t locate it and view it within context, it’s useless.

Instead, companies should look at their data in a holistic manner, as you would the grains of sand that make up a beach. Rather than try to build that sand into a castle that won’t hold up to new waves crashing upon it, the better approach is to examine and recognize the value of your data where it lies.

Tools for information discovery are advanced enough to allow data to be found in its natural habitat. For unstructured data, this can provide a real benefit. Some data types are going to function best when stored in a purpose-built repository rather than being shoehorned in to a one-size-fits-all system.

Taking this holistic approach to your data will not only save you time in the long run, it will provide new value to your data in the form of cost savings, efficiency gains and risk reductions. Moving your data delays delivery of its value and hastens its staleness. Leaving the data where it resides and providing unified access to it not only helps make the most of existing IT investments, it provides a full view of the information that matters most to you, which helps you make better decisions.

Anyone who’s ever watched a sandcastle demolished by the waves understands that you can’t make sand retain any form or location. To get the most value out of data, we must embrace the real-time nature of ever-changing business data, and harness it where it resides.

Photo by tommco

Discuss