Published/updated: October 2012
By Tony Lock
Anyone currently employed in any area of the IT business will be aware, however reluctantly, of the considerable amount of effort being put into marketing ‘Big Data’. During August and September of 2012 Freeform Dynamics surveyed 502 readers of The Register (for sample breakdown see here) to gauge how far organisations have advanced in their adoption of Big Data solutions and the current state of play concerning business analytics. This article looks at where organisations report they currently hold business data, how well they exploit the value locked in their data stores and their thoughts on how things may develop in the near future.
Today the words ‘Big Data’ are currently used and abused in a myriad of ways. From these it is possible to distil the term to be shorthand to refer to a number of advanced data storage, access and analytics technologies aimed at handling high volume and/or fast moving data in a variety of scenarios. These typically involve low signal-to-noise ratios, including but not limited to brand monitoring, log file analysis, high volume transaction monitoring for fraud detection, to name just a few.
Data volumes are growing
It is no news that the amount of data being generated by organisations continues to grow rapidly and this survey illustrates just how much of a challenge this is with over half of respondents reporting that they are experiencing high, or extremely high, rates of data growth and very few experiencing no growth at all (Figure 1).
Of considerably more importance with the vendor focus firmly directed at “Big Data”, is the fact that whilst the expansion of volumes associated with unstructured data sources such as email, office productivity documents, video files etc. is very strong, the reports of growth in structured data sources, such as those associated with relational database systems, is not that much lower.
This begs the question, just how valuable or important are the types of information held in structured sources compared with unstructured repositories?
The value of data
It is evident that the majority of organisations today still keep most of their business critical information in structured repositories, typically relational database management systems (RDBMS). Far fewer hold such data in unstructured sources (Figure 2).
That said, it is worth noting that nearly one in ten of the respondents indicate that they are not sure how such data is held. It is easy to understand why this should be so when the question of data storage architecture has never been an area of intense study for most IT professionals and has never even been thought about by the vast majority of business users.
When asked how things are changing, the research indicates that roughly equal numbers of respondents expect there to be a shift towards either structured or unstructured sources as the repositories for business critical data. That said, the most frequently offered opinion is that there is likely to be little change evident in the near future. This illustrates that Big Data will not be sweeping away the very strong position that relational databases and other established solutions hold in the near term. With so much data out there just how well are organisations exploiting what they already store?
Data value is still ‘underexploited’
The Reg readers who responded to the survey tell us that only rarely is the data their organisations hold exploited as well as it could be. Even a cursory glance at Figure 3 shows very clearly that only around one in eight organisations fully exploit their structured data whilst an even smaller fraction does so for unstructured data.
By Dale Vile
By Bryan Betts and Dale Vile
Yesterdays software delivery processes are not up to dealing with today’s demands, but modernising you approach is not just about implementing Agile, even creating a DevOps culture. You need to focus on some specific, hard-core principles. ...more
By Dale Vile & Jack Vile
Cloud services are increasingly becoming part of the IT delivery mix, but a recent study of 378 senior IT professionals suggests a parallel commitment to ongoing investment in the datacentre. This in turn shines a light on the key role of modern application platforms. ...more
By Tony Lock & Dale Vile
Despite the advent to cloud computing the datacentre remains central to corporate IT. But with demands continuing to escalate, how do you ensure your infrastructure is powered robustly and efficiently? ...more
By Bryan Betts
Many are exploiting cloud computing to drive business advantage, while others are enjoying the flexibility and efficiency of DevOps. But what happens if you use both together in a coordinated manner? The answer is a significant amplification of the benefits of each. ...more
By Dale Vile
Securing the applications and services that underpin your online and mobile presence is one thing, but keeping them secure secure on an ongoing basis is another. How well do your business execs understand this? ...more