Big Data Initiatives: Most Organizations Still Coming to Grips
What's driving big data initiatives? According to a new report out of the University of Oxford, key factors include risk and financial management and the need to understand and predict customer behaviors to find new ways to engage with them. However, despite these imperatives, few organizations are actually in the "execute" phase of their big data strategies, as most are still educating themselves, exploring options, or, at best, testing out ideas and technologies.
Other driving factors behind big data initiatives include operational optimization, new business model development, and employee collaboration, according to the report, "Analytics: The real-world use of Big Data," produced by the IBM Institute for Business Value in collaboration with Oxford's Saïd Business School.
The report was the result of the 2012 Big Data @ Work Study, a global survey of 1,144 academics, subject matter experts, and executives from the business and IT fields across 26 industries and 95 countries. The report looked at how organizations view, approach, and use big data to benefit their business.
The report's authors noted there was some confusion among survey respondents as to the definition of big data. Some described it as a greater volume of data, some as a new type of data and analysis, and others as a new need for real-time information analysis.
The report defined big data as having four characteristics:
- Volume, which refers to the sheer quantity of data involved;
- Variety, which refers to different types of data and data sources, including structured, semi-structured, and unstructured data, where unstructured data is data that doesn't fit into traditional databases and includes text, audio, images, and video;
- Velocity, which refers to the increasing speed of data creation, processing, and analysis; and
- Veracity, which refers to the level of uncertainty associated with some types of data, such as the weather or people's future decisions, and the need for organizations to acknowledge and plan for uncertainty in some of their big data.
Dealing with data that has considerable volume, variety, velocity, and uncertain veracity requires a solid foundation of information technology, according to the report. The survey found the nearly two-thirds of organizations pursuing big data initiatives started with technology that was integrated, scalable, extensible, and secure. The next most common information management foundation components were a scalable storage infrastructure and high capacity data warehouse to support the rapid growth of data.
Once organizations embarked on their big data initiatives, more than half reported that they used internal data as their primary source.
"Internal data is the most mature, well understood data available to organizations," stated the report. "It has been collected, integrated, structured, and standardized through years of enterprise resource planning, master data management, business intelligence, and other related work."
But the whole point of collecting big data is to use it to solve problems, and that's where analytics comes in, according to the report. More than 75 percent of survey respondents that had big data initiatives underway reported that they used core analytics capabilities, such as querying and reporting or data mining to analyze the data. More than 67 percent used predictive modeling, and 71 percent used data visualization skills to view and analyze the huge quantity of data.
"Acquiring or developing these more advanced technical and analytic capabilities required for big data advancement is becoming a top challenge among many organizations with active big data efforts," stated the report.
The report identified four stages of big data adoption as organizations' awareness and involvement in big data increased:
- Twenty-four percent of survey respondents were in the "educate" stage, meaning they weren't using big data yet, either because they were unaware of it or because they hadn't yet determined its benefits to their organization;
- Forty-seven percent of respondents were in the "explore" stage, which involves developing the business case and roadmap for using big data;
- Twenty-two percent were in the "engage" stage, which involves developing and testing the skills and technologies necessary for big data initiatives; and
- Only 6 percent of respondents were in the "execute" stage, where big data and analytics capabilities were widely implemented in the organization.
Following up on the findings of the survey, the report provided five key recommendations for organizations considering big data initiatives:
- Commit initial efforts to customer-centric outcomes, meaning that organizations should start with analytics that will help them understand their customers' needs and anticipate their future behaviors;
- Develop an enterprise-wide big data blueprint that covers the organization's vision, strategy, and requirements for big data so everybody in the organization understands why and how they plan to use big data;
- Start with existing data to achieve near-term results;
- Build analytics capabilities based on business priorities by investing in the necessary tools and skills; and
- Create a business case based on measurable outcomes to create a comprehensive and workable plan.
"The most effective big data solutions identify the business requirements first, and then tailor the infrastructure, data sources, and quantitative analysis to support that business opportunity," stated the report.
According to the report's authors, big data initiatives can help organizations make decisions smarter, faster, and in a way that actually makes a difference.
The complete report is available on IBM's site.