The subject of Big Data is both fascinating and challenging – fascinating from the perspective of how organisations need to adapt their Business Intelligence (BI) strategies to meet the changing demands from sources and types of data, and challenging from the perspective that this is an evolving area.
From analysis conducted by organisations such as Gartner and Intel, organisations that adopt Big Data strategies are likely to outperform their competitors by around 20%.
So let’s try and understand what Big Data actual is. Data analysis and visualisation is not new, organisations have been using a variety of BI tools for decades, but the traditional tools can no longer handle the Volume, Variety and Velocity of data now flowing through an organisation.
The data is there somewhere – the key is to close the gap between the data and business needs.
Most organisations are overwhelmed by the amount and variety of data flowing through the organisation that they struggle to store the data let alone analyse and present it in a real-time meaningful manner.
What is Big Data?
|Traditional BI||Big Data|
|Structured data||Structured, Semi Unstructured and Unstructured Data|
|Deductive Analysis1||Inductive Analysis2|
|Data obtained from pre-defined processes and systems||Data obtained from a variety of different sources and platforms|
|Data consolidated to a data warehouse to facilitate analysis||Data analysed at source|
|Used to analyse and monitor the health of an organisation||Used to optimise processes, identify trends and develop new services and products|
|Part real-time / Part Batch||Real-time Analysis|
|Static Data Analysis||End User Interaction|
1 Assumes understanding of patterns based on structured data sets – analysing organisation’s goals and translating them into performance indicators and relevant business questions.2 Predictive Analytics applies inductive reasoning to identify interrelations, patterns and trends.
Traditional BI analyses structured relational data sets (i.e. customer invoices). Analytics for Big Data is different – it analyses structured and unstructured data to optimise processes, reduce operational costs and identify new service / product requirements. Success is in reconciling both approaches within the same strategic BI framework.
The strategies surrounding predictive modelling is a whole different matter but analytics teams need to weigh the benefits of using the full assortment of data at their disposal. That might be necessary for some applications — for example, fraud detection, which depends on identifying outliers in a data set that point toward fraudulent activity. In other cases, predictive modelling in big data environments can be done effectively, and more quickly, with smaller data sets through the use of data sampling techniques.
The Challenges of Big Data
- Information Strategy – Big data is causing enterprises to find new ways to leverage information sources to drive growth.
- Data Analytics – You need to predict future customer behaviours, trends and outcomes!
- Governance – How will you govern your organization’s information assets?
- User expectations – Your employees are demanding more access to big data sources. What’s your plan to manage access to these information sources? What are the use cases?
- Costs – How can you deliver access to big data in a rapid and cost-effective way to support better decision-making?
- Tools – How will you link these new sources of diverse data?
- Resource / Expertise – Do you have the necessary skills?
The Future – HR example
“Big data is the future of recruiting” – Driven by “People Analytics”
- Xerox recently cut the attrition rate at its call centres by 20% by using big-data tools! How? The answers lie, in large part, within the very information that HR already encounters daily…
- Companies comb the Internet – social media
- Data aggregation also happens in-house
- HR departments can also deploy tests and games
- Once this material is in hand, algorithms go to work
- Of course, all of this takes some expertise.
Big Data Intelligence using SharePoint as a tool in its own right and /or a presentation layer to surface information generated by other tools to deliver a collaboration platform. The SharePoint BI tool-set includes:
- Excel services
- Report builder
- Performance Point
- Visio Services
- Access Services
- Big Data is already here
- The strategic value of big data is compelling
- Real-time analysis of structured and unstructured data is critical to secure competitive advantage
- A formal strategy to exploit Big Data is key
- Predictive analysis should complement deductive analysis
- Governance data policies must underpin a data strategy
- In order for a business to thrive end users must have access and interact with data intelligence
- Collaboration and information sharing must be a given
- Have the right infrastructure to deliver a scalable platform
If you would like to view the Big Data Infrastructure presentation delivered by our partners, Onyx, please visit: http://onyx.net/media/view/24/big-data-presentation