Big Data & Big Data Outsourcing (BDO) – what’s the big deal?

The meteoric rise of all things Internet and social media related creates vast amounts of unstructured data every hour of every day; facts, figures, opinions, references, page views, click -throughs, likes, tweets & re-tweets, produce terabyte on terabyte of valuable data, together of course with a large amount of inane & unimportant drivel. 

A recent IDC report showed a 1000% explosion of on-line and off-line data volumes over the past 5 years, while global consulting firm McKinsey estimates that the universe of digital information will increase 20-fold by the year 2020.   This increase in data about you and your company, provides unparalleled opportunities for listening, analysing, predicting, forecasting and then acting upon the analysis to attract & retain customers, increase sales, reduce costs & improve efficiency. 

But the volume, scale and nature of the data, makes it incredibly difficult to analyse and this complexity moves it beyond the means of many traditional data warehouse or business intelligence suites which were designed for the structured data world of CRM, before the explosion of social media and cloud computing & the complex merger of structured and unstructured data sets made analysis far more complicated.

The potential benefits of improving a business’s understanding of data analytics would however seem to be well worth the effort, a recent University of Texas Report, sponsored by Sybase predicts that if an average Fortune 1000 company could increase the usability of it’s data by 10%, it could result in an annual revenue increase of $2.1billion.

So how do organisations tackle the problem of Big Data and can external support in the form of outsourcing help to overcome the challenges?  General Atlantic & Sequoia would seem to think so, as this week they have announced $108m private equity investment into Mu Sigma a Chicago based big data firm who provide data analytics outsourcing to some of the world’s largest businesses.  The rapidly growing firm takes customer’s data and turns it into business insights, meaning customers don’t have to build their own in-house big data expertise.  Mu Sigma describe their services as integrating the disciplines of business, math, and technology in a sustainable global delivery model.

The necessary skills required to do big data are often in short supply internally and the increasing data challenge combined with the significant revenue opportunity mean the use of external support to data mine, analyse, report and recommend are in hot demand.   The solution on the face of it is a simple one and lends itself to an outsourcing model – the customers provide their data to the outsourcer, the vendors experts then go to work on the data, using their sophisticated algorithms and tools designed for huge workloads and based on cloud technology.

Mu Sigma are following in he footsteps of Opera Solutions who raised $84 million in September for its similar service offering and who already turn over in excess of $100m p/a.  Mu Sigma grew 886% between 2008 and 2010 and that growth looks set to continue over the coming years as Big Data, Cloud and Social Media continue to become ever more established.  The large-scale ITO and BPO players are also developing their skills in data analytics through both acquisition (Cognizant acquired Market RX in 2007) and internal development and expansion.

The logic of BDO make perfect sense, there is however an understandable level of customer reticence in handing over the data equivalent of their crown jewels to a third party with the responsibility to make critical business decisions - but with a balance of internal and external oversight and a sensible structure, the right balance can be reached.

Many customers say that the project structure is critical, following a “4-1-1”: four offshore analytics consultants, one consultant at the client site, and one well-connected client employee on the team for example; The onshore consultant is responsible for communication and coordination between the offshore team and the client as in traditional outsourcing models.  The employee’s job is to ensure that the analysis is consistent with the decisions the organization wants to make, and to communicate the outcome to the appropriate executives. The result of a decision analysis is useful only if it’s implemented, and offshore analysts can’t easily influence executives to adopt a recommended action. It’s also important that consultants on the vendor’s team have deep domain expertise.

However it’s structured, there is no doubt that outsourcing of big data analytics has the potential to be huge business and companies like Mu Sigma & Opera Solutions are likely to be key players in defining the market place.

There aren’t many organisations with the internal capability to provide the mix of mathematics, engineering, programming and analysis to mine the vast data sets in the right way to yield the results that they want and a more cost effective option that can yield more meaningful results will be to outsource the analytics to companies with proven expertise and who’s core competency is to provide your business with the answers to the big data challenge.

So is Big Data a Big Deal?   Absolutely…watch out for the unstoppable rise of BDO during 2012 and beyond

Barry Matthews