What is big data?

Big Data: water wordscape (Photo credit: Marius B)

This post gives you a brief introduction to “big data”, a term used in many circles and in many businesses. The following posts will then give some examples from real business to help you understand the effects this might have on accounting and accountants.

Although the term big data has become mainstream in recent years, it has been used for a decade or more by scientists to simply describe very large amounts of data. Diebold (2003)  defines big data as follows:

Big data refers to the explosion of quantity (and sometimes, quality) of available and potentially relevant data, largely the result of recent and unprecedented advances in data recording and storage technology.

It is hard to believe that this definition although only a decade or so old, bears little resemblance to what can be achieved today in terms of data collection. Devices such as smartphones and tablets in a cloud-computing environment allow users to use cloud-based services (such as software or social networks) and, in turn, data can be collected through these devices and stored elsewhere in the cloud. The result is vast potentially vast amount of data, which can be analysed for many purposes, including business decisions. Facebook has about a billion users, there are about 500 million tweets per day sent on Twitter and Google handles about 3 billion search queries per day. These vast uses of each of the mentioned websites/network generates hitherto unknown amounts of data, some of which may be useful, some of which may not. In an article for Forbes, Feinleib notes three issues with big data, which give a good insight into what it is, and the problems facing business:

1) big-data is ill-defined.. We are not sure what exactly big data is,  but a Jevons Paradox seems to exist in the world of big data. As technology evolved to allow the storage and analysis of large volumes of data, more data is being stored and analysed by organisations.

2) big data is intimidating. He asks “how do we make big data approachable” from perspectives such as having tools to analyse data, to getting the right insights and information from the data.

3) big data is immediate. Huge volumes of data are generated, but the analytical value of this data can decay rapidly. For example, in the near future companies like Google and Groupon may display adverts on mobile devices for businesses in the immediate proximity of a consumer – the time to analyse and act on this data could be a matter of minutes, or even seconds.

References:

Diebold, F. 2003, “ ‘Big Data’ Dynamic Factor Models for Macroeconomic Measurement and Forecasting” (Discussion of Reichlin and Watson papers), in M. Dewatripont, L.P. Hansen and S.Turnovsky (eds.), Advances in Economics and Econometrics, Eighth World Congress of the Econometric Society. Cambridge: Cambridge University Press, 115-122.