What is Big Data?
The world's data is increasing at exponential rates. This massive, unstructured amount of data coming from non-traditional sources is Big Data.
Wikipedia defines it as:
“Big Data is a broad term for data sets that are so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, data curation, search, sharing, storage, transfer, visualization, and information privacy.”
The infographic from Big Data | Visual.ly is a good starting point for those who still don't quite understand it:
(click to enlarge)
Why is it important?
This data explosion and its analysis have become crucial for innovating and gaining a competitive advantage. It generates various insights in ways that were previously impossible. The Big Data industry is poised to grow from $25 billion in 2015 to over $50 billion in 2017.
And it affects practically every other industry:
What can Big Data analytics do?
Previously, companies relied mainly on transactional data stored in an orderly fashion, but that has changed. A massive amount of data is generated daily from various sources: from an email to a tweet. In the past, organizations discarded their data. Today, investing in storage and analysis has become both affordable and mandatory.
For example, there are 12 terabytes of tweets per day. After filtering out the noise, this data can generate several insights into consumer behavior.
Chart: different communities interacting.
What are the challenges?
- Filtering data: Which data is reliable? Which is up to date?
- Real-time processing: Some data can lose all its value after just a few hours.
- Dealing with the high volume and variety of data.
Read our article on the 5 Vs of Big Data.
This text was originally written by Amar Prabhu on the social network Quora.