What is Big Data?

Big Data Is Only Getting Bigger Size Matters How Big Is 50 Petabytes? According to Wikipedia, Big Data is defined as data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. 90% of the data in the world today was created within the last two years THE CHALLENGE IS TO DERIVE MEANING FROM 100% OF BIG DATA. 50 petabytes could hold 25 trillion pages of printed text. It would take about 25 billion floppy disks to store the same amount of data. Big Data is made of structured and unstructured information. Structured information is the data in databases and is about 10% of the story. Unstructured information is 90% of Big Data and is human information like emails, videos, tweets, Facebook posts, callcenter conversations, closed circuit TV footage, mobile phone calls, website clicks. 2.2 million terabytes of new data is created every day A retailer comparing sensor data with pictures of store shelves to see in real-time what products are selling best To analyze Big Data for customers, Autonomy has built the largest private cloud on the planet, at 50 Petabytes (April 2012) As of January 2012, the Library of Congress had collected about 285 terabytes of web archive data. 50 PB would hold the entire contents of the Library of Congress 179 times. 665 77 Million 700MB CDs would occupy 50 PB, so: Combined worldwide album sales of Pink Floyd (all works) Combined worldwide album sales of U2 & Nirvana combined (all works) The human mind processes about one petabyte a second , so 50 PB can store everything in your brain for nearly a minute 64 GB iPhones The amount of information you can store on 2,560,000,000 trees turned into paper In 2010 40% 5.7% 179 25 TRILLION PAGES 50 PB= 2.56 bn 77 m $3.2 BILLION BIG DATA MARKET OVERALL TECHNOLOGY MARKET (Information & Communications) $16.9 BILLION In 2015 GROWTH IN THE BIG DATA MARKET International Data Corporation Forecast WHAT IS 10% STRUCTURED 90% UNSTRUCTURED FOR EXAMPLE: 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2.2m 2012 2011 2010 2009 2008 2007 2006 2005 50 A bank green-lighting a loan application from a credit risk assessment based upon sentiment in a phone conversation that is statistically correlated with credit risk data COMPOUND ANNUAL GROWTH RATE SOLUTION PETABYTE CLOUD = 1 trillion Years of HD-TV video 820,300