Slide 1

Slide 1 text

BIG DATA Arnon Rotem-Gal-Oz Director of Technology Research, Amdocs The blind men and the elephant. Poem by John Godfrey Saxe (Cartoon originally copyrighted by the authors; G. Renee Guzlas, artists http://www.nature.com/ki/journal/v62/n5/fig_tab/4493262f1.html

Slide 2

Slide 2 text

1880 US Census

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Hollerith 
 Tabulating 
 Machine Hollerith photos by Martin Wichary : http://www.flickr.com/photos/ mwichary/4358926764/in/photostream/

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

ource: Silicon Angle http://siliconangle.com/blog/2013/11/13/how-big-is-big-data-really/ Big data happens when the data you have to process is bigger than what you can process in the given time with current technologies

Slide 7

Slide 7 text

Myth: Big data = keep all data Source: Big Data Public Private Forum : http://www.big-project.eu/sites/default/files/ D2.2.1_First%20draft%20of%20Technical%20white%20papers_FINAL_v1.01_0.pdf

Slide 8

Slide 8 text

Source: Big Data Public Private Forum : http://www.big-project.eu/sites/default/files/ D2.2.1_First%20draft%20of%20Technical%20white%20papers_FINAL_v1.01_0.pdf

Slide 9

Slide 9 text

Some Telco Numbers Source: Wikipedia http://upload.wikimedia.org/wikipedia/commons/5/50/Telephone_operators,_1952.jpg

Slide 10

Slide 10 text

So, what do we do 
 with all this data? Wikipedia http://upload.wikimedia.org/wikipedia/commons/0/06/UPS_Truck.jpg

Slide 11

Slide 11 text

It’s the insights, stupid* * With apologies to Bill Clinton

Slide 12

Slide 12 text

ource: Silicon Angle http://siliconangle.com/blog/2013/11/13/how-big-is-big-data-really/ Big data analytics is when sample = N • Big data happens when the data you have to process is bigger than what you can process in the given time with current technologies

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

“My daughter got this in the mail!, She’s still in high school, and you’re sending her coupons for baby clothes and cribs? Are you trying to encourage her to get pregnant?” Source: Forbes http://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her- father-did/

Slide 15

Slide 15 text

We need to watch out that Analytics won’t get too creepy

Slide 16

Slide 16 text

When people hear big data they think fast data Source: Steve Jones Cap Gemini http://www.no.capgemini.com/node/778541

Slide 17

Slide 17 text

Subscribers Collect & Filter Correlate (simplified) Network proactive care flow Account Event Store Identify & Predict Network Failures Reimburse VIPs Prioritize technicians Identify impact on high valued Accounts

Slide 18

Slide 18 text

ource: Silicon Angle http://siliconangle.com/blog/2013/11/13/how-big-is-big-data-really/ Big data is when we can handle data fast enough to make a difference • Big data happens when the data you have to process is bigger than what you can process in the given time with current technologies • Big data analytics is when sample = N

Slide 19

Slide 19 text

Technology space

Slide 20

Slide 20 text

The Elephant in the room

Slide 21

Slide 21 text

Hadoop Stack Map/ Reduce HDFS HBase Pig Hive Zoo Keeper Oozie Mahout Giraph

Slide 22

Slide 22 text

Schema on read

Slide 23

Slide 23 text

Move data to computation

Slide 24

Slide 24 text

Maybe we should rethink moving data to computation… Source : http://my-inner-voice.blogspot.co.il/2012/06/haddop-101-paper-by-miha-ahronovitz-and.html

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

Map/reduce Source: http://www.bodhtree.com/blog/2012/10/18/ever-wondered-what-happens-between-map-and-reduce/

Slide 27

Slide 27 text

Customer Segmentation First name Last name ARPU Age Device Country … Mr. Smith 100 22 iPhone 5s,White USA John Doe 87 42 Samsung Galaxy S5,Gold France Lady In Red 105 21 Samsung Note 3, White UK … Uluru, Australia by Stuart Edwards (cc) http://en.wikipedia.org/wiki/Uluru#mediaviewer/File:Uluru_Panorama.jpg

Slide 28

Slide 28 text

K-Means ARPU Age Source : http://pypr.sourceforge.net/kmeans.html

Slide 29

Slide 29 text

K=3 ARPU Age ARPU Age Source : http://pypr.sourceforge.net/kmeans.html

Slide 30

Slide 30 text

New paradigms Map/ Reduce HDFS HBase Pig Hive Zoo Keeper Oozie Mahout Giraph

Slide 31

Slide 31 text

New Paradigms Map/ Reduce HDFS HBase Pig Hive Zoo Keeper Oozie Mahout YARN Giraph

Slide 32

Slide 32 text

New Paradigms Map/ Reduce HDFS HBase Pig Hive Zoo Keeper Oozie Mahout YARN Giraph Spark Storm Slider Flink Impala Tez Presto

Slide 33

Slide 33 text

Amdocs Analytics & Data Management Heritage 2013 • Proactive Care • TerraScale • Network optimization • Real time analytics platform • Single product catalog • BSS–OSS Integration • CRM-Billing Integration OSS Analytics Platform, 16 Analytics Patents • aLDM logical data model • Policy control Network Analytics CRM 2000 2008 Acquisitions Portfolio

Slide 34

Slide 34 text

Information Security Level 2 – Sensitive © 2014 – Proprietary and Confidential Information of Amdocs 34 Touchpoints & Applications CRM Self Service E-Mail PCRF SMS Other Wi-Fi Offload Campaign Mng. • • • • • • • Operational Envelope & Platform Administration • Security Management • Configuration Management • Services Inventory • Performance Management • Fault Management • Logger Collect & Ingest Transform & Enrich Aggregate & Correlate Drive Insight Close the Loop Machine Learn & Score Application-Ready Data and Analytics/ML Insights Entities and Profiles Detailed Data OSS Probes Social RAN Inventory Usage & 
 Charging CRM Real-Time & Batch Connectors Insight Platform Marketing Analytical Application 
 Framework: Dashboards & Visualisation Decisioning Engine Dynamic Micro Segmentation Network Care Operations

Slide 35

Slide 35 text

ource: Silicon Angle http://siliconangle.com/blog/2013/11/13/how-big-is-big-data-really/ • Big data happens when the data you have to process is bigger than what you can process in the given time with current technologies • Big data analytics is when sample = N • Big data is when we can handle data fast enough to make a difference

Slide 36

Slide 36 text

Additional takeaways • CSPs have always been in the big data business – they just didn’t know it • Big data is not a panacea • Hadoop is shaping up as the big data OS – Though there are alternatives arriving from the cloud arena (mesos, kubernetes)

Slide 37

Slide 37 text

What we covered here is not even the tip of the iceberg Source: wikimedia http://commons.wikimedia.org/wiki/File:Iceberg.jpg

Slide 38

Slide 38 text

Arnon Rotem-Gal-Oz Director of Technology Research, Amdocs [email protected] / [email protected]