Why big data is not data science

"A data scientist has become one of the hottest jobs in the tech world. And when people hear about big data they usually think that this is some part of the data scientist's toolbox. In this talk, we will figure out why it is not true by discussing specific cases, how these fields co-exist, and what are the main differences between the two."
Presented at BuildStuff 2020 (Vilnius, Lithuania) - virtually
Presented at Women Who Code Connect Forward 2020 (San Francisco, US) - virtually

Transcript

1. Why big data is not data science
2. Roksolana Diachuk • Big Data Developer at Captify • Diversity & Inclusion ambassador for Captify Kyiv office • Women Who Code Kyiv Data Engineering Lead and Mentor • Speaker and traveller
3. Agenda 1. What is big Data 2. Difference between big data and data science 3. Practical cases
4. BIG DATA
5. BIG DATA
6. BIG DATA
7. 5 VS
8. Volume Petabytes Terrabytes Gigabytes Exabytes
9. Velocity Batch data Streaming data
10. Variety
11. Veracity
12. Value
13. Big data Structured Unstructured Semi-structured
14. BIG DATA DATA SCIENCE
15. Garbage in - garbage out
16. General ML workflow
17. RDBMS ML model Metrics
18. RDBMS ML model Metrics BIG DATA DATA SCIENCE
19. Background
20. Data Scientist Data Analyst Data Engineer Data Communication Math, Stats, Algorithms Software Engineering
21. Software engineer
22. Data analysis
23. RESPONSIBILITIES
24. • Data processing and cleaning • Developing data pipelines • Storing data • Infrastructure • Data Pre-Processing • Data Analysis • Building ML models • ML models tuning
27. PRACTICAL CASES
29. CASE 1
30. Anomalies detected Streaming data Batch data Anomalies detection
31. Prophet Tableau Anomalies detection
32. CASE 2
33. Data Lake Feature engineering Credit score Credit scoring
34. Psycopg Tensorflow Credit scoring
35. Psycopg Tensorflow
36. CASE 3
37. Data Lake Recommendations User and item ratings Recommender systems
39. CASE 4
40. ETL ETL ETL Data Lake Data lake
41. Tableau Data lake
42. CONCLUSIONS
43. Differences • Goals • Responsibilities • Background • Results of work
44. dead_flowers22 roksolana-d roksolanadiachuk roksolanad My contact info
45. Thank you for attention