Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Data Science: It’s Easy as Py!

Avatar for Michael Becker Michael Becker
February 25, 2016
Technology
44
0

Data Science: It’s Easy as Py!

Over the last several years, many amazing Python projects have been created for working with data, making Python one of the top languages for data scientists. In this talk, Michael Becker will cover all five stages of the OSEMN data science process, and several Python libraries that can be used at each stage. To demo, he will illustrate the creation of a language classification algorithm, using Wikipedia articles.

Avatar for Michael Becker

Michael Becker

February 25, 2016

More Decks by Michael Becker

See All by Michael Becker
Data Science: It’s Easy as Py! (With Notes)
Avatar for Michael Becker beckerfuffle
0
42

Other Decks in Technology

See All in Technology
実体験から学ぶ分析エージェントの開発と運⽤
Avatar for Recruit recruitengineers
PRO
8
2.4k
パッチ更新でterraform initが落ちた日
Avatar for gr1m0h grimoh
1
140
分析の民主化を支えるメタデータ整備の実践
Avatar for KAKEHASHI kakehashi
PRO
4
1.4k
Tofu-Controller lightning talk @ Containerday 2026
Avatar for Giovanni Toraldo gionn
0
100
V8のArray.prototype.flatを最大約20倍速くするまでと、巨大OSSへの大規模コミットの道のり
Avatar for Riya Amemiya riyaamemiya
0
290
サブエージェントと Claude Mods で作る独自 Advisor ツールの話
Avatar for Kenichi Odo kenichiodo
2
160
Getting Started with Google Antigravity
Avatar for Guillaume Laforge glaforge
0
300
Rustuna: Rustによる高速なOptuna実装
Avatar for Preferred Networks pfn
PRO
0
190
DevelopersIO 2026 知識の地図を描く ─ オントロジーとは何か？
Avatar for Satoru Ishikawa ishikawa_satoru
1
1.7k
How to Design Configuration for Rails Applications
Avatar for Shinichi Maeshima willnet
1
770
Gemini Intelligence 入門 ― AppFunctions と Gemini Nano
Avatar for takuro abe takuroabe
0
250
会社説明資料
Avatar for パーソル＆サーバーワークス株式会社 persolserverworks
0
9.8k

Featured

See All Featured
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
Avatar for Tammy Everts tammyeverts
11
1.1k
Designing Dashboards & Data Visualisations in Web Apps
Avatar for Des Traynor destraynor
232
55k
Taking LLMs out of the black box: A practical guide to human-in-the-loop distillation
Avatar for Ines Montani inesmontani
PRO
3
2.5k
Keith and Marios Guide to Fast Websites
Avatar for Keith Pitt keithpitt
413
23k
The Cult of Friendly URLs
Avatar for Andy Hume andyhume
79
7k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
9
1.6k
Navigating Weather and Climate Data
Avatar for Ryan Abernathey rabernat
0
560
WCS-LA-2024
Avatar for Leonardo Collado-Torres lcolladotor
0
890
The Success of Rails: Ensuring Growth for the Next 100 Years
Avatar for Eileen M. Uchitelle eileencodes
47
8.4k
JavaScript: Past, Present, and Future - NDC Porto 2020
Avatar for David Neal reverentgeek
52
6.1k
Helping Users Find Their Own Way: Creating Modern Search Experiences
Avatar for Dan Newman danielanewman
31
3.4k
A Soul's Torment
Avatar for Nat Ma seathinner
8
3.8k

Transcript

  1. Data Science: It’s Easy as Py! Michael Becker #PhillyPUG @beckerfuffle

  2. What My Coworkers Think I Do #PhillyPUG @beckerfuffle

  3. What I Actually Do #PhillyPUG @beckerfuffle

  4. What I'll Cover • What is Data Science? #PhillyPUG @beckerfuffle

  5. What I'll Cover • What is Data Science? • The

    OSEMN process #PhillyPUG @beckerfuffle

  6. What I'll Cover • What is Data Science? • The

    OSEMN process • Writing a custom web scraper #PhillyPUG @beckerfuffle

  7. What I'll Cover • What is Data Science? • The

    OSEMN process • Writing a custom web scraper • Scrubbing the data #PhillyPUG @beckerfuffle

  8. What I'll Cover • What is Data Science? • The

    OSEMN process • Writing a custom web scraper • Scrubbing the data • Data Exploration #PhillyPUG @beckerfuffle

  9. What I'll Cover • What is Data Science? • The

    OSEMN process • Writing a custom web scraper • Scrubbing the data • Data Exploration • Predictive Modeling #PhillyPUG @beckerfuffle

  10. What is Data Science? http://www.dataists.com/2010/09/the-data-science-venn-diagram/ #PhillyPUG @beckerfuffle

  11. #PhillyPUG @beckerfuffle

  12. OSEMN! • Obtain #PhillyPUG @beckerfuffle

  13. OSEMN! • Obtain • Scrub #PhillyPUG @beckerfuffle

  14. OSEMN! • Obtain • Scrub • Explore #PhillyPUG @beckerfuffle

  15. OSEMN! • Obtain • Scrub • Explore • Model #PhillyPUG

    @beckerfuffle

  16. OSEMN! • Obtain • Scrub • Explore • Model •

    iNterpret #PhillyPUG @beckerfuffle

  17. Language Classification #PhillyPUG @beckerfuffle

  18. Language Classification #PhillyPUG @beckerfuffle

  19. Language Classification #PhillyPUG @beckerfuffle

  20. Obtain @beckerfuffle #PyOhio API?! Where we’re going we don’t need

    APIs! #PhillyPUG @beckerfuffle

  21. Scrape All The Things! #PhillyPUG @beckerfuffle

  22. Scrape All The Things! #PhillyPUG @beckerfuffle

  23. Scrape All The Things! #PhillyPUG @beckerfuffle

  24. Scrape All The Things! #PhillyPUG @beckerfuffle

  25. Scrub @beckerfuffle #PyOhio @beckerfuffle #PyOhio #PhillyPUG @beckerfuffle

  26. Scrub #PhillyPUG @beckerfuffle

  27. Scrub #PhillyPUG @beckerfuffle

  28. Scrub #PhillyPUG @beckerfuffle

  29. Explore @beckerfuffle #PyOhio #PhillyPUG @beckerfuffle

  30. Pandas • 2 basic data structures #PhillyPUG @beckerfuffle

  31. Pandas • Series #PhillyPUG @beckerfuffle

  32. Pandas • Series #PhillyPUG @beckerfuffle

  33. Pandas • Series #PhillyPUG @beckerfuffle

  34. Pandas • Series #PhillyPUG @beckerfuffle

  35. Pandas • Series #PhillyPUG @beckerfuffle

  36. Pandas #PhillyPUG @beckerfuffle

  37. Pandas #PhillyPUG @beckerfuffle

  38. Pandas #PhillyPUG @beckerfuffle

  39. Pandas #PhillyPUG @beckerfuffle

  40. Pandas @beckerfuffle #PyOhio #PhillyPUG @beckerfuffle

  41. Pandas • Series #PhillyPUG @beckerfuffle

  42. Pandas • Series #PhillyPUG @beckerfuffle

  43. Pandas • Series #PhillyPUG @beckerfuffle

  44. Pandas • Series #PhillyPUG @beckerfuffle

  45. Pandas #PhillyPUG @beckerfuffle

  46. Pandas #PhillyPUG @beckerfuffle

  47. Pandas #PhillyPUG @beckerfuffle

  48. Pandas #PhillyPUG @beckerfuffle

  49. Pandas #PhillyPUG @beckerfuffle

  50. Pandas #PhillyPUG @beckerfuffle

  51. Pandas #PhillyPUG @beckerfuffle

  52. Pandas #PhillyPUG @beckerfuffle

  53. Pandas #PhillyPUG @beckerfuffle

  54. Pandas #PhillyPUG @beckerfuffle

  55. Machine Learning #PhillyPUG @beckerfuffle

  56. Machine Learning #PhillyPUG @beckerfuffle

  57. Machine Learning #PhillyPUG @beckerfuffle

  58. Supervised Learning #PhillyPUG @beckerfuffle

  59. Unsupervised Learning #PhillyPUG @beckerfuffle

  60. Visualize #PhillyPUG @beckerfuffle

  61. Visualize #PhillyPUG @beckerfuffle

  62. KMeans Clustering #PhillyPUG @beckerfuffle

  63. KMeans Clustering #PhillyPUG @beckerfuffle

  64. KMeans Clustering #PhillyPUG @beckerfuffle

  65. KMeans Clustering bytemuse.com/post/k-means-clustering- visualization/ #PhillyPUG @beckerfuffle

  66. Visualize #PhillyPUG @beckerfuffle

  67. Visualize #PhillyPUG @beckerfuffle

  68. Visualize #PhillyPUG @beckerfuffle

  69. Visualize #PhillyPUG @beckerfuffle

  70. Visualize #PhillyPUG @beckerfuffle

  71. Explore More #PhillyPUG @beckerfuffle

  72. Explore More #PhillyPUG @beckerfuffle

  73. Explore More #PhillyPUG @beckerfuffle

  74. Explore More #PhillyPUG @beckerfuffle

  75. Explore More #PhillyPUG @beckerfuffle

  76. Explore More #PhillyPUG @beckerfuffle

  77. Explore Even More #PhillyPUG @beckerfuffle

  78. But There’s More! @beckerfuffle #PyOhio #PhillyPUG @beckerfuffle

  79. Supervised Learning • Scikit-learn overview • Model Distribution • Data

    flow • RabbitMQ • Demo • Scalability • Other considerations @beckerfuffle #PyOhio @beckerfuffle #PyOhio

  80. Supervised Learning #PhillyPUG @beckerfuffle

  81. Supervised Learning #PhillyPUG @beckerfuffle

  82. Supervised Learning #PhillyPUG @beckerfuffle

  83. Supervised Learning #PhillyPUG @beckerfuffle

  84. The Model #PhillyPUG @beckerfuffle

  85. The Model #PhillyPUG @beckerfuffle

  86. The Model #PhillyPUG @beckerfuffle

  87. The Model • Scikit-learn overview • Model Distribution • Data

    flow • RabbitMQ • Demo • Scalability • Other considerations #PhillyPUG @beckerfuffle

  88. The Model • Scikit-learn overview • Model Distribution • Data

    flow • RabbitMQ • Demo • Scalability • Other considerations #PhillyPUG @beckerfuffle

  89. The Model • Scikit-learn overview • Model Distribution • Data

    flow • RabbitMQ • Demo • Scalability • Other considerations #PhillyPUG @beckerfuffle

  90. The Model • Scikit-learn overview • Model Distribution • Data

    flow • RabbitMQ • Demo • Scalability • Other considerations #PhillyPUG @beckerfuffle

  91. The Model #PhillyPUG @beckerfuffle

  92. The Model • Scikit-learn overview • Model Distribution • Data

    flow • RabbitMQ • Demo • Scalability • Other considerations #PhillyPUG @beckerfuffle

  93. The Model • Scikit-learn overview • Model Distribution • Data

    flow • RabbitMQ • Demo • Scalability • Other considerations #PhillyPUG @beckerfuffle

  94. The Model #PhillyPUG @beckerfuffle

  95. The Model #PhillyPUG @beckerfuffle

  96. The Model #PhillyPUG @beckerfuffle

  97. The Model • Scikit-learn overview • Model Distribution • Data

    flow • RabbitMQ • Demo • Scalability • Other considerations #PhillyPUG @beckerfuffle

  98. The Model • Scikit-learn overview • Model Distribution • Data

    flow • RabbitMQ • Demo • Scalability • Other considerations #PhillyPUG @beckerfuffle

  99. The Model • Scikit-learn overview • Model Distribution • Data

    flow • RabbitMQ • Demo • Scalability • Other considerations #PhillyPUG @beckerfuffle

  100. The Model #PhillyPUG @beckerfuffle

  101. The Model #PhillyPUG @beckerfuffle

  102. The Model #PhillyPUG @beckerfuffle

  103. The Model #PhillyPUG @beckerfuffle

  104. The Model #PhillyPUG @beckerfuffle

  105. The Model #PhillyPUG @beckerfuffle

  106. Supervised Learning #PhillyPUG @beckerfuffle

  107. X_train #PhillyPUG @beckerfuffle

  108. Supervised Learning #PhillyPUG @beckerfuffle

  109. y_train #PhillyPUG @beckerfuffle

  110. y_train #PhillyPUG @beckerfuffle

  111. Tuning #PhillyPUG @beckerfuffle

  112. RandomizedSearchCV #PhillyPUG @beckerfuffle

  113. RandomizedSearchCV #PhillyPUG @beckerfuffle

  114. RandomizedSearchCV #PhillyPUG @beckerfuffle

  115. Skull github.com/EducationalTestingService/ skll #PhillyPUG @beckerfuffle

  116. sklearn.metrics • accuracy_score • classification_report • confusion_matrix #PhillyPUG @beckerfuffle

  117. sklearn.metrics • accuracy_score • classification_report • confusion_matrix #PhillyPUG @beckerfuffle

  118. Overfitting #PhillyPUG @beckerfuffle

  119. Overfitting #PhillyPUG @beckerfuffle

  120. accuracy_score #PhillyPUG @beckerfuffle

  121. accuracy_score #PhillyPUG @beckerfuffle

  122. classification_report • classification_report #PhillyPUG @beckerfuffle

  123. classification_report • classification_report #PhillyPUG @beckerfuffle

  124. classification_report • classification_report #PhillyPUG @beckerfuffle

  125. I’m Confused • classification_report • accuracy_score • confusion_matrix #PhillyPUG @beckerfuffle

  126. I’m Confused • classification_report • accuracy_score • confusion_matrix #PhillyPUG @beckerfuffle

  127. I’m Confused #PhillyPUG @beckerfuffle

  128. I’m Confused #PhillyPUG @beckerfuffle

  129. I’m Confused #PhillyPUG @beckerfuffle

  130. I’m Confused #PhillyPUG @beckerfuffle

  131. I’m Confused #PhillyPUG @beckerfuffle

  132. I’m Confused #PhillyPUG @beckerfuffle

  133. y_train #PhillyPUG @beckerfuffle

  134. INterpret #PhillyPUG @beckerfuffle

  135. More Info jupyter.org #PhillyPUG @beckerfuffle

  136. More Info jupyter.org pandas.pydata.org #PhillyPUG @beckerfuffle

  137. More Info jupyter.org goo.gl/TWHeeo pandas.pydata.org #PhillyPUG @beckerfuffle

  138. More Info jupyter.org goo.gl/TWHeeo pandas.pydata.org scikit-learn.org #PhillyPUG @beckerfuffle

  139. More Info jupyter.org goo.gl/TWHeeo pandas.pydata.org scikit-learn.org #PhillyPUG @beckerfuffle

  140. More Info jupyter.org goo.gl/TWHeeo pandas.pydata.org scikit-learn.org github.com/jakevdp/sklearn_pydata2015 #PhillyPUG @beckerfuffle

  141. More Info jupyter.org goo.gl/TWHeeo pandas.pydata.org scikit-learn.org github.com/jakevdp/sklearn_pydata2015 github.com/EducationalTestingService/skll #PhillyPUG @beckerfuffle

  142. What I do • PennSignals: a framework on which to

    build predictive applications in health care #PhillyPUG @beckerfuffle

  143. Ingest Clean Feature Extraction Model fit Pt ID Time Adm

    Age Lab/Stats … Zip TS/Data Resample Local+Feature+Engineering Data+Labeling Global+Feature+ Engineering Fusion+of++ Multimodal Data Feeding/the/Data/Model Payload Raw+Time+Series Column+Form Fast/query/and/data/mining: Patient+static+data+with+statistical+ summary+of+temporal+data. Open/source/processing/tools Data/Schema:+Simple,+complete+and+self+contained #PhillyPUG @beckerfuffle

  144. All The Things! Twitter: @beckerfuffle Blog: beckerfuffle.com Github: github.com/mdbecker Work

    with me: http://pennmedicine.jobs #PhillyPUG @beckerfuffle