Tweet
Tweet

Slide 1

Slide 1 text

Realtime predictive analytics using scikit-learn & RabbitMQ Michael Becker @beckerfuffle #pycon2014 1

Slide 2

Slide 2 text

Who Is This Guy? Data guy @ AWeber @beckerfuffle beckerfuffle.com These slides and more @ github.com/mdbecker @beckerfuffle #pycon2014 2

Slide 3

Slide 3 text

What My Coworkers Think I Do @beckerfuffle #pycon2014 3

Slide 4

Slide 4 text

What I Actually Do @beckerfuffle #pycon2014 4

Slide 5

Slide 5 text

What I'll Cover • Scikit-learn overview @beckerfuffle #pycon2014 5

Slide 6

Slide 6 text

What I'll Cover • Scikit-learn overview • Model Distribution @beckerfuffle #pycon2014 6

Slide 7

Slide 7 text

What I'll Cover • Scikit-learn overview • Model Distribution • Data flow @beckerfuffle #pycon2014 7

Slide 8

Slide 8 text

What I'll Cover • Scikit-learn overview • Model Distribution • Data flow • RabbitMQ @beckerfuffle #pycon2014 8

Slide 9

Slide 9 text

What I'll Cover • Scikit-learn overview • Model Distribution • Data flow • RabbitMQ • Demo @beckerfuffle #pycon2014 9

Slide 10

Slide 10 text

What I'll Cover • Scikit-learn overview • Model Distribution • Data flow • RabbitMQ • Demo • Scalability @beckerfuffle #pycon2014 10

Slide 11

Slide 11 text

What I'll Cover • Scikit-learn overview • Model Distribution • Data flow • RabbitMQ • Demo • Scalability • Other considerations @beckerfuffle #pycon2014 11

Slide 12

Slide 12 text

Supervised Learning @beckerfuffle #pycon2014 12

Slide 13

Slide 13 text

Supervised Learning @beckerfuffle #pycon2014 13

Slide 14

Slide 14 text

Supervised Learning @beckerfuffle #pycon2014 14

Slide 15

Slide 15 text

Supervised Learning @beckerfuffle #pycon2014 15

Slide 16

Slide 16 text

Supervised Learning @beckerfuffle #pycon2014 16

Slide 17

Slide 17 text

Supervised Learning @beckerfuffle #pycon2014 17

Slide 18

Slide 18 text

Supervised Learning @beckerfuffle #pycon2014 18

Slide 19

Slide 19 text

Supervised Learning @beckerfuffle #pycon2014 19

Slide 20

Slide 20 text

38 Top Wikipedias Arabic: !"#$%&ا Bulgarian: Български Catalan: Català Czech: Čeština Danish: Dansk German: Deutsch English: English Spanish: Español Estonian: Eesti Basque: Euskara Persian: ()ر+, Finnish: Suomi French: Français Hebrew: תירבע Hindi: !ह#दी Croatian: Hrvatski Hungarian: Magyar Indonesian: Bahasa Indonesia Italian: Italiano Japanese: ೔ຊޠ Kazakh: Қазақша Korean: ೠҴয Lithuanian: Lietuvių Malay Bahasa: Melayu Dutch: Nederlands Norwegian: Norsk (Bokmål) Polish: Polski Portuguese: Português Romanian: Română Russian: Русский Slovak: Slovenčina Slovenian: Slovenščina Serbian: Српски / Srpski Swedish: Svenska Turkish: Türkçe Ukrainian: Українська Vietnamese: Tiếng Việt Waray-Waray: Winaray @beckerfuffle #pycon2014 20

Slide 21

Slide 21 text

Vectorizing Text @beckerfuffle #pycon2014 21

Slide 22

Slide 22 text

@beckerfuffle #pycon2014 22

Slide 23

Slide 23 text

@beckerfuffle #pycon2014 23

Slide 24

Slide 24 text

@beckerfuffle #pycon2014 24

Slide 25

Slide 25 text

@beckerfuffle #pycon 25

Slide 26

Slide 26 text

The Model @beckerfuffle #pycon2014 26

Slide 27

Slide 27 text

Supervised Learning @beckerfuffle #pycon2014 27

Slide 28

Slide 28 text

X_train @beckerfuffle #pycon2014 28

Slide 29

Slide 29 text

Supervised Learning @beckerfuffle #pycon2014 29

Slide 30

Slide 30 text

y_train @beckerfuffle #pycon2014 30

Slide 31

Slide 31 text

Dimensionality Reduction @beckerfuffle #pycon2014 31

Slide 32

Slide 32 text

Dimensionality Reduction @beckerfuffle #pycon2014 32

Slide 33

Slide 33 text

Dimensionality Reduction @beckerfuffle #pycon2014 33

Slide 34

Slide 34 text

Dimensionality Reduction @beckerfuffle #pycon2014 34

Slide 35

Slide 35 text

Dimensionality Reduction @beckerfuffle #pycon2014 35

Slide 36

Slide 36 text

Don’t Do What I Just Did! @beckerfuffle #pycon2014 36

Slide 37

Slide 37 text

Tuning @beckerfuffle #pycon2014 37

Slide 38

Slide 38 text

RandomizedSearchCV @beckerfuffle #pycon2014 38

Slide 39

Slide 39 text

RandomizedSearchCV @beckerfuffle #pycon2014 39

Slide 40

Slide 40 text

Distributing The Model @beckerfuffle #pycon2014 40

Slide 41

Slide 41 text

Data Input @beckerfuffle #pycon2014 41

Slide 42

Slide 42 text

The Client @beckerfuffle #pycon2014 42

Slide 43

Slide 43 text

Message Loss @beckerfuffle #pycon2014 43

Slide 44

Slide 44 text

Enter RabbitMQ Reliability Flexible Routing Clustering Many clients HA Queues @beckerfuffle #pycon2014 44

Slide 45

Slide 45 text

Enter RabbitMQ Reliability Flexible Routing Clustering Many clients HA Queues @beckerfuffle #pycon2014 45

Slide 46

Slide 46 text

Pika @beckerfuffle #pycon2014 46

Slide 47

Slide 47 text

Pika @beckerfuffle #pycon2014 47

Slide 48

Slide 48 text

@beckerfuffle #pycon2014 48

Slide 49

Slide 49 text

All the things! @beckerfuffle #pycon2014 49

Slide 50

Slide 50 text

Data Processing @beckerfuffle #pycon2014 50

Slide 51

Slide 51 text

The Consumer @beckerfuffle #pycon2014 51

Slide 52

Slide 52 text

The Consumer @beckerfuffle #pycon2014 52

Slide 53

Slide 53 text

The Consumer @beckerfuffle #pycon2014 53

Slide 54

Slide 54 text

The Consumer @beckerfuffle #pycon2014 54

Slide 55

Slide 55 text

The Design                  @beckerfuffle #pycon2014 55

Slide 56

Slide 56 text

The Design                  @beckerfuffle #pycon2014 56

Slide 57

Slide 57 text

The Design                  @beckerfuffle #pycon2014 57

Slide 58

Slide 58 text

The Design                  @beckerfuffle #pycon2014 58

Slide 59

Slide 59 text

The Design                  @beckerfuffle #pycon2014 59

Slide 60

Slide 60 text

The Design                  @beckerfuffle #pycon2014 60

Slide 61

Slide 61 text

The Design                  @beckerfuffle #pycon2014 61

Slide 62

Slide 62 text

The Design                  @beckerfuffle #pycon2014 62

Slide 63

Slide 63 text

The Design                  @beckerfuffle #pycon2014 63

Slide 64

Slide 64 text

Demo Time! @beckerfuffle #pycon2014 64

Slide 65

Slide 65 text

Scaling @beckerfuffle #pycon2014 65

Slide 66

Slide 66 text

Realtime vs Batch @beckerfuffle #pycon2014 66

Slide 67

Slide 67 text

Monitoring @beckerfuffle #pycon2014 67

Slide 68

Slide 68 text

Load @beckerfuffle #pycon2014 68

Slide 69

Slide 69 text

Re-verify @beckerfuffle #pycon2014 69

Slide 70

Slide 70 text

Re-verify @beckerfuffle #pycon2014 70

Slide 71

Slide 71 text

Thank You! API & Consumer: Kelly O’Brien (linkedin.com/in/kellyobie) UI: Matt Parke (ordinaryrobot.com) Classifier: Michael Becker (github.com/mdbecker) Images: Wikipedia; flickr.com users: kohsa, spenceyc, statefarm, klara, rh2ox, nasahqphoto, frickfrack @beckerfuffle #pycon2014 71

Slide 72

Slide 72 text

My Info Twitter: @beckerfuffle Blog: beckerfuffle.com These slides and more @ github.com/mdbecker @beckerfuffle #pycon2014 72

Slide 73

Slide 73 text

Demo time! @beckerfuffle #pycon2014 73

Slide 74

Slide 74 text

Demo time! @beckerfuffle #pycon2014 74

Slide 75

Slide 75 text

@beckerfuffle #pycon2014 Demo time! 75