PyData Meetup Group Presentation

Presentation on py-earth to the San Francisco PyData Meetup group on 2013-05-29.

Transcript

1. MARS in Python or A Tale of Two Planets 1
2. Outline • Motivating use case • MARS algorithm • Py-earth • Examples 2
5. M A R S ultivariate daptive egression plines 5
6. Not MARS •MARSplines •MARegressionSplines •ARES •earth 6
8. HbA1c Age Gender Etc. Cost X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X 8
9. Constraints •Non-monotone relationships among variables •Interactions among predictors •Simple model 9
12. Illustration by Yi-Ke Peng 12
14. Python R My Brain Raw data processing Object-relational mapping Feature extraction Plotting Bootstrapping Normalization Multivariate Adaptive Regression Splines 14
17. Regression: The search for f(x) yj = f ( x1j, . . . , xnj) + ✏j 17
18. Linear Regression ˆ f ( x ) = a0 + P X i=1 aixi 18
19. Multivariate Adaptive Regression Splines ˆ f ( x ) = a0 + M X m=1 am Km Y k=1 ⇥ skm xv(k,m) tkm ⇤ + 19
20. Hinge Functions CDify h ( x t ) = [ x t ]+ = ( x t, x > t 0 , x  t 20
21. Multivariate Adaptive Regression Splines ˆ f ( x ) = a0 + M X m=1 am Km Y k=1 ⇥ skm xv(k,m) tkm ⇤ + 21
22. y = 1 2h (1 x ) + 1 2 h ( x 1) Multivariate Adaptive Regression Splines 22
23. Multivariate Adaptive Regression Splines y = h ( x 1) h ( x 1) + h (1 x ) h (1 x ) 23
24. Multivariate Adaptive Regression Splines y = 2 + 0 . 1h ( x 1) + h (1 x ) + 3h ( x 1) h (4 x ) 24
25. Multivariate Example z = h ( 3 x ) + h ( 3 x ) h (5 y ) 25
26. Multivariate Adaptive Regression Splines ˆ f ( x ) = a0 + M X m=1 am Km Y k=1 ⇥ skm xv(k,m) tkm ⇤ + 26
27. Forward Pass Pruning Pass 27
28. Forward Pass • while True: • best_err = Infinity • for each term, predictor, knot candidate: • err = get_squared_error(term, predictor, knot) • if err < best_err: • best_err = err • best_term, best_pred, best_knot = term, predictor, knot • add term pair for best_term, best_pred, best_knot • check stopping conditions 28
29. Forward Pass 1 Start Iteration 1 Iteration 2 h( x t ) h( t x ) h( x t ) ⇥ h ( x s ) h( x t ) ⇥ h ( s x ) 29
30. Forward Pass • while True: • best_err = Infinity • for each term, predictor, knot candidate: • err = get_squared_error(term, predictor, knot) • if err < best_err: • best_err = err • best_term, best_pred, best_knot = term, predictor, knot • add term pair for best_term, best_pred, best_knot • check stopping conditions 30
31. O N2P3 31
32. Forward Pass 1 Start Iteration 1 Iteration 2 h( x t ) h( t x ) h( x t ) ⇥ h ( x s ) h( x t ) ⇥ h ( s x ) 32
33. Generalized Cross Validation GCV = 1 N PN i=1 [yi ˆ yi]2 1 N2 (N Q d (Q 1))2 33
34. Pruning Pass • for i in range(num_terms): • best_score = Infinity • for term in terms: • score = GCV(model \ term) • if score < best_score: • best_score = score • term_to_drop = term • remove term_to_drop from model • models[i] = model.copy() • scores[i] = score • selected_model = models[argmin(scores)] 34
35. Pruning Pass 1 h( x t ) h( t x ) h( x t ) ⇥ h ( x s ) h( x t ) ⇥ h ( s x ) 35
36. Final Model [yi ˆ yi]2 d(Q 1))2 y = a0 + a1 h ( t x ) + a2 h ( x t ) h ( x s ) 36
38. Implementation Goals •Compatible with numpy ecosystem •Fast and reliable •Easy to maintain 38
41. >git clone git://github.com/jcrudy/py-earth.git >cd py-earth >sudo python setup.py install Installation 41
42. Important Earth Methods •fit(X,y) •transform(X) •predict(X) 42
43. Simple Example 43
44. Simple Example 44
46. With Pandas 46
47. With Patsy 47
48. Classification 48
49. Classification 49
51. Future Plans •Documentation •Integrate into scikit-learn •Multiple responses •Sample weights 51
52. Summary • MARS is a simple but flexible regression method • py-earth is MARS for Python data stack • Try it! 52
53. py-earth A far better thing than I have ever done • https://github.com/jcrudy/py-earth 53