Making Scores with HiScore

Video here:

Transcript

1. Making Scores with HiScore Abe Othman
6. HiScore is a python library for creating and maintaining scores
7. It uses a novel quasi-Kriging solution to a new methodology, supervised scoring
8. What are scores?
9. Scores are a tool for domain experts to communicate their expertise to a broad audience
10. 88 51 27
11. } 58 Score Function Dimensions Score
12. There is no one correct scoring function
13. Scores are typically developed using the dual approach
14. 1. Select a set of basis functions f(x ⃗) = ∑ γᵢφᵢ(x ⃗)
15. 2. Adjust coefficients until things look right f(x ⃗) = ∑ γᵢφᵢ(x ⃗)
16. Dual scores ossify
17. Walkscore Problems Score of 100, but the highest crime in SF
18. Supervised scoring: a primal approach
19. Experts start by labeling a reference set and the objects’ dimensions
20. Algorithm makes a scoring function that interpolates and obeys the monotone relationship
21. Some nice features
22. Monotonicity is important for score acceptance and understanding
23. See a mis-scored point? Add it to the reference set and re-run!
24. OK, but what algorithm?
25. Easy in one dimension
29. Hard in many dimensions
30. Failed approach: simplical interpolation
32. Failed approach: 
 B-spline product bases
33. Supervised Scoring with Monotone
 Multidimensional Splines, AAAI 2014
34. Curse of dimensionality!
38. Failed approach: RBF with monotone row generation constraints
39. Failed approach: Neural Networks
42. Success: Beliakov
43. Reminder: 
 Lipschitz Continuity |f(a)-f(b)| < C |a-b|
45. Monotone Lipschitz continuity
47. 1. Project monotone Lipschitz cones from each point to generate upper and lower bounds
48. 2. Find the sup and inf constraints from the bounding cones
49. 3. Function value is halfway in-between the sup and inf bounds
50. Beliakov example
51. Beliakov plateaux
52. Beliakov plateaux
53. How can we smooth and improve this?
54. Abandon Lipschitz, just project minimal cones from each point
56. `
57. HiScore solution
58. Using HiScore: Simplified Water Well Score
61. Two factors: Distance from nearest latrine and platform size
62. Label a reference set by taking high, middle and low values in each dimension
63. Distance: 
 0m, 10m, 50m Size: 1SF, 25SF, 100SF
64. Score Distance Size 0 0 1 5 0 25 10 0 100 20 10 1 50 10 25 60 10 100 65 50 1 90 50 25 100 50 100 Monotone Relationship: (+, +)
65. import hiscore reference_set = {(0,1): 0, (0,25): 5, (0,100): 10, (10,1): 20, (10,25): 50, … } mono_rel = [1,1] hiscore.create(reference_set, mono_rel, minval=0, maxval=100)
67. Complicate the model with additional factors
68. Avoid curse of dimensionality by building a tree
70. Possible to easily construct and understand scores with dozens of input dimensions
71. Making dimensions monotone: blood pressure
73. S+ > 0 S- = 0 D+ > 0 D- = 0 D+ = 0 D- > 0 S+ = 0 S- > 0
74. What do you want to score? github.com/aothman/ hiscore $ pip install hiscore
75. Thanks! aothman@cs.cmu.edu