# Efficient Second-Order Shape-Constrained Function Fitting

Slides for a talk I gave at WADS 2019; see paper website for details
https://www.wild-inter.net/publications/durfee-gao-rao-wild-2019

August 06, 2019

## Transcript

1. ### Second-Order Shape-Constrained Function Fitting Sebastian Wild www.wild-inter.net based on joint

work with David Durfee, Yu Gao, and Anup B. Rao WADS Algorithms and Data Structures Symposium 2019 Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 0 / 12
2. ### Outline 1 Introduction 1 Introduction 2 Our Result 2 Our

Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 0 / 12
3. ### Regression Goal: Given observations here: 2D points (xi, yi), ﬁnd

best function explainin “ﬁt” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: eﬃciently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
4. ### Regression Goal: Given observations here: 2D points (xi, yi), ﬁnd

best function explainin “ﬁt” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: eﬃciently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
5. ### Regression Goal: Given observations here: 2D points (xi, yi), ﬁnd

best function explainin “ﬁt” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: eﬃciently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
6. ### Regression Goal: Given observations here: 2D points (xi, yi), ﬁnd

best function explainin “ﬁt” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: eﬃciently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
7. ### Regression Goal: Given observations here: 2D points (xi, yi), ﬁnd

best function explainin “ﬁt” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: eﬃciently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
8. ### Regression Goal: Given observations here: 2D points (xi, yi), ﬁnd

best function explainin “ﬁt” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: eﬃciently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
9. ### Regression Goal: Given observations here: 2D points (xi, yi), ﬁnd

best function explainin “ﬁt” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: eﬃciently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
10. ### Regression Goal: Given observations here: 2D points (xi, yi), ﬁnd

best function explainin “ﬁt” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: eﬃciently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
11. ### Regression Goal: Given observations here: 2D points (xi, yi), ﬁnd

best function explainin “ﬁt” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: eﬃciently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
12. ### Regression Goal: Given observations here: 2D points (xi, yi), ﬁnd

best function explainin “ﬁt” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: eﬃciently compute best f for data s. t. shape constraints eﬃciently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
13. ### Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overﬁtting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
14. ### Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overﬁtting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
15. ### Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overﬁtting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
16. ### Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overﬁtting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
17. ### Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overﬁtting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
18. ### Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overﬁtting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
19. ### Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overﬁtting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
20. ### Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overﬁtting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
21. ### Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overﬁtting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth all expressible by bounds on ﬁrst and second derivatives! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
22. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min error between

f and y s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
23. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi · |fi − yi| s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
24. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
25. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
26. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i f (xi) f + i (i = 2, . . . , n) bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
27. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
28. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
29. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
30. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: f − i y− i fi − fi−1 xi − xi−1 f + i y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
31. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
32. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
33. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
34. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
35. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
36. ### Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper y− i fi − fi−1 y+ i z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
37. ### Previous Work Shape Constraint Error Norm Time Source linear function

weighted L2 O(n) [linear least squares] linear comb. of k ﬁxed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
38. ### Previous Work Shape Constraint Error Norm Time Source linear function

weighted L2 O(n) [linear least squares] linear comb. of k ﬁxed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for ﬁrst-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
39. ### Previous Work Shape Constraint Error Norm Time Source linear function

weighted L2 O(n) [linear least squares] linear comb. of k ﬁxed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for ﬁrst-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
40. ### Previous Work Shape Constraint Error Norm Time Source linear function

weighted L2 O(n) [linear least squares] linear comb. of k ﬁxed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for ﬁrst-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
41. ### Outline 1 Introduction 1 Introduction 2 Our Result 2 Our

Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
42. ### Our result Main result: There is a single algorithm computing

f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
43. ### Our result Main result: There is a single algorithm computing

f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
44. ### Our result Main result: There is a single algorithm computing

f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
45. ### Our result Main result: There is a single algorithm computing

f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
46. ### Our result Main result: There is a single algorithm computing

f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
47. ### Our result Main result: There is a single algorithm computing

f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
48. ### Our result Main result: There is a single algorithm computing

f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can ﬁnd f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
49. ### Dynamic Programming with Inﬁnite States? 2nd-order-diﬀ constrained decision problem: ∃f

s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
50. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
51. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

x− 1 = 1.5 x+ 1 = 3 x− 2 = 0.4 x+ 2 = 2 x− 3 = 1.7 x+ 3 = 2.2 x− 4 = 2.5 x+ 4 = 4 x− 5 = 2 x+ 5 = 3.3 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
52. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

y− 2 = −.5 y+ 2 = 1 y− 3 = 0 y+ 3 = 1 y− 4 = −1 y+ 4 = 10 y− 5 = −1 y+ 5 = 0.5 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
53. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

z− 3 = −.5 z+ 3 = 1.5 z− 4 = −.8 z+ 4 = −.2 z− 5 = −.5 z+ 5 = .5 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
54. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
55. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
56. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
57. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 ﬁxed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
58. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

f4 f3 f2 f1 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 ﬁxed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
59. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

f4 f3 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 ﬁxed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
60. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

f5 f4 f3 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 ﬁxed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
61. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

f5 f4 f3 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 ﬁxed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Dynamic programming over preﬁxes of problem! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
62. ### Dynamic Programming with Inﬁnite States? 1 2 3 4 5

f5 f4 f3 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 ﬁxed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Dynamic programming over preﬁxes of problem! Continuous problem! Inﬁnitely many candidates f! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
63. ### Outline 1 Introduction 1 Introduction 2 Our Result 2 Our

Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
64. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
65. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
66. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
67. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
68. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions last value Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
69. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions last value incoming slope Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
70. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions last value incoming slope Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
71. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions last value incoming slope 2nd-order-diﬀ constrained decision problem ≡ Pn = {}? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
72. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions last value incoming slope 2nd-order-diﬀ constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (deﬁned by linear inequalities) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
73. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions last value incoming slope 2nd-order-diﬀ constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (deﬁned by linear inequalities) Can represent them by their vertex sequence! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
74. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions last value incoming slope 2nd-order-diﬀ constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (deﬁned by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
75. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions last value incoming slope 2nd-order-diﬀ constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (deﬁned by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to eﬃciently compute Pi from Pi−1? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
76. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions last value incoming slope 2nd-order-diﬀ constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (deﬁned by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to eﬃciently compute Pi from Pi−1? ... by the magic of geometry and data structures! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
77. ### Dynamic Programming with Inﬁnite States! There is a ﬁnite representation

for all preﬁx candidates 2nd-order-diﬀ constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of preﬁx solutions last value incoming slope 2nd-order-diﬀ constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (deﬁned by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to eﬃciently compute Pi from Pi−1? ... by the magic of geometry and data structures! stay tuned Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
78. ### From Pi−1 to Pi – Step 1 Pi = (fi

, fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
79. ### From Pi−1 to Pi – Step 1 Pi = (fi

, fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
80. ### From Pi−1 to Pi – Step 1 Pi = (fi

, fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
81. ### From Pi−1 to Pi – Step 1 Pi = (fi

, fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
82. ### From Pi−1 to Pi – Step 1 Pi = (fi

, fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
83. ### From Pi−1 to Pi – Step 1 Pi = (fi

, fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
84. ### From Pi−1 to Pi – Step 1 Pi = (fi

, fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 x y Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
85. ### From Pi−1 to Pi – Step 1 Pi = (fi

, fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 horizontal shear + y (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
86. ### From Pi−1 to Pi – Step 1 Pi = (fi

, fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 horizontal shear (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) slide along x = y + z + z z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
87. ### From Pi−1 to Pi – Step 2 2 Add x-

and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12
88. ### From Pi−1 to Pi – Step 2 2 Add x-

and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] Pi = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i intersect w/ halfplanes (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi x− 4 = 2.5 x+ 4 = 4 y− 4 = −1 y+ 4 = 10 Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12
89. ### From Pi−1 to Pi – Step 2 2 Add x-

and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] Pi = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i intersect w/ halfplanes (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi x− 4 = 2.5 x+ 4 = 4 y− 4 = −1 y+ 4 = 10 Can compute P2, . . . , Pn iteratively, check if Pn nonempty. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12
90. ### Data Structures & Implicit Updates But: Not yet done! Pi

has O(i) vertices Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
91. ### Data Structures & Implicit Updates But: Not yet done! Pi

has O(i) vertices Can’t aﬀord to update the vertices(!) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
92. ### Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t aﬀord to update the vertices(!) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
93. ### Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t aﬀord to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
94. ### Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t aﬀord to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
95. ### Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t aﬀord to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to ﬁnd edges intersecting a cutting plane eﬃciently. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
96. ### Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t aﬀord to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to ﬁnd edges intersecting a cutting plane eﬃciently. Claim: Two sorted deques for upper and lower hull vertices suﬃce. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
97. ### Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t aﬀord to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to ﬁnd edges intersecting a cutting plane eﬃciently. Claim: Two sorted deques for upper and lower hull vertices suﬃce. Lemma: All edges have (weakly) positive slope. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
98. ### Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t aﬀord to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to ﬁnd edges intersecting a cutting plane eﬃciently. Claim: Two sorted deques for upper and lower hull vertices suﬃce. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
99. ### Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t aﬀord to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to ﬁnd edges intersecting a cutting plane eﬃciently. Claim: Two sorted deques for upper and lower hull vertices suﬃce. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Lemma: We only add O(n) new vertices for all polygons. ( 1 per cut edge) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
100. ### Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t aﬀord to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to ﬁnd edges intersecting a cutting plane eﬃciently. Claim: Two sorted deques for upper and lower hull vertices suﬃce. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Lemma: We only add O(n) new vertices for all polygons. ( 1 per cut edge) Linear search for intersections costs O(n) in total (search cost = # deleted vertices) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
101. ### Our result Main result: There is a single algorithm computing

f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can ﬁnd f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12
102. ### Our result Main result: There is a single algorithm computing

f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diﬀ constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can ﬁnd f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12
103. ### Outline 1 Introduction 1 Introduction 2 Our Result 2 Our

Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12
104. ### Conclusion You’ve seen First near-linear algorithm for a regression problem

with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
105. ### Conclusion You’ve seen First near-linear algorithm for a regression problem

with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
106. ### Conclusion You’ve seen First near-linear algorithm for a regression problem

with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
107. ### Conclusion You’ve seen First near-linear algorithm for a regression problem

with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
108. ### Conclusion You’ve seen First near-linear algorithm for a regression problem

with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
109. ### Conclusion You’ve seen First near-linear algorithm for a regression problem

with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
110. ### Conclusion You’ve seen First near-linear algorithm for a regression problem

with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12

112. ### Icons made by Freepik, Gregor Cresnar, Those Icons, and Madebyoliver

from www.flaticon.com. Other photos from www.pixabay.com. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 13 / 12