Efficient Second-Order Shape-Constrained Function Fitting

Slides for a talk I gave at WADS 2019; see paper website for details
https://www.wild-inter.net/publications/durfee-gao-rao-wild-2019

Transcript

1. Second-Order Shape-Constrained Function Fitting Sebastian Wild www.wild-inter.net based on joint work with David Durfee, Yu Gao, and Anup B. Rao WADS Algorithms and Data Structures Symposium 2019 Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 0 / 12
2. Outline 1 Introduction 1 Introduction 2 Our Result 2 Our Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 0 / 12
3. Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
4. Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
5. Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
6. Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
7. Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
8. Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
9. Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
10. Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
11. Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
12. Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
13. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
14. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
15. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
16. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
17. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
18. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
19. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
20. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
21. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth all expressible by bounds on first and second derivatives! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
22. Weighted L∞ Function Fitting With 2nd-Order Constraints min error between f and y s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
23. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi · |fi − yi| s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
24. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
25. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
26. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i f (xi) f + i (i = 2, . . . , n) bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
27. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
28. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
29. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
30. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: f − i y− i fi − fi−1 xi − xi−1 f + i y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
31. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
32. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
33. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
34. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
35. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
36. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper y− i fi − fi−1 y+ i z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
37. Previous Work Shape Constraint Error Norm Time Source linear function weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
38. Previous Work Shape Constraint Error Norm Time Source linear function weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for first-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
39. Previous Work Shape Constraint Error Norm Time Source linear function weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for first-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
40. Previous Work Shape Constraint Error Norm Time Source linear function weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for first-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
41. Outline 1 Introduction 1 Introduction 2 Our Result 2 Our Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
42. Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
43. Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
44. Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
45. Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
46. Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
47. Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
48. Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can find f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
49. Dynamic Programming with Infinite States? 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
50. Dynamic Programming with Infinite States? 1 2 3 4 5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
51. Dynamic Programming with Infinite States? 1 2 3 4 5 x− 1 = 1.5 x+ 1 = 3 x− 2 = 0.4 x+ 2 = 2 x− 3 = 1.7 x+ 3 = 2.2 x− 4 = 2.5 x+ 4 = 4 x− 5 = 2 x+ 5 = 3.3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
52. Dynamic Programming with Infinite States? 1 2 3 4 5 y− 2 = −.5 y+ 2 = 1 y− 3 = 0 y+ 3 = 1 y− 4 = −1 y+ 4 = 10 y− 5 = −1 y+ 5 = 0.5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
53. Dynamic Programming with Infinite States? 1 2 3 4 5 z− 3 = −.5 z+ 3 = 1.5 z− 4 = −.8 z+ 4 = −.2 z− 5 = −.5 z+ 5 = .5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
54. Dynamic Programming with Infinite States? 1 2 3 4 5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
55. Dynamic Programming with Infinite States? 1 2 3 4 5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
56. Dynamic Programming with Infinite States? 1 2 3 4 5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
57. Dynamic Programming with Infinite States? 1 2 3 4 5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
58. Dynamic Programming with Infinite States? 1 2 3 4 5 f4 f3 f2 f1 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
59. Dynamic Programming with Infinite States? 1 2 3 4 5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
60. Dynamic Programming with Infinite States? 1 2 3 4 5 f5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
61. Dynamic Programming with Infinite States? 1 2 3 4 5 f5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Dynamic programming over prefixes of problem! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
62. Dynamic Programming with Infinite States? 1 2 3 4 5 f5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Dynamic programming over prefixes of problem! Continuous problem! Infinitely many candidates f! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
63. Outline 1 Introduction 1 Introduction 2 Our Result 2 Our Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
64. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
65. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
66. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
67. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
68. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
69. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
70. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
71. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
72. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
73. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
74. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
75. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to efficiently compute Pi from Pi−1? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
76. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to efficiently compute Pi from Pi−1? ... by the magic of geometry and data structures! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
77. Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to efficiently compute Pi from Pi−1? ... by the magic of geometry and data structures! stay tuned Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
78. From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
79. From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
80. From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
81. From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
82. From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
83. From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
84. From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 x y Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
85. From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 horizontal shear + y (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
86. From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 horizontal shear (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) slide along x = y + z + z z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
87. From Pi−1 to Pi – Step 2 2 Add x- and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12
88. From Pi−1 to Pi – Step 2 2 Add x- and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] Pi = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i intersect w/ halfplanes (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi x− 4 = 2.5 x+ 4 = 4 y− 4 = −1 y+ 4 = 10 Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12
89. From Pi−1 to Pi – Step 2 2 Add x- and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] Pi = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i intersect w/ halfplanes (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi x− 4 = 2.5 x+ 4 = 4 y− 4 = −1 y+ 4 = 10 Can compute P2, . . . , Pn iteratively, check if Pn nonempty. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12
90. Data Structures & Implicit Updates But: Not yet done! Pi has O(i) vertices Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
91. Data Structures & Implicit Updates But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
92. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
93. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
94. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
95. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
96. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
97. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
98. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
99. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Lemma: We only add O(n) new vertices for all polygons. ( 1 per cut edge) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
100. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Lemma: We only add O(n) new vertices for all polygons. ( 1 per cut edge) Linear search for intersections costs O(n) in total (search cost = # deleted vertices) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
101. Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can find f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12
102. Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can find f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12
103. Outline 1 Introduction 1 Introduction 2 Our Result 2 Our Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12
104. Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
105. Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
106. Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
107. Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
108. Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
109. Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
110. Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
111. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
112. Icons made by Freepik, Gregor Cresnar, Those Icons, and Madebyoliver from www.flaticon.com. Other photos from www.pixabay.com. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 13 / 12