Efficient Second-Order Shape-Constrained Function Fitting

Efficient Second-Order Shape-Constrained Function Fitting

Slides for a talk I gave at WADS 2019; see paper website for details
https://www.wild-inter.net/publications/durfee-gao-rao-wild-2019

2a8430a8f786043f73f42dc9d88a7e5a?s=128

Sebastian Wild

August 06, 2019
Tweet

Transcript

  1. Second-Order Shape-Constrained Function Fitting Sebastian Wild www.wild-inter.net based on joint

    work with David Durfee, Yu Gao, and Anup B. Rao WADS Algorithms and Data Structures Symposium 2019 Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 0 / 12
  2. Outline 1 Introduction 1 Introduction 2 Our Result 2 Our

    Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 0 / 12
  3. Regression Goal: Given observations here: 2D points (xi, yi), find

    best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
  4. Regression Goal: Given observations here: 2D points (xi, yi), find

    best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
  5. Regression Goal: Given observations here: 2D points (xi, yi), find

    best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
  6. Regression Goal: Given observations here: 2D points (xi, yi), find

    best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
  7. Regression Goal: Given observations here: 2D points (xi, yi), find

    best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
  8. Regression Goal: Given observations here: 2D points (xi, yi), find

    best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
  9. Regression Goal: Given observations here: 2D points (xi, yi), find

    best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
  10. Regression Goal: Given observations here: 2D points (xi, yi), find

    best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
  11. Regression Goal: Given observations here: 2D points (xi, yi), find

    best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
  12. Regression Goal: Given observations here: 2D points (xi, yi), find

    best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12
  13. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

    cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
  14. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

    cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
  15. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

    cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
  16. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

    cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
  17. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

    cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
  18. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

    cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
  19. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

    cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
  20. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

    cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
  21. Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic

    cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth all expressible by bounds on first and second derivatives! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12
  22. Weighted L∞ Function Fitting With 2nd-Order Constraints min error between

    f and y s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  23. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi · |fi − yi| s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  24. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  25. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  26. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i f (xi) f + i (i = 2, . . . , n) bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  27. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  28. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  29. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  30. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: f − i y− i fi − fi−1 xi − xi−1 f + i y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  31. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  32. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  33. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  34. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  35. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  36. Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1

    i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper y− i fi − fi−1 y+ i z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12
  37. Previous Work Shape Constraint Error Norm Time Source linear function

    weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
  38. Previous Work Shape Constraint Error Norm Time Source linear function

    weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for first-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
  39. Previous Work Shape Constraint Error Norm Time Source linear function

    weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for first-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
  40. Previous Work Shape Constraint Error Norm Time Source linear function

    weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for first-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
  41. Outline 1 Introduction 1 Introduction 2 Our Result 2 Our

    Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12
  42. Our result Main result: There is a single algorithm computing

    f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
  43. Our result Main result: There is a single algorithm computing

    f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
  44. Our result Main result: There is a single algorithm computing

    f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
  45. Our result Main result: There is a single algorithm computing

    f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
  46. Our result Main result: There is a single algorithm computing

    f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
  47. Our result Main result: There is a single algorithm computing

    f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
  48. Our result Main result: There is a single algorithm computing

    f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can find f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12
  49. Dynamic Programming with Infinite States? 2nd-order-diff constrained decision problem: ∃f

    s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  50. Dynamic Programming with Infinite States? 1 2 3 4 5

    2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  51. Dynamic Programming with Infinite States? 1 2 3 4 5

    x− 1 = 1.5 x+ 1 = 3 x− 2 = 0.4 x+ 2 = 2 x− 3 = 1.7 x+ 3 = 2.2 x− 4 = 2.5 x+ 4 = 4 x− 5 = 2 x+ 5 = 3.3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  52. Dynamic Programming with Infinite States? 1 2 3 4 5

    y− 2 = −.5 y+ 2 = 1 y− 3 = 0 y+ 3 = 1 y− 4 = −1 y+ 4 = 10 y− 5 = −1 y+ 5 = 0.5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  53. Dynamic Programming with Infinite States? 1 2 3 4 5

    z− 3 = −.5 z+ 3 = 1.5 z− 4 = −.8 z+ 4 = −.2 z− 5 = −.5 z+ 5 = .5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  54. Dynamic Programming with Infinite States? 1 2 3 4 5

    2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  55. Dynamic Programming with Infinite States? 1 2 3 4 5

    2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  56. Dynamic Programming with Infinite States? 1 2 3 4 5

    2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  57. Dynamic Programming with Infinite States? 1 2 3 4 5

    2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  58. Dynamic Programming with Infinite States? 1 2 3 4 5

    f4 f3 f2 f1 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  59. Dynamic Programming with Infinite States? 1 2 3 4 5

    f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  60. Dynamic Programming with Infinite States? 1 2 3 4 5

    f5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  61. Dynamic Programming with Infinite States? 1 2 3 4 5

    f5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Dynamic programming over prefixes of problem! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  62. Dynamic Programming with Infinite States? 1 2 3 4 5

    f5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Dynamic programming over prefixes of problem! Continuous problem! Infinitely many candidates f! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  63. Outline 1 Introduction 1 Introduction 2 Our Result 2 Our

    Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12
  64. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  65. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  66. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  67. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  68. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  69. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  70. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  71. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  72. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  73. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  74. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  75. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to efficiently compute Pi from Pi−1? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  76. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to efficiently compute Pi from Pi−1? ... by the magic of geometry and data structures! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  77. Dynamic Programming with Infinite States! There is a finite representation

    for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to efficiently compute Pi from Pi−1? ... by the magic of geometry and data structures! stay tuned Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12
  78. From Pi−1 to Pi – Step 1 Pi = (fi

    , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
  79. From Pi−1 to Pi – Step 1 Pi = (fi

    , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
  80. From Pi−1 to Pi – Step 1 Pi = (fi

    , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
  81. From Pi−1 to Pi – Step 1 Pi = (fi

    , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
  82. From Pi−1 to Pi – Step 1 Pi = (fi

    , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
  83. From Pi−1 to Pi – Step 1 Pi = (fi

    , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
  84. From Pi−1 to Pi – Step 1 Pi = (fi

    , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 x y Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
  85. From Pi−1 to Pi – Step 1 Pi = (fi

    , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 horizontal shear + y (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
  86. From Pi−1 to Pi – Step 1 Pi = (fi

    , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 horizontal shear (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) slide along x = y + z + z z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12
  87. From Pi−1 to Pi – Step 2 2 Add x-

    and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12
  88. From Pi−1 to Pi – Step 2 2 Add x-

    and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] Pi = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i intersect w/ halfplanes (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi x− 4 = 2.5 x+ 4 = 4 y− 4 = −1 y+ 4 = 10 Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12
  89. From Pi−1 to Pi – Step 2 2 Add x-

    and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] Pi = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i intersect w/ halfplanes (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi x− 4 = 2.5 x+ 4 = 4 y− 4 = −1 y+ 4 = 10 Can compute P2, . . . , Pn iteratively, check if Pn nonempty. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12
  90. Data Structures & Implicit Updates But: Not yet done! Pi

    has O(i) vertices Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  91. Data Structures & Implicit Updates But: Not yet done! Pi

    has O(i) vertices Can’t afford to update the vertices(!) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  92. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

    0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  93. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

    0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  94. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

    0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  95. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

    0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  96. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

    0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  97. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

    0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  98. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

    0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  99. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

    0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Lemma: We only add O(n) new vertices for all polygons. ( 1 per cut edge) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  100. Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2,

    0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Lemma: We only add O(n) new vertices for all polygons. ( 1 per cut edge) Linear search for intersections costs O(n) in total (search cost = # deleted vertices) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12
  101. Our result Main result: There is a single algorithm computing

    f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can find f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12
  102. Our result Main result: There is a single algorithm computing

    f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can find f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12
  103. Outline 1 Introduction 1 Introduction 2 Our Result 2 Our

    Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12
  104. Conclusion You’ve seen First near-linear algorithm for a regression problem

    with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
  105. Conclusion You’ve seen First near-linear algorithm for a regression problem

    with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
  106. Conclusion You’ve seen First near-linear algorithm for a regression problem

    with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
  107. Conclusion You’ve seen First near-linear algorithm for a regression problem

    with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
  108. Conclusion You’ve seen First near-linear algorithm for a regression problem

    with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
  109. Conclusion You’ve seen First near-linear algorithm for a regression problem

    with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
  110. Conclusion You’ve seen First near-linear algorithm for a regression problem

    with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12
  111. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12

  112. Icons made by Freepik, Gregor Cresnar, Those Icons, and Madebyoliver

    from www.flaticon.com. Other photos from www.pixabay.com. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 13 / 12