Tweet
Tweet

Slide 1

Slide 1 text

Second-Order Shape-Constrained Function Fitting Sebastian Wild www.wild-inter.net based on joint work with David Durfee, Yu Gao, and Anup B. Rao WADS Algorithms and Data Structures Symposium 2019 Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 0 / 12

Slide 2

Slide 2 text

Outline 1 Introduction 1 Introduction 2 Our Result 2 Our Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 0 / 12

Slide 3

Slide 3 text

Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12

Slide 4

Slide 4 text

Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12

Slide 5

Slide 5 text

Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12

Slide 6

Slide 6 text

Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12

Slide 7

Slide 7 text

Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12

Slide 8

Slide 8 text

Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression x y stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12

Slide 9

Slide 9 text

Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12

Slide 10

Slide 10 text

Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12

Slide 11

Slide 11 text

Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12

Slide 12

Slide 12 text

Regression Goal: Given observations here: 2D points (xi, yi), find best function explainin “fit” function to points g yi ≈ f(xi). “regression analysis” ubiquitous in statistics / machine learning parametric regression linear regression x y f(x) = ax + b nonparametric regression isotonic regression x y f = any weakly increasing function stats challenge: identify reasonable shape restrictions for f (model, prior knowledge) cs challenge: efficiently compute best f for data s. t. shape constraints efficiently compute best f for data s. t. shape constraints Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 1 / 12

Slide 13

Slide 13 text

Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12

Slide 14

Slide 14 text

Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12

Slide 15

Slide 15 text

Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12

Slide 16

Slide 16 text

Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12

Slide 17

Slide 17 text

Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12

Slide 18

Slide 18 text

Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12

Slide 19

Slide 19 text

Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12

Slide 20

Slide 20 text

Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12

Slide 21

Slide 21 text

Shape Constraints Typical shape constraints for 1D functions (bounded) monotonic cumulative density functions, low-dim. metric embedding unimodal dose-response curves Lipschitz (bounded derivative) avoiding overfitting in isotonic regression increasing & concave production functions, utility functions (diminishing returns) convex consumer preferences, value functions in reinforcement learning smooth all expressible by bounds on first and second derivatives! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 2 / 12

Slide 22

Slide 22 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min error between f and y s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 23

Slide 23 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi · |fi − yi| s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 24

Slide 24 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 25

Slide 25 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. bounds on 1st derivative of f bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 26

Slide 26 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i f (xi) f + i (i = 2, . . . , n) bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 27

Slide 27 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 28

Slide 28 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative bounds on 2nd derivative of f given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 29

Slide 29 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 30

Slide 30 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: f − i y− i fi − fi−1 xi − xi−1 f + i y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 31

Slide 31 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 32

Slide 32 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 33

Slide 33 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 34

Slide 34 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 35

Slide 35 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper (xi − xi−1 )f − i y− i fi − fi−1 f + i (xi − xi−1 ) y+ i (xi − xi−1 )2f − i z− i (fi − fi−1 ) − xi − xi−1 xi−1 − xi−2 αi · (fi−1 − fi−2 ) f + i (xi − xi−1 )2 z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 36

Slide 36 text

Weighted L∞ Function Fitting With 2nd-Order Constraints min max 1 i n wi weights · |fi − yi| weighted f − y ∞ (L∞) s. t. f − i fi − fi−1 xi − xi−1 f + i (i = 2, . . . , n) discrete derivative f − i fi−fi−1 xi−xi−1 − fi−1−fi−2 xi−1−xi−2 xi − xi−1 f + i (i = 3, . . . , n) discrete 2nd derivative given quantities variables to compute More convenient formulation: For this talk: xi = i αi = 1 (equidistant points) general α in paper y− i fi − fi−1 y+ i z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 3 / 12

Slide 37

Slide 37 text

Previous Work Shape Constraint Error Norm Time Source linear function weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12

Slide 38

Slide 38 text

Previous Work Shape Constraint Error Norm Time Source linear function weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for first-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12

Slide 39

Slide 39 text

Previous Work Shape Constraint Error Norm Time Source linear function weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for first-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12

Slide 40

Slide 40 text

Previous Work Shape Constraint Error Norm Time Source linear function weighted L2 O(n) [linear least squares] linear comb. of k fixed functions weighted L2 O(nk2) [linear least squares] monotonic (isotonic regr. on path) weighted L2 O(n) [pool adjacent violators algorithm, Rote 2019] monotonic weighted L1 O(n log n) [Ahuja&Orlin 2001, Rote 2019] monotonic weighted L∞ O(n) [Stout 2015] isotonic regr. on general partial orders various results surveyed [Stout 2019] unimodal weighted L2 O(n) [Stout 2008] unimodal weighted L1 O(n log n) [Stout 2008] unimodal L∞ O(n) [Stout 2008] piecewise constant (k-step function) weighted L∞ O(n log n) [Fournier&Vigneron 2013] piecewise constant (k-step function) L∞ O(n) sorted input [Fournier&Vigneron 2011] Lipschitz & monotonic L2 O(n log n) [Agarwal&Phillips&Sadri 2010] Lipschitz & unimodal L2 O(n polylog n) [Agarwal&Phillips&Sadri 2010] multivar. convex (& Lipschitz) L2 quadr. prog. Θ(n2) constraints [Mazumder et al. 2018] Summary: Near-linear time exact methods for first-order constraints Best prior approach for second-order: solve LP Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12

Slide 41

Slide 41 text

Outline 1 Introduction 1 Introduction 2 Our Result 2 Our Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 4 / 12

Slide 42

Slide 42 text

Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12

Slide 43

Slide 43 text

Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12

Slide 44

Slide 44 text

Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12

Slide 45

Slide 45 text

Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12

Slide 46

Slide 46 text

Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12

Slide 47

Slide 47 text

Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12

Slide 48

Slide 48 text

Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can find f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 5 / 12

Slide 49

Slide 49 text

Dynamic Programming with Infinite States? 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 50

Slide 50 text

Dynamic Programming with Infinite States? 1 2 3 4 5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 51

Slide 51 text

Dynamic Programming with Infinite States? 1 2 3 4 5 x− 1 = 1.5 x+ 1 = 3 x− 2 = 0.4 x+ 2 = 2 x− 3 = 1.7 x+ 3 = 2.2 x− 4 = 2.5 x+ 4 = 4 x− 5 = 2 x+ 5 = 3.3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 52

Slide 52 text

Dynamic Programming with Infinite States? 1 2 3 4 5 y− 2 = −.5 y+ 2 = 1 y− 3 = 0 y+ 3 = 1 y− 4 = −1 y+ 4 = 10 y− 5 = −1 y+ 5 = 0.5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 53

Slide 53 text

Dynamic Programming with Infinite States? 1 2 3 4 5 z− 3 = −.5 z+ 3 = 1.5 z− 4 = −.8 z+ 4 = −.2 z− 5 = −.5 z+ 5 = .5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 54

Slide 54 text

Dynamic Programming with Infinite States? 1 2 3 4 5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 55

Slide 55 text

Dynamic Programming with Infinite States? 1 2 3 4 5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 56

Slide 56 text

Dynamic Programming with Infinite States? 1 2 3 4 5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 57

Slide 57 text

Dynamic Programming with Infinite States? 1 2 3 4 5 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 58

Slide 58 text

Dynamic Programming with Infinite States? 1 2 3 4 5 f4 f3 f2 f1 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 59

Slide 59 text

Dynamic Programming with Infinite States? 1 2 3 4 5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 60

Slide 60 text

Dynamic Programming with Infinite States? 1 2 3 4 5 f5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 61

Slide 61 text

Dynamic Programming with Infinite States? 1 2 3 4 5 f5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Dynamic programming over prefixes of problem! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 62

Slide 62 text

Dynamic Programming with Infinite States? 1 2 3 4 5 f5 f4 f3 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (This is a ‘Yes’ instance.) “Induction” Suppose candidate values for f1 , . . . , fn−1 fixed Need only fn−2 and fn−1 to decide if feasible fn exists (for this candidate). Dynamic programming over prefixes of problem! Continuous problem! Infinitely many candidates f! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 63

Slide 63 text

Outline 1 Introduction 1 Introduction 2 Our Result 2 Our Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 6 / 12

Slide 64

Slide 64 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 65

Slide 65 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 66

Slide 66 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 67

Slide 67 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 68

Slide 68 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi−1 ) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 69

Slide 69 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 70

Slide 70 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 71

Slide 71 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 72

Slide 72 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 73

Slide 73 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 74

Slide 74 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 75

Slide 75 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to efficiently compute Pi from Pi−1? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 76

Slide 76 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to efficiently compute Pi from Pi−1? ... by the magic of geometry and data structures! Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 77

Slide 77 text

Dynamic Programming with Infinite States! There is a finite representation for all prefix candidates 2nd-order-diff constrained decision problem: ∃f s. t. (Xi ) x− i fi x+ i i = 1..n (Yi ) y− i fi − fi−1 y+ i i = 2..n (Zi ) z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n Pi := (fi, fi − fi−1) : ∃(f1, . . . , fi−2) ∀j ≤ i (Xj) ∧ (Yj) ∧ (Zj) possible last components of prefix solutions last value incoming slope 2nd-order-diff constrained decision problem ≡ Pn = {}? P2, . . . , Pn are all convex polygons. (defined by linear inequalities) Can represent them by their vertex sequence! P2 can be computed directly. How to efficiently compute Pi from Pi−1? ... by the magic of geometry and data structures! stay tuned Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 7 / 12

Slide 78

Slide 78 text

From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12

Slide 79

Slide 79 text

From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12

Slide 80

Slide 80 text

From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12

Slide 81

Slide 81 text

From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12

Slide 82

Slide 82 text

From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12

Slide 83

Slide 83 text

From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12

Slide 84

Slide 84 text

From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 x y Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12

Slide 85

Slide 85 text

From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 horizontal shear + y (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12

Slide 86

Slide 86 text

From Pi−1 to Pi – Step 1 Pi = (fi , fi − fi−1 ) : ∃(f1 , . . . , fi−2 ) ∀j i (Xj ), (Yj ), (Zj ) = (fi , fi − fi−1 ) : ∃fi−2 (fi−1 , fi−1 − fi−2 ) ∈ Pi−1 , (Xi ), (Yi ), (Zi ) = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] 1 Only z-constraints: Pi−1 P(z) i P(z) i = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1 , y − u ∈ [z− i , z+ i ] = (x + y + z, y + z) : (x, y) ∈ Pi−1 , z ∈ [z− i , z+ i ] (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 horizontal shear (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) slide along x = y + z + z z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 8 / 12

Slide 87

Slide 87 text

From Pi−1 to Pi – Step 2 2 Add x- and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12

Slide 88

Slide 88 text

From Pi−1 to Pi – Step 2 2 Add x- and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] Pi = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i intersect w/ halfplanes (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi x− 4 = 2.5 x+ 4 = 4 y− 4 = −1 y+ 4 = 10 Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12

Slide 89

Slide 89 text

From Pi−1 to Pi – Step 2 2 Add x- and y-constraints: P(z) i Pi P(z) i = (x, y) : ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] Pi = (x, y) : x ∈ [x− i , x+ i ], y ∈ [y− i , y+ i ], ∃u (x − y, u) ∈ Pi−1, y − u ∈ [z− i , z+ i ] (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i intersect w/ halfplanes (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi x− 4 = 2.5 x+ 4 = 4 y− 4 = −1 y+ 4 = 10 Can compute P2, . . . , Pn iteratively, check if Pn nonempty. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 9 / 12

Slide 90

Slide 90 text

Data Structures & Implicit Updates But: Not yet done! Pi has O(i) vertices Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 91

Slide 91 text

Data Structures & Implicit Updates But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 92

Slide 92 text

Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 93

Slide 93 text

Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 94

Slide 94 text

Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 95

Slide 95 text

Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 96

Slide 96 text

Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 97

Slide 97 text

Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 98

Slide 98 text

Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 99

Slide 99 text

Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Lemma: We only add O(n) new vertices for all polygons. ( 1 per cut edge) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 100

Slide 100 text

Data Structures & Implicit Updates (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) Pi−1 1 (1.7, 0) (2, 0) (2.4, 0.2) (3.2, 1) (3, 1) (2.4, 0.7) 1 z − 4 = − 0.8 z + 4 = − 0.2 (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) (0.9, −0.8) (1.2, −0.8) (1.6, −0.6) (3, 0.8) (2.8, 0.8) (2.2, 0.5) P(z) i 1 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, 0.65) Pi 2 But: Not yet done! Pi has O(i) vertices Can’t afford to update the vertices(!) Step 1 (mostly) +append 1 vertex each applies same linear transformations to vertices in upper resp. lower hull. Transform vertices lazily and compose just a matrix product! (homogeneous coordinates) transformations. Step 2 has to find edges intersecting a cutting plane efficiently. Claim: Two sorted deques for upper and lower hull vertices suffice. Lemma: All edges have (weakly) positive slope. Hulls remain sorted by x and y. Lemma: We only add O(n) new vertices for all polygons. ( 1 per cut edge) Linear search for intersections costs O(n) in total (search cost = # deleted vertices) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 10 / 12

Slide 101

Slide 101 text

Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can find f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12

Slide 102

Slide 102 text

Our result Main result: There is a single algorithm computing f s. t. 2nd-order constraints with maxi wi · |fi − yi| OPT + ε (ε-approximation) in time O n log(U ε ) for U = (max wj )(max yj − min yj ). Overview: O(n) alg. for decision version: Given L, ∃f with error L? Binary search for smallest L O log(U ε ) iterations 2nd-order-diff constrained decision problem: ∃f s. t. x− i fi x+ i i = 1..n y− i fi − fi−1 y+ i i = 2..n z− i (fi − fi−1 ) − (fi−1 − fi−2 ) z+ i i = 3..n This talk: only Yes/No But: can find f by backtracing ( paper) Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12

Slide 103

Slide 103 text

Outline 1 Introduction 1 Introduction 2 Our Result 2 Our Result 3 Geometric Dynamic Programming 3 Geometric Dynamic Programming 4 Conclusion 4 Conclusion Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 11 / 12

Slide 104

Slide 104 text

Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12

Slide 105

Slide 105 text

Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12

Slide 106

Slide 106 text

Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12

Slide 107

Slide 107 text

Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12

Slide 108

Slide 108 text

Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12

Slide 109

Slide 109 text

Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12

Slide 110

Slide 110 text

Conclusion You’ve seen First near-linear algorithm for a regression problem with 2nd-order shape constraints Geometric dynamic programming (1, −0.5)(2, −0.5) (2, 0.5) P2 (1.7, 0) (2, 0) (2.2, 0.2) (2.2, 1) (2, 1) (1.7, 0.7) P3 (2.5, 0.3) (3, 0.8) (2.8, 0.8) (2.5, .65) P4 (2.3, −0.2) (3. (3. (3, 0.5) P5 You have not seen → paper Handling general α (non-equidistant xi ’s) Simple O(n) algorithm for special case of unweighted L∞ convex regression. Hardness result for generalization to arbitrary DAGs (instead of path) separation from isotonic regression Open Problems Other error norms beyond L∞ Exact algorithms Parametric search? Extension to higher dimensions? Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12

Slide 111

Slide 111 text

Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 12 / 12

Slide 112

Slide 112 text

Icons made by Freepik, Gregor Cresnar, Those Icons, and Madebyoliver from www.flaticon.com. Other photos from www.pixabay.com. Sebastian Wild Second-Order Shape-Constrained Function Fitting 2019-08-06 13 / 12