Consider a metric space (X, dX ) of predictors and a space Y of observations. Given n observations Yn := {y1, . . . , yn } 2 Y⌦n, we would like to recover a tar- get µ? in M(X), the space of signed measures on X, from these n observations.
Consider a metric space (X, dX ) of predictors and a space Y of observations. Given n observations Yn := {y1, . . . , yn } 2 Y⌦n, we would like to recover a tar- get µ? in M(X), the space of signed measures on X, from these n observations. <latexit sha1_base64="O0NpgCD+4IH7y54tD3h5OgyevMs=">AAAGY3icjVLrahNBFD5tbKypl7b6T4TBriXFEpJWVAqBiiCCKNXaG512md2dJEN2Z+PMbNuwpI8p+ACCvoVnJpuaplrdkOTsd77znWvQi4U29fq3qenSjZnyzdlblbnbd+7em19Y3NVppkK+E6ZxqvYDpnksJN8xwsR8v6c4S4KY7wXd19a/d8KVFqn8bPo9fpSwthQtETKDkL8w/fWV1lnCiekwgz9oMNXmhghNIqFDxQ0nHkWzF7O+Nv2Y0yQ7phppTYqRft5tNgbH+bshNmDDf79LIx4b5udnI2DgESYjcspJh52g6LpHdMglU0JsVGjA20LmLBZtOahQw9q5t+03NiihCTMdleTbLMGByPbAQ7/FghY58CUlyyOGqIlaNCgi0Eu3OuKi2NUKpRe6a2O6bzIZ2lmwmBIhW1xxGfJLSchy88+S9EvGIlKVzcbKuPr6mPqHVOg+JXY9TFGScKYzxRMujZ5oZCILdjbK83SEc19W6Cp2wmU0GtVpBysmHsOBCzkkBuSTt0q8i8GPHCGLyb71jOfx7KaH5ZGWSpPCa7nvq7/DVjxiUlLtqTRggYiF6a/Y+zBKBJmdniZVu6/hiu2EvZWJRL6cTHV+Taox38FxTlMjEq6JxBuyedatOmY698ZGg+eU4iFLnDiv+fNL9VrdPeSq0SiMJSierXRhagMoRJBCCBkkwEGCQTsGBho/h9CAOvQQO4IcMYWWcH4OA6hgbIYsjgyGaBd/2/h2WKAS362mdtEhZonxqzCSwBOMSZGn0LbZiPNnTtmif9POnaatrY//QaGVIGqgg+i/4kbM/42zvRhowUvXg8Ceeg6x3YWFSuamYisnY10ZVOghZu0I/Qrt0EWO5kxcjHa929ky5//hmBa172HBzeDntd1Z1nCDyvV03X5yrEdjT6lTHvIsy7h6JHLsTKiLUq7nqKgiL67hut1rOEHr8u7PLm2/gnfamLzKq8buWq3xvPbs49rS5kZxsbPwEB5DFa/yBWzCW9iCHQhLzVJYikvJzPfyXHmx/GBInZ4qYu7Dpaf86Be8QLVl</latexit> Assume that the target is discrete µ? = K? X k=1 a? k x? k and we have 3 scenarii: Yn i.i.d µ?, (S1 : Sampling) Y = µ?, (n = 1) (S2 : Functional inference) Yn = nµ? + en , (S3 : Noisy linear measurements) where ak 2 R, x? k 2 X, is linear from M(X) to (probability) distributions (S1 and S2), n is linear from M(X) to Y⌦n (S3), and en some noise.
Consider a metric space (X, dX ) of predictors and a space Y of observations. Given n observations Yn := {y1, . . . , yn } 2 Y⌦n, we would like to recover a tar- get µ? in M(X), the space of signed measures on X, from these n observations. <latexit sha1_base64="O0NpgCD+4IH7y54tD3h5OgyevMs=">AAAGY3icjVLrahNBFD5tbKypl7b6T4TBriXFEpJWVAqBiiCCKNXaG512md2dJEN2Z+PMbNuwpI8p+ACCvoVnJpuaplrdkOTsd77znWvQi4U29fq3qenSjZnyzdlblbnbd+7em19Y3NVppkK+E6ZxqvYDpnksJN8xwsR8v6c4S4KY7wXd19a/d8KVFqn8bPo9fpSwthQtETKDkL8w/fWV1lnCiekwgz9oMNXmhghNIqFDxQ0nHkWzF7O+Nv2Y0yQ7phppTYqRft5tNgbH+bshNmDDf79LIx4b5udnI2DgESYjcspJh52g6LpHdMglU0JsVGjA20LmLBZtOahQw9q5t+03NiihCTMdleTbLMGByPbAQ7/FghY58CUlyyOGqIlaNCgi0Eu3OuKi2NUKpRe6a2O6bzIZ2lmwmBIhW1xxGfJLSchy88+S9EvGIlKVzcbKuPr6mPqHVOg+JXY9TFGScKYzxRMujZ5oZCILdjbK83SEc19W6Cp2wmU0GtVpBysmHsOBCzkkBuSTt0q8i8GPHCGLyb71jOfx7KaH5ZGWSpPCa7nvq7/DVjxiUlLtqTRggYiF6a/Y+zBKBJmdniZVu6/hiu2EvZWJRL6cTHV+Taox38FxTlMjEq6JxBuyedatOmY698ZGg+eU4iFLnDiv+fNL9VrdPeSq0SiMJSierXRhagMoRJBCCBkkwEGCQTsGBho/h9CAOvQQO4IcMYWWcH4OA6hgbIYsjgyGaBd/2/h2WKAS362mdtEhZonxqzCSwBOMSZGn0LbZiPNnTtmif9POnaatrY//QaGVIGqgg+i/4kbM/42zvRhowUvXg8Ceeg6x3YWFSuamYisnY10ZVOghZu0I/Qrt0EWO5kxcjHa929ky5//hmBa172HBzeDntd1Z1nCDyvV03X5yrEdjT6lTHvIsy7h6JHLsTKiLUq7nqKgiL67hut1rOEHr8u7PLm2/gnfamLzKq8buWq3xvPbs49rS5kZxsbPwEB5DFa/yBWzCW9iCHQhLzVJYikvJzPfyXHmx/GBInZ4qYu7Dpaf86Be8QLVl</latexit> Assume that the target is discrete µ? = K? X k=1 a? k x? k and we have 3 scenarii: Yn i.i.d µ?, (S1 : Sampling) Y = µ?, (n = 1) (S2 : Functional inference) Yn = nµ? + en , (S3 : Noisy linear measurements) where ak 2 R, x? k 2 X, is linear from M(X) to (probability) distributions (S1 and S2), n is linear from M(X) to Y⌦n (S3), and en some noise. <latexit sha1_base64="gmwHebuBZCprnzZYONC+UYLknpc=">AAAEG3icjVLNbtNAEB7X/BTzl9JjLxZJJSSkyGkRoJ4qcekxCNJWygZr7WzsVW2vtbsuVFafgSfgabghrhz6BuUBuDO72SDSitC1nMx+833j/XYmqQuudBRdeGv+rdt37q7fC+4/ePjocWfjyaESjUzZKBWFkMcJVazgFRtprgt2XEtGy6RgR8nJG5M/OmVScVG912c1m5Q0q/iMp1QjFHc+k3FANM3a3rt4t3cekLL5QJSmkhSiyiTPcoyl+BiSkuo8mYVkmPO4+kMLb8h7vsAZglclOdVIjKuATOJON+pHdoXXg4ELuuDWUGx4e0BgCgJSaKAEBhVojAugoPAZwwAiqBGbQIuYxIjbPINzCFDbIIshgyJ6gr8Z7sYOrXBvaiqrTvErBb4SlSFso0YgT2JsvhbafGMrG/RftVtb05ztDP8TV6tEVEOO6P90C+ZNdcaLhhm8th44eqotYtylrkpjb8WcPPzLlcYKNWImnmJeYpxa5eKeQ6tR1ru5W2rzl5ZpULNPHbeBnyvdGda8g9J6WtWfFs+j0JOwlec8w9L2PBVyzJ0Qq5LW89SdonXTsKr3Ck4xWu79p6XuBzing6tTeT043OkPXvZfvN3p7u+5iV2HLXgKz3AqX8E+HMAQRlj5l7fl9bxt/4v/1f/mf59T1zyn2YSl5f/4DX0g7h0=</latexit> µ? ! nµ? ! nµ? + en ! ˆ µn (S3)
[P1] (Mixtures): With scenario S1, the observation yk is sampled from a mixture density f?, and target µ? is a mixing law where a? k are mixture weights and x? k are mixture parameters: f? = K? X k=1 a? k 'x? k , and µ := Z X 'xdµ(x) , where 'x denotes a parametric density function with parameters x.
[P1] (Mixtures): With scenario S1, the observation yk is sampled from a mixture density f?, and target µ? is a mixing law where a? k are mixture weights and x? k are mixture parameters: f? = K? X k=1 a? k 'x? k , and µ := Z X 'xdµ(x) , where 'x denotes a parametric density function with parameters x. <latexit sha1_base64="M134ZZ9zsJf6tUf5gnQbbF9v3Bs=">AAAE03icjVJdb9MwFL0ZBUb42AaPvFgsSJ2YqnYgQJUmTZqEeCwa3SbNI3ISd7Wa2MF2tlVRXxCv/BR+DU9I8A7/gms3QxsTA1dtb8495+Qe20mZC2O73a/BwrXW9Rs3F2+Ft+/cvbe0vHJ/16hKp3yYqlzp/YQZngvJh1bYnO+XmrMiyfleMtl2/b1jro1Q8q2dlvywYEdSjETKLELxSvAqqmkyIgeDjcNZRNrbSlohK1UZslMybfhan+wJOyYm5ZJpoUi0Ez+N1okdc6ISw/WxdyLRNJ5ERBjCiFTCTImbiGlScGYqzQsuLVEjEtGiekeNZTrqhzThR0LW/H3lPWYhemy2acHsGEeig7GI5W/+Wjx5wuNJSJGdhdTyU1szmc0aoF2f1828ECX9TSqkjefNlOVkf0ZoaUQ88YguSIZEur4eUi6zc6OcjLnmOK7n+mBGFZxMpDqRJFGVzHhGRpVMHb1D4uXVbqfrF7lc9JpiFZo1UCtBHyhkoCCFCgrgIMFinQMDg58D6EEXSsQOoUZMYyV8n8MMQtRWyOLIYIhO8PcInw4aVOKz8zReneJbcvxqVBJ4jBqFPI21exvx/co7O/Rv3rX3dLNN8T9pvApELYwR/ZfujPm/OpfFwghe+gwCM5UecenSxqXyu+ImJ+dSWXQoEXN1hn2NdeqVZ/tMvMb47G5vme//8EyHuue04Vbw88p0jjU/Qe0zXXU+Nc5jMJPyznOeY1k/j0SO2xPqVdpnzpop6uY2XHX2Bo6xunj2pxdOP8R72vvzVl4udjc6veedZ282Vrf6zY1dhIfwCNp4K1/AFryGAQwhDT4HX4JvwffWsFW3PrQ+zqkLQaN5ABdW69MvLtMtBQ==</latexit> [P2] (Continuous Sparse): With scenario S3, the observation yk is a noisy linear measurement of µ?: yk = ( nµ?)k + ek and ( nµ)k := Z X kdµ , (1) where k is some known bounded function.
[P3] (Two-layer neural networks): With scenario S3, one observes a couple (yk, zk) of input data zk and response yk as a linear measurement of µ?: f? = K? X k=1 a? k 'x? k , yk = ( nµ?)k + ek and ( nµ)k := Z X 'x(zk)dµ(x) , where f? is the target function, 'x(z) := (hx, (1, z)i) = (x1 + Pd j=2 xjzj) is the neuron outcome with activation and weights x at input point z.
[P3] (Two-layer neural networks): With scenario S3, one observes a couple (yk, zk) of input data zk and response yk as a linear measurement of µ?: f? = K? X k=1 a? k 'x? k , yk = ( nµ?)k + ek and ( nµ)k := Z X 'x(zk)dµ(x) , where f? is the target function, 'x(z) := (hx, (1, z)i) = (x1 + Pd j=2 xjzj) is the neuron outcome with activation and weights x at input point z. <latexit sha1_base64="Z9gMox+zgoYJPkyd8jz8JDlZmqA=">AAAFnHicjVLrahNBFD6NRut6aas/BRnsFhKoISlFJVAoWIpQhEjtBTJpmN1MkiF7c2Y23bKmT+JffSffQN/CM5NNbVKsbsjume985/KdM14SCKXr9R9LpTt3y/fuLz9wHj56/GRlde3psYpT6fMjPw5ieeoxxQMR8SMtdMBPE8lZ6AX8xBu9M/6TMZdKxNEnfZHwTsgGkegLn2mEumulFTenXp+0W9udiUsqB1xGPCCHCZOKkz2uxCBS1SY5EXpI3MPulrtJBmLMo0u3f0aVZhKBc06YCAnTxKVhWsBE1XTNoW2HhkwPscR+BZ3V5g71xIB+maG0NRSIv6Ii0t3coj4LyOmE0DGTyVB0swr1e7GuFvUqWdWyZEh62TTXtbj9Cd3cdOjnlPWcDarSsJuPdhqTs/xgGj1h0293NEufZzNksrPQ1NRBMJHNRzXPdH6Ok5gUFRb4qG1RxZWIq56RhxIc22bHEYroISceV5qwJJFxJkK7GlLpx9L6fCk0lwZy50fpVkncJ9f2wKLepXtV0SV9znQqOQlZYpYzO4ooSTVJYuz10s2w5T/9urXu6nq9VrcPuWk0CmMdiqcVry01gUIPYvAhhRA4RKDRDoCBwl8bGlCHBLEO5IhJtIT1c5iAg7EpsjgyGKIjfA/w1C7QCM8mp7LRPlYJ8C8xksAGxsTIk2ibasT6U5vZoH/LnducprcL/HpFrhBRDUNE/xU3Y/5vnNGioQ9vrQaBmhKLGHV+kSW1UzGdk2uqNGZIEDN2D/0Sbd9GzuZMbIyy2s1smfX/tEyDmrNfcFP4das6w5puUFpNt+0nx34Uaopt5inPsLTtJ0KOmQm1UdJq7hVd5MVtuG33CsZoze8+m9u+g/e0sXgrbxrHW7XG69r2x6313WZxY5fhObyECt7KN7AL76EFR+CXxqWvpW+l7+UX5b3yQfnDlFpaKmKewdxTPv4NoBBxaQ==</latexit> [P4] (Kernel Sparse Designs): With S2, given f?, we aim at µ? s.t. F(µ) := µ Z X 'x(·)f?(x)dx F , with µ := Z X 'xdµ(x) , is the best approximation (for the criterion F(µ)) of f?, and 'x feature map at feature input point x 2 X.
Tensors): With scenario S3, we aim at finding a K?-rank d-way symmetric tensor ? from the noisy linear measurements yk: ? := K? X k=1 a? k x? k ⌦d = Z X x⌦ddµ?(x) , ( nµ)k := Z X k(x)dµ(x) , with a? k > 0, x? k are normalized (kx? k k2 = 1), k(x) := h⌧k , x⌦di for some ⌧? k 2 (Rn)⌦d (e.g., the canonical basis), h· , ·i the standard dot product of tensors, and ek 2 R some centered random variable. Note that ( nµ?)k = h⌧k , ?i is some linear form evaluated at the target point ?.
Tensors): With scenario S3, we aim at finding a K?-rank d-way symmetric tensor ? from the noisy linear measurements yk: ? := K? X k=1 a? k x? k ⌦d = Z X x⌦ddµ?(x) , ( nµ)k := Z X k(x)dµ(x) , with a? k > 0, x? k are normalized (kx? k k2 = 1), k(x) := h⌧k , x⌦di for some ⌧? k 2 (Rn)⌦d (e.g., the canonical basis), h· , ·i the standard dot product of tensors, and ek 2 R some centered random variable. Note that ( nµ?)k = h⌧k , ?i is some linear form evaluated at the target point ?. <latexit sha1_base64="GECKhd4GeikVHMbseN7NpZhlxbM=">AAAHY3icnVVtTxNBEB6wUgRfAP1AYkw2UiPE2rTEiCHRoCTGDxoxyouhhFyv13Lhrldu70BS6v/0B5joF3+Dz8zu0RewGu/S3b15n2dmp7V24OukXP42Nn4ld3UiP3ltavr6jZu3ZmbntnSUxq636UZBFO/UHO0FfsvbTPwk8HbaseeEtcDbrh2uM3/72Iu1H7U+Jadtby90mi2/4btOAlI0O/aLqtSiiHysdfKwJjRF2zgpckFvkQaPOTEor+gtvaSPeN/jS4MfUAoN30quQrdKu7IekANOlUJI7INbFR9VUGNqCp2/96ljZTJ+KHoH8O7AuqJ3tHgJdYeWqCt+atBje50+qRo1IPNa/C5eoH8WetH6XaJHOAWQCMGvY2euojPLPxvwYnzuYT2BPUaFkUrkzHuEn4nwWDL1xbPBRwnSMawqUDRWBTuFPk+KnuNXBS+ltpx62SmLUHKO2TAiXawNkct47KuDnLqWmspeRJ2qkmFDfAeI/QjflQGPXURWlAiz7Ljej60/1qmTUvNqXqLxIaUEPc6f42BKIhE3YeshXsbHs/kX/rtaBdCOcHIEC188eYLmAeI7wc5yLr56dQmxOoKqqVfBWiucVyLAfgheQKe2jlkOPStZdl8vid5EyfYWz/ui2KerRd7gFvfRa1gTi8uSaDhyE02Mw135gsriwxELKSgGX4Vu4W5zJFNjrURqf2ahXCrLoy4eKvawQPbZwDxYgdc6sndhPbTzwJU4NN5d9EgZvhLcgI545Aq4kpW5GSmkPInGBZ4OYvOgZagtG58WbVemB98H7s4H0ImkPkq8memTimWm/sl2R2xybKeClLGV1Ub/VS+T/Fe9XZl3DXomOTD6baFwdq61YiZiIPXtZcWd07bVr4PPHeKKZoazEh0tuZtqMv+HSGqpuovXyKb0c2R2dVAbMnWSkZXRmFPNocp8GajNKAQDnCKZcoxaKL0wSiOATGT/TVosiQ6tDPfjxcPWcqnytPTkw/LC2qrt1Um6S/dx1yq0Qmv0hjZok9zces7PxTk98T0/nZ/L3zGi42NW5zYNPPl7vwGp0mHp</latexit> We consider BLASSO solutions: ˆ µn œ arg min µœM(X) ) Fn(Yn, µ) + ⁄n |µ| * where the total variation norm is |µ| = sup ) s X fdµ : |f| Æ 1 * , the so-called “ data fitting ” term Fn(Yn, µ) quantifies how much the measure µ is likely to fit the data Yn (here, the smaller the better), and ⁄n > 0 a tuning parameter.
[P1] (Mixtures): With scenario S1, the observation yk is sampled from a mixture density f?, and target µ? is a mixing law where a? k are mixture weights and x? k are mixture parameters: f? = K? X k=1 a? k 'x? k , and µ := Z X 'xdµ(x) , where 'x denotes a parametric density function with parameters x.
[P1] (Mixtures): With scenario S1, the observation yk is sampled from a mixture density f?, and target µ? is a mixing law where a? k are mixture weights and x? k are mixture parameters: f? = K? X k=1 a? k 'x? k , and µ := Z X 'xdµ(x) , where 'x denotes a parametric density function with parameters x. • MLE is a non-convex program • Usually solved by EM • K is assumed to be known • Convergence of EM to global maximum can be tedious <latexit sha1_base64="BVLgunTX85W28llXDg5PV2q6OaM=">AAAENnicjVJdi9NAFL3Z+LHWr66CL74EW6GFbmkWcWVxYVEUofuwgt1d2FnDJJmmQyaZkExKS+h/8tG/4otvor75E7wzzaJ1sTql7Z1zzzkz9871M8ELNRh8sjbsK1evXd+80bh56/adu82te8eFLPOAjQIpZH7q04IJnrKR4kqw0yxnNPEFO/Hjlzp/MmV5wWX6Ts0zdp7QKOVjHlCFkNf8QFLJ05ClyjmU0fYhj9FqImXoUOW0iZowRfc71HN7RIRSFT3qDXuzX9uZN+y29whptEnIi0zQeaHmghGaRyThqVctLRbkBY9ItU2KMvEqvu8u3qdEyIj4POoswViDQ+rFZErzbMK9aubFi87ci7ua1TUOi7bXbA36A7Ocy4FbBy2o15HcsnaBQAgSAighAQYpKIwFUCjwcwYuDCBD7BwqxHKMuMkzWEADtSWyGDIoojH+Rrg7q9EU99qzMOoATxH4zVHpwGPUSOTlGOvTHJMvjbNG/+ZdGU99tzn++7VXgqiCCaL/0l0w/1ena1EwhmemBo41ZQbR1QW1S2m6om/u/FaVQocMMR2HmM8xDozyos+O0RSmdt1bavLfDFOjeh/U3BK+r60uRHSMZ2vuupcpYIrR6svMVt5mXQcFRprFTdcSMwvrFAI5Eu+mu5BqJk6o++c8Xg6Od/ru0/6Ttzutg716VjfhITyCDs7jLhzAGziCEQTWA+u59cp6bX+0P9tf7K9L6oZVa+7DyrJ//ATakfdq</latexit> Log-Likelihood at ◊ = (a1, . . . , aK, x1, . . . , xK ): arg min ◊ Ó ≠ n ÿ i=1 log ! K ÿ k=1 akÏxk (yk ) "Ô
ˆ fn = 1 n n ÿ i=1 ”yi • PDF: µı = K ÿ k=1 aı k Ï(xı k ≠ ·) • Inverse problem: µı æ µı æ ˆ fn <latexit sha1_base64="X3h4wR49MjjcuKavIL2YzSkCzHA=">AAAETnicjVJdaxNBFL3bVI3xo6k++jKYFSJISIqoFIRCofggUqFpCtkQZiez3aWzO+vMbGrZ5s/5C3z1H+gf8E30zmQKxmJ0lk3unHvu2Tlzb1yKTJt+/0uw0di8cfNW83brzt1797fa2w+OtawU40MmhVQnMdVcZAUfmswIflIqTvNY8FF8tm/zozlXOpPFkbko+SSnp0WWZIwahKbtT2EUV0JwQ0KyL4u5FJVNEFlyRY1UCL/tJk/JaxIJVJ3RSBuqSBI+I+cpV5yEHg9JpomuWEpMSlEtyqlJGRX1wWLsKZOuQSGXiJN6sJjWkYgVZZxcmstI8A/kKFJLYBGS88ykKzIhKnNygM4zrohRtNCJVHlv2u70e323yPVg4IMO+HUot4NdiGAGEhhUkAOHAgzGAihofMYwgD6UiE2gRkxhlLk8hwW0sLZCFkcGRfQMf09xN/ZogXurqV01w68IfBVWEniCNRJ5CmP7NeLylVO26N+0a6dpz3aB/7HXyhE1kCL6r7or5v/WWS8GEnjlPGToqXSIdce8SuVuxZ6c/ObKoEKJmI1nmFcYM1d5dc/E1Wjn3d4tdfmvjmlRu2eeW8G3te4sa9lB5Tyt60+N59HoSTrlJc+yjDtPgRx7J5GrUs7zzJ+i9tOwrvca5hit9v7jSvdbOKeDP6fyenC80xu86D1/v9PZ2/UT24RH8Bi6OJUvYQ/ewCEMgQXd4F0wCk4anxvfGz8aP5fUjcDXPISVtdn8BYAA/FE=</latexit> • Convolution operator L(f) = ? f, where is such that F[ ](t) = 1{|t|T } with F the Fourier transform. • Kernel defines a RKHS L <latexit sha1_base64="QIRCQoDVQB+x6okIctKE3iKHVUA=">AAADwnicjVLLbhMxFL3p8GjDK4UlG4sMEqtoUiEeXVWCRaWyKI+0lZqq8sw47VDPA9tTUQX+hK9hCz/QP4C/4PjGlQgVAY8muT73nDO+vjdtdGFdkpx3lqIrV69dX17p3rh56/ad3urdHVu3JlOjrNa12UulVbqo1MgVTqu9xihZplrtpicvfH73VBlb1NU7d9aog1IeVcWkyKQDdNh7Eo/TVmvlRCy2lKmUFvFYQ5/LWORqAlsrpHiztfkWiVK649yKV/Fhr58MEl7icjAMQZ/C2q5XO+s0ppxqyqilkhRV5BBrkmTx7NOQEmqAHdAUmEFUcF7RZ+pC24KlwJBAT/B7hN1+QCvsvadldYavaLwGSkEPoanBM4j91wTnW3b26N+8p+zpz3aG/zR4lUAdHQP9l+6C+b86X4ujCT3jGgrU1DDiq8uCS8u34k8ufqvKwaEB5uMceYM4Y+XFPQvWWK7d363k/A9metTvs8Bt6efC6jxr1kHDNS3qzxTnsaipZucZz7Mcn6cCx9/JmFWGa87DKaZhGhb13tIpovnef5zrfhdzOvxzKi8HO2uD4ePB89dr/Y31MLHLdJ8e0CNM5VPaoE3aphGcv9BX+kbfo5fR++hDZGfUpU7Q3KO5FX36BVLXyN4=</latexit> <latexit sha1_base64="jmbsFTv5ZHoWQ4CG7VqaOLAZjuQ=">AAAERHicjVJdaxNBFL3b9aPGj6b6KMhgESpqSIKohAoFRXxQqZK0hWwSZmdnk6H7xexsNW720f+k7+J/8B9U8E18Fe/MbqqxGJ2Q3TvnnnP23pnrJoFIVbP52VqxT50+c3b1XO38hYuX1urrl3fTOJOM91gcxHLfpSkPRMR7SqiA7yeS09AN+J578Ejn9w65TEUcddU04YOQjiPhC0YVQqP6eyekauKl5Bl5SBxXjJ3AlZRx4nfmmVdDjzhSjCeKShm/Jsc4ccpYhnnaUI1C72fEx8coP7Ythm3tLCL1C+zf6d7uDoZeQXIyMyCjQf6k6PuDTXVzNmwXc2PiEUW2yC3U+2paFihNgaP6RrPRNIucDFpVsLG9NXz34ejj25143eqAAx7EwCCDEDhEoDAOgEKKvz60oAkJYgPIEZMYCZPnUEANtRmyODIoogf4HOOuX6ER7rVnatQMvxLgX6KSwA3UxMiTGOuvEZPPjLNG/+adG09d2xTfbuUVIqpggui/dHPm/+p0Lwp8eGB6ENhTYhDdHatcMnMqunLyW1cKHRLEdOxhXmLMjHJ+zsRoUtO7Pltq8keGqVG9ZxU3gy9Lu9Os8gal6WnZ/eRYT4o9xca55GmWMvVEyNFn4hiVND17VRV5NQ3L7j6FQ4wW7/7Nwu3XcE5bf07lyWC33Wjda9x9iQPbgXKtwlW4Dps4lfdhG57CDvSAWdesx9Zz64X9yf5qf7O/l9QVq9JcgYVl//gJ2sz9fg==</latexit> L = f : Rd ! R s.t. kfk2 L = Z [ T,T ]d |F[f](t)|2dt < +1 Solving the Mixture: Hilbert space for data fi delity
8 <latexit sha1_base64="1VI0CH+BB7gRZgGWPDCmmFIooDQ=">AAAEUnicjVLBbhMxEJ1tApRQIIUjF4sEhEQb7UaIokpIFVw4FpSkkeoq8nq9iVWvvex6A1GaP+TIhT+AX+DE2GwlQiHgKMnzm/fGM2PHuZKlDcMvwVajee36je2brVs7t+/cbe/eG5WmKrgYcqNMMY5ZKZTUYmilVWKcF4JlsRIn8flrFz+Zi6KURg/sIhdnGZtqmUrOLFKT9ieqjdSJ0JZ0aVwpJWyXDGaCDEb72hQZ6V7QrLqYRC9pWeX0lZzSJZXaTpY0Y3bGmSLjVeoxihPUUnJISbSf0ql4H9I9+pjuRU/rnbOvupS2/nTqO6GYFQmxhlgsoMwZF8SkZEnjlGijtZhi0XNB0kpzV325IlITRpIKq/jAFr1JuxP2Qr/IVRDVoAP1Oja7wQFQSMAAhwoyEKDBIlbAoMTPKUQQQo7cGSyRKxBJHxewghZ6K1QJVDBkz/F3irvTmtW4dzlL7+Z4isJvgU4Cj9BjUFcgdqcRH698Zsf+LffS53S1LfA/rnNlyFqYIfsv36Xyf32uFwspvPA9SOwp94zrjtdZKj8VVzn5pSuLGXLkHE4wXiDm3nk5Z+I9pe/dzZb5+FevdKzb81pbwbeN3SXIpni20266mRLmiNZv5uPa3WyaoELkVNJPLfNvYZNDocZgbW4K2inxhUa/v8erYNTvRc97z972O0eH9VvdhgfwEJ7gezyAI3gDxzAEHvSDccCCuPG58b0ZNBs/pVtB7bkPa6u58wPZ4/0x</latexit> • The TV-norm |µ|1 = sup Ó s X fdµ : 1 ≠ f Ø 0 & 1 + f Ø 0 Ô • Related to the space of nonnegative functions in a dual way.
8 <latexit sha1_base64="iNyzvIOGxDyw2V6O8QKKDZ44hEI=">AAAEF3icjVJNbxMxEJ3t8lHCV1qOCMkiApLLalMhWlVCqoIqcakUpKatFIeVs3G2Vrzele2tWqX9ERz5NdwQV478A/gFXBmbrSBUBBwleX7z3tgznnEphbFx/CVYCa9dv3Fz9Vbj9p279+4319YPTFHplA/SQhb6aMwMl0LxgRVW8qNSc5aPJT8cz165+OEJ10YUat+elXyUs0yJqUiZRSppvqOqEGrClSU9XmlMk5E9oUTOJNk9tZqVhfRS0u7t7XYIi2YRi0iPGWFIv9KmErZBhw1qWTZHxUWD5kIlNK9oT2R0fo7oPOlSsk0J7RuR7LeR6bz8hd9SY5nueDm6R0mzFUexX+Qq6NagBfXqF2vBJlCYQAEpVJADBwUWsQQGBj9D6EIMJXIjmCOnEQkf53ABDfRWqOKoYMjO8DfD3bBmFe5dTuPdKZ4i8avRSeAJegrUacTuNOLjlc/s2L/lnvuc7m5n+D+uc+XIWjhG9l++S+X/+lwtFqaw5WsQWFPpGVddWmepfFfczclvVVnMUCLn8ATjGnHqnZd9Jt5jfO2ut8zHv3qlY90+rbUVfFta3QTZKZ7ttMtexsAJosWXOV14m2UdlIicSviu5X4Wljkkagq8m+uCckqc0O6f83gVHGxE3RfR8zcbrZ3telZX4SE8hjbO4ybswGvowwBP+h48Cp4Gz8L34YfwY/jpp3QlqD0PYGGFn38A4ZbpdQ==</latexit> Beurling Minimal Extrapolation (BME) a.k.a. Basis Pursuit min µ Ó |µ|1 : T (µ) = T (µı) Ô (BME) <latexit sha1_base64="1VI0CH+BB7gRZgGWPDCmmFIooDQ=">AAAEUnicjVLBbhMxEJ1tApRQIIUjF4sEhEQb7UaIokpIFVw4FpSkkeoq8nq9iVWvvex6A1GaP+TIhT+AX+DE2GwlQiHgKMnzm/fGM2PHuZKlDcMvwVajee36je2brVs7t+/cbe/eG5WmKrgYcqNMMY5ZKZTUYmilVWKcF4JlsRIn8flrFz+Zi6KURg/sIhdnGZtqmUrOLFKT9ieqjdSJ0JZ0aVwpJWyXDGaCDEb72hQZ6V7QrLqYRC9pWeX0lZzSJZXaTpY0Y3bGmSLjVeoxihPUUnJISbSf0ql4H9I9+pjuRU/rnbOvupS2/nTqO6GYFQmxhlgsoMwZF8SkZEnjlGijtZhi0XNB0kpzV325IlITRpIKq/jAFr1JuxP2Qr/IVRDVoAP1Oja7wQFQSMAAhwoyEKDBIlbAoMTPKUQQQo7cGSyRKxBJHxewghZ6K1QJVDBkz/F3irvTmtW4dzlL7+Z4isJvgU4Cj9BjUFcgdqcRH698Zsf+LffS53S1LfA/rnNlyFqYIfsv36Xyf32uFwspvPA9SOwp94zrjtdZKj8VVzn5pSuLGXLkHE4wXiDm3nk5Z+I9pe/dzZb5+FevdKzb81pbwbeN3SXIpni20266mRLmiNZv5uPa3WyaoELkVNJPLfNvYZNDocZgbW4K2inxhUa/v8erYNTvRc97z972O0eH9VvdhgfwEJ7gezyAI3gDxzAEHvSDccCCuPG58b0ZNBs/pVtB7bkPa6u58wPZ4/0x</latexit> • The TV-norm |µ|1 = sup Ó s X fdµ : 1 ≠ f Ø 0 & 1 + f Ø 0 Ô • Related to the space of nonnegative functions in a dual way.
8 <latexit sha1_base64="iNyzvIOGxDyw2V6O8QKKDZ44hEI=">AAAEF3icjVJNbxMxEJ3t8lHCV1qOCMkiApLLalMhWlVCqoIqcakUpKatFIeVs3G2Vrzele2tWqX9ERz5NdwQV478A/gFXBmbrSBUBBwleX7z3tgznnEphbFx/CVYCa9dv3Fz9Vbj9p279+4319YPTFHplA/SQhb6aMwMl0LxgRVW8qNSc5aPJT8cz165+OEJ10YUat+elXyUs0yJqUiZRSppvqOqEGrClSU9XmlMk5E9oUTOJNk9tZqVhfRS0u7t7XYIi2YRi0iPGWFIv9KmErZBhw1qWTZHxUWD5kIlNK9oT2R0fo7oPOlSsk0J7RuR7LeR6bz8hd9SY5nueDm6R0mzFUexX+Qq6NagBfXqF2vBJlCYQAEpVJADBwUWsQQGBj9D6EIMJXIjmCOnEQkf53ABDfRWqOKoYMjO8DfD3bBmFe5dTuPdKZ4i8avRSeAJegrUacTuNOLjlc/s2L/lnvuc7m5n+D+uc+XIWjhG9l++S+X/+lwtFqaw5WsQWFPpGVddWmepfFfczclvVVnMUCLn8ATjGnHqnZd9Jt5jfO2ut8zHv3qlY90+rbUVfFta3QTZKZ7ttMtexsAJosWXOV14m2UdlIicSviu5X4Wljkkagq8m+uCckqc0O6f83gVHGxE3RfR8zcbrZ3telZX4SE8hjbO4ybswGvowwBP+h48Cp4Gz8L34YfwY/jpp3QlqD0PYGGFn38A4ZbpdQ==</latexit> Beurling Minimal Extrapolation (BME) a.k.a. Basis Pursuit min µ Ó |µ|1 : T (µ) = T (µı) Ô (BME) <latexit sha1_base64="VMTRwKSazNinO2y/01JuwO8BCm4=">AAAFTnicjVJdbxJBFL0g1YpfrT76MrFoloQQaIw1VZNaY2JSTdDQj4ShZFlmYcLsznZmtqWh/D5fffUf6B/wzeidYanFKrpklztnzrn3njvTTQTXplb7nMtfKSxdvbZ8vXjj5q3bd1ZW7+5pmaqA7QZSSHXQ9TUTPGa7hhvBDhLF/Kgr2H53+Mru7x8zpbmMm+Y0Ye3I78c85IFvEOqs5j7SWPK4x2JDxrQbkuaAScWiCSl5L8v0rYz7goVG8f7A+ErJE2+7XCInA6YYpUXLKW2SEo3SQ6qRUCJaitSmJkYSb/vd6/Izx9u2PGNlhI3QlrYi3wwCX5AG5bFbqGj8wY/7bOLRhuad5iE1MsFyOg0GKPYNpqIt4vEyPUr9Hjn7leKMCnZE6pTQR/iew96oM5x2Vn4xK6F5P554/vkGrVRQEkrlC0GG2Etrp12k7WKRRqynhzzB6HxGftybs8u1dUXSmB+l7LJ5wkMSpsoaj3CsZGbgDw68UfnsOTq42M0ICxscznjUqU8rVqjoSaMro87OFKATWqliu52VtVq15h5yOahnwRpkT0Ou5jaAQg8kBJBCBAxiMBgL8EHjrwV1qEGCWBvGiCmMuNtnMIEialNkMWT4iA7x28dVK0NjXNuc2qkDrCLwVagk8BA1EnkKY1uNuP3UZbbo33KPXU7b2yn+d7NcEaIGBoj+Szdj/q/OejEQwlPngaOnxCHWXZBlSd1UbOfkgiuDGRLEbNzDfYVx4JSzOROn0c67na3v9r84pkXtOsi4KXxd6K6HaIi1LXfRyWg4xmj+ZEZzZ7NoggIjy+JuapG7C4sUAjkSe7NTiC0Tb2j99/t4Odhbr9afVB+/X1/b2szu6jLchwfg4X3cgC14Aw3YhSBfzTfz7fxh4VPhW+F74ceUms9lmnsw9ywt/wSJX1Oh</latexit> Theorem (A) ≈∆ (B) where (A): µı solution to (BME); (B): there exists P œ Range( € T ) such that (i) |P| Æ 1 & P(xı k ) = sign(aı k ) , ’k œ [K] and µı is the unique solution to (BME) if furthermore (ii) |P(x)| < 1 , ’x / œ {xı 1, . . . , xı K } . <latexit sha1_base64="1VI0CH+BB7gRZgGWPDCmmFIooDQ=">AAAEUnicjVLBbhMxEJ1tApRQIIUjF4sEhEQb7UaIokpIFVw4FpSkkeoq8nq9iVWvvex6A1GaP+TIhT+AX+DE2GwlQiHgKMnzm/fGM2PHuZKlDcMvwVajee36je2brVs7t+/cbe/eG5WmKrgYcqNMMY5ZKZTUYmilVWKcF4JlsRIn8flrFz+Zi6KURg/sIhdnGZtqmUrOLFKT9ieqjdSJ0JZ0aVwpJWyXDGaCDEb72hQZ6V7QrLqYRC9pWeX0lZzSJZXaTpY0Y3bGmSLjVeoxihPUUnJISbSf0ql4H9I9+pjuRU/rnbOvupS2/nTqO6GYFQmxhlgsoMwZF8SkZEnjlGijtZhi0XNB0kpzV325IlITRpIKq/jAFr1JuxP2Qr/IVRDVoAP1Oja7wQFQSMAAhwoyEKDBIlbAoMTPKUQQQo7cGSyRKxBJHxewghZ6K1QJVDBkz/F3irvTmtW4dzlL7+Z4isJvgU4Cj9BjUFcgdqcRH698Zsf+LffS53S1LfA/rnNlyFqYIfsv36Xyf32uFwspvPA9SOwp94zrjtdZKj8VVzn5pSuLGXLkHE4wXiDm3nk5Z+I9pe/dzZb5+FevdKzb81pbwbeN3SXIpni20266mRLmiNZv5uPa3WyaoELkVNJPLfNvYZNDocZgbW4K2inxhUa/v8erYNTvRc97z972O0eH9VvdhgfwEJ7gezyAI3gDxzAEHvSDccCCuPG58b0ZNBs/pVtB7bkPa6u58wPZ4/0x</latexit> • The TV-norm |µ|1 = sup Ó s X fdµ : 1 ≠ f Ø 0 & 1 + f Ø 0 Ô • Related to the space of nonnegative functions in a dual way.
• For all µı œ M ™ M(X), Alice sends T (µı) and Bob has to uncover µı; • Impossible for M = M(X); • Solved by (BME) decoder when M = M Ø C · K1/2 · d3/2 · # 1 T $ [≠T, T]d ™ Supp(F[Ï]) where M := Ó µ : ÷K Ø 1 , ÷x1, . . . , xK œ X s.t. min i”=j dX (xi, xj ) Ø Ô with X ™ Rd. <latexit sha1_base64="3Ipix8gf3cGb9wE4t6lecxXS9As=">AAADnXicjVJdSxtBFD1x22rtV2wffBDapVKwUMKmSC19EmpFSikKxghGwuxkooOzH+zOSiX4a/qqP6j/oD75FzwzrtBUTDshyZlzz7kz986Nc6NLG0W/GlPBvfsPpmcezj56/OTps+bc850yqwqpOjIzWbEbi1IZnaqO1dao3bxQIomN6sZHn128e6yKUmfptj3J1X4iDlI91FJYUv3m/FIvEfZQChNu9Efb73prylhx+rbfXIxakV/hbdCuweLqy/VRf+Hy62Y211hBDwNkkKiQQCGFJTYQKPnZQxsRcnL7GJEriLSPK5xilt6KKkWFIHvE3wPu9mo25d7lLL1b8hTDb0FniDf0ZNQVxO600Mcrn9mxd+Ue+Zzubif8j+tcCVmLQ7L/8t0o/9fnarEY4qOvQbOm3DOuOllnqXxX3M3DP6qyzJCTc3jAeEEsvfOmz6H3lL5211vh47+90rFuL2tthYuJ1Q3IDnm20056mRLHROMv82PsbSZ10BA5lfZdS/wsTHIYajLezXUhdUpOaPvvebwNdt632h9ay1sc1U+4XjNYwGsscR5XsIoNbKLjz/6JM5wHr4Ivwbfg+7V0qlF7XmBsBd0rEJK/ug==</latexit> (HT, )
• What can be said about BLASSO? ˆ µn œ arg min µ Ó 1 2 ÎL( ˆ fn) ≠ T (µ)Î2 L + Ÿn |µ|1 Ô <latexit sha1_base64="wrwevCvnGhDGplZjSDp+DOrJYao=">AAAGNXicjVJdbxtFFL2JcSlLaRsQT7yMiFvZyHXtgChKFamkHwJFkQJ12kgeezW7O7anmZ3Z7symjlb+S7zyW3joW4Xgib/AnfHarVuaMpbtM+d+nXvnRpkUxna7f2xs1j6qX/r48ifBp1c+u3rt+tbnT4wu8pgfx1rq/CRihkuh+LEVVvKTLOcsjSR/Gp3ed/anZzw3Qqu+Pc/4MGUTJcYiZhapcGvzN6q0UAlXljRoVEjJbYM80jlhUiKTFiNqLMupUDRldpoYckhNERlu+XPPxEySw+YKnrQabfKjFDEnhit0b9AjI8J+c5Wq1SBMJWRfR2TKDLGaFCrWKPKNco27QUBTnphTkSH6D4k/p5k2RmCfZKx9aKVu7z2qPpjxsZZnPCHROWnuHz5skYTHOkFVL6ZckYBGfCJUyaSYqG/mwaocubm3wiF9wKVllAYLQG7SCX9O7tM40ZYcjMre7Z354pKMym+XFxqJyYCOcxb3+g4PMcHgVr/dH44STLE27TwtHxdZNn/d2qMBPWN5NhXDVkBx4iuJqDvnAR0Eb+vb3aP7YkJLnDYluxR74zPcNEMOnNwebbeXxCzstalEiaY9Cw+WO+AHSttLPaZjO3OaChWWgirs99ncm6IxScLydcS8OQsF5nnWcmUWUgKvBMc5DF4IO62ecVFgre8oIr+OkkYnvL7d7XT9Ie+CXgW2oTpHemvjDlBIQEMMBaTAQYFFLIGBwc8AetCFDLkhlMjliIS3c5hDgLEFenH0YMie4u8Eb4OKVXh3OY2PjrGKxG+OkQRuYIxGvxyxq0a8vfCZHfu+3KXP6bSd439U5UqRtTBF9kNxS8//G+d6sTCGH3wPAnvKPOO6i6sshZ+KU07e6Mpihgw5hxO054hjH7mcM/ExxvfuZsu8/S/v6Vh3jyvfAv6+sLsE2THWdr4XvYyBM0TrLzNbe5uLJigROS/hp5b6XbgoQqKPRm1uCsp54ob23t7Hd8GTnU7v+853v+xs39utdvUyfAVfQxP38Q7cg5/gCI4hrn1Zu1t7UHtY/73+sv6q/ufCdXOjivkC1k79n38BIiWnSw==</latexit> • For all µı œ M ™ M(X), Alice sends T (µı) and Bob has to uncover µı; • Impossible for M = M(X); • Solved by (BME) decoder when M = M Ø C · K1/2 · d3/2 · # 1 T $ [≠T, T]d ™ Supp(F[Ï]) where M := Ó µ : ÷K Ø 1 , ÷x1, . . . , xK œ X s.t. min i”=j dX (xi, xj ) Ø Ô with X ™ Rd. <latexit sha1_base64="3Ipix8gf3cGb9wE4t6lecxXS9As=">AAADnXicjVJdSxtBFD1x22rtV2wffBDapVKwUMKmSC19EmpFSikKxghGwuxkooOzH+zOSiX4a/qqP6j/oD75FzwzrtBUTDshyZlzz7kz986Nc6NLG0W/GlPBvfsPpmcezj56/OTps+bc850yqwqpOjIzWbEbi1IZnaqO1dao3bxQIomN6sZHn128e6yKUmfptj3J1X4iDlI91FJYUv3m/FIvEfZQChNu9Efb73prylhx+rbfXIxakV/hbdCuweLqy/VRf+Hy62Y211hBDwNkkKiQQCGFJTYQKPnZQxsRcnL7GJEriLSPK5xilt6KKkWFIHvE3wPu9mo25d7lLL1b8hTDb0FniDf0ZNQVxO600Mcrn9mxd+Ue+Zzubif8j+tcCVmLQ7L/8t0o/9fnarEY4qOvQbOm3DOuOllnqXxX3M3DP6qyzJCTc3jAeEEsvfOmz6H3lL5211vh47+90rFuL2tthYuJ1Q3IDnm20056mRLHROMv82PsbSZ10BA5lfZdS/wsTHIYajLezXUhdUpOaPvvebwNdt632h9ay1sc1U+4XjNYwGsscR5XsIoNbKLjz/6JM5wHr4Ivwbfg+7V0qlF7XmBsBd0rEJK/ug==</latexit> (HT, )
BLASSO 10 <latexit sha1_base64="GTNTOG51j+E1fILa9tM2+cUwkwo=">AAAFbHicjVJ/axNBEJ3ERmv81ar/FWGxZ0kgjUkQKxWh0CIFRWJJWqHbHJvLJllytxd390pKkg/qBxD0Hz+Ds5trTVtMe+Eus2/ee7Mzu+1hKLSpVH5ksneWcnfvLd/PP3j46PGTldWnhzpOVMCbQRzG6lubaR4KyZtGmJB/GyrOonbIj9qDXZs/OuVKi1g2zNmQn0SsJ0VXBMwg5K9mflIZC9nh0pAxbXfJXsJCssuVmZH4lDQxq4hXoBEz/QCz+/64UaJ7PDRsWvRKxPS54oSPcLeaeBe0ukd0EvQxzUyetnlPyDGXScSVtc1TYXg0T6dCuoWKxgdM9vi0QOta+I0WNfGw6L0/V0z+SSY05N9JlRK6ge8FXBj5gxbVhqnihyotlTDXjRULQzLAIsefTrxSauZa/sKZIge4v1huT+d3VBgVZwU2dxutGp2MNkczW39AJ36tVcPm0ZhYZ2+E7ZZNGfVXeeihtRZRofq6UfTmK3+8sTDlht2ySs+ouSJcduam7a+sV8oV95DrQTUN1iF96vFqZgsodCCGABKIgIMEg3EIDDT+jqEKFRgidgJjxBRGwuU5TCGP2gRZHBkM0QF+e7g6TlGJa+upnTrAKiG+CpUEXqEmRp7C2FYjLp84Z4v+z3vsPO3ezvC/nXpFiBroI3qT7px5W53txUAX3rkeBPY0dIjtLkhdEjcVu3My15VBhyFiNu5gXmEcOOX5nInTaNe7nS1z+V+OaVG7DlJuAr8XdtdBtIu1LXfRyWg4xejyyYwunc2iCYYYWZZwU4vcXVikCJET497sFKRl4g2tXr2P14PDWrn6tvzma219Zzu9q8uwBi+hgPdxC3ZgH+rQhCD7Oauy4+xk6U/ueW4t92JGzWZSzTO49OQ2/gKF91s3</latexit> Dual Certificate Under (HT, ), there exists P such that 1. P œ Range( € T ); 2. |P| Æ 1 & P(xı k ) = 1 , ’k œ [K], 3. Near Region: P(x) Æ 1≠CT2 Îx≠xı kÎ 2 2, for all x s.t. Îx≠xı kÎ2 . (1/T) 4. Far Region: P(x) Æ 1 ≠ ÷, for all x s.t. Îx ≠ xı kÎ2 & (1/T)
sha1_base64="vn+1IurrFk0ExfzGyC12qoxgmpM=">AAAFFnicjVJNbxMxEJ2EAGX5auAIB4umqKhplFSIolSVKuDAsUhNGikOkXfX2Vr1ehevt2qU9jdw5ddwQ1y58g/gD3BmbLZNQ0uooyTPb+a98Yztp1Jkptn8Xipfq1y/cXPhlnf7zt179xerD7pZkuuAd4JEJrrns4xLoXjHCCN5L9Wcxb7ke/7BaxvfO+Q6E4naNeOUD2IWKTESATNIDaulj1QlQoVcGVKjfi4lNzVEMTP7AZNkp0ZERhjJcj/SLBQ2LxkRs8/JbndNJTomzJAULaw+zt/TzDBd26TUu8z4leZRzBQJBR4q4irgHu17Z9XeDCfTyicr6Fc/83zW3jrGzfGwtXZ8RtodxdrndL2TqYVDeMTQWq1NrWjEP5AmrW96dHDpOXfZGCdH+FHKlJ1dm9SK4t7WTHVvlUqmIsmnResztah24dWr9ejVhotLzUbTLXIRtAqwBMXaSaqlDaAQQgIB5BADBwUGsQQGGX760IImpMgNYIKcRiRcnMMJeKjNMYtjBkP2AH8j3PULVuHeemZOHWAViV+NSgLLqEkwTyO21YiL587Zsv/ynjhPe7Yx/vuFV4ysgX1k/6c7zbyqzvZiYAQvXQ8Ce0odY7sLCpfcTcWenJzryqBDipzFIcY14sApT+dMnCZzvdvZMhf/4TIta/dBkZvDz7ndhciOsLbNnXczGRwimr2Zo5m7mTdBichmCTe12L2FeQqJOQmezU5B2Ux8oa2/3+NF0F1vtF40nr9bX9puF291AR7BE1jB97gB2/AWdqADQelX+XF5ufy08qnyufKl8vVParlUaB7CzKp8+w138EU/</latexit> • P is a subgradient of the TV-norm at point µı; • Bregman divergence DP (µ, µı) := |µ|1 ≠ |µı|1 ≠ ⁄ X Pd(µ ≠ µı) Ø 0 ; • Taylor expansion: |µ|1 = |µı|1 + ÈP, µ ≠ µıÍ + DP (µ, µı) <latexit sha1_base64="KaucstCv7PmRe5gw+snna/FVOQo=">AAAGB3icjVTdahNBFD5pjNZotVXQgjeLUVSIIQliRRBCreBFwQjpDzRp2J1skqX7E3ZnqyX2AXwa78RbH8MnUN/C75ydVJPY6A7MnjnnOz/fzJlxRr6X6Gr1W24pf6Fw8dLy5eKVqyvXrq+u3dhNojRW7o6K/Cjed+zE9b3Q3dGe9t39UezageO7e87RS7bvHbtx4kVhS5+M3E5gD0Kv7ylbQxWt5RrUppAi8jD3yMWsyaIxtA71IbVoCG1EMeaATqHZMcgY8j16CGRANryGpPD3oX1NXURoURm2LSB9WG34PgK+DHtf4rF3m45gGWHY8AnpxV+ivREL54mhjWTFkSx6J7VxZRYVYT+QmXMNkH8T2FRqtSCFmLXgLWvdWjecE6wfYJyK5ySzg2HRK2g2gRkgbps+0LbUMBQMc8jqeAxdE3E8rFtmN1I6xD+RSmJg2LsrezqdYRt5D6l+lqcDyUdFiQwP6OIU6wxbxmB9B7MntbDdB89kah8WsZnd462Z+ib6Jir8k3XGjmspn8N0wqQ4x8WaYWM4dFdL1UpVPmteqBmhROZromc34NlDFIX8gelZhWy25DqgGlXRURrRx1KXRn4l1WTnnMq5c88p6b8BVgdGG2IdSN3srWRnuVu58+/Dh3uKe5ezWWJPJTJrz4s9lphc2wn+jok12evkn34T5P/6MReNDn0mHPhmj0TD7JSJksqu+HJ3frPSchuzu92Dne+WEs/g7G3ITtWSHoklr6YfgmQtr5XBpvRzITvu2L7cTb3wZBI6hjR9Mu+nzmbRDvqQGOXJrgXSC4s8fGAi88KFjESH1mb7cV7YrVdqTytP3tZLjeemV5fpDt3F/anRBjXwKjbxdqql7/mV/K387cLHwqfC58KXDLqUMz43aeorfP0FRdAktA==</latexit> Theorem Under (HT, ), for Ÿn = On (fln ) where E Ë ÎL( ˆ fn ) ≠ T (µı)Î2 L È . fl2 n , (Bound on the “noise”) it holds E Ë DP (ˆ µn, µı) È . fln
Theorem Under (HT, ), for Ÿn = On (fln ) where E Ë ÎL( ˆ fn ) ≠ T (µı)Î2 L È . fl2 n , (Bound on the “noise”) it holds T 2 Ë Kı ÿ k=1 |ˆ µn |(Nk )”xı k , |µı| È . fln , |ˆ µn | Ë X\ ! Kı € k=1 Nk "È Æ fln , and max kœ[Kı] |ˆ ak ≠aı k | . fln where T 2 is a partial transport distance close to unbalanced Wasserstein-2, Nk is the ball centered at xı k with radius Ô fln and ˆ ak = ˆ µn (Nk ). Furthermore, one can prove that fln = On (1/ Ô n) .
sha1_base64="gWT57j7NEYDBub1HllnLdJrZUsQ=">AAAMbXicnVb9T91UGD7M6caGbmhMlmhMI5hBciGwGGdITObYmIsuQxkwI9vS3nu4VHrbpqflY8gf6n+gP/kv+LzPe1rai6DxEtpz3vN+Pu/HaZQnsSuXln6fuPLO1Xffu3Z98sbNqfc/uHV7+sMtl1VF3272syQrXkahs0mc2s0yLhP7Mi9sOIoSux3tr8r59oEtXJylL8rj3L4ahcM03o37YQlSNn3VmR0TGWuGJjapOcGzxG6E91u8T80NnNe0AOfCvYvVqsnAH5s+1usmNAV4ZJeAMzBPsA/NABQLrhKUR1g5nOv+1KyAlpmcUrU1B1qI/11TgavPM7ES+OcsOGfxzmkvxM7Ss4KSs2bOHJk3Zt/0cCbveXLPQVMIv8SzgY8hpk/5mN8Ofs2DXpo9nAiHSIrliFEdglskS0YhvqrdHXgSUkqxWYP1FHbH6T+T3gPdIcIRdifw8huzDLuvsR7hrZ7vwFMLn0ruT3xcpz4i0ageHNKv2kuHkwqSxw1PAB2Kb+nx28EqIwIpfCrBqxhl5oBILoyhW/jcDIhMRN195sPhZEAZ0TVs2czA9ytWmsED0hSlymOS09fsfyAnOhQFrYnapqPffW9fZMT/kEgXTW3JM+7IHYK7ALaukXtIPXvkT1kxq9gJCuJR7wL64oWdssEcDBqU1ohuCpsLZhs2E3BZ3xGO+wPvr+IsuyNWa4H90OfGNQgcMs59yNW5GXksSmqULDrm2MLLIf575EvA6XxPRL5DzlCo8Zk1L8wWnguwJUiN2J+yamOoHiwwy45VW/hKlR7PaGsI/+ZhPYBGwU0zkoCe+b7ao/a4mQMa/4BnOgmEP/AYtKfLoDNd2h0sdXTcwbKAvG3qQOdNzFlWkp6yNyNy1mg4nuUex0e0k0Gv9XOlxwlXQb942ONMtDgvzJ+ditmAVAXdMf2Vjqp4orYcc2f/MXJ3SYV9T/mU3qiNDV9PD1nzmmOtloJxaEcoWmFnzu4SAdmlvBXCVg+fn8tRS38dY8hIJErHSs1Z/1r7jhMlZAxSDz1fobbVd0L7iXMs7mD3zE9rqbUKtbbGqK3XUnP9QI8czwqfGZEcsntS7i0rQzJ1MabPfXUX7Dyp4gr8Ifc6GfdAHfDOWGG+Vf8xPJOpmjdTSedg4Wu3K6tI1dPG0ddNZO8pZstZTI9BWQcmT9E57Vg1Twtjc0Kmq84KxVNnSXSuEobEx/r7YuR7YNRMtDrH3bkxB8+ew8t1Vm/tSRDcCe74b4KSmipWujN38SdZ+Rb+b0CmnfHHnHUxsRH6E86ByEdTd9KG16T12o7/O+bnLRGVTEouLc/Ofce8uT2ztLjEX3B+sewXM8b/1rPpifu8hTNgWDFrElmfNSC+/IJbe4nfL69grfsdoV9NFT3WSbzPCrSQqnwcer86Svf9hCwgGZgvWG8VKy1p5mBFzUK9SPeJvxcc8inzS3XV96j7V7ma87/KSSwleuVrxiA5z0mxnBGqpSIq+k14FlXJGtXeGjD/2iO2wTnw32P6bVP42+yP1k3eb6Z9hRl7WXTSZ7u+Mi/LjEPNDccyc9TJzWUIjt9jMTG4WCIBT+YnfyqcqNDl8Xo8v9i6t7j81eKXP96bebDia/W6+cR8jr5cNvfNA/TEOiZIf/LZpJv8bfL05l9TH099OvWZsl6Z8DIfmc5v6u7fTcFJfQ==</latexit> • Conic Particle Gradient Descent: optimizes a function on m parame- ters (xk, ak ) (called particles) that can be written as Fn (Yn, qm k=1 ak”xk ) then we can study the limit m æ Œ (over-parametrized) by considering the objective µ ‘æ Fn (Yn, µ) on the space of measures as in the works of Bach and Chizat, and Chizat. • Sliding Frank-Wolfe: solves convex programs on weakly compact sets (e.g., closed balls of the TV -norm for the weak-ı topology). This algorithm is a conditional gradient descent that may converge in a finite number of steps (Denoyelle, Duval, Peyr´ e, and Soubies) under some conditions. • Kernel SoS: Based on representation of nonnegative function based and a subsampling strategy, see Bach, Rudi, and Marteau-Ferey, and Lasserre, Magron, et al. • Other popular methods: Prony-type spectral methods such as MUSIC and ESPRIT, and non-convex approaches based on greedy minimization (e.g., (COMP) and “Continuous” LARS), see Elvira, Gribonval, Soussen, and Herzet.
ydecastro
flekschas
takasehideki
k141303
hf149
tomonom
kobayashi31
takehaya
.png){kind=avatar}
satai
yukako_nakano
junkurihara
kinoue_smarthr
rfujii
.png){kind=avatar}
helenjbeal
csswizardry
addyosmani
scottboms
jdejongh
mfonobong
greggifford
lemiorhan
kimpetersen
techseoconnect
palkan
![MAS - MODE 2022 BLASSO 5 <latexit sha1_base64="DRjIGdxUkUokfYj55AvDsBAsdRk=">AAAHF3icjVPNbiNFEO5sglnMz27gwIFLQRrJlryWHX5lKWglLkhIKLCbTaR0MmrPtO2Wp7tNd09iM+sX4A14Gm6IK0feAI68AdU94yEmu4GxnNTUz1ffV1UeL3Lp/GDw+8693b1XWq/ef639+htvvvXg4f7bz5wpbCpOUpMbezbmTuRSixMvfS7OFlZwNc7F6Xj+ZYifXgnrpNFP/WohLhSfajmRKffoSvZ3f6QlG0/g/PiTizWFzpOVUsJbmcJToZ2xrjuCU+ln4FKhuZUG6JPkI9qDawFcKuAeJlJnUk+BA/36kjnPLX1kuZ4Dzeija74C12D6iAmUKe5n2JU5OVW8LoKJNQr8TIA20q0gSOIWlOCusEIJ7R3QVTKnozY7b78IYnTEXKGScn40XF+WNZl1HUzmrFcuNzbGmfFSCQfZ+ohJ7ZOImPIczsrlVjgGrIKMqaKq7yy7rNdrs+8LnrU7DZXjmUw0JnWT+ajCLP8BXQNbOJnMQ+0NwA3URfs6jJk2bL8Y4JRpQ5gCt2EyVvFc/iAy6FD2vImy58nh0ZB2saTpghRyrqe5gIag50UYQ4/1llsCbczDDeBynFHixoZiSdUGFdVax/DdZanX3ZsgeDyiP+334gZTro2WQTcep3RdVNKQSTPjI4fK2vQOZdhFZ9xmgAFYWJMVqQczqe/G9QDDQAVqkLphQivKeJ9eWBwMAmZ4SFd4rRx/Bn34xniB8Hiq9PauKmm4sbum9aJja4hLVxGo7xVnqEBc8bzgHtlg16AMC6YCNRm8ipfcfz95eDDoD+IDt41hbRyQ+jk2+zsjwkhGDElJQRQRRBOPdk44cfg5J0MyIAv0XZASfRYtGeOCrEkbawvMEpjB0TvHv1N8O6+9Gt8DpovVKXbJ8WuxEsiHWGMwz6IdukGMFxE5eF+GXUbMwG2F/8c1lkKvJzP0/lfdJvP/1gUtnkzI51GDRE2L6Anq0hqliFMJzOGGKo8IC/QFO8O4RTuNlZs5Q6xxUXuYLY/xP2Jm8Ib3tM4tyJ93qgtZ1QZt1HTXfkrk41CTichVXsjykY/GnDATFqts1JzVLMr6Gu7avSNXaG3vfrm1/Tbe6fDfV3nbeHbYH37a//jbw4PHo/pi75P3yAekg1f5GXlMviLH5ISku3/tvbsHe++3fmr93Pql9WuVem+nrnmHbD2t3/4GMw76xA==</latexit> [P5] (Symmetric](https://files.speakerdeck.com/presentations/dbfebfa781964ea5a275ffcfb69d9948/slide_8.jpg){kind=link}
![MAS - MODE 2022 BLASSO 5 <latexit sha1_base64="DRjIGdxUkUokfYj55AvDsBAsdRk=">AAAHF3icjVPNbiNFEO5sglnMz27gwIFLQRrJlryWHX5lKWglLkhIKLCbTaR0MmrPtO2Wp7tNd09iM+sX4A14Gm6IK0feAI68AdU94yEmu4GxnNTUz1ffV1UeL3Lp/GDw+8693b1XWq/ef639+htvvvXg4f7bz5wpbCpOUpMbezbmTuRSixMvfS7OFlZwNc7F6Xj+ZYifXgnrpNFP/WohLhSfajmRKffoSvZ3f6QlG0/g/PiTizWFzpOVUsJbmcJToZ2xrjuCU+ln4FKhuZUG6JPkI9qDawFcKuAeJlJnUk+BA/36kjnPLX1kuZ4Dzeija74C12D6iAmUKe5n2JU5OVW8LoKJNQr8TIA20q0gSOIWlOCusEIJ7R3QVTKnozY7b78IYnTEXKGScn40XF+WNZl1HUzmrFcuNzbGmfFSCQfZ+ohJ7ZOImPIczsrlVjgGrIKMqaKq7yy7rNdrs+8LnrU7DZXjmUw0JnWT+ajCLP8BXQNbOJnMQ+0NwA3URfs6jJk2bL8Y4JRpQ5gCt2EyVvFc/iAy6FD2vImy58nh0ZB2saTpghRyrqe5gIag50UYQ4/1llsCbczDDeBynFHixoZiSdUGFdVax/DdZanX3ZsgeDyiP+334gZTro2WQTcep3RdVNKQSTPjI4fK2vQOZdhFZ9xmgAFYWJMVqQczqe/G9QDDQAVqkLphQivKeJ9eWBwMAmZ4SFd4rRx/Bn34xniB8Hiq9PauKmm4sbum9aJja4hLVxGo7xVnqEBc8bzgHtlg16AMC6YCNRm8ipfcfz95eDDoD+IDt41hbRyQ+jk2+zsjwkhGDElJQRQRRBOPdk44cfg5J0MyIAv0XZASfRYtGeOCrEkbawvMEpjB0TvHv1N8O6+9Gt8DpovVKXbJ8WuxEsiHWGMwz6IdukGMFxE5eF+GXUbMwG2F/8c1lkKvJzP0/lfdJvP/1gUtnkzI51GDRE2L6Anq0hqliFMJzOGGKo8IC/QFO8O4RTuNlZs5Q6xxUXuYLY/xP2Jm8Ib3tM4tyJ93qgtZ1QZt1HTXfkrk41CTichVXsjykY/GnDATFqts1JzVLMr6Gu7avSNXaG3vfrm1/Tbe6fDfV3nbeHbYH37a//jbw4PHo/pi75P3yAekg1f5GXlMviLH5ISku3/tvbsHe++3fmr93Pql9WuVem+nrnmHbD2t3/4GMw76xA==</latexit> [P5] (Symmetric](https://files.speakerdeck.com/presentations/dbfebfa781964ea5a275ffcfb69d9948/slide_9.jpg){kind=link}
