Random Features 〜Shift invariant kernelからGraph kernelまで〜

Transcript

1. Random Features Shift invariant kernelからGraph kernel まで ⽊村⼤翼 Retrieva Engineer Casual Talk 2019/10/16
2. ⾃⼰紹介 • ⽊村⼤翼(@big_wing) • 株式会社レトリバリサーチャー • 興味のある研究分野 • 機械学習 • ⾼速で省メモリなデータ構造, アルゴリズム • 趣味 • 野球観戦 • ⽇本酒, ビール • 経歴 • 理研AIP (テクニカルスタッフ@圧縮情報処理ユニット〜2019/3) • 株式会社レトリバ(リサーチャー 2019/4〜) © 2019, Retrieva, Inc. All rights reserved. 2
3. 本⽇の内容: カーネル関数の⾼速な近似計算⼿法について紹介 • Random Features • カーネル法はデータ数が増えると計算が重い • カーネル関数を近似的に⾼速計算するための⼿法の⼀つ • 内積の期待値がカーネル関数と⼀致するように特徴ベクトルをランダ ム射影によって作成 • もともとShift invariant なクラスのカーネル関数に対して提案 • その後様々なクラスのカーネル関数に対して拡張 • 多項式カーネル • 構造データに対するカーネル © 2019, Retrieva, Inc. All rights reserved. 3
4. カーネル法の概略: 特徴ベクトルの内積であるカーネル関数を通して学習する枠組み (·) <latexit sha1_base64="uNrFUI5/wH1ZJBBdbMPYpWk1Qtg=">AAACbnichVHLSsNAFD2N7/poVRBBRFEqFaFM6kJxVXTjsj6qQisliaMdmiYhmRZq8QdcKy4ERUFE/Aw3/oCLfoK4ERTcuPA2KYiKesNkzpy5586ZO7pjCk8yVg8pLa1t7R2dXeHunt6+SLR/YMOzy67BM4Zt2u6WrnncFBbPSCFNvuW4XCvpJt/Ui0uN/c0Kdz1hW+uy6vDtkrZniV1haJKobC5dEPGcsWPL6Xx0kiWYH+M/gdoEk6mJ3MxRPVVN29Fr5LADGwbKKIHDgiRsQoNHXxYqGBzitlEjziUk/H2OA4RJW6YsThkasUX679Eq22QtWjdqer7aoFNMGi4pxxFjD+yGvbB7dsse2fuvtWp+jYaXKs16oOVOPnI4vPb2r6pEs0ThU/WnZ4ldzPteBXl3fKZxCyPQV/ZPXtYWVmO1KXbJnsj/BauzO7qBVXk1rlb46inC9ADq93b/BBvJhDqbSK7QSywiiE6MYAJx6vccUlhGGhm/Y8c4w3noWRlSRpWxIFUJNTWD+BJK/APPhJDH</latexit> © 2019, Retrieva, Inc. All rights reserved. 特徴マッピング 元のデータ空間 線形分離不可 特徴空間 線形分離可 k(x, y) = (x)> (y) <latexit sha1_base64="KZk8ZFWlFp/kYucxanvwPGV8tXc=">AAACf3icSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQtIZWtU6FRq2sYEZGRqVGjGxZTkF4DZlZrxAsoGegZgoIDJMIQylB2UYrSn3HSoDMgXWM4Qw5DCkM+QzFDKkMuQypDHUAJk5zAkMhQDYTSDIYMBQwFQLJahGihWBGRlguVTGWoZuIB6S4GqUoEqEoGi2UAyHciLhormAfkgM4vBupOBtuQAcRFQpwKDqsFVg5UGnw1OGKw2eGnwB6dZ1WAzQG6pBNJJEL2pBfH8XRLB3wnqygXSJQwZCF143VzCkMZgAXZrJtDtBWARkC+SIfrLqqZ/DrYKUq1WM1hk8Bro/oUGNw0OA32QV/YleWlgatBsBi5gBBiiBzcmI8xIz9BYzygQGBNODBDAwSDNoMSgAQxvcwYHBg+GAIZQoL0NDMsY1jNsYGJkUmfSYzKAKGVihOoRZkABTJYAc3aVaw==</latexit> (x) <latexit sha1_base64="Cw84B9t3waQQJFRmzhNXqX7V3kE=">AAACanichVFNSwJBGH7cvu3L7FJ4EcswAhntUHSSunTUSg1UZHebdGjdXXZXyaQ/0C0IgjoVREQ/o0t/oIM/Iepm0KVDr6sQJdU7zMwzz7zPO8/MKKYmbIexpkfq6x8YHBoe8Y6OjU9M+qb8GduoWipPq4ZmWLuKbHNN6DztCEfju6bF5Yqi8axysNHez9a4ZQtD33HqJi9U5JIu9oUqO0Rl88myiBwuFn1zLMrcCPaCWBfMJUL5pbNmop40fLfIYw8GVFRRAYcOh7AGGTa1HGJgMIkroEGcRUi4+xzH8JK2SlmcMmRiD2gs0SrXZXVat2varlqlUzTqFimDCLMndsda7JHds2f28Wuthluj7aVOs9LRcrM4eTKz/f6vqkKzg/KX6k/PDvax6noV5N10mfYt1I6+dnTe2l7bCjcW2DV7If9XrMke6AZ67U29SfGtS3jpA2I/n7sXZOLR2HI0nqKfWEcnhhFACBF67xUksIkk0q67U1zg0vMq+aVZKdBJlTxdzTS+hTT/CRGpjxE=</latexit> を陽を求めずに を直接計算することで特徴空間の次元に依存しない k(x, y) <latexit sha1_base64="jZm7wX3AcgVqkMHzquekkPICsAo=">AAACaXichVFNSwJBGH7cvsy+tC5RF1GMopCxDkUnqUvH1DRBRXa3yTb3i91V2qQ/0KlDENWpICL6GV36Ax38CeGxoEuHXlchKqp3mJlnnnmfd56ZkUxVsR3Gmj6hp7evf8A/GBgaHhkdC4bGc7ZRs2SelQ3VsPKSaHNV0XnWURyV502Li5qk8m2put7e365zy1YMfctxTV7SxIqu7Cqy6BCVq84eLLhz5WCUxZkX4Z8g0QXRZKQ4f9pMuptG8BZF7MCAjBo0cOhwCKsQYVMrIAEGk7gSGsRZhBRvn+MIAdLWKItThkhslcYKrQpdVqd1u6btqWU6RaVukTKMGHtid+yFPbJ79szef63V8Gq0vbg0Sx0tN8tjx5OZt39VGs0O9j5Vf3p2sIsVz6tC3k2Pad9C7ujrh2cvmdV0rDHDrlmL/F+xJnugG+j1V/kmxdOXCNAHJL4/90+QW4wnluKLKfqJNXTCj2lEMEvvvYwkNrCJLJ27jxOc48LXEkLCpDDVSRV8Xc0EvoQQ/QAPxY6a</latexit> 4
5. 正定値カーネル: 正定値カーネルと特徴空間が⼀対⼀に対応 • カーネル関数 が以下を満たすとき正定値 カーネルと呼ぶ 1. 対称性: 2. 正定値性: に対してグラム⾏列 が半正定値⾏列 © 2019, Retrieva, Inc. All rights reserved. k : ⌦ ⇥ ⌦ ! R <latexit sha1_base64="x1GrpMtSB9TRVrSWJwJyr2I8Occ=">AAAClHichVFNSxtBGH6yaquprVFBCl4Wg6WndGIFRTyIQeipxtioYCTMrpNkyH4xO4loyB/wLh48KZRS/AkebcE/4MGfID1a6KUH390sllbavsvOPO8z7/POMzNW4MhQM3aTMvr6B548HRxKPxt+/mIkMzq2EfotZYuy7Tu+2rJ4KBzpibKW2hFbgRLctRyxaTUL0fpmW6hQ+t4HvR+IHZfXPVmTNtdEVTNvmgtmZdUVdW5WtHRF+JApWW9orpS/Z1ZcrhtWrVPqpquZLMuxOMzHIJ+ALJIo+plPqGAXPmy04ELAgybsgCOkbxt5MATE7aBDnCIk43WBLtKkbVGVoApObJPGOmXbCetRHvUMY7VNuzj0K1KamGbX7DO7Y1fsnN2yn3/t1Yl7RF72abZ6WhFURw5frv/4r8qlWaPxS/VPzxo1zMdeJXkPYiY6hd3Ttw+O79YXStOdV+yMfSP/p+yGXdIJvPZ3++OaKJ0geoD8n9f9GGzM5PJvczNrs9ml5eQpBjGJKbym+57DEt6hiDLte4QLfMFXY8JYNArGSq/USCWacfwWxvt71Dablg==</latexit> k(x, y) = k(y, x) <latexit sha1_base64="x3lMZiRBEnFlxjw4E8IgriojF6g=">AAACdXichVHLSsNAFD2N7/qKuhFECNaWFkqdqqAIgujGpa1WhVpKEkcNTZOQpMFa/AF/wIVuFFTEz3DjD7jwE8SlghsX3qQB0aLeYWbOnLnnzpkZxdI1x2XsKSK0tXd0dnX3RHv7+gcGxaHhTces2SovqKZu2tuK7HBdM3jB1Vydb1s2l6uKzreUyoq/v+Vx29FMY8OtW7xUlfcNbU9TZZeosihWkofpekpalCrJevowFS2LMZZhQUitIBuCGMJYM8Ub7GAXJlTUUAWHAZewDhkOtSKyYLCIK6FBnE1IC/Y5jhElbY2yOGXIxFZo3KdVMWQNWvs1nUCt0ik6dZuUEuLskd2yV/bA7tgz+/i1ViOo4Xup06w0tdwqD56Mrr//q6rS7OLgS/WnZxd7mA+8auTdChj/FmpT7x2dvq4v5OONBLtkL+T/gj2xe7qB4b2pVzmeP4P/Admfz90KNqcz2ZnMdG42trQcfkU3xjCBJL33HJawijUU6FwP57jCdeRdGBcmhUQzVYiEmhF8C2HqE1SmjoQ=</latexit> 8x1, ..., xn 2 ⌦ <latexit sha1_base64="IRM9JsQ+iJztJTc0Vg+Ysvtw6Nk=">AAAChXichVHLLgRBFD3aezxmsJHYdEyIBZ1qBLEhbOw8B4mRSXerGRXVj3T3TDCxtfADFlYkIsKWH7DxAxY+QSxJbCzc6elEENxKVZ06dc+tU1WmJ0UQMvZYo9TW1Tc0NjUnWlrb2pOpjs7VwC36Fs9YrnT9ddMIuBQOz4QilHzd87lhm5KvmTuzlf21EvcD4Tor4Z7HN22j4Ii8sIyQqFxKzeZd35BS3c3pg6qmaYOEHDUrqM/bvGDkUmmmsSjUn0CPQRpxLLipC2SxBRcWirDB4SAkLGEgoLYBHQwecZsoE+cTEtE+xwESpC1SFqcMg9gdGgu02ohZh9aVmkGktugUSd0npYo+9sAu2Qu7Z1fsib3/Wqsc1ah42aPZrGq5l0sedS+//auyaQ6x/an603OIPCYir4K8exFTuYVV1Zf2j1+WJ5f6yv3sjD2T/1P2yO7oBk7p1Tpf5EsnSNAH6N+f+ydYHdb0EW14cTQ9PRN/RRN60IsBeu9xTGMOC8jQuYe4xg1ulUZlSBlVxqqpSk2s6cKXUKY+ANi2lA8=</latexit> G = 0 B @ k(x1, x1) · · · k(x1, xn) . . . ... . . . k(xn, x1) · · · k(xn, xn) 1 C A <latexit sha1_base64="OzQ7WjzNjb4pADUjUXI/zVmiths=">AAAC9HichVHLSuRAFL2JOmrUsdWN0JswbYvC0FRUUASh0YUufbUKRpokXbZFJ5WQVAe16R9w49KFbmakERH8CTf+wCz8BHGp4saFN+kwzoyPuSFVp06dc++tKtOzWSAIuZHklta2L+0dnUpXd8/X3lRf/1rgVn2LFizXdv0N0wiozTgtCCZsuuH51HBMm66blblofz2kfsBcvir2PLrlGGXOtpllCKSKqcN5dUbRTVpmvOY5hvDZbl2pjOwWte8qDqPqsKpbJVcECH7TfFTVdUUPYz4OVJUSVUKjINLz99PwP9JQXnotXUxlSI7Eob4FWgIykMSimzoDHUrgggVVcIACB4HYBgMC/DZBAwIecltQQ85HxOJ9CnVQ0FtFFUWFgWwFxzKuNhOW4zrKGcRuC6vY+PvoVCFLfpFzck+uyQW5Jc8f5qrFOaJe9nA2m17qFXsPBlee/utycBaw8+r6tGcB2zAV98qwdy9molNYTX+4f3S/Mr2crQ2Tn+QO+/9BbsgVnoCHD1ZjiS4fQ/QA2r/X/RasjeW08dzY0kQmP5s8RQek4RuM4H1PQh4WYBEKWPdRSktDUlYO5RP5VG40pbKUeAbgr5AvXwCLPbd3</latexit> G <latexit sha1_base64="2GAFnKXap8EB31HMBh4eqEDhzQk=">AAACZnichVFNLwNBGH66vqqoIkLi0mgqTs0sEuLUcOCorSKhaXbXtCa2u5vdaRMaf0DiysGJRET8DBd/wME/II4kLg7e3W4iNHgnM/PMM+/zzjMzumMKTzL2GFE6Oru6e6K9sb7+gfhgYmh4w7PrrsGLhm3a7pauedwUFi9KIU2+5bhcq+km39T3l/39zQZ3PWFb6/LA4aWaVrVERRiaJKqwkoyVEymWYUEk24EaghTCWLMT19jBLmwYqKMGDguSsAkNHrVtqGBwiCuhSZxLSAT7HEeIkbZOWZwyNGL3aazSajtkLVr7Nb1AbdApJnWXlEmk2QO7Ya/snt2yZ/bxa61mUMP3ckCz3tJypzx4PF54/1dVo1li70v1p2eJChYCr4K8OwHj38Jo6RuHZ6+FxXy6OcUu2Qv5v2CP7I5uYDXejKscz5/D/wD153O3g42ZjDqbmcnNpbJL4VdEMYFJTNN7zyOLVayhSOdWcYJTnEWelLgyqoy1UpVIqBnBt1CSnzUZigk=</latexit> 5
6. 正定値カーネルの例: Shift invariant kernel 平⾏移動に対して不変なカーネル関数 • Shift invariant kernel: に対して と書くことができる正定値カーネル関数 • ガウスカーネル: • ラプラスカーネル: © 2019, Retrieva, Inc. All rights reserved. k(x, y) = (x y) <latexit sha1_base64="kCncEYjEc4Vv+NXPunQURcF9l6o=">AAACeHichVHLSsNAFD2N7/po1Y3gJliqLWiZVkERBNGNS1+tgkpJ4rQOTZOQpMVY/AF/wIULURAVP8ONP+DCTxCXCoK48CYNiIp6h5k5c+aeO2dmVEsXjsvYQ0RqaW1r7+jsinb39PbF4v0DBces2RrPa6Zu2puq4nBdGDzvClfnm5bNlaqq8w21sujvb9S57QjTWHc9i+9UlbIhSkJTXKKK8cFKan/cS8tz8ra1J1L7E146WownWIYFIf8E2RAkEMayGb/ENnZhQkMNVXAYcAnrUOBQ20IWDBZxO2gQZxMSwT7HIaKkrVEWpwyF2AqNZVpthaxBa7+mE6g1OkWnbpNSRpLds2v2zO7YDXtk77/WagQ1fC8ezWpTy61i7Gho7fVfVZVmF3ufqj89uyhhJvAqyLsVMP4ttKa+fnD8vDa7mmyMsnP2RP7P2AO7pRsY9RftYoWvnsD/gOz35/4JCrlMdjKTW5lKzC+EX9GJYYwgRe89jXksYRl5OtfDKa5wHXmTZGlMSjdTpUioGcSXkHIfPDuP1Q==</latexit> x, y 2 Rm <latexit sha1_base64="r0LWOm1jq5IMuD6OI3eDqVPrzp4=">AAACe3ichVHdLgNBGD1df1V/RSISNxsNEZFmWoS4Em5cUopEaXbXlIn9y+60UeUFvIALVyQiwlu48QIuPIK4JHFD4tvtJoLgm8zMmTPf+ebMjO6awpeMPcSUhsam5pZ4a6KtvaOzK9nds+o7Zc/gecMxHW9d13xuCpvnpZAmX3c9rlm6ydf0vflgf63CPV849oqsunzT0nZsURKGJokqJvv2x9SqWhC2WrA0uavrtdzRlpUoJlMszcJQf4JMBFKIYtFJXqKAbTgwUIYFDhuSsAkNPrUNZMDgEreJGnEeIRHucxwhQdoyZXHK0Ijdo3GHVhsRa9M6qOmHaoNOMal7pFQxxO7ZFXtmd+yaPbK3X2vVwhqBlyrNel3L3WLXcf/y678qi2aJ3U/Vn54lSpgOvQry7oZMcAujrq8cnDwvz+SGasPsnD2R/zP2wG7pBnblxbhY4rlTBB+Q+f7cP8FqNp0ZT2eXJlKzc9FXxDGAQYzQe09hFgtYRJ7OPcQ5rnETe1dSyqgyVk9VYpGmF19CmfwAIgiSIA==</latexit> k(x, y) = e kx yk2 2 2 <latexit sha1_base64="/piaHNSloJ30X5f+LYd3rNHZsW4=">AAACmHichVFNSxtBGH5cP6rxK7WX0l6CwaKgYRIFRSiE9tB602hUcDXMrpM4ZL/YnQTTNX/AP1BKTy0UEX+EBy/6Azz4E8SjhV566LubhaKivsPMPPPM+7zzzIzhWTJQjF11ad09vX0v+gdSg0PDI6Ppl2PrgdvwTVE2Xcv1Nw0eCEs6oqykssSm5wtuG5bYMOofo/2NpvAD6TprquWJbZvXHFmVJldEVdKz9cn96dZU5n1Gt7na8+1QtHdCvepzM5zRD/ZnWvrBTqEdFvRA1mxOsJ2qpLMsx+LIPAT5BGSRxLKbPoKOXbgw0YANAQeKsAWOgNoW8mDwiNtGSJxPSMb7Am2kSNugLEEZnNg6jTVabSWsQ+uoZhCrTTrFou6TMoMJdsmO2S07Zyfsmv19tFYY14i8tGg2OlrhVUYPX6/+eVZl06yw91/1pGeFKhZir5K8ezET3cLs6Jtfvt6uLpYmwnfsJ7sh/z/YFTujGzjN3+avFVH6jugD8vef+yFYL+Tys7nCyly2+CH5in68xTgm6b3nUcRnLKNM537DKc5xob3RitonbamTqnUlmle4E1rpH8CAnUQ=</latexit> k(x, y) = e |x y| <latexit sha1_base64="T6myfCX2tbVGrsRPpF8N0yKi5Wo=">AAACiHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQsoZ2tU6FRqKtgqxOQmlmQU5Van1sZV68YkpZYkKtRU6FbW1HIBVRnoGYCBAibDEMpQZoCCgHyB5QwxDCkM+QzJDKUMuQypDHkMJUB2DkMiQzEQRjMYMhgwFADFYhmqgWJFQFYmWD6VoZaBC6i3FKgqFagiESiaDSTTgbxoqGgekA8ysxisOxloSw4QFwF1KjCoGlw1WGnw2eCEwWqDlwZ/cJpVDTYD5JZKIJ0E0ZtaEM/fJRH8naCuXCBdwpCB0IXXzSUMaQwWYLdmAt1eABYB+SIZor+savrnYKsg1Wo1g0UGr4HuX2hw0+Aw0Ad5ZV+SlwamBs1mAEWAIXpwYzLCjPQMjfWMAk2UHZygUcHBIM2gxKABDG9zBgcGD4YAhlCgvZ0M6xl2MOxk4mIyYDJnsoQoZWKE6hFmQAFMTgDW4ZZd</latexit> 6
7. 正定値カーネルの例: 構造データに対するカーネル 2つの⼊⼒に共通する部分構造を数え上げる • 正定値カーネルであれば⼊⼒は実数ベクトルでなくてもよい • ⽂字列, ⽊構造, グラフ構造 © 2019, Retrieva, Inc. All rights reserved. k(x, y) = X x02S(x) X y02S(y) k0 S (x0, y0) <latexit sha1_base64="v4uRHV1WpR3PDDe+hL6I1JcxmBg=">AAACoHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQtYZ2tU6ChUairYKsQUl+bGV1eoK8Rk5ikEa1Ro1kKFKmFClZq12erxQCl1oBZ1zXgBZQM9AzBQwGQYQhnKDFAQkC+wnCGGIYUhnyGZoZQhlyGVIY+hBMjOYUhkKAbCaAZDBgOGAqBYLEM1UKwIyMoEy6cy1DJwAfWWAlWlAlUkAkWzgWQ6kBcNFc0D8kFmFoN1JwNtyQHiIqBOBQZVg6sGKw0+G5wwWG3w0uAPTrOqwWaA3FIJpJMgelML4vm7JIK/E9SVC6RLGDIQuvC6uYQhjcEC7NZMoNsLwCIgXyRD9JdVTf8cbBWkWq1msMjgNdD9Cw1uGhwG+iCv7Evy0sDUoNkMXMAIMEQPbkxGmJGeobGeUaCJsoMTNCo4GKQZlBg0gOFtzuDA4MEQwBAKtHchw3GGCwwXmZSYPJj8mQIhSpkYoXqEGVAAUxQAVBydHg==</latexit> S(x) <latexit sha1_base64="Njtyiqqac/Zr6fC2abFKXMsgtIQ=">AAACZ3ichVHLSgMxFD0d3/XRqiAFN9Wi1E1JVVBciW5c9mFVUCkzY2xDpzPDTFrU4g+4cFvBlYKI+Blu/AEX/YTiUsGNC+9MB0RFvSHJyck9NyeJZhvClYy1QkpXd09vX/9AeHBoeCQSHR3bcq2ao/OCbhmWs6OpLjeEyQtSSIPv2A5Xq5rBt7XKure/XeeOKyxzUx7bfL+qlkxxKHRVelQ+eTRXjCZYivkR/wnSAUggiIwVvcUeDmBBRw1VcJiQhA2ocKntIg0Gm7h9NIhzCAl/n+MUYdLWKItThkpshcYSrXYD1qS1V9P11TqdYlB3SBnHDHtid+yFPbJ71mbvv9Zq+DU8L8c0ax0tt4uRs1j+7V9VlWaJ8qfqT88Sh1j2vQrybvuMdwu9o6+fNF/yK7mZxiy7Zs/k/4q12APdwKy/6jdZnrtEmD4g/f25f4Kt+VR6ITWfXUysrgVf0Y9JTCNJ772EVWwggwKdW8Y5mrgItZWIMqHEOqlKKNCM40soUx+xn4q+</latexit> : の部分構造の集合 x <latexit sha1_base64="kqksgV29kEVi+nuGsxDaY7DP4ng=">AAACZHichVFNSwJBGH7cvswsLQmCICQxOsloQdFJ6tLRj/wAE9ndxlpcd5fdVTLpD9S16NCpICL6GV36Ax38A0F0NOjSodd1IUqqd5iZZ555n3eemZEMVbFsxjoeYWh4ZHTMO+6b8E9OBYLTM3lLb5gyz8m6qptFSbS4qmg8Zyu2youGycW6pPKCVNvq7Rea3LQUXduxWwYv18V9TakqsmgTlT6sBCMsxpwID4K4CyJwI6UHb7GLPeiQ0UAdHBpswipEWNRKiIPBIK6MNnEmIcXZ5ziGj7QNyuKUIRJbo3GfViWX1Wjdq2k5aplOUambpAwjyp7YHeuyR3bPXtjHr7XaTo2elxbNUl/LjUrgZC77/q+qTrONgy/Vn55tVLHueFXIu+EwvVvIfX3z6KKb3chE20vsmr2S/yvWYQ90A635Jt+keeYSPvqA+M/nHgT5RCy+EkukVyPJTfcrvJjHIpbpvdeQxDZSyNG5HKc4w7nnWfALIWG2nyp4XE0I30JY+AT3mon8</latexit> k0 S (x0, y0) <latexit sha1_base64="TXE7w1OuflYO7LLMYWQVqenADvg=">AAACb3ichVHLSsNAFD2Nr1pfVRcKghRLbQUpUxUUV0U3LltrH9CWksSphqZJSNJiLf6AH6ALFz5ARPwMN/6ACz9BXEkFNy68TQOiRb3DzJw5c8+dMzOSoSqWzdiTR+jp7esf8A76hoZHRsf84xMZS6+ZMk/LuqqbOUm0uKpoPG0rtspzhsnFqqTyrFTZbO9n69y0FF3bsRsGL1bFPU0pK7JoE1WohEupyEF4MdAIL5T8QRZlTgS6QcwFQbiR0P03KGAXOmTUUAWHBpuwChEWtTxiYDCIK6JJnElIcfY5juAjbY2yOGWIxFZo3KNV3mU1WrdrWo5aplNU6iYpAwixR3bLWuyB3bFn9vFrraZTo+2lQbPU0XKjNHY8nXr/V1Wl2cb+l+pPzzbKWHO8KuTdcJj2LeSOvn542kqtb4ea8+yKvZD/S/bE7ukGWv1Nvk7y7TP46ANiP5+7G2SWorHl6FJyJRjfcL/CixnMIULvvYo4tpBAms41cIJzXHhehSlhVgh0UgWPq5nEtxAWPgH2c40S</latexit> : 部分構造間のカーネル関数 7
8. カーネル法の問題点: データ数についてスケールしない • カーネル法は⾼次元の特徴空間での計算を, データ点の全ての ペアのカーネル関数の計算で⾼次元の影響を回避 • 主にグラム⾏列を利⽤: (データ数×データ数)サイズの⾏列 • データ数について の計算量 © 2019, Retrieva, Inc. All rights reserved. G = 0 B @ k(x1, x1) · · · k(x1, xn) . . . ... . . . k(xn, x1) · · · k(xn, xn) 1 C A <latexit sha1_base64="OzQ7WjzNjb4pADUjUXI/zVmiths=">AAAC9HichVHLSuRAFL2JOmrUsdWN0JswbYvC0FRUUASh0YUufbUKRpokXbZFJ5WQVAe16R9w49KFbmakERH8CTf+wCz8BHGp4saFN+kwzoyPuSFVp06dc++tKtOzWSAIuZHklta2L+0dnUpXd8/X3lRf/1rgVn2LFizXdv0N0wiozTgtCCZsuuH51HBMm66blblofz2kfsBcvir2PLrlGGXOtpllCKSKqcN5dUbRTVpmvOY5hvDZbl2pjOwWte8qDqPqsKpbJVcECH7TfFTVdUUPYz4OVJUSVUKjINLz99PwP9JQXnotXUxlSI7Eob4FWgIykMSimzoDHUrgggVVcIACB4HYBgMC/DZBAwIecltQQ85HxOJ9CnVQ0FtFFUWFgWwFxzKuNhOW4zrKGcRuC6vY+PvoVCFLfpFzck+uyQW5Jc8f5qrFOaJe9nA2m17qFXsPBlee/utycBaw8+r6tGcB2zAV98qwdy9molNYTX+4f3S/Mr2crQ2Tn+QO+/9BbsgVnoCHD1ZjiS4fQ/QA2r/X/RasjeW08dzY0kQmP5s8RQek4RuM4H1PQh4WYBEKWPdRSktDUlYO5RP5VG40pbKUeAbgr5AvXwCLPbd3</latexit> O(n3 ) <latexit sha1_base64="YNTJzUMjvTBOceWyI/OZ02aLKLI=">AAACeXichVFNLwNBGH66vuurPg4Sl9KQcmimKiFOwsVNiyJRmt01atL9yu60CZv+AX/AwQWJCH6Giz/g4CeII4kDB2+3mwiCd7Mzz/vM+7zzzIzmGMKTjD1ElKbmlta29o5oZ1d3T2+sr3/dsyuuzvO6bdjupqZ63BAWz0shDb7puFw1NYNvaOXF+vpGlbuesK01eeDwbVMtWWJP6KokqhgbLLimv1xLFoT0rRol8Z3MRDGWYCkWRPwnSIcggTCyduwSBezCho4KTHBYkIQNqPDo20IaDA5x2/CJcwmJYJ2jhihpK1TFqUIltkxjibKtkLUor/f0ArVOuxj0u6SMY4zdsyv2zO7YDXtk77/28oMedS8HNGsNLXeKvUdDq6//qkyaJfY/VX96ltjDbOBVkHcnYOqn0Bv66uHx8+rcypg/zs7ZE/k/Yw/slk5gVV/0ixxfOUGUHiD9/bp/gvWpVDqTmspNJ+YXwqdoxzBGkaT7nsE8lpBFnvY9xCmucB15U0aUpDLZKFUioWYAX0LJfABepJHR</latexit> 8
9. カーネル法の⾼速化⼿法: Nyström近似とRandom Features • 厳密計算は諦めて近似的に⾼速計算することを考える 1. Nyström近似 • データ点の⼀部をサンプリング + グラム⾏列を低ランク近似 • データ依存の⼿法 2. Random Features • 内積がカーネル関数の値と⼀致するようにランダム射影によって特徴 ベクトルを作成 • データ⾮依存の⼿法 © 2019, Retrieva, Inc. All rights reserved. 9
10. Random Featuresの概要: ランダム射影によって線形学習で近似的な⾮線形学習⾏う • ランダム射影によって内積の期待値がカーネル関数値と⼀致す るようなベクトル表現を得るのが⽬標 • は直接扱うことができないほど⾼次元 or そもそもわからない • 直接扱うことができる程度の次元のベクトル で近似したい • が得られれば線形の学習でカーネル関数を⽤いた⾮線形学習が近 似できる © 2019, Retrieva, Inc. All rights reserved. k(x, y) = (x)> (y) ⇡ E[s(x)>s(y)] <latexit sha1_base64="xtoJ3skNrwXkRuwO/522g16Bb3Q=">AAACm3icSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQuYZWtU6FRqKtgqxARkZGpUaMbFlOQXQDiVmjGJBQVF+RUKrtHFcKlioHhsvICygZ4BGChgMgyhDGUGKAjIF1jOEMOQwpDPkMxQypDLkMqQx1ACZOcwJDIUA2E0gyGDAUMBUCyWoRooVgRkZYLlUxlqGbiAekuBqlKBKhKBotlAMh3Ii4aK5gH5IDOLwbqTgbbkAHERUKcCg6rBVYOVBp8NThisNnhp8AenWdVgM0BuqQTSSRC9qQXx/F0Swd8J6soF0iUMGQhdeN1cwpDGYAF2aybQ7QVgEZAvkiH6y6qmfw62ClKtVjNYZPAa6P6FBjcNDgN9kFf2JXlpYGrQbAYuYAQYogc3JiPMSM/QWM8o0ETZwQkaFRwM0gxKDBrA8DZncGDwYAhgCAXaO4fhEMNphjNMskzOTF5MPhClTIxQPcIMKIApFACTkZyR</latexit> (x) <latexit sha1_base64="o3tqQ+fDtwJnqUkSayN3upeCoTY=">AAACanichVFNSwJBGH7cvsw+NLsUXSQr7CKjBUWnqEtHrUxBQ3a3SQfX3WV3lUz6A906BXkqiIh+Rpf+QAd/QtStoEuHXteFKKneYWaeeeZ93nlmRjE1YTuMtX1SX//A4JB/ODAyOjYeDE2E92yjZqk8oxqaYeUU2eaa0HnGEY7Gc6bF5aqi8axS2ezsZ+vcsoWh7zoNk+9X5ZIuDoUqO0RlC6myiB0tFkNRFmduRHpBwgNReJEyQjco4AAGVNRQBYcOh7AGGTa1PBJgMInbR5M4i5Bw9zlOECBtjbI4ZcjEVmgs0SrvsTqtOzVtV63SKRp1i5QRzLNHdste2QO7Y0/s49daTbdGx0uDZqWr5WYxeDq18/6vqkqzg/KX6k/PDg6x6noV5N10mc4t1K6+fnz+urO2Pd9cYFfsmfxfsja7pxvo9Tf1Os23WwjQByR+Pncv2EvGE0vxZHo5ur7hfYUfM5hFjN57BevYQgoZ190ZLtDyvUhhaVqa6aZKPk8ziW8hzX0Ce56MBg==</latexit> s(x) <latexit sha1_base64="HRdJng2dsKIkIS7wnhng6/AEgp8=">AAACZ3ichVHLSgMxFD0dX7U+WhWk4EYtFd2UVAXFlejGZau2CrXIzBjb4HRmmEmLtfgDLtxWcKUgIn6GG3/AhZ8gLiu4ceGd6YBoUW9IcnJyz81JotmGcCVjzyGlq7unty/cHxkYHBqOxkZG865VdXSe0y3DcnY11eWGMHlOCmnwXdvhakUz+I52tO7t79S44wrL3JZ1mxcraskUh0JXpUe5s8dz+7EESzE/JjtBOgAJBJGxYrfYwwEs6KiiAg4TkrABFS61AtJgsIkrokGcQ0j4+xyniJC2SlmcMlRij2gs0aoQsCatvZqur9bpFIO6Q8pJJNkTu2Mt9sju2Qv7+LVWw6/heanTrLW13N6PnsW33v9VVWiWKH+p/vQscYhl36sg77bPeLfQ2/raSbO1tbKZbMywa/ZK/q/YM3ugG5i1N/0myzcvEaEPSP987k6Qn0+lF1Lz2cXE6lrwFWFMYBqz9N5LWMUGMsjRuWWco4mL0IsSVcaVeDtVCQWaMXwLZeoT8f+K3g==</latexit> s(x) <latexit sha1_base64="HRdJng2dsKIkIS7wnhng6/AEgp8=">AAACZ3ichVHLSgMxFD0dX7U+WhWk4EYtFd2UVAXFlejGZau2CrXIzBjb4HRmmEmLtfgDLtxWcKUgIn6GG3/AhZ8gLiu4ceGd6YBoUW9IcnJyz81JotmGcCVjzyGlq7unty/cHxkYHBqOxkZG865VdXSe0y3DcnY11eWGMHlOCmnwXdvhakUz+I52tO7t79S44wrL3JZ1mxcraskUh0JXpUe5s8dz+7EESzE/JjtBOgAJBJGxYrfYwwEs6KiiAg4TkrABFS61AtJgsIkrokGcQ0j4+xyniJC2SlmcMlRij2gs0aoQsCatvZqur9bpFIO6Q8pJJNkTu2Mt9sju2Qv7+LVWw6/heanTrLW13N6PnsW33v9VVWiWKH+p/vQscYhl36sg77bPeLfQ2/raSbO1tbKZbMywa/ZK/q/YM3ugG5i1N/0myzcvEaEPSP987k6Qn0+lF1Lz2cXE6lrwFWFMYBqz9N5LWMUGMsjRuWWco4mL0IsSVcaVeDtVCQWaMXwLZeoT8f+K3g==</latexit> 10
11. Bochnerの定理: 正定値関数と確率測度がフーリエ変換で⼀対⼀対応 • Bochnerの定理: 上の正規化された連続関数 が正定値で あるための必要⼗分条件は 上の確率測度 が存在し以下 が成⽴すること • 正定値関数 と確率測度 は(逆)フーリエ変換として⼀対⼀に 対応 © 2019, Retrieva, Inc. All rights reserved. Rm <latexit sha1_base64="aO6tMwkowq/r7pDRs+lgYQWfjJU=">AAACcHichVFNLwNBGH66vqq+iksTB6Uh4tDMloQ4NVwcKUVCye4abLpfdqdNauMP+AM9uKhERPwMF3/AoT9B3JC4OHh3u4kgeCcz88wz7/POMzOqY+ieYKwZk9raOzq74t2Jnt6+/oHk4NCGZ1dcjRc127DdLVXxuKFbvCh0YfAtx+WKqRp8Uy0vBfubVe56um2ti5rDS6ZyaOkHuqYIoko7piKOVNUvnO6aib1khmVZGOmfQI5ABlGs2Mlr7GAfNjRUYILDgiBsQIFHbRsyGBziSvCJcwnp4T7HKRKkrVAWpwyF2DKNh7TajliL1kFNL1RrdIpB3SVlGhPsgd2wF3bPbtkje/+1lh/WCLzUaFZbWu7sDZyl1t7+VZk0Cxx9qv70LHCA+dCrTt6dkAluobX01ZP6y9pCYcKfZJfsifw3WJPd0Q2s6qt2tcoL5wg+QP7+3D/BRi4rz2Rzq7OZ/GL0FXGMYBxT9N5zyGMZKyjSuceo4wKN2LOUkkalsVaqFIs0w/gS0vQHuqmO5Q==</latexit> (z) <latexit sha1_base64="ZXqeKmzu8D4FZnkZGCMs92JjIVI=">AAACanichVG7SgNBFD1ZXzE+ErVRbMQYiU2YjYJiJdpY+koiqITddUyGbHaX3UlAgz9gZyVopSAifoaNP2DhJ4h2EWwsvLtZEA3qHWbmzJl77pyZ0R1TeJKxp4jS0dnV3RPtjfX1DwzGE0PDec+uuQbPGbZpu9u65nFTWDwnhTT5tuNyraqbvKBXVvz9Qp27nrCtLXno8L2qVrLEgTA0SVRh1ymL9NFMMZFkGRbERDtQQ5BEGGt24ga72IcNAzVUwWFBEjahwaO2AxUMDnF7aBDnEhLBPscxYqStURanDI3YCo0lWu2ErEVrv6YXqA06xaTuknICKfbIblmTPbA79sw+fq3VCGr4Xg5p1lta7hTjJ6Ob7/+qqjRLlL9Uf3qWOMBC4FWQdydg/FsYLX396Ky5ubiRakyzK/ZC/i/ZE7unG1j1N+N6nW9cIEYfoP587naQz2bU2Ux2fS65tBx+RRTjmESa3nseS1jFGnKBu1Oc4yLyqgwrY8p4K1WJhJoRfAtl6hPAQIwo</latexit> Rm <latexit sha1_base64="aO6tMwkowq/r7pDRs+lgYQWfjJU=">AAACcHichVFNLwNBGH66vqq+iksTB6Uh4tDMloQ4NVwcKUVCye4abLpfdqdNauMP+AM9uKhERPwMF3/AoT9B3JC4OHh3u4kgeCcz88wz7/POMzOqY+ieYKwZk9raOzq74t2Jnt6+/oHk4NCGZ1dcjRc127DdLVXxuKFbvCh0YfAtx+WKqRp8Uy0vBfubVe56um2ti5rDS6ZyaOkHuqYIoko7piKOVNUvnO6aib1khmVZGOmfQI5ABlGs2Mlr7GAfNjRUYILDgiBsQIFHbRsyGBziSvCJcwnp4T7HKRKkrVAWpwyF2DKNh7TajliL1kFNL1RrdIpB3SVlGhPsgd2wF3bPbtkje/+1lh/WCLzUaFZbWu7sDZyl1t7+VZk0Cxx9qv70LHCA+dCrTt6dkAluobX01ZP6y9pCYcKfZJfsifw3WJPd0Q2s6qt2tcoL5wg+QP7+3D/BRi4rz2Rzq7OZ/GL0FXGMYBxT9N5zyGMZKyjSuceo4wKN2LOUkkalsVaqFIs0w/gS0vQHuqmO5Q==</latexit> p(w) <latexit sha1_base64="VTQmHLOSWoIB8pRvlVI6Eo+hU/o=">AAACZ3ichVFNLwNBGH66vqo+WiTSxAVNhUszi4Q4CRfHFkWCyO4a7cR+ZXdaqcYfcHCtxIlERPwMF3/AwU8QRxIXB+9uNxEavJOZeeaZ93nnmRndNYUvGXuKKW3tHZ1d8e5ET29ffzI1MLjhOxXP4EXDMR1vS9d8bgqbF6WQJt9yPa5Zusk39cPlYH+zyj1fOPa6rLl819JKtjgQhiYDyp08mtpLZViOhTHaCtQIZBBF3kndYAf7cGCgAgscNiRhExp8attQweASt4s6cR4hEe5znCBB2gplccrQiD2ksUSr7Yi1aR3U9EO1QaeY1D1SjiLLHtkte2UP7I49s49fa9XDGoGXGs16U8vdveRpeu39X5VFs0T5S/WnZ4kDzIdeBXl3Qya4hdHUV48br2sLq9n6BLtiL+T/kj2xe7qBXX0zrgt89QIJ+gD153O3go3pnDqTmy7MZhaXoq+IYwTjmKT3nsMiVpBHkc4t4wwNnMeelaQyrKSbqUos0gzhWyhjn+n1ito=</latexit> (z) = Z eiz>w p(w)dw <latexit sha1_base64="IUbDb2NJcUUZYPSdxrJc4J02Q/E=">AAAClnichVFNSxtBGH7cVpumaqJeBC9LQyRewqwWWgSLWIoeozZRcDXsrqMZsh/D7iTBLPkD/QMeemqhlOJv6KlQ/QM95CeUHlPopYe+2SyISuu77MzzPvM+7zwzY0tXRIqx/pj24OH4xKPM4+yTyanpXH5mthYFrdDhVSdwg3DftiLuCp9XlVAu35chtzzb5Xt289Vwfa/Nw0gE/ht1JvmhZ5364kQ4liKqnjdM2RCl7pK+ppvCV7oZejHvHcXCFCruHpkqkHqn16NMl6XOkn7cqecLrMyS0O8CIwUFpFEJ8p9g4hgBHLTggcOHIuzCQkTfAQwwSOIOERMXEhLJOkcPWdK2qIpThUVsk8ZTyg5S1qd82DNK1A7t4tIfklJHkX1nn9mAXbEL9oP9+WevOOkx9HJGsz3SclnPvZ3f/X2vyqNZoXGt+q9nhRO8SLwK8i4TZngKZ6Rvd88Hu6s7xXiRfWA/yf971mdf6QR++5fzcZvvvEOWHsC4fd13QW25bKyUl7efFdY30qfIYAFPUaL7fo51bKGCKu17ji/4hkttXnupvdY2R6XaWKqZw43QKn8B3PicdA==</latexit> (z) <latexit sha1_base64="ZXqeKmzu8D4FZnkZGCMs92JjIVI=">AAACanichVG7SgNBFD1ZXzE+ErVRbMQYiU2YjYJiJdpY+koiqITddUyGbHaX3UlAgz9gZyVopSAifoaNP2DhJ4h2EWwsvLtZEA3qHWbmzJl77pyZ0R1TeJKxp4jS0dnV3RPtjfX1DwzGE0PDec+uuQbPGbZpu9u65nFTWDwnhTT5tuNyraqbvKBXVvz9Qp27nrCtLXno8L2qVrLEgTA0SVRh1ymL9NFMMZFkGRbERDtQQ5BEGGt24ga72IcNAzVUwWFBEjahwaO2AxUMDnF7aBDnEhLBPscxYqStURanDI3YCo0lWu2ErEVrv6YXqA06xaTuknICKfbIblmTPbA79sw+fq3VCGr4Xg5p1lta7hTjJ6Ob7/+qqjRLlL9Uf3qWOMBC4FWQdydg/FsYLX396Ky5ubiRakyzK/ZC/i/ZE7unG1j1N+N6nW9cIEYfoP587naQz2bU2Ux2fS65tBx+RRTjmESa3nseS1jFGnKBu1Oc4yLyqgwrY8p4K1WJhJoRfAtl6hPAQIwo</latexit> p(w) <latexit sha1_base64="VTQmHLOSWoIB8pRvlVI6Eo+hU/o=">AAACZ3ichVFNLwNBGH66vqo+WiTSxAVNhUszi4Q4CRfHFkWCyO4a7cR+ZXdaqcYfcHCtxIlERPwMF3/AwU8QRxIXB+9uNxEavJOZeeaZ93nnmRndNYUvGXuKKW3tHZ1d8e5ET29ffzI1MLjhOxXP4EXDMR1vS9d8bgqbF6WQJt9yPa5Zusk39cPlYH+zyj1fOPa6rLl819JKtjgQhiYDyp08mtpLZViOhTHaCtQIZBBF3kndYAf7cGCgAgscNiRhExp8attQweASt4s6cR4hEe5znCBB2gplccrQiD2ksUSr7Yi1aR3U9EO1QaeY1D1SjiLLHtkte2UP7I49s49fa9XDGoGXGs16U8vdveRpeu39X5VFs0T5S/WnZ4kDzIdeBXl3Qya4hdHUV48br2sLq9n6BLtiL+T/kj2xe7qBXX0zrgt89QIJ+gD153O3go3pnDqTmy7MZhaXoq+IYwTjmKT3nsMiVpBHkc4t4wwNnMeelaQyrKSbqUos0gzhWyhjn+n1ito=</latexit> 11
12. Random Fourier Features [Rahimi & Recht, ʼ07]: Shift invariant kernelに対する近似⼿法 • Shift invariant kernel: © 2019, Retrieva, Inc. All rights reserved. k(x, y) = (x y) <latexit sha1_base64="kCncEYjEc4Vv+NXPunQURcF9l6o=">AAACeHichVHLSsNAFD2N7/po1Y3gJliqLWiZVkERBNGNS1+tgkpJ4rQOTZOQpMVY/AF/wIULURAVP8ONP+DCTxCXCoK48CYNiIp6h5k5c+aeO2dmVEsXjsvYQ0RqaW1r7+jsinb39PbF4v0DBces2RrPa6Zu2puq4nBdGDzvClfnm5bNlaqq8w21sujvb9S57QjTWHc9i+9UlbIhSkJTXKKK8cFKan/cS8tz8ra1J1L7E146WownWIYFIf8E2RAkEMayGb/ENnZhQkMNVXAYcAnrUOBQ20IWDBZxO2gQZxMSwT7HIaKkrVEWpwyF2AqNZVpthaxBa7+mE6g1OkWnbpNSRpLds2v2zO7YDXtk77/WagQ1fC8ezWpTy61i7Gho7fVfVZVmF3ufqj89uyhhJvAqyLsVMP4ttKa+fnD8vDa7mmyMsnP2RP7P2AO7pRsY9RftYoWvnsD/gOz35/4JCrlMdjKTW5lKzC+EX9GJYYwgRe89jXksYRl5OtfDKa5wHXmTZGlMSjdTpUioGcSXkHIfPDuP1Q==</latexit> k(x, y) = (x y) = Z ei(x y)>w p(w)dw <latexit sha1_base64="tct2vbyS8TP2vuHy/rXvdubbO/o=">AAACpnichVHLShxBFD22GnU0cdSNkE2RwdBCMtSYgCIIJtm4Eh8ZR7B10t2WWkw/iu6aGSfN/EB+IIusEhAJ+Qw3brJU8RNClgbcuMidnoaQiOY2XXXq3HtunapylCdjzfllj9Hb1/9gYHAoNzzy8NFofmx8Iw7rkSvKbuiF0aZjx8KTgShrqT2xqSJh+44nKk7tTSdfaYgolmHwVreU2Pbt/UDuSdfWRFXzr2rm4TPWmmYLzFIH0jx83sUy0MyK/ES0dxJpSZ2kmR1Lh4o1221imDKb02y3Wc0XeJGnwW6DUgYKyGIlzB/Dwi5CuKjDh0AATdiDjZi+LZTAoYjbRkJcREimeYE2cqStU5WgCpvYGo37tNrK2IDWnZ5xqnZpF4/+iJQMU/yMf+VX/JR/4z/4zZ29krRHx0uLZqerFao6+mFy/fq/Kp9mjYM/qns9a+xhLvUqybtKmc4p3K6+8f7j1fr82lTylH/hP8n/Z37JT+gEQeOXe7Qq1j4hRw9Q+ve6b4ONmWLpRXFm9WVh8XX2FIN4jCcw6b5nsYglrKBM+x7jO85xYZjGslE2Kt1SoyfTTOCvMN79BlBfoSM=</latexit> = Z p(w)eix>w · e iy>w dw <latexit sha1_base64="eI9Ni4bRWfDigAv+yAiSk9xGl9U=">AAACrHichVHLSiNBFD22z4mjRt0MuGkMDrowVFRwGBiQceEsfUWFdAzdnUos7BfdlWRikx+YH3AxqxEGGQa/wo17cSHoB4hLBTcuvOm0iop6i6q695x7bt2qMjxLBJKx0zalvaOzq7vnQ6L3Y1//QHJwaC1wK77Js6Zruf6GoQfcEg7PSiEtvuH5XLcNi68b2/NNfr3K/UC4zqqsezxv62VHlISpS4IKyYVvqiYcqXrjtQnNt0Pe2AyFJmT4c1OTrqfWGg1VM4uuVO/ZyYiuP9AUqcVaIZliaRaZ+tLJxE4KsS26yX1oKMKFiQpscDiQ5FvQEdDIIQMGj7A8QsJ88kTEczSQIG2Fsjhl6IRu01qmKBejDsXNmkGkNukUi6ZPShVj7IT9Y1fsiP1nF+z21VphVKPZS512o6XlXmHg16eVm3dVNu0SW4+qN3uWKOFL1Kug3r0Iad7CbOmrO7tXK1+Xx8LPbI9dUv9/2Ck7pBs41Wvz7xJf/o0EfUDm+XO/dNam0pnp9NTSTGrue/wVPRjBKMbpvWcxhx9YRJbOPcAxznCupJVVJafkW6lKW6wZxhNTSnf6zqa+</latexit> = Ew[⇠w(x)⇠w(y)] <latexit sha1_base64="ug4EAOTUT7RsDY6RV6LbeMUuR2g=">AAAChnichVHLSsNAFD3GV62vqhvBTbUouilTHyiCUBTBZVutCm0JSRx1ME1Ckr4srgV/wIUrBRERt/oBbvwBF36CuFRw48KbNCAq6h1m5syZe+6cmVEtXTguY49NUnNLa1t7qCPc2dXd0xvp6193zJKt8axm6qa9qSoO14XBs65wdb5p2VwpqjrfUPeWvP2NMrcdYRprbs3ihaKyY4htoSkuUXJkeGFZruTyVSFXxqsTeZNyvVL1BlObOCjIkRiLMz+iP0EiADEEkTIjF8hjCyY0lFAEhwGXsA4FDrUcEmCwiCugTpxNSPj7HAcIk7ZEWZwyFGL3aNyhVS5gDVp7NR1frdEpOnWblFGMsgd2yV7YPbtiT+z911p1v4bnpUaz2tByS+49Glx9+1dVpNnF7qfqT88utjHnexXk3fIZ7xZaQ1/eP35Znc+M1sfYGXsm/6fskd3RDYzyq3ae5pkThOkDEt+f+ydYn4wnpuKT6elYcjH4ihCGMIJxeu9ZJLGCFLJ07iGucYNbKSTFpRlptpEqNQWaAXwJKfkB4lOWdw==</latexit> (Bochnerの定理) (⇠w(x) = eix>w ) <latexit sha1_base64="lExsZnBDCC/FD37Q9kYKW2Zn1o8=">AAACiXichVG7SiRBFD2273HVURPBpHFWGZOhWgUHQRBNDH2NCrYO3W2NFvaL7pqHNvMD/oCBkYKImJi66Sb7Axv4CWKoYGLgnZ4G2RX1FlV16tQ9t05Vmb4tQsnYfYvS2tbe0dnVner50dvXnx4Y3Ai9cmDxguXZXrBlGiG3hcsLUkibb/kBNxzT5pvm4WJjf7PCg1B47ro88vmOY+y7oiQsQxJVTP/M6jVRrGZrE+qcqgdOxOu7kdCFjGr1XV16vlqtTxTTGZZjcagfgZaADJJY9tJX0LEHDxbKcMDhQhK2YSCktg0NDD5xO4iICwiJeJ+jjhRpy5TFKcMg9pDGfVptJ6xL60bNMFZbdIpNPSClijH2l12zJ/aH3bAH9vpprSiu0fByRLPZ1HK/2H8yvPbyrcqhWeLgXfWlZ4kS8rFXQd79mGncwmrqK8enT2uzq2PROLtgj+T/nN2z33QDt/JsXa7w1TOk6AO0/5/7I9iYzGlTucmV6cz8QvIVXRjBKLL03jOYxxKWUaBzT3CLO/xSehRNySuzzVSlJdEM4Z9QFt8AB66XYg==</latexit> 12
13. Random Fourier Features [Rahimi & Recht, ʼ07]: 期待値の実部をモンテカルロ積分で近似計算 © 2019, Retrieva, Inc. All rights reserved. w1, ..., wD i.i.d. ⇠ p(w) <latexit sha1_base64="C2DCyoAXUNKaPrHKCKwjtIp7CDA=">AAACk3icSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvolccb6ujp6emUx7soxOSDlKaWVMeUpFaUVGfqZeql6NXWVscUZ+bWFmiUa3LFCygb6BmAgQImwxDKUGaAgoB8geUMMQwpDPkMyQylDLkMqQx5DCVAdg5DIkMxEEYzGDIYMBQAxWIZqoFiRUBWJlg+laGWgQuotxSoKhWoIhEomg0k04G8aKhoHpAPMrMYrDsZaEsOEBcBdSowqBpcNVhp8NnghMFqg5cGf3CaVQ02A+SWSiCdBNGbWhDP3yUR/J2grlwgXcKQgdCF180lDGkMFmC3ZgLdXgAWAfkiGaK/rGr652CrINVqNYNFBq+B7l9ocNPgMNAHeWVfkpcGpgbNZgBFgCF6cGMywoz0DI31jAJNlB2coFHBwSDNoMSgAQxvcwYHBg+GAIZQoL1TGHYxHGY4wiTKZM3kxOQCUcrECNUjzIACmHwB1y6arw==</latexit> s(x) = 1 p D 0 B B B B B @ cos(w> 1 x) sin(w> 1 x) . . . cos(w> D x) sin(w> D x) 1 C C C C C A <latexit sha1_base64="cUCiYl990kP2UI+zCUsZIQ/FXJE=">AAAC+3ichVHNThRBEK4ZUHH9YdGLCZcJG8wSzKYHSTQkJEQ5eOTHBRIGNzO9vUuHme6xu3dZnMwL+AB68ISEEH+uPoEXX8ADj2A8YuKFgzWzY4gsanW6+6uv66uq7g7ikGtDyLFlDw1funxl5Grp2vUbN0fLY7fWtOwoyupUhlJtBL5mIResbrgJ2UasmB8FIVsPdh5n5+tdpjSX4qnZi9lW5LcFb3HqG6Qa5Ve62puad7yW8mnipomnnyuTLKZpyQtYm4skjnyjeA99KnV1t+E+Szwj47Q35XheydNcDJLdpjQakfNbtHiR6BzJRPOsWKNcITWSmzMI3AJUoLAlWT4CD5oggUIHImAgwCAOwQeNYxNcIBAjtwUJcgoRz88ZpFBCbQejGEb4yO7g2kZvs2AF+llOnaspVglxKlQ6MEm+knfkhHwhH8g3cvrXXEmeI+tlD/egr2VxY/TlndWf/1VFuBvYPlP9s2cDLXiY98qx9zhnslvQvr774vXJ6tzKZHKXvCXfsf99ckw+4w1E9wc9XGYrbyD7APf8cw+CtZmae782szxbWXhUfMUIjMMEVPG9H8ACPIElqGPdU2vCmrbu2al9YL+3P/ZDbavQ3IY/zP70C3hywck=</latexit> k(x, y) ⇡ 1 D D X i=1 ⇠wi (x)⇠wi (y) <latexit sha1_base64="mVQ/pBCig46Ip/7AZKksdbDDfkk=">AAACp3ichVFNSxtBGH5cbbXpR6K9FHpZDJYESpi1giIIoj148zMxYOwyu07skP1idxOTLvsH/AM99FSh2OLP6KUnb6L+hOJRoZce+u5maVHRvsvOPO8z7/POMzOGZ8kgZOx8QBkcevBweORR7vGTp8/yhdGxWuC2fVNUTddy/brBA2FJR1RDGVqi7vmC24YlNo3WYrK+2RF+IF1nI+x5Ytvmu45sSpOHROmFhVap+1rtldUG9zzf7aqNps/NSIujt3EjaNt6JOe0+F2SdaUe7ekyLnWp+m/SK+uFIquwNNTbQMtAEVmsuIVDNLADFybasCHgICRsgSOgbwsaGDzithER5xOS6bpAjBxp21QlqIIT26Jxl7KtjHUoT3oGqdqkXSz6fVKqmGAn7Bu7ZD/YEfvJft/ZK0p7JF56NBt9rfD0/P6L9V//Vdk0h3j/T3Wv5xBNzKReJXn3UiY5hdnXdz58vFyfXZuIXrEDdkH+P7Nz9p1O4HSuzC+rYu0TcvQA2s3rvg1qkxXtTWVydao4v5A9xQheYhwluu9pzGMJK6jSvl9xjFOcKWVlWakp9X6pMpBpnuNaKPwPwwmjng==</latexit> = 1 D D X i=1 ✓ cos(w> i x) sin(w> i x) ◆>✓ cos(w> i y) sin(w> i y) ◆ <latexit sha1_base64="dij1WMXeFSUokvWo1fAqAfD8gqE=">AAAC33ichVHPS9xAFH5J/dWtdVO9FLwsXSx6WSZWUAqCaA8e/dFVwWhIxtntYJIJmdm165Cz4EF6k9KThSKlf0Yv/Qc8ePNaerTgxUNfsoFi17YvZOZ737zvzTczfhxwqQi5NMwHff0Dg0MPS4+GH4+UrSejG1K0EsrqVAQi2fI9yQIesbriKmBbccK80A/Ypr+/lK1vtlkiuYheq07MdkKvGfEGp55CyrUO5ytOI/GotlP9KnVkK3Q1n7fT3SzzeSRC7VAhJw9cvqsdJeL07VSqHcmju1SB7pd0eiVIuVaV1EgelV5gF6AKRawI6xwc2AMBFFoQAoMIFOIAPJD4bYMNBGLkdkAjlyDi+TqDFEqobWEVwwoP2X0cm5htF2yEedZT5mqKuwT4J6iswAS5IJ/JNflGvpDv5PavvXTeI/PSwdnvalnslo+frt/8VxXirODNb9U/PStowFzulaP3OGeyU9Cuvn14er3+cm1CPycfyQ/0f0YuyVc8QdT+ST+tsrUPUMIHsP+87l6wMV2zX9SmV2eqC4vFUwzBODyDSbzvWViAZViBOu57ZQwYZcMyPfPIPDHfdUtNo9CMwZ0w3/8CtO67MQ==</latexit> = s(x)>s(y) <latexit sha1_base64="WbCGTMRmuP0aa72hKFph5WsH/Ek=">AAACdXichVFNLwNBGH66vqq+FheJSBrVpi41LQmRSISLI6VIimZ3TWtju7vZnTaq8Qf8AQcuJIj4GS7+gEN/gjiSuDh4d7uJ0OCdzMwzz7zPO8/MqLahu4KxRkhqa+/o7Ap3R3p6+/oH5MGhTdeqOBrPaZZhOduq4nJDN3lO6MLg27bDlbJq8C31cNnb36pyx9Utc0PUbL5bVkqmXtQ1RRBVkOUFN3k0ubcjLDvqJmuTkYIcYynmR7QVpAMQQxCrlnyLHezDgoYKyuAwIQgbUOBSyyMNBpu4XdSJcwjp/j7HCSKkrVAWpwyF2EMaS7TKB6xJa6+m66s1OsWg7pAyijh7YnfslT2ye/bMPn6tVfdreF5qNKtNLbcLA6cj6+//qso0Cxx8qf70LFDEnO9VJ++2z3i30Jr66vHZ6/p8Nl5PsCv2Qv4vWYM90A3M6pt2vcaz5/A+IP3zuVvBZiaVnk5l1mZii0vBV4QxinEk6b1nsYgVrCJH51ZxgWvchN6lMWlCSjRTpVCgGca3kKY+AcIQjzg=</latexit> (モンテカルロ積分で近似) (実部のみを考える) (これを特徴ベクトルとする) 13
14. Random Fourier Featuresの理論解析[Rahimi & Recht, ʼ07]: ⼀様ノルムでワーストケースを評価 1. がコンパクトかつ直径が 2. : の⼆次モーメント © 2019, Retrieva, Inc. All rights reserved. ⇢ Rm <latexit sha1_base64="lZd63HWeWX8I7STRC3NCuc8GXkk=">AAACf3ichVG7ThtBFD1e8nCchw00EWmsWCSprGsTiUdlQZPSj/gh2cTa3Qx45H1pd2zJWJao+QEKKpAIQhTwDzT8QAo+IUpJpDQpcne9UpSgwB3NzJkz99w5M2N4lgwU0XVCm3nw8NHj5JPU02fPX6Qzs3ONwB34pqibruX6LUMPhCUdUVdSWaLl+UK3DUs0jf5GuN8cCj+QrvNRjTyxaevbjtySpq6Y6mYWOmZPZjvBwAiEynZsXfUMY1ydfLJT3UyO8hRF9jYoxCCHOMpu5gQdfIYLEwPYEHCgGFvQEXBrowCCx9wmxsz5jGS0LzBBirUDzhKcoTPb53GbV+2YdXgd1gwitcmnWNx9VmaxSF/plG7ois7oG/36b61xVCP0MuLZmGqF103vvaz9vFdl86zQ+6O607PCFlYir5K9exET3sKc6oc7+ze1teri+A0d0Xf2f0jXdMk3cIY/zOOKqB4g/IDCv899GzSK+cJSvlh5nyutx1+RxCu8xjt+72WU8AFl1PncXXzBOS60hPZWy2s0TdUSsWYef4W2+hvjEZNb</latexit> l <latexit sha1_base64="eXd03/4xmuDAAkCU1NgLH7gDsgs=">AAACZHichVFNSwJBGH7cvswsLQmCICQxOsloQdFJ6tLRj/wAE9ndRltcd5fdVTDpD9S16NCpICL6GV36Ax38A0F0NOjSodd1IUqqd5iZZ555n3eemZEMVbFsxroeYWR0bHzCO+mb8k/PBIKzc3lLb5oyz8m6qptFSbS4qmg8Zyu2youGycWGpPKCVN/p7xda3LQUXduz2wYvN8SaplQVWbSJSquVYITFmBPhYRB3QQRupPTgLfZxAB0ymmiAQ4NNWIUIi1oJcTAYxJXRIc4kpDj7HMfwkbZJWZwyRGLrNNZoVXJZjdb9mpajlukUlbpJyjCi7IndsR57ZPfshX38Wqvj1Oh7adMsDbTcqAROFrLv/6oaNNs4/FL96dlGFZuOV4W8Gw7Tv4U80LeOLnrZrUy0s8Ku2Sv5v2Jd9kA30Fpv8k2aZy7how+I/3zuYZBPxOJrsUR6PZLcdr/Ci0UsY5XeewNJ7CKFHJ3LcYoznHueBb8QEuYHqYLH1YTwLYSlT9+aifA=</latexit> 2 p = E[w>w] <latexit sha1_base64="ipPTgZN0syjtlt6eM56LPISxJUo=">AAACenichVHLSgMxFD0d3/VVFUFwUywVRShpLSiCIIrg0lcf0NYyM8YaOi9m0ooWf8AfcOFKQUT9DDf+gAs/QVwquHHh7XRAVNQbkpyc3HNzkmiOITzJ2GNIaWvv6Ozq7gn39vUPDEaGhrOeXXN1ntFtw3bzmupxQ1g8I4U0eN5xuWpqBs9p1ZXmfq7OXU/Y1rY8dHjJVCuW2BO6KokqR0aLnqiYatnZSS2uFg6K0naiB6VyJMYSzI/oT5AMQAxBrNuRKxSxCxs6ajDBYUESNqDCo1ZAEgwOcSU0iHMJCX+f4xhh0tYoi1OGSmyVxgqtCgFr0bpZ0/PVOp1iUHdJGUWcPbBr9sLu2S17Yu+/1mr4NZpeDmnWWlrulAdPxrbe/lWZNEvsf6r+9Cyxh3nfqyDvjs80b6G39PWj05ethc14Y5JdsGfyf84e2R3dwKq/6pcbfPMMYfqA5Pfn/gmyqURyNpHaSMeWloOv6MY4JjBF7z2HJaxhHRk6t4Fz3OA29K5MKNPKTCtVCQWaEXwJJf0BUh6SQA==</latexit> p(w) <latexit sha1_base64="VTQmHLOSWoIB8pRvlVI6Eo+hU/o=">AAACZ3ichVFNLwNBGH66vqo+WiTSxAVNhUszi4Q4CRfHFkWCyO4a7cR+ZXdaqcYfcHCtxIlERPwMF3/AwU8QRxIXB+9uNxEavJOZeeaZ93nnmRndNYUvGXuKKW3tHZ1d8e5ET29ffzI1MLjhOxXP4EXDMR1vS9d8bgqbF6WQJt9yPa5Zusk39cPlYH+zyj1fOPa6rLl819JKtjgQhiYDyp08mtpLZViOhTHaCtQIZBBF3kndYAf7cGCgAgscNiRhExp8attQweASt4s6cR4hEe5znCBB2gplccrQiD2ksUSr7Yi1aR3U9EO1QaeY1D1SjiLLHtkte2UP7I49s49fa9XDGoGXGs16U8vdveRpeu39X5VFs0T5S/WnZ4kDzIdeBXl3Qya4hdHUV48br2sLq9n6BLtiL+T/kj2xe7qBXX0zrgt89QIJ+gD153O3go3pnDqTmy7MZhaXoq+IYwTjmKT3nsMiVpBHkc4t4wwNnMeelaQyrKSbqUos0gzhWyhjn+n1ito=</latexit> Pr( sup x,y2 |s(x)>s(y) k(x, y)| ✏)  256( pl ✏ )2exp( D✏2 2(m + 2) ) <latexit sha1_base64="S5pYTsWKCVxURljSnkDEXQjPK5s=">AAADMXichVG7bhNBFL27vIJ5xECDRDPCCvIKYo3NU1QRUFA6CU4sZYK1uxk7I+9j2BlbNpP5AX6AggokhBAln0BDQUvhDtGhlEGioeDu2lEEEXBXu/feM+ecvTMTyEgoTenEcY8cPXb8xNzJ0qnTZ87Ol8+dX1PpIAt5K0yjNGsHvuKRSHhLCx3xtsy4HwcRXw/69/P19SHPlEiTR3os+Wbs9xLRFaGvEeqUP7EsNs3MVtkg2UIe14YJbUbXxpYwkRAWbgtr1EBahMmOqo68x4bpVFpVHXuL/SoyvR3CevwJYVwqEaWJR1iUt+jcuHkLrVHJupkfGqZEL/Y7RtrImn26zR1zrrV5JnwkC8niVPPggFjwyD6vUbDiq6RoPOvZTrlCa7QIcrioz4oKzKKZlt8Agy1IIYQBxMAhAY11BD4ofDagDhQkYptgEMuwEsU6Bwsl1A6QxZHhI9rHbw+7jRmaYJ97qkId4l8ifDNUElign+lbukc/0nf0G/35Vy9TeOSzjDEHUy2XnflnF1d//FcVY9awfaD658waunCnmFXg7LJA8l2EU/3w6fO91bsrC+YKfUV3cf6XdEI/4A6S4ffw9TJfeQElvID6n8d9uFhr1OrXa43lG5Wle7OrmINLcBmqeN63YQkeQhNaEDoNp+34TuC+dyfuF/frlOo6M80F+C3c3V+VQ9iS</latexit> カーネル関数値と近似後の最⼤の ズレがε以上となる確率 近似後の次元について 指数的に減少 14
15. Random Fourier Featuresのアルゴリズムまとめ: カーネル関数のフーリエ変換からサンプリングして特徴⽣成 • ⼊⼒: shift invariant kernel: データ集合 , 所望の特徴ベクトル次元 • 出⼒: 各データの特徴ベクトル 1. カーネル関数の(逆)フーリエ変換を計算 2. から 個のパラメタをサンプリング 3. 各データ について特徴ベクトルを得る © 2019, Retrieva, Inc. All rights reserved. w1, ..., wD i.i.d. ⇠ p(w) <latexit sha1_base64="C2DCyoAXUNKaPrHKCKwjtIp7CDA=">AAACk3icSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvolccb6ujp6emUx7soxOSDlKaWVMeUpFaUVGfqZeql6NXWVscUZ+bWFmiUa3LFCygb6BmAgQImwxDKUGaAgoB8geUMMQwpDPkMyQylDLkMqQx5DCVAdg5DIkMxEEYzGDIYMBQAxWIZqoFiRUBWJlg+laGWgQuotxSoKhWoIhEomg0k04G8aKhoHpAPMrMYrDsZaEsOEBcBdSowqBpcNVhp8NnghMFqg5cGf3CaVQ02A+SWSiCdBNGbWhDP3yUR/J2grlwgXcKQgdCF180lDGkMFmC3ZgLdXgAWAfkiGaK/rGr652CrINVqNYNFBq+B7l9ocNPgMNAHeWVfkpcGpgbNZgBFgCF6cGMywoz0DI31jAJNlB2coFHBwSDNoMSgAQxvcwYHBg+GAIZQoL1TGHYxHGY4wiTKZM3kxOQCUcrECNUjzIACmHwB1y6arw==</latexit> x1, ..., xn 2 Rm <latexit sha1_base64="TnOY48l35AEZxr3G+/nHHP+2eLk=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIV8Qb6ijo6enpKFTE5ynEZAJxbmJJRlJSdVBtXC5XvICygZ4BGChgMgyhDGUGKAjIF1jOEMOQwpDPkMxQypDLkMqQx1ACZOcwJDIUA2E0gyGDAUMBUCyWoRooVgRkZYLlUxlqGbiAekuBqlKBKhKBotlAMh3Ii4aK5gH5IDOLwbqTgbbkAHERUKcCg6rBVYOVBp8NThisNnhp8AenWdVgM0BuqQTSSRC9qQXx/F0Swd8J6soF0iUMGQhdeN1cwpDGYAF2aybQ7QVgEZAvkiH6y6qmfw62ClKtVjNYZPAa6P6FBjcNDgN9kFf2JXlpYGrQbAZQBBiiBzcmI8xIz9BYzyjQRNnBCRoVHAzSDEoMGsDwNmdwYPBgCGAIBdrbyrCaYQvDViY2Jh0mYyZTiFImRqgeYQYUwGQHAOwek60=</latexit> 2D <latexit sha1_base64="kiV/HxuMbsz5A+fbngaucw+BFlM=">AAACZXichVHLSsNAFD2Nr1qtrQ9EcGGxKK7KtAqKq6IuXPZhH1BLSeK0hqZJSNJCLf6AuFUXrhRExM9w4w+46BeIuKzgxoW3aUC0qHeYmTNn7rlzZkYyVMWyGWt7hIHBoeER76hvbNw/EQhOTmUtvW7KPCPrqm7mJdHiqqLxjK3YKs8bJhdrkspzUnW7u59rcNNSdG3Pbhq8WBMrmlJWZNEmKhXbKQXDLMKcCPWDqAvCcCOhB2+xjwPokFFHDRwabMIqRFjUCoiCwSCuiBZxJiHF2ec4ho+0dcrilCESW6WxQquCy2q07ta0HLVMp6jUTVKGsMSe2B3rsEd2z17Yx6+1Wk6NrpcmzVJPy41S4GQu/f6vqkazjcMv1Z+ebZSx4XhVyLvhMN1byD194+iik95MLbWW2TV7Jf9XrM0e6AZa402+SfLUJXz0AdGfz90PsrFIdDUSS66F41vuV3gxj0Ws0HuvI45dJJChc8s4xRnOPc+CX5gRZnupgsfVTONbCAufGTuKBA==</latexit> s(x1), ..., s(xn) 2 R2D <latexit sha1_base64="QQOl/GMSCQhsbBVE+1U0wHeCWJ0=">AAACinichVFNSxtBGH6y2mqjrVEvBS/BkKIgy2wU1HqRmoNHjSYKxobdddTB/WJ3ErRL/kD7A3rwpCAinry2Ry/+AQ/+BPEYwYsH390siIr6DjPzzDPv884zM4ZniUAydpVSOjo/fOzq/pTu6f38pS/TP1AJ3Lpv8rLpWq6/augBt4TDy1JIi696Ptdtw+Irxs5ctL/S4H4gXGdZ7nl83da3HLEpTF0SVcvkg5HdmjY6pqrqWASd0WxVONmqrcttwwhLzZ9hodisZXJMZXFkXwItATkkseBmjlHFBlyYqMMGhwNJ2IKOgNoaNDB4xK0jJM4nJOJ9jibSpK1TFqcMndgdGrdotZawDq2jmkGsNukUi7pPyizy7JKdsBa7YKfsmt2/WiuMa0Re9mg22lru1fp+f126e1dl0yyx/ah607PEJqZir4K8ezET3cJs6xu//raWvpfy4Td2yG7I/wG7Yud0A6dxax4t8tI+0vQB2vPnfgkqBVUbVwuLE7nZH8lXdGMIwxih957ELOaxgDKd+wdn+If/Sq9SUKaVmXaqkko0g3gSSvEBjKOWKA==</latexit> p(w) = 1 (2⇡)m Z e iw>z k(z)dz <latexit sha1_base64="ZWsXwK8r3/sZzWUBNb4Dc+DMYWA=">AAACpnichVFNT9RAGH6ooriIrHIx8dK4wXQPbKaricaEBPXiyfDhsiSUXdoyi5Ptx6SdXcI2/QP+AQ+eNDHE+DO8cOEohJ9gOGLihQNvu00IEOBtOvO8z7zPO8/MONITsWLscES7dXv0zt2xe6Xx+xMPJssPHy3HYS9yecMNvTBaceyYeyLgDSWUx1dkxG3f8XjT6b7L1pt9HsUiDD6qbcnXfHszEB3h2oqodvmNNLaq+qxudSLbTcw0MeqWFNWWn1oiULoV+QlPW8mMsIRKttKWpUKpD9Is6xqDaroxaJcrrMby0C8DswAVFDEflndgYQMhXPTggyOAIuzBRkzfKkwwSOLWkBAXERL5OkeKEml7VMWpwia2S+MmZasFG1Ce9YxztUu7ePRHpNQxzf6wn+yY7bJf7C87ubJXkvfIvGzT7Ay1XLYnPz9e+n+jyqdZ4dOZ6lrPCh28yr0K8i5zJjuFO9T3B1+Ol14vTifP2Hd2RP6/sUP2m04Q9P+5Pxb44leU6AHMi9d9GSzXa+bzWn3hRWXubfEUY3iCpzDovl9iDu8xjwbtu4M97ONAM7QPWkNrDku1kUIzhXOhrZ8CGE+jZg==</latexit> p(w) <latexit sha1_base64="VTQmHLOSWoIB8pRvlVI6Eo+hU/o=">AAACZ3ichVFNLwNBGH66vqo+WiTSxAVNhUszi4Q4CRfHFkWCyO4a7cR+ZXdaqcYfcHCtxIlERPwMF3/AwU8QRxIXB+9uNxEavJOZeeaZ93nnmRndNYUvGXuKKW3tHZ1d8e5ET29ffzI1MLjhOxXP4EXDMR1vS9d8bgqbF6WQJt9yPa5Zusk39cPlYH+zyj1fOPa6rLl819JKtjgQhiYDyp08mtpLZViOhTHaCtQIZBBF3kndYAf7cGCgAgscNiRhExp8attQweASt4s6cR4hEe5znCBB2gplccrQiD2ksUSr7Yi1aR3U9EO1QaeY1D1SjiLLHtkte2UP7I49s49fa9XDGoGXGs16U8vdveRpeu39X5VFs0T5S/WnZ4kDzIdeBXl3Qya4hdHUV48br2sLq9n6BLtiL+T/kj2xe7qBXX0zrgt89QIJ+gD153O3go3pnDqTmy7MZhaXoq+IYwTjmKT3nsMiVpBHkc4t4wwNnMeelaQyrKSbqUos0gzhWyhjn+n1ito=</latexit> D <latexit sha1_base64="8eSQWchmR3f/b2eGJyudXC45MVg=">AAACZHichVFNSwJBGH7cvswsLQmCICQxOsloQdFJqkNHP/IDTGR3m2xx3V12V8GkP1DXokOngojoZ3TpD3TwDwTR0aBLh17XhSip3mFmnnnmfd55ZkYyVMWyGet4hKHhkdEx77hvwj85FQhOz+QtvWHKPCfrqm4WJdHiqqLxnK3YKi8aJhfrksoLUm2rt19octNSdG3Xbhm8XBermnKgyKJNVHq7EoywGHMiPAjiLojAjZQevMUe9qFDRgN1cGiwCasQYVErIQ4Gg7gy2sSZhBRnn+MYPtI2KItThkhsjcYqrUouq9G6V9Ny1DKdolI3SRlGlD2xO9Zlj+yevbCPX2u1nRo9Ly2apb6WG5XAyVz2/V9VnWYbh1+qPz3bOMC641Uh74bD9G4h9/XNo4tudiMTbS+xa/ZK/q9Yhz3QDbTmm3yT5plL+OgD4j+fexDkE7H4SiyRXo0kN92v8GIei1im915DEjtIIUfncpziDOeeZ8EvhITZfqrgcTUhfAth4ROPmonI</latexit> xi <latexit sha1_base64="xSkKUs5rpr3tXin1AlN0bIQMSp0=">AAACZnichVFNLwNBGH66vou2iJC4NBri1LwtCXESLo60WhKk2V3TmnS7u9ndNmj8AYkrBycSEfEzXPwBh/4D4liJi4O3200EwTuZmWeeeZ93npnRbEO6HlEjpHR0dnX39PaF+wcGI9HY0HDetaqOLnK6ZVjOlqa6wpCmyHnSM8SW7Qi1ohliUyuvtPY3a8JxpWVueIe22K2oJVMWpa56TGUPCrIQS1CS/Ij/BKkAJBDEmhW7wQ72YEFHFRUImPAYG1DhcttGCgSbuV3UmXMYSX9f4Bhh1lY5S3CGymyZxxKvtgPW5HWrpuurdT7F4O6wMo4peqRbatID3dEzvf9aq+7XaHk55Flra4VdiJ6MZ9/+VVV49rD/qfrTs4ciFnyvkr3bPtO6hd7W147Om9nFzFR9mq7ohf1fUoPu+QZm7VW/XheZC4T5A1Lfn/snyKeTqdlken0usbQcfEUvJjCJGX7veSxhFWvI8bklnOIM56EnJaKMKmPtVCUUaEbwJZT4B9PJitg=</latexit> s(xi) = 1 p D (cos(w> 1 xi), sin(w> 1 xi), ..., cos(w> D xi), sin(w> D xi))> <latexit sha1_base64="Yd2D/7O4AMrmX70azRmEfsYTJQ4=">AAAC1HichVFNS+NAGH4bP7ertrqXBS/FolSQMFFBEQRxe9ijH9sqGA3JOK1D0yRmplU39rTsZQ979bCnFUTEn+HFP+DBnyAeK3jYPfgmDfhR1Ddk5pnnfZ533pmxPJsLSch1Quno7Oru6f2Q/NjXP5BKDw4VhVvzKStQ13b9dcsUzOYOK0gubbbu+cysWjZbsypfwvxanfmCu843eeCxzapZdniJU1MiZaRdkds3+Pi8XvJNGmiNQBe7vgzyjUZOp5jcM7QtXbpeJlRN6II7LyhVVSdiZb5d+YQaj6CRzhKVRJFpB1oMshDHkps+BR22wQUKNagCAwckYhtMEPhtgAYEPOQ2IUDOR8SjPIMGJNFbQxVDhYlsBccyrjZi1sF1WFNEboq72Pj76MzAKLkiZ6RJLsk5uSH/X60VRDXCXg5wtlpe5hmpX59X7991VXGWsPPoerNnCSWYjXrl2LsXMeEpaMtf/37UXJ1bGQ3GyDG5xf7/kmtygSdw6nf0ZJmt/IEkPoD28rrbQXFS1abUyeXp7MJi/BS9MAwjkMP7noEF+ApLUMB9r+BfojPRpRSVQ+WH8rMlVRKx5xM8C+X3AyL7siA=</latexit> k(x, y) = (x y) <latexit sha1_base64="kCncEYjEc4Vv+NXPunQURcF9l6o=">AAACeHichVHLSsNAFD2N7/po1Y3gJliqLWiZVkERBNGNS1+tgkpJ4rQOTZOQpMVY/AF/wIULURAVP8ONP+DCTxCXCoK48CYNiIp6h5k5c+aeO2dmVEsXjsvYQ0RqaW1r7+jsinb39PbF4v0DBces2RrPa6Zu2puq4nBdGDzvClfnm5bNlaqq8w21sujvb9S57QjTWHc9i+9UlbIhSkJTXKKK8cFKan/cS8tz8ra1J1L7E146WownWIYFIf8E2RAkEMayGb/ENnZhQkMNVXAYcAnrUOBQ20IWDBZxO2gQZxMSwT7HIaKkrVEWpwyF2AqNZVpthaxBa7+mE6g1OkWnbpNSRpLds2v2zO7YDXtk77/WagQ1fC8ezWpTy61i7Gho7fVfVZVmF3ufqj89uyhhJvAqyLsVMP4ttKa+fnD8vDa7mmyMsnP2RP7P2AO7pRsY9RftYoWvnsD/gOz35/4JCrlMdjKTW5lKzC+EX9GJYYwgRe89jXksYRl5OtfDKa5wHXmTZGlMSjdTpUioGcSXkHIfPDuP1Q==</latexit> 15
16. Random Fourier Features(RFF)の⾼速化と拡張: ⾼速化, 他カーネルに対する提案, エラー解析 • ガウスカーネルに対するRFFの⾼速化 • Fastfood[Le et al., ʼ13]:ガウス⾏列をアダマール⾏列などで近似+FFT的な⾼速計算 • Orthogonal Random Features[Felix et al., ʼ16]: ガウス⾏列を直交⾏列を⽤いて近似 • 多項式カーネルに対するRandom Features • Fast and Scalable polynomial kernels via explicit feature maps[Pham & Pagh, ʼ13] • Spherical Random Features for Polynomial Kernels[Pennington et al., ʼ16] • 距離から導出されるカーネル関数に対するRandom Features • Random Warping Series[Wu et al., ʼ18]: 時系列データカーネルに対するRF • Global String Kernel with RF [Wu et al., ʼ19a]: ⽂字列カーネルに対するRF • Global Alignment Graph Kernel Using RF [Wu et al., ʼ19b]: グラフカーネルに対するRF © 2019, Retrieva, Inc. All rights reserved. 16
17. 距離関数 • が距離関数 1. ⾮負性: 2. 3. 対称性: 4. 三⾓不等式: • ユークリッド距離, 編集距離, .. © 2019, Retrieva, Inc. All rights reserved. d : X ⇥ X ! R <latexit sha1_base64="43pnRp/gJ9qrz22tIlb6MUZ9lRI=">AAACjnichVE9SyNBGH5cPfXi1+o1B9cshog2YaKiIohyNpYmMTFgJOyuYzK4X8xOIhr8A9dcecVVdyCHWF97Fjb+AQt/glgq2Fj47mbhUFHfYWaeeeZ93nlmxgocESrGrrq07p4PvX39H1MDg0PDI/roWDn0m9LmJdt3fFmxzJA7wuMlJZTDK4Hkpms5fNPaW432N1tchsL3NtRBwLdds+6JXWGbiqiaPrWzaFSMqhIuDyPg+F5dinpDmVL6+0bVNVXDstqFo1RNT7Msi8N4CXIJSCOJdV//gyp24MNGEy44PCjCDkyE1LaQA0NA3DbaxElCIt7nOEKKtE3K4pRhErtHY51WWwnr0TqqGcZqm05xqEtSGsiwS3bCbtkFO2XX7OHVWu24RuTlgGaro+VBbeTb5+L9uyqXZoXGf9WbnhV2sRB7FeQ9iJnoFnZH3zr8cVtcLGTaE+w3uyH/v9gVO6cbeK07+zjPCz8RfUDu+XO/BOXpbG4mO52fTa98Tb6iH18wjkl673msYA3rKNG53/EX/3Cm6dqctqQtd1K1rkTzCU9CW3sEJrGZJQ==</latexit> d(x, y) 0 <latexit sha1_base64="cOOK0B74ULz7mW6UwLEacJxwwhI=">AAACcXichVHLSsNAFD2N7/po1Y3iJlgURSm3Kiiuim5c2mpV8EUSpzU0TWKSFmvxB/wBBVcVRMTPcOMPuPATxGUFNy68SQOiot5hZs6cuefOmRnVNnTXI3qKSC2tbe0dnV3R7p7evli8f2DDtcqOJnKaZVjOlqq4wtBNkfN0zxBbtiOUkmqITbW47O9vVoTj6pa57lVtsVtSCqae1zXFY2rvYOJ4ujop7xTEkUzR/XiCkhSE/BOkQpBAGKtW/AY7OIAFDWWUIGDCY2xAgcttGykQbOZ2UWPOYaQH+wKniLK2zFmCMxRmizwWeLUdsiav/ZpuoNb4FIO7w0oZY/RIt9SgB7qjZ3r/tVYtqOF7qfKsNrXC3o+dDa29/asq8ezh8FP1p2cPeSwEXnX2bgeMfwutqa+cnDfWFrNjtXG6ohf2X6cnuucbmJVX7TojspfwPyD1/bl/go2ZZGo2OZOZS6SXwq/oxAhGMcHvPY80VrCKHJ/r4AJ1XEUa0rAkS6PNVCkSagbxJaSpD9Rcjes=</latexit> d(x, y) = 0 , x = y <latexit sha1_base64="vT5D3H7oBc1wAUAU7PlA837GwTw=">AAAChnicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQsopmhU6FRqKtgqGCjE+KSmlRRlpmeUJBYV5ZcrVNhWcikoxAsoG+gZgIECJsMQylBmgIKAfIHlDDEMKQz5DMkMpQy5DKkMeQwlQHYOQyJDMRBGMxgyGDAUAMViGaqBYkVAViZYPpWhloELqLcUqCoVqCIRKJoNJNOBvGioaB6QDzKzGKw7GWhLDhAXAXUqMKgaXDVYafDZ4ITBaoOXBn9wmlUNNgPklkognQTRm1oQz98lEfydoK5cIF3CkIHQhdfNJQxpDBZgt2YC3V4AFgH5Ihmiv6xq+udgqyDVajWDRQavge5faHDT4DDQB3llX5KXBqYGzWbgAkaAIXpwYzLCjPQMjfWMAk2UHZygUcHBIM2gxKABDG9zBgcGD4YAhlCgve0Maxm2MWxn4mDSYzJlMocoZWKE6hFmQAFMDgCMyJRd</latexit> d(x, y) = d(y, x) <latexit sha1_base64="0skMukJH2EiJPmAOm3DDktriWJk=">AAACd3ichVHLSsNAFD2N7/po1Y3gwmCxtFDKRAVFEEQ3LrXaB1QpSTqtwTQJSVpaiz/gD7gQBAVR8TPc+AMu+gniUkEEF96mAdGi3mFmzpy5586ZGcXSNcdlrBUQenr7+gcGh4LDI6NjofD4RMYxq7bK06qpm3ZOkR2uawZPu5qr85xlc7mi6DyrHG6097M1bjuaaey6DYvvV+SyoZU0VXaJKoQnirF6ohEXV8VirJGox4OiWAhHWJJ5IXYDyQcR+LFlhq+xhyJMqKiiAg4DLmEdMhxqeUhgsIjbR5M4m5Dm7XMcI0jaKmVxypCJPaSxTKu8zxq0btd0PLVKp+jUbVKKmGOP7Ja9sAd2x57Yx6+1ml6NtpcGzUpHy61C6GRq5+1fVYVmFwdfqj89uyhh2fOqkXfLY9q3UDv62tHpy85Kaq4ZZZfsmfxfsBa7pxsYtVf1apunzhCkD5B+Pnc3yMwnpYXk/PZiZG3d/4pBTGMWMXrvJaxhE1tI07l1nOMaN4F3YUaICrFOqhDwNZP4FoL0CQvUjso=</latexit> d(x, z)  d(x, y) + d(y, z) <latexit sha1_base64="9qKN8iNodO1M+MinV6zIkttHiP4=">AAACg3ichVHLSsNAFD3Gd3006kYQJFgqLUqZVkERBNGNS21tFVRKkk5rME1ikhZrcefKH3DhSkFEXeofuPEHXPgJ4lLBjQtv0oBoUe8wM2fO3HPnzIxi6ZrjMvbUIrS2tXd0dnWHenr7+sPiwGDOMSu2yrOqqZv2hiI7XNcMnnU1V+cbls3lsqLzdWV3ydtfr3Lb0Uxjza1ZfLsslwytqKmyS1ReHC3E9icP4tKWzvckD9fi0gSBGpEhScqLEZZgfkjNIBmACIJYMcVLbKEAEyoqKIPDgEtYhwyH2iaSYLCI20adOJuQ5u9zHCJE2gplccqQid2lsUSrzYA1aO3VdHy1Sqfo1G1SSoiyR3bFXtkDu2HP7OPXWnW/huelRrPS0HIrHz4ezrz/qyrT7GLnS/WnZxdFzPpeNfJu+Yx3C7Whrx6cvGbm0tH6ODtnL+T/jD2xe7qBUX1TL1Z5+hQh+oDkz+duBrlUIjmVSK1ORxYWg6/owgjGEKP3nsEClrGCLJ17hGvc4k5oFyaElDDdSBVaAs0QvoUw/wlBqJHk</latexit> , <latexit sha1_base64="wgOPEMHGkhAJ7oAa6vQ1r4zPZMA=">AAACdnichVHLSiNBFD1p3/EVdSMMSDBEXYUbHRhxJbpx4cJXVFAJ3W0lKex0N9WVhEyYH/AHXIgLBUfEz5jN/IALP0FcKujChTedBlFRb1FVp07dc+tUleU7MtBENzGjrb2js6u7J97b1z8wmBga3gy8irJFzvYcT21bZiAc6YqcltoR274SZtlyxJZ1sNjc36oKFUjP3dB1X+yVzaIrC9I2NVP5xFAyubssClrJYkmbSnm1fCJFGQoj+RFkI5BCFCte4gK72IcHGxWUIeBCM3ZgIuC2gywIPnN7aDCnGMlwX+AP4qytcJbgDJPZAx6LvNqJWJfXzZpBqLb5FIe7YmUSabqmS7qn/3RFt/T8aa1GWKPppc6z1dIKPz94OLr++K2qzLNG6VX1pWeNAmZDr5K9+yHTvIXd0ld/H92vz62lGxN0Rnfs/5Ru6B/fwK0+2OerYu0Ycf6A7Pvn/gg2pzPZmcz06s/U/EL0Fd34gXFM8Xv/wjyWsIIcn1vDCf7iIvZkjBlpY7KVasQizQjehEEv88iQxQ==</latexit> 17
18. D2KE: Distance to Kernel Embedding[Wu et al., ʼ18] 距離関数を⽤いてカーネル関数を定義する • 距離関数 と確率密度関数 を⽤いてカーネル関数を定義 • 上で定義されるカーネル関数は正定値 • で定義すると正定値になるとは限らない • がnegative definite symmetric の時のみ正定値 • 例: ⽂字列の編集距離はアルファベットサイズ1の場合のみに限る © 2019, Retrieva, Inc. All rights reserved. k(x, y) := Z p(w)e d(x,w)e d(y,w)dw <latexit sha1_base64="HRTb81ONqAHOUmPSqVtlVF8qucU=">AAACtHichVFNSxtBGH6ytmpjq9FeBC+LISWBGiYqKoIQ9OKtapoouDbsbsY4ZL/YnSSmS/5A/4CHnlooUnruL+jFP+BBvOpBekyhlx76ZrNQ1La+y84887zv884zM4ZniUAydpFQhh49Hh4ZfZIce/psfCI1OVUJ3KZv8rLpWq6/Z+gBt4TDy1JIi+95Ptdtw+K7RmOjn99tcT8QrvNadjx+YOt1RxwKU5dEVVOvGtnjl52curqmasKRqpdt5zTfDnn3TagJqc5pdd22dbVGZe1c9++pTpQiqtauptIsz6JQ74NCDNKIY8tNnUJDDS5MNGGDw4EkbEFHQN8+CmDwiDtASJxPSER5ji6SpG1SFacKndgGjXVa7cesQ+t+zyBSm7SLRb9PShUZds4+sx47Y1/YDfv1z15h1KPvpUOzMdByrzrxbrr080GVTbPE0R/Vfz1LHGIl8irIuxcx/VOYA33r7UmvtLqTCV+wj+w7+f/ALtg3OoHT+mF+2uY775GkByjcve77oDKfLyzk57cX08X1+ClGMYNZZOm+l1HEJrZQpn2/4hJXuFaWFE0xFT4oVRKx5jluheL8BpBHpww=</latexit> d <latexit sha1_base64="F6t6w3U2rj9Mz4tRs97de8VpFQE=">AAACZHichVFNSwJBGH7cvswsLQmCICQxOsloQdFJ6tLRj/wAE9ldx1pcd5fdVTDpD9S16NCpICL6GV36Ax38A0F0NOjSodd1IUqqd5iZZ555n3eemZEMVbFsxroeYWR0bHzCO+mb8k/PBIKzc3lLb5oyz8m6qptFSbS4qmg8Zyu2youGycWGpPKCVN/p7xda3LQUXduz2wYvN8QDTakpsmgTla5WghEWY06Eh0HcBRG4kdKDt9hHFTpkNNEAhwabsAoRFrUS4mAwiCujQ5xJSHH2OY7hI22TsjhliMTWaTygVcllNVr3a1qOWqZTVOomKcOIsid2x3rskd2zF/bxa62OU6PvpU2zNNByoxI4Wci+/6tq0Gzj8Ev1p2cbNWw6XhXybjhM/xbyQN86uuhltzLRzgq7Zq/k/4p12QPdQGu9yTdpnrmEjz4g/vO5h0E+EYuvxRLp9Uhy2/0KLxaxjFV67w0ksYsUcnQuxynOcO55FvxCSJgfpAoeVxPCtxCWPgHPmono</latexit> k(x, y) = e d(x,y) <latexit sha1_base64="3hwCAtAU4hPPDMQIL8jmZNO9rh0=">AAACiXicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQuoZGtU6FRqKtgqxBTlVqfWxlXHZJYo6MakJ+bmJiqkgCVr4wWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGxnpGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3i6GDQw7GXYxcTMZMlkwWUGUMjFC9QgzoAAmZwCTD5Yv</latexit> p(w) <latexit sha1_base64="VTQmHLOSWoIB8pRvlVI6Eo+hU/o=">AAACZ3ichVFNLwNBGH66vqo+WiTSxAVNhUszi4Q4CRfHFkWCyO4a7cR+ZXdaqcYfcHCtxIlERPwMF3/AwU8QRxIXB+9uNxEavJOZeeaZ93nnmRndNYUvGXuKKW3tHZ1d8e5ET29ffzI1MLjhOxXP4EXDMR1vS9d8bgqbF6WQJt9yPa5Zusk39cPlYH+zyj1fOPa6rLl819JKtjgQhiYDyp08mtpLZViOhTHaCtQIZBBF3kndYAf7cGCgAgscNiRhExp8attQweASt4s6cR4hEe5znCBB2gplccrQiD2ksUSr7Yi1aR3U9EO1QaeY1D1SjiLLHtkte2UP7I49s49fa9XDGoGXGs16U8vdveRpeu39X5VFs0T5S/WnZ4kDzIdeBXl3Qya4hdHUV48br2sLq9n6BLtiL+T/kj2xe7qBXX0zrgt89QIJ+gD153O3go3pnDqTmy7MZhaXoq+IYwTjmKT3nsMiVpBHkc4t4wwNnMeelaQyrKSbqUos0gzhWyhjn+n1ito=</latexit> d <latexit sha1_base64="F6t6w3U2rj9Mz4tRs97de8VpFQE=">AAACZHichVFNSwJBGH7cvswsLQmCICQxOsloQdFJ6tLRj/wAE9ldx1pcd5fdVTDpD9S16NCpICL6GV36Ax38A0F0NOjSodd1IUqqd5iZZ555n3eemZEMVbFsxroeYWR0bHzCO+mb8k/PBIKzc3lLb5oyz8m6qptFSbS4qmg8Zyu2youGycWGpPKCVN/p7xda3LQUXduz2wYvN8QDTakpsmgTla5WghEWY06Eh0HcBRG4kdKDt9hHFTpkNNEAhwabsAoRFrUS4mAwiCujQ5xJSHH2OY7hI22TsjhliMTWaTygVcllNVr3a1qOWqZTVOomKcOIsid2x3rskd2zF/bxa62OU6PvpU2zNNByoxI4Wci+/6tq0Gzj8Ev1p2cbNWw6XhXybjhM/xbyQN86uuhltzLRzgq7Zq/k/4p12QPdQGu9yTdpnrmEjz4g/vO5h0E+EYuvxRLp9Uhy2/0KLxaxjFV67w0ksYsUcnQuxynOcO55FvxCSJgfpAoeVxPCtxCWPgHPmono</latexit> 18
19. D2KEのRandom Features[Wu et al., ʼ18] ランダムに⽣成したサンプルとの距離を計算し特徴ベクトル化 • (サンプリング)は ”適当” に決める (RFFは⼀意に決まる) • データから直接サンプリング • データの統計量などを⾒てランダムに⽣成 © 2019, Retrieva, Inc. All rights reserved. p(w) <latexit sha1_base64="VTQmHLOSWoIB8pRvlVI6Eo+hU/o=">AAACZ3ichVFNLwNBGH66vqo+WiTSxAVNhUszi4Q4CRfHFkWCyO4a7cR+ZXdaqcYfcHCtxIlERPwMF3/AwU8QRxIXB+9uNxEavJOZeeaZ93nnmRndNYUvGXuKKW3tHZ1d8e5ET29ffzI1MLjhOxXP4EXDMR1vS9d8bgqbF6WQJt9yPa5Zusk39cPlYH+zyj1fOPa6rLl819JKtjgQhiYDyp08mtpLZViOhTHaCtQIZBBF3kndYAf7cGCgAgscNiRhExp8attQweASt4s6cR4hEe5znCBB2gplccrQiD2ksUSr7Yi1aR3U9EO1QaeY1D1SjiLLHtkte2UP7I49s49fa9XDGoGXGs16U8vdveRpeu39X5VFs0T5S/WnZ4kDzIdeBXl3Qya4hdHUV48br2sLq9n6BLtiL+T/kj2xe7qBXX0zrgt89QIJ+gD153O3go3pnDqTmy7MZhaXoq+IYwTjmKT3nsMiVpBHkc4t4wwNnMeelaQyrKSbqUos0gzhWyhjn+n1ito=</latexit> k(x, y) ⇡ 1 D D X i=1 e d(x,wi )e d(y,wi ) <latexit sha1_base64="eXlNOs4ryZf7H4zrNJsIzEZ4UCM=">AAACyHichVHLahRBFD1pX3F8ZNSN4KZxiCSgQ3UUFEEImoW4ysNJAumkqa7UjMX0o6iumaRteuPSH3DhSkFE/Aw3+gEu8gniwkUEFVx4p6dBNERv01Wnzr3n1qmqUEcqs4ztTThHjh47fmLyZOPU6TNnp5rnzq9m6cAI2RFplJr1kGcyUonsWGUjua6N5HEYybWwf2+UXxtKk6k0eWhzLTdj3ktUVwluiQqaoj+zezWfdX2utUl3Xb9ruCi8slgo/WwQB4W645VbC65v4kKWW4WvrHvN7/E45u42SXcCNVu6h6TzcTpotlibVeEeBF4NWqhjMW2+ho9tpBAYIIZEAks4AkdG3wY8MGjiNlEQZwipKi9RokHaAVVJquDE9mns0WqjZhNaj3pmlVrQLhH9hpQuptlH9obts/fsLfvEfh7aq6h6jLzkNIdjrdTB1NOLK9/+q4pptnj0W/VPzxZd3Kq8KvKuK2Z0CjHWDx8/21+5vTxdXGEv2Wfy/4LtsXd0gmT4VbxaksvP0aAH8P6+7oNgda7tXW/PLd1ozd+tn2ISl3AZM3TfNzGP+1hEh/b9gC/4jh/OA0c7O04+LnUmas0F/BHOk1+J9q/u</latexit> = s(x)>s(y) <latexit sha1_base64="G8znCRJYa8zBCjloqnly8B8S43U=">AAACdHichVHLSsNAFD2Nr1ofrboRdFEslbop0yooglB047JV+4CqJYnTGpomIZkWa/EH/AEXrhRFxM9w4w+46CeIy4puXHibBkRFvWEyZ87cc+fMXMXSNUcw1vZJff0Dg0P+4cDI6Nh4MDQxmXPMuq3yrGrqpl1QZIfrmsGzQhM6L1g2l2uKzvNKdaO7n29w29FMY0c0Lb5XkyuGVtZUWRBVCgXXnNjRwv6uMK2wE2sulEIRFmduhH+ChAci8CJthm6wiwOYUFFHDRwGBGEdMhz6ikiAwSJuDy3ibEKau89xggBp65TFKUMmtkr/Cq2KHmvQulvTcdUqnaLTsEkZRpQ9slvWYQ/sjj2x919rtdwaXS9NmpWellul4On09tu/qhrNAoefqj89C5Sx4nrVyLvlMt1bqD194/iss726FW3Ns0v2TP4vWJvd0w2Mxot6neFb5whQAxLfn/snyCXjicV4MrMUSa17rfBjBnOI0XsvI4VNpJF1e3KOK1z7XqVZKSJFe6mSz9NM4UtI8Q+Dko8k</latexit> w1, ..., wD i.i.d. ⇠ p(w) <latexit sha1_base64="C2DCyoAXUNKaPrHKCKwjtIp7CDA=">AAACk3icSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvolccb6ujp6emUx7soxOSDlKaWVMeUpFaUVGfqZeql6NXWVscUZ+bWFmiUa3LFCygb6BmAgQImwxDKUGaAgoB8geUMMQwpDPkMyQylDLkMqQx5DCVAdg5DIkMxEEYzGDIYMBQAxWIZqoFiRUBWJlg+laGWgQuotxSoKhWoIhEomg0k04G8aKhoHpAPMrMYrDsZaEsOEBcBdSowqBpcNVhp8NnghMFqg5cGf3CaVQ02A+SWSiCdBNGbWhDP3yUR/J2grlwgXcKQgdCF180lDGkMFmC3ZgLdXgAWAfkiGaK/rGr652CrINVqNYNFBq+B7l9ocNPgMNAHeWVfkpcGpgbNZgBFgCF6cGMywoz0DI31jAJNlB2coFHBwSDNoMSgAQxvcwYHBg+GAIZQoL1TGHYxHGY4wiTKZM3kxOQCUcrECNUjzIACmHwB1y6arw==</latexit> 19
20. D2KEのRandom Features理論解析[Wu et al., ʼ18]: ⼀様ノルムでワーストケースを評価 © 2019, Retrieva, Inc. All rights reserved. カーネル関数値と近似後の最⼤の ズレがε以上となる確率 近似後の次元について 指数的に減少 Pr( max x1,x2 2X |s(x)>s(y) k(x, y)| ✏)  2( 24 ✏ )2PX ,d exp( D✏2 8 ) <latexit sha1_base64="hUJYYspBJVjDQ+iq+pevrokc0V4=">AAADInichVHLbtNAFL02rxIeDbBBYjMiKrKlNBqbSlSsKmABuzQhbaROYznuJB3FL2wnSnDnB/gBFqxAIIT4BlZsEDuEWFTiBxDLIrFhwR3HqIIKuJZ9z5x7z/WZmX7sizSjdF/Tjx0/cfLUwunKmbPnzi9WL1zcSKNx4vGOF/lR0u27KfdFyDuZyHzejRPuBn2fb/ZHt1V9c8KTVETh/WwW8+3AHYZiIDw3Q8qpvmFJkDcTabDAnTo5ExlRKQmIJeslsiVhIiRdSfZUPTWmZo9lUYxoZpJlMjKm9Zm5x4b8AWE8ToUfhSZhvloqucEGievl9gobukHgyvxXkzR7ua1GNp28W9+RUpnh0xjdIDlXLd85bO+VdmSRV6XpVGu0QYsgR4FVghqU0YyqL4HBDkTgwRgC4BBChtgHF1J8tsACCjFy25AjlyASRZ2DhApqx9jFscNFdoTfIa62SjbEtZqZFmoP/+Ljm6CSwBL9RF/RA/qOvqZf6I+/zsqLGcrLDHN/ruWxs/jocvv7f1UB5gx2D1X/9JzBAFYLrwK9xwWjduHN9ZOHjw/aN1tL+TX6jH5F/0/pPn2LOwgn37wX67z1BCp4Adafx30UbNgN63rDXl+prd0qr2IBrsBVMPC8b8Aa3IUmdMDTato9raW19ef6e/2D/nHeqmul5hL8Fvrnn9FH0Do=</latexit> 20
21. D2KEのRandom Featuresアルゴリズムまとめ: 適当にデータを⽣成して距離関数を計算して特徴⽣成 • ⼊⼒: 距離関数 , ハイパーパラメータ データ集合 , 所望の特徴ベクトル次元 • 出⼒: 各データの特徴ベクトル 1. 個のデータを からサンプリング(⽣成) 2. 各データ についてサンプリングしたデータ間と距離を計算して特徴 ベクトルを得る © 2019, Retrieva, Inc. All rights reserved. w1, ..., wD i.i.d. ⇠ p(w) <latexit sha1_base64="C2DCyoAXUNKaPrHKCKwjtIp7CDA=">AAACk3icSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvolccb6ujp6emUx7soxOSDlKaWVMeUpFaUVGfqZeql6NXWVscUZ+bWFmiUa3LFCygb6BmAgQImwxDKUGaAgoB8geUMMQwpDPkMyQylDLkMqQx5DCVAdg5DIkMxEEYzGDIYMBQAxWIZqoFiRUBWJlg+laGWgQuotxSoKhWoIhEomg0k04G8aKhoHpAPMrMYrDsZaEsOEBcBdSowqBpcNVhp8NnghMFqg5cGf3CaVQ02A+SWSiCdBNGbWhDP3yUR/J2grlwgXcKQgdCF180lDGkMFmC3ZgLdXgAWAfkiGaK/rGr652CrINVqNYNFBq+B7l9ocNPgMNAHeWVfkpcGpgbNZgBFgCF6cGMywoz0DI31jAJNlB2coFHBwSDNoMSgAQxvcwYHBg+GAIZQoL1TGHYxHGY4wiTKZM3kxOQCUcrECNUjzIACmHwB1y6arw==</latexit> x1, ..., xn 2 Rm <latexit sha1_base64="TnOY48l35AEZxr3G+/nHHP+2eLk=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIV8Qb6ijo6enpKFTE5ynEZAJxbmJJRlJSdVBtXC5XvICygZ4BGChgMgyhDGUGKAjIF1jOEMOQwpDPkMxQypDLkMqQx1ACZOcwJDIUA2E0gyGDAUMBUCyWoRooVgRkZYLlUxlqGbiAekuBqlKBKhKBotlAMh3Ii4aK5gH5IDOLwbqTgbbkAHERUKcCg6rBVYOVBp8NThisNnhp8AenWdVgM0BuqQTSSRC9qQXx/F0Swd8J6soF0iUMGQhdeN1cwpDGYAF2aybQ7QVgEZAvkiH6y6qmfw62ClKtVjNYZPAa6P6FBjcNDgN9kFf2JXlpYGrQbAZQBBiiBzcmI8xIz9BYzyjQRNnBCRoVHAzSDEoMGsDwNmdwYPBgCGAIBdrbyrCaYQvDViY2Jh0mYyZTiFImRqgeYQYUwGQHAOwek60=</latexit> p(w) <latexit sha1_base64="VTQmHLOSWoIB8pRvlVI6Eo+hU/o=">AAACZ3ichVFNLwNBGH66vqo+WiTSxAVNhUszi4Q4CRfHFkWCyO4a7cR+ZXdaqcYfcHCtxIlERPwMF3/AwU8QRxIXB+9uNxEavJOZeeaZ93nnmRndNYUvGXuKKW3tHZ1d8e5ET29ffzI1MLjhOxXP4EXDMR1vS9d8bgqbF6WQJt9yPa5Zusk39cPlYH+zyj1fOPa6rLl819JKtjgQhiYDyp08mtpLZViOhTHaCtQIZBBF3kndYAf7cGCgAgscNiRhExp8attQweASt4s6cR4hEe5znCBB2gplccrQiD2ksUSr7Yi1aR3U9EO1QaeY1D1SjiLLHtkte2UP7I49s49fa9XDGoGXGs16U8vdveRpeu39X5VFs0T5S/WnZ4kDzIdeBXl3Qya4hdHUV48br2sLq9n6BLtiL+T/kj2xe7qBXX0zrgt89QIJ+gD153O3go3pnDqTmy7MZhaXoq+IYwTjmKT3nsMiVpBHkc4t4wwNnMeelaQyrKSbqUos0gzhWyhjn+n1ito=</latexit> D <latexit sha1_base64="8eSQWchmR3f/b2eGJyudXC45MVg=">AAACZHichVFNSwJBGH7cvswsLQmCICQxOsloQdFJqkNHP/IDTGR3m2xx3V12V8GkP1DXokOngojoZ3TpD3TwDwTR0aBLh17XhSip3mFmnnnmfd55ZkYyVMWyGet4hKHhkdEx77hvwj85FQhOz+QtvWHKPCfrqm4WJdHiqqLxnK3YKi8aJhfrksoLUm2rt19octNSdG3Xbhm8XBermnKgyKJNVHq7EoywGHMiPAjiLojAjZQevMUe9qFDRgN1cGiwCasQYVErIQ4Gg7gy2sSZhBRnn+MYPtI2KItThkhsjcYqrUouq9G6V9Ny1DKdolI3SRlGlD2xO9Zlj+yevbCPX2u1nRo9Ly2apb6WG5XAyVz2/V9VnWYbh1+qPz3bOMC641Uh74bD9G4h9/XNo4tudiMTbS+xa/ZK/q9Yhz3QDbTmm3yT5plL+OgD4j+fexDkE7H4SiyRXo0kN92v8GIei1im915DEjtIIUfncpziDOeeZ8EvhITZfqrgcTUhfAth4ROPmonI</latexit> xi <latexit sha1_base64="xSkKUs5rpr3tXin1AlN0bIQMSp0=">AAACZnichVFNLwNBGH66vou2iJC4NBri1LwtCXESLo60WhKk2V3TmnS7u9ndNmj8AYkrBycSEfEzXPwBh/4D4liJi4O3200EwTuZmWeeeZ93npnRbEO6HlEjpHR0dnX39PaF+wcGI9HY0HDetaqOLnK6ZVjOlqa6wpCmyHnSM8SW7Qi1ohliUyuvtPY3a8JxpWVueIe22K2oJVMWpa56TGUPCrIQS1CS/Ij/BKkAJBDEmhW7wQ72YEFHFRUImPAYG1DhcttGCgSbuV3UmXMYSX9f4Bhh1lY5S3CGymyZxxKvtgPW5HWrpuurdT7F4O6wMo4peqRbatID3dEzvf9aq+7XaHk55Flra4VdiJ6MZ9/+VVV49rD/qfrTs4ciFnyvkr3bPtO6hd7W147Om9nFzFR9mq7ohf1fUoPu+QZm7VW/XheZC4T5A1Lfn/snyKeTqdlken0usbQcfEUvJjCJGX7veSxhFWvI8bklnOIM56EnJaKMKmPtVCUUaEbwJZT4B9PJitg=</latexit> d(x, y) <latexit sha1_base64="7AcGvPS+Mo+4vXWSK+5daE5bdvQ=">AAACaXichVFNSwJBGH7cvsy+tC5SF0kMg5DRgqKT1KWjZX6Aieyuk22tu8vuKpn0Bzp1i+pUEBH9jC79gQ7+hPBo0KVDr+tClFTvMDPPPPM+7zwzIxmqYtmMtTzCwODQ8Ih31Dc2PjE55Q9MZy29Zso8I+uqbuYl0eKqovGMrdgqzxsmF6uSynPS0WZ3P1fnpqXo2q7dMHixKlY0ZV+RRZuobDl6vNRYLPnDLMacCPWDuAvCcCOl+++xhzJ0yKihCg4NNmEVIixqBcTBYBBXRJM4k5Di7HOcwkfaGmVxyhCJPaKxQquCy2q07ta0HLVMp6jUTVKGEGEv7IF12DN7ZK/s49daTadG10uDZqmn5UZp6iyYfv9XVaXZxsGX6k/PNvax5nhVyLvhMN1byD19/eSik17fiTQX2C1rk/8b1mJPdAOt/ibfbfOda/joA+I/n7sfZBOx+HIssb0STm64X+HFHOYRpfdeRRJbSCFD5x7iHJe48rSFgBAUZnupgsfVzOBbCOFPa5eLiA==</latexit> <latexit sha1_base64="hmbSIjNHGW90U/2Eq3fmvX3IV34=">AAACaXichVG7SgNBFD1ZXzE+kmgTtBFDxCrMRkGxCtpYJtE8QEPYXce4Zl/sbgIx+ANWdqJWCiLiZ9j4Axb5BLFUsLHw7mZBNKh3mJkzZ+65c2ZGtjTVcRnrhoSBwaHhkfBoZGx8YjIai0+VHLNpK7yomJppV2TJ4Zpq8KKruhqvWDaXdFnjZbmx4e2XW9x2VNPYdtsWr+pS3VD3VUVyiSrt1iVdl2qxJEszP+b6gRiAJILImbFb7GIPJhQ0oYPDgEtYgwSH2g5EMFjEVdEhziak+vscx4iQtklZnDIkYhs01mm1E7AGrb2ajq9W6BSNuk3KOaTYE7tjr+yR3bNn9vFrrY5fw/PSplnuablVi54ktt7/Vek0uzj4Uv3p2cU+Vn2vKnm3fMa7hdLTt47OXrfWCqnOArtmL+T/inXZA93AaL0pN3leuESEPkD8+dz9oJRJi0vpTH45mV0PviKMWcxjkd57BVlsIocinXuIU5zjIvQixIWEMNNLFUKBZhrfQkh+AoarjBU=</latexit> D <latexit sha1_base64="8eSQWchmR3f/b2eGJyudXC45MVg=">AAACZHichVFNSwJBGH7cvswsLQmCICQxOsloQdFJqkNHP/IDTGR3m2xx3V12V8GkP1DXokOngojoZ3TpD3TwDwTR0aBLh17XhSip3mFmnnnmfd55ZkYyVMWyGet4hKHhkdEx77hvwj85FQhOz+QtvWHKPCfrqm4WJdHiqqLxnK3YKi8aJhfrksoLUm2rt19octNSdG3Xbhm8XBermnKgyKJNVHq7EoywGHMiPAjiLojAjZQevMUe9qFDRgN1cGiwCasQYVErIQ4Gg7gy2sSZhBRnn+MYPtI2KItThkhsjcYqrUouq9G6V9Ny1DKdolI3SRlGlD2xO9Zlj+yevbCPX2u1nRo9Ly2apb6WG5XAyVz2/V9VnWYbh1+qPz3bOMC641Uh74bD9G4h9/XNo4tudiMTbS+xa/ZK/q9Yhz3QDbTmm3yT5plL+OgD4j+fexDkE7H4SiyRXo0kN92v8GIei1im915DEjtIIUfncpziDOeeZ8EvhITZfqrgcTUhfAth4ROPmonI</latexit> s(x1), ..., s(xn) 2 RD <latexit sha1_base64="EQEtkaYT5Q88kjVjZe4QrUEQB8A=">AAACi3icSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQuoFWtUxBtq6ijo6ekp6CiAeHmaCjGZeQoxuYklGUlJ1UG1cS5c8QLKBnoGYKCAyTCEMpQZoCAgX2A5QwxDCkM+QzJDKUMuQypDHkMJkJ3DkMhQDITRDIYMBgwFQLFYhmqgWBGQlQmWT2WoZeAC6i0FqkoFqkgEimYDyXQgLxoqmgfkg8wsButOBtqSA8RFQJ0KDKoGVw1WGnw2OGGw2uClwR+cZlWDzQC5pRJIJ0H0phbE83dJBH8nqCsXSJcwZCB04XVzCUMagwXYrZlAtxeARUC+SIboL6ua/jnYKki1Ws1gkcFroPsXGtw0OAz0QV7Zl+SlgalBsxlAEWCIHtyYjDAjPUNjPaNAE2UHJ2hUcDBIMygxaADD25zBgcGDIYAhFGhvD8Mmht0Me5h4mYyZrJhsIEqZGKF6hBlQAJMrACx2lXI=</latexit> 21
22. Efficient Global String Kernel with Random Features [Wu et al., ʼ19a] ⾼い予測精度と⾼速な計算を⽰した • 距離関数を編集距離とした⽂字列カーネルのRFを提案 • s © 2019, Retrieva, Inc. All rights reserved. 22 [Wu et al., ʼ19a]より引⽤ Random Feature カーネル + SVM ニューラルネット
23. 特徴ベクトルの次元を変化させた時の予測精度 数千次元程度で⼗分な精度 © 2019, Retrieva, Inc. All rights reserved. 23 [Wu et al., ʼ19a]より引⽤ better 縦軸: 予測精度 横軸: 特徴ベクトルの次元数
24. 特徴ベクトルの次元を変化させた時の計算時間 計算時間は特徴ベクトルの次元について線形に増加 © 2019, Retrieva, Inc. All rights reserved. 24 [Wu et al., ʼ19a]より引⽤ 縦軸: 計算時間 横軸: 特徴ベクトルの次元数
25. 学習データ数, ⽂字列の⻑さを変化させた時の計算時間 計算時間は学習データ数, ⽂字列の⻑さについて線形に増加 © 2019, Retrieva, Inc. All rights reserved. 25 [Wu et al., ʼ19a]より引⽤ 縦軸: 計算時間 横軸: 学習データ数 縦軸: 計算時間 横軸: ⽂字列の⻑さ
26. まとめ • カーネル法はデータ数についてスケールしない • Random Featuresを⽤いた⾼速な近似計算 • Random Fourier Features • Shift invariant kernelに対するものが始まり • 内積の期待値がカーネル関数と⼀致するようにランダム射影によって作成 • 距離関数からカーネル関数を定義してRFに持ち込む枠組み • ⼊⼒が数値ベクトルでなくてよい © 2019, Retrieva, Inc. All rights reserved. 26
27. © 2019, Retrieva, Inc. All rights reserved.