論文紹介：Neural Architecture Search with Bayesian Optimisation and Optimal Transport [Kandasamy et al., NIPS 2018]

Neural Architecture Search with Bayesian
Optimisation and Optimal Transport
K. Kandasamy, W. Neiswanger, J. Schneider, B. Poczos, and E. Xing
NIPS 2018
2018.11.27｜文献紹介
斉藤翔汰
横浜国立大学大学院環境情報学府情報メディア環境学専攻
情報メディア学コース白川真一研究室博士課程前期2年

View Slide

文献情報と選んだ理由
文献情報
K. Kandasamy, W. Neiswanger, J. Schneider, B. Poczos, and E. Xing,
“Neural Architecture Search with Bayesian Optimisation and Optimal
Transport,” in Proceedings of Thirty-second Conference on Neural
Information Processing Systems (NIPS 2018), 2018.
選んだ理由
n 構造探索に関する最新の論文であるため
n 「構造間の距離」を定義したとのことで，他の構造探索の
論文とは異なる視点であったため興味を持った
2

View Slide

概要
構造間の距離 OTMANN ＋ Bayesian Optimization
n NNの構造間の距離 OTMANN を提案
n OTMANNを使ってカーネルを設計し，Bayesian Optimization
で最適化する手法NASBOTを提案
n 6つの回帰問題＋CIFAR-10において，NASBOTは他の手法と
同等以上の精度を持つ構造を獲得
n NASBOTで得た構造は，どのデータセットにおいても
複数の出力層と長いスキップ結合を持っていた
3

View Slide

Introduction
Bayesian Optimization (BO)
n ベイズモデルに基づくBlack-Box最適化手法のひとつ
n 機械学習ではSVMの正則化係数などの最適化に使用
n 従来研究では探索空間が連続であるものが多い
Gaussian Process (GP) based BO
n BOでは事前分布にガウス過程を置くことが多い
n GP based BOでは2点の類似度をカーネルの形で表現
n 探索空間が連続であるとき，カーネルとしてGaussian，
Laplacian，Matérnがよく用いられる
4

View Slide

Introduction
BOを構造探索に適用する際の問題点
n ニューラルネットワーク（NN）の構造探索では，
探索空間が離散であるため先ほどのカーネルを使用できない
n BOを適用するためには構造の類似度を定量化する必要がある
Contribution
1. ニューラルネットワークの構造間の(擬)距離OTMANNを開発
- OTMANNは最適輸送問題を解くことで効率的に求められる
2. 構造探索のためのBOフレームワークNASBOTを提案
- NASBOTにおける獲得関数の最適化は進化計算で実行
5

View Slide

Set up｜A brief review of GP based BO
Notation
n 目的関数（validation accuracy）：
n 探索空間：
n 構造：
n 構造に対する評価値：
n 観測点：
n 平均関数：
n カーネル（共分散）関数：
n 評価値のベクトル表現：
n カーネルに関する変数：
6
f
AAAEk3icnVNNb9NAEJ0UAyV89AMhIXGxSIsQQtE6h6bApYIe4FDRpE1bqY4q290EU8e2bCe0WPkDcAVx4AQSB8R/4MKFP8ChPwFxLBIXDrwdO0F8JEisZXvm7cybt7O7dui5cSLEYWHimHb8xMnJU8XTZ86em5qemd2Ig27kyIYTeEG0ZVux9FxfNhI38eRWGEmrY3ty0967o+Y3ezKK3cBfTw5C2exYbd9tuY6VAKq1dqZLolypLhrC0NkQNyoDo6obZcGjRPlYDWYK78mkXQrIoS51SJJPCWyPLIrxbJNBgkJgTUrpIRCLIngux0jqUxH5XeASURbQPXzb8LZz1IeveGNmcFDJwxshU6d58Um8FUfio3gnPovvI7lS5lB6DvC3s1wZ7kw9ubj27Z9ZHfwTevAza6zmhFq0yFpdaA8ZUatwsvze4xdHazfr8+kV8Vp8gf5X4lB8wAr83lfnTU3WXzL7MnKyLkawVnIF98EngShP9eAaqpgc04ZKVa/PqpVv0vUh9r+MFu3/wZhho3uWYL4FLnUO4rGRNtiyeR/+I97bDlfwcaKUjjpXN4c7YONJGeVe4lVrsbAqHTVd2B4yxzGu/8YYwUoZHTAm3A0fikNkK93j+Fy2+nzOm3/Reg9zJv4lzJtcpcj3RUKvST3ek6zzKiLLt+Hvs23zad8dcuo0x3FzOVMR93VwKfXRxkalbMCuGaWlhfzmTtIlukxXwVWlJbpLq9SAEklP6Rk91y5ot7Tb2nIWOlHIc87TL0Nb+QHvpPUd
AAAEhHicnVM9b9RAEJ0LhgQTSCIaJJoTlyCEULRHQ6CKgAIKRHLJJZHiU2T79o4l/pK9dyRY9wfSUlBQgUSB+A80NPwBivwERBkkGgrejs0hcuSQWMv2zNuZN29nd70kUJkW4rAycco6fWZy6qx9bto+f2Fmdnoji3upL5t+HMTpludmMlCRbGqlA7mVpNINvUBuerv3zPxmX6aZiqN1vZ/IVuh2I9VRvqsBrezM1sSi4FEdNeqlUaNyxHOVD+RQm2LyqUchSYpIww7IpQzPNtVJUAKsRTk9BeJSCk9xjKQB2cjvAZeIcoHu4tuFt12iEXzDmzGDj0oB3hSZVVoQn8U7cSQ+iffii/hxIlfOHEbPPv5ekSuTnZmDS2vf/5kV4q/pye+ssZo1dWiJtSpoTxgxq/CL/P7zl0drdxoL+VXxRnyF/tfiUHzECqL+N//tqmy8Yvb7yCm6mMJ6VCp4DD4JxHimB9dRxeGYLlSaegNWbXyHbgyx/2V0aW+EscBO7pnGfAdc5hxkYyM9sBXzEfxnvLchV4hwooyOBld3hjvg4ckZ5V7iNWtxsaoqairYATLHMa4fY0xh5Yz+YtTcjQiKE2Qb3eP4FFsDPuetv2h9iDkH/xrmHa5i832R0OtQn/ek6LyJKPI9+Htse3za20POKs1z3HzJZOO61o9fzlFj4+ZiHfaqoCm6TFfoGihu0TI9oBVqQkCbDuiFddG6bS0X13qiUt7vOfpjWHd/Ar4v8nw=
AAAEiHicnVPBbtNAEH0pAUootEVCQuISkRYhhKoNFxBcEPQAB0STNm2lOqpsdxNMHduyndAS5QfgCuLACSQOiH/gwoUf4NBPQByLxIUDb8cmiIYGibVsz7ydefN2dteJfC9JldorTBwpHj12fPJE6eTUqdPTM7NTq0nYjV3dcEM/jNcdO9G+F+hG6qW+Xo9ibXccX68523fM/FpPx4kXBivpbqSbHbsdeC3PtVNCtdbmTEUtKBnlUaOaGxXkYymcLXyAhS2EcNFFBxoBUto+bCR8NlCFQkSsiT4eEbER0/MkRmOAEvO7xDWjbKLb/LbpbeRoQN/wJsLgspLPN2ZmGfPqs3qn9tUn9V59UT8O5eoLh9Gzy7+T5epoc/rpueXv/8zq8J/i4e+ssZpTtHBdtHrUHgliVuFm+b0nL/eXb9Tn+xfVG/WV+l+rPfWRKwh639y3NV1/JeyLzMm6GNO6nyt4QD5NxHimB5dZxZKYNlWaegNRbXwLV4bY/zLa2BlhzLDDe5ZyvkUucw6SsZEO2bL5gP5j2duOVAh4ooyOulS3hjvg8OkLKr3ka9Zic1Vl1vRo+8wcx7hygDGm1Rf0F2Mq3QioOGK20T2OzxNrIOe8+Ret9zhn8V/hvCVVSnJfNPVa6MmeZJ03EVm+Q39HbEdO+9aQs4w5iZvLmUq8r9WDt3PUWL26UKVdU5jEeVzAJVJcwy3cxRIaFKDxDM/xoni2eLN4O7vZE4X8ip/BH6O4+BMKuvPb
AAAEiHicnVNNb9NAEJ0UAyUU2iIhIXGxSIsQQtE6lxS4IOgBDogmbdpKdVTZ7iY19ZdsJ7RY+QNwBXHgBBIHxH/gwoU/wKE/AXEsEhcOvB07QXwkSKxle+btzJu3s7t25LlJKsRhaeqYdvzEyelT5dMzZ87Ozs3PrCdhL3Zkywm9MN60rUR6biBbqZt6cjOKpeXbntyw9+6o+Y2+jBM3DNbSg0i2fasbuB3XsVJAjc72XEVUa/UlQxg6G+J6bWjUdaMqeFSoGCvhfOk9mbRDITnUI58kBZTC9siiBM8WGSQoAtamjB4CsSiG53KMpAGVkd8DLhFlAd3Dtwtvq0AD+Io3YQYHlTy8MTJ1WhSfxFtxJD6Kd+Kz+D6WK2MOpecAfzvPldH27JMLq9/+meXjn9Luz6yJmlPq0BJrdaE9YkStwsnz+49fHK3eaC5ml8Vr8QX6X4lD8QErCPpfnTcN2XzJ7MvIybsYw7pfKHgAPglEeaoHV1HF5JguVKp6A1atfJOujbD/ZbRo/w/GHBvfsxTzHXCpc5BMjLTBls8H8B/x3vpcIcCJUjqaXN0c7YCNJ2OUe4lXrcXCqnTUdGF7yJzEuPYbYwwrY3TImHI3AiiOkK10T+Jz2RrwOW//Res9zJn4VzBvcpUy3xcJvSb1eU/yzquIPN+Gv8+2zad9Z8Sp0wLHLRRMZdzX4aXUxxvrtaoBuyFomi7SJboCijrdoru0Qi0IkPSUntFz7bx2U7ud3+ypUnHFz9EvQ1v+AVAZ9Cs=
AAAEk3icnVPLbtNAFL0pBkp49IGQkNhYpEUIoWiSRVNgU0EXsKho0qatVEeV7U6CqV+yndBi5QdgC2LBCiQWiH9gw4YfYNFPQCyLxIYFZ66dIB4JEmPZvvfMveeeuTNjha4TJ0IcFiaOacdPnJw8VTx95uy5qemZ2Y046Ea2bNqBG0RblhlL1/FlM3ESV26FkTQ9y5Wb1t4dNb/Zk1HsBP56chDKlmd2fKft2GYCqN7emS6JcrW2WBEVnQ1xozowanqlLHiUKB+rwUzhPRm0SwHZ1CWPJPmUwHbJpBjPNlVIUAisRSk9BGJSBM/hGEl9KiK/C1wiygS6h28H3naO+vAVb8wMNiq5eCNk6jQvPom34kh8FO/EZ/F9JFfKHErPAf5WlivDnaknF9e+/TPLwz+hBz+zxmpOqE2LrNWB9pARtQo7y+89fnG0drMxn14Rr8UX6H8lDsUHrMDvfbXf1GXjJbMvIyfrYgRrJVdwH3wSiPJUD66hisExHahU9fqsWvkGXR9i/8to0v4fjBk2umcJ5tvgUucgHhtpgS2b9+E/4r31uIKPE6V0NLi6MdwBC0/KKPcSr1qLiVXpqOnAdpE5jnH9N8YIVsrogDHhbvhQHCJb6R7H57DV53Pe+ovWe5gz8C9h3uAqRb4vEnoN6vGeZJ1XEVm+BX+fbYtP++6QU6c5jpvLmYq4r4NLqY82NqrlCuy6KC0t5Dd3ki7RZboKrhot0V1apSaUSHpKz+i5dkG7pd3WlrPQiUKec55+GdrKD+9U9Rw=
X
AAAEnXicnVPLbtNAFL0pBkp4tIUNEgsi0iKEUDTOghRWlQAJJB59pYkUR5XtTlJTv2Q7ocXKD7BHLJBAILFA/AMbNvwAi34CYlkkNiw4c+0E8UiQGMv2vWfuPffMnRkrdJ04EWK/MHVIO3zk6PSx4vETJ0/NzM6d3oiDXmTLuh24QdS0zFi6ji/riZO4shlG0vQsVzasnRtqvtGXUewE/nqyF8q2Z3Z9p+PYZgKoZXhmsm2bbtocbM6WRaVaW9SFXmJDXKsOjVpJrwgeZcrHcjBXeE8GbVFANvXII0k+JbBdMinG0yKdBIXA2pTSQyAmRfAcjpE0oCLye8AlokygO/h24bVy1IeveGNmsFHJxRshs0QL4pN4Kw7ER/FOfBbfx3KlzKH07OFvZbky3Jx5cnbt2z+zPPwT2v6ZNVFzQh1aZK0OtIeMqFXYWX7/8bODteurC+lF8Vp8gf5XYl98wAr8/lf7zYpcfc7sN5GTdTGCdS9X8AB8EojyVA8uo4rBMV2oVPUGrFr5Bl0ZYf/LaNLuH4wZNr5nCeY74FLnIJ4YaYEtm/fhP+K99biCjxOldKxydWO0AxaelFHuJV61FhOrKqGmA9tF5iTG9d8YI1gpo0PGhLvhQ3GIbKV7Ep/D1oDPefsvWu9gzsC/jHmDqxT5vkjoNajPe5J1XkVk+Rb8XbYtPu1bI84SzXPcfM5UxH0dXsrSeGOjWtFhr+jlpav5zZ2mc3SBLoGrRkt0m5apzqt7Si/opXZeu6Xd1e5noVOFPOcM/TK0xg/Up/mh
x 2 X
AAAEpXicnVPNbtNAEJ6UACX8NIELEheLtIBQFW1yIIVTJTjAoaJNkzRSHUW2u0lN/SfbSVOivAAvwAFxAAkQ4h24cOEFOPQREMciceHAt2MniJ8GibVsz3w78823s7tm4NhRLMRhZu5E9uSp0/NncmfPnb+wkC9cbEZ+P7Rkw/IdP2yZRiQd25ON2I4d2QpCabimI7fMvbtqfmsgw8j2vXp8EMi2a/Q8u2tbRgyoky8MNd32NN014l3LcEatcSdfFKVKdaUsyhob4nZlYlS1cknwKFI61v1C5j3ptEM+WdQnlyR5FMN2yKAIzzaVSVAArE0jegTEoBCezTGSxpRDfh+4RJQBdA/fHrztFPXgK96IGSxUcvCGyNRoSXwSb8WR+Cjeic/i+7FcI+ZQeg7wN5NcGXQWnlze/PbPLBf/mHZ/Zs3UHFOXVlirDe0BI2oVVpI/ePz0aPNObWl0TbwUX6D/hTgUH7ACb/DVerUha8+Y/R5yki6GsNZSBQ/BJ4EoT/XgJqroHNODSlVvzKqVr9PyFPtfRoOGfzAm2PE9izHfBZc6B9HMSBNsybwHf5/31uUKHk6U0lHj6vp0B0w8I0a5l3jVWgysSkNNG7aDzFmM9d8YQ1gjRieMMXfDg+IA2Ur3LD6brTGf8/ZftD7AnI5/EfM6V8nxfZHQq9OA9yTpvIpI8k34Q7ZNPu07U06NFjluMWXK4b5OLqV2vNGslMqwN8rF1VvpzZ2nK3SVboCrSqt0n9apASX79Jxe05vs9exatp5tJqFzmTTnEv0ysp0frj/7+Q==
Dn = {(xi, yi
)}n
i=1
AAAEv3icnVNNb9NAEJ2UACV8NIULEheLtKhAFa1zoAWpUhE9wAHR71aqi2W7m3Sp41i2Exos/wFunDhwAokD4gdwQyAu/AEO/QmIY5G4cODtOA3io0FiLdszb2fevJ3ddUNfxYkQe4WhI8Wjx44PnyidPHX6zEh59Oxq3GpHnlzxWn4rWnedWPoqkCuJSny5HkbSabq+XHN3bun5tY6MYtUKlpNuKDebTiNQdeU5CSC7fNVqOsm25/jpXGanQWbMGFY6sWurSaNrq8uGld0HaqdqxszsckVUa1PTpjANNsT12oExZZhVwaNCvTHfGi28JYu2qEUetalJkgJKYPvkUIxng0wSFALbpJQeAHEogqc4RlJGJeS3gUtEOUB38G3A2+ihAXzNGzODh0o+3giZBo2LT+KV2BcfxWvxWXw/lCtlDq2ni7+b58rQHnl8funbP7Oa+Ce0/TNroOaE6jTNWhW0h4zoVXh5fufR0/2lG4vj6SXxQnyB/udiT3zACoLOV+/lglx8xuxzyMm7GMG621NwD3wSiPZ0D66gisUxDajU9TJWrX2LJvvY/zI6tPsHY44d3rME83Vw6XMQD4x0wZbPB/Af8t42uUKAE6V1LHJ1q78DLp6UUe4lXr0WB6syUFPB9pE5iHH5N8YIVsroAWPC3QigOES21j2IT7GV8Tnf/IvWO5iz8K9g3uIqJb4vEnot6vCe5J3XEXm+C3+XbZdP+1af06AxjhvrMZVwXw8upXG4sVqrmrAXzMrstd7NHaYLdJEmwDVFs3Sb5mkFSp7QG3pH74s3i41iUAzz0KFCL+cc/TKK3R8TzQYv
AAAEtHicnVPBbtNAEJ2UACUUmnJC4mKRFhWoqg0XEFIlED3AAdGmTVupDpbtbtKltmPZTmiw/APcOHHgBBIHxAdwQyAu/ACHfgLiWCQuHHg7DkE0NJVYy/bM25k3b2d3ndBTcSLEXmHsWPH4iZPjp0qnJ86cnSxPTazF7U7kyrrb9trRhmPH0lOBrCcq8eRGGEnbdzy57uzc0fPrXRnFqh2sJr1QNny7Faimcu0EkFW+avp2su3aXrqYWWmQGQuGmc7uWmrO6FnqsmFmD4FaqVqoZla5IuYFD2PYqPaNCvXHUnuq8J5M2qI2udQhnyQFlMD2yKYYzyZVSVAIrEEpPQJiUwRPcYykjErI7wCXiLKB7uDbgrfZRwP4mjdmBheVPLwRMg2aEV/EG7EvPou34qv4eShXyhxaTw9/J8+VoTX59PzKjyOzfPwT2v6TNVJzQk26wVoVtIeM6FW4eX73yfP9lZu1mfSSeCW+Qf9LsSc+YQVB97v7elnWXjD7InLyLkaw7vcVPACfBKI93YMrqGJyTAsqdb2MVWvfpLkB9r+MNu0OMebY4T1LMN8Elz4H8chIB2z5fAD/Me+tzxUCnCito8bVzcEOOHhSRrmXePVabKzKQE0F20PmKMbVA4wRrJTR34wJdyOA4hDZWvcoPsVWxue88Q+t9zBn4l/BvMlVSnxfJPSa1OU9yTuvI/J8B/4u2w6f9q0Bp0HTHDfdZyrhvlYP3s5hY+3afBX2sqBxukAXaRYU1+kW3aUlqkPAM3pHH+hj8XaxVQzymz1W6F/xc/TXKIa/AHuMBME=
AAAEtHicnVNNb9NAEJ2UACUUmnJC4mKRFhWoqk0uKUiVQPQAB0SbNm2lOli2u02XOo5lOyHB8h/gxokDJ5A4IH4ANwTiwh/g0J+AOBaJCwfejpMiPhok1rI983bmzdvZXSfwVBQLsZ8bO5Y/fuLk+KnC6YkzZyeLUxPrUbsTurLutr12uOnYkfSUL+uxij25GYTSbjme3HD2bun5ja4MI9X21+J+IBstu+mrHeXaMSCreNVs2fGua3vJUmolfmosGmYy27PUnNG31GXDTO8DtRK1WE6tYknMV6oLZVE22BDXKkOjapTnBY8SDcZyeyr3lkzapja51KEWSfIphu2RTRGeLSqToABYgxJ6AMSmEJ7iGEkpFZDfAS4RZQPdw7cJb2uA+vA1b8QMLip5eENkGjQjPolX4kB8FK/FZ/H9SK6EObSePv5OlisDa/Lx+dVv/8xq4R/T7s+skZpj2qEF1qqgPWBEr8LN8ruPnh6sXq/NJJfEC/EF+p+LffEBK/C7X92XK7L2jNmXkJN1MYR1d6DgHvgkEO3pHlxBFZNjmlCp66WsWvsmzR1i/8toU+8Pxgw7umcx5nfApc9BNDLSAVs278N/yHvb4go+TpTWUePq5uEOOHgSRrmXePVabKzKQE0F20PmKMa13xhDWAmjQ8aYu+FDcYBsrXsUn2Ir5XPe+IvWO5gz8S9h3uQqBb4vEnpN6vKeZJ3XEVm+A7/HtsOnffuQ06BpjpseMBVwX4eX0jjaWK/Ml2GvCBqnC3SRZkFRpRt0m5apDgFP6A29o/f5m/lm3s9u9lhucMXP0S8jH/wAwOsFEQ==
AAAEv3icnVO7btRAFL0JC4TlkQQaJBqLTVCAKJrdgg1IkYJIAQUi70SKg2U7s5shfsl2liyWf4COioIKJArEB9AhEA0/QJFPQJRBoqHgzPXuIh5ZJMayfe+Ze889c2fGiTyVpELsDwweKR09dnzoRPnkqdNnhkdGz64m4W7syhU39MJ43bET6alArqQq9eR6FEvbdzy55uzc0vNrLRknKgyW03YkN327GaiGcu0UkDVy1fTtdNu1vWwut7IgN2YMM5vYs9Sk0bbUZcPM7wO1MjVTza2Ripiq1aeromqwIa7XukbdqE4JHhXqjPlwdOAtmbRFIbm0Sz5JCiiF7ZFNCZ4NqpKgCNgmZfQAiE0xPMUxknIqI38XuESUDXQH3ya8jQ4awNe8CTO4qOThjZFp0Lj4JF6JA/FRvBafxfdDuTLm0Hra+DtFroys4cfnl779M8vHP6Xtn1l9NafUoGnWqqA9YkSvwi3yW4+eHizdWBzPLokX4gv0Pxf74gNWELS+ui8X5OIzZp9DTtHFGNbdjoJ74JNAtKd7cAVVTI5pQqWul7Nq7Zs02cP+l9GmvT8YC+zwnqWYb4BLn4Okb6QDtmI+gP+Q99bnCgFOlNaxyNXN3g44eDJGuZd49VpsrMpATQXbQ2Y/xuXfGGNYGaNdxpS7EUBxhGytux+fYivnc775F613MGfiX8G8yVXKfF8k9JrU4j0pOq8jinwH/h7bDp/2rR6nQWMcN9ZhKuO+di+lcbixWpuqwl4QldlrnZs7RBfoIk2Aq06zdJvmaQVKntAbekfvSzdLzVJQiorQwYFOzjn6ZZTaPwATfQYu
y = f(x) + ✏ 2 R, ✏ ⇠ N(0, ⌘2)
AAAE3nicnVM9axRBGH4TT43nRxJtBJvFS+SiIcxd4UVBCGihgppcPiF7nrubucuY/WJ3cyYu11oIYqlgpWAhNv4CGxv/gEIqbcUygo2Fz7x7OT9zgrPMzvv5vM+8M2OHrooTIbb6+vfk9u7bP3Agf/DQ4SODQ8NHF+JgPXLkvBO4QbRkW7F0lS/nE5W4cimMpOXZrly01y5q/2JLRrEK/LlkM5Q1z2r6qqEcK4GpPnR107hgNIobY8YZw5RhrNzAN0yF6VnJqm2n1fb4T45YeZnHsdz0ersotDOxbqbl9lh9qCAmypXJkigZLIhz5R2hYpQmBI8CdcZ0MNz3mkxaoYAcWiePJPmUQHbJohjfMpVIUAhbjVK6DYtFETTFMZLalEf+OuwSURasa/g3oS13rD50jRszgoNKLmaETINGxTvxQmyLt+Kl+CS+7YqVMobms4nVznJlWB+8f3z26z+zPKwJrf7I6sk5oQZNMlcF7iFb9C6cLL9199H27PnqaHpKPBOfwf+p2BJvsAO/9cV5PiOrTxj9EnKyLkaQrnUY3ACehEVrugenUcXkmCZY6nptZq11k8a7tv9FtGjjD8TMtnvPEvgbwNL3IO4ZaQMt8/vQ7/DZelzBx43SPKpc3eyegI0vZSv3ElPvxcKuDNRUkF1k9kKc+w0xgpSydQcx4W74YBwiW/PuhadYavM9r/2F6xX4TKwF+E2ukuf3IsHXpBafSdZ5HZHl29A3WLb5tq90MQ0a4biRDlIe73XnURq7CwvliRLkmVJh6mzn5Q7QCTpJRWBVaIou0zTNg8krek8f6GPuVu5e7kHuYRba39fJOUa/jNzj7xMPESw=
AAAE03icnVNNaxNBGH5To9ZYbepJ8LKYVlItZdKLIgiCHlRQ27RpC93Y7m4n6dj9YncSE5e9ehDEowdPCh7Ei7/Aixf/gEJPehWPFbx48JnZGLWxEZxldt7P533mnRk7dEUsGdvJjRzIHzx0ePRI4ejYsePjxYmx5ThoRQ6vOYEbRKu2FXNX+LwmhXT5ahhxy7NdvmJvX1H+lTaPYhH4S7Ib8rpnNX3REI4lYVov3ugal4xGuTNtnDNMHsbCDXzDFJieJbdsO6mmM785YuFlHsdyk1tpmSmntO4kc+n0erHEZpkexqBQ6Qkl6o35YCL3hkzapIAcapFHnHySkF2yKMa3RhViFMJWp4TuwmJRBE3oGE4pFZDfgp0jyoJ1G/8mtLWe1YeucGON4KCSixkh06Ap9p69ZLvsHXvFPrPv+2IlGkPx6WK1s1wero8/PLn47Z9ZHlZJW7+yhnKW1KALmqsA91Bb1C6cLL99/8nu4sXqVHKGPWdfwP8Z22FvsQO//dV5scCrTzX6VeRkXYwg3ewxuA08DovSVA/OooqpY5pgqeqlmrXSTZrp2/4X0aLOAGJm279nEv4GsNQ9iIdG2kDL/D70e/psPV3Bx41SPKq6utk/ARtfoq26l5hqLxZ2ZaCmgOwicxji0h7ECFKirT8Rpe6GD8YhshXvYXhCS6m+5/W/cL0On4m1BL+pqxT0e+Hga1Jbn0nWeRWR5dvQO1q29W3f7GMaNKnjJntIBbzXyt7XOSgsz81WIC8wGqVTdJrKgDhPl+kazVMNBF7TB/pIn/Ib+Qf5R9nLHsn1nvgJ+mPkH/8AB4EPnw==
AAAE03icnVNNaxNBGH5To9ZYbepJ8LKYVlotYZJLqyAIelBBbdNP6Ma6u520Y/eL3W1MXXL1IIhHD54UPIgXf4EXL/4BhZ70Kh4rePHgM+8m1aqJ4Cyz834+7zPvzNihq+JEiJ3cwIH8wUOHB48Ujg4dOz5cHBlajIOtyJELTuAG0bJtxdJVvlxIVOLK5TCSlme7csnevKz9S00ZxSrw55PtUNY9a91XDeVYCUyrxevbxkWjMd6aMM4Zpgxj5Qa+YSpMz0o2bDuttSd/ccTKyzyO5aY32+NCOxPrdlptT6wWS6JcnZquiIrBgjhf7QpTRqUseJSoM2aCkdwbMmmNAnJoizyS5FMC2SWLYnwrVCFBIWx1SukuLBZF0BTHSGpTAflbsEtEWbBu4r8ObaVj9aFr3JgRHFRyMSNkGjQm3ouXYle8E6/EZ/G9J1bKGJrPNlY7y5Xh6vDDk3Pf/pnlYU1o42dWX84JNWiauSpwD9mid+Fk+c37T3bnLtTG0jPiufgC/s/EjniLHfjNr86LWVl7yuhXkJN1MYJ0o8PgFvAkLFrTPTiLKibHrIOlrtdm1lo3aXLP9r+IFrX+QMxsvXuWwN8Alr4Hcd9IG2iZ34d+j8/W4wo+bpTmUePq5t4J2PhStnIvMfVeLOzKQE0F2UVmP8T53xAjSClbu4gJd8MH4xDZmnc/PMVSm+95/S9cr8FnYi3Bb3KVAr8XCb4mNflMss7riCzfht5i2ebbvraHadAox412kAp4r91HafQWFqvlCuRZQYN0ik7TOCCm6BJdpRlaAIHX9IE+0qf8nfyD/KPsZQ/kOk/8BO0b+cc/AEzgD+8=
AAAE3nicnVM9axRBGH4TT43nRxJtBJvFS+SiIcxd4UVBCGihgppcPiF7nrubucuY/WJ3cyYu11oIYqlgpWAhNv4CGxv/gEIqbcUygo2Fz7x7OT9zgrPMzvv5vM+8M2OHrooTIbb6+vfk9u7bP3Agf/DQ4SODQ8NHF+JgPXLkvBO4QbRkW7F0lS/nE5W4cimMpOXZrly01y5q/2JLRrEK/LlkM5Q1z2r6qqEcK4GpPnR107hgNIobY8YZw5RhrNzAN0yF6VnJqm2n1fb4T45YeZnHsdz0ersotDOxbqbl9lh9qCAmypXJkigZLIhz5R2hYpQmBI8CdcZ0MNz3mkxaoYAcWiePJPmUQHbJohjfMpVIUAhbjVK6DYtFETTFMZLalEf+OuwSURasa/g3oS13rD50jRszgoNKLmaETINGxTvxQmyLt+Kl+CS+7YqVMobms4nVznJlWB+8f3z26z+zPKwJrf7I6sk5oQZNMlcF7iFb9C6cLL9199H27PnqaHpKPBOfwf+p2BJvsAO/9cV5PiOrTxj9EnKyLkaQrnUY3ACehEVrugenUcXkmCZY6nptZq11k8a7tv9FtGjjD8TMtnvPEvgbwNL3IO4ZaQMt8/vQ7/DZelzBx43SPKpc3eyegI0vZSv3ElPvxcKuDNRUkF1k9kKc+w0xgpSydQcx4W74YBwiW/PuhadYavM9r/2F6xX4TKwF+E2ukuf3IsHXpBafSdZ5HZHl29A3WLb5tq90MQ0a4biRDlIe73XnURq7CwvliRLkGVGYOtt5uQN0gk5SEVgVmqLLNE3zYPKK3tMH+pi7lbuXe5B7mIX293VyjtEvI/f4OxK/ESs=
µ : X ! R
AAAEu3icnVM9bxNBEB0HA8F8xIEGieaEE0AIWXsucIiEFAkKKBCJEyeWcpZ1d1nbl9yX7s5Owsl/AImaggokCkRJCzQ0/AGK/AREGSQaCt7O2UZ8xEjs6e5m3s68eTu7a4WuEydCHOSmjuWPnzg5fapw+szZczPF2fPrcdCLbFm3AzeIGpYZS9fxZT1xElc2wkianuXKDWvnjprf6MsodgJ/LdkPZdMzO77TdmwzAdQqXjW83qJmeGbStU03bQw0I3I63cSMomA3wy0rrQ1axZIoV6oLutA1NsStysioanpZ8CjRcCwHs7n3ZNAWBWRTjzyS5FMC2yWTYjybpJOgEFiTUtoGYlIEz+EYSQMqIL8HXCLKBLqDbwfe5hD14SvemBlsVHLxRsjUaF58Eq/EofgoXovP4vuRXClzKD37+FtZrgxbM48vrn77Z5aHf0Ldn1kTNSfUpgXW6kB7yIhahZ3l9x89PVxdrM2nV8QL8QX6n4sD8QEr8Ptf7ZcrsvaM2e8iJ+tiBOvBUMFD8EkgylM9uI4qBsd0oFLVG7Bq5Rt0Y4z9L6NJe38wZtjRPUsw3waXOgfxxEgLbNm8D3+X99bjCj5OlNJR4+rGeAcsPCmj3Eu8ai0mVqWhpgPbReYkxrXfGCNYKaMjxoS74UNxiGylexKfw9aAz3nzL1rvY87Av4R5g6sU+L5I6DWoz3uSdV5FZPkW/D22LT7tW2NOjeY4bm7IVMB9HV1K7WhjvVLWYa/opaWbw5s7TZfoMl0DV5WW6B4tUx1KntAbekvv8rfzdn4772ahU7lhzgX6ZeR7PwA0HgWK
 : X2 ! R
AAAEwnicnVO7bhNBFL0JBoJ5xIEGiWaFE4QQssYuSEgViRRQIBLnZSlrrNnN2N5kX+xunITFP0CNREEFEgXiDyhoQsEPUOQTEGWQaCg4c3djxCNGYla7e++Ze889c2fGCl0nToQ4GBk9UTh56vTYmeLZc+cvjJcmLq7GwXZkqxU7cIOoYclYuY6vVhIncVUjjJT0LFetWVt39PxaT0WxE/jLyV6omp7s+E7bsWUCqFWqmFsyDOWsYXoy6drSTRv9h2mtb5iR0+kmMoqCnWzOstJ6v1Uqi0pteqYqqgYb4nbtyJg2qhXBo0z5WAgmRt6TSRsUkE3b5JEinxLYLkmK8axTlQSFwJqU0iYQSRE8h2MU9amI/G3gClES6Ba+HXjrOerD17wxM9io5OKNkGnQlPgk3ohD8VG8FZ/F92O5UubQevbwt7JcFbbGn15e+vbPLA//hLo/s4ZqTqhNM6zVgfaQEb0KO8vvPX5+uDRbn0qviVfiC/S/FAdiHyvwe1/t14uq/oLZ55GTdTGCdT9X8AB8Coj2dA9uoIrJMR2o1PX6rFr7Jt0cYP/LKGn3D8YMO75nCebb4NLnIB4aaYEtm/fh7/DeelzBx4nSOupc3RzsgIUnZZR7iVevRWJVBmo6sF1kDmNc/o0xgpUyesSYcDd8KA6RrXUP43PY6vM5b/5F6z3MmfiXMW9ylSLfFwW9JvV4T7LO64gs34K/y7bFp31jwGnQJMdN5kxF3NejS2kcb6zWKlXYi9Xy3K385o7RFbpK18E1TXN0lxZoBUqe0Tvapw+F+cJm4VEhzkJHR/KcS/TLKDz5AQkuCIM=
AAAEt3icnVM9b9NQFL0pAUooNGVCYrFIixBC0UuXok5IdIAB0aZNG6kO0bP7krjxF/ZL2mL5DzAjMTCBxID4BwwsZeAPMPQnIMYisTBw3rMJoqFB4lm27z333nPP+7JC14klY0eFqTPFs+fOT18oXZy5dHm2PDezGQeDyBYNO3CDqGnxWLiOLxrSka5ohpHgnuWKLat/T8W3hiKKncDfkAehaHm86zsdx+YSULtcNfs8DPmyYXpc9mzuJs30cbKYGmbkdHuSR1Gwl8UsK6mn7XKFVZkexrhRy40K5WM1mCt8IJN2KCCbBuSRIJ8kbJc4xXi2qUaMQmAtSmgXCKcInqNzBKVUQv0AuEAWB9rHtwtvO0d9+Io31gw2Orl4I1QatMA+s7fsmH1i79gX9uNUrkRzKD0H+FtZrQjbs8+urn//Z5WHv6Te76qJmiV16I7W6kB7qBE1CzurHz59cby+XF9IbrDX7Cv0v2JH7BAz8Iff7Ddrov5Ss6+gJlvFCNbDXMEj8AkgylNrcAtdTJ3ThUrVL9WqlW/S7RH2v4yc9scYM+z0NZOId8ClzkE8MdMCWxb34e/pvfV0Bx8nSumo6+7maAcsPIlG9VriVXPhmJWBng5sF5WTGDdOMEawEo3+YpR6NXwoDlGtdE/ic7SV6nPe+ovWB4iZ+FcQN3WXkr4vAnpNGuo9yVZeZWT1Fvx9bVv6tO+MOA2a13nzOVMJ97V28naOG5uL1RrsNUbTdI2u001QLNFduk+r1ICA5/SeDuljcaW4W3yS3eypQn7Fr9Afoxj/BFfbBxI=
AAAEt3icnVO7bhNREJ0EQ4IJxKFColnhBCGErGs3DqmQSAEFInFelrLGuru5tjfeF7sbJ2HlH6BGoqACiQLxBxQ0oeAHKPIJiDJINBScO+sY8bCRuKvdnTkzc+bclxW6TpwIcTwxeSZ39tzU9Pn8hZmLl2YLczObcbAX2WrDDtwgqlsyVq7jq43ESVxVDyMlPctVW1b3ro5v9VQUO4G/nhyGquHJtu+0HFsmgJqFktmVYSiXDNOTSceWblrvP0orfcOMnHYnkVEU7Gcxy0pr/WahKEqV6mJZlA02xO3KqVE1yiXBo0iDsRLMTbwnk3YoIJv2yCNFPiWwXZIU49mmMgkKgTUopV0gkiJ4Duco6lMe9XvAFbIk0C6+bXjbA9SHr3ljZrDRycUbodKgBfFJvBEn4qN4Kz6L7yO5UubQeg7xt7JaFTZnn15Z+/bPKg//hDo/q8ZqTqhFi6zVgfaQET0LO6vvPXl+srZUW0ivi1fiC/S/FMfiCDPwe1/t16uq9oLZl1GTrWIE68FAwUPwKSDa02twE11MzmlDpe7XZ9XaN+nWEPtfRkkHfzBm2Og1SxBvgUufg3hspgW2LO7D3+e99biDjxOlddS4uzncAQtPyiivJV49F4lZGejpwHZROY5x/TfGCFbK6CljwqvhQ3GIaq17HJ/DVp/PeeMvWu8jZuJfRNzkLnm+Lwp6TerxnmQrrzOyegv+AdsWn/adIadB85w3P2DK476eXkpjtLFZKZVhrwqapqt0jW6Aokp36B6t0AYEPKN3dEQfcsu53dzj7GZPTgyu+GX6ZeTiH506B2I=
AAAEwnicnVNNb9NAEJ0WAyV8NIULEheLtAghFG1yoKWnSvQAB0SbfkWqQ7R2N4lbxza2m7aY/AHOSBw4gcQB8Q84cCkH/gCH/gTEsUhcOPB27Abx0SCxlu2ZtzNv3s7u2qHnxokQByOjJ4yTp06PnSmcPXf+wnhx4uJqHGxHjlpxAi+I6raMlef6aiVxE0/Vw0jJru2pNXvrjp5f66kodgN/OdkLVaMr277bch2ZAGoWy9aWDEM5a1pdmXQc6aX1/sO02jetyG13EhlFwU42Z9tprd8slkS5Oj1TERWTDXG7emRMm5Wy4FGifCwEEyPvyaINCsihbeqSIp8S2B5JivGsU4UEhcAalNImEEkRPJdjFPWpgPxt4ApREugWvm146znqw9e8MTM4qOThjZBp0pT4JN6IQ/FRvBWfxfdjuVLm0Hr28LezXBU2x59eXvr2z6wu/gl1fmYN1ZxQi2ZYqwvtISN6FU6W33v8/HBptjaVXhOvxBfofykOxD5W4Pe+Oq8XVe0Fs88jJ+tiBOt+ruAB+BQQ7eke3EAVi2PaUKnr9Vm19i26OcD+l1HS7h+MGXZ8zxLMt8Clz0E8NNIGWzbvw9/hve1yBR8nSuuocXVrsAM2npRR7iVevRaJVZmo6cL2kDmMcfk3xghWyugRY8Ld8KE4RLbWPYzPZavP57zxF633MGfhX8K8xVUKfF8U9FrU4z3JOq8jsnwb/i7bNp/2jQGnSZMcN5kzFXBfjy6lebyxWi1XYC+K0tyt/OaO0RW6StfBNU1zdJcWaAVKntE72qcPxryxaTwy4ix0dCTPuUS/DOPJDwjeCII=
Y = yi 2 Rn
AAAEsHicnVNNb9NAEJ2UACV8NIULEhws0iKEULTOgRQkpEpwgAOiTb+C6hDZ7iZ169iW7YQGKxeO/AEOnEBCAnGCH8CFC3+AQ38C4lgkLhx4O3aC+EiQWMv2zNuZN29nd63AdaJYiP3c1KH84SNHp48Vjp84eWqmOHt6PfK7oS3XbN/1w7plRtJ1PLkWO7Er60EozY7lyg1r96aa3+jJMHJ8bzXuB7LRMdue03JsMwbULJ6/r93Q+s3EGWiG42lGx4y3LSupDR4k3qBZLIlypbqgC11jQ1yrDI2qppcFjxJlY8mfzb0ng7bIJ5u61CFJHsWwXTIpwrNJOgkKgDUooR0gJoXwHI6RNKAC8rvAJaJMoLv4tuFtZqgHX/FGzGCjkos3RKZG8+KTeC0OxEfxRnwW38dyJcyh9PTxt9JcGTRnnpxd+fbPrA7+MW3/zJqoOaYWLbBWB9oDRtQq7DS/9+jpwcr12nxyUbwQX6D/udgXH7ACr/fVfrksa8+Y/RZy0i6GsO5mCu6BTwJRnurBZVQxOKYNlaregFUr36ArI+x/GU3a+4Mxxcb3LMZ8C1zqHEQTIy2wpfMe/Ie8tx2u4OFEKR01rm6MdsDCkzDKvcSr1mJiVRpqOrBdZE5iXP2NMYSVMDpkjLkbHhQHyFa6J/E5bA34nDf+ovUO5gz8S5g3uEqB74uEXoN6vCdp51VEmm/B32Pb4tO+NeLUaI7j5jKmAu7r8FJq4431SlmHvayXFq9mN3eaztEFugSuKi3SbVqiNSh5TK/oLb3LV/L1fDNvpqFTuSznDP0y8js/ABEwAF8=
AAAEpXicnVNNb9NAEJ2UACUUmnJCgoNFWoQQqja9gJCQkOAAB0SbNm1QHSLb3aRL/SXbCQ2WLxz5Axw4gYQE4gQ/gAsX/gCH/gTEsUhcOPB2bIJoaZC6lu2ZtzNv3s7u2qGr4kSIndLEkfLRY8cnT1ROTp06PV2dmVqNg37kyKYTuEHUsq1YusqXzUQlrmyFkbQ825Vr9tYtPb82kFGsAn8lGYay7Vk9X3WVYyWAOtXzD4wbxrCTqswwlW+YnpVs2nbayB6mftap1sS84GHsN+qFUaNiLAYzpY9k0gYF5FCfPJLkUwLbJYtiPOtUJ0EhsDal9AiIRRE8xTGSMqogvw9cIsoCuoVvD956gfrwNW/MDA4quXgjZBo0J76It2JXfBbvxFfx80CulDm0niH+dp4rw870s7PLP/6b5eGf0OafrLGaE+rSNdaqoD1kRK/CyfMHT57vLl9vzKUXxSvxDfpfih3xCSvwB9+d10uy8YLZbyMn72IE616h4D74JBDt6R5cRhWTY3pQqetlrFr7Jl0ZYYdltGh7H2OOHdyzBPNdcOlzEI+NtMGWz/vwH/PeelzBx4nSOhpc3RztgI0nZZR7iVevxcKqDNRUsF1kjmNc2cMYwUoZ/c2YcDd8KA6RrXWP41NsZXzO2//QehdzJv41zJtcpcL3RUKvSQPek7zzOiLPt+Fvs23zad8YcRo0y3GzBVMF97W+93buN1YX5uuwlwRN0jm6QJdAcZVu0h1apCYEPKU39J4+lBfKrXInv9kTpeKKn6G/Rtn6BbYt/vE=
AAAEpXicnVNNb9NAEJ2UACUUmnJCgoNFWoQQija5pCAhIcEBDog2bdqgOkS2u0lN/SXbCQ2WLxz5Axw4gYQE4gQ/gAsX/gCH/gTEsUhcOPB2nATxkVRiLdszb2fevJ3dNQPHjmIh9nMzR/JHjx2fPVE4OXfq9HxxYW4j8nuhJRuW7/hh0zQi6diebMR27MhmEErDNR25ae7eVPObfRlGtu+tx4NAtlyj69kd2zJiQO3i+fvadW3QTuxU021P010j3jHNpJ4+SLy0XSyJcrW2XBEVjQ1xtToyalqlLHiUaDhW/IXcB9Jpm3yyqEcuSfIohu2QQRGeLaqQoABYixJ6CMSgEJ7NMZJSKiC/B1wiygC6i28X3tYQ9eAr3ogZLFRy8IbI1GhJfBZvxIH4JN6KL+LHRK6EOZSeAf5mliuD9vzTs2vfD81y8Y9p51fWVM0xdWiZtdrQHjCiVmFl+f3Hzw7WrtWXkovipfgK/S/EvviIFXj9b9arVVl/zuy3kJN1MYR1d6jgHvgkEOWpHlxGFZ1julCp6qWsWvk6XRlj/8to0N5fjBk2uWcx5jvgUucgmhppgi2b9+A/4r11uYKHE6V01Lm6Pt4BE0/CKPcSr1qLgVVpqGnDdpA5jXH9D8YQVsLoiDHmbnhQHCBb6Z7GZ7OV8jlv/UPrHczp+Jcwr3OVAt8XCb069XlPss6riCzfhL/HtsmnfXvMqdEixy0OmQq4r6NLqU02NqrlCuxVQbN0ji7QJVDU6AbdphVqQMATek3v6H2+mm/m29nNnskNr/gZ+m3kjZ/7jP9B
AAAEsHicnVNNb9NAEJ0UAyV8NIULEhws0iKEULTJgRQkpEpwgAOiTb+C6hDZ7iZ1azuW7YQGKxeO/AEOnEBCAnGCH8CFC3+AQ38C4lgkLhx4O3aC+EiQWMv2zNuZN29nd63AdaJYiP3c1CHt8JGj08fyx0+cPDVTmD29HnW6oS3X7I7bCeuWGUnX8eVa7MSurAehND3LlRvW7k01v9GTYeR0/NW4H8iGZ7Z9p+XYZgyoWTh/X7+h95uJM9ANx9cNz4y3LSupDR4k/qBZKIpSpbpQFmWdDXGtMjSqerkkeBQpG0ud2dx7MmiLOmRTlzyS5FMM2yWTIjybVCZBAbAGJbQDxKQQnsMxkgaUR34XuESUCXQX3za8zQz14SveiBlsVHLxhsjUaV58Eq/Fgfgo3ojP4vtYroQ5lJ4+/laaK4PmzJOzK9/+meXhH9P2z6yJmmNq0QJrdaA9YEStwk7ze4+eHqxcr80nF8UL8QX6n4t98QEr8Htf7ZfLsvaM2W8hJ+1iCOtupuAe+CQQ5akeXEYVg2PaUKnqDVi18g26MsL+l9GkvT8YU2x8z2LMt8ClzkE0MdICWzrvw3/Ie+txBR8nSumocXVjtAMWnoRR7iVetRYTq9JR04HtInMS4+pvjCGshNEhY8zd8KE4QLbSPYnPYWvA57zxF613MGfgX8S8wVXyfF8k9BrU4z1JO68i0nwL/h7bFp/2rRGnTnMcN5cx5XFfh5dSH2+sV0pl2MuiuHg1u7nTdI4u0CVwVWmRbtMSrUHJY3pFb+mdVtHqWlMz09CpXJZzhn4Z2s4PEOAAXg==
ki = (x, xi
), k0
i
= (x0, xi
)
AAAEyHicnVNNaxNRFL2po9b40VQ3gpvBtLZKCC9BTBWEgi5UENu0aQudEGamL3HMZDLMTNLEkI1L/4CgKwUX4n8QwY26d9GfIC4ruHHheXeS+FETwTfMzL3n3nvueV+W7zphJMReYuqQdvjI0eljyeMnTp6aSc2e3gibrcCWJbvpNoMtywyl63iyFDmRK7f8QJoNy5WbVv2Gim+2ZRA6TW896vqy3DBrnlN1bDMCVEldrld6Tl+/rht10/fNxU5G7yjkYkbnyMIvoYVhrJJKi2y+sJQTOZ0NcTU/NAp6Lit4pGkwVpqziTdk0A41yaYWNUiSRxFsl0wK8WxTjgT5wMrUowdATArgOZwjqU9J1LeAS2SZQOv41uBtD1APvuINmcFGJxdvgEqd5sUn8Ursi/fitfgsvo/l6jGH0tPF34prpV+ZeXx27ds/qxr4R3T/Z9VEzRFVaYm1OtDuM6JmYcf17YdP9teuFed7F8QL8QX6n4s98Q4z8Npf7ZersviM2W+iJl7FANbdgYJ74JNAlKfW4BK6GJxTg0rVr8+qlW9QZoT9L6NJnQOMMTZ+zSLEq+BS5yCcmGmBLY578Hd5bxvcwcOJUjqK3N0Y7YCFp8coryVeNRcTs9LR04HtonIS4/ofjAGsHqNDxohXw4NiH9VK9yQ+h60+n/PyX7TeRszAP424wV2SfF8k9BrU5j2JV15lxPUW/A7bFp/2nRGnTnOcNzdgSuK+Di+lPt7YyGdzsFdz6eUrg5s7TefoPC2Cq0DLdItWqAQlT+ktfaCP2h3N13a1bpw6lRjUnKHfhvboB0vcCEs=
K 2 Rn⇥n, Kij = (xi, xj
)
AAAE0HicnVNNb9NAEJ2UACV8tIULEheLtKigKtrkQAsCqRIcQBWiTT+luo1sd5NuajuW7YYUy0Jc+wc4cKISB8R/4MKlV5A49CcgjkXiwoG347SIjwQJW/bOvJ1583Z21w5cFcVCHOQGTuRPnjo9eKZw9tz5C0PDIxeXotZ26MhFp+W2whXbiqSrfLkYq9iVK0EoLc925bK9dU/PL7dlGKmWvxDvBHLNsxq+qivHigHVhu/MGKbyDdOz4k3bTqrpegIvVp6MDD+dMGZqiWqmxl3D3LKCwBrvwAeMoZlerw0XRakyOVUWZYMNcatyZEwa5ZLgp0jdZ7Y1kntHJm1QixzaJo8k+RTDdsmiCO8qlUlQAGyNEmoCsSiEpzhGUkoF5G8Dl4iygG7h34C32kV9+Jo3YgYHlVx8ITINGhOfxBtxKPbFW/FZfO/JlTCH1rOD0c5yZVAb2r08/+2fWR7GmDZ/ZvXVHFOdplirgvaAEb0KJ8tvP31xOH+7OpZcE3viC/S/EgfiPVbgt786r+dk9SWz30dO1sUQ1qOugsfgk0C0p3twA1VMjmlApa6XsmrtmzRxjP0vo0WdPxgzrHfPYszXwaXPQdQ30gZbNu/Df8J763EFHydK66hydfN4B2y8CaPcS3x6LRZWZaCmgu0isx/jwm+MIayE0SPGmLvhQ3GAbK27H59iK+VzvvYXrQ8xZ2IsYt7kKgW+LxJ6TWrznmSd1xFZvg2/w7bNp33jmNOgUY4b7TIVcF+PLqXR21iqlMqw58rF6ZvdmztIV+gqjYNrkqbpAc3SIpTs0T59oI/5ar6Tf5Z/noUO5Lo5l+iXJ7/7A4qcDLw=

View Slide

Set up｜A brief review of GP based BO
事後分布の計算
n ある観測点が与えられたときの事後分布を考える
n 事後分布の平均関数：
n 事後分布のカーネル：
探索点の決定と獲得関数
n 事後分布のもとで獲得関数を最大化する点を次に探索
n 獲得関数として，Expected improvementを採用
- ガウス過程のもとであれば，期待値部分は明に計算可能
7
Dn
AAAEo3icnVPNbtNAEJ4UQ0v4SQoXJC4RaRBCKNoUxN+pEj2AEKJNk7ZSHUW2u0lN/SfbCS2WX4AX4NATiB4Q78CFCy/AoY+AOBaJCwe+HbtBUBIk1rI98+3MN9/O7pqBY0exEAeFqRPayVPTM6eLZ86eO18qz15YjfxBaMm25Tt+uG4akXRsT7ZjO3bkehBKwzUduWZuP1Dza0MZRrbvteLdQHZco+/ZPdsyYkDdckl3jXjLMpxkMe0mXtotV0X95q17Yl5UjhuNuuBRpXws+bOFD6TTJvlk0YBckuRRDNshgyI8G9QgQQGwDiX0DIhBITybYySlVET+ALhElAF0G98+vI0c9eAr3ogZLFRy8IbIrFBNfBbvxKH4JN6LL+LHWK6EOZSeXfzNLFcG3dLLSyvf/5nl4h/T1q+siZpj6tFd1mpDe8CIWoWV5Q9fvDpcud+sJVfFG/EV+l+LA/ERK/CG36z9ZdncY/ZF5GRdDGE9yRU8BZ8EojzVg+uoonNMHypVvZRVK1+nGyPsfxkN2jnGmGHjexZjvgcudQ6iiZEm2LJ5D/5z3luXK3g4UUpHk6vrox0w8SSMci/xqrUYWFUFNW3YDjInMbb+YAxhJYweMcbcDQ+KA2Qr3ZP4bLZSPuedv2h9hDkd/yrmda5S5PsioVenIe9J1nkVkeWb8HfYNvm0b444KzTHcXM5UxH39ehSVsYbq/P1hqg3lkV14XZ+c2foMl2ha+C6Qwv0kJaozTd4j97SvlbTHmtNrZWFThXynIv029A6PwH7KfuQ
f | Dn
AAAEqnicnVPNbtNAEJ4UAyX8NAUJIXGxSIsQqqJNQfydKtEDHBBt2rRFcRTZziY19Z9sJ7SYvAAvwIETlTig8gxcuPACHPoIiGORuHDg27EbBCVBwpa9M9/OfPPt7K4Vuk6cCLFfmDimHT9xcvJU8fSZs+emStPn1+KgF9mybgduEG1YZixdx5f1xElcuRFG0vQsV65bW/fV/HpfRrET+KvJTiibntn1nY5jmwmgVuliRzc8p42fmWzappsuDlqpP2iVyqJy4+ZdMS/0o0a1IvgpU/4sBdOFD2RQmwKyqUceSfIpge2SSTHeBlVJUAisSSk9BWJSBM/hGEkDKiK/B1wiygS6hX8XXiNHffiKN2YGG5VcfBEydZoVn8U7cSA+iT3xRfwYyZUyh9Kzg9HKcmXYmnp5aeX7P7M8jAlt/soaqzmhDt1hrQ60h4yoVdhZfv/5q4OVe7XZ9KrYFV+h/43YFx+xAr//zX67LGuvmX0ROVkXI1iPcgWPwSeBKE/14DqqGBzThUpVb8CqlW/Q3BD7X0aTto8wZtjoniWY74BLnYN4bKQFtmzeh/+M99bjCj5OlNJR4+rGcAcsvCmj3Et8ai0mVqWjpgPbReY4xtU/GCNYKaOHjAl3w4fiENlK9zg+h60Bn/PmX7Q+xJyBsYx5g6sU+b5I6DWoz3uSdV5FZPkW/G22LT7t7SGnTjMcN5MzFXFfDy+lPtpYm69URaW6LMoLt/KbO0mX6QpdA9dtWqAHtER1KHlBu7RH77U5raY90RpZ6EQhz7lAvz1a+yeT9/4S
µn
(x) = k>
⇣
K + ⌘2I
⌘ 1
Y
AAAEzXicnVPLahRBFL2JrcbxkUQ3gpvCSSTxEWpGUSMIAV0YRPJ+SHpm6O7UTNr0i+6aMbFtt4I/4MKFKLgQ/8GNCK505SKfIC4juHHhqepJROOMYDXdfe+pe889davKjjw3kZxv9fTuM/YfONh3qHD4yNFj/QODxxeTsBk7YsEJvTBetq1EeG4gFqQrPbEcxcLybU8s2es31PxSS8SJGwbzcjMSFd9qBG7ddSwJqDYwbvrNWhpkIxuj7Dpbr6amDKOMmZ6oyxF2m51jppBWtcwmmRm7jTU5Wk0vlDJ2l9UGinzs4qVxXuZsr1Ea43oUqT2mw8Get2TSKoXkUJN8EhSQhO2RRQmeFSoRpwhYhVK6B8SiGJ6rYwRlVEB+E7hAlAV0Hd8GvJU2GsBXvIlmcFDJwxsjk9Ew/8xf823+gb/hX/iPjlyp5lB6NvG381wR1fqfnJz7/s8sH39Ja7+yumqWVKerWqsL7ZFG1CqcPL/14On23LXZ4fQMf8m/Qv8LvsXfYQVB65vzakbMPtPsN5GTdzGGdaetYAp8AojyVA/OooqpYxpQqeplWrXyTTq/i/0vo0UbexhzrHPPJObr4FLnIOkaaYMtnw/g39d76+sKAU6U0jGrq5u7O2DjSTWqe4lXrcXCqhhqurA9ZHZjnP+DMYaVanSHUepuBFAcIVvp7sbnaivT57zyF62TmDPxL2Le1FUK+r4I6DWppfck77yKyPNt+BvatvVpX93lZDSk44baTAXc151LyTobi+WxEuwZXpy43L65fXSKTtMIuK7QBN2iaVqAkuf0nj7SJ2PKaBoPjUd5aG9PO+cE/TaMxz8B2KwJiQ==
n
x, x0 =  x, x0 k>
⇣
K + ⌘2I
⌘ 1
k0
AAAFAHicnVO/b9NAFH4pAUr40RQWJJYTaWmBtroUxC8JqRIMVAjR361UJ5HtXlITx7bsS0ixsjDyDzAwgYQEYkCsDCwszEgM/RMQY5FgYOC7c5IK2gSJs2y/99173/vu3Z0VuE4kOd9ODRxIHzx0ePBI5uix4yeGssMnVyK/Htpi2fZdP1yzzEi4jieWpSNdsRaEwqxZrli1qrfU/GpDhJHje0tyKxCFmlnxnLJjmxJQKesaVTMIzFLstZjhirIcZ80J1hxjRuhUNuV5dpMlEfvPTrJqMTakH3Sz77KLzBDSLE6z2U5YMZ7Mt1h1rJTN8alLl6/zac72GvkprkeO2mPOH059IIM2yCeb6lQjQR5J2C6ZFOFZpzxxCoAVKKYHQEwK4Tk6RlCLMsivAxeIMoFW8a3AW2+jHnzFG2kGG5VcvCEyGY3yL/w13+Gf+Bv+lf/qyRVrDqVnC38ryRVBaejJ6cUf/8yq4S9pczerr2ZJZbqmtTrQHmhErcJO8huPnu4s3lgYjc/xF/wb9D/n2/wjVuA1vtsv58XCM81+GzlJF0NY99oK7oNPAFGe6sEFVDF0TAUqVb2WVq18gya62P8ymtTcw5hgvXsmMV8GlzoHUd9IC2zJvAf/od7bmq7g4UQpHQu6utHdAQtPrFHdS7xqLSZWxVDTge0isx/j0l+MIaxYox1GqbvhQXGAbKW7H5+jrZY+54V9tM5izsA/h3lDV8no+yKg16CG3pOk8yoiybfgN7Vt6dO+0eVkNKLjRtpMGdzXzqVkvY2V6ak87Hmem7nSvrmDdIbO0ji4rtIM3aE5WoaSz/QzlUoNpB+nX6Xfpt8loQOpds4p+mOk3/8G51oaBA==
'
AAAEmXicnVNNb9NAEJ0UAyV8tIULUi8RaRFCKNoUxNepCA4FCdEmTVOpjirb3aRL/SXbCS1W/gA3Tkj0BBIHxH/gwoU/wKE/AXEsEhcOvB27QVCSSqxle+btzJu3s7t26Ko4EWKvMHbMOH7i5Pip4ukzZ89NTE6dX4mDbuTIhhO4QbRqW7F0lS8biUpcuRpG0vJsVzbtrft6vtmTUawCfznZCWXLszq+aivHSgA1zZ4VhZtqfbIsKtdv3BFzonTYqFYEjzLlYzGYKnwkkzYoIIe65JEknxLYLlkU41mjKgkKgbUopadALIrgKY6R1Kci8rvAJaIsoFv4duCt5agPX/PGzOCgkos3QmaJZsUX8V7si8/ig/gqfg7lSplD69nB385yZbg+8eJi/ceRWR7+CW3+zhqpOaE23WatCtpDRvQqnCy/9/zVfv1ubTa9LN6Kb9D/RuyJT1iB3/vuvFuStV1mf4CcrIsRrMe5gifgk0C0p3twFVVMjulApa7XZ9XaN+naAPtfRou2DzFm2PCeJZhvg0ufg3hkpA22bN6H/4z31uMKPk6U1lHj6uZgB2w8KaPcS7x6LRZWVUJNBdtF5ijG5b8YI1gpoweMCXfDh+IQ2Vr3KD7FVp/PeesfWh9izsS/jHmTqxT5vkjoNanHe5J1Xkdk+Tb8bbZtPu0bA84SzXDcTM5UxH09uJSl4cbKXKUKe0mU52/mN3ecpukSXQHXLZqnBVqkBu/NS3pNu8a0cc9YMB5loWOFPOcC/TGM+i/xL/e+
't
(x) = E
h
max
n
0, f(x) arg maxi t 1
f(xi
)
o
| Dn
i
AAAFIHicnVPLbtNAFL0pAUp4NIVNJTZD06IWtdWkIF4SUiWKBAtE361UV9HYnSRDbcfYbkix/AOIPQtWVGKB+Ac2bLpDLFiUP0DsKBIbFpwZp0FQGiRs2XPvufeee+ZlB66KYs53cz1H8kePHe89UTh56vSZvmL/2aWosRk6ctFpuI1wxRaRdJUvF2MVu3IlCKXwbFcu2xu3dXy5KcNINfyFeCuQa56o+aqqHBEDqhSfWU0RBnVVSeJ0pDXKbjHLE3HdtpM7KbNcWY3xt0PhbOhAax9K+Bir6vxxZomwhkglUYxZNfmIxWy8nOpoRY0yK1S1OgpA5qn1jNwRbjKdVhI/7YRD06JSLPGJy1du8EnODhrlCW6eErWfmUZ/7i1ZtE4NcmiTPJLkUwzbJUER3lUqE6cA2Bol9BCIoBCeMjmSUiqgfhO4RJYAuoF/Dd5qG/Xha97IMDjo5OILUclomH/kr/ke3+Fv+Gf+41CuxHBoPVsY7axWBpW+pwPz3/9Z5WGMqf6rqqvmmKp03WhV0B4YRM/CyeqbT57vzd+cG04u8m3+Bfpf8l3+DjPwm9+cV7Ny7oVhn0ZNtoohrPttBQ/AJ4FoT6/BJXSxTE4NKnW/1KjWvkVjHex/GQW1DjBm2OFrFiNeBZc+B1HXTBtsWdyH/9jsrWc6+DhRWsec6W51dsDGmxjUrCU+PReBWTH0VLBdVHZjXPiDMYSVGHSfMTar4UNxgGqtuxufMlZqzvnaX7TeQ8zCWELcMl0K5r5I6LWoafYkW3mdkdXb8FvGts1pX+9wMhoyeUNtpgLu6/6lZIcbS5MTZdiz5dLU1fbN7aXzNEgj4LpGU3SXZmgRSr7mBnIXcoP57fxO/n3+Q5bak2vXnKPfnvynn2lPKB8=
AAAFFXicnVM9b9RAEJ0LB4QjkIQqEs2SD5SgJNqjASEhIREkKBD5TqQ4Oq19e3dLbJ+xN8cFy38A0VNQEYkC8R9oaNIhCor8BERHkGgoeLu+HCIhh4Qte2fezLx5++VGvko05/uFvlPF02fO9p8rnR+4cHFwaHhgNWlux55c8Zp+M153RSJ9FcoVrbQv16NYisD15Zq7ddfE11oyTlQzXNY7kdwMRD1UNeUJDagy9MJpiThqqEqqs8n2FLvNnEDohuum9zLm+LKm8Xdj4W2ZQPsQSvk0q5n8GeaIuI5IJVWMOXX5hGk2U85MtKKmmBOregMFIAtUNSf3hJ/OZZU0zLrh2LaoDI3xWW4fdtwod4wx6jzzzeHCe3KoSk3yaJsCkhSShu2ToATvBpWJUwRsk1J6DERQDE/ZHEkZlVC/DVwiSwDdwr8Ob6ODhvANb2IZPHTy8cWoZDTBP/O3/IDv8Xf8C/95IldqOYyeHYxuXiujyuDzkaUf/6wKMGpq/K7qqVlTjW5arQraI4uYWXh5fevZy4OlW4sT6VW+y79C/2u+zz9gBmHru/dmQS6+suxzqMlXMYb1sKPgEfgkEOOZNbiGLo7NqUOl6ZdZ1cZ3aLqL/S+joPYxxhw7ec004jVwmXOQ9Mx0wZbHQ/hP7d4GtkOIE2V0LNruTncHXLypRe1a4jNzEZgVQ08F20dlL8blI4wxrNSih4zarkYIxRGqje5efMpamT3nm3/R+gAxB+MY4o7tUrL3RUKvQy27J/nKm4y83oXftrZrT3u1y8lo3OaNd5hKuK/lo7fzuLF6fbYMe4FTP12mUZoExQ26Q/dpnlYg4FthpHClMFrcLe4VP+Y3u6/QueKX6I+n+OkXu3Qmag==
AAAFFXicnVPLahRBFL0TR41jNImrgJsyD0kkCTVR8AGCYARdiHknkA5Ddadmpkx3T9tdGSc2/QPi3oUrAy7Ef3DjJjtx4SKfIO6M4MaFp6onIz4ygt10173n3nvuqZcb+SrRnO8Xeo4Vj5842XuqdLrvzNn+gcG+laSxHXty2Wv4jXjNFYn0VSiXtdK+XItiKQLXl6vu1m0TX23KOFGNcEnvRHIjELVQVZUnNKDKwDOnKeKoriqpzsZbE+wmcwKh666b3smY48uqxt+NhbdlAq1DKOWTrGryp5gj4hoilVQx5tTkI6bZVDkz0YqaYE6sanUUgCxQmzm5J/x0NqukYdYJx7ZFZWCET1++cp3PcPanUZ7m9hmh9jPXGCy8JYc2qUEebVNAkkLSsH0SlOBdpzJxioBtUEoPgQiK4SmbIymjEuq3gUtkCaBb+NfgrbfREL7hTSyDh04+vhiVjMb4R/6aH/A9/oZ/4t+P5Eoth9Gzg9HNa2VU6X86tPjtn1UBRk31n1VdNWuq0jWrVUF7ZBEzCy+vbz55frB4Y2Esvch3+Wfof8n3+TvMIGx+9V7Ny4UXln0WNfkqxrDutxU8AJ8EYjyzBpfQxbE5Nag0/TKr2vgOTXaw/2UU1PqDMceOXjONeBVc5hwkXTNdsOXxEP5ju7eB7RDiRBkdC7a709kBF29qUbuW+MxcBGbF0FPB9lHZjXHpN8YYVmrRQ0ZtVyOE4gjVRnc3PmWtzJ7zjb9ovYeYg3EEccd2Kdn7IqHXoabdk3zlTUZe78JvWdu1p32zw8lo1OaNtplKuK+Hl5IdbazMTJdhz3PqpfM0TOOguEq36C7N0TIEfCkMFS4Uhou7xb3i+/xm9xTaV/wc/fIUP/wAl5cmoA==
AAAFIHicnVPLbtNAFL0pAUp4NIVNJTZD06IWtdWkIF4SUiWKBAtE361UV5HtTpKhtmPsaUix/AOIPQtWVGKB+Ac2bLpDLFiUP0DsKBIbFpwZp0FQGiRs2XPvufeee+blhJ6MFee7uZ4j+aPHjveeKJw8dfpMX7H/7FLc2Ixcseg2vEa04tix8GQgFpVUnlgJI2H7jieWnY3bOr7cFFEsG8GC2grFmm/XAlmVrq0AVYrPrKYdhXVZSVQ60hplt5jl26ruOMmdlFmeqCr8nch2N3SgtQ8lfIxVdf44s+yohkglkYxZNfGIKTZeTnW0IkeZFclaHQUg8+V6Ru7aXjKdVpIg7YQj06JSLPGJy1du8EnODhrlCW6eErWfmUZ/7i1ZtE4NcmmTfBIUkILtkU0x3lUqE6cQ2Bol9BCITRE8aXIEpVRA/SZwgSwb6Ab+NXirbTSAr3ljw+Cik4cvQiWjYf6Rv+Z7fIe/4Z/5j0O5EsOh9WxhdLJaEVb6ng7Mf/9nlY9RUf1XVVfNiqp03WiV0B4aRM/CzeqbT57vzd+cG04u8m3+Bfpf8l3+DjMImt/cV7Ni7oVhn0ZNtooRrPttBQ/AJ4BoT6/BJXSxTE4NKnW/1KjWvkVjHex/GW1qHWDMsMPXTCFeBZc+B3HXTAdsWTyA/9jsrW86BDhRWsec6W51dsDBmxjUrCU+PRcbs2LoKWF7qOzGuPAHYwQrMeg+ozKrEUBxiGqtuxufNFZqzvnaX7TeQ8zCWELcMl0K5r4I6LWoafYkW3mdkdU78FvGdsxpX+9wMhoyeUNtpgLu6/6lZIcbS5MTZdizvDR1tX1ze+k8DdIIuK7RFN2lGVqEkq+5gdyF3GB+O7+Tf5//kKX25No15+i3J//pJ2j/KB4=

View Slide

Set up｜A mathematical formalism for NN
構造の表現方法
n NNは層の集合と有向接続の集合
で構成：
n 有向接続は層番号のペアで表現
n ある層は種類とユニット数
で表現
n 入力が複数ある場合は，
入力を全てチャンネル方向に結合
8
0: ip
(235)
1: conv3, 16
(16)
2: conv3, 16
(256)
3: conv3, 32
(512)
4: conv5, 32
(1024)
5: max-pool, 1
(32)
6: fc, 16
(512)
7: softmax
(235)
8: op
(235)
(a)
0: ip
(235)
1: conv3, 16
(16)
2: conv3, 16
(256)
3: conv3, 16
(256)
4: conv3, 16
(256)
5: conv5, 32
(1024)
6: max-pool, 1
(32)
7: fc, 16
(512)
8: softmax
(235)
9: op
(235)
(b)
0: ip
(240)
1: conv7, 16
(16)
2: conv5, 32
(512)
3: conv3 /2, 16
(256)
4: conv3, 16
(256)
5: avg-pool, 1
(32)
6: max-pool, 1
(16)
7: max-pool, 1
(16)
8: fc, 16
(512)
12: fc, 16
(512)
9: conv3, 16
(256)
10: softmax
(120)
13: softmax
(120)
11: max-pool, 1
(16)
14: op
(240)
(c)
Figure 1: An illustration of
architectures. In each layer
the layer, followed by the
conv3), and then the numb
(e.g. number of ﬁlters). Th
output layers are pink while
(softmax) layers are green.
From Section 3: The layer
noted in parentheses. The fo
the normalised and unnorm
tances d,
¯
d . All self dista
i.e. d(G, G) = ¯
d(G, G) =
malised: d(a, b) = 175.1,
1479.3, d(b, c) = 1621.4. N
¯
d(a, b) = 0.0286, ¯
d(a, c)
¯
d(b, c) = 0.2625.
2
Figure 1
L
AAAEnXicnVNNaxNRFL2po9b40VY3gguDaUVEwksVv1YFFRSqtmnTFDKhzExf0rHzxcxLbB3yB9yLC0FRcCH+Bzdu/AMu+hPEZQU3LjzvzjSiNRF8w8zce969555333t25LmJEmKnMHbAOHjo8PiR4tFjx09MTE6dXEnCbuzIuhN6YbxqW4n03EDWlas8uRrF0vJtTzbszVt6vtGTceKGwbLajmTLtzqB23YdSwFqmr6lNhzLS+f7a5NlUbl85YaYFaX9RrUieJQpHwvhVOEDmbROITnUJZ8kBaRge2RRgqdJVRIUAWtRSo+AWBTDczlGUp+KyO8Cl4iygG7i24HXzNEAvuZNmMFBJQ9vjMwSzYjP4p3YFZ/Ee/FF/BjKlTKH1rONv53lymht4unppe//zPLxV7TxK2ukZkVtus5aXWiPGNGrcLL83pPnu0s3azPpefFGfIX+12JHfMQKgt435+2irL1g9tvIyboYw7qfK3gIPglEe7oHF1HF5JgOVOp6fVatfZMuDbD/ZbRoax9jhg3vmcJ8G1z6HCQjI22wZfMB/Me8tz5XCHCitI4aVzcHO2DjSRnlXuLVa7GwqhJqurA9ZI5iXP6DMYaVMrrHqLgbARRHyNa6R/G5bPX5nLf+ovUe5kz8y5g3uUqR74uEXpN6vCdZ53VElm/D32Lb5tO+PuAs0TTHTedMRdzXvUtZGm6szFaqsBdFee5qfnPH6QydowvgukZzdJcWqM6re0Yv6ZVx1rhjzBsPstCxQp5zin4bRuMnN735eg==
AAAEknicnVNNaxRBEK2No8Y1msST4MHFTUREQq8XQ06CCgpRk002CewsoWfSuxkzX8z0ronD/gHv4kFQFDyI/8GLF/+Ah/wE8RjBiwdf14wrZs0K9jAzVa+rXr2u7nZi30u1EPulsWPW8RMnx0+VT0+cOTs5NT2xlkbdxFUNN/KjZMORqfK9UDW0p321ESdKBo6v1p2dW2Z+vaeS1IvCVb0Xq1YgO6HX9lypATXtQOptV/rZYn9zqirmBI/KsFErjCoVYymaLn0gm7YoIpe6FJCikDRsnySleJpUI0ExsBZl9AiIpASexzGK+lRGfhe4QpQEuoNvB16zQEP4hjdlBheVfLwJMis0Kz6Ld+JAfBLvxRfx40iujDmMnj38nTxXxZuTT8+vfP9nVoC/pu3fWSM1a2rTPGv1oD1mxKzCzfN7T54frCzUZ7PL4o34Cv2vxb74iBWEvW/u22VVf8Hst5GTdzGBdb9Q8BB8CojxTA+uoorNMR2oNPX6rNr4Nl0bYP/LKGl3iDHHju6ZxnwbXOYcpCMjHbDl8yH8x7y3AVcIcaKMjjpXtwc74ODJGOVe4jVrkVhVBTU92D4yRzGuHmJMYGWM/mLU3I0QimNkG92j+Dy2+nzOW3/Reg9zNv5VzNtcpcz3RUGvTT3ek7zzJiLPd+Dvsu3wad8acFZohuNmCqYy7mvt8O0cNtauz9VgLwsapwt0ia6A4gbdpLu0RA1e1DN6Sa+si9YdazG/2WOl4oqfoz+G9eAnjhL4SQ==
AAAEknicnVPPaxNBFH6pUWustvUkeDCYVkQkTKrgj5OggkLVNm3aQjaU2e0kXbu/2J3E1iX/gHfxICgKHsT/wYsX/wEP/RPEYwUvHvzm7TaiNRGcZXff++a9733zZsaOPDfRQuwWxg4VDx85On6sdHzixMnJqemJlSTsxo5qOKEXxmu2TJTnBqqhXe2ptShW0rc9tWpv3TLzqz0VJ24YLOudSLV82QnctutIDahp+VJvOtJL5/vrUxVRvXzlupgT5YNGrSp4VCgfC+F04QNZtEEhOdQlnxQFpGF7JCnB06QaCYqAtSilR0AkxfBcjlHUpxLyu8AVoiTQLXw78Jo5GsA3vAkzOKjk4Y2RWaZZ8Vm8E3vik3gvvogfQ7lS5jB6dvC3s1wVrU8+Pb30/Z9ZPv6aNn9ljdSsqU3XWKsL7REjZhVOlt978nxv6UZ9Nj0v3oiv0P9a7IqPWEHQ++a8XVT1F8x+GzlZF2NY93MFD8GngBjP9OAiqlgc04FKU6/Pqo1v0aUB9r+MkrYPMGbY8J5pzLfBZc5BMjLSBls2H8B/zHvrc4UAJ8roqHN1a7ADNp6UUe4lXrMWiVWVUdOF7SFzFOPyH4wxrJTRfUbN3QigOEK20T2Kz2Wrz+e89Ret9zBn4V/BvMVVSnxfFPRa1OM9yTpvIrJ8G/422zaf9o0BZ5lmOG4mZyrhvu5fyvJwY2WuWoO9KGicztA5ugCKq3ST7tICNXhRz+glvSqeLd4pzmc3e6yQX/FT9NsoPvgJajX4fw==
E
AAAEnXicnVNLaxNRFD6po9b4aKsbwYXBtCIi4aaKr1VBCwo+2rRpCplQ7kxv0rHzYmYSW4f8AffiQlAUXIj/wY0b/4CL/gRxWcGNC797ZhrRmgjeYWbO+e453/nuufdaoevEiRA7hbEDxsFDh8ePFI8eO35iYnLq5EocdCNb1e3ADaJVS8bKdXxVT5zEVathpKRnuaphbd7S842eimIn8JeT7VC1PNnxnbZjywRQ0/RksmFLN53vr02WReXylRtiVpT2G9WK4FGmfCwEU4UPZNI6BWRTlzxS5FMC2yVJMZ4mVUlQCKxFKT0CIimC53CMoj4Vkd8FrhAlgW7i24HXzFEfvuaNmcFGJRdvhMwSzYjP4p3YFZ/Ee/FF/BjKlTKH1rONv5XlqnBt4unppe//zPLwT2jjV9ZIzQm16TprdaA9ZESvws7ye0+e7y7drM2k58Ub8RX6X4sd8REr8Hvf7LeLqvaC2W8jJ+tiBOt+ruAh+BQQ7ekeXEQVk2M6UKnr9Vm19k26NMD+l1HS1j7GDBveswTzbXDpcxCPjLTAls378B/z3npcwceJ0jpqXN0c7ICFJ2WUe4lXr0ViVSXUdGC7yBzFuPwHYwQrZXSPMeFu+FAcIlvrHsXnsNXnc976i9a7mDPxL2Pe5CpFvi8Kek3q8Z5kndcRWb4Ff4tti0/7+oCzRNMcN50zFXFf9y5labixMlupwl4U5bmr+c0dpzN0ji6A6xrN0R1aoDqv7hm9pFfGWWPeuGc8yELHCnnOKfptGI2fGm35cw==
ll(u)
AAAEl3icnVNNb9NAEJ0UAyV8tIULiEtEWlQQijYF8XWhEgiVA6JJm7ZSHVW2u0m39ZdsJ7RY+QNcOALiBBIHxH/gwoU/wKE/AXEsEhcOvB27QVASJNayPfN25s3b2V07dFWcCLFbGDlkHD5ydPRY8fiJk6fGxidOL8VBJ3JkwwncIFqxrVi6ypeNRCWuXAkjaXm2K5ftrbt6frkro1gF/mKyE8qmZ7V91VKOlQBquO5059LaeFlUrl67JWZE6aBRrQgeZcrHfDBR+EAmrVNADnXII0k+JbBdsijGs0pVEhQCa1JKm0AsiuApjpHUoyLyO8AloiygW/i24a3mqA9f88bM4KCSizdCZommxGfxTuyJT+K9+CJ+DORKmUPr2cHfznJluDb29OzC939mefgntPEra6jmhFp0k7UqaA8Z0atwsvzuk+d7C7frU+lF8UZ8hf7XYld8xAr87jfnbU3WXzH7PeRkXYxgPcwVPAKfBKI93YPLqGJyTBsqdb0eq9a+SVf62P8yWrR9gDHDBvcswXwLXPocxEMjbbBl8z78x7y3HlfwcaK0jjpXN/s7YONJGeVe4tVrsbCqEmoq2C4yhzEu/sEYwUoZ3WdMuBs+FIfI1rqH8Sm2enzOm3/R+gBzJv5lzJtcpcj3RUKvSV3ek6zzOiLLt+Fvs23zaV/vc5ZokuMmc6Yi7uv+pSwNNpZmKlXYNVGevZ7f3FE6TxdoGlw3aJbmaJ4aUKLoGb2gl8Y5445x35jLQkcKec4Z+m0YtZ9AnfZi
AAAEjHicnVM9bxNBEB0HA8EEklCBaCycoIBQtKYB0YAEQqFAJE6cRMpZ0d1lbZbsfehubRJO/gM0lICoQKJA/AcaGv4ARX4CogwSDQVv5w4jYmIk9nR3M29n3ryd3fVirVIjxF5p7Ej56LHj4ycqJydOnZ6cmp5YTaNu4sumH+koWffcVGoVyqZRRsv1OJFu4Gm55m3ftvNrPZmkKgpXzG4sW4HbCVVb+a4B1NR6rntpc6om5gWP6rBRL4waFWMxmi59IIe2KCKfuhSQpJAMbE0upXg2qE6CYmAtyugREJcSeIpjJPWpgvwucIkoF+g2vh14GwUawre8KTP4qKTxJsis0qz4LN6JffFJvBdfxI9DuTLmsHp28ffyXBlvTj49u/z9n1kB/oYe/s4aqdlQm66zVgXtMSN2FX6e33vyfH/5RmM2uyjeiK/Q/1rsiY9YQdj75r9dko1XzH4HOXkXE1j3CwUPwCeBWM/24DKqOBzTgUpbr8+qre/QlQH2v4wu7Qwx5tjhPTOYb4PLnoN0ZKQHtnw+hP+Y9zbgCiFOlNXR4OrOYAc8PBmj3Eu8di0uVlVFTQVbI3MU48oBxgRWxugvRsPdCKE4RrbVPYpPsdXnc976i9Z7mHPwr2He4SoVvi8Seh3q8Z7knbcReb4Hf4dtj0/71oCzSjMcN1MwVXBf6wdv57CxenW+DntJ0Didpws0B4prdIsWaJGaEKDoGb2gl+Vz5Zvlu/nNHisVV/wM/THKCz8Bt+b1Nw==
AAAEjHicnVNNb9NAEJ0UAyUU2nICcbFIiwpC0aatxMcFJBAqB0SbNm2lOqpsdxOW+ku2E1qs/AEuHAFxAokD4j9w4cIf4NCfgDgWiQsH3o7dICgJEmvZnnk78+bt7K4TeSpJhdgrjRwxjh47PnqifHLs1Onxicmx1STsxK5suKEXxuuOnUhPBbKRqtST61Esbd/x5JqzfVvPr3VlnKgwWEl3I9n07XagWsq1U0ANz5vpXNqcqIjq3Px1MSvMw0atKnhUqBiL4WTpA1m0RSG51CGfJAWUwvbIpgTPBtVIUASsSRk9AmJTDE9xjKQelZHfAS4RZQPdxrcNb6NAA/iaN2EGF5U8vDEyTZoWn8U7sS8+iffii/gxkCtjDq1nF38nz5XR5vjTs8vf/5nl45/Sw19ZQzWn1KJrrFVBe8SIXoWb53efPN9fvlGfzi6KN+Ir9L8We+IjVhB0v7lvl2T9FbPfQU7exRjW/ULBA/BJINrTPbiMKhbHtKFS1+uxau1bdKWP/S+jTTuHGHNscM9SzLfApc9BMjTSAVs+H8B/zHvrc4UAJ0rrqHN1q78DDp6MUe4lXr0WG6syUVPB9pA5jHHlD8YYVsboAWPK3QigOEK21j2MT7HV43Pe/IvWe5iz8K9g3uIqZb4vEnot6vKe5J3XEXm+A3+HbYdP+1af06QpjpsqmMq4rweX0hxsrM5Wa7CXBI3SebpAM6C4SrdogRapAQGKntELemmcM24ad/ObPVIqrvgZ+m0YCz8BlAn1bQ==
lu(u)
AAAEl3icnVNNb9NAEJ0UAyV8tIULiEtEWlQQijYF8XWhEgiVA6JJm7ZSHVW2u0m39ZdsJ7RY+QNcOALiBBIHxH/gwoU/wKE/AXEsEhcOvB27QVASJNayPfN25s3b2V07dFWcCLFbGDlkHD5ydPRY8fiJk6fGxidOL8VBJ3JkwwncIFqxrVi6ypeNRCWuXAkjaXm2K5ftrbt6frkro1gF/mKyE8qmZ7V91VKOlQBquJ3pzqW18bKoXL12S8yI0kGjWhE8ypSP+WCi8IFMWqeAHOqQR5J8SmC7ZFGMZ5WqJCgE1qSUNoFYFMFTHCOpR0Xkd4BLRFlAt/Btw1vNUR++5o2ZwUElF2+EzBJNic/indgTn8R78UX8GMiVMofWs4O/neXKcG3s6dmF7//M8vBPaONX1lDNCbXoJmtV0B4yolfhZPndJ8/3Fm7Xp9KL4o34Cv2vxa74iBX43W/O25qsv2L2e8jJuhjBepgreAQ+CUR7ugeXUcXkmDZU6no9Vq19k670sf9ltGj7AGOGDe5ZgvkWuPQ5iIdG2mDL5n34j3lvPa7g40RpHXWubvZ3wMaTMsq9xKvXYmFVJdRUsF1kDmNc/IMxgpUyus+YcDd8KA6RrXUP41Ns9ficN/+i9QHmTPzLmDe5SpHvi4Rek7q8J1nndUSWb8PfZtvm077e5yzRJMdN5kxF3Nf9S1kabCzNVKqwa6I8ez2/uaN0ni7QNLhu0CzN0Tw1oETRM3pBL41zxh3jvjGXhY4U8pwz9Nswaj8BZl/2aw==
G = (L, E)
AAAEvHicnVNLb9NAEJ6UACU8msIFiYtFWlRQFW0K4iWBKgECJBBt+pTqKLKdTWril2wntFj+Axy4cuAEEgfElSu9cOEPcOhPQByLxIUD347TVLQkSKxle+bbmW++nd01A8eOYiG2cyOH8oePHB09Vjh+4uSpseL46eXI74SWXLJ8xw9XTSOSju3JpdiOHbkahNJwTUeumO07an6lK8PI9r3FeDOQNddoeXbTtowYUL04pbtGvG4ZTnI/1W5pe+6jdFrrO/fSi/ViSZQvX7khZoR20KiUBY8S9cacP57bIp0a5JNFHXJJkkcxbIcMivCsUYUEBcBqlNBTIAaF8GyOkZRSAfkd4BJRBtA2vi14az3Ug694I2awUMnBGyJTo0nxVbwXO+KL+CC+iV8DuRLmUHo28TezXBnUx16cXfj5zywX/5jW97KGao6pSddZqw3tASNqFVaW333+amfhZnUyuSDeiu/Q/0Zsi89Ygdf9Yb2bl9XXzH4XOVkXQ1iPewqegE8CUZ7qwSVU0TmmBZWqXsqqla/TdB/7X0aDNg4wZtjgnsWYb4JLnYNoaKQJtmzeg/+M99blCh5OlNJR5ep6fwdMPAmj3Eu8ai0GVqWhpg3bQeYwxsV9jCGshNFdxpi74UFxgGylexifzVbK57z2F60PMafjX8K8zlUKfF8k9OrU5T3JOq8isnwT/gbbJp/2Rp9TowmOm+gxFXBfdy+lNthYnilXYM+L0uzV3s0dpXN0nqbAdY1m6QHN0RKUvKSP9Im28rfzjXw772ahI7lezhn6Y+S7vwFsOgTe
AAAEsXicnVNNb9NAEJ2UACUUmnJC4mKRFhVUVRsurZBASIAACUSbNm2lOopsZ5Oa+Eu2E1os/wEOXDlwAokD4sqVXrjwBzj0JyCOReLCgbdjk4qGphJr2Z55O/Pm7eyuGTh2FAuxVxg7UTx56vT4mdLZiXPnJ8tTE2uR3wstWbd8xw83TCOSju3JemzHjtwIQmm4piPXze4dNb/el2Fk+95qvBPIhmt0PLttW0YMqFme1V0j3rIMJ7mfaje1A/dROqcNnHvp1Wa5IuYFD23YqOZGhfKx5E8VdkmnFvlkUY9ckuRRDNshgyI8m1QlQQGwBiX0FIhBITybYySlVEJ+D7hElAG0i28H3maOevAVb8QMFio5eENkajQjvor3Yl98ER/EN/HrSK6EOZSeHfzNLFcGzckXF1d+Hpvl4h/T1kHWSM0xtWmRtdrQHjCiVmFl+f3nr/ZXbtRmkivirfgO/W/EnviMFXj9H9a7ZVl7zex3kZN1MYT1OFfwBHwSiPJUD66his4xHahU9VJWrXyd5gbY/zIatD3EmGFH9yzGfBtc6hxEIyNNsGXzHvxnvLcuV/BwopSOGlfXBztg4kkY5V7iVWsxsCoNNW3YDjJHMa4eYgxhJYz+YYy5Gx4UB8hWukfx2WylfM4b/9D6EHM6/hXM61ylxPdFQq9Ofd6TrPMqIss34W+zbfJpbw04NZrmuOmcqYT7Wj18O4eNtevzVdjLgsbpEl2mWVAs0G16QEtUh4CX9JE+0W7xVrFV7GY3e6yQX/EL9Ncour8BS3oDjg==
AAAEsXicnVNNb9NAEJ2UACUUmnJC4mKRFhVURZuCxIcEQgIESCDatGkr1VFkO5vUxF+yndBi+Q9w4MqBE0gcEFeu9MKFP8ChPwFxLBIXDrwdp6mgTZBYy/bM25k3b2d3zcCxo1iIndzYkfzRY8fHTxROTpw6PVmcmliJ/G5oyZrlO364ZhqRdGxP1mI7duRaEErDNR25anbuqPnVngwj2/eW461A1l2j7dkt2zJiQI3irO4a8YZlOMn9VLup7buP0jlt4NxLLzaKJVG+fOW6mBfaQaNSFjxK1B8L/lRum3Rqkk8WdcklSR7FsB0yKMKzThUSFACrU0JPgRgUwrM5RlJKBeR3gUtEGUA7+LbhrfdRD77ijZjBQiUHb4hMjWbEV/Fe7Iov4oP4Jn4N5UqYQ+nZwt/McmXQmHxxdunnP7Nc/GPa2M8aqTmmFl1jrTa0B4yoVVhZfu/5q92lG9WZ5IJ4K75D/xuxIz5jBV7vh/VuUVZfM/td5GRdDGE97it4Aj4JRHmqB5dQReeYNlSqeimrVr5OcwPsfxkN2jzAmGHDexZjvgUudQ6ikZEm2LJ5D/4z3luXK3g4UUpHlavrgx0w8SSMci/xqrUYWJWGmjZsB5mjGJf/YgxhJYzuMcbcDQ+KA2Qr3aP4bLZSPuf1Q7Q+xJyOfwnzOlcp8H2R0KtTj/ck67yKyPJN+Jtsm3zamwNOjaY5brrPVMB93buU2nBjZb5cgb0oaJzO0XmaBcVVuk0PaIFqEPCSPtIn2s7fyjfznexmj+X6V/wM/THy7m8nnQPE
AAAEpXicnVPNbtNAEJ4UAyX8NIELEheLtIAQijYF8XeqBAeQqGjTJK1UR5HtblJT/8l20hYrL8ALcEAcQAKEeAcuXHgBDn0ExLFIXDjw7dgNgtJUYi3bM9/OfPPt7K4Vuk6cCLFTmDiiHT12fPJE8eSp02emSuWzrTjoR7Zs2oEbRCuWGUvX8WUzcRJXroSRND3LlcvWxj01vzyQUewEfiPZDmXbM3u+03VsMwHUKZX7uuH4uuGZybptuumjYadUEdXrN+6IWaHvN2pVwaNC+VgIyoWPZNAaBWRTnzyS5FMC2yWTYjyrVCNBIbA2pfQEiEkRPIdjJA2piPw+cIkoE+gGvj14qznqw1e8MTPYqOTijZCp04z4It6LXfFZfBBfxc8DuVLmUHq28beyXBl2pp6dX/pxaJaHf0Lrv7PGak6oS7dZqwPtISNqFXaWP3j6fHfpbn0mvSRei2/Q/0rsiE9YgT/4br9ZlPUXzH4fOVkXI1jzuYLH4JNAlKd6cBVVDI7pQaWqN2TVyjfo2gj7X0aTtvYxZtjBPUsw3wWXOgfx2EgLbNm8D3+T99bjCj5OlNJR5+rGaAcsPCmj3Eu8ai0mVqWjpgPbReY4xsZfjBGslNE9xoS74UNxiGylexyfw9aQz3n7H1ofYs7Av4J5g6sU+b5I6DVowHuSdV5FZPkW/C22LT7tayNOnaY5bjpnKuK+7l1K/WCjNVutwV4Ulbmb+c2dpAt0ka6A6xbN0QNaoCaUbNJLekvvtMvavNbQWlnoRCHPOUd/DK3zCwSY+88=

View Slide

Set up｜A mathematical formalism for NN
出力層は複数存在してもよい
n NNは少なくとも1つ以上の出力を
持つものとする
n 複数の出力層が存在する場合は，
各出力層からの出力を平均した結果
を最終出力と定義
n 文献中では出力層をdecision layer，
最終出力をoutput layerと表記
9
6
0: ip
(240)
1: conv7, 16
(16)
2: conv5, 32
(512)
3: conv3 /2, 16
(256)
4: conv3, 16
(256)
5: avg-pool, 1
(32)
6: max-pool, 1
(16)
7: max-pool, 1
(16)
8: fc, 16
(512)
12: fc, 16
(512)
9: conv3, 16
(256)
10: softmax
(120)
13: softmax
(120)
11: max-pool, 1
(16)
14: op
(240)
(c)
Figure 1: An illustration of some CNN
architectures. In each layer, i: indexes
the layer, followed by the label (e.g
conv3), and then the number of units
(e.g. number of ﬁlters). The input and
output layers are pink while the decision
(softmax) layers are green.
From Section 3: The layer mass is de-
noted in parentheses. The following are
the normalised and unnormalised dis-
tances d,
¯
d . All self distances are 0,
i.e. d(G, G) = ¯
d(G, G) = 0. Unnor-
malised: d(a, b) = 175.1, d(a, c) =
1479.3, d(b, c) = 1621.4. Normalised:
¯
d(a, b) = 0.0286, ¯
d(a, c) = 0.2395,
¯
d(b, c) = 0.2625.
Figure 1

View Slide

The OTMANN Distance
距離の設計指針
n 要件：層の計算量・種類・接続方法の違いを表現可能
n 2つの構造を一致させるために必要な変更コストに着目
n OTMANN：2つの構造を一致させるための最小コスト
- OTMANN：Optimal Transport Metrics for Architectures of Neural Networks
OTMANNの構成要素
1. 接続や種類などの構造の変更コスト
2. 2つの構造を一致させるための最小コストの求め方
10

View Slide

The OTMANN Distance｜Layer masses
計算量を考慮した層の重み
n 各層に応じた重みを考える
n 層の計算量を考慮したいため，チャンネル数に着目
11
層の種類重みの計算方法
入力層・出力層・最終出力 0.1 × すべての層の重みの和
畳み込み層・全結合層前層の出力チャンネル数 × 出力チャンネル数
プーリング層入力チャンネル数
lm(u)
AAAEl3icnVNNb9NAEJ0UAyV8tIULiEtEWlQQijYF8XWhEgiVA6JJm7ZSHVW2u0m39ZdsJ7RY+QNcOALiBBIHxH/gwoU/wKE/AXEsEhcOvB27QVASJNayPfN25s3b2V07dFWcCLFbGDlkHD5ydPRY8fiJk6fGxidOL8VBJ3JkwwncIFqxrVi6ypeNRCWuXAkjaXm2K5ftrbt6frkro1gF/mKyE8qmZ7V91VKOlQBquN5059LaeFlUrl67JWZE6aBRrQgeZcrHfDBR+EAmrVNADnXII0k+JbBdsijGs0pVEhQCa1JKm0AsiuApjpHUoyLyO8AloiygW/i24a3mqA9f88bM4KCSizdCZommxGfxTuyJT+K9+CJ+DORKmUPr2cHfznJluDb29OzC939mefgntPEra6jmhFp0k7UqaA8Z0atwsvzuk+d7C7frU+lF8UZ8hf7XYld8xAr87jfnbU3WXzH7PeRkXYxgPcwVPAKfBKI93YPLqGJyTBsqdb0eq9a+SVf62P8yWrR9gDHDBvcswXwLXPocxEMjbbBl8z78x7y3HlfwcaK0jjpXN/s7YONJGeVe4tVrsbCqEmoq2C4yhzEu/sEYwUoZ3WdMuBs+FIfI1rqH8Sm2enzOm3/R+gBzJv5lzJtcpcj3RUKvSV3ek6zzOiLLt+Fvs23zaV/vc5ZokuMmc6Yi7uv+pSwNNpZmKlXYNVGevZ7f3FE6TxdoGlw3aJbmaJ4aUKLoGb2gl8Y5445x35jLQkcKec4Z+m0YtZ9Ez/Zj
0: ip
(235)
1: conv3, 16
(16)
2: conv3, 16
(256)
3: conv3, 32
(512)
4: conv5, 32
(1024)
5: max-pool, 1
0: ip
(235)
1: conv3, 16
(16)
2: conv3, 16
(256)
3: conv3, 16
(256)
4: conv3, 16
(256)
5: conv5, 32
(1024)
6: max-pool, 1
0: ip
(240)
1: conv7, 16
(16)
2: conv5, 32
(512)
3: conv3 /2, 16
(256)
4: conv3, 16
(256)
5: avg-pool, 1
(32)
6: max-pool, 1
7: max-pool, 1
(16)
9: conv3, 16
(256)
11: max-pool, 1
Fig
arc
the
co
(e.
ou
(so
Fro
no
0: ip
(235)
conv3, 16
(16)
conv3, 16
0: ip
(235)
1: conv3, 16
(16)
2: conv3, 16
0: ip
(240)
1: conv7, 16
(16)
2: conv5, 32 4: conv3, 16
Figure 1: An il
architectures. I
the layer, follo
conv3), and th
0: ip
(235)
1: conv3, 16
(16)
2: conv3, 16
(256)
3: conv3, 32
(512)
4: conv5, 32
(1024)
5: max-pool, 1
(32)
6: fc, 16
(512)
0: ip
(235)
1: conv3, 16
(16)
2: conv3, 16
(256)
3: conv3, 16
(256)
4: conv3, 16
(256)
5: conv5, 32
(1024)
6: max-pool, 1
(32)
7: fc, 16
(512)
0: ip
(240)
1: conv7, 16
(16)
2: conv5, 32
(512)
3: conv3 /2, 16
(256)
4:
5: avg-pool, 1
(32)
6: max-pool, 1
(16)
8: fc, 16
(512)
12: fc, 16
(512)
0: ip
(235)
1: conv3, 16
(16)
2: conv3, 16
(256)
3: conv3, 32
(512)
4: conv5, 32
(1024)
5: max-pool, 1
(32)
6: fc, 16
(512)
7: softmax
(235)
8: op
(235)
0: ip
(235)
1: conv3, 16
(16)
2: conv3, 16
(256)
3: conv3, 16
(256)
4: conv3, 16
(256)
5: conv5, 32
(1024)
6: max-pool, 1
(32)
7: fc, 16
(512)
8: softmax
(235)
9: op
(235)
2:
5: a
10:
(
Figure 1. (b)
層の重みの計算例： () 内が層の重み

View Slide

The OTMANN Distance｜Path lengths
接続関係の定量化
n 接続関係を定量化するために，2つの層の間の経路長を導入
n 入力層からある層までの経路長と
ある層から最終出力までの経路長を3種類計算
- 最短経路長，最大経路長，ランダムウォーク時の経路長の期待値
n 経路長はそれぞれ線形オーダーで計算可能
12
sp
AAAEqHicnVPNbtNAEJ6UACX8NAUOSFwi0iJUoWhTkPg5VYIDHBBp2jRFdYhsZ5Oa+k/2JrS18gK8AAcugMSh8A5cuPACHPoIiGORuHDg27EbBKWpxFq2Z76d+ebb2V0rdJ1YCbGbmziWP37i5OSpwukzZ89NFafPr8RBP7Jlww7cIFq1zFi6ji8bylGuXA0jaXqWK5vWxj093xzIKHYCf1lthbLlmT3f6Tq2qQC1ixeNjnSV+TQxPFOtR14Sh8Nhu1gWlRs374h5UTpoVCuCR5myUQumcx/JoA4FZFOfPJLkk4LtkkkxnjWqkqAQWIsSegbEpAiewzGShlRAfh+4RJQJdAPfHry1DPXha96YGWxUcvFGyCzRrPgidsSe+Cw+iK/i56FcCXNoPVv4W2muDNtTLy4t/Tgyy8Nf0frvrLGaFXXpNmt1oD1kRK/CTvMH2y/3lu7WZ5Or4q34Bv1vxK74hBX4g+/2u0VZf8Xs95GTdjGC9ShT8Bh8Eoj2dA/mUMXgmB5U6npDVq19g66PsP9lNGnzAGOKHd4zhfkuuPQ5iMdGWmBL5334z3lvPa7g40RpHXWubox2wMKTMMq9xKvXYmJVJdR0YLvIHMe4/BdjBCthdJ9RcTd8KA6RrXWP43PYGvI5b/1D60PMGfiXMW9wlQLfFwm9Bg14T9LO64g034K/ybbFp70z4izRDMfNZEwF3Nf9S1k63FiZr1RhL4ryQjW7uZN0ma7QNXDdogV6QDVqQMk2vaYdep+fy9fyzfyTNHQil+VcoD9G3voFYzD+Ag==
AAAEnXicnVO9bhNBEB4HA8EE4iAKJBoLJwhFKFrTgKiQoIAC4ThxHJQz1t557Ry5P92tTZKTX4AXoKABJIrAO9DQ8AIUeQREGSQaCr6dO4xIiJHY093NfDvzzbezu3bkuYkWYr8wdaJ48tTp6TOlszPnzs+W52bWknAQO6rphF4Yr9syUZ4bqKZ2tafWo1hJ3/ZUy966a+ZbQxUnbhis6p1ItX3ZD9ye60gNqFO+aHWVp+WT1PKl3oz9NIlGo065KpYEj8pRo5YbVcpHPZwrfCCLuhSSQwPySVFAGrZHkhI8G1QjQRGwNqX0FIikGJ7LMYpGVEL+ALhClAS6hW8f3kaOBvANb8IMDip5eGNkVmhBfBZ74kB8Eu/FF/HjWK6UOYyeHfztLFdFndnnl1a+/zPLx1/T5u+siZo19egWa3WhPWLErMLJ8oe7Lw5WbjcW0qvijfgK/a/FvviIFQTDb87bZdV4yez3kJN1MYb1MFfwCHwKiPFMDxZRxeKYPlSaeiNWbXyLro+x/2WUtH2EMcOO75nGfA9c5hwkEyNtsGXzAfxnvLc+VwhwooyOBle3xjtg40kZ5V7iNWuRWFUFNV3YHjInMa4eYoxhpYz+YtTcjQCKI2Qb3ZP4XLZGfM7bf9H6AHMW/lXMW1ylxPdFQa9FQ96TrPMmIsu34W+zbfNp7445KzTPcfM5Uwn3tXb4dh411m4s1WAvC5qmy3SFroHiJt2h+1SnJgTs0ivao3fFxWK92Mpu9lQhv+IX6I9RfPwTivH8yw==
AAAEnXicnVPNbtNAEJ6UACUUmiIOSFwi0iJUoWhTkPg5IcEBDog0bZqiOkS2s0lN/Sd7E9pafgFegAMXQOJQeAcuXHgBDn0ExLFIXDjw7TgNgpIgsZbtmW9nvvl2dtcKXSdWQuznpo7lj584OX2qcHrmzNnZ4tzMWhz0I1s27MANonXLjKXr+LKhHOXK9TCSpme5smlt3dXzzYGMYifwV9VOKFue2fOdrmObClC7eN7oSFeZTxLDM9Vm5CVxmKbtYllUrl2/JZZE6ahRrQgeZRqOWjCX+0AGdSggm/rkkSSfFGyXTIrxbFCVBIXAWpTQUyAmRfAcjpGUUgH5feASUSbQLXx78DaGqA9f88bMYKOSizdCZokWxGexJw7EJ/FefBE/xnIlzKH17OBvZbkybM8+v7Dy/Z9ZHv6KNn9lTdSsqEs3WasD7SEjehV2lj/YfXGwcru+kFwWb8RX6H8t9sVHrMAffLPfLsv6S2a/h5ysixGsh0MFj8AngWhP92ARVQyO6UGlrpeyau0bdHWE/S+jSdtHGDNsfM8U5rvg0ucgnhhpgS2b9+E/4731uIKPE6V11Lm6MdoBC0/CKPcSr16LiVWVUNOB7SJzEuPqH4wRrITRQ0bF3fChOES21j2Jz2Er5XPe+ovWB5gz8C9j3uAqBb4vEnoNGvCeZJ3XEVm+BX+bbYtPe2fEWaJ5jpsfMhVwXw8vZWm8sbZUqcJeFjRNF+kSXQHFDbpD96lGDQjYpVe0R+/yi/lavpnd7Knc8Iqfo99G/vFPZxT9AQ==
lp
AAAEqHicnVPNbtNAEJ6UACX8NAUOSFwi0iJUoWhTkPg5VYIDHBBp2jRFdYhsZ5Oa+k/2JrS18gK8AAcugMSh8A5cuPACHPoIiGORuHDg27EbBKWpxFq2Z76d+ebb2V0rdJ1YCbGbmziWP37i5OSpwukzZ89NFafPr8RBP7Jlww7cIFq1zFi6ji8bylGuXA0jaXqWK5vWxj093xzIKHYCf1lthbLlmT3f6Tq2qQC1ixeNjnSV+TQxPFOtR17ihsNhu1gWlRs374h5UTpoVCuCR5myUQumcx/JoA4FZFOfPJLkk4LtkkkxnjWqkqAQWIsSegbEpAiewzGShlRAfh+4RJQJdAPfHry1DPXha96YGWxUcvFGyCzRrPgidsSe+Cw+iK/i56FcCXNoPVv4W2muDNtTLy4t/Tgyy8Nf0frvrLGaFXXpNmt1oD1kRK/CTvMH2y/3lu7WZ5Or4q34Bv1vxK74hBX4g+/2u0VZf8Xs95GTdjGC9ShT8Bh8Eoj2dA/mUMXgmB5U6npDVq19g66PsP9lNGnzAGOKHd4zhfkuuPQ5iMdGWmBL5334z3lvPa7g40RpHXWubox2wMKTMMq9xKvXYmJVJdR0YLvIHMe4/BdjBCthdJ9RcTd8KA6RrXWP43PYGvI5b/1D60PMGfiXMW9wlQLfFwm9Bg14T9LO64g034K/ybbFp70z4izRDMfNZEwF3Nf9S1k63FiZr1RhL4ryQjW7uZN0ma7QNXDdogV6QDVqQMk2vaYdep+fy9fyzfyTNHQil+VcoD9G3voFRdL9+w==
rw
AAAEqHicnVPLbtNAFL0pAUp4NAUWSGwi0iJUoWhSkHisKsECFog0bZqiukS2M0lN/ZI9SR9WfoAfYMEGkFgU/oENG36ART8BsSwSGxacuXaDoCRIjGX73jP3nnvmzowVuk6shNjPTRzLHz9xcvJU4fSZs+emitPnV+KgF9myYQduEK1aZixdx5cN5ShXroaRND3LlU1r856eb/ZlFDuBv6x2QrnumV3f6Ti2qQC1iheNtnSV+TQxPFNtRF4SbQ0GrWJZVG7cvCPmRemoUa0IHmXKRi2Yzn0gg9oUkE098kiSTwq2SybFeNaoSoJCYOuU0DMgJkXwHI6RNKAC8nvAJaJMoJv4duGtZagPX/PGzGCjkos3QmaJZsVnsScOxCfxXnwRP0ZyJcyh9ezgb6W5MmxNPb+09P2fWR7+ijZ+ZY3VrKhDt1mrA+0hI3oVdprf331xsHS3PptcFW/EV+h/LfbFR6zA73+z3y7K+ktmv4+ctIsRrEeZgsfgk0C0p3swhyoGx3ShUtcbsGrtG3R9iP0vo0nbRxhTbHTPFOY74NLnIB4baYEtnffhb/HeelzBx4nSOupc3RjugIUnYZR7iVevxcSqSqjpwHaROY5x+Q/GCFbC6CGj4m74UBwiW+sex+ewNeBzvv4XrQ8xZ+BfxrzBVQp8XyT0GtTnPUk7ryPSfAv+NtsWn/b2kLNEMxw3kzEVcF8PL2VptLEyX6nCXhTlhWp2cyfpMl2ha+C6RQv0gGrUgJJdekV79C4/l6/lm/knaehELsu5QL+NvPUTfFX+CA==
AAAEnXicnVPNbtNAEJ6UACUUmiIOSFwi0iJUoWrDBcQJCQ5wQKRp0xTVIbKdTerWf7I36Y/lF+AFOHABJA6Fd+DChRfg0EdAHIvEhQPfjk0QDQ0Sa9me+Xbmm29nd63QdWIlxGFh6lTx9Jmz0+dK52cuXJwtz82sxcEgsmXTDtwgWrfMWLqOL5vKUa5cDyNpepYrW9b2fT3fGsoodgJ/Ve2Fsu2Zfd/pObapAHXKl42udJX5LDE8U21GXhLtpGmnXBVLgkdl3KjlRpXyUQ/mCh/IoC4FZNOAPJLkk4Ltkkkxng2qkaAQWJsS2gJiUgTP4RhJKZWQPwAuEWUC3ca3D28jR334mjdmBhuVXLwRMiu0ID6LA3EkPon34ov4cSJXwhxazx7+VpYrw87s8ysr3/+Z5eGvaPN31kTNinp0h7U60B4yoldhZ/nD/RdHK3cbC8l18UZ8hf7X4lB8xAr84Tf77bJsvGT2B8jJuhjBepwreAI+CUR7ugeLqGJwTB8qdb2UVWvfoJsj7H8ZTdodY8ywk3umMN8Dlz4H8cRIC2zZvA9/h/fW4wo+TpTW0eDqxmgHLDwJo9xLvHotJlZVQU0HtovMSYyrxxgjWAmjvxgVd8OH4hDZWvckPoetlM95+y9aH2HOwL+KeYOrlPi+SOg1aMh7knVeR2T5Fvxdti0+7d0RZ4XmOW4+ZyrhvtaO385xY+3WUg32sqBpukrX6AYobtM9ekh1akLAPr2iA3pXXCzWi63sZk8V8it+if4Yxac/AaPU/NE=
AAAEnXicnVO9bhNBEB4HA8EE4iAKJBoLJwhFyFonkfipkKCAAuE4cRyUM9bdee0cuT/drZ2fk1+AF6CgASSKwDvQ0PACFHkERBkkGgq+nXOMINhI2dPdzXw78823s7tW6DqxEuIgM3Eqe/rM2clzufNTFy5O52em1uKgG9myZgduEK1bZixdx5c15ShXroeRND3LlXVr676er/dkFDuBv6p2Q9nwzI7vtB3bVICa+ctGS7rKfJYYnqk2Iy+Jtvv9Zr4oSotLd8SCKBw3yiXBo0iDUQlmMh/JoBYFZFOXPJLkk4Ltkkkxng0qk6AQWIMSeg7EpAiewzGS+pRDfhe4RJQJdAvfDryNAerD17wxM9io5OKNkFmgOfFF7ItD8Vl8EF/Fz5FcCXNoPbv4W2muDJvTL66s/Phvloe/os3fWWM1K2rTbdbqQHvIiF6Fneb39l4ertytziXXxVvxDfrfiAPxCSvwe9/td8uy+orZHyAn7WIE6/FAwRPwSSDa0z2YRxWDYzpQqev1WbX2Dbo5xE7KaNLOMcYUG90zhfk2uPQ5iMdGWmBL533427y3HlfwcaK0jipXN4Y7YOFJGOVe4tVrMbGqAmo6sF1kjmNc/YsxgpUwesSouBs+FIfI1rrH8Tls9fmcN/6h9RHmDPyLmDe4So7vi4Reg3q8J2nndUSab8HfYdvi094achZoluNmB0w53NejS1kYbawtlMqwlwVN0lW6RjdAcYvu0UOqUA0C9ug17dP77Hy2kq2nN3siM7jil+iPkX36C3/3/Qc=
O(|E|)
AAAErnicnVNNb9NAEJ0UAyV8NIULgktEWlQQijYF8XWqBEhwQG3TJq1UR5HtbhJTf8l2QosbiTN/gAMnkEBCHOAXcOHCH+DQn4A4FokLB96O3SBoEyTWsj3zdubN29ldM3DsKBZiJzd2SDt85Oj4sfzxEydPTRQmT9cjvxtasmb5jh+umkYkHduTtdiOHbkahNJwTUeumBt31PxKT4aR7XvL8VYgG67R9uyWbRkxoGbhnO4acccynGS+P7M9cO71ty81CyVRvnrtlpgVxf1GpSx4lCgbC/5k7iPptE4+WdQllyR5FMN2yKAIzxpVSFAArEEJPQJiUAjP5hhJfcojvwtcIsoAuoFvG95ahnrwFW/EDBYqOXhDZBZpWnwRb8Wu+Czeia/i51CuhDmUni38zTRXBs2JZ2eXfvwzy8U/ps7vrJGaY2rRTdZqQ3vAiFqFleb3njzfXbpdnU4uilfiG/S/FDviE1bg9b5brxdl9QWz30VO2sUQ1sNMwTz4JBDlqR5cRhWdY9pQqer1WbXydboywP6X0aDNfYwpNrxnMeZb4FLnIBoZaYItnffgP+a9dbmChxOldFS5uj7YARNPwij3Eq9ai4FVFVHThu0gcxTj8l+MIayE0T3GmLvhQXGAbKV7FJ/NVp/PeeMArQ8wp+NfwrzOVfJ8XyT06tTjPUk7ryLSfBP+Jtsmn/b1AWeRpjhuKmPK477uXcricKM+W67AXhSluevZzR2n83SBZsB1g+boPi1QDUqe0ht6Tx80odW1htZMQ8dyWc4Z+mNonV/NuAAP

View Slide

The OTMANN Distance
OTMANN
n 2つのNN との距離を測りたい
n 2つの層の重みを一致させる変換行列を考える
n 各ペナルティを最小にする変換行列を求め，
そのときのペナルティの合計値を距離とする
13
G1 = (L1, E1)
AAACmXichVHLSiNBFD22r0x8Rd0MuGkMioKEGw2+YECRQREXaowKKqG7LbWxX3RXAk6TH/AHBF0pDCL+hOBG3bvwE2SWDrhx4U0nKiLqLarq1Kl7bp2q0j3LDCTRXY1SW1ff0Bj7EW9qbmltS7R3LAduwTdEznAt11/VtUBYpiNy0pSWWPV8odm6JVb03any/kpR+IHpOktyzxMbtrbtmFumoUmm8onMuq3JHUOzwulSPkyX1F9q3ys1F1ED6ivxOyL684kkpQbHRmloTP0I0imKIolqzLuJU6xjEy4MFGBDwIFkbEFDwG0NaRA85jYQMuczMqN9gRLirC1wluAMjdldHrd5tVZlHV6XawaR2uBTLO4+K1X00C2d0QNd0Tnd09OntcKoRtnLHs96RSu8fNv+z+zjtyqbZ4mdN9WXniW2MBp5Ndm7FzHlWxgVffHPwUN2fLEn7KUT+sf+j+mOLvkGTvG/8XdBLB4hzh/w8srq52B5MJVmvJBJTgxXvyKGLnSjj997BBOYwTxyfO4hLnCNG6VLmVRmlNlKqlJT1XTiXSjZZ6rMnhU=
G2 = (L2, E2)
AAACmXichVHLSiNBFD224yvjaNSN4KYxKApDuMmILxAUEUVcaDJRQSV091RiY7/orgS0yQ/4A4KuFESG+YkBN+rehZ8wuIzgxsXcdKIyiM4tqurUqXtunarSPcsMJNFdk9L8qaW1rb0j9rnzS1d3vKd3PXBLviFyhmu5/qauBcIyHZGTprTEpucLzdYtsaHvzdf2N8rCD0zX+S73PbFja0XHLJiGJpnKx8e2bU3uGpoVLlbyYbqizqgjL9RKRH1VX4iFiBjNxxOUTE9N0rcp9S1IJSmKBBqx6sYvsI0fcGGgBBsCDiRjCxoCbltIgeAxt4OQOZ+RGe0LVBBjbYmzBGdozO7xWOTVVoN1eF2rGURqg0+xuPusVDFEt/STqnRFv+gPPb1bK4xq1Lzs86zXtcLLdx/2Zx//q7J5lth9VX3oWaKAyciryd69iKndwqjrywdH1ex0ZigcpjO6Z/+ndEeXfAOn/GCcr4nMCWL8Ac+vrL4P1tPJFOO1scTseOMr2jGAQYzwe09gFktYRY7PPcZvXONGGVDmlCVluZ6qNDU0ffgnlOxfsQeeGA==
Z 2 Rn1
⇥n2
+
AAACinichVFBSxtBGH1Za9Voa9peBC9LQ6RQCF+itIleBD14jLFRqbHL7jrqkN3ZZXcS0CV/oP4AD54UpBRPXttjL/0DPfgTxKNCLz30yyallGL7DTPz5s33vnkz44SejDXRVcYYejD8cGR0LDs+8ejxZO7J0/U4aEeuaLiBF0Sbjh0LTyrR0FJ7YjOMhO07nthwWku9/Y2OiGIZqDf6IBTbvr2n5K50bc2UlSu8NZtSmU3f1vuOk9S77xJllcymlr6ITWWVu1bysmvl8lQsVys0WzX/BqUipZHHIGpB7gOa2EEAF234EFDQjD3YiLltoQRCyNw2EuYiRjLdF+giy9o2ZwnOsJlt8bjHq60Bq3jdqxmnapdP8bhHrDRRoG/0kW7pK13QNf24t1aS1uh5OeDZ6WtFaE2+n1r7/l+Vz7PG/m/VPz1r7KKSepXsPUyZ3i3cvr5zeHy7Nl8vJDN0Rjfs/5Su6AvfQHXu3PNVUT9Blj/g1yub94P1crHEeHUuv/hq8BWjmMZzvOD3fo1FrKCGBp97hEt8wmdjwigbVWOhn2pkBppn+COM5Z84rJff
Z
AAACZHichVFNSwJBGH7cvsysLAmCICQxOsmrRV8noUvHrLTIRHa3sRbX3WV3FUz6A3UtOnQqiIh+Rpf+QIf+QBAdDbp06HU1IsJ6h5l55pn3eeeZGcXSNcclevJJXd09vX3+/sBAcHBoODQymnXMiq2KjGrqpr2tyI7QNUNkXM3VxbZlC7ms6GJLKa0097eqwnY009h0a5bIl+V9Qytqquwyld4phKIUTy4t0uxS5DdIxMmLKNqxZoZusIs9mFBRQRkCBlzGOmQ43HJIgGAxl0edOZuR5u0LHCHA2gpnCc6QmS3xuM+rXJs1eN2s6XhqlU/RudusjCBGj3RLDXqgO3qhj4616l6Nppcaz0pLK6zC8PH4xvu/qjLPLg6+VX96dlHEoudVY++WxzRvobb01cPzxsbyeqw+TVf0yv4v6Ynu+QZG9U29Tov1CwT4A75eOdIZZJPxBOP0XDQ13/4KPyYwhRl+7wWksIo1ZPhcgROc4sz3LAWlsDTWSpV8bU0YP0Ka/ARqWIor
層の種類の違い
に関するペナルティ
層数の差
conv3 conv5 max-pool avg-pool fc
v3 0 0.2 1 1 1
v5 0.2 0 1 1 1
-pool 1 1 0 0.25 1
-pool 1 1 0.25 0 1
1 1 1 1 0
Table 1: An example label mis
cost matrix M. There is zero c
matching identical layers, < 1 c
similar layers, and infinite cost f
parate layers.
lks from the input uip
to u. As the layers of a neural network can be topologically ordere
path lengths are well defined and finite. Further, for any s 2 {sp,lp,rw} and t 2 {ip,op}
e computed for all u 2 L, in O(|E|) time (see Appendix A.3 for details).
e now ready to describe OTMANN. Given two networks G1 = (L1, E1), G2 = (L2, E2
layers respectively, we will attempt to match the layer masses in both networks.
Rn1
⇥n2
+
be such that Z(i, j) denotes the amount of mass matched between layer i 2
2
. The OTMANN distance is computed by solving the following optimisation problem
minimise
Z
lmm(Z) + nas(Z) + ⌫str str(Z)
subject to
X
j2L2
Zij  `m(i),
X
i2L1
Zij  `m(j), 8i, j
abel mismatch term lmm
, penalises matching masses that have different labels, wh
経路長

View Slide

The OTMANN Distance
：Label mismatch penalty
n ，
n 層の種類の差異を定量化したペナルティ
n 「3x3 convと5x5 convは近く，3x3 convとmax poolingは
遠い」などの層の種類の差異を表現
14
conv3 conv5 max-pool avg-pool fc
conv3 0 0.2 1 1 1
conv5 0.2 0 1 1 1
max-pool 1 1 0 0.25 1
avg-pool 1 1 0.25 0 1
fc 1 1 1 1 0
Table 1: An example label mismatch
cost matrix M. There is zero cost for
matching identical layers, < 1 cost for
similar layers, and infinite cost for dis-
parate layers.
for walks from the input uip
to u. As the layers of a neural network can be topologically ordered1, the
above path lengths are well defined and finite. Further, for any s 2 {sp,lp,rw} and t 2 {ip,op}, s
t
(u)
can be computed for all u 2 L, in O(|E|) time (see Appendix A.3 for details).
lmm
AAAEp3icnVPNbtNAEJ6UACX8NKUXJC4RaREgFG2ipiWIQyU4wAG1TZs0Uh1FtrtJTG3Hst3QYOUFeAEOHBBISFS8AxcuvACHPgLiWCQuHPh27Abxk1RiLdsz38588+3sruHZVhAKcZiaOpU+febs9LnM+QsXL81kZy/Xg96eb8qa2bN7fsPQA2lbrqyFVmjLhudL3TFsuWXs3lfzW33pB1bP3QwHnmw6ese12paph4Ba2TnN61qtSHP0sOs7ke04w2ErmxeFYrm8WFrKicJSuVKqKGN5EVAlVywIHnlKxlpvNvWBNNqhHpm0Rw5JcimEbZNOAZ5tKpIgD1iTInoCRCcfnsUxkoaUQf4ecIkoHeguvh142wnqwle8ATOYqGTj9ZGZowXxWRyII/FJvBdfxI+xXBFzKD0D/I04V3qtmedXNr6fmOXgH1L3V9ZEzSG16Q5rtaDdY0Stwozz+89eHG3crS5E18Ub8RX6X4tD8RErcPvfzLfrsvqS2R8gJ+6iD+txomAVfBKI8lQPbqGKxjEdqFT1hqxa+RrdHmH/y6jT/l+MMTa+ZyHm2+BS5yCYGGmALZ534T/lvXW4gosTpXRUubo22gEDT8Qo9xKvWouOVeVQ04JtI3MS4+YfjD6siNFjxpC74UKxh2ylexKfxdaQz3nzH1ofYU7DP495jatk+L5I6NWoz3sSd15FxPkG/H22DT7tOyPOHM1z3HzClMF9Pb6UufFGvVQowl4X+ZV7yc2dpqt0jW6Aa5lW6CGtUQ1KBvSK3tFB+mZ6NV1PN+LQqVSSM0e/jbT+EzUT/cY=
AAAEp3icnVPNbtNAEJ6UACX8NKUXJC4RaREgFG2ipiWIQyU4wAG1TZs0Uh1FtrtJTG3Hst3QYOUFeAEOHBBISFS8AxcuvACHPgLiWCQuHPh27Abxk1RiLdsz38588+3sruHZVhAKcZiaOpU+febs9LnM+QsXL81kZy/Xg96eb8qa2bN7fsPQA2lbrqyFVmjLhudL3TFsuWXs3lfzW33pB1bP3QwHnmw6ese12paph4Ba2TnN61qtSHP0sOs7ke04w2ErmxeFYrm8WFrKicJSuVKqKGN5EVAlVywIHnlKxlpvNvWBNNqhHpm0Rw5JcimEbZNOAZ5tKpIgD1iTInoCRCcfnsUxkoaUQf4ecIkoHeguvh142wnqwle8ATOYqGTj9ZGZowXxWRyII/FJvBdfxI+xXBFzKD0D/I04V3qtmedXNr6fmOXgH1L3V9ZEzSG16Q5rtaDdY0Stwozz+89eHG3crS5E18Ub8RX6X4tD8RErcPvfzLfrsvqS2R8gJ+6iD+txomAVfBKI8lQPbqGKxjEdqFT1hqxa+RrdHmH/y6jT/l+MMTa+ZyHm2+BS5yCYGGmALZ534T/lvXW4gosTpXRUubo22gEDT8Qo9xKvWouOVeVQ04JtI3MS4+YfjD6siNFjxpC74UKxh2ylexKfxdaQz3nzH1ofYU7DP495jatk+L5I6NWoz3sSd15FxPkG/H22DT7tOyPOHM1z3HzClMF9Pb6UufFGvVQowl4X+ZV7yc2dpqt0jW6Aa5lW6CGtUQ1KBvSK3tFB+mZ6NV1PN+LQqVSSM0e/jbT+EzUT/cY=
AAAEp3icnVPNbtNAEJ6UACX8NKUXJC4RaREgFG2ipiWIQyU4wAG1TZs0Uh1FtrtJTP0n2wkNVl6AF+DAAYGERMU7cOHCC3DoIyCOReLCgW/HaRCUBom1bM98O/PNt7O7hm9bYSTEQWrqVPr0mbPT5zLnL1y8NJOdvVwPvV5gyprp2V7QMPRQ2pYra5EV2bLhB1J3DFtuGbv31fxWXwah5bmb0cCXTUfvuFbbMvUIUCs7p/ldqxVrjh51Aye2HWc4bGXzolAslxdLSzlRWCpXShVlLC8CquSKBcEjT6Ox5s2mPpBGO+SRST1ySJJLEWybdArxbFORBPnAmhTTEyA6BfAsjpE0pAzye8AlonSgu/h24G2PUBe+4g2ZwUQlG2+AzBwtiM9iXxyKT+K9+CJ+nMgVM4fSM8DfSHKl35p5fmXj+z+zHPwj6v7Kmqg5ojbdYa0WtPuMqFWYSX7/2YvDjbvVhfi6eCO+Qv9rcSA+YgVu/5v5dl1WXzL7A+QkXQxgPR4pWAWfBKI81YNbqKJxTAcqVb0hq1a+RrfH2P8y6rR3jDHBTu5ZhPk2uNQ5CCdGGmBL5l34T3lvHa7g4kQpHVWuro13wMATM8q9xKvWomNVOdS0YNvInMS4+QdjACtm9Igx4m64UOwjW+mexGexNeRz3vyL1keY0/DPY17jKhm+LxJ6NerzniSdVxFJvgF/j22DT/vOmDNH8xw3P2LK8H1NLmXuuHF0X+ulQhH2usiv3Bvd3Gm6StfoBriWaYUe0hrVoGRAr+gd7advplfT9XQjCZ1KjXLm6LeR1n8CM739ww==
Cij = M(ll(i), ll( j))
AAAErHicnVPNbtNAEJ6UACX8NIVLJS4WaVGCqmidAylISJXKAQ4VbZq0leIost1NcOo4lu2Elig8AC/AgRMIDgjBI3Dhwgtw6CMgjkXiwoFvx0kQP0klbNk78+3MN9/O7lq+64SREEeJmVPJ02fOzp5Lnb9w8dJcev7ydtjpBras2B23E+xaZihdx5OVyIlcuesH0mxbrtyx9tfU/E5PBqHT8crRoS9rbbPpOQ3HNiNA9fTCWr3vtAbaHW0967pZJ7esYWjlcvV0RuQLxRVd6Bob4lZhZBQ1PS/4ydDw2ejMJz6QQXvUIZu61CZJHkWwXTIpxFslnQT5wGrUpxYQkwJ4DsdIGlAK+V3gElEm0H38m/CqQ9SDr3hDZrBRycUXIFOjJfFZvBHH4pN4K76IHxO5+syh9BxitOJc6dfnni5sfT8xq40xooe/sqZqjqhBK6zVgXafEbUKO87vPX52vHW7tNS/Ll6Kr9D/QhyJj1iB1/tmv96UpefMfhc5cRcDWOtDBQ/AJ4EoT/XgBqoYHNOESlVvwKqVb9DyGPtfRpMO/mKMsck9izDfAJc6B+HUSAts8bwH/xHvbZsreDhRSkeJqxvjHbDw9hnlXuJTazGxKg01HdguMqcxlv9gDGD1GR0xRtwND4p9ZCvd0/gctgZ8zmv/0HofcwbGDOYNrpLi+yKh16Ae70nceRUR51vwD9i2+LTvjTk1WuS4xSFTCvd1dCm1ycZ2Ia/D3tQzqzeHN3eWrtI1yoKrSKt0jzaoAiVP6BW9o/fJfLKcrCZrcehMYphzhX57ko2fJIb8xw==
AAAEoXicnVPBbtNAEJ2UACUUmnKqxMUiLUpQFW24gJCQkMoBDhVtmrSV4iiy3U3Y1LEt2wktVvgAfoADJxAcEIJP4MKFH+DQT0Aci8SFA2/HIYiGBglb9s68nXnzdnbXDlwVxUIcZmZOZU+fOTt7Lnd+7sLF+fzC3Fbk90NH1h3f9cMd24qkqzxZj1Xsyp0glFbPduW2vbeq57cHMoyU79Xig0A2e1bHU23lWDGgVn5xtZWo7tC4bawVXbeoSisGhm6p1MoXRFnwY0walZFRoNGz7i9kPpBJu+STQ33qkSSPYtguWRThbVCFBAXAmpRQF4hFITzFMZKGlEN+H7hElAV0D/8OvMYI9eBr3ogZHFRy8YXINGhZfBZvxJH4JN6KL+LHiVwJc2g9BxjtNFcGrfmni5vf/5nVwxjTw99ZUzXH1KabrFVBe8CIXoWT5g8ePzvavFVdTq6Kl+Ir9L8Qh+IjVuANvjmvN2T1ObPfRU7axRDW2kjBA/BJINrTPbiGKibHdKBS1xuyau2btDLG/pfRov0JxhQ7uWcx5tvg0ucgmhppgy2d9+A/4r3tcQUPJ0rrqHJ1c7wDNt6EUe4lPr0WC6syUFPBdpE5jbF2jDGElTD6izHmbnhQHCBb657Gp9ga8jlv/kXrfcyZGAuYN7lKju+LhF6TBrwnaed1RJpvw99n2+bTvjvmNGiJ45ZGTDnc18rx2zlpbF0vV2BvCJqly3SFiqC4QXfoHq1THQKe0Ct6R++z5Wwt20hv9kxmdMUv0R9PtvkT7677bA==
AAAEoXicnVPNbtNAEJ6UACUUmnKqxMUiLUpQFW1ySUFCQioHOFS0adJWiqPIdjepU8e2bCe0WOEBeAEOnEBwQAgegQsXXoBDHwFxLBIXDnw7ToL4aZCwZe/MtzPffDu7a/qOHUZCHKdmzqTPnjs/eyFzce7S5fnswtx26PUDS9Ytz/GCXdMIpWO7sh7ZkSN3/UAaPdORO+bBmprfGcggtD23Fh35stkzOq7dti0jAtTKLq61Yrs71G5r63nHyduFFQ1Dt1BoZXOiWK6slkRJY0PcLI+NilYqCn5yNHo2vIXUe9JpjzyyqE89kuRSBNshg0K8DSqRIB9Yk2LqAjEogGdzjKQhZZDfBy4RZQA9wL8DrzFCXfiKN2QGC5UcfAEyNVoWn8RrcSI+ijfis/h+KlfMHErPEUYzyZV+a/7J4ta3f2b1MEa0/zNrquaI2rTKWm1o9xlRq7CS/MGjpydbt6rL8XXxQnyB/ufiWHzACtzBV+vVpqw+Y/a7yEm6GMBaHyl4AD4JRHmqBzdQReeYDlSqekNWrXydVibY/zIadPgHY4Kd3rMI821wqXMQTo00wZbMu/Af8t72uIKLE6V0VLm6PtkBE2/MKPcSn1qLgVVpqGnDdpA5jbH2G2MAK2Z0zBhxN1wo9pGtdE/js9ka8jlv/kXrfczpGHOY17lKhu+LhF6dBrwnSedVRJJvwj9k2+TTvjfh1GiJ45ZGTBnc1/Gl1E43tsvFEuxNQbN0la5RHhQVukP3aIPqEPCYXtJbepcupmvpRnKzZ1KjK36FfnnSzR81HPu8
AAAErHicnVPNbtNAEJ6UACX8NIVLJS4WaVGCqmiTAylISJXKAQ4VbZq0leIost1NcOrYlu2Elig8AC/AgRMIDgjBI3Dhwgtw6CMgjkXiwoFvx0kQP0klbNk78+3MN9/O7pq+Y4eREEeJmVPJ02fOzp5Lnb9w8dJcev7yduh1A0tWLc/xgl3TCKVju7Ia2ZEjd/1AGh3TkTvm/pqa3+nJILQ9txId+rLeMVqu3bQtIwLUSC+sNfp2e6Dd0dazjpO1c8sahnYu10hnRL5YWimIgsaGuFUcGSWtkBf8ZGj4bHjziQ+k0x55ZFGXOiTJpQi2QwaFeGtUIEE+sDr1qQ3EoACezTGSBpRCfhe4RJQBdB//FrzaEHXhK96QGSxUcvAFyNRoSXwWb8Sx+CTeii/ix0SuPnMoPYcYzThX+o25pwtb30/M6mCM6OGvrKmaI2rSCmu1od1nRK3CivN7j58db90uL/Wvi5fiK/S/EEfiI1bg9r5Zrzdl+Tmz30VO3MUA1vpQwQPwSSDKUz24gSo6x7SgUtUbsGrl67Q8xv6X0aCDvxhjbHLPIsw3waXOQTg10gRbPO/Cf8R72+EKLk6U0lHm6vp4B0y8fUa5l/jUWgysSkNNG7aDzGmMlT8YA1h9RkeMEXfDhWIf2Ur3ND6brQGf8/o/tN7HnI4xg3mdq6T4vkjo1anHexJ3XkXE+Sb8A7ZNPu17Y06NFjlucciUwn0dXUptsrFdzBdgb4rM6s3hzZ2lq3SNsuAq0Srdow2qQskTekXv6H0yn6wka8l6HDqTGOZcod+eZPMnJDb8xg==
AAAEoXicnVPBbtNAEJ2UACUUmnKqxMUiLUpQFW24gJCQkMoBDhVtmrSV4iiy3U3Y1LEt2wktVvgAfoADJxAcEIJP4MKFH+DQT0Aci8SFA2/HIYiGBglb9s68nXnzdnbXDlwVxUIcZmZOZU+fOTt7Lnd+7sLF+fzC3Fbk90NH1h3f9cMd24qkqzxZj1Xsyp0glFbPduW2vbeq57cHMoyU79Xig0A2e1bHU23lWDGgVn5xtZWo7tC4bawVXbeoSisGhm6p1MoXRFnwY0walZFRoNGz7i9kPpBJu+STQ33qkSSPYtguWRThbVCFBAXAmpRQF4hFITzFMZKGlEN+H7hElAV0D/8OvMYI9eBr3ogZHFRy8YXINGhZfBZvxJH4JN6KL+LHiVwJc2g9BxjtNFcGrfmni5vf/5nVwxjTw99ZUzXH1KabrFVBe8CIXoWT5g8ePzvavFVdTq6Kl+Ir9L8Qh+IjVuANvjmvN2T1ObPfRU7axRDW2kjBA/BJINrTPbiGKibHdKBS1xuyau2btDLG/pfRov0JxhQ7uWcx5tvg0ucgmhppgy2d9+A/4r3tcQUPJ0rrqHJ1c7wDNt6EUe4lPr0WC6syUFPBdpE5jbF2jDGElTD6izHmbnhQHCBb657Gp9ga8jlv/kXrfcyZGAuYN7lKju+LhF6TBrwnaed1RJpvw99n2+bTvjvmNGiJ45ZGTDnc18rx2zlpbF0vV2BvCJqly3SFiqC4QXfoHq1THQKe0Ct6R++z5Wwt20hv9kxmdMUv0R9PtvkT7677bA==
AAAEoXicnVPNbtNAEJ6UACUUmnKqxMUiLUpQFW1ySUFCQioHOFS0adJWiqPIdjepU8e2bCe0WOEBeAEOnEBwQAgegQsXXoBDHwFxLBIXDnw7ToL4aZCwZe/MtzPffDu7a/qOHUZCHKdmzqTPnjs/eyFzce7S5fnswtx26PUDS9Ytz/GCXdMIpWO7sh7ZkSN3/UAaPdORO+bBmprfGcggtD23Fh35stkzOq7dti0jAtTKLq61Yrs71G5r63nHyduFFQ1Dt1BoZXOiWK6slkRJY0PcLI+NilYqCn5yNHo2vIXUe9JpjzyyqE89kuRSBNshg0K8DSqRIB9Yk2LqAjEogGdzjKQhZZDfBy4RZQA9wL8DrzFCXfiKN2QGC5UcfAEyNVoWn8RrcSI+ijfis/h+KlfMHErPEUYzyZV+a/7J4ta3f2b1MEa0/zNrquaI2rTKWm1o9xlRq7CS/MGjpydbt6rL8XXxQnyB/ufiWHzACtzBV+vVpqw+Y/a7yEm6GMBaHyl4AD4JRHmqBzdQReeYDlSqekNWrXydVibY/zIadPgHY4Kd3rMI821wqXMQTo00wZbMu/Af8t72uIKLE6V0VLm6PtkBE2/MKPcSn1qLgVVpqGnDdpA5jbH2G2MAK2Z0zBhxN1wo9pGtdE/js9ka8jlv/kXrfczpGHOY17lKhu+LhF6dBrwnSedVRJJvwj9k2+TTvjfh1GiJ45ZGTBnc1/Gl1E43tsvFEuxNQbN0la5RHhQVukP3aIPqEPCYXtJbepcupmvpRnKzZ1KjK36FfnnSzR81HPu8
AAAErHicnVPNbtNAEJ6UACX8NIVLJS4WaVGCqmiTAylISJXKAQ4VbZq0leIost1NcOrYlu2Elig8AC/AgRMIDgjBI3Dhwgtw6CMgjkXiwoFvx0kQP0klbNk78+3MN9/O7pq+Y4eREEeJmVPJ02fOzp5Lnb9w8dJcev7yduh1A0tWLc/xgl3TCKVju7Ia2ZEjd/1AGh3TkTvm/pqa3+nJILQ9txId+rLeMVqu3bQtIwLUSC+sNfp2e6Dd0dazjpO1c8sahnYu10hnRL5YWimIgsaGuFUcGSWtkBf8ZGj4bHjziQ+k0x55ZFGXOiTJpQi2QwaFeGtUIEE+sDr1qQ3EoACezTGSBpRCfhe4RJQBdB//FrzaEHXhK96QGSxUcvAFyNRoSXwWb8Sx+CTeii/ix0SuPnMoPYcYzThX+o25pwtb30/M6mCM6OGvrKmaI2rSCmu1od1nRK3CivN7j58db90uL/Wvi5fiK/S/EEfiI1bg9r5Zrzdl+Tmz30VO3MUA1vpQwQPwSSDKUz24gSo6x7SgUtUbsGrl67Q8xv6X0aCDvxhjbHLPIsw3waXOQTg10gRbPO/Cf8R72+EKLk6U0lHm6vp4B0y8fUa5l/jUWgysSkNNG7aDzGmMlT8YA1h9RkeMEXfDhWIf2Ur3ND6brQGf8/o/tN7HnI4xg3mdq6T4vkjo1anHexJ3XkXE+Sb8A7ZNPu17Y06NFjlucciUwn0dXUptsrFdzBdgb4rM6s3hzZ2lq3SNsuAq0Srdow2qQskTekXv6H0yn6wka8l6HDqTGOZcod+eZPMnJDb8xg==
lmm
(Z) =
P
i2L1, j2L2
Zij
Cij
AAAE6nicnVPPT9RAFH7gqrj+APRi4mXigkFDyHQPgiYmJHjQRCO/IVCyacsAA223absr2PQf8OjFGE4aPRj/BePFi/+AB67eDN4w8eLBb14XjAJrYpt23vvmve9982bGjXydpFLudHSeKJ08dbrrTPnsufMXunt6L84m9UbsqRmv7tfjeddJlK9DNZPq1FfzUaycwPXVnLsxZubnmipOdD2cTrcitRQ4q6Fe0Z6TAqr1zNrRmq5lduCka3GQ+UGQ5wML18UdYSeNoJZpYetQ8LTn+NmDvGYNivVDYDUXCwhez8UYD7WeihyqDo9Y0hJsyFvVfWNYWEOSnwq1nvF6b8cHsmmZ6uRRgwJSFFIK2yeHEryLZJGkCNgSZbQOxKEYnuYYRTmVkd8ArhDlAN3AfxXeYgsN4RvehBk8VPLxxcgU1C8/y7dyT36S7+RX+fNYrow5jJ4tjG6Rq6Ja99PLUz/+mRVgTGntd1ZbzSmt0Ahr1dAeMWJW4RX5zSfP96ZuT/Zn1+QruQv9L+WO/IgVhM3v3psJNbnN7HeRU3QxhvWwpeAR+BQQ45ke3EAVm2NWodLUy1m18W0aPMD+l9GhzUOMBXZ8z1LMr4DLnIOkbaQLtmI+hP+Y9zbgCiFOlNExydXtgx1w8WaMci/xmbU4WJVATQ3bR2Y7xum/GGNYGaP7jCl3I4TiCNlGdzs+zVbO53zpCK33MWdjrGDe5iplvi8Kem1q8p4UnTcRRb4Lf5Ntl0/78gGnoD6O62sxlXFf9y+lON6YrQ5ZsCesyujN1s3toit0lQbANUyjdI/GaQZK3tMX2qVvJb/0rPSitF2Edna0ci7RH0/p9S+8cxfb

View Slide

The OTMANN Distance
：Non-assignment penalty
n
n 層数の違いに対応したペナルティ
：Structural penalty
n ，
n 経路長を用いて接続の違いを表現したペナルティ
n は6パターンの経路長の平均値を意味する
15
nas
AAAEp3icnVPLbtNAFL0pAUp4NKUbJDYWaREgFE2ipiWIRSVYwAK1TZs0Uh1FY3eSmPol2w0NVn6AH2DBAoGERMU/sGHDD7DoJyCWRWLDgjNjN4hHUomxbN975t5zz9yZMXzbCiPGDjNTp7Knz5ydPpc7f+HipZn87OVG6O0Fpqibnu0FTYOHwrZcUY+syBZNPxDcMWyxZezel/NbfRGEluduRgNftBzeda2OZfIIUDs/p/s9qx3rDo96gRO7PBwO2/kCK5YqlcXyksaKS5VquSqN5UVAVa1UZGoUKB1r3mzmA+m0Qx6ZtEcOCXIpgm0TpxDPNpWIkQ+sRTE9AcIpgGepGEFDyiF/D7hAFAe6i28X3naKuvAlb6gYTFSy8QbI1GiBfWYH7Ih9Yu/ZF/ZjLFesOKSeAf5Gkiv89szzKxvfT8xy8I+o9ytrouaIOnRHabWg3VeIXIWZ5PefvTjauFtbiK+zN+wr9L9mh+wjVuD2v5lv10XtpWJ/gJykiwGsx6mCVfAJINKTPbiFKrqK6UKlrDdUqqWv0+0R9r+MnPb/Ykyw8T2LMN8BlzwH4cRIA2zJvAv/qdpbR1VwcaKkjpqqro92wMATK1T1Eq9cC8eqNNS0YNvInMS4+QdjACtW6DFjpLrhQrGPbKl7Ep+lrKE6561/aH2EOR3/AuZ1VSWn7ouAXp36ak+SzsuIJN+Av69sQ532nRGnRvMqbj5lyuG+Hl9KbbzRKBdLsNdLhZV76c2dpqt0jW6Aa5lW6CGtUR1KBvSK3tFB9mZ2NdvINpPQqUyaM0e/jSz/CSSX/cM=
AAAEp3icnVPLbtNAFL0pAUp4NKUbJDYWaREgFE2ipiWIRSVYwAK1TZs0Uh1FY3eSmPol2w0NVn6AH2DBAoGERMU/sGHDD7DoJyCWRWLDgjNjN4hHUomxbN975t5zz9yZMXzbCiPGDjNTp7Knz5ydPpc7f+HipZn87OVG6O0Fpqibnu0FTYOHwrZcUY+syBZNPxDcMWyxZezel/NbfRGEluduRgNftBzeda2OZfIIUDs/p/s9qx3rDo96gRO7PBwO2/kCK5YqlcXyksaKS5VquSqN5UVAVa1UZGoUKB1r3mzmA+m0Qx6ZtEcOCXIpgm0TpxDPNpWIkQ+sRTE9AcIpgGepGEFDyiF/D7hAFAe6i28X3naKuvAlb6gYTFSy8QbI1GiBfWYH7Ih9Yu/ZF/ZjLFesOKSeAf5Gkiv89szzKxvfT8xy8I+o9ytrouaIOnRHabWg3VeIXIWZ5PefvTjauFtbiK+zN+wr9L9mh+wjVuD2v5lv10XtpWJ/gJykiwGsx6mCVfAJINKTPbiFKrqK6UKlrDdUqqWv0+0R9r+MnPb/Ykyw8T2LMN8BlzwH4cRIA2zJvAv/qdpbR1VwcaKkjpqqro92wMATK1T1Eq9cC8eqNNS0YNvInMS4+QdjACtW6DFjpLrhQrGPbKl7Ep+lrKE6561/aH2EOR3/AuZ1VSWn7ouAXp36ak+SzsuIJN+Av69sQ532nRGnRvMqbj5lyuG+Hl9KbbzRKBdLsNdLhZV76c2dpqt0jW6Aa5lW6CGtUR1KBvSK3tFB9mZ2NdvINpPQqUyaM0e/jSz/CSSX/cM=
AAAEp3icnVPNbtNAEJ6UACX8NKUXJC4RaREgFG2ipiWIQyU4wAG1TZs0Uh1FtrtJTP0n2wkNVl6AF+DAAYGERMU7cOHCC3DoIyCOReLCgW/HaRCUBom1bM98O/PNt7O7hm9bYSTEQWrqVPr0mbPT5zLnL1y8NJOdvVwPvV5gyprp2V7QMPRQ2pYra5EV2bLhB1J3DFtuGbv31fxWXwah5bmb0cCXTUfvuFbbMvUIUCs7p/ldqxVrjh51Ayd29XA4bGXzolAslxdLSzlRWCpXShVlLC8CquSKBcEjT6Ox5s2mPpBGO+SRST1ySJJLEWybdArxbFORBPnAmhTTEyA6BfAsjpE0pAzye8AlonSgu/h24G2PUBe+4g2ZwUQlG2+AzBwtiM9iXxyKT+K9+CJ+nMgVM4fSM8DfSHKl35p5fmXj+z+zHPwj6v7Kmqg5ojbdYa0WtPuMqFWYSX7/2YvDjbvVhfi6eCO+Qv9rcSA+YgVu/5v5dl1WXzL7A+QkXQxgPR4pWAWfBKI81YNbqKJxTAcqVb0hq1a+RrfH2P8y6rR3jDHBTu5ZhPk2uNQ5CCdGGmBL5l34T3lvHa7g4kQpHVWuro13wMATM8q9xKvWomNVOdS0YNvInMS4+QdjACtm9Igx4m64UOwjW+mexGexNeRz3vyL1keY0/DPY17jKhm+LxJ6NerzniSdVxFJvgF/j22DT/vOmDNH8xw3P2LK8H1NLmXuuHF0X+ulQhH2ejG/cm90c6fpKl2jG+BaphV6SGtUg5IBvaJ3tJ++mV5N19ONJHQqNcqZo99GWv8JI0H9wA==
str
AAAEp3icnVPLbtNAFL0pAUp4NKUbJDYWaREgFE2ipiWIRSVYwAK1TZs0Uh1FtjtJTP2SPQ0NVn6AH2DBAoGERMU/sGHDD7DoJyCWRWLDgjNjN4hHUomxbN975t5zz9yZMQPHjgRjh5mpU9nTZ85On8udv3Dx0kx+9nIj8vdCi9ct3/HDpmlE3LE9Xhe2cHgzCLnhmg7fMnfvy/mtPg8j2/c2xSDgLdfoenbHtgwBqJ2f04Oe3Y511xC90I0jEQ6H7XyBFUuVymJ5SWPFpUq1XJXG8iKgqlYqMjUKlI41fzbzgXTaIZ8s2iOXOHkkYDtkUIRnm0rEKADWopieADEohGerGE5DyiF/DzhHlAF0F98uvO0U9eBL3kgxWKjk4A2RqdEC+8wO2BH7xN6zL+zHWK5YcUg9A/zNJJcH7ZnnVza+n5jl4i+o9ytromZBHbqjtNrQHihErsJK8vvPXhxt3K0txNfZG/YV+l+zQ/YRK/D636y367z2UrE/QE7SxRDW41TBKvg4EOnJHtxCFV3FdKFS1hsq1dLX6fYI+19Gg/b/Ykyw8T0TmO+AS56DaGKkCbZk3oP/VO2tqyp4OFFSR01V10c7YOKJFap6iVeuxcCqNNS0YTvInMS4+QdjCCtW6DGjUN3woDhAttQ9ic9W1lCd89Y/tD7CnI5/AfO6qpJT94VDr059tSdJ52VEkm/C31e2qU77zohTo3kVN58y5XBfjy+lNt5olIsl2Oulwsq99OZO01W6RjfAtUwr9JDWqA4lA3pF7+ggezO7mm1km0noVCbNmaPfRtb4CYUb/do=
AAAEp3icnVPLbtNAFL0pAUp4NKUbJDYWaREgFE2ipiWIRSVYwAK1TZs0Uh1FtjtJTP2SPQ0NVn6AH2DBAoGERMU/sGHDD7DoJyCWRWLDgjNjN4hHUomxbN975t5zz9yZMQPHjgRjh5mpU9nTZ85On8udv3Dx0kx+9nIj8vdCi9ct3/HDpmlE3LE9Xhe2cHgzCLnhmg7fMnfvy/mtPg8j2/c2xSDgLdfoenbHtgwBqJ2f04Oe3Y511xC90I0jEQ6H7XyBFUuVymJ5SWPFpUq1XJXG8iKgqlYqMjUKlI41fzbzgXTaIZ8s2iOXOHkkYDtkUIRnm0rEKADWopieADEohGerGE5DyiF/DzhHlAF0F98uvO0U9eBL3kgxWKjk4A2RqdEC+8wO2BH7xN6zL+zHWK5YcUg9A/zNJJcH7ZnnVza+n5jl4i+o9ytromZBHbqjtNrQHihErsJK8vvPXhxt3K0txNfZG/YV+l+zQ/YRK/D636y367z2UrE/QE7SxRDW41TBKvg4EOnJHtxCFV3FdKFS1hsq1dLX6fYI+19Gg/b/Ykyw8T0TmO+AS56DaGKkCbZk3oP/VO2tqyp4OFFSR01V10c7YOKJFap6iVeuxcCqNNS0YTvInMS4+QdjCCtW6DGjUN3woDhAttQ9ic9W1lCd89Y/tD7CnI5/AfO6qpJT94VDr059tSdJ52VEkm/C31e2qU77zohTo3kVN58y5XBfjy+lNt5olIsl2Oulwsq99OZO01W6RjfAtUwr9JDWqA4lA3pF7+ggezO7mm1km0noVCbNmaPfRtb4CYUb/do=
AAAEp3icnVPNbtNAEJ6UACX8NKUXJC4RaREgFG2ipiWIQyU4wAG1TZs0Uh1FtrtJTP0n2wkNVl6AF+DAAYGERMU7cOHCC3DoIyCOReLCgW/HaRCUBom1bM98O/PNt7O7hm9bYSTEQWrqVPr0mbPT5zLnL1y8NJOdvVwPvV5gyprp2V7QMPRQ2pYra5EV2bLhB1J3DFtuGbv31fxWXwah5bmb0cCXTUfvuFbbMvUIUCs7p/ldqxVrjh51AycOo2A4bGXzolAslxdLSzlRWCpXShVlLC8CquSKBcEjT6Ox5s2mPpBGO+SRST1ySJJLEWybdArxbFORBPnAmhTTEyA6BfAsjpE0pAzye8AlonSgu/h24G2PUBe+4g2ZwUQlG2+AzBwtiM9iXxyKT+K9+CJ+nMgVM4fSM8DfSHKl35p5fmXj+z+zHPwj6v7Kmqg5ojbdYa0WtPuMqFWYSX7/2YvDjbvVhfi6eCO+Qv9rcSA+YgVu/5v5dl1WXzL7A+QkXQxgPR4pWAWfBKI81YNbqKJxTAcqVb0hq1a+RrfH2P8y6rR3jDHBTu5ZhPk2uNQ5CCdGGmBL5l34T3lvHa7g4kQpHVWuro13wMATM8q9xKvWomNVOdS0YNvInMS4+QdjACtm9Igx4m64UOwjW+mexGexNeRz3vyL1keY0/DPY17jKhm+LxJ6NerzniSdVxFJvgF/j22DT/vOmDNH8xw3P2LK8H1NLmXuuHF0X+ulQhH2ejG/cm90c6fpKl2jG+BaphV6SGtUg5IBvaJ3tJ++mV5N19ONJHQqNcqZo99GWv8Jg8X91w==
nas
(Z) =
P
i2L1
(lm(i)
P
j2L2
Zij
) +
P
j2L2
(lm( j)
P
i2L1
Zij
)
AAAFPHicnVPNbtNAEB6XACVAm8IFicuKtCjlJ1pHiBakokpwAAlE/1u1riLb3SSb2o5lO6HF8gvwAhw4gYQE4h164cILcOidC6o4FcGFA7NjNxW0CRJr2Z75duabb2d3Ld+RYcT5rjZwInfy1OnBM/mz584PDRdGLiyFrXZgi0W75bSCFcsMhSM9sRjJyBErfiBM13LEsrV5X80vd0QQypa3EG37Yt01656sSduMEKoW3hl+Q1ZjwzWjRuDGnhkmSWl1nE0xI2y71VgyQ3qMpm3TiR8nVT1hJcctyXF2M4tpHompJGwVc5vJOLveL0gRNQ+Jji2WEVULRV6uTEzqXGdk8DuVA2OC6WVOowjZmGmNaDtgwAa0wIY2uCDAgwhtB0wI8VkDHTj4iK1DDE1ETAjQkxQjIIE85rcRFxhlIrqJ3zp6axnqoa94Q2KwsZKDb4CZDMb4Z/6e7/NP/AP/yn/15IqJQ+nZxr+V5gq/Ovzi0vzPf2a5+I+gcZjVV3MENZgkrRK1+4SoVdhpfuf5y/35u3Nj8VX+hu+h/td8l3/EFXid7/bbWTH3itgfYE7axQCtJ5mCp8gnEFGe6sE1rGJQTB1VqnoJqVa+ATe62P8ymrB1hDHFevcswvkacqlzEPaNtJAtnffQf0Z761IFD0+U0jFH1Y3uDlj4xIRSL/FVazFxVQxrSrQdzOzHuPAXY4BWTOgBY0Td8FCxj9lKdz8+SVZC53z9GK2PcM7AfxHnDaqSp/siUK8BHdqTtPMqIs230N8i26LTvtHlZDBKcaMZUx7v68GlZL2NpUpZR3tWL07fzm7uIFyGK1BCrgmYhocwA4tga0PaLW1Ku5fbyX3J7eW+paEDWpZzEf4YuR+/AX00Mw4=
AAAFMXicnVNNa9RQFL2po9axtlNXgptgW5n6Ud4UQREqgi4UFPvd0qYMSfravmmSCUlmbA35A/4BF64UBMX/0I0b/4CL7t1IcVXBjQvPu4ktdtoRfCHJvefee+55X07oqTgRYtfoOVU6feZs77ny+b4L/QOVwb75uNmKXDnnNr1mtOjYsfRUIOcSlXhyMYyk7TueXHA2H+j4QltGsWoGs8l2KFd8ez1Qa8q1E0D1ynsr3FD11PLtZCPy08COs6y6NGpOmFbc8uupMi0VmBx2bS99ktVrmVn1/KoaNW8WOY2OnPHMXEJtIxs1r3dL0kSNQ6JjmxVE9cqQGBM8zE6jVhhDVIzJ5qCxQxatUpNcapFPkgJKYHtkU4xnmWokKAS2Qik1gNgUwVOcIymjMupbwCWybKCb+K7DWy7QAL7mjZnBRScPb4RKk0bEF/FB7IvP4qP4Jn6dyJUyh9azjb+T18qwPvDy0szPf1b5+Ce0cVjVVXNCa3SHtSpoDxnRs3Dz+vaLV/szd6dH0qvirdiD/jdiV3zCDIL2D/fdlJx+zewPUZOvYgTraaHgGfgkEO3pNbiGLhbnrEOl7pexau1bdOMA+19Gm7Y6GHPs5DVLEF8Dlz4HcddMB2x5PID/nPfW5w4BTpTWMc3drYMdcPCkjPJa4tVzsTErEz0VbA+V3RhnjzBGsFJG/zAmvBoBFIeo1rq78Sm2Mj7nK8dofYyYhf8Q4hZ3KfN9kdBrUZv3JF95nZHXO/C32Hb4tK8ecJo0zHnDBVMZ97V29HZ2GvPjYzXYU4J66TJdoSoobtN9ekSTNEeu0W/cMiaMe6Wd0tfSXn6ze4ziil+kv0bp+2/p2jEj
AAAFMXicnVNLa9RQFD6po9ax9uFKcHOxrUx9DDeD0CpUBF0oKPbd0qaEJL2dudO8SDJja8gf8A+4cKUgKP6Hbtz4B1x070aKqwpuXHjuSTpF25mCNyQ55zvnfOe7Lzt0ZZxwvqf1nSmdPXe+/0L54sClwaHhkYGlOGhFjlh0AjeIVmwrFq70xWIiE1eshJGwPNsVy/bWQxVfbosoloG/kOyEYt2z6r7clI6VIGQOfzDChjRTw7OSRuSlvhVnWWV1gk0zI255ZiqZIX1GYcdy06eZqWes4noVOcFuFznNYzm1jK1ibTObYDd7JSmi5hHRic0KInN4lFdrk1M61xkZ/G7t0JhkepXTGIVizAQj2i4YsAEBONACDwT4kKDtggUxPmugA4cQsXVIoYmIBRF6knIEZFDG+hbiArMsRLfwW0dvrUB99BVvTAwOdnLxjbCSwTj/yj/yA/6Ff+Lf+e+uXClxKD07+LfzWhGaQ6+uzP86tcrDfwKNo6qemhPYhCnSKlF7SIiahZPXt1++Ppi/NzeeXufv+D7qf8v3+Gecgd/+6byfFXNviP0R1uSrGKH1rFDwHPkEIspTa3ADuxiUU0eVql9GqpVvwK0O9r+MFmwfY8yx7muWYHwTudQ5iHtm2siWx330X9DeetTBxxOldMxRd6OzAzY+KaG0lviquVg4K4Y9JdouVvZiXPiHMUIrJfSQMaHV8FFxiNVKdy8+SVZG53z9BK1PMGbgfxTjBnUp030RqNeANu1JvvIqI6+30d8m26bTvtHhZDBGeWMFUxnv6+GlZN2NpVpVR3uWQz9chWtQQYpJeACPYQYWwdEGtTvatHa/tFv6VtrPb3afVlzxy/DXKP34Ay9IMXM=
AAAFPHicnVNLb9NAEB4XAyVAH3BB4rIiLUp5RJsIkYJUVAkOIIFo06daV5HtbtpN/ZLthBbLf4A/wIETSEgg/kMvXPgDHHrngipORXDhwOzYTQVtg8Ratme+nfnm29ldK3BkFHO+o/Wd0E+eOt1/pnD23PmBwaHhC/OR3w5tMWf7jh8uWmYkHOmJuVjGjlgMQmG6liMWrI37an6hI8JI+t5svBWIFddc82RT2maMUGPonRGsy0ZiuGa8HrqJZ0ZpWloaYxPMiNpuI5HMkB6jadt0ksdpo5KykuOW5Bi7mce0DsVUU7aEua10jF3vFaSIWgdERxbLiRpDRV6u1sYrvMLI4Heq+0aNVcqcRhHyMeUPa9tgwCr4YEMbXBDgQYy2AyZE+CxDBTgEiK1AAi1ETAjRkxQjIIUC5rcRFxhlIrqB3zX0lnPUQ1/xRsRgYyUH3xAzGYzyz/w93+Of+Af+lf86lishDqVnC/9WliuCxuCLSzM//5nl4j+G9YOsnppjaMI4aZWoPSBErcLO8jvPX+7N3K2PJlf5G76L+l/zHf4RV+B1vttvp0X9FbE/wJysiyFaT3IFT5FPIKI81YNrWMWgmDVUqeqlpFr5BtzoYv/LaMLmIcYMO75nMc43kUudg6hnpIVs2byH/jPaW5cqeHiilI46VTe6O2DhkxBKvcRXrcXEVTGsKdF2MLMX4+xfjCFaCaH7jDF1w0PFAWYr3b34JFkpnfOVI7Q+wjkD/0WcN6hKge6LQL0GdGhPss6riCzfQn+TbItO+2qXk8EIxY3kTAW8r/uXkh1vzFfLFbSneXHydn5z++EyXIESctVgEh7CFMyBrQ1ot7QJ7Z6+rX/Rd/VvWWifludchD+G/uM3fOQzDQ==
str
(Z) =
P
i2L1, j2L2
Zij
Cij
AAAE6nicnVPPT9RAFH7gqrj+APRi4mXigkFDyHQPgiYmJHjQRCO/IVCyacsAA223absr2PQf8OjFGE4aPRj/BePFi/+AB67eDN4w8eLBb14XjAJrYpt23vvmve9982bGjXydpFLudHSeKJ08dbrrTPnsufMXunt6L84m9UbsqRmv7tfjeddJlK9DNZPq1FfzUaycwPXVnLsxZubnmipOdD2cTrcitRQ4q6Fe0Z6TAqr1zNrRmq5lduCka3GQJWmc5wML18UdYSeNoJZpYetQ8LTn+NmDvGYNivVDYDUXCwhez8UYD7WeihyqDo9Y0hJsyFvVfWNYWEOSnwq1nvF6b8cHsmmZ6uRRgwJSFFIK2yeHEryLZJGkCNgSZbQOxKEYnuYYRTmVkd8ArhDlAN3AfxXeYgsN4RvehBk8VPLxxcgU1C8/y7dyT36S7+RX+fNYrow5jJ4tjG6Rq6Ja99PLUz/+mRVgTGntd1ZbzSmt0Ahr1dAeMWJW4RX5zSfP96ZuT/Zn1+QruQv9L+WO/IgVhM3v3psJNbnN7HeRU3QxhvWwpeAR+BQQ45ke3EAVm2NWodLUy1m18W0aPMD+l9GhzUOMBXZ8z1LMr4DLnIOkbaQLtmI+hP+Y9zbgCiFOlNExydXtgx1w8WaMci/xmbU4WJVATQ3bR2Y7xum/GGNYGaP7jCl3I4TiCNlGdzs+zVbO53zpCK33MWdjrGDe5iplvi8Kem1q8p4UnTcRRb4Lf5Ntl0/78gGnoD6O62sxlXFf9y+lON6YrQ5ZsCesyujN1s3toit0lQbANUyjdI/GaQZK3tMX2qVvJb/0rPSitF2Edna0ci7RH0/p9S8RMxfu
Cij = 1/6
P
s2{sp,lp,rw}
P
t2{ip,op}
| s
t
(i) s
t
( j)|
AAAFFHicnVPLThRBFL2Do+KIAhqNiZuKAwYMYjULQBMTElzowsgbEhon3U0BBf1Kd88ANp249gN04UoTF8ZfMG6IiSt3LvgBE+MSEzcuPFXdYBxlTKxOdd176t5zT73s0JVxwvleqe1Y+fiJk+2nKqc7zpzt7Oo+NxcH9cgRs07gBtGCbcXClb6YTWTiioUwEpZnu2Le3hhX8/MNEcUy8GeS7VAsedaqL1ekYyWAal2Px2upXM/YbWbcGGZmXPdqacxM6TMzZaZnJWuRl8bhAHPRo83MzLIiKmmOkogIQkxnLGM7zFwWbmI9TOMMsVmf7GfXm7H1frZT66rywaGRUYMbTBv85tCBMcKMQa5blYo2EXSX3pFJyxSQQ3XySJBPCWyXLIrxLZJBnEJgS5TSOhCLInhSxwjKqIL8OnCBKAvoBv6r8BYL1IeveGPN4KCSix4hk1Ev/8Rf833+gb/hX/iPI7lSzaH0bGO081wR1jqfXJr+/s8sD2NCa7+yWmpOaIVGtVYJ7aFG1CqcPL/x6Nn+9K2p3vQqf8m/Qv8Lvsd3sQK/8c15NSmmnmv2O8jJdzGCdb9Q8AB8Aojy1B5cQxVTx6xCpaqXadXKN2ngEPtfRou2/mDMsaP3LMH8CrjUPYhbRtpgy+d9+Jv6bD1dwceNUjqmdHXz8ARsfKlG9V6iq7VYWBVDTQnbRWYrxpkmxghWqtEDxkTvhg/FIbKV7lZ8UluZvudLf9F6D3MmxirmTV2lot+LgF6TGvpM8p1XEXm+DX9L27a+7cuHnIx6dFxPwVTBez14lOxoY25o0IA9aVTHhouX206X6Qr1gWuExuguTdAslHwudZQulC6Wn5bflnfL7/PQtlKRc55+a+WPPwG2riSZ
Cij
AAAEmHicnVNNb9NAEJ0UAyV8tIULgotFWoQQitY5kMIpUpGgB0SbNmmlOopsdxOcOrZlO6Elyh/gxA0BJ5A4IP4DFy78AQ79CYhjkbhw4O3YCeIjQWIt2zNvZ968nd21Q8+NEyEOczPHtOMnTs6eyp8+c/bc3PzC+Xoc9CJH1pzAC6Jt24ql5/qylriJJ7fDSFpd25Nb9t6Kmt/qyyh2A38zOQhlo2u1fbflOlYCqL7SHLidYXO+IIql8rIhDJ0Ncas0Msq6URQ8CpSNtWAh955M2qWAHOpRlyT5lMD2yKIYzw4ZJCgE1qABdYBYFMFzOUbSkPLI7wGXiLKA7uHbhreToT58xRszg4NKHt4ImTotiU/irTgSH8U78Vl8n8g1YA6l5wB/O82VYXPuycWNb//M6uKf0MOfWVM1J9SiZdbqQnvIiFqFk+b3Hz872rhdXRpcFa/FF+h/JQ7FB6zA73913qzL6ktmv4OctIsRrPuZggfgk0CUp3pwHVVMjmlDpao3ZNXKN+nGGPtfRov2/2BMsck9SzDfApc6B/HUSBts6bwP/xHvbZcr+DhRSkeVq5vjHbDxDBjlXuJVa7GwKh01XdgeMqcxbv7GGMEaMDpiTLgbPhSHyFa6p/G5bA35nDf+onUVcyb+BcybXCXP90VCr0l93pO08yoizbfh77Nt82nfHXPqtMhxixlTHvd1dCn1yUa9VDRgrxuFys3s5s7SZbpC18BVpgrdozWqQUmHntJzeqFd0iraXW01DZ3JZTkX6JehVX8ANtr3Vg==

View Slide

The OTMANN Distance
Theorem 1｜OTMANNは擬距離である
n OTMANNは擬距離であることをAppendix A.2にて証明
n （OTMANNを含む）擬距離の性質：
- 非負性：
- 対称性：
- 三角不等式：
- ※通常の距離と異なりとは限らない
n 異なる構造であってもOTMANNがゼロとなるケースが存在
16
d(G1, G2
) 0
AAAEvnicnVM9b9RAEJ0LBsLxkQQaJBqLS1CIotP6mgSqk0ACCkRy+ZTi08n27V1M/IXtOxIs/wEqOgoqkCgQPSUg0fAHKPITEGWQaCh4O74EBchFYi3bM29n3ryd3bUjz01SIXZLIye0k6dOj54pnz13/sLY+MTF1STsxY5ccUIvjNdtK5GeG8iV1E09uR7F0vJtT67ZW7fU/FpfxokbBsvpTiSbvtUN3I7rWCmg1vhMe9r0rXTTsbzsTt7KjHxWPwTU8uu62ZWPdNEar4hqbW7eEIbOhrhR2zfmdKMqeFRoMBbCidJ7MqlNITnUI58kBZTC9siiBM8GGSQoAtakjB4CsSiG53KMpJzKyO8Bl4iygG7h24W3MUAD+Io3YQYHlTy8MTJ1mhJfxBuxJz6Lt+Kr+HkkV8YcSs8O/naRK6PW2NPLSz+OzfLxT2nzd9ZQzSl1aJ61utAeMaJW4RT5/SfP95ZuNqaya+KV+Ab9L8Wu+IQVBP3vzutF2XjB7LeRU3QxhnV/oOAB+CQQ5akezKCKyTFdqFT1clatfJNmD7D/ZbRo+y/GAju6ZynmO+BS5yAZGmmDrZgP4D/mvfW5QoATpXQ0uLp5sAM2noxR7iVetRYLq9JR04XtIXMY4/IfjDGsjNF9xpS7EUBxhGylexify1bO57z5D633MGfiX8G8yVXKfF8k9JrU5z0pOq8iinwb/jbbNp/29gGnTpMcNzlgKuO+7l9K/WhjtVY1YC8alboxuLmjdIWu0jS45qhOd2mBVqDkGb2jD/RRq2sdzdfCInSkNMi5RIeGtv0LhggFkQ==
d(G1, G2
) = d(G2, G1
)
AAAE3XicnVO/b9NAFH4pAUr40RQWJBaLtKhFVXTOkoKEVAkk6IDor7SV6iiynUtq6jiW7YYWyyMLQqwdmEBiQAz8Byws/AMdusCMGIvEwsB3z0lQC0klzrL93nfvfe+7d3eW7zphJMRBZuRU9vSZs6PncucvXLw0lh+/vBq2twNbVuy22w7WLTOUruPJSuRErlz3A2m2LFeuWVt31fxaRwah0/ZWol1fVltm03Majm1GgGr5+fqU0TKjTdt04/tJLdaTGe0IUEqmtTvasajS8Sg9ma7lC6JYKs/qQtfYELdKPaOs6UXBo0DdsdAez3wkg+rUJpu2qUWSPIpgu2RSiGeDdBLkA6tSTI+BmBTAczhGUkI55G8Dl4gygW7h24S30UU9+Io3ZAYblVy8ATI1mhT74p04FJ/Fe/FN/BrIFTOH0rOLv5XmSr829vzq8s8Ts1r4R7T5J2uo5ogaNMtaHWj3GVGrsNP8ztO9w+XbS5PxDfFGfIf+1+JAfMIKvM4P++2iXHrF7PeQk3YxgPWwq+AR+CQQ5ake3EQVg2OaUKnqJaxa+QbN9LH/ZTRp5y/GFBvcswjzDXCpcxAOjbTAls578J/w3ra4gocTpXQscXWjvwMWnphR7iVetRYTq9JQ04HtInMY48oxxgBWzGiPMeJueFDsI1vpHsbnsJXwOa/+Q+s85gz8C5g3uEqO74uEXoM6vCdp51VEmm/B32Hb4tNe73NqNMFxE12mHO5r71Jqg43VUlGHvagX5vTuzR2la3SdpsBVpjl6QAtUgZIPtE9f6Gu2ln2WfZF9mYaOZLo5V+jIyO79BrmiEjc=
AAAE0nicnVO/b9NAFH4pAUooNGVCYrFIi1pUVecsICQkJJCgA6K/0laqo8h2Lqmpf8l2QovlkQUhVgYmkBgQA/8BCwv/QIcuMCPGIrEw8N1zCGpCg8RZtt/77r3vfffuzgpdJ06EOCiMnSiePHV6/Ezp7MS585PlqYn1OOhEtqzZgRtEm5YZS9fxZS1xElduhpE0PcuVG9bObTW/0ZVR7AT+WrIXyrpntn2n5dhmAqhRXmzOGp6ZbNumm97NGqmezWtHgGo2p93UBqKqg1F6NtcoV8SC4KENG3rPqFBvLAVThQ9kUJMCsqlDHknyKYHtkkkxni3SSVAIrE4pPQRiUgTP4RhJGZWQ3wEuEWUC3cG3DW+rh/rwFW/MDDYquXgjZGo0I/bFW3EoPol34qv4eSxXyhxKzx7+Vp4rw8bk04urP/6Z5eGf0PafrJGaE2rRddbqQHvIiFqFned3H784XL2xMpNeEa/FN+h/JQ7ER6zA73633yzLlZfMfgc5eRcjWPd7Ch6ATwJRnurBVVQxOKYNlapexqqVb9B8H/tfRpN2hxhz7PieJZhvgUudg3hkpAW2fN6H/4j31uMKPk6U0rHC1Y3+Dlh4Uka5l3jVWkysSkNNB7aLzFGMawOMEayU0d+MCXfDh+IQ2Ur3KD6HrYzPef0vWhcxZ+BfwbzBVUp8XyT0GtTlPck7ryLyfAv+LtsWn/Zmn1OjaY6b7jGVcF/1wds5bKxXF3TYy4LG6RJdpllQXKNbdI+WqAYB72mfPtOXYqP4pPgsv9ljhd4Vv0BHRvH5L6O3ELA=
AAAE0nicnVMxb9NAFH4pAUooNGVCYrFoi1pUVecsKUhISCBBB0SbNm2lOops95qa2o5lO6HF8siCECsDE0gMiIF/wMLCH+jQBWbEWCQWBr57ToIaSCpxlu33vnvve9+9u7MC14liIQ5zI6fyp8+cHT1XOD924eJ4cWJsLWq2QltW7abbDDcsM5Ku48tq7MSu3AhCaXqWK9et3Ttqfr0tw8hp+qvxfiBrntnwnW3HNmNA9eLi1ozhmfGObbrJvbSe6OmcdgwopbPaLa0vqtQfpaez9eKkmC+VF3Sha2yIG6WuUdb0ecFjkjpjqTmR+0gGbVGTbGqRR5J8imG7ZFKEZ5N0EhQAq1FCj4CYFMJzOEZSSgXkt4BLRJlAd/FtwNvsoD58xRsxg41KLt4QmRpNiwPxThyJz+K9+CZ+DeRKmEPp2cffynJlUB9/dnnl54lZHv4x7fzJGqo5pm1aYK0OtAeMqFXYWX77ycujlZuV6eSaeCO+Q/9rcSg+YQV++4f9dllWXjH7XeRkXQxhPegoeAg+CUR5qgfXUcXgmAZUqnopq1a+QXM97H8ZTdr7izHDBvcsxvw2uNQ5iIZGWmDL5n34j3lvPa7g40QpHRWubvR2wMKTMMq9xKvWYmJVGmo6sF1kDmNc7WMMYSWMdhlj7oYPxQGyle5hfA5bKZ/z2j+0LmLOwH8S8wZXKfB9kdBrUJv3JOu8isjyLfh7bFt82rd6nBpNcdxUh6mA+9q9lNpgY600r8NeFjRKV+gqzYCiTLfpPi1RFQI+0AF9oa/5ev5p/nl2s0dynSt+iY6N/Ivf6RYRAA==
AAAE3XicnVM9b9RAEJ0LBsLxkQQaJBqLS1CCotP6mgQkpEggQQpEvhMpjk62s5eY+Eu270iwXNIgRJuCCiQKRME/oKHhD6RIAzWiDBINBW/Hl0M5yEViLdszb2fevJ3dtSPPTVIh9kt9p7TTZ872nyufv3Dx0sDg0OWlJGzGjlx0Qi+MV2wrkZ4byMXUTT25EsXS8m1PLttbd9X8ckvGiRsGC+lOJNd8ayNwG65jpYDqg9Pro6ZvpZuO5WX383pm5OP6EaCWj+l39K6oWneUkY/VByuiWpuYNIShsyFu1Q6NCd2oCh4Vao+ZcKj0kUxap5AcapJPkgJKYXtkUYJnlQwSFAFbo4weA7EohudyjKScyshvApeIsoBu4bsBb7WNBvAVb8IMDip5eGNk6jQi9sQ7cSA+i/fim/h1LFfGHErPDv52kSuj+sDzq/M/T8zy8U9p809WT80pNWiStbrQHjGiVuEU+a2nuwfzt+dGshvijfgO/a/FvviEFQStH87bWTn3itnvIafoYgzrYVvBI/BJIMpTPbiJKibHbEClqpezauWbNN7B/pfRou2/GAvs+J6lmG+AS52DpGekDbZiPoD/hPfW5woBTpTSMcfVzc4O2HgyRrmXeNVaLKxKR00XtofMXowLXYwxrIzRQ8aUuxFAcYRspbsXn8tWzud87R9apzFn4l/BvMlVynxfJPSa1OI9KTqvIop8G/422zaf9vUOp07DHDfcZirjvh5eSv14Y6lWNWDPisqU0b65/XSNrtMouCZoih7QDC1CyQfaoy/0Vatrz7QX2ssitK/UzrlCR4a2+xu5UhI2
d(G1, G3
)  d(G1, G2
) + d(G2, G3
)
AAAFBnicnVNNa9RQFL1TR1vHj7a6EdwEp5WqpbyMi1ZXBQV1IbbTT2iGIcm8aWMzSUzSsTVkL/4BF64UXIjgTtyJ4MY/INifIC4ruBHxvJtMpR8zBV9Icu9595573n3vWYHrRLEQ24W+Y8XjJ/oHTpZOnT5zdnBo+Nxi5G+EtlywfdcPly0zkq7jyYXYiV25HITSbFmuXLLWb6n5pbYMI8f35uOtQNZa5qrnNB3bjAHVh8LGmNEy4zXbdJM7aT3R03FtD3A9vaIZrnykHRVYQeC1/VGVQ+jqQ2UxUZmc0oWusSFuVDrGpKZPCB5lyseMP1z4SAY1yCebNqhFkjyKYbtkUoRnhXQSFACrUUIPgZgUwnM4RlJKJeRvAJeIMoGu47sKbyVHPfiKN2IGG5VcvCEyNRoVX8UbsSO+iLfiu/jdlSthDqVnC38ry5VBffDZhblfR2a18I9p7V9WT80xNWmKtTrQHjCiVmFn+e0nz3fmblZHk8vilfgB/S/FtviMFXjtn/brWVl9wey3kZN1MYR1P1fwAHwSiPJUD66iisExq1Cp6qWsWvkGje9i/8to0uYBxgzr3rMY801wqXMQ9Yy0wJbNe/Af8962uIKHE6V0VLm6sbsDFp6EUe4lXrUWE6vSUNOB7SKzF+P8PsYQVsJohzHmbnhQHCBb6e7F57CV8jmvHaL1HuYM/MuYN7hKie+LhF6D2rwnWedVRJZvwd9k2+LT3tjl1GiE40ZyphLua+dSat2NxcqEDntWL0/r+c0doIt0icbANUnTdJdmaAFKvtGfQn9hoPi0+K74vvghC+0r5Dnnac8ofvoLL7Ygng==
AAAE+3icnVPPaxNBFH6p0cZYbepJ8LKYVqqWMokHxZOgoB7ENm3aQreE3c0kXbu/3N3E1mXv4j/gwZOCBxG8iTcvXvwHBPsniMcKXkT85m2MNGlTcJbdfe+b9773zZsZM3DsKBZiNzd2LH/8xHjhZPHUxOkzk6WpiZXI74SWrFu+44drphFJx/ZkPbZjR64FoTRc05Gr5tYtNb/alWFk+95yvBPIDddoe3bLtowYUKMUNmd114g3LcNJ7qSNpJLOafuAq+klTXfkI+2owCoCrwxGVQ+ga5TKYl7w0IaNSs8oU28s+FO5j6RTk3yyqEMuSfIohu2QQRGedaqQoADYBiX0EIhBITybYySlVER+B7hElAF0C982vPUe6sFXvBEzWKjk4A2RqdGM+CLeiD3xWbwV38SvQ7kS5lB6dvA3s1wZNCafnVv6eWSWi39Mm/+yRmqOqUXXWasN7QEjahVWlt998nxv6UZtJrkoXonv0P9S7IpPWIHX/WG9XpS1F8x+GzlZF0NY93sKHoBPAlGe6sFlVNE5pg2Vql7KqpWv01wf+19Gg7aHGDPs8J7FmG+BS52DaGSkCbZs3oP/mPfW5QoeTpTSUePqen8HTDwJo9xLvGotBlaloaYN20HmKMblAcYQVsLoX8aYu+FBcYBspXsUn81Wyud84wCt9zCn41/GvM5VinxfJPTq1OU9yTqvIrJ8E/422yaf9mafU6NpjpvuMRVxXyuDt3PYWKnOV2AvCirQebpAs6C4RjfpLi1QHQK+0u/ceK6Qf5p/l3+f3eyxXO+Kn6V9I//hD3C/Hu4=
AAAE+3icnVPBbtNAEJ2UQEMoNOWExMUiLSpQRetwSOGEBBJwQLRp01aqo8h2Nqmp4xjbCS1W7ogf4MAJJA4IiRvixoULP4BEPwFxLBIXhHg7ToNamlTqWrZn3s68eTu7a/muE0ZC7KTGTqRPnhrPnM6emTh7bjI3NbEStjuBLSt2220Ha5YZStfxZCVyIleu+YE0W5YrV63N22p+tSuD0Gl7y9G2L6sts+k5Dcc2I0C1XFCfNVpmtGGbbny3V4v13py2D7jeu6IZrnysHRVYROC1g1HFQ+hqubwoFEvzutA1NsSN4p5R0vSC4JGn/lhoT6U+kUF1apNNHWqRJI8i2C6ZFOJZJ50E+cCqFNMjICYF8ByOkdSjLPI7wCWiTKCb+DbhrfdRD77iDZnBRiUXb4BMjWbEV/FW7Iov4p34Ln4P5YqZQ+nZxt9KcqVfm3x+YenXkVkt/CPa+Jc1UnNEDZpnrQ60+4yoVdhJfvfpi92lm+WZ+LJ4LX5A/yuxIz5jBV73p/1mUZZfMvsd5CRdDGA96Ct4CD4JRHmqB1dRxeCYJlSqej1WrXyD5gbYcRlN2vqPMcGG9yzCfANc6hyEIyMtsCXzHvwnvLctruDhRCkdZa5uDHbAwhMzyr3Eq9ZiYlUaajqwXWSOYlw+wBjAihndY4y4Gx4U+8hWukfxOWz1+JxXD9F6H3MG/nnMG1wly/dFQq9BXd6TpPMqIsm34G+xbfFprw84NZrmuOk+Uxb3de9SasONlWJBh70oKEMX6RLNgqJEt+geLVAFAr7Rn9R4KpN+ln6f/pDc7LFU/4qfp30j/fEvth4fPg==
AAAFBnicnVNNa9RQFL1Tox3Hj051I7gJTitVS3kZF62uCgrqQmynnbbQlCFJ37Sx+TJJx9Ywe/EPuHCl4EIEd+JOBDf+AcH+BHFZwY2I591MR/o1BV9Icu9595573n3v2ZHnJqkQW4W+Y9rxE/3Fk6VTp8+cHSgPnptLwvXYkXUn9MJ4wbYS6bmBrKdu6smFKJaWb3ty3l67pebnWzJO3DCYTTcjueRbK4HbdB0rBdQox8sjpm+lq47lZXfajcxoj+q7gOvtK7rpyUf6UYFVBF7bG1U9gK5Rroix6viEIQydDXGjumOM68aY4FGhzpgKBwsfyaRlCsmhdfJJUkApbI8sSvAskkGCImBLlNFDIBbF8FyOkdSmEvLXgUtEWUDX8F2Bt9hBA/iKN2EGB5U8vDEydRoWX8UbsS2+iLfiu/h9KFfGHErPJv52niujxsCzCzO/jszy8U9p9V9WT80pNWmCtbrQHjGiVuHk+a0nz7dnbtaGs8vilfgB/S/FlviMFQStn87raVl7wey3kZN3MYZ1v6PgAfgkEOWpHlxFFZNjVqBS1WuzauWbNNrF/pfRoo19jDl2eM9SzDfBpc5B0jPSBls+H8B/zHvrc4UAJ0rpqHF1s7sDNp6MUe4lXrUWC6vSUdOF7SGzF+PsHsYYVsboDmPK3QigOEK20t2Lz2Wrzed86QCt9zBn4l/BvMlVSnxfJPSa1OI9yTuvIvJ8G/4G2zaf9uUup05DHDfUYSrhvu5cSv1wY646ZsCeFpVJo3Nzi3SRLtEIuMZpku7SFNWh5Bv9KfQXitpT7Z32XvuQh/YVOjnnadfQPv0FL2YgnQ==
d(G1, G1
) = 0
AAAEu3icnVNNb9NAEJ2UACV8NIULEheLtFBQFa1zSUGqVAkk4IBo06atVEeR7WxSt7Zj2U5osfIHkDhz4AQSB8SRK3Dhwh/g0J+AOBaJCwfejpMgCkkl1rI983bmzdvZXStwnSgW4iAzcSJ78tTpyTO5s+fOX5jKT19cj9qd0JZVu+22w03LjKTr+LIaO7ErN4NQmp7lyg1r946a3+jKMHLa/lq8H8iaZ7Z8p+nYZgyonr/emDM8M962TTe516snem9eOwLc0BY1Uc8XRLFUXtCFrrEhbpUGRlnTi4JHgfpjuT2d+UgGNahNNnXII0k+xbBdMinCs0U6CQqA1SihHSAmhfAcjpHUoxzyO8Alokygu/i24G31UR++4o2YwUYlF2+ITI1mxRfxRhyKz+Kt+Cp+juRKmEPp2cffSnNlUJ96enn1x7FZHv4xbf/OGqs5piYtsFYH2gNG1CrsNL/75Pnh6u3KbHJNvBLfoP+lOBCfsAK/+91+vSIrL5j9LnLSLoawHvYVPAKfBKI81YObqGJwTAsqVb0eq1a+QfND7H8ZTdr7izHFRvcsxnwTXOocRGMjLbCl8z78x7y3HlfwcaKUjgpXN4Y7YOFJGOVe4lVrMbEqDTUd2C4yxzGuHWEMYSWMDhhj7oYPxQGyle5xfA5bPT7ntX9ofYA5A/8C5g2ukuP7IqHXoC7vSdp5FZHmW/D32Lb4tDeGnBrNcNxMnymH+zq4lNpoY71U1GGv6IUlvX9zJ+kKXaU5cJVpie7TMlWh5Bm9o/f0IbuYtbM7WTcNncj0cy7RHyPb+QUtTgQW
AAAEsHicnVPPaxNBFH6pUWusNvUkeFlMq1VKmXhRhIKgoB7ENm3aQjeE3c0k3XZ/sTuJrUv+AcGzB08KHsSjV/XixX/AQ/8E8VjBiwe/ebtGTGwEZ9nd975573vfvJmxI89NlBAHhYljxeMnTk6eKp2eOnN2ujwztZ6E3diRdSf0wnjTthLpuYGsK1d5cjOKpeXbntywd2/r+Y2ejBM3DNbUfiQbvtUJ3LbrWApQs3y5NW/6ltp2LC+922+m1f6CMQRcMZYM0SxXxKLgYYwa1dyoUD6Ww5nCBzKpRSE51CWfJAWkYHtkUYJni6okKALWoJR2gFgUw3M5RlKfSsjvApeIsoDu4tuBt5WjAXzNmzCDg0oe3hiZBs2Jz+K1OBSfxBvxRfw4kitlDq1nH387y5VRc/rJ+dXv/8zy8Ve0/TtrrGZFbbrBWl1ojxjRq3Cy/N7jZ4erN2tz6SXxUnyF/hfiQHzECoLeN+fViqw9Z/Y7yMm6GMN6kCt4CD4JRHu6B1dRxeSYDlTqen1WrX2TFgbY/zJatDfCmGFH90xhvg0ufQ6SsZE22LL5AP4j3lufKwQ4UVpHjaubgx2w8aSMci/x6rVYWJWBmi5sD5njGNeGGGNYKaO/GBV3I4DiCNla9zg+l60+n/PGX7Tex5yJfwXzJlcp8X2R0GtSj/ck67yOyPJt+Hts23zaWwNOg2Y5bjZnKuG+Vodv56ixfm2xCntF0CRdoIs0D4rrdIvu0TLVIeApvaV39L64VHSKO9nNnijkV/wc/TGK3k+ssgKx
AAAEsHicnVNNb9NAEJ2UACUUmnJC4mKRFgqqonUuKUiVkEACDog2bdpKdRTZziZ16y/ZTmix8geQOHPgBBIHxJErcOHCH+DQn4A4FokLB96OkyAKDRJr2Z55O/Pm7eyuFbpOnAhxkJs4kT956vTkmcLZqXPnp4szU+tx0I1sWbcDN4g2LTOWruPLeuIkrtwMI2l6lis3rN3ban6jJ6PYCfy1ZD+UDc/s+E7bsc0EULN4tTVveGaybZtuerffTPX+gnYEuKYtaaJZLIlypbqoC11jQ9yoDI2qppcFjxINxnIwk/tABrUoIJu65JEknxLYLpkU49kinQSFwBqU0g4QkyJ4DsdI6lMB+V3gElEm0F18O/C2BqgPX/HGzGCjkos3QqZGc+KzeC0OxSfxRnwRP47lSplD6dnH38pyZdicfnJx9fs/szz8E9r+lTVWc0JtWmStDrSHjKhV2Fl+7/Gzw9Wbtbn0ingpvkL/C3EgPmIFfu+b/WpF1p4z+x3kZF2MYD0YKHgIPglEeaoH11HF4JgOVKp6fVatfIMWRtj/Mpq09wdjhh3fswTzbXCpcxCPjbTAls378B/x3npcwceJUjpqXN0Y7YCFJ2WUe4lXrcXEqjTUdGC7yBzHuHaEMYKVMjpkTLgbPhSHyFa6x/E5bPX5nDf+ovU+5gz8S5g3uEqB74uEXoN6vCdZ51VElm/B32Pb4tPeGnFqNMtxswOmAu7r8FJqxxvrlbIOe0XQJF2iyzQPiirdonu0THUIeEpv6R29zy/l7fxOdrMncoMrfoF+G3n3J/IRAwE=
AAAEu3icnVNNb9NAEJ2UACV8NIULEheLtFBQFa1zSUGqVAkk4IBo06atVEeR7WxSt7Zj2U5osfIHkDhz4AQSB8SRK3Dhwh/g0J+AOBaJCwfejpMgCkkl1rI983bmzdvZXStwnSgW4iAzcSJ78tTpyTO5s+fOX5jKT19cj9qd0JZVu+22w03LjKTr+LIaO7ErN4NQmp7lyg1r946a3+jKMHLa/lq8H8iaZ7Z8p+nYZgyonr/emDM8M962TTe516snem9eOwLc0BY1Uc8XRLFUXtCFrrEhbpUGRlnTi4JHgfpjuT2d+UgGNahNNnXII0k+xbBdMinCs0U6CQqA1SihHSAmhfAcjpHUoxzyO8Alokygu/i24G31UR++4o2YwUYlF2+ITI1mxRfxRhyKz+Kt+Cp+juRKmEPp2cffSnNlUJ96enn1x7FZHv4xbf/OGqs5piYtsFYH2gNG1CrsNL/75Pnh6u3KbHJNvBLfoP+lOBCfsAK/+91+vSIrL5j9LnLSLoawHvYVPAKfBKI81YObqGJwTAsqVb0eq1a+QfND7H8ZTdr7izHFRvcsxnwTXOocRGMjLbCl8z78x7y3HlfwcaKUjgpXN4Y7YOFJGOVe4lVrMbEqDTUd2C4yxzGuHWEMYSWMDhhj7oYPxQGyle5xfA5bPT7ntX9ofYA5A/8C5g2ukuP7IqHXoC7vSdp5FZHmW/D32Lb4tDeGnBrNcNxMnymH+zq4lNpoY71U1GGviMKS3r+5k3SFrtIcuMq0RPdpmapQ8oze0Xv6kF3M2tmdrJuGTmT6OZfoj5Ht/AIs/gQV
d(G1, G2
) = 0 , G1 = G2
AAAE73icnVNda9RAFL2tq9b1o62+CL4MbitVSpnsy1ZBKCiooNivbQtNWZJ0dhubTWIy3baG/QP+gT4IooIPxf8giC/+AR/6C0R8USr44oNnbraVVruCCUnuPXPPuWcmM24c+KmWcqen91jh+ImTfaeKp8+cPdc/MHh+Lo3WEk9VvSiIkgXXSVXgh6qqfR2ohThRTtMN1Ly7esuMz7dUkvpROKs3Y7XUdBqhX/c9RwOqDSwuj9hNR694TpDdadcyqz0qDgDl9lVxU0hh31d1nfiNFe0kSbQuxCEaig7xRG2gJMfKlXFLWoIDeb28F1SENSb5KlHnmowGe96STcsUkUdr1CRFIWnEATmU4l4kiyTFwJYoo0dAHEqQ+VyjqE1F8NeAK1Q5QFfxbiBb7KAhcqObsoKHTgGeBExBw/Kj3Ja78oN8Iz/Ln0dqZaxh/Gzi6+ZcFdf6n16c+fFPVhNfTSu/WV09a6rTOHv14T1mxMzCy/mtJ1u7Mzemh7Mr8pX8Av8v5Y58jxmEre/e6yk1/YzVb4OTr2KC6EHHwUPoKSAmM2twDV1srmnApenXZtcmt2l0H/tfRYc2/lDMsaPXTGO8Di2zD9KulS7U8vEQ+Tr/2yZ3CLGjjI9p7m7v/wEXd8YoryUeMxcHsxLo6SMOwOymOHtIMUGUMbqnqHk1QjiOwTa+u+n5HLV5ny/9xes9jNn4ljBuc5cinxcFvza1+J/kK28qcr6LfINjl3f78r6moCGuG+ooFXFe9w6lODqYK49ZiKes0oTVObl9dIku0wi0KjRBd2mSqnDyjj7RV/pWeFzYKjwvvMhLe3s6nAt04Cps/wI2WRlH
AAAE5HicnVPdahNBFD6pUWusNvVK8GYxrVQpZdIbRRAEBRUU27RpC9kSZjeTZO3+uTtJW5d9AV+gF4Ko4IX4DoJ44wt40ScQ8Uap4I0XfjMbI01sCs6yu+d8c853vjkzY4WuE0vG9nJjx/LHT5wcP1U4PXHm7GRxamI1DjqRLap24AbRusVj4Tq+qEpHumI9jAT3LFesWZu31PxaV0SxE/grcicUGx5v+U7TsbkEVC/WGrOmx2Xb5m5yJ60n5XTOOAAspJeNGwYzzPuiKSOn1ZY8ioItwxhIQ9BAnlEvltg808MYNso9o0S9sRhM5d6RSQ0KyKYOeSTIJwnbJU4xnhqViVEIbIMSegSEUwTP0TGCUiogvwNcIIoD3cS3Ba/WQ334ijfWDDYquXgjZBo0wz6xN2yffWRv2Rf261CuRHMoPTv4W1muCOuTT88v/zwyy8NfUvtv1kjNkpp0TWt1oD3UiFqFneV3n+zuL1+vzCSX2Cv2Ffpfsj32ASvwuz/s10ui8kyz30ZO1sUI1oOegofgE0CUp3pwBVVMHdOCSlUv1aqVb9JcH/tfRk7bQ4wZdnjPJOab4FLnIB4ZaYEtm/fhb+m99XQFHydK6ajo6mZ/Byw8iUZ1L/GqtXCsykBNB7aLzFGMKwOMEaxEo38Ype6GD8UhspXuUXyOtlJ9zjf+ofUe5kz8S5g3dZWCvi8Cek3q6j3JOq8isnwL/ra2LX3aG31Og6Z13HSPqYD7Wh68ncPG6sJ8GfYSo3G6QBdpFhRX6SbdpUWqQsB7+kzf6Hv+cX43/zy72WO53hU/RwdG/sVv1esXrg==
AAAE5HicnVPNahRBEK6Nq8Y1mo0nwUtjEokSQu9eNgqCoKCCYv4T2AnLzKR3M2Z2ZpzpbBKHfQFfIAdBVPAgvoMgXnwBD3kCES9KBC8e/Lpms5LormAPM1P1ddVXX1d3O5HvJVrKvdzAsfzxEycHTxVOD505O1wcGVpKws3YVYtu6IfximMnyvcCtag97auVKFZ20/HVsrNx08wvt1SceGGwoHcitdq0G4FX91xbA6oVq2sTVtPW667tp7fbtbTUnhSHgHL7srgupLDuqbqOvca6tuM43BLiSBqCjuSJWnFUTpUr0yVZEmzIq+UDoyJKU5LHKHXGTDiSe0sWrVFILm1SkxQFpGH7ZFOCp0olkhQBW6WUHgKxKYbncYyiNhWQvwlcIcoGuoFvA161gwbwDW/CDC4q+XhjZAoalx/la7kvP8g38rP82ZMrZQ6jZwd/J8tVUW34yfn5H//MauKvaf13Vl/Nmuo0zVo9aI8YMatws/zW4939+Wtz4+kl+VJ+gf4Xck++xwqC1nf31ayae8rst5CTdTGGdb+j4AH4FBDjmR5cQRWLYxpQaeq1WbXxLZrsYv/LaNP2H4wZ1rtnGvN1cJlzkPSNdMCWzQfwt3hvm1whwIkyOua4utXdAQdPyij3Eq9Zi41VCdT0YPvI7Me4cIQxhpUyesCouRsBFEfINrr78Xlstfmcr/5F613MWfiPYt7iKgW+Lwp6LWrxnmSdNxFZvgN/m22HT/tal1PQGMeNdZgKuK8Hl1L0NpbKUyXYs5IG6QJdpAlQVOgG3aEZWoSAd/SJvtK3/KP8bv5ZdrMHcp0rfo4OjfzzXxtZF/4=
AAAE73icnVNda9RAFL2tq9b1o62+CL4MbitVSpnsy1ZBKCiooNivbQtNWZJ0dhubTWIy3baG/QP+gT4IooIPxf8giC/+AR/6C0R8USr44oNnbraVVruCCUnuPXPPuWcmM24c+KmWcqen91jh+ImTfaeKp8+cPdc/MHh+Lo3WEk9VvSiIkgXXSVXgh6qqfR2ohThRTtMN1Ly7esuMz7dUkvpROKs3Y7XUdBqhX/c9RwOqDSwuj9hNR694TpDdadcyqz0qDgDl9lVxU0hh31d1nfiNFe0kSbQuxCEaig7xRG2gJMfKlXFLWoIDeb28F1SENSb5KlHnmowGe96STcsUkUdr1CRFIWnEATmU4l4kiyTFwJYoo0dAHEqQ+VyjqE1F8NeAK1Q5QFfxbiBb7KAhcqObsoKHTgGeBExBw/Kj3Ja78oN8Iz/Ln0dqZaxh/Gzi6+ZcFdf6n16c+fFPVhNfTSu/WV09a6rTOHv14T1mxMzCy/mtJ1u7Mzemh7Mr8pX8Av8v5Y58jxmEre/e6yk1/YzVb4OTr2KC6EHHwUPoKSAmM2twDV1srmnApenXZtcmt2l0H/tfRYc2/lDMsaPXTGO8Di2zD9KulS7U8vEQ+Tr/2yZ3CLGjjI9p7m7v/wEXd8YoryUeMxcHsxLo6SMOwOymOHtIMUGUMbqnqHk1QjiOwTa+u+n5HLV5ny/9xes9jNn4ljBuc5cinxcFvza1+J/kK28qcr6LfINjl3f78r6moCGuG+ooFXFe9w6lODqYK49ZiKdkacLqnNw+ukSXaQRaFZqguzRJVTh5R5/oK30rPC5sFZ4XXuSlvT0dzgU6cBW2fwE2CRlG

View Slide

The OTMANN Distance
構造は異なるがOTMANN = 0となる例
17
0: ip
(100)
1: conv3, 16
(16)
2: conv3, 16
(256)
3: conv3, 32
(512)
4: max-pool, 1
(32)
5: fc, 16
(51)
6: softmax
(100)
7: op
(100)
0: ip
(100)
1: conv3, 16
(16)
2: conv3, 8
(128)
3: conv3, 8
(128)
4: conv3, 32
(512)
5: max-pool, 1
(32)
6: fc, 16
(51)
7: softmax
(100)
8: op
(100)
Figure 3: An example of 2 CN
d = ¯
d = 0 distance. The OT solu
mass in each layer in the netwo
the layer horizontally opposite
with 0 cost. For layer 2 on the
mapped to layers 2 and 3 on th
while the descriptor of these netw
their functional behaviour is the
0: ip
(100)
1: conv3, 16
(16)
2: conv3, 16
(256)
3: conv3, 32
(512)
4: max-pool, 1
(32)
5: fc, 16
(51)
6: softmax
(100)
7: op
(100)
0: ip
(100)
1: conv3, 16
(16)
2: conv3, 8
(128)
3: conv3, 8
(128)
4: conv3, 32
(512)
5: max-pool, 1
(32)
6: fc, 16
(51)
7: softmax
(100)
8: op
(100)
Figure 3: An example of 2 CNNs which have
d = ¯
d = 0 distance. The OT solution matches the
mass in each layer in the network on the left to
the layer horizontally opposite to it on the right
with 0 cost. For layer 2 on the left, its mass is
mapped to layers 2 and 3 on the left. However,
while the descriptor of these networks is different,
their functional behaviour is the same.
A.2 Distance Properties of OTMANN

View Slide

NASBOT
カーネルの設計
n 次式で定義されるnegative exponentiated distanceを
カーネルとして採用：
- LaplacianカーネルやGaussカーネルに類似したカーネル
n はOTMANN，は正規化されたOTMANN
n 著名なカーネルのように正定値性を満たすかは未確認
n 予備実験では固有値が負になるケースはなかったとのこと
18
 = ↵ exp( d) + ¯
↵ exp( ¯ ¯
d)
AAAE3XicnVM7b9NQFD4pBkp4tIUFicUiLWp5RDcZaEFCqgQDHRBt+pTqKLp2blMTx7FsN6REGVkQYu3ABBIDYuAfsLDwBzp0gRkxFomFge8eO0E8EiSuZfuc75zzne++7MBzo1iIg8zIEePoseOjJ7InT50+MzY+cXYtau6Ejlp1ml4z3LBlpDzXV6uxG3tqIwiVbNieWrfrt3V8vaXCyG36K/FuoMoNWfPdLdeRMaDK+IJVl0EgzVumJb1gW5qWagfT1yxbxdKszphXTMuWYScJdtNoiumcbmJXuzOV8ZzIF2fnCqJgsiFuFHvGrFnICx45SsdicyLzjiyqUpMc2qEGKfIphu2RpAjPJhVIUACsTB16AERSCM/lHEVdyqJ+B7hClgRax7cGbzNFffiaN2IGB508vCEqTZoS++K1OBQfxBvxWXwfyNVhDq1nF387qVVBZezJ+eVv/6xq4B/T9s+qoZpj2qI51upCe8CInoWT1Lce7R0u3yxNdS6Jl+IL9L8QB+I9ZuC3vjqvllTpObPfQU2yiiGse6mC++BTQLSn1+AyulicU4NK3a/LqrVv0dU+9r+Mktp/MCbY4DWLEd8Clz4H0dBMG2xJ3If/kPe2wR18nCito8Tdrf4O2Hg6jPJa4tVzkZiViZ4ubA+VwxhXfmMMYXUY7THGvBo+FAeo1rqH8blsdfmcl/+idQExC/8c4hZ3yfJ9UdBrUYv3JFl5nZHU2/DbbNt82qt9TpMmOW8yZcrivvYupTnYWCvmC7CXCrn56+nNHaULdJGmwTVL83SXFmkVSt7SPn2kT0bFeGw8NZ4lqSOZtOYc/TKMvR/LgxEd
d
AAAEk3icnVNNb9NAEJ0UAyV8tAUhIXGxSIsQQtE6B1LgUkEPcKho0qatVEeV7WyCqWNbthNarPwBuII4cAKJA+I/cOHCH+DQn4A4FokLB96OnSA+EiTWsj3zdubN29ldO/TcOBHioDB1RDt67Pj0ieLJU6fPzMzOnd2Ig17kyIYTeEG0ZVux9FxfNhI38eRWGEmra3ty0969o+Y3+zKK3cBfT/ZD2exaHd9tu46VAKq1dmZLolypLhrC0NkQNypDo6obZcGjRPlYDeYK78mkFgXkUI+6JMmnBLZHFsV4tskgQSGwJqX0EIhFETyXYyQNqIj8HnCJKAvoLr4deNs56sNXvDEzOKjk4Y2QqdOC+CTeikPxUbwTn8X3sVwpcyg9+/jbWa4Md2aeXFj79s+sLv4JPfiZNVFzQm1aZK0utIeMqFU4WX7/8YvDtZv1hfSyeC2+QP8rcSA+YAV+/6vzpibrL5l9GTlZFyNYK7mC++CTQJSnenAVVUyO6UClqjdg1co36doI+19Gi/b+YMyw8T1LMN8GlzoH8cRIG2zZvA//Ee9tlyv4OFFKR52rm6MdsPGkjHIv8aq1WFiVjpoubA+ZkxjXf2OMYKWMDhkT7oYPxSGyle5JfC5bAz7nzb9ovYc5E/8S5k2uUuT7IqHXpD7vSdZ5FZHl2/D32Lb5tLdGnDrNc9x8zlTEfR1eSn28sVEpG7BrRmnpen5zp+kiXaIr4KrSEt2lVWpAiaSn9Iyea+e1W9ptbTkLnSrkOefol6Gt/ADnRvUb
¯
d
AAAEmXicnVNNb9NAEJ0UAyV8tIULUi8WaRFCKFrnQAqnIjgUJESbtE2lOqpsZ5OaOrZlO6HFyh/gxgmJnkDigPgPXLjwBzj0JyCOReLCgbdjJ4iPBIm1bM+8nXnzdnbXDj03ToQ4Kkyd0E6eOj19pnj23PkLM7NzFzfjoBc5csMJvCDasq1Yeq4vNxI38eRWGEmra3uyYe/dVfONvoxiN/DXk4NQNrtWx3fbrmMlgBqmbUVpa7AzWxLlSnXJEIbOhrhVGRpV3SgLHiXKx2owV3hPJrUoIId61CVJPiWwPbIoxrNNBgkKgTUppcdALIrguRwjaUBF5PeAS0RZQPfw7cDbzlEfvuKNmcFBJQ9vhEydFsUn8VYci4/infgsvo/lSplD6TnA385yZbgz8+xy/ds/s7r4J7T7M2ui5oTatMRaXWgPGVGrcLL8/tMXx/XbtcX0qngtvkD/K3EkPmAFfv+r82ZN1g6Z/R5ysi5GsB7mCh6BTwJRnurBdVQxOaYDlaregFUr36QbI+x/GS3a/4Mxw8b3LMF8G1zqHMQTI22wZfM+/Ce8t12u4ONEKR01rm6OdsDGkzLKvcSr1mJhVTpqurA9ZE5iXP+NMYKVMjpkTLgbPhSHyFa6J/G5bA34nDf/ovU+5kz8S5g3uUqR74uEXpP6vCdZ51VElm/D32fb5tPeGnHqtMBxCzlTEfd1eCn18cZmpWzAXjNKyzfzmztN83SFroGrSsu0Qqu0wXvznF7SoTav3dFWtAdZ6FQhz7lEvwyt/gN43/fg

View Slide

NASBOT
進化計算による獲得関数の最適化
1. 獲得関数値が高い個体ほど選択確率を高くなるように設定し，
親個体をランダムに個選ぶ
2. 選んだ個体に以下の操作のいずれかをランダムに適用し，
子個体を生成
3. 子個体の獲得関数値を評価し，全て親個体グループに追加
19
AAAEpHicnVPLbtNAFL0pBkp4NIUNEhuLtBVCKBpnQQqrSrAAiUebNG2lOkS2O0lN/ZLthBYrP8APsIANSEVC/AMbNvwAi34CYlkkNiw4c50E8WiQGMv2vWfuPffMnRk78twkFeKgMHVMO37i5PSp4ukzZ8/NlGbPryVhL3Zk0wm9MN6wrUR6biCbqZt6ciOKpeXbnly3d26p+fW+jBM3DFbTvUi2fKsbuB3XsVJA7VLpQTszfSvdjv3M73mDQbtUFpVqbdEQhs6GuFEdGTXdqAgeZRqO5XC28J5M2qKQHOqRT5ICSmF7ZFGCZ5MMEhQBa1FGj4FYFMNzOUbSgIrI7wGXiLKA7uDbhbc5RAP4ijdhBgeVPLwxMnWaF5/EW3EoPop34rP4fiRXxhxKzx7+dp4ro/bMs4uNb//M8vFPaftn1kTNKXVokbW60B4xolbh5Pn9p88PGzfr89mCeC2+QP8rcSA+YAVB/6uzvyLrL5j9NnLyLsaw7g8VPASfBKI81YOrqGJyTBcqVb0Bq1a+SdfG2P8yWrT7B2OOHd2zFPMdcKlzkEyMtMGWzwfwn/De+lwhwIlSOupc3RzvgI0nY5R7iVetxcKqdNR0YXvInMS4+htjDCtjdMSYcjcCKI6QrXRP4nPZGvA5b/1F613MmfiXMW9ylSLfFwm9JvV5T/LOq4g834a/y7bNp31rzKnTHMfNDZmKuK+jS6kfbaxVKwbsFaO8dH14c6fpEl2mK+Cq0RLdoWVqQkmfXtI+vdEWtHtaQ2vmoVOFYc4F+mVoj34AFEL8Tg==
Operation Description
dec_single Pick a layer at random and decrease the number of units by 1/8.
dec_en_masse Pick several layers at random and decrease the number of units by 1/8 for all of them.
inc_single Pick a layer at random and increase the number of units by 1/8.
inc_en_masse Pick several layers at random and increase the number of units by 1/8 for all of them.
dup_path Pick a random path u1, . . . , uk
, duplicate u2, . . . , uk 1
and connect them to u1
and uk
.
remove_layer Pick a layer at random and remove it. Connect the layer’s parents to its children if necessary.
skip Randomly pick layers u, v where u is topologically before v. Add (u, v) to E.
swap_label Randomly pick a layer and change its label.
wedge_layer Randomly remove an edge (u, v) from E. Create a new layer w and add (u, w), (w, v) to E.
Table 2: Descriptions of modifiers to transform one network to another. The first four change the number of
units in the layers but do not change the architecture, while the last five change the architecture.

View Slide

NASBOT
NASBOTの柔軟性
n NASBOTの枠組みを拡張することでDropoutの確率やバッチ
正規化のパラメータも層ごとに最適化可能
n 学習率などのハイパーパラメータも既存のカーネルを使用し
同時にBOで最適化可能
n 今回は構造最適化に限定して実験
20

View Slide

Experiments｜Settings
比較手法
n ランダムサーチ（NASBOTの獲得関数の値を一様乱数で決定）
n 獲得関数の最適化と同じスキームの進化計算法
n TreeBO（順伝播構造を探索するBO）
データセットとモデル
n 回帰（MLP）：Blog, Indoor, Slice, Naval, Protein, News
- 訓練：検証：テスト＝ 60%：20%：20%
n 分類（CNN）：CIFAR-10
- 訓練：検証：テスト＝ 40,000：10,000：10,000
21

View Slide

Experiments｜Settings
計算資源
n 全ての手法で非同期並列処理を使用
- 概ね70 ~ 120個の構造を評価する実験設定
n Blog, Indoor, Slice, Naval, Protein：Geforce 970 2枚 × 8時間
n News： Geforce 980 4枚 × 6時間
n CIFAR-10：K80 4枚 × 10時間
SGDによるNNの学習
n 回帰：学習率10-5，バッチサイズ256，更新回数20K回
n 分類：初期学習率10-2，バッチサイズ32，更新回数60K回
22

View Slide

Experiments｜Results
検証精度の最良値の推移（5試行）
23
Time (hours)
0 2 4 6 8
Cross Validation MSE
0.7
0.8
0.9
1
1.1
1.2
Blog Feedback, #workers = 2
Time (hours)
0 2 4 6 8
0.1
0.15
0.2
0.25
Indoor Location, #workers = 2
Time (hours)
0 2 4 6 8
0.6
0.7
0.8
0.9
1
Slice Localisation, #workers = 2
Time (hours)
0 2 4 6 8
10 -2
10 -1
Naval Propulsion, #workers = 2
Time (hours)
0 2 4 6 8
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
Protein, #workers = 2
Time (hours)
0 1 2 3 4 5 6
0.7
0.8
0.9
1
1.1
News, #workers = 4
Time (hours)
0 2 4 6 8 10
Cross Validation Error
0.12
0.13
0.14
0.15
0.16
0.17
Cifar10, #workers = 4
EA
RAND
TreeBO
NASBOT
Figure 2: Cross validation results: In all figures, the x axis is time. The y axis is the mean squared error
(MSE) in the first 6 figures and the classification error in the last. Lower is better in all cases. The title of each
figure states the dataset and the number of parallel workers (GPUs). All figures were averaged over at least 5
independent runs of each method. Error bars indicate one standard error.
increasing or decreasing the number of computational units in a layer, by adding or deleting layers,

View Slide

検証精度が最良であるモデルのテスト精度（5試行）
24
Method
Blog
(60K, 281)
Indoor
(21K, 529)
Slice
(54K, 385)
Naval
(12K, 17)
Protein
(46K, 9)
News
(40K, 61)
Cifar10
(60K, 1K)
Cifar10
150K iters
RAND
0.780
± 0.034
0.115
±0.023
0.758
± 0.041
0.0103
± 0.002
0.948
± 0.024
0.762
±0.013
0.1342
± 0.002
0.0914
± 0.008
EA
0.806
± 0.040
0.147
± 0.010
0.733
± 0.041
0.0079
±0.004
1.010
± 0.038
0.758
±0.038
0.1411
± 0.002
0.0915
± 0.010
TreeBO
0.928
± 0.053
0.168
± 0.023
0.759
± 0.079
0.0102
± 0.002
0.998
± 0.007
0.866
± 0.085
0.1533
± 0.004
0.1121
± 0.004
NASBOT
0.731
±0.029
0.117
±0.008
0.615
±0.044
0.0075
±0.002
0.902
±0.033
0.752
±0.024
0.1209
±0.003
0.0869
±0.004
Table 3: The ﬁrst row gives the number of samples N and the dimensionality D of each dataset in the form
(N, D). The subsequent rows show the regression MSE or classiﬁcation error (lower is better) on the test set
for each method. The last column is for Cifar10 where we took the best models found by each method in 24K
iterations and trained it for 120K iterations. When we trained the VGG-19 architecture using our training
procedure, we got test errors 0.1718 (60K iterations) and 0.1018 (150K iterations).
Experimental Set up: Each method is executed in an asynchronously parallel set up of 2-4 GPUs,
That is, it can evaluate multiple models in parallel, with each model on a single GPU. When the

View Slide

学習する構造を決定するまでに要する時間
NASBOTでの探索結果
n すべてのデータセットで他の手法と同等以上のスコア
n CIFAR-10で他の構造探索に比べてスコアが劣るのは探索空間
が広いことが原因と考えている
- VGG-19（テストエラー10.18%）よりは良い構造を発見できている
25
手法構造の決定に要する時間 [sec]
NASBOT 46.13
ランダムサーチ（RAND） 26.43
進化計算（EA） 0.19
TreeBO 7.83

View Slide

NASBOTが得たベスト構造（Indoorデータセット）
26
#0 ip, 64, (28891)
#1 crelu, 144, (144)
#2 softplus, 576, (82944)
#6 logistic, 256, (69632)
#9 linear, 256, (14445)
#3 leaky-relu, 72, (41472)
#4 logistic, 128, (73728)
#5 elu, 64, (4608)
#7 logistic, 256, (16384)
#8 linear, 256, (14445)
#10 op, 512, (28891)
#0 ip, 64, (542390)
#1 elu, 128, (128)
#2 elu, 256, (32768)
#3 logistic, 512, (131072)
#27 logistic, 512, (393216)
#29 linear, 512, (542390)
#4 crelu, 512, (262144)
#5 logistic, 512, (262144)
#6 logistic, 512, (262144)
#7 crelu, 512, (262144)
#8 elu, 512, (262144)
#9 crelu, 512, (262144)
#10 tanh, 512, (262144)
#11 elu, 512, (262144)
#23 tanh, 324, (259200)
#12 softplus, 64, (32768)
#13 tanh, 512, (262144)
#16 logistic, 72, (9216)
#14 softplus, 512, (262144)
#15 softplus, 64, (32768)
#17 relu, 128, (8192) #18 logistic, 128, (9216)
#19 tanh, 576, (73728) #20 relu, 128, (16384)
#21 leaky-relu, 576, (331776) #22 relu, 288, (36864)
#26 leaky-relu, 512, (589824)
#24 tanh, 648, (209952)
#25 leaky-relu, 576, (373248)
#28 logistic, 512, (262144)
#30 op, 512, (542390)
#0 ip, 64, (423488)
#1 elu, 128, (128)
#2 elu, 256, (32768)
#3 linear, 512, (211744)
#25 tanh, 576, (700416)
#4 logistic, 512, (131072)
#21 tanh, 512, (262144)
#27 op, 512, (423488)
#5 logistic, 512, (262144)
#6 logistic, 512, (262144)
#7 leaky-relu, 512, (262144)
#8 leaky-relu, 512, (262144)
#9 leaky-relu, 576, (294912)
#10 tanh, 64, (32768)
#11 leaky-relu, 512, (262144)
#12 tanh, 512, (294912)
#20 crelu, 256, (81920)
#13 tanh, 512, (262144)
#14 tanh, 64, (32768)
#15 relu, 64, (32768)
#16 relu, 64, (4096)
#17 relu, 128, (16384)
#18 logistic, 256, (32768)
#19 logistic, 256, (32768)
#22 crelu, 512, (131072)
#23 elu, 504, (258048)
#24 tanh, 576, (290304)
#26 linear, 512, (211744)
#0 ip, 64, (206092)
#1 relu, 112, (112)
#2 relu, 112, (112)
#3 relu, 112, (112)
#4 relu, 224, (25088)
#20 logistic, 512, (417792)
#5 logistic, 448, (50176)
#8 linear, 512, (103046)
#6 logistic, 392, (87808)
#7 logistic, 441, (98784)
#9 logistic, 496, (416640)
#10 leaky-relu, 62, (27342)
#22 op, 512, (206092)
#11 leaky-relu, 496, (246016)
#12 logistic, 512, (253952)
#19 logistic, 256, (192512)
#13 tanh, 128, (7936)
#14 leaky-relu, 64, (31744)
#18 softplus, 256, (159744)
#21 linear, 512, (103046)
#17 softplus, 128, (32768)
#15 tanh, 64, (4096)
#16 tanh, 128, (8192)
Figure 12: Optimal network architectures found with NASBOTon Indoor data.
どのデータセットでも
複数の出力層・長いスキップ結合
を持つ構造がベスト構造

View Slide

Conclusion
NASBOT
n NNの構造探索のためのBOフレームワークNASBOTを提案
n NASBOTは他の手法よりも良いMLP・CNNの構造を発見
OTMANN
n この論文における重要な貢献はOTMANN
n BO以外の手法への応用も考えられる
27

View Slide

Appendix｜他の手法で獲得した構造
EAで獲得したベスト構造（Indoor データセット）
28
Figure 12: Optimal network architectures found with NASBOTon Indoor data.
#0 ip, 64, (232665)
#1 relu, 128, (128)
#2 relu, 256, (32768)
#3 logistic, 512, (131072)
#14 crelu, 512, (262144)
#4 logistic, 512, (262144)
#5 elu, 512, (262144)
#6 elu, 512, (262144)
#13 crelu, 256, (196608)
#7 tanh, 576, (294912)
#8 tanh, 64, (36864)
#9 softplus, 64, (4096)
#10 softplus, 128, (8192)
#11 logistic, 128, (16384)
#12 logistic, 256, (32768)
#15 tanh, 512, (262144)
#16 tanh, 512, (262144)
#17 linear, 512, (232665)
#18 op, 512, (232665)
#0 ip, 64, (9121)
#1 leaky-relu, 128, (128)
#2 leaky-relu, 128, (128)
#3 leaky-relu, 224, (28672)
#4 crelu, 126, (16128)
#5 logistic, 64, (14336)
#9 linear, 256, (9121)
#6 logistic, 72, (4608)
#7 crelu, 126, (9072)
#8 crelu, 144, (18144)
#10 op, 256, (9121)
#0 ip, 64, (12209)
#1 relu, 144, (144)
#2 relu, 252, (36288)
#7 linear, 256, (12209)
#3 tanh, 72, (18144)
#6 logistic, 144, (54720)
#4 tanh, 64, (4608)
#5 leaky-relu, 128, (8192)
#8 op, 256, (12209)
#0 ip, 64, (30336)
#1 softplus, 128, (128)
#2 softplus, 128, (128)
#3 softplus, 256, (32768)
#4 softplus, 256, (32768)
#5 crelu, 160, (40960)
#8 tanh, 64, (20480)
#6 softplus, 64, (10240)
#7 softplus, 64, (4096)
#12 elu, 128, (24576)
#9 crelu, 64, (4096)
#10 tanh, 128, (8192)
#11 tanh, 128, (16384)
#13 elu, 112, (14336)
#14 elu, 256, (28672)
#15 elu, 256, (65536)
#16 linear, 256, (30336)
#17 op, 256, (30336)
Figure 13: Optimal network architectures found with EA on Indoor data.

View Slide

RANDで獲得したベスト構造（Indoor データセット）
29
#0 ip, 64, (7795)
#1 relu, 128, (128)
#2 relu, 256, (32768)
#7 linear, 256, (7795)
#3 logistic, 64, (16384)
#4 logistic, 64, (4096)
#5 crelu, 128, (8192)
#6 crelu, 128, (16384)
#8 op, 256, (7795)
#0 ip, 64, (133952)
#1 crelu, 128, (128)
#2 crelu, 256, (32768)
#3 crelu, 512, (131072)
#4 crelu, 64, (32768)
#5 crelu, 64, (4096)
#6 crelu, 64, (4096)
#19 tanh, 256, (81920)
#7 logistic, 64, (4096)
#8 logistic, 64, (4096)
#9 logistic, 64, (4096)
#10 logistic, 128, (8192)
#11 logistic, 128, (16384)
#12 logistic, 128, (16384)
#13 leaky-relu, 128, (16384)
#14 leaky-relu, 128, (16384)
#15 leaky-relu, 128, (16384)
#16 leaky-relu, 256, (32768)
#17 leaky-relu, 256, (65536)
#18 leaky-relu, 256, (65536)
#20 tanh, 256, (65536)
#21 tanh, 256, (65536)
#22 tanh, 512, (131072)
#23 tanh, 512, (262144)
#24 tanh, 512, (262144)
#25 linear, 512, (133952)
#26 op, 512, (133952)
#0 ip, 64, (68428)
#1 relu, 128, (128) #2 relu, 128, (128)
#3 relu, 256, (32768) #4 leaky-relu, 64, (8192)
#10 linear, 512, (22809)
#5 leaky-relu, 512, (131072)
#11 leaky-relu, 64, (20480)
#13 tanh, 128, (49152)
#6 tanh, 128, (8192)
#7 logistic, 512, (262144) #8 softplus, 256, (32768)
#9 leaky-relu, 64, (32768)
#18 op, 512, (68428)
#12 tanh, 128, (8192)
#14 softplus, 256, (32768)
#15 softplus, 256, (65536)
#16 linear, 512, (22809)
#17 linear, 512, (22809)
#0 ip, 64, (68006)
#1 crelu, 128, (128)
#2 crelu, 256, (32768)
#15 linear, 512, (22668)
#3 elu, 512, (131072)
#4 logistic, 512, (262144)
#5 leaky-relu, 64, (32768)
#6 leaky-relu, 64, (32768) #7 leaky-relu, 64, (4096)
#8 tanh, 144, (9216) #9 tanh, 144, (9216)
#10 linear, 512, (22668)
#11 softplus, 288, (41472)
#12 softplus, 288, (41472)
#16 op, 512, (68006)
#13 softplus, 288, (82944)
#14 linear, 512, (22668)
Figure 14: Optimal network architectures found with RAND on Indoor data.

View Slide

TreeBOで獲得したベスト構造（Indoor データセット）
30
Figure 14: Optimal network architectures found with RAND on Indoor data.
#0 ip, 64, (19993)
#1 relu, 256, (256)
#2 logistic, 512, (131072)
#3 elu, 56, (28672)
#4 elu, 128, (7168)
#5 tanh, 256, (32768)
#6 linear, 256, (19993)
#7 op, 512, (19993)
#0 ip, 64, (15146)
#1 leaky-relu, 128, (128)
#2 leaky-relu, 216, (27648)
#3 leaky-relu, 256, (55296)
#4 softplus, 164, (41984)
#5 tanh, 81, (13284)
#6 relu, 162, (13122)
#7 linear, 256, (15146)
#8 op, 256, (15146)
#0 ip, 64, (100)
#1 softplus, 128, (128)
#2 linear, 256, (100)
#3 op, 256, (100)
#0 ip, 64, (632)
#1 leaky-relu, 56, (56)
#2 crelu, 112, (6272)
#3 linear, 256, (632)
#4 op, 256, (632)
Figure 15: Optimal network architectures found with TreeBO on Indoor data.

View Slide

論文紹介：Neural Architecture Search with Bayesian Optimisation and Optimal Transport [Kandasamy et al., NIPS 2018]

論文紹介：Neural Architecture Search with Bayesian Optimisation and Optimal Transport [Kandasamy et al., NIPS 2018]

S.Shota

More Decks by S.Shota

Other Decks in Technology

Featured

Transcript