A discussion on scaling Gaussian process regression
Review of some literature on accelerating Gaussian process regression with a special focus on fast solvers based on hierarchical decomposition of the covariance matrix.
stochastic process such that any finite collection of random variables are jointly distributed as Gaussian. A GP is fully specified by a mean function and a covariance kernel so that: {Xt }t 0, s.t.{Xt1 , Xt2 , · · · , Xtk } ⇠ N(m, K) <latexit sha1_base64="ueKukNNnXl+++J3uZc9hKxWo4Rs=">AAADwnicdVLbbtNAEHVsLiVcmsIjLyOiVHFVRXaFBAJVVJAHJKSqINJGitNovVknS2yvuzuGRMY/yRP8DWvHJUkbRrI0e+bM+MzR+EnIFTrOn5pp3bl77/7Og/rDR4+f7Db2np4rkUrKelSEQvZ9oljIY9ZDjiHrJ5KRyA/ZhT/7UNQvvjOpuIi/4iJhw4hMYh5wSlBDo73a75YXEZxSEmbdHPaPwcugXUJ+kM3zy6zN7fwQrpFFhdjg5aOMH7v6fVrUPVg1wb7H4+rtZ180pau5cb4xB7ZwRIp5Xm8Fb7b0gif5ZIpESvHjP33elGAW5G2PjgW+BQ+nDImt1ZAkkWIOwbJiaxnI5pipDnZK7fUW/OtdW/0g3z5kg2Ifrm1+eQBeLPB6s6WrdW1pf4RQWobgTdgVOPmGCICSk+HI1YUyOSoYhVxVAbOCpW1QPFpNP10pjtbs/ZTbo0bT6ThlwO3ErZKmUcXZqPHLGwuaRixGGhKlBq6T4DAjEjkNmd4iVSwhdEYmbKDTmERMDbPyBHNoaWQMgZD6ixFKdL0jI5FSi8jXzEKiulkrwG21QYrB66E+gCRFFtPlj4I0BBRQ3DOMuWQUw4VOCJVcawU6JZJQ1Fdf1ya4N1e+nZwfdVyn435+2Tx5X9mxYzw3XhhtwzVeGSfGR+PM6BnUfGcyMzaF1bW+WVeWWlLNWtXzzNgI6+dfLzc26g==</latexit> <latexit sha1_base64="ueKukNNnXl+++J3uZc9hKxWo4Rs=">AAADwnicdVLbbtNAEHVsLiVcmsIjLyOiVHFVRXaFBAJVVJAHJKSqINJGitNovVknS2yvuzuGRMY/yRP8DWvHJUkbRrI0e+bM+MzR+EnIFTrOn5pp3bl77/7Og/rDR4+f7Db2np4rkUrKelSEQvZ9oljIY9ZDjiHrJ5KRyA/ZhT/7UNQvvjOpuIi/4iJhw4hMYh5wSlBDo73a75YXEZxSEmbdHPaPwcugXUJ+kM3zy6zN7fwQrpFFhdjg5aOMH7v6fVrUPVg1wb7H4+rtZ180pau5cb4xB7ZwRIp5Xm8Fb7b0gif5ZIpESvHjP33elGAW5G2PjgW+BQ+nDImt1ZAkkWIOwbJiaxnI5pipDnZK7fUW/OtdW/0g3z5kg2Ifrm1+eQBeLPB6s6WrdW1pf4RQWobgTdgVOPmGCICSk+HI1YUyOSoYhVxVAbOCpW1QPFpNP10pjtbs/ZTbo0bT6ThlwO3ErZKmUcXZqPHLGwuaRixGGhKlBq6T4DAjEjkNmd4iVSwhdEYmbKDTmERMDbPyBHNoaWQMgZD6ixFKdL0jI5FSi8jXzEKiulkrwG21QYrB66E+gCRFFtPlj4I0BBRQ3DOMuWQUw4VOCJVcawU6JZJQ1Fdf1ya4N1e+nZwfdVyn435+2Tx5X9mxYzw3XhhtwzVeGSfGR+PM6BnUfGcyMzaF1bW+WVeWWlLNWtXzzNgI6+dfLzc26g==</latexit> <latexit sha1_base64="ueKukNNnXl+++J3uZc9hKxWo4Rs=">AAADwnicdVLbbtNAEHVsLiVcmsIjLyOiVHFVRXaFBAJVVJAHJKSqINJGitNovVknS2yvuzuGRMY/yRP8DWvHJUkbRrI0e+bM+MzR+EnIFTrOn5pp3bl77/7Og/rDR4+f7Db2np4rkUrKelSEQvZ9oljIY9ZDjiHrJ5KRyA/ZhT/7UNQvvjOpuIi/4iJhw4hMYh5wSlBDo73a75YXEZxSEmbdHPaPwcugXUJ+kM3zy6zN7fwQrpFFhdjg5aOMH7v6fVrUPVg1wb7H4+rtZ180pau5cb4xB7ZwRIp5Xm8Fb7b0gif5ZIpESvHjP33elGAW5G2PjgW+BQ+nDImt1ZAkkWIOwbJiaxnI5pipDnZK7fUW/OtdW/0g3z5kg2Ifrm1+eQBeLPB6s6WrdW1pf4RQWobgTdgVOPmGCICSk+HI1YUyOSoYhVxVAbOCpW1QPFpNP10pjtbs/ZTbo0bT6ThlwO3ErZKmUcXZqPHLGwuaRixGGhKlBq6T4DAjEjkNmd4iVSwhdEYmbKDTmERMDbPyBHNoaWQMgZD6ixFKdL0jI5FSi8jXzEKiulkrwG21QYrB66E+gCRFFtPlj4I0BBRQ3DOMuWQUw4VOCJVcawU6JZJQ1Fdf1ya4N1e+nZwfdVyn435+2Tx5X9mxYzw3XhhtwzVeGSfGR+PM6BnUfGcyMzaF1bW+WVeWWlLNWtXzzNgI6+dfLzc26g==</latexit> µt = E[Xt] = m(t), cov(Xt, Xt0 ) = E[XtXt0 ] E[Xt]E[Xt0 ] = k(t, t0) <latexit sha1_base64="qy2CwNYqB5G+9WUmJ/2ZOObDuTs=">AAAEQ3icdVNba9RAFE6zXmq8tfroy8GybVLqkhRBURaKVhCEUsVeZGe7TGYnu8PmZuakdgnz33zxD/jmH/DFB0V8FZxk03a3XQcCZ77vO2e+cybjp6GQ6LrfFszGlavXri/esG7eun3n7tLyvX2Z5BnjeywJk+zQp5KHIuZ7KDDkh2nGaeSH/MAfvSz5g2OeSZHE73Gc8m5EB7EIBKOood6y+aFJIopDRsNiW8FqG0gBdgX5QXGijgpbOGoDTpFxjThAVK8QbU/vd0qewHkSrBIR13u/eKcl21obq5k6MEeT5KiU1QyezckFkonBEGmWJZ/+k0eGFItA2YT1E3wOBIccqaPd0DTNkhMIJoyjbSA/wUK2sFV5t5pwljvV+rqaX2RG4mxMdX60DiRO8LSzyVS1sQIOewjVzBDIgH8EV824AKg0BfY8TVTBZqko/coaGJUqPQcpovPyO+eWo6n5vlGORaJcH9o+w16pjnbRLS85sqspEIslx7YGJyes6WvV5IWEmurCo0uVZvYTURtGti6Ha05vacVtudWCy4FXBytGvXZ7S19JP2F5xGNkIZWy47kpdguaoWAhVxbJJU8pG9EB7+gwphGX3aJ6AwqaGulDkGT6ixEqdDqjoJGU48jXytK0vMiV4Dyuk2PwtKv/wDRHHrPJQUEeAiZQPijoi4wzDMc6oCwT2iuwIc0oQ/3sLD0E72LLl4P9zZbntry3j1e2XtTjWDQeGA8N2/CMJ8aW8drYNfYMZn42v5s/zV+NL40fjd+NPxOpuVDn3DdmVuPvP79SZRQ=</latexit> <latexit sha1_base64="qy2CwNYqB5G+9WUmJ/2ZOObDuTs=">AAAEQ3icdVNba9RAFE6zXmq8tfroy8GybVLqkhRBURaKVhCEUsVeZGe7TGYnu8PmZuakdgnz33zxD/jmH/DFB0V8FZxk03a3XQcCZ77vO2e+cybjp6GQ6LrfFszGlavXri/esG7eun3n7tLyvX2Z5BnjeywJk+zQp5KHIuZ7KDDkh2nGaeSH/MAfvSz5g2OeSZHE73Gc8m5EB7EIBKOood6y+aFJIopDRsNiW8FqG0gBdgX5QXGijgpbOGoDTpFxjThAVK8QbU/vd0qewHkSrBIR13u/eKcl21obq5k6MEeT5KiU1QyezckFkonBEGmWJZ/+k0eGFItA2YT1E3wOBIccqaPd0DTNkhMIJoyjbSA/wUK2sFV5t5pwljvV+rqaX2RG4mxMdX60DiRO8LSzyVS1sQIOewjVzBDIgH8EV824AKg0BfY8TVTBZqko/coaGJUqPQcpovPyO+eWo6n5vlGORaJcH9o+w16pjnbRLS85sqspEIslx7YGJyes6WvV5IWEmurCo0uVZvYTURtGti6Ha05vacVtudWCy4FXBytGvXZ7S19JP2F5xGNkIZWy47kpdguaoWAhVxbJJU8pG9EB7+gwphGX3aJ6AwqaGulDkGT6ixEqdDqjoJGU48jXytK0vMiV4Dyuk2PwtKv/wDRHHrPJQUEeAiZQPijoi4wzDMc6oCwT2iuwIc0oQ/3sLD0E72LLl4P9zZbntry3j1e2XtTjWDQeGA8N2/CMJ8aW8drYNfYMZn42v5s/zV+NL40fjd+NPxOpuVDn3DdmVuPvP79SZRQ=</latexit> <latexit sha1_base64="qy2CwNYqB5G+9WUmJ/2ZOObDuTs=">AAAEQ3icdVNba9RAFE6zXmq8tfroy8GybVLqkhRBURaKVhCEUsVeZGe7TGYnu8PmZuakdgnz33zxD/jmH/DFB0V8FZxk03a3XQcCZ77vO2e+cybjp6GQ6LrfFszGlavXri/esG7eun3n7tLyvX2Z5BnjeywJk+zQp5KHIuZ7KDDkh2nGaeSH/MAfvSz5g2OeSZHE73Gc8m5EB7EIBKOood6y+aFJIopDRsNiW8FqG0gBdgX5QXGijgpbOGoDTpFxjThAVK8QbU/vd0qewHkSrBIR13u/eKcl21obq5k6MEeT5KiU1QyezckFkonBEGmWJZ/+k0eGFItA2YT1E3wOBIccqaPd0DTNkhMIJoyjbSA/wUK2sFV5t5pwljvV+rqaX2RG4mxMdX60DiRO8LSzyVS1sQIOewjVzBDIgH8EV824AKg0BfY8TVTBZqko/coaGJUqPQcpovPyO+eWo6n5vlGORaJcH9o+w16pjnbRLS85sqspEIslx7YGJyes6WvV5IWEmurCo0uVZvYTURtGti6Ha05vacVtudWCy4FXBytGvXZ7S19JP2F5xGNkIZWy47kpdguaoWAhVxbJJU8pG9EB7+gwphGX3aJ6AwqaGulDkGT6ixEqdDqjoJGU48jXytK0vMiV4Dyuk2PwtKv/wDRHHrPJQUEeAiZQPijoi4wzDMc6oCwT2iuwIc0oQ/3sLD0E72LLl4P9zZbntry3j1e2XtTjWDQeGA8N2/CMJ8aW8drYNfYMZn42v5s/zV+NL40fjd+NPxOpuVDn3DdmVuPvP79SZRQ=</latexit>
linear models, deep neural networks etc. ) – • Hypothesize that the unknown function resides in a space of parameterized functions. • Pose a risk minimization problem of the form: ✓⇤ = argmin ✓ 1 N N X i=1 L(y(i), ˆ f(x(i); ✓)) + R(✓) <latexit sha1_base64="rJ0gPCqJuqt0XtHAY/MGsexelmE=">AAAE8XicdVNbaxNBFN420dZ4a/XRl4MlbbbWkC2CogSKVhCUUou9QDcNk8lssmRv7pytDcP8C198UMRX/41v/hvPXtpNmjgwcOY73znznXNmepHnSmy1/i4sVqo3bi4t36rdvnP33v2V1QdHMkxiLg556IXxSY9J4bmBOEQXPXESxYL5PU8c90ZvUv/xuYilGwafcByJjs8Ggeu4nCFB3dXKUt32GQ4589SuhvU22AoaGdRz1IU+Uw3X1FtwiYwLxARbd5Xbtui8l/ptKINg3XaD4txTB0TZJW6gp/LAHE6YoNa1uvNyTizYsTsYIovj8Mt/4uwhQ+Xohs37Ib4CG4cCmUlqWBTF4QU4ucckGSguUMkmNjPttTpcxU6UvqnnJ5mimFsTlZ9tgh2EeFlZ3lUSpuCki5D1DMEeiM/Q0lMqADKOwq5FjszYThmpXlkAo5RFfZCuX6bfKyX7E/19r0261k/o1vYV+FafkoxOOmW/kbWBKufheYPQ/I4NGix5r0UUrg48nUk1dc5JbRg1KB1umM1a3r20TamMJOjTUxSoclirLDr2FYsHfj5jJ2ZcWZrKojoTv3xiZcUfyorH5fvMxzf7dK8maMITsD36GX1W5jqgXLm7u7LWarayBbOGVRhrRrH2uyt/7H7IE18EyD0m5anVirBDpaDLPaFrdiJFxPiIDcQpmQHzheyo7MdqqBPSByeMaQcIGToZoZgv5djvETOVKq/7UnCe7zRB50WH/kuUoAh4fpGTeIAhpN8f+m4sOHpjMhiPXdIKfMio6UiTqVETrOslzxpH202r1bQ+PlvbeV20Y9l4ZDw2GoZlPDd2jHfGvnFo8EpQ+Vr5XvlRldVv1Z/VXzl1caGIeWhMrervfx9/ohw=</latexit> <latexit sha1_base64="rJ0gPCqJuqt0XtHAY/MGsexelmE=">AAAE8XicdVNbaxNBFN420dZ4a/XRl4MlbbbWkC2CogSKVhCUUou9QDcNk8lssmRv7pytDcP8C198UMRX/41v/hvPXtpNmjgwcOY73znznXNmepHnSmy1/i4sVqo3bi4t36rdvnP33v2V1QdHMkxiLg556IXxSY9J4bmBOEQXPXESxYL5PU8c90ZvUv/xuYilGwafcByJjs8Ggeu4nCFB3dXKUt32GQ4589SuhvU22AoaGdRz1IU+Uw3X1FtwiYwLxARbd5Xbtui8l/ptKINg3XaD4txTB0TZJW6gp/LAHE6YoNa1uvNyTizYsTsYIovj8Mt/4uwhQ+Xohs37Ib4CG4cCmUlqWBTF4QU4ucckGSguUMkmNjPttTpcxU6UvqnnJ5mimFsTlZ9tgh2EeFlZ3lUSpuCki5D1DMEeiM/Q0lMqADKOwq5FjszYThmpXlkAo5RFfZCuX6bfKyX7E/19r0261k/o1vYV+FafkoxOOmW/kbWBKufheYPQ/I4NGix5r0UUrg48nUk1dc5JbRg1KB1umM1a3r20TamMJOjTUxSoclirLDr2FYsHfj5jJ2ZcWZrKojoTv3xiZcUfyorH5fvMxzf7dK8maMITsD36GX1W5jqgXLm7u7LWarayBbOGVRhrRrH2uyt/7H7IE18EyD0m5anVirBDpaDLPaFrdiJFxPiIDcQpmQHzheyo7MdqqBPSByeMaQcIGToZoZgv5djvETOVKq/7UnCe7zRB50WH/kuUoAh4fpGTeIAhpN8f+m4sOHpjMhiPXdIKfMio6UiTqVETrOslzxpH202r1bQ+PlvbeV20Y9l4ZDw2GoZlPDd2jHfGvnFo8EpQ+Vr5XvlRldVv1Z/VXzl1caGIeWhMrervfx9/ohw=</latexit> <latexit sha1_base64="rJ0gPCqJuqt0XtHAY/MGsexelmE=">AAAE8XicdVNbaxNBFN420dZ4a/XRl4MlbbbWkC2CogSKVhCUUou9QDcNk8lssmRv7pytDcP8C198UMRX/41v/hvPXtpNmjgwcOY73znznXNmepHnSmy1/i4sVqo3bi4t36rdvnP33v2V1QdHMkxiLg556IXxSY9J4bmBOEQXPXESxYL5PU8c90ZvUv/xuYilGwafcByJjs8Ggeu4nCFB3dXKUt32GQ4589SuhvU22AoaGdRz1IU+Uw3X1FtwiYwLxARbd5Xbtui8l/ptKINg3XaD4txTB0TZJW6gp/LAHE6YoNa1uvNyTizYsTsYIovj8Mt/4uwhQ+Xohs37Ib4CG4cCmUlqWBTF4QU4ucckGSguUMkmNjPttTpcxU6UvqnnJ5mimFsTlZ9tgh2EeFlZ3lUSpuCki5D1DMEeiM/Q0lMqADKOwq5FjszYThmpXlkAo5RFfZCuX6bfKyX7E/19r0261k/o1vYV+FafkoxOOmW/kbWBKufheYPQ/I4NGix5r0UUrg48nUk1dc5JbRg1KB1umM1a3r20TamMJOjTUxSoclirLDr2FYsHfj5jJ2ZcWZrKojoTv3xiZcUfyorH5fvMxzf7dK8maMITsD36GX1W5jqgXLm7u7LWarayBbOGVRhrRrH2uyt/7H7IE18EyD0m5anVirBDpaDLPaFrdiJFxPiIDcQpmQHzheyo7MdqqBPSByeMaQcIGToZoZgv5djvETOVKq/7UnCe7zRB50WH/kuUoAh4fpGTeIAhpN8f+m4sOHpjMhiPXdIKfMio6UiTqVETrOslzxpH202r1bQ+PlvbeV20Y9l4ZDw2GoZlPDd2jHfGvnFo8EpQ+Vr5XvlRldVv1Z/VXzl1caGIeWhMrervfx9/ohw=</latexit> LOG LIKELIHOOD LOG PRIOR
Approximation of the posterior at the mode: Optimization task: HMAP = argmin H log p(y|X, H) <latexit sha1_base64="IDn6HW7pMXIct8TFfa4DkDVzbFc=">AAAHeXictVVJb9NAFHYLTUrYWjjC4UGV1i5pFBckEFWlAkUUAVWpukSq02jijBMr3rDHJdF0fgP/jRt/hAsXnpfEdhoQF0aKNPPW73tL3PEsM2CNxo+5+WvXF0rlxRuVm7du37m7tHzvJHBDX6fHumu5frNDAmqZDj1mJrNo0/MpsTsWPe0M3kT60wvqB6brHLGRR1s26TmmYeqEoai9vPCtqtmE9XVi8V0Bq9ugcZBjUcfgQ3HOZVMRNRhLRqlEAU20ubmt4ns/0muQOcGqZjrpu8MP0WQXbR1RiAMzbNyQCVGpGi9n+ILmm70+I77vfv2Dn9YnjBtC1vSuy7ZAY33KiIJoiOf57hCMRKMgDEaHjAd1Vo+xV6ow8c1RXxezgxRMlFqO+fk6aI7LxsySqiIwDs02g7hmDLQe/QINUUABENtw1lZREV82I4sIb5AKBpEV1iEw7Sz8fgbZztX3g1AwrR1i1u2J8K04QxitqMu2HJcBmevuhYzSJMcaNha1Ux6pqgUbV0IV3onRNgxkDMfWlDpCiMsX1SnCETpdnEXKeCIWPHb3bU78np002fCJzlWBvJBoaGczllH+mFEeZQOa9O/q7E5aqMAT0CxcjS7JYh1irERdqY6HCrkYEeLLYqNr+Z6mRZwaBcwwkGd64bsZLY1FDSZDrk0RqMDs2aTNHXTYFBPd+/HIK+d8Q80xw93ZyOXGuGkrxwRO/pXA/wP718jnR7jjuFGFQX53IGQDLgtFRV4DeViD4dpWZrg34Wvgul3mSc2MmRgx0+pSXJFi9FQ8EDOz1Cu5Z3syrJ9eHYipcc7MZsx0tDaW2wMv18HLrBy1QtL20kqj3ogPXL2o6WVFSs9Be+m71nX10KYO0y0SBGdqw2MtzM5M3aKiooUB9Yg+ID16hleH2DRo8fjLIaCKki4Yro8/h0EszXtwYgfByO6gZYQxmNZFwlm6s5AZL1r4v+2FjDp6ksgILWAuRJ8h6Jo+1Zk1wgvRfROxgt4nuPsMK1rBIqjTlK9eTjbraqOufn62svM6Lcei9EB6LMmSKj2XdqQ96UA6lvSFn6WHpWpptfSr/Kgsl9cT0/m51Oe+VDjlp78BNDOIew==</latexit> <latexit sha1_base64="IDn6HW7pMXIct8TFfa4DkDVzbFc=">AAAHeXictVVJb9NAFHYLTUrYWjjC4UGV1i5pFBckEFWlAkUUAVWpukSq02jijBMr3rDHJdF0fgP/jRt/hAsXnpfEdhoQF0aKNPPW73tL3PEsM2CNxo+5+WvXF0rlxRuVm7du37m7tHzvJHBDX6fHumu5frNDAmqZDj1mJrNo0/MpsTsWPe0M3kT60wvqB6brHLGRR1s26TmmYeqEoai9vPCtqtmE9XVi8V0Bq9ugcZBjUcfgQ3HOZVMRNRhLRqlEAU20ubmt4ns/0muQOcGqZjrpu8MP0WQXbR1RiAMzbNyQCVGpGi9n+ILmm70+I77vfv2Dn9YnjBtC1vSuy7ZAY33KiIJoiOf57hCMRKMgDEaHjAd1Vo+xV6ow8c1RXxezgxRMlFqO+fk6aI7LxsySqiIwDs02g7hmDLQe/QINUUABENtw1lZREV82I4sIb5AKBpEV1iEw7Sz8fgbZztX3g1AwrR1i1u2J8K04QxitqMu2HJcBmevuhYzSJMcaNha1Ux6pqgUbV0IV3onRNgxkDMfWlDpCiMsX1SnCETpdnEXKeCIWPHb3bU78np002fCJzlWBvJBoaGczllH+mFEeZQOa9O/q7E5aqMAT0CxcjS7JYh1irERdqY6HCrkYEeLLYqNr+Z6mRZwaBcwwkGd64bsZLY1FDSZDrk0RqMDs2aTNHXTYFBPd+/HIK+d8Q80xw93ZyOXGuGkrxwRO/pXA/wP718jnR7jjuFGFQX53IGQDLgtFRV4DeViD4dpWZrg34Wvgul3mSc2MmRgx0+pSXJFi9FQ8EDOz1Cu5Z3syrJ9eHYipcc7MZsx0tDaW2wMv18HLrBy1QtL20kqj3ogPXL2o6WVFSs9Be+m71nX10KYO0y0SBGdqw2MtzM5M3aKiooUB9Yg+ID16hleH2DRo8fjLIaCKki4Yro8/h0EszXtwYgfByO6gZYQxmNZFwlm6s5AZL1r4v+2FjDp6ksgILWAuRJ8h6Jo+1Zk1wgvRfROxgt4nuPsMK1rBIqjTlK9eTjbraqOufn62svM6Lcei9EB6LMmSKj2XdqQ96UA6lvSFn6WHpWpptfSr/Kgsl9cT0/m51Oe+VDjlp78BNDOIew==</latexit> <latexit sha1_base64="IDn6HW7pMXIct8TFfa4DkDVzbFc=">AAAHeXictVVJb9NAFHYLTUrYWjjC4UGV1i5pFBckEFWlAkUUAVWpukSq02jijBMr3rDHJdF0fgP/jRt/hAsXnpfEdhoQF0aKNPPW73tL3PEsM2CNxo+5+WvXF0rlxRuVm7du37m7tHzvJHBDX6fHumu5frNDAmqZDj1mJrNo0/MpsTsWPe0M3kT60wvqB6brHLGRR1s26TmmYeqEoai9vPCtqtmE9XVi8V0Bq9ugcZBjUcfgQ3HOZVMRNRhLRqlEAU20ubmt4ns/0muQOcGqZjrpu8MP0WQXbR1RiAMzbNyQCVGpGi9n+ILmm70+I77vfv2Dn9YnjBtC1vSuy7ZAY33KiIJoiOf57hCMRKMgDEaHjAd1Vo+xV6ow8c1RXxezgxRMlFqO+fk6aI7LxsySqiIwDs02g7hmDLQe/QINUUABENtw1lZREV82I4sIb5AKBpEV1iEw7Sz8fgbZztX3g1AwrR1i1u2J8K04QxitqMu2HJcBmevuhYzSJMcaNha1Ux6pqgUbV0IV3onRNgxkDMfWlDpCiMsX1SnCETpdnEXKeCIWPHb3bU78np002fCJzlWBvJBoaGczllH+mFEeZQOa9O/q7E5aqMAT0CxcjS7JYh1irERdqY6HCrkYEeLLYqNr+Z6mRZwaBcwwkGd64bsZLY1FDSZDrk0RqMDs2aTNHXTYFBPd+/HIK+d8Q80xw93ZyOXGuGkrxwRO/pXA/wP718jnR7jjuFGFQX53IGQDLgtFRV4DeViD4dpWZrg34Wvgul3mSc2MmRgx0+pSXJFi9FQ8EDOz1Cu5Z3syrJ9eHYipcc7MZsx0tDaW2wMv18HLrBy1QtL20kqj3ogPXL2o6WVFSs9Be+m71nX10KYO0y0SBGdqw2MtzM5M3aKiooUB9Yg+ID16hleH2DRo8fjLIaCKki4Yro8/h0EszXtwYgfByO6gZYQxmNZFwlm6s5AZL1r4v+2FjDp6ksgILWAuRJ8h6Jo+1Zk1wgvRfROxgt4nuPsMK1rBIqjTlK9eTjbraqOufn62svM6Lcei9EB6LMmSKj2XdqQ96UA6lvSFn6WHpWpptfSr/Kgsl9cT0/m51Oe+VDjlp78BNDOIew==</latexit> log p(y|X, H) = 1 2 yT KXX 2 n I 1 y 1 2 log det KXX 2 n I N 2 log 2⇡ <latexit sha1_base64="HZeOviACPCFicnOHJScOC6+fIN4=">AAAIsHictVVbj9NGFDbQkjS9LfDYl6Ouwto0ieKoEgi0EgUqqKCrLd1dIu1krYk9Tkx8wx7TRLPz9/oD+tZ/wxnbie1sWu1Da8nSzLl+37loprHvpXw4/PvGzVuffX671f6i8+VXX3/z7d6du2dplCU2O7UjP0rGU5oy3wvZKfe4z8Zxwmgw9dm76eK50r/7yJLUi8ITvorZJKCz0HM9m3IUWXdu/9klAeVzm/rihYT7h0AE6Llo6oqlvBC6Z8gerCWrUmIAkZbwDk28Hyk9gcoJ7hMvLO9T8RZNXqBtKBtxYIdNlHEpO1338Q5fIIk3m3OaJNEf/+BH5pQLV+rEdiL+BAifM04NREPjOImW4BYaA2FwtuQiHfBBjr3ThY1vjfoDuTtIw8To1ZhfPAASRnzNrKgqAhMwtjjkNeNAZuwDDGUDBUBuI7hloiI/jJSFwpuWgoWywjqkXlCFP6ogB7X6vpYGpg0yzHq4Ef4szxHGRHU50PMyIHM7+qijtMhxgI1F7ZZHqZpA/0qoxr0wOoSFjuH4gTFACHn5VJ0Ujix0cBYZF4VYitw9CQRNZkHRZDehtjAl8kKiWVDNWEX5TUV5VQ1o0b+rs7tpoQE/APFxNRxaxXqLsQp1p7seKuTiKsSXzUb36j0ti7g1Cphhoe/0wvtYLY3PXK5DrU0KVOrNAmqJEB1GcqP7ZT3yxoXomzVmuDv9Wm6MW7ZyTeDsugT+P7D/GvniBHccN6oxyC+Ppe7CZaOoyGuhL3uwPHhSGb7a8HVx3S7rpHbGLIy45zsMV6QZvRQv5M4sg063drc20/rrT8dya54rsx1DrfbGj2YQ11p4WdWj18jaub4pQuhvFqbWDNyKExAkfx1Ewhy5bmXVyZJMHrQ64WBdr7/1XIpdDURBdV0Dh/H/NPsm2VHBWOUakdiz9vaHg2H+wdWDWR72tfI7tvb+Ik5kZwELue3TND03hzGfYM+4Z/tMdkiWspjaCzpj53gMacDSichLKqGLEgfcKME/5JBL6x6CBmm6CqZoqRik2zol3KU7z7j7aILPXZxxFtpFIjfzgUegXm9wvITZ3F/hgdqJh1jBnlOsB8c57GARzG3KVw9no4E5HJi//bj/9FlZjrb2nfa9pmum9lB7qr3SjrVTzW71W7+3SGvSHrXHbatNC9ObN0qfe1rja7//BPhD/Nc=</latexit> <latexit sha1_base64="HZeOviACPCFicnOHJScOC6+fIN4=">AAAIsHictVVbj9NGFDbQkjS9LfDYl6Ouwto0ieKoEgi0EgUqqKCrLd1dIu1krYk9Tkx8wx7TRLPz9/oD+tZ/wxnbie1sWu1Da8nSzLl+37loprHvpXw4/PvGzVuffX671f6i8+VXX3/z7d6du2dplCU2O7UjP0rGU5oy3wvZKfe4z8Zxwmgw9dm76eK50r/7yJLUi8ITvorZJKCz0HM9m3IUWXdu/9klAeVzm/rihYT7h0AE6Llo6oqlvBC6Z8gerCWrUmIAkZbwDk28Hyk9gcoJ7hMvLO9T8RZNXqBtKBtxYIdNlHEpO1338Q5fIIk3m3OaJNEf/+BH5pQLV+rEdiL+BAifM04NREPjOImW4BYaA2FwtuQiHfBBjr3ThY1vjfoDuTtIw8To1ZhfPAASRnzNrKgqAhMwtjjkNeNAZuwDDGUDBUBuI7hloiI/jJSFwpuWgoWywjqkXlCFP6ogB7X6vpYGpg0yzHq4Ef4szxHGRHU50PMyIHM7+qijtMhxgI1F7ZZHqZpA/0qoxr0wOoSFjuH4gTFACHn5VJ0Ujix0cBYZF4VYitw9CQRNZkHRZDehtjAl8kKiWVDNWEX5TUV5VQ1o0b+rs7tpoQE/APFxNRxaxXqLsQp1p7seKuTiKsSXzUb36j0ti7g1Cphhoe/0wvtYLY3PXK5DrU0KVOrNAmqJEB1GcqP7ZT3yxoXomzVmuDv9Wm6MW7ZyTeDsugT+P7D/GvniBHccN6oxyC+Ppe7CZaOoyGuhL3uwPHhSGb7a8HVx3S7rpHbGLIy45zsMV6QZvRQv5M4sg063drc20/rrT8dya54rsx1DrfbGj2YQ11p4WdWj18jaub4pQuhvFqbWDNyKExAkfx1Ewhy5bmXVyZJMHrQ64WBdr7/1XIpdDURBdV0Dh/H/NPsm2VHBWOUakdiz9vaHg2H+wdWDWR72tfI7tvb+Ik5kZwELue3TND03hzGfYM+4Z/tMdkiWspjaCzpj53gMacDSichLKqGLEgfcKME/5JBL6x6CBmm6CqZoqRik2zol3KU7z7j7aILPXZxxFtpFIjfzgUegXm9wvITZ3F/hgdqJh1jBnlOsB8c57GARzG3KVw9no4E5HJi//bj/9FlZjrb2nfa9pmum9lB7qr3SjrVTzW71W7+3SGvSHrXHbatNC9ObN0qfe1rja7//BPhD/Nc=</latexit> <latexit sha1_base64="HZeOviACPCFicnOHJScOC6+fIN4=">AAAIsHictVVbj9NGFDbQkjS9LfDYl6Ouwto0ieKoEgi0EgUqqKCrLd1dIu1krYk9Tkx8wx7TRLPz9/oD+tZ/wxnbie1sWu1Da8nSzLl+37loprHvpXw4/PvGzVuffX671f6i8+VXX3/z7d6du2dplCU2O7UjP0rGU5oy3wvZKfe4z8Zxwmgw9dm76eK50r/7yJLUi8ITvorZJKCz0HM9m3IUWXdu/9klAeVzm/rihYT7h0AE6Llo6oqlvBC6Z8gerCWrUmIAkZbwDk28Hyk9gcoJ7hMvLO9T8RZNXqBtKBtxYIdNlHEpO1338Q5fIIk3m3OaJNEf/+BH5pQLV+rEdiL+BAifM04NREPjOImW4BYaA2FwtuQiHfBBjr3ThY1vjfoDuTtIw8To1ZhfPAASRnzNrKgqAhMwtjjkNeNAZuwDDGUDBUBuI7hloiI/jJSFwpuWgoWywjqkXlCFP6ogB7X6vpYGpg0yzHq4Ef4szxHGRHU50PMyIHM7+qijtMhxgI1F7ZZHqZpA/0qoxr0wOoSFjuH4gTFACHn5VJ0Ujix0cBYZF4VYitw9CQRNZkHRZDehtjAl8kKiWVDNWEX5TUV5VQ1o0b+rs7tpoQE/APFxNRxaxXqLsQp1p7seKuTiKsSXzUb36j0ti7g1Cphhoe/0wvtYLY3PXK5DrU0KVOrNAmqJEB1GcqP7ZT3yxoXomzVmuDv9Wm6MW7ZyTeDsugT+P7D/GvniBHccN6oxyC+Ppe7CZaOoyGuhL3uwPHhSGb7a8HVx3S7rpHbGLIy45zsMV6QZvRQv5M4sg063drc20/rrT8dya54rsx1DrfbGj2YQ11p4WdWj18jaub4pQuhvFqbWDNyKExAkfx1Ewhy5bmXVyZJMHrQ64WBdr7/1XIpdDURBdV0Dh/H/NPsm2VHBWOUakdiz9vaHg2H+wdWDWR72tfI7tvb+Ik5kZwELue3TND03hzGfYM+4Z/tMdkiWspjaCzpj53gMacDSichLKqGLEgfcKME/5JBL6x6CBmm6CqZoqRik2zol3KU7z7j7aILPXZxxFtpFIjfzgUegXm9wvITZ3F/hgdqJh1jBnlOsB8c57GARzG3KVw9no4E5HJi//bj/9FlZjrb2nfa9pmum9lB7qr3SjrVTzW71W7+3SGvSHrXHbatNC9ObN0qfe1rja7//BPhD/Nc=</latexit>
THE ABSTRACT `` By partitioning and distributing kernel matrix multiplies, we demonstrate that an exact GP can be trained on over a million points in 3 days using 8GPUs and can compute predictive means and variances in under a second using 1 GPU attest time. Moreover, we perform the first-ever comparison of exact GPs against state-of-the-art scalable approximations on large-scale regression datasets with104−106 data points, showing dramatic performance improvements. ‘’[1] Ref.: 1. Wang, Ke Alexander, Geoff Pleiss, Jacob R. Gardner, Stephen Tyree, Kilian Q. Weinberger, and Andrew Gordon Wilson. "Exact Gaussian Processes on a Million Data Points." arXiv preprint arXiv:1903.08114 (2019).
(Sparse Gaussian process regression) • ``Summarize” N training data points with M training data points (or pseudo-inputs) – different choices of pseudo- inputs lead to different Sparse GP strategies (M << N). • Reduce training cost from O(N3) to O(NM2) and storage cost from O(N2) to O(NM). • Do not require special structure in the data, i.e., meant to be usable ‘off-the-shelf’. • Key examples – Snelson, Gahramani (2006)[1], Titsias (2009)[2] (and many others). § Structure-exploiting approaches: • Exploit the fact the dataset is generated with some meaningful structure (such as grid inputs). • Main types of structure – Kronecker[3] and Toeplitz[4]. • Kronecker structure arises when inputs are on a multidimensional lattice; Toeplitz structure arises when the data lies on a regularly spaced 1D – making such methods highly useful for some scientific applications (thinking surrogate models for PDE solvers) but not so much for general purpose datasets. Ref.: 1. Snelson, Edward and Ghahramani, Zoubin. Sparse Gaussian processes using pseudo-inputs. In Advances in neural information processing systems (NIPS), volume 18, pp. 1257. MIT Press, 2006. 2. Titsias, Michalis. Variational learning of inducing variables in sparse Gaussian processes. In Artificial Intelligence and Statistics, pp. 567-574. 2009. 3. Saatçi, Yunus. Scalable inference for structured Gaussian process models. PhD thesis., University of Cambridge, 2012. 4. Cunningham, John P., Krishna V. Shenoy, and Maneesh Sahani. Fast Gaussian process methods for point process intensity estimation. In Proceedings of the 25th international conference on Machine learning, pp. 192-199. ACM, 2008.
the covariance kernel / matrix: § Optimal rank q approximation through SVD is not of practical interest; we need an alternative approach to computing the low rank approximation of K. § Nystrom approximation[1] – first q eigenfunctions of the following integral operator: § Random Fourier Features[2] – Approximating the kernel by sampling it’s spectral density: Ref.: 1. Williams, Christopher KI, and Matthias Seeger. "Using the Nyström method to speed up kernel machines." In Advances in neural information processing systems, pp. 682-688. 2001. 2. Rahimi, Ali, and Benjamin Recht. "Random features for large-scale kernel machines." In Advances in neural information processing systems, pp. 1177-1184. 2008. K = QQT , where, Q 2 RN⇥q =) Fast inversion of QQT + 2 n I using Matrix inversion Lemma. <latexit sha1_base64="r3AVmTFhBiJF8lTEWf2l6efeaf8=">AAAJbHictVZbbxtFFN6WS2xzaQp9KKqQjoicrItjeSMkqlaWChQooo2SKkkjZW1rvJ61R95bd2dbW5N54Sfyxk/ghd/Amd31XuwF5QFWijRzrt93Lp5MAodFvN//49bt997/4MOdRrP10ceffHpn9+5nF5EfhxY9t3zHDy8nJKIO8+g5Z9yhl0FIiTtx6OvJ4gelf/2WhhHzvTO+CujQJTOP2cwiHEXjuzu/tU2X8LlFHPFMwv4ATAF6IprYYilHQmcd2YW1ZJVJOmDKsWADA+/HSm9C4QT7JvOy+0S8QpNnaOvJShyosfFjLmWrbT+u8QUzZLM5J2Hov/sHP3NOuLClblpTnz8Bk88pJx1EQ4Ig9Jdgp5oOwuB0yUXU470Ee6sNuW+J+kNZH6Ri0umWmI8egun5fM0srSoCE3A55pDUjIM5o2+gLysoABIbwccGKpLDkbJQeKNMsFBWWIeIuUX44wKyW6rvr7KDad0Ysw5y4Y/yCmEMVZddPSkDMrf8tzpK0xwH2FjUbnhkqiEcboWq3FOjASx0DMcPOj2EkJRP1UnhiL0pziLlIhVLkbiHriDhzE2bbIfEEoZEXkg0dosZKyi/KCivigFN+7c9u3kLO/A1mA6uxpQUsV5hrFTdaq+HCrnYCvF1tdHdck+zIm6MAmZY6LVeeL9US+NQm+tQapMCFbGZS8bCQ4cjmet+WY98ZyQOjRIz3J3DUm6Mm7VyTeDipgT+P7D/Gnl0hjuOG1UZ5J9PpG7DdaWoyGuhL7uwPHhSGD7P+dq4btdlUrUxUyPOnCnFFalGz8QLWZul12qX7uN8Wl9+dyI35rkwqxlqtTeOP4Og1MLroh7dSlbMeWNbxHCYb0ypG7gWZyDM5HkQIZ3KdS+LVmZskqDFCSfrZg0u51L0SiBSrusiTCn/T7PnyY5TxirXkRmwVmlKB3B6Ojpb/77CuzkNaRcknG69OcdqAlwawRv1Hrn44uI5dfuJRByYlz2e4NsgTTMjosLnu6CAKsB1YDMEccS8GbwkPGTLUswX1HVJT4539/q9fvLB9sHIDnta9p2Md383p74Vu9TjlkOi6MroB3yI88aZ5VDZMuOIBsRakBm9wqNHkOFQJNMgoY2SKdh+iH8eh0Ra9hDEjaKVO0FLxSfa1Clhne4q5vajIT7VQcypZ6WJ7NgB7oP6zwOmLKQWd1Z4IFbIECtYc4Kt5FiOFhbB2KS8fbg46hn9nnH6zd7T77NyNLQH2learhnat9pT7bl2op1r1s6fjTuN+40vGn817zUfNL9MTW/fynw+1ypfc/9vL7dAEw==</latexit> <latexit sha1_base64="r3AVmTFhBiJF8lTEWf2l6efeaf8=">AAAJbHictVZbbxtFFN6WS2xzaQp9KKqQjoicrItjeSMkqlaWChQooo2SKkkjZW1rvJ61R95bd2dbW5N54Sfyxk/ghd/Amd31XuwF5QFWijRzrt93Lp5MAodFvN//49bt997/4MOdRrP10ceffHpn9+5nF5EfhxY9t3zHDy8nJKIO8+g5Z9yhl0FIiTtx6OvJ4gelf/2WhhHzvTO+CujQJTOP2cwiHEXjuzu/tU2X8LlFHPFMwv4ATAF6IprYYilHQmcd2YW1ZJVJOmDKsWADA+/HSm9C4QT7JvOy+0S8QpNnaOvJShyosfFjLmWrbT+u8QUzZLM5J2Hov/sHP3NOuLClblpTnz8Bk88pJx1EQ4Ig9Jdgp5oOwuB0yUXU470Ee6sNuW+J+kNZH6Ri0umWmI8egun5fM0srSoCE3A55pDUjIM5o2+gLysoABIbwccGKpLDkbJQeKNMsFBWWIeIuUX44wKyW6rvr7KDad0Ysw5y4Y/yCmEMVZddPSkDMrf8tzpK0xwH2FjUbnhkqiEcboWq3FOjASx0DMcPOj2EkJRP1UnhiL0pziLlIhVLkbiHriDhzE2bbIfEEoZEXkg0dosZKyi/KCivigFN+7c9u3kLO/A1mA6uxpQUsV5hrFTdaq+HCrnYCvF1tdHdck+zIm6MAmZY6LVeeL9US+NQm+tQapMCFbGZS8bCQ4cjmet+WY98ZyQOjRIz3J3DUm6Mm7VyTeDipgT+P7D/Gnl0hjuOG1UZ5J9PpG7DdaWoyGuhL7uwPHhSGD7P+dq4btdlUrUxUyPOnCnFFalGz8QLWZul12qX7uN8Wl9+dyI35rkwqxlqtTeOP4Og1MLroh7dSlbMeWNbxHCYb0ypG7gWZyDM5HkQIZ3KdS+LVmZskqDFCSfrZg0u51L0SiBSrusiTCn/T7PnyY5TxirXkRmwVmlKB3B6Ojpb/77CuzkNaRcknG69OcdqAlwawRv1Hrn44uI5dfuJRByYlz2e4NsgTTMjosLnu6CAKsB1YDMEccS8GbwkPGTLUswX1HVJT4539/q9fvLB9sHIDnta9p2Md383p74Vu9TjlkOi6MroB3yI88aZ5VDZMuOIBsRakBm9wqNHkOFQJNMgoY2SKdh+iH8eh0Ra9hDEjaKVO0FLxSfa1Clhne4q5vajIT7VQcypZ6WJ7NgB7oP6zwOmLKQWd1Z4IFbIECtYc4Kt5FiOFhbB2KS8fbg46hn9nnH6zd7T77NyNLQH2learhnat9pT7bl2op1r1s6fjTuN+40vGn817zUfNL9MTW/fynw+1ypfc/9vL7dAEw==</latexit> <latexit sha1_base64="r3AVmTFhBiJF8lTEWf2l6efeaf8=">AAAJbHictVZbbxtFFN6WS2xzaQp9KKqQjoicrItjeSMkqlaWChQooo2SKkkjZW1rvJ61R95bd2dbW5N54Sfyxk/ghd/Amd31XuwF5QFWijRzrt93Lp5MAodFvN//49bt997/4MOdRrP10ceffHpn9+5nF5EfhxY9t3zHDy8nJKIO8+g5Z9yhl0FIiTtx6OvJ4gelf/2WhhHzvTO+CujQJTOP2cwiHEXjuzu/tU2X8LlFHPFMwv4ATAF6IprYYilHQmcd2YW1ZJVJOmDKsWADA+/HSm9C4QT7JvOy+0S8QpNnaOvJShyosfFjLmWrbT+u8QUzZLM5J2Hov/sHP3NOuLClblpTnz8Bk88pJx1EQ4Ig9Jdgp5oOwuB0yUXU470Ee6sNuW+J+kNZH6Ri0umWmI8egun5fM0srSoCE3A55pDUjIM5o2+gLysoABIbwccGKpLDkbJQeKNMsFBWWIeIuUX44wKyW6rvr7KDad0Ysw5y4Y/yCmEMVZddPSkDMrf8tzpK0xwH2FjUbnhkqiEcboWq3FOjASx0DMcPOj2EkJRP1UnhiL0pziLlIhVLkbiHriDhzE2bbIfEEoZEXkg0dosZKyi/KCivigFN+7c9u3kLO/A1mA6uxpQUsV5hrFTdaq+HCrnYCvF1tdHdck+zIm6MAmZY6LVeeL9US+NQm+tQapMCFbGZS8bCQ4cjmet+WY98ZyQOjRIz3J3DUm6Mm7VyTeDipgT+P7D/Gnl0hjuOG1UZ5J9PpG7DdaWoyGuhL7uwPHhSGD7P+dq4btdlUrUxUyPOnCnFFalGz8QLWZul12qX7uN8Wl9+dyI35rkwqxlqtTeOP4Og1MLroh7dSlbMeWNbxHCYb0ypG7gWZyDM5HkQIZ3KdS+LVmZskqDFCSfrZg0u51L0SiBSrusiTCn/T7PnyY5TxirXkRmwVmlKB3B6Ojpb/77CuzkNaRcknG69OcdqAlwawRv1Hrn44uI5dfuJRByYlz2e4NsgTTMjosLnu6CAKsB1YDMEccS8GbwkPGTLUswX1HVJT4539/q9fvLB9sHIDnta9p2Md383p74Vu9TjlkOi6MroB3yI88aZ5VDZMuOIBsRakBm9wqNHkOFQJNMgoY2SKdh+iH8eh0Ra9hDEjaKVO0FLxSfa1Clhne4q5vajIT7VQcypZ6WJ7NgB7oP6zwOmLKQWd1Z4IFbIECtYc4Kt5FiOFhbB2KS8fbg46hn9nnH6zd7T77NyNLQH2learhnat9pT7bl2op1r1s6fjTuN+40vGn817zUfNL9MTW/fynw+1ypfc/9vL7dAEw==</latexit> Z k(x, x0) i(x)dx = i i(x), ˜ k(x, x0) = q X i=1 i i(x) i(x0) <latexit sha1_base64="oMwPnQqXDITjs2HSELMG+2XPyhY=">AAAKQXictVbLj9tEGHdbHkl4bcuRyydWaeySjeIVEogqUoECRbSr3Wpf0jqJJs44GcWv2uM20ez8a1z4D7hx58IBhLhy4Rvbie3ERXsAS5HG3/P3+x6eTEKXxbzf/+XW7TtvvPnW241m651333v/g727987jIIlsemYHbhBdTkhMXebTM864Sy/DiBJv4tKLyeJrpb94SaOYBf4pX4V06JGZzxxmE46i8d3GRdvyCJ/bxBWPJdwfgCVAT0UTRyzlSOjMkF1YS1a5xABLjgUbmPh+pPQWFE5w32J+/j4Rz9HkMdr6shIHamyChEvZajtf1PiCFbHZnJMoCl69xs+aEy4cqVv2NOAPweJzyomBaEgYRsESnExjIAxOl1zEPd5LsbfasPEtUX8g64NUTIxuifnoAVh+wNfMsqoiMAGXYw5pzThYM/oC+rKCAiC1EXxsoiI9HCoLhTfOBQtlhXWImVeEPyoge6X6/iANTOslmHWwEX4jrxDGUHXZ09MyIHM7eKmjNMvRwcaidssjVw3hYCdU5T0zGsBCx3C8Y/QQQlo+VSeFI/GnOIuUi0wsReoeeYJEMy9rshMRW5gSeSHRxCtmrKD8tKC8KgY069/u7G5aaMAnYLm4GlNSxHqOsTJ1q70eKuTiKMTX1UZ3yz3Ni7g1Cphhodd64fulWhqXOlyHUpsUqJjNPDIWPjocyo3u+/XIGyNxYJaY4e4clHJj3LyVawLnNyXw/4H918ijU9xx3KjKIH93LHUHritFRV4LfdmFZedhYfhkw9fBdbsuk6qNmRlx5k4prkg1ei5eyNosvVa79D7eTOuzL4/l1jwXZjVDrfbGDWYQllp4XdSjW8mKOW9sixgONhtT6gauxSkIK70eRESnct3LopU5mzRoccLJulmDy7kUvRKIjOu6CFPK/9Psm2RHGWOV69AK2Xr60zEdwMnJ6HT9gYVXcxrRLkg42bl0jtQIeDSGF+pC8vDKxXPm9i2JOTA/vz0hcEBaVs5Ehd8sg0KqENehzREkMfNn8IzwiC1LMZ9SzyM92UJUvLIy5Tulg1HCOcMvYWV4iwrLQqqGMvvEjVmtVxfv6dLUvzbjoPL1VcWphmVVKNuijjHe2+/3+ukDuwczP+xr+XM83vvZmgZ24lGf2y6J4yuzH/IhLhFntkuxRklMQ2IvyIxe4dEn2LWhSEdcQhslU3CCCH9YylRa9hDEi+OVN0FLhTHe1ilhne4q4c7nQ/z/ESac+naWyElc4AGov1MwZRG1ubvCA7EjhljBnhOcT44tbmERzG3Ku4fzw57Z75knn+4/+iovR0P7SPtY0zVT+0x7pD3RjrUzzW782Pi18Xvjj+ZPzd+afzb/ykxv38p9PtQqT/PvfwBelpQO</latexit> <latexit sha1_base64="oMwPnQqXDITjs2HSELMG+2XPyhY=">AAAKQXictVbLj9tEGHdbHkl4bcuRyydWaeySjeIVEogqUoECRbSr3Wpf0jqJJs44GcWv2uM20ez8a1z4D7hx58IBhLhy4Rvbie3ERXsAS5HG3/P3+x6eTEKXxbzf/+XW7TtvvPnW241m651333v/g727987jIIlsemYHbhBdTkhMXebTM864Sy/DiBJv4tKLyeJrpb94SaOYBf4pX4V06JGZzxxmE46i8d3GRdvyCJ/bxBWPJdwfgCVAT0UTRyzlSOjMkF1YS1a5xABLjgUbmPh+pPQWFE5w32J+/j4Rz9HkMdr6shIHamyChEvZajtf1PiCFbHZnJMoCl69xs+aEy4cqVv2NOAPweJzyomBaEgYRsESnExjIAxOl1zEPd5LsbfasPEtUX8g64NUTIxuifnoAVh+wNfMsqoiMAGXYw5pzThYM/oC+rKCAiC1EXxsoiI9HCoLhTfOBQtlhXWImVeEPyoge6X6/iANTOslmHWwEX4jrxDGUHXZ09MyIHM7eKmjNMvRwcaidssjVw3hYCdU5T0zGsBCx3C8Y/QQQlo+VSeFI/GnOIuUi0wsReoeeYJEMy9rshMRW5gSeSHRxCtmrKD8tKC8KgY069/u7G5aaMAnYLm4GlNSxHqOsTJ1q70eKuTiKMTX1UZ3yz3Ni7g1Cphhodd64fulWhqXOlyHUpsUqJjNPDIWPjocyo3u+/XIGyNxYJaY4e4clHJj3LyVawLnNyXw/4H918ijU9xx3KjKIH93LHUHritFRV4LfdmFZedhYfhkw9fBdbsuk6qNmRlx5k4prkg1ei5eyNosvVa79D7eTOuzL4/l1jwXZjVDrfbGDWYQllp4XdSjW8mKOW9sixgONhtT6gauxSkIK70eRESnct3LopU5mzRoccLJulmDy7kUvRKIjOu6CFPK/9Psm2RHGWOV69AK2Xr60zEdwMnJ6HT9gYVXcxrRLkg42bl0jtQIeDSGF+pC8vDKxXPm9i2JOTA/vz0hcEBaVs5Ehd8sg0KqENehzREkMfNn8IzwiC1LMZ9SzyM92UJUvLIy5Tulg1HCOcMvYWV4iwrLQqqGMvvEjVmtVxfv6dLUvzbjoPL1VcWphmVVKNuijjHe2+/3+ukDuwczP+xr+XM83vvZmgZ24lGf2y6J4yuzH/IhLhFntkuxRklMQ2IvyIxe4dEn2LWhSEdcQhslU3CCCH9YylRa9hDEi+OVN0FLhTHe1ilhne4q4c7nQ/z/ESac+naWyElc4AGov1MwZRG1ubvCA7EjhljBnhOcT44tbmERzG3Ku4fzw57Z75knn+4/+iovR0P7SPtY0zVT+0x7pD3RjrUzzW782Pi18Xvjj+ZPzd+afzb/ykxv38p9PtQqT/PvfwBelpQO</latexit> <latexit sha1_base64="oMwPnQqXDITjs2HSELMG+2XPyhY=">AAAKQXictVbLj9tEGHdbHkl4bcuRyydWaeySjeIVEogqUoECRbSr3Wpf0jqJJs44GcWv2uM20ez8a1z4D7hx58IBhLhy4Rvbie3ERXsAS5HG3/P3+x6eTEKXxbzf/+XW7TtvvPnW241m651333v/g727987jIIlsemYHbhBdTkhMXebTM864Sy/DiBJv4tKLyeJrpb94SaOYBf4pX4V06JGZzxxmE46i8d3GRdvyCJ/bxBWPJdwfgCVAT0UTRyzlSOjMkF1YS1a5xABLjgUbmPh+pPQWFE5w32J+/j4Rz9HkMdr6shIHamyChEvZajtf1PiCFbHZnJMoCl69xs+aEy4cqVv2NOAPweJzyomBaEgYRsESnExjIAxOl1zEPd5LsbfasPEtUX8g64NUTIxuifnoAVh+wNfMsqoiMAGXYw5pzThYM/oC+rKCAiC1EXxsoiI9HCoLhTfOBQtlhXWImVeEPyoge6X6/iANTOslmHWwEX4jrxDGUHXZ09MyIHM7eKmjNMvRwcaidssjVw3hYCdU5T0zGsBCx3C8Y/QQQlo+VSeFI/GnOIuUi0wsReoeeYJEMy9rshMRW5gSeSHRxCtmrKD8tKC8KgY069/u7G5aaMAnYLm4GlNSxHqOsTJ1q70eKuTiKMTX1UZ3yz3Ni7g1Cphhodd64fulWhqXOlyHUpsUqJjNPDIWPjocyo3u+/XIGyNxYJaY4e4clHJj3LyVawLnNyXw/4H918ijU9xx3KjKIH93LHUHritFRV4LfdmFZedhYfhkw9fBdbsuk6qNmRlx5k4prkg1ei5eyNosvVa79D7eTOuzL4/l1jwXZjVDrfbGDWYQllp4XdSjW8mKOW9sixgONhtT6gauxSkIK70eRESnct3LopU5mzRoccLJulmDy7kUvRKIjOu6CFPK/9Psm2RHGWOV69AK2Xr60zEdwMnJ6HT9gYVXcxrRLkg42bl0jtQIeDSGF+pC8vDKxXPm9i2JOTA/vz0hcEBaVs5Ehd8sg0KqENehzREkMfNn8IzwiC1LMZ9SzyM92UJUvLIy5Tulg1HCOcMvYWV4iwrLQqqGMvvEjVmtVxfv6dLUvzbjoPL1VcWphmVVKNuijjHe2+/3+ukDuwczP+xr+XM83vvZmgZ24lGf2y6J4yuzH/IhLhFntkuxRklMQ2IvyIxe4dEn2LWhSEdcQhslU3CCCH9YylRa9hDEi+OVN0FLhTHe1ilhne4q4c7nQ/z/ESac+naWyElc4AGov1MwZRG1ubvCA7EjhljBnhOcT44tbmERzG3Ku4fzw57Z75knn+4/+iovR0P7SPtY0zVT+0x7pD3RjrUzzW782Pi18Xvjj+ZPzd+afzb/ykxv38p9PtQqT/PvfwBelpQO</latexit> s(!) = F{k(r)} = Z 1 1 k(r)e i!rdr, k(r) = F 1{s(!)} = Z 1 1 s(!)ei!rdr, ˜ k(r) = 1 M q X i=1 cos(!ir). <latexit sha1_base64="6D4igp2AJWHDGLPuqNJaMBdmvmQ=">AAALT3ictVZLb9tGEGbSV0S3jdMeexnUkE2mkiAZBVo0EJC2aZqisWEHtmPAlIQVtZQW4ivLZSKB3n/YS3vr3+ilhxZFZ/kQSUkOfGgJCFrOzuub+WaX49Blkeh2f79z951333v/g3sNfefDjz6+v/vgk4soiLlNz+3ADfjlmETUZT49F0y49DLklHhjl74cz79X+y9fUx6xwD8Ty5AOPDL1mcNsIlA0etCgTcsjYmYTN3kiYb8PVgJGKho7yUIOE4OZsgWFZJlLTLDkKGH9Hr4fq30LSiPYt5ifv4+TF6jyBHV9WfMDW3SCWEipN51vttiCxdl0JgjnwZsb7KwZEYkjDcueBOIRWGJGBTExGxKGPFiAk+2YmIagC5FEHdFJc9ebsLKtQH8otzupqZitCvLhQ7D8QBTIsqpiYglcjgSkNRNgTekr6MpaFgCpTiJGPdxIF4dKQ+Ub5YK50sI6RMwr3R+XKXuV+v4sTQzrxRi1vxL+IK8wjYHqsmekZUDkdvDaQGkW4wAbi7trFvnWANobrmrvmVIf5ga6EwdmB1NIy6fqpPKI/QlykYokE8skNedeQvjUy5rscGInPYm4EGjslRwrIT8vIS9Lgmb92+TuqoUmfAGWi6MxIaWvF+gr29abBakQi6Myvq43ulXtaV7ENSpghLmx1QrfL9XQuNQRBlTapJKK2NQjo8RHg0O52vupoLw5TNq9CjKcnXYlNvrNW1kAuLgtgP8v2bd6Hp7hjONE1Yj844k0HLiuFRVxzY1FCxYHj0rFZyu8Do7bdRXUVp+ZkmDuhOKI1L3n4rncGqWjNyvvoxVbj749kWt8LtW2kFrNjRtMIay08LqsR6sWFWPeWhdzaK8mptINHIszSKz0ekg4nciil2UrczSp03KFzLpdg6uxFLxKEhnWoggTKv7T6KtgxxliFevQClnB/pSmfTg9HZ4VByy8mVFOWyDhdOPSOVYU8GgEr9SF5OGVi+vM7CmJBDA/vz0hcEBaVo5EuV8Ng8pUZbwt2zyDOGL+FI6I4GxR8fmceh7pqAuC+aI2M9VL5QDdhDOGR2GNvWWJZSlVrMzOuBHbatXCi7pC+xsj9mvHr6pO3S2rp7IuOjD1/ciwAo9OiVlcQYq1TyVedHODqw8IJUfgo6St/h2hTvN8kalQ5BrGS70AlxXEvAW6lSnVnKfkxAhl7LeHKfUw1o2h8LDZL2uWxSwYfyQ3KmUHhVusFjc7AKPdvW6nmz6wuejliz0tf05Gu79Zk8COPeoL2yVRdNXrhmKAR4pgtkulbsURDYk9J1N6hUufIIcHSTrwEpoomYATcPwhr1Jp1SIhXhQtvTFqKpTR+p4Sbtu7ioXz9QC/xsJYUN/OAjmxCyIA9XEJE8apLdwlLojNGeYK9oxgoQQSXsci9NYhby4uDju9bqd3+uXe4+/yctzTPtM+1wytp32lPdaeaSfauWY3fmn80fir8bf+q/6n/s9Ornr3Tr74VKs9O41/AVEU8BU=</latexit> <latexit sha1_base64="6D4igp2AJWHDGLPuqNJaMBdmvmQ=">AAALT3ictVZLb9tGEGbSV0S3jdMeexnUkE2mkiAZBVo0EJC2aZqisWEHtmPAlIQVtZQW4ivLZSKB3n/YS3vr3+ilhxZFZ/kQSUkOfGgJCFrOzuub+WaX49Blkeh2f79z951333v/g3sNfefDjz6+v/vgk4soiLlNz+3ADfjlmETUZT49F0y49DLklHhjl74cz79X+y9fUx6xwD8Ty5AOPDL1mcNsIlA0etCgTcsjYmYTN3kiYb8PVgJGKho7yUIOE4OZsgWFZJlLTLDkKGH9Hr4fq30LSiPYt5ifv4+TF6jyBHV9WfMDW3SCWEipN51vttiCxdl0JgjnwZsb7KwZEYkjDcueBOIRWGJGBTExGxKGPFiAk+2YmIagC5FEHdFJc9ebsLKtQH8otzupqZitCvLhQ7D8QBTIsqpiYglcjgSkNRNgTekr6MpaFgCpTiJGPdxIF4dKQ+Ub5YK50sI6RMwr3R+XKXuV+v4sTQzrxRi1vxL+IK8wjYHqsmekZUDkdvDaQGkW4wAbi7trFvnWANobrmrvmVIf5ga6EwdmB1NIy6fqpPKI/QlykYokE8skNedeQvjUy5rscGInPYm4EGjslRwrIT8vIS9Lgmb92+TuqoUmfAGWi6MxIaWvF+gr29abBakQi6Myvq43ulXtaV7ENSpghLmx1QrfL9XQuNQRBlTapJKK2NQjo8RHg0O52vupoLw5TNq9CjKcnXYlNvrNW1kAuLgtgP8v2bd6Hp7hjONE1Yj844k0HLiuFRVxzY1FCxYHj0rFZyu8Do7bdRXUVp+ZkmDuhOKI1L3n4rncGqWjNyvvoxVbj749kWt8LtW2kFrNjRtMIay08LqsR6sWFWPeWhdzaK8mptINHIszSKz0ekg4nciil2UrczSp03KFzLpdg6uxFLxKEhnWoggTKv7T6KtgxxliFevQClnB/pSmfTg9HZ4VByy8mVFOWyDhdOPSOVYU8GgEr9SF5OGVi+vM7CmJBDA/vz0hcEBaVo5EuV8Ng8pUZbwt2zyDOGL+FI6I4GxR8fmceh7pqAuC+aI2M9VL5QDdhDOGR2GNvWWJZSlVrMzOuBHbatXCi7pC+xsj9mvHr6pO3S2rp7IuOjD1/ciwAo9OiVlcQYq1TyVedHODqw8IJUfgo6St/h2hTvN8kalQ5BrGS70AlxXEvAW6lSnVnKfkxAhl7LeHKfUw1o2h8LDZL2uWxSwYfyQ3KmUHhVusFjc7AKPdvW6nmz6wuejliz0tf05Gu79Zk8COPeoL2yVRdNXrhmKAR4pgtkulbsURDYk9J1N6hUufIIcHSTrwEpoomYATcPwhr1Jp1SIhXhQtvTFqKpTR+p4Sbtu7ioXz9QC/xsJYUN/OAjmxCyIA9XEJE8apLdwlLojNGeYK9oxgoQQSXsci9NYhby4uDju9bqd3+uXe4+/yctzTPtM+1wytp32lPdaeaSfauWY3fmn80fir8bf+q/6n/s9Ornr3Tr74VKs9O41/AVEU8BU=</latexit> <latexit sha1_base64="6D4igp2AJWHDGLPuqNJaMBdmvmQ=">AAALT3ictVZLb9tGEGbSV0S3jdMeexnUkE2mkiAZBVo0EJC2aZqisWEHtmPAlIQVtZQW4ivLZSKB3n/YS3vr3+ilhxZFZ/kQSUkOfGgJCFrOzuub+WaX49Blkeh2f79z951333v/g3sNfefDjz6+v/vgk4soiLlNz+3ADfjlmETUZT49F0y49DLklHhjl74cz79X+y9fUx6xwD8Ty5AOPDL1mcNsIlA0etCgTcsjYmYTN3kiYb8PVgJGKho7yUIOE4OZsgWFZJlLTLDkKGH9Hr4fq30LSiPYt5ifv4+TF6jyBHV9WfMDW3SCWEipN51vttiCxdl0JgjnwZsb7KwZEYkjDcueBOIRWGJGBTExGxKGPFiAk+2YmIagC5FEHdFJc9ebsLKtQH8otzupqZitCvLhQ7D8QBTIsqpiYglcjgSkNRNgTekr6MpaFgCpTiJGPdxIF4dKQ+Ub5YK50sI6RMwr3R+XKXuV+v4sTQzrxRi1vxL+IK8wjYHqsmekZUDkdvDaQGkW4wAbi7trFvnWANobrmrvmVIf5ga6EwdmB1NIy6fqpPKI/QlykYokE8skNedeQvjUy5rscGInPYm4EGjslRwrIT8vIS9Lgmb92+TuqoUmfAGWi6MxIaWvF+gr29abBakQi6Myvq43ulXtaV7ENSpghLmx1QrfL9XQuNQRBlTapJKK2NQjo8RHg0O52vupoLw5TNq9CjKcnXYlNvrNW1kAuLgtgP8v2bd6Hp7hjONE1Yj844k0HLiuFRVxzY1FCxYHj0rFZyu8Do7bdRXUVp+ZkmDuhOKI1L3n4rncGqWjNyvvoxVbj749kWt8LtW2kFrNjRtMIay08LqsR6sWFWPeWhdzaK8mptINHIszSKz0ekg4nciil2UrczSp03KFzLpdg6uxFLxKEhnWoggTKv7T6KtgxxliFevQClnB/pSmfTg9HZ4VByy8mVFOWyDhdOPSOVYU8GgEr9SF5OGVi+vM7CmJBDA/vz0hcEBaVo5EuV8Ng8pUZbwt2zyDOGL+FI6I4GxR8fmceh7pqAuC+aI2M9VL5QDdhDOGR2GNvWWJZSlVrMzOuBHbatXCi7pC+xsj9mvHr6pO3S2rp7IuOjD1/ciwAo9OiVlcQYq1TyVedHODqw8IJUfgo6St/h2hTvN8kalQ5BrGS70AlxXEvAW6lSnVnKfkxAhl7LeHKfUw1o2h8LDZL2uWxSwYfyQ3KmUHhVusFjc7AKPdvW6nmz6wuejliz0tf05Gu79Zk8COPeoL2yVRdNXrhmKAR4pgtkulbsURDYk9J1N6hUufIIcHSTrwEpoomYATcPwhr1Jp1SIhXhQtvTFqKpTR+p4Sbtu7ioXz9QC/xsJYUN/OAjmxCyIA9XEJE8apLdwlLojNGeYK9oxgoQQSXsci9NYhby4uDju9bqd3+uXe4+/yctzTPtM+1wytp32lPdaeaSfauWY3fmn80fir8bf+q/6n/s9Ornr3Tr74VKs9O41/AVEU8BU=</latexit>
of dense matrices arising in scientific applications can be represented as hierarchical matrices. § Goal – Accelerate the computation of determinant/inverse of the Gram matrix by making hierarchical approximations to the covariance matrix. § We discuss the methodology and results in Ambikasaran (2014) [1]. Ref.: 1. Ambikasaran, Sivaram, Daniel Foreman-Mackey, Leslie Greengard, David W. Hogg, and Michael O'Neil. "Fast direct methods for gaussian processes." arXiv preprint arXiv:1403.6015 (2014).
Rn⇥n <latexit sha1_base64="PIFlW0otVBg45kxOOaeFPjrHsfM=">AAAL+HictVZbj9tEFHbLrThQtvDIy4FVdu2SRMmCBKKKVGgpRe0uu9XudqV1Ek2ccTKKb7XHbZbZ+SW88ABCvPJTeOPfcMaX2E6y1T5QS1HGZ87lO+d8Z8bj0GUx73b/vXHzrbffefe9W+/rjQ8+vP3R1p2PT+MgiWx6YgduEJ2NSUxd5tMTzrhLz8KIEm/s0ufj+QO1//wljWIW+Mf8IqQDj0x95jCbcBSN7ui3m5ZH+MwmrngoYacPlgAjFY0dsZBDYTBTtqCQXOQSEyw5Eqzfw/cDtW9BaQQ7FvPz97F4hioPUdeXNT+wQSdIuJR60/l2gy1YEZvOOImi4NUVdtaMcOFIw7InAb8HFp9RTkxEQ8IwChbgZDsmwuB0wUXc4Z0Uu96EpW0l9btys5OaitmqZD68C5Yf8CKzrKoITMDZiENaMw7WlL6ArqyhAEh1BB/1cCNd7CkNhTfOBXOlhXWImVe6Pyghe5X6PpEmhvUSjNpfCn+Q5whjoLrsGWkZMHM7eGmgNIuxi43F3RWLfGsA7TVXtfdMqQ9zA93xXbODENLyqTopHIk/QS5SLjKxFKl55AkSTb2syU5EbNGTmBcmmnglx8qUn5YpX5QEzfq3zt1lC034AiwXR2NCSl/P0Fe2rTcLUmEujkJ8WW90q9rTvIgrVMAIc2OjFb6fqaFxqcMNqLRJgYrZ1CMj4aPBnlzu/VRQ3hyKdq+SGc5OuxIb/eatLBI4vW4Cbw7saz0Pj3HGcaJqRP7xUBoOXNaKinnNjUULFrv3SsXHy3wdHLfLalIbfWZKnLkTiiNS956L53JjlI7erLyPlmzd/+5QrvC5VNtAajU3bjCFsNLCy7IerVpUjHltXcTQXk5MpRs4FscgrPR6EBGdyKKXZSvzbFKn5QqZdb0GV2Op9CogslyLIkwo/1+jL4MdZBmrWHtWyAr2pzTtw9HR8Lg4YOHVjEa0BRKO1i6dA0UBj8bwQl1IHl65uM7MHpGYA/Pz2xMCB6Rl5Zko98thUEgV4k1ocwRJzPwp7BMesUXF51PqeaSjLgjm89rMVC+VXXQTzhgehTX2liWWpVSxMjvjRmyjVQsv6grtr4zYrx2/qjp1t6wOZVW0izzeiQ0r8OiUmMUdpGj7SOJNNzci9QWh5Jj5SLTVv8PVcZ4vMhWKZMOAqReIZCXlqAVYtkyr5j2lJ4Yog78+TqmHwa6MpY6bnbJsWdCC9PtyrVh2UPjFgkVmB6Ak6IOsS9nheo2huO6hqzcF1Ga+KMrP0jgYfomFgM9A6k/WhsAvhsCXo63tbqebPrC+6OWLbS1/Dkdb/1iTwE486nPbJXF83uuGfIBnH2e2S6VuJTENiT0nU3qOS59gnIFIUUpoomQCThDhDwcglVYtBPHi+MIbo6aCG6/uKeGmvfOEO98M8LMxTDj17SyQk7jAA1BfwTBhEbW5e4ELYkcMsYI9I9hOjpOpYxF6qymvL073Or1up3f01fb97/Ny3NI+1T7XDK2nfa3d1x5rh9qJZuuJ/qv+u/5H45fGb40/G39lqjdv5DafaLWn8fd/YvIqUw==</latexit> <latexit sha1_base64="PIFlW0otVBg45kxOOaeFPjrHsfM=">AAAL+HictVZbj9tEFHbLrThQtvDIy4FVdu2SRMmCBKKKVGgpRe0uu9XudqV1Ek2ccTKKb7XHbZbZ+SW88ABCvPJTeOPfcMaX2E6y1T5QS1HGZ87lO+d8Z8bj0GUx73b/vXHzrbffefe9W+/rjQ8+vP3R1p2PT+MgiWx6YgduEJ2NSUxd5tMTzrhLz8KIEm/s0ufj+QO1//wljWIW+Mf8IqQDj0x95jCbcBSN7ui3m5ZH+MwmrngoYacPlgAjFY0dsZBDYTBTtqCQXOQSEyw5Eqzfw/cDtW9BaQQ7FvPz97F4hioPUdeXNT+wQSdIuJR60/l2gy1YEZvOOImi4NUVdtaMcOFIw7InAb8HFp9RTkxEQ8IwChbgZDsmwuB0wUXc4Z0Uu96EpW0l9btys5OaitmqZD68C5Yf8CKzrKoITMDZiENaMw7WlL6ArqyhAEh1BB/1cCNd7CkNhTfOBXOlhXWImVe6Pyghe5X6PpEmhvUSjNpfCn+Q5whjoLrsGWkZMHM7eGmgNIuxi43F3RWLfGsA7TVXtfdMqQ9zA93xXbODENLyqTopHIk/QS5SLjKxFKl55AkSTb2syU5EbNGTmBcmmnglx8qUn5YpX5QEzfq3zt1lC034AiwXR2NCSl/P0Fe2rTcLUmEujkJ8WW90q9rTvIgrVMAIc2OjFb6fqaFxqcMNqLRJgYrZ1CMj4aPBnlzu/VRQ3hyKdq+SGc5OuxIb/eatLBI4vW4Cbw7saz0Pj3HGcaJqRP7xUBoOXNaKinnNjUULFrv3SsXHy3wdHLfLalIbfWZKnLkTiiNS956L53JjlI7erLyPlmzd/+5QrvC5VNtAajU3bjCFsNLCy7IerVpUjHltXcTQXk5MpRs4FscgrPR6EBGdyKKXZSvzbFKn5QqZdb0GV2Op9CogslyLIkwo/1+jL4MdZBmrWHtWyAr2pzTtw9HR8Lg4YOHVjEa0BRKO1i6dA0UBj8bwQl1IHl65uM7MHpGYA/Pz2xMCB6Rl5Zko98thUEgV4k1ocwRJzPwp7BMesUXF51PqeaSjLgjm89rMVC+VXXQTzhgehTX2liWWpVSxMjvjRmyjVQsv6grtr4zYrx2/qjp1t6wOZVW0izzeiQ0r8OiUmMUdpGj7SOJNNzci9QWh5Jj5SLTVv8PVcZ4vMhWKZMOAqReIZCXlqAVYtkyr5j2lJ4Yog78+TqmHwa6MpY6bnbJsWdCC9PtyrVh2UPjFgkVmB6Ak6IOsS9nheo2huO6hqzcF1Ga+KMrP0jgYfomFgM9A6k/WhsAvhsCXo63tbqebPrC+6OWLbS1/Dkdb/1iTwE486nPbJXF83uuGfIBnH2e2S6VuJTENiT0nU3qOS59gnIFIUUpoomQCThDhDwcglVYtBPHi+MIbo6aCG6/uKeGmvfOEO98M8LMxTDj17SyQk7jAA1BfwTBhEbW5e4ELYkcMsYI9I9hOjpOpYxF6qymvL073Or1up3f01fb97/Ny3NI+1T7XDK2nfa3d1x5rh9qJZuuJ/qv+u/5H45fGb40/G39lqjdv5DafaLWn8fd/YvIqUw==</latexit> <latexit sha1_base64="PIFlW0otVBg45kxOOaeFPjrHsfM=">AAAL+HictVZbj9tEFHbLrThQtvDIy4FVdu2SRMmCBKKKVGgpRe0uu9XudqV1Ek2ccTKKb7XHbZbZ+SW88ABCvPJTeOPfcMaX2E6y1T5QS1HGZ87lO+d8Z8bj0GUx73b/vXHzrbffefe9W+/rjQ8+vP3R1p2PT+MgiWx6YgduEJ2NSUxd5tMTzrhLz8KIEm/s0ufj+QO1//wljWIW+Mf8IqQDj0x95jCbcBSN7ui3m5ZH+MwmrngoYacPlgAjFY0dsZBDYTBTtqCQXOQSEyw5Eqzfw/cDtW9BaQQ7FvPz97F4hioPUdeXNT+wQSdIuJR60/l2gy1YEZvOOImi4NUVdtaMcOFIw7InAb8HFp9RTkxEQ8IwChbgZDsmwuB0wUXc4Z0Uu96EpW0l9btys5OaitmqZD68C5Yf8CKzrKoITMDZiENaMw7WlL6ArqyhAEh1BB/1cCNd7CkNhTfOBXOlhXWImVe6Pyghe5X6PpEmhvUSjNpfCn+Q5whjoLrsGWkZMHM7eGmgNIuxi43F3RWLfGsA7TVXtfdMqQ9zA93xXbODENLyqTopHIk/QS5SLjKxFKl55AkSTb2syU5EbNGTmBcmmnglx8qUn5YpX5QEzfq3zt1lC034AiwXR2NCSl/P0Fe2rTcLUmEujkJ8WW90q9rTvIgrVMAIc2OjFb6fqaFxqcMNqLRJgYrZ1CMj4aPBnlzu/VRQ3hyKdq+SGc5OuxIb/eatLBI4vW4Cbw7saz0Pj3HGcaJqRP7xUBoOXNaKinnNjUULFrv3SsXHy3wdHLfLalIbfWZKnLkTiiNS956L53JjlI7erLyPlmzd/+5QrvC5VNtAajU3bjCFsNLCy7IerVpUjHltXcTQXk5MpRs4FscgrPR6EBGdyKKXZSvzbFKn5QqZdb0GV2Op9CogslyLIkwo/1+jL4MdZBmrWHtWyAr2pzTtw9HR8Lg4YOHVjEa0BRKO1i6dA0UBj8bwQl1IHl65uM7MHpGYA/Pz2xMCB6Rl5Zko98thUEgV4k1ocwRJzPwp7BMesUXF51PqeaSjLgjm89rMVC+VXXQTzhgehTX2liWWpVSxMjvjRmyjVQsv6grtr4zYrx2/qjp1t6wOZVW0izzeiQ0r8OiUmMUdpGj7SOJNNzci9QWh5Jj5SLTVv8PVcZ4vMhWKZMOAqReIZCXlqAVYtkyr5j2lJ4Yog78+TqmHwa6MpY6bnbJsWdCC9PtyrVh2UPjFgkVmB6Ak6IOsS9nheo2huO6hqzcF1Ga+KMrP0jgYfomFgM9A6k/WhsAvhsCXo63tbqebPrC+6OWLbS1/Dkdb/1iTwE486nPbJXF83uuGfIBnH2e2S6VuJTENiT0nU3qOS59gnIFIUUpoomQCThDhDwcglVYtBPHi+MIbo6aCG6/uKeGmvfOEO98M8LMxTDj17SyQk7jAA1BfwTBhEbW5e4ELYkcMsYI9I9hOjpOpYxF6qymvL073Or1up3f01fb97/Ny3NI+1T7XDK2nfa3d1x5rh9qJZuuJ/qv+u/5H45fGb40/G39lqjdv5DafaLWn8fd/YvIqUw==</latexit> Real, symmetric matrix: K = " K(1) 1 U(1) 1 V (1) 1 T V (1) 1 U(1) 1 T K(1) 2 # <latexit sha1_base64="XBDKBKdotevTutkFVnRlS3sMIGs=">AAAMeHictVZbbxtFFN6WW9kESOGxLweC493iWF6DBKKyVGgpRWlCUtlppKxtjdez9sh7cXfHraPJ/AV+HG/8EF544sxevLu2U+UBLFmeOXNu3znfmfFo7rGYt1p/3bn73vsffPjRvY/1nd1PPv1s7/7n53G4iBzac0IvjC5GJKYeC2iPM+7Ri3lEiT/y6KvR7Ik6f/WGRjELgy6/mtO+TyYBc5lDOIqG9/U/arZP+NQhnngq4aADtgAjEY1csZQDYTBTNiCXXGUSE2w5FKxj4f5EndtQGMGBzYJsPxIvUeUp6gay4ge26IQLLqVec3/cYgt2xCZTTqIofHuDnT0lXLjSsJ1xyB+BzaeUExOzIfN5FC7BTU9MTIPTJRdxkzeT3PUarGxL0B/K7U4qKmajhHzwEOwg5DmytKqYmICLIYekZhzsCX0NLVnJAiDREXxo4UGyaCsNlW+cCWZKC+sQM79wf1Kk7JfqeyRNDOsvMGpnJfxFXmIafdVl30jKgMid8I2B0jRGHRuLp2sW2VEfDjdcVfapUgdmBrrjdbOJKSTlU3VSeSyCMXKRcpGKpUjMI1+QaOKnTXYj4ghLIi4EuvALjhWQXxSQrwqCpv3b5O6qhSZ8A7aHozEmha+X6Cs91ms5qRCLqzK+rja6Ue5pVsQ1KmCEmbHVCvcXamg86nIDSm1SScVs4pOhCNCgLVdnv+WUNwfi0Cohw9k5LMVGv1krcwDntwXw/yX7Ts+DLs44TlSFyL+eSsOF60pREdfMWDZgWX9UKD5f4XVx3K7LoLb6TJU488YUR6TqPRPP5NYoTb1W2g9XbD3+6VSu8blQ20JqNTdeOIF5qYXXRT0alagY89a6mMPhamJK3cCx6IKwk+dBRHQs814WrczQJE6LFTLrdg0ux1LwSkmkWPMijCn/T6Ovgp2kiFWstj1nOfsTmnbg7GzQzS9YeDulEW2AhLONR+dEUcCnMbxWD5KPTy6uU7NnJObAguz1hNAFadsZEuV+NQwqU5XxtmyzDBYxCyZwTHjEliWfL6jvk6Z6IFjAKzNTflTq6GY+ZXgVVthblFgWUsXK9I4bsq1WDXyoS7S/MWKncv2q6lTdsmoq66I68vggNuzQpxNi5m+Qou0ziS/dzIjUPwglR+RDcah+Xa6u82yRqlAkGwZMvEAkS5CjBmDZUq2K94SeGKII/u44hR4GuzGWum4OirKlQXPSH8uNYjlh7hcLFplNgIKgT9IupZfrLYbitpeuXhNQmfm8KL9L42TwLRYCvgSpQ+1oYwyCfAwCqR9hdro9ohMWiJGfUFaCfjQUCpthmXjLQq+0E+fFBn+6UhXrvKzQKyt00RydtfNT3abBeBVouLffaraSD2wurGyxr2Wf0+Hen/Y4dBY+DbjjkTi+tFpz3se7lzPHo1K3FzGdE2dGJvQSlwFBlH2RVElCDSVjcMMIvziAibRsIYgfx1f+CDVVseL1MyXcdna54O4PffzbOl9wGjhpIHfhAQ9B/QuHMYuow70rXBAnYpgrOFOCdOJ4M+hYBGsd8ubivN20Wk3r7Lv9xz9n5binPdC+0gzN0r7XHmvPtVOtpzn63zsPdr7eqe38swu79V0zVb17J7P5Qqt8dtv/AsQ6Ugg=</latexit> <latexit sha1_base64="XBDKBKdotevTutkFVnRlS3sMIGs=">AAAMeHictVZbbxtFFN6WW9kESOGxLweC493iWF6DBKKyVGgpRWlCUtlppKxtjdez9sh7cXfHraPJ/AV+HG/8EF544sxevLu2U+UBLFmeOXNu3znfmfFo7rGYt1p/3bn73vsffPjRvY/1nd1PPv1s7/7n53G4iBzac0IvjC5GJKYeC2iPM+7Ri3lEiT/y6KvR7Ik6f/WGRjELgy6/mtO+TyYBc5lDOIqG9/U/arZP+NQhnngq4aADtgAjEY1csZQDYTBTNiCXXGUSE2w5FKxj4f5EndtQGMGBzYJsPxIvUeUp6gay4ge26IQLLqVec3/cYgt2xCZTTqIofHuDnT0lXLjSsJ1xyB+BzaeUExOzIfN5FC7BTU9MTIPTJRdxkzeT3PUarGxL0B/K7U4qKmajhHzwEOwg5DmytKqYmICLIYekZhzsCX0NLVnJAiDREXxo4UGyaCsNlW+cCWZKC+sQM79wf1Kk7JfqeyRNDOsvMGpnJfxFXmIafdVl30jKgMid8I2B0jRGHRuLp2sW2VEfDjdcVfapUgdmBrrjdbOJKSTlU3VSeSyCMXKRcpGKpUjMI1+QaOKnTXYj4ghLIi4EuvALjhWQXxSQrwqCpv3b5O6qhSZ8A7aHozEmha+X6Cs91ms5qRCLqzK+rja6Ue5pVsQ1KmCEmbHVCvcXamg86nIDSm1SScVs4pOhCNCgLVdnv+WUNwfi0Cohw9k5LMVGv1krcwDntwXw/yX7Ts+DLs44TlSFyL+eSsOF60pREdfMWDZgWX9UKD5f4XVx3K7LoLb6TJU488YUR6TqPRPP5NYoTb1W2g9XbD3+6VSu8blQ20JqNTdeOIF5qYXXRT0alagY89a6mMPhamJK3cCx6IKwk+dBRHQs814WrczQJE6LFTLrdg0ux1LwSkmkWPMijCn/T6Ovgp2kiFWstj1nOfsTmnbg7GzQzS9YeDulEW2AhLONR+dEUcCnMbxWD5KPTy6uU7NnJObAguz1hNAFadsZEuV+NQwqU5XxtmyzDBYxCyZwTHjEliWfL6jvk6Z6IFjAKzNTflTq6GY+ZXgVVthblFgWUsXK9I4bsq1WDXyoS7S/MWKncv2q6lTdsmoq66I68vggNuzQpxNi5m+Qou0ziS/dzIjUPwglR+RDcah+Xa6u82yRqlAkGwZMvEAkS5CjBmDZUq2K94SeGKII/u44hR4GuzGWum4OirKlQXPSH8uNYjlh7hcLFplNgIKgT9IupZfrLYbitpeuXhNQmfm8KL9L42TwLRYCvgSpQ+1oYwyCfAwCqR9hdro9ohMWiJGfUFaCfjQUCpthmXjLQq+0E+fFBn+6UhXrvKzQKyt00RydtfNT3abBeBVouLffaraSD2wurGyxr2Wf0+Hen/Y4dBY+DbjjkTi+tFpz3se7lzPHo1K3FzGdE2dGJvQSlwFBlH2RVElCDSVjcMMIvziAibRsIYgfx1f+CDVVseL1MyXcdna54O4PffzbOl9wGjhpIHfhAQ9B/QuHMYuow70rXBAnYpgrOFOCdOJ4M+hYBGsd8ubivN20Wk3r7Lv9xz9n5binPdC+0gzN0r7XHmvPtVOtpzn63zsPdr7eqe38swu79V0zVb17J7P5Qqt8dtv/AsQ6Ugg=</latexit> <latexit sha1_base64="XBDKBKdotevTutkFVnRlS3sMIGs=">AAAMeHictVZbbxtFFN6WW9kESOGxLweC493iWF6DBKKyVGgpRWlCUtlppKxtjdez9sh7cXfHraPJ/AV+HG/8EF544sxevLu2U+UBLFmeOXNu3znfmfFo7rGYt1p/3bn73vsffPjRvY/1nd1PPv1s7/7n53G4iBzac0IvjC5GJKYeC2iPM+7Ri3lEiT/y6KvR7Ik6f/WGRjELgy6/mtO+TyYBc5lDOIqG9/U/arZP+NQhnngq4aADtgAjEY1csZQDYTBTNiCXXGUSE2w5FKxj4f5EndtQGMGBzYJsPxIvUeUp6gay4ge26IQLLqVec3/cYgt2xCZTTqIofHuDnT0lXLjSsJ1xyB+BzaeUExOzIfN5FC7BTU9MTIPTJRdxkzeT3PUarGxL0B/K7U4qKmajhHzwEOwg5DmytKqYmICLIYekZhzsCX0NLVnJAiDREXxo4UGyaCsNlW+cCWZKC+sQM79wf1Kk7JfqeyRNDOsvMGpnJfxFXmIafdVl30jKgMid8I2B0jRGHRuLp2sW2VEfDjdcVfapUgdmBrrjdbOJKSTlU3VSeSyCMXKRcpGKpUjMI1+QaOKnTXYj4ghLIi4EuvALjhWQXxSQrwqCpv3b5O6qhSZ8A7aHozEmha+X6Cs91ms5qRCLqzK+rja6Ue5pVsQ1KmCEmbHVCvcXamg86nIDSm1SScVs4pOhCNCgLVdnv+WUNwfi0Cohw9k5LMVGv1krcwDntwXw/yX7Ts+DLs44TlSFyL+eSsOF60pREdfMWDZgWX9UKD5f4XVx3K7LoLb6TJU488YUR6TqPRPP5NYoTb1W2g9XbD3+6VSu8blQ20JqNTdeOIF5qYXXRT0alagY89a6mMPhamJK3cCx6IKwk+dBRHQs814WrczQJE6LFTLrdg0ux1LwSkmkWPMijCn/T6Ovgp2kiFWstj1nOfsTmnbg7GzQzS9YeDulEW2AhLONR+dEUcCnMbxWD5KPTy6uU7NnJObAguz1hNAFadsZEuV+NQwqU5XxtmyzDBYxCyZwTHjEliWfL6jvk6Z6IFjAKzNTflTq6GY+ZXgVVthblFgWUsXK9I4bsq1WDXyoS7S/MWKncv2q6lTdsmoq66I68vggNuzQpxNi5m+Qou0ziS/dzIjUPwglR+RDcah+Xa6u82yRqlAkGwZMvEAkS5CjBmDZUq2K94SeGKII/u44hR4GuzGWum4OirKlQXPSH8uNYjlh7hcLFplNgIKgT9IupZfrLYbitpeuXhNQmfm8KL9L42TwLRYCvgSpQ+1oYwyCfAwCqR9hdro9ohMWiJGfUFaCfjQUCpthmXjLQq+0E+fFBn+6UhXrvKzQKyt00RydtfNT3abBeBVouLffaraSD2wurGyxr2Wf0+Hen/Y4dBY+DbjjkTi+tFpz3se7lzPHo1K3FzGdE2dGJvQSlwFBlH2RVElCDSVjcMMIvziAibRsIYgfx1f+CDVVseL1MyXcdna54O4PffzbOl9wGjhpIHfhAQ9B/QuHMYuow70rXBAnYpgrOFOCdOJ4M+hYBGsd8ubivN20Wk3r7Lv9xz9n5binPdC+0gzN0r7XHmvPtVOtpzn63zsPdr7eqe38swu79V0zVb17J7P5Qqt8dtv/AsQ6Ugg=</latexit> The matrix K can be written as: K(1) 1 = " K(2) 1 U(2) 1 V (2) 1 T V (2) 1 U(2) 1 T K(2) 2 # , K(1) 2 = " K(2) 3 U(2) 2 V (2) 2 T V (2) 2 U(2) 2 T K(2) 4 # . <latexit sha1_base64="ANKc6ZuCSx9CJgBqBOqrwL6RPI8=">AAANkHictVdbbxpHFCbpLSWmddLHvpzWwl5SjGAdqVFT1KRJ07SOXTsCYskLaFhmYcReyO6Q2BrP3+kP6lv/Tc/shZ0FnPqhRbKYOXMu3znnOzN4NHdZxJvNv2/d/ujjTz797M7n5btblS++3L53vxcFi9CmXTtwg/BsRCLqMp92OeMuPZuHlHgjl74ZzZ6p8zfvaBixwO/wyznte2TiM4fZhKNoeO/un1XLI3xqE1c8l7DbBkuAEYtGjriQA2GwmqxDJrlMJTWw5FCwdgv3x+rcgtwIdi3mp/uReI0qz1HXlwU/sEEnWHApy1Xnhw22YIVsMuUkDIP319hZU8KFIw3LHgf8MVh8SjmpIRoyn4fBBTjJSQ1hcHrBRdTgjRh7uQpLWy31B3Kzk4JKra5lPngAlh/wLLOkqghMwNmQQ1wzDtaEvoWmLKAAiHUEH7bwIF6YSkPhjVLBTGlhHSLm5e6Pc8ieVt9DWcOw3gKjtpfCX+Q5wuirLntGXAbM3A7eGShNYuxhY/F0xSI96sP+mqvCPlFqw8xAd3yv1kAIcflUnRSOhT9GLlIuErEUsXnoCRJOvKTJTkhs0ZKYFya68HKO5Sm/ylO+zAma9G+du8sW1uA7sFwcjTHJfb1GX8lxuZqRCnNxFOKrYqPrek/TIq5QASPMjI1WuD9TQ+NShxugtUmBitjEI0Pho4Epl2e/ZZSvDcR+S8sMZ2dfi41+01ZmCfRumsD/B/aDngcdnHGcqAKRfz2RhgNXhaJiXjPjog4Xe49zxZfLfB0ctys9qY0+EyXO3DHFESl6T8UzuTFKo1zV9sMlW4+ensgVPudqG0it5sYNJjDXWniV16NeiIoxb6yLGPaXE6N1A8eiA8KKnwcR0rHMepm3Ms0mdpqvkFk3a7AeS6WngUhyzYowpvw/jb4MdpxkrGKZ1pxl7I9p2obT00Enu2Dh/ZSGtA4STtcenWNFAY9G8FY9SB4+ubhOzF6QiAPz09cTAgekZaWZKPfLYVBIFeJNaFMEi4j5EzgiPGQXms9X1PNIQz0QzOeFmdEflT10M58yvAoL7M1LLHOpYmVyxw3ZRqs6PtQa7a+N2C5cv6o6RbesCGVVtIc83o0MK/DohNSyN0jR9oXEl25mhOoXhJJj5kOxr74drq7zdJGoUCQbBoy9QCi1lMM6YNkSrYL3mJ4YIg/+4Ti5Hga7Npa6bnbzsiVBM9IfybVi2UHmFwsW1hoAOUGfJV1KLtcbDMVNL91yVUBh5rOi/CGN48EBFgK+AVmG6uHaGPjZGPjo5RDhIdoRnTBfjLyYtBIlh0Oh0jNaNbxooavtRC/f4FdHxvXq6RpdXaOD9ujNzE4xGvXHy1hlPRJiWYOSnZsFJKaOxNSQ9HSFrq6gA1Gn5QKOehzJ/BckBwUkpo7E3IDE1JGYq0gebkbSGG7vNBvN+APri1a62Cmln5Ph9l/WOLAXHvW57ZIoOm8157yPbxJntktl2VpEdE7sGZnQc1z6BLvfFzF7JFRRMgYnCPEPL6ZYqlsI4kXRpTdCTUWiaPVMCTednS+486iPP+fnC059OwnkLFzgAaj/TmDMQmpz9xIXxA4ZYgV7SnDMON6YZSxCazXl9UXPbLSajdbpw50nP6fluFP6uvRtySi1St+XnpRelk5K3ZK9Vdk62Ppxq125X3lU+anyNFG9fSu1+apU+FR+/wfuw6ve</latexit> <latexit sha1_base64="ANKc6ZuCSx9CJgBqBOqrwL6RPI8=">AAANkHictVdbbxpHFCbpLSWmddLHvpzWwl5SjGAdqVFT1KRJ07SOXTsCYskLaFhmYcReyO6Q2BrP3+kP6lv/Tc/shZ0FnPqhRbKYOXMu3znnOzN4NHdZxJvNv2/d/ujjTz797M7n5btblS++3L53vxcFi9CmXTtwg/BsRCLqMp92OeMuPZuHlHgjl74ZzZ6p8zfvaBixwO/wyznte2TiM4fZhKNoeO/un1XLI3xqE1c8l7DbBkuAEYtGjriQA2GwmqxDJrlMJTWw5FCwdgv3x+rcgtwIdi3mp/uReI0qz1HXlwU/sEEnWHApy1Xnhw22YIVsMuUkDIP319hZU8KFIw3LHgf8MVh8SjmpIRoyn4fBBTjJSQ1hcHrBRdTgjRh7uQpLWy31B3Kzk4JKra5lPngAlh/wLLOkqghMwNmQQ1wzDtaEvoWmLKAAiHUEH7bwIF6YSkPhjVLBTGlhHSLm5e6Pc8ieVt9DWcOw3gKjtpfCX+Q5wuirLntGXAbM3A7eGShNYuxhY/F0xSI96sP+mqvCPlFqw8xAd3yv1kAIcflUnRSOhT9GLlIuErEUsXnoCRJOvKTJTkhs0ZKYFya68HKO5Sm/ylO+zAma9G+du8sW1uA7sFwcjTHJfb1GX8lxuZqRCnNxFOKrYqPrek/TIq5QASPMjI1WuD9TQ+NShxugtUmBitjEI0Pho4Epl2e/ZZSvDcR+S8sMZ2dfi41+01ZmCfRumsD/B/aDngcdnHGcqAKRfz2RhgNXhaJiXjPjog4Xe49zxZfLfB0ctys9qY0+EyXO3DHFESl6T8UzuTFKo1zV9sMlW4+ensgVPudqG0it5sYNJjDXWniV16NeiIoxb6yLGPaXE6N1A8eiA8KKnwcR0rHMepm3Ms0mdpqvkFk3a7AeS6WngUhyzYowpvw/jb4MdpxkrGKZ1pxl7I9p2obT00Enu2Dh/ZSGtA4STtcenWNFAY9G8FY9SB4+ubhOzF6QiAPz09cTAgekZaWZKPfLYVBIFeJNaFMEi4j5EzgiPGQXms9X1PNIQz0QzOeFmdEflT10M58yvAoL7M1LLHOpYmVyxw3ZRqs6PtQa7a+N2C5cv6o6RbesCGVVtIc83o0MK/DohNSyN0jR9oXEl25mhOoXhJJj5kOxr74drq7zdJGoUCQbBoy9QCi1lMM6YNkSrYL3mJ4YIg/+4Ti5Hga7Npa6bnbzsiVBM9IfybVi2UHmFwsW1hoAOUGfJV1KLtcbDMVNL91yVUBh5rOi/CGN48EBFgK+AVmG6uHaGPjZGPjo5RDhIdoRnTBfjLyYtBIlh0Oh0jNaNbxooavtRC/f4FdHxvXq6RpdXaOD9ujNzE4xGvXHy1hlPRJiWYOSnZsFJKaOxNSQ9HSFrq6gA1Gn5QKOehzJ/BckBwUkpo7E3IDE1JGYq0gebkbSGG7vNBvN+APri1a62Cmln5Ph9l/WOLAXHvW57ZIoOm8157yPbxJntktl2VpEdE7sGZnQc1z6BLvfFzF7JFRRMgYnCPEPL6ZYqlsI4kXRpTdCTUWiaPVMCTednS+486iPP+fnC059OwnkLFzgAaj/TmDMQmpz9xIXxA4ZYgV7SnDMON6YZSxCazXl9UXPbLSajdbpw50nP6fluFP6uvRtySi1St+XnpRelk5K3ZK9Vdk62Ppxq125X3lU+anyNFG9fSu1+apU+FR+/wfuw6ve</latexit> <latexit sha1_base64="ANKc6ZuCSx9CJgBqBOqrwL6RPI8=">AAANkHictVdbbxpHFCbpLSWmddLHvpzWwl5SjGAdqVFT1KRJ07SOXTsCYskLaFhmYcReyO6Q2BrP3+kP6lv/Tc/shZ0FnPqhRbKYOXMu3znnOzN4NHdZxJvNv2/d/ujjTz797M7n5btblS++3L53vxcFi9CmXTtwg/BsRCLqMp92OeMuPZuHlHgjl74ZzZ6p8zfvaBixwO/wyznte2TiM4fZhKNoeO/un1XLI3xqE1c8l7DbBkuAEYtGjriQA2GwmqxDJrlMJTWw5FCwdgv3x+rcgtwIdi3mp/uReI0qz1HXlwU/sEEnWHApy1Xnhw22YIVsMuUkDIP319hZU8KFIw3LHgf8MVh8SjmpIRoyn4fBBTjJSQ1hcHrBRdTgjRh7uQpLWy31B3Kzk4JKra5lPngAlh/wLLOkqghMwNmQQ1wzDtaEvoWmLKAAiHUEH7bwIF6YSkPhjVLBTGlhHSLm5e6Pc8ieVt9DWcOw3gKjtpfCX+Q5wuirLntGXAbM3A7eGShNYuxhY/F0xSI96sP+mqvCPlFqw8xAd3yv1kAIcflUnRSOhT9GLlIuErEUsXnoCRJOvKTJTkhs0ZKYFya68HKO5Sm/ylO+zAma9G+du8sW1uA7sFwcjTHJfb1GX8lxuZqRCnNxFOKrYqPrek/TIq5QASPMjI1WuD9TQ+NShxugtUmBitjEI0Pho4Epl2e/ZZSvDcR+S8sMZ2dfi41+01ZmCfRumsD/B/aDngcdnHGcqAKRfz2RhgNXhaJiXjPjog4Xe49zxZfLfB0ctys9qY0+EyXO3DHFESl6T8UzuTFKo1zV9sMlW4+ensgVPudqG0it5sYNJjDXWniV16NeiIoxb6yLGPaXE6N1A8eiA8KKnwcR0rHMepm3Ms0mdpqvkFk3a7AeS6WngUhyzYowpvw/jb4MdpxkrGKZ1pxl7I9p2obT00Enu2Dh/ZSGtA4STtcenWNFAY9G8FY9SB4+ubhOzF6QiAPz09cTAgekZaWZKPfLYVBIFeJNaFMEi4j5EzgiPGQXms9X1PNIQz0QzOeFmdEflT10M58yvAoL7M1LLHOpYmVyxw3ZRqs6PtQa7a+N2C5cv6o6RbesCGVVtIc83o0MK/DohNSyN0jR9oXEl25mhOoXhJJj5kOxr74drq7zdJGoUCQbBoy9QCi1lMM6YNkSrYL3mJ4YIg/+4Ti5Hga7Npa6bnbzsiVBM9IfybVi2UHmFwsW1hoAOUGfJV1KLtcbDMVNL91yVUBh5rOi/CGN48EBFgK+AVmG6uHaGPjZGPjo5RDhIdoRnTBfjLyYtBIlh0Oh0jNaNbxooavtRC/f4FdHxvXq6RpdXaOD9ujNzE4xGvXHy1hlPRJiWYOSnZsFJKaOxNSQ9HSFrq6gA1Gn5QKOehzJ/BckBwUkpo7E3IDE1JGYq0gebkbSGG7vNBvN+APri1a62Cmln5Ph9l/WOLAXHvW57ZIoOm8157yPbxJntktl2VpEdE7sGZnQc1z6BLvfFzF7JFRRMgYnCPEPL6ZYqlsI4kXRpTdCTUWiaPVMCTednS+486iPP+fnC059OwnkLFzgAaj/TmDMQmpz9xIXxA4ZYgV7SnDMON6YZSxCazXl9UXPbLSajdbpw50nP6fluFP6uvRtySi1St+XnpRelk5K3ZK9Vdk62Ppxq125X3lU+anyNFG9fSu1+apU+FR+/wfuw6ve</latexit> 1st level 2nd level where, U(j) i , V (j) i 2 R n 2j ⇥r, are tall-and-thin matrices. <latexit sha1_base64="DJsmoxywTHeqOhXlSY4QLFDOkGs=">AAAOD3ictVdbbxtFFHbLrZgYUnjk5UDkZF1sy95WAlFZKrSUojQhqeI0Uja2xutZe+q9uLvj1tFk/gEv/BVeeAAhXnnljX/Dmb14Z20nzQNYijJz5ly+c853ZuzB1GURb7X+uXHzrbffefe9W++XP9iofPjR5u2Pj6NgFtq0awduEJ4MSERd5tMuZ9ylJ9OQEm/g0ueDyUN1/vwVDSMW+Ef8fErPPDLymcNswlHUv71RrVoe4WObuOKRhO0OWAKMWDRwxFz2hMFqsg6Z5DyV1MCSfcE6bdzvq3MLciPYtpif7gfiGao8Ql1fFvzAGp1gxqUsV52v19iCFbLRmJMwDF5fYmeNCReONCx7GPD7YPEx5aSGaMh0GgZzcJKTGsLgdM5F1OTNGHu5CgtbLfU7cr2TgkqtrmXeuwOWH/Ass6SqCEzASZ9DXDMO1oi+hJYsoACIdQTvt/EgXphKQ+GNUsFEaWEdIubl7vdzyJ5W311Zw7DeDKN2FsLv5CnCOFNd9oy4DJi5HbwyUJrE2MHG4umSRXp0Bo0VV4V9otSBiYHu+E6tiRDi8qk6KRwzf4hcpFwkYili89ATJBx5SZOdkNiiLTEvTHTm5RzLU36ap3yeEzTp3yp3Fy2swRdguTgaQ5L7eoa+kuNyNSMV5uIoxBfFRtf1nqZFXKICRpgYa61wf6KGxqUON0BrkwIVsZFH+sJHA1Muzn7IKF/riUZbywxnp6HFRr9pK7MEjq+bwP8H9krPvSOccZyoApG/P5CGAxeFomJeE2Neh/nO/VzxySJfB8ftQk9qrc9EiTN3SHFEit5T8USujdIsV7V9f8HWvW8O5BKfc7U1pFZz4wYjmGotvMjrUS9ExZjX1kUMjcXEaN3AsTgCYcXPgwjpUGa9zFuZZhM7zVfIrOs1WI+l0tNAJLlmRRhS/p9GXwTbTzJWsUxryjL2xzTtwOFh7yi7YOH1mIa0DhIOVx6dfUUBj0bwUj1IHj65uE7MHpOIA/PT1xMCB6RlpZko94thUEgV4nVoUwSziPkj2CM8ZHPN51PqeaSpHgjm88LM6I/KDrqZjhlehQX25iWWuVSxMrnj+mytVR0fao32l0bsFK5fVZ2iW1aEsizaQR5vR4YVeHREatkbpGj7WOJLNzFC9Q1CyTHzvmio/w5X13m6SFQokg0Dxl4glFrKYR2wbIlWwXtMTwyRB786Tq6HwS6Npa6b7bxsSdCM9HtypVh2kPnFgoW1JkBO0IdJl5LL9RpDcd1Lt1wVUJj5rCg/SmO/dxcLAZ+BLEN1d2UM/GwMfPSyi/AQ7YCOmC8GXkxaiZLdvlDpGe0aXrTQ1XbiON/gvyMZ1+tY1+jqGkdoj97M7BSjUX+4iFUMdTUYswDG1MGY68CYOhhzGYy5CqaeBDPfCOduAY6pwzHXwTF1OOYynHuXwGmWkytlcad11YSj7gv1Leg436z0OKGrj9dm74XM+q14HvurAwkpcOK6DeIPG3yM1nFIm0ZN2d/cajVb8QdWF+10sVVKPwf9zb+tYWDPPOpz2yVRdNpuTfkZPomc2S6VZWsW0SmxJ2RET3HpEwRzJmLySqiiZAhOEOIf3ouxVLcQxIuic2+Amiq/aPlMCdednc6489UZ/pqYzjj17SSQM3OBB6B+HMGQhdTm7jkuiB0yxAr2mGDZOF7YZSxCeznl1cWx2Wy3mu3De1sPvk3Lcav0aenzklFql74sPSg9KR2UuiV746eNXzZ+2/i98nPl18oflT8T1Zs3UptPSoVP5a9/ATs93Fc=</latexit> <latexit sha1_base64="DJsmoxywTHeqOhXlSY4QLFDOkGs=">AAAOD3ictVdbbxtFFHbLrZgYUnjk5UDkZF1sy95WAlFZKrSUojQhqeI0Uja2xutZe+q9uLvj1tFk/gEv/BVeeAAhXnnljX/Dmb14Z20nzQNYijJz5ly+c853ZuzB1GURb7X+uXHzrbffefe9W++XP9iofPjR5u2Pj6NgFtq0awduEJ4MSERd5tMuZ9ylJ9OQEm/g0ueDyUN1/vwVDSMW+Ef8fErPPDLymcNswlHUv71RrVoe4WObuOKRhO0OWAKMWDRwxFz2hMFqsg6Z5DyV1MCSfcE6bdzvq3MLciPYtpif7gfiGao8Ql1fFvzAGp1gxqUsV52v19iCFbLRmJMwDF5fYmeNCReONCx7GPD7YPEx5aSGaMh0GgZzcJKTGsLgdM5F1OTNGHu5CgtbLfU7cr2TgkqtrmXeuwOWH/Ass6SqCEzASZ9DXDMO1oi+hJYsoACIdQTvt/EgXphKQ+GNUsFEaWEdIubl7vdzyJ5W311Zw7DeDKN2FsLv5CnCOFNd9oy4DJi5HbwyUJrE2MHG4umSRXp0Bo0VV4V9otSBiYHu+E6tiRDi8qk6KRwzf4hcpFwkYili89ATJBx5SZOdkNiiLTEvTHTm5RzLU36ap3yeEzTp3yp3Fy2swRdguTgaQ5L7eoa+kuNyNSMV5uIoxBfFRtf1nqZFXKICRpgYa61wf6KGxqUON0BrkwIVsZFH+sJHA1Muzn7IKF/riUZbywxnp6HFRr9pK7MEjq+bwP8H9krPvSOccZyoApG/P5CGAxeFomJeE2Neh/nO/VzxySJfB8ftQk9qrc9EiTN3SHFEit5T8USujdIsV7V9f8HWvW8O5BKfc7U1pFZz4wYjmGotvMjrUS9ExZjX1kUMjcXEaN3AsTgCYcXPgwjpUGa9zFuZZhM7zVfIrOs1WI+l0tNAJLlmRRhS/p9GXwTbTzJWsUxryjL2xzTtwOFh7yi7YOH1mIa0DhIOVx6dfUUBj0bwUj1IHj65uE7MHpOIA/PT1xMCB6RlpZko94thUEgV4nVoUwSziPkj2CM8ZHPN51PqeaSpHgjm88LM6I/KDrqZjhlehQX25iWWuVSxMrnj+mytVR0fao32l0bsFK5fVZ2iW1aEsizaQR5vR4YVeHREatkbpGj7WOJLNzFC9Q1CyTHzvmio/w5X13m6SFQokg0Dxl4glFrKYR2wbIlWwXtMTwyRB786Tq6HwS6Npa6b7bxsSdCM9HtypVh2kPnFgoW1JkBO0IdJl5LL9RpDcd1Lt1wVUJj5rCg/SmO/dxcLAZ+BLEN1d2UM/GwMfPSyi/AQ7YCOmC8GXkxaiZLdvlDpGe0aXrTQ1XbiON/gvyMZ1+tY1+jqGkdoj97M7BSjUX+4iFUMdTUYswDG1MGY68CYOhhzGYy5CqaeBDPfCOduAY6pwzHXwTF1OOYynHuXwGmWkytlcad11YSj7gv1Leg436z0OKGrj9dm74XM+q14HvurAwkpcOK6DeIPG3yM1nFIm0ZN2d/cajVb8QdWF+10sVVKPwf9zb+tYWDPPOpz2yVRdNpuTfkZPomc2S6VZWsW0SmxJ2RET3HpEwRzJmLySqiiZAhOEOIf3ouxVLcQxIuic2+Amiq/aPlMCdednc6489UZ/pqYzjj17SSQM3OBB6B+HMGQhdTm7jkuiB0yxAr2mGDZOF7YZSxCeznl1cWx2Wy3mu3De1sPvk3Lcav0aenzklFql74sPSg9KR2UuiV746eNXzZ+2/i98nPl18oflT8T1Zs3UptPSoVP5a9/ATs93Fc=</latexit> <latexit sha1_base64="DJsmoxywTHeqOhXlSY4QLFDOkGs=">AAAOD3ictVdbbxtFFHbLrZgYUnjk5UDkZF1sy95WAlFZKrSUojQhqeI0Uja2xutZe+q9uLvj1tFk/gEv/BVeeAAhXnnljX/Dmb14Z20nzQNYijJz5ly+c853ZuzB1GURb7X+uXHzrbffefe9W++XP9iofPjR5u2Pj6NgFtq0awduEJ4MSERd5tMuZ9ylJ9OQEm/g0ueDyUN1/vwVDSMW+Ef8fErPPDLymcNswlHUv71RrVoe4WObuOKRhO0OWAKMWDRwxFz2hMFqsg6Z5DyV1MCSfcE6bdzvq3MLciPYtpif7gfiGao8Ql1fFvzAGp1gxqUsV52v19iCFbLRmJMwDF5fYmeNCReONCx7GPD7YPEx5aSGaMh0GgZzcJKTGsLgdM5F1OTNGHu5CgtbLfU7cr2TgkqtrmXeuwOWH/Ass6SqCEzASZ9DXDMO1oi+hJYsoACIdQTvt/EgXphKQ+GNUsFEaWEdIubl7vdzyJ5W311Zw7DeDKN2FsLv5CnCOFNd9oy4DJi5HbwyUJrE2MHG4umSRXp0Bo0VV4V9otSBiYHu+E6tiRDi8qk6KRwzf4hcpFwkYili89ATJBx5SZOdkNiiLTEvTHTm5RzLU36ap3yeEzTp3yp3Fy2swRdguTgaQ5L7eoa+kuNyNSMV5uIoxBfFRtf1nqZFXKICRpgYa61wf6KGxqUON0BrkwIVsZFH+sJHA1Muzn7IKF/riUZbywxnp6HFRr9pK7MEjq+bwP8H9krPvSOccZyoApG/P5CGAxeFomJeE2Neh/nO/VzxySJfB8ftQk9qrc9EiTN3SHFEit5T8USujdIsV7V9f8HWvW8O5BKfc7U1pFZz4wYjmGotvMjrUS9ExZjX1kUMjcXEaN3AsTgCYcXPgwjpUGa9zFuZZhM7zVfIrOs1WI+l0tNAJLlmRRhS/p9GXwTbTzJWsUxryjL2xzTtwOFh7yi7YOH1mIa0DhIOVx6dfUUBj0bwUj1IHj65uE7MHpOIA/PT1xMCB6RlpZko94thUEgV4nVoUwSziPkj2CM8ZHPN51PqeaSpHgjm88LM6I/KDrqZjhlehQX25iWWuVSxMrnj+mytVR0fao32l0bsFK5fVZ2iW1aEsizaQR5vR4YVeHREatkbpGj7WOJLNzFC9Q1CyTHzvmio/w5X13m6SFQokg0Dxl4glFrKYR2wbIlWwXtMTwyRB786Tq6HwS6Npa6b7bxsSdCM9HtypVh2kPnFgoW1JkBO0IdJl5LL9RpDcd1Lt1wVUJj5rCg/SmO/dxcLAZ+BLEN1d2UM/GwMfPSyi/AQ7YCOmC8GXkxaiZLdvlDpGe0aXrTQ1XbiON/gvyMZ1+tY1+jqGkdoj97M7BSjUX+4iFUMdTUYswDG1MGY68CYOhhzGYy5CqaeBDPfCOduAY6pwzHXwTF1OOYynHuXwGmWkytlcad11YSj7gv1Leg436z0OKGrj9dm74XM+q14HvurAwkpcOK6DeIPG3yM1nFIm0ZN2d/cajVb8QdWF+10sVVKPwf9zb+tYWDPPOpz2yVRdNpuTfkZPomc2S6VZWsW0SmxJ2RET3HpEwRzJmLySqiiZAhOEOIf3ouxVLcQxIuic2+Amiq/aPlMCdednc6489UZ/pqYzjj17SSQM3OBB6B+HMGQhdTm7jkuiB0yxAr2mGDZOF7YZSxCeznl1cWx2Wy3mu3De1sPvk3Lcav0aenzklFql74sPSg9KR2UuiV746eNXzZ+2/i98nPl18oflT8T1Zs3UptPSoVP5a9/ATs93Fc=</latexit>
can obtain a k-level HODLR approximation of the original matrix K, we can factorize it into a product of k+1 matrices where one of them is block diagonally dense, and the rest are low-rank updates to the identity matrix (easy inversion by SMW formula). Dense block Low-rank block Zero block Identity block
to be addressed: 1. How to efficiently construct the low rank blocks? 2. How to construct the factorization? 3. How to exploit the factorization to compute the covariance matrix inverse and determinant?
matrix A, the optimal low-rank approximation is estimated from the SVD which costs O(rn2). we can do better ! HOW: 1. Approximation theoretic techniques – eigenfunction expansion, Taylor series expansion of the covariance kernel function (suitable for simple, stationary covariance kernels). 2. Linear algebraic methods – Rank-revealing LU factorization, rank-revealing QR factorization, partial-pivoted LU decomposition.
§ Number of low-rank blocks in level j = n/2j § Cost of computing low-rank approximation per off-diagonal block = O(rn/2j) § Total number of off-diagonal blocks = O(log n) § Total cost of all off-diagonal low-rank approximations = O(n log n) COST OF COMPUTING ALL THE LOW RANK OFF DIAGONAL BLOCKS COST OF PERFORMING THE HODLR FACTORIZATION § Number of levels, = O(log n) § Cost of inverting the first block diagonally dense block – O(p3) § Cost of applying the inverses to the low-rank blocks in the next factor – O(prn/2j) § Total number of times the inverse application product has to be performed – O(2) § Cost for obtaining the 1st factorization level – O(prn log n) § Total cost of estimating all factors – O(prn log2n) ≅ O(n log2n) § Total cost of estimating all factors = O(n log n) (Assuming p, r << n)
FACTORIZATION § Cost of inverting first block diagonally dense matrix – None (Done in previous step) § Cost of inverting each of the remaining factors – O(n) (By SMW formula) § Total cost of inversion – O(n log n) § Cost of computing the determinant of each factor – O(n) (By Sylvester’s Determinant Theorem) § Total cost of computing determinant - O(n log n)
HODLR covariance matrix inversion when the input is 1,2,3 dimensional. The scaling of inversion of the HODLR factors is dimension size dependent whereas the direct inversion of K does not dependent on the dimensionality of x but has cubic scaling. These results are for the RBF kernel.
Ambikasaran, Sivaram, Daniel Foreman-Mackey, Leslie Greengard, David W. Hogg, and Michael O'Neil. "Fast direct methods for gaussian processes." arXiv preprint arXiv:1403.6015 (2014).