機械学習と機械発見：自然科学研究におけるデータ利活用の再考

[email protected] https://itakigawa.github.io/ IBISML 2022.12.22 @

( ) ( ) / (7 ) (7 ) JST
(3.5 ) ( ) ⾒ (4 ) (4 ) ( ) ( ) AIP ATR ( )

Materials Informatics (Bioinformatics) Materials Informatics ( )

. . o -the-shelf . PDF https://itakigawa.page.link/IBISML taki

‣ ‣ ( ) ( )

( ) <latexit sha1_base64="rREik5SVI+19We5hP0uVzuGffGw=">AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==</latexit> x⇤ <latexit sha1_base64="ggumTYNxk72HcgOPYsD+vGiQOG0=">AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==</latexit> x <latexit sha1_base64="Nr2ImZk9Gvp+/1cBWPwAK6yswp0=">AAACiXichVFNLwNRFD0dX1VfxUZi02iIVXNHBOmq0Y1lP7QkiMyMh9H5ysy0UU3/gJWdYEViIX6AH2DjD1j0J4gliY2F2+kkguBO3rzzzrvnvvPeVR1D93yiVkTq6u7p7Yv2xwYGh4ZH4qNjZc+uupooabZhu+uq4glDt0TJ131DrDuuUEzVEGtqJdveX6sJ19Nta9WvO2LLVPYsfVfXFJ+p8qZqNg6b2/EkpSiIxE8ghyCJMHJ2/A6b2IENDVWYELDgMzagwONvAzIIDnNbaDDnMtKDfYEmYqytcpbgDIXZCv/3eLURshav2zW9QK3xKQYPl5UJTNMj3dALPdAtPdH7r7UaQY22lzrPakcrnO2R44ni278qk2cf+5+qPz372MVS4FVn707AtG+hdfS1o9OXYrow3ZihK3pm/5fUonu+gVV71a7zonDxhx+VvfCLcYPk7+34CcpzKXkhRfn5ZGY5bFUUk5jCLPdjERmsIIcS1z/ACc5wLg1IsrQkpTupUiTUjONLSNkPIVKSSQ==</latexit>
x <latexit sha1_base64="Qqfga3YaBoUI0Yb7fQVzumPP1Sw=">AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=</latexit> y <latexit sha1_base64="TEyxI6Ffe+b+OEFRulxYJ0sUB8w=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBGBdk8IVxRXTjkoc8EkTS1gEbStu0hQSJP2DiVhauNHFh/AA/wI0/4IJPMC4xcePCS2lilIi3mc6ZM/fcOTNXMlTFshnreISR0bHxCe+kb2p6ZtYfmJvPWnrdlHlG1lXdzEuixVVF4xlbsVWeN0wu1iSV56Tqfm8/1+Cmpejaod00eLEmVjSlrMiiTVS6ebxWCoRYhDkRHARRF4TgRkIPPOIIJ9Aho44aODTYhFWIsOgrIAoGg7giWsSZhBRnn+McPtLWKYtThkhslf4VWhVcVqN1r6blqGU6RaVhkjKIMHth96zLntkDe2Wff9ZqOTV6Xpo0S30tN0r+i8X0x7+qGs02Tr9VQz3bKGPH8aqQd8NhereQ+/rGWbub3k2FWyvslr2R/xvWYU90A63xLt8leep6iB+JvNCLUYOiv9sxCLLrkeh2ZCO5GYrvua3yYgnLWKV+xBDHARLIUP0KLnGFtuAVIsKWEOunCh5Xs4AfIcS/AEWBkJQ=</latexit> y⇤ <latexit sha1_base64="rREik5SVI+19We5hP0uVzuGffGw=">AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==</latexit> x⇤

( ) <latexit sha1_base64="rREik5SVI+19We5hP0uVzuGffGw=">AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==</latexit> x⇤ <latexit sha1_base64="ggumTYNxk72HcgOPYsD+vGiQOG0=">AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==</latexit> x <latexit sha1_base64="Nr2ImZk9Gvp+/1cBWPwAK6yswp0=">AAACiXichVFNLwNRFD0dX1VfxUZi02iIVXNHBOmq0Y1lP7QkiMyMh9H5ysy0UU3/gJWdYEViIX6AH2DjD1j0J4gliY2F2+kkguBO3rzzzrvnvvPeVR1D93yiVkTq6u7p7Yv2xwYGh4ZH4qNjZc+uupooabZhu+uq4glDt0TJ131DrDuuUEzVEGtqJdveX6sJ19Nta9WvO2LLVPYsfVfXFJ+p8qZqNg6b2/EkpSiIxE8ghyCJMHJ2/A6b2IENDVWYELDgMzagwONvAzIIDnNbaDDnMtKDfYEmYqytcpbgDIXZCv/3eLURshav2zW9QK3xKQYPl5UJTNMj3dALPdAtPdH7r7UaQY22lzrPakcrnO2R44ni278qk2cf+5+qPz372MVS4FVn707AtG+hdfS1o9OXYrow3ZihK3pm/5fUonu+gVV71a7zonDxhx+VvfCLcYPk7+34CcpzKXkhRfn5ZGY5bFUUk5jCLPdjERmsIIcS1z/ACc5wLg1IsrQkpTupUiTUjONLSNkPIVKSSQ==</latexit>
x <latexit sha1_base64="Qqfga3YaBoUI0Yb7fQVzumPP1Sw=">AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=</latexit> y <latexit sha1_base64="TEyxI6Ffe+b+OEFRulxYJ0sUB8w=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBGBdk8IVxRXTjkoc8EkTS1gEbStu0hQSJP2DiVhauNHFh/AA/wI0/4IJPMC4xcePCS2lilIi3mc6ZM/fcOTNXMlTFshnreISR0bHxCe+kb2p6ZtYfmJvPWnrdlHlG1lXdzEuixVVF4xlbsVWeN0wu1iSV56Tqfm8/1+Cmpejaod00eLEmVjSlrMiiTVS6ebxWCoRYhDkRHARRF4TgRkIPPOIIJ9Aho44aODTYhFWIsOgrIAoGg7giWsSZhBRnn+McPtLWKYtThkhslf4VWhVcVqN1r6blqGU6RaVhkjKIMHth96zLntkDe2Wff9ZqOTV6Xpo0S30tN0r+i8X0x7+qGs02Tr9VQz3bKGPH8aqQd8NhereQ+/rGWbub3k2FWyvslr2R/xvWYU90A63xLt8leep6iB+JvNCLUYOiv9sxCLLrkeh2ZCO5GYrvua3yYgnLWKV+xBDHARLIUP0KLnGFtuAVIsKWEOunCh5Xs4AfIcS/AEWBkJQ=</latexit> y⇤ <latexit sha1_base64="qj6YHRPaHOekeYfY3235sFbWBxQ=">AAAChHichVHNLgNRFP6M/98WG4lNoyEW0pzpn9YCYWOpqDYpkZlxMel0ZjIzbULjBdgSCysSC/EAHsDGC1h4BLEksbFwZloRi3Ju7r3nfud85373HtU2dNcjem6T2js6u7p7evv6BwaHQuHhkU3XqjqayGuWYTlFVXGFoZsi7+meIYq2I5SKaoiCWl7244WacFzdMje8Q1tsV5R9U9/TNcVjKLewE45SLJtJUzIeoRhRJk5pdlIkZ+VsRGbEtyiatmqF77GFXVjQUEUFAiY89g0ocHmUIINgM7aNOmMOe3oQFzhGH3OrnCU4Q2G0zOs+n0pN1OSzX9MN2BrfYvB0mBnBJD3RLb3RI93RC322rFUPavhaDnlXG1xh74ROxtY//mVVePdw8MP6U7OHPWQCrTprtwPEf4XW4NeOLt7W59Ym61N0Ta+s/4qe6YFfYNbetZucWLv8Q4/KWvjHuEHfXYi0djbjMTkdS+SS0cWlZqt6MI4JTHM/ZrGIFawiz/UFTnGGc6lLmpESUqqRKrU1OaP4ZdL8F+zrkAQ=</latexit> ? <latexit sha1_base64="qj6YHRPaHOekeYfY3235sFbWBxQ=">AAAChHichVHNLgNRFP6M/98WG4lNoyEW0pzpn9YCYWOpqDYpkZlxMel0ZjIzbULjBdgSCysSC/EAHsDGC1h4BLEksbFwZloRi3Ju7r3nfud85373HtU2dNcjem6T2js6u7p7evv6BwaHQuHhkU3XqjqayGuWYTlFVXGFoZsi7+meIYq2I5SKaoiCWl7244WacFzdMje8Q1tsV5R9U9/TNcVjKLewE45SLJtJUzIeoRhRJk5pdlIkZ+VsRGbEtyiatmqF77GFXVjQUEUFAiY89g0ocHmUIINgM7aNOmMOe3oQFzhGH3OrnCU4Q2G0zOs+n0pN1OSzX9MN2BrfYvB0mBnBJD3RLb3RI93RC322rFUPavhaDnlXG1xh74ROxtY//mVVePdw8MP6U7OHPWQCrTprtwPEf4XW4NeOLt7W59Ym61N0Ta+s/4qe6YFfYNbetZucWLv8Q4/KWvjHuEHfXYi0djbjMTkdS+SS0cWlZqt6MI4JTHM/ZrGIFawiz/UFTnGGc6lLmpESUqqRKrU1OaP4ZdL8F+zrkAQ=</latexit> ? <latexit sha1_base64="qj6YHRPaHOekeYfY3235sFbWBxQ=">AAAChHichVHNLgNRFP6M/98WG4lNoyEW0pzpn9YCYWOpqDYpkZlxMel0ZjIzbULjBdgSCysSC/EAHsDGC1h4BLEksbFwZloRi3Ju7r3nfud85373HtU2dNcjem6T2js6u7p7evv6BwaHQuHhkU3XqjqayGuWYTlFVXGFoZsi7+meIYq2I5SKaoiCWl7244WacFzdMje8Q1tsV5R9U9/TNcVjKLewE45SLJtJUzIeoRhRJk5pdlIkZ+VsRGbEtyiatmqF77GFXVjQUEUFAiY89g0ocHmUIINgM7aNOmMOe3oQFzhGH3OrnCU4Q2G0zOs+n0pN1OSzX9MN2BrfYvB0mBnBJD3RLb3RI93RC322rFUPavhaDnlXG1xh74ROxtY//mVVePdw8MP6U7OHPWQCrTprtwPEf4XW4NeOLt7W59Ym61N0Ta+s/4qe6YFfYNbetZucWLv8Q4/KWvjHuEHfXYi0djbjMTkdS+SS0cWlZqt6MI4JTHM/ZrGIFawiz/UFTnGGc6lLmpESUqqRKrU1OaP4ZdL8F+zrkAQ=</latexit> ? <latexit sha1_base64="qj6YHRPaHOekeYfY3235sFbWBxQ=">AAAChHichVHNLgNRFP6M/98WG4lNoyEW0pzpn9YCYWOpqDYpkZlxMel0ZjIzbULjBdgSCysSC/EAHsDGC1h4BLEksbFwZloRi3Ju7r3nfud85373HtU2dNcjem6T2js6u7p7evv6BwaHQuHhkU3XqjqayGuWYTlFVXGFoZsi7+meIYq2I5SKaoiCWl7244WacFzdMje8Q1tsV5R9U9/TNcVjKLewE45SLJtJUzIeoRhRJk5pdlIkZ+VsRGbEtyiatmqF77GFXVjQUEUFAiY89g0ocHmUIINgM7aNOmMOe3oQFzhGH3OrnCU4Q2G0zOs+n0pN1OSzX9MN2BrfYvB0mBnBJD3RLb3RI93RC322rFUPavhaDnlXG1xh74ROxtY//mVVePdw8MP6U7OHPWQCrTprtwPEf4XW4NeOLt7W59Ym61N0Ta+s/4qe6YFfYNbetZucWLv8Q4/KWvjHuEHfXYi0djbjMTkdS+SS0cWlZqt6MI4JTHM/ZrGIFawiz/UFTnGGc6lLmpESUqqRKrU1OaP4ZdL8F+zrkAQ=</latexit> ? <latexit sha1_base64="qj6YHRPaHOekeYfY3235sFbWBxQ=">AAAChHichVHNLgNRFP6M/98WG4lNoyEW0pzpn9YCYWOpqDYpkZlxMel0ZjIzbULjBdgSCysSC/EAHsDGC1h4BLEksbFwZloRi3Ju7r3nfud85373HtU2dNcjem6T2js6u7p7evv6BwaHQuHhkU3XqjqayGuWYTlFVXGFoZsi7+meIYq2I5SKaoiCWl7244WacFzdMje8Q1tsV5R9U9/TNcVjKLewE45SLJtJUzIeoRhRJk5pdlIkZ+VsRGbEtyiatmqF77GFXVjQUEUFAiY89g0ocHmUIINgM7aNOmMOe3oQFzhGH3OrnCU4Q2G0zOs+n0pN1OSzX9MN2BrfYvB0mBnBJD3RLb3RI93RC322rFUPavhaDnlXG1xh74ROxtY//mVVePdw8MP6U7OHPWQCrTprtwPEf4XW4NeOLt7W59Ym61N0Ta+s/4qe6YFfYNbetZucWLv8Q4/KWvjHuEHfXYi0djbjMTkdS+SS0cWlZqt6MI4JTHM/ZrGIFawiz/UFTnGGc6lLmpESUqqRKrU1OaP4ZdL8F+zrkAQ=</latexit> ? <latexit sha1_base64="rREik5SVI+19We5hP0uVzuGffGw=">AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==</latexit> x⇤

( ) <latexit sha1_base64="Nr2ImZk9Gvp+/1cBWPwAK6yswp0=">AAACiXichVFNLwNRFD0dX1VfxUZi02iIVXNHBOmq0Y1lP7QkiMyMh9H5ysy0UU3/gJWdYEViIX6AH2DjD1j0J4gliY2F2+kkguBO3rzzzrvnvvPeVR1D93yiVkTq6u7p7Yv2xwYGh4ZH4qNjZc+uupooabZhu+uq4glDt0TJ131DrDuuUEzVEGtqJdveX6sJ19Nta9WvO2LLVPYsfVfXFJ+p8qZqNg6b2/EkpSiIxE8ghyCJMHJ2/A6b2IENDVWYELDgMzagwONvAzIIDnNbaDDnMtKDfYEmYqytcpbgDIXZCv/3eLURshav2zW9QK3xKQYPl5UJTNMj3dALPdAtPdH7r7UaQY22lzrPakcrnO2R44ni278qk2cf+5+qPz372MVS4FVn707AtG+hdfS1o9OXYrow3ZihK3pm/5fUonu+gVV71a7zonDxhx+VvfCLcYPk7+34CcpzKXkhRfn5ZGY5bFUUk5jCLPdjERmsIIcS1z/ACc5wLg1IsrQkpTupUiTUjONLSNkPIVKSSQ==</latexit> x <latexit sha1_base64="Qqfga3YaBoUI0Yb7fQVzumPP1Sw=">AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=</latexit> y <latexit sha1_base64="TEyxI6Ffe+b+OEFRulxYJ0sUB8w=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBGBdk8IVxRXTjkoc8EkTS1gEbStu0hQSJP2DiVhauNHFh/AA/wI0/4IJPMC4xcePCS2lilIi3mc6ZM/fcOTNXMlTFshnreISR0bHxCe+kb2p6ZtYfmJvPWnrdlHlG1lXdzEuixVVF4xlbsVWeN0wu1iSV56Tqfm8/1+Cmpejaod00eLEmVjSlrMiiTVS6ebxWCoRYhDkRHARRF4TgRkIPPOIIJ9Aho44aODTYhFWIsOgrIAoGg7giWsSZhBRnn+McPtLWKYtThkhslf4VWhVcVqN1r6blqGU6RaVhkjKIMHth96zLntkDe2Wff9ZqOTV6Xpo0S30tN0r+i8X0x7+qGs02Tr9VQz3bKGPH8aqQd8NhereQ+/rGWbub3k2FWyvslr2R/xvWYU90A63xLt8leep6iB+JvNCLUYOiv9sxCLLrkeh2ZCO5GYrvua3yYgnLWKV+xBDHARLIUP0KLnGFtuAVIsKWEOunCh5Xs4AfIcS/AEWBkJQ=</latexit>
y⇤ <latexit sha1_base64="rREik5SVI+19We5hP0uVzuGffGw=">AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==</latexit> x⇤ <latexit sha1_base64="rREik5SVI+19We5hP0uVzuGffGw=">AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==</latexit> x⇤ <latexit sha1_base64="ggumTYNxk72HcgOPYsD+vGiQOG0=">AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==</latexit> x <latexit sha1_base64="YCdrxHBq8hWthguuj4LwkMwyoug=">AAAB93icdVDLSsNAFJ3UV62vqks3g0VwFZKatM2u6MZlC9YW2lAm00k7dCYJMxMhhH6BW127E7d+jkv/xOlDsKIHLhzOuZd77wkSRqWyrA+jsLG5tb1T3C3t7R8cHpWPT+5lnApMOjhmsegFSBJGI9JRVDHSSwRBPGCkG0xv5n73gQhJ4+hOZQnxORpHNKQYKS21w2G5YpmW6zn1KrRM17I950oTz2s4NRfaprVABazQGpY/B6MYp5xECjMkZd+2EuXnSCiKGZmVBqkkCcJTNCZ9TSPEifTzxaEzeKGVEQxjoStScKH+nMgRlzLjge7kSE3kb28u/uX1UxU2/JxGSapIhJeLwpRBFcP513BEBcGKZZogLKi+FeIJEggrnc3aloDPdCbfj8P/yX3VtGum23YqzetVOkVwBs7BJbBBHTTBLWiBDsCAgEfwBJ6NzHgxXo23ZWvBWM2cgjUY71/5QpPk</latexit> f <latexit sha1_base64="DcbiaPwQEMQnrQxSiqXKigyS2IY=">AAACNXicbVDLSgMxFM34tr6qLt0Ei6Agw4z2YReC6MZlBdsKM0PJpJk2mMwMSUYsYb7Gj/Ab3OrKhQtB3PoLpo+FWg8EDuecm3s5YcqoVI7zas3Mzs0vLC4tF1ZW19Y3iptbLZlkApMmTlgibkIkCaMxaSqqGLlJBUE8ZKQd3l4M/fYdEZIm8bUapCTgqBfTiGKkjNQpnvp9pPQgh6dQ+6PvPNELA+3YTqVerh0dOnbFcevlY0Pq9ZNytZLDKN/3Q67v84NOsWSCI8Bp4k5ICUzQ6BTf/W6CM05ihRmS0nOdVAUaCUUxI3nBzyRJEb5FPeIZGiNOZKBHd+VwzyhdGCXCvFjBkfpzQiMu5YCHJsmR6su/3lD8z/MyFZ0EmsZppkiMx4uijEGVwGFnsEsFwYoNDEFYUHMrxH0kEFam2V9bQp6bTty/DUyT1pHtVu3KVbl0dj5pZwnsgF2wD1xQA2fgEjRAE2DwAJ7AM3ixHq0368P6HEdnrMnMNvgF6+sbdw6oMA==</latexit> ˆ y = f(x)

<latexit sha1_base64="ou7gs1Bt14yFxh1zGfB0tITvvDk=">AAACpXicbVFda9swFJW9ry77aNY+7kUsjKVQgp04H34YlJaNvQw6aNKCbYKkyK6oJBtJLgvCP2k/aI/7J1OcPKzpLggdzr3n3KsrXHGmTRD89vwnT589f3HwsvPq9Zu3h913Rwtd1orQOSl5qW4w0pQzSeeGGU5vKkWRwJxe47uLTf76nirNSnll1hXNBCokyxlBxlHL7i+btiaJKnBmg0EwjqPp8DQYjIMwjkYOxPEsmowbmDfwM0yRKmAqmFzaHKZMOozMLUHcfm2azp7XNA4mYegs3DWKJg6Ew2A0i5qtCmP7xYlSzAqedFpKCUuVKlXTX5/CvJ9iYX82JydtjcpgZ9ntObc24GMQ7kAP7OJy2f2TrkpSCyoN4UjrJAwqk1mkDCOcuua1phUid6igiYMSCaoz276igR8ds4J5qdyRBrbsvwqLhNZrgV3lZni9n9uQ/8sltclnmWWyqg2VZNsorzk0Jdz8EVwxRYnhawcQUczNCsktUogY95MPumDRuJ2E+xt4DBbDQTgZjH9EvbPz3XYOwHvwAfRBCKbgDHwDl2AOiHfsxd65d+F/8r/7V/5iW+p7O80xeBD+8i9BGsel</latexit> f = arg min f2F E ⇥ error(y,
f(x)) ⇤ ( ) <latexit sha1_base64="Nr2ImZk9Gvp+/1cBWPwAK6yswp0=">AAACiXichVFNLwNRFD0dX1VfxUZi02iIVXNHBOmq0Y1lP7QkiMyMh9H5ysy0UU3/gJWdYEViIX6AH2DjD1j0J4gliY2F2+kkguBO3rzzzrvnvvPeVR1D93yiVkTq6u7p7Yv2xwYGh4ZH4qNjZc+uupooabZhu+uq4glDt0TJ131DrDuuUEzVEGtqJdveX6sJ19Nta9WvO2LLVPYsfVfXFJ+p8qZqNg6b2/EkpSiIxE8ghyCJMHJ2/A6b2IENDVWYELDgMzagwONvAzIIDnNbaDDnMtKDfYEmYqytcpbgDIXZCv/3eLURshav2zW9QK3xKQYPl5UJTNMj3dALPdAtPdH7r7UaQY22lzrPakcrnO2R44ni278qk2cf+5+qPz372MVS4FVn707AtG+hdfS1o9OXYrow3ZihK3pm/5fUonu+gVV71a7zonDxhx+VvfCLcYPk7+34CcpzKXkhRfn5ZGY5bFUUk5jCLPdjERmsIIcS1z/ACc5wLg1IsrQkpTupUiTUjONLSNkPIVKSSQ==</latexit> x <latexit sha1_base64="Qqfga3YaBoUI0Yb7fQVzumPP1Sw=">AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=</latexit> y <latexit sha1_base64="TEyxI6Ffe+b+OEFRulxYJ0sUB8w=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBGBdk8IVxRXTjkoc8EkTS1gEbStu0hQSJP2DiVhauNHFh/AA/wI0/4IJPMC4xcePCS2lilIi3mc6ZM/fcOTNXMlTFshnreISR0bHxCe+kb2p6ZtYfmJvPWnrdlHlG1lXdzEuixVVF4xlbsVWeN0wu1iSV56Tqfm8/1+Cmpejaod00eLEmVjSlrMiiTVS6ebxWCoRYhDkRHARRF4TgRkIPPOIIJ9Aho44aODTYhFWIsOgrIAoGg7giWsSZhBRnn+McPtLWKYtThkhslf4VWhVcVqN1r6blqGU6RaVhkjKIMHth96zLntkDe2Wff9ZqOTV6Xpo0S30tN0r+i8X0x7+qGs02Tr9VQz3bKGPH8aqQd8NhereQ+/rGWbub3k2FWyvslr2R/xvWYU90A63xLt8leep6iB+JvNCLUYOiv9sxCLLrkeh2ZCO5GYrvua3yYgnLWKV+xBDHARLIUP0KLnGFtuAVIsKWEOunCh5Xs4AfIcS/AEWBkJQ=</latexit> y⇤ <latexit sha1_base64="rREik5SVI+19We5hP0uVzuGffGw=">AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==</latexit> x⇤ <latexit sha1_base64="rREik5SVI+19We5hP0uVzuGffGw=">AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==</latexit> x⇤ <latexit sha1_base64="ggumTYNxk72HcgOPYsD+vGiQOG0=">AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==</latexit> x <latexit sha1_base64="YCdrxHBq8hWthguuj4LwkMwyoug=">AAAB93icdVDLSsNAFJ3UV62vqks3g0VwFZKatM2u6MZlC9YW2lAm00k7dCYJMxMhhH6BW127E7d+jkv/xOlDsKIHLhzOuZd77wkSRqWyrA+jsLG5tb1T3C3t7R8cHpWPT+5lnApMOjhmsegFSBJGI9JRVDHSSwRBPGCkG0xv5n73gQhJ4+hOZQnxORpHNKQYKS21w2G5YpmW6zn1KrRM17I950oTz2s4NRfaprVABazQGpY/B6MYp5xECjMkZd+2EuXnSCiKGZmVBqkkCcJTNCZ9TSPEifTzxaEzeKGVEQxjoStScKH+nMgRlzLjge7kSE3kb28u/uX1UxU2/JxGSapIhJeLwpRBFcP513BEBcGKZZogLKi+FeIJEggrnc3aloDPdCbfj8P/yX3VtGum23YqzetVOkVwBs7BJbBBHTTBLWiBDsCAgEfwBJ6NzHgxXo23ZWvBWM2cgjUY71/5QpPk</latexit> f <latexit sha1_base64="DcbiaPwQEMQnrQxSiqXKigyS2IY=">AAACNXicbVDLSgMxFM34tr6qLt0Ei6Agw4z2YReC6MZlBdsKM0PJpJk2mMwMSUYsYb7Gj/Ab3OrKhQtB3PoLpo+FWg8EDuecm3s5YcqoVI7zas3Mzs0vLC4tF1ZW19Y3iptbLZlkApMmTlgibkIkCaMxaSqqGLlJBUE8ZKQd3l4M/fYdEZIm8bUapCTgqBfTiGKkjNQpnvp9pPQgh6dQ+6PvPNELA+3YTqVerh0dOnbFcevlY0Pq9ZNytZLDKN/3Q67v84NOsWSCI8Bp4k5ICUzQ6BTf/W6CM05ihRmS0nOdVAUaCUUxI3nBzyRJEb5FPeIZGiNOZKBHd+VwzyhdGCXCvFjBkfpzQiMu5YCHJsmR6su/3lD8z/MyFZ0EmsZppkiMx4uijEGVwGFnsEsFwYoNDEFYUHMrxH0kEFam2V9bQp6bTty/DUyT1pHtVu3KVbl0dj5pZwnsgF2wD1xQA2fgEjRAE2DwAJ7AM3ixHq0368P6HEdnrMnMNvgF6+sbdw6oMA==</latexit> ˆ y = f(x) <latexit sha1_base64="0fXeD1cakvAs7WfPkj1UglydrqI=">AAACPXicbVDNSgMxGMz6b/2revQSLEIVKbu1VXtQiiJ4VLAqdNeSTbNtaLK7JFlxCftEPoTP4FE9efAmXr2a1h60OhAYZr7J9zF+zKhUtv1kjY1PTE5Nz8zm5uYXFpfyyyuXMkoEJg0csUhc+0gSRkPSUFQxch0LgrjPyJXfO+77V7dESBqFFyqNicdRJ6QBxUgZqZU/cbtI6TSDB1C7g++aouN72i7Z1Vplr7xtl6q2U6vsGFKr7Vd2qxkMsqLrc32XbcJDmN7orayVL5jAAPAvcYakAIY4a+Vf3XaEE05ChRmSsunYsfI0EopiRrKcm0gSI9xDHdI0NEScSE8P7svghlHaMIiEeaGCA/VnQiMuZcp9M8mR6spRry/+5zUTFex7moZxokiIvxcFCYMqgv3uYJsKghVLDUFYUHMrxF0kEFam4V9bfN7vxBlt4C+5LJec3VL1vFKoHw3bmQFrYB0UgQP2QB2cgjPQABjcg0fwDF6sB+vNerc+vkfHrGFmFfyC9fkFIG+q9w==</latexit> ˆ y = f(x) > y⇤ <latexit sha1_base64="ggumTYNxk72HcgOPYsD+vGiQOG0=">AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==</latexit> x

(Uncertainty Quantification, UQ) <latexit sha1_base64="Nr2ImZk9Gvp+/1cBWPwAK6yswp0=">AAACiXichVFNLwNRFD0dX1VfxUZi02iIVXNHBOmq0Y1lP7QkiMyMh9H5ysy0UU3/gJWdYEViIX6AH2DjD1j0J4gliY2F2+kkguBO3rzzzrvnvvPeVR1D93yiVkTq6u7p7Yv2xwYGh4ZH4qNjZc+uupooabZhu+uq4glDt0TJ131DrDuuUEzVEGtqJdveX6sJ19Nta9WvO2LLVPYsfVfXFJ+p8qZqNg6b2/EkpSiIxE8ghyCJMHJ2/A6b2IENDVWYELDgMzagwONvAzIIDnNbaDDnMtKDfYEmYqytcpbgDIXZCv/3eLURshav2zW9QK3xKQYPl5UJTNMj3dALPdAtPdH7r7UaQY22lzrPakcrnO2R44ni278qk2cf+5+qPz372MVS4FVn707AtG+hdfS1o9OXYrow3ZihK3pm/5fUonu+gVV71a7zonDxhx+VvfCLcYPk7+34CcpzKXkhRfn5ZGY5bFUUk5jCLPdjERmsIIcS1z/ACc5wLg1IsrQkpTupUiTUjONLSNkPIVKSSQ==</latexit> x <latexit sha1_base64="Qqfga3YaBoUI0Yb7fQVzumPP1Sw=">AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=</latexit> y <latexit
sha1_base64="rREik5SVI+19We5hP0uVzuGffGw=">AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==</latexit> x⇤ <latexit sha1_base64="rREik5SVI+19We5hP0uVzuGffGw=">AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==</latexit> x⇤ <latexit sha1_base64="ggumTYNxk72HcgOPYsD+vGiQOG0=">AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==</latexit> x <latexit sha1_base64="Qqfga3YaBoUI0Yb7fQVzumPP1Sw=">AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=</latexit> y <latexit sha1_base64="Qqfga3YaBoUI0Yb7fQVzumPP1Sw=">AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=</latexit> y <latexit sha1_base64="uHwbcp1I7cfAojOCGzVZ2+fOJ7M=">AAACh3ichVE7TsNAEH2Yb/glQINEExGBoAnjACGh4tNQQiCAFKLINhuwcGzLdiKFiAtQ0AaJCiQKxAE4AA0XoMgRECVINBSMnSBEEZjV7s6+mTf7dke1Dd31iBodUmdXd09vX6h/YHBoOBwZGd11rbKjiaxmGZazryquMHRTZD3dM8S+7QilpBpiTz1Z9+N7FeG4umXueFVb5EvKkakXdU3xfMieqc4WIjGKp1NJWkhEKU6USlCSnUWS03I6KjPiWwwt27QiDzjAISxoKKMEARMe+wYUuDxykEGwGcujxpjDnh7EBc7Qz9wyZwnOUBg94fWIT7kWavLZr+kGbI1vMXg6zIxiip7pjt7oie7phT7b1qoFNXwtVd7VJlfYhfD5+PbHv6wS7x6Of1h/avZQRCrQqrN2O0D8V2hNfuW0/ra9nJmqTdMNvbL+a2rQI7/ArLxrt1sic/WHHpW18I9xg767EG3v7CbicjI+v7UQW1lrtaoPE5jEDPdjCSvYwCayXP8YF6jjUgpJc1JSSjVTpY4WZwy/TFr9AnUXkR0=</latexit> p(y) <latexit sha1_base64="xAYAr5eatCRkt85fJM86k8bLQd0=">AAAChnichVE7TsNAEH2YXwi/AA0STUQEQhTWOCQhoYqgoSSEABI/2WYTLBzbsp1IIeICSLRQUIFEgTgAB6DhAhQcAVGCREPB2AlCFMCsdnf2zbzZtzuaYxqeT/TUIXV2dff0Rvqi/QODQ8OxkdF1z665uijptmm7m5rqCdOwRMk3fFNsOq5Qq5opNrTDpSC+UReuZ9jWmt9wxE5VrVhG2dBVn6FiY3d2L5YgOZfNUCoZJ5kom6QMO2lSckourjASWAJtW7Fjd9jGPmzoqKEKAQs++yZUeDy2oIDgMLaDJmMue0YYFzhGlLk1zhKcoTJ6yGuFT1tt1OJzUNML2TrfYvJ0mRnHFD3SDb3SA93SM338WqsZ1gi0NHjXWlzh7A2fjBff/2VVefdx8M36U7OPMrKhVoO1OyESvEJv8etH56/FhdWp5jRd0Qvrv6QnuucXWPU3/bogVi/+0KOxFv4xbtBXF+K/O+tJWcnIc4VUIr/YblUEE5jEDPdjHnksYwUlrl/BKc5wLkUkWUpL861UqaPNGcMPk/Kf1ZuQ2g==</latexit> y⇤ <latexit sha1_base64="Cx8vtQI6wD6rFemnJtzP0VxR74M=">AAACp3ichVE9T9tQFD0Y2vJRSqALUheLiCphiK5TGhIkKlSWboSPhEh8RLbzQp9wbMt+ieRG7FX/QIdOrdQBIXWFnYU/wMBPqDqCxMLQaydVxQBc6/mdd9499533ruU7MlRElwPa4NCTp8+GR0bHno+/mEhNTlVDrx3YomJ7jhfULDMUjnRFRUnliJofCLNlOWLLOliJ97c6Igil526qyBe7LXPflU1pm4qpemqmnIneRXtz2aUd6ap6l+HhHsOminQ/E2X1RlRPpSlXKhZoPq9TjqiYpwKDt2SUjJJuMBNHGv0oe6kT7KABDzbaaEHAhWLswETI3zYMEHzmdtFlLmAkk32BQ4yyts1ZgjNMZg/4v8+r7T7r8jquGSZqm09xeASs1DFLF3REV3ROx/Sbbu+t1U1qxF4inq2eVvj1iS/TGzePqlo8K3z8r3rQs0ITxcSrZO9+wsS3sHv6zqevVxuL67Pd1/SD/rD/73RJZ3wDt3Nt/1wT698e8GOxF34xbtC/Luj3g2o+ZxRyb9bm08vv+60axivMIMP9WMAyPqCMCtf/jF84wamW1Va1qlbrpWoDfc1L3AnN/AuqB50Q</latexit> P(y > y⇤) = Z 1 y⇤ p(y)dy <latexit sha1_base64="svoh2wRjAJ9IgkLWLNY4QZ6NPX0=">AAACwnichVHLTttAFL24vMsjlA0Sm1EjKlpF0XWANEECIWilLnk0gBSHyHYmMOBX7UkkY/ID/AALVi3qouoH9AO6abetWPAJqEsqsWHRayeoYgFcazxnztxz58xcw7NEIBEvupQn3T29ff0Dg0+HhkdGU2PPNgO34Zu8ZLqW628besAt4fCSFNLi257Pdduw+JZxsBLvbzW5HwjXeS9Dj1dsfdcRdWHqkqhqakGzdblnGNHbVlnLhEzLHGkZFi6GO68qC5pwZDUi2NohWJchowSz5krmTYcvWS2sptKYLRbyOJtjmEUs5DBPYA7VolpkKjFxpKETq27qG2hQAxdMaIANHByQhC3QIaCvDCogeMRVICLOJySSfQ4tGCRtg7I4ZejEHtB/l1blDuvQOq4ZJGqTTrFo+KRkMIXn+AWv8Ad+xUu8ubdWlNSIvYQ0G20t96qjxxMb14+qbJol7P1XPehZQh0KiVdB3r2EiW9htvXNw5Orjfn1qegFfsI/5P8jXuB3uoHT/Gt+XuPrpw/4McgLvRg16LYL7H6wmcuq+ezM2mx6abnTqn6YhOcwTf14DUvwDlahRPXP4Cf8gt/KG2Vf+aAE7VSlq6MZhzuhHP0DArmnyA==</latexit> E[ y | y > y⇤] = Z 1 y⇤ y · p(y)dy Probability of improvement (PI) Expected Improvement (EI) Upper Conﬁdence Bound (UCB) (UQ)

( ) <latexit sha1_base64="mpuRLmJhhTRKlPt+QsNIZJRw5HU=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==</latexit> x1 <latexit sha1_base64="C1ohMOVeKO2tnLKPo8dfEeHHC5o=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==</latexit> x2 <latexit sha1_base64="ms/uPGAGeTg0lt5op+vO01P7rjg=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==</latexit>
y <latexit sha1_base64="WQjRG4bPVEcrBhk0IPMU7zTCz9E=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==</latexit> f <latexit sha1_base64="WQjRG4bPVEcrBhk0IPMU7zTCz9E=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==</latexit> f <latexit sha1_base64="ms/uPGAGeTg0lt5op+vO01P7rjg=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==</latexit> y <latexit sha1_base64="mpuRLmJhhTRKlPt+QsNIZJRw5HU=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==</latexit> x1 <latexit sha1_base64="C1ohMOVeKO2tnLKPo8dfEeHHC5o=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==</latexit> x2 <latexit sha1_base64="mpuRLmJhhTRKlPt+QsNIZJRw5HU=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==</latexit> x1 <latexit sha1_base64="C1ohMOVeKO2tnLKPo8dfEeHHC5o=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==</latexit> x2 <latexit sha1_base64="ms/uPGAGeTg0lt5op+vO01P7rjg=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==</latexit> y

y <latexit sha1_base64="WQjRG4bPVEcrBhk0IPMU7zTCz9E=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==</latexit> f <latexit sha1_base64="WQjRG4bPVEcrBhk0IPMU7zTCz9E=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==</latexit> f <latexit sha1_base64="ms/uPGAGeTg0lt5op+vO01P7rjg=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==</latexit> y <latexit sha1_base64="mpuRLmJhhTRKlPt+QsNIZJRw5HU=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==</latexit> x1 <latexit sha1_base64="C1ohMOVeKO2tnLKPo8dfEeHHC5o=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==</latexit> x2 <latexit sha1_base64="mpuRLmJhhTRKlPt+QsNIZJRw5HU=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==</latexit> x1 <latexit sha1_base64="C1ohMOVeKO2tnLKPo8dfEeHHC5o=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==</latexit> x2 <latexit sha1_base64="ms/uPGAGeTg0lt5op+vO01P7rjg=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==</latexit> y ‣ (= ) (x , x ) y ( )

y <latexit sha1_base64="WQjRG4bPVEcrBhk0IPMU7zTCz9E=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==</latexit> f <latexit sha1_base64="WQjRG4bPVEcrBhk0IPMU7zTCz9E=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==</latexit> f <latexit sha1_base64="ms/uPGAGeTg0lt5op+vO01P7rjg=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==</latexit> y <latexit sha1_base64="mpuRLmJhhTRKlPt+QsNIZJRw5HU=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==</latexit> x1 <latexit sha1_base64="C1ohMOVeKO2tnLKPo8dfEeHHC5o=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==</latexit> x2 <latexit sha1_base64="mpuRLmJhhTRKlPt+QsNIZJRw5HU=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==</latexit> x1 <latexit sha1_base64="C1ohMOVeKO2tnLKPo8dfEeHHC5o=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==</latexit> x2 <latexit sha1_base64="ms/uPGAGeTg0lt5op+vO01P7rjg=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==</latexit> y ‣ (= ) (x , x ) y ( ) ‣ (x , x ) ( ) y (x , x )

<latexit sha1_base64="mpuRLmJhhTRKlPt+QsNIZJRw5HU=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==</latexit> x1 <latexit sha1_base64="C1ohMOVeKO2tnLKPo8dfEeHHC5o=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==</latexit> x2 <latexit sha1_base64="ms/uPGAGeTg0lt5op+vO01P7rjg=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==</latexit> y <latexit
sha1_base64="WQjRG4bPVEcrBhk0IPMU7zTCz9E=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==</latexit> f <latexit sha1_base64="WQjRG4bPVEcrBhk0IPMU7zTCz9E=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==</latexit> f <latexit sha1_base64="ms/uPGAGeTg0lt5op+vO01P7rjg=">AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==</latexit> y <latexit sha1_base64="mpuRLmJhhTRKlPt+QsNIZJRw5HU=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==</latexit> x1 <latexit sha1_base64="C1ohMOVeKO2tnLKPo8dfEeHHC5o=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==</latexit> x2 ⾒ ( ) N = = N = = N = = 10 10 =100 Full Factorial Design

Fisher ( ) . (Replication) ⾒ ( ) . (Local
Control) . (Randomization) Randomized Controlled Trial (Fisher ) Data Leakage

(or ?) UQ ⾒

(or ?) UQ ⾒ ( ) ⾒ ( ?) ༗઒અ෉
Herbert A. Simon Simon HA, Machine Discovery. ( ) Langley PW, Simon HA, Bradshaw G, Zytkow JM, Scienti c Discovery: Computational Explorations of the Creative Process ( ). , . (1996) , . (2001)

o -the-shelf

( big data ) ( ) interpolator ( ) o
-the-shelf

KernelRidge(kernel='rbf', alpha=0.05, gamma=0.1) KernelRidge(kernel='rbf', alpha=1e-4, gamma=0.1) KernelRidge(kernel='rbf', alpha=1e-4, gamma=2.0) ExtraTreesRegressor(n_estimators=50)
DecisionTreeRegressor()

DecisionTreeRegressor() ( )

DecisionTreeRegressor() ( ) / (Activity Cli s, Selectivity Cli s, )

(UQ) skopt.learning. ExtraTreesRegressor(n_estimators=50) skopt.learning. ExtraTreesRegressor(n_estimators=50, bootstrap=True) GaussianProcessRegressor(kernel=1*RBF(),  alpha=1e-2) skopt.learning. RandomForestRegressor(n_estimators=50)
base = ExtraTreesRegressor(n_estimators=50,   bootstrap=True) MapieRegressor(base, method="plus", cv=-1) GaussianProcessRegressor(kernel=1*RBF(),  alpha=1e-5) GaussianProcessRegressor(kernel=1*RBF(),  alpha=1e-4) MapieRegressor(base, method="plus", cv=Subsample(n_resamplings=50))

(>100) ( ) ( !?)

(>100) ( ) ( !?) ⾒ Balestriero R, Pesenti J,
LeCun Y. Learning in High Dimension Always Amounts to Extrapolation. arXiv [cs.LG]. 2021. http://arxiv.org/abs/2110.09485 (interpolation) y • “Our goal in this paper is to demonstrate both theoretically and empirically for both synthetic and real data that interpolation almost surely never occurs in high-dimensional spaces (>100) regardless of the underlying intrinsic dimension of the data manifold.“ • "Those results challenge the validity of our current interpolation/ extrapolation deﬁnition as an indicator of generalization performances. "

/ 1/2 d (r= ) <latexit sha1_base64="mchf2gvio17p/Fz9iF84xhnMH3c=">AAAC73ichVHNThRBEK4dVFZAWeBi4mXiBrJ72dQSIsTEhOhBj/ztQkLjpqenZ2mZv53p3WSdzAvwAhxMjJB4ID6AD+DFoxcOPILhCMGLidbOTiJKxJ7M9Fdf1VfzdZcVuirWiKcFY+TW7Tujxbtj4xP37k+WpqabcdCNhGyIwA2iLYvH0lW+bGilXbkVRpJ7lis3rb3ng/xmT0axCvwN3Q/ljsfbvnKU4JqoVul1s2JXzacmcyIuEhaqV8kQ2mkyn6bp1YgJO9DsBfc8zlzp6MrVXKTau7qamkwHJpqs0+W2WbGzkCnf0f1qq1TGGmbLvA7qOShDvlaC0idgYEMAArrggQQfNGEXOMT0bEMdEELidiAhLiKksryEFMZI26UqSRWc2D36tinazlmf4kHPOFML+otLb0RKE2bxBI/xHL/gR/yGP/7ZK8l6DLz0abeGWhm2JvcfrH//r8qjXcPub9WNnjU4sJR5VeQ9zJjBKcRQ33tzcL7+ZG02mcMjPCP/h3iKn+kEfu9CfFiVa29v8GORF7oxGlD973FcB835Wv1xbWF1obz8LB9VER7CI6jQPBZhGV7CCjSo/1e4hJ8FMDrGgfHOeD8sNQq5Zgb+WMbxL1/dugg=</latexit> V (d) =
⇡ d 2 d 2 · d 2 ! 0 (d ! 1) https://www.math.ucdavis.edu/~strohmer/courses/180BigData/180lecture1.pdf <latexit sha1_base64="eWTwm+uSU1d/PxikW7VncIaHMRY=">AAAChHichVHLSsNAFD2Nr/ps1Y3gRiwVF1JutD5wIUU3LltrVdAiSZzW0DQJSVqoxR/QreLClYIL8QP8ADf+gAs/QVxWcOPC2zQgWtQbJnPmzD13zsxVbUN3PaLnkNTR2dXdE+7t6x8YHIpEh0e2XKviaCKnWYbl7KiKKwzdFDlP9wyxYztCKauG2FZLa8397apwXN0yN72aLfJlpWjqBV1TPKYy8n40RgnyY6IdyAGIIYi0Fb3HHg5gQUMFZQiY8BgbUODytwsZBJu5POrMOYx0f1/gGH2srXCW4AyF2RL/i7zaDViT182arq/W+BSDh8PKCcTpiW6pQY90Ry/08Wutul+j6aXGs9rSCns/cjKWff9XVebZw+GX6k/PHgpY8r3q7N32meYttJa+enTRyC5vxOtTdE2v7P+KnumBb2BW37SbjNi4/MOPyl74xbhB8s92tIOt2YS8kEhmkrHUatCqMMYxiWnuxyJSWEcaOa4vcIoznEvd0ow0J823UqVQoBnFt5BWPgE/Y4+x</latexit> 1 <latexit sha1_base64="z3Ba/qsiF6tEgbjc9DFSGYR9udI=">AAACjnichVHNSgJRFD5Of2Y/Wm2CNpIYreQYZhFEUhuX/uQPqMjMeLPBcWaauQomvkD7aBEUBS2iB+gB2vQCLXyEaGnQpkXHcSBKsjPcud/97vnO/e49kqEqFkfsuoSx8YnJKfe0Z2Z2bt7rW1jMWnrDlFlG1lXdzEuixVRFYxmucJXlDZOJdUllOal20N/PNZlpKbp2yFsGK9XFqqYcKbLIiSpgaLNonZi8XemUfQEMoR3+YRB2QACcSOi+RyhCBXSQoQF1YKABJ6yCCBZ9BQgDgkFcCdrEmYQUe59BBzykbVAWowyR2Br9q7QqOKxG635Ny1bLdIpKwySlH4L4gvfYw2d8wFf8/LNW267R99KiWRpomVH2ni2nP/5V1WnmcPytGumZwxFs214V8m7YTP8W8kDfPL3opXdSwfYa3uIb+b/BLj7RDbTmu3yXZKnLEX4k8kIvRg0K/27HMMhuhMLRUCQZCcT2nVa5YQVWYZ36sQUxiEMCMvaLnsMVXAs+ISrsCnuDVMHlaJbgRwjxLwzik/k=</latexit> 0.5 p d <latexit sha1_base64="rzd8+rC/ag8/gv3jUp4Y6gUrSiI=">AAAChnichVHLSsNAFD3GV62PVt0IbopFcRVupVVxVXTjsg9rC1okidMamiYhSQta/AHBrV24UnAhfoAf4MYfcOEniEsFNy68TQOixXrDZM6cuefOmbmqbeiuR/Q8IA0ODY+MhsbC4xOTU5Ho9MyuazUcTRQ0y7Cckqq4wtBNUfB0zxAl2xFKXTVEUa1tdfaLTeG4umXueMe2KNeVqqlXdE3xmMqTnDqIxkkmP2K9IBGAOILIWNF77OMQFjQ0UIeACY+xAQUuf3tIgGAzV0aLOYeR7u8LnCLM2gZnCc5QmK3xv8qrvYA1ed2p6fpqjU8xeDisjGGRnuiW3uiR7uiFPv+s1fJrdLwc86x2tcI+iJzN5T/+VdV59nD0rerr2UMF675Xnb3bPtO5hdbVN0/ab/mN3GJria7plf1f0TM98A3M5rt2kxW5yz5+VPbCL8YNSvxuRy/YXZETq3Iym4ynN4NWhTCPBSxzP9aQxjYyKHD9Ks5xgbYUkmQpJa11U6WBQDOLHyGlvwBbQpAn</latexit> 0.5 <latexit sha1_base64="Nn/M5/H5iD9Z/hOTHxuHUqAZH+I=">AAAChnichVHLSsNAFD2Nr1ofrboR3BRLxVWZlmpFEIpuXPZhH1BLSdKxhqZJSNJCLf6A4NYuXCm4ED/AD3DjD7joJ4jLCm5ceJsGRIv1hsmcOXPPnTNzJUNVLJuxnkeYmJyanvHO+ubmFxb9gaXlvKU3TZnnZF3VzaIkWlxVNJ6zFVvlRcPkYkNSeUGqHwz2Cy1uWoquHdltg5cbYk1TThRZtInKVvdilUCIRZgTwVEQdUEIbqT0wCOOUYUOGU00wKHBJqxChEVfCVEwGMSV0SHOJKQ4+xzn8JG2SVmcMkRi6/Sv0arkshqtBzUtRy3TKSoNk5RBhNkLu2d99swe2Cv7/LNWx6kx8NKmWRpquVHxX6xmP/5VNWi2cfqtGuvZxgl2HK8KeTccZnALeahvnXX72d1MuLPBbtkb+b9hPfZEN9Ba7/Jdmmeux/iRyAu9GDUo+rsdoyAfi0S3I/F0PJTcd1vlxRrWsUn9SCCJQ6SQo/o1XOIKXcErRIQtITFMFTyuZgU/Qkh+AeO5kGc=</latexit> d = 2 <latexit sha1_base64="LVcqe4i4MA2A5XBAmTvFXnqLayw=">AAAChnichVHLSsNAFD2Nr1ofrboR3BRLxVWZSrUiCEU3LvuwD6ilJOm0hqZJSNJCLf6A4NYuXCm4ED/AD3DjD7joJ4jLCm5ceJsGRIv1hsmcOXPPnTNzJUNVLJuxnkeYmJyanvHO+ubmFxb9gaXlnKU3TZlnZV3VzYIkWlxVNJ61FVvlBcPkYkNSeV6qHw728y1uWoquHdttg5caYk1Tqoos2kRlKvuxciDEIsyJ4CiIuiAEN5J64BEnqECHjCYa4NBgE1YhwqKviCgYDOJK6BBnElKcfY5z+EjbpCxOGSKxdfrXaFV0WY3Wg5qWo5bpFJWGScogwuyF3bM+e2YP7JV9/lmr49QYeGnTLA213Cj7L1YzH/+qGjTbOP1WjfVso4pdx6tC3g2HGdxCHupbZ91+Zi8d7mywW/ZG/m9Yjz3RDbTWu3yX4unrMX4k8kIvRg2K/m7HKMhtRaI7kVgqFkocuK3yYg3r2KR+xJHAEZLIUv0aLnGFruAVIsK2EB+mCh5Xs4IfISS+AOf5kGk=</latexit> d = 4 <latexit sha1_base64="eWTwm+uSU1d/PxikW7VncIaHMRY=">AAAChHichVHLSsNAFD2Nr/ps1Y3gRiwVF1JutD5wIUU3LltrVdAiSZzW0DQJSVqoxR/QreLClYIL8QP8ADf+gAs/QVxWcOPC2zQgWtQbJnPmzD13zsxVbUN3PaLnkNTR2dXdE+7t6x8YHIpEh0e2XKviaCKnWYbl7KiKKwzdFDlP9wyxYztCKauG2FZLa8397apwXN0yN72aLfJlpWjqBV1TPKYy8n40RgnyY6IdyAGIIYi0Fb3HHg5gQUMFZQiY8BgbUODytwsZBJu5POrMOYx0f1/gGH2srXCW4AyF2RL/i7zaDViT182arq/W+BSDh8PKCcTpiW6pQY90Ry/08Wutul+j6aXGs9rSCns/cjKWff9XVebZw+GX6k/PHgpY8r3q7N32meYttJa+enTRyC5vxOtTdE2v7P+KnumBb2BW37SbjNi4/MOPyl74xbhB8s92tIOt2YS8kEhmkrHUatCqMMYxiWnuxyJSWEcaOa4vcIoznEvd0ow0J823UqVQoBnFt5BWPgE/Y4+x</latexit> 1 <latexit sha1_base64="UR9ymFZxc/Q/TCzOORzighhjAlM=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCKDQTEuDNGNSx7ySJCQtgzYUNqmLSRI/AETt7JwpYkL4wf4AW78ARd8gnGJiRsXXkoTo0S8zXTOnLnnzpm5kqEqls1YzyNMTE5Nz3hnfXPzC4v+wNJyztKbpsyzsq7qZkESLa4qGs/aiq3ygmFysSGpPC/VDwf7+RY3LUXXju22wUsNsaYpVUUWbaIylf1YORBiEeZEcBREXRCCG0k98IgTVKBDRhMNcGiwCasQYdFXRBQMBnEldIgzCSnOPsc5fKRtUhanDJHYOv1rtCq6rEbrQU3LUct0ikrDJGUQYfbC7lmfPbMH9so+/6zVcWoMvLRploZabpT9F6uZj39VDZptnH6rxnq2UcWu41Uh74bDDG4hD/Wts24/s5cOdzbYLXsj/zesx57oBlrrXb5L8fT1GD8SeaEXowZFf7djFOS2ItGdSCwVCyUO3FZ5sYZ1bFI/4kjgCElkqX4Nl7hCV/AKEWFbiA9TBY+rWcGPEBJf6hqQag==</latexit> d > 4 <latexit sha1_base64="nHSiJ7YlA7Y9nChVeD/tJhyz2yo=">AAACjnichVHNSgJRFD5Of2Y/Wm2CNpIYreQoZhFEUhuX/uQPqMjMdLPBcWaauQomvkD7aBEUBS2iB+gB2vQCLXyEaGnQpkXHcSBKsjPcud/97vnO/e49kqEqFkfsuoSx8YnJKfe0Z2Z2bt7rW1jMWXrDlFlW1lXdLEiixVRFY1mucJUVDJOJdUlleam239/PN5lpKbp2wFsGK9fFqqYcKbLIiSpiaKNknZi8HelUfAEMoR3+YRB2QACcSOq+RyjBIeggQwPqwEADTlgFESz6ihAGBIO4MrSJMwkp9j6DDnhI26AsRhkisTX6V2lVdFiN1v2alq2W6RSVhklKPwTxBe+xh8/4gK/4+Wettl2j76VFszTQMqPiPVvOfPyrqtPM4fhbNdIzhyPYsr0q5N2wmf4t5IG+eXrRy2yng+01vMU38n+DXXyiG2jNd/kuxdKXI/xI5IVejBoU/t2OYZCLhMKxUDQVDcT3nFa5YQVWYZ36sQlxSEASsvaLnsMVXAs+ISbsCLuDVMHlaJbgRwiJL6Jhk8c=</latexit> 0.5 p 2 <latexit sha1_base64="z3Ba/qsiF6tEgbjc9DFSGYR9udI=">AAACjnichVHNSgJRFD5Of2Y/Wm2CNpIYreQYZhFEUhuX/uQPqMjMeLPBcWaauQomvkD7aBEUBS2iB+gB2vQCLXyEaGnQpkXHcSBKsjPcud/97vnO/e49kqEqFkfsuoSx8YnJKfe0Z2Z2bt7rW1jMWnrDlFlG1lXdzEuixVRFYxmucJXlDZOJdUllOal20N/PNZlpKbp2yFsGK9XFqqYcKbLIiSpgaLNonZi8XemUfQEMoR3+YRB2QACcSOi+RyhCBXSQoQF1YKABJ6yCCBZ9BQgDgkFcCdrEmYQUe59BBzykbVAWowyR2Br9q7QqOKxG635Ny1bLdIpKwySlH4L4gvfYw2d8wFf8/LNW267R99KiWRpomVH2ni2nP/5V1WnmcPytGumZwxFs214V8m7YTP8W8kDfPL3opXdSwfYa3uIb+b/BLj7RDbTmu3yXZKnLEX4k8kIvRg0K/27HMMhuhMLRUCQZCcT2nVa5YQVWYZ36sQUxiEMCMvaLnsMVXAs+ISrsCnuDVMHlaJbgRwjxLwzik/k=</latexit> 0.5 p d !? Unit Cube in Unit Ball <latexit sha1_base64="eWTwm+uSU1d/PxikW7VncIaHMRY=">AAAChHichVHLSsNAFD2Nr/ps1Y3gRiwVF1JutD5wIUU3LltrVdAiSZzW0DQJSVqoxR/QreLClYIL8QP8ADf+gAs/QVxWcOPC2zQgWtQbJnPmzD13zsxVbUN3PaLnkNTR2dXdE+7t6x8YHIpEh0e2XKviaCKnWYbl7KiKKwzdFDlP9wyxYztCKauG2FZLa8397apwXN0yN72aLfJlpWjqBV1TPKYy8n40RgnyY6IdyAGIIYi0Fb3HHg5gQUMFZQiY8BgbUODytwsZBJu5POrMOYx0f1/gGH2srXCW4AyF2RL/i7zaDViT182arq/W+BSDh8PKCcTpiW6pQY90Ry/08Wutul+j6aXGs9rSCns/cjKWff9XVebZw+GX6k/PHgpY8r3q7N32meYttJa+enTRyC5vxOtTdE2v7P+KnumBb2BW37SbjNi4/MOPyl74xbhB8s92tIOt2YS8kEhmkrHUatCqMMYxiWnuxyJSWEcaOa4vcIoznEvd0ow0J823UqVQoBnFt5BWPgE/Y4+x</latexit> 1 <latexit sha1_base64="PtefnFPcLJximPXvkayt2U9xPRk=">AAAChXichVHLSsNAFD2Nr1pfVTeCm2KpuLHcSFFxoUU3LvuwWqhFkjjVYJqEJC1o8QfErbpwpeBC/AA/wI0/4KKfIC4ruHHhbRoQFfWGyZw5c8+dM3NV29Bdj6gZkrq6e3r7wv2RgcGh4ZHo6Nima9UcTRQ0y7Ccoqq4wtBNUfB0zxBF2xFKVTXElnqw1t7fqgvH1S1zwzu0Rbmq7Jl6RdcUj6ncsrwTjVOS/Ij9BHIA4ggiY0XvsY1dWNBQQxUCJjzGBhS4/JUgg2AzV0aDOYeR7u8LHCPC2hpnCc5QmD3g/x6vSgFr8rpd0/XVGp9i8HBYGUOCnuiWWvRId/RM77/Wavg12l4OeVY7WmHvjJxM5N/+VVV59rD/qfrTs4cKFn2vOnu3faZ9C62jrx9dtPJLuURjmq7phf1fUZMe+AZm/VW7yYrc5R9+VPbCL8YNkr+34yfYnEvK88lUNhVPrwatCmMSU5jhfiwgjXVkUOD6FZziDOdSnzQrpaT5TqoUCjTj+BLSygfo4Y/5</latexit> > 1 !? ( ⾒) 0

/ 2/2 Bronstein MM, Bruna J, Cohen T, Veličković P.
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. arXiv [cs.LG]. 2021. http://arxiv.org/abs/2104.13478 for all <latexit sha1_base64="NqBYbofjRLhsRuy5gs3o5FfAjVo=">AAAConichVG7SgNBFD1Z3/EVtRFECEbFKkwkqKQK2ohVHkaFJIbddRKX7IvdSUCDlZ0/YGGlYCF2WvgBNv6ART5BLBVsLLzZLAQN6l1m58yZe+6cmavYuuYKxpoBqae3r39gcCg4PDI6Nh6amNxxrZqj8pxq6Zazp8gu1zWT54QmdL5nO1w2FJ3vKtWN1v5unTuuZpnb4sjmRUOumFpZU2VBVCk0W04UDFkcKkojc7J/EC4IK9whSqEIizIvwt0g5oMI/EhZoQcUcAALKmowwGFCENYhw6UvjxgYbOKKaBDnENK8fY4TBElboyxOGTKxVfpXaJX3WZPWrZqup1bpFJ2GQ8owFtgzu2Fv7Indshf2+Wuthlej5eWIZqWt5XZp/Gw6+/GvyqBZ4LCj+tOzQBlrnleNvNse07qF2tbXj8/fsonMQmORXbFX8n/JmuyRbmDW39XrNM9c/OFHIS/0YtSg2M92dIOd5WhsJRpPxyPJdb9Vg5jBHJaoH6tIYhMp5Kj+KW5wh3tpXtqS0lK2nSoFfM0UvoVU+AKNf5vS</latexit> f : Rd ! R <latexit sha1_base64="bLp7Jqe5Ke+bRuLIs0xt/zwFMHo=">AAACmnichVHLLgRBFD3T3u9BJBIWExOPhUxqRBCrCRtiM8a8EsOkuxUq+pXumgk68wN+wMKKxAIf4ANs/ICFTxBLEhsLd3o6EQS3U12nTt1z61RdzTGEJxl7jChNzS2tbe0dnV3dPb190f6BvGdXXJ3ndNuw3aKmetwQFs9JIQ1edFyumprBC9rBcn2/UOWuJ2wrK48cvmWqe5bYFboqiSpHhw6nDydjJWHFSqYq9zXNz9S2d8rROEuwIGI/QTIEcYSRtqO3KGEHNnRUYILDgiRsQIVH3yaSYHCI24JPnEtIBPscNXSStkJZnDJUYg/ov0erzZC1aF2v6QVqnU4xaLikjGGcPbAr9sLu2Q17Yu+/1vKDGnUvRzRrDS13yn0nwxtv/6pMmiX2P1V/epbYxULgVZB3J2Dqt9Ab+urx6cvGYmbcn2AX7Jn8n7NHdkc3sKqv+uU6z5z94UcjL/Ri1KDk93b8BPmZRHIuMbs+G08tha1qxwjGMEX9mEcKK0gjR/V9nOMaN8qosqSsKmuNVCUSagbxJZTsB6MZmAU=</latexit> x, x0 2 Rd -Lipschitz <latexit sha1_base64="VAwN9BEfNuoROOZ0f78ZUuUItlA=">AAACqHichVFNLwNRFD0d39/FRmIz0fhM2ryKIFYNGwuL+qgSFZkZr0y8zoyZ16a0fgB/wMKKxEIsLLG28Qcs/ASxJLGxcDudRBDcybx73nn33Dlvru4I05OMPYaUmtq6+obGpuaW1rb2jnBn17Jn512Dpwxb2O6KrnlcmBZPSVMKvuK4XMvpgqf1nZnKebrAXc+0rSW55/D1nLZlmVnT0CRRG+FIOTtUHI7SMjhcVjOC73pCs6Q6p2bKajFaHKRMVSzG/FB/gngAIggiaYevkcEmbBjIIwcOC5KwgAaPnjXEweAQt44ScS4h0z/nOEAzafNUxalCI3aH1i3arQWsRftKT89XG/QVQa9LShX97IFdsBd2zy7ZE3v/tVfJ71HxskdZr2q5s9Fx1LP49q8qR1li+1P1p2eJLCZ9ryZ5d3ymcgujqi/sH78sTi30lwbYGXsm/6fskd3RDazCq3E+zxdO/vCjkxf6YzSg+Pdx/ATLo7H4eGxsfiySmA5G1Yhe9GGI5jGBBGaRRIr6H+IKN7hVRpSkklZWq6VKKNB040so+gci3ZxP</latexit> |f(x) f(x0)| 6 Lkx x0k <latexit sha1_base64="+ZiSkRyCDmMX1dfLuFjrVp2j6hw=">AAAChHichVG7SgNBFD1ZNcZ31EawCQbFQsJE4wMLCdpYWJjERCGK7K6TOLjZXXYngRj8AW0VCysFC/ED/AAbf8AinyCWEWwsvNksiAb1LrNz5sw9d87M1WxDuJKxekDp6OwKdod6evv6BwaHwsMjOdcqOzrP6pZhOTua6nJDmDwrhTT4ju1wtaQZfFs7Wmvub1e44wrL3JJVm++V1KIpCkJXJVGpjf1wlMWYF5F2EPdBFH5sWuEH7OIAFnSUUQKHCUnYgAqXvjziYLCJ20ONOIeQ8PY5TtBL2jJlccpQiT2if5FWeZ81ad2s6XpqnU4xaDikjGCSPbM71mBP7J69sI9fa9W8Gk0vVZq1lpbb+0OnY5n3f1UlmiUOv1R/epYoYMnzKsi77THNW+gtfeX4spFZTk/WptgNeyX/16zOHukGZuVNv03x9NUffjTyQi9GDYr/bEc7yM3G4guxRCoRTa76rQphHBOYpn4sIol1bCJL9TnOcI4LJajMKHPKfCtVCfiaUXwLZeUTeMOPzA==</latexit> L Donoho DL, High-dimensional data analysis: The curses and blessings of dimensionality. Plenary Lecture, AMS National Meeting on Mathematical Challenges of the 21st Century. 2000. (Lipschitz ) ⾒ ε ( /ε)d Sobolev class d= 100 d= 10 =100

Underspecification Rashomon ( ) CV Underspeci cation ( specify )
D’Amour et al., Underspeciﬁcation Presents Challenges for Credibility in Modern Machine Learning. J Mach Learn Res, 2022; 23: 1-61. https://ai.googleblog.com/2021/10/how-underspeciﬁcation-presents.html

Underspecification Rashomon Underspeci cation ( specify ) specify

Underspecification Rashomon Underspeci cation ( specify ) specify (Underspeci cation)
α

2 ( big data ) ( )

Kernel Ridge (RBF) Neural Network (MLP) Gradient Boosted Trees SVR
(RBF) Gaussian Process (RBF) Random Forest Nearest Neighbors Decision Tree ⾒ Nearest Neighbor (Neural Network )

= if-then <latexit sha1_base64="KLK3uKDnOLsAzd1uz4imbactEN0=">AAAClnichVHLSsNAFD3Gd3201Y3gplgUV2UiRcWFFEV02YfVgpWQxLENpklMphUt/oA/4EJcKKiIH+AHuPEHXPQTxGUFNy68TQOiot4wmTNn7rlzZq7mmIYnGGt0SJ1d3T29ff2hgcGh4XAkOrLh2VVX53ndNm23oKkeNw2L54UhTF5wXK5WNJNvanvLrf3NGnc9w7bWxaHDtytqyTJ2DV0VRCmRaEGZiRVNvh8rijIXqiIrkThLMD9iP4EcgDiCSNuRexSxAxs6qqiAw4IgbEKFR98WZDA4xG2jTpxLyPD3OY4RIm2VsjhlqMTu0b9Eq62AtWjdqun5ap1OMWm4pIxhkj2xW9Zkj+yOPbP3X2vV/RotL4c0a20td5TwyVju7V9VhWaB8qfqT88Cu5j3vRrk3fGZ1i30tr52dNrMLWQn61Pskr2Q/wvWYA90A6v2ql9lePbsDz8aeaEXowbJ39vxE2zMJOTZRDKTjKeWglb1YRwTmKZ+zCGFNaSRp/oHOMc1bqQxaVFakVbbqVJHoBnFl5DSHzOslho=</latexit> X2  ✓1 yes no <latexit
sha1_base64="fUwPPBtoir3sq20H215QULGdmUg=">AAAClnichVHLSsNAFD3Gd3201Y3gplgUV2UiRcWFFEV02YfVgpWQxLENpklMphUt/oA/4EJcKKiIH+AHuPEHXPQTxGUFNy68TQOiot4wmTNn7rlzZq7mmIYnGGt0SJ1d3T29ff2hgcGh4XAkOrLh2VVX53ndNm23oKkeNw2L54UhTF5wXK5WNJNvanvLrf3NGnc9w7bWxaHDtytqyTJ2DV0VRCmRaEGRY0WT78eKosyFqswokThLMD9iP4EcgDiCSNuRexSxAxs6qqiAw4IgbEKFR98WZDA4xG2jTpxLyPD3OY4RIm2VsjhlqMTu0b9Eq62AtWjdqun5ap1OMWm4pIxhkj2xW9Zkj+yOPbP3X2vV/RotL4c0a20td5TwyVju7V9VhWaB8qfqT88Cu5j3vRrk3fGZ1i30tr52dNrMLWQn61Pskr2Q/wvWYA90A6v2ql9lePbsDz8aeaEXowbJ39vxE2zMJOTZRDKTjKeWglb1YRwTmKZ+zCGFNaSRp/oHOMc1bqQxaVFakVbbqVJHoBnFl5DSHzOelho=</latexit> X1  ✓2 yes no Blue <latexit sha1_base64="TQ4GrszeMgM294G1rFaPjchY/V8=">AAAClnichVHLSsNAFD2N7/qquim4KZaKqzItRcWFiCK6rNZqoZWQxGkbTJOYTCta/AF/wIW4UFARP8APcOMPuOgniMsKblx4mwZExXrDZM6cuefOmbmqbeiuYKwRkLq6e3r7+geCg0PDI6OhsfFt16o6Gs9qlmE5OVVxuaGbPCt0YfCc7XClohp8R91fae3v1Ljj6pa5JY5svltRSqZe1DVFECWHxnJyMlIw+EGkIMpcKHJKDkVZnHkR+Q0SPojCj7QVekABe7CgoYoKOEwIwgYUuPTlkQCDTdwu6sQ5hHRvn+MEQdJWKYtThkLsPv1LtMr7rEnrVk3XU2t0ikHDIWUEMfbM7liTPbF79sI+/qxV92q0vBzRrLa13JZHT8OZ939VFZoFyl+qjp4Fipj3vOrk3faY1i20tr52fNbMLGzG6tPsir2S/0vWYI90A7P2pl1v8M3zDn5U8kIvRg1K/GzHb7CdjCdm46mNVHRp2W9VPyYxhRnqxxyWsI40slT/EBe4wa0UlhalVWmtnSoFfM0EvoWU/gQ6DJYd</latexit> X2  ✓4 yes no Red <latexit sha1_base64="FaDvqDDJ/DHLyBEhkovQS4NqCn0=">AAAClnichVHLSsNAFD3GV62vVjeCm2JRXJWJ+MKFFEV0WVurBSshiVMNTZOYTCta/AF/wIW4UFARP8APcOMPuOgniEsFNy68SQOiot4wmTNn7rlzZq7mmIYnGGu0SK1t7R2dka5od09vX38sPrDu2VVX53ndNm23oKkeNw2L54UhTF5wXK5WNJNvaOVFf3+jxl3PsK01ceDwrYq6YxklQ1cFUUosXlDkRNHke4mi2OVCVaaUWJKlWBCJn0AOQRJhZOzYHYrYhg0dVVTAYUEQNqHCo28TMhgc4rZQJ84lZAT7HEeIkrZKWZwyVGLL9N+h1WbIWrT2a3qBWqdTTBouKRMYZY/shr2wB3bLntj7r7XqQQ3fywHNWlPLHaX/eCj39q+qQrPA7qfqT88CJcwGXg3y7gSMfwu9qa8dnrzk5rKj9TF2wZ7J/zlrsHu6gVV71S9Xefb0Dz8aeaEXowbJ39vxE6xPpOTp1OTqZDK9ELYqgmGMYJz6MYM0VpBBnurv4wxXuJaGpHlpSVpupkotoWYQX0LKfAA5/pYd</latexit> X1  ✓5 yes no Red <latexit sha1_base64="j6aRObEml8aKBOmlM9//dP7CY5I=">AAAClnichVHLSsNAFD3Gd321uhHcFIviqkykVHEhRRFd9mG1YCUkcWyDaRKTaUWLP+APuBAXCiriB/gBbvwBF/0EcVnBjQtv04CoqDdM5syZe+6cmas5puEJxhodUmdXd09vX39oYHBoeCQcGd3w7Kqr87xum7Zb0FSPm4bF88IQJi84Llcrmsk3tb3l1v5mjbueYVvr4tDh2xW1ZBm7hq4KopRwpKDI0aLJ96NFUeZCVZJKOMbizI/oTyAHIIYg0nb4HkXswIaOKirgsCAIm1Dh0bcFGQwOcduoE+cSMvx9jmOESFulLE4ZKrF79C/RaitgLVq3anq+WqdTTBouKaOYYk/sljXZI7tjz+z911p1v0bLyyHNWlvLHWXkZDz39q+qQrNA+VP1p2eBXcz7Xg3y7vhM6xZ6W187Om3mFrJT9Wl2yV7I/wVrsAe6gVV71a8yPHv2hx+NvNCLUYPk7+34CTZm43IynsgkYqmloFV9mMAkZqgfc0hhDWnkqf4BznGNG2lcWpRWpNV2qtQRaMbwJaT0Bzwelh4=</latexit> X1  ✓6 yes no Red Blue <latexit sha1_base64="3d4/BR7YXLLt+AHkvejt6OKrM6I=">AAAClnichVHLSsNAFD3GV3221Y3gplgUV2WiRcWFFEV0Wa3VgpWQxKmGpklMphUt/oA/4EJcKKiIH+AHuPEHXPQTxKWCGxfepAFRUW+YzJkz99w5M1dzTMMTjDVapNa29o7OSFd3T29ffzQWH1j37Kqr87xum7Zb0FSPm4bF88IQJi84Llcrmsk3tPKCv79R465n2NaaOHD4VkXdsYySoauCKCUWLyhyomjyvURR7HKhKpNKLMlSLIjETyCHIIkwsnbsDkVsw4aOKirgsCAIm1Dh0bcJGQwOcVuoE+cSMoJ9jiN0k7ZKWZwyVGLL9N+h1WbIWrT2a3qBWqdTTBouKRMYZY/shr2wB3bLntj7r7XqQQ3fywHNWlPLHSV6PJR7+1dVoVlg91P1p2eBEmYCrwZ5dwLGv4Xe1NcOT15ys6uj9TF2wZ7J/zlrsHu6gVV71S9X+OrpH3408kIvRg2Sv7fjJ1ifSMlTqfRKOpmZD1sVwTBGME79mEYGy8giT/X3cYYrXEtD0py0KC01U6WWUDOILyFlPwA1vpYb</latexit> X1  ✓3 yes no Red <latexit sha1_base64="bG79VND5DJn1e15l/47tj0nzmbk=">AAAClnichVHLSsNAFD3Gd321uhHcFIviqkykWHEhRRFd9mG1YCUkcWyDaRKTaUWLP+APuBAXCiriB/gBbvwBF/0EcVnBjQtv04CoqDdM5syZe+6cmas5puEJxhodUmdXd09vX39oYHBoeCQcGd3w7Kqr87xum7Zb0FSPm4bF88IQJi84Llcrmsk3tb3l1v5mjbueYVvr4tDh2xW1ZBm7hq4KopRwpKDI0aLJ96NFUeZCVZJKOMbizI/oTyAHIIYg0nb4HkXswIaOKirgsCAIm1Dh0bcFGQwOcduoE+cSMvx9jmOESFulLE4ZKrF79C/RaitgLVq3anq+WqdTTBouKaOYYk/sljXZI7tjz+z911p1v0bLyyHNWlvLHWXkZDz39q+qQrNA+VP1p2eBXcz7Xg3y7vhM6xZ6W187Om3mFrJT9Wl2yV7I/wVrsAe6gVV71a8yPHv2hx+NvNCLUYPk7+34CTZm4/JcPJFJxFJLQav6MIFJzFA/kkhhDWnkqf4BznGNG2lcWpRWpNV2qtQRaMbwJaT0Bz4+lh8=</latexit> X1  ✓7 yes no Blue Blue if x2 ≤ θ1 then if x1 ≤ θ2 then return Blue else if x2 ≤ θ4 then return Red else if x1 ≤ θ5 then return Red else if x1 ≤ θ6 then return Blue else return Red else if x1 ≤ θ3 then if x2 ≤ θ7 then return Blue else return Blue else return Red <latexit sha1_base64="tvPTQvEnR42QahYYPRslvA0PAkQ=">AAAB+XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5CUMugjWVE8wHJEfY2e8mS3b1jd08IR36CrdZ2YuuvsfSfuLlcYRIfDDzem2FmXhBzpo3rfjuFjc2t7Z3ibmlv/+DwqHx80tZRoghtkYhHqhtgTTmTtGWY4bQbK4pFwGknmNzN/c4zVZpF8slMY+oLPJIsZAQbKz12B7VBueJW3QxonXg5qUCO5qD80x9GJBFUGsKx1j3PjY2fYmUY4XRW6ieaxphM8Ij2LJVYUO2n2akzdGGVIQojZUsalKl/J1IstJ6KwHYKbMZ61ZuL/3m9xIQ3fspknBgqyWJRmHBkIjT/Gw2ZosTwqSWYKGZvRWSMFSbGprO0JRAzm4m3msA6adeq3lW1/lCvNG7zdIpwBudwCR5cQwPuoQktIDCCF3iFNyd13p0P53PRWnDymVNYgvP1C4sglCA=</latexit> X2 <latexit sha1_base64="wDeZ5yQA5kysGCdsNSdw9QhGcj8=">AAAB+XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKqMegF48RzQOSJcxOZpMh81hmZoWw5BO86tmbePVrPPonTpI9mMSChqKqm+6uKOHMWN//9gpr6xubW8Xt0s7u3v5B+fCoaVSqCW0QxZVuR9hQziRtWGY5bSeaYhFx2opGd1O/9Uy1YUo+2XFCQ4EHksWMYOukx3Yv6JUrftWfAa2SICcVyFHvlX+6fUVSQaUlHBvTCfzEhhnWlhFOJ6VuamiCyQgPaMdRiQU1YTY7dYLOnNJHsdKupEUz9e9EhoUxYxG5ToHt0Cx7U/E/r5Pa+CbMmExSSyWZL4pTjqxC079Rn2lKLB87golm7lZEhlhjYl06C1siMXGZBMsJrJLmRTW4ql4+XFZqt3k6RTiBUziHAK6hBvdQhwYQGMALvMKbl3nv3of3OW8tePnMMSzA+/oFiY2UHw==</latexit> X1 <latexit sha1_base64="FVULVmCNg47QKtvg46uxkYeCQJE=">AAACi3ichVG7SgNBFL1ZXzEaE7URbIIhYhVmNagEi6AIlnmYByQh7K5jMmRf7E4CMfgDljYWsVGwED/AD7DxByzyCWIZwcbCu5sF0WC8y+ycOXPPnTNzZVNlNiek7xMmJqemZ/yzgbn54EIovLhUsI2WpdC8YqiGVZIlm6pMp3nOuEpLpkUlTVZpUW4eOPvFNrVsZujHvGPSqibVdXbKFIkjVarwBuVSTayFoyRO3IiMAtEDUfAibYQfoQInYIACLdCAgg4csQoS2PiVQQQCJnJV6CJnIWLuPoVzCKC2hVkUMyRkm/iv46rssTqunZq2q1bwFBWHhcoIxMgLuScD8kweyCv5/LNW163heOngLA+11KyFLlZyH/+qNJw5NL5VYz1zOIVd1ytD76bLOLdQhvr22dUgl8zGuuvklryh/xvSJ094A739rtxlaLY3xo+MXvDFsEHi73aMgsJmXNyOJzKJaGrfa5UfVmENNrAfO5CCI0hD3u3DJfTgWggKW0JS2BumCj5Psww/Qjj8Ak3rksg=</latexit> ✓1 <latexit sha1_base64="OBBuUzKAHrHcbt2xZlrMzfgelZg=">AAACi3ichVG7TgJBFL2sL0QR1MbEhkgwVmRAooZYEI2JJQ95JEDI7jrAhH1ldyBB4g9Y2lhgo4mF8QP8ABt/wIJPMJaY2Fh4d9nEKBHvZnbOnLnnzpm5kqEwixMy8AhT0zOzc95538KifykQXF4pWHrblGle1hXdLEmiRRWm0TxnXKElw6SiKim0KLUO7f1ih5oW07UT3jVoVRUbGqszWeRIlSq8SblYi9eCYRIlToTGQcwFYXAjrQcfoQKnoIMMbVCBggYcsQIiWPiVIQYEDOSq0EPORMScfQrn4ENtG7MoZojItvDfwFXZZTVc2zUtRy3jKQoOE5UhiJAXck+G5Jk8kFfy+WetnlPD9tLFWRppqVELXKzlPv5VqThzaH6rJnrmUIc9xytD74bD2LeQR/rO2dUwl8xGepvklryh/xsyIE94A63zLt9laLY/wY+EXvDFsEGx3+0YB4V4NLYTTWQS4dSB2yovrMMGbGE/diEFx5CGvNOHS+jDteAXtoWksD9KFTyuZhV+hHD0BVALksk=</latexit> ✓2 <latexit sha1_base64="587jdLEUJKtkMM/L+3DOcHwPLKw=">AAACi3ichVG7SgNBFL1ZXzEaE7URbIIhYhVmNagEi6AIlnmYByQh7K5jMmRf7E4CMfgDljYWsVGwED/AD7DxByzyCWIZwcbCu5sF0WC8y+ycOXPPnTNzZVNlNiek7xMmJqemZ/yzgbn54EIovLhUsI2WpdC8YqiGVZIlm6pMp3nOuEpLpkUlTVZpUW4eOPvFNrVsZujHvGPSqibVdXbKFIkjVarwBuVSLVELR0mcuBEZBaIHouBF2gg/QgVOwAAFWqABBR04YhUksPErgwgETOSq0EXOQsTcfQrnEEBtC7MoZkjINvFfx1XZY3VcOzVtV63gKSoOC5URiJEXck8G5Jk8kFfy+WetrlvD8dLBWR5qqVkLXazkPv5VaThzaHyrxnrmcAq7rleG3k2XcW6hDPXts6tBLpmNddfJLXlD/zekT57wBnr7XbnL0GxvjB8ZveCLYYPE3+0YBYXNuLgdT2QS0dS+1yo/rMIabGA/diAFR5CGvNuHS+jBtRAUtoSksDdMFXyeZhl+hHD4BVRLkss=</latexit> ✓4 <latexit sha1_base64="e77Z2aqe8KZFIqh8R+ZYHDOKN+I=">AAACi3ichVG7TgJBFL2sL0QR1MbEhkgwVmRQfIRYEI2JJQ95JEDI7jrAhH1ldyBB4g9Y2lhgo4mF8QP8ABt/wIJPMJaY2Fh4d9nEKBHvZnbOnLnnzpm5kqEwixPS9wgTk1PTM95Z39y8fyEQXFzKW3rLlGlO1hXdLEqiRRWm0RxnXKFFw6SiKim0IDUP7f1Cm5oW07UT3jFoRRXrGqsxWeRIFcu8QblY3a4GwyRKnAiNgpgLwuBGSg8+QhlOQQcZWqACBQ04YgVEsPArQQwIGMhVoIuciYg5+xTOwYfaFmZRzBCRbeK/jquSy2q4tmtajlrGUxQcJipDECEv5J4MyDN5IK/k889aXaeG7aWDszTUUqMauFjJfvyrUnHm0PhWjfXMoQZ7jleG3g2HsW8hD/Xts6tBNpGJdNfJLXlD/zekT57wBlr7Xb5L00xvjB8JveCLYYNiv9sxCvKb0dhONJ6Oh5MHbqu8sAprsIH92IUkHEMKck4fLqEH14Jf2BISwv4wVfC4mmX4EcLRF1Zrksw=</latexit> ✓5 <latexit sha1_base64="eu7hkk7xInDVK5lVQXXLPV+MjYw=">AAACi3ichVG7TgJBFL2sL0QR1MbEhkgwVmRQgoZYEI2JJQ95JEDI7jrAhH1ldyBB4g9Y2lhgo4mF8QP8ABt/wIJPMJaY2Fh4d9nEKBHvZnbOnLnnzpm5kqEwixMy8AhT0zOzc95538KifykQXF4pWHrblGle1hXdLEmiRRWm0TxnXKElw6SiKim0KLUO7f1ih5oW07UT3jVoVRUbGqszWeRIlSq8SblYS9SCYRIlToTGQcwFYXAjrQcfoQKnoIMMbVCBggYcsQIiWPiVIQYEDOSq0EPORMScfQrn4ENtG7MoZojItvDfwFXZZTVc2zUtRy3jKQoOE5UhiJAXck+G5Jk8kFfy+WetnlPD9tLFWRppqVELXKzlPv5VqThzaH6rJnrmUIc9xytD74bD2LeQR/rO2dUwl8xGepvklryh/xsyIE94A63zLt9laLY/wY+EXvDFsEGx3+0YB4XtaCwRjWfi4dSB2yovrMMGbGE/diEFx5CGvNOHS+jDteAXdoSksD9KFTyuZhV+hHD0BViLks0=</latexit> ✓6 <latexit sha1_base64="YU9dEL0zbRAMaEFNhHkGnXZKQpA=">AAACi3ichVG7TgJBFL2sL0QR1MbEhkgwVmQQooZYEI2JJQ95JEDI7jrAhH1ldyBB4g9Y2lhgo4mF8QP8ABt/wIJPMJaY2Fh4d9nEKBHvZnbOnLnnzpm5kqEwixMy8AhT0zOzc95538KifykQXF4pWHrblGle1hXdLEmiRRWm0TxnXKElw6SiKim0KLUO7f1ih5oW07UT3jVoVRUbGqszWeRIlSq8SblYi9eCYRIlToTGQcwFYXAjrQcfoQKnoIMMbVCBggYcsQIiWPiVIQYEDOSq0EPORMScfQrn4ENtG7MoZojItvDfwFXZZTVc2zUtRy3jKQoOE5UhiJAXck+G5Jk8kFfy+WetnlPD9tLFWRppqVELXKzlPv5VqThzaH6rJnrmUIc9xytD74bD2LeQR/rO2dUwl8xGepvklryh/xsyIE94A63zLt9laLY/wY+EXvDFsEGx3+0YB4XtaGwnmsgkwqkDt1VeWIcN2MJ+7EIKjiENeacPl9CHa8EvxIWksD9KFTyuZhV+hHD0BVIrkso=</latexit> ✓3 <latexit sha1_base64="VJpvibITWQ0L0zaDACUdk6SsPiA=">AAACi3ichVG7TgJBFL2sL0QR1MbEhkgwVmRQIoZYEI2JJQ95JEDI7jrAhH1ldyBB4g9Y2lhgo4mF8QP8ABt/wIJPMJaY2Fh4d9nEKBHvZnbOnLnnzpm5kqEwixMy8AhT0zOzc95538KifykQXF4pWHrblGle1hXdLEmiRRWm0TxnXKElw6SiKim0KLUO7f1ih5oW07UT3jVoVRUbGqszWeRIlSq8SblYS9SCYRIlToTGQcwFYXAjrQcfoQKnoIMMbVCBggYcsQIiWPiVIQYEDOSq0EPORMScfQrn4ENtG7MoZojItvDfwFXZZTVc2zUtRy3jKQoOE5UhiJAXck+G5Jk8kFfy+WetnlPD9tLFWRppqVELXKzlPv5VqThzaH6rJnrmUIc9xytD74bD2LeQR/rO2dUwl8xGepvklryh/xsyIE94A63zLt9laLY/wY+EXvDFsEGx3+0YB4XtaGw3Gs/Ew6kDt1VeWIcN2MJ+JCAFx5CGvNOHS+jDteAXdoSksD9KFTyuZhV+hHD0BVqrks4=</latexit> ✓7 <latexit sha1_base64="wDeZ5yQA5kysGCdsNSdw9QhGcj8=">AAAB+XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKqMegF48RzQOSJcxOZpMh81hmZoWw5BO86tmbePVrPPonTpI9mMSChqKqm+6uKOHMWN//9gpr6xubW8Xt0s7u3v5B+fCoaVSqCW0QxZVuR9hQziRtWGY5bSeaYhFx2opGd1O/9Uy1YUo+2XFCQ4EHksWMYOukx3Yv6JUrftWfAa2SICcVyFHvlX+6fUVSQaUlHBvTCfzEhhnWlhFOJ6VuamiCyQgPaMdRiQU1YTY7dYLOnNJHsdKupEUz9e9EhoUxYxG5ToHt0Cx7U/E/r5Pa+CbMmExSSyWZL4pTjqxC079Rn2lKLB87golm7lZEhlhjYl06C1siMXGZBMsJrJLmRTW4ql4+XFZqt3k6RTiBUziHAK6hBvdQhwYQGMALvMKbl3nv3of3OW8tePnMMSzA+/oFiY2UHw==</latexit> X1 <latexit sha1_base64="tvPTQvEnR42QahYYPRslvA0PAkQ=">AAAB+XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5CUMugjWVE8wHJEfY2e8mS3b1jd08IR36CrdZ2YuuvsfSfuLlcYRIfDDzem2FmXhBzpo3rfjuFjc2t7Z3ibmlv/+DwqHx80tZRoghtkYhHqhtgTTmTtGWY4bQbK4pFwGknmNzN/c4zVZpF8slMY+oLPJIsZAQbKz12B7VBueJW3QxonXg5qUCO5qD80x9GJBFUGsKx1j3PjY2fYmUY4XRW6ieaxphM8Ij2LJVYUO2n2akzdGGVIQojZUsalKl/J1IstJ6KwHYKbMZ61ZuL/3m9xIQ3fspknBgqyWJRmHBkIjT/Gw2ZosTwqSWYKGZvRWSMFSbGprO0JRAzm4m3msA6adeq3lW1/lCvNG7zdIpwBudwCR5cQwPuoQktIDCCF3iFNyd13p0P53PRWnDymVNYgvP1C4sglCA=</latexit> X2

⾒ d 1 ⾒ ( )

( ) RandomForestRegressor (n_estimators=6, max_leaf_nodes=8) 6 × DecisionTreeRegressor(max_leaf_nodes=8) = +
+ + + + 8 ⾒ 8 ( ) min_samples_leaf

/ ⾒ PolyReg(1) RMSE 0.299 PolyReg(3) RMSE 0.28 PolyReg(5) RMSE
0.225 PolyReg(7) RMSE 0.113 PolyReg(10) RMSE 0.0189 PolyReg(15) RMSE 0.00737 PolyReg(20) RMSE 0.000 PolyReg(30) RMSE 0.000 ExtraTrees (no bootstrap) RMSE 0.000 ExtraTrees (bootstrap) RMSE 0.0121 Random Forest RMSE 0.012 LGBM RMSE 0.0508 95%༧ଌ۠ؒ 95%༧ଌ۠ؒ 95%༧ଌ۠ؒ 95%༧ଌ۠ؒ Problematic overﬁtting by polynomial regression of order k clearly overﬁtted but harmless (still informative)

Interpolator 0 ( ) ExtraTrees (no bootstrap) ExtraTrees (no bootstrap)
ExtraTrees (no bootstrap) ExtraTrees (no bootstrap) Gradient Boosted Trees Gradient Boosted Trees Gradient Boosted Trees Gradient Boosted Trees

Interpolator Nearest Neighbor (k=1) Nearest Neighbor (k=1) Nearest Neighbor (k=1)
Nearest Neighbor (k=1) Decision Tree Decision Tree Decision Tree Decision Tree 0 ( GBDT, -NN, DT )

Extra Trees Geurts, Ernst, Wehenkel, Extremely randomized trees. Mach Learn
63, 3–42 (2006). https://doi.org/10.1007/s10994-006-6226-1 ExtraTreesRegressor(n_estimators=10) RandomForestRegressor(n_estimators=10) ( )

Local-averaging estimators Nearest Neighbor

Local-averaging estimators Nearest Neighbor Nearest Neighbor (X, y) (Stone ,
1977) Nadaraya-Watson Attention Mechanism ICML A Tutorial on Attention in Deep Learning https://icml.cc/Conferences/ /ScheduleMultitrack?event= Histogram rules on data-dependent partitions (or data-driven histogram methods) 90 (Nobel, Ann. Statist. ( ), ; Lugosi & Nobel, Ann. Statist. (2), 1996) ( Yellow Terror) Devroye, Györ , Lugosi, A Probabilistic Theory of Pattern Recognition, . https://www.szit.bme.hu/~gyor /pbook.pdf

Interpolation ⾒ ( 0 or 0) Harmless over tting Benign
over tting ( )

over tting ( ) Hastie et al, Surprises in high-dimensional ridgeless least squares interpolation (2020) https://arxiv.org/abs/1903.08560 Muthukumar et al, Harmless interpolation of noisy data in regression (2019) https://arxiv.org/abs/1903.09139 Bartlett et al, Benign overfitting in linear regression (2019) https://arxiv.org/abs/1906.11300 Belkin et al, Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate (2018) https://arxiv.org/abs/1806.05161 <latexit sha1_base64="V6gnlRc34dPuxi8leGWVweNz3Zg=">AAACG3icbVDLSsNAFJ3UV62vqks3g0VaEUsivjZC0Y3uKtg20MQymUzaoZMHMxMhhHyAH+E3uNW1O3HrwqV/4rTNwloPDBzOOZd75zgRo0Lq+pdWmJtfWFwqLpdWVtfWN8qbW20RxhyTFg5ZyE0HCcJoQFqSSkbMiBPkO4x0nOHVyO88EC5oGNzJJCK2j/oB9ShGUkm9csVyiETwAtbMqgkPoMXUrIvgzf59emhk0KzCRKX0uj4GnCVGTiogR7NX/rbcEMc+CSRmSIiuoUfSThGXFDOSlaxYkAjhIeqTrqIB8omw0/FnMrinFBd6IVcvkHCs/p5IkS9E4jsq6SM5EH+9kfif142ld26nNIhiSQI8WeTFDMoQjpqBLuUES5YogjCn6laIB4gjLFV/U1scP1OdGH8bmCXto7pxWj+5Pa40LvN2imAH7IIaMMAZaIBr0AQtgMEjeAYv4FV70t60d+1jEi1o+cw2mIL2+QPP7p7D</latexit> = (X0X + I) 1X0y <latexit sha1_base64="VsNVIh2b6vsuVSv8LjXaQ8stH3Y=">AAACDnicbVDLSsNAFJ3UV62vqODGzWCRVoSSiK+NUHTjsoJtA20sk+m0HTqZhJmJEGL+wW9wq2t34tZfcOmfOG2zsK0HLhzOuZdzOV7IqFSW9W3kFhaXllfyq4W19Y3NLXN7pyGDSGBSxwELhOMhSRjlpK6oYsQJBUG+x0jTG96M/OYjEZIG/F7FIXF91Oe0RzFSWuqYe22PKASvYNkpOUcPyXEKnRKMO2bRqlhjwHliZ6QIMtQ65k+7G+DIJ1xhhqRs2Vao3AQJRTEjaaEdSRIiPER90tKUI59INxn/n8JDrXRhLxB6uIJj9e9FgnwpY9/Tmz5SAznrjcT/vFakepduQnkYKcLxJKgXMagCOCoDdqkgWLFYE4QF1b9CPEACYaUrm0rx/FR3Ys82ME8aJxX7vHJ2d1qsXmft5ME+OABlYIMLUAW3oAbqAIMn8AJewZvxbLwbH8bnZDVnZDe7YArG1y+Uw5p9</latexit> = (X0X)+X0y Ridgeless Ridge

over tting ( ) Belkin M, Hsu D, Mitra PP. Overfitting or perfect fitting? risk bounds for classification and regression rules that interpolate. NIPS 2018. https://dl.acm.org/doi/10.5555/3327144.3327157 "Many modern machine learning models are trained to achieve zero or near-zero training error in order to obtain near-optimal (but non-zero) test error. This phenomenon of strong generalization performance for "overfitted" / interpolated classifiers appears to be ubiquitous in high-dimensional data, having been observed in deep networks, kernel machines, boosting and random forests. Their performance is consistently robust even when the data contain large amounts of label noise." Belkin M. Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation. Acta Numerica, 2021; 30: 203-248: https://doi.org/10.1017/S0962492921000039 Weyner AJ, Olson M, Bleich J, and Mease D. Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers. J Mach Learn Res. 2017; 18(48): 1-33. https://jmlr.org/papers/v18/15-240.html

(LSH) . ( ) ReLU = max(x, ) (= )
cf.

(LSH) Balestriero, Randall. "Max-Aﬃne Splines Insights Into Deep Learning." (2021)
Diss., Rice University. https://hdl.handle.net/1911/110439 Balestriero & Baraniuk. A Spline Theory of Deep Learning (ICML 2018) https://proceedings.mlr.press/v80/balestriero18b.html Balestriero et al., The Geometry of Deep Networks: Power Diagram Subdivision (NeurIPS 2019) https://arxiv.org/abs/1905.08443 . ( ) ReLU = max(x, ) (= ) cf.

https://www.kaggle.com/kaggle-survey-2021 State of Data Science and Machine Learning 2021 (Kaggle
Survey) Q17. Which of the following ML algorithms do you use on a regular basis? Linear or Logistic Regression Decision Trees or Random Forests Gradient Boosting Machines (xgboost, lightgbm, etc) Convolutional Neural Networks Dense Neural Networks (MLPs, etc) Bayesian Approaches Recurrent Neural Networks Transformer Networks (BERT, gpt-3, etc) None or Other Generative Adversarial Networks Evolutionary Approaches 0 3,500 7,000 10,500 14,000 13,852 11,863 7,566 7,410 4,468 4,392 4,228 2,273 1,953 1,353 963 # respondents: 25,973 ( ⾒ ?)

ExtraTrees Random Forests ( Boosting ⾒ Quantile ) scikit-optimize https://scikit-optimize.github.io/

Conformal Prediction https://github.com/scikit-learn-contrib/MAPIE regressor = ExtraTreesRegressor(max_leaf_nodes=32, bootstrap=True) MapieRegressor(regressor, method="plus", cv=-1)
MapieRegressor(regressor, method="plus", cv=Subsample(n_resampling s=50)) MapieRegressor(regressor, method="plus", cv=-1) MapieRegressor(regressor, method="plus", cv=Subsample(n_resampling s=50)) 95% prediction intervals 95% prediction intervals 90% prediction intervals 90% prediction intervals Jacknife+ Jacknife+ after bootstrap Jacknife+ Jacknife+ after bootstrap Conformal Prediction

SWED-8 SWED-3 Wang et al., Accelerated discovery of multi-elemental reverse
water-gas shift catalysts using extrapolative machine learning approach. (2022) https://doi.org/10.26434/chemrxiv-2022-695rj ( )

CV ( ) CV ( ) f (Xi , y
i ) y i ≈ ̂ yi = f(Xi ) ( ) ⾒ ( )

Physics-informed ML, Geometric ML, Causal ML https://doi.org/10.1038/s42254-021-00314-5

( ) ( ) ⾒ ( ) ⾒⾒

( ) ( ) ⾒ ( ) ⾒⾒ (
Symmetry Geometric ML )

( ) ( ) ⾒ ( ) ⾒⾒ (
Symmetry Geometric ML ) ⾒課 https://doi.org/10.1038/s42254-021-00314-5

( ) X Y Y

. Two Cultures ( vs ) ‣ Data Modeling (
) vs. Algorithmic Modeling ( ) L. Breiman Statistical Modeling: The Two Cultures ( ) + Cox Efron ‣ 20 Observational Studies (2021) + Breiman's main point is: If you want prediction, do prediction for its own sake and forget about the illusion of representing nature. (Judea Pearl)

. Two Cultures ( vs ) ‣ Data Modeling (
) vs. Algorithmic Modeling ( ) L. Breiman Statistical Modeling: The Two Cultures ( ) + Cox Efron ‣ 20 Observational Studies (2021) + Breiman's main point is: If you want prediction, do prediction for its own sake and forget about the illusion of representing nature. (Judea Pearl) . p ( HARKing ) ‣ (A. Lang, ) ‣ 2019 Nature Retire statistical signi cance Don t say statistically signi cant (ASA) (2016) (2019) (800 )

vs (= ) • “Theory-driven models can be wrong. But
data-driven models cannot be wrong or right. Data-driven are not trying to describe an underlying reality.” David Hand, KDD2018 (Keynote Talk) http://videolectures.net/kdd2018_hand_data_science/ ⾒ ( )

⾒ (NP-hard ) Schrittweiser et al. Mastering Atari, Go, chess
and shogi by planning with a learned model. (“MuZero”, Nature, 2020) Davies et al. Advancing mathematics by guiding human intuition with AI. (Nature, 2021) Fawzi et al. Discovering faster matrix multiplication algorithms with reinforcement learning. (“AlphaTensor”, Nature, 2022) × closed /

( ) ( ) ( ) ⾒ (Physics-infomed ML, Causal
ML, Geometric ML, )

. UQ ⾒ . o -the-shelf ( ) Underspeci cation
. ⾒ (Physics-informed ML, Causal ML, Geometric ML) + (Pure Math, Pure CS, ⾒ , / ) PDF https://itakigawa.page.link/IBISML taki

機械学習と機械発見：自然科学研究におけるデータ利活用の再考

機械学習と機械発見：自然科学研究におけるデータ利活用の再考

More Decks by itakigawa

Other Decks in Research

Featured

Transcript