Upgrade to Pro — share decks privately, control downloads, hide ads and more …

機械学習と機械発見:自然科学研究におけるデータ利活用の再考

itakigawa
December 23, 2022

 機械学習と機械発見:自然科学研究におけるデータ利活用の再考

情報論的学習理論と機械学習研究会(IBISML), 京都大学, 2022年12月22日-23日

itakigawa

December 23, 2022
Tweet

More Decks by itakigawa

Other Decks in Research

Transcript

  1. [email protected]
    https://itakigawa.github.io/
    IBISML 2022.12.22 @

    View full-size slide

  2. ( )
    ( )
    / (7 )
    (7 )
    JST (3.5 )
    ( )
    ⾒ (4 )
    (4 ) ( )
    ( )
    AIP ATR ( )

    View full-size slide

  3. Materials Informatics
    (Bioinformatics)
    Materials Informatics
    ( )

    View full-size slide

  4. .
    . o -the-shelf
    .
    PDF https://itakigawa.page.link/IBISML taki

    View full-size slide

  5. .
    . o -the-shelf
    .
    PDF https://itakigawa.page.link/IBISML taki

    View full-size slide


  6. ‣ ( )
    ( )

    View full-size slide

  7. ( )
    AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==
    x⇤
    AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==
    x
    AAACiXichVFNLwNRFD0dX1VfxUZi02iIVXNHBOmq0Y1lP7QkiMyMh9H5ysy0UU3/gJWdYEViIX6AH2DjD1j0J4gliY2F2+kkguBO3rzzzrvnvvPeVR1D93yiVkTq6u7p7Yv2xwYGh4ZH4qNjZc+uupooabZhu+uq4glDt0TJ131DrDuuUEzVEGtqJdveX6sJ19Nta9WvO2LLVPYsfVfXFJ+p8qZqNg6b2/EkpSiIxE8ghyCJMHJ2/A6b2IENDVWYELDgMzagwONvAzIIDnNbaDDnMtKDfYEmYqytcpbgDIXZCv/3eLURshav2zW9QK3xKQYPl5UJTNMj3dALPdAtPdH7r7UaQY22lzrPakcrnO2R44ni278qk2cf+5+qPz372MVS4FVn707AtG+hdfS1o9OXYrow3ZihK3pm/5fUonu+gVV71a7zonDxhx+VvfCLcYPk7+34CcpzKXkhRfn5ZGY5bFUUk5jCLPdjERmsIIcS1z/ACc5wLg1IsrQkpTupUiTUjONLSNkPIVKSSQ==
    x
    AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=
    y
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBGBdk8IVxRXTjkoc8EkTS1gEbStu0hQSJP2DiVhauNHFh/AA/wI0/4IJPMC4xcePCS2lilIi3mc6ZM/fcOTNXMlTFshnreISR0bHxCe+kb2p6ZtYfmJvPWnrdlHlG1lXdzEuixVVF4xlbsVWeN0wu1iSV56Tqfm8/1+Cmpejaod00eLEmVjSlrMiiTVS6ebxWCoRYhDkRHARRF4TgRkIPPOIIJ9Aho44aODTYhFWIsOgrIAoGg7giWsSZhBRnn+McPtLWKYtThkhslf4VWhVcVqN1r6blqGU6RaVhkjKIMHth96zLntkDe2Wff9ZqOTV6Xpo0S30tN0r+i8X0x7+qGs02Tr9VQz3bKGPH8aqQd8NhereQ+/rGWbub3k2FWyvslr2R/xvWYU90A63xLt8leep6iB+JvNCLUYOiv9sxCLLrkeh2ZCO5GYrvua3yYgnLWKV+xBDHARLIUP0KLnGFtuAVIsKWEOunCh5Xs4AfIcS/AEWBkJQ=
    y⇤
    AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==
    x⇤

    View full-size slide

  8. ( )
    AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==
    x⇤
    AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==
    x
    AAACiXichVFNLwNRFD0dX1VfxUZi02iIVXNHBOmq0Y1lP7QkiMyMh9H5ysy0UU3/gJWdYEViIX6AH2DjD1j0J4gliY2F2+kkguBO3rzzzrvnvvPeVR1D93yiVkTq6u7p7Yv2xwYGh4ZH4qNjZc+uupooabZhu+uq4glDt0TJ131DrDuuUEzVEGtqJdveX6sJ19Nta9WvO2LLVPYsfVfXFJ+p8qZqNg6b2/EkpSiIxE8ghyCJMHJ2/A6b2IENDVWYELDgMzagwONvAzIIDnNbaDDnMtKDfYEmYqytcpbgDIXZCv/3eLURshav2zW9QK3xKQYPl5UJTNMj3dALPdAtPdH7r7UaQY22lzrPakcrnO2R44ni278qk2cf+5+qPz372MVS4FVn707AtG+hdfS1o9OXYrow3ZihK3pm/5fUonu+gVV71a7zonDxhx+VvfCLcYPk7+34CcpzKXkhRfn5ZGY5bFUUk5jCLPdjERmsIIcS1z/ACc5wLg1IsrQkpTupUiTUjONLSNkPIVKSSQ==
    x
    AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=
    y
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBGBdk8IVxRXTjkoc8EkTS1gEbStu0hQSJP2DiVhauNHFh/AA/wI0/4IJPMC4xcePCS2lilIi3mc6ZM/fcOTNXMlTFshnreISR0bHxCe+kb2p6ZtYfmJvPWnrdlHlG1lXdzEuixVVF4xlbsVWeN0wu1iSV56Tqfm8/1+Cmpejaod00eLEmVjSlrMiiTVS6ebxWCoRYhDkRHARRF4TgRkIPPOIIJ9Aho44aODTYhFWIsOgrIAoGg7giWsSZhBRnn+McPtLWKYtThkhslf4VWhVcVqN1r6blqGU6RaVhkjKIMHth96zLntkDe2Wff9ZqOTV6Xpo0S30tN0r+i8X0x7+qGs02Tr9VQz3bKGPH8aqQd8NhereQ+/rGWbub3k2FWyvslr2R/xvWYU90A63xLt8leep6iB+JvNCLUYOiv9sxCLLrkeh2ZCO5GYrvua3yYgnLWKV+xBDHARLIUP0KLnGFtuAVIsKWEOunCh5Xs4AfIcS/AEWBkJQ=
    y⇤
    AAAChHichVHNLgNRFP6M/98WG4lNoyEW0pzpn9YCYWOpqDYpkZlxMel0ZjIzbULjBdgSCysSC/EAHsDGC1h4BLEksbFwZloRi3Ju7r3nfud85373HtU2dNcjem6T2js6u7p7evv6BwaHQuHhkU3XqjqayGuWYTlFVXGFoZsi7+meIYq2I5SKaoiCWl7244WacFzdMje8Q1tsV5R9U9/TNcVjKLewE45SLJtJUzIeoRhRJk5pdlIkZ+VsRGbEtyiatmqF77GFXVjQUEUFAiY89g0ocHmUIINgM7aNOmMOe3oQFzhGH3OrnCU4Q2G0zOs+n0pN1OSzX9MN2BrfYvB0mBnBJD3RLb3RI93RC322rFUPavhaDnlXG1xh74ROxtY//mVVePdw8MP6U7OHPWQCrTprtwPEf4XW4NeOLt7W59Ym61N0Ta+s/4qe6YFfYNbetZucWLv8Q4/KWvjHuEHfXYi0djbjMTkdS+SS0cWlZqt6MI4JTHM/ZrGIFawiz/UFTnGGc6lLmpESUqqRKrU1OaP4ZdL8F+zrkAQ=
    ? AAAChHichVHNLgNRFP6M/98WG4lNoyEW0pzpn9YCYWOpqDYpkZlxMel0ZjIzbULjBdgSCysSC/EAHsDGC1h4BLEksbFwZloRi3Ju7r3nfud85373HtU2dNcjem6T2js6u7p7evv6BwaHQuHhkU3XqjqayGuWYTlFVXGFoZsi7+meIYq2I5SKaoiCWl7244WacFzdMje8Q1tsV5R9U9/TNcVjKLewE45SLJtJUzIeoRhRJk5pdlIkZ+VsRGbEtyiatmqF77GFXVjQUEUFAiY89g0ocHmUIINgM7aNOmMOe3oQFzhGH3OrnCU4Q2G0zOs+n0pN1OSzX9MN2BrfYvB0mBnBJD3RLb3RI93RC322rFUPavhaDnlXG1xh74ROxtY//mVVePdw8MP6U7OHPWQCrTprtwPEf4XW4NeOLt7W59Ym61N0Ta+s/4qe6YFfYNbetZucWLv8Q4/KWvjHuEHfXYi0djbjMTkdS+SS0cWlZqt6MI4JTHM/ZrGIFawiz/UFTnGGc6lLmpESUqqRKrU1OaP4ZdL8F+zrkAQ=
    ?
    AAAChHichVHNLgNRFP6M/98WG4lNoyEW0pzpn9YCYWOpqDYpkZlxMel0ZjIzbULjBdgSCysSC/EAHsDGC1h4BLEksbFwZloRi3Ju7r3nfud85373HtU2dNcjem6T2js6u7p7evv6BwaHQuHhkU3XqjqayGuWYTlFVXGFoZsi7+meIYq2I5SKaoiCWl7244WacFzdMje8Q1tsV5R9U9/TNcVjKLewE45SLJtJUzIeoRhRJk5pdlIkZ+VsRGbEtyiatmqF77GFXVjQUEUFAiY89g0ocHmUIINgM7aNOmMOe3oQFzhGH3OrnCU4Q2G0zOs+n0pN1OSzX9MN2BrfYvB0mBnBJD3RLb3RI93RC322rFUPavhaDnlXG1xh74ROxtY//mVVePdw8MP6U7OHPWQCrTprtwPEf4XW4NeOLt7W59Ym61N0Ta+s/4qe6YFfYNbetZucWLv8Q4/KWvjHuEHfXYi0djbjMTkdS+SS0cWlZqt6MI4JTHM/ZrGIFawiz/UFTnGGc6lLmpESUqqRKrU1OaP4ZdL8F+zrkAQ=
    ?
    AAAChHichVHNLgNRFP6M/98WG4lNoyEW0pzpn9YCYWOpqDYpkZlxMel0ZjIzbULjBdgSCysSC/EAHsDGC1h4BLEksbFwZloRi3Ju7r3nfud85373HtU2dNcjem6T2js6u7p7evv6BwaHQuHhkU3XqjqayGuWYTlFVXGFoZsi7+meIYq2I5SKaoiCWl7244WacFzdMje8Q1tsV5R9U9/TNcVjKLewE45SLJtJUzIeoRhRJk5pdlIkZ+VsRGbEtyiatmqF77GFXVjQUEUFAiY89g0ocHmUIINgM7aNOmMOe3oQFzhGH3OrnCU4Q2G0zOs+n0pN1OSzX9MN2BrfYvB0mBnBJD3RLb3RI93RC322rFUPavhaDnlXG1xh74ROxtY//mVVePdw8MP6U7OHPWQCrTprtwPEf4XW4NeOLt7W59Ym61N0Ta+s/4qe6YFfYNbetZucWLv8Q4/KWvjHuEHfXYi0djbjMTkdS+SS0cWlZqt6MI4JTHM/ZrGIFawiz/UFTnGGc6lLmpESUqqRKrU1OaP4ZdL8F+zrkAQ=
    ? AAAChHichVHNLgNRFP6M/98WG4lNoyEW0pzpn9YCYWOpqDYpkZlxMel0ZjIzbULjBdgSCysSC/EAHsDGC1h4BLEksbFwZloRi3Ju7r3nfud85373HtU2dNcjem6T2js6u7p7evv6BwaHQuHhkU3XqjqayGuWYTlFVXGFoZsi7+meIYq2I5SKaoiCWl7244WacFzdMje8Q1tsV5R9U9/TNcVjKLewE45SLJtJUzIeoRhRJk5pdlIkZ+VsRGbEtyiatmqF77GFXVjQUEUFAiY89g0ocHmUIINgM7aNOmMOe3oQFzhGH3OrnCU4Q2G0zOs+n0pN1OSzX9MN2BrfYvB0mBnBJD3RLb3RI93RC322rFUPavhaDnlXG1xh74ROxtY//mVVePdw8MP6U7OHPWQCrTprtwPEf4XW4NeOLt7W59Ym61N0Ta+s/4qe6YFfYNbetZucWLv8Q4/KWvjHuEHfXYi0djbjMTkdS+SS0cWlZqt6MI4JTHM/ZrGIFawiz/UFTnGGc6lLmpESUqqRKrU1OaP4ZdL8F+zrkAQ=
    ? AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==
    x⇤

    View full-size slide

  9. ( )
    AAACiXichVFNLwNRFD0dX1VfxUZi02iIVXNHBOmq0Y1lP7QkiMyMh9H5ysy0UU3/gJWdYEViIX6AH2DjD1j0J4gliY2F2+kkguBO3rzzzrvnvvPeVR1D93yiVkTq6u7p7Yv2xwYGh4ZH4qNjZc+uupooabZhu+uq4glDt0TJ131DrDuuUEzVEGtqJdveX6sJ19Nta9WvO2LLVPYsfVfXFJ+p8qZqNg6b2/EkpSiIxE8ghyCJMHJ2/A6b2IENDVWYELDgMzagwONvAzIIDnNbaDDnMtKDfYEmYqytcpbgDIXZCv/3eLURshav2zW9QK3xKQYPl5UJTNMj3dALPdAtPdH7r7UaQY22lzrPakcrnO2R44ni278qk2cf+5+qPz372MVS4FVn707AtG+hdfS1o9OXYrow3ZihK3pm/5fUonu+gVV71a7zonDxhx+VvfCLcYPk7+34CcpzKXkhRfn5ZGY5bFUUk5jCLPdjERmsIIcS1z/ACc5wLg1IsrQkpTupUiTUjONLSNkPIVKSSQ==
    x
    AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=
    y
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBGBdk8IVxRXTjkoc8EkTS1gEbStu0hQSJP2DiVhauNHFh/AA/wI0/4IJPMC4xcePCS2lilIi3mc6ZM/fcOTNXMlTFshnreISR0bHxCe+kb2p6ZtYfmJvPWnrdlHlG1lXdzEuixVVF4xlbsVWeN0wu1iSV56Tqfm8/1+Cmpejaod00eLEmVjSlrMiiTVS6ebxWCoRYhDkRHARRF4TgRkIPPOIIJ9Aho44aODTYhFWIsOgrIAoGg7giWsSZhBRnn+McPtLWKYtThkhslf4VWhVcVqN1r6blqGU6RaVhkjKIMHth96zLntkDe2Wff9ZqOTV6Xpo0S30tN0r+i8X0x7+qGs02Tr9VQz3bKGPH8aqQd8NhereQ+/rGWbub3k2FWyvslr2R/xvWYU90A63xLt8leep6iB+JvNCLUYOiv9sxCLLrkeh2ZCO5GYrvua3yYgnLWKV+xBDHARLIUP0KLnGFtuAVIsKWEOunCh5Xs4AfIcS/AEWBkJQ=
    y⇤
    AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==
    x⇤
    AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==
    x⇤
    AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==
    x AAAB93icdVDLSsNAFJ3UV62vqks3g0VwFZKatM2u6MZlC9YW2lAm00k7dCYJMxMhhH6BW127E7d+jkv/xOlDsKIHLhzOuZd77wkSRqWyrA+jsLG5tb1T3C3t7R8cHpWPT+5lnApMOjhmsegFSBJGI9JRVDHSSwRBPGCkG0xv5n73gQhJ4+hOZQnxORpHNKQYKS21w2G5YpmW6zn1KrRM17I950oTz2s4NRfaprVABazQGpY/B6MYp5xECjMkZd+2EuXnSCiKGZmVBqkkCcJTNCZ9TSPEifTzxaEzeKGVEQxjoStScKH+nMgRlzLjge7kSE3kb28u/uX1UxU2/JxGSapIhJeLwpRBFcP513BEBcGKZZogLKi+FeIJEggrnc3aloDPdCbfj8P/yX3VtGum23YqzetVOkVwBs7BJbBBHTTBLWiBDsCAgEfwBJ6NzHgxXo23ZWvBWM2cgjUY71/5QpPk
    f
    AAACNXicbVDLSgMxFM34tr6qLt0Ei6Agw4z2YReC6MZlBdsKM0PJpJk2mMwMSUYsYb7Gj/Ab3OrKhQtB3PoLpo+FWg8EDuecm3s5YcqoVI7zas3Mzs0vLC4tF1ZW19Y3iptbLZlkApMmTlgibkIkCaMxaSqqGLlJBUE8ZKQd3l4M/fYdEZIm8bUapCTgqBfTiGKkjNQpnvp9pPQgh6dQ+6PvPNELA+3YTqVerh0dOnbFcevlY0Pq9ZNytZLDKN/3Q67v84NOsWSCI8Bp4k5ICUzQ6BTf/W6CM05ihRmS0nOdVAUaCUUxI3nBzyRJEb5FPeIZGiNOZKBHd+VwzyhdGCXCvFjBkfpzQiMu5YCHJsmR6su/3lD8z/MyFZ0EmsZppkiMx4uijEGVwGFnsEsFwYoNDEFYUHMrxH0kEFam2V9bQp6bTty/DUyT1pHtVu3KVbl0dj5pZwnsgF2wD1xQA2fgEjRAE2DwAJ7AM3ixHq0368P6HEdnrMnMNvgF6+sbdw6oMA==
    ˆ
    y = f(x)

    View full-size slide

  10. AAACpXicbVFda9swFJW9ry77aNY+7kUsjKVQgp04H34YlJaNvQw6aNKCbYKkyK6oJBtJLgvCP2k/aI/7J1OcPKzpLggdzr3n3KsrXHGmTRD89vwnT589f3HwsvPq9Zu3h913Rwtd1orQOSl5qW4w0pQzSeeGGU5vKkWRwJxe47uLTf76nirNSnll1hXNBCokyxlBxlHL7i+btiaJKnBmg0EwjqPp8DQYjIMwjkYOxPEsmowbmDfwM0yRKmAqmFzaHKZMOozMLUHcfm2azp7XNA4mYegs3DWKJg6Ew2A0i5qtCmP7xYlSzAqedFpKCUuVKlXTX5/CvJ9iYX82JydtjcpgZ9ntObc24GMQ7kAP7OJy2f2TrkpSCyoN4UjrJAwqk1mkDCOcuua1phUid6igiYMSCaoz276igR8ds4J5qdyRBrbsvwqLhNZrgV3lZni9n9uQ/8sltclnmWWyqg2VZNsorzk0Jdz8EVwxRYnhawcQUczNCsktUogY95MPumDRuJ2E+xt4DBbDQTgZjH9EvbPz3XYOwHvwAfRBCKbgDHwDl2AOiHfsxd65d+F/8r/7V/5iW+p7O80xeBD+8i9BGsel
    f = arg min
    f2F
    E

    error(y, f(x))

    ( )
    AAACiXichVFNLwNRFD0dX1VfxUZi02iIVXNHBOmq0Y1lP7QkiMyMh9H5ysy0UU3/gJWdYEViIX6AH2DjD1j0J4gliY2F2+kkguBO3rzzzrvnvvPeVR1D93yiVkTq6u7p7Yv2xwYGh4ZH4qNjZc+uupooabZhu+uq4glDt0TJ131DrDuuUEzVEGtqJdveX6sJ19Nta9WvO2LLVPYsfVfXFJ+p8qZqNg6b2/EkpSiIxE8ghyCJMHJ2/A6b2IENDVWYELDgMzagwONvAzIIDnNbaDDnMtKDfYEmYqytcpbgDIXZCv/3eLURshav2zW9QK3xKQYPl5UJTNMj3dALPdAtPdH7r7UaQY22lzrPakcrnO2R44ni278qk2cf+5+qPz372MVS4FVn707AtG+hdfS1o9OXYrow3ZihK3pm/5fUonu+gVV71a7zonDxhx+VvfCLcYPk7+34CcpzKXkhRfn5ZGY5bFUUk5jCLPdjERmsIIcS1z/ACc5wLg1IsrQkpTupUiTUjONLSNkPIVKSSQ==
    x
    AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=
    y
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBGBdk8IVxRXTjkoc8EkTS1gEbStu0hQSJP2DiVhauNHFh/AA/wI0/4IJPMC4xcePCS2lilIi3mc6ZM/fcOTNXMlTFshnreISR0bHxCe+kb2p6ZtYfmJvPWnrdlHlG1lXdzEuixVVF4xlbsVWeN0wu1iSV56Tqfm8/1+Cmpejaod00eLEmVjSlrMiiTVS6ebxWCoRYhDkRHARRF4TgRkIPPOIIJ9Aho44aODTYhFWIsOgrIAoGg7giWsSZhBRnn+McPtLWKYtThkhslf4VWhVcVqN1r6blqGU6RaVhkjKIMHth96zLntkDe2Wff9ZqOTV6Xpo0S30tN0r+i8X0x7+qGs02Tr9VQz3bKGPH8aqQd8NhereQ+/rGWbub3k2FWyvslr2R/xvWYU90A63xLt8leep6iB+JvNCLUYOiv9sxCLLrkeh2ZCO5GYrvua3yYgnLWKV+xBDHARLIUP0KLnGFtuAVIsKWEOunCh5Xs4AfIcS/AEWBkJQ=
    y⇤
    AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==
    x⇤
    AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==
    x⇤
    AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==
    x AAAB93icdVDLSsNAFJ3UV62vqks3g0VwFZKatM2u6MZlC9YW2lAm00k7dCYJMxMhhH6BW127E7d+jkv/xOlDsKIHLhzOuZd77wkSRqWyrA+jsLG5tb1T3C3t7R8cHpWPT+5lnApMOjhmsegFSBJGI9JRVDHSSwRBPGCkG0xv5n73gQhJ4+hOZQnxORpHNKQYKS21w2G5YpmW6zn1KrRM17I950oTz2s4NRfaprVABazQGpY/B6MYp5xECjMkZd+2EuXnSCiKGZmVBqkkCcJTNCZ9TSPEifTzxaEzeKGVEQxjoStScKH+nMgRlzLjge7kSE3kb28u/uX1UxU2/JxGSapIhJeLwpRBFcP513BEBcGKZZogLKi+FeIJEggrnc3aloDPdCbfj8P/yX3VtGum23YqzetVOkVwBs7BJbBBHTTBLWiBDsCAgEfwBJ6NzHgxXo23ZWvBWM2cgjUY71/5QpPk
    f
    AAACNXicbVDLSgMxFM34tr6qLt0Ei6Agw4z2YReC6MZlBdsKM0PJpJk2mMwMSUYsYb7Gj/Ab3OrKhQtB3PoLpo+FWg8EDuecm3s5YcqoVI7zas3Mzs0vLC4tF1ZW19Y3iptbLZlkApMmTlgibkIkCaMxaSqqGLlJBUE8ZKQd3l4M/fYdEZIm8bUapCTgqBfTiGKkjNQpnvp9pPQgh6dQ+6PvPNELA+3YTqVerh0dOnbFcevlY0Pq9ZNytZLDKN/3Q67v84NOsWSCI8Bp4k5ICUzQ6BTf/W6CM05ihRmS0nOdVAUaCUUxI3nBzyRJEb5FPeIZGiNOZKBHd+VwzyhdGCXCvFjBkfpzQiMu5YCHJsmR6su/3lD8z/MyFZ0EmsZppkiMx4uijEGVwGFnsEsFwYoNDEFYUHMrxH0kEFam2V9bQp6bTty/DUyT1pHtVu3KVbl0dj5pZwnsgF2wD1xQA2fgEjRAE2DwAJ7AM3ixHq0368P6HEdnrMnMNvgF6+sbdw6oMA==
    ˆ
    y = f(x)
    AAACPXicbVDNSgMxGMz6b/2revQSLEIVKbu1VXtQiiJ4VLAqdNeSTbNtaLK7JFlxCftEPoTP4FE9efAmXr2a1h60OhAYZr7J9zF+zKhUtv1kjY1PTE5Nz8zm5uYXFpfyyyuXMkoEJg0csUhc+0gSRkPSUFQxch0LgrjPyJXfO+77V7dESBqFFyqNicdRJ6QBxUgZqZU/cbtI6TSDB1C7g++aouN72i7Z1Vplr7xtl6q2U6vsGFKr7Vd2qxkMsqLrc32XbcJDmN7orayVL5jAAPAvcYakAIY4a+Vf3XaEE05ChRmSsunYsfI0EopiRrKcm0gSI9xDHdI0NEScSE8P7svghlHaMIiEeaGCA/VnQiMuZcp9M8mR6spRry/+5zUTFex7moZxokiIvxcFCYMqgv3uYJsKghVLDUFYUHMrxF0kEFam4V9bfN7vxBlt4C+5LJec3VL1vFKoHw3bmQFrYB0UgQP2QB2cgjPQABjcg0fwDF6sB+vNerc+vkfHrGFmFfyC9fkFIG+q9w==
    ˆ
    y = f(x) > y⇤
    AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==
    x

    View full-size slide

  11. (Uncertainty Quantification, UQ)
    AAACiXichVFNLwNRFD0dX1VfxUZi02iIVXNHBOmq0Y1lP7QkiMyMh9H5ysy0UU3/gJWdYEViIX6AH2DjD1j0J4gliY2F2+kkguBO3rzzzrvnvvPeVR1D93yiVkTq6u7p7Yv2xwYGh4ZH4qNjZc+uupooabZhu+uq4glDt0TJ131DrDuuUEzVEGtqJdveX6sJ19Nta9WvO2LLVPYsfVfXFJ+p8qZqNg6b2/EkpSiIxE8ghyCJMHJ2/A6b2IENDVWYELDgMzagwONvAzIIDnNbaDDnMtKDfYEmYqytcpbgDIXZCv/3eLURshav2zW9QK3xKQYPl5UJTNMj3dALPdAtPdH7r7UaQY22lzrPakcrnO2R44ni278qk2cf+5+qPz372MVS4FVn707AtG+hdfS1o9OXYrow3ZihK3pm/5fUonu+gVV71a7zonDxhx+VvfCLcYPk7+34CcpzKXkhRfn5ZGY5bFUUk5jCLPdjERmsIIcS1z/ACc5wLg1IsrQkpTupUiTUjONLSNkPIVKSSQ==
    x
    AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=
    y
    AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==
    x⇤
    AAAB/nicbVDJSgNBEK2JW4xb1KOXxiCIhzAjbsegF48RzALJGHo6PUmT7p6hu0cMw4Df4FXP3sSrv+LRP7GzHEzig4LHe1VU1QtizrRx3W8nt7S8srqWXy9sbG5t7xR39+o6ShShNRLxSDUDrClnktYMM5w2Y0WxCDhtBIObkd94pEqzSN6bYUx9gXuShYxgY6VmOxDpU/Zw0imW3LI7Blok3pSUYIpqp/jT7kYkEVQawrHWLc+NjZ9iZRjhNCu0E01jTAa4R1uWSiyo9tPxvRk6skoXhZGyJQ0aq38nUiy0HorAdgps+nreG4n/ea3EhFd+ymScGCrJZFGYcGQiNHoedZmixPChJZgoZm9FpI8VJsZGNLMlEJnNxJtPYJHUT8veRfn87qxUuZ6mk4cDOIRj8OASKnALVagBAQ4v8ApvzrPz7nw4n5PWnDOd2YcZOF+/0HGWjQ==
    x⇤
    AAAB/HicbVC7TsNAEFzzDOEVoKQ5ESFRRTbiVUbQUAaJPKTEis6Xc3Lk7mzdnRGRFb6BFmo6RMu/UPInnBMXJGGklUYzu9rdCWLOtHHdb2dpeWV1bb2wUdzc2t7ZLe3tN3SUKELrJOKRagVYU84krRtmOG3FimIRcNoMhjeZ33ykSrNI3ptRTH2B+5KFjGBjpUYnEOnTuFsquxV3ArRIvJyUIUetW/rp9CKSCCoN4VjrtufGxk+xMoxwOi52Ek1jTIa4T9uWSiyo9tPJtWN0bJUeCiNlSxo0Uf9OpFhoPRKB7RTYDPS8l4n/ee3EhFd+ymScGCrJdFGYcGQilL2OekxRYvjIEkwUs7ciMsAKE2MDmtkSiCwTbz6BRdI4rXgXlfO7s3L1Ok+nAIdwBCfgwSVU4RZqUAcCD/ACr/DmPDvvzofzOW1dcvKZA5iB8/ULqrqV8Q==
    x
    AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=
    y AAAChHichVHLSsNAFD1GrbW+qm4EN8WiuJBy6xsXUnTjsg9rC1VKEqc1mCYhSQu1+AO6VVy4UnAhfoAf4MYfcNFPEJcKblx4mwZEi3rDZM6cuefOmbmKpWuOS9Tskrp7egN9wf7QwODQ8Eh4dGzHMau2KrKqqZt2XpEdoWuGyLqaq4u8ZQu5ougipxxutvZzNWE7mmlsu3VL7FXksqGVNFV2mUrVi+EoxciLSCeI+yAKP5Jm+B672IcJFVVUIGDAZaxDhsNfAXEQLOb20GDOZqR5+wLHCLG2ylmCM2RmD/lf5lXBZw1et2o6nlrlU3QeNisjmKYnuqVXeqQ7eqaPX2s1vBotL3WelbZWWMWRk4nM+7+qCs8uDr5Uf3p2UcKq51Vj75bHtG6htvW1o4vXzFp6ujFD1/TC/q+oSQ98A6P2pt6kRPryDz8Ke+EX4wbFf7ajE+zMx+LLMUotRhMbfquCmMQUZrkfK0hgC0lkub7AKc5wLgWkOWlBWmqnSl2+ZhzfQlr/BNcbj/U=
    y
    AAACh3ichVE7TsNAEH2Yb/glQINEExGBoAnjACGh4tNQQiCAFKLINhuwcGzLdiKFiAtQ0AaJCiQKxAE4AA0XoMgRECVINBSMnSBEEZjV7s6+mTf7dke1Dd31iBodUmdXd09vX6h/YHBoOBwZGd11rbKjiaxmGZazryquMHRTZD3dM8S+7QilpBpiTz1Z9+N7FeG4umXueFVb5EvKkakXdU3xfMieqc4WIjGKp1NJWkhEKU6USlCSnUWS03I6KjPiWwwt27QiDzjAISxoKKMEARMe+wYUuDxykEGwGcujxpjDnh7EBc7Qz9wyZwnOUBg94fWIT7kWavLZr+kGbI1vMXg6zIxiip7pjt7oie7phT7b1qoFNXwtVd7VJlfYhfD5+PbHv6wS7x6Of1h/avZQRCrQqrN2O0D8V2hNfuW0/ra9nJmqTdMNvbL+a2rQI7/ArLxrt1sic/WHHpW18I9xg767EG3v7CbicjI+v7UQW1lrtaoPE5jEDPdjCSvYwCayXP8YF6jjUgpJc1JSSjVTpY4WZwy/TFr9AnUXkR0=
    p(y)
    AAAChnichVE7TsNAEH2YXwi/AA0STUQEQhTWOCQhoYqgoSSEABI/2WYTLBzbsp1IIeICSLRQUIFEgTgAB6DhAhQcAVGCREPB2AlCFMCsdnf2zbzZtzuaYxqeT/TUIXV2dff0Rvqi/QODQ8OxkdF1z665uijptmm7m5rqCdOwRMk3fFNsOq5Qq5opNrTDpSC+UReuZ9jWmt9wxE5VrVhG2dBVn6FiY3d2L5YgOZfNUCoZJ5kom6QMO2lSckourjASWAJtW7Fjd9jGPmzoqKEKAQs++yZUeDy2oIDgMLaDJmMue0YYFzhGlLk1zhKcoTJ6yGuFT1tt1OJzUNML2TrfYvJ0mRnHFD3SDb3SA93SM338WqsZ1gi0NHjXWlzh7A2fjBff/2VVefdx8M36U7OPMrKhVoO1OyESvEJv8etH56/FhdWp5jRd0Qvrv6QnuucXWPU3/bogVi/+0KOxFv4xbtBXF+K/O+tJWcnIc4VUIr/YblUEE5jEDPdjHnksYwUlrl/BKc5wLkUkWUpL861UqaPNGcMPk/Kf1ZuQ2g==
    y⇤
    AAACp3ichVE9T9tQFD0Y2vJRSqALUheLiCphiK5TGhIkKlSWboSPhEh8RLbzQp9wbMt+ieRG7FX/QIdOrdQBIXWFnYU/wMBPqDqCxMLQaydVxQBc6/mdd9499533ruU7MlRElwPa4NCTp8+GR0bHno+/mEhNTlVDrx3YomJ7jhfULDMUjnRFRUnliJofCLNlOWLLOliJ97c6Igil526qyBe7LXPflU1pm4qpemqmnIneRXtz2aUd6ap6l+HhHsOminQ/E2X1RlRPpSlXKhZoPq9TjqiYpwKDt2SUjJJuMBNHGv0oe6kT7KABDzbaaEHAhWLswETI3zYMEHzmdtFlLmAkk32BQ4yyts1ZgjNMZg/4v8+r7T7r8jquGSZqm09xeASs1DFLF3REV3ROx/Sbbu+t1U1qxF4inq2eVvj1iS/TGzePqlo8K3z8r3rQs0ITxcSrZO9+wsS3sHv6zqevVxuL67Pd1/SD/rD/73RJZ3wDt3Nt/1wT698e8GOxF34xbtC/Luj3g2o+ZxRyb9bm08vv+60axivMIMP9WMAyPqCMCtf/jF84wamW1Va1qlbrpWoDfc1L3AnN/AuqB50Q
    P(y > y⇤) =
    Z 1
    y⇤
    p(y)dy
    AAACwnichVHLTttAFL24vMsjlA0Sm1EjKlpF0XWANEECIWilLnk0gBSHyHYmMOBX7UkkY/ID/AALVi3qouoH9AO6abetWPAJqEsqsWHRayeoYgFcazxnztxz58xcw7NEIBEvupQn3T29ff0Dg0+HhkdGU2PPNgO34Zu8ZLqW628besAt4fCSFNLi257Pdduw+JZxsBLvbzW5HwjXeS9Dj1dsfdcRdWHqkqhqakGzdblnGNHbVlnLhEzLHGkZFi6GO68qC5pwZDUi2NohWJchowSz5krmTYcvWS2sptKYLRbyOJtjmEUs5DBPYA7VolpkKjFxpKETq27qG2hQAxdMaIANHByQhC3QIaCvDCogeMRVICLOJySSfQ4tGCRtg7I4ZejEHtB/l1blDuvQOq4ZJGqTTrFo+KRkMIXn+AWv8Ad+xUu8ubdWlNSIvYQ0G20t96qjxxMb14+qbJol7P1XPehZQh0KiVdB3r2EiW9htvXNw5Orjfn1qegFfsI/5P8jXuB3uoHT/Gt+XuPrpw/4McgLvRg16LYL7H6wmcuq+ezM2mx6abnTqn6YhOcwTf14DUvwDlahRPXP4Cf8gt/KG2Vf+aAE7VSlq6MZhzuhHP0DArmnyA==
    E[ y | y > y⇤] =
    Z 1
    y⇤
    y · p(y)dy
    Probability of
    improvement (PI)
    Expected
    Improvement (EI)
    Upper Confidence
    Bound (UCB)
    (UQ)

    View full-size slide

  12. ( )
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==
    f
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==
    f
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1 AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1 AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y

    View full-size slide

  13. ( )
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==
    f
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==
    f
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1 AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1 AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y

    View full-size slide

  14. ( )
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==
    f
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==
    f
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1 AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1 AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y

    (= )
    (x , x ) y
    ( )

    View full-size slide

  15. ( )
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==
    f
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==
    f
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1 AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1 AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y

    (= )
    (x , x ) y
    ( )
    ‣ (x , x ) ( )
    y (x , x )

    View full-size slide

  16. AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==
    f
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYqWS4FQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkCsAOP5g==
    f
    AAAChHichVHLTsJAFD3UF+ID1I2JGyLBuDBkUHzEhSG6cclDHgkS0tYBG0rbtIUEiT+gW40LV5q4MH6AH+DGH3DBJxiXmLhx4aU0MUrE20znzJl77pyZKxmqYtmMtT3C0PDI6Jh33DcxOTXtD8zMZi29bso8I+uqbuYl0eKqovGMrdgqzxsmF2uSynNSda+7n2tw01J07cBuGrxYEyuaUlZk0SYq2SwFQizCnAj2g6gLQnAjoQcecYgj6JBRRw0cGmzCKkRY9BUQBYNBXBEt4kxCirPPcQofaeuUxSlDJLZK/wqtCi6r0bpb03LUMp2i0jBJGUSYvbB71mHP7IG9ss8/a7WcGl0vTZqlnpYbJf/ZfPrjX1WNZhvH36qBnm2UseV4Vci74TDdW8g9fePkqpPeToVbS+yWvZH/G9ZmT3QDrfEu3yV56nqAH4m80ItRg6K/29EPsquR6EYkloyF4rtuq7xYwCKWqR+biGMfCWSoPsc5LnApjAorwpqw3ksVPK5mDj9C2PkC2GOP+Q==
    y
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTg2JcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkN+IiVuxeDoeSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVLKQnA==
    x1 AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDQTGuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmM9jzAxOTU94531zc0vLPoDS8t5S2+aMs/JuqqbRUm0uKpoPGcrtsqLhsnFhqTyglTfH+wXWty0FF07tNsGLzfEmqZUFVm0icqeVmKVQIhFmBPBURB1QQhupPTAI45wDB0ymmiAQ4NNWIUIi74SomAwiCujQ5xJSHH2Oc7hI22TsjhliMTW6V+jVcllNVoPalqOWqZTVBomKYMIsxd2z/rsmT2wV/b5Z62OU2PgpU2zNNRyo+K/WM1+/Ktq0Gzj5Fs11rONKnYcrwp5NxxmcAt5qG+ddfvZ3Uy4s8Fu2Rv5v2E99kQ30Frv8l2aZ67H+JHIC70YNSj6ux2jIB+LRLcj8XQ8lNxzW+XFGtaxSf1IIIkDpJCj+jVc4gpdwStEhC0hMUwVPK5mBT9CSH4BVtKQnQ==
    x2
    ⾒ ( )
    N = = N = = N = =
    10 10 =100
    Full Factorial Design

    View full-size slide

  17. Fisher
    ( )
    . (Replication)
    ⾒ ( )
    . (Local Control)
    . (Randomization) Randomized Controlled Trial (Fisher )
    Data Leakage

    View full-size slide

  18. (or ?)
    UQ ⾒

    View full-size slide

  19. (or ?)
    UQ ⾒
    ( ) ⾒ ( ?)
    ༗઒અ෉
    Herbert A. Simon
    Simon HA, Machine Discovery. ( )
    Langley PW, Simon HA, Bradshaw G, Zytkow JM, Scienti c Discovery:
    Computational Explorations of the Creative Process ( ).
    , . (1996)
    , . (2001)

    View full-size slide

  20. .
    . o -the-shelf
    .
    PDF https://itakigawa.page.link/IBISML taki

    View full-size slide

  21. o -the-shelf

    View full-size slide

  22. ( big data )
    ( )
    interpolator ( )
    o -the-shelf

    View full-size slide

  23. KernelRidge(kernel='rbf', alpha=0.05, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=2.0)
    ExtraTreesRegressor(n_estimators=50)
    DecisionTreeRegressor()

    View full-size slide

  24. KernelRidge(kernel='rbf', alpha=0.05, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=2.0)
    ExtraTreesRegressor(n_estimators=50)
    DecisionTreeRegressor()

    View full-size slide

  25. KernelRidge(kernel='rbf', alpha=0.05, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=2.0)
    ExtraTreesRegressor(n_estimators=50)
    DecisionTreeRegressor()

    View full-size slide

  26. KernelRidge(kernel='rbf', alpha=0.05, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=2.0)
    ExtraTreesRegressor(n_estimators=50)
    DecisionTreeRegressor()

    View full-size slide

  27. KernelRidge(kernel='rbf', alpha=0.05, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=2.0)
    ExtraTreesRegressor(n_estimators=50)
    DecisionTreeRegressor()
    ( )

    View full-size slide

  28. KernelRidge(kernel='rbf', alpha=0.05, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=0.1)
    KernelRidge(kernel='rbf', alpha=1e-4, gamma=2.0)
    ExtraTreesRegressor(n_estimators=50)
    DecisionTreeRegressor()
    ( )
    /
    (Activity Cli s,
    Selectivity Cli s, )

    View full-size slide

  29. (UQ)
    skopt.learning.
    ExtraTreesRegressor(n_estimators=50)
    skopt.learning.
    ExtraTreesRegressor(n_estimators=50,
    bootstrap=True)
    GaussianProcessRegressor(kernel=1*RBF(),

    alpha=1e-2)
    skopt.learning.
    RandomForestRegressor(n_estimators=50)
    base = ExtraTreesRegressor(n_estimators=50, 

    bootstrap=True)
    MapieRegressor(base, method="plus", cv=-1)
    GaussianProcessRegressor(kernel=1*RBF(),

    alpha=1e-5)
    GaussianProcessRegressor(kernel=1*RBF(),

    alpha=1e-4)
    MapieRegressor(base, method="plus",
    cv=Subsample(n_resamplings=50))

    View full-size slide

  30. (>100)
    ( ) ( !?)

    View full-size slide

  31. (>100)
    ( ) ( !?)

    Balestriero R, Pesenti J, LeCun Y.
    Learning in High Dimension Always Amounts to Extrapolation.
    arXiv [cs.LG]. 2021. http://arxiv.org/abs/2110.09485
    (interpolation)
    y
    • “Our goal in this paper is to demonstrate both theoretically and
    empirically for both synthetic and real data that interpolation almost
    surely never occurs in high-dimensional spaces (>100) regardless of the
    underlying intrinsic dimension of the data manifold.“
    • "Those results challenge the validity of our current interpolation/
    extrapolation definition as an indicator of generalization performances. "

    View full-size slide

  32. / 1/2
    d (r= )
    AAAC73ichVHNThRBEK4dVFZAWeBi4mXiBrJ72dQSIsTEhOhBj/ztQkLjpqenZ2mZv53p3WSdzAvwAhxMjJB4ID6AD+DFoxcOPILhCMGLidbOTiJKxJ7M9Fdf1VfzdZcVuirWiKcFY+TW7Tujxbtj4xP37k+WpqabcdCNhGyIwA2iLYvH0lW+bGilXbkVRpJ7lis3rb3ng/xmT0axCvwN3Q/ljsfbvnKU4JqoVul1s2JXzacmcyIuEhaqV8kQ2mkyn6bp1YgJO9DsBfc8zlzp6MrVXKTau7qamkwHJpqs0+W2WbGzkCnf0f1qq1TGGmbLvA7qOShDvlaC0idgYEMAArrggQQfNGEXOMT0bEMdEELidiAhLiKksryEFMZI26UqSRWc2D36tinazlmf4kHPOFML+otLb0RKE2bxBI/xHL/gR/yGP/7ZK8l6DLz0abeGWhm2JvcfrH//r8qjXcPub9WNnjU4sJR5VeQ9zJjBKcRQ33tzcL7+ZG02mcMjPCP/h3iKn+kEfu9CfFiVa29v8GORF7oxGlD973FcB835Wv1xbWF1obz8LB9VER7CI6jQPBZhGV7CCjSo/1e4hJ8FMDrGgfHOeD8sNQq5Zgb+WMbxL1/dugg=
    V (d) =
    ⇡ d
    2
    d
    2
    · d
    2
    ! 0 (d ! 1)
    https://www.math.ucdavis.edu/~strohmer/courses/180BigData/180lecture1.pdf
    AAAChHichVHLSsNAFD2Nr/ps1Y3gRiwVF1JutD5wIUU3LltrVdAiSZzW0DQJSVqoxR/QreLClYIL8QP8ADf+gAs/QVxWcOPC2zQgWtQbJnPmzD13zsxVbUN3PaLnkNTR2dXdE+7t6x8YHIpEh0e2XKviaCKnWYbl7KiKKwzdFDlP9wyxYztCKauG2FZLa8397apwXN0yN72aLfJlpWjqBV1TPKYy8n40RgnyY6IdyAGIIYi0Fb3HHg5gQUMFZQiY8BgbUODytwsZBJu5POrMOYx0f1/gGH2srXCW4AyF2RL/i7zaDViT182arq/W+BSDh8PKCcTpiW6pQY90Ry/08Wutul+j6aXGs9rSCns/cjKWff9XVebZw+GX6k/PHgpY8r3q7N32meYttJa+enTRyC5vxOtTdE2v7P+KnumBb2BW37SbjNi4/MOPyl74xbhB8s92tIOt2YS8kEhmkrHUatCqMMYxiWnuxyJSWEcaOa4vcIoznEvd0ow0J823UqVQoBnFt5BWPgE/Y4+x
    1
    AAACjnichVHNSgJRFD5Of2Y/Wm2CNpIYreQYZhFEUhuX/uQPqMjMeLPBcWaauQomvkD7aBEUBS2iB+gB2vQCLXyEaGnQpkXHcSBKsjPcud/97vnO/e49kqEqFkfsuoSx8YnJKfe0Z2Z2bt7rW1jMWnrDlFlG1lXdzEuixVRFYxmucJXlDZOJdUllOal20N/PNZlpKbp2yFsGK9XFqqYcKbLIiSpgaLNonZi8XemUfQEMoR3+YRB2QACcSOi+RyhCBXSQoQF1YKABJ6yCCBZ9BQgDgkFcCdrEmYQUe59BBzykbVAWowyR2Br9q7QqOKxG635Ny1bLdIpKwySlH4L4gvfYw2d8wFf8/LNW267R99KiWRpomVH2ni2nP/5V1WnmcPytGumZwxFs214V8m7YTP8W8kDfPL3opXdSwfYa3uIb+b/BLj7RDbTmu3yXZKnLEX4k8kIvRg0K/27HMMhuhMLRUCQZCcT2nVa5YQVWYZ36sQUxiEMCMvaLnsMVXAs+ISrsCnuDVMHlaJbgRwjxLwzik/k=
    0.5
    p
    d
    AAAChnichVHLSsNAFD3GV62PVt0IbopFcRVupVVxVXTjsg9rC1okidMamiYhSQta/AHBrV24UnAhfoAf4MYfcOEniEsFNy68TQOixXrDZM6cuefOmbmqbeiuR/Q8IA0ODY+MhsbC4xOTU5Ho9MyuazUcTRQ0y7Cckqq4wtBNUfB0zxAl2xFKXTVEUa1tdfaLTeG4umXueMe2KNeVqqlXdE3xmMqTnDqIxkkmP2K9IBGAOILIWNF77OMQFjQ0UIeACY+xAQUuf3tIgGAzV0aLOYeR7u8LnCLM2gZnCc5QmK3xv8qrvYA1ed2p6fpqjU8xeDisjGGRnuiW3uiR7uiFPv+s1fJrdLwc86x2tcI+iJzN5T/+VdV59nD0rerr2UMF675Xnb3bPtO5hdbVN0/ab/mN3GJria7plf1f0TM98A3M5rt2kxW5yz5+VPbCL8YNSvxuRy/YXZETq3Iym4ynN4NWhTCPBSxzP9aQxjYyKHD9Ks5xgbYUkmQpJa11U6WBQDOLHyGlvwBbQpAn
    0.5 AAAChnichVHLSsNAFD2Nr1ofrboR3BRLxVWZlmpFEIpuXPZhH1BLSdKxhqZJSNJCLf6A4NYuXCm4ED/AD3DjD7joJ4jLCm5ceJsGRIv1hsmcOXPPnTNzJUNVLJuxnkeYmJyanvHO+ubmFxb9gaXlvKU3TZnnZF3VzaIkWlxVNJ6zFVvlRcPkYkNSeUGqHwz2Cy1uWoquHdltg5cbYk1TThRZtInKVvdilUCIRZgTwVEQdUEIbqT0wCOOUYUOGU00wKHBJqxChEVfCVEwGMSV0SHOJKQ4+xzn8JG2SVmcMkRi6/Sv0arkshqtBzUtRy3TKSoNk5RBhNkLu2d99swe2Cv7/LNWx6kx8NKmWRpquVHxX6xmP/5VNWi2cfqtGuvZxgl2HK8KeTccZnALeahvnXX72d1MuLPBbtkb+b9hPfZEN9Ba7/Jdmmeux/iRyAu9GDUo+rsdoyAfi0S3I/F0PJTcd1vlxRrWsUn9SCCJQ6SQo/o1XOIKXcErRIQtITFMFTyuZgU/Qkh+AeO5kGc=
    d = 2 AAAChnichVHLSsNAFD2Nr1ofrboR3BRLxVWZSrUiCEU3LvuwD6ilJOm0hqZJSNJCLf6A4NYuXCm4ED/AD3DjD7joJ4jLCm5ceJsGRIv1hsmcOXPPnTNzJUNVLJuxnkeYmJyanvHO+ubmFxb9gaXlnKU3TZlnZV3VzYIkWlxVNJ61FVvlBcPkYkNSeV6qHw728y1uWoquHdttg5caYk1Tqoos2kRlKvuxciDEIsyJ4CiIuiAEN5J64BEnqECHjCYa4NBgE1YhwqKviCgYDOJK6BBnElKcfY5z+EjbpCxOGSKxdfrXaFV0WY3Wg5qWo5bpFJWGScogwuyF3bM+e2YP7JV9/lmr49QYeGnTLA213Cj7L1YzH/+qGjTbOP1WjfVso4pdx6tC3g2HGdxCHupbZ91+Zi8d7mywW/ZG/m9Yjz3RDbTWu3yX4unrMX4k8kIvRg2K/m7HKMhtRaI7kVgqFkocuK3yYg3r2KR+xJHAEZLIUv0aLnGFruAVIsK2EB+mCh5Xs4IfISS+AOf5kGk=
    d = 4
    AAAChHichVHLSsNAFD2Nr/ps1Y3gRiwVF1JutD5wIUU3LltrVdAiSZzW0DQJSVqoxR/QreLClYIL8QP8ADf+gAs/QVxWcOPC2zQgWtQbJnPmzD13zsxVbUN3PaLnkNTR2dXdE+7t6x8YHIpEh0e2XKviaCKnWYbl7KiKKwzdFDlP9wyxYztCKauG2FZLa8397apwXN0yN72aLfJlpWjqBV1TPKYy8n40RgnyY6IdyAGIIYi0Fb3HHg5gQUMFZQiY8BgbUODytwsZBJu5POrMOYx0f1/gGH2srXCW4AyF2RL/i7zaDViT182arq/W+BSDh8PKCcTpiW6pQY90Ry/08Wutul+j6aXGs9rSCns/cjKWff9XVebZw+GX6k/PHgpY8r3q7N32meYttJa+enTRyC5vxOtTdE2v7P+KnumBb2BW37SbjNi4/MOPyl74xbhB8s92tIOt2YS8kEhmkrHUatCqMMYxiWnuxyJSWEcaOa4vcIoznEvd0ow0J823UqVQoBnFt5BWPgE/Y4+x
    1
    AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCKDQTEuDNGNSx7ySJCQtgzYUNqmLSRI/AETt7JwpYkL4wf4AW78ARd8gnGJiRsXXkoTo0S8zXTOnLnnzpm5kqEqls1YzyNMTE5Nz3hnfXPzC4v+wNJyztKbpsyzsq7qZkESLa4qGs/aiq3ygmFysSGpPC/VDwf7+RY3LUXXju22wUsNsaYpVUUWbaIylf1YORBiEeZEcBREXRCCG0k98IgTVKBDRhMNcGiwCasQYdFXRBQMBnEldIgzCSnOPsc5fKRtUhanDJHYOv1rtCq6rEbrQU3LUct0ikrDJGUQYfbC7lmfPbMH9so+/6zVcWoMvLRploZabpT9F6uZj39VDZptnH6rxnq2UcWu41Uh74bDDG4hD/Wts24/s5cOdzbYLXsj/zesx57oBlrrXb5L8fT1GD8SeaEXowZFf7djFOS2ItGdSCwVCyUO3FZ5sYZ1bFI/4kjgCElkqX4Nl7hCV/AKEWFbiA9TBY+rWcGPEBJf6hqQag==
    d > 4
    AAACjnichVHNSgJRFD5Of2Y/Wm2CNpIYreQoZhFEUhuX/uQPqMjMdLPBcWaauQomvkD7aBEUBS2iB+gB2vQCLXyEaGnQpkXHcSBKsjPcud/97vnO/e49kqEqFkfsuoSx8YnJKfe0Z2Z2bt7rW1jMWXrDlFlW1lXdLEiixVRFY1mucJUVDJOJdUlleam239/PN5lpKbp2wFsGK9fFqqYcKbLIiSpiaKNknZi8HelUfAEMoR3+YRB2QACcSOq+RyjBIeggQwPqwEADTlgFESz6ihAGBIO4MrSJMwkp9j6DDnhI26AsRhkisTX6V2lVdFiN1v2alq2W6RSVhklKPwTxBe+xh8/4gK/4+Wettl2j76VFszTQMqPiPVvOfPyrqtPM4fhbNdIzhyPYsr0q5N2wmf4t5IG+eXrRy2yng+01vMU38n+DXXyiG2jNd/kuxdKXI/xI5IVejBoU/t2OYZCLhMKxUDQVDcT3nFa5YQVWYZ36sQlxSEASsvaLnsMVXAs+ISbsCLuDVMHlaJbgRwiJL6Jhk8c=
    0.5
    p
    2
    AAACjnichVHNSgJRFD5Of2Y/Wm2CNpIYreQYZhFEUhuX/uQPqMjMeLPBcWaauQomvkD7aBEUBS2iB+gB2vQCLXyEaGnQpkXHcSBKsjPcud/97vnO/e49kqEqFkfsuoSx8YnJKfe0Z2Z2bt7rW1jMWnrDlFlG1lXdzEuixVRFYxmucJXlDZOJdUllOal20N/PNZlpKbp2yFsGK9XFqqYcKbLIiSpgaLNonZi8XemUfQEMoR3+YRB2QACcSOi+RyhCBXSQoQF1YKABJ6yCCBZ9BQgDgkFcCdrEmYQUe59BBzykbVAWowyR2Br9q7QqOKxG635Ny1bLdIpKwySlH4L4gvfYw2d8wFf8/LNW267R99KiWRpomVH2ni2nP/5V1WnmcPytGumZwxFs214V8m7YTP8W8kDfPL3opXdSwfYa3uIb+b/BLj7RDbTmu3yXZKnLEX4k8kIvRg0K/27HMMhuhMLRUCQZCcT2nVa5YQVWYZ36sQUxiEMCMvaLnsMVXAs+ISrsCnuDVMHlaJbgRwjxLwzik/k=
    0.5
    p
    d
    !?
    Unit Cube in
    Unit Ball
    AAAChHichVHLSsNAFD2Nr/ps1Y3gRiwVF1JutD5wIUU3LltrVdAiSZzW0DQJSVqoxR/QreLClYIL8QP8ADf+gAs/QVxWcOPC2zQgWtQbJnPmzD13zsxVbUN3PaLnkNTR2dXdE+7t6x8YHIpEh0e2XKviaCKnWYbl7KiKKwzdFDlP9wyxYztCKauG2FZLa8397apwXN0yN72aLfJlpWjqBV1TPKYy8n40RgnyY6IdyAGIIYi0Fb3HHg5gQUMFZQiY8BgbUODytwsZBJu5POrMOYx0f1/gGH2srXCW4AyF2RL/i7zaDViT182arq/W+BSDh8PKCcTpiW6pQY90Ry/08Wutul+j6aXGs9rSCns/cjKWff9XVebZw+GX6k/PHgpY8r3q7N32meYttJa+enTRyC5vxOtTdE2v7P+KnumBb2BW37SbjNi4/MOPyl74xbhB8s92tIOt2YS8kEhmkrHUatCqMMYxiWnuxyJSWEcaOa4vcIoznEvd0ow0J823UqVQoBnFt5BWPgE/Y4+x
    1
    AAAChXichVHLSsNAFD2Nr1pfVTeCm2KpuLHcSFFxoUU3LvuwWqhFkjjVYJqEJC1o8QfErbpwpeBC/AA/wI0/4KKfIC4ruHHhbRoQFfWGyZw5c8+dM3NV29Bdj6gZkrq6e3r7wv2RgcGh4ZHo6Nima9UcTRQ0y7Ccoqq4wtBNUfB0zxBF2xFKVTXElnqw1t7fqgvH1S1zwzu0Rbmq7Jl6RdcUj6ncsrwTjVOS/Ij9BHIA4ggiY0XvsY1dWNBQQxUCJjzGBhS4/JUgg2AzV0aDOYeR7u8LHCPC2hpnCc5QmD3g/x6vSgFr8rpd0/XVGp9i8HBYGUOCnuiWWvRId/RM77/Wavg12l4OeVY7WmHvjJxM5N/+VVV59rD/qfrTs4cKFn2vOnu3faZ9C62jrx9dtPJLuURjmq7phf1fUZMe+AZm/VW7yYrc5R9+VPbCL8YNkr+34yfYnEvK88lUNhVPrwatCmMSU5jhfiwgjXVkUOD6FZziDOdSnzQrpaT5TqoUCjTj+BLSygfo4Y/5
    > 1
    !?
    ( ⾒) 0

    View full-size slide

  33. / 2/2
    Bronstein MM, Bruna J, Cohen T, Veličković P.
    Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges.
    arXiv [cs.LG]. 2021. http://arxiv.org/abs/2104.13478
    for all
    AAAConichVG7SgNBFD1Z3/EVtRFECEbFKkwkqKQK2ohVHkaFJIbddRKX7IvdSUCDlZ0/YGGlYCF2WvgBNv6ART5BLBVsLLzZLAQN6l1m58yZe+6cmavYuuYKxpoBqae3r39gcCg4PDI6Nh6amNxxrZqj8pxq6Zazp8gu1zWT54QmdL5nO1w2FJ3vKtWN1v5unTuuZpnb4sjmRUOumFpZU2VBVCk0W04UDFkcKkojc7J/EC4IK9whSqEIizIvwt0g5oMI/EhZoQcUcAALKmowwGFCENYhw6UvjxgYbOKKaBDnENK8fY4TBElboyxOGTKxVfpXaJX3WZPWrZqup1bpFJ2GQ8owFtgzu2Fv7Indshf2+Wuthlej5eWIZqWt5XZp/Gw6+/GvyqBZ4LCj+tOzQBlrnleNvNse07qF2tbXj8/fsonMQmORXbFX8n/JmuyRbmDW39XrNM9c/OFHIS/0YtSg2M92dIOd5WhsJRpPxyPJdb9Vg5jBHJaoH6tIYhMp5Kj+KW5wh3tpXtqS0lK2nSoFfM0UvoVU+AKNf5vS
    f : Rd ! R
    AAACmnichVHLLgRBFD3T3u9BJBIWExOPhUxqRBCrCRtiM8a8EsOkuxUq+pXumgk68wN+wMKKxAIf4ANs/ICFTxBLEhsLd3o6EQS3U12nTt1z61RdzTGEJxl7jChNzS2tbe0dnV3dPb190f6BvGdXXJ3ndNuw3aKmetwQFs9JIQ1edFyumprBC9rBcn2/UOWuJ2wrK48cvmWqe5bYFboqiSpHhw6nDydjJWHFSqYq9zXNz9S2d8rROEuwIGI/QTIEcYSRtqO3KGEHNnRUYILDgiRsQIVH3yaSYHCI24JPnEtIBPscNXSStkJZnDJUYg/ov0erzZC1aF2v6QVqnU4xaLikjGGcPbAr9sLu2Q17Yu+/1vKDGnUvRzRrDS13yn0nwxtv/6pMmiX2P1V/epbYxULgVZB3J2Dqt9Ab+urx6cvGYmbcn2AX7Jn8n7NHdkc3sKqv+uU6z5z94UcjL/Ri1KDk93b8BPmZRHIuMbs+G08tha1qxwjGMEX9mEcKK0gjR/V9nOMaN8qosqSsKmuNVCUSagbxJZTsB6MZmAU=
    x, x0 2 Rd
    -Lipschitz
    AAACqHichVFNLwNRFD0d39/FRmIz0fhM2ryKIFYNGwuL+qgSFZkZr0y8zoyZ16a0fgB/wMKKxEIsLLG28Qcs/ASxJLGxcDudRBDcybx73nn33Dlvru4I05OMPYaUmtq6+obGpuaW1rb2jnBn17Jn512Dpwxb2O6KrnlcmBZPSVMKvuK4XMvpgqf1nZnKebrAXc+0rSW55/D1nLZlmVnT0CRRG+FIOTtUHI7SMjhcVjOC73pCs6Q6p2bKajFaHKRMVSzG/FB/gngAIggiaYevkcEmbBjIIwcOC5KwgAaPnjXEweAQt44ScS4h0z/nOEAzafNUxalCI3aH1i3arQWsRftKT89XG/QVQa9LShX97IFdsBd2zy7ZE3v/tVfJ71HxskdZr2q5s9Fx1LP49q8qR1li+1P1p2eJLCZ9ryZ5d3ymcgujqi/sH78sTi30lwbYGXsm/6fskd3RDazCq3E+zxdO/vCjkxf6YzSg+Pdx/ATLo7H4eGxsfiySmA5G1Yhe9GGI5jGBBGaRRIr6H+IKN7hVRpSkklZWq6VKKNB040so+gci3ZxP
    |f(x) f(x0)| 6 Lkx x0k
    AAAChHichVG7SgNBFD1ZNcZ31EawCQbFQsJE4wMLCdpYWJjERCGK7K6TOLjZXXYngRj8AW0VCysFC/ED/AAbf8AinyCWEWwsvNksiAb1LrNz5sw9d87M1WxDuJKxekDp6OwKdod6evv6BwaHwsMjOdcqOzrP6pZhOTua6nJDmDwrhTT4ju1wtaQZfFs7Wmvub1e44wrL3JJVm++V1KIpCkJXJVGpjf1wlMWYF5F2EPdBFH5sWuEH7OIAFnSUUQKHCUnYgAqXvjziYLCJ20ONOIeQ8PY5TtBL2jJlccpQiT2if5FWeZ81ad2s6XpqnU4xaDikjGCSPbM71mBP7J69sI9fa9W8Gk0vVZq1lpbb+0OnY5n3f1UlmiUOv1R/epYoYMnzKsi77THNW+gtfeX4spFZTk/WptgNeyX/16zOHukGZuVNv03x9NUffjTyQi9GDYr/bEc7yM3G4guxRCoRTa76rQphHBOYpn4sIol1bCJL9TnOcI4LJajMKHPKfCtVCfiaUXwLZeUTeMOPzA==
    L
    Donoho DL,
    High-dimensional data analysis: The curses and blessings of dimensionality.
    Plenary Lecture, AMS National Meeting on Mathematical Challenges of the 21st Century. 2000.
    (Lipschitz ) ⾒
    ε ( /ε)d Sobolev class
    d= 100 d= 10 =100

    View full-size slide

  34. Underspecification Rashomon
    ( )
    CV
    Underspeci cation
    (
    specify )
    D’Amour et al., Underspecification Presents Challenges for Credibility in Modern Machine Learning.
    J Mach Learn Res, 2022; 23: 1-61. https://ai.googleblog.com/2021/10/how-underspecification-presents.html

    View full-size slide

  35. Underspecification Rashomon
    Underspeci cation
    (
    specify )
    specify

    View full-size slide

  36. Underspecification Rashomon
    Underspeci cation
    (
    specify )
    specify
    (Underspeci cation)
    α

    View full-size slide

  37. 2
    ( big data )
    ( )

    View full-size slide

  38. 2
    ( big data )
    ( )

    View full-size slide

  39. Kernel Ridge (RBF)
    Neural Network (MLP)
    Gradient Boosted Trees
    SVR (RBF) Gaussian Process (RBF)
    Random Forest
    Nearest Neighbors Decision Tree

    Nearest Neighbor
    (Neural Network
    )

    View full-size slide

  40. =
    if-then
    AAAClnichVHLSsNAFD3Gd3201Y3gplgUV2UiRcWFFEV02YfVgpWQxLENpklMphUt/oA/4EJcKKiIH+AHuPEHXPQTxGUFNy68TQOiot4wmTNn7rlzZq7mmIYnGGt0SJ1d3T29ff2hgcGh4XAkOrLh2VVX53ndNm23oKkeNw2L54UhTF5wXK5WNJNvanvLrf3NGnc9w7bWxaHDtytqyTJ2DV0VRCmRaEGZiRVNvh8rijIXqiIrkThLMD9iP4EcgDiCSNuRexSxAxs6qqiAw4IgbEKFR98WZDA4xG2jTpxLyPD3OY4RIm2VsjhlqMTu0b9Eq62AtWjdqun5ap1OMWm4pIxhkj2xW9Zkj+yOPbP3X2vV/RotL4c0a20td5TwyVju7V9VhWaB8qfqT88Cu5j3vRrk3fGZ1i30tr52dNrMLWQn61Pskr2Q/wvWYA90A6v2ql9lePbsDz8aeaEXowbJ39vxE2zMJOTZRDKTjKeWglb1YRwTmKZ+zCGFNaSRp/oHOMc1bqQxaVFakVbbqVJHoBnFl5DSHzOslho=
    X2
     ✓1
    yes no
    AAAClnichVHLSsNAFD3Gd3201Y3gplgUV2UiRcWFFEV02YfVgpWQxLENpklMphUt/oA/4EJcKKiIH+AHuPEHXPQTxGUFNy68TQOiot4wmTNn7rlzZq7mmIYnGGt0SJ1d3T29ff2hgcGh4XAkOrLh2VVX53ndNm23oKkeNw2L54UhTF5wXK5WNJNvanvLrf3NGnc9w7bWxaHDtytqyTJ2DV0VRCmRaEGRY0WT78eKosyFqswokThLMD9iP4EcgDiCSNuRexSxAxs6qqiAw4IgbEKFR98WZDA4xG2jTpxLyPD3OY4RIm2VsjhlqMTu0b9Eq62AtWjdqun5ap1OMWm4pIxhkj2xW9Zkj+yOPbP3X2vV/RotL4c0a20td5TwyVju7V9VhWaB8qfqT88Cu5j3vRrk3fGZ1i30tr52dNrMLWQn61Pskr2Q/wvWYA90A6v2ql9lePbsDz8aeaEXowbJ39vxE2zMJOTZRDKTjKeWglb1YRwTmKZ+zCGFNaSRp/oHOMc1bqQxaVFakVbbqVJHoBnFl5DSHzOelho=
    X1
     ✓2
    yes no
    Blue
    AAAClnichVHLSsNAFD2N7/qquim4KZaKqzItRcWFiCK6rNZqoZWQxGkbTJOYTCta/AF/wIW4UFARP8APcOMPuOgniMsKblx4mwZExXrDZM6cuefOmbmqbeiuYKwRkLq6e3r7+geCg0PDI6OhsfFt16o6Gs9qlmE5OVVxuaGbPCt0YfCc7XClohp8R91fae3v1Ljj6pa5JY5svltRSqZe1DVFECWHxnJyMlIw+EGkIMpcKHJKDkVZnHkR+Q0SPojCj7QVekABe7CgoYoKOEwIwgYUuPTlkQCDTdwu6sQ5hHRvn+MEQdJWKYtThkLsPv1LtMr7rEnrVk3XU2t0ikHDIWUEMfbM7liTPbF79sI+/qxV92q0vBzRrLa13JZHT8OZ939VFZoFyl+qjp4Fipj3vOrk3faY1i20tr52fNbMLGzG6tPsir2S/0vWYI90A7P2pl1v8M3zDn5U8kIvRg1K/GzHb7CdjCdm46mNVHRp2W9VPyYxhRnqxxyWsI40slT/EBe4wa0UlhalVWmtnSoFfM0EvoWU/gQ6DJYd
    X2
     ✓4
    yes no
    Red AAAClnichVHLSsNAFD3GV62vVjeCm2JRXJWJ+MKFFEV0WVurBSshiVMNTZOYTCta/AF/wIW4UFARP8APcOMPuOgniEsFNy68SQOiot4wmTNn7rlzZq7mmIYnGGu0SK1t7R2dka5od09vX38sPrDu2VVX53ndNm23oKkeNw2L54UhTF5wXK5WNJNvaOVFf3+jxl3PsK01ceDwrYq6YxklQ1cFUUosXlDkRNHke4mi2OVCVaaUWJKlWBCJn0AOQRJhZOzYHYrYhg0dVVTAYUEQNqHCo28TMhgc4rZQJ84lZAT7HEeIkrZKWZwyVGLL9N+h1WbIWrT2a3qBWqdTTBouKRMYZY/shr2wB3bLntj7r7XqQQ3fywHNWlPLHaX/eCj39q+qQrPA7qfqT88CJcwGXg3y7gSMfwu9qa8dnrzk5rKj9TF2wZ7J/zlrsHu6gVV71S9Xefb0Dz8aeaEXowbJ39vxE6xPpOTp1OTqZDK9ELYqgmGMYJz6MYM0VpBBnurv4wxXuJaGpHlpSVpupkotoWYQX0LKfAA5/pYd
    X1
     ✓5
    yes no
    Red AAAClnichVHLSsNAFD3Gd321uhHcFIviqkykVHEhRRFd9mG1YCUkcWyDaRKTaUWLP+APuBAXCiriB/gBbvwBF/0EcVnBjQtv04CoqDdM5syZe+6cmas5puEJxhodUmdXd09vX39oYHBoeCQcGd3w7Kqr87xum7Zb0FSPm4bF88IQJi84Llcrmsk3tb3l1v5mjbueYVvr4tDh2xW1ZBm7hq4KopRwpKDI0aLJ96NFUeZCVZJKOMbizI/oTyAHIIYg0nb4HkXswIaOKirgsCAIm1Dh0bcFGQwOcduoE+cSMvx9jmOESFulLE4ZKrF79C/RaitgLVq3anq+WqdTTBouKaOYYk/sljXZI7tjz+z911p1v0bLyyHNWlvLHWXkZDz39q+qQrNA+VP1p2eBXcz7Xg3y7vhM6xZ6W187Om3mFrJT9Wl2yV7I/wVrsAe6gVV71a8yPHv2hx+NvNCLUYPk7+34CTZm43IynsgkYqmloFV9mMAkZqgfc0hhDWnkqf4BznGNG2lcWpRWpNV2qtQRaMbwJaT0Bzwelh4=
    X1
     ✓6
    yes no
    Red
    Blue
    AAAClnichVHLSsNAFD3GV3221Y3gplgUV2WiRcWFFEV0Wa3VgpWQxKmGpklMphUt/oA/4EJcKKiIH+AHuPEHXPQTxKWCGxfepAFRUW+YzJkz99w5M1dzTMMTjDVapNa29o7OSFd3T29ffzQWH1j37Kqr87xum7Zb0FSPm4bF88IQJi84Llcrmsk3tPKCv79R465n2NaaOHD4VkXdsYySoauCKCUWLyhyomjyvURR7HKhKpNKLMlSLIjETyCHIIkwsnbsDkVsw4aOKirgsCAIm1Dh0bcJGQwOcVuoE+cSMoJ9jiN0k7ZKWZwyVGLL9N+h1WbIWrT2a3qBWqdTTBouKRMYZY/shr2wB3bLntj7r7XqQQ3fywHNWlPLHSV6PJR7+1dVoVlg91P1p2eBEmYCrwZ5dwLGv4Xe1NcOT15ys6uj9TF2wZ7J/zlrsHu6gVV71S9X+OrpH3408kIvRg2Sv7fjJ1ifSMlTqfRKOpmZD1sVwTBGME79mEYGy8giT/X3cYYrXEtD0py0KC01U6WWUDOILyFlPwA1vpYb
    X1
     ✓3
    yes no
    Red
    AAAClnichVHLSsNAFD3Gd321uhHcFIviqkykWHEhRRFd9mG1YCUkcWyDaRKTaUWLP+APuBAXCiriB/gBbvwBF/0EcVnBjQtv04CoqDdM5syZe+6cmas5puEJxhodUmdXd09vX39oYHBoeCQcGd3w7Kqr87xum7Zb0FSPm4bF88IQJi84Llcrmsk3tb3l1v5mjbueYVvr4tDh2xW1ZBm7hq4KopRwpKDI0aLJ96NFUeZCVZJKOMbizI/oTyAHIIYg0nb4HkXswIaOKirgsCAIm1Dh0bcFGQwOcduoE+cSMvx9jmOESFulLE4ZKrF79C/RaitgLVq3anq+WqdTTBouKaOYYk/sljXZI7tjz+z911p1v0bLyyHNWlvLHWXkZDz39q+qQrNA+VP1p2eBXcz7Xg3y7vhM6xZ6W187Om3mFrJT9Wl2yV7I/wVrsAe6gVV71a8yPHv2hx+NvNCLUYPk7+34CTZm4/JcPJFJxFJLQav6MIFJzFA/kkhhDWnkqf4BznGNG2lcWpRWpNV2qtQRaMbwJaT0Bz4+lh8=
    X1
     ✓7
    yes no
    Blue Blue
    if x2 ≤ θ1 then
    if x1 ≤ θ2 then
    return Blue
    else
    if x2 ≤ θ4 then
    return Red
    else
    if x1 ≤ θ5 then
    return Red
    else
    if x1 ≤ θ6 then
    return Blue
    else
    return Red
    else
    if x1 ≤ θ3 then
    if x2 ≤ θ7 then
    return Blue
    else
    return Blue
    else
    return Red
    AAAB+XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5CUMugjWVE8wHJEfY2e8mS3b1jd08IR36CrdZ2YuuvsfSfuLlcYRIfDDzem2FmXhBzpo3rfjuFjc2t7Z3ibmlv/+DwqHx80tZRoghtkYhHqhtgTTmTtGWY4bQbK4pFwGknmNzN/c4zVZpF8slMY+oLPJIsZAQbKz12B7VBueJW3QxonXg5qUCO5qD80x9GJBFUGsKx1j3PjY2fYmUY4XRW6ieaxphM8Ij2LJVYUO2n2akzdGGVIQojZUsalKl/J1IstJ6KwHYKbMZ61ZuL/3m9xIQ3fspknBgqyWJRmHBkIjT/Gw2ZosTwqSWYKGZvRWSMFSbGprO0JRAzm4m3msA6adeq3lW1/lCvNG7zdIpwBudwCR5cQwPuoQktIDCCF3iFNyd13p0P53PRWnDymVNYgvP1C4sglCA=
    X2
    AAAB+XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKqMegF48RzQOSJcxOZpMh81hmZoWw5BO86tmbePVrPPonTpI9mMSChqKqm+6uKOHMWN//9gpr6xubW8Xt0s7u3v5B+fCoaVSqCW0QxZVuR9hQziRtWGY5bSeaYhFx2opGd1O/9Uy1YUo+2XFCQ4EHksWMYOukx3Yv6JUrftWfAa2SICcVyFHvlX+6fUVSQaUlHBvTCfzEhhnWlhFOJ6VuamiCyQgPaMdRiQU1YTY7dYLOnNJHsdKupEUz9e9EhoUxYxG5ToHt0Cx7U/E/r5Pa+CbMmExSSyWZL4pTjqxC079Rn2lKLB87golm7lZEhlhjYl06C1siMXGZBMsJrJLmRTW4ql4+XFZqt3k6RTiBUziHAK6hBvdQhwYQGMALvMKbl3nv3of3OW8tePnMMSzA+/oFiY2UHw==
    X1
    AAACi3ichVG7SgNBFL1ZXzEaE7URbIIhYhVmNagEi6AIlnmYByQh7K5jMmRf7E4CMfgDljYWsVGwED/AD7DxByzyCWIZwcbCu5sF0WC8y+ycOXPPnTNzZVNlNiek7xMmJqemZ/yzgbn54EIovLhUsI2WpdC8YqiGVZIlm6pMp3nOuEpLpkUlTVZpUW4eOPvFNrVsZujHvGPSqibVdXbKFIkjVarwBuVSTayFoyRO3IiMAtEDUfAibYQfoQInYIACLdCAgg4csQoS2PiVQQQCJnJV6CJnIWLuPoVzCKC2hVkUMyRkm/iv46rssTqunZq2q1bwFBWHhcoIxMgLuScD8kweyCv5/LNW163heOngLA+11KyFLlZyH/+qNJw5NL5VYz1zOIVd1ytD76bLOLdQhvr22dUgl8zGuuvklryh/xvSJ094A739rtxlaLY3xo+MXvDFsEHi73aMgsJmXNyOJzKJaGrfa5UfVmENNrAfO5CCI0hD3u3DJfTgWggKW0JS2BumCj5Psww/Qjj8Ak3rksg=
    ✓1
    AAACi3ichVG7TgJBFL2sL0QR1MbEhkgwVmRAooZYEI2JJQ95JEDI7jrAhH1ldyBB4g9Y2lhgo4mF8QP8ABt/wIJPMJaY2Fh4d9nEKBHvZnbOnLnnzpm5kqEwixMy8AhT0zOzc95538KifykQXF4pWHrblGle1hXdLEmiRRWm0TxnXKElw6SiKim0KLUO7f1ih5oW07UT3jVoVRUbGqszWeRIlSq8SblYi9eCYRIlToTGQcwFYXAjrQcfoQKnoIMMbVCBggYcsQIiWPiVIQYEDOSq0EPORMScfQrn4ENtG7MoZojItvDfwFXZZTVc2zUtRy3jKQoOE5UhiJAXck+G5Jk8kFfy+WetnlPD9tLFWRppqVELXKzlPv5VqThzaH6rJnrmUIc9xytD74bD2LeQR/rO2dUwl8xGepvklryh/xsyIE94A63zLt9laLY/wY+EXvDFsEGx3+0YB4V4NLYTTWQS4dSB2yovrMMGbGE/diEFx5CGvNOHS+jDteAXtoWksD9KFTyuZhV+hHD0BVALksk=
    ✓2
    AAACi3ichVG7SgNBFL1ZXzEaE7URbIIhYhVmNagEi6AIlnmYByQh7K5jMmRf7E4CMfgDljYWsVGwED/AD7DxByzyCWIZwcbCu5sF0WC8y+ycOXPPnTNzZVNlNiek7xMmJqemZ/yzgbn54EIovLhUsI2WpdC8YqiGVZIlm6pMp3nOuEpLpkUlTVZpUW4eOPvFNrVsZujHvGPSqibVdXbKFIkjVarwBuVSLVELR0mcuBEZBaIHouBF2gg/QgVOwAAFWqABBR04YhUksPErgwgETOSq0EXOQsTcfQrnEEBtC7MoZkjINvFfx1XZY3VcOzVtV63gKSoOC5URiJEXck8G5Jk8kFfy+WetrlvD8dLBWR5qqVkLXazkPv5VaThzaHyrxnrmcAq7rleG3k2XcW6hDPXts6tBLpmNddfJLXlD/zekT57wBnr7XbnL0GxvjB8ZveCLYYPE3+0YBYXNuLgdT2QS0dS+1yo/rMIabGA/diAFR5CGvNuHS+jBtRAUtoSksDdMFXyeZhl+hHD4BVRLkss=
    ✓4
    AAACi3ichVG7TgJBFL2sL0QR1MbEhkgwVmRQfIRYEI2JJQ95JEDI7jrAhH1ldyBB4g9Y2lhgo4mF8QP8ABt/wIJPMJaY2Fh4d9nEKBHvZnbOnLnnzpm5kqEwixPS9wgTk1PTM95Z39y8fyEQXFzKW3rLlGlO1hXdLEqiRRWm0RxnXKFFw6SiKim0IDUP7f1Cm5oW07UT3jFoRRXrGqsxWeRIFcu8QblY3a4GwyRKnAiNgpgLwuBGSg8+QhlOQQcZWqACBQ04YgVEsPArQQwIGMhVoIuciYg5+xTOwYfaFmZRzBCRbeK/jquSy2q4tmtajlrGUxQcJipDECEv5J4MyDN5IK/k889aXaeG7aWDszTUUqMauFjJfvyrUnHm0PhWjfXMoQZ7jleG3g2HsW8hD/Xts6tBNpGJdNfJLXlD/zekT57wBlr7Xb5L00xvjB8JveCLYYNiv9sxCvKb0dhONJ6Oh5MHbqu8sAprsIH92IUkHEMKck4fLqEH14Jf2BISwv4wVfC4mmX4EcLRF1Zrksw=
    ✓5
    AAACi3ichVG7TgJBFL2sL0QR1MbEhkgwVmRQgoZYEI2JJQ95JEDI7jrAhH1ldyBB4g9Y2lhgo4mF8QP8ABt/wIJPMJaY2Fh4d9nEKBHvZnbOnLnnzpm5kqEwixMy8AhT0zOzc95538KifykQXF4pWHrblGle1hXdLEmiRRWm0TxnXKElw6SiKim0KLUO7f1ih5oW07UT3jVoVRUbGqszWeRIlSq8SblYS9SCYRIlToTGQcwFYXAjrQcfoQKnoIMMbVCBggYcsQIiWPiVIQYEDOSq0EPORMScfQrn4ENtG7MoZojItvDfwFXZZTVc2zUtRy3jKQoOE5UhiJAXck+G5Jk8kFfy+WetnlPD9tLFWRppqVELXKzlPv5VqThzaH6rJnrmUIc9xytD74bD2LeQR/rO2dUwl8xGepvklryh/xsyIE94A63zLt9laLY/wY+EXvDFsEGx3+0YB4XtaCwRjWfi4dSB2yovrMMGbGE/diEFx5CGvNOHS+jDteAXdoSksD9KFTyuZhV+hHD0BViLks0=
    ✓6
    AAACi3ichVG7TgJBFL2sL0QR1MbEhkgwVmQQooZYEI2JJQ95JEDI7jrAhH1ldyBB4g9Y2lhgo4mF8QP8ABt/wIJPMJaY2Fh4d9nEKBHvZnbOnLnnzpm5kqEwixMy8AhT0zOzc95538KifykQXF4pWHrblGle1hXdLEmiRRWm0TxnXKElw6SiKim0KLUO7f1ih5oW07UT3jVoVRUbGqszWeRIlSq8SblYi9eCYRIlToTGQcwFYXAjrQcfoQKnoIMMbVCBggYcsQIiWPiVIQYEDOSq0EPORMScfQrn4ENtG7MoZojItvDfwFXZZTVc2zUtRy3jKQoOE5UhiJAXck+G5Jk8kFfy+WetnlPD9tLFWRppqVELXKzlPv5VqThzaH6rJnrmUIc9xytD74bD2LeQR/rO2dUwl8xGepvklryh/xsyIE94A63zLt9laLY/wY+EXvDFsEGx3+0YB4XtaGwnmsgkwqkDt1VeWIcN2MJ+7EIKjiENeacPl9CHa8EvxIWksD9KFTyuZhV+hHD0BVIrkso=
    ✓3
    AAACi3ichVG7TgJBFL2sL0QR1MbEhkgwVmRQIoZYEI2JJQ95JEDI7jrAhH1ldyBB4g9Y2lhgo4mF8QP8ABt/wIJPMJaY2Fh4d9nEKBHvZnbOnLnnzpm5kqEwixMy8AhT0zOzc95538KifykQXF4pWHrblGle1hXdLEmiRRWm0TxnXKElw6SiKim0KLUO7f1ih5oW07UT3jVoVRUbGqszWeRIlSq8SblYS9SCYRIlToTGQcwFYXAjrQcfoQKnoIMMbVCBggYcsQIiWPiVIQYEDOSq0EPORMScfQrn4ENtG7MoZojItvDfwFXZZTVc2zUtRy3jKQoOE5UhiJAXck+G5Jk8kFfy+WetnlPD9tLFWRppqVELXKzlPv5VqThzaH6rJnrmUIc9xytD74bD2LeQR/rO2dUwl8xGepvklryh/xsyIE94A63zLt9laLY/wY+EXvDFsEGx3+0YB4XtaGw3Gs/Ew6kDt1VeWIcN2MJ+JCAFx5CGvNOHS+jDteAXdoSksD9KFTyuZhV+hHD0BVqrks4=
    ✓7
    AAAB+XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKqMegF48RzQOSJcxOZpMh81hmZoWw5BO86tmbePVrPPonTpI9mMSChqKqm+6uKOHMWN//9gpr6xubW8Xt0s7u3v5B+fCoaVSqCW0QxZVuR9hQziRtWGY5bSeaYhFx2opGd1O/9Uy1YUo+2XFCQ4EHksWMYOukx3Yv6JUrftWfAa2SICcVyFHvlX+6fUVSQaUlHBvTCfzEhhnWlhFOJ6VuamiCyQgPaMdRiQU1YTY7dYLOnNJHsdKupEUz9e9EhoUxYxG5ToHt0Cx7U/E/r5Pa+CbMmExSSyWZL4pTjqxC079Rn2lKLB87golm7lZEhlhjYl06C1siMXGZBMsJrJLmRTW4ql4+XFZqt3k6RTiBUziHAK6hBvdQhwYQGMALvMKbl3nv3of3OW8tePnMMSzA+/oFiY2UHw==
    X1
    AAAB+XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5CUMugjWVE8wHJEfY2e8mS3b1jd08IR36CrdZ2YuuvsfSfuLlcYRIfDDzem2FmXhBzpo3rfjuFjc2t7Z3ibmlv/+DwqHx80tZRoghtkYhHqhtgTTmTtGWY4bQbK4pFwGknmNzN/c4zVZpF8slMY+oLPJIsZAQbKz12B7VBueJW3QxonXg5qUCO5qD80x9GJBFUGsKx1j3PjY2fYmUY4XRW6ieaxphM8Ij2LJVYUO2n2akzdGGVIQojZUsalKl/J1IstJ6KwHYKbMZ61ZuL/3m9xIQ3fspknBgqyWJRmHBkIjT/Gw2ZosTwqSWYKGZvRWSMFSbGprO0JRAzm4m3msA6adeq3lW1/lCvNG7zdIpwBudwCR5cQwPuoQktIDCCF3iFNyd13p0P53PRWnDymVNYgvP1C4sglCA=
    X2

    View full-size slide

  41. ⾒ d
    1
    ⾒ ( )

    View full-size slide

  42. ( )
    RandomForestRegressor
    (n_estimators=6,
    max_leaf_nodes=8)
    6 × DecisionTreeRegressor(max_leaf_nodes=8)
    = + +
    + + +
    8

    8
    ( )
    min_samples_leaf

    View full-size slide

  43. ( )
    RandomForestRegressor
    (n_estimators=6,
    max_leaf_nodes=8)
    6 × DecisionTreeRegressor(max_leaf_nodes=8)
    = + +
    + + +
    8

    8
    ( )
    min_samples_leaf

    View full-size slide

  44. ( )
    RandomForestRegressor
    (n_estimators=6,
    max_leaf_nodes=8)
    6 × DecisionTreeRegressor(max_leaf_nodes=8)
    = + +
    + + +
    8

    8
    ( )
    min_samples_leaf

    View full-size slide

  45. ( )
    RandomForestRegressor
    (n_estimators=6,
    max_leaf_nodes=8)
    6 × DecisionTreeRegressor(max_leaf_nodes=8)
    = + +
    + + +
    8

    8
    ( )
    min_samples_leaf

    View full-size slide

  46. / ⾒
    PolyReg(1)
    RMSE 0.299
    PolyReg(3)
    RMSE 0.28
    PolyReg(5)
    RMSE 0.225
    PolyReg(7)
    RMSE 0.113
    PolyReg(10)
    RMSE 0.0189
    PolyReg(15)
    RMSE 0.00737
    PolyReg(20)
    RMSE 0.000
    PolyReg(30)
    RMSE 0.000
    ExtraTrees (no bootstrap)
    RMSE 0.000
    ExtraTrees (bootstrap)
    RMSE 0.0121
    Random Forest
    RMSE 0.012
    LGBM
    RMSE 0.0508
    95%༧ଌ۠ؒ 95%༧ଌ۠ؒ 95%༧ଌ۠ؒ 95%༧ଌ۠ؒ
    Problematic overfitting by polynomial regression of order k
    clearly overfitted but harmless (still informative)

    View full-size slide

  47. Interpolator
    0 ( )
    ExtraTrees (no bootstrap) ExtraTrees (no bootstrap) ExtraTrees (no bootstrap) ExtraTrees (no bootstrap)
    Gradient Boosted Trees Gradient Boosted Trees Gradient Boosted Trees Gradient Boosted Trees

    View full-size slide

  48. Interpolator
    Nearest Neighbor (k=1) Nearest Neighbor (k=1) Nearest Neighbor (k=1) Nearest Neighbor (k=1)
    Decision Tree Decision Tree Decision Tree Decision Tree
    0 ( GBDT, -NN, DT )

    View full-size slide

  49. Extra Trees
    Geurts, Ernst, Wehenkel, Extremely randomized trees. Mach Learn 63, 3–42 (2006).
    https://doi.org/10.1007/s10994-006-6226-1
    ExtraTreesRegressor(n_estimators=10)
    RandomForestRegressor(n_estimators=10)
    ( )

    View full-size slide

  50. Local-averaging estimators
    Nearest Neighbor

    View full-size slide

  51. Local-averaging estimators
    Nearest Neighbor
    Nearest Neighbor
    (X, y)
    (Stone , 1977)
    Nadaraya-Watson
    Attention Mechanism
    ICML A Tutorial on Attention in Deep Learning
    https://icml.cc/Conferences/ /ScheduleMultitrack?event=
    Histogram rules on data-dependent partitions (or data-driven histogram methods)
    90 (Nobel, Ann. Statist. ( ), ; Lugosi & Nobel,
    Ann. Statist. (2), 1996) ( Yellow Terror)
    Devroye, Györ , Lugosi, A Probabilistic Theory of Pattern Recognition, . https://www.szit.bme.hu/~gyor /pbook.pdf

    View full-size slide

  52. Interpolation
    ⾒ ( 0 or 0)
    Harmless over tting Benign over tting
    ( )

    View full-size slide

  53. Interpolation
    ⾒ ( 0 or 0)
    Harmless over tting Benign over tting
    ( )
    Hastie et al, Surprises in high-dimensional ridgeless least squares interpolation (2020)
    https://arxiv.org/abs/1903.08560
    Muthukumar et al, Harmless interpolation of noisy data in regression (2019)
    https://arxiv.org/abs/1903.09139
    Bartlett et al, Benign overfitting in linear regression (2019)
    https://arxiv.org/abs/1906.11300
    Belkin et al, Overfitting or perfect fitting? Risk bounds for classification and regression rules
    that interpolate (2018)
    https://arxiv.org/abs/1806.05161
    AAACG3icbVDLSsNAFJ3UV62vqks3g0VaEUsivjZC0Y3uKtg20MQymUzaoZMHMxMhhHyAH+E3uNW1O3HrwqV/4rTNwloPDBzOOZd75zgRo0Lq+pdWmJtfWFwqLpdWVtfWN8qbW20RxhyTFg5ZyE0HCcJoQFqSSkbMiBPkO4x0nOHVyO88EC5oGNzJJCK2j/oB9ShGUkm9csVyiETwAtbMqgkPoMXUrIvgzf59emhk0KzCRKX0uj4GnCVGTiogR7NX/rbcEMc+CSRmSIiuoUfSThGXFDOSlaxYkAjhIeqTrqIB8omw0/FnMrinFBd6IVcvkHCs/p5IkS9E4jsq6SM5EH+9kfif142ld26nNIhiSQI8WeTFDMoQjpqBLuUES5YogjCn6laIB4gjLFV/U1scP1OdGH8bmCXto7pxWj+5Pa40LvN2imAH7IIaMMAZaIBr0AQtgMEjeAYv4FV70t60d+1jEi1o+cw2mIL2+QPP7p7D
    = (X0X + I) 1X0y
    AAACDnicbVDLSsNAFJ3UV62vqODGzWCRVoSSiK+NUHTjsoJtA20sk+m0HTqZhJmJEGL+wW9wq2t34tZfcOmfOG2zsK0HLhzOuZdzOV7IqFSW9W3kFhaXllfyq4W19Y3NLXN7pyGDSGBSxwELhOMhSRjlpK6oYsQJBUG+x0jTG96M/OYjEZIG/F7FIXF91Oe0RzFSWuqYe22PKASvYNkpOUcPyXEKnRKMO2bRqlhjwHliZ6QIMtQ65k+7G+DIJ1xhhqRs2Vao3AQJRTEjaaEdSRIiPER90tKUI59INxn/n8JDrXRhLxB6uIJj9e9FgnwpY9/Tmz5SAznrjcT/vFakepduQnkYKcLxJKgXMagCOCoDdqkgWLFYE4QF1b9CPEACYaUrm0rx/FR3Ys82ME8aJxX7vHJ2d1qsXmft5ME+OABlYIMLUAW3oAbqAIMn8AJewZvxbLwbH8bnZDVnZDe7YArG1y+Uw5p9
    = (X0X)+X0y
    Ridgeless
    Ridge

    View full-size slide

  54. Interpolation
    ⾒ ( 0 or 0)
    Harmless over tting Benign over tting
    ( )
    Belkin M, Hsu D, Mitra PP.
    Overfitting or perfect fitting? risk bounds for classification and regression rules that interpolate. NIPS 2018.
    https://dl.acm.org/doi/10.5555/3327144.3327157
    "Many modern machine learning models are trained to achieve zero or near-zero training error in order to obtain
    near-optimal (but non-zero) test error. This phenomenon of strong generalization performance for "overfitted" /
    interpolated classifiers appears to be ubiquitous in high-dimensional data, having been observed in deep networks,
    kernel machines, boosting and random forests. Their performance is consistently robust even when the data
    contain large amounts of label noise."
    Belkin M.
    Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation.
    Acta Numerica, 2021; 30: 203-248: https://doi.org/10.1017/S0962492921000039
    Weyner AJ, Olson M, Bleich J, and Mease D.
    Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers. J Mach Learn Res.
    2017; 18(48): 1-33. https://jmlr.org/papers/v18/15-240.html

    View full-size slide

  55. (LSH)
    .
    ( )
    ReLU = max(x, )
    (= )
    cf.

    View full-size slide

  56. (LSH)
    Balestriero, Randall. "Max-Affine Splines Insights Into Deep Learning." (2021)
    Diss., Rice University. https://hdl.handle.net/1911/110439
    Balestriero & Baraniuk. A Spline Theory of Deep Learning (ICML 2018)
    https://proceedings.mlr.press/v80/balestriero18b.html
    Balestriero et al., The Geometry of Deep Networks: Power Diagram Subdivision (NeurIPS 2019)
    https://arxiv.org/abs/1905.08443
    .
    ( )
    ReLU = max(x, )
    (= )
    cf.

    View full-size slide

  57. https://www.kaggle.com/kaggle-survey-2021
    State of Data Science and Machine Learning 2021 (Kaggle Survey)
    Q17. Which of the following ML algorithms do you use on a regular basis?
    Linear or Logistic Regression
    Decision Trees or Random Forests
    Gradient Boosting Machines (xgboost, lightgbm, etc)
    Convolutional Neural Networks
    Dense Neural Networks (MLPs, etc)
    Bayesian Approaches
    Recurrent Neural Networks
    Transformer Networks (BERT, gpt-3, etc)
    None or Other
    Generative Adversarial Networks
    Evolutionary Approaches
    0 3,500 7,000 10,500 14,000
    13,852
    11,863
    7,566
    7,410
    4,468
    4,392
    4,228
    2,273
    1,953
    1,353
    963
    # respondents: 25,973
    ( ⾒ ?)

    View full-size slide

  58. ExtraTrees Random Forests
    ( Boosting ⾒ Quantile )
    scikit-optimize
    https://scikit-optimize.github.io/

    View full-size slide

  59. Conformal Prediction
    https://github.com/scikit-learn-contrib/MAPIE
    regressor = ExtraTreesRegressor(max_leaf_nodes=32, bootstrap=True)
    MapieRegressor(regressor,
    method="plus", cv=-1)
    MapieRegressor(regressor,
    method="plus",
    cv=Subsample(n_resampling
    s=50))
    MapieRegressor(regressor,
    method="plus", cv=-1)
    MapieRegressor(regressor,
    method="plus",
    cv=Subsample(n_resampling
    s=50))
    95% prediction intervals
    95% prediction intervals
    90% prediction intervals
    90% prediction intervals
    Jacknife+
    Jacknife+ after bootstrap
    Jacknife+
    Jacknife+ after bootstrap
    Conformal Prediction

    View full-size slide

  60. SWED-8
    SWED-3
    Wang et al., Accelerated discovery of multi-elemental reverse water-gas shift catalysts using extrapolative machine
    learning approach. (2022) https://doi.org/10.26434/chemrxiv-2022-695rj
    ( )

    View full-size slide

  61. .
    . o -the-shelf
    .
    PDF https://itakigawa.page.link/IBISML taki

    View full-size slide

  62. CV ( )
    CV
    ( )
    f
    (Xi
    , y
    i
    ) y
    i
    ≈ ̂
    yi
    = f(Xi
    )
    ( ) ⾒ ( )

    View full-size slide

  63. Physics-informed ML, Geometric ML, Causal ML
    https://doi.org/10.1038/s42254-021-00314-5

    View full-size slide

  64. ( ) ( ) ⾒
    ( ) ⾒

    View full-size slide

  65. ( ) ( ) ⾒
    ( ) ⾒

    ( Symmetry Geometric ML )

    View full-size slide

  66. ( ) ( ) ⾒
    ( ) ⾒

    ( Symmetry Geometric ML )


    https://doi.org/10.1038/s42254-021-00314-5

    View full-size slide

  67. . Two Cultures ( vs )
    ‣ Data Modeling ( ) vs. Algorithmic Modeling ( )
    L. Breiman Statistical Modeling: The Two Cultures ( ) + Cox Efron
    ‣ 20 Observational Studies (2021) +
    Breiman's main point is: If you want prediction, do prediction for its own sake and
    forget about the illusion of representing nature. (Judea Pearl)

    View full-size slide

  68. . Two Cultures ( vs )
    ‣ Data Modeling ( ) vs. Algorithmic Modeling ( )
    L. Breiman Statistical Modeling: The Two Cultures ( ) + Cox Efron
    ‣ 20 Observational Studies (2021) +
    Breiman's main point is: If you want prediction, do prediction for its own sake and
    forget about the illusion of representing nature. (Judea Pearl)
    . p ( HARKing )
    ‣ (A. Lang, )
    ‣ 2019 Nature Retire statistical signi cance Don t say statistically signi cant
    (ASA) (2016) (2019)
    (800 )

    View full-size slide

  69. vs
    (= )
    • “Theory-driven models can be wrong. But data-driven models cannot be wrong or
    right. Data-driven are not trying to describe an underlying reality.”
    David Hand, KDD2018 (Keynote Talk)
    http://videolectures.net/kdd2018_hand_data_science/

    ( )

    View full-size slide

  70. ⾒ (NP-hard )
    Schrittweiser et al. Mastering Atari, Go, chess and shogi by planning with a learned model.
    (“MuZero”, Nature, 2020)
    Davies et al. Advancing mathematics by guiding human intuition with AI. (Nature, 2021)
    Fawzi et al. Discovering faster matrix multiplication algorithms with reinforcement learning.
    (“AlphaTensor”, Nature, 2022)
    ×
    closed /

    View full-size slide

  71. ( )
    ( )
    ( )
    ⾒ (Physics-infomed ML, Causal ML, Geometric ML, )

    View full-size slide

  72. .
    UQ ⾒
    . o -the-shelf
    ( ) Underspeci cation
    .
    ⾒ (Physics-informed ML, Causal ML, Geometric ML)
    + (Pure Math, Pure CS, ⾒ , / )
    PDF https://itakigawa.page.link/IBISML taki

    View full-size slide