Share
Embed
[解説資料] Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl
進化的アルゴリズムに基づくシンボリック回帰のライブラリであるPySRの論文についてまとめた資料です。
元論文: https://arxiv.org/abs/2305.01582
Shota Kato
September 16, 2024
Transcript
9 slides
-
Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl Miles
Cranmer จϦϯΫɿhttps://arxiv.org/abs/2305.01582 ژେֶ Ճ౻ ↅଠ ͱ͘ʹऍ͕ͳ͍ݶΓɼਤදࣄྫհจ͔ΒͷҾ༻Ͱ͢ ˞հऀͷίϝϯτͰ͢ -
l 4ZNCPMJD3FHSFTTJPO 43 ͷ֓؍ΛѲ͢ΔͨΊɽ l 1ZUIPOͰ༻Ͱ͖Δ43ϥΠϒϥϦͷதͰ ͓ͦΒ͘࠷༗໊Ͱɼཧղ͖͢ͱஅͨͨ͠Ίɽ l ඃҾ༻ɿʢ(PPHMF4DIPMBSʣ (JU)VCTUBSɿL̈ʢ݄࣌ʣ
ຊհͷత 1Z43ͷΞϧΰϦζϜΛཧղ͢Δɽ ˞1Z434ZNCPMJD3FHSFTTJPOKMΛݺͼग़͍ͯ͠Δɽ ຊࢿྉͰࣔ͢จதͷͱ࠷৽ͷ1Z43ͷσϑΥϧτҟͳΔ߹ ͕͋ΔͨΊҙɽ ͳͥ͜ͷจΛબΜ͔ͩʁ 1 †: https://github.com/MilesCranmer/PySR -
PySRͷΞϧΰϦζϜʢAlgorithm 1ʣ 2 ೖྗ දݱΛݟ͚͍ͭͨ σʔληοτ 𝑋 𝐻 = pysr(𝑋)
ਐԽతΞϧΰϦζϜ [Koza, 1994]ͷվྑ൛ ग़ྗ ֤ෳࡶʹରͯ͠ ࠷ਖ਼֬ͳදݱ 𝐻 ॳظूஂͷ࡞ 𝐿ݸͷࣜΛؚΉू߹𝑃! ɼ࠷ਖ਼֬ͳදݱΛอଘ͢Δू߹𝑀! ͱ𝐻Λ ࡞͢Δ (𝑖 = 1, … , 𝑛") ɽ ਐԽ୯७Խ࠷దԽ 1) 𝑃! ʹਐԽΛద༻͢Δɽ 2) 𝑃! தͷ֤ࣜΛ୯७Խ͠ɼ࠷దԽͰఆΛٻΊΔɽ ࠷ਖ਼֬ͳදݱͷอଘ 1) ֤ෳࡶʹରͯ͠𝑃! ͷதͰ࠷ਖ਼֬ͳදݱΛ𝑀! ͱ͢Δɽ 2) 𝑀! ∪ 𝐻ͷதͰ࠷ਖ਼֬ͳදݱΛ𝐻ͱ͢Δɽ Ҡॅ 𝑃! தͷ֤ࣜΛ֬𝛼Ͱ𝑀! (𝑖 ≠ 𝑗)ʢ·ͨ𝐻ʣͷཁૉͰஔ͖͑Δɽ 𝑖 = 1, … , 𝑛" ʹରͯ͠ਐԽʙҠॅΛద༻ͯ͠ 𝑃! , 𝑀! , 𝐻Λߋ৽͢Δɽ 𝑛#$%& ճ܁Γฦͨ͠Βऴྃ͢Δɽ -
𝐿ݸͷࣜΛؚΉू߹𝑃! ɼ࠷ਖ਼֬ͳදݱΛอଘ͢Δू߹𝑀! ͱ𝐻Λ࡞͢Δ (𝑖 = 1, … , 𝑛" )
ɽ l σϑΥϧτͷύϥϝʔλ § ֤ू߹ʹؚ·ΕΔࣜͷɿ𝐿 = 1000 § ू߹ͷɿ𝑛" = 40 l ॳظूஂʹؚ·ΕΔࣜͷෳࡶͱ͢Δɽ ॳظूஂͷ࡞ 3 ෳࡶ ࣜΛߏͰදͨ͠ͱ͖ͷϊʔυͷ ʢʹมɾԋࢉࢠɾఆͷ૯ʣɽ ˞ϢʔβಠࣗͷෳࡶͷఆٛՄೳɽ 0.86 𝑦 1.15 × + 𝑥' 3.2 + ෳࡶɿ3 ෳࡶɿ5 -
τʔφϝϯτબΞϧΰϦζϜʢઆ໌Qʣʹج͍ͮͯબͨࣜ͠ʹରͯ͠ɼ ಥવมҟ·ͨަ伹Λ𝑛# ճ܁Γฦ͠ద༻ͯ͠ߋ৽͢Δɽ l ಥવมҟʢཚͷʼ 𝑝(&)** ͷͱ͖ʣ l ࣄલʹܾΊͨنଇͷ͍ͣΕ͔Λ༻͍ͯࣜΛมߋ͢Δʢઆ໌Qʣɽ l
ަ伹ʢཚͷ ≤ 𝑝(&)** ͷͱ͖ʣ ͭͷࣜ𝐸' ͱ𝐸+ ͷҰ෦ΛೖΕସ͑ͯ ɼ ৽ͨͳࣜ𝐸' ∗ͱ𝐸+ ∗Λੜ͢Δɽ ࠷ݹ͍ͭͷࣜΛ 𝐸' ∗ ͱ 𝐸+ ∗ʹஔ͖͑Δɽ ࡞ͨࣜ͠𝐸' ∗ ͱ 𝐸+ ∗ ੍͕݅Λຬͨ͢·ͰrΛ܁Γฦ͢ɽ ਐԽʢAlgorithm 2ʣ 4 0.86 𝑦 1.15 × + 𝑦 𝑥 ^ ަ伹 𝑥 0.86 + 𝑦 ^ 𝑦 1.15 × -
l Թʹґଘ͢Δ֬Ͱࣜ𝐸Λಥવมҟͤ͞Δɽ l ಥવมҟͷύλʔϯ௨Γʢύλʔϯ𝑖Λ༻͍Δ֬ύϥϝʔλ 𝑤! ͱͯ͠ઃఆ͢Δʣɽ ఆΛมߋ͢Δɽ ԋࢉࢠΛมߋ͢Δɽ
ࠜϊʔυ͔ࢠϊʔυʹϥϯμϜͳϊʔυΛՃ͢Δɽ ϥϯμϜͳϊʔυΛૠೖ͢Δɽ ͋ΔϊʔυͱͦͷϊʔυҎԼͷࢠϊʔυΛ ఆ·ͨมʹஔ͖͑Δɽ ࣜΛ୯७Խ͢Δɽ શ͘৽͍ࣜ͠ʹมߋ͢Δɽ Կ͠ͳ͍ɽ ಥવมҟʢAlgorithm 3ʣ 5 0.86 𝑦 1.15 × − 0.86 𝑦 1.15 × + ྫʣύλʔϯ2 ԋࢉࢠͷมߋ -
ࣜू߹𝑃͔Β𝑛$ ݸͷࣜΛؚΉαϒू߹𝑄Λ࡞͠ɼ𝑄 ͔ΒͭͷཁૉΛબ͢Δɽ l ࠷ྑʹଛࣦ͕࠷খɽଛࣦ༧ଌޡ͕ࠩେ͖͍ࣜಉ͡ෳࡶͷࣜΛආ͚ΔΑ͏ʹઃܭɽ l σϑΥϧτͷύϥϝʔλɿ𝑛* = 12ɼ 𝑝-./012341-
= 0.90 τʔφϝϯτબʢAlgorithm 4ʣ 6 :&4 1.21 $ 2.4 × + ( − ࠷ྑͷཁૉ 𝐸 rand() ≥ 𝑝-./012341- PS𝑙𝑒𝑛𝑔𝑡ℎ(𝑄) == 1 /0 𝐸 Λ 𝑄 ͔Βআ͠ɼΓͷࣜ ͷதͰ࠷ྑͷཁૉΛ 𝐸 ͱ͢Δɽ 𝐸 Λग़ྗ͢Δɽ ࠷ྑͷཁૉ 𝐸 Λ 𝑄 ͔Βબ͢Δɽ ࣜू߹ 𝑃ʢࣜ 𝐿 ݸʣ αϒू߹ 𝑄ʢࣜ 𝑛* ݸʣ 0.86 % 1.15 × + ! " ^ 1.21 $ 2.4 × + ( − -
1Z43Ͱෳͷ͕ࣜಘΒΕΔɽ࠷ऴతʹɼ ʢMPTTʻ࠷খͷMPTTYʣΛຬͨࣜ͢ͷதͰɼTDPSF͕࠷ߴͷࣜΛग़ྗ͢Δɽ 𝑠𝑐𝑜𝑟𝑒 = − log( OPQQ! OPQQ!"# )/(𝐶R −
𝐶RST ) 𝑙𝑜𝑠𝑠! ࣜ 𝑖 ͷMPTT ༧ଌޡࠩ ɼ 𝐶! ࣜ 𝑖 ͷෳࡶ ײతʹɼෳࡶͷ্͕Γ෯ʹରͯ͠MPTT͕େ෯ʹݮগͨ͠ͱ͖ͷࣜΛग़ྗ͢Δɽ ࠷ऴతʹग़ྗ͢ΔࣜҾʢNPEFM@TFMFDUJPOʣͰઃఆՄೳʢ࠷MPTTPS࠷ߴTDPSFʣɽ ࠷ऴతʹಘΒΕΔࣜ 7 1Z43Λ༻͍ͨͱ͖ͷग़ྗྫɽෳͷෳࡶΛ༗͢ΔࣜΛMPTTͱTDPSFͱͱʹग़ྗ͢Δɽ -
1Z43ɺՊֶσʔλʹಛԽͨ͠43ϥΠϒϥϦ l ϚϧνίΞɾϚϧνϊʔυͰͷࢄॲཧ͕ՄೳͰɼ େنσʔλʹద༻Մೳɽ l ༷ʑͳσʔλʹద༻͢ΔͨΊͷػೳΛ࣋ͭɽ ྫ͑ɼΨεաఔճؼʹΑΔϊΠζআڈɼ ޯϒʔεςΟϯάʹجͮ͘ಛྔબͳͲɽ l 1ZUIPOͱ+VMJBͰར༻ՄೳͳΦʔϓϯιʔεϥΠϒϥϦ
l ֦ுੑ͕ߴ͘ɼࣗ࡞ͷԋࢉࢠଛࣦؔΛ؆୯ʹಋೖͰ͖Δɽ l 4ZN1Zɼ1Z5PSDIɼ+"9ͱͷ࿈ܞ͕༰қͰɼଞπʔϧͱͷ Έ߹Θͤॊೈɽ $PMBCσϞϦϯΫ IUUQTDPMBCSFTFBSDIHPPHMFDPNHJUIVC.JMFT$SBONFS1Z43 CMPCNBTUFSFYBNQMFTQZTS@EFNPJQZOC ·ͱΊʴଞͷSRख๏ͱൺͨಛ 8 PySR Eureqa Compiled X X Multi-core X X Multi-node X ⇥ Scalability GPU-capable ⇥ ⇥ No pre-training X X Denoising X X Feature selection X X Differential equations ⇥ X High-dimensional ⇥ ⇥ Practicality Full Pareto curve X X API X ⇥ SymPy Interface X ⇥ Interfacing Deep Learning export X ⇥ Expressivity score 4 5 Open-source X ⇥ Real Constants X X Custom operators X ⇥ Discontinuous operators X X Custom losses X X Symbolic Constraints X ⇥ Custom complexity X X Extensibility Custom types X ⇥ Citation [self] [11] - Code Expressivity scores: (1a) Pre-trained on equations gener 13