XGBoost: A Scalable Tree Boosting System_free

XGBoost論文の解説用資料の公開版です。
詳細版を確認されたい方は、下記よりご購入いただけますのでご確認ください。
https://note.mu/lib_arts/n/nefb511ba4fde

Transcript

1. Copyright @ Liberal Arts Community. All Rights Reserved. XGBoost: A Scalable Tree Boosting System 1 SOK@LiberalArtsCommunity
2. Copyright @ Liberal Arts Community. All Rights Reserved. ໨࣍ • ࣗݾ঺հ • ࿦จ֓؍ • ࿦จৄࡉ • ܾఆ໦(CART)ͷ෮श • Tree Boosting in a Nutshell • Split Finding Algorithms • System Design • ࢀߟจݙ 2
3. Copyright @ Liberal Arts Community. All Rights Reserved. ࣗݾ঺հ 3
4. Copyright @ Liberal Arts Community. All Rights Reserved. ࣗݾ঺հ twitter: @sokei14 ౦ژେֶେֶӃ਺ཧՊֶݚڀՊम࢜՝ఔमྃɻઐ໳͸ෳૉزԿֶɻ ͦͷޙɺϝΨόϯΫͰΫΦϯπͱͯ͠ࢢ৔ϦεΫ؅ཧۀ຿ʹैࣄɻ ݱࡏ͸ϕϯνϟʔͰAI༥ࢿ৹ࠪϞσϧͷ։ൃʹܞΘΔɻAIͰۚ༥αʔϏεͷ มֵΛເݟΔػցֶशΤϯδχΞɻ 4
5. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จ֓؍ 5
6. Copyright @ Liberal Arts Community. All Rights Reserved. ABSTRACT • ͜ͷ࿦จͰ͸XGBoostͱݺ͹ΕΔεέʔϥϒϧ͔ͭend-to-endͳπϦʔϒʔεςΟϯάΞϧΰϦζϜΛ ঺հ͢Δɽ • ఏҊ͢Δख๏ͱͯ͠ҎԼ͕ڍ͛ΒΕ͍ͯΔɽ 1. sparcity-aware-algorithm, weighted quantile sketch → 3ষͰઆ໌ 2. cache-aware access, data compression and shading → 4ষͰઆ໌ 6
7. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จ֓؍ 6ͭͷষͰߏ੒͞Ε͍ͯ·͢ɻ 1. INTRODUCTION 2. TREE BOOSTING IN A NUTSHELL 3. SPLIT FINDING ALGORITHMS 4. SYSTEM DESIGN 5. RELATED WORKS 6. END TO END EVALUATIONS ͕͜͜ϝΠϯ 7
8. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จ֓؍ 2. TREE BOOSTING IN A NUTSHELL XGBoostͷίΞͱͳΔΞϧΰϦζϜʹ͍ͭͯ·ͱΊΒΕ͍ͯΔɽ • tree boostingͷΞϧΰϦζϜͷղઆ • ςΠϥʔల։ʹΑΔϩεؔ਺ͷۙࣅ • Shrinkage • Column Subsampling 8
9. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จ֓؍ 3. SPLIT FINDING ALGORITHMS XGBoostʹ͓͚Δ෼ׂ఺୳ࡧͷ޻෉఺ʹ͍ͭͯड़΂ΒΕ͍ͯΔɽ • جຊͱͳΔExact Greedy Algorithmͷղઆ • ෼ׂީิ఺Λߜͬͯ୳ࡧʢApproximate Algorithmʣ • ॏΈ෇͖෼Ґ఺ͷ࠾༻ • ॏΈ෇͖෼Ґ఺ͷࢉग़ͷߴ଎ԽʢWeighted Quantile Sketchʣ • ܽଛσʔλʹରͯ͠͸default directionΛ࠾༻ʢSparcity-aware Split Findingʣ 9
10. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จ֓؍ 4. SYSTEM DESIGN XGBoostʹ͓͚ΔγεςϜଆͷ޻෉఺ʹ͍ͭͯड़΂ΒΕ͍ͯΔɽ • ιʔτͷܭࢉίετͷ࡟ݮʢColumn Block for Parallel Learningʣ • CSCʹΑΔεύʔεߦྻσʔλѹॖ • σʔλͷϒϩοΫԽ • ܭࢉྔͷൺֱ • ޯ഑৘ใͷϓϦϑΣονʢCache-aware Accessʣ • ϒϩοΫαΠζͷ࠷దԽ • σΟεΫIOͷεϧʔϓοτ޲্ʢϒϩοΫѹॖɾ ϒϩοΫஅยԽʣ 10
11. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จৄࡉ 11
12. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จৄࡉɹͦͷલʹ…ܾఆ໦ͷ෮श ▪ ܾఆ໦ʢCARTʣͱ͸ ͋Δಛ௃࣠ͱᮢ஋ͷେখؔ܎ͷ൑அͷ૊Έ߹ΘͤͰ෼ྨ໰୊΍ճؼ໰୊Λղ ͘ΞϧΰϦζϜͷ͜ͱɽ ܾఆ໦ͷ͏ͪɼԼਤͷΑ͏ʹඞͣೋ෼͞ΕΔ΋ͷΛCARTͱ͍͏ 12
13. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จৄࡉɹͦͷલʹ…ܾఆ໦ͷ෮श ▪ ܾఆ໦ʢCARTʣͱ͸ ϊʔυͱϊʔυΛ݁ͿϦϯΫ͔Βߏ੒͞Ε͍ͯΔɽϊʔυʹ͍ͭͯ͸໦ͷͲ ͷ෦෼ʹҐஔ͢ΔʹΑͬͯ࣍ͷΑ͏ʹ۠ผ͞Ε͍ͯΔɽ ໊લ ҙຯ ࠜϊʔυ ໦ͷҰ൪্ʹ͋Δϊʔυ ༿ϊʔυʢϦʔϑʣ ໦ͷҰ൪Լʹ͋Δϊʔυ ಺෦ϊʔυ ࠜϊʔυͱ༿ϊʔυҎ֎ͷϊʔυ ༿ϊʔυ ࠜϊʔυ ಺෦ϊʔυ ϦϯΫ 13
14. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จৄࡉɹͦͷલʹ…ܾఆ໦ͷ෮श • 14
15. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จৄࡉɹͦͷલʹ…ܾఆ໦ͷ෮श • 15
16. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จৄࡉɹͦͷલʹ…ܾఆ໦ͷ෮श • 16
17. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จৄࡉɹͦͷલʹ…ܾఆ໦ͷ෮श • 17
18. Copyright @ Liberal Arts Community. All Rights Reserved. ࿦จৄࡉɹͦͷલʹ…ܾఆ໦ͷ෮श • 18
19. Copyright @ Liberal Arts Community. All Rights Reserved. TREE BOOSTING IN A NUTSHELL 19