Slide 1
Slide 1 text
Copyright @ Liberal Arts Community. All Rights Reserved.
XGBoost: A Scalable Tree
Boosting System
1
SOK@LiberalArtsCommunity
Slide 2
Slide 2 text
Copyright @ Liberal Arts Community. All Rights Reserved.
࣍
• ࣗݾհ
• จ֓؍
• จৄࡉ
• ܾఆ(CART)ͷ෮श
• Tree Boosting in a Nutshell
• Split Finding Algorithms
• System Design
• ࢀߟจݙ
2
Slide 3
Slide 3 text
Copyright @ Liberal Arts Community. All Rights Reserved.
ࣗݾհ
3
Slide 4
Slide 4 text
Copyright @ Liberal Arts Community. All Rights Reserved.
ࣗݾհ
twitter: @sokei14
౦ژେֶେֶӃཧՊֶݚڀՊम࢜՝ఔमྃɻઐෳૉزԿֶɻ
ͦͷޙɺϝΨόϯΫͰΫΦϯπͱͯ͠ࢢϦεΫཧۀʹैࣄɻ
ݱࡏϕϯνϟʔͰAI༥ࢿ৹ࠪϞσϧͷ։ൃʹܞΘΔɻAIͰۚ༥αʔϏεͷ
มֵΛເݟΔػցֶशΤϯδχΞɻ
4
Slide 5
Slide 5 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จ֓؍
5
Slide 6
Slide 6 text
Copyright @ Liberal Arts Community. All Rights Reserved.
ABSTRACT
• ͜ͷจͰXGBoostͱݺΕΔεέʔϥϒϧ͔ͭend-to-endͳπϦʔϒʔεςΟϯάΞϧΰϦζϜΛ
հ͢Δɽ
• ఏҊ͢Δख๏ͱͯ͠ҎԼ͕ڍ͛ΒΕ͍ͯΔɽ
1. sparcity-aware-algorithm, weighted quantile sketch → 3ষͰઆ໌
2. cache-aware access, data compression and shading → 4ষͰઆ໌
6
Slide 7
Slide 7 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จ֓؍
6ͭͷষͰߏ͞Ε͍ͯ·͢ɻ
1. INTRODUCTION
2. TREE BOOSTING IN A NUTSHELL
3. SPLIT FINDING ALGORITHMS
4. SYSTEM DESIGN
5. RELATED WORKS
6. END TO END EVALUATIONS
͕͜͜ϝΠϯ
7
Slide 8
Slide 8 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จ֓؍
2. TREE BOOSTING IN A NUTSHELL
XGBoostͷίΞͱͳΔΞϧΰϦζϜʹ͍ͭͯ·ͱΊΒΕ͍ͯΔɽ
• tree boostingͷΞϧΰϦζϜͷղઆ
• ςΠϥʔల։ʹΑΔϩεؔͷۙࣅ
• Shrinkage
• Column Subsampling
8
Slide 9
Slide 9 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จ֓؍
3. SPLIT FINDING ALGORITHMS
XGBoostʹ͓͚Δׂ୳ࡧͷʹ͍ͭͯड़ΒΕ͍ͯΔɽ
• جຊͱͳΔExact Greedy Algorithmͷղઆ
• ׂީิΛߜͬͯ୳ࡧʢApproximate Algorithmʣ
• ॏΈ͖Ґͷ࠾༻
• ॏΈ͖Ґͷࢉग़ͷߴԽʢWeighted Quantile Sketchʣ
• ܽଛσʔλʹରͯ͠default directionΛ࠾༻ʢSparcity-aware Split
Findingʣ
9
Slide 10
Slide 10 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จ֓؍
4. SYSTEM DESIGN
XGBoostʹ͓͚ΔγεςϜଆͷʹ͍ͭͯड़ΒΕ͍ͯΔɽ
• ιʔτͷܭࢉίετͷݮʢColumn Block for Parallel Learningʣ
• CSCʹΑΔεύʔεߦྻσʔλѹॖ
• σʔλͷϒϩοΫԽ
• ܭࢉྔͷൺֱ
• ޯใͷϓϦϑΣονʢCache-aware Accessʣ
• ϒϩοΫαΠζͷ࠷దԽ
• σΟεΫIOͷεϧʔϓοτ্ʢϒϩοΫѹॖɾ ϒϩοΫஅยԽʣ
10
Slide 11
Slide 11 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จৄࡉ
11
Slide 12
Slide 12 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จৄࡉɹͦͷલʹ…ܾఆͷ෮श
■ ܾఆʢCARTʣͱ
͋Δಛ࣠ͱᮢͷେখؔͷஅͷΈ߹ΘͤͰྨճؼΛղ
͘ΞϧΰϦζϜͷ͜ͱɽ
ܾఆͷ͏ͪɼԼਤͷΑ͏ʹඞͣೋ͞ΕΔͷΛCARTͱ͍͏
12
Slide 13
Slide 13 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จৄࡉɹͦͷલʹ…ܾఆͷ෮श
■ ܾఆʢCARTʣͱ
ϊʔυͱϊʔυΛ݁ͿϦϯΫ͔Βߏ͞Ε͍ͯΔɽϊʔυʹ͍ͭͯͷͲ
ͷ෦ʹҐஔ͢ΔʹΑͬͯ࣍ͷΑ͏ʹ۠ผ͞Ε͍ͯΔɽ
໊લ ҙຯ
ࠜϊʔυ ͷҰ൪্ʹ͋Δϊʔυ
༿ϊʔυʢϦʔϑʣ ͷҰ൪Լʹ͋Δϊʔυ
෦ϊʔυ ࠜϊʔυͱ༿ϊʔυҎ֎ͷϊʔυ
༿ϊʔυ
ࠜϊʔυ
෦ϊʔυ
ϦϯΫ
13
Slide 14
Slide 14 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จৄࡉɹͦͷલʹ…ܾఆͷ෮श
•
14
Slide 15
Slide 15 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จৄࡉɹͦͷલʹ…ܾఆͷ෮श
•
15
Slide 16
Slide 16 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จৄࡉɹͦͷલʹ…ܾఆͷ෮श
•
16
Slide 17
Slide 17 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จৄࡉɹͦͷલʹ…ܾఆͷ෮श
•
17
Slide 18
Slide 18 text
Copyright @ Liberal Arts Community. All Rights Reserved.
จৄࡉɹͦͷલʹ…ܾఆͷ෮श
•
18
Slide 19
Slide 19 text
Copyright @ Liberal Arts Community. All Rights Reserved.
TREE BOOSTING IN A NUTSHELL
19