Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
XGBoost: A Scalable Tree Boosting System_free
LiberalArts
October 20, 2019
Technology
0
1.5k
XGBoost: A Scalable Tree Boosting System_free
XGBoost論文の解説用資料の公開版です。
詳細版を確認されたい方は、下記よりご購入いただけますのでご確認ください。
https://note.mu/lib_arts/n/nefb511ba4fde
LiberalArts
October 20, 2019
Tweet
Share
More Decks by LiberalArts
See All by LiberalArts
Inductive Bias and Graph Networks
liberalarts
2
800
Tree-based Algorithm by Python
liberalarts
1
78
Math Exercise about Maximum likelihood estimation
liberalarts
1
64
Probability distribution and Maximum likelihood estimation
liberalarts
1
1.1k
How to Read Papers
liberalarts
0
920
Transformer and Graph Neural Network
liberalarts
6
3.1k
Wasserstein GAN
liberalarts
0
730
NGBoost: Natural Gradient Boosting for Probabilistic Prediction
liberalarts
0
1.5k
On the Number of Linear Regions of Deep Neural Networks
liberalarts
0
620
Other Decks in Technology
See All in Technology
Istioを活用したセキュアなマイクロサービスの実現/Secure Microservices with Istio
ido_kara_deru
3
440
大声で伝えたい!定時に帰る方法
sbtechnight
0
250
テスト自動化を最速で軌道に乗せるために
nozomiito
0
150
CityGMLとFBXの連携で地理空間のエンタメ化
soh_mitian
0
760
EC/CRMの自社サービス開発をマネジメントするようになって1年でやってきたこととこれから / devio2022-takano-sho-road-to-good-development-team-management
masaru_b_cl
0
440
JAWS-UG 朝会 #36 登壇資料
takakuni
1
580
Continuous Architecture Design for Modernization
humank
3
530
ECS Fargate+Mackerelにおける監視費用を削減するまでの話
nulabinc
PRO
1
510
私のAWS愛を聞け!ここが好きだよAmazon FSx for NetApp ONTAP
non97
0
800
Settlement simulation testing to ensure correct settlement processing
applepine1125
2
1.4k
#awsbasics [LT] サーバレスECにおける Step Functions の使い方
miu_crescent
0
860
DeFiChain Tech Talk - DFI Uniswap Staking, DeFi Options & DeFi Meta Chain
uzyn
0
120
Featured
See All Featured
A Philosophy of Restraint
colly
192
15k
How to name files
jennybc
40
63k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
6
580
GitHub's CSS Performance
jonrohan
1020
420k
Gamification - CAS2011
davidbonilla
75
3.9k
A better future with KSS
kneath
226
16k
Designing for Performance
lara
597
64k
The World Runs on Bad Software
bkeepers
PRO
57
5.4k
From Idea to $5000 a Month in 5 Months
shpigford
373
44k
Fireside Chat
paigeccino
13
1.4k
Building Your Own Lightsaber
phodgson
95
4.7k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
11
4.9k
Transcript
Copyright @ Liberal Arts Community. All Rights Reserved. XGBoost: A
Scalable Tree Boosting System 1 SOK@LiberalArtsCommunity
Copyright @ Liberal Arts Community. All Rights Reserved. ࣍ •
ࣗݾհ • จ֓؍ • จৄࡉ • ܾఆ(CART)ͷ෮श • Tree Boosting in a Nutshell • Split Finding Algorithms • System Design • ࢀߟจݙ 2
Copyright @ Liberal Arts Community. All Rights Reserved. ࣗݾհ 3
Copyright @ Liberal Arts Community. All Rights Reserved. ࣗݾհ twitter:
@sokei14 ౦ژେֶେֶӃཧՊֶݚڀՊम࢜՝ఔमྃɻઐෳૉزԿֶɻ ͦͷޙɺϝΨόϯΫͰΫΦϯπͱͯ͠ࢢϦεΫཧۀʹैࣄɻ ݱࡏϕϯνϟʔͰAI༥ࢿ৹ࠪϞσϧͷ։ൃʹܞΘΔɻAIͰۚ༥αʔϏεͷ มֵΛເݟΔػցֶशΤϯδχΞɻ 4
Copyright @ Liberal Arts Community. All Rights Reserved. จ֓؍ 5
Copyright @ Liberal Arts Community. All Rights Reserved. ABSTRACT •
͜ͷจͰXGBoostͱݺΕΔεέʔϥϒϧ͔ͭend-to-endͳπϦʔϒʔεςΟϯάΞϧΰϦζϜΛ հ͢Δɽ • ఏҊ͢Δख๏ͱͯ͠ҎԼ͕ڍ͛ΒΕ͍ͯΔɽ 1. sparcity-aware-algorithm, weighted quantile sketch → 3ষͰઆ໌ 2. cache-aware access, data compression and shading → 4ষͰઆ໌ 6
Copyright @ Liberal Arts Community. All Rights Reserved. จ֓؍ 6ͭͷষͰߏ͞Ε͍ͯ·͢ɻ
1. INTRODUCTION 2. TREE BOOSTING IN A NUTSHELL 3. SPLIT FINDING ALGORITHMS 4. SYSTEM DESIGN 5. RELATED WORKS 6. END TO END EVALUATIONS ͕͜͜ϝΠϯ 7
Copyright @ Liberal Arts Community. All Rights Reserved. จ֓؍ 2.
TREE BOOSTING IN A NUTSHELL XGBoostͷίΞͱͳΔΞϧΰϦζϜʹ͍ͭͯ·ͱΊΒΕ͍ͯΔɽ • tree boostingͷΞϧΰϦζϜͷղઆ • ςΠϥʔల։ʹΑΔϩεؔͷۙࣅ • Shrinkage • Column Subsampling 8
Copyright @ Liberal Arts Community. All Rights Reserved. จ֓؍ 3.
SPLIT FINDING ALGORITHMS XGBoostʹ͓͚Δׂ୳ࡧͷʹ͍ͭͯड़ΒΕ͍ͯΔɽ • جຊͱͳΔExact Greedy Algorithmͷղઆ • ׂީิΛߜͬͯ୳ࡧʢApproximate Algorithmʣ • ॏΈ͖Ґͷ࠾༻ • ॏΈ͖Ґͷࢉग़ͷߴԽʢWeighted Quantile Sketchʣ • ܽଛσʔλʹରͯ͠default directionΛ࠾༻ʢSparcity-aware Split Findingʣ 9
Copyright @ Liberal Arts Community. All Rights Reserved. จ֓؍ 4.
SYSTEM DESIGN XGBoostʹ͓͚ΔγεςϜଆͷʹ͍ͭͯड़ΒΕ͍ͯΔɽ • ιʔτͷܭࢉίετͷݮʢColumn Block for Parallel Learningʣ • CSCʹΑΔεύʔεߦྻσʔλѹॖ • σʔλͷϒϩοΫԽ • ܭࢉྔͷൺֱ • ޯใͷϓϦϑΣονʢCache-aware Accessʣ • ϒϩοΫαΠζͷ࠷దԽ • σΟεΫIOͷεϧʔϓοτ্ʢϒϩοΫѹॖɾ ϒϩοΫஅยԽʣ 10
Copyright @ Liberal Arts Community. All Rights Reserved. จৄࡉ 11
Copyright @ Liberal Arts Community. All Rights Reserved. จৄࡉɹͦͷલʹ…ܾఆͷ෮श ▪
ܾఆʢCARTʣͱ ͋Δಛ࣠ͱᮢͷେখؔͷஅͷΈ߹ΘͤͰྨճؼΛղ ͘ΞϧΰϦζϜͷ͜ͱɽ ܾఆͷ͏ͪɼԼਤͷΑ͏ʹඞͣೋ͞ΕΔͷΛCARTͱ͍͏ 12
Copyright @ Liberal Arts Community. All Rights Reserved. จৄࡉɹͦͷલʹ…ܾఆͷ෮श ▪
ܾఆʢCARTʣͱ ϊʔυͱϊʔυΛ݁ͿϦϯΫ͔Βߏ͞Ε͍ͯΔɽϊʔυʹ͍ͭͯͷͲ ͷ෦ʹҐஔ͢ΔʹΑͬͯ࣍ͷΑ͏ʹ۠ผ͞Ε͍ͯΔɽ ໊લ ҙຯ ࠜϊʔυ ͷҰ൪্ʹ͋Δϊʔυ ༿ϊʔυʢϦʔϑʣ ͷҰ൪Լʹ͋Δϊʔυ ෦ϊʔυ ࠜϊʔυͱ༿ϊʔυҎ֎ͷϊʔυ ༿ϊʔυ ࠜϊʔυ ෦ϊʔυ ϦϯΫ 13
Copyright @ Liberal Arts Community. All Rights Reserved. จৄࡉɹͦͷલʹ…ܾఆͷ෮श •
14
Copyright @ Liberal Arts Community. All Rights Reserved. จৄࡉɹͦͷલʹ…ܾఆͷ෮श •
15
Copyright @ Liberal Arts Community. All Rights Reserved. จৄࡉɹͦͷલʹ…ܾఆͷ෮श •
16
Copyright @ Liberal Arts Community. All Rights Reserved. จৄࡉɹͦͷલʹ…ܾఆͷ෮श •
17
Copyright @ Liberal Arts Community. All Rights Reserved. จৄࡉɹͦͷલʹ…ܾఆͷ෮श •
18
Copyright @ Liberal Arts Community. All Rights Reserved. TREE BOOSTING
IN A NUTSHELL 19