Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

論文紹介:Gradient Boosted Feature Selection

S.Shota
October 06, 2017

論文紹介:Gradient Boosted Feature Selection

2017/10/6の論文紹介で使用したスライドです.

論文:
Z. Xu, G. Huang, K. Q. Weinberger, and A. X. Zheng, “Gradient boosted feature selection,” in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD ’14, pp. 522-531, 2014.

S.Shota

October 06, 2017
Tweet

More Decks by S.Shota

Other Decks in Technology

Transcript

  1. (SBEJFOU#PPTUFE 'FBUVSF4FMFDUJPO ;9V ()VBOH ,28FJOCFSHFS BOE"9;IFOH JO,%%ˏ QQˊ  橆㞮䞔㜠㷕䏍

    䞔㜠ًر؍،橆㞮㷕㼔余 䞔㜠ًر؍،㷕؝٦أ 涯䊛灇瑔㹓 ⥜㡦䎃 俕谏 绤實 䎃剢傈
  2. 俑柃䞔㜠٥鼅׿׌椚歋 • ;9V ()VBOH ,28FJOCFSHFS BOE"9;IFOH  ˑ(SBEJFOUCPPTUFEGFBUVSFTFMFDUJPO ˒JO1SPDFFEJOHTPG UIFUI"$.4*(,%%JOUFSOBUJPOBMDPOGFSFODFPO

    ,OPXMFEHFEJTDPWFSZBOEEBUBNJOJOH ,%%ˏ QQ  • -"440הכ殯ז׷暴䗙鼅䫛ך،فٗ٦ثד 莆㄂׾䭯׏׋׋׭ • (SBEJFOU#PPTUJOH׉ך׮ךח׮ 莆㄂ָ֮׏׋׋׭ • LBHHMFדכ㢳欽ׁ׸גְ׷䩛岀׵׃ְ 
  3. *OUSPEVDUJPO  (SBEJFOU#PPTUFE'FBUVSF4FMFDUJPO (#'4 简䕎陎ⴽ㐻 - ظٕي (SBEJFOU #PPTUFE5SFF (#'4׮-"440הずֻׄ

    &NCFEEFE"QQSPBDIךא ꬊ简䕎㼎䘔 㷕统ה暴䗙鼅䫛׾ ず儗ח遤ֲ単穈׫
  4. 3FMBUFE8PSL  • -"440 • 湡涸ꟼ侧ח- ظٕيח״׷姻⵱⻉׾鷄⸇ • )4*$-"440 •

    אך暴䗙׀הח醱侧ךؕ٦طٕ׾黝欽ׅ׷ -"440 • N3.3 • 暴䗙꟦ך湱✼䞔㜠ꆀח㛇בֻ暴䗙鼅䫛
  5. -"440חֶֽ׷剑黝⻉㉏겗 䖤׵׸׷⸬卓 • 䖤׵׸׷鍑ָأػ٦أ暴䗙鼅䫛 • 害⻉䚍腉ךぢ♳姻⵱⻉ 'FBUVSF4FMFDUJPOXJUIUIF- OPSN  log(1

    + exp( yi wT xi )) 䴦㣟ꟼ侧 min w X (xi,yi ) l (xi, yi, w) + k w k 1 - ظٕي |w1| + · · · + |wd| אך⸬卓ךغٓٝأכוֲ锃侭ׅ׷
  6. 5IFDBQQFE- OPSN  ˟ ך㹀纏ָ꟦麩׏גְ׷孡ָׅ׷㢳ⴓ如䒭דכ ♧䏝鼅䫛ׁ׸׋暴䗙חכ׉׸⟃♳ لشٕذ؍׾Ⰵ׸זְ״ֲח何葺 q✏ (w) =

    min(|w1|, ✏) + · · · + min(|wd|, ✏) q✏ (w) ぐ銲稆ך窫㼎⦼חٍؗحفׅ׷ꟼ侧 ׾欽ְג q✏ (w) = [q✏ (w1 ), . . . , q✏ (wd )] q✏ (wi ) = min(|wi|, ✏) min w X (xi,yi ) l (xi, yi, w) + k w k 1 + µq✏(w) - ظٕيך 葺ְ鵚⡂
  7. (SBEJFOU#PPTUFE'FBUVSF4FMFDUJPO  $BQQFE- ظٕيך《׶䪔ְ • ֮׷暴䗙G׾欽ְ׷加ח㼎ׅ׷ꅾ׫ךさ鎘⦼ • さ鎘⦼׾暴䗙׀החꟼ侧 חⰅ׸ג筨ㄤ T

    X t=1 |Fft t| Fft 2 {0, 1} ׋׌׃ q✏ min l( ) + k k1 + µ d X f=1 q✏ 0 B B B B B @ T X t=1 |Fft t| 1 C C C C C A (#'4ך剑黝⻉㉏겗ꬊ⳻ַא䗍ⴓ♶〳
  8. 0QUJNJ[BUJPO  (SBEJFOU#PPTUJOHד剑黝⻉㉏겗׾鍑ֻ • 嫣أذحف׀הח剑䚈倯ぢח鹌׬如⯋׾刿倜 • 㔐䌓加 ח㼎׃ ך㶷㖈׾鏩ׇל 頾ך⺟ꂁ倯ぢח⹛ַֿׅהד䌢ח

    כ㟓⸇ • 剑䚈倯ぢח鹌׬如⯋ ך䱱稊 h 2 H h 2 H t⇤ t⇤ = arg min t rL( )t 䏟垥鯥ח屟׏ג剑䚈倯ぢח刿倜 䏟垥꣬♴岀$%岀
  9. 0QUJNJ[BUJPO  أذحف湡חֶֽ׷湡涸ꟼ侧ך⺟ꂁ T0 + 1 rL( )t = @l

    @ t + µ d X f=1 rq✏ 0 B B B B B B @ T0 X t=1 Fft t 1 C C C C C C A T0 X t=1 Fft t < ✏ T0 X t=1 Fft t ✏ Fft 0 if if f Fft ה邌鎸 f 暴䗙Gָ鼅䫛 PUIFSXJTF 
  10. 加ך䱱稊㉏겗פך㢌䳔 • 加כⰋג姻鋉⻉ׁ׸גְ׷ה⟎㹀 • ⺟ꂁח如ךאך㹀侧갪׾⸇ִג侭椚 • 侭椚׃׋穠卓ָ加ך䱱稊㉏겗ך㹀䒭⻉ ✳⛦铎䊴  姻⵱⻉갪

    0QUJNJ[BUJPO  X i h2 t (xi ) = 1 1 2 X i h2 t (xi ) , 1 2 X i g2 i ht = arg min ht 2H 1 2 n X i= 1 ( gi hi (xi))2 + µ d X f= 1 f Fft
  11. 3FTVMU  • 㹋꿀갪湡 • ➂䊨涸זさ䧭ر٦ة • 㹋⚅歲ךر٦ةإحز • 圓鸡⻉ׁ׸׋ر٦ة穠艌ָ׿ر٦ةإحز

    • كٝثو٦ؙر٦ةإحز • 㹋꿀橆㞮 • *OUFMJ ()[ (#3". -JOVY04CJU
  12. 4USVDUVSFE'FBUVSF4FMFDUJPO  㹋꿀穠卓 • (#'4כָ׿חꟼ鸬ׅ׷ؚٕ٦ف׾ אח穾׶鴥׬ֿהָדֹגְ׷ • ずׄؒٓ٦桦׾䭯א3BOEPN'PSFTUכ圫ղז CBHַ׵暴䗙׾鼅䫛׃גֶ׶鍑ꅸָ㔭ꨇ •

    (#'4ך㜥さ鼅䫛׃׋暴䗙חכ⡭ⴓז لشٕذ؍׾♷ִזְ׋׭✮庠礵䏝ָ넝ְ • -"440ך㜥さ鼅䫛ׅץֹCBHך暴䗙ח׮ 姻⵱⻉ח״׷⡭ⴓזلشٕذ؍ָⰅ׏גְ׷
  13. #FODINBSL%BUBTFUT  儗꟦鎘皾ꆀך嫰鯰 • N3.3װ)4*$-"440 PS • 3BOEPN'PSFTU • (#'4

    • (#'4כأذحف侧װ加ך帾ׁכⵖꣲ〳腉 • (#'4כ,%%$VQך״ֲז䊬㣐זر٦ة إحزד넝鸞ח⹛⡲ O(d2) O(n2) O ( p dn log n ) O(dn)
  14. %JTDVTTJPO  תה׭ה'VUVSF8PSL • (SBEJFOU#PPTUJOH׾欽ְג暴䗙䬄⳿הٌرٕ ך㷕统׾ず儗ח遤ֲ،ٕ؞ٔؤي׾䲿周 • 儗꟦鎘皾ꆀכ暴䗙侧הر٦ةꆀך琎 $"35ך儗꟦鎘皾ꆀ •

    暴䗙侧חꟼׅ׷儗꟦鎘皾ꆀ׾何㊣ׅ׷䗳銲䚍 • 3BOEPN'PSFTUה殯ז׶暴䗙ך鼅䫛然桦ח غ؎،أַַָ׷ךד♶銲ז暴䗙׾䱖ꤐ〳腉