Share
Embed
minne meets Hivemall/pepabo_minne_matrix_factorization_in_hivemall
第三回 Hivemall Meetup
https://eventdots.jp/event/597518
monochromegane
September 08, 2016
Transcript
64 slides
-
͓͢͢Ί࡞Ոͷબఆ wNJOOFʮ͋ͳͨʹ͓͢͢Ίͷ࡞Ոʯ wϢʔβʔͷߦಈΛجʹ࡞ՈΛϨʔςΟϯά Activity Filter and shuffle Users fav, follow
etc… Matrix Factorization Recommendation import DB -
ڠௐϑΟϧλϦϯά ᅂͷऩू ϢʔβʔͱΞΠςϜʹΑΔධՁߦྻΛ࡞͢Δ ࣅ͍ͯΔϢʔβʔΛ୳ͩ͢͠ ྫ͑ɺϢʔβʔؒͷධՁΛݩʹྨࣅੑείΞΛࢉग़͢Δ
ϢʔΫϦουڑɺϐΞιϯ૬ؔɺ+BDDBSEͳͲ ΞΠςϜΛਪન͢Δ ࣅ͍ͯΔϢʔβʔ͕ධՁ͍ͯͯ͠ɺ͕ࣗධՁ͍ͯ͠ͳ͍ΞΠςϜ ྫ͑ɺϢʔβʔྨࣅੑείΞʹج͍ͯධՁ༧ଌΛࢉग़͢Δ -
.BUSJY'BDUPSJ[BUJPO wϢʔβʔΞΠςϜ͝ͱͷධՁͷภΓ͕͋ͬͯɺେྔ͔ͭૄͳσʔλʹର͢Δ ධՁ༧ଌ wLݸͷજࡏҼࢠΛ࣋ͭϢʔβʔߦྻͱΞΠςϜߦྻΛ༻͍ͯۙࣅʢ࣍ݩݮʣ wϢʔβʔΞΠςϜʹର͢ΔόΠΞεͷߟྀ
Item User R = ≈ m P n n Q × m k k -
͓͞Β͍ߦྻͷੵ 0 B B @ 1 3 1 4 2
1 1 3 1 C C A ✓ 4 1 2 2 4 2 ◆ = 0 B B @ 10 13 8 12 17 10 10 6 6 10 13 8 1 C C A -
͓͞Β͍ߦྻͷੵ w ߦͱྻͷੵ͕ߦྻ 0 B B @ 1 3 1
4 2 1 1 3 1 C C A ✓ 4 1 2 2 4 2 ◆ = 0 B B @ 10 13 8 12 17 10 10 6 6 10 13 8 1 C C A (1 ⇤ 4) + (3 ⇤ 2) = 10 -
͓͞Β͍ߦྻͷੵ w ߦͱྻͷੵ͕ߦྻ w NºLͱLºOͷߦྻͷੵNºO w ٯ·ͨΓཱͭ 0 B B
@ 1 3 1 4 2 1 1 3 1 C C A ✓ 4 1 2 2 4 2 ◆ = 0 B B @ 10 13 8 12 17 10 10 6 6 10 13 8 1 C C A 3 4 3 4 (4×2) ͱ (2×3) Ͱ (4×3)ͷߦྻ -
.BUSJY'BDUPSJ[BUJPO ϥϯμϜͳͰॳظԽ͞ΕͨղޙͷߦྻΛͭ༻ҙ͢Δ ݩͷධՁߦྻͰ͕ೖ͍ͬͯΔશͯʹରͯ͠ɺղޙͷߦྻͷੵͷ݁Ռ ͱൺֱ͢Δ ޡࠩΛิਖ਼͢Δ ݩͷධՁߦྻͱۙࣅʹͳΔ·Ͱ
Λ܁Γฦ͢ Item User R = ≈ m P n n Q × m k k -
ͭ·Γ R = m P n n Q × m
k k LݸͷજࡏҼࢠΛ࣋ͭͭͷߦྻͷੵʹ ΑΓɺݩͷධՁߦྻͷʹۙͮ͘Α͏ ͳA AͷΈ߹ΘͤΛ୳͢͜ͱ 0 B B @ ? ? ? ? ? ? ? ? 1 C C A ✓ ? ? ? ? ? ? ◆ = 0 B B @ 10 13 8 12 17 10 10 6 6 10 13 8 1 C C A ٻΊΒΕ֤ͨߦྻɺݩͷಛΛ·ͱΊͨߦྻͱΈͳ ͢͜ͱ͕Ͱ͖Δ w 6TFSͷಛΛL࣍ݩʹ·ͱΊͨߦྻ w *UFNͷಛΛL࣍ݩʹ·ͱΊͨߦྻ -
ͭ·Γ R = m P n n Q × m
k k LݸͷજࡏҼࢠΛ࣋ͭͭͷߦྻͷੵʹ ΑΓɺݩͷධՁߦྻͷʹۙͮ͘Α͏ ͳA AͷΈ߹ΘͤΛ୳͢͜ͱ 0 B B @ ? ? ? ? ? ? ? ? 1 C C A ✓ ? ? ? ? ? ? ◆ = 0 B B @ 10 13 8 12 17 10 10 6 6 10 13 8 1 C C A ˠ ϥϯμϜͳA AͷΈ߹Θ͔ͤΒٻΊ ͨͱɺਖ਼ղͷͷޡࠩΛখͯ͘͞͠ ͍͚Α͍ ˠޯ߱Լ๏ -
.BUSJY'BDUPSJ[BUJPO w༧ଌ ˆ rij = pT i qj = k
X k=1 pikqkj e2 ij = (rij ˆ rij)2 = (rij k X k=1 pikqkj)2 p0 ik = pik + ↵ @ @pik e2 ij = pik + 2↵eijqkj q0 kj = qkj + ↵ @ @qkj e2 ij = qkj + 2↵eijpik wೋޡࠩͷతؔ wύϥϝλߋ৽ࣜ -
.BUSJY'BDUPSJ[BUJPO3FHVMBSJ[BUJPO w༧ଌ ˆ rij = pT i qj = k
X k=1 pikqkj wೋޡࠩͷతؔ wύϥϝλߋ৽ࣜ e2 ij = (rij ˆ rij)2 + 2 ( P 2 + Q 2 ) p0 ik = pik + ↵ @ @pik e2 ij = pik + ↵(2eijqkj pik) q0 kj = qkj + ↵ @ @qkj e2 ij = qkj + ↵(2eijpik qkj) -
.BUSJY'BDUPSJ[BUJPO#JBT w༧ଌ wೋޡࠩͷతؔ wύϥϝλߋ৽ࣜ ˆ rij = µ + buser(i)
+ bitem(j) + k X k=1 pikqkj e2 ij = (rij ˆ rij)2 + 2 ( buser 2 + bitem 2 + P 2 + Q 2 ) p0 ik = pik + ↵ @ @pik e2 ij = pik + ↵(2eijqkj pik) q0 kj = qkj + ↵ @ @qkj e2 ij = qkj + ↵(2eijpik qkj) buser(i) = buser(i) + ↵ @ @buser(i) e2 ij = buser(i) + ↵(2eij buser(i)) bitem(j) = bitem(j) + ↵ @ @bitem(j) e2 ij = bitem(j) + ↵(2eij bitem(j)) -
.BUSJY'BDUPSJ[BUJPO rows, cols := R.Dims() p := mat64.NewDense(rows, K, randomSlice(rows*K))
q := mat64.NewDense(K, cols, randomSlice(rows*K)) for step := 0; step < 5000; step++ { for i := 0; i < rows; i++ { for j := 0; j < cols; j++ { rij := R.At(i, j) if rij == 0.0 { continue } pi := p.RowView(i) qj := q.ColView(j) err := rij - mat64.Dot(pi, qj) for k := 0; k < K; k++ { pik := p.At(i, k) qkj := q.At(k, j) p.Set(i, k, pik+alpha*(2*err*qkj)) q.Set(k, j, qkj+alpha*(2*err*pik)) } } } } return p, q # R ⽷5 0 3⽹ ⾇0 2 2⾇ ⽸4 4 0⽺ # R^ ⽷ 4.867 4.489 3.194⽹ ⾇ 2.482 2.266 1.629⾇ ⽸ 4.155 3.834 2.726⽺ -
Ϩίϝϯυ wNJOOFʮ͋ͳͨʹ͓͢͢Ίͷ࡞Ոʯ wϢʔβʔͷߦಈΛجʹ࡞ՈΛϨʔςΟϯά Activity Filter and shuffle Users fav, follow
etc… Matrix Factorization Recommendation import DB -
#JHGPPU IDFA/GAID UID rack-bigfoot Service Request Activity log Services DB
Attribute Big Cube Cube https://icons8.com BI Recommendation Bandit algorithm Re-marketing Feedback Name identification Cookie Sync -
Ϩίϝϯυ wNJOOFʮ͋ͳͨʹ͓͢͢Ίͷ࡞Ոʯ wϢʔβʔͷߦಈΛجʹ࡞ՈΛϨʔςΟϯά Activity Filter and shuffle Users fav, follow
etc… Matrix Factorization Recommendation import DB -
.BUSJY'BDUPSJ[BUJPOCZ)JWFNBMM Activity Rating Ratings 20% 80% testings trainings sgd_model predictions
evaluations MAE RMSE Bias P, Q Service fav, follow etc… Recommendation Matrix Factorization -
SBUJOH wධՁߦྻΛೖ͢Δఔ wNJOOFͰϢʔβʔʹ໌ࣔతͳධՁΛೖྗͯ͠Β͏ͷͰͳ͘ɺલड़ͷߦ ಈϩάΛݩʹAߪೖऀAͷA࡞ՈAʹର͢ΔධՁΛಋग़͍ͯ͠Δ Activity Filter and shuffle Users fav,
follow etc… Matrix Factorization Recommendation import DB -
SBUJOH INSERT OVERWRITE TABLE ratings SELECT r.account_id, r.creator_id, ( --
rating ) AS rating, rand(31) as rnd FROM ( ... 3FTVMU&YQPSUΑΓ*/4&35 07&383*5& 5"#-& ͷ΄͏͕ߴ ޙఔͰτϨʔχϯάͱςετσʔλʹׂΓৼΔͨΊ ϥϯμϜΛׂΓৼ͓ͬͯ͘ͱศར -
USBJOJOH UFTUJOH INSERT OVERWRITE TABLE training SELECT account_id, creator_id, rating,
rnd FROM ratings WHERE rnd > 0.2 -- rnd <= 0.2 ; ΛτϨʔχϯά༻ USBJOJOH ʹɻ ಉ͡Α͏ʹͯ͠Λςετ༻ UFTUJOH ʹׂΓৼΔɻ -
.BUSJY'BDUPSJ[BUJPO w.BUSJY'BDUPSJ[BUJPOΛͬͯɺධՁߦྻ͔ΒLݸͷજࡏҼࢠΛ࣋ͭϢʔβʔ ߦྻͱΞΠςϜߦྻʢNJOOFͷ߹ɺ࡞ՈߦྻʣΛಋ͘ఔ w)JWFNBMMͰόΠΞε߲͖ͭͷߦྻΛಋग़͢Δ
Item User R = ≈ m P n n Q × m k k -
.BUSJY'BDUPSJ[BUJPO INSERT OVERWRITE TABLE sgd_model SELECT idx, array_avg(u_rank) as Pu,
array_avg(i_rank) as Qi, avg(u_bias) as Bu, avg(i_bias) as Bi, min(mu) as mu FROM ( SELECT train_mf_sgd(account_id, creator_id, rating, '- factor 20 -iter 50 -update_mu') AS (idx, u_rank, i_rank, u_bias, i_bias, mu) FROM training ) t GROUP BY idx; -
.BUSJY'BDUPSJ[BUJPO INSERT OVERWRITE TABLE sgd_model SELECT idx, array_avg(u_rank) as Pu,
array_avg(i_rank) as Qi, avg(u_bias) as Bu, avg(i_bias) as Bi, min(mu) as mu FROM ( SELECT train_mf_sgd(account_id, creator_id, rating, '- factor 20 -iter 50 -update_mu') AS (idx, u_rank, i_rank, u_bias, i_bias, mu) FROM training ) t GROUP BY idx; ޙఔͰධՁͷฏۉΛ͏͕ɺ5SFBTVSF%BUB্ Ͱม͕͑ͳ͍ͷͰʢશͯಉ͡ʹͳΔ͕ʣ VQEBUFNVΦϓγϣϯΛͬͯɺ͜͜ͰٻΊ͓ͯ͘ ཧߏͱͯ͠ߦྻ͕ͩJEYڞ௨Խ͞Ε͓ͯΓ ͻͱͭͷςʔϒϧʹ֨ೲ͞ΕΔ 1V 2JͦΕͧΕϢʔβʔɺΞΠςϜʢ࡞Ոʣ͋ͨ ΓͷLݸͷજࡏҼࢠΛྻͱͯ࣋ͭ͠ USBJO@NG@THEʹΑΓ֬తޯ߱Լ๏ʹΑΔ .BUSJY'BDUPSJ[BUJPO͕ߦΘΕΔɻ ଞʹ"EB(SBE࠷దԽΛ͏USBJO@NG@BEBHSBE ͏͜ͱ͕Ͱ͖Δ -
FWBMVBUJPO SELECT mae(t3.predicted, t3.actual) as mae, rmse(t3.predicted, t3.actual) as rmse
FROM ( SELECT t2.actual, mf_predict(if(size(t2.Pu)=0, null, t2.Pu), if(size(p2.Qi)=0, null, Qi), t2.Bu, p2.Bi, t2.mu) as predicted FROM ( SELECT t1.account_id, t1.creator_id, t1.rating as actual, p1.Pu, p1.Bu, p1.mu FROM testing t1 LEFT OUTER JOIN sgd_model p1 ON (t1.account_id = p1.idx) WHERE p1.Bu IS NOT NULL AND p1.mu IS NOT NULL ) t2 LEFT OUTER JOIN sgd_model p2 ON (t2.creator_id = p2.idx) ) t3; ֶशࡁΈϞσϧΛͬͯςετσʔλʹର͢Δ ."& 3.4&ΛٻΊΔʢখ͍͞΄ͲΑ͍ʣ -
QSFEJDUJPO INSERT OVERWRITE TABLE prediction SELECT t2.account_id, t2.creator_id, t2.rank, t2.predicted
FROM ( SELECT ROW_NUMBER() OVER(PARTITION BY t1.account_id ORDER BY t1.predicted DESC) AS rank, t1.account_id, t1.creator_id, t1.predicted FROM ( SELECT accounts_creators.account_id, accounts_creators.creator_id, mf_predict(if(size(accounts.Pu)=0, null, accounts.Pu), if(size(creators.Qi)=0, null, creators.Qi), accounts.Bu, creators.Bi, accounts.mu) AS predicted FROM -- prediction for all accounts and creators ) t1 ) t2 WHERE t2.rank <= 100; ֤ΞΧϯτͷ༧ଌϨʔςΟϯά͕ߴ͍ͷΛ݅ NG@QSFEJDUʹΑΓֶशࡁΈͷϞσϧ͔Β֤ΞΠςϜʹର͢Δ ༧ଌϨʔςΟϯάΛٻΊΔ ΞΧϯτ͝ͱͷ༧ଌϨʔςΟϯάͰฒͼସ͑ -
1FOEVMVN schedule 'test-scheduled-job' do database 'db_name' query 'select time from
access;' retry_limit 0 priority :normal cron '30 0 * * *' timezone 'Asia/Tokyo' delay 0 result_url 'td://@/db_name/table_name' end Schedfile Apply $ pendulum --apikey='...' -a --dry-run $ pendulum --apikey='...' -a