論文紹介: The Indian Buffet Hawkes Process to Model Evolving Latent Influences

Tan, X., Rao, V., Neville, J., The Indian Buffet Hawkes Process to Model Evolving Latent Influences, The Conference on Uncertainty in Artificial Intelligence, 2018.
original paper: http://auai.org/uai2018/proceedings/papers/289.pdf

Transcript

1. ࿦จ঺հɿThe Indian Buffet Hawkes Process to Model Evolving Latent Influences ઒ౡوେ September 17, 2019 ೔໺ݚڀࣨΠϯλʔϯɼి௨େ M1
2. ࿦จ֓ཁ “The Indian Buffet Hawkes Process to Mode Evolving Latent Influences”, Xi Tan, Vianyak Rao, and Jennifer Neville • UAI2018 ͷ࿦จ • ఺աఔͷ Hawkes Process ͱϊϯύϥϕΠζͷ Indian Buffet Process Λ౷߹ͨ͠Ϟσϧ • ঢ়ଶۭؒϞσϧͷϊϯύϥϕΠζ֦ுͱ΋ଊ͑ΒΕΔ • ෳ਺࣌఺ͰಘΒΕͨυΩϡϝϯτ͔ΒɼτϐοΫ਺΍ͦͷڧ ౓ͷਪҠͳͲΛਪఆͰ͖Δʢجຊతʹ͸ࣗવݴޠσʔλΛ ૝ఆʣ 2
3. Hawkes Process Hawkes Process ʮ͋ΔΠϕϯτͷൃੜޙ͸࣍ͷΠϕϯτ͕ى͜Γ΍͍͢ʯͱ͍ ͏ઌݟ৘ใΛ൓өͨࣗ͠ݾྭىաఔͷϞσϧͷͻͱͭ e.g.) ஍਒ɼਆܦࡉ๔ͷൃՐ ఺աఔͷڧ౓ؔ਺ λ(t) Λ λ(t) = λ0 + ∫ t 0 κ(t − s)dN(s) (1) ͱදݱɽ λ0 ∈ R+ɿ ϕʔεͷڧ౓ύϥϝʔλ κ(·)ɿ Χʔωϧؔ਺ʢਖ਼ఆ஋Χʔωϧͱ͸ҟͳΔ֓೦ʣ N(s)ɿ [0, s) ظؒͰͷΠϕϯτൃੜ਺ 3
4. Hawkes Process λ(t) ॴ༩ͷ΋ͱͰΠϕϯτൃੜ࣌ࠁͷཤྺ͕ H(0,T] = (t1, . . . , tn) ͱ༩͑ΒΕͨͱ͖ͷ Hawkes Process ͷ໬౓ؔ਺͸ L(H) = exp{−Λ(0, T)} n ∏ i=1 λ(ti) (2) Λ(0, T) = ∫ T 0 λ(t)dt (3) ͱදͤΔ 4
5. Indian Buffet Process Indian Buffet Process ྻ਺͕ N < ∞ɼߦ਺ K ͕જࡏతʹແݶେͱͳΓ͏ΔόΠφϦ ߦྻ C ∈ {0, 1}N×K ͷੜ੒Ϟσϧ ൃ૝ͱͯ͠͸όΠφϦߦྻ C ͷཁૉ cik ʹؔ͢Δ p(cik|πk) = Bernoulli(cik|πk) p(πk|α) = Beta(πk|α, 1) ͱ͍͏Ϟσϧʹ͍ͭͯ πk ΛपลԽআڈ͠ɼp(cik|α) ʹ͍ͭͯ K → ∞ ͷۃݶΛධՁ͢Δ͜ͱͰಋग़ 5
6. Indian Buffet Process ͷ֓೦ਤ 1. N×0行列からスタート 3. 2列目の既存要素を 確率pで1にする 4. 2列目のK'個を新たにactivate 5. i列目について同様に 既存要素を確率pで1に 2. 1列目のK'個をactivate 6. i列目のK'個を新たにactivate 6
7. Indian Buffet Hawkes Process Indian Buffet Hawkes Process ͸ latent ͳঢ়ଶ zn = {Kn, V n} ͱ Πϕϯτ yn = {tn, Tn} ͔ΒͳΔ ˠঢ়ଶۭؒϞσϧͷҰछͱΈͳ͢͜ͱ͕Մೳ tn ɿΠϕϯτൃੜ࣌ࠁ Tn ɿൃੜͨ͠ςΩετ જࡏม਺ zn ʹ͍ͭͯ͸࣍ͷεϥΠυͰ 1 1ຊ࿦จͰ͸Ұ௨Γ NLP ͷจ຺ͰϞσϧͷઆ໌͕ͳ͞Ε͍ͯΔ 7
8. Indian Buffet Hawkes Process K ݸͷ “Ҽࢠ” Λ௨ͯ͠աڈͷσʔλ͕࣍ͷΠϕϯτͷൃੜ࣌ࠁ ͱରԠ͢ΔςΩετΛੜ੒͢Δͱߟ͑Δ NLP ͷจ຺Ͱ͸Ҽࢠ͸τϐοΫʹ૬౰ 2 SɿϘΩϟϒϥϦू߹ DɿςΩετͷ௕͞ʢ؆୯ͷͨΊ͢΂ͯҰఆͱ͢Δʣ zn = {Kn, V n}ɿIBHP ͷજࡏม਺ 1 × K ߦྻ Kn ͸τϐοΫ k = 1, . . . , K ʹ͓͚ΔΠϕϯτͷൃੜ ͠΍͢͞ʢHP ʹ͓͚Δڧ౓ؔ਺ʣΛද͢ |S| × K ߦྻ V n ͸τϐοΫ k ͔Βͷ͋Δ୯ޠ s ∈ S ͷݱΕ΍͢ ͞Λද͢ 2ͻͱͭͷ୯ޠ͕ෳ਺ͷτϐοΫʹଐ͢Δ͜ͱ͕Մೳ 8
9. Indian Buffet Hawkes Process ·͢ୈ k ൪໨ͷҼࢠͰͷࣙॻॏΈΛ vk ∈ [0, 1]|S| ͱ͠ɼ͜Ε͕࣍ ʹੜ੒͞ΕΔςΩετΛܾఆ ͜͜Ͱ vk ͷཁૉͷ࿨͸ 1 ͱ͢Δ vk ͷࣄલ෼෍͸ύϥϝʔλ v0 ͷσΟϦΫϨ෼෍ vk ∼ Dir(vk|v0) (4) 9
10. Indian Buffet Hawkes Process Πϕϯτ࣌ࠁʹ͍ͭͯ͸ɼL ݸͷࢦ਺ܕͷجఈΧʔωϧ γl(δ) = βl exp ( − δ τl ) (5) ͷ wk ∈ [0, 1]L ʹΑΔॏΈ෇͚࿨ͱͯ͠ಘΒΕΔҼࢠΧʔωϧ κik(δ|wk, cik) =        L ∑ l=1 wklγl(δ), if cik = 1 0. if cik = 0 (6) ͱϕʔεڧ౓ λ0 ʹΑͬͯ Hawkes Process ͷڧ౓ؔ਺ λk(t) Λ ఆٛɽ {(βl, τl)} ͸ύϥϝʔλ wk ͷࣄલ෼෍͸ wk ∼ Dir(wk|w0) 10
11. Indian Buffet Hawkes Process ʢॳظԽʣ ΑͬͯॳΊͷσʔλ y1 = {t1, T1} ͷੜ੒աఔ͸ 1. K ∼ Poisson(K|λ0) ʹΑΓҼࢠ਺Λܾఆ 2. c ∈ {0, 1}K, w1:K ͷॳظԽɽc1k = 1 (k = 1, . . . , K) ͱ͠ɼ wk ∼ Dir(wk|w0) Λαϯϓϧ 3. ͋ͱ͸ vk ∼ Dir(vk|v0) Λαϯϓϧ͢Δͱજࡏม਺ z1 = {K1, V 1} ͕ಘΒΕΔɽجఈΧʔωϧ͸ γl(0) = βl ͳͷ Ͱ κ1 = w⊤ k β ʹΑͬͯ K1 = (κ1 · · · κK) ∈ R1×K ͱ͢Δ 4. t1 Λڧ౓ؔ਺ λ0 ͷϙΞιϯաఔ 3 ͔ΒɼςΩετΛ T1 ∼ Multinomial(T1|D, ∑ k vk/K) ͔Βੜ੒ ͱϞσϦϯά͞ΕΔ 3p(t1 ) = λ0 exp(λ0 t1 ) 11
12. Indian Buffet Hawkes Process ʢn ≥ 2 ʹର͢Δੜ੒Ϟσϧʣ k ൪໨ͷҼࢠͷ Hawkes Process ͷڧ౓ λk(tn−1) ΛɼͦΕ·Ͱͷ શΠϕϯτʹΑΔҼࢠΧʔωϧͷ࿨Ͱఆٛɿ λk(tn−1) = n−1 ∑ i=1 κik(tn−1 − ti) ∥κi∥0 . (7) ·ͨόΠφϦߦྻ C ͷطʹ activate ͞Ε͍ͯΔྻ಺ͷ֤ཁૉΛ pk = λk(tn−1) λ0/K + λk(tn − 1) (8) ͷ֬཰Ͱ 1 ʹ͢Δɽͦͷޙ৽ͨʹ K+ ∼ Poisson ( λ0 λ0 + ∑ K k=1 λk(tn − 1) ) (9) ݸͷҼࢠͷ 1 Λ C ʹ௥Ճ 12
13. Indian Buffet Hawkes Process ʢn ≥ 2 ʹର͢Δੜ੒Ϟσϧʣ ࣍ͷΠϕϯτൃੜ࣌ࠁ tn ͸ڧ౓ؔ਺ λ(tn) = ∑ κnk̸=0 λk(tn) (10) ͷϙΞιϯաఔ͔Βੜ੒ ·ͨੜ੒͢ΔςΩετ Tn ͸ Tn ∼ Multinomial  D, ∑ κnk̸=0 vk ∥κn∥0   (11) ͔Βαϯϓϧ 13
14. Indian Buffet Hawkes Process ʢશମͷεΩʔϜʣ 14
15. ࣮ݧ • ਓ޻ςΩετσʔλɼͳΒͼʹ͍͔ͭ͘ͷ࣮ςΩετσʔλ ͰධՁ • SMC Ͱֶश • HP, DHP (Dirichlet Hawkes Process), HDHP (Hierarchical Diriclet Hawkes Process) ͱൺֱ DHP ͸ HP ͷڧ౓ λ(t) ʹΑͬͯΫϥελͷׂΓ౰ͯʢCRP ͷύ ϥϝʔλʣ͕ܾఆ͞ΕΔϞσϧ HDHP ͸ DHP Λ֊૚σΟϦΫϨաఔʹͨ͠Ϟσϧ 15
16. ਓ޻σʔλ NIPS Dataset ͷ্Ґ 1000 ୯ޠ͔Βࣙॻ S Λߏ੒͠ɼIBHP ʹैͬ ͯ௕͞ D = 20 ͷςΩετσʔλΛ N = 1000 ηοτੜ੒ ਅͷҼࢠ਺ K ͸ K = 5, 10, 20 ʹ͍࣮ͭͯݧɽ IBHP ͸Ϟσϧ͕ਖ਼ ͍ͨ͠Ί࠷ྑ 16
17. ࣮σʔλ - ༧ଌର਺໬౓ Facebook Dataset, NIPS Dataset, Santa Barbara Corpus Dataset, Enron Email Dataset ͷ 4 ͭͰ༧ଌର਺໬౓ΛධՁ 17
18. ࣮σʔλ - τϐοΫͷਪҠʢFBʣ Facebook Dataset ͔ΒಘΒΕͨ͏ͪɼ2 ͭͷτϐοΫʹ͍ͭͯڧ ౓ؔ਺ͱϫʔυΫϥ΢υΛՄࢹԽ • τϐοΫ 1 ͸ “class”, “work” ͳͲΛؚΈɼฏ೔ʹڧ౓ؔ਺͕ େ͖͘ͳΔ • τϐοΫ 2 ͸ “happy”, “weekend” ͳͲΛؚΈɼि຤ʹڧ౓ؔ ਺͕େ͖͘ͳΔ 18
19. ࣮σʔλ - τϐοΫͷਪҠʢNIPSʣ NIPS Dataset ͷτϐοΫ͝ͱͷڧ౓ͱͦ୅දతͳ୯ޠྫΛՄࢹԽ Χʔωϧ๏ɼਂ૚ֶशɼϕΠζʹؔ͢ΔτϐοΫ͕ग़ݱ NeurIPS ಺ͷࢧ഑తͳ෼໺ͷਪҠΛ͏·͔ͭ͘Ί͍ͯΔʁ 19
20. ࣮σʔλ - τϐοΫͷ༧ଌʢNIPSʣ কདྷͷ࣌఺ 1, 2, 3 ʹ͓͍ͯ NeurIPS ͷ࿦จதͰසग़ʹͳΓͦ͏ͳ τϐοΫɾ୯ޠΛ༧ଌ ࠓޙ͸ϕΠζͷ࿦จ͕ԼՐʹͳΔʢʁʣ 20