Share
Embed
差分プライバシーによる 安全な連合学習の実現
LINEヤフー株式会社 Privacy & Trust Team / Federated Learning Team
LINEヤフー研究所 上席研究員
髙橋翼 Tsubasa TAKAHASHI
2024/1/24 名古屋大学での講義資料です
LINEヤフーTech (LY Corporation Tech) PRO
January 24, 2024
Transcript
59 slides
-
-:$PSQPSBUJPO ࠩϓϥΠόγʔʹΑΔ ҆શͳ࿈߹ֶशͷ࣮ݱ ∁ڮ ཌྷ c5TVCBTB 5",")"4)* -*/&Ϡϑʔגࣜձࣾ 1SJWBDZ5SVTU5FBN'FEFSBUFE-FBSOJOH5FBN -*/&Ϡϑʔݚڀॴ
্੮ݚڀһ UTVCBTBUBLBIBTIJ!MZDPSQDPKQ -
-:$PSQPSBUJPO ࿈߹ֶशͱϓϥΠόγʔ σʔλ׆༻࣌ͷϓϥΠόγʔϦεΫ %JGGFSFOUJBM1SJWBDZʢࠩϓϥΠόγʔʣ ࠩϓϥΠόγʔʹΑΔ҆શͳ࿈߹ֶश ۙͷൃలతͳʢ͕࣌ؒ͋Εʣ
-*/&ϠϑʔͰͷݚڀ։ൃͷࣄྫʹ͍ͭͯॴʑͰ؆ܿʹհ͠·͢ ຊߨٛͷίϯςϯπ -
-:$PSQPSBUJPO ࿈߹ֶश 'FEFSBUFE-FBSOJOH • ΫϥΠΞϯτ܈ͱαʔόʔͱ͕ڠௐ͢Δࢄܕͷػցֶशʢ$SPTTEFWJDF'-Λఆʣ • ΫϥΠΞϯτͷσʔλ͔Βܭࢉͨ͠ߋ৽ใ ޯ ͚ͩΛαʔόʔʹఏڙ
• σʔλΫϥΠΞϯτʹཹ·Δ à ϓϥΠόγʔʹྀ /POQBSUJDJQBOUTPG'- (MPCBM.PEFM -
-:$PSQPSBUJPO 1SJWBDZ&OIBODJOH5FDIOPMPHJFT 1&5T 1&5Tʹɺ ҉߸ʹΑΔΞΫηε੍ޚٕज़ͱ ใͦͷͷΛՃ͢Δٕज़ ͕͋Δ ࿈߹ֶशɾ
ہॴࠩϓϥΠόγʔ ಗ໊Խɾσʔλ߹ ࠩϓϥΠόγʔ ͍ઢΑΓࠨଆ͕ϓϥΠόγʔอޢͷൣғ FH ಗ໊ԽࠩϓϥΠόγʔ FH ࿈߹ֶशൿີܭࢉ -
-:$PSQPSBUJPO /FUGMJY1SJ[Fͷ࠶ࣝผͷ • ಗ໊Խ͞Ε͍ͯͳ͍֎෦ͷσʔλϕʔε *.%# ͱͷಥ߹ʹΑͬͯɺಗ໊Խ͞Ε͍ͯͨίϝ ϯτͱϢʔβ͕ରԠ͍ͯ͠·ͬͨ •
ίϝϯτʹ࣏৴͕Θ͔Δͷ͋ͬͨ • ݸͷධՁͱ िؒҎͷਫ਼ ͕͔Εͷਫ਼ͰݸਓΛಛఆͰ͖ͨ 6TFSOBNF 5JUMF 3BUF "MJDF """ ### %%% #PC """ 999 1TFVEP*% 5JUMF 3BUF $PNNFOU Y """ ### %%% *UTFFNTYYY B ::: 999 "OPOZNJ[FE/FUGMJY.PWJFSBOLJOHEBUB /POBOPOZNPVT*.%#NPWJFSBUJOH +0*/ -
-:$PSQPSBUJPO ࠩ߈ܸ ͋Δσʔλϕʔε͔Βੜͨ͠ෳͷ౷ܭใͷʮࠩʯ͔Βಛఆͷݸਓʹؔ͢ΔใΛਪఆ BWHTBMBSZ ສԁ BWHTBMBSZ ສԁ ʲ2VJ[ʳ"MJDFͷऩʁ ʜ
ʜ ʜ ʜ "MJDF "MJDF͕ୀ৬ FOHJOFFST FOHJOFFST -
-:$PSQPSBUJPO 2VJ[ͷճ BWHTBMBSZ ສԁ BWHTBMBSZ ສԁ ʲ2VJ[ʳ"MJDFͷऩʁ ʜ ʜ
ʜ ʜ "MJDF "MJDF͕ୀ৬ FOHJOFFST FOHJOFFST Yr Y ສԁ ͑ -
-:$PSQPSBUJPO ࠩ߈ܸͷࣄྫ 'BDFCPPLͷ1**CBTFE5BSHFUJOH ݱࡏमਖ਼ࡁΈ • ࠂλʔήςΟϯάͷޮՌΛܭଌ͢ΔͨΊͷ౷ܭੳ"1* • ։੍ࣔޚͷͨΊʹᮢʹΑΔอޢΛಋೖ͍͕ͯͨ͠ɺόΠύε͞Εͨ •
ి൪߸ϝʔϧΞυϨεɺ8FCӾཡཤྺ͕࿙Ӯ IUUQTXXXZPVUVCFDPNXBUDI W -Q *X:WY(QL IUUQTXXXGUDHPWTZTUFNGJMFTEPDVNFOUTQVCMJD@FWFO UTQQSJWBDZDPONJTMPWF@QEG -
-:$PSQPSBUJPO ػցֶशϞσϧʹର͢ΔڴҖ ܇࿅ʹ༻ͨ͠ σʔλ ܇࿅ʹະ༻ͷ σʔλ 0.7 0.1 ⋮
0.05 0.25 0.12 ⋮ 0.45 5BSHFU.PEFM .BMJDJPVT *OTJEFSʹΑΔ ܇࿅σʔλͷ ࡡऔɾ౪ௌ ѱҙͷ͋Δୈࡾऀʹ ΑΔϞσϧߏ ܇࿅σʔλͷਪఆ 5SBJOJOH 1IBTF *OGFSFODF 1IBTF -
-:$PSQPSBUJPO ϝϯόʔγοϓਪ߈ܸ ͋Δσʔλ͕Ϟσϧͷ܇࿅ʹར༻͞Ε͔ͨ൱͔Λਪఆ͢Δ߈ܸ ܇࿅ʹ༻ͨ͠ σʔλ ܇࿅ʹະ༻ͷ σʔλ 0.7 0.1
⋮ 0.05 0.25 0.12 ⋮ 0.45 5BSHFU.PEFM "UUBDL.PEFM &TUJNBUFBT JOUSBJOEBUB &TUJNBUFBT PVUPGUSBJOEBUB "UUBDL.PEFMͷ܇࿅ʹɺ5BSHF.PEFMΛ฿ͨ͠4IBEPX.PEFMΛར༻ ϝϯόʔγοϓਪͷରࡦͱͯ͠ɺࠩϓϥΠόγʔͷอূ͕༗ޮ -
-:$PSQPSBUJPO ػցֶशٕज़ʹର͢ΔϓϥΠόγʔอޢٕज़͕ඞཁ TBNQMF𝑥 ϓϥΠόγʔอޢ͞Εͨ ػցֶशϞσϧ Ϟσϧ͔Βͷग़ྗ ܇࿅σʔλͷਪఆ͕ࠔ ܇࿅σʔλ TBNQMF𝑥
ػցֶशϞσϧ Ϟσϧ͔Βͷग़ྗ ܇࿅σʔλ ʮࠩϓϥΠόγʔʯʹΑΔ ϓϥΠόγʔอޢػցֶश -
-:$PSQPSBUJPO ࠩϓϥΠόγʔͱʁ ϊΠζʢ ʣ͕Ճࢉ͞Ε͍ͯΔͱϓϥΠόγʔอޢ͞Ε͍ͯΔΑ͏ʹײ͡·ͤΜ͔ʁ ౷ܭత ͳग़ྗ ϊΠζ ΛՃࢉ 𝜖
∞ TUSPOH XFBL ʜ ࠩϓϥΠόγʔ͕ఏڙ͢Δͷ • ϊΠζͷՃࢉʹର͢Δʢ͋ΔલఏͰͷʣ ཧతͳϓϥΠόγʔอޢͷई • ॴఆͷϓϥΠόγʔڧ 𝜖 ͷ ୡʹඞཁͳϊΠζͷಋग़ -
-:$PSQPSBUJPO ͭͷओཁͳϓϥΠόγʔϞσϧ $FOUSBM %JGGFSFOUJBM1SJWBDZ -PDBM%JGGFSFOUJBM1SJWBDZ ୈࡾऀͷ ౷ܭͷެ։ ػඍͳ ౷ܭͷऩू
αʔόʔΛ ৴༻͠ͳ͍ αʔόʔΛ ৴༻͢Δ ಋೖࣄྫ • $ISPNFϒϥβͰͷ౷ܭऩू (PPHMF • J04σόΠεͰͷ౷ܭऩू "QQMF • ελϯϓαδΣετͷ࿈߹ֶश -*/&Ϡϑʔ ಋೖࣄྫ • ถࠃࠃௐࠪ 64$FOTVT • 'VMM63-TEBUBTFU .FUB • /FYUXPSEQSFEJDUJPOJO(CPBSE (PPHMF • "VEJFODF&OHBHFNFOU"1* -JOLFE*O ·ͣͪ͜Β͔Β ʢ͝ࢀߟʣ"MJTUPGSFBMXPSMEVTFPGEJGGFSFOUJBMQSJWBDZIUUQTEFTGPOUBJOFTQSJWBDZSFBMXPSMEEJGGFSFOUJBMQSJWBDZIUNM -
-:$PSQPSBUJPO 𝝐, 𝜹 %JGGFSFOUJBM1SJWBDZ ϝΧχζϜ ℳ: 𝒟 → 𝒮
͕ 𝜖, 𝛿 %1Λຬͨ͢ͱɺྡσʔλϕʔε 𝐷, 𝐷) ∈ 𝒟 ͓Αͼҙͷग़ྗͷू߹ 𝑆 ⊆ 𝒮 ʹରͯ͠ҎԼ͕Γཱͭͱ͖Ͱ͋Δ %BUBCBTF𝑫 0VUQVU 𝜖 ∞ TUSPOH XFBL ʜ Pr ℳ 𝐷 ∈ 𝑆 ≤ exp 𝜖 Pr ℳ 𝐷! ∈ 𝑆 + 𝛿 ℳ 𝑫′ɿ BEKBDFOUPG𝑫 ℳ 𝜖 ≥ 0, 𝛿 ∈ 0,1 ) ೖྗ͕มԽͯ͠ग़ྗʹ΄ͱΜͲʢexp 𝜖 ఔ͔͠ʣӨڹ͕ͳ͍ %XPSL%JGGFSFOUJBMQSJWBDZ*$"-1 -
-:$PSQPSBUJPO -BQMBDFϝΧχζϜ ࠷Α͘ΒΕͨϥϯμϜԽॲཧΛ࣮ݱ͢ΔϝΧχζϜʢ𝛿 = 0Λఆʣ ℳ 𝐷 = 𝑓
𝐷 + Lap 0, Δ- 𝜖 -BQMBDF.FDIBOJTN 𝜖 = 10, Δ! = 1 𝜖 = 1, Δ! = 1 ฏۉɿ ඪ४ภࠩɿ 2 "! # ͷ ϥϓϥε͔ΒϊΠζΛαϯϓϦϯά Δ. = sup /,/!∈𝒟 𝑓 𝐷 − 𝑓 𝐷) 3 ℓ𝟏 TFOTJUJWJUZ Δ45678 = 1 Δ9:;7 = 1 𝑛 ˞ఆ͢Δྡ%#ʹґଘ &Y 4FOTJUJWJUZ𝚫𝒇 ҙͷҰϨίʔυͷมԽ˞͕ؔͷग़ྗʹӨڹΛ༩͑Δ߹͍ %XPSL%JGGFSFOUJBMQSJWBDZ*$"-1 -
-:$PSQPSBUJPO ʢ͝ࢀߟʣ-BQMBDFϝΧχζϜͷূ໌ Pr[𝑀 𝐷 = 𝑦] Pr[𝑀 𝐷) =
𝑦] = Π< 𝑃=;> 𝑦< − 𝑓 𝐷 < Π< 𝑃=;> 𝑦< − 𝑓 𝐷) < = Π< exp 𝑏?3 𝑦< − 𝑓 𝐷 < − 𝑦< − 𝑓 𝐷) < ≤ Π< exp 𝑏?3 𝑓 𝐷 < − 𝑓 𝐷) < = exp 𝑏?3 C < 𝑓 𝐷 < − 𝑓 𝐷) < = exp 𝑏?3 𝑓 𝐷 − 𝑓 𝐷) 3 = exp 𝜖 Δ. 𝑓 𝐷 − 𝑓 𝐷) 3 ≤ exp 𝜖 𝑃=;> 𝑥 = 1 2𝑏 exp(−𝑏?3|𝑥|) 𝑏 = Δ. 𝜖 Δ. ≥ 𝑓 𝐷 − 𝑓 𝐷) 3 𝑥3 − 𝑥@ ≤ |𝑥3 − 𝑥@| -
-:$PSQPSBUJPO ྡσʔλϕʔε ҙͷཁૉ͚͕ͩҟͳΔσʔλϕʔεͷɻ%1Ͱఆ͢ΔʮࠩʯΛϞσϧԽ /".& $BODFS "MJDF :FT #PC /P
$ZOUIJB /P %BWJE :FT ʜ /".& $BODFS "MJDF :FT #PC /P $ZOUIJB /P %BWJE :FT &WF :FT /".& $BODFS "MJDF :FT $ZOUIJB /P %BWJE :FT /".& $BODFS "MJDF :FT #PC /P $ZOUIJB /P %BWJE :FT 'SBOD /P อޢରͷ%# 𝑑# 𝐷, 𝐷$ = 1 𝑑# ⋅,⋅ ɿϋϛϯάڑ /".& $BODFS "MJDF :FT #PC /P %BWJE :FT /".& $BODFS "MJDF /P #PC /P $ZOUIJB /P %BWJE :FT /".& $BODFS "MJDF :FT #PC /P $ZOUIJB :FT %BWJE :FT /".& $BODFS "MJDF :FT #PC :FT $ZOUIJB /P %BWJE :FT /".& $BODFS "MJDF :FT #PC /P $ZOUIJB /P %BWJE /P ྡ%# BEESFNPWBM ྡ%# SFQMBDFNFOU ϊΠζͷେ͖͞࠷େͰ BEESFNPWBMͷഒ -
-:$PSQPSBUJPO 6TFSMFWFM1SJWBDZ&WFOUMFWFM1SJWBDZ ϓϥΠόγʔอޢରͷ୯ҐʢݸਓΠϕϯτΞΠςϜFUDʣ /".& $BODFS "MJDF :FT #PC /P
$ZOUIJB /P %BWJE :FT /".& %JTFBTF "MJDF ප "MJDF നো #PC ഏ͕Μ #PC ң͕Μ $ZOUIJB ΠϯϑϧΤϯβ %BWJE ң௵ᙾ %BWJE ेೋࢦ௵ᙾ %BWJE ด࠹ Ϩίʔυਓ ෳϨίʔυਓ ࠩϓϥΠόγʔ͕ ҉తʹԾఆ͢Δܗࣜ ҰਓͨΓͷϨίʔυ͕NഒʹͳΔͱɺϊΠζͷେ͖֓͞ͶNഒʹͳΔ à ϊΠζ͕େ͖͘ͳΓա͗Δ͜ͱΛ͙ͨΊɺ ҰਓͨΓͷ࠷େϨίʔυ NBYDPOUSJCVUJPO Λ੍ݶ͢Δ͜ͱ͕͋Δ &WFOU %JTFBTF ප നো ഏ͕Μ ң͕Μ ΠϯϑϧΤϯβ ң௵ᙾ ेೋࢦ௵ᙾ ด࠹ ϨίʔυFWFOU 6TFSMFWFM1SJWBDZ &WFOUMFWFM1SJWBDZ -
-:$PSQPSBUJPO -BQMBDFϝΧχζϜͷڍಈ ϥϯμϜੑΛ࣋ͬͨڍಈΛ͢ΔͨΊɺಉ͡ઃఆͰຖճग़ྗ͕ҟͳΔ 𝜖 = 1, Δ% = 1
𝜖 = 1, Δ% = 1 𝜖 = 1, Δ% = 1 -
-:$PSQPSBUJPO -BQMBDFϝΧχζϜͷڍಈ ϓϥΠόγʔύϥϝʔλ𝜖Λมಈʢ𝜖খ à ϓϥΠόγʔอޢڧʣ 𝜖 = 0.1 𝜖
= 0.5 𝜖 = 2 𝜖 = 0.05 𝜖 = 10 Δ% = 1 -
-:$PSQPSBUJPO ࠩϓϥΠόγʔͷղऍɿ౷ܭతԾઆݕఆͷࢹ ϥϯμϜԽ͞Εͨग़ྗΛ؍ଌͨ͠ͱ͖ʹɺݩͷೖྗ %PS%`ΛͯΒΕΔ͔ à ܦݧతͳ ࠩϓϥΠόγʔ͜ͷਪͷਖ਼ޡͷ͔֬Βಋग़Ͱ͖Δ 𝜖&'( =
max log 1 − 𝛿 − FP FN , log 1 − 𝛿 − FN FP &NQJSJDBMEJGGFSFOUJBMQSJWBDZ Kairouz+. The composition theorem for differential privacy. ICML2015 5SVF*OQVU (VFTT 'BMTF1PTJUJWF 𝐷 𝐷$ 'BMTF/FHBUJWF 𝐷$ 𝐷 ℳ 𝑦 𝐷 PS𝐷) 𝐷 PS 𝐷) 𝐷 𝐷′ ྡ%# -
-:$PSQPSBUJPO ϓϥΠόγʔ༧ࢉͷཧ σʔλͷ܁Γฦ͠ར༻ʹ͏ྦྷੵతͳϓϥΠόγʔͷফඅΛ 𝜖) ͷ߹ʹΑͬͯಋग़ à ࣄલʹఆٛͨ͠ϓϥΠόγʔ༧ࢉͷൣғͰɺԠՄ൱ΛஅͰ͖Δ TBUJTGZJOH𝝐𝟏 %1
TBUJTGZJOH𝝐𝒌 %1 ʜ 2VFSZ𝒒𝟏 2VFSZ𝒒𝒌 𝝐𝟏 𝝐𝟏 𝝐𝟐 𝝐𝟏 𝝐𝟐 𝝐𝒌 ʜ ʜ EFDMJOF ྦྷੵϓϥΠόγʔফඅ 2VFSZ EBUBBDDFTT ϓϥΠόγʔͷ߹ 1SJWBDZ$PNQPTJUJPO 𝜖*+*,- = ∑𝜖) ΫΤϦԠ γεςϜ ػցֶश -
-:$PSQPSBUJPO 1SJWBDZ$PNQPTJUJPO 4FRVFOUJBM$PNQPTJUJPOʢྻ߹ʣ 𝜖 = B )∈ / 𝜖/
𝜖 = max 𝜖0, … , 𝜖/ -FUℳ% , … , ℳ& TBUJTGZ𝜖% , … , 𝜖& SFTQFDUJWFMZ 𝐷`TUPUBMQSJWBDZDPOTVNQUJPOCZℳ% 𝐷 , … , ℳ& 𝐷 JT 1BSBMMFM$PNQPTJUJPOʢฒྻ߹ʣ -FUℳ%, … , ℳ& TBUJTGZ𝜖%, … , 𝜖& SFTQFDUJWFMZ -FU𝐷 = 𝐷% ∪ ⋯ ∪ 𝐷& XIFSF𝐷' ∩ 𝐷()' = ∅ 𝐷`TUPUBMQSJWBDZDPOTVNQUJPOCZℳ% 𝐷% , … , ℳ& 𝐷& JT &YBNQMF ℳ0 ℳ1 𝜖0 = 1 ℳ2 𝜖1 = 0.5 𝜖2 = 1.5 ℳ$ ℳ% ℳ& 1 0.5 1.5 1.5 1 1 2 2.5 46. NBY ൃలతͳϓϥΠόγʔ߹ "EWBODFE$PNQPTJUJPO [$%1 3ÉOZJ %JGGFSFOUJBM1SJWBDZ 3%1 3%1[$%1͕Α͘ར༻͞Ε͍ͯΔ • ྻ߹ϧʔζ • ཧతͳ߹ະ è ༷ʑͳ߹ఆཧ 𝜖 DPNQPTJUJPOT 4FRVFOUJBM $PNQPTJUJPO JEFBM 3%1 -
-:$PSQPSBUJPO ʲݚڀࣄྫʳ)%17JFX GPS&YQMPSJOH%BUBCBTF $POTUSVDUJPOPGJOUFSNFEJBUFQSJWBUJ[FElWJFXz 1WJFX UPXBSETBDUVBMJ[JOHBOZ RVFSZSFTQPOTFTXJUITNBMMFSOPJTF 3FRVJSFNFOU 2VFSZBHOPTUJD/PJTFSFTJTUBODF4QBDFFGGJDJFOU"OBMZUJDBMMZSFMJBCMF
"DDFQUFEBU7-%# *EFOUJUZ 1SJWUSFF )%.. 1SJWCBZFT )%17JFX PVST "33 1.94×10! 7.05 35.34 3.79 𝟏. 𝟎𝟎 "WFSBHF3FMBUJWF&SSPSPWFSEBUBTFUT 4J[FPGQWJFX IUUQTBSYJWPSHBCT 4JNQMF4DBMBCMF 3BOEPN3FDVSTJWF #JTFDUJPO -
-:$PSQPSBUJPO ػցֶशͱࠩϓϥΠόγʔ ࿈߹ֶशͱதԝࠩϓϥΠόγʔ ہॴࠩϓϥΠόγʔ ࿈߹ֶशͱہॴࠩϓϥΠόγʔ
ࠩϓϥΠόγʔʹΑΔ҆શͳ࿈߹ֶश -
-:$PSQPSBUJPO 4(%ɿ4UPDIBTUJD(SBEJFOU%FTDFOU ඪ४తͳػցֶशख๏ͷҰͭɻ܇࿅σʔλ͔Βޯ HSBEJFOU Λಋग़ͯ͠Ϟσϧͷߋ৽ʹར༻ 𝐚𝐯𝐠. .JOJCBUDI 4BNQMJOH 1FSTBNQMF
(SBEJFOU %BUBCBTF𝐷 𝐷 = 𝑛 𝑥%, … , 𝑥5 ∈ 𝐷 𝑔' = ∇𝑓(𝑥'; 𝜃) 𝑔% = ∇𝑓(𝑥% ; 𝜃) 𝑔5 = ∇𝑓(𝑥5; 𝜃) 𝜃6 = 𝜃67% − 𝜂 1 𝑛 < '∈ 5 ∇𝑓 𝑥'; 𝜃67% 4(% NJOJCBUDIUSBJOJOH .PEFM6QEBUF ऩଋ͢Δ·Ͱ෮܇࿅ -
-:$PSQPSBUJPO %1ͳػցֶशϑϨʔϜϫʔΫɿ%14(% ϞσϧΛߋ৽͢Δޯ HSBEJFOU ΛϥϯμϜԽ à ܇࿅ޙͷύϥϝʔλ(𝜖, 𝛿)%1Λอূ 𝐚𝐯𝐠.
(BVTTJBO /PJTF (SBEJFOU $MJQQJOH 𝑔% = ∇𝑓(𝑥% ; 𝜃) 𝝅𝑪 𝒈𝟏 𝑔5 = ∇𝑓(𝑥5 ; 𝜃) 3BOEPN 4BNQMJOH $ 1FSTBNQMF (SBEJFOU %BUBCBTF𝐷 𝐷 = 𝑛 𝑥%, … , 𝑥5 ∈ 𝐷 𝑔' = ∇𝑓(𝑥'; 𝜃) 𝜋' 𝑔( = 𝑔( ⋅ min 1, 𝐶 𝑔( % (SBEJFOU $MJQQJOH (SBEJOUͷℓ: ϊϧϜΛఆ$Ͱ੍ݶ à ηϯγςΟϏςΟΛ$ʹ͢Δ ʢ1PTUQSPDFTTJOHఆཧΑΓʣ IUUQTBSYJWPSHBCT .PEFM6QEBUF 𝜃) = 𝜃)*$ − 𝜂 1 𝑛 P (∈ , 𝜋' 𝑔( ; 𝜃)*$ + 𝑁 0, 𝜎 %14(% ϓϥΠόγʔ༧ࢉͷൣғͰ෮܇࿅ "CBEJ %FFQ-FBSOJOHXJUI%JGGFSFOUJBM1SJWBDZ$44 -
-:$PSQPSBUJPO ʲݚڀࣄྫʳ1SJWBUF%BUB4ZOUIFTJT σʔλΛ฿͢ΔੜϞσϧΛࠩϓϥΠόγʔΛอূ͠ͳ͕Β܇࿅ à σʔλྲྀ௨Λଅਐ 1(. PVST 𝜖, 𝛿
= (1,107;) IUUQTBSYJWPSHBCT *$%& /BÏWF.FUIPE 7"&X%14(% ՝ɿ("/7"&ͷσʔλ߹ΞϧΰϦζϜෳࡶ͕ߴ͘ɺ &ODͱ%FDͷ&OEUP&OEͷ܇࿅ʹେͳ෮͕ඞཁ à ॴఆͷϓϥΠόγʔ༧ࢉͰ܇࿅͕ࠔ à ϊΠζͷӨڹΛड͚͘͢ɺੜσʔλͷ࣭͕ѱ͍ ఏҊɿ &ODͷSPVOE܇࿅ %11$" à %FDͷ෮܇࿅ %14(% "DDFQUFEBU*$%& -
-:$PSQPSBUJPO ࿈߹ֶश 'FEFSBUFE-FBSOJOH ࿈߹ֶशͷखॱ αʔόʔ͕άϩʔόϧϞσϧΛΫϥΠΞϯτ܈ʹ ΫϥΠΞϯτ͕ͦΕͧΕݻ༗ͷϩʔΧϧσʔλͰϞσϧΛධՁɺ ޯͳͲͷߋ৽ใΛܭࢉ
ߋ৽ใΛαʔόʔʹૹ ऩूͨ͠ߋ৽ใΛूͯ͠άϩʔόϧϞσϧΛߋ৽ ಛ • ϩʔΧϧσʔλ͕ΫϥΠΞϯτʹཹ·Δ • αʔόʔ͕σʔλΛཧ͠ͳͯ͘Α͍ • ΫϥΠΞϯτݻ༗ͷϩʔΧϧϞσϧʹΑΓ ϢʔβʔͷᅂͷϩʔΧϧͳಛΛө • ΫϥΠΞϯτͷܭࢉࢿݯʹΑΔ੍ • σʔλ௨৴͕8JGJଓ࣌ʹݶΒΕΔ • ΫϥΠΞϯτͷESPQPVU͕͋ΔʢిݯPGGʣ /POQBSUJDJQBOUTPG'- (MPCBM.PEFM ̍ -
-:$PSQPSBUJPO 'FE4(%"WH ΫϥΠΞϯτอ༗ͷσʔλͰޯΛܭࢉ à αʔόʔͰूޙʹάϩʔόϧϞσϧΛߋ৽ 𝐚𝐯𝐠. 𝑔% = ∇𝑓(𝑥%
; 𝜃) 𝑔5 = ∇𝑓(𝑥5 ; 𝜃) 3BOEPN 4BNQMJOH $ 1FSDMJFOU (SBEJFOU (MPCBM.PEFM 𝜃 𝑥%, … , 𝑥5 ∈ 𝐷 𝑔' = ∇𝑓(𝑥'; 𝜃) (MPCBM.PEFM6QEBUF 0OEFWJDF $PNQVUJOH -
-:$PSQPSBUJPO ࿈߹ֶश ࠩϓϥΠόγʔ (MPCBM.PEFM (MPCBM.PEFM 3BX (SBEJFOU 3BOEPNJ[FE (SBEJFOU
/PJTF*OKFDUJPO $FOUSBM.PEFM -PDBM.PEFM -
-:$PSQPSBUJPO $FOUSBM %1'FE4(%"WH ΫϥΠΞϯτอ༗ͷσʔλͰޯΛܭࢉ à αʔόʔͰूͱϥϯμϜԽޙʹϞσϧΛߋ৽ 𝐚𝐯𝐠. (SBEJFOU $MJQQJOH
𝑔% = ∇𝑓(𝑥% ; 𝜃) 𝝅𝑪 𝒈𝟏 𝑔5 = ∇𝑓(𝑥5 ; 𝜃) 3BOEPN 4BNQMJOH $ 1FSDMJFOU (SBEJFOU 𝑥%, … , 𝑥5 ∈ 𝐷 𝑔' = ∇𝑓(𝑥'; 𝜃) (MPCBM.PEFM 𝜃 (BVTTJBO /PJTF .PEFM6QEBUF ϓϥΠόγʔ༧ࢉͷൣғͰ෮܇࿅ .D.BIBO -FBSOJOH%JGGFSFOUJBMMZ1SJWBUF3FDVSSFOU-BOHVBHF.PEFMT*$-3IUUQTBSYJWPSHBCT -
-:$PSQPSBUJPO -PDBM%JGGFSFOUJBM1SJWBDZʢہॴࠩϓϥΠόγʔʣ ࢄڥԼͰͷ౷ܭऩू࿈߹ֶशͷͨΊͷϓϥΠόγʔϞσϧɻΫϥΠΞϯτͰϥϯμϜԽ 1SJWBDZ.PEFM ػඍͳ ౷ܭͷऩू αʔόʔΛ ৴༻͠ͳ͍ ϝΧχζϜ
ℳ: 𝒳 → 𝒮 ͕ 𝜖, 𝛿 %1Λຬͨ͢ͱɺ ҙͷೖྗͷ 𝑥$ , 𝑥% ∈ 𝒳 ͓Αͼҙͷग़ྗͷू߹ 𝑆 ⊆ 𝒮 ʹରͯ͠ҎԼ͕Γཱͭͱ͖Ͱ͋Δ Pr ℳ 𝑥0 ∈ 𝑆 ≤ exp 𝜖 Pr ℳ 𝑥1 ∈ 𝑆 + 𝛿 𝑥0 𝑥1 ℳ ℳ ݟ͚͕͔ͭͳ͍ (𝝐, 𝜹)-PDBM%1 ྡ%# SFQMBDFNFOUΛఆ -
-:$PSQPSBUJPO L3BOEPNJ[FE3FTQPOTF L33 ίΠϯΛ͛ͯදͳΒਅͷΛɺཪͳΒͦΕҎ֎ͷΛϥϯμϜʹԠ 𝒳 ∈ { } 3BOEPNJ[FE
0SJHJOBM 𝑅𝑅 𝑥 = M 𝑥 𝑤. 𝑝. exp 𝜖 exp 𝜖 + 𝑘 − 1 𝑥) ∼ 𝒳 ∖ 𝑥 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 ℳ SBOEPNMZTFMFDU 𝑘: |𝒳| JUFNT -
-:$PSQPSBUJPO $%1ͱ-%1ͷൺֱ $FOUSBM%1 -PDBM%1 ྡσʔλϕʔε BEESFNPWBM͕ඪ४త %#ͷαΠζҎ্ SFQMBDFNFOU͕ඪ४త %#ͷαΠζ
ϥϯμϜԽॲཧd౷ܭॲཧ αʔόʔͰ౷ܭॲཧͨ͠ग़ྗʹ ճ͚ͩϊΠζΛՃࢉ ΫϥΠΞϯτ͕࡞ͨ͠ϥϯμϜ ԽϨϙʔτΛαʔόʔʹૹ৴ ϥϯμϜԽϨϙʔτ܈Λूͯ͠ ౷ܭॲཧ à ૬ରతʹϊΠζɿେ ʢTVNͷ߹$%1ൺ 𝑛ഒʣ ஶ໊ͳϥϯμϜԽॲཧ • -BQMBDFϝΧχζϜ • (BVTTJBOϝΧχζϜ • ࢦϝΧχζϜ • 3BOEPNJ[FE3FTQPOTF • %JTDSFUF-BQMBDFϝΧχζϜ • 1PJTTPO#JOPNJOBMϝΧχζϜ ৴པϞσϧ αʔόʔΛ৴པ͢Δ αʔόʔͷ৴པඞཁͳ͍ -
-:$PSQPSBUJPO ࠩϓϥΠόγʔΛอূ͢Δ҆શͳ࿈߹ֶशʢ-%1ʣ (MPCBM.PEFM 3BOEPNJ[FE (SBEJFOU -%1ͳ$SPTTEFWJDF'-ͷಛ • αʔόʔΛ৴༻͠ͳ͍ࢄϞσϧ •
ΫϥΠΞϯτͰॴఆͷ-%1Λอূͨ͠ ϥϯμϜԽϨϙʔτΛ࡞ͯ͠ૹ৴ • ϓϥΠόγʔ༧ࢉΫϥΠΞϯτຖʹཧ • ϊΠζ͕େ͖͘ͳΓ͕ͪͳͨΊɺͰ͖Δ͚ͩ ଟ͘ͷϥϯμϜԽϨϙʔτΛฏۉԽͯ͠ ϊΠζͷӨڹΛݮ͍ͨ͠ ༗ӹͳ-%1'-Λ࣮ݱ͢Δʹ • େͳͷΫϥΠΞϯτͷࢀՃ͕ඞཁ • ϓϥΠόγʔϞσϧͷ؇ͷݕ౼ඞཁ • %JTUSJCVUFE%14FDVSF"HH ޙड़ -
-:$PSQPSBUJPO ࿈߹ֶश͚ͷϥϯμϜԽॲཧʢ-%1ʣ 1SJW6OJUϝΧχζϜ • 𝜖< -%1Λอূ • ޯΛϥϯμϜʹαϯϓϦϯά •
PSനΛ 33Ͱબ • ͞ΒʹબྖҬʢ PSനʣ͔Β Ұ༷αϯϓϧ (BVTTJBOϝΧχζϜ • $%1͚ͷ(BVTTJBOϝΧχζϜͷస༻ • ϊΠζΛ$%1͚ख๏ͷഒʹ͢Δ ̀ $%1ͱ-%1ͷྡੑͷҧ͍ʹΑΓ 33ϕʔεͷख๏ $POUJOVPVT/PJTF %JTDSFUF/PJTF %JTDSFUF-BQMBDFϝΧχζϜ • ಠཱͨͭ͠ͷ֬ม d1PJTTPO ͷࠩͷ 4LFMMBNϝΧχζϜ ˞࠶ੜੑͷ͋ΔΛ༻͍Δඞཁ͕͋Δ IUUQTFOXJLJQFEJBPSHXJLJ4LFMMBN@EJTUSJCVUJPO 4LFMMBN #IPXNJDL 1SPUFDUJPO"HBJOTU3FDPOTUSVDUJPOBOE *UT"QQMJDBUJPOTJO1SJWBUF'FEFSBUFE-FBSOJOH IUUQTBSYJWPSHBCT "HBSXBM 5IF4LFMMBN .FDIBOJTNGPS%JGGFSFOUJBMMZ1SJWBUF 'FEFSBUFE-FBSOJOH/FVS*14IUUQTBSYJWPSHBCT -
-:$PSQPSBUJPO -%1'FE4(%"WH ΫϥΠΞϯτͰޯʢ·ͨࠩʣͷܭࢉͱϥϯμϜԽ à αʔόʔͰूޙʹϞσϧΛߋ৽ 𝐚𝐯𝐠. (SBEJFOU $MJQQJOH 𝑔%
= ∇𝑓(𝑥% ; 𝜃) 𝝅𝑪 𝒈𝟏 𝑔5 = ∇𝑓(𝑥5 ; 𝜃) 3BOEPN 4BNQMJOH $ 1FSDMJFOU (SBEJFOU (MPCBM.PEFM 6QEBUF 𝑥%, … , 𝑥5 ∈ 𝐷 𝑔' = ∇𝑓(𝑥'; 𝜃) (MPCBM.PEFM 𝜃 1FSUVSCBUJPO ΫϥΠΞϯτͷϓϥΠόγʔ༧ࢉͷ ൣғͰ෮܇࿅ -
-:$PSQPSBUJPO ۙͷಈɿ1SJWBDZ5FDI$PNCP ϓϥΠόγʔอޢͷਫ४ͱσʔλར׆༻ͷ༗༻ੑͷόϥϯεͷվળΛతͱͯ͠ɺηΩϡΞͳ ϋʔυΣΞϓϩτίϧΛલఏͱ͢ΔࠩϓϥΠόγʔͷݚڀ͕׆ൃԽ &/1"ΞʔΩςΫνϟ ൿີࢄ ࠩϓϥΠόγʔ IUUQTXXXBQQMFDPNKQOFXTSPPN BQQMFBOEHPPHMFQBSUOFSPODPWJEDPOUBDU
USBDJOHUFDIOPMPHZ IUUQTDPWJETUBUJDDEO BQQMFDPNBQQMJDBUJPOTDPWJEDVSSFOUTUBUJDDPOUBDU USBDJOHQEG&/1"@8IJUF@1BQFSQEG 'FEFSBUFE"OBMZUJDT IUUQTBSYJWPSHBCT -
-:$PSQPSBUJPO 4IVGGMFNPEFMr BOJOUFSNFEJBUFQSJWBDZNPEFM • *OUFSNFEJBUFUSVTUFEFOUJUZlTIVGGMFSz BOPOZNJ[FTMPDBMVTFST`JEFOUJUZ • &BDIDMJFOUFODSZQUTUIFJSSBOEPNJ[FEDPOUFOUXUIFTFSWFS`TQVCMJDLFZ UIFO
TIVGGMFSPOMZNJYFTUIFJSJEFOUJGJFTXPMPPLJOHBUUIFDPOUFOUT Q 𝑥0 Q 𝑥1 Q 𝑥3 R 𝑥0 (R 𝑥0, R 𝑥1, … , R 𝑥3) R 𝑥1 R 𝑥3 4FSWFS 3BOEPNJ[FEX𝝐𝟎 4IVGGMF 4IVGGMFS 4FOEUIFTIVGGMFECBUDI BOPOZNJ[FE -
-:$PSQPSBUJPO 1SJWBDZ"NQMJGJDBUJPOWJB4IVGGMJOH • 4IVGGMFSDBOBNQMJGZEJGGFSFOUJBMQSJWBDZà QPTTJCJMJUZUPEFDSFBTFMPDBMOPJTF • 5IFBNQMJGJDBUJPOPOTIVGGMFSUSBOTMBUFT-%1PODMJFOUTJOUP$%1 𝜖5 =
8 -%1 𝛿 = 1067 𝑘 = 10 CZ)JEJOHBNPOHDMPOFT &YBNQMFJOLSBOEPNJ[FESFTQPOTF IUUQTBSYJWPSHBCT 1SJWBDZ"NQMJGJDBUJPO A 𝑥% A 𝑥: A 𝑥= B 𝑥% (B 𝑥% , B 𝑥: , … , B 𝑥= ) B 𝑥: B 𝑥= 4IVGGMFS 4FSWFS 𝝐𝟎 -%1 𝝐 < 𝝐𝟎 $%1 -
-:$PSQPSBUJPO 4IVGGMFE$IFDL*OTGPS'- ࿈߹ֶशʹ4IVGGMOHΛಋೖ • ΫϥΠΞϯτ܈ʢ𝑛ʣͷҰ෦ʢ𝑚 < 𝑛ʣΛαϯϓϧͯ͠ϞσϧΛ෮తʹ܇࿅͍ͨ͠ à Ұ܇࿅ͷࢀՃΫϥΠΞϯτ𝑚͕গͳ͍
à 4IVGGMJOH͕ෆे à 4VCTBNQMJOHΛߟྀ • 4VCTBNQMJOHΫϥΠΞϯτͷESPQPVUʹ੬ऑ à ؤ݈ͳϞσϧΛಋೖ͍ͨ͠ )JHI6UJMJUZBU4USPOH1SJWBDZ-FWFM "NQMJGJFE 4IVGGMFE$IFDL*OT 𝛾 = 𝑝 1 − 𝑝$ 𝑝DIFDLJOSBUF 𝑝 𝑝$ESPQPVUSBUF 𝛾FGGFDUJWFDIFDLJOSBUF Q 𝑥0 Q 𝑥8 Q 𝑥3 {Q 𝑥8, Q 𝑥3, … , Q 𝑥0} -
-:$PSQPSBUJPO ʲݚڀࣄྫʳ/FUXPSL4IVGGMJOH %FDFOUSBMJ[FETIVGGMJOHWJBNVMUJSPVOESBOEPNXBMLT POBHSBQI *OFBDISPVOE FWFSZDMJFOUSFMBZTIFSSBOEPNJ[FESFQPSUTUPPOFPGIFSOFJHICPST FH GSJFOETPOBTPDJBMOFUXPSL WJBBOFODSZQUFEDIBOOFM
5IFMBSHFSHSBQIBNQMJGJFTQSJWBDZUIFNPSF "DDFQUFEBU4*(.0% IUUQTBSYJWPSHBCT -
-:$PSQPSBUJPO 4FDVSF"HHSFHBUJPO "NBUVSFQSJWBDZUFDIDPNCPUIBUBMMPXTVTUPFNQMPZXFBLFS%1NPEFMUIBU SFTVMUTJOTJHOJGJDBOUOPJTFSFEVDUJPO ℎ𝑖𝑠𝑡/ = B )∈['>] 𝑥)
+ 𝑧) ℎ𝑖𝑠𝑡/ = B )∈['>] 𝑥) + 𝑧) 𝒎𝒌 -PDBMNPEFM %JTUSJCVUFENPEFM ∑ 4FDVSF"HH %JTUSJCVUFE%1 BOUJNFNPSJ[BUJPO "OPOZNPVT $IBOOFM 0OEFWJDF $PNQVUJOH ʜ ʜ )JHIVUJMJUZ WJB SFMBYFE%1NPEFM VOEFSTUSPOHFOPVHI QSJWBDZUFDIDPNCP -
-:$PSQPSBUJPO %JTUSJCVUFE%JGGFSFOUJBM1SJWBDZ "OJOUFSNFEJBUFQSJWBDZNPEFMXMPDBMQSJWBUJ[BUJPOBOETFDVSFQSPUPDPMT /PJTFTDBMF-%1 -%14IVGGMF ʾ%%1 ʾ$%1 ℎ𝑖𝑠𝑡/ = B
)∈ '> 𝑥) + 𝑧) ℎ𝑖𝑠𝑡/ = B )∈ '> 𝑥) + 𝑧/ ∑ ∑ ∑ -PDBM%1 %JTUSJCVUFE%1 $FOUSBM%1 )VHFBNPVOUPGOPJTF OPUSVTUFEFOUJUJFT • -FTTOPJTFBHBJOTUMPDBMEQ FRVJWBMFOUUP$%1BGUFSTVNNBUJPO • *OTUBMMJOHTFDVSFUSVTUFEFOUJUZ 'BJSOPJTF CVUUSVTUFEDVSBUPSJTSFRVJSFE ℎ𝑖𝑠𝑡/ = B )∈ '> 𝑥) + 𝑧) 𝑚/
lycorptech_jp
ryudai
birgitta410
.jpg){kind=avatar}
ryouto715
iwamot
jirtosterone
takuyay0ne
daishukojima
fortegp05
munakata
ama_ch
inariku
mongodb
pwiseman
eileencodes
konakalab
geshan
davetheseo
mfonobong
palkan
bermonpainter
sergeychernyshev
tammielis
lara
![-:$PSQPSBUJPO ʢ͝ࢀߟʣ-BQMBDFϝΧχζϜͷূ໌ Pr[𝑀 𝐷 = 𝑦] Pr[𝑀 𝐷) =](https://files.speakerdeck.com/presentations/821effefe6ad416b96a272f519052bb9/slide_22.jpg){kind=link}
![-:$PSQPSBUJPO 4FDVSF"HHSFHBUJPO "NBUVSFQSJWBDZUFDIDPNCPUIBUBMMPXTVTUPFNQMPZXFBLFS%1NPEFMUIBU SFTVMUTJOTJHOJGJDBOUOPJTFSFEVDUJPO ℎ𝑖𝑠𝑡/ = B )∈['>] 𝑥)](https://files.speakerdeck.com/presentations/821effefe6ad416b96a272f519052bb9/slide_54.jpg){kind=link}
