対数変換ってしてもいいの？ / Is it really okey to do Log Transformation

This presentation close in the secret behind Log Transformation.

Transcript

1. ର਺ม׵ͬͯͯ͠΋͍͍ͷʁ Is it really okey to do Log Transformation?
2. Ͳ͏ͯͦ͠Β͸͓͍͋ͷ͔ͳ
   Ͳ͏ͯ͠ର਺ม׵ͯ͠΋͍͍ͷ͔ͳ ͱ͍͏ࢲ΋ؚΊΈͳ͞Μ͕༮গظ͔Β๊͍͍ͯΔ ૉ๿ͳٙ໰ʹऔΓ૊Έ·͕ͨ͠ɺ࠳͚ͦ͏ʹͳΓ ·ͨ͠ͷͰɺҰ୴தؒ݁ՌΛ͝ใࠂ͠·͢
3. ಛ௃ྔΤϯδχΞϦϯάʹ͓͚Δ਺஋ม׵ Q.ʮม׵ͯ͠΋͍͍ͷ͔ʁͦΕͰࣦΘΕΔ΋ͷ͸ͳ͍ͷ͔ʁʯ A. ʮৗʹτϨʔυΦϑͰ͋Δʯ ࣦΘΕΔ΋ͷ͸͔֬ʹ͋ΓɺͦΕʹରͯ͠ಘΒΕΔ΋ͷ΋͋Δ ࣦΘΕΔ͹͔Γͷ͜ͱ΋͋ΔͷͰɺΉ΍Έʹߦ͑͹ྑ͍ͱ͍͏Θ͚Ͱ͸ͳ͍ ʢͨͩɺ༧ଌੑೳ΋൚Խੑೳ΋े෼ߴ͍Ϟσϧ͕Ͱ͖ΔͳΒͦΕ͕ਖ਼ٛͳͷͰ͸ͱ͸ࢥ͏ʣ
4. ର਺ม׵Ͱ۩ମతʹݟͯΈΑ͏
5. ର਺ม׵ ͋Δม਺xʹ͍ͭͯɺln(x)ʹม׵Λߦ͏͜ͱ
6. ର਺ม׵Λߦ͏Ϟνϕʔγϣϯ 1. ਖ਼ن෼෍ʹ͍ͨ͠ 2. ෼ࢄΛݮΒ͍ͨ͠ ▶ͦΕͧΕϝϦοτɾσϝϦοτ͕͋Δ
7. 1. ਖ਼ن෼෍ʹ͍ͨ͠ ਖ਼ن෼෍Λલఏͱ͢ΔύϥϝτϦοΫϞσϧ΍ ౷ܭతݕఆʹ͓͍ͯɺͦͷલఏʹద߹ͤ͞Δ ྫʣޡ͕ࠩਖ਼ن෼෍Ͱ͋Δ͜ͱΛཁٻ͢ΔҰൠઢܗϞσϧͳͲ ಛʹɺର਺ਖ਼ن෼෍ʹै͏ม਺ͷ৔߹͸ɺɹɹ ਖ਼ن෼෍ʹม׵Ͱ͖ΔͨΊ༗༻ ʢ෋ͷ෼෍ͳͲɺ͜ͷ෼෍͸ଟ͘ݟΒΕΔʣ ର਺ਖ਼ن෼෍
8. ͔͠͠ٯʹݴ͏ͱɺͦ͏Ͱͳ͍৔߹͸ޡͬͨੑ࣭ʹม׵ͯ͠͠·͏͜ͱʹͳΔ ྫ͑͹ɺԼਤͷ෼෍ʹै͏σʔλͷ৔߹ɺର਺ม׵͢Δͱ෼෍͕ߋʹ࿪ΜͰ͠ ·͍ɺ໌Β͔ʹਖ਼ن෼෍ʹ͸ݟ͑ͳ͍
9. 2. ෼ࢄΛݮΒ͍ͨ͠ ֎Ε஋ͷӨڹΛখ͍ͨ͘͞͠ɺͱ΋ݴ͍͔͑ΒΕΔ ࿪Μͩ෼෍Λਖ਼ن෼෍ʹม׵͢Δ͜ͱʹΑͬͯɺ ֎Ε஋ͷӨڹΛখ͘͞Ͱ͖Δ
10. ͨͩ͠ɺ෼෍ʹΑͬͯ͸ٯʹ෼ࢄ͕େ͖͘ͳΔ͜ͱ΋͋Δ ྫ͑͹ɺԼਤͷ෼෍ʹै͏σʔλͷ৔߹ɺβʹΑͬͯ͸ඪ४ޡࠩʢSEʣʢ㲈ඪ४ภࠩ˞ʣ͕େ͖͘ͳ͍ͬͯΔ ·ͨɺ֎Ε஋ͷ༗ແʹΑͬͯඪ४ޡࠩ͸มΘΒͳ͍ͨΊɺର਺ม׵ʹ͸֎Ε஋ʹΑΔมಈੑΛௐ੔͢Δೳྗ͸ͳ͍ ※ภճؼ܎਺ͷඪ४ޡࠩ͸ɺޡࠩͷ෼ࢄʗೋ৐ޡࠩͷ૯࿨ͰܭࢉͰ͖ΔͨΊ
11. ର਺ม׵ͷ࢖͍ॴʢ௒ࢲݟʣ ະݕূͳͷͰ࿩൒෼Ͱ ύϥϝτϦοΫͳख๏ʹ͓͍ͯɺਖ਼ن෼෍Ͱ͋Δ͜ͱΛલఏͱͯ͠ཁٻ͞ΕΔ࣌ A. ର਺ਖ਼ن෼෍ͱࢥΘΕΔσʔλ B. αϯϓϧ਺͕গͳ͘࿪ΜͰ͍Δ͕ɺܦݧతʹਖ਼ن෼෍ʹै͏ͱΘ͔͍ͬͯΔσʔλ ϊϯύϥϝτϦοΫͳख๏ʹ͓͍ͯ͸ɺಛஈ࢖͏ҙຯ͕ݟ͑ͳ͍ આ໌΋͠ʹ͘͘ͳΔ ֎Ε஋Λॲཧ͍͚ͨͩ͠ͳΒɺଞͷख๏ʢΫϦοϐϯά΍ϏχϯάɺϥϯΫԽʣΛࢼ͢΂͖ͱࢥ͏ ಛʹର਺ΛऔΔͱฏۉͷେখ͕ٯస͢ΔՄೳੑ͕͋ΔͨΊɺେখΛൺֱ͢ΔΞϧΰϦζϜͷػցֶश ख๏ʹͱͬͯ͸ѻ͍ʹ஫ҙΛཁ͢Δ ྫʣܾఆ໦ܥͷख๏Ͱ৘ใརಘܭࢉʹδχෆ७౓Λ࠾༻ͨ͠৔߹ʢδχෆ७౓͸෼ࢄΛܭࢉ͢ΔࣄͱಉٛͳͷͰฏۉΛར༻͍ͯ͠Δʣ
12. ࢀߟࢿྉ Changyong FENG, Hongyue WANG, Naiji LU, Tian CHEN, Hua HE, Ying LU, Xin M. TU(2014), “Log-transformation and its implications for data analysis” ໳࿬େีɼࡕాོ࢘ɼอࡔܡ༎ɼฏদ༤࢘ ʢ̎̌̍̕ʣ, KaggleͰউͭσʔλ෼ੳͷٕज़, ٕज़ධ ࿦ࣾ, ୈࡾষ ಛ௃ྔͷ࡞੒, page.123-135 Kaggle discussion “Why do so many Kagglers perform log transformation on independent variables?” https://www.kaggle.com/c/house-prices-advanced-regression-techniques/discussion/60861 Wikipedia (ja) ϙʔϧɾϫΠε ର਺ਖ਼ن෼෍