ABテストが上手く行かなかった話

Slide 1

Slide 1 text

鸐濼鏩閉桦ぢ♳ך׋׭ך倵瘻ך "#ذأزדֲתְַֻזַ׏׋鑧傈劤穗幥倜耀爡խ⚥䊛♰蛁㣽 Otemachi.swift #02

Slide 2

Slide 2 text

Marina Nakagawa @marina10172013 ೔ຊܦࡁ৽ฉࣾͷiOSΤϯδχΞ  ిࢠ൛ΞϓϦɼࢴ໘ϏϡʔΞʔΞϓϦͷ։ൃ

Slide 3

Slide 3 text

ຊ೔࿩͢͜ͱ ࣮ࡍʹऔΓ૊Μͩ͜ͱ ௨஌ڐ୚཰Λ޲্ͤ͞ΔͨΊͷࢪࡦͷ֓ཁ 1 2 3 ݁Ռͱ·ͱΊ

Slide 4

Slide 4 text

௨஌ڐ୚཰Λ޲্ͤ͞ΔͨΊͷࢪࡦͷ֓ཁ 1

Slide 5

Slide 5 text

೔ܦిࢠ൛ͷ௨஌ڐ୚཰ʹ͍ͭͯ • ిࢠ൛ͷ௨஌ͷڐ୚ͷΞϥʔτΛग़͢ͷ͸ىಈ࣌ • ςετ։࢝࣌ͷڐ୚཰͸໿52ˋҐ Կ΋͠ͳͯ͘΋໿52%͘Β͍ͷਓ͸ ڐ୚ͯ͘͠Ε͍ͯΔ

Slide 6

Slide 6 text

ڐ୚཰52%ͷײ֮ New data shows up to 60% of users opt-out of push notiﬁcations (Guest Post)ΑΓҾ༻  (https://andrewchen.co/why-people-are-turning-oﬀ-push/)

Slide 7

Slide 7 text

ڐ୚཰52%ͷײ֮ New data shows up to 60% of users opt-out of push notiﬁcations (Guest Post)ΑΓҾ༻  (https://andrewchen.co/why-people-are-turning-oﬀ-push/)

Slide 8

Slide 8 text

ڐ୚཰52%ͷײ֮ New data shows up to 60% of users opt-out of push notiﬁcations (Guest Post)ΑΓҾ༻  (https://andrewchen.co/why-people-are-turning-oﬀ-push/) ҰൠతͳχϡʔεͷΞϓϦͱ  ൺֱ͢Δͱଟ͍

Slide 9

Slide 9 text

௨஌Λ։͘ͱ௚઀هࣄʹඈͿΑ͏ʹͳͬͨ

Slide 10

Slide 10 text

ࠓͦ͜௨஌Λ࢖ͬͯ΄͍͠ɾɾɾ

Slide 11

Slide 11 text

௨஌ڐ୚཰Λ΋ͬͱ΋ͬͱ্͛ͯ! ϢʔβʔʹχϡʔεΛಧ͚͍ͨ

Slide 12

Slide 12 text

௨஌ΛڐՄͯ͠΋Β͏ʹ͸ ৐Γӽ͑Δඞཁ͕͋Δน͕͋Δ

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Ծઆ ΞϓϦͷॳճΠϯετʔϧ࣌ʹ  ϓογϡ௨஌ͷҰྫΛࣔ͠༗༻Ͱ͋Δ͜ͱΛࣔͤΕ͹  ௨஌ͷڐ୚཰্͕͕ΔͷͰ͸ͳ͍͔

Slide 15

Slide 15 text

࣮ࡍʹऔΓ૊Μͩ͜ͱ 2

Slide 16

Slide 16 text

ը໘ભҠਤ

Slide 17

Slide 17 text

ϙοϓΞοϓը໘ͷৄࡉ ͜ͷϘλϯΛԡͨ͠ਓ͚͕ͩ  OSͷڐ୚௨஌ͷϙοϓΞοϓ͕Ͱͯ͘Δ

Slide 18

Slide 18 text

ϙοϓΞοϓը໘ͷৄࡉ

Slide 19

Slide 19 text

ϙοϓΞοϓը໘ͷৄࡉ ࠓ͸΍Ί͓ͯ͘ϘλϯΛԡͨ͠ਓ ࿮֎λοϓΛͨ͠ਓ ԼεϫΠϓΛͨ͠ਓ

Slide 20

Slide 20 text

ϙοϓΞοϓը໘ͷৄࡉ

Slide 21

Slide 21 text

݁Ռͱ·ͱΊ 3

Slide 22

Slide 22 text

▪ ϙοϓΞοϓͳ͠ɿ໿40% ▪ ϙοϓΞοϓ͋Γɿ໿21% ௨஌ڐ୚཰:52% ࣮ࢪޙ ࣮ࢪલ A/Bςετͷ݁Ռ

Slide 23

Slide 23 text

A/Bςετͷ݁Ռ ϙοϓΞοϓΛදࣔͨ͠σόΠεͷํ͕  ௨஌ONͷׂ߹͕൒਺ఔ౓ͩͬͨ

Slide 24

Slide 24 text

A/Bςετͷ݁Ռ ϙοϓΞοϓΛදࣔͨ͠σόΠεͷํ͕  ௨஌ONͷׂ߹͕൒਺ఔ౓ͩͬͨ མͪͯΔɾɾɾɾɾ "

Slide 25

Slide 25 text

ݪҼ͸ʁ Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1 2 ϝϦοτΛϢʔβʔʹ఻͖͑Ε͍ͯͳ͔ͬͨ

Slide 26

Slide 26 text

Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1

Slide 27

Slide 27 text

Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1

Slide 28

Slide 28 text

Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1 ͜ͷϘλϯΛԡͨ͠ਓ͚͕ͩ  OSͷڐ୚௨஌ͷϙοϓΞοϓ͕Ͱͯ͘Δ

Slide 29

Slide 29 text

ࠓ͸΍Ί͓ͯ͘ϘλϯΛԡͨ͠ਓ ࿮֎λοϓΛͨ͠ਓ ԼεϫΠϓΛͨ͠ਓ ໿70%ͷਓ͕OSͷڐ୚௨஌Λ  ݟͳ͍͜ͱʹͳͬͯ͠·ͬͨ Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1

Slide 30

Slide 30 text

○ཁૉΛͻͱ͚ͭͩม͑ͯςετ͢Δͷ͕A/Bςετͳͷʹɺ ෳ਺ཁૉͰൺֱͤͯ͞͠·ͬͨ  ○ࣗ෼͕ίϯςϯπͷத਎(ಈըͷ੍࡞΍දهͷ಺༰)ʹूத͠ ͯ͠·͍ભҠͷ͜ͱ·Ͱߟ͍͑ͯͳ͔ͬͨ ○ԼϑϦοΫɺແࢹ͢Δͱ͍͏બ୒ࢶ͸ແͯ͘͠Α͔ͬͨ ഊҼ Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1

Slide 31

Slide 31 text

Ծઆ ϓογϡ௨஌ͷҰྫΛࣔ͠༗༻Ͱ͋Δ͜ͱΛࣔͤΕ͹ ௨஌ͷڐ୚཰্͕͕Δ͔͔֬ΊΔ ࢼ͢͜ͱ OSͷ௨஌ڐ୚͕ग़ΔલʹϙοϓΞοϓը໘Ͱ  ༗༻Ͱ͋Δ͜ͱΛઆ໌͢Δ͜ͱͰ௨஌ڐ୚͸มΘΔͷ͔ Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1

Slide 32

Slide 32 text

Ծઆ ϓογϡ௨஌ͷҰྫΛࣔ͠༗༻Ͱ͋Δ͜ͱΛࣔͤΕ͹ ௨஌ͷڐ୚཰্͕͕Δ͔͔֬ΊΔ ࢼ͢͜ͱ OSͷ௨஌ڐ୚͕ग़ΔલʹϙοϓΞοϓը໘Ͱ  ༗༻Ͱ͋Δ͜ͱΛઆ໌͢Δ͜ͱͰ௨஌ڐ୚͸มΘΔͷ͔ Ϣʔβʔ͕཭୤͢Δͱ͜ΖΛ࡞ͬͯ͠·ͬͨ 1 มߋ͢Δ΂͖͸ϙοϓΞοϓը໘Λग़͔͢ग़͞ͳ͍͔Ͱ͋Γ  OSͷ௨஌ڐ୚ը໘Λग़͔͢ग़͞ͳ͍͔มߋΛՃ͑Δ΂͖Ͱ͸ͳ͍

Slide 33

Slide 33 text

ʮࠓ஌Δ΂͖ʯΛಀ͞ͳ͍ ศརͳϓογϡ௨஌ 2 ϝϦοτΛϢʔβʔʹ఻͖͑Ε͍ͯͳ͔ͬͨ ○ҙࢥܾఆΛઌʹ͖͚࣋ͬͯͨͲ৘ใ͕গͳ͍͔Β͔΋͠Εͳ͍  ○ಈը͸ॏཁͰ͸ͳ͍͔΋͠Εͳ͍  ഑ஔͷ໰୊ͰจݴͱϘλϯΛ͚ۙͮͨ΄͏͕Α͔͔ͬͨ΋ ○Ϙλϯͷબ୒ࢶͰ͋Δ  ʮ͍·͸΍Ί͓ͯ͘ʯ͸·ͩνϟϯε͕༗ΔΑ͏ʹݟ͑Δ

Slide 34

Slide 34 text

·ͱΊ A/BςετΛ࣮ࢪ͢Δ࣌͸ɺ  ຊ౰ʹཁૉΛͻͱ͚ͭͩม͑ͯൺֱͯ͠Δͷ͔֬ೝ͢Δ ϙοϓΞοϓʹදࣔͨ͠ίϯςϯπ͕ຊ౰ʹྑ͔ͬͨͷ͔Ͳ͏͔͸Θ͔Βͳ͍  จݴ΍ίϯςϯπʹΑͬͯҧ͍͕͋Δͷ͔ͱ͍͏ͷ΋ࢼͯ͠Έ͍ͨ

Slide 35

Slide 35 text

͋Γ͕ͱ͏͍͟͝·ͨ͠