Upgrade to Pro — share decks privately, control downloads, hide ads and more …

FiNCの分析基盤の概要

Yoshimi Kondo
April 26, 2017
4.6k

 FiNCの分析基盤の概要

Yoshimi Kondo

April 26, 2017
Tweet

Transcript

  1. FiNCͷαʔϏε঺հ 3 ݕٕࠪज़΍ઐ໳ՈɾଞۀքωοτϫʔΫΛ׆༻͠ɺݸਓ΍๏ਓͷ݈߁Λࢧԉ QPXFSFECZ ຖ৯ʹαϥμΛͭ ͚Δ ఏܞ ళฮ޲͚ ๏ਓ޲͚ "*ύʔιφϧίʔνΞϓϦʢ'J/$ʣ

    ΢ΣϧωεϔϧεέΞϓϥοτϑΥʔϜ  'J/$Ϟʔϧʢ&$ʣ ϔϧεέΞʹಛԽͨ͠ΦϯϥΠϯετΞ ݕࠪࣄۀ Ҩ఻ࢠݕࠪɾ݂ӷɾετϨενΣοΫ౳ Ұൠ޲͚ ΦϯϥΠϯϫʔΫε ϔϧεέΞઐ໳Ո΍ΞεϦʔτͷΫϥ΢υιʔγϯά 'J/$Ոఉڭࢣ ಛఆอ݈ࢦಋɾݮྔɾӫཆӡಈࢦಋͷεϚϗԕִࢦಋ
  2. ϩάऩूج൫ 15 BQQGSBNFXPSL LJOFTJTpSFIPTF MBNCEB 4 "NB[PO%.4 4 API Access

    Custom Event TFSWFSBQQ Ϣʔβʔͷߦಈϩάͱͯ͠ɺAPIΞΫηεͰ͸ิ׬͖͠Εͳ͍ߦಈΛ෼ੳ͢Δχʔζ͕ੜ·Εͨ →ޙ͔ΒCustomEventͱͯ͠firehoseʹྲྀ͠ࠐΉΑ͏ʹͳͬͨʢͳ͓ࠓ΋APIΞΫηεϩά΋ར༻ʣ ΞϓϦέʔγϣϯଆͷσʔλϕʔε͸ɺDMSͰRedshiftʹྲྀ͠ࠐΜͰ͍Δ AccessLog APIॲཧ
  3. ϩάऩूج൫ 16 BQQGSBNFXPSL LJOFTJTpSFIPTF MBNCEB 4 "NB[PO%.4 4 ৯ࣄΛه࿥ͨ͠ʂ ϨγϐಈըΛ࠶ੜͨ͠ʂ

    TFSWFSBQQ ྫɿಈը࠶ੜϩάͱ৯ࣄه࿥ػೳར༻ ΞϓϦέʔγϣϯϩδοΫʹؔΘΔσʔλ→αʔόʔαΠυΞϓϦέʔγϣϯͷRDS্ʹอଘ ෼ੳ͚ͩʹ͔͠࢖Θͳ͍σʔλ→෼ੳ༻framework(firehose)͔Βอଘ →શͯΛ෼ੳϩάʹྲྀ͢ͷ͠ΜͲ͍&෼ੳ༻ϩάͷ੹຿ΛΞϓϦέʔγϣϯʹ࣋ͪࠐΈͨ͘ͳ͍ͨΊ ৯ࣄΛه࿥͢Δલʹ ϨγϐಈըΛ࠶ੜͨ͠ʂ AccessLog
  4. ϩάूܭج൫ 19 ੜϩάΛͦͷ··ͭͬ͜Μͩςʔϒϧ͕ͻ͠Ί͘εΩʔϚ firehose͔Βͷϩάςʔϒϧ APIΞΫηεϩάςʔϒϧ ֤αʔόʔΞϓϦέʔγϣϯ͔ΒҾͬு͖ͬͯͨεΩʔϚ ϥΠϑϩάܥςʔϒϧͱ͔ ϑΥϩʔͱ͔͍͍Ͷͱ͔ ޿ࠂπʔϧͳͲ֎෦αʔϏε͔ΒҾͬு͖ͬͯͨεΩʔϚ ྲྀೖܦ࿏ͱ͔

    ޿ࠂΫϦΤΠςΟϒͱ͔ Ϣʔβʔ͝ͱ/ػೳ͝ͱʹαϚͬͨϏϡʔ(ςʔϒϧ)ͷεΩʔϚ σϞάϥ৘ใxΞΫςΟϏςΟ ޿ࠂ৘ใxΞΫςΟϏςΟ DailyͷϢʔβʔߋ৽σʔλ ੜϩάܥεΩʔϚ ूܭؔ਺ͰKPIʹ͍ۙܗͰ·ͱΊͨςʔϒϧͨͪͷεΩʔϚ DAU/RR/ActivationRate ֤ػೳར༻཰ूܭ ਓؾίϯςϯπ ͍ΘΏΔ෼ੳ༻εΩʔϚ ͻͱͭͷRedshift clusterʹू໿
  5. ϩάूܭج൫ 20 ੜϩάΛͦͷ··ͭͬ͜Μͩςʔϒϧ͕ͻ͠Ί͘εΩʔϚ firehose͔Βͷϩάςʔϒϧ APIΞΫηεϩάςʔϒϧ ϥΠϑϩάܥςʔϒϧͱ͔ ϑΥϩʔͱ͔͍͍Ͷͱ͔ ޿ࠂπʔϧͳͲ֎෦αʔϏε͔ΒҾͬு͖ͬͯͨεΩʔϚ ྲྀೖܦ࿏ͱ͔ ޿ࠂΫϦΤΠςΟϒͱ͔

    ੜϩάܥεΩʔϚ ෼ੳ༻εΩʔϚͷݩσʔλ σʔλΛਂ௥͍͍ͨ࣌͠ʹɺ σʔλΞφϦετ΍ ΫΤϦΛୟ͖͍ͨΤϯδχΞ͕ ੜϩάܥͷεΩʔϚΛ࢖͏ ೉қ౓ ͋Δఔ౓ͷΞϓϦϩδοΫ΍ SQLͷ஌͕ࣝͦΕͳΓඞཁ ֤αʔόʔΞϓϦέʔγϣϯ͔ΒҾͬு͖ͬͯͨεΩʔϚ
  6. ϩάूܭج൫ 21 σϞάϥ৘ใxΞΫςΟϏςΟ ޿ࠂ৘ใxΞΫςΟϏςΟ DailyͷϢʔβʔߋ৽σʔλ ूܭؔ਺ͰKPIʹ͍ۙܗͰ·ͱΊͨςʔϒϧͨͪͷεΩʔϚ DAU/RR/ActivationRate ֤ػೳར༻཰ूܭ ਓؾίϯςϯπ ͍ΘΏΔ෼ੳ༻εΩʔϚ

    KPIͷ೺Ѳ΍ جຊతͳσʔλ෼ੳ͸ ෼ੳ༻εΩʔϚͰ࿫͏ ೉қ౓ ΤΫηϧΛ࢖͑ΔਓͰ͋Ε͹ σʔλΤΫεϙʔτΛͯ͠ ͓͓Αͦͷ෼ੳ͕Մೳ
 
 SQLͷ෍ڭ΋େࣄɺͰ΋ΤΫηϧͰͰ͖Δ͜ͱ΋ଟ͍ͷͰɺ ෼ੳ༻ͷσʔλςʔϒϧΛ༻ҙͨ͠΄͏͕ ίεύ͕ྑ͍͜ͱ΋ଟ͍Ͱ͢ ※αΠζେ͖͗͢ͳ͍Α͏ɺΤΫεϙʔτͷ৚݅ൣғ͸ཁࢦఆ Ϣʔβʔ͝ͱ/ػೳ͝ͱʹαϚͬͨϏϡʔ(ςʔϒϧ)ͷεΩʔϚ
  7. ϏδϡΞϥΠθʔγϣϯ 23 redash͕γϯϓϧͰศར - ΞυϗοΫͳΫΤϦ - ୯ҰͷKPI΢Υον - ؆қͳμογϡϘʔυ redashͱͷಉظػೳΛ׆༻

    - Ϩϙʔτͱͯ͠·ͱΊ͍ͨ࣌ - ෳ਺ͷKPIΛҰؾʹฒ΂ͯݟ͍ͨ࣌ ͖Ε͍ͳUIͷμογϡϘʔυ΋େࣄ!Ͱ΋ ·ͣ͸ݟΔ΂͖਺ࣈ͕ݟΕ͍ͯΔ͜ͱ͕ॏཁ