Upgrade to Pro — share decks privately, control downloads, hide ads and more …

모빌리티데이터팀 신입 데이터 분석가의 1년 회고

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
SOCAR
October 19, 2019

모빌리티데이터팀 신입 데이터 분석가의 1년 회고

데이터야놀자 2019에서 발표한 자료입니다

SOCAR

October 19, 2019

Transcript

95 slides
  1. ࣗѐ ӂਮജ • ҃৔೟ + ஹೊఠҕ೟ ੹ҕ੉ ݏ૑ ঋই ੉ز

    • झఋ౟সҗ ؘ੉ఠী ҙब੉ ݆਺ • 2019.01 ॑஠ ؘ੉ఠ ࠙ࢳо ੑࢎ, ఋ׮ ؘ੉ఠ౱ ѐੋ੸ੋ ҙब
 • بद ੤ࢤ, بद੄ ӝמ ١ بदী ҙब੉ ݆਺ • ׮নೠ بदܳ ҃೷ೞח Ѫਸ જইೣ • ࢲ਎݅ఀ Ѣ؀ೠ بदח ଺ইࠁӝ ൨ٜ׮ ఋ׮ؘ੉ఠ౱ • ର۝ ਍৔ ബਯച • ࣻਃ ৘ஏ, ର۝ ੤ߓ஖ ঌҊ્ܻ • ࢲ૚ ੗زച
  2. 1 ъթҳ 2 ઺ҳ 3 ࢲୡҳ 4 ৔١ನҳ 5 ઙ۽ҳ

    6 ਊ࢑ҳ 7 ݃ನҳ 8 ࣠౵ҳ 9 ࢿزҳ 10 ࢲ؀ޙҳ 11 ҙঈҳ 12 ъࢲҳ 0.0147 13 ز੘ҳ 0.0135 14 ࢿ࠘ҳ 0.0123 15 ҳ۽ҳ 0.0106 ইஜ 8:00 ب଱૑ ࣽਤ (%) ইஜীࢎۈٜ੉ೱೞחҔ
  3. ইஜ 8:00 ب଱૑ ࣽਤ (%) ইஜীࢎۈٜ੉ೱೞחҔ ب଱૑ ࢚ਤ 5ѐ ҳח

    ੹୓ ੉ز੄ ݻ %ܳ ର૑ೡөਃ? ࢲ਎੄ ҳח ୨ 25ѐ ੑפ׮. 1 ъթҳ 2 ઺ҳ 3 ࢲୡҳ 4 ৔١ನҳ 5 ઙ۽ҳ 6 ਊ࢑ҳ 7 ݃ನҳ 8 ࣠౵ҳ 9 ࢿزҳ 10 ࢲ؀ޙҳ 11 ҙঈҳ 12 ъࢲҳ 0.0147 13 ز੘ҳ 0.0135 14 ࢿ࠘ҳ 0.0123 15 ҳ۽ҳ 0.0106
  4. ইஜ 8:00 ب଱૑ ࣽਤ (%) 24% 76% ইஜীࢎۈٜ੉ೱೞחҔ 1 ъթҳ

    0.3041 2 ઺ҳ 0.1353 3 ࢲୡҳ 0.1324 4 ৔١ನҳ 0.0874 5 ઙ۽ҳ 0.0784 6 ਊ࢑ҳ 0.0657 7 ݃ನҳ 0.0369 8 ࣠౵ҳ 0.0318 9 ࢿزҳ 0.0282 10 ࢲ؀ޙҳ 0.0275 11 ҙঈҳ 0.0213 12 ъࢲҳ 0.0147 13 ز੘ҳ 0.0135 14 ࢿ࠘ҳ 0.0123 15 ҳ۽ҳ 0.0106
  5. ࢎۈٜ਷য٣ࢲೞܖܳ݃ޖܻೡө ঠр 23:00 ഐ୹૑ ࣽਤ (%) 1 ъթҳ 2 ઺ҳ

    3 ࢲୡҳ 4 ઙ۽ҳ 5 ਊ࢑ҳ 6 ݃ನҳ 7 ৔١ನҳ 8 ࢿزҳ 9 ࣠౵ҳ 10 ࢲ؀ޙҳ 11 ҙঈҳ 0.0109 12 ز੘ҳ 0.0101 13 ҟ૓ҳ 0.0079 14 ҳ۽ҳ 0.0065 15 ъࢲҳ 0.0064 ഐ୹૑ ࢚ਤ 5ѐ ҳח ੹୓ ੉ز੄ ݻ %ܳ ର૑ೡөਃ?
  6. ࢎۈٜ਷য٣ࢲೞܖܳ݃ޖܻೡө ঠр 23:00 ഐ୹૑ ࣽਤ (%) 1 ъթҳ 0.4241 2

    ઺ҳ 0.1248 3 ࢲୡҳ 0.0982 4 ઙ۽ҳ 0.0791 5 ਊ࢑ҳ 0.0783 6 ݃ನҳ 0.052 7 ৔١ನҳ 0.0435 8 ࢿزҳ 0.0216 9 ࣠౵ҳ 0.0159 10 ࢲ؀ޙҳ 0.0146 11 ҙঈҳ 0.0109 12 ز੘ҳ 0.0101 13 ҟ૓ҳ 0.0079 14 ҳ۽ҳ 0.0065 15 ъࢲҳ 0.0064 20% 80%
  7. ࢎۈٜ਷য٣ࢲೞܖܳ݃ޖܻೡө ঠр 23:00 ഐ୹૑ ࣽਤ (%) 1 ъթҳ 0.1897 2

    ࢲୡҳ 0.1086 3 ਊ࢑ҳ 0.0817 4 ҙঈҳ 0.0741 5 ࣠౵ҳ 0.0694 6 ࢿزҳ 0.0694 7 ݃ನҳ 0.0667 8 ز੘ҳ 0.0607 9 ৔١ನҳ 0.0531 10 ҟ૓ҳ 0.0431 11 ઺ҳ 0.0406 12 ࢿ࠘ҳ 0.0398 13 ࢲ؀ޙҳ 0.0379 14 ਷ಣҳ 0.0339 15 ઙ۽ҳ 0.0312 ঠр 23:00 ب଱૑ ࣽਤ (%) 1 ъթҳ 0.4241 2 ઺ҳ 0.1248 3 ࢲୡҳ 0.0982 4 ઙ۽ҳ 0.0791 5 ਊ࢑ҳ 0.0783 6 ݃ನҳ 0.052 7 ৔١ನҳ 0.0435 8 ࢿزҳ 0.0216 9 ࣠౵ҳ 0.0159 10 ࢲ؀ޙҳ 0.0146 11 ҙঈҳ 0.0109 12 ز੘ҳ 0.0101 13 ҟ૓ҳ 0.0079 14 ҳ۽ҳ 0.0065 15 ъࢲҳ 0.0064
  8. ݽ࠽ܻ౭ؘ੉ఠ࠙ࢳоח઱۽যڃؘ੉ఠܳࠁաਃ Origin - Destination data • ౠ੿ दр زউ যו

    ૑৉੄ ࢎۈٜ੉ য٣۽ ੉زೞӡ ਗೞחо? • ࠂ੟ೠ بद ࣘ ੉ز ಁఢী ؀ೠ Ѫ
  9. ݽ࠽ܻ౭ؘ੉ఠ࠙ࢳоח઱۽যڃؘ੉ఠܳࠁաਃ Origin - Destination data • ؘ੉ఠࣁ౟ ৘द id created_at_kr

    status origin_lng origin_lat origin_gu destination_lng destination_lat dest_si dest_gu 1 2019-01-30T00:25:20 CANCELED 127.0414496 37.51002284 ъթҳ 127.1909267 37.56209539 ҃ӝ ೞթद 2 2019-011-27T15:37:40 ACCEPTED 127.0563954 37.54280202 ࢿزҳ 126.8721171 37.45949251 ҃ӝ ҟݺद 3 2019-09-16T17:52:19 ARRIVED_AT 127.0613762 37.51048997 ъթҳ 127.1375613 37.59445381 ҃ӝ ҳܻद 4 2019-04-22T19:43:07 PICKED_UP 127.1105661 37.51265727 ࣠౵ҳ 126.9714429 37.40572274 ҃ӝ উনद 5 2019-07-07T21:33:56 RIDING 127.0127873 37.49320677 ࢲୡҳ 127.0709374 37.27701722 ҃ӝ ਊੋद 6 2019-10-30T21:35:33 DROPPED_OFF 127.0018494 37.58132881 ࢿزҳ 127.1243009 37.32402575 ҃ӝ ౵઱द 7 2019-09-27T21:41:36 DISPATCHING 127.0730746 37.5113235 ࣠౵ҳ 126.983202 37.39739958 ҃ӝ ੄৴द 8 2019-12-24T22:53:23 CANCELED 126.9696439 37.5635487 ઺ҳ 126.9936623 37.43394209 ҃ӝ җୌद ഐ୹૑ / ਤ҃ب ب଱૑ / ਤ҃ب दр
  10. ݽ࠽ܻ౭ؘ੉ఠ࠙ࢳоח઱۽যڃؘ੉ఠܳࠁաਃ Origin - Destination data • ؘ੉ఠࣁ౟ ৘द id created_at_kr

    status origin_lng origin_lat origin_gu destination_lng destination_lat dest_si dest_gu 1 2019-01-30T00:25:20 CANCELED 127.0414496 37.51002284 ъթҳ 127.1909267 37.56209539 ҃ӝ ೞթद 2 2019-011-27T15:37:40 ACCEPTED 127.0563954 37.54280202 ࢿزҳ 126.8721171 37.45949251 ҃ӝ ҟݺद 3 2019-09-16T17:52:19 ARRIVED_AT 127.0613762 37.51048997 ъթҳ 127.1375613 37.59445381 ҃ӝ ҳܻद 4 2019-04-22T19:43:07 PICKED_UP 127.1105661 37.51265727 ࣠౵ҳ 126.9714429 37.40572274 ҃ӝ উনद 5 2019-07-07T21:33:56 RIDING 127.0127873 37.49320677 ࢲୡҳ 127.0709374 37.27701722 ҃ӝ ਊੋद 6 2019-10-30T21:35:33 DROPPED_OFF 127.0018494 37.58132881 ࢿزҳ 127.1243009 37.32402575 ҃ӝ ౵઱द 7 2019-09-27T21:41:36 DISPATCHING 127.0730746 37.5113235 ࣠౵ҳ 126.983202 37.39739958 ҃ӝ ੄৴द 8 2019-12-24T22:53:23 CANCELED 126.9696439 37.5635487 ઺ҳ 126.9936623 37.43394209 ҃ӝ җୌद ഐ୹ ஂࣗ ٘ۄ੉ߡ ഐ୹ ࣻۅ ٘ۄ੉ߡ ب଱ थё ఑थ ਍೯ थё ೞର ؀ӝ ࢚క 다양한 상태 값을 가짐
  11. ର۝ഐ୹ ର۝ب଱ ݾ੸૑୹ߊ ݾ੸૑ب଱ Icons made by Freepik from www.flaticon.com

    is licensed by CC 3.0 BY ఋ׮҃೷ৈ੿ 데이터로 사용자 경험을 개선한 이야기
  12. ETA ۆ? • Estimated time of arrival ( ৘࢚ ب଱

    दр ) • ର۝ਸ ഐ୹ೞҊ Ҋёীѱ ب଱ೞӝө૑ ݽ࠽ܻ౭ী.-੸ਊೞӝ
  13. ٘ۄ੉ߡী ٮܲ ಞରо ௼׮ &5"ҙ۲ؘ੉ఠఐ࢝ ৘࢚ ب଱ दрࠁ׮ ןח ҃ೱࢿ

    = SUM(੉੹ ۄ੉٘ Ѥٜ੄ पઁ ࣗਃ दр) / SUM(੉੹ ۄ੉٘ Ѥٜ੄ ৘࢚ ࣗਃ दр)
  14. ݽ࠽ܻ౭ী.-੸ਊೞӝ ಣо ૑಴ ੤੿੄ • ੌ߈੸ਵ۽ ॳ੉ח MSE ࣚप ೣࣻ۽

    ೟णೠ׮ݶ • Ҋёীѱ ןח Ѫҗ ࡈܻ য়ח Ѫਸ ڙэ੉ ੋध
  15. ݽ࠽ܻ౭ী.-੸ਊೞӝ ಣо ૑಴ ੤੿੄ • Ҋё ҃೷ী ؊ աࢂ ৔ೱਸ

    ઱ח Ѫ਷ ןѱ য়ח Ѫ • ٮۄࢲ ןѱ য়ח Ѫী ಕօ౭ܳ ؊ ઱ب۾ • ࣚप ೣࣻܳ ੤੿੄ • ࠺؀ட੸ ਤ೷ (Asymmetrical risk) • ੌ߈੸ਵ۽ ॳ੉ח MSE ࣚप ೣࣻ۽ ೟णೠ׮ݶ • Ҋёীѱ ןח Ѫҗ ࡈܻ য়ח Ѫਸ ڙэ੉ ੋध
  16. ݽ࠽ܻ౭ী.-੸ਊೞӝ ಣо ૑಴ ੤੿੄ weight = 0.4
 residual = y_true

    - y_predict
 grad = np.where(residual<0, -2.0*residual, -2.0*weight*residual) য়ରо 0ࠁ׮ ௾ ҃਋৬ ੘਷ ҃਋ী ಁօ౭ܳ ׮ܰѱ ઱ח ೣࣻܳ ੿੄ Params = {
 ‘objective’ : custom_asymmetric_objective } ݾ੸ೣࣻܳ ੤੿੄ೠ ೣࣻ۽ ૑੿
  17. ݽ؛݂ ױ҅ীࢲ ו՛ Ѫ • അपী ੸ਊೡ ٸח ׮নೠ ૑಴ٜਸ

    ೣԋ Ҋ޹೧ঠೠ׮ • ఋ׮ ࢎ۹ • ੿ഛب ૑಴ : MAE • ࠺૑פझ ૑಴ : ૑п ࠺ਯਸ ծ୶ח Ѫ / Ҋё ҃೷ ೱ࢚ ੿ഛب
 ૑಴ ࠺૑פझ ҙ੼ 
 ૑಴ ݽ࠽ܻ౭ী.-੸ਊೞӝ
  18. ߓನ ળ࠺ ױ҅ • ݽ؛݂਷ ৮߷ೞѱ ՘լ׮ • ੉ઁ ࣁ࢚ਵ۽

    ղࠁղ੗ ݽ࠽ܻ౭ী.-੸ਊೞӝ ࠁ੿ ݽ؛ ӝઓ ETA ࠁ੿ ETA
  19. ঌ ࣻ হח ਗੋ ݽ࠽ܻ౭ী.-੸ਊೞӝ ࠗपೠ ۽Ӧ • पઁ ч

    • ৘ஏ ч • যڃ ࠙ನ੄ ೖ୛ٜ੉ ٜয৳ח૑ী ؀ೠ ۽Ӧਸ ೞ૑ ঋও׮ • ٮۄࢲ ݽ؛੉ ৵ Ӓۧѱ ৘ஏ೮ח૑ ঌ ࣻо হ׮ ࠁ੿ ݽ؛ ӝઓ ETA ࠁ੿ ETA
  20. ੌױ ܀ߔ ݽ࠽ܻ౭ী.-੸ਊೞӝ • ٘ۄ੉۠ - ࢲ࠺झীח ੸ਊೞ૑ ঋҊ पઁ

    ജ҃ীࢲ పझ౟݅ ૓೯ ࠁ੿ ݽ؛ ӝઓ ETA ࠁ੿ ETA ӝઓ ETA
  21. ੌױ ܀ߔ ݽ࠽ܻ౭ী.-੸ਊೞӝ • ٘ۄ੉۠ - ࢲ࠺झীח ੸ਊೞ૑ ঋҊ पઁ

    ജ҃ীࢲ పझ౟݅ ૓೯ • ࠻௪ܻ৬ క࠶۽۽ ݽפఠ݂ بҳܳ ݃۲ೞҊ ۽Ӓܳ ऺইࠆ ࠁ੿ ݽ؛ ӝઓ ETA ࠁ੿ ETA ӝઓ ETA
  22. য়ܨо լ؍ ਗੋ ݽ࠽ܻ౭ী.-੸ਊೞӝ • ઱ਃೠ ೖ୛ ઺ ೞաо ࢲ۽

    ׮ܲ ࠙ನܳ о૑Ҋ ੓঻׮ (ࢲ۽ ׮ܲ ױਤ ޙઁ) Train ױ҅
  23. য়ܨо լ؍ ਗੋ ݽ࠽ܻ౭ী.-੸ਊೞӝ • ઱ਃೠ ೖ୛ ઺ ೞաо ࢲ۽

    ׮ܲ ࠙ನܳ о૑Ҋ ੓঻׮ (ࢲ۽ ׮ܲ ױਤ ޙઁ) Train ױ҅ Test ױ҅
  24. য়ܨо լ؍ ਗੋ ݽ࠽ܻ౭ী.-੸ਊೞӝ • ઱ਃೠ ೖ୛ ઺ ೞաо ࢲ۽

    ׮ܲ ࠙ನܳ о૓׮ • ߓನ ੉੹ী ݽפఠ݂ਸ ؀࠺೮׮ݶ Әߑ ೧Ѿ೮ਸ ޙઁ • ѐߊ੗৬ ഈস җ੿ীࢲ ੄ࢎ ࣗా ޙઁ Train ױ҅ Test ױ҅
  25. ઱য૓
 ؘ੉ఠࣇ ࠙ࢳ ߂ ݽ؛݂ ઱য૓ 
 ಣо ૑಴
 (MAE,RMSE)

    ࠙ࢳ ߂ ݽ؛݂ ࢜۽ ੿੄ೞח 
 ಣо૑಴ ࢜۽ ݅٘ח
 ؘ੉ఠࣇ ߓನ ߂ 
 ݽפఠ݂ ঌ؍ Ѫ ೧ঠ೮؍ Ѫ ݽ࠽ܻ౭ী.-੸ਊೞӝ
  26. • Linear Programming • Integer Programming • Routing • Packing

    • Network Flows • Assignment • Scheduling Operation research
  27. • Linear Programming • Integer Programming • Routing • Packing

    • Network Flows • Assignment • Scheduling Operation research
  28. ֢٘ /PEF ੿੄ೞӝ -20 -20 -20 -10 -50 -40 90

    30 • ֢٘ח ౠ੿ೠ ૑৉੉ۄҊ ೡ ࣻ ੓׮ • ҕә ֢٘ : թח ର۝੉ ߊࢤೞח ૑৉ • ࣻਃ ֢٘ : ࣻਃо ߊࢤೞח ૑৉
  29. ݂௼ -JOL ੿੄ೞӝ • ݂௼ח ֢٘৬ ֢٘ܳ োѾೠ׮ • ݂௼ܳ

    ా೧ ੉زਸ ೡ ҃਋ীח ࠺ਊ੉ ߊࢤ • അपীࢲח Ѣܻ ഑਷ ࣗਃ दр 30 5 10 12 12 20 5 10
  30. -20 -20 -20 -10 -50 -40 90 30 • ਋ܻо

    ૒੽ ੿੄ೠ ֎౟ਕ௼ 30 5 10 12 12 20 5 10 ֎౟ਕ௼
  31. ݽ࠽ܻ౭৬୭੸ച ݂௼੄ ࠺ਊਸ ੿੄ п ֢٘੄ ҕәҗ ࣻਃܳ ੿੄ ࣛߡܳ

    ࢶ঱ೞҊ, ֎౟ਕ௼ܳ ҳࢿ https://developers.google.com/optimization/flow/mincostflow
  32. -20 -10 -50 -40 90 30 ݽ࠽ܻ౭৬୭੸ച • ࠺ਊਸ ୭ࣗചೞݶࢲ

    ҕә ֢٘ীࢲ ࣻਃ ֢٘۽ ੉زदఃח ߑध • Min cost flow problem
  33. ݽ࠽ܻ౭৬୭੸ച 0 4 5 3 1 2 20 10 50

    40 50 • ࣛߡ(Solver)о ળ ׹
  34. ݽ࠽ܻ౭৬୭੸ച 0 4 5 3 1 2 20 10 50

    40 50 • ࣛߡ(Solver)о ળ ׹ • അपীࢲ Ҋ۰೧ঠೡ ࢎ೦ٜ • ੉ز ઺ী ௒੉ ੟൧ ࣻب ੓׮ • ҕә਷ োࣘ੸ਵ۽ ߊࢤೠ׮
  35. .-ݽ؛ਸ݅٘חѪҗपઁജ҃ীࢲ࠺झೞחѪ ݽ؛݂ Ӓ ੉റܳ ࢤпೞӝ • ߓನ റীب উ੿੸ਵ۽ ҙܻೡ

    ࣻ ੓ח ജ҃ਸ ݃۲೧فח Ѫ੉ ઺ਃ (ݽפఠ݂) ࢚ടী ݏח ݾ੸ೣࣻ ଺ӝ • ೙ਃೞ׮ݶ ࣚप ೣࣻܳ ੤੿੄೧ঠೡ ࣻب ੓׮ • Objective is subjective !
  36. بݫੋীҙೠ੉ঠӝ ־ҳաীѱ ੓ਸ п੗੄ بݫੋ • ೧׼ بݫੋীࢲח ݠन۞׬, ٩۞׬੉

    ੹ࠗо ইק ࣻ ੓Ҋ • ഑਷ ׮ܲ ӝߨٜҗ ೣԋ ॳৈঠ ࡄਸ ߊೡ ٸо ੓ח Ѫ э׮
  37. ౱೒ۨ੉੄઺ਃࢿ ഒ੗ࢲ ݽٚ Ѫਸ ೡ ࣻח হӝী • ؘ੉ఠূ૑פয +

    ؘ੉ఠ࠙ࢳо + ѐߊ੗ + ਍৔౱ • ׮নೠ ૒ҵ੄ ࢎۈٜҗ ഈসਵ۽ ޙઁܳ ೧Ѿ೧աоҊ ੓਺ ઱߸ਵ۽ࠗఠ ݆੉ ߓ਋ӝ • જ਷ ࢎࣻ, ׮ܲ ૒ҵ੄ زܐ
  38. ଻ਊ ؘ੉ఠӒܛ • ఋ׮ؘ੉ఠ౱ • ؘ੉ఠࢎ੉঱झ౱ • оѺ੹ۚ౱ • ؘ੉ఠূ૑פয݂౱

    • URL : bit.ly/ؘ੉ఠঠ֥੗॑஠଻ਊ ؘ੉ఠ౱ • ؘ੉ఠূ૑פয • ؘ੉ఠࢎ੉঱౭झ౟ • URL : https://tadacareer.vcnc.co.kr/

More Decks by SOCAR

Other Decks in Programming