Upgrade to Pro — share decks privately, control downloads, hide ads and more …

모빌리티데이터팀 신입 데이터 분석가의 1년 회고

SOCAR
October 19, 2019

모빌리티데이터팀 신입 데이터 분석가의 1년 회고

데이터야놀자 2019에서 발표한 자료입니다

SOCAR

October 19, 2019
Tweet

More Decks by SOCAR

Other Decks in Programming

Transcript

  1. ࣗѐ ӂਮജ • ҃৔೟ + ஹೊఠҕ೟ ੹ҕ੉ ݏ૑ ঋই ੉ز

    • झఋ౟সҗ ؘ੉ఠী ҙब੉ ݆਺ • 2019.01 ॑஠ ؘ੉ఠ ࠙ࢳо ੑࢎ, ఋ׮ ؘ੉ఠ౱ ѐੋ੸ੋ ҙब
 • بद ੤ࢤ, بद੄ ӝמ ١ بदী ҙब੉ ݆਺ • ׮নೠ بदܳ ҃೷ೞח Ѫਸ જইೣ • ࢲ਎݅ఀ Ѣ؀ೠ بदח ଺ইࠁӝ ൨ٜ׮ ఋ׮ؘ੉ఠ౱ • ର۝ ਍৔ ബਯച • ࣻਃ ৘ஏ, ର۝ ੤ߓ஖ ঌҊ્ܻ • ࢲ૚ ੗زച
  2. 1 ъթҳ 2 ઺ҳ 3 ࢲୡҳ 4 ৔١ನҳ 5 ઙ۽ҳ

    6 ਊ࢑ҳ 7 ݃ನҳ 8 ࣠౵ҳ 9 ࢿزҳ 10 ࢲ؀ޙҳ 11 ҙঈҳ 12 ъࢲҳ 0.0147 13 ز੘ҳ 0.0135 14 ࢿ࠘ҳ 0.0123 15 ҳ۽ҳ 0.0106 ইஜ 8:00 ب଱૑ ࣽਤ (%) ইஜীࢎۈٜ੉ೱೞחҔ
  3. ইஜ 8:00 ب଱૑ ࣽਤ (%) ইஜীࢎۈٜ੉ೱೞחҔ ب଱૑ ࢚ਤ 5ѐ ҳח

    ੹୓ ੉ز੄ ݻ %ܳ ର૑ೡөਃ? ࢲ਎੄ ҳח ୨ 25ѐ ੑפ׮. 1 ъթҳ 2 ઺ҳ 3 ࢲୡҳ 4 ৔١ನҳ 5 ઙ۽ҳ 6 ਊ࢑ҳ 7 ݃ನҳ 8 ࣠౵ҳ 9 ࢿزҳ 10 ࢲ؀ޙҳ 11 ҙঈҳ 12 ъࢲҳ 0.0147 13 ز੘ҳ 0.0135 14 ࢿ࠘ҳ 0.0123 15 ҳ۽ҳ 0.0106
  4. ইஜ 8:00 ب଱૑ ࣽਤ (%) 24% 76% ইஜীࢎۈٜ੉ೱೞחҔ 1 ъթҳ

    0.3041 2 ઺ҳ 0.1353 3 ࢲୡҳ 0.1324 4 ৔١ನҳ 0.0874 5 ઙ۽ҳ 0.0784 6 ਊ࢑ҳ 0.0657 7 ݃ನҳ 0.0369 8 ࣠౵ҳ 0.0318 9 ࢿزҳ 0.0282 10 ࢲ؀ޙҳ 0.0275 11 ҙঈҳ 0.0213 12 ъࢲҳ 0.0147 13 ز੘ҳ 0.0135 14 ࢿ࠘ҳ 0.0123 15 ҳ۽ҳ 0.0106
  5. ࢎۈٜ਷য٣ࢲೞܖܳ݃ޖܻೡө ঠр 23:00 ഐ୹૑ ࣽਤ (%) 1 ъթҳ 2 ઺ҳ

    3 ࢲୡҳ 4 ઙ۽ҳ 5 ਊ࢑ҳ 6 ݃ನҳ 7 ৔١ನҳ 8 ࢿزҳ 9 ࣠౵ҳ 10 ࢲ؀ޙҳ 11 ҙঈҳ 0.0109 12 ز੘ҳ 0.0101 13 ҟ૓ҳ 0.0079 14 ҳ۽ҳ 0.0065 15 ъࢲҳ 0.0064 ഐ୹૑ ࢚ਤ 5ѐ ҳח ੹୓ ੉ز੄ ݻ %ܳ ର૑ೡөਃ?
  6. ࢎۈٜ਷য٣ࢲೞܖܳ݃ޖܻೡө ঠр 23:00 ഐ୹૑ ࣽਤ (%) 1 ъթҳ 0.4241 2

    ઺ҳ 0.1248 3 ࢲୡҳ 0.0982 4 ઙ۽ҳ 0.0791 5 ਊ࢑ҳ 0.0783 6 ݃ನҳ 0.052 7 ৔١ನҳ 0.0435 8 ࢿزҳ 0.0216 9 ࣠౵ҳ 0.0159 10 ࢲ؀ޙҳ 0.0146 11 ҙঈҳ 0.0109 12 ز੘ҳ 0.0101 13 ҟ૓ҳ 0.0079 14 ҳ۽ҳ 0.0065 15 ъࢲҳ 0.0064 20% 80%
  7. ࢎۈٜ਷য٣ࢲೞܖܳ݃ޖܻೡө ঠр 23:00 ഐ୹૑ ࣽਤ (%) 1 ъթҳ 0.1897 2

    ࢲୡҳ 0.1086 3 ਊ࢑ҳ 0.0817 4 ҙঈҳ 0.0741 5 ࣠౵ҳ 0.0694 6 ࢿزҳ 0.0694 7 ݃ನҳ 0.0667 8 ز੘ҳ 0.0607 9 ৔١ನҳ 0.0531 10 ҟ૓ҳ 0.0431 11 ઺ҳ 0.0406 12 ࢿ࠘ҳ 0.0398 13 ࢲ؀ޙҳ 0.0379 14 ਷ಣҳ 0.0339 15 ઙ۽ҳ 0.0312 ঠр 23:00 ب଱૑ ࣽਤ (%) 1 ъթҳ 0.4241 2 ઺ҳ 0.1248 3 ࢲୡҳ 0.0982 4 ઙ۽ҳ 0.0791 5 ਊ࢑ҳ 0.0783 6 ݃ನҳ 0.052 7 ৔١ನҳ 0.0435 8 ࢿزҳ 0.0216 9 ࣠౵ҳ 0.0159 10 ࢲ؀ޙҳ 0.0146 11 ҙঈҳ 0.0109 12 ز੘ҳ 0.0101 13 ҟ૓ҳ 0.0079 14 ҳ۽ҳ 0.0065 15 ъࢲҳ 0.0064
  8. ݽ࠽ܻ౭ؘ੉ఠ࠙ࢳоח઱۽যڃؘ੉ఠܳࠁաਃ Origin - Destination data • ౠ੿ दр زউ যו

    ૑৉੄ ࢎۈٜ੉ য٣۽ ੉زೞӡ ਗೞחо? • ࠂ੟ೠ بद ࣘ ੉ز ಁఢী ؀ೠ Ѫ
  9. ݽ࠽ܻ౭ؘ੉ఠ࠙ࢳоח઱۽যڃؘ੉ఠܳࠁաਃ Origin - Destination data • ؘ੉ఠࣁ౟ ৘द id created_at_kr

    status origin_lng origin_lat origin_gu destination_lng destination_lat dest_si dest_gu 1 2019-01-30T00:25:20 CANCELED 127.0414496 37.51002284 ъթҳ 127.1909267 37.56209539 ҃ӝ ೞթद 2 2019-011-27T15:37:40 ACCEPTED 127.0563954 37.54280202 ࢿزҳ 126.8721171 37.45949251 ҃ӝ ҟݺद 3 2019-09-16T17:52:19 ARRIVED_AT 127.0613762 37.51048997 ъթҳ 127.1375613 37.59445381 ҃ӝ ҳܻद 4 2019-04-22T19:43:07 PICKED_UP 127.1105661 37.51265727 ࣠౵ҳ 126.9714429 37.40572274 ҃ӝ উনद 5 2019-07-07T21:33:56 RIDING 127.0127873 37.49320677 ࢲୡҳ 127.0709374 37.27701722 ҃ӝ ਊੋद 6 2019-10-30T21:35:33 DROPPED_OFF 127.0018494 37.58132881 ࢿزҳ 127.1243009 37.32402575 ҃ӝ ౵઱द 7 2019-09-27T21:41:36 DISPATCHING 127.0730746 37.5113235 ࣠౵ҳ 126.983202 37.39739958 ҃ӝ ੄৴द 8 2019-12-24T22:53:23 CANCELED 126.9696439 37.5635487 ઺ҳ 126.9936623 37.43394209 ҃ӝ җୌद ഐ୹૑ / ਤ҃ب ب଱૑ / ਤ҃ب दр
  10. ݽ࠽ܻ౭ؘ੉ఠ࠙ࢳоח઱۽যڃؘ੉ఠܳࠁաਃ Origin - Destination data • ؘ੉ఠࣁ౟ ৘द id created_at_kr

    status origin_lng origin_lat origin_gu destination_lng destination_lat dest_si dest_gu 1 2019-01-30T00:25:20 CANCELED 127.0414496 37.51002284 ъթҳ 127.1909267 37.56209539 ҃ӝ ೞթद 2 2019-011-27T15:37:40 ACCEPTED 127.0563954 37.54280202 ࢿزҳ 126.8721171 37.45949251 ҃ӝ ҟݺद 3 2019-09-16T17:52:19 ARRIVED_AT 127.0613762 37.51048997 ъթҳ 127.1375613 37.59445381 ҃ӝ ҳܻद 4 2019-04-22T19:43:07 PICKED_UP 127.1105661 37.51265727 ࣠౵ҳ 126.9714429 37.40572274 ҃ӝ উনद 5 2019-07-07T21:33:56 RIDING 127.0127873 37.49320677 ࢲୡҳ 127.0709374 37.27701722 ҃ӝ ਊੋद 6 2019-10-30T21:35:33 DROPPED_OFF 127.0018494 37.58132881 ࢿزҳ 127.1243009 37.32402575 ҃ӝ ౵઱द 7 2019-09-27T21:41:36 DISPATCHING 127.0730746 37.5113235 ࣠౵ҳ 126.983202 37.39739958 ҃ӝ ੄৴द 8 2019-12-24T22:53:23 CANCELED 126.9696439 37.5635487 ઺ҳ 126.9936623 37.43394209 ҃ӝ җୌद ഐ୹ ஂࣗ ٘ۄ੉ߡ ഐ୹ ࣻۅ ٘ۄ੉ߡ ب଱ थё ఑थ ਍೯ थё ೞର ؀ӝ ࢚క 다양한 상태 값을 가짐
  11. ର۝ഐ୹ ର۝ب଱ ݾ੸૑୹ߊ ݾ੸૑ب଱ Icons made by Freepik from www.flaticon.com

    is licensed by CC 3.0 BY ఋ׮҃೷ৈ੿ 데이터로 사용자 경험을 개선한 이야기
  12. ETA ۆ? • Estimated time of arrival ( ৘࢚ ب଱

    दр ) • ର۝ਸ ഐ୹ೞҊ Ҋёীѱ ب଱ೞӝө૑ ݽ࠽ܻ౭ী.-੸ਊೞӝ
  13. ٘ۄ੉ߡী ٮܲ ಞରо ௼׮ &5"ҙ۲ؘ੉ఠఐ࢝ ৘࢚ ب଱ दрࠁ׮ ןח ҃ೱࢿ

    = SUM(੉੹ ۄ੉٘ Ѥٜ੄ पઁ ࣗਃ दр) / SUM(੉੹ ۄ੉٘ Ѥٜ੄ ৘࢚ ࣗਃ दр)
  14. ݽ࠽ܻ౭ী.-੸ਊೞӝ ಣо ૑಴ ੤੿੄ • ੌ߈੸ਵ۽ ॳ੉ח MSE ࣚप ೣࣻ۽

    ೟णೠ׮ݶ • Ҋёীѱ ןח Ѫҗ ࡈܻ য়ח Ѫਸ ڙэ੉ ੋध
  15. ݽ࠽ܻ౭ী.-੸ਊೞӝ ಣо ૑಴ ੤੿੄ • Ҋё ҃೷ী ؊ աࢂ ৔ೱਸ

    ઱ח Ѫ਷ ןѱ য়ח Ѫ • ٮۄࢲ ןѱ য়ח Ѫী ಕօ౭ܳ ؊ ઱ب۾ • ࣚप ೣࣻܳ ੤੿੄ • ࠺؀ட੸ ਤ೷ (Asymmetrical risk) • ੌ߈੸ਵ۽ ॳ੉ח MSE ࣚप ೣࣻ۽ ೟णೠ׮ݶ • Ҋёীѱ ןח Ѫҗ ࡈܻ য়ח Ѫਸ ڙэ੉ ੋध
  16. ݽ࠽ܻ౭ী.-੸ਊೞӝ ಣо ૑಴ ੤੿੄ weight = 0.4
 residual = y_true

    - y_predict
 grad = np.where(residual<0, -2.0*residual, -2.0*weight*residual) য়ରо 0ࠁ׮ ௾ ҃਋৬ ੘਷ ҃਋ী ಁօ౭ܳ ׮ܰѱ ઱ח ೣࣻܳ ੿੄ Params = {
 ‘objective’ : custom_asymmetric_objective } ݾ੸ೣࣻܳ ੤੿੄ೠ ೣࣻ۽ ૑੿
  17. ݽ؛݂ ױ҅ীࢲ ו՛ Ѫ • അपী ੸ਊೡ ٸח ׮নೠ ૑಴ٜਸ

    ೣԋ Ҋ޹೧ঠೠ׮ • ఋ׮ ࢎ۹ • ੿ഛب ૑಴ : MAE • ࠺૑פझ ૑಴ : ૑п ࠺ਯਸ ծ୶ח Ѫ / Ҋё ҃೷ ೱ࢚ ੿ഛب
 ૑಴ ࠺૑פझ ҙ੼ 
 ૑಴ ݽ࠽ܻ౭ী.-੸ਊೞӝ
  18. ߓನ ળ࠺ ױ҅ • ݽ؛݂਷ ৮߷ೞѱ ՘լ׮ • ੉ઁ ࣁ࢚ਵ۽

    ղࠁղ੗ ݽ࠽ܻ౭ী.-੸ਊೞӝ ࠁ੿ ݽ؛ ӝઓ ETA ࠁ੿ ETA
  19. ঌ ࣻ হח ਗੋ ݽ࠽ܻ౭ী.-੸ਊೞӝ ࠗपೠ ۽Ӧ • पઁ ч

    • ৘ஏ ч • যڃ ࠙ನ੄ ೖ୛ٜ੉ ٜয৳ח૑ী ؀ೠ ۽Ӧਸ ೞ૑ ঋও׮ • ٮۄࢲ ݽ؛੉ ৵ Ӓۧѱ ৘ஏ೮ח૑ ঌ ࣻо হ׮ ࠁ੿ ݽ؛ ӝઓ ETA ࠁ੿ ETA
  20. ੌױ ܀ߔ ݽ࠽ܻ౭ী.-੸ਊೞӝ • ٘ۄ੉۠ - ࢲ࠺झীח ੸ਊೞ૑ ঋҊ पઁ

    ജ҃ীࢲ పझ౟݅ ૓೯ ࠁ੿ ݽ؛ ӝઓ ETA ࠁ੿ ETA ӝઓ ETA
  21. ੌױ ܀ߔ ݽ࠽ܻ౭ী.-੸ਊೞӝ • ٘ۄ੉۠ - ࢲ࠺झীח ੸ਊೞ૑ ঋҊ पઁ

    ജ҃ীࢲ పझ౟݅ ૓೯ • ࠻௪ܻ৬ క࠶۽۽ ݽפఠ݂ بҳܳ ݃۲ೞҊ ۽Ӓܳ ऺইࠆ ࠁ੿ ݽ؛ ӝઓ ETA ࠁ੿ ETA ӝઓ ETA
  22. য়ܨо լ؍ ਗੋ ݽ࠽ܻ౭ী.-੸ਊೞӝ • ઱ਃೠ ೖ୛ ઺ ೞաо ࢲ۽

    ׮ܲ ࠙ನܳ о૑Ҋ ੓঻׮ (ࢲ۽ ׮ܲ ױਤ ޙઁ) Train ױ҅
  23. য়ܨо լ؍ ਗੋ ݽ࠽ܻ౭ী.-੸ਊೞӝ • ઱ਃೠ ೖ୛ ઺ ೞաо ࢲ۽

    ׮ܲ ࠙ನܳ о૑Ҋ ੓঻׮ (ࢲ۽ ׮ܲ ױਤ ޙઁ) Train ױ҅ Test ױ҅
  24. য়ܨо լ؍ ਗੋ ݽ࠽ܻ౭ী.-੸ਊೞӝ • ઱ਃೠ ೖ୛ ઺ ೞաо ࢲ۽

    ׮ܲ ࠙ನܳ о૓׮ • ߓನ ੉੹ী ݽפఠ݂ਸ ؀࠺೮׮ݶ Әߑ ೧Ѿ೮ਸ ޙઁ • ѐߊ੗৬ ഈস җ੿ীࢲ ੄ࢎ ࣗా ޙઁ Train ױ҅ Test ױ҅
  25. ઱য૓
 ؘ੉ఠࣇ ࠙ࢳ ߂ ݽ؛݂ ઱য૓ 
 ಣо ૑಴
 (MAE,RMSE)

    ࠙ࢳ ߂ ݽ؛݂ ࢜۽ ੿੄ೞח 
 ಣо૑಴ ࢜۽ ݅٘ח
 ؘ੉ఠࣇ ߓನ ߂ 
 ݽפఠ݂ ঌ؍ Ѫ ೧ঠ೮؍ Ѫ ݽ࠽ܻ౭ী.-੸ਊೞӝ
  26. • Linear Programming • Integer Programming • Routing • Packing

    • Network Flows • Assignment • Scheduling Operation research
  27. • Linear Programming • Integer Programming • Routing • Packing

    • Network Flows • Assignment • Scheduling Operation research
  28. ֢٘ /PEF ੿੄ೞӝ -20 -20 -20 -10 -50 -40 90

    30 • ֢٘ח ౠ੿ೠ ૑৉੉ۄҊ ೡ ࣻ ੓׮ • ҕә ֢٘ : թח ର۝੉ ߊࢤೞח ૑৉ • ࣻਃ ֢٘ : ࣻਃо ߊࢤೞח ૑৉
  29. ݂௼ -JOL ੿੄ೞӝ • ݂௼ח ֢٘৬ ֢٘ܳ োѾೠ׮ • ݂௼ܳ

    ా೧ ੉زਸ ೡ ҃਋ীח ࠺ਊ੉ ߊࢤ • അपীࢲח Ѣܻ ഑਷ ࣗਃ दр 30 5 10 12 12 20 5 10
  30. -20 -20 -20 -10 -50 -40 90 30 • ਋ܻо

    ૒੽ ੿੄ೠ ֎౟ਕ௼ 30 5 10 12 12 20 5 10 ֎౟ਕ௼
  31. ݽ࠽ܻ౭৬୭੸ച ݂௼੄ ࠺ਊਸ ੿੄ п ֢٘੄ ҕәҗ ࣻਃܳ ੿੄ ࣛߡܳ

    ࢶ঱ೞҊ, ֎౟ਕ௼ܳ ҳࢿ https://developers.google.com/optimization/flow/mincostflow
  32. -20 -10 -50 -40 90 30 ݽ࠽ܻ౭৬୭੸ച • ࠺ਊਸ ୭ࣗചೞݶࢲ

    ҕә ֢٘ীࢲ ࣻਃ ֢٘۽ ੉زदఃח ߑध • Min cost flow problem
  33. ݽ࠽ܻ౭৬୭੸ച 0 4 5 3 1 2 20 10 50

    40 50 • ࣛߡ(Solver)о ળ ׹
  34. ݽ࠽ܻ౭৬୭੸ച 0 4 5 3 1 2 20 10 50

    40 50 • ࣛߡ(Solver)о ળ ׹ • അपীࢲ Ҋ۰೧ঠೡ ࢎ೦ٜ • ੉ز ઺ী ௒੉ ੟൧ ࣻب ੓׮ • ҕә਷ োࣘ੸ਵ۽ ߊࢤೠ׮
  35. .-ݽ؛ਸ݅٘חѪҗपઁജ҃ীࢲ࠺झೞחѪ ݽ؛݂ Ӓ ੉റܳ ࢤпೞӝ • ߓನ റীب উ੿੸ਵ۽ ҙܻೡ

    ࣻ ੓ח ജ҃ਸ ݃۲೧فח Ѫ੉ ઺ਃ (ݽפఠ݂) ࢚ടী ݏח ݾ੸ೣࣻ ଺ӝ • ೙ਃೞ׮ݶ ࣚप ೣࣻܳ ੤੿੄೧ঠೡ ࣻب ੓׮ • Objective is subjective !
  36. بݫੋীҙೠ੉ঠӝ ־ҳաীѱ ੓ਸ п੗੄ بݫੋ • ೧׼ بݫੋীࢲח ݠन۞׬, ٩۞׬੉

    ੹ࠗо ইק ࣻ ੓Ҋ • ഑਷ ׮ܲ ӝߨٜҗ ೣԋ ॳৈঠ ࡄਸ ߊೡ ٸо ੓ח Ѫ э׮
  37. ౱೒ۨ੉੄઺ਃࢿ ഒ੗ࢲ ݽٚ Ѫਸ ೡ ࣻח হӝী • ؘ੉ఠূ૑פয +

    ؘ੉ఠ࠙ࢳо + ѐߊ੗ + ਍৔౱ • ׮নೠ ૒ҵ੄ ࢎۈٜҗ ഈসਵ۽ ޙઁܳ ೧Ѿ೧աоҊ ੓਺ ઱߸ਵ۽ࠗఠ ݆੉ ߓ਋ӝ • જ਷ ࢎࣻ, ׮ܲ ૒ҵ੄ زܐ
  38. ଻ਊ ؘ੉ఠӒܛ • ఋ׮ؘ੉ఠ౱ • ؘ੉ఠࢎ੉঱झ౱ • оѺ੹ۚ౱ • ؘ੉ఠূ૑פয݂౱

    • URL : bit.ly/ؘ੉ఠঠ֥੗॑஠଻ਊ ؘ੉ఠ౱ • ؘ੉ఠূ૑פয • ؘ੉ఠࢎ੉঱౭झ౟ • URL : https://tadacareer.vcnc.co.kr/