Upgrade to Pro — share decks privately, control downloads, hide ads and more …

データ基盤品質向上のための一年

Toshiki Tsuchikawa
December 13, 2021
7.8k

 データ基盤品質向上のための一年

DataEngineeringStudy #11

Toshiki Tsuchikawa

December 13, 2021
Tweet

Transcript

  1. ᅵᕝ⛱⏕ 7VXFKLNDZD7RVKLNL  Ɣ ឡ▱┴ฟ㌟ Ɣ ᖺᮾᕤ኱᝟ሗ⌮ᕤᏛ㝔༞ Ɣ ᰴᘧ఍♫䝍䜲䝭䞊 ż

    '5( 'DWD5HOLDELOLW\(QJLQHHULQJ 䝏䞊䝮 ż 䝕䞊䝍ᇶ┙䛾㛤Ⓨ䞉ಖᏲ䞉㐠⏝ ż ศᯒᇶ┙䛾㛤Ⓨ䞉ಖᏲ䞉㐠⏝ Ɣ 7ZLWWHU#WYWJB  ⮬ᕫ⤂௓
  2.  ̐᭦᪂䝍䜲䝭䞁䜾䛾㐜ᘏ %LJ4XHU\䝴䞊䝄䞊䛛䜙᭦᪂䝍䜲䝭䞁䜾䛜㐜䜜䛶䛔䜛 䛸䛾ධಙ䛜ቑຍ 䛭䛾㒔ᗘ'5(䝏䞊䝮䛜༶ᑐᛂ ┘ど䛿䛔䛟䛴䛛ධ䜜䛶䛔䛯䛜䚸䜲䞁䝣䝷䜔䚸䝕䞊䝍䝟 䜲䝥䝷䜲䞁䛺䛹ከᒱ䛻䜟䛯䜛䝞䜾䛾Ⓨ⏕※䛜䛒䜚ᑐᛂ 䛻᫬㛫䛜ྲྀ䜙䜜䜛䜘䛖䛻 䝕䞊䝍ရ㉁䛾ၥ㢟Ⅼ ʍ᭦᪂㢖ᗘ┘ど6/26/$䛾ᑟධ

    㐺᫬ᛶ䜢6/,䛸䛧䛶ྛ䝕䞊䝍䝋䞊䝇䛤䛸䛻 6/$䜢ศᯒ ⪅䛸ᐃ⩏ 䜶䝷䞊䝞䝆䜵䝑䝖䜢タ䛡䜛䛣䛸䛷᪂つ㛤Ⓨ䛻᫬㛫䛜๭ 䛡䜛䜘䛖䛻䛺䜛 䝫䝇䝖䝰䞊䝔䝮䛷ᨵၿ䜒ᅇ䜛 %LJ4XHU\䛾᭱⤊᭦᪂᫬้䜢┘ど䛩䜛䛣䛸䛷 '5(䝏䞊 䝮䛜䜶䝷䞊䛻᪩䛟Ẽ䛵䛡䜛䜘䛖䛻䛺䜛 ̐ ̐ ̐ ̐
  3.  '5(䝏䞊䝮䛻䛚䛡䜛6/,6/26/$䛾ᐃ⩏ 6HUYLFH/HYHO,QGLFDWRU 䝕䞊䝍䝟䜲䝥䝷䜲䞁䛾㐺᫬ᛶ 䝕䞊䝍䝋䞊䝇䛾᭦᪂䛛䜙䛹 䛾䛟䜙䛔㐜䜜䛶㌿㏦ඛ䛷ᐇ⏝ྍ⬟䛻䛺䜛䛛 6/, 6/$ 6/2 6HUYLFH/HYHO$JUHHPHQW

    䝕䞊䝍䝋䞊䝇䛤䛸䛻%LJ4XHU\౑⏝⪅䛸⤖䜀䜜䛯㐺 ᫬ᛶ䛻㛵䛩䜛ዎ⣙ ◚䛳䛯ሙྜ䛿䝫䝇䝖䝰䞊䝔䝮䜢ᐇ᪋ ౛䝕䞊䝍䝋䞊䝇$䛿᪥䛾㐺᫬ᛶ䛷䛾㌿㏦ 6HUYLFH/HYHO2EMHFWLYH '5(䝏䞊䝮ෆ䛷Ỵᐃ䛥䜜䛯䝕䞊䝍䝋䞊䝇䛤䛸䛾㐺 ᫬ᛶ䛾┠ᶆ ౛䝕䞊䝍䝋䞊䝇$䛿KRXU䛾㐺᫬ᛶ䛷䛾㌿㏦
  4. 䜽䜶䝸䛾ಟṇ䛚 㢪䛔䛧䜎䛩 䛣䛾䝎䝑䝅䝳䝪䞊䝗䛾 సᡂ䛚㢪䛔䛧䜎䛩  䝕䞊䝍ရ㉁䛾ၥ㢟Ⅼ ̐ศᯒ䝏䞊䝮䜈䛾౫㢗㐣ከ ศᯒ䝏䞊䝮䛻䜽䜶䝸䛾సᡂ䚸ಟṇ䛺䛹䛾౫㢗䛜ቑຍ ㄡ䛛䛜సᡂ䛧䛯䜽䜶䝸䜢෌฼⏝䛩䜛䛣䛸䛷ᑡ䛧䛪䛴䝈 䝺䛜Ⓨ⏕

    ʍ/RRNHU䛾ᑟධ /RRNHU䛷஦๓䛻64/䜢ᐃ⩏䛧䛶䛒䛢䜛䛣䛸䛷䚸䜽䜶䝸 䛾᭩䛝᪉䛻䜘䜛䝈䝺䜢ῶ䜙䛩䚹 ㄡ䛷䜒䝎䝑䝅䝳䝪䞊䝗䜢⡆༢䛻సᡂ䛷䛝䜛䜘䛖䛻䛩䜛 䛣䛸䛷䚸䝕䞊䝍ά⏝䛾ಁ㐍䜢ᮇᚅ
  5.  ̐(7/䛾7ฎ⌮䛾䝒䞊䝹౫Ꮡᛶ䚸⫧኱໬䚸୙㏱᫂ᛶ 䝕䞊䝍㔞䛾ቑຍ䛻క䛔䝬䝇䜻䞁䜾䛺䛹䛾ຍᕤฎ⌮䜢 ⾜䛳䛶䛔䜛HPEXONෆ䛾7ฎ⌮䝁䞊䝗䛾ᐇ⾜᫬㛫䛜⫧኱ ໬䛧䛶䛝䛯 WURFFR䛺䛹䛾HPEXON௨እ䛾䝕䞊䝍䝟䜲䝥䝷䜲䞁䛾ᑟ ධ䛻䜘䜚7ฎ⌮䛜HPEXON䛻౫Ꮡ䛧䛶䛔䜛 HPEXON䛾䝁䞊䝗䜢ㄞ䜎䛺䛔䛸 7ฎ⌮䛾ෆᐜ䛜ᢕᥱ䛷 䛝䛪䚸ศᯒ⪅䛛䜙䛩䜛䛸୙㏱᫂䛺ฎ⌮䛷䛒䜛

    䝕䞊䝍ရ㉁䛾ၥ㢟Ⅼ ʍGEW&ORXG䛾ᑟධ䛻䜘䜛(/7䝟䜲䝥䝷䜲䞁䛾ᵓ⠏ GEW&ORXG䜢⏝䛔䛶HPEXON䛷⾜䛳䛶䛔䛯7ฎ⌮䜢௦᭰ 䝕䞊䝍䝟䜲䝥䝷䜲䞁䛻⏝䛔䛶䛔䜛䝒䞊䝹䛻౫Ꮡ䛫䛪 䛻䚸%LJ4XHU\䛻཰㞟䛧䛯䝕䞊䝍䛻ᑐ䛧䛶ᵝ䚻䛺ຍᕤ䛜 ྍ⬟䛻 ຍᕤ䜢୍⟠ᡤ䛻㞟䜑䜛䛣䛸䛷䚸ศᯒ⪅䛻ຍᕤ᝟ሗ䜢 㐺ษ䛻ఏ䛘䜙䜜䜛䜘䛖䛻ᮇᚅ 䝬䝇䜻䞁䜾䛺䛹䛾ຍᕤฎ⌮ ¡