Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Amazon EMR でのバッチ処理 Spark アプリケーション運用で 気をつけたいこと

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for soonraah soonraah
March 29, 2015
Technology
650
1

Amazon EMR でのバッチ処理 Spark アプリケーション運用で 気をつけたいこと

2015-03-29 第2回 “Learning Spark” 読書会 LT

Avatar for soonraah

soonraah

March 29, 2015

More Decks by soonraah

See All by soonraah
MLCT #5 Pipeline Jungles とたたかう話
Avatar for soonraah soonraah
0
2k
Apache Spark による推薦システム案件例
Avatar for soonraah soonraah
11
8.1k
Spark MLlib でやってみる協調フィルタリング
Avatar for soonraah soonraah
8
7.8k

Other Decks in Technology

See All in Technology
Agentic AI 時代のテスト手法: Kiro とはじめるプロパティベーステスト (AWS Summit Japan 2026 | DEV212)
Avatar for Hiroki Yamazaki ymhiroki
0
220
「ちゃんとやっている」は独りよがりだった ― 不安に寄り添うインシデント対応へ / Towards incident response that addresses anxieties
Avatar for mekka chmikata
1
3.3k
AIDLC_ヤフーショッピングの取り組み
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
560
AI Agent SaaS を支える自社仮想化基盤への挑戦と実運用 / ai-agent-saas-virtualization
Avatar for GMO Flatt Security flatt_security
2
2.7k
Control Planeで育てるBtoB SaaSの認証基盤 - SRE NEXT 2026
Avatar for pokohide pokohide
1
1.3k
CIで使うClaude
Avatar for TomoyaIwata iwatatomoya
0
160
SRE歴2ヶ月でも開発6年の知見を活かして、チームで止まっていた環境改善を前に進めた話
Avatar for a_ono a_ono
1
220
勉強会企画をアプリで構造化してみた 〜そこで見えた、AIとの付き合い方〜 / I've structured a study group plan using an app.
Avatar for Pauli pauli
0
330
AI時代における最適なQA組織の作り方
Avatar for ymty ymty
3
460
プライバシー保護の理論と実践
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
1
250
テスト設計の本質を改めて考えてみる～生成AIを活用する時代だからこそ、作ったテストの説明性を高めよう～
Avatar for YAMASAKI Takashi yamasaki696
1
390
SRE Lounge Hiroshimaへの招待
Avatar for gr1m0h grimoh
0
380

Featured

See All Featured
HTML-Aware ERB: The Path to Reactive Rendering @ RubyCon 2026, Rimini, Italy
Avatar for Marco Roth marcoroth
2
300
A designer walks into a library…
Avatar for Paul Jervis Heath pauljervisheath
211
24k
SERP Conf. Vienna - Web Accessibility: Optimizing for Inclusivity and SEO
Avatar for Sara Fernández Carmona sarafernandez
2
1.5k
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
Avatar for UX Y'all uxyall
0
340
SEO for Brand Visibility & Recognition
Avatar for Aleyda Solis aleyda
0
4.6k
Making Projects Easy
Avatar for Brett Harned brettharned
120
6.7k
The State of eCommerce SEO: How to Win in Today's Products SERPs - #SEOweek
Avatar for Aleyda Solis aleyda
2
11k
Introduction to Domain-Driven Design and Collaborative software design
Avatar for Kenny Baas-Schwegler baasie
1
890
End of SEO as We Know It (SMX Advanced Version)
Avatar for Michael King ipullrank
3
4.3k
The Psychology of Web Performance [Beyond Tellerrand 2023]
Avatar for Tammy Everts tammyeverts
49
3.5k
Ten Tips & Tricks for a 🌱 transition
Avatar for Manu Carrasco Molina stuffmc
0
150
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
51
52k

Transcript

  1. "NB[PO&.3Ͱͷόονॲཧ 4QBSLΞϓϦέʔγϣϯӡ༻Ͱ ؾΛ͚͍ͭͨ͜ͱ  ୈճl-FBSOJOH4QBSLzಡॻձ-5 !TPPOSBBI

  2. "(&/%" •  "NB[PO&.3 •  ΠϯελϯεੑೳWTίετ •  +PCqPXΛ։࢝Ͱ͖ͳ͍৔߹ͷอݥ •  XFC6*ͷݟํ • 

    ·ͱΊ 2

  3. "NB[PO&.3 &MBTUJD.BQ3FEVDF  •  )BEPPQϑϨʔϜϫʔΫΛΫϥ΢υఏڙ •  )%'4Λར༻Մೳ •  4QBSL΍1SFTUP౳Λ࣮ߦՄೳ • 

    ॳظඅ༻Λ཈͑ͯ෼ࢄॲཧΛࢼͤΔ •  όονॲཧ޲͖ʁ  3

  4. ΠϯελϯεੑೳWTίετ •  ҎԼͷཁૉΛߟྀܾͯ͠ఆ •  ඞཁͳϝϞϦྔ͸ʁ •  ϩʔΧϧͰͷ࣮ݧ΍4QBSLXFC6*͔Βݟੵ •  ཁٻ͞ΕΔॲཧ࣌ؒ͸ʁ • 

    ϩʔΧϧͰͷ࣮ݧͰݟੵ •  ༧ࢉʹऩ·Δ͔ʁ •  εϙοτΠϯελϯεͷ৔߹͸࣌Ձ •  ೖࡳՁ֨ͷࢦఆ 4

  5. ΠϯελϯεՁ֨มಈ͸େ͖͍ʜ 5

  6. +PCqPXΛ։࢝Ͱ͖ͳ͍৔߹ͷอݥ •  +PCΛ։࢝Ͱ͖ͳ͍ •  #PPUTUSBQ"DUJPOͰͷࣦഊ •  ೖࡳՁ͕֨࣌ՁΑΓ௿͍ •  ΘΓͱى͜Δ • 

    &.3KPCqPXϦΫΤετଆʹอݥͷॲཧΛ •  +PCqPXΛ؂ࢹ •  ࣦഊͨ͠ΒΠϯελϯελΠϓ΍ೖࡳՁ֨Λ ม͑ͯϦτϥΠ͢Δ࢓૊Έ 6

  7. 8FC6* •  ࣮ߦதͷϝτϦΫε͕ݟΕΔ •  εςʔδ͝ͱͷλεΫ࣮ߦঢ়گ •  ֤3%%ͷ࢖༻ϝϞϦྔ •  ϊʔυ৘ใ • 

    Ϧιʔεௐ੔΍νϡʔχϯάͷͨΊʹ 7

  8. 8FC6* 8

  9. 8FC6*ͷݟํʢʣ 9

  10. 8FC6*ͷݟํʢʣ 10

  11. 8FC6*ͷݟํʢʣ 11

  12. 8FC6*ͷݟํʢʣ 12

  13. 8FC6*ͷݟํʢʣ 13

  14. ·ͱΊ •  ඞཁͳΠϯελϯεੑೳΛݟੵ΋ͬͯίετΛ ܭࢉ͠Α͏ •  +PCqPXͷ։࢝ʹࣦഊͨ͠ͱ͖ͷอݥ͕͋Δͱ ӡ༻Λָʹ͠Α͏ •  8FC6*ΛݟΑ͏ 14