Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
0
91

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
2025年になってもまだMySQLが好き
Avatar for yoku0825 yoku0825
8
4.8k
Generative AI Japan 第一回生成AI実践研究会「AI駆動開発の現在地──ブレイクスルーの鍵を握るのはデータ領域」
Avatar for KoheiOgawa shisyu_gaku
0
300
普通のチームがスクラムを会得するたった一つの冴えたやり方 / the best way to scrum
Avatar for 岡本卓也 okamototakuyasr2
0
100
要件定義・デザインフェーズでもAIを活用して、コミュニケーションの密度を高める
Avatar for KazukiHayase kazukihayase
0
120
La gouvernance territoriale des données grâce à la plateforme Terreze
Avatar for BlueHats bluehats
0
180
現場で効くClaude Code ─ 最新動向と企業導入
Avatar for TakaakiKakei takaakikakei
1
250
複数サービスを支える マルチテナント型 Batch MLプラットフォーム
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
1
730
TS-S205_昨年対比2倍以上の機能追加を実現するデータ基盤プロジェクトでのAI活用について
Avatar for K.Mitsuhashi kaz3284
1
190
Agile PBL at New Grads Trainings
Avatar for Yasunobu Kawaguchi kawaguti
PRO
1
440
なぜスクラムはこうなったのか？歴史が教えてくれたこと/Shall we explore the roots of Scrum
Avatar for Genki Sano sanogemaru
5
1.6k
これでもう迷わない！Jetpack Composeの書き方実践ガイド
Avatar for ZOZO Developers zozotech
PRO
0
1k
初めてAWSを使うときのセキュリティ覚書〜初心者支部編〜
Avatar for cm-usuda-keisuke cmusudakeisuke
1
270

Featured

See All Featured
Put a Button on it: Removing Barriers to Going Fast.
Avatar for kastner kastner
60
4k
GitHub's CSS Performance
Avatar for Jon Rohan jonrohan
1032
460k
Agile that works and the tools we love
Avatar for Rasmus Luckow-Nielsen rasmusluckow
330
21k
How STYLIGHT went responsive
Avatar for nonsquared nonsquared
100
5.8k
Building Adaptive Systems
Avatar for Chris Keathley keathley
43
2.7k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
8
920
Side Projects
Avatar for Sacha Greif sachag
455
43k
Measuring & Analyzing Core Web Vitals
Avatar for Philip Tellis bluesmoon
9
580
Music & Morning Musume
Avatar for Bryan Veloso bryan
46
6.8k
Six Lessons from altMBA
Avatar for Skipper Chong Warson skipperchong
28
4k
Fireside Chat
Avatar for paigeccino paigeccino
39
3.6k
Bootstrapping a Software Product
Avatar for Garrett Dimon garrettdimon
PRO
307
110k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo