Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Victor Wibisono
May 31, 2018
Technology
0
83

Serverless Data Processing

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
devacto
1
1.6k

Other Decks in Technology

See All in Technology
CEL(Common Expression Language)で書いた条件にマッチしたIAM Policyを見つける / iam-policy-finder
fujiwara3
0
720
簡単に始めるSnowflakeの機械学習
nayuts
1
190
「単なる OAuth 2.0 を認証に使うと、車が通れるほどのどでかいセキュリティー・ホールができる」のか検証してみた
terara
0
380
年間一億円削減した時系列データベースのアーキテクチャ改善～不確実性の高いプロジェクトへの挑戦～
lycorptech_jp
PRO
3
2.9k
[I/O Extended Android 2024] What`s new in Android 2024
kyeongwan
0
220
MySQLのロックの種類とその競合
yoku0825
6
1.6k
dxd2024-生成AIに振り回された3か月間の成功と失敗/dxd2024-link-and-motivation
lmi
2
260
ゆめみのアクセシビリティの現在地と今後
ryokatsuse
3
290
エンジニアの生存戦略 〜クラウド潮流の経験から紐解く技術トレンドのメカニズムと乗りこなし方〜
shimy
9
1.9k
地理情報とAPIのトレンド
nagix
0
160
大規模ドラレコデータ収集・機械学習基盤を支える AWS CDK 〜導入・運用事例紹介〜
pemugi
0
110
シフトレフトで挑む　セキュリティの生産性向上
sekido
PRO
0
270

Featured

See All Featured
The Pragmatic Product Professional
lauravandoore
29
6.1k
Debugging Ruby Performance
tmm1
71
11k
The Cult of Friendly URLs
andyhume
75
5.9k
A Philosophy of Restraint
colly
200
16k
Build The Right Thing And Hit Your Dates
maggiecrowley
28
2.2k
For a Future-Friendly Web
brad_frost
173
9.2k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
90
47k
Documentation Writing (for coders)
carmenintech
63
4.2k
Speed Design
sergeychernyshev
9
270
Principles of Awesome APIs and How to Build Them.
keavy
124
16k
Designing for Performance
lara
604
67k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
149
45k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo