Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Victor Wibisono
May 31, 2018
Technology
0
83

Serverless Data Processing

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
devacto
1
1.7k

Other Decks in Technology

See All in Technology
日本版とグローバル版のモバイルアプリ統合の開発の裏側と今後の展望
miichan
1
130
PHP ユーザのための OpenTelemetry 入門 / phpcon2024-opentelemetry
shin1x1
1
170
UI State設計とテスト方針
rmakiyama
2
470
生成AIのガバナンスの全体像と現実解
fnifni
1
180
re:Invent 2024 Innovation Talks(NET201)で語られた大切なこと
shotashiratori
0
310
非機能品質を作り込むための実践アーキテクチャ
knih
3
1k
KubeCon NA 2024 Recap: How to Move from Ingress to Gateway API with Minimal Hassle
ysakotch
0
200
サービスでLLMを採用したばっかりに振り回され続けたこの一年のあれやこれや
segavvy
2
390
組織に自動テストを書く文化を根付かせる戦略（2024冬版） / Building Automated Test Culture 2024 Winter Edition
twada
PRO
13
3.6k
小学3年生夏休みの自由研究「夏休みに Copilot で遊んでみた」
taichinakamura
0
150
コンテナセキュリティのためのLandlock入門
nullpo_head
2
320
私なりのAIのご紹介 [2024年版]
qt_luigi
1
120

Featured

See All Featured
Adopting Sorbet at Scale
ufuk
73
9.1k
Building Better People: How to give real-time feedback that sticks.
wjessup
365
19k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
26
1.9k
Designing for humans not robots
tammielis
250
25k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
6
510
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
226
22k
Why Our Code Smells
bkeepers
PRO
335
57k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
247
1.3M
A Tale of Four Properties
chriscoyier
157
23k
Designing Experiences People Love
moore
138
23k
Speed Design
sergeychernyshev
25
670
Done Done
chrislema
181
16k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo