$30 off During Our Annual Pro Sale. View Details »

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
0
100

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
子育てで想像してなかった「見えないダメージ」 / Unforeseen "hidden burdens" of raising children.
Avatar for Pauli pauli
2
310
Agent Skillsがハーネスの垣根を超える日
Avatar for Gota gotalab555
5
2.8k
会社紹介資料 / Sansan Company Profile
Avatar for Sansan, Inc. sansan33
PRO
11
390k
ESXi のAIOps だ！2025冬
Avatar for Unno Wataru unnowataru
0
150
フィッシュボウルのやり方 / How to do a fishbowl
Avatar for Pauli pauli
2
340
たまに起きる外部サービスの障害に備えたり備えなかったりする話
Avatar for Sohei Iwahori egmc
0
350
M&Aで拡大し続けるGENDAのデータ活用を促すためのDatabricks権限管理 / AEON TECH HUB #22
Avatar for GENDA genda
0
180
AI-DLCを現場にインストールしてみた：プロトタイプ開発で分かったこと･やめたこと
Avatar for Recruit recruitengineers
PRO
2
220
re:Invent 2025 ~何をする者であり、どこへいくのか~
Avatar for tetutetu214 tetutetu214
0
240
ウェルネス SaaS × AI、1,000万ユーザーを支える 業界特化 AI プロダクト開発への道のり
Avatar for hacomono Inc. hacomono
PRO
0
340
Strands Agents × インタリーブ思考 で変わるAIエージェント設計 / Strands Agents x Interleaved Thinking AI Agents
Avatar for Takanori Suzuki takanorig
4
1.6k
AWS re:Invent 2025~初参加の成果と学び~
Avatar for Kubo kubomasataka
0
170

Featured

See All Featured
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
Avatar for Kevin Liebholz kevinliebholz
37
6.2k
Navigating Weather and Climate Data
Avatar for Ryan Abernathey rabernat
0
45
VelocityConf: Rendering Performance Case Studies
Avatar for Addy Osmani addyosmani
333
24k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
Avatar for Morgane Peng morganepeng
132
19k
Claude Code のすすめ
Avatar for schroneko schroneko
65
200k
Leveraging Curiosity to Care for An Aging Population
Avatar for Cassini Nazir cassininazir
1
130
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
65
8.3k
The innovator’s Mindset - 
Leading Through an Era of Exponential Change - McGill University 2025
Avatar for Jean-Marc De Jonghe jdejongh
PRO
1
65
Building the Perfect Custom Keyboard
Avatar for Naoto Takai takai
1
660
Marketing Yourself as an Engineer | Alaka | Gurzu
Avatar for Gurzu gurzu
0
87
How To Stay Up To Date on Web Technology
Avatar for Chris Coyier chriscoyier
791
250k
Navigating Team Friction
Avatar for Lara Hogan lara
191
16k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo