Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
0
89

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.8k

Other Decks in Technology

See All in Technology
Amazon Bedrockで実現する 新たな学習体験
Avatar for Kazuki Maeda kzkmaeda
1
500
2年でここまで成長！AWSで育てたAI Slack botの軌跡
Avatar for iwamot iwamot
PRO
4
640
Clineを含めたAIエージェントを 大規模組織に導入し、投資対効果を考える / Introducing AI agents into your organization
Avatar for Masato Ishigaki / 石垣雅人 i35_267
4
1.5k
本当に使える？AutoUpgrade の新機能を実践検証してみた
Avatar for oracle4engineer oracle4engineer
PRO
1
140
～宇宙最速～ 2025年AWS Summit レポート
Avatar for sleepingsato satodesu
1
1.7k
初めてのAzure FunctionsをClaude Codeで作ってみた / My first Azure Functions using Claude Code
Avatar for Hideaki Aoyagi hideakiaoyagi
1
210
「Chatwork」の認証基盤の移行とログ活用によるプロダクト改善
Avatar for kubell kubell_hr
1
110
AIエージェント最前線！ Amazon Bedrock、Amazon Q、そしてMCPを使いこなそう
Avatar for みのるん minorun365
PRO
13
4.8k
Snowflake Summit 2025全体振り返り / Snowflake Summit 2025 Overall Review
Avatar for k.muguruma mtpooh
2
390
Javaで作る RAGを活用した Q＆Aアプリケーション
Avatar for Recruit recruitengineers
PRO
1
100
Observability в PHP без боли. Олег Мифле, тимлид Altenar
Avatar for Lamoda Tech lamodatech
0
330
Witchcraft for Memory
Avatar for pocke pocke
1
210

Featured

See All Featured
We Have a Design System, Now What?
Avatar for Morgane Peng morganepeng
53
7.7k
Documentation Writing (for coders)
Avatar for Carmen Chung carmenintech
71
4.9k
Visualization
Avatar for Eitan Lees eitanlees
146
16k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
Avatar for panda_program panda_program
107
19k
GitHub's CSS Performance
Avatar for Jon Rohan jonrohan
1031
460k
Building a Scalable Design System with Sketch
Avatar for Laura Van Doore lauravandoore
462
33k
Dealing with People You Can't Stand - Big Design 2015
Avatar for Cassini Nazir cassininazir
367
26k
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
36
2.8k
A Tale of Four Properties
Avatar for Chris Coyier chriscoyier
160
23k
The Cult of Friendly URLs
Avatar for Andy Hume andyhume
79
6.5k
Typedesign – Prime Four
Avatar for Hannes Fritz hannesfritz
42
2.7k
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo