Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
110
0

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
その Valkey 使いこなしていますか？
Avatar for Takafumi ONAKA onk
PRO
5
1.5k
SAENM ~Jupyterで残すPoCの試行錯誤、未来の自分と仲間を助ける記録！~
Avatar for Manabu TERADA terapyon
0
110
Local CI × GitHub Actions
Avatar for gogutan gogutan
2
190
Browser Spec: Testing Hotwire with Ruby
Avatar for Masayoshi Takahashi takahashim
1
120
re:Invent初参加で変わったこと、2回目にやってみたいこと（JAWS-UG彩の国埼玉 #12）
Avatar for 株式会社みずほ銀⾏ 情報数理⼯学研究所 デジタル技術開発部 mhrtech
2
220
分析AIエージェントを育てる、動く環境を育てる
Avatar for Codatum codatum
1
1.1k
今さら理解するウェブフォントver.2026
Avatar for takanorip takanorip
3
770
Tofu-Controller lightning talk @ Containerday 2026
Avatar for Giovanni Toraldo gionn
0
100
DGX Spark でどこまでハッキングができるのか？
Avatar for kash1064 kash1064
3
1.3k
Active Record Connection Adaptersにおける長年の課題に再び取り組む
Avatar for Yasuo Honda yahonda
0
140
アウトプット初心者、半年間のふりかえり
Avatar for aoi 4o1sobblue
0
430
日本語に強いオープンな大規模言語モデル「LLM-jp」
Avatar for Yusuke Oda odashi
4
1.1k

Featured

See All Featured
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
9
1.6k
How to Ace a Technical Interview
Avatar for Jacob Kaplan-Moss jacobian
280
24k
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
51
53k
svc-hook: hooking system calls on ARM64 by binary rewriting
Avatar for Akira Moroo retrage
2
630
Redefining SEO in the New Era of Traffic Generation
Avatar for Szymon Słowik szymonslowik
1
480
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
239
140k
HDC tutorial
Avatar for Michiel Stock michielstock
2
960
Prompt Engineering for Job Search
Avatar for Mfonobong Umondia mfonobong
0
520
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
Avatar for Stéphanie Walter stephaniewalter
287
14k
Evolving SEO for Evolving Search Engines
Avatar for Ryan Jones ryanjones
0
330
It's Worth the Effort
Avatar for 3n 3n
188
29k
Into the Great Unknown - MozCon
Avatar for Noah Learner thekraken
41
2.8k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo