Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
0
100

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
15 years with Rails and DDD (AI Edition)
Avatar for Andrzej Krzywda andrzejkrzywda
0
170
2026年、サーバーレスの現在地 -「制約と戦う技術」から「当たり前の実行基盤」へ- /serverless2026
Avatar for Serverless Operations slsops
2
210
あたらしい上流工程の形。 0日導入からはじめるAI駆動PM
Avatar for 熊井悠 kumaiu
5
760
ブロックテーマでサイトをリニューアルした話 / 2026-01-31 Kansai WordPress Meetup
Avatar for Toro_Unit (Hiroshi Urabe) torounit
0
450
Databricks Free Edition講座 データサイエンス編
Avatar for Takaaki Yayoi taka_aki
0
290
SREじゃなかった僕らがenablingを通じて「SRE実践者」になるまでのリアル / SRE Kaigi 2026
Avatar for AEON aeonpeople
6
2.1k
会社紹介資料 / Sansan Company Profile
Avatar for Sansan, Inc. sansan33
PRO
15
400k
ClickHouseはどのように大規模データを活用したAIエージェントを全社展開しているのか
Avatar for Miki Matsumoto mikimatsumoto
0
190
M&A 後の統合をどう進めるか ─ ナレッジワーク × Poetics が実践した組織とシステムの融合
Avatar for KNOWLEDGE WORK / 株式会社ナレッジワーク kworkdev
PRO
1
410
Ruby版 JSXのRuxが気になる
Avatar for SansanTech sansantech
PRO
0
100
クレジットカード決済基盤を支えるSRE - 厳格な監査とSRE運用の両立 (SRE Kaigi 2026)
Avatar for capytan capytan
6
2.6k
入社1ヶ月でデータパイプライン講座を作った話
Avatar for Yuta Ozaki waiwai2111
1
250

Featured

See All Featured
No one is an island. Learnings from fostering a developers community.
Avatar for Antonio thoeni
21
3.6k
Building Better People: How to give real-time feedback that sticks.
Avatar for Will Jessup wjessup
370
20k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
Avatar for Sam Stephenson sstephenson
162
16k
How to Talk to Developers About Accessibility
Avatar for Jason CranfordTeague jct
2
130
世界の人気アプリ100個を分析して見えたペイウォール設計の心得
Avatar for Akihiro Kokubo akihiro_kokubo
PRO
66
36k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
9.8k
Visual Storytelling: How to be a Superhuman Communicator
Avatar for David Neal reverentgeek
2
430
What does AI have to do with Human Rights?
Avatar for Per Axbom axbom
PRO
0
2k
Embracing the Ebb and Flow
Avatar for Simon Collison colly
88
5k
The Language of Interfaces
Avatar for Des Traynor destraynor
162
26k
Chasing Engaging Ingredients in Design
Avatar for Sebastian Deterding codingconduct
0
110
Tips & Tricks on How to Get Your First Job In Tech
Avatar for Honza Javorek honzajavorek
0
430

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo