Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
0
91

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
Devinを使ったモバイルアプリ開発 / Mobile app development with Devin
Avatar for Yuki Anzai yanzm
0
190
LLMエージェント時代に適応した開発フロー
Avatar for Yuya Hirayama hiragram
1
410
Goss: New Production-Ready Go Binding for Faiss #coefl_go_jp
Avatar for 弁護士ドットコム bengo4com
0
1.1k
AIエージェントの開発に必須な「コンテキスト・エンジニアリング」とは何か──プロンプト・エンジニアリングとの違いを手がかりに考える
Avatar for Masaya Mori (森正弥) / CAIO (Chief AI Officer) masayamoriofficial
0
390
Yahoo!ニュースにおけるソフトウェア開発
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
360
実践アプリケーション設計 ①データモデルとドメインモデル
Avatar for Recruit recruitengineers
PRO
3
260
新卒(ほぼ)専業Kagglerという選択肢
Avatar for Ryota nocchi1
1
2.3k
KiroでGameDay開催してみよう(準備編)
Avatar for Yuuuuuuu yuuuuuuu168
1
130
LLM時代の検索とコンテキストエンジニアリング
Avatar for shibuiwilliam shibuiwilliam
2
1.1k
JavaScript 研修
Avatar for Recruit recruitengineers
PRO
3
180
株式会社ARAV 採用案内
Avatar for ARAV maqui
0
350
サービスロボット最前線：ugoが挑むPhysical AI活用
Avatar for Ken Matsui kmatsuiugo
0
190

Featured

See All Featured
A designer walks into a library…
Avatar for Paul Jervis Heath pauljervisheath
207
24k
The Cost Of JavaScript in 2023
Avatar for Addy Osmani addyosmani
53
8.8k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
Avatar for panda_program panda_program
110
20k
XXLCSS - How to scale CSS and keep your sanity
Avatar for Zaharenia Atzitzikaki sugarenia
248
1.3M
BBQ
Avatar for Matthew Crist matthewcrist
89
9.8k
We Have a Design System, Now What?
Avatar for Morgane Peng morganepeng
53
7.7k
Fireside Chat
Avatar for paigeccino paigeccino
39
3.6k
For a Future-Friendly Web
Avatar for Brad Frost brad_frost
179
9.9k
A Tale of Four Properties
Avatar for Chris Coyier chriscoyier
160
23k
Gamification - CAS2011
Avatar for David Bonilla davidbonilla
81
5.4k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
53
2.9k
Optimizing for Happiness
Avatar for Tom Preston-Werner mojombo
379
70k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo