Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
0
110

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
20260321_エンベディングってなに？RAGってなに？エンベディングの説明とGemini Embedding 2 の紹介
Avatar for tsho tsho
0
130
Zero Data Loss Autonomous Recovery Service サービス概要
Avatar for oracle4engineer oracle4engineer
PRO
2
13k
Phase12_総括_自走化
Avatar for overflow ,Inc overflowinc
0
450
「通るまでRe-run」から卒業！落ちないテストを書く勘所
Avatar for asumikam asumikam
2
430
Phase04_ターミナル基礎
Avatar for overflow ,Inc overflowinc
0
750
Bill One 開発エンジニア 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
5
18k
モジュラモノリス導入から4年間の総括：アーキテクチャと組織の相互作用について / Architecture and Organizational Interaction
Avatar for Satoshi Kawashima nazonohito51
3
1.2k
AgentCoreとLINEを使った飲食店おすすめアプリを作ってみた
Avatar for やくも yakumo
2
140
大規模ECサイトのあるバッチのパフォーマンスを改善するために僕たちのチームがしてきたこと
Avatar for panda_program panda_program
1
320
Astro Islandsの 内部実装を 「日本で一番わかりやすく」 ざっくり解説！
Avatar for Wu Kenji knj
0
100
CyberAgentの生成AI戦略 〜変わるものと変わらないもの〜
Avatar for katayan katayan
0
280
A Casual Introduction to RISC-V
Avatar for Masanori Ogino omasanori
0
510

Featured

See All Featured
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
82
6.2k
Introduction to Domain-Driven Design and Collaborative software design
Avatar for Kenny Baas-Schwegler baasie
1
650
Exploring anti-patterns in Rails
Avatar for Elle Meredith aemeredith
2
290
Leveraging LLMs for student feedback in introductory data science courses - posit::conf(2025)
Avatar for Mine Cetinkaya-Rundel minecr
1
200
Context Engineering - Making Every Token Count
Avatar for Addy Osmani addyosmani
9
770
Leading Effective Engineering Teams in the AI Era
Avatar for Addy Osmani addyosmani
9
1.7k
WENDY [Excerpt]
Avatar for Tessa Abrams tessaabrams
9
37k
Keith and Marios Guide to Fast Websites
Avatar for Keith Pitt keithpitt
413
23k
Distributed Sagas: A Protocol for Coordinating Microservices
Avatar for Caitie McCaffrey caitiem20
333
22k
Making the Leap to Tech Lead
Avatar for Ryan Cromwell cromwellryan
135
9.8k
Conquering PDFs: document understanding beyond plain text
Avatar for Ines Montani inesmontani
PRO
4
2.5k
Producing Creativity
Avatar for Steve Smith orderedlist
PRO
348
40k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo