Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
0
97

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
Active Directory 勉強会 第 6 回目 Active Directory セキュリティについて学ぶ回
Avatar for Eureka eurekaberry
16
5.9k
AI 時代のデータ戦略
Avatar for na0 na0
8
3.2k
MAP-7thplaceSolution
Avatar for yukichi0403 yukichi0403
2
240
小さな判断で育つ、大きな意思決定力 / 20251204 Takahiro Kinjo
Avatar for SHIFT EVOLVE shift_evolve
PRO
1
260
履歴テーブル、今回はこう作りました 〜 Delegated Types編 〜 / How We Built Our History Table This Time — With Delegated Types
Avatar for moznion moznion
15
9.3k
MS Ignite 2025で発表されたFoundry IQをRecap
Avatar for Akira Sato satodayo
3
230
こがヘンだよ！Snowflake?サービス名称へのこだわり
Avatar for タロウ tarotaro0129
0
110
プラットフォームエンジニアリングとは何であり、なぜプラットフォームエンジニアリングなのか
Avatar for doublemarket doublemarket
1
540
ECMAScript仕様の最新動向: プロセスの変化と仕様のトレンド
Avatar for uhyo uhyo
2
400
なぜ使われないのか？──定量×定性で見極める本当のボトルネック
Avatar for KAKEHASHI kakehashi
PRO
1
670
プロダクトマネジメントの分業が生む「デリバリーの渋滞」を解消するTPMの越境
Avatar for Recruit recruitengineers
PRO
3
380
Introduction to Sansan, inc / Sansan Global Development Center, Inc.
Avatar for Sansan, Inc. sansan33
PRO
0
2.9k

Featured

See All Featured
Optimizing for Happiness
Avatar for Tom Preston-Werner mojombo
379
70k
Facilitating Awesome Meetings
Avatar for Lara Hogan lara
57
6.7k
Building Adaptive Systems
Avatar for Chris Keathley keathley
44
2.9k
RailsConf 2023
Avatar for Aaron Patterson tenderlove
30
1.3k
Navigating Team Friction
Avatar for Lara Hogan lara
191
16k
We Have a Design System, Now What?
Avatar for Morgane Peng morganepeng
54
7.9k
The Pragmatic Product Professional
Avatar for Laura Van Doore lauravandoore
37
7k
The Web Performance Landscape in 2024 [PerfNow 2024]
Avatar for Tammy Everts tammyeverts
12
960
Docker and Python
Avatar for Tania Allard trallard
46
3.7k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
Avatar for Chris Coyier chriscoyier
508
140k
Why Our Code Smells
Avatar for Brandon Keepers bkeepers
PRO
340
57k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
9
1.1k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo