Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
110
0

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
20260912_スクフェス三河
Avatar for hikari kgnkhkr
0
100
スクラムで身についていた動き方を、XPで捉え直してみた
Avatar for コドモン開発チーム codmoninc
PRO
1
320
[DroidKaigi 2026] Making UI specifications visible: Android UI development in the AI agent era supported by Compose Screenshot Testing and galleries
Avatar for syarihu syarihu
0
610
10分で知る最近のOmarchy
Avatar for komagata komagata
0
170
AI時代に、プロダクトの数だけ積み上がる所有コストをどうエンジニアリングするか / Engineering the Cost of Ownership
Avatar for Kazuki Maeda kzkmaeda
1
1.1k
Redmine 7.0で私が開発した新機能の狙いと背景
Avatar for MAEDA Go vividtone
1
130
Driving AI Adoption Using In-House GPUs to Serve Qwen
Avatar for po3rin po3rin
2
470
Podは生きているのにGoだけが落ちる：GOGCとGOMEMLIMITで追うInvisible OOM Killの謎
Avatar for Watanabe Takeshi(Buzz) tkc66buzz
1
220
Omarchy Quattro の日本語設定周り
Avatar for Akira Shimosako simosako
2
100
TinyGo 開発サイクルを高速化する：Go で作るエミュレータ入門
Avatar for ZOZO Developers zozotech
PRO
1
130
WAF 運用改善の承認サイクル/SRE_BizReach_MIXI_1
Avatar for Visional Engineering ＆ Design visional_engineering_and_design
2
380
Bet AI Day 2026丨Agentは、「金融」という巨大産業の何を変えられるのか
Avatar for LayerX layerx
PRO
0
1.2k

Featured

See All Featured
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
Avatar for Joe Casabona jcasabona
35
2.8k
Everyday Curiosity
Avatar for Cassini Nazir cassininazir
0
310
The untapped power of vector embeddings
Avatar for Frank van Dijk frankvandijk
2
1.9k
Have SEOs Ruined the Internet? - User Awareness of SEO in 2025
Avatar for Akash Hashmi akashhashmi
0
490
The Spectacular Lies of Maps
Avatar for Per Axbom axbom
PRO
1
980
Leading Effective Engineering Teams in the AI Era
Avatar for Addy Osmani addyosmani
9
2.6k
Skip the Path - Find Your Career Trail
Avatar for Mark Kilby mkilby
1
220
技術選定の審美眼（2025年版） / Understanding the Spiral of Technologies 2025 edition
Avatar for Takuto WADA twada
PRO
120
120k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
Avatar for Kevin Liebholz kevinliebholz
37
6.6k
Dealing with People You Can't Stand - Big Design 2015
Avatar for Cassini Nazir cassininazir
367
27k
Agile Leadership in an Agile Organization
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
230
Embracing the Ebb and Flow
Avatar for Simon Collison colly
88
5.2k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo