Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
0
91

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.8k

Other Decks in Technology

See All in Technology
「AI駆動開発」のボトルネック『言語化』を効率化するには
Avatar for Tomohiro Tani taniiicom
1
230
ファインディにおける Dataform ブランチ戦略
Avatar for Noriaki Hiraki hiracky16
0
230
【CEDEC2025】『Shadowverse: Worlds Beyond』二度目のDCG開発でゲームをリデザインする～遊びやすさと競技性の両立～
Avatar for Cygames cygames
PRO
1
150
生成AI時代におけるAI・機械学習技術を用いたプロダクト開発の深化と進化 #BetAIDay
Avatar for LayerX layerx
PRO
0
110
AI エンジニアの立場からみた、AI コーディング時代の開発の品質向上の取り組みと妄想
Avatar for Soh Ohara soh9834
8
620
みんなのSRE 〜チーム全員でのSRE活動にするための4つの取り組み〜
Avatar for KAKEHASHI kakehashi
PRO
2
110
帳票構造化タスクにおけるLLMファインチューニングの性能評価
Avatar for yosukeyoshida yosukeyoshida
1
190
robocopy の怖い話/scary-story-about-robocopy
Avatar for emi emiki
0
420
From Live Coding to Vibe Coding with Firebase Studio
Avatar for Firebase Thailand firebasethailand
1
330
猫でもわかるQ_CLI(CDK開発編)+ちょっとだけKiro
Avatar for Hiroo Katoh kentapapa
0
140
テキストからの実世界知能の実現に向けて
Avatar for SUMO.ai sumoai
0
110
Unson OS｜48時間で「売れるか」を判定する AI 市場検証プラットフォーム
Avatar for 佐藤圭吾 unson
0
140

Featured

See All Featured
Put a Button on it: Removing Barriers to Going Fast.
Avatar for kastner kastner
60
3.9k
Bash Introduction
Avatar for André Augusto Costa Santos 62gerente
613
210k
GraphQLとの向き合い方2022年版
Avatar for Yosuke Kurami quramy
49
14k
How to train your dragon (web standard)
Avatar for Monica Dinculescu notwaldorf
96
6.1k
Writing Fast Ruby
Avatar for Erik Berlin sferik
628
62k
The Psychology of Web Performance [Beyond Tellerrand 2023]
Avatar for Tammy Everts tammyeverts
48
2.9k
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
235
140k
GitHub's CSS Performance
Avatar for Jon Rohan jonrohan
1031
460k
Gamification - CAS2011
Avatar for David Bonilla davidbonilla
81
5.4k
A Tale of Four Properties
Avatar for Chris Coyier chriscoyier
160
23k
The Web Performance Landscape in 2024 [PerfNow 2024]
Avatar for Tammy Everts tammyeverts
8
720
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
58
9.5k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo