Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Victor Wibisono
May 31, 2018
Technology
0
86

Serverless Data Processing

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
devacto
1
1.8k

Other Decks in Technology

See All in Technology
2025年の挑戦 コーポレートエンジニアの技術広報/techpr5
nishiuma
0
140
Amazon Route 53, 待ちに待った TLSAレコードのサポート開始
kenichinakamura
0
150
あなたの知らないクラフトビールの世界
miura55
0
120
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
6
54k
【JAWS-UG大阪 reInvent reCap LT大会 サンバが始まったら強制終了】“1分”で初めてのソロ参戦reInventを数字で振り返りながら反省する
ttelltte
0
130
KMP with Crashlytics
sansantech
PRO
0
240
JuliaTokaiとJuliaLangJaの紹介 for NGK2025S
antimon2
1
110
デジタルアイデンティティ技術 認可・ID連携・認証 応用 / 20250114-OIDF-J-EduWG-TechSWG
oidfj
2
630
.NET 最新アップデート ～ AI とクラウド時代のアプリモダナイゼーション
chack411
0
200
dbtを中心にして組織のアジリティとガバナンスのトレードオンを考えてみた
gappy50
0
160
生成AI × 旅行 LLMを活用した旅行プラン生成・チャットボット
kominet_ava
0
150
Evolving Architecture
rainerhahnekamp
3
250

Featured

See All Featured
Become a Pro
speakerdeck
PRO
26
5.1k
Bash Introduction
62gerente
610
210k
Reflections from 52 weeks, 52 projects
jeffersonlam
348
20k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.6k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Building an army of robots
kneath
302
45k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
49k
KATA
mclloyd
29
14k
Rebuilding a faster, lazier Slack
samanthasiow
79
8.8k
The Invisible Side of Design
smashingmag
299
50k
StorybookのUI Testing Handbookを読んだ
zakiyama
28
5.4k
Speed Design
sergeychernyshev
25
730

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo