Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
110
0

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
GuardrailからGovernanceへ～AIエージェント運用の次の課題～
Avatar for 吉岡駿 sbspsy
1
220
次世代ランサムウェア対策の考察 / 20260704 Mitsutoshi Matsuo
Avatar for SHIFT EVOLVE shift_evolve
PRO
5
1.7k
SRE Lounge Hiroshimaへの招待
Avatar for gr1m0h grimoh
0
410
最近評価が難しくなった
Avatar for Mitsui maroon8021
0
260
AWS Summit の片隅で、体育座りしながらコミュニティがにぎわう理由を考えた
Avatar for Kazuki Adachi k_adachi_01
2
360
ローカルLLMとLINE Botの組み合わせ その3 / LINE DC Generative AI Meetup #8
Avatar for you（@youtoy） you
PRO
0
120
テスト設計の本質を改めて考えてみる～生成AIを活用する時代だからこそ、作ったテストの説明性を高めよう～
Avatar for YAMASAKI Takashi yamasaki696
1
390
NDIAS CTF 2026 問題解説会資料
Avatar for bata_24 bata_24
0
180
依頼文化をやめる日 EM視点で語るPlatform EngineeringとInclusive SRE / Discussing Platform Engineering and Inclusive SRE from an EM's Perspective
Avatar for katsukamaru shin1988
4
4.2k
初めてのDatabricks勉強会
Avatar for Takaaki Yayoi taka_aki
2
240
“全部コピーしない”ファイルデータの活用 ： — FSx for ONTAP × S3 Tables × Icebergで作るメタデータカタログ
Avatar for Yoshiki Fujiwara yoshiki0705
0
560
美しいコードを書くためにF#を学んでみた話
Avatar for yud0uhu yud0uhu
1
350

Featured

See All Featured
Unsuck your backbone
Avatar for Amy Palamountain ammeep
672
58k
Building a Modern Day 
E-commerce SEO Strategy
Avatar for Aleyda Solis aleyda
45
9.1k
Fantastic passwords and where to find them - at NoRuKo
Avatar for Phil Nash philnash
52
3.8k
Impact Scores and Hybrid Strategies: The future of link building
Avatar for Tamara Novitovic tamaranovitovic
0
330
For a Future-Friendly Web
Avatar for Brad Frost brad_frost
183
10k
Dealing with People You Can't Stand - Big Design 2015
Avatar for Cassini Nazir cassininazir
367
27k
Learning to Love Humans: Emotional Interface Design
Avatar for Aarron Walter aarron
275
41k
Efficient Content Optimization with Google Search Console & Apps Script
Avatar for Katarina Dahlin katarinadahlin
PRO
1
670
Game over? The fight for quality and originality in the time of robots
Avatar for Wayne Barker wayneb77
1
220
Primal Persuasion: How to Engage the Brain for Learning That Lasts
Avatar for Mike Taylor tmiket
0
380
Conquering PDFs: document understanding beyond plain text
Avatar for Ines Montani inesmontani
PRO
4
2.9k
Leadership Guide Workshop - DevTernity 2021
Avatar for David Neal reverentgeek
1
320

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo