Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Victor Wibisono
May 31, 2018
Technology
0
86

Serverless Data Processing

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
devacto
1
1.8k

Other Decks in Technology

See All in Technology
Aspire をカスタマイズしよう & Aspire 9.2
nenonaninu
0
170
3D生成AIのための画像生成
kosukeito
1
160
日経電子版 for Android の技術的課題と取り組み(令和最新版)/android-20250423
nikkei_engineer_recruiting
1
460
PicoRabbit: a Tiny Presentation Device Powered by Ruby
harukasan
PRO
2
250
Winning at PHP in Production in 2025
beberlei
1
150
コスト最適重視でAurora PostgreSQLのログ分析基盤を作ってみた #jawsug_tokyo
non97
1
600
Amazon CloudWatch を使って NW 監視を行うには
o11yfes2023
0
180
Mastraに入門してみた ~AWS CDKを添えて~
tsukuboshi
0
320
Cursor AgentによるパーソナルAIアシスタント育成入門―業務のプロンプト化・MCPの活用
os1ma
15
5.2k
PagerDuty×ポストモーテムで築く障害対応文化/Building a culture of incident response with PagerDuty and postmortems
aeonpeople
2
400
SDカードフォレンジック
su3158
1
640
2025-04-24 "Manga AI Understanding & Localization" Furukawa Arata (CyberAgent, Inc)
ornew
2
260

Featured

See All Featured
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Thoughts on Productivity
jonyablonski
69
4.6k
Testing 201, or: Great Expectations
jmmastey
42
7.5k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.9k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
227
22k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
45
7.2k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
135
33k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Being A Developer After 40
akosma
91
590k
The Art of Programming - Codeland 2020
erikaheidi
53
13k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo