Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
0
100

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
こんなところでも（地味に）活躍するImage Modeさんを知ってるかい？- Image Mode for OpenShift -
Avatar for Masataka Tsukamoto tsukaman
0
120
【５分でわかる】セーフィー エンジニア向け会社紹介
Avatar for Safie safie_recruit
0
42k
Data Hubグループ 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
2.7k
Context Engineeringの取り組み
Avatar for nutslove nutslove
0
310
広告の効果検証を題材にした因果推論の精度検証について
Avatar for ZOZO Developers zozotech
PRO
0
140
名刺メーカーDevグループ 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
1k
ブロックテーマでサイトをリニューアルした話 / 2026-01-31 Kansai WordPress Meetup
Avatar for Toro_Unit (Hiroshi Urabe) torounit
0
460
顧客との商談議事録をみんなで読んで顧客解像度を上げよう
Avatar for shibayu36 shibayu36
0
200
Deno・Bunの標準機能やElysiaJSを使ったWebSocketサーバー実装 / ラーメン屋を貸し切ってLT会! IoTLT 2026新年会
Avatar for you（@youtoy） you
PRO
0
300
生成AIを活用した音声文字起こしシステムの2つの構築パターンについて
Avatar for Kazuki Miura miu_crescent
PRO
1
160
Context Engineeringが企業で不可欠になる理由
Avatar for Hirosato Gamo hirosatogamo
PRO
3
510
GitHub Issue Templates + Coding Agentで簡単みんなでIaC/Easy IaC for Everyone with GitHub Issue Templates + Coding Agent
Avatar for AEON aeonpeople
1
200

Featured

See All Featured
Leading Effective Engineering Teams in the AI Era
Avatar for Addy Osmani addyosmani
9
1.6k
KATA
Avatar for Megan Lloyd mclloyd
PRO
34
15k
So, you think you're a good person
Avatar for Per Axbom axbom
PRO
2
1.9k
How to audit for AI Accessibility on your Front & Back End
Avatar for davetheseo davetheseo
0
180
エンジニアに許された特別な時間の終わり
Avatar for watany watany
106
230k
Avoiding the “Bad Training, Faster” Trap in the Age of AI
Avatar for Mike Taylor tmiket
0
72
Redefining SEO in the New Era of Traffic Generation
Avatar for Szymon szymonslowik
1
210
HDC tutorial
Avatar for Michiel Stock michielstock
1
370
Exploring anti-patterns in Rails
Avatar for Elle Meredith aemeredith
2
250
How To Stay Up To Date on Web Technology
Avatar for Chris Coyier chriscoyier
791
250k
The Director’s Chair: Orchestrating AI for Truly Effective Learning
Avatar for Mike Taylor tmiket
1
96
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
61
9.7k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo