Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Avatar for Victor Wibisono Victor Wibisono
May 31, 2018
Technology
110
0

Serverless Data Processing

Avatar for Victor Wibisono

Victor Wibisono

May 31, 2018

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
Avatar for Victor Wibisono devacto
1
1.9k

Other Decks in Technology

See All in Technology
ガバメント AI 源内を地方自治体は活用できるのか 可能性と課題、期待について
Avatar for takeda_h takeda_h
2
460
Go 1.27 の標準パッケージに uuid が入った！のでいろいろ喋る / go_127_std_go_uuid
Avatar for convto convto
3
570
Invisible to AI? Making TYPO3 Sites Quotable by AI Search Systems
Avatar for Wolfgang Wagner wolfgangwagner
0
240
AIに狂うスタートアップが、あえて「人との協働」に全振りした新卒エンジニア研修 / New Graduate Engineer Training at a Startup Accelerating AI Adoption
Avatar for akira ohno ohnoeight
0
180
Eight Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
3
8.2k
Kiro入門｜仕様駆動開発で変わるAI時代の開発スタイル
Avatar for 工藤皓夢 cmkudo
0
300
AIペネトレーションテスト・ セキュリティ検証「AgenticSec」紹介資料
Avatar for Sho Nakatani laysakura
2
9.3k
Head First モブプログラミング / Head First Mobprogramming
Avatar for TAKAKING22 takaking22
10
12k
AI時代に、人は何を、どう学ぶのか #pbl_pub / What and How Do We Learn in the AI Era?
Avatar for TAKAKING22 takaking22
1
200
AIのためのEthernet技術動向 (SerDes)
Avatar for Masayuki Kobayashi markunet
1
270
2026-08-15 JAWS-UG 茨城 #16 Secrets ManagerにおけるSecret値の管理(Terraformの場合) / Secrets Manager Secrets
Avatar for SUZUKI Masashi masasuzu
0
470
What's new in Go 1.27?
Avatar for Koya IWAMURA ciarana
0
290

Featured

See All Featured
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
39
3.4k
[RailsConf 2023] Rails as a piece of cake
Avatar for Vladimir Dementyev palkan
59
6.9k
Dominate Local Search Results - an insider guide to GBP, reviews, and Local SEO
Avatar for Greg Gifford greggifford
PRO
0
300
Visualization
Avatar for Eitan Lees eitanlees
152
17k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
Avatar for Stéphanie Walter stephaniewalter
287
14k
The Art of Delivering Value - GDevCon NA Keynote
Avatar for David Neal reverentgeek
16
2.1k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
10k
Product Roadmaps are Hard
Avatar for C. Todd Lombardo iamctodd
55
12k
Making the Leap to Tech Lead
Avatar for Ryan Cromwell cromwellryan
135
10k
Groundhog Day: Seeking Process in Gaming for Health
Avatar for Sebastian Deterding codingconduct
0
290
How To Speak Unicorn (iThemes Webinar)
Avatar for Michelle Schulp Hunt marktimemedia
1
550
sira's awesome portfolio website redesign presentation
Avatar for ElsiraPls elsirapls
0
340

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo