Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Serverless Data Processing

Victor Wibisono
May 31, 2018
Technology
0
83

Serverless Data Processing

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
devacto
1
1.7k

Other Decks in Technology

See All in Technology
EthernetベースのGPUクラスタ導入による学びと展望
lycorptech_jp
PRO
0
400
総会員数1,500万人のレストランWeb予約サービスにおけるRustの活用
kymmt90
3
2.7k
iOS 18 から追加された SwiftUI の傾向について調べてみる
swiftty
2
100
情シスの引継ぎが大変という話
miyu_dev
2
520
GAS × Discord bot × Gemini で作ったさいきょーの情報収集ツール
ysknsid25
1
350
ご挨拶
iotcomjpadmin
0
190
ヤプリのデータカタログ整備 1年間の歩み / Progress of Building a Data Catalog at Yappli
yamamotoyuta
3
620
Amazon ECSとCloud Runの相互理解で広げる クラウドネイティブの景色 / Mutually understanding Amazon ECS and Cloud Run
iselegant
18
2.2k
Mastering Quickfix
daisuzu
2
480
SLMをエッジAIとして検証してみて分かったこと
iotcomjpadmin
0
190
専門領域に特化したチームの挑戦
leveragestech
0
200
LY Accessibility Guidelines @fukuoka_a11yconf_前夜祭
lycorptech_jp
PRO
1
130

Featured

See All Featured
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Fireside Chat
paigeccino
34
3k
Optimising Largest Contentful Paint
csswizardry
33
2.9k
RailsConf 2023
tenderlove
29
910
The Art of Programming - Codeland 2020
erikaheidi
53
13k
Why You Should Never Use an ORM
jnunemaker
PRO
54
9.1k
Building a Modern Day 
E-commerce SEO Strategy
aleyda
38
7k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
27
870
The World Runs on Bad Software
bkeepers
PRO
65
11k
Build The Right Thing And Hit Your Dates
maggiecrowley
33
2.4k
Designing for humans not robots
tammielis
250
25k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
10
760

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo