Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Victor Wibisono
May 31, 2018
Technology
0
83

Serverless Data Processing

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

See All by Victor Wibisono
Predictive Maintenance Pipeline using Kafka Connect, Streams and KSQL
devacto
1
1.7k

Other Decks in Technology

See All in Technology
OCI Network Firewall 概要
oracle4engineer
PRO
0
4.1k
Incident Response Practices: Waroom's Features and Future Challenges
rrreeeyyy
0
160
マルチプロダクトな開発組織で 「開発生産性」に向き合うために試みたこと / Improving Multi-Product Dev Productivity
sugamasao
1
300
Terraform Stacks入門 #HashiTalks
msato
0
350
オープンソースAIとは何か？ --「オープンソースAIの定義 v1.0」詳細解説
shujisado
7
810
なぜ今 AI Agent なのか _近藤憲児
kenjikondobai
4
1.4k
Shopifyアプリ開発における Shopifyの機能活用
sonatard
4
250
Taming you application's environments
salaboy
0
180
リンクアンドモチベーション ソフトウェアエンジニア向け紹介資料 / Introduction to Link and Motivation for Software Engineers
lmi
4
300k
複雑なState管理からの脱却
sansantech
PRO
1
140
Exadata Database Service on Dedicated Infrastructure(ExaDB-D) UI スクリーン・キャプチャ集
oracle4engineer
PRO
2
3.2k
テストコード品質を高めるためにMutation Testingライブラリ・Strykerを実戦導入してみた話
ysknsid25
7
2.6k

Featured

See All Featured
Building an army of robots
kneath
302
43k
A Philosophy of Restraint
colly
203
16k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
25
1.8k
The Invisible Side of Design
smashingmag
298
50k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
27
840
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
131
33k
Being A Developer After 40
akosma
86
590k
Side Projects
sachag
452
42k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
232
17k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Gamification - CAS2011
davidbonilla
80
5k
StorybookのUI Testing Handbookを読んだ
zakiyama
27
5.3k

Transcript

  1. Serverless Data Processing

  2. Contents • What it is and why • Architecture •

    Demo

  3. Serverless?

  4. Serverless is about focusing your efforts on what provides value

    to users.

  5. No servers?

  6. None
  7. None

  8. https://serverless.com/learn/

  9. Data processing?

  10. None
  11. None
  12. None

  13. Architecture

  14. None

  15. • Infinitely scalable • 99.9999999% data durability • Pay-per-use, no

    pre- provisioning

  16. AWS Glue • Spark-as-a-service • No cluster management • Pay-per-use,

    no pre-provisioning • Services include: data crawling, cataloging (Hive metastore)

  17. AWS Athena • Pay-per-query • No pre-provisioning • Infinitely scalable

  18. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

  19. None

  20. Learn more...

  21. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

  22. Demo