Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Data Processing

Serverless Data Processing

Victor Wibisono

May 31, 2018
Tweet

More Decks by Victor Wibisono

Other Decks in Technology

Transcript

  1. Serverless Data Processing

    View full-size slide

  2. Contents
    • What it is and why

    • Architecture

    • Demo

    View full-size slide

  3. Serverless is about focusing your efforts on what provides
    value to users.

    View full-size slide

  4. https://serverless.com/learn/

    View full-size slide

  5. Data processing?

    View full-size slide

  6. Architecture

    View full-size slide

  7. • Infinitely scalable

    • 99.9999999% data durability

    • Pay-per-use, no pre-
    provisioning

    View full-size slide

  8. AWS Glue
    • Spark-as-a-service

    • No cluster management

    • Pay-per-use, no pre-provisioning

    • Services include: data crawling, cataloging (Hive
    metastore)

    View full-size slide

  9. AWS Athena
    • Pay-per-query

    • No pre-provisioning

    • Infinitely scalable

    View full-size slide

  10. https://blog.panoply.io/an-amazonian-battle-comparing-athena-and-redshift

    View full-size slide

  11. Learn more...

    View full-size slide

  12. https://unnik.s3.amazonaws.com/public-files/unnik-lab-guides/aws-summit-2018/datalake/unnik-aws-summit-2018-datalake-demo.html

    View full-size slide