Easy Data Processing on Azure with Serverless Functions

Easy Data Processing on Azure with Serverless Functions

Serverless or Function as a Service (FaaS) enables you to focus more on your code while making it easier to deploy your applications. FaaS can be great tools to perform data processing scenarios. Azure functions allow you to leverage Azure's robust, managed, and scalable cloud computing platform. In this workshop, Tania Allard will teach you how you can get started with Azure functions and Python for data processing scenarios.

✨ Check the tutorial at https://aka.ms/pycon2020-azurefunctions

✨ GitHub repository https://github.com/trallard/pycon2020-azure-functions

🐍 PyCon schedule description: https://us.pycon.org/2020/schedule/presentation/230/

Ecdea9b9714877b86cee08458f085481?s=128

Tania Allard

April 24, 2020
Tweet

Transcript

  1. Tania Allard, PhD PyCon 2020 Easy data processing with Python

    and Azure functions @ixek
  2. • Detailed step-by-step tutorial: aka.ms/pycon2020- azurefunctions • GitHub Repository: https:!//github.com/trallard/pycon2020-

    azure-functions • Slides:aka.ms/pycon2020-azurefunctions-slides Resources: @ixek
  3. Serverless computing Are there no servers at all? @ixek

  4. Serverless computing Are there no servers at all? Yes there

    are but: - You are not responsible for the management or maintenance - Your cloud provider takes care of the provisioning @ixek
  5. Serverless computing You focus on the code not the infrastructure

    @ixek
  6. Managed Pay only for what you use Scalable Why Serverless

    computing? @ixek
  7. Managed serverless Microsoft Azure functions @ixek Learn more at: https:!//cda.ms/1g7

  8. Handles: software, monitoring, scaling, and hardware Host management Microsoft Azure

    functions @ixek
  9. Are responsible for the application code Decide what services to

    integrate (i.e databases, Blob Storage, email provider) You - the developer @ixek
  10. Why functions? Microsoft Azure functions @ixek

  11. Serverless computing A.K.A Functions as a Service (FasS) @ixek

  12. Microsoft Azure functions Why functions? FaaS Self contained Code snippets

    In the cloud @ixek
  13. Microsoft Azure functions FaaS Self contained Code snippets In the

    cloud a.k.a function @ixek
  14. Event triggered (idle in the meantime) Coupled services Stateless and

    short-lived Asynchronous (do not wait for response) Serverless characteristics @ixek
  15. Image and video processing Internet of Things Data pipelines Good

    use cases for FaaS @ixek
  16. • You want to collect data from the StackExchange API

    - to monitor questions from a specific topic or tool • The data needs to be stored for later wrangling and reporting • Ideally, you want to get a daily digest in your inbox with the new questions created as well as a plot of the most commonly used tags in said questions The Scenario @ixek
  17. • Python 3.x (supported 3.6, 3.7, 3.8) • VS Code

    • Python VS Code Extension • Azure Functions VS Code Extension • StackExchange API app key https:!//api.stackexchange.com/ Requirements: @ixek
  18. Getting started - timer @ixek Timer trigger: Every day at

    9am Function gets executed Data is collected from StackExchange
  19. Triggers vs bindings @ixek Triggers Idle function Trigger Active function

  20. Triggers vs bindings @ixek Bindings Idle function Trigger Active function

    In Binding Out Bindings Trigger Binding
  21. Extend function @ixek Timer trigger: Every day at 9am Function

    gets executed Data is collected from StackExchange Store in Blob Storage (csv) Trigger Binding
  22. Completing the scenario @ixek Function gets executed File added in

    Blob Storage (csv) Process data / Create plots Save plots Send digest Retrieve Data Trigger Binding
  23. • Detailed step-by-step tutorial: aka.ms/pycon2020- azurefunctions • GitHub Repository: https:!//github.com/trallard/pycon2020-

    azure-functions • Slides:aka.ms/pycon2020-azurefunctions-slides Resources: @ixek
  24. Hope you enjoyed the tutorial