DockerSF Meetup: Building a Serverless Platform on Docker

Transcript

1. Building a Serverless Platform on Docker
   Chad Arimura
   VP Serverless, Oracle
   @chadarimura
   Docker SF Meetup: 2/1/18
   

   View Slide

2. About Me
   ● Engineer and builder
   ● 3x entrepreneur
   ○ 1999-2006: Founding CTO of AllDorm
   ○ 2008-2011: Founding Partner of Appoxy
   ○ 2011-2017: Founding CEO of Iron.io
   ● Now VP Serverless, Oracle
   me
   

   View Slide

3. What is Serverless?
   ● Serverless is an abstraction of infrastructure and its operations
   including provisioning, scaling, patching, etc.
   ● Serverless architecture is when an app is built entirely on
   serverless components (compute, storage, networking)
   ● FaaS is the compute component in a serverless architecture
   

   View Slide

4. Why Serverless?
   ● Easier: Just think about your code, not infrastructure
   ● Powerful: Transparent and limitless scaling
   ● Faster: Deploy faster, iterate faster, innovate faster
   ● Cheaper: Only pay for what you use to the 100ms (never idle)
   

   View Slide

5. Functions-as-a-Service
   ● Functions are small bits of code that do one thing well and are
   easy to understand and maintain
   ● As a service means no complicated plumbing, the system
   takes care of provisioning, scaling, patching, maintaining, etc.
   Each function scales independently.
   In mathematics, a function is a relation between a set
   of inputs and a set of permissible outputs with the
   property that each input is related to exactly one output.
   Function (mathematics) - Wikipedia
   https://en.wikipedia.org/wiki/Function_(mathematics)
   

   View Slide

6. Containers vs Functions
   Function is a container with a
   set of known traits:
   ● Short running
   ● Ephemeral
   ● Stateless
   ● Invoked
   ● Single Purpose
   ● Self-contained
   

   View Slide

7. Introducing the Fn Project
   ● Open-source serverless compute platform
   ● Can be deployed to any cloud and on-premise
   ● Simple, elegant, and extensible by design
   ● Containers are primitives
   ● Hot containers provide fast response times (< 100ms)
   ● Active w/ 2500+ commits across 50+ contributors
   ● Independently governed with plans for foundation
   

   View Slide

8. For Developers
   

   View Slide

9. An Fn Function
   ● Small chunk of code wrapped into a container image
   ● Gets input via STDIN and environment
   ● Produces output to STDOUT
   ● Logs to STDERR
   The Fn server handles everything else, like the API gateway, piping
   things around, storing logs, etc.
   

   View Slide

10. Fn CLI
    ● fn init --runtime go
    ● fn run
    ● fn test
    ● fn deploy --app myapp
    ● fn call myapp myfunc
    → http://localhost:8080/r/myapp/myfunc
    

    View Slide

11. fn deploy details
    1. Builds container (multi-stage) + bumps version
    2. Pushes container to registry
    3. Creates/updates function route (servers lazy load images)
    MyFunc:0.0.2
    MyFunc:0.0.2 MyFunc:0.0.2
    Your code
    Fn Service
    myfunc →
    /r/myapp/myfunc:0.0.2
    1 2 3
    

    View Slide

12. Function Development Kits (FDKs)
    ● Used to help with parsing input and writing output
    ● Familiar syntax for Lambda developers
    ● Simply write a `handler` function that adheres to the FDK’s
    interface and it will parse STDIN and provide the input data to
    your function and deal with writing the proper output format.
    ● Makes it a lot easier to write hot functions
    

    View Slide

13. Debugging
    ● fn calls list myapp
    ● fn calls get myapp
    ● fn logs get myapp
    ● Metrics created using OpenTracing w/ initial collectors and
    extensions for Prometheus, ZipKin, and soon Jaeger
    

    View Slide

14. Fn UI
    

    View Slide

15. Demo #1
    

    View Slide

16. For Operators
    

    View Slide

17. Architecture
    

    View Slide

18. Fn Server
    ● Handles CRUD operations for setting up routes and functions
    ● Executes sync functions, returning responses to clients
    immediately
    ● Queues async function calls
    ● Executes async functions when capacity is available
    ● Written in Go, easy to extend via plugin module system
    

    View Slide

19. Fn LB
    ● Simple, fast load balancer that routes functions to certain nodes
    consistently for hot function efficiency
    ● Scales each function independently based on traffic to any
    particular function
    ● Can be used to scale Fn servers and infrastructure as well as it
    has a view of global state of all fn servers
    

    View Slide

20. Fn LB Details
    

    View Slide

21. Supporting Services
    ● DB, MQ, blob store are all pluggable modules that are thin
    wrappers around their respective drivers.
    ○ DB: MySQL, sqlite3, Postgres
    ○ Queue: Redis, Kafka
    ○ Registry: Any Docker v2-compliant, even private
    ● Metrics/Monitoring
    ○ OpenTracing API for metrics
    ○ Prometheus support, pluggable backends
    ○ Logging via syslog
    

    View Slide

22. Open Tracing and Prometheus
    ● OpenTracing spans with plugins to send to prom where
    Grafana can be used to view Fn data
    

    View Slide

23. Request Flow
    

    View Slide

24. Sync Request
    

    View Slide

25. Async Request
    

    View Slide

26. Scheduling
    

    View Slide

27. Fn Scheduling
    t0
    

    View Slide

28. Fn Scheduling
    t1
    

    View Slide

29. Fn Scheduling
    t2
    

    View Slide

30. Fn Scheduling
    t3
    

    View Slide

31. Fn Scheduling
    t4
    

    View Slide

32. Fn Scheduling
    t5
    

    View Slide

33. Fn Scheduling
    t6
    

    View Slide

34. Fn Scheduling
    t7
    

    View Slide

35. Kubernetes
    

    View Slide

36. Kubernetes
    ● Fn is scheduler agnostic but lots of optimization/management
    work in process to optimize on Kubernetes
    ● Helm chart available at https://github.com/fnproject/fn-helm
    ● Thinking about deeper Kubernetes integrations including CRD’s
    to model functions
    

    View Slide

37. Kubernetes
    Deployment
    

    View Slide

38. Why not K8s Scheduling?
    1. Speed
    a. Pod launch time is too slow for sync requests
    b. Coordinating all resource alloc to one k8s master is slow
    c. Yes we can preload + hot pod like we do with current scheduling
    but…
    2. Scale
    a. Runs out of addressable network space quickly
    b. Functions easily scale to the hundreds of thousands / millions
    

    View Slide

39. Docker
    

    View Slide

40. Storage Stuff
    1. Tried a lot of various storage drivers
    a. Overlay1 ran out of file descriptors / blocks
    b. Btrfs container creation / deletion under load was minutes
    c. Overlay2 was the only one that ended up tenable
    

    View Slide

41. Why DinD?
    1. Contained blast radius
    a. Doesn’t affect outer VM
    b. Kick the fn container and fire up a new one
    c. Systemd can manage Fn nicely
    2. Control Docker version / config easier for our customers
    a. Inner can differ from outer
    

    View Slide

42. Fn Flow
    

    View Slide

43. Fn Flow
    ● Build long-running, reliable, scalable functions with rich sets of
    language-specific primitives including fork-join, chaining, delays
    and error handling
    ● Supports complex parallel processes that are readable and
    testable (including unit tests) with standard programming tools
    ● Java support using CompletableFuture API from Java 8 with
    JS, Python, Go language support on the way!
    

    View Slide

44. Plate Detect
    Function
    Draw
    Function
    Slack
    Function
    Twitter
    Function
    Scraper
    

    View Slide

45. Scraper
    Plate Detect
    Function
    Draw
    Function
    Slack
    Function
    Twitter
    Function
    FlowFuture.invoke()
    thenCompose()
    thenCompose()
    allOf()
    whenComplete()
    

    View Slide

46. Scraper
    Plate Detect
    Function
    Draw
    Function
    Slack
    Function
    Twitter
    Function
    FlowFuture.invoke()
    thenCompose()
    thenCompose()
    allOf()
    whenComplete()
    Flow Function
    

    View Slide

47. View Slide

48. Demo #2
    

    View Slide

49. Thank you!
    1. Star the project: github.com/fnproject/fn
    2. Join the conversation: slack.fnproject.io
    3. Learn more: fnproject.io
    4. We’re hiring engineers and evangelists:
    chad.arimura@oracle.com
    Name
    Title
    @twitter
    Get Involved
    

    View Slide

50. Appendix
    

    View Slide