FaaS Measurement Fundamentals

"Don't Worry About Servers, Still Worry About Metrics: FaaS Measurement Fundamentals"

Talk at Gluecon 2017.

Transcript

1. Don't Worry About Servers
   Still Worry About Metrics
   FaaS Measurement Fundamentals
   @smithclay
   New Relic
   Gluecon 5/24/17
   

   View Slide

2. Metrics are what we
   measure
   *hopefully useful things
   

   View Slide

3. λ
   This Thing Appeared
   The Magic Hat
   of Werner Vogels
   How do we
   understand it?
   

   View Slide

4. Hyped tech wish list
   Metrics Trends
   Alerting CTA
   Logging Detail
   Tracing Cause
   Analytics All of the above
   

   View Slide

5. MALTA Observability Index for FaaS
   Metrics
   Alerting
   Logging
   Tracing
   Analytics
   Maturity Level
   

   View Slide

6. Built-in FaaS Metrics*
   Error Count
   Function Invocation Count
   Function Duration (ms)
   * not comprehensive, but the important ones
   

   View Slide

7. Why does function
   invocation time vary so
   much?
   

   View Slide

8. Event
   Trigger
   1. Invoke
   λ
   2. Run 3. End
   Result
   Error
   Timeout
   or
   or
   

   View Slide

9. Cold Start vs Warm Start
   Event
   Trigger
   Handler Code
   Warm
   Function Invocation Time
   Create Initialize Handler Code
   Cold
   

   View Slide

10. What's inside AWS
    Lambda?
    "It's containers" — Person waving their hands
    

    View Slide

11. λ: Running Commands for Discovery
    const exec = require('child_process').exec;
    exports.handler = (trigger, cb) => {
    exec('whoami', (err, stdout) => {
    console.log(stdout);
    return cb(null);
    });
    }
    [LOG TIME] sbxuser_1066
    

    View Slide

12. λ is a UNIX system?!
    I know this!
    

    View Slide

13. Let's run SSH in λ
    λ
    ssh process
    
    SSH Tunnel
    Firewall: no inbound ports
    

    View Slide

14. SSH in Lambda Architecture
    λ
    node.js wrapper
    go sshd
    binary (x64) Go SSH Crypto Libs
    process.exec()
    https://github.com/smithclay/faassh
    

    View Slide

15. Max Session Length: 5 minutes
    (custom prompt optional)
    https://github.com/smithclay/faassh
    

    View Slide

16. Info from /proc
    2x Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    cat /proc/cpuinfo
    3857664 kB
    cat /proc/meminfo
    ixgbevf (EC2 10Gbps Network Driver)
    cat /proc/modules
    c4.large EC2 Compute-Optimized Instance (?)
    

    View Slide

17. c4.large instance
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λs in theory on a VM
    10 Gbps
    λ = Running 128 MB Function
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ = Not Running 128 MB Function
    

    View Slide

18. Frozen functions help avoid cold starts
    https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-
    subsystem.txt
    cgroup freezer
    subsystem
    λ λ λ
    

    View Slide

19. Internals Recap
    It's just containers on a VM
    Functions frozen when not running
    No magic unikernels involved
    

    View Slide

20. How do we measure and
    prevent cold starts?
    "Use Kubernetes" — Troll
    

    View Slide

21. Cold Start Discovery
    var SO_SO_COLD = true;
    exports.handler = function(trigger, cb) {
    console.log('Cold? %s', SO_SO_COLD);
    SO_SO_COLD = false;
    return callback(cb)
    }
    https://github.com/smithclay/lambda-proc-info
    

    View Slide

22. Warming automation
    Scheduled
    Event
    λ Only effective for non-
    concurrent execution!
    4 minute
    interval
    http://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html
    

    View Slide

23. Sending cold start events to analytics
    λ
    console.log(coldStart)
    Logs λ
    POST to Event DB
    (Insights)
    CloudWatch
    Log Filter Trigger
    

    View Slide

24. Cold Starts Visualized
    ~7 hrs ~8 hrs
    

    View Slide

25. λ Host Uptime
    Cold starts happen when hosts change!
    ~8hrs
    

    View Slide

26. λ Host Subnet Hopping
    10.13
    10.12 10.11 10.13 10.12 10.12 10.12
    10.
    11
    # of AZs in us-west-2: 3
    

    View Slide

27. What's the maximum concurrency of your function?
    one > 1
    A scheduled event
    will warm it
    until host retires.
    More advanced
    strategy needed*
    

    View Slide

28. "Advanced" Strategy
    for i in `seq 1 $NUM_EXECUTIONS`;
    do
    echo "[$i] Executing $AWS_LAMBDA_FUNCTION_NAME..."
    aws lambda invoke ...
    done
    https://gist.github.com/smithclay/e89dfe35fe2a4938db56bb12df76777c
    

    View Slide

29. Multiple containers running on a
    single host to serve parallel requests
    Tracking /proc/sys/kernel/random/boot_id and hostname
    Confirmed: cold start happens on container init.
    

    View Slide

30. So is this just a container PaaS?
    High-availability/multiple zones
    Elastic fleet of compute-optimized VMs
    A very good scheduling algorithm
    Design (freezing, limits, etc) for very fast invocation
    Only if your PaaS has...
    

    View Slide

31. FaaS in Production Reality
    λ λ
    Dev Prod
    Orchestration (!?)
    Version/Deploy
    Monitoring
    Security
    Cold Start Mgmt
    The "learning
    cliff"
    Great tweet from @mfdii
    

    View Slide

32. FaaS Isn't a Silver Bullet
    "I've got a fast, computationally-intensive task that I need to
    perform occasionally in response to a well-defined event that
    isn't that sensitive to latency."
    —The Ideal FaaS Developer
    // TO DO:
    measure &&
    share results
    

    View Slide

33. Thanks.
    @smithclay
    New Relic
    Gluecon 5/24/17
    

    View Slide