$30 off During Our Annual Pro Sale. View Details »

Serverless Measurement Limits

Serverless Measurement Limits

Exploring cold starts and warm starts in serverless compute platforms.

Expanded version of the 2017 dotScale talk, "Searching for the Server in Serverless"

Clay Smith

May 05, 2017
Tweet

More Decks by Clay Smith

Other Decks in Programming

Transcript

  1. SERVERLESS
    MEASUREMENT
    LIMITS
    @smithclay
    New Relic
    Based on my dotScale 2017 talk:
    "Searching for the Server in Serverless"

    View Slide

  2. λ
    HTTP Requests
    Client App
    App Architecture with FaaS
    Functions + API Gateway
    Is this going to
    (not) work for me?
    Managed Infrastructure
    λ
    λ
    λ

    View Slide

  3. Crazy idea: Let's run SSH in a λ
    λ
    ssh process

    View Slide

  4. FaaS: Typical Metrics
    Error Count
    Function Invocation Count
    Function Duration (ms)

    View Slide

  5. Event
    Trigger
    1. Invoke
    λ
    2. Run 3. End
    Result
    Error
    Timeout
    or
    or

    View Slide

  6. Why does my function invocation
    time vary so much?
    Extremely common question with FaaS...

    View Slide

  7. Is your λ triggered by user interaction?
    Yes No
    Monitor &
    make it fast
    (> 100ms)
    Monitor &
    still make it fast-ish

    View Slide

  8. Cold Start vs Warm Start
    Event
    Trigger Handler
    Warm
    Function Invocation Time
    Create Initialize Handler
    Cold

    View Slide

  9. λ Running Commands
    const exec = require('child_process').exec;
    exports.handler = (trigger, cb) => {
    exec('whoami', (err, stdout) => {
    console.log(stdout);
    return cb(null);
    });
    }
    [LOG TIME] sbxuser_1066

    View Slide

  10. λ is a UNIX system?!
    I know this!

    View Slide

  11. Interacting this way is
    slow.

    View Slide

  12. Go as a SSH Server + Tunnel-er
    Copy and paste Go SSH server + tunnel code
    (no inbound connections allowed)
    Execute binary on trigger
    Max(λ timeout)

    View Slide

  13. Aside: Old But Still Good Advice
    λ
    Cloud vendor code
    "Secret Sauce
    Library" Keep it portable
    Write short handlers

    View Slide

  14. https://github.com/smithclay/faassh

    View Slide

  15. Info from /proc
    2x Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    cat /proc/cpuinfo
    3857664 kB
    cat /proc/meminfo
    ixgbevf (EC2 10Gbps Network Driver)
    cat /proc/modules
    c4.large EC2 Compute-Optimized Instance (?)

    View Slide

  16. c4.large instance
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λs in theory on a VM
    10 Gbps
    λ = 128 MB Function
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ

    View Slide

  17. Cold Start Discovery in Code
    var SO_SO_COLD = true;
    exports.handler = function(trigger, cb) {
    console.log('Cold? %s', SO_SO_COLD);
    SO_SO_COLD = false;
    return callback(cb)
    }
    https://github.com/smithclay/lambda-proc-info

    View Slide

  18. How are functions frozen?
    https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt
    cgroup freezer!

    View Slide

  19. Warming methodology
    Scheduled
    Event
    λ Only effective for non-
    concurrent execution!
    4 minute
    interval

    View Slide

  20. Data collection from event-driven logs
    λ
    console.log(data)
    Logs λ
    POST to Metrics DB
    (Insights)
    Log Filter Trigger

    View Slide

  21. Cold Starts Visualized
    ~7 hrs ~8 hrs

    View Slide

  22. λ Host Hopping
    10.13
    10.12 10.11 10.13 10.12 10.12 10.12
    10.
    11
    # of AZs in us-west-2: 3

    View Slide

  23. Serverless, in brief
    High availability (multi-AZ or region)
    Elastic VMs and Automagic Freezing Containers
    Really good scheduling algorithm

    View Slide

  24. 4. Better utilization @ cloud scale...
    for the environment?

    View Slide

  25. Measure Your Own FAAS
    Cost,
    Worst-case perf
    Ops/dev
    benefits
    Easier to measure Harder to measure
    tech people

    View Slide

  26. FaaS Production Reality
    λ λ
    Dev Prod
    Orchestration (!?)
    Version/Deploy
    Monitoring
    Security
    Cold Start Mgmt
    The "learning
    cliff"
    Great tweet from @mfdii

    View Slide

  27. FaaS Isn't a Silver Bullet
    "I've got a quick, computationally-intensive task that I need to
    perform occasionally in response to a well-defined event that
    isn't that sensitive to latency."
    —The Ideal FaaS Developer
    // TO DO:
    measure &&
    share

    View Slide

  28. Thanks.
    @smithclay
    New Relic

    View Slide