Serverless Measurement Limits

Exploring cold starts and warm starts in serverless compute platforms.

Expanded version of the 2017 dotScale talk, "Searching for the Server in Serverless"

Transcript

1. SERVERLESS
   MEASUREMENT
   LIMITS
   @smithclay
   New Relic
   Based on my dotScale 2017 talk:
   "Searching for the Server in Serverless"
   

   View Slide

2. λ
   HTTP Requests
   Client App
   App Architecture with FaaS
   Functions + API Gateway
   Is this going to
   (not) work for me?
   Managed Infrastructure
   λ
   λ
   λ
   

   View Slide

3. Crazy idea: Let's run SSH in a λ
   λ
   ssh process
   
   

   View Slide

4. FaaS: Typical Metrics
   Error Count
   Function Invocation Count
   Function Duration (ms)
   

   View Slide

5. Event
   Trigger
   1. Invoke
   λ
   2. Run 3. End
   Result
   Error
   Timeout
   or
   or
   

   View Slide

6. Why does my function invocation
   time vary so much?
   Extremely common question with FaaS...
   

   View Slide

7. Is your λ triggered by user interaction?
   Yes No
   Monitor &
   make it fast
   (> 100ms)
   Monitor &
   still make it fast-ish
   

   View Slide

8. Cold Start vs Warm Start
   Event
   Trigger Handler
   Warm
   Function Invocation Time
   Create Initialize Handler
   Cold
   

   View Slide

9. λ Running Commands
   const exec = require('child_process').exec;
   exports.handler = (trigger, cb) => {
   exec('whoami', (err, stdout) => {
   console.log(stdout);
   return cb(null);
   });
   }
   [LOG TIME] sbxuser_1066
   

   View Slide

10. λ is a UNIX system?!
    I know this!
    

    View Slide

11. Interacting this way is
    slow.
    

    View Slide

12. Go as a SSH Server + Tunnel-er
    Copy and paste Go SSH server + tunnel code
    (no inbound connections allowed)
    Execute binary on trigger
    Max(λ timeout)
    

    View Slide

13. Aside: Old But Still Good Advice
    λ
    Cloud vendor code
    "Secret Sauce
    Library" Keep it portable
    Write short handlers
    

    View Slide

14. https://github.com/smithclay/faassh
    

    View Slide

15. Info from /proc
    2x Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz
    cat /proc/cpuinfo
    3857664 kB
    cat /proc/meminfo
    ixgbevf (EC2 10Gbps Network Driver)
    cat /proc/modules
    c4.large EC2 Compute-Optimized Instance (?)
    

    View Slide

16. c4.large instance
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λs in theory on a VM
    10 Gbps
    λ = 128 MB Function
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    

    View Slide

17. Cold Start Discovery in Code
    var SO_SO_COLD = true;
    exports.handler = function(trigger, cb) {
    console.log('Cold? %s', SO_SO_COLD);
    SO_SO_COLD = false;
    return callback(cb)
    }
    https://github.com/smithclay/lambda-proc-info
    

    View Slide

18. How are functions frozen?
    https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt
    cgroup freezer!
    

    View Slide

19. Warming methodology
    Scheduled
    Event
    λ Only effective for non-
    concurrent execution!
    4 minute
    interval
    

    View Slide

20. Data collection from event-driven logs
    λ
    console.log(data)
    Logs λ
    POST to Metrics DB
    (Insights)
    Log Filter Trigger
    

    View Slide

21. Cold Starts Visualized
    ~7 hrs ~8 hrs
    

    View Slide

22. λ Host Hopping
    10.13
    10.12 10.11 10.13 10.12 10.12 10.12
    10.
    11
    # of AZs in us-west-2: 3
    

    View Slide

23. Serverless, in brief
    High availability (multi-AZ or region)
    Elastic VMs and Automagic Freezing Containers
    Really good scheduling algorithm
    

    View Slide

24. 4. Better utilization @ cloud scale...
    for the environment?
    

    View Slide

25. Measure Your Own FAAS
    Cost,
    Worst-case perf
    Ops/dev
    benefits
    Easier to measure Harder to measure
    tech people
    

    View Slide

26. FaaS Production Reality
    λ λ
    Dev Prod
    Orchestration (!?)
    Version/Deploy
    Monitoring
    Security
    Cold Start Mgmt
    The "learning
    cliff"
    Great tweet from @mfdii
    

    View Slide

27. FaaS Isn't a Silver Bullet
    "I've got a quick, computationally-intensive task that I need to
    perform occasionally in response to a well-defined event that
    isn't that sensitive to latency."
    —The Ideal FaaS Developer
    // TO DO:
    measure &&
    share
    

    View Slide

28. Thanks.
    @smithclay
    New Relic
    

    View Slide