Serverless Measurement Limits

Serverless Measurement Limits

Exploring cold starts and warm starts in serverless compute platforms.

Expanded version of the 2017 dotScale talk, "Searching for the Server in Serverless"

2ad55a66b8325eb097612c069ab76231?s=128

Clay Smith

May 05, 2017
Tweet

Transcript

  1. SERVERLESS MEASUREMENT LIMITS @smithclay New Relic Based on my dotScale

    2017 talk: "Searching for the Server in Serverless"
  2. λ HTTP Requests Client App App Architecture with FaaS Functions

    + API Gateway Is this going to (not) work for me? Managed Infrastructure λ λ λ
  3. Crazy idea: Let's run SSH in a λ λ ssh

    process
  4. FaaS: Typical Metrics Error Count Function Invocation Count Function Duration

    (ms)
  5. Event Trigger 1. Invoke λ 2. Run 3. End Result

    Error Timeout or or
  6. Why does my function invocation time vary so much? Extremely

    common question with FaaS...
  7. Is your λ triggered by user interaction? Yes No Monitor

    & make it fast (> 100ms) Monitor & still make it fast-ish
  8. Cold Start vs Warm Start Event Trigger Handler Warm Function

    Invocation Time Create Initialize Handler Cold
  9. λ Running Commands const exec = require('child_process').exec; exports.handler = (trigger,

    cb) => { exec('whoami', (err, stdout) => { console.log(stdout); return cb(null); }); } [LOG TIME] sbxuser_1066
  10. λ is a UNIX system?! I know this!

  11. Interacting this way is slow.

  12. Go as a SSH Server + Tunnel-er Copy and paste

    Go SSH server + tunnel code (no inbound connections allowed) Execute binary on trigger Max(λ timeout)
  13. Aside: Old But Still Good Advice λ Cloud vendor code

    "Secret Sauce Library" Keep it portable Write short handlers
  14. https://github.com/smithclay/faassh

  15. Info from /proc 2x Intel(R) Xeon(R) CPU E5-2666 v3 @

    2.90GHz cat /proc/cpuinfo 3857664 kB cat /proc/meminfo ixgbevf (EC2 10Gbps Network Driver) cat /proc/modules c4.large EC2 Compute-Optimized Instance (?)
  16. c4.large instance λ λ λ λ λ λ λ λ

    λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λs in theory on a VM 10 Gbps λ = 128 MB Function λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ
  17. Cold Start Discovery in Code var SO_SO_COLD = true; exports.handler

    = function(trigger, cb) { console.log('Cold? %s', SO_SO_COLD); SO_SO_COLD = false; return callback(cb) } https://github.com/smithclay/lambda-proc-info
  18. How are functions frozen? https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt cgroup freezer!

  19. Warming methodology Scheduled Event λ Only effective for non- concurrent

    execution! 4 minute interval
  20. Data collection from event-driven logs λ console.log(data) Logs λ POST

    to Metrics DB (Insights) Log Filter Trigger
  21. Cold Starts Visualized ~7 hrs ~8 hrs

  22. λ Host Hopping 10.13 10.12 10.11 10.13 10.12 10.12 10.12

    10. 11 # of AZs in us-west-2: 3
  23. Serverless, in brief High availability (multi-AZ or region) Elastic VMs

    and Automagic Freezing Containers Really good scheduling algorithm
  24. 4. Better utilization @ cloud scale... for the environment?

  25. Measure Your Own FAAS Cost, Worst-case perf Ops/dev benefits Easier

    to measure Harder to measure tech people
  26. FaaS Production Reality λ λ Dev Prod Orchestration (!?) Version/Deploy

    Monitoring Security Cold Start Mgmt The "learning cliff" Great tweet from @mfdii
  27. FaaS Isn't a Silver Bullet "I've got a quick, computationally-intensive

    task that I need to perform occasionally in response to a well-defined event that isn't that sensitive to latency." —The Ideal FaaS Developer // TO DO: measure && share
  28. Thanks. @smithclay New Relic