Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Measurement Limits

Serverless Measurement Limits

Exploring cold starts and warm starts in serverless compute platforms.

Expanded version of the 2017 dotScale talk, "Searching for the Server in Serverless"

Clay Smith

May 05, 2017

More Decks by Clay Smith

Other Decks in Programming


  1. SERVERLESS MEASUREMENT LIMITS @smithclay New Relic Based on my dotScale

    2017 talk: "Searching for the Server in Serverless"
  2. λ HTTP Requests Client App App Architecture with FaaS Functions

    + API Gateway Is this going to (not) work for me? Managed Infrastructure λ λ λ
  3. Crazy idea: Let's run SSH in a λ λ ssh

  4. FaaS: Typical Metrics Error Count Function Invocation Count Function Duration

  5. Event Trigger 1. Invoke λ 2. Run 3. End Result

    Error Timeout or or
  6. Why does my function invocation time vary so much? Extremely

    common question with FaaS...
  7. Is your λ triggered by user interaction? Yes No Monitor

    & make it fast (> 100ms) Monitor & still make it fast-ish
  8. Cold Start vs Warm Start Event Trigger Handler Warm Function

    Invocation Time Create Initialize Handler Cold
  9. λ Running Commands const exec = require('child_process').exec; exports.handler = (trigger,

    cb) => { exec('whoami', (err, stdout) => { console.log(stdout); return cb(null); }); } [LOG TIME] sbxuser_1066
  10. λ is a UNIX system?! I know this!

  11. Interacting this way is slow.

  12. Go as a SSH Server + Tunnel-er Copy and paste

    Go SSH server + tunnel code (no inbound connections allowed) Execute binary on trigger Max(λ timeout)
  13. Aside: Old But Still Good Advice λ Cloud vendor code

    "Secret Sauce Library" Keep it portable Write short handlers
  14. https://github.com/smithclay/faassh

  15. Info from /proc 2x Intel(R) Xeon(R) CPU E5-2666 v3 @

    2.90GHz cat /proc/cpuinfo 3857664 kB cat /proc/meminfo ixgbevf (EC2 10Gbps Network Driver) cat /proc/modules c4.large EC2 Compute-Optimized Instance (?)
  16. c4.large instance λ λ λ λ λ λ λ λ

    λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λs in theory on a VM 10 Gbps λ = 128 MB Function λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ
  17. Cold Start Discovery in Code var SO_SO_COLD = true; exports.handler

    = function(trigger, cb) { console.log('Cold? %s', SO_SO_COLD); SO_SO_COLD = false; return callback(cb) } https://github.com/smithclay/lambda-proc-info
  18. How are functions frozen? https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-subsystem.txt cgroup freezer!

  19. Warming methodology Scheduled Event λ Only effective for non- concurrent

    execution! 4 minute interval
  20. Data collection from event-driven logs λ console.log(data) Logs λ POST

    to Metrics DB (Insights) Log Filter Trigger
  21. Cold Starts Visualized ~7 hrs ~8 hrs

  22. λ Host Hopping 10.13 10.12 10.11 10.13 10.12 10.12 10.12

    10. 11 # of AZs in us-west-2: 3
  23. Serverless, in brief High availability (multi-AZ or region) Elastic VMs

    and Automagic Freezing Containers Really good scheduling algorithm
  24. 4. Better utilization @ cloud scale... for the environment?

  25. Measure Your Own FAAS Cost, Worst-case perf Ops/dev benefits Easier

    to measure Harder to measure tech people
  26. FaaS Production Reality λ λ Dev Prod Orchestration (!?) Version/Deploy

    Monitoring Security Cold Start Mgmt The "learning cliff" Great tweet from @mfdii
  27. FaaS Isn't a Silver Bullet "I've got a quick, computationally-intensive

    task that I need to perform occasionally in response to a well-defined event that isn't that sensitive to latency." —The Ideal FaaS Developer // TO DO: measure && share
  28. Thanks. @smithclay New Relic