Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Application Ops 1.0

Ba4b0b302231bffb0f685bbef25db0d6?s=47 Bryan Liles
November 05, 2015

Application Ops 1.0

An introduction to Application Ops (AppOps)

Ba4b0b302231bffb0f685bbef25db0d6?s=128

Bryan Liles

November 05, 2015
Tweet

More Decks by Bryan Liles

Other Decks in Programming

Transcript

  1. A P P L I C AT I O N

    O P S 1 . 0
  2. C O N C L U S I O N

    S • Having a continuous integration story • Deploy with confidence • Control your logs • Feel what your application is telling you • Experiment and constantly improve
  3. @ B RYA N L @ D I G I

    TA L O C E A N U R B A N S O F T WA R E D E V E L O P E R B RYA N L I L E S
  4. D O Y O U A P P ?

  5. – D E V E L O P E R

    S E V E RY W H E R E “I write software, someone else deploys and maintains it”
  6. D O Y O U O P S ?

  7. A P P O P S L A D D

    E R
  8. None
  9. Continuous Integration

  10. Continuous Integration Continuous Deployment

  11. Continuous Integration Continuous Deployment Logging

  12. Continuous Integration Continuous Deployment Logging Metrics

  13. Continuous Integration Continuous Deployment Logging Metrics Error Handling

  14. C O N T I N U O U S

    I N T E G R AT I O N
  15. C H O O S I N G A C

    I T O O L • Do you want to manage it? • How much control do you want? • What else can it do? • What can’t it do? • What are the real costs?
  16. C O N T I N U O U S

    I N T E G R AT I O N B U I L D I N G A D E L I V E RY P I P E L I N E Garbage -> -> Stuff Magic
  17. C O N T I N U O U S

    I N T E G R AT I O N Garbage -> -> Stuff Magic
  18. C O N T I N U O U S

    I N T E G R AT I O N Commit -> -> Artifact Test/Build
  19. C O N T I N U O U S

    I N T E G R AT I O N What’s an artifact?
  20. D E P L O Y A L L T

    H E T H I N G S
  21. C H O O S I N G A D

    E P L O Y M E N T S T R AT E G Y • Who does the deploys? • Is it easy to set up? • What happens when it breaks? • Where are we deploying to? • What happens when I oops?
  22. D E P L O Y A L L T

    H E T H I N G S
  23. D E P L O Y A L L T

    H E T H I N G S S E R V E R S E R V E R S E R V E R
  24. D E P L O Y A L L T

    H E T H I N G S I M M U TA B L E I N F R A S T R U C T U R E
  25. D E P L O Y A L L T

    H E T H I N G S I M M U TA B L E I N F R A S T R U C T U R E http://bit.ly/iminfra
  26. D E P L O Y A L L T

    H E T H I N G S I bet your deploys look kind of like this
  27. D E P L O Y A L L T

    H E T H I N G S A RT I FA C T 1 S E R V E R 1 S E R V E R 2 S E R V E R 3 Deploy 1
  28. D E P L O Y A L L T

    H E T H I N G S A RT I FA C T 2 S E R V E R 1 S E R V E R 2 S E R V E R 3 Deploy 2
  29. D E P L O Y A L L T

    H E T H I N G S • What do you servers look like after Deploy 2? • What happens when someone logs in to “fix something”? • Are my servers pets or cattle?
  30. D E P L O Y A L L T

    H E T H I N G S Pets or Cattle?
  31. http://bestpetpics.com/wp-content/uploads/2015/03/Cute-Pets.jpg http://media.graytvinc.com/images/cattle48.jpg http://www.slideshare.net/gmccance/cern-data-centre-evolution

  32. D E P L O Y A L L T

    H E T H I N G S Pets
  33. P E T S • Are given cute names like

    odin.example.com • Are unique and loving raised • Require you to nurse them back to health when they get ill
  34. D E P L O Y A L L T

    H E T H I N G S Cattle
  35. C AT T L E • Cattle are given numbers

    like app001.dc2.example.com • They are identical to all the other cattle • When they get ill, you get another one
  36. D E P L O Y A L L T

    H E T H I N G S A RT I FA C T 1 S E R V E R 1 S E R V E R 2 S E R V E R 3 Deploy 1
  37. D E P L O Y A L L T

    H E T H I N G S A RT I FA C T 2 S E R V E R 4 S E R V E R 5 S E R V E R 6 Deploy 2
  38. D E P L O Y A L L T

    H E T H I N G S So WHAT does this mean for me?
  39. We always want to deploy to fresh servers and retire

    the old ones to pasture when we’re done.
  40. When using the “Cloud”, you boot new server instances when

    you start new apps.
  41. or don’t

  42. W H AT ’ S M Y A P P

    D O I N G ?
  43. H O W D O I K N O W

    T H I N G S A R E W O R K I N G C O R R E C T LY ? • What does “Working OK mean?” • Where do I go to see if things are working OK? • What was my app doing at 1545UTC last Thursday?
  44. W H AT ’ S M Y A P P

    D O I N G ? You have logs, right?
  45. You aren’t alone if your process looks like this:

  46. You aren’t alone if your process looks like this: ssh

    app@zeus.example.com
  47. You aren’t alone if your process looks like this: cd

    /app/myapp/shared/logs
  48. You aren’t alone if your process looks like this: tail

    -f production.log | grep “foo”
  49. W H AT ’ S M Y A P P

    D O I N G ? Where are your logs?
  50. W H AT ’ S M Y A P P

    D O I N G ? S E R V E R 1 L O G S S E R V E R 2 L O G S S E R V E R 3 L O G S
  51. W H AT ’ S M Y A P P

    D O I N G ? S E R V E R 2 L O G S S E R V E R 2 L O G S S E R V E R 2 L O G S L O G S E R V E R
  52. W H AT ’ S M Y A P P

    D O I N G ? S E R V E R 2 L O G S S E R V E R 2 L O G S S E R V E R 2 L O G S
  53. W H AT ’ S M Y A P P

    D O I N G ? S E R V E R 2 L O G S S E R V E R 2 L O G S S E R V E R 2 L O G S
  54. W H AT ’ S M Y A P P

    D O I N G ?
  55. W H AT ’ S M Y A P P

    D O I N G ? PS. You don’t have to use logstash
  56. W H AT ’ S M Y A P P

    D O I N G ? Rant: Why you might not use logstash
  57. W H AT ’ S M Y A P P

    D O I N G ? What kinds of things should I log?
  58. W H AT ’ S M Y A P P

    D O I N G ? Answer these questions:
  59. W H AT ’ S M Y A P P

    D O I N G ? If I were to trace through my application and rebuild a flow, what data would I need?
  60. W H AT ’ S M Y A P P

    D O I N G ? What data do I need when hits the fan?
  61. W H AT ’ S M Y A P P

    D O I N G ? Does my code capture errors?
  62. W H AT ’ S G O I N G

    O N ?
  63. Y O U H AV E T O B E

    T H E A P P W H I S P E R E R • Your app has a pulse, what is it? • If your app crashes when no one is paying attention, does it make a sound?
  64. W H AT ’ S G O I N G

    O N ? Disclaimer: This is not another section about logging.
  65. W H AT ’ S G O I N G

    O N ? L O G S O T H E R S T U F F P E A C E O F M I N D
  66. W H AT ’ S G O I N G

    O N ? O T H E R S T U F F
  67. W H AT ’ S G O I N G

    O N ? O T H E R S T U F F
  68. W H AT ’ S G O I N G

    O N ? O T H E R S T U F F E X C E P T I O N S
  69. W H AT ’ S G O I N G

    O N ? O T H E R S T U F F E X C E P T I O N S T E L E M E T RY
  70. W H AT ’ S G O I N G

    O N ? E X C E P T I O N S
  71. W H AT ’ S G O I N G

    O N ? E X C E P T I O N S
  72. W H AT ’ S G O I N G

    O N ? Pro Tip: This shouldn’t be a dumping ground
  73. W H AT ’ S G O I N G

    O N ? O T H E R S T U F F E X C E P T I O N S T E L E M E T RY
  74. W H AT ’ S G O I N G

    O N ? T E L E M E T RY
  75. W H AT ’ S G O I N G

    O N ? T E L E M E T RY
  76. E V E RY O N E L O V

    E S P R E T T Y C O L O R S
  77. O S S TAT E A P P R E

    S O U R C E U S A G E N E T W O R K S TAT S C E N T R A L R E P O O F A L L M E T R I C S A S A T I M E S E R I E S
  78. W H AT E L S E C A N

    I D O ?
  79. W H AT E L S E C A N

    I D O ?
  80. W H AT E L S E C A N

    I D O ?
  81. W H AT E L S E C A N

    I D O ?
  82. W H AT E L S E C A N

    I D O ?
  83. W H AT E L S E C A N

    I D O ? Decrease your infrastructure woes Boot a DigitalOcean Droplet
  84. None
  85. - B RYA N L I L E S (

    @ B RYA N L ) The end